New blog at sgwhitehat.com

I am starting a new cyber security blog at sgwhitehat.com. I would probably not be spending much time on this blog anymore. Cheers!

AWS DynamoDB API calls not working? Could be the region is wrong

I was playing with Amazon Web Services recently, and one cool NoSQL database SAS (Software as a Service) they have is DynamoDB. As a ruby guy, I naturally used the AWS Ruby SDK to access it. Took me a while to figure out that I had to create a user in my account AND then add DynamoDB access permissions to the user before I can make DynamoDB API calls (AWS takes their security seriously, not bad ...).

I created a table and populated it with data via the browser, and I made API calls to it. But I kept getting  

AWS::DynamoDB::Errors::ResourceNotFoundException: Requested resource not found

Huh? The table is right there in the browser but the API call cannot find it! Turns out that the response is correct - the table does exist, but in a different AWS datacenter (or region). Just had to add the :region option.


dynamo_db = AWS::DynamoDB.new(:access_key_id => 'xxxx',:secret_access_key => 'xxxx',:region => 'your-region')


The exact region string can be (hopefully) inferred from the table details in the browser. An example is 'us-west-1'

I suspect the same pitfall may apply to other AWS services that are tied to regions.

Copying iPhoto Library too slow? Here is a trick.

Found to my surprise that my mac needs 24 hours to copy a 45 GB iPhoto Library to the USB hard disk. Why? Because the iPhoto Library itself is a package of tens of thousands of files. It doesn't help that iPhoto itself generates thousands of files on its own.

The trick is that it takes far shorter time if you copy 45 GB as a single file into the USB media (because you avoid tens of thousands slower filesystem calls / writes onto the USB media).

Try this method to reduce the copying to a "mere" few hours:

1) Tar up the iPhoto Library. Prefer tar to compression as your JPEG and MOV are already highly compressed, so doing compression would be a waste.

Open up the Terminal in Mac OS. Use "tar -cf your_name.tar your_original_iPhoto_Library".

This took a long enough time for my iPhoto Library, maybe 1 hour.

2) Copy the your_name.tar into the USB. As one big binary blob, the copying is much faster. Takes about 20 mins in my case.

3) At the destination mac, just untar it in a directory of your choosing.

Open up the Terminal. Use "tar -xf your_name.tar". This took less than 1 hour.

SWIG Ruby constants

This is my further adventures in the use of SWIG for Ruby 1.9

I defined constants in interface.i for SWIG

#define constant1 10

#define constant1 20

and I got the ruby module created. However how do I use the constants? Turns out that SWIG creates the constants in a funny (though still usable) way. Assuming you call your module 'm', you can see the relevant methods in irb by executing M.methods. The most important ones are

M.constants <= returns an array of the constants

M.const_get(symbol) <= returns actual value

For example

irb>M.const_get :constant1

10

There is another method M.const_set(symbol,x) which is useful if you are extending the module with more constants. If you just call

irb> M.const_set :constant1 2

this returns an error, as :constant1 is probably declared final (or its equivalent).

DLL wrapper for Ruby 1.9 in Windows

I found myself in a situation where I had to access a Windows DLL to control some external PC peripheral. Hey I thought, this is a good opportunity to hone my Ruby skills.

At first I used Ruby's Win32API. It works fine, provided that the dll function does not return a pointer. For example, if the DLL function is

char * controlx (int a, int b, some_struct * c)

In ruby you can process arguments a, b, and even c, but I almost vomited blood trying to process the return pointer. The thing is, Ruby do not use pointers ! This makes it very tricky to process return pointers from functions.

I was about to use plain old C to do the job, or maybe even Python (I got a Python friend - or fiend - who is telling me that this is no prob in Python, even though Python doesn't use pointers too). Then I googled upon the SWIG project (www.swig.org).

It do me a while to get SWIG working for Ruby. SWIG is pretty well documented, but the below gives the juice of the process and some gotchas. SWIG essentially generates a Ruby wrapper module (in c) for the DLL.

To save hassles, get the same c compiler and environment that created the Ruby windows version you are using. Ruby 1.9 uses MINGW compiler (basically an open source Windows C compiler that uses the Windows libraries, unlike CYGWIN that uses the POSIX libraries). You also need the make program. So I installed MINGW and MSYS, which gives a POSIX-like shell with the essential make program and other usual shell goodies. I think it may be possible to use Microsoft Visual C and nmake.exe (same Windows library calls right?), but I didn't to bother to try that.

You have to create a interface.i file that lists down the DLL functions. It looks like this.

%module your_desired_ruby_name

%{

/* Put header files here or struct declarations that are needed by the functions below */

%}

%feature("autodoc", "1"); /* this is for Ruby's rdoc */

extern char * DLL_function1(void);

%feature("autodoc", "1");

extern bool DLL_function2(void);

...............

Then just execute

> swig -ruby interface.i

or for c++

> swig -c++ -ruby interface.i

SWIG is supposed to be able to wrap both C and C++ functions for Ruby. However when I tried it with the c++ option, the generated c++ wrapper code is not compilable. No problem, I just changed all the bool types in my interface to char, and I had switched to a nice C interface. Anyway why would anyone export C++ functions in a DLL? DLLs are meant to be called from any language.

SWIG generates a c program named interface_wrapper.c. This is just a Ruby c module. So you can now use the standard Ruby ways to compile the module. You can either go the gcc way, or use Ruby built-in mkmf class. mkmf is recommended because it handles all the gcc settings for your Ruby installation (mostly the include dirs etc).

To use mkmf, create a extconf.rb file that looks like this

require 'mkmf'

$libs = append_library($libs, "your_dll_name")

create_makefile('your_ruby_module_name')

Now execute

>ruby extconf.rb

This creates a Makefile in the local directory

>make

This compiles the wrapper Ruby module. Put the DLL in the same directory as the wrapper c code to avoid any linking problems. Then just

>make install

Viola! Your Ruby wrapper module had been compiled and copied to your Ruby installation. In my case it is C:\Ruby19\lib\ruby\site_ruby\1.9.1\i386-msvcrt

To use your DLL functions, just fire up irb or type in a ruby program

irb> require 'your_module_name'

irb> YOUR_MODULE_NAME_IN_CAPITAL.dll_function_xxxx

Oh yeah one last thing. Use rdoc to generate some nice documentation of what the DLL input/output parameters are.

rdoc -f html interface_wrap.c

Removing Nokia E71's camera

Many companies do not allow their employees to use camera phones. Unfortunately my company was one of them. But the nokia E71 is such a great phone - I like the full QWERTY keyboard very much, SMS will no longer a chore like my old phones - that I decided to get one and just remove the camera myself.

One needs the T5 and T6 special screwdrivers, and some sharp plastic edge to dissassemble the E71. No metal tools for prying open some of the parts! I learnt too late from a forum that this is VERY unwise and will just damage your cover (which it did a little to mine)



1. First remove the back covers and battery and SIM card.



2. Use T5 or T6 tool to unscrew (RED)

3. Just depress the latches (BLUE) to push out the front cover

4. As there are metal latches at the head of the phone, you got to clear the latches a bit from the body first, then just slide it towards the head (maintain the angle shown in the next pic). Make sure that the latches do not hit the flexible PCB strip that goes to the volume controls. I was trying to slide the cover out so hard that I didn't notice until quite late that the latches were damaging flex PCB. Luckily the volume controls are still working in the end (I think).



You will get this:



There are two more screws at the upper left and right of the LCD that you need to unscrew using T5 or T6 before you can remove the back cover.

By the way, I am not sure if you need to take out the antenna cover at the base of the device in order to remove the front cover, but I did anyway.

5. Pry the PCB assembly off the back cover. Camera is highlighted in RED.



6. The 3.2 Megapixel camera is attached to the underside of the PCB by a simple flex connector, so its easy to remove. You do have to open up the two pieces of the PCB assembly a bit to remove the connector

See how slim the thing is :



Split to access the camera connector:



The camera, with the mechanical auto focus:



7. If you look closely, there is a pin hole camera (RED) at the head of the phone for 3G video calls. You can either desolder this or just seal it up with opaque tape or something.



8. Now assemble back. This is probably the most DIFFICULT part. You need to snap the PCB assembly into the cover, and you got to do it carefully so that you don't irrevocably damage some plastic. I find it easier to disassemble the PCB assembly. Put in the green piece first into the back cover then the LCD piece. When you put the front cover back, do remember to slide in the front cover from the top a bit.

Ta ta! Assembled back. The cool sticker at the back is from a recent Japan trip.




Just for curiosity, I disassembled everything in the process to see how the E71 is constructed. Surprisingly, most of the PCB is bare. Most of the electronics is at the lower half, shielded behind EMI enclosures.



You can see three antennas. The bigger one is for WCDMA (or 3G). The two smaller white ceramic-like antennas are for Bluetooth and WiFi. I believe the design is called folded dipole?



The back cover is carefully designed too. You can see a speaker, vibrator (the rectangular metal box - I know there's a magnet inside because it attracts the screws), the camera flash (PCB). The parallel strips thingy is the headset jack. No idea whats the circular object though.

Internet Radio Device

I decided to use the pico-ITX computer mainly as an internet radio device. Ok, so this would be an under-utilization of the 1 GHz CPU, 512 MB, Win XP appliance, but I can't think of any other more interesting applications that I would need at the moment. Of course I can use it to watch streaming video, or play my mp3 collections, but internet radio (SHOUTCAST) would be its main purpose.

I googled and installed the following software
- Winamp : mp3, internet radio player
- CLamp : command line control application to Winamp's API
- AutoHotkey : a key mapping tool with surprisingly powerful scripting capabilties.

I also got myself a nice little numeric keyboard. With a simple script, I am now able to select radio stations, control volume, and shutdown the device with a single keypress. I have also configured XP so that it will autoplay the first internet radio station upon boot up. No user input required. On the power switch, wait 30s, you got the music. The audio-out goes straight to my hi-fi system auxilary input, so the sound is great.


The device doesn't need a monitor to boot up, but I linked the VGA out to my HDTV. I usually off the HDTV when I turned on the device, but if I wanted to find out the name of a particularly good song, I can check it out on screen.