Interesting article about unikernels

I came across this interesting article about unikernels:

http://queue.acm.org/detail.cfm?id=2566628

It will be interesting to see how all of this change in the cloud shapes up in the future and how it will effect the traditional operating system Jobbing Software Engineer!

Fascinating DevOps Talk

While browsing The Register, I came across this video Adam Jacob’s keynote speech at ChefTalk 2015 where he introduced DevOps, it’s fascinating stuff and well worth a watch!

What interests me is how methods of software development and deployment are changing so quickly and how this is bringing in new ideas and talent beyond that of the traditional small pool of software developers and engineers.

libexif binaries for Windows and Visual Studio

32bit libexif binaries for Windows (and Visual Studio) which were compiled on Windows using this method, can be downloaded from here. These are unsupported & use them at your own peril, but they work fine for me!

libexif-win32-0.6.21

Build libexif for Windows and Visual Studio

This is a record of how to build / compile libexif (v0.6.21) binaries for windows. It is another entry in a Software Engineer’s daily ‘trial log’, however this time thanks to the excellent MinGW32 and the well written libexif things went rather well!

This readme says that you have 2 options, either hack together a project in Visual Studio to build the library or use MinGW32 to build it.

I tried the first option, but the main problem is that poor old Visual Studio can’t handle some of the more modern standard C (C99) constructs that libexif uses (e.g. the inline keyword).

So that left the MinGW route which is detailed here:

1.) Install MinGW32 onto your windows machine if you don’t already have it. Florian Wolters has a good description of how to do this here (thanks!).

I found it vital to put the path to WinGW32’s bin directory at the beginning of my system PATH variable, not at the end! Check that MinGW32 is working ok by trying Florian’s little test program.

2.) Get the libexif source here. Extract it to somewhere.

3.) Open a windows console window (CMD), don’t use GitBash or anything, just cmd.exe, cd into the extracted libexif folder. I used GitBash at the begining but was getting make errors, it turned out that some of GitBash’s tools were conflicting with MinGW’s tools.

4.) Make libexif by issuing the following commands:

#
sh ./configure --prefix=/tmp/install_libexif
make
make install
#

Now during make, you may get an error like this:

libtool: link: cannot find the library `/home/keith/staged/mingw32/lib/libiconv.’

If this happens, go to your MinGW lib directory (e.g. C:\MinGW\lib) and delete this file:

libintl.la

And try running make again, if you continue to get the above error, then the advice out there us to delete any files with ‘keith’ in them, but luckily I didn’t have to.

If make & make install succeed, you should then see the install directories in /tmp, which you can access using MS explorer in your MinGW\msys directory:

e.g. maybe in C:\MinGW\msys\1.0\tmp

Thankfully that all worked for me, I was then able to link to the libexif libraries from a Visual Studio 2012 C++ project. The only gotcha I found is that when freeing the char* buffer allocated during a call to exif_data_save_data(), I found it important not to use free() but to use libexif’s own memory deallacator like this:

// 
   unsigned char *exif_data;
    
    /* Get a pointer to the EXIF data block we just created */
    exif_data_save_data(exif, &exif_data, &exif_data_len);
    // Use exif_data here, save it to file etc..
    // Now we want to free this memory
    // don't  do this, we will get a heap error:
    // free(exif_data) 
    // instead can do this (as long as you used
    // the default mem allocator earlier in your
    // code):
    ExifMem *mem = exif_mem_new_default();
    exif_mem_free(mem, exif_data);
//

This is probably due to my test application using a different malloc() to the lib.

To link to libEXIF from visual studio you can directly add libexif.dll.a to Properties / Linker / Input / Additional Dependencies and then just make sure that libexif-12.dll is somewhere on the execution path.

If you don’t fancy all of this hassle the library binaries can be downloaded from here.

Geotag – EXIF GPS Latitude field format with libEXIF

I have been developing some software to geotag jpeg images by adding EXIF GPS information using libEXIF. This is very handy as loads of applications like GIS systems and google maps etc can correctly geographically position your images.

As usual I started in the middle rather than starting at the beginning and got a bit confused by how GPS latitude and longitude fields are specified in EXIF, so I decided to try to describe it here with pictures (so that I can test out the google drawing app).

So, latitude (and longitude) can be expressed in different ways bit it is essentially just an angle. Common ways of expressing these angles are:

Degrees, minutes & seconds (with decimal places)
N 52 58 40.44

Degrees & minutes (with decimal places)
N 52 58.674

Degrees (with decimal places)
52.97790

The EXIF latitude field allows you to specify the angle in all of these forms, it is made up of 3 parts as follows:

1.) Degrees – Rational (8 bytes)
2.) Minutes – Rational (8 bytes)
3.) Seconds – Rational (8 bytes)

Each part is an EXIF Rational, it is hard to find a description of its format, but a an EXIF rational contains two 4-byte words and is like a fraction. The first word specifies the value’s magnitude while the second denominates the units. Consider the following values, (where ‘/’ should be read as ‘over’ or ‘divided by’):

a.) 52 = 52 / 1 (52 units)
b.) 40.44 = 4044 / 100 (4044 hundredths)
c.) 52.97790 = 52977900 / 1000000 (52977900 millionths)
d.) 0 = 0/1

the last value ( 0/1 ) is handy as it allows us to specify, say, 0 seconds if we only want to provide degrees and fractional minutes.

To set a rational we can use libEXIF’s set_rational() function like this:

//
// 40.44 = 4044 / 100 (4044 hundredths)
exif_set_rational(entry->data, EXIF_BYTE_ORDER_INTEL, { 4044, 100 });
//

or more generally, if, for example, you want to set a value to 6 decimal places:

//
float lat = 52.977900;
exif_set_rational(entry->data, FILE_BYTE_ORDER, { (unsigned)(lat * 1000000.0), 1000000 });
//

So now, given a latitude value in degrees, minutes and seconds all we have to do is create a EXIF_TAG_GPS_LATITUDE tag and add a rational for each. Imagine that we want to encode 52, 58, 44.44 then the tag data will then end up looking like this:

efix gps latitude format degrees minutes seconds

This is all very well but I don’t normally bother holding minutes and seconds in my code, instead I preferr to use a degree value to many decimal places, e.g. 52.97790, no problem, this is where our rational value 0 / 1 comes in handy – it can be represented as follows:

exif gps latitude format decimal degrees

So wrapping all of this up, here is some example code that sets a decimal degree value for latitude:

/*
 * create_tag() is from the write-exif.c sample code that is floating
 * around the interweb - with thanks to whoever created it!
 */
/* Create a brand-new tag with a data field of the given length, in the
 * given IFD. This is needed when exif_entry_initialize() isn't able to create
 * this type of tag itself, or the default data length it creates isn't the
 * correct length.
 */
static ExifEntry *create_tag(ExifData *exif, ExifIfd ifd, ExifTag tag, size_t len)
{
	void *buf;
	ExifEntry *entry;
	/* Create a memory allocator to manage this ExifEntry */
	ExifMem *mem = exif_mem_new_default();
	/* Create a new ExifEntry using our allocator */
	entry = exif_entry_new_mem (mem);
	/* Allocate memory to use for holding the tag data */
	buf = exif_mem_alloc(mem, len);
	/* Fill in the entry */
	entry->data = buf;
	entry->size = len;
	entry->tag = tag;
	entry->components = len;
	entry->format = EXIF_FORMAT_UNDEFINED;
	/* Attach the ExifEntry to an IFD */
	exif_content_add_entry (exif->ifd[ifd], entry);
	/* The ExifMem and ExifEntry are now owned elsewhere */
	exif_mem_unref(mem);
	exif_entry_unref(entry);
	return entry;
}
// Set a decimal degree value with support for 6 decimal places
//
//
  // create our latitude tag, the whole  field is 24 bytes long
  entry = create_tag(exif, EXIF_IFD_GPS, EXIF_TAG_GPS_LATITUDE, 24);
  // Set the field's format and number of components, this is very important!
  entry->format = EXIF_FORMAT_RATIONAL;
  entry->components = 3;
  // Degrees
  float lat = 52.977900;
  exif_set_rational(entry->data, EXIF_BYTE_ORDER_INTEL, { (unsigned)(lat * 1000000.0), 1000000 });
//
//
    

I will probably do another post that details how to write EXIF data into a jpeg image’s header using libEXIF and libJPEG

The google drawing app actually worked quite well!

UDP Broadcast not working when no Default Gateway configured on Linux

udp_broadcastThere are a few pitfalls for the burdened Software Engineer getting going with UDP datagram broadcast. One that I hit this week involved clients not receiving broadcast messages unless a default gateway was configured on the broadcasting host.

Very strange, why should broadcast need a gateway? Well it turns out that it doesn’t, it really just needed to know which network adapter to use when sending (and failing that it uses the gateway).

So to fix this we have a few options, including:

1.) Add a broadcast route to the network device in question (e.g. in /etc/network/interfaces), or

2.) Update the UDP code so that the outgoing network device is specified – which can be done like this (to target eth0):

  //
  char buffer[]="eth0";
  setsockopt(socket, SOL_SOCKET, SO_BINDTODEVICE, buffer, 5);
  //

PS, I think this may be non-portable an may only work on Linux. Some more info here.

A Machine Vision Engineer’s take on Fifty Shades of Grey by James Mahon

A Machine Vision Engineer’s take on Fifty Shades of Grey by James Mahon (generated by pure C source code):

50_shades_of_grey_machine_vision

And the source code:

//
//
// 50Shades.cpp : Something for the weekend - JM
//  I am sure that you could to this in abaout 6 lines of Python, but here it is in olde C
//
#include 
#include 
#include 
#define IMAGE unsigned char
void draw_grey_box ( IMAGE *vram, int x_size, int ix, int iy, int dx, int dy, IMAGE grey )
{
	unsigned char *ptr;
	int   y;
	for ( y = iy; y < iy+dy; ++y ) {
		ptr = vram + ix + y * x_size;
		memset ( ptr, grey, dx );
	}
}

int save_any_pgm_image2 ( IMAGE *ram, char *file, char *com, int x1, int y_1, int x2, int y2, int X_SIZE )
{
	int    dx, y;
	IMAGE  *ptr;
	FILE   *fd;
	dx = x2-x1;
	if ( (fd = fopen ( file, "wb" )) == NULL ) {  /*  1.29  */
		printf   ( "save_any_image File <%s> open failed to write\n", file );
		perror   ( "pgm write" );
		return -1;
	}
	fprintf ( fd, "P5 #%s\n%d\n%d\n255\n", com, dx, y2-y_1 );
	for ( y = y_1; y < y2; y++ ) {             /*  Write it all out from gram */
		ptr = ram + x1 + y * X_SIZE;
		if ( fwrite ( ptr, dx, sizeof( IMAGE ), fd ) != sizeof( IMAGE ) ) {
			fclose  ( fd );
			return -2;
		}
	}
	fclose ( fd );
	return y;
}

int main(int argc, char* argv[])
{
	int x_size = 1024, y_size = 768, i, x, y, dx, dy, nx = 10, ny = 5, ix, iy, grey = 1;
	printf ( "Image fILE in c:\\temp\n" );
	IMAGE *vram;
	dx = x_size / ( nx+1 );
	dy = y_size / ( ny+1 );
	vram = (unsigned char *)malloc ( x_size * y_size );
	memset ( vram, 0, x_size * y_size );
	for ( y = 0; y < ny; ++y ) {
		for ( x = 0; x < nx; ++x ) {
			ix = x * dx + dx / 2;
			iy = y * dy + dy / 2;
			draw_grey_box ( vram, x_size, ix, iy, dx*9/10, dy*9/10, grey*5+3 );
			++grey;
		}
	}
	save_any_pgm_image2 ( vram, "c:\\Temp\\50_shades.pgm", "for Valentines day", 0, 0, x_size, y_size, x_size );
	return 0;
}

Quadcopter, Camera and Software to Monitor Coastal Erosion

We are based in Wicklow Town near to the Murrough which is a piece of coastline directly to the North of Wicklow town. Recently a part of the Murrough has been undergoing rather alarming & incredibly fast erosion, differences in the shore line can be observed on a weekly basis. The erosion is occurring mostly at the end of some new rock armour that was quite recently placed to the North of Wicklow town.

Anyway I started to think about how an amateur could monitor the progress of the erosion and maybe plot time-laps type pictures as it progresses, I couldn’t think of any kind of cheap and accessible survey method that could be used to monitor the erosion until I thought of a quadcopter with a downward facing camera along with some offline computer vision software, here’s the kind of thing that I am thinking of:

1.) Fly a quadcopter on a pre-programmed route low over the area of erosion every few days (may need to fly a grid)

2.) Use a downward facing camera on the quadcopter to acquire images of the shore directly below

3.) Use some computer vision software (OpenGL based) to detect features and use them register the images in space to one another yielding a photo-mosaic for each flight.

4.) Use static features (like the rock armour) to register the photo-mosaics from one flight to another.

This may yield a photo-mosaic of the coast from each flight which can be spatially registered to one another (using static features). This could allow us to accurately monitor the erosion as a function of time in detail along this section of coast.

Now if only I had some spare time (oh and a quadcopter)….

#AndNowBackToWork

Affordable logic Analyzer for the jobbing Software Engineer

Today I am working on some embedded code that generates lighting and camera triggers for a computer vision application that I am consulting on, my main thought today is that I am so glad to have my little USB logic analyser from Saleae Logic.

I can remember when the most basic logic analyzer has huge and cost thousands! I have no connections to the company, I just think its a great device and when you are trying to debug camera and lighting synchronization a tool like this invaluable!

computer_vision_pulse_train

A good / recommended JSON serializer for .NET

One of the great advances in software engineering over the last while is the rise of Json over the more ungainly and awkward XML. It is even making inroads in the more conservative .NET world.

In terms of working with Json in .NET, in my opinion, you can’t really do much better than NewtonSoft’s Json.NET lib, it’s great stuff and makes working with Json .NET very easy (as it should be).

Whatever you do don’t be tempted to use Microsoft’s default Json support as its very awkward and removes much of the advantage of using Json in the first place.

So many thanks to James Newton-King for a great library!

You can add this library to your Visual Studio project via the package manager console bu issuing the following command:

Install-Package Newtonsoft.Json

Happy Json-ing!!