Posts

Accuracy of the various Windows software timers

This (admittedly quite old) article provides an interesting comparison of the accuracy of the various software timers available under windows, it looks like the Multimedia timers are still the best despite apparently being frowned upon my Microsoft.

http://omeg.pl/blog/2011/11/on-winapi-timers-and-their-resolution/

I plan to trial camera acquisition using a software trigger, and in order to obtain as accurate and reliable a trigger as possible I want to get as close to a timer interrupt handler as windows will allow, in the hopes that it will be ‘good enough’ for my computer vision application. For a really reliable trigger time-base I normally use a hardware trigger generated by a micro-controller or similar, but this time as the timing constraints aren’t as onerous I have decided to see if I can get away without one, so we will see…. I guess I will have to test the various options and see which is best and if any of them are (generally) within spec..

Image Storage and Indexing for Machine Vision Images

Every Software Engineer needs a hobby – to this end I have been toying with an idea for the last while.

There are many machine vision and computer vision applications that capture images from cameras and store them on disk. These applications can generate so many images that working with them can be quite difficult. For example consider an application that acquires from two cameras each acquiring at 30 frames per second – this application will save 216K images per hour, a 5 hour run would generate 1 million images!

Very often the images will be stored on a file system (local or networked) in some sort of hierarchical directory structure. Using a file system is a very efficient way of storing images, database systems (Relational or NoSQL) don’t offer many advantages and indeed can have associated disadvantages.

But how can we effectively work with so many images, we have possibly millions of images sitting in a set of directories, how can we interact with them and efficiently and query them based on attributes that are interest to us so the we can perform more analysis?

For example consider this set of (contrived) image queries:

Give me all of the images:

+ from camera 1
+ from camera 1 acquired on Sunday between 13:00 and 13:10
+ whose file size > 1MB
+ acquired within 100 meters of this GPS location
+ that have an average brightness > 63 Grey levels

Some people have attacked this image query problem by using a relational database to store image meta-data, if designed well this can allow for efficient image retrieval, however it seems to me that a schema-less approach is a better fit for images with dynamic attributes and I like the idea of not being tied down to any particular database technology and all of the baggage that comes with it.

So my idea is to start out on the road of implementing (for fun) a simple image indexing system for rather large sets of images, it will have an associated tool set, API and maybe even a query language in the future.

The system will:

Allow indexing of large numbers of images in arbitrary hierarchical directory structures

Index images based on standard attributes such as:

+ Acquisition Date/Time
+ Name
+ Source (e.g. camera)
+ Type
+ Size
+ Bit Depth
+ Exif Data, e.g.:
–> Location
–> Author
–> Acquisition parameters (aperture, exposure time etc.)
+ Etc.

Index images optionally based on Computer Vision metrics, e.g.
+ Brightness
+ Sharpness
+ Etc.

Allow users to define their own attributes for indexing, e.g.:
+ Define image attributes based on an OpenCV algorithm
+ Define attributes based on the contents of the image fie name.

The system will:

+ Have no dependencies on technologies such as Database systems etc.
+ Be cross platform

To get the ball rolling and so that we can say the first sod has been turned, here is some (naive) python which scans a directory tree of images and creates a flat CSV file of the image name, path and size:

Run it like this:

Once the directory tree has been walked and the CSV file generated we can use the following script to query images:

This allows us to run queries like this:

This will quickly list the images whose file size > 76000 bytes and whose name contains ‘_43’

This is a really simple first step but it does demonstrate how even a flat ‘index’ of attributes can be of great use.

Next Step:

+ Add more image attributes to the CSV file

Software Algorithm Complexity Cheat-Sheet

Software Algorithm Complexities – Having trouble remembering your O(n)’s from your O(log N)’s, not to worry I just found this very handy complexity cheat-sheet –

http://bigocheatsheet.com/

I am currently doing some research into possible indexing mechanisms for very large sets of images in Computer Vision applications, so this sheet is very handy, thanks Eric!

Quadcopter, Camera and Software to Monitor Coastal Erosion

We are based in Wicklow Town near to the Murrough which is a piece of coastline directly to the North of Wicklow town. Recently a part of the Murrough has been undergoing rather alarming & incredibly fast erosion, differences in the shore line can be observed on a weekly basis. The erosion is occurring mostly at the end of some new rock armour that was quite recently placed to the North of Wicklow town.

Anyway I started to think about how an amateur could monitor the progress of the erosion and maybe plot time-laps type pictures as it progresses, I couldn’t think of any kind of cheap and accessible survey method that could be used to monitor the erosion until I thought of a quadcopter with a downward facing camera along with some offline computer vision software, here’s the kind of thing that I am thinking of:

1.) Fly a quadcopter on a pre-programmed route low over the area of erosion every few days (may need to fly a grid)

2.) Use a downward facing camera on the quadcopter to acquire images of the shore directly below

3.) Use some computer vision software (OpenGL based) to detect features and use them register the images in space to one another yielding a photo-mosaic for each flight.

4.) Use static features (like the rock armour) to register the photo-mosaics from one flight to another.

This may yield a photo-mosaic of the coast from each flight which can be spatially registered to one another (using static features). This could allow us to accurately monitor the erosion as a function of time in detail along this section of coast.

Now if only I had some spare time (oh and a quadcopter)….

#AndNowBackToWork

Integrating Intercom to cloud App keeps me out of trouble… (ARM NEON)

I have been busy over the last few days integrating Intercom into a client’s cloud app. I have integrated quite a few SaaS systems of late (and written not a few REST interfaces myself) and I am quite enjoying this integration as the data model is good and the REST interface is reasonably easy to use!

So far so good, and at least it keeps me out of trouble, the specific trouble I have in mind is to attempt to implement a visual programming front-end for the ARM NEON SIMD instructions using the very exciting noFlo Javascript library!!

The ARM NEON instructions are very powerful SIMD instructions that are very useful for optimising image processing and computer vision tasks on ARM devices. They are however quite difficult to use as each instruction has so many variants for the different input and output element types. I reckon it could be a problem well suited to visual/graph flow programming and as far as I can see the most capable people at those races are those in the noFlo team – I am getting seriously excited about their work!

Anyway back to Intercom for the moment!

Computer Vision in the Cloud? – Amazon makes Nvidia GPUs available

I just came across this article from The Register that mentions that Amazon is making Nvidia GPU cores available:

http://www.theregister.co.uk/2013/11/05/aws_gpu_grid_service/

Makes me think again about the possibilities of doing Computer Vision / Machine Vision in the cloud (taking advantage of GPGPU techniques via OpenCL etc.), however like most of Amazon’s stuff it would probably be way too expensive, interesting though – I must look into it if I ever get the time…