Web2py and python – Change to model definition may break web site

In Web2py if you want to change a model’s definition, you can first of all change the model’s software definition via its python code and then, (depending on your migration settings), the next time you launch your web app, web2py will attempt to alter your database structure to match your updated model.

 

If this works, then it works and your database will reflect your new model, however sometimes web2py can’t make the necessary changes and your app may not subsequently load any more at all. If this happens then I have found the best way to fix the app is to delete all of the files in the app’s database directory – the next time you launch your app web2py will recreate your DB from scratch and it should launch again.

 

Now this is fine assuming that your database didn’t have loads of important data in it that you didn’t want to loose. So, to get around this problem, before trying to change a model’s definition I export all of the data in the database to a file (using a tool like this) so that if the problem happens and I have to get web2py to recreate the database ‘in its own image’ I can later reimport the previously saved data…

 

C# Create a Cognex 8bit image (CogImage8Grey) from an 8bit Grayscale image array (byte[])

Here is some code that shows how to create an 8bit grayscale cognex image (CogImage8Grey) from an 8bit raw image stored as a byte array (byte[]). This type of memory messing is difficult in .NET at the best of times and it’s just a pity the the cognex library doesn’t help much more than it does.

 

I also have a feeling that the cognex library is doing more copying than it strictly needs to, but in true style its software documentation does not detail if, or when, it copies image data (or much else for that matter!) So I copy the image data from the byte array into a malloc’ed buffer before creating the cognex image.

 

First we have to define a SafeBuffer through which cognex can free up the allocated memory when it is finished with it, to do this we can derive a class from SafeBuffer like this:
[crayon lang=”csharp”]
///

/// A wrapper around malloc so that FreeHGlobal() is called
/// when the object is disposed.
///

class SafeMalloc : SafeBuffer
{
///

/// Allocates memory and initialises the SaveBuffer
///

///The number of bytes to allocate public SafeMalloc(int size) : base(true)
{
this.SetHandle(Marshal.AllocHGlobal(size));
this.Initialize((ulong)size);
}

///

/// Called when the object is disposed, ferr the
/// memory via FreeHGlobal().
///

///
protected override bool ReleaseHandle()
{
Marshal.FreeHGlobal(this.handle);
return true;
}

///

/// Cast to IntPtr
///

public static implicit operator IntPtr(SafeMalloc h)
{
return h.handle;
}
}
[/crayon]

 

Its constructor mallocs the memory and when it is disposed ReleaseHandle() is called, and this frees the memory. I also added a cast to IntPtr so that we can pass it into functions that expect an IntPtr.

 

Now that we have SafeMalloc we can write function to create the cognex image like this:
[crayon lang=”csharp”]
class CognexStuff
{
public ICogImage Convert8BitRawImageToCognexImage(
byte[] imageData, int width, int height)
{
// no padding etc. so size calculation
// is simple.
var rawSize = width * height;

var buf = new SafeMalloc(rawSize);

// Copy from the byte array into the
// previously allocated. memory
Marshal.Copy(imageData, 0, buf, rawSize);

// Create Cognex Root thing.
var cogRoot = new CogImage8Root();

// Initialise the image root, the stride is the
// same as the widthas the input image is byte alligned and
// has no padding etc.
cogRoot.Initialize(width, height, buf, width, buf);

// Create cognex 8 bit image.
var cogImage = new CogImage8Grey();

// And set the image roor
cogImage.SetRoot(cogRoot);

return cogImage;
}
}
[/crayon]

 

This function allocates memory via SafeMalloc, it then copies the raw image data from the input array into this memory. Then CogImage8Root.Initialize() is called passing in a pointer to this memory. not that in this case the image’s stride is the same as its width. Once the CogImage8Root has been initialised we can create a CogImage8Grey image and set the root via a call to SetRoot()!

 

They certainly make you work for it!!

 

If you know that nobody else will be using your image array and are willing to go ‘unsafe'(!) then you could avoid this extra memory copy by pinning the array and getting a pointer to it, you could then pass this pointer directly to the root Initialize() function. In this case you won’t need the SafeMalloc class etc.

 

Thanks to all on this thread for hints on SafeBuffer!

 

C# Save Grayscale byte array Image (byte[]) as Bitmap file (.bmp) example

Here is a quick, dirty and inefficient example of how to save an 8bit Grey scale image stored in a C# byte array as a 32bit bitmap file (.bmp). Saving bitmaps can be quite suprisingly difficult in .NET so I am posting this for future reference!

This code copies each byte (8 bit pixel) in the 8bit image into an array of 32bit pixels (4 bytes per pixel) and then saves it to disk. Note that you have to build your project with the ‘Allow unsafe code’ checkbox checked (go to project properties / Build and you will see the ‘Allow unsafe code’ checkbox.)

public void SaveAsBitmap(string fileName, int width, int height, byte[] imageData)
{
    // Need to copy our 8 bit greyscale image into a 32bit layout.
    // Choosing 32bit rather than 24 bit as its easier to calculate stride etc.
    // This will be slow enough and isn't the most efficient method.
    var data = new byte[width * height * 4];
    int o = 0;
    for (var i = 0; i < width * height; i++)
    {
        var value = imageData[i];
        // Greyscale image so r, g, b, get the same
        // intensity value.
        data[o++] = value;
        data[o++] = value;
        data[o++] = value;
        data[o++] = 0; // Alpha isn't actually used
    }
    unsafe
    {
        fixed (byte* ptr = data)
        {
            // Create a bitmap wit a raw pointer to the data
            using (Bitmap image = new Bitmap(width, height, width * 4,
            PixelFormat.Format32bppRgb, new IntPtr(ptr)))
            {
                // And save it.
                image.Save(Path.ChangeExtension(fileName, ".bmp"));
            }
        }
    }
}

Thanks to all on this thread for the pointers!

Update, 02/2014:

This code will save the 8 bit bitmap in a compressed (but perfectly valid) format, to have your software save it in an uncompressed format take a look at this post.

Machine Vision on the Raspberry Pi anybody?

Here’s is a blog post that has some pictures of the (hopefully) soon to arrive camera for the Pi.

 

http://www.raspberrypi.org/archives/3224

 

It will be interesting to see what cone be done from a machine vision or computer vision point of view with the Pi once cameras are available.

 

Create an SQLExpress database using Visual Studio

This took me a bit of time to figure out – how to create an empty SQL express database without using SQL Mangement Studio (I didn’t have it installed..) but instead using Visual Studio 2010 (and without having to add data sources, model definitions and other such muck into the context of a VS project)

 

Once you have a database you can use Visual Studio’s ‘Server Explorer’ pane to edit the DB and add tables etc. etc. but it isn’t obvious how to create an empty database in the first place, so in the end I did the following:

 

1.) Check that the SQL services are running – start them if necessary.

 

2.) Use Visual Studio to connect to the SQL Express master database – Choose the Tools / Connect to Database menu item. Specify ‘Microsoft SQL Server’ as the data source. Enter ./SQLEXPRESS as the ‘Server name’, or hit the browse button if you the have the browser service running. Leave the database name field blank. Press the OK button and VS should connect to the local master database, you should see it in the ‘Server Explorer’ pane.

 

3.) Now right click on the master database in the server explorer pane and choose the ‘New Query’ menu item. In the query window type:

 

[crayon]
create database DatabaseName
[/crayon]

 

Where DatabaseName is the name of the database that you want to create. Execute the query (press Ctrl – E) – this should create your new database.

 

3.) To connect to your new database repeat step 2.) and specify the database name – Visual Studio should now connect to it. Now you can add tables etc.

 

4.) You can now remove the connection to the master database in the Server Explorer pane if you like.

 

Magento – Cannot set property ‘disabled’ of undefined error in one page checkout

We hit a strange Magento problem yesterday, on one of our sites the one page checkout stopped working. Once the various fields were filled out on the billing details pane and the ‘Continue’ button was pressed, then the checkout would not progress onto the next pane. Debugging on chrome showed that the following JavaScript error was occurring:

 

Uncaught Type Error: Cannot set property ‘disabled’ of undefined.

 

Googling the error produced some hits but no concrete solutions were offered, so we started switching things off one by one to see if we could get rid of the problem…

 

It turned out that removing the twitter feed from the footer fixed the problem…. who knows why, I’ll try to debug it tomorrow…

 

I hope this helps somebody in a similar situation.

 

Realex – Your transaction has been successful but there was a problem connecting back to the merchant’s web site

If your are trying to get realex e-payment integration working on your website and are getting the following message at the end of the realex redirect process:

 

Your transaction has been successful but there was a problem connecting back to the merchant’s web site. Please contact the merchant and advise them that you received this error message. Thank you.

 

Then the good news is that most of you integration is working and its just the very last step that’s failing, this means that you have sent to Realex correct values for: Merchant Id, Shared Secret, Account Id, Request URL etc. etc.

 

Now, nine times out of ten the above error is caused by a badly configured Return or Callback URL, this is a URL that you must tell Realex about by phone or email, Realex redirects back to this URL once the transaction is complete. So there are 3 possibilities why the redirect is failing:

 

1.) You haven’t yet told Realex what your return URL is. You need to find out what the URL should be and let Realex know, phoning them is often the most efficient route. The URL will depend on which CMS you are using etc. Ask Google for hints.

 

In ubercat you can get the URL by going to the Realex settings page (Store administration / Configuration / Payment settings / Overview / Payment Methods / Credit Card [realex] settings) and copying the URL from the ‘Realex Callback Page:’ field.

 

Details about the Magento return URL can be found here.

 

2.) You have told realex what the URL is but they forgot to configure your account or made a mistake while configuring it – this happens more often than you might think, phone up Realex and get them to confirm that the return URL is set correctly – double check the configured URL it in detail! Do you need to add or remove the www.?

 

Hopefully it is only a matter of short time before you get everything working as it should!

 

+441315531100 keeps phoning me multiple times a day…

I have been getting 2 or 3 calls to my mobile from +44 131 5531100 for the last few weeks. The first time I received a call I was suspicious and wary enough not to answer and I haven’t answered any of the calls since as the caller never leaves a voice-mail! It’s all getting rather annoying.

 

I did some googling on the number and after reading this post (and the comments that follow it), I am very glad that didn’t answer the calls!

 

I have the number diverting to my voicemail as I can’t directly block numbers on my mobile.

 

If you start to get calls from this number it is up to you whether you answer and engage, but before you do make sure to do some research and read the above article first!

 

Visual Studio, TFS – Cheeps checking files out, There appears to be a discrepancy between the solution’s source control information about some project(s)

Over the last few hours I started seeing an annoying problem with Visual Studio and TFS on a geographically distributed software project that I am working on, Visual Studio kept on automatically checking out files that hadn’t been edited, I also started getting the following message before the files were automatically checked out:

 

“There appears to be a discrepancy between the solution’s source control information about some project(s) and the information in the project file(s).
Inbox”

 

After some investigation we figured out what the problem was – it was annoying problem, but in this case quite easy to fix.

 

When the TFS server was set-up originally we were all told to access it by its IP address over a partially set-up VPN. Now once the VPN was properly set-up, project contributors started to use the TFS server’s host-name instead of the IP address (as they should).

 

However, not all of the contributors were notified of the change and continued to use the IP address. Visual studio would update any files that had TFS info within them, checking them out and altering them to use either the TFS host name or IP address depending on the particulars of the local Visual Studio TFS set-up – hence all the trouble and all of the annoying error messages!

 

As soon as we changed from using the IP address to using the host name the problem went away.

 

It may well appear again however, as soon as someone else who uses the IP address instead of the host name checks in some files…

 

TFS is quite a good source control system but every now and again you can hear worrying echoes of its ignoble Visual Source Safe history and some of the hair-brained and crazy stuff that comes with it!

 

Visual Studio – Error The type or namespace name ‘CodedUITest’ could not be found

If you are trying to build some code with Visual Studio 2010 and get build errors like this:

 

The type or namespace name ‘CodedUITest’ could not be found …

 

Then it is probably because you are building with an edition of Visual Studio that is not Premium or Ultimate – the professional edition does not have the UI Test Tools built in.

 

If you are lucky you will be able to just remove the offending test project(s) with the broken references from the solution and build the rest – typically it will be only the test projects that won’t build.

 

To do this, choose the Build / Configuration manager… menu and un-check under the ‘Build’ column for any test projects that aren’t building.

 

Or shell out for an upgrade of Visual Studio!