A recipe for building a Web UI using MQTT for your IoT / Embedded App

Cook up a simple & tasty web UI for your embedded project with just a few simple ingredients!  Served on a bed of wholesome Linux goodness on an SBC, (my favourites are: Raspbery Pi or Odroid N2 but substitute alternatives for your own taste) – this is sure to delight family and friends alike!

Ingredients

For the back-end you will need:

Your embedded App  – this is the centrepiece of your creation, it will do some magic and communicate with the UI via MQTT.

python3 – always handy for thickening the sauce!

Flask – tiny & easy to set-up python system for serving your UI’s web pages and static files, perhaps a little lighter on the palette than the ubiquitous Node.js?

Mosquito – for sending MQTT messages between your app and the web UI for updating status and sending commands, MQTT lends a more contemporary taste than the sometimes overused AJAX!

For the front-end you will need:

bootstrap – presentation is soo important!

paho-js – for communicating with MQTT.

HTML & Javascript – the prefect seasoning, no need for some of the heavier spices like Angular.

How To Cook

(details coming soon, hopefully!)

Format std::time_point in ISO 8601 format with fractional seconds / microseconds in C++

A C++ function to format an std::time_point as an ISO 8601 string.

The C++ std chrono stuff is very useful but a bit of a head-wreck!  One of the things I had problems with was how to take an std::time_point value and format it as a string with the fractional seconds / microseconds included.  This sort of time resolution is often required for accurately time stamping machine vision images, especially when acquiring at  high rates from multiple cameras – accurate timestamps allow you to compare images from different cameras that were taken at the ‘same time’.

Anyway if you’re happy to wait for C++20 then you will have access to a format() function; but if you’re more eager to format your time strings now, then here is a function which may fit the bill, it (the function) has to jump through some hoops, but gets there in the end.

Note: This function uses the std::chrono::system_clock but it could be converted (or templated) for other std clocks..

//  Licensed under the Apache License, Version 2.0 (the "License");
//   you may not use this file except in compliance with the License.
//   You may obtain a copy of the License at
//
//       http://www.apache.org/licenses/LICENSE-2.0
//
//   Unless required by applicable law or agreed to in writing, software
//   distributed under the License is distributed on an "AS IS" BASIS,
//   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
//   See the License for the specific language governing permissions and
//   limitations under the License.
/////////////////////////////////////////////////////////////////////////////////
// Format an std::time_point as an ISO 8601 string with fractional seconds to 6
// decimal places, e.g. 2014-08-30T08:18:51.867479
// Warning will not work for any date/times before the start of the UNIX epoch.
//
#include 
inline std::string to_iso_8601(std::chrono::time_point t) {
	// convert to time_t which will represent the number of
	// seconds since the UNIX epoch, UTC 00:00:00 Thursday, 1st. January 1970
	auto epoch_seconds = std::chrono::system_clock::to_time_t(t);
	// Format this as date time to seconds resolution
	// e.g. 2016-08-30T08:18:51
	std::stringstream stream;
	stream << std::put_time(gmtime(&epoch_seconds), "%FT%T");
	// If we now convert back to a time_point we will get the time truncated
	// to whole seconds 
	auto truncated = std::chrono::system_clock::from_time_t(epoch_seconds);
	// Now we subtract this seconds count from the original time to
	// get the number of extra microseconds..
	auto delta_us = std::chrono::duration_cast(t - truncated).count();
	// And append this to the output stream as fractional seconds
	// e.g. 2016-08-30T08:18:51.867479
	stream << "." << std::fixed << std::setw(6) << std::setfill('0') << delta_us;
	return stream.str();
}

Microsoft Teams stuck in a loop saying it needs sign in after a password change

An amusing MS Teams related problem hit me this morning after I changed my Domain Password – The MS Teams App piped up and said it need Sign In, when I hit OK it seemd to restart and then repeated the message, we were in an infinite loop of fun!

Now as much as I can happily live without Teams bugging me, I unfortunately needed it for a conference call so I had to coax it back to life.  What worked for me in the end was to wait until Teams came up with its message and then right-clicked on the teams icon in the task bar and chose the ‘Sign Out’ option.  I then shutdown and restarted Teams and everything was OK again.

 

Launching Flask on Port 80 without using sudo

Flask is a great python based HTTP server that’s really small and easy/fast to setup, it is really useful for deploying small Web based User Interfaces for IoT type devices.  By default flask will attach itself to port 5000.  To get my flask script to attach to port 80 I use:

if __name__ == '__main__':
    app.run(host='0.0.0.0', port=80, debug=True)

This works fine, the only problem is that in order to successfully attach to port 80 the script must be run as root, so instead of just running:

./my_ui.py

I have to run:

sudo ./my_ui.py

Which isn’t great.

To get around this problem I used the tool: authbind

To Install:

sudo apt-get install authbind

And configure it for access to port 80:

sudo touch /etc/authbind/byport/80
sudo chmod 777 /etc/authbind/byport/807

(Not sure if the very loose 777 permission is required, must experiment)

Now I just have to launch my flask UI script using authbind and it will take care of the script’s permissions to bind to port 80 and I can connect from a browser:

authbind -deep python3 ./ui.py

More info on authbind here

Octave – Can’t scroll Window, Workaround

Strange problem with Octave (my version is 4.4.0), in the GUI I can’t scroll the ‘Command Window’, so if some code outputs lots of info I can’t scroll up to see it all, the window keeps jumping to the bottom as I try to scroll!

I am not sure why its happening, but a workaround is to enter the ‘pause’ command within the window, once pause executes I can scroll to my heart’s content.  I  then hit ctrl-c to exit pause.

It is possible that installing a newer version of Octave would fix this but I am not bothered to upgrade at the moment as all of the Maths bits seem to work very well!

 

Paho Javascript Client – Figure out received message’s MQTT topic

When using the Paho Javascript client from MQTT; when a message arrives via client.onMessageArrived(), how can we figure out the message’s topic.

This threw me for a bit as the documentation for the message object doesn’t mention ‘topic’ at all – but it turns out that the topic name is stored in the message.destinationName field! (panic over)

KDevelop hangs during C++ build on ARM Odroid N2

I have been using Hardkernel’s new Odroid N2 to develop a computer vision system with multiple Basler USB3 cameras. I have been using KDevelop on the Odroid to engineer the C++ code and in general all has been going very well (the Odroid N2 is a fantastic device). I did however hit an occasional problem where KDevelop would cause the Odroid to ‘hang’ during a build – especially when re-building a lot of files.

On investigation it seemed like KDevelop was just using up too much memory and putting the Odroid into a very bad place, sometimes the system would free up after a (long) period of time but most often it wouldn’t, it just ground to a halt thrashing memory (I presume). I am using the boost libraries and I think that a lot of IDEs and build systems have problems with boost as it’s very big! (Clion does especially!)

Anyway the solution was to limit how many parallel build instances (or jobs as it calls them) that KDevelop can run, to do this open the ‘Project / Open Configuration..’ menu and click on the ‘Make’ tab in the left-hand column, now check the ‘Override number of simultaneous jobs’ checkbox, and enter a number in the ‘Number of Simultaneous jobs’ box. A value of 3 works well for me, the builds are a little slower but no longer hang the system! I might try increasing it to see if I can get away with quicker builds…

A faster alternative to the very slow GetPixel() and SetPixel() for .Net System.Drawing.Bitmap

Anybody who works with images often have probably come across .Net Bitmaps (System.Drawing.Bitmap) with their staggeringly slow GetPixel() and SetPixel() methods. Now, if you are going to work directly with images then you’re probably in the wrong place if you are using C# and .Net. However, sometimes you may want to do a small amount image analysis or manipulation from within .Net without the pain of having to pull in any other image libraries – but you find that GetPixel() and SetPixel() are just way too slow to use!

Now there is a faster way to access or manipulate the pixel data stored in a .Net Bitmap, that is to lock it (using LockBits()) and then directly access the raw image data in memory – you unlock it when you’re finished. This method is a lot faster than using Get/SetPixel() but is quite complicated to implement and it’s very messy to look at!

To get around this problem, and to avoid littering my code with gibberish I have written a bitmap wrapper class that wraps a bitmap, locks it, and provides it’s own GetPixel() and SetPixel() functions with which the original bitmap’s image data can be accessed. Using this class you can get fast access to a bitmap while using the familiar Get/SetPixel() paradigm – in this way it should act as a fairly easy drop in replacement for accessing the Bitmap objects directly.

The class is called BmpPixelSnoop and it is used like this:

// Calculate a simple sum over all of the pixels
// in the snooped bitmap. bitmap is a valid Bitmap object
long snoopSum = 0;
// Create a BmpPixelSnoop wrapper for bitmap
using (var snoop = new BmpPixelSnoop(bitmap))
{
    for (int j = 0; j != snoop.Height; j++)
    {
        for (int i = 0; i != snoop.Width; i++)
        {
            // We call GetPixel() on snoop rather
            // than bitmap as it's much faster!
            var col = snoop.GetPixel(i, j);
            snoopSum += col.R +
                        col.G +
                        col.B;
        }
    }
}

First a BmpPixelSnoop object is created to wrap the bitmap, GetPixel() and SetPixel() can then be called on it. When the BmpPixelSnoop object is destroyed (on leaving the using() block) the original bitmap will be unlocked. It is important to note, that while the bitmap is being snooped the original bitmap object cannot be accessed as it’s locked! Currently BmpPixelSnoop only works for bitmaps with a Pixel Format of PixelFormat.Format32bppArgb which is the default format for Bitmaps (if you don’t specify an alternative when creating them).

So for a little extra complication you get easy & fast access to the bitmap data – but how much faster than the native Get/SetPixel() is it? My (non scientific) tests seem to indicate that it’s about 10 times faster, which is fast enough for simple imaging tasks. It is still quite inefficient however, this is a result of wanting to provide the same Get/SetPixel() interface as System.Drawing.Bitmap – for example, GetPixel() always returns all of the pixel data even if you just want to access the red component and hence is slower than it needs to be in this case. I may add extra accessor methods in the future to cater for other usage patterns and greater efficiency.

The code can be found in the git-hub repo: https://github.com/kgodden/DotNetPixelSnoop.

The class is defined in BmpPixelSnoop.cs, there is also some test code to check correctness and performance in Program.cs.

Here is the code:

//   Copyright 2019 Kevin Godden
//
//   Licensed under the Apache License, Version 2.0 (the "License");
//   you may not use this file except in compliance with the License.
//   You may obtain a copy of the License at
//
//       http://www.apache.org/licenses/LICENSE-2.0
//
//   Unless required by applicable law or agreed to in writing, software
//   distributed under the License is distributed on an "AS IS" BASIS,
//   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
//   See the License for the specific language governing permissions and
//   limitations under the License.
using System;
using System.Drawing;
using System.Drawing.Imaging;
/// 
/// Wraps a System.Drawing.Bitmap and provides faster
/// GetPixel() and SetPixel() functions for pixel access.
/// 
/// NB While the snoop object is in scope the wrapped
/// bitmap object is locked and cannot be used 
/// as normal.  Once you have finished snooping
/// on a bitmap object, dispose of the snooper to
/// unlock the bitmap and gain normal access to 
/// it again, it is best to employ the 'using' keyword
/// to effectivly manage the snooper's scope as follows:
/// 
/// 
/// using (var snoop = new BmpPixelSnoop(myBitmap))
/// { 
/// 
///     // Snoop away!
///     var pixel = snoop.GetPixel(0, 0);
///     
/// } // Snoop goes out of scope here and bitmap is unlocked
/// 
/// This class is marked as 'unsafe' so to use it in your project
/// you must have the 'Allow unsafe code' setting checked in the
/// project settings.
/// 
/// 
unsafe class BmpPixelSnoop : IDisposable
{
    // A reference to the bitmap to be wrapped
    private readonly Bitmap wrappedBitmap;
    // The bitmap's data (once it has been locked)
    private BitmapData data = null;
    // Pointer to the first pixel
    private readonly byte* scan0;
    // Number of bytes per pixel
    private readonly int depth;
    // Number of bytes in an image row
    private readonly int stride;
    // The bitmap's width
    private readonly int width;
    // The bitmap's height
    private readonly int height;
    /// 
    /// Constructs a BmpPixelSnoop object, the bitmap
    /// object to be wraped is passed as a parameter.
    /// 
    /// The bitmap to snoop
    public BmpPixelSnoop(Bitmap bitmap)
    {
        wrappedBitmap = bitmap ?? throw new ArgumentException("Bitmap parameter cannot be null", "bitmap");
        // Currently works only for: PixelFormat.Format32bppArgb
        if (wrappedBitmap.PixelFormat != PixelFormat.Format32bppArgb)
            throw new System.ArgumentException("Only PixelFormat.Format32bppArgb is supported", "bitmap");
        // Record the width & height
        width = wrappedBitmap.Width;
        height = wrappedBitmap.Height;
        // So now we need to lock the bitmap so that we can gain access
        // to it's raw pixel data.  It will be unlocked when this snoop is 
        // disposed.
        var rect = new Rectangle(0, 0, wrappedBitmap.Width, wrappedBitmap.Height);
        try
        {
            data = wrappedBitmap.LockBits(rect, ImageLockMode.ReadWrite, wrappedBitmap.PixelFormat);
        }
        catch (Exception ex)
        {
            throw new System.InvalidOperationException("Could not lock bitmap, is it already being snooped somewhere else?", ex);
        }
        // Calculate number of bytes per pixel
        depth = Bitmap.GetPixelFormatSize(data.PixelFormat) / 8; // bits per channel
        // Get pointer to first pixel
        scan0 = (byte*)data.Scan0.ToPointer();
        // Get the number of bytes in an image row
        // this will be used when determining a pixel's
        // memory address.
        stride = data.Stride;
    }
    /// 
    /// Disposes BmpPixelSnoop object
    /// 
    public void Dispose()
    {
        Dispose(true);
        GC.SuppressFinalize(this);
    }
     /// 
    /// Disposes BmpPixelSnoop object, we unlock
    /// the wrapped bitmap.
    /// 
    protected virtual void Dispose(bool disposing)
    {
        if (disposing)
        {
            if (wrappedBitmap != null)
                wrappedBitmap.UnlockBits(data);
        }
        // free native resources if there are any.
    }
    /// 
    /// Calculate the pointer to a pixel at (x, x)
    /// 
    /// The pixel's x coordinate
    /// The pixel's y coordinate
    /// A byte* pointer to the pixel's data
    private byte* PixelPointer(int x, int y)
    {
        return scan0 + y * stride + x * depth;
    }
    /// 
    /// Snoop's implemetation of GetPixel() which is similar to
    /// Bitmap's GetPixel() but should be faster.
    /// 
    /// The pixel's x coordinate
    /// The pixel's y coordinate
    /// The pixel's colour
    public System.Drawing.Color GetPixel(int x, int y)
    {
        // Better do the 'decent thing' and bounds check x & y
        if (x < 0 || y < 0 || x >= width || y >= width)
            throw new ArgumentException("x or y coordinate is out of range");
        int a, r, g, b;
        // Get a pointer to this pixel
        byte* p = PixelPointer(x, y);
        // Pull out its colour data
        b = *p++;
        g = *p++;
        r = *p++;
        a = *p;
        // And return a color value for it (this is quite slow
        // but allows us to look like Bitmap.GetPixel())
        return System.Drawing.Color.FromArgb(a, r, g, b);
    }
    /// 
    /// Sets the passed colour to the pixel at (x, y)
    /// 
    /// The pixel's x coordinate
    /// The pixel's y coordinate
    /// The value to be assigned to the pixel
    public void SetPixel(int x, int y, System.Drawing.Color col)
    {
        // Better do the 'decent thing' and bounds check x & y
        if (x < 0 || y < 0 || x >= width || y >= width)
            throw new ArgumentException("x or y coordinate is out of range");
        // Get a pointer to this pixel
        byte* p = PixelPointer(x, y);
        // Set the data
        *p++ = col.B;
        *p++ = col.G;
        *p++ = col.R;
        *p = col.A;
    }
    /// 
    /// The bitmap's width
    /// 
    public int Width { get { return width; } }
    // The bitmap's height
    public int Height { get { return height; } }
}

Here is some sample output from the colsone based test program, showing relative times:

Testing GetPixel()
GetPixel() OK
Testing SetPixel()
SetPixel() OK
Testing GetPixel() Speed
Bitmap.GetPixel() took 759ms, BmpPixelSnoop.GetPixel() took 67ms
Testing SetPixel() Speed
Bitmap.SetPixel() took 907ms, BmpPixelSnoop.SetPixel() took 72ms

Boost ASIO Simple UDP Send Packet Example

Update: I have written a simple Fire-And-Forget wrapper class for sending datagrams via UDP can be found here. It handles simple transmission use cases while hiding the (sometimes confusing) boost::asio details. However, if you are interested in the details then read on!

Boost.ASIO is great but if you don’t use it everyday it can be hard to remember how to use it to do even the simplest of things. I have included below a sample of simply sending a packet via UDP (ipv4), see the function called send_message(), this example code aims to be as minimal as it can be:

Those spouting software engineering dogma will often tell you to steer well clear of UDP for the usual, well understood reasons, but for a certain type of application where very low latency is important, it just can’t be beat!!

#include 
#include 
using namespace boost::asio;
//
// Send a string via UDP to the specified destination
// ip addresss at the specified port (point-to-point
// not broadcast)
//
bool send_udp_message(const std::string& message, const std::string& destination_ip,
						const unsigned short port) {
	io_service io_service;
	ip::udp::socket socket(io_service);
	// Create the remote endpoint using the destination ip address and
	// the target port number.  This is not a broadcast
	auto remote = ip::udp::endpoint(ip::address::from_string(destination_ip), port);
	try {
	
		// Open the socket, socket's destructor will
		// automatically close it.
		socket.open(boost::asio::ip::udp::v4());
		// And send the string... (synchronous / blocking)
		socket.send_to(buffer(message), remote);
	
	} catch (const boost::system::system_error& ex) {
		// Exception thrown!
		// Examine ex.code() and ex.what() to see what went wrong!
		return false;
	}
	return true;
}

This is the bare-bones code, no error reporting etc. Also it won’t broadcast, to allow for broadcast you need to include the following two lines and supply a broadcast ip address when calling the function. Be careful if broadcasting a lot of data as it can really overload & mess-up network equipment!

socket_base::broadcast option(true);
socket.set_option(option);

Fixed – KDevelop not stopping at breakpoints on Ubuntu Mate

I couldn’t get KDevelop to stop at breakpoints even on simple ‘hello world’ C++ projects. It appeared that CMAKE_BUILD_TYPE was being correctly set and GDB worked fine from the command-line, but from within kdevelop breakpoints were never respected! I think the problem stemmed from the Cache Value for CMAKE_BUILD_TYPE being empty, this value can be seen in the Project / Open Configuration… menu:

kdevelop breakpoint broken

Following the advice from this post I added the following into the project’s CMakeLists.txt file:

# Set a default build type if none was specified
if(NOT CMAKE_BUILD_TYPE AND NOT CMAKE_CONFIGURATION_TYPES)
  message(STATUS "Setting build type to 'Debug' as none was specified.")
  set(CMAKE_BUILD_TYPE Debug CACHE STRING "Choose the type of build." FORCE)
  # Set the possible values of build type for cmake-gui
  set_property(CACHE CMAKE_BUILD_TYPE PROPERTY STRINGS "Debug" "Release"
    "MinSizeRel" "RelWithDebInfo")
endif()

[Originally from here]

I ran clean & rebuild etc. and then cache value was set correctly to ‘Debug’ and the debugger happily stops at breakpoints!

I hope this post helps folks as I spent _ages_ trying to get the breakpoints to work!!