Using ‘Mod’ on (small) Embedded Systems while Avoiding Time Penalties

The Mod/Modulo (% in C) operator is incredibly useful in many situations, it calculates the remainder from an integer division, for example 10 Mod 3 = 1 (10 Div 3 = 3, 3 * 3 = 9, 10 – 9 = 1).

Although you can happily call the % operator from your embedded C program it is important to be aware that it is typically implemented using integer division and multiplication, which is fine if your embedded system support these in hardware, if not they will be implemented in software sometimes resulting in huge time penalties (for example, on many PIC chips).

Often this won’t be a problem, but if you’re programming a real-time system these delays can be devastating! A big sting-in-the-tail from an innocent looking operator %

So what can be done:

a.) Avoid using % in your time-sensitive code, perhaps use running sums and remainders instead.

b.) Use a power-of-2 in your Mod call instead.

If the value of the right-hand operator isn’t crucial and both arguments are positive, then switching to a power of 2 will make it much easier and faster to calculate, a power-of-2 Mod can be calculated with an AND instruction.

a % b == a & (b - 1)    // if b is a power of 2 and a, b are both +ve.

Your compiler should figure out that you are using a power-of-2 and automatically carry out the optimisation, but to be safe, check the output assembly code. If it hasn’t optimised then you can replace your code with a & (b – 1)

Python Monty Hall Problem Simulation

The Monty Hall Problem is a very (to me at least) counter-intuitive probability mind-experiment which contorts my brain and fascinates me at the same time, I have been mulling it over the last few weeks and wanted to write a little simulator to see if the numbers come out as predicted (if not expected, and indeed they do!). I can just about understand the probabilistic arguments, but I still find it very confusing, as soon as I think that I grok it – my ‘understanding’ disappears into the night! I am used to being this bamboozled when reading about quantum mechanics or something, but I find it fascinating that such an apparently simple problem can be so deceptively deep!

Summary

There are 3 doors, behind one lies a car, while behind the other two are goats. A player chooses a door at random. Monty opens one of the other doors to show that there is a goat behind it. Monty then asks the player if they would like to stick with their original choice of door or switch to the other un-opened door.

If the player sticks with their door then their chance of winning the car should be 1/3. If the player switches door then their chances of winning the car increases to 2/3!!!

If you are having problems understanding the outcome, I find it helps to imagine that there are a million doors rather than 3. After you choose your door (1/1,000,000 chance of hiding the car) Monty opens up 999,998 doors that hide goats to leave one door still closed. Now which door do you think is most likely to hide the car? The one you choose, or the one that Monty avoided opening while he opened all 999,998 other doors?! It seems obvious to me that the other door that Monty left un-opened has a massively higher chance of hiding the car than your original choice! As N reduces to 3 this ‘obviousness’ reduces greatly however! This simulator allows you to experiment with more then 3 doors for this reason.

For more info: https://en.wikipedia.org/wiki/Monty_Hall_problem

This program is an experimental simulator to see what numbers we get when the player decides to stick or switch. It can be found on GitHub.

#! /usr/bin/python
#   Copyright 2019 Kevin Godden
#
#   Licensed under the Apache License, Version 2.0 (the "License");
#   you may not use this file except in compliance with the License.
#   You may obtain a copy of the License at
#
#       http://www.apache.org/licenses/LICENSE-2.0
#
#   Unless required by applicable law or agreed to in writing, software
#   distributed under the License is distributed on an "AS IS" BASIS,
#   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
#   See the License for the specific language governing permissions and
#   limitations under the License.
import random
"""
    Simulates the Monty Hall problem with 3 doors
    The Monty Hall problem is a very (to me at least) counter-intuitive probability mind experiment which
    contorts my brain and fascinates me at the same time. I can just about understand the
    probabilistic arguments, but I still find it confusing.  This program is an experimental
    simulator to see what numbers we get when the player decides to stick or switch.
    If you are having problems understanding the outcome, I find it helps to imagine that there
    are a million doors rather than 3.  After you choose your door (1/1,000,000 chance of hiding the car)
    Monty opens up 999,998 doors that hide goats to leave one door still closed.  Now which door do you think
    is most likely to hide the car? The one you choose, or the one that Monty avoided opening while
    he opened all 999,998 other doors?!  It seems obvious to me that the other door that Monty left un opened
    has a massively higher chance of hiding the car than the original choice!  As N reduces to 3 this
    obviousness reduces greatly however!  This simulation can be run with more than 3 doors to experiment
    with this...
    Summary:
    --------
    There are 3 doors, behind one lies a car, while behind the other two are goats.
    A player chooses a door at random.  Monty then opens one of the other doors to show that
    there is no car behind it.  Monty asks the player if they would like to stick with their
    door or switch to the other un-opened door.
    If the player sticks with their door then their chance of winning the car should be 1/3
    If the player switches door then their chances of winning the car increases to 2/3!!!
    If you want to run the game with more doors to see how the probabilities change
    just change the door_count value below.
    For more info: https://en.wikipedia.org/wiki/Monty_Hall_problem
"""
# Should the player switch their guess or stick
# with their original one?
switch = True
# The number of doors in the game, should be 3 for
# the problem as normally stated.
door_count = 3
# The number of times to run the simulation
game_count = 10000
# Should we print lots of messages?
# If door_count is large change this to True to
# make simulation run faster.
quiet = False
def pick_random_door():
    """
    Pick a door at 'random', doors are 0-indexed so this will
    return a number between 0 and door_count - 1
    :return: the chosen door
    """
    return random.randint(0, door_count - 1)
def pick_monty_doors(car_door, players_guess):
    """
    Pick the doors for Monty to open, of the number
    of doors is 3 then Monty opens up the first door that doesn't
    have a car and that play hasn't already chosen...
    If door_count > 3 then Monty will open up all of the doors
    that hide goats avoiding the door that has been already picked
    :param car_door: The door behind which the car resides
    :param players_guess: The door which the play choose
    :return: A list of the indices of the doors that Monty opens
    """
    monty_doors = []
    for d in range(0, door_count):
        if len(monty_doors) == door_count - 2:
            break
        if d == car_door:
            continue
        if d == players_guess:
            continue
        monty_doors.append(d)
    return monty_doors
def pick_other_door(players_guess, monty_doors):
    """
    The player has chosen to switch their chosen door, so we
    need to pick another one for them.  We pick the door that
    a.) Doesn't match the player's original guess
    b.) Wasn't chosen by Monty
    :param players_guess:  The door that the player originally picked
    :param monty_doors: The doors that Monty has opened (usually just 1)
    :return: The other door's index
    """
    for d in range(0, door_count):
        # Can't switch to the player's original guess
        if d == players_guess:
            continue
        # Don;t pick a door that Monty has opened
        if d in monty_doors:
            continue
        return d
def run_game():
    """
    Run a single game
    :return:  win/loose --> True/False
    """
    # Pick the door with the car
    car_door = pick_random_door()
    if not quiet:
        print("Car is behind door %d" % car_door)
    # Player chooses a door
    players_guess = pick_random_door()
    if not quiet:
        print("Player has guessed door %d" % players_guess)
    # Monty opens up a door that doesn't have a
    # car behind it.
    monty_doors = pick_monty_doors(car_door, players_guess)
    if not quiet:
        print("Monty opens: " + str(monty_doors))
    # Does the player switch doors after Monty
    # opens his??
    if switch:
        players_guess = pick_other_door(players_guess, monty_doors)
        if not quiet:
            print("Player switches to door %d" % players_guess)
    if not quiet:
        if players_guess == car_door:
            print("Player wins!")
        else:
            print("Player looses ;-(")
    return players_guess == car_door
def run_games():
    wins = 0
    game = 0
    while game < game_count:
        game += 1
        # Run a game and see if we win!
        if run_game():
            wins += 1
    print("Games: %d" % game)
    print("Wins: %d" % wins)
    print("------------------------------")
    if wins == 0:
        print("you didn't win any games!")
    else:
        win_rate = wins / game
        print("Win rate: %f" % win_rate)
if __name__ == "__main__":
    run_games()

boost::split – warning C4996: ‘std::copy::_Unchecked_iterators::_Deprecate’: Call to ‘std::copy’ with parameters that may be unsafe

If you’re using the very handy boost::split() on Visual Studio, then you may run into the following annoying warning:

warning C4996: ‘std::copy::_Unchecked_iterators::_Deprecate’: Call to ‘std::copy’ with parameters that may be unsafe – this call relies on the caller to check that the passed values are correct. To disable this warning, use -D_SCL_SECURE_NO_WARNINGS. See documentation on how to use Visual C++ ‘Checked Iterators’

It’s a big warning that rightly litters the compiler’s output!!

Everything’s OK, it’s just VS whinging about std::copy() being called with pointers. As the warning mentions, you can disable the warnings by defining _SCL_SECURE_NO_WARNINGS, its probably best to define it only for the affected files and not your whole project however!

Another way is to explicitly instruct that warning 4996 be ignored in the code like this:

//
// Suppress annoying warning for boost:split()
//
#if defined(_MSC_VER) && _MSC_VER >= 1400 
#pragma warning(push) 
#pragma warning(disable:4996) 
#endif 
#include 
#if defined(_MSC_VER) && _MSC_VER >= 1400 
#pragma warning(pop) 
//

More Info

Remote Desktop Connection Closes / Disappears silently soon after Connecting

I had a strange problem with my Windows 10 Remote Desktop connections over the last few days, soon after connecting the connection would simply disappear without any message or warning, this would happen typically under a minute after connecting to the remote machine – amusing, but quite frustrating if you wanted to get anything done!

I searched around and couldn’t find any mention of similar things happening to other folk, but there was some mention of cleaning out stale connection data, so I managed to make the problem go away by going to the Remote Desktop connection manager and deleting the ‘Saved Credentials’ from the connection. When I reconnected everything was fine and has remained so ever since!

Connections from multiple Browsers to Mosquitto MQTT via Web Sockets not Working

If you are using the paho javascript MQTT client library to connect to MQTT via web sockets, you might notice that you can’t connect from multiple browser sessions at the same time. This may be because you are using the same client ID for each connection – you should use a different client ID for each connection.

client = new Paho.MQTT.Client('192.168.1.123', 9001, 'some-client-id');  // Doesn't work

To get around this you could generate a random client Id like this:

client = new Paho.MQTT.Client('192.168.1.123', 9001, Math.random().toString());

This generates a thoroughly meaningless/ugly client Id, but it works…

There is some worry that using an approach like this can lead to a buildup of stale sessions on the server side as the browser will pass a completely different client Id on each connection/refresh but I am not sure, I think this may only affect persistent sessions, will keep an eye on it…

A recipe for building a Web UI using MQTT for your IoT / Embedded App

Cook up a simple & tasty web UI for your embedded project with just a few simple ingredients!  Served on a bed of wholesome Linux goodness on an SBC, (my favourites are: Raspbery Pi or Odroid N2 but substitute alternatives for your own taste) – this is sure to delight family and friends alike!

Ingredients

For the back-end you will need:

Your embedded App  – this is the centrepiece of your creation, it will do some magic and communicate with the UI via MQTT.

python3 – always handy for thickening the sauce!

Flask – tiny & easy to set-up python system for serving your UI’s web pages and static files, perhaps a little lighter on the palette than the ubiquitous Node.js?

Mosquito – for sending MQTT messages between your app and the web UI for updating status and sending commands, MQTT lends a more contemporary taste than the sometimes overused AJAX!

For the front-end you will need:

bootstrap – presentation is soo important!

paho-js – for communicating with MQTT.

HTML & Javascript – the prefect seasoning, no need for some of the heavier spices like Angular.

How To Cook

(details coming soon, hopefully!)

Format std::time_point in ISO 8601 format with fractional seconds / microseconds in C++

A C++ function to format an std::time_point as an ISO 8601 string.

The C++ std chrono stuff is very useful but a bit of a head-wreck!  One of the things I had problems with was how to take an std::time_point value and format it as a string with the fractional seconds / microseconds included.  This sort of time resolution is often required for accurately time stamping machine vision images, especially when acquiring at  high rates from multiple cameras – accurate timestamps allow you to compare images from different cameras that were taken at the ‘same time’.

Anyway if you’re happy to wait for C++20 then you will have access to a format() function; but if you’re more eager to format your time strings now, then here is a function which may fit the bill, it (the function) has to jump through some hoops, but gets there in the end.

Note: This function uses the std::chrono::system_clock but it could be converted (or templated) for other std clocks..

//  Licensed under the Apache License, Version 2.0 (the "License");
//   you may not use this file except in compliance with the License.
//   You may obtain a copy of the License at
//
//       http://www.apache.org/licenses/LICENSE-2.0
//
//   Unless required by applicable law or agreed to in writing, software
//   distributed under the License is distributed on an "AS IS" BASIS,
//   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
//   See the License for the specific language governing permissions and
//   limitations under the License.
/////////////////////////////////////////////////////////////////////////////////
// Format an std::time_point as an ISO 8601 string with fractional seconds to 6
// decimal places, e.g. 2014-08-30T08:18:51.867479
// Warning will not work for any date/times before the start of the UNIX epoch.
//
#include 
inline std::string to_iso_8601(std::chrono::time_point t) {
	// convert to time_t which will represent the number of
	// seconds since the UNIX epoch, UTC 00:00:00 Thursday, 1st. January 1970
	auto epoch_seconds = std::chrono::system_clock::to_time_t(t);
	// Format this as date time to seconds resolution
	// e.g. 2016-08-30T08:18:51
	std::stringstream stream;
	stream << std::put_time(gmtime(&epoch_seconds), "%FT%T");
	// If we now convert back to a time_point we will get the time truncated
	// to whole seconds 
	auto truncated = std::chrono::system_clock::from_time_t(epoch_seconds);
	// Now we subtract this seconds count from the original time to
	// get the number of extra microseconds..
	auto delta_us = std::chrono::duration_cast(t - truncated).count();
	// And append this to the output stream as fractional seconds
	// e.g. 2016-08-30T08:18:51.867479
	stream << "." << std::fixed << std::setw(6) << std::setfill('0') << delta_us;
	return stream.str();
}

Microsoft Teams stuck in a loop saying it needs sign in after a password change

An amusing MS Teams related problem hit me this morning after I changed my Domain Password – The MS Teams App piped up and said it need Sign In, when I hit OK it seemd to restart and then repeated the message, we were in an infinite loop of fun!

Now as much as I can happily live without Teams bugging me, I unfortunately needed it for a conference call so I had to coax it back to life.  What worked for me in the end was to wait until Teams came up with its message and then right-clicked on the teams icon in the task bar and chose the ‘Sign Out’ option.  I then shutdown and restarted Teams and everything was OK again.

 

Launching Flask on Port 80 without using sudo

Flask is a great python based HTTP server that’s really small and easy/fast to setup, it is really useful for deploying small Web based User Interfaces for IoT type devices.  By default flask will attach itself to port 5000.  To get my flask script to attach to port 80 I use:

if __name__ == '__main__':
    app.run(host='0.0.0.0', port=80, debug=True)

This works fine, the only problem is that in order to successfully attach to port 80 the script must be run as root, so instead of just running:

./my_ui.py

I have to run:

sudo ./my_ui.py

Which isn’t great.

To get around this problem I used the tool: authbind

To Install:

sudo apt-get install authbind

And configure it for access to port 80:

sudo touch /etc/authbind/byport/80
sudo chmod 777 /etc/authbind/byport/807

(Not sure if the very loose 777 permission is required, must experiment)

Now I just have to launch my flask UI script using authbind and it will take care of the script’s permissions to bind to port 80 and I can connect from a browser:

authbind -deep python3 ./ui.py

More info on authbind here

Octave – Can’t scroll Window, Workaround

Strange problem with Octave (my version is 4.4.0), in the GUI I can’t scroll the ‘Command Window’, so if some code outputs lots of info I can’t scroll up to see it all, the window keeps jumping to the bottom as I try to scroll!

I am not sure why its happening, but a workaround is to enter the ‘pause’ command within the window, once pause executes I can scroll to my heart’s content.  I  then hit ctrl-c to exit pause.

It is possible that installing a newer version of Octave would fix this but I am not bothered to upgrade at the moment as all of the Maths bits seem to work very well!