Posts

Posted on

CMake Makefile

CMake has plenty of advantages, I don’t need to re-iterate them here. However, if you’re like me, who lives on the command line and has ./configure and make seemingly hard-wired into my fingers, it seems a bit cumbersome to have to type mkdir build, cd build, and (for example) cmake .. -DCMAKE_BUILD_TYPE=Release -DBUILD_SHARED_LIBS=ON each time you want to build something.

It got me thinking, and I haven’t come across anyone else suggesting it (I’m sure plenty of people have though)… why not drive CMake through a Makefile? Sure you can script it up easily enough, but it seems a bit single-purpose each time I do it, and my fingers are just itching to type `make`.

It seems a weird concept, because of course CMake is, amongst other things, a Makefile generator. But here’s my take on it. I love being able to type make or make release:

Makefile: 
# Usage:
#   make          -> debug build
#   make release  -> release build
#   make test     -> debug build + run tests
#   make clean    -> remove all build directories and symlink

APP_NAME := <<SET BINARY NAME HERE>>

BUILD_DIR_DEBUG   := build/Debug
BUILD_DIR_RELEASE := build/Release

MAKEFLAGS += --no-print-directory

CMAKE       := cmake
CMAKE_FLAGS :=
JOBS        := $(shell nproc 2>/dev/null || sysctl -n hw.logicalcpu 2>/dev/null || echo 4)

.PHONY: all debug release test clean

# -----------------------
# Default target -> debug
# -----------------------
all: debug

# -----------
# Debug build
# -----------
debug: $(BUILD_DIR_DEBUG)/Makefile
	$(CMAKE) --build $(BUILD_DIR_DEBUG) -- -j$(JOBS)

$(BUILD_DIR_DEBUG)/Makefile:
	@mkdir -p $(BUILD_DIR_DEBUG)
	$(CMAKE) -S . -B $(BUILD_DIR_DEBUG) -DCMAKE_EXPORT_COMPILE_COMMANDS=1 -DCMAKE_BUILD_TYPE=Debug $(CMAKE_FLAGS)

# -------------
# Release build
# -------------
release: $(BUILD_DIR_RELEASE)/Makefile
	$(CMAKE) --build $(BUILD_DIR_RELEASE) -- -j$(JOBS)

$(BUILD_DIR_RELEASE)/Makefile:
	@mkdir -p $(BUILD_DIR_RELEASE)
	$(CMAKE) -S . -B $(BUILD_DIR_RELEASE) -DCMAKE_EXPORT_COMPILE_COMMANDS=1 -DCMAKE_BUILD_TYPE=Release $(CMAKE_FLAGS)

# ------------------------
# Test (debug build + run)
# ------------------------
test: $(BUILD_DIR_DEBUG)/Makefile
	$(CMAKE) --build $(BUILD_DIR_DEBUG) --target $(APP_NAME)_tests -- -j$(JOBS)
	@$(BUILD_DIR_DEBUG)/$(APP_NAME)_tests --reporter console || true

# -----
# Clean
# -----
clean:
	@rm -rf build $(APP_NAME)
	@echo "-> build directories and symlink removed"

Set APP_NAME to the same as your CMake add_executable() name. This assumes you have a unit test framework and compiles a test suite in <APP_NAME>_tests (runnable with make test). It also symlinks the normal binary into your main project directory for easy running from the command line. Verbose building is as simple as typing VERBOSE=1 make. Tweak as you see fit.
Posted on

std::visit and std::variant

If you’ve got a set of types, and you need to iterate over them and do something with each of them, your initial instinct would probably be to make them related. For example, if we had a set of application menu commands, we might make them all inherit from a pure virtual base class (let’s call it “MenuCommand”). The sub-classes might be “FileMenuCommand”, or “ToolsMenuCommand”. We could then simply create a vector of smart pointers to MenuCommand and iterate over that. Right?

This gives us flexibility, but when we want to do something against each MenuCommand type, what do we do? Let’s assume our design is such that we have, in the base class, a member function called “print”. We can iterate over each element of the vector and simply call MenuCommand->print().

This has several disadvantages we might want to consider. Firstly, there’s the overhead of virtual function calls. Each call to print() requires a redirect to the actual type’s print() member function. Another issue is that we are iterating over smart pointers, so again, we have to dereference the pointer to look on the heap where the actual object is stored.  Also, there’s the issue that the actual objects could potentially be scattered indeterminately on the heap, which means the CPU can’t use its caching algorithms effectively. Finally, we might want to treat some MenuCommands completely differently from others – we might want to call a completely different member function, or several. Adding a “type” enum to each MenuCommand and a switch to determine type becomes a maintenance problem, and dynamic casting has similar issues.

std::visit and std::variant allows us to use the visitor pattern to avoid these concerns. Consider instead that we could use disparate types, with no inheritance. If we create a vector of std::variant<FileMenuCommand, ToolsMenuCommand> (for example), we can (a) create a vector of concrete types (so no heap allocation) and (b) we don’t have any dereferencing or virtual calls, thus improving both speed, and locality (enhancing our chances of CPU cache hits). Here’s a trivial example:

#include <iostream>
#include <variant>
#include <vector>

struct FileMenuCommand {
    void print() const
    {
        std::cout << "File Menu\n";
    }
};

struct ToolsMenuCommand {
    void display() const
    {
        std::cout << "Tools Menu\n";
    }
};

template <typename... Ts> struct overloaded : Ts... {
    using Ts::operator()...;
};

int main()
{
    std::vector<std::variant<FileMenuCommand, ToolsMenuCommand>> vec;
    vec.emplace_back(ToolsMenuCommand {});
    vec.emplace_back(FileMenuCommand {});
    for (const auto& o : vec) {
        std::visit(
            overloaded {
                [](const ToolsMenuCommand& tmc) { tmc.display(); },
                [](const FileMenuCommand& fmc) { fmc.print(); },
            },
            o);
    }
    return 0;
}

You can see we have no heap allocations for the types in the vector – obviously its storage is heap-based, but the types are all local to it.

We can also tailor our calls to each type (note that we call display() for ToolsMenuCommand, but print() for FileMenuCommand – confusing, but it serves to illustrate the point that we can act differently depending on the object).

The one confusing item above might be the “overloaded” template – This is a small utility that lets us construct a single callable object from multiple lambdas, so that std::visit can dispatch to the right one based on the active type in the variant.

It inherits from all of the types we pass in (in this case, the lambdas — each lambda is its own anonymous struct type with an operator()). The using Ts::operator()...; line pulls all of those operator() overloads into the same scope, so the compiler can see them together and perform normal overload resolution.

Without it, we’d have a problem: std::visit requires a single callable, but we have two separate lambdas. overloaded merges them into one object that has both call operators, so when std::visit calls it with a ToolsMenuCommand, overload resolution picks the first lambda, and with a FileMenuCommand it picks the second.

Posted on

“littlefs” Missing When Compiling with ESPHome

If you’ve got ESPHome installed on an Apple Silicon Mac via Homebrew, you might have noticed a compilation error complaining about littlefs missing on all projects.

I haven’t dug too deeply into this. I suspect it’s a Homebrew dependency issue, but the solution was to go to the following directory (note this might change depending on the version of ESPHome you have installed):

cd /opt/homebrew/Cellar/esphome/2026.3.0/libexec/bin

Then run the following commands:

./python3 -m pip install littlefs-python

./python3 -m pip install fatfs-ng

Until the homebrew dependency (if that is indeed the issue) is resolved, you’ll need to do this each time ESPHome gets updated via homebrew.

Update: the issue seems to have now been solved by the latest ESPHome package now up on Homebrew (I tested with version 2026.4.1)

Posted on

An Interesting Bug

A few years ago I wrote the C++ program latheControl which is designed to control my lathe (and subsequently my milling maching, and rotary table) by moving the machine’s axes via stepper motors. It’s designed to be an aid to manual machining rather than going the full CNC route.

It consists of a low-priority UI thread which handles display updates and input, and multiple realtime-scheduled threads which handle stepper motors and sensors. These are high priority as they need microsecond-level timing to handle operations like cutting threads, where the program needs to know exactly where the spindle is (rotation-wise) and how far to advance the z-axis.

All this runs on a Rasberry Pi, with a regular keyboard attached for input. It’s been rock solid and I’ve used it to great effect in many machining projects.

I recently relocated my mill, and started a new cut on a fairly large chunk of steel. To my surprise, the latheControl program stopped responding to any keypresses about half way through the first cut. I had to physically cut power to the stepper motor to stop the tool’s motion. Strangely, although the program wasn’t responding, I could get a virtual console (Ctrl-Alt-F1) and reboot it fine. So the Pi hadn’t crashed.

After the reboot, I tried again. About half way through the cut, it exhibited the same unresponsiveness.

The program is written such that I can run it on my laptop with all the hardware (steppers, sensors, rotary encoders etc) mocked out. This allows for unit testing as well as manual testing. I could not replicate the issue at all.

Back to the “real” hardware, I started to wonder about hardware failures. I tried installing the latest Pi OS. Again, same issue. Interestingly, if I traversed the x-axis without actually cutting metal, the problem didn’t appear. Now it was getting strange.

It’s not easy (but not impossible) to debug the program while running on the mill. I started thinking about running “strace” on the program while running. Initially I tried the trusty “got here” messages in the program’s log output. It confirmed it simply didn’t return keyboard input after a while, but only while cutting metal. I wondered whether the extra load of the mill’s spindle was causing some form of brown-out on the Pi. But this was discounted because the Pi remained working fine if I opened a virtual console for rebooting it. I wondered if the keyboard was physically failing (it does occasionally get showered with hot metal chips), but the fact I could switch to a virtual console and type normally discounted that idea.

I started looking more closely at the code, puzzled. The input loop has a function to get input from the keyboard. The program has a virtual function for this – I do this so I can encapsulate and abstract out the UI library in case I want to swap it in future (I’ve already done this once – the first version of the program used a TUI rather than a GUI). The current one uses SFML to get input and display graphical output.

In pseudocode, the input function does this:

    •  get event (this is any input, e.g. window interactions, keypresses, joystick input, mouse motion etc)
    • if event is not keypress return “no key pressed”
    • otherwise return the key pressed

This is called from the low-priority UI thread, in a loop, which has a 50ms delay each iteration. Handling one keypress every 50ms is fine, right?

After staring at the lower-level input code for a few hours, I suddenly realised that if many non-keyboard events are received, the event queue will back up (because only one event is popped off the queue every 50ms).

The keyboard attached to the mill’s Pi has a rollerball on it, to control the mouse. Can you see the issue?

It turned out that the rollerball was very slightly being vibrated by the mill when cutting, causing a slew of “mouse moved” events to be added to the event queue when chewing metal. These were only being popped off every 50ms, so  the queue grew to the point that keypresses were pushed deep down the queue, rendering the program unresponsive.

When the cause finally dawned on me, it was a simple job to put a loop in the input processing function  to discard any non-keyboard events in one go, and the problem disappeared. One of the most interesting bugs I’ve encountered during decades of C++ development.

Posted on

Solving Word Games with Rust

If you enjoy word games (such as cryptic crosswords) as I do, then this is a really useful tool for you.

It can be used in many ways. For example, if you’re trying to solve an anagram, you could type lookup GSNIGNI and it will give you output like this:

      S
        G
  N       G
    I   N
      I

  _ _ _ _ _ _ _

If you have some found letters, you can add them, for example, lookup GSNIGNI -f ..g...g and it will show:

    I   N

  S       I

        N

  _ _ G _ _ _ G

Where it becomes more clear that the answer would be “signing”.

If you’re totally stuck, you could append -a to your command, i.e. lookup GSNIGNI -f ..g...g -a and it will show you all valid words which are anagrams of your search string which fit the “found” letters. In this case it just returns “signing”.

There are lots of other options, including straight lookup, regex searches (if you understand how to use them), definitions, and support for other word games like Wordle, Panagrams, and Spelling Bee.

It’s written in Rust and the source is all available at https://github.com/md81544/lookup

Posted on

Rust Projects

I’ve been having a lot of fun recently learning Rust. It’s a breath of fresh air in so many ways compared to C++: for instance the standardised package management, a consistent way of building projects, and simple-to-add unit tests. These are in addition to the much-touted safer memory management. We won’t be giving up with C++ at work, but we will be using Rust for several new projects.

As a learning exercise, I’ve re-written a game I first wrote for the ZX-Spectrum forty years ago, called Xtarda Rescue. It’s not the cleanest code – there was a lot of learning along the way, and I’d definitely refactor parts of it if I had the time, but it plays really well and, while I tried to keep it retro, it’s far more advanced than the original.

I also wrote probably about the millionth version of Conway’s Life.

There’s also a tiny little CLI utility called Temperature I’ve just created, which is a good example of how to do async calls, in this case to a couple of REST APIs, one for IP-based geolocation (so not very accurate!) and then, given that location, a call to get localised weather information.

I’m definitely looking forward to continuing learning more Rust.

Posted on

Non-Metric Threads on the Lathe Using C++

I continue to improve the lathe application I’ve written – it’s working really well. Several major improvements recently include a graphical user interface to allow better display of information, stepper control of the X axis, and the ability to cut tapers by controlling the Z and X axis simultaneously.

Yesterday I wanted to cut a non-metric thread (normally I use metric exclusively) – specifically 3/8″ x 16 TPI. I needed a part for an old engineer’s clamp I’ve had kicking around for ages; it was missing one of its two screws. I have very few imperial dies, and not that one.

Now I can cut threads programmatically, that’s no problem – I just added the thread dimensions to my program, ground up a 55° tool, and started cutting. The test thread (not the finished replacement bolt) is shown above. It fits beautifully!

Update: here’s a picture of the finished version, in situ on the old clamp which I’ve never been able to use owing to the missing screw (the new one’s on the left). I guess the next project is cleaning the thing!

Posted on

Controlling a Rotary Table with C++

As part of the project to control my metalworking lathe with a stepper motor (see https://www.martyndavis.com/?p=726), I’ve decided it would be useful to make some additional gears to step down the speed of the rotary encoder in relation to the lathe’s spindle.

Here’s the first one I created (I haven’t cut a keyway in it yet).

A z45 module 1.0 gear I cut with this software

To cut gears, typically people use a rotary table, which has a worm drive on it to allow for accurate turning of the workpiece via a handwheel which is marked with a vernier scale. I figured it would be easier to get a computer to do that for me.

Creating the software was simple, because I had all the components already created for the previous project. Unlike the lathe project, there are no microsecond-critical timing issues – just rotate, cut, rotate, and repeat until you’ve turned 360°. See it in action at https://youtu.be/4pt8MZhDIB4 – quite literally this was my first test, and it all just worked as expected.

The code can be found at https://github.com/md81544/electronicRotaryTable

Update: here’s a new video showing the code in action for real, cutting an aluminium z40 spur gear. The result is shown below:

Posted on

Controlling a Lathe with C++

This is a quick post – I’m intending on going into far more detail about this at a later stage – but recently, for fun, I’ve been working on automating some functions on a metalworking lathe. It’s been a really interesting project – I’ve been driving a fairly powerful stepper motor to give accuracy of positioning of tooling down to near-micron level, and measuring the position of the lathe’s spindle (via a rotary encoder) while it spins, at an accuracy required to start a cut at the same position over multiple passes, for cutting threads. I’ve been driving all this from a Raspberry Pi 3 Model B. It’s working beautifully, as you can see from the thread I cut below:

From a C++ perspective, this has required microsecond-level accuracy, and plenty of multi-threading to get suitable real-time performance – it’s interesting to have a real-world requirement for this kind of coding outside of my normal work, and incredibly rewarding to see the lathe chuck spinning in a blur but knowing that the code can determine exactly what its rotational position is at any moment.

Usage of mock objects and unit tests have allowed me to develop large chunks of the code without access to the hardware when I’m away from my workshop.

The code can be found at https://github.com/md81544/electronicLeadScrew

See the thread cutting in action at https://www.youtube.com/watch?v=VlhPiRscNeY

Posted on

Using libjpeg in C++

Just for fun I thought it would be interesting to write a quick program to convert .jpg images to an ASCII representation. Here’s a sample output:

I was surprised that there are not more examples online showing how to use libjpeg, and even fewer wrapping it in a more cpp-friendly manner.

I decided to make the functionality re-usable, so if you have a need to read in a .jpg file, modify it, and save it, then this is going to be useful to you.

One point of interest, the shrink() member function – initially I wrote it very simply. I went through each (new) pixel in turn and calculated it as the average of a box of pixels in the original, larger, bitmap. This worked fine, but at the time I knew it was inefficient. Jumping around a surrounding box means cache misses galore. The best way to do this is one line at a time – this means we’re processing data already in processor cache lines, and is simpler for the CPU to do correct branch prediction. The version you see in the current source was my more considered version – it ended up three times faster than the initial version (and as it’s on github you can compare it to the old version in the history). I’ve left the getAverage() member function in, in case it is useful for anyone, but no longer use it in shrink().

If you’re on a linux distro you’ll undoubtedly have libjpeg (or one of its forks) installed. To use it and my source, you’ll need to install the header files for it, with:

sudo apt install libjpeg-dev

You can also install it on Windows, as a test I used Microsoft’s vcpkg to get the header files.

You can see the code at https://github.com/md81544/libjpeg_cpp -you’ll want a C++11-compatible compiler for it. I’ve tested the latest version with both g++ and clang++.