October 06, 2020

Iain R. Learmonth


OpenBSD Worrying RAID

I wanted to move a couple of USB hard drives from one OpenBSD machine to another. They are configured with softraid(4) as RAID 1 (mirrored). When I plugged the drives into the new machine though, nothing happened with softraid. This was pretty worrying.

Both the drives showed in dmesg output so the issue was specifically to do with softraid. The man page for bioctl(8) talks about -c creating a “new” RAID device which sounded a little too destructive. I asked for help in #openbsd and apparently the language in the man page is misleading. The -d flag has recently been updated to say “detach” rather than “delete” to try to address this.

I went for it and did:

bioctl -c 1 -l /dev/sd5a,/dev/sd6a

It worked and I got the dmesg output:

softraid0: sd3 was not shutdown properly
sd7 at scsibus4 targ 1 lun 0: <OPENBSD, SR RAID 1, 006>
sd7: 7630885MB, 512 bytes/sector, 15628052512 sectors
softraid0: volume sd7 is roaming, it used to be sd3, updating metadata
softraid0: roaming device sd2a -> sd6a
softraid0: roaming device sd1a -> sd5a

I guess if it’s not cleanly shutdown it doesn’t just automatically set up the RAID device again, it could also have been the renumbering that stopped it.

Mounting the device:

mount_ffs: /dev/sd7i on /external: filesystem must be mounted read-only; you may need to run fsck
WARNING: R/W mount of /mnt denied.  Filesystem is not clean - run fsck

I guess that was expected. An fsck later and everything was working again.

Backup your stuff.

October 06, 2020 11:50 AM

October 03, 2020



Radio Power in the Field - Follow on

Previously, I wrote about building a battery for my portable radio operations. This project has been a long and agonising one. Batteries are really difficult, and I spent a good chunk of June, July and August thinking about them. The Winning Design As I discussed in that last blog post, the architecture I settled upon was: I’ve learned quite a lot, made a large number of wrong assumptions and joined a lot of dots in my head as this has gone on.

October 03, 2020 05:00 PM

Tom Jones


Presentations with mdp

It feels like work is just a constant stream of preparing, travelling for and giving presentations. Brief words and pictures is an excellent for conveying information between small groups of humans. All of these presentations I write in keynote, keynote manages to be light weight, powerful and not horrific to use. As a bonus, my boss feels at home in keynote and is happy to make edits there.

The keynote workflow does not match well to how I think. When dreaming up a presentation I want to shit of a stream of conciousness and have it magically become slides in the right shape.

I might write a series of headings like:

# intro
# who
# meat 
# details
# questions?

I will iterate on these to add bodies, details and more slides.

For quite a while I have wanted a system where I could write plain text and have it become slides. I [wrote][3] about the [sent][4] tool from suckless, but in the end I found it wanting. I have also considered just showing screens of text, but a nightmare DEFCON wireess village talk by Hak5 scared me away. They attempted to just present using just a plain text file and less, but the window size got out of whack and it all fell apart.

Enter mdp

mdp is a terminal presentation program, it takes slides it approximately markdown and takes over the entire terminal as its presentation surface.

Intrigued I used an opportunity to speak at a [local tech event][5] to try out mdp. [The slides][6] from that presentation can be found on [my talks page][7] and overall I thought mdp worked quite well.

I was able to draft in the stream of conciousness style I want, getting the bulk of the slides written very quickly. Adding diagrams required resorting to ASCII art which isn't so bad, I like [ascii][10] [art][11]. mdp worked great in practice, I had to find readable dimensions for the text by trial and error, but overall it went well.

Plain text as a format does have some major downsides, mdp has a way to encode builds for text (see below), but I couldn't use it with my tools. ASCII art diagrams also meant that the builds I did use were eggregious to maintain, any modification required manual propigation through the build chain.

mdp does not support a portable output format. You may say the source markdown is an excellent format for portability, but I find it lacks the crispness of having a single slide in view at once.

I wanted to be able to point at a viewable copy of my slides and so I hacked together some tools to export the mdp presentation to html, but for this I had to sacrifice the built in build mechanism of mdp

Finally there was no way to include images in the mdp presentation let alone the sacride gif format required to correctly convey nyan cat. I played with some terminal graphics viewers, but none of them worked well and after a while I started to think 'what is the point of reinventing everything'.

Drafting the presentation in markdown fit very well with my work flow, but the difficulties in getting a complete presentation with mdp meant that I didn't want to use it for future presentations.

Exporting to html

Getting html of the mdp presentation hinged on a complete hack. There is a tool I had seen in the past that can output a html dump of a tmux session unsurprisingly called [tmux2html][12]. With some playing around I was able to automate a tmux session to work through the slides and use tmux2html to grab each slide as a frame.

Finding the number of slides in the deck required splitting on the slide seperator from the markdown, this ruled out using the built in build mechanism as I would end up with the wrong number of slides.

The output script runs through the markdown to find the number of slides then uses tmux send-keys to control moving through the deck.


set -e 

command -v tmux >/dev/null 2>&1 || { echo >&2 "I require tmux but it's not installed.  Aborting."; exit 1; }
command -v tmux2html >/dev/null 2>&1 || { echo >&2 "I require tmux2html but it's not installed.  Aborting."; exit 1; }
command -v mdp >/dev/null 2>&1 || { echo >&2 "I require mdp but it's not installed.  Aborting."; exit 1; }

if [ -z "$1" ]
    echo "tohtml [outfile.html]"


if [ ! -z "$2" ]

javascript="<script>function page(){var e=!1,n=document.getElementsByClassName('tmux-html'),l=0; document.onkeydown=function(t){if(t=t||window.event,key=t.keyCode,e)if(13==key){e=!1,l=0;for(var i=0;i<n.length;i++)n[i].style.display='inline'}else{37==key&&--l<0&&(l=0),39==key&&++l>=n.length&&(l=n.length-1);for(i=0;i<n.length;i++)n[i].style.display='none';n[l].style.display='inline'}else if(13==key){e=!0,n[0].style.display='inline',l=0;for(i=1;i<n.length;i++)n[i].style.display='none'}}}window.onload=function(){page()};</script>"


slides=`grep -e "^---" $file | wc -l`

tmux new-session -s $tmux -d -x 96 -y 25

tmux send-keys -t $tmux "mdp $file"
tmux send-keys -t $tmux "Enter"

tmux send-keys -t $tmux 'g'
tmux2html -o $tmpfile $tmux 1>/dev/null

# insert javascript
lines=`cat $tmpfile | wc -l`
styleend=`cat -n $tmpfile | grep -e "</style>" | awk '{print \$1}'`
head -n $styleend $tmpfile > $outfile
echo $javascript >> $outfile
tail -n $((lines-styleend)) $tmpfile >> $outfile
mv $outfile $tmpfile

# remove closing tag
lines=`cat $tmpfile | wc -l `
end=`tail -n 1 $tmpfile`
head -n $((lines-1)) $tmpfile > $outfile

echo turning $file into $((slides+1)) slides 

while [ $i -lt $((slides+1)) ]
    printf "\rSlide $i"
    tmux send-keys -t $tmux 'j'

    tmux2html -o $tmpfile $tmux 1>/dev/null 
    grep -e "^<div" $tmpfile >> $outfile
    (( i++ ))

echo $end >> $outfile
tmux kill-session -t $tmux 
rm $tmpfile
printf "\rwritten to $outfile \n"

[If you view the presentation page][6] you will see the entire slide deck, this was the first output I got from this script. All the slides in a nice order. After a little pondering I wrote up some javascript to give controls, if you hit enter it will go from all slides to single slide. Arrow keys in single slide mode will allow you to move through the slide deck. The unminified javascript for this is below.

function page() 
    var presenting = false
    var elements = document.getElementsByClassName('tmux-html');
    var current = 0;

    document.onkeydown = function(evt) {
        evt = evt || window.event;
        key = evt.keyCode 

        if (presenting) {
            if (key == 13) {
                presenting = false;
                current = 0;
                for (var i = 0; i < elements.length;i++)
            } else {
                if (key == 37) {    //left
                    if (current < 0)
                        current = 0;
                if (key == 39) {    //right
                    if (current >= elements.length)
                        current = elements.length-1;
                for (var i = 0; i < elements.length;i++)
        } else {
            if (key == 13) {
                presenting = true;

                current = 0;
                for (var i = 1; i < elements.length;i++)

window.onload = function () {

[3]: sent blog post [4]: sent link [5]: techmeetup aberdeen [6]: mdp slides [7]: talks page

[10]: ietf ascii art guidelines [11]: draft cco [12]: tmux to html

October 03, 2020 12:00 AM

September 27, 2020

Iain R. Learmonth


Multicast IPTV

For almost a decade, I’ve been very slowly making progress on a multicast IPTV system. Recently I’ve made a significant leap forward in this project, and I wanted to write a little on the topic so I’ll have something to look at when I pick this up next. I was aspiring to have a useable system by the end of today, but for a couple of reasons, it wasn’t possible.

When I started thinking about this project, it was still common to watch broadcast television. Over time the design of this system has been changing as new technologies have become available. Multicast IP is probably the only constant, although I’m now looking at IPv6 rather than IPv4.

Initially, I’d been looking at DVB-T PCI cards. USB devices have become common and are available cheaply. There are also DVB-T hats available for the Raspberry Pi. I’m now looking at a combination of Raspberry Pi hats and USB devices with one of each on a couple of Pis.

Two Raspberry Pis with DVB hats installed, TV antenna sockets showing

Two Raspberry Pis with DVB hats installed, TV antenna sockets showing

The Raspberry Pi devices will run DVBlast, an open-source DVB demultiplexer and streaming server. Each of the tuners will be tuned to a different transponder giving me the ability to stream any combination of available channels simultaneously. This is everything that would be needed to watch TV on PCs on the home network with VLC.

I’ve not yet worked out if Kodi will accept multicast streams as a TV source, but I do know that Tvheadend will. Tvheadend can also act as a PVR to record programmes for later playback so is useful even if the multicast streams can be viewed directly.

So how far did I get? I have built two Raspberry Pis in cases with the DVB-T hats on. They need to sit in the lounge as that’s where the antenna comes down from the roof. There’s no wired network connection in the lounge. I planned to use an OpenBSD box as a gateway, bridging the wireless network to a wired network.

Two problems quickly emerged. The first being that the wireless card I had purchased only supported 2.4GHz, no 5GHz, and I have enough noise from neighbours that the throughput rate and packet loss are unacceptable.

The second problem is that I had forgotten the problems with bridging wireless networks. To create a bridge, you need to be able to spoof the MAC addresses of wired devices on the wireless interface, but this can only be done when the wireless interface is in access point mode.

So when I come back to this, I will have to look at routing rather than bridging to work around the MAC address issue, and I’ll also be on the lookout for a cheap OpenBSD supported mini-PCIe wireless card that can do 5GHz.

September 27, 2020 09:35 PM

September 24, 2020

Ana Custura

Firefox modding with containers and SOCKS proxies

Here’s a guide to my lazy setup for running multiple Firefox tabs in the same session over different networks using the magic of SOCKS.

The use-case is that I sometimes want to access a web app or page which is only accessible via a specific network (i.e., my work network or Tor), but I most definitely don’t want the rest of my browsing traffic going through there.

The general idea is to use long-running SSH tunnels to provide one or more SOCKS5 proxies that can be used by Firefox (or your browser of choice). SOCKS is a protocol that allows applications to request connections through a proxy server. Applications, such as Firefox, must be configured to use it.

Generally, to do this manually, you’d first SSH with dynamic forwarding into a host on the desired network:

ssh -D1080 user@host

…and now a SOCKS proxy on localhost port 1080 is ready to forward connections to the remote host. Tor also provides a SOCKS proxy that can be used in much the same way by default on port 9050. This does not conflict with Tor Browser, which runs its own Tor daemon listening on port 9051, separate from the system Tor.

So, to use the proxy in a browser, the browser’s network settings should be changed to resemble something like this:

Firefox settings

Firefox also has a checkbox for proxying DNS requests through the same connection.

It’s a good idea to proxy your DNS requests because 1) the remote DNS resolver might know names of resources you can’t access locally and 2) due to the prevalence of CDNs in the Internet, the IP addresses obtained this way will often correspond to servers physically closer to the tunnel endpoint, speeding up connections.

These settings could be saved under a separate Firefox profile that can be fired up whenever the SSH connection is active. Any browser requests will be forwarded to the network of the host you’re SSHed into.

Now, this is an easy substitute for a VPN, but still requires launching a new SSH connection and browser instance every time you want to browse via the remote network. Plus, multiple networks mean multiple profiles or multiple SSH connections which is a pain to manage.

Enter Firefox containers and autossh. The first is an extension that allows you to keep website data, cookies, and cache separate between tabs and websites by assigning them to different containers.

The second is a way to maintain an SSH tunnel indefinitely. The way to glue them together is Container Proxy, another Firefox extension that allows per-container proxy settings.

Here’s how it works:

Autossh and Tor

This is a wrapper around ssh to keep tunnels open indefinitely in the background. It can use any SSH option or config. For simplicity, I have the following config specified for my proxy host in ~/.ssh/config:

Host pxhost
    ServerAliveInterval 30
    ServerAliveCountMax 3
    DynamicForward 1080

This command will run autossh in the background, forever keeping the connection alive.

autossh -M 0 -f -N pxhost 

To persist this on reboot, I use a systemd service file for Linux and a @reboot cronjob for macOS.

I also have Tor configured to run at startup, allowing me to use it alongside other connections. Tor can run as a service on distros using systemd. On macOS, I modified the .torrc file in my home directory to include RunAsDaemon 1, and just running tor with no options on the command line starts the SOCKS proxy.

Firefox Multi-Account Containers

The extension can be found in the official Firefox store. I have three containers: a Direct container for day-to-day browsing without a proxy, a Work container for accessing some infrastructure at work via an SSH connection into my work computer, and a Tor container for looking at .onion addresses or other web pages over Tor:

Firefox Containers

Tabs opened in each container are colour coded, and easy to keep track of.

An important note about the Tor container: using Tor as a proxy and not using Tor Browser does not provide anonymity, because any other browsers will leak client information allowing 3rd parties to identify users. I do this mostly for convenience and sometimes to avoid eavesdropping from my ISP. However, if you want anonymity, USE TOR BROWSER!

Container Proxy

Proxies for containers are not supported natively in the official Firefox extension. At the moment, another extension is required to make the containers use the tunnels. While not checked by Mozilla, this is open source and the code is auditable at

It lets you configure and test the proxies with DuckDuckGo, and assign them to containers:

Container proxy setup

If the tunnels are set up to persist on reboot, and your Firefox profile is not entirely erased with the latest update, this is how it looks/works:

Example container

That’s it. Containers are cool. Use more containers, before Mozilla dies off.

September 24, 2020 12:00 AM

September 21, 2020

Iain R. Learmonth


Matrix Synapse Certificates

I’ve had a need to set up and run a Matrix homeserver and I wanted to try to set it up for general use as well as just for this project.

The one thing I wanted to do that is a little non-standard is to use the “plain” domain for my user ID while hosting the homeserver on a subdomain. In this case I wanted to use “” as the user ID domain, and “” to host the Synapse homeserver.

If you’ve run an XMPP server before then you’ll know about this sort of arrangement allowing you to seperate identity from the actual infrastructure providing the services. In XMPP this is achieved with DNS SRV records, for example: SRV 5 0 5222 SRV 5 0 5269

There are two records here, one for clients and one for servers. Both of these endpoints should use a TLS certificate with the plain domain “” in the name as this indirection doesn’t change the identity of the service you’re connecting to.

Matrix is considerably more complex unfortunately. There are still two types of discovery required: one for clients and one for servers. The implementation of these however are different. Let’s start with server to server (federation) discovery.

The specification outlines the 5 step process to be used to discover the server that should be connected to. Matrix prefers to use a well known file to be fetched via HTTPS to discover the server to connect to.

In my case, that means connecting to and expecting to receive a JSON object that looks something like:

{"m.server": ""}

If this succeeds then it will connect to the Synapse homeserver and expect that the TLS certificate will have the name I’ve not really got a web server set up there yet though which means that this will fail and move on to the next step.

The next step looks up a DNS SRV record, similar to that used by XMPP, that looks like: SRV 0 0 8448

This allows other Matrix homeservers to now discover my homeserver and federate, but the important thing to watch out for here is that now the name expected on the certificate is the plain domain name,

For clients, there is no DNS SRV option available, only the use of a JSON file at a well known URL. Clients always expect the name on the domain to be the name of the host running Synapse. As I’m the only user on this server I’m quite happy for now to just type into clients that I use rather than automatically discovering the server.

This means there’s a need for both the plain domain name and the server domain name to be present in certificates for federation and for client use. I suspect that the use of the server name is to better support operation from a browser. A browser can lookup the well known URL to get the JSON file, and then create a new HTTPS request to the homeserver. Without the server’s name on the certificate that would fail in the browser. For federation it’s possible to use the plain domain (and so have some better authenticity checks when you consider that the user IDs have the plain domain) because it’s the homeserver that is checking the certificates.

With XMPP, you just have the plain domain because both servers and clients are all XMPP aware and you’re not having to compromise on security checks based on what a browser can do for you.

So now I need to get two certificates into the Matrix homeserver: and I’m using OpenBSD which has acme-client(1) available, but this only implements the http-01 challenge. This means I won’t be able to use acme for the plain domain.

To set up acme-client for the server hostname, I used the hambsd-acme role to configure the client and set up periodic renewals. For the plain domain name, I paid for a certificate to be issued which gives it a longer expiry meaning I’m not having to constantly update it manually.

In the relayd.conf, I added:

tls keypair
tls keypair

This means that when a request comes in, the correct keypair will be chosen based on the SNI TLS option.

In the future, I’ll be aiming to switch from DNS SRV discovery to the well-known URL which will then allow me to use only the certificate for the server hostname and drop the certificate for the plain domain name.

September 21, 2020 09:32 AM

August 31, 2020

Iain R. Learmonth


Portable Radio Setup

I’ve been wanting to document my portable radio setup for a while, so finally here is that blog post. This is a QRP setup, intended primarily for digimodes on HF.

HF antenna set up on Elrick Hill, Aberdeen

HF antenna set up on Elrick Hill, Aberdeen

First I want to talk about the antenna setup. Pictured above is the antenna set up on Elrick Hill in Aberdeen. I walked straight up the hill last weekend with the kit and had the antenna up in under 10 minutes. The lower guy ropes can be put in without assistance as they are short enough to be able to hold the mast while you push the pegs in. If it had been windier I’d have done those and then I have a second set of guy ropes that go half way between those and the top.

The antenna is the SOTABEAMS Bandspringer Midi, an end-fed antenna good for 10m-60m with a tuner. There’s the wire that goes over the mast and a second wire as a counterpoise that runs along the ground beneath the first wire. I particularly like the wire winder that makes it near impossible to get either the guy ropes or the antenna wires tangled.

The mast is the SOTABEAMS Tactical Mini, a 6 meter fibreglass mast. I’ve had at least 50 hours of use of it so far, and I’ve never had any issues with it. If I do, spares are available to replace any sections that get broken. It’s lightweight and can sit in my backpack nicely (although it pokes out a couple of inches).

Laptop, Yaesu FT-817, Travel Tuner and connected antenna

Laptop, Yaesu FT-817, Travel Tuner and connected antenna

The antenna wire and counterpoise both terminate with 4mm banana plugs. These are then connected to an adapter to convert to BNC, which is connected to an adapter to convert to PL259. This part of the setup is not optimal and while the tuner, a MFJ-902H Travel Tuner, does have 4mm sockets on the back I’m not sure how they would work with this antenna or how I should wire them up. If you have ideas for this I’d be interested to hear them.

The tuner is then connected to the Yaesu FT-817ND with a short RG58 patch lead. For now I’m running this from the internal battery (which has just been replaced) but I’m seriously considering investing in an external battery. Hibby has been invesitgating building his own battery packs with lithium ion cells, a route I may go down later too.

Finally, there is a Dell laptop running Debian connected to the radio via a data mode cable. A USB sound card is used as laptops don’t have microphone jacks anymore. On the particular trip in these pictures, the laptop didn’t actually work as there were issues with PulseAudio refusing to emit any sound that I’ll need to investigate more.

Packing everything away at the end of the day, the mast and antenna sit in the backpack with the laptop. The radio, tuner, connectors, sound card, and data cable are all stored in the kit bag.

RUSH 24 backpack and 5.11 Kit Bag packed away

RUSH 24 backpack and 5.11 Kit Bag packed away

This post isn’t as comprehensive as I’d have liked, but I’ve not blogged for a while and I figured that writing something was better than waiting for the perfect post to materialise.

August 31, 2020 04:15 PM

August 29, 2020

Ana Custura


Rethinking ACKs at the Transport Layer. Ana Custura, Tom Jones, Gorry Fairhurst. FIT 2020: 731-736

Measuring mobile performance in the Tor network with OnionPerf. Ana Custura, Iain Learmonth, Gorry Fairhurst. MNM 2019: 233-238

Exploring Usable Path MTU in the Internet. Ana Custura, Gorry Fairhurst, Iain Learmonth. Traffic Measurement Analysis 2018: 1-8

Experience: Implications of Roaming in Europe. Anna Maria Mandalari, Andra Lutu, Ana Custura, Ali Safari Khatouni, Özgü Alay, Marcelo Bagnulo, Vaibhav Bajpai, Anna Brunström, Jörg Ott, Marco Mellia, Gorry Fairhurst. MobiCom 2018: 179-189

Exploring DSCP modification pathologies in the Internet. Ana Custura, Raffaello Secchi, Gorry Fairhurst. Comput. Commun. 127: 86-94 (2018)

Exploring DSCP modification pathologies in mobile edge networks. Ana Custura, Andre Venne, Gorry Fairhurst. MNM 2017: 1-6

August 29, 2020 12:00 AM

August 04, 2020



SOTA Trip Reports: July 12, August 01, 02 2020

Here’re my first sota trip reports. I made some mistakes and have lots to learn still, but they were great days out and I’m dead keen to do more. Most importantly, I’ve got better at radio in general - I’m much more comfortable on a microphone, I understand the flow and format of an HF contact far better. It’s very different from the datamodes I’ve been doing for years! One thing that I’ve noticed, and it probably deserves mentioning, is that the SOTA web infrastructure is some of the best I’ve seen in ham radio.

August 04, 2020 07:50 PM

July 10, 2020

Iain R. Learmonth


Light OpenStreetMapping with GPS

Now that lockdown is lifting a bit in Scotland, I’ve been going a bit further for exercise. One location I’ve been to a few times is Tyrebagger Woods. In theory, I can walk here from my house via Brimmond Hill although I’m not yet fit enough to do that in one go.

Instead of following the main path, I took a detour along some route that looked like it wanted to be a path but it hadn’t been maintained for a while. When I decided I’d had enough of this, I looked for a way back to the main path but OpenStreetMap didn’t seem to have the footpaths mapped out here yet.

I’ve done some OpenStreetMap surveying before so I thought I’d take a look at improving this, and moving some of the tracks on the map closer to where they are in reality. In the past I’ve used OSMTracker which was great, but now I’m on iOS there doesn’t seem to be anything that matches up.

My new handheld radio, a Kenwood TH-D74 has the ability to record GPS logs so I thought I’d give this a go. It records the logs to the SD card with one file per session. It’s a very simple logger that records the NMEA strings as they are received. The only sentences I see in the file are GPGGA (Global Positioning System Fix Data) and GPRMC (Recommended Minimum Specific GPS/Transit Data).

I tried to import this directly with JOSM but it seemed to throw an error and crash. I’ve not investigated this, but I thought a way around could be to convert this to GPX format. This was easier than expected:

apt install gpsbabel
gpsbabel -i nmea -f "/sdcard/KENWOOD/TH-D74/GPS_LOG/25062020_165017.nme" \
                 -o gpx,gpxver=1.1 -F "/tmp/tyrebagger.gpx"

This imported into JOSM just fine and I was able to adjust some of the tracks to better fit where they actually are.

I’ll take the radio with me when I go in future and explore some of the other paths, to see if I can get the whole woods mapped out nicely. It is fun to just dive into the trees sometimes, along the paths that looks a little forgotten and overgrown, but also it’s nice to be able to find your way out again when you get lost.

July 10, 2020 02:00 PM

July 06, 2020



Splat for UHF radio station coverage modelling

I’ve been talking to Iain, MM0ROR a bit on the radio. After I persuaded him to build a 70cm version of the 1/4 wave vertical antenna I built earlier this year, I helped him test it on the radio tonight from my new /m setup. Initially I drove to the picturesque Aberdeen Beach, then the less pretty 57North Hacklab, and finally I took the long road to the very lovely Balmedie Beach, and for the bulk of the drive nattered away to Iain about nonsense and radios.

July 06, 2020 10:59 PM

July 02, 2020

Tom Jones


Blog more in 2020

In June I tried to write 4 blog posts and I elicited help from some of my friends to do this. I managed to write 5 posts beyond the announcement I would blog:

Of course it wasn't just me, I asked other people to blog to help me stay on track. The idea here was that seeing other peoples blog posts would inspire and force me to keep going. This worked reasonably well. The pressure to write the blog posts was there, but publishing was harder. This ended up with me pushing several posts in the final few days of June.

The pressure didn't really show up either, I know that the others wrote blog posts, but they didn't tell me!

They were great sports to get involved and help me with this, you should look up their blogs and drop them into your rss reader.

Because this wort of worked I think we should aim to keep doing this. Now 4 posts a month is a lot (maybe even too much) and so I thought that 8 more this year would be good. That is about 1.3333333... a month and seems entirely achievable.

I am going to try and blow this number out the water, but even if I fail completely and only manage one or two more post that will still be great.

July 02, 2020 12:00 AM

July 01, 2020

Tom Jones


Simple ipfw NAT for bhyve virtual machines and vnet jails

Most of the time, I want to do some throw away networking temporally to play with something or to try something out. I really don't like changing all the config on a machine just to try something. The FreeBSD documentation leans the other way first showing you what to edit in rc.conf before maybe mentioning that actual commands to run.

The ipfw documentation has a different problem. The example in the handbook and online are both very verbose and very complicated. Because ipfw is normally configured with a shell script the authors go absolutely wild with all the features they can.

I had a hard time figuring out ipfw in-kernel NAT from these guides. Instead here I present the simplest set of commands I could find to set up a NAT and a little explanation to help you debug when it doesn't work.

This is based on a great email from Allan Jude on the freebsd-virtualization list from 2014 that laid out the basics of this setup.

Set up Overview

For testing I want to run virtual machines and vnet jails on my laptop and give them have access to the internet. I want a throw away NAT setup that is ready to go quickly.

My laptop connects to my home network (and eventually the internet) over wifi. The wifi network offers me an address in the subnet. On my laptop I want to have multiple guests. To do this we are going to use ipfw NAT and a bridge interface. It will look something like this:

         TO INTERNET
          +-------+  192.168.1.x
+-------------| wlan0 |---------------+
|             +-------+               |
|                 ^                   |
|                 |                   |
|              ipfw nat               |
|                 |                   |
|                 V                   |
|            +---------+              |
|   | bridge0 |              |
|            +----+----+              |
|                 ^                   |
|                 |       |
|      ___________+_______________    |
|      |       |        |        |    |
|      v       v        v        v    |
|  +---+--+ +--+---+ +--+---+ +--+---+|
|  | jail | |  vm  | | jail | | ...  ||
|  +------+ +------+ +------+ +------+|
+-------------- laptop ---------------+

The interfaces in the jails (the b half of the epair) and the virtual machines (the vtnet in the V) won't be visible to ipfw, but will exist in their own world. To work around this we will use a bridge with the epairs and tap interfaces.

Setting up ipfw NAT

We need to load the kernel modules for ipfw and the ipfw in kernel NAT. ipfw has the frustrating default (and annoyingly different to ipf and pf) of to dening all traffic. This default has the great property of locking you out of a machine you are setting up remotely.

This is control by a sysctl that cannot be changed at run time, but we can change the default behaviour with kenv before we load the module:

# kenv net.inet.ip.fw.default_to_accept=1

Now we can safely load ipfw and the in-kernel NAT.

# kldload ipfw ipfw_nat

ipfw should load enabled, if you are having trouble later on double check that the firewall is actually enabled.

# sysctl net.inet.ip.fw.enable
net.inet.ip.fw.enable: 1

When we do NAT we are acting as a gateway between the traffic on the NATd interface and the real interface. For any packets to be passed we need to enable forwarding.

# sysctl net.inet.ip.forwarding=1
# sysctl net.inet6.ip6.forwarding=1

ipfw rule set

We need to create an IPFW NAT instance configured with the interface we want to NAT (wlan0 in this case) and configure rules to pass all traffic from the bridge through the NAT.

# ipfw nat 1 config if wlan0
# ipfw add 101 nat 1 ip from to any out via wlan0
# ipfw add 103 nat 1 ip from any to any in via wlan0

I like to leave a gap between rules like this so I can insert an ipfw log command for the eventual case that nothing makes sense and everything is broken.

set up interfaces

A bridge is the center of our guest network, we will give it the default root address that all of our guests will speak to.

# ifconfig bridge create
# ifconfig bridge0 inet up

Our jail will use an epair interface to speak to the outside world. They come as an a and a b part, ifconfig only tells us about the a part when it clones the interface. When we give a vnet jail an interface it is no longer visible to the host system. An epair gives us two interfaces that act like a virtual ethernet cable, we stick one end into the jail and the other is connected to the bridge.

# ifconfig epair create

Our virtual machine will use a tap interface to access the world. The tap interface needs to be brought up. There is a helpful sysctl that is off by default which will trigger the interface to be brought up when it is first opened. I like to set this to one, otherwise I find myself debugging networking inside the VM alot with little success.

# ifconfig tap create
# sysctl

With all the interfaces set up we need to add them to our bridge.

# ifconfig bridge0 addm epair0a addm tap0

Create jail

Never spoken about is the bsdinstall jail command. It takes a directory and installs a jail into it. This command will ask you some questions, it would be cool if it didn't, that would make automating jail creation in scripts much easier for me.

# mkdir testjail
# bsdinstall jail testjail

We make our jail persist so it will stick around as we experiment. The following command creates the jail on the host:

# jail -c name=testjail persist vnet path=testjail vnet.interface=epair0b 

Now we can jexec into the jail and configure the epair. When you bring one end of an epair up, the other end comes up, when it goes down the other end goes down. We just need to configure an address and a default route in our jail.

# jexec testjail sh
[testjail] # ifconfig epair0b inet up
[testjail] # route add default
[testjail] # ping -c 1
[testjail] # ping -c 1
[testjail] # ping -c 1

With this setup the jail can speak to our bridge, the local network and the wider Internet.

Create and config a VM

The FreeBSD offers prebuilt virtual machine images, The latest current one is available from a url like this:

# fetch

It would be cool if there was a latest symlink that gave you a new head VM from one static place. The image comes xz compressed, we need to unpack it and I like to move it to a consistent place:

# xz -d FreeBSD-13.0-CURRENT-amd64.raw.xz
# mv FreeBSD-13.0-CURRENT-amd64.raw /vms/freebsd-current

bhyve requires we load the vmm kernel module, with that we can use the excellent script to launch our vm.

# kldload vmm
# sh /usr/share/examples/bhyve/ -c 4 -m 1024 -t tap0 -d /vms/freebsd-current freebsd-current

Once that comes up you can log in and do some manual config.

[vm] # ifconfig vtnet0 inet up
[vm] # route add default
[vm] # ping

For DNS in both the jail and the virtual machines I have to manually set up the name server local from my network.


search lan

This won't be valid as I move to other networks, but I am sure I will remember after only a little confusion and debugging.


That is all it takes. The NAT configuration is 3 firewall rules and enabling forwarding. None of this is persistent and that isn't great practice for a production environment, but it you just want to experiment with ipfw and NAT, or spin up a VM for today knowing how to do this in a non-persistent way is really helpful.

July 01, 2020 12:00 AM

June 30, 2020



Debian as a Ham Radio Station Laptop

This post is part of [tj]’s blog more in June challenge. You should join us and take part - just make 4 posts in June! As part of the general move to do more radio / do radio in a field, I’ve been doing radio in a field instead of preparing for it. This laptop was first set up and somewhat extensively tested at home, and then used in the field to prove it worked.

June 30, 2020 12:00 PM

Tom Jones


Quick and Dirty Network Scanning

Ever want to scan a subnet in the nosiest, least reliable way and generate too many processes while doing so? Yes? Well do I have a script for you:



if [ -z $1 ]

pinghost ()
        ping -t 1 -c 1 $1 > /dev/null
        if [ $? -eq 0 ]
                echo hit $1

for x in `jot 254`
        pinghost $prefix.$x &

I wrote this while I was doing hack the box challenges and it was a fun and quick way to look to actually find things on my test network. I do not recommend using this. Some operating systems won't let you run it twice in succession as it generates a lot of processes.

June 30, 2020 12:00 AM

My Streaming Setup

Streaming Title Card

In April and May I did some streaming on twitch of hardware hacking projects. I started this as a way to work through the material for my cancelled BSDCan hardware hacking tutorial. With the tutorial cancelled I have been left with quite a few NanoPi Neo LTS and I was thinking about doing the tutorial as a series of videos with the idea of selling intro kits with the boards.

So far I have done four streams aiming for about an hour for each. I will say now that I haven't streamed again in June, I am not saying it is the end, but I (and I bet you too) need to control my commitments or I just never get anything done.

I have also been asked to write up my streaming set up so other people can use it. This vanity pic I tooted seems to include most of it.

Streaming Setup


I have a bunch of equipment because I want to stream stuff in the real world. If you just wanted to share some windows and your webcam as Kristof Provost does then you can get by just with your laptop.

  • Canon 600D
  • Nikon J1
  • MacBook Pro 13,1 for streaming
  • Zoom H3 microphone
  • Aputure AL-M9 Mini LED Light
  • Tripods for the cameras
  • An arm for the microphone
  • LKV373 HDMI as a capture device for a camera

This is quite a lot of stuff, but other than the Aputure light I had all of it already from other projects. I would like to be able to capture the display of my work oscilloscope, thankfully the scope has VGA out so it is just a matter of figuring out a way to capture.


I am using Open Broadcast Studio (OBS) on macOS to mix video and feed it to twitch.

OBS supports a rich variety of input sources that can mixed to make a scene. I have been using three scenes to make it easy to setup what is streamed in advance:

  1. Everything
  2. Just the camera
  3. Just the presentation

Everything is the mixed view of my terminal, the camera output and the slides in a web browser. My tutorial slides are derived from my EuroBSDCan Tutorial and render in a web browser natively. Having a browser in the video mix quickly turned out to be really helpful.

OBS has been rock solid and I have had no problems with stability while streaming. The interface is a bit of a maze, but I suspect that is a natural result of the power it offers.

Canon Camera

For most of the top down shots I used a canon 600D with a 18-55mm kit lens. The battery in this thing is quite old and only manages about an hour and a half of video output. I have since gotten a USB powered battery insert that should allow me to run forever, but am yet to try streaming with it.

My canon camera does not support acting as a webcam, but gphoto2 does support grabbing live video from it. This should have been easy to feed into OBS, but I couldn't get it working and ended up instead using ffplay to render the video and grabbed that with a video capture. To get video from gphoto2 into ffplay I ran:

gphoto2 --stdout --capture-movie | ffplay -

This turned out to be very stable and easy to set up. There is quite high latency between the capture output and the gphoto2 capture, but it worked fine if I didn't have to move the camera much.

Nikon Camera

I also have a Nikon J1 and while the camera is awful to use I do have a 10mm lens for it with a macro extender. This allows me to take very high detail pictures of PCBs. I wanted to add this to the streaming set up too.

The Nikon firmware is garbage and while gphoto2 does support the camera I found I could only get a single image from the camera before having to reset it and was completely unable to get video from it. Never mind, my 2018 EuroBSDCon Talk used a HDMI network extender as a capture device. In the stream Crimes Against Computers3 I did something awful with a BeagleBone Black and got video from the Nikon through the HDMI extender into OBS.

This was a lot of faff to setup (though I since did get a USB-C Ethernet for the MacBook) and I am still kind of dubious of the stability of the LKV373 so I have only used this in the stream where I figured it out. I might try this again later.

Since doing that stream cheap USB2 and USB3 HDMI capture devices have appeared. These are supposed to appear as UVC webcams to the system. I suspect these will be a better method than the LKV373, but I want to try before recommending them.


For audio I am using a Zoom H3 dictaphone thing. I really like the audio from the Zoom, but I seem to be in the minority. In the end it is the microphone I have.

It is stereo which should be mixed down or you will drift across the channels when you move your head. OBS supports this so it wasn't a problem, but I did have to be told about it.

The firmware for this thing isn't so great either, it cannot record to the Micro SD card while acting as a USB interface and Annoyingly there is a bug where it defaults to 44KHz for the audio. The USB driver doesn't seem to advertise this to the properly and if you continue with the default you get weird audio. I dealt with this before in FreeeBSD, but was surprised to see it appear in macOS. You should always double check your audio before starting a stream.

Why not stream from FreeBSD?

I wanted a pain free approach where I could set up and go for my first streams so while OBS is ported to FreeBSD I expected it to be a lot more work than the much more common macOS/OBS combination. I expect FreeBSD/OBS support to get better if people continue streaming FreeBSD stuff.

The MacBook is similar specs to my Thinkpad, but the wireless card in the Thinkpad doesn't have great support and only manages 80211g rates. That might just be enough to stream out the video, but it seems very risky to me.

I will look at streaming with OBS from FreeBSD in the future.

The streams have done okay so far, there is a big social media boost network of FreeBSD users and developers and that has helped people find the content. I am sure the next stream will have fewer viewers as there has been a gap in my streaming.

You can support this

Finally, I set up a ko-fi page to support me writing and streaming. I plan to write more this year and if there is interest I will do more streams too. The streams so far have mostly been using equipment and parts I already had. Doing more is going to require me spending money.

If you have enjoyed my streams, or if you just want me to do more a very solid signal of your support would be tipping me a cup of coffee or something through ko-fi. You can support my hardware habbit here.

June 30, 2020 12:00 AM

June 28, 2020



HF Portable Setup V1

This post is part of [tj]’s blog more in June challenge. You should join us and take part - just make 4 posts in June! As part of the general move to do more radio / do radio in a field, I’ve been doing radio in a field instead of preparing for it. Here’s a breakdown of the portable station, and what I’ve been doing with it. Radio My Radio is a Yaesu FT-891.

June 28, 2020 09:00 PM

June 27, 2020

Tom Jones


Command Line Bug Hunting in FreeBSD

FreeBSD uses bugzilla for tracking bugs, taking feature requests, regressions and issues in the Operating System. The web interface for bugzilla is okay, but if you want to do a lot of batch operations it is slow to deal with. We are planning to run a bugsquash in July and that really needs some tooling to help any hackers that show up process the giant bug list we have.

Thankfully there is a python3 command line tool for interacting with bugzilla, called pybugz. bugz allows you to search through, up date and modify bugs without having to use a web browser. Getting bugz going was not very intuitive and it took me a bit of faffing.

bugz ships with a configuration for connecting to the FreeBSD bugzilla you use it by selecting it as a connection. The supported connections are dumped out if you try and do an operation with the -d 3 debug flag. This flag is really helpful for figuring out how to use bugz because while it documents itself, it holds on to the documentation like a powerful secret.

bugz really wants you to authenticate before you do anything, it won't show you help for commands without auth. There is however a --skip-auth flag. With this you can search for bugs, lets look for ipv6 issues with a patch in the base system:

$ bugz --connection FreeBSD --skip-auth search --product "Base System" "patch ipv6"
 * Info: Using [FreeBSD] (
 * Info: Searching for bugs meeting the following criteria:
 * Info:    product              = ['Base System']
 * Info:    status               = ['New', 'Open', 'In Progress', 'UNCONFIRMED']
 * Info:    summary              = ['patch ipv6']
88821 bugs                 [patch] IPv6 support for ggated(8)
186133 bugs                 [patch] tcpdump(1): zero checksums are invalid for UDP over IPv6
174225 bugs                 [network.subr] [patch] add support for ipv6_addrs_IF style aliases to rc.conf(5)
178881 bdrewery             [patch] getifaddrs(3) does not report IPv6 addresses properly in 32-bit compatibility mode
180572 rc                   [network.subr] [patch] SLAAC is enabled for ipv6_cpe_wanif
133227 bugs                 [patch] whois(1): add support for SLD whois server lookups and IPv6 address lookups
104851 bugs                 [inet6] [patch] On link routes not configured when using both IPv6 autoconfiguration and manual configuration
147681 bugs                 [network.subr][patch] Add inet6 keyword if it wasn't specified in ifconfig_IF_ipv6
130657 bugs                 [ip6] [patch] ipv6 class option
165190 bugs                 [ipfw] [lo] [patch] loopback interface is not marking ipv6 packets
245103 bz                   [patch] [ipv6] IPv6: update v6 temporary address lifetime according to rfc4941bis
 * Info: 11 bug(s) found.

We can also filter our search by component:

$ bugz --connection FreeBSD --skip-auth search --product "Base System" --component bhyve 

bugz supports modifying and updating bugz from the command line, this is the main focus of the README on github. To authenticate bugz takes a username and password on the command line, I am not suggesting you fill your history with your bugzilla password, I did something like:

$ bugz --connection FreeBSD -u --password `pass show FreeBSD/bugz | head -n 1` search udp

There is a -k flag that takes a key file, but I didn't want to dig into the source of bugz to figure out what this actually is.

Our bug squash is probably going to focus on clearing bugs with patches and the readme has a workflow for finding bugs and grabbing any diffs. More tools can and should be written around bugz this is just a start. Just playing with this I have spotted bugs than can easily be closed from the tracker.

Finally, you are going to need the help while using bugz, it took me longer than I liked to figure out that each sub command documents its own help and they all take the -h after the command. You need auth (or to skip auth) before you can use this flag.

$ bugz --connection FreeBSD --skip-auth search -h

June 27, 2020 12:00 AM

June 26, 2020

Tom Jones


FreeBSD on the Intel 10th Gen i3 NUC

I have ended up with some 10th Gen i3 NUC's (NUC10i3FNH to be specific) to put to work in my testbed. These are quite new devices, the build date on the boxes is 13APR2020. Before I figure out what their true role is (one of them might have to run linux) I need to install FreeBSD -CURRENT and see how performance and hardware support is.

They have an Intel i3-10110U with 2 cores and 4 threads at 2ish GHz (with 4GHz boost), I got a single 32GB DIMM of RAM for each and a 480GB Western Digital M.2 SSD. This configuration came in just under £500 for each NUC.

The NUCs are pretty small, they have pretty beefy fans taking up about a cm of the top of the enclosure. They certainly aren't silent, without any load I could hear the NUC sat on the desk next to me. When building at full steam the fan in the NUC is about as loud as my x270 Thinkpad is when it is building.

What works?

Out of the box I had to break into the bios and disable secure boot to boot the FreeBSD installer. I did this by hitting every FN key as the NUC booted, I think FN2 was the correct choice. At the time my keyboard was being fought over by USB and Bluetooth on my MacBook Pro.

FreeBSD install was problem free. I set up the M.2 as a single drive with ZFS, datasets and snapshots are a magic power.

Before I tried anything else I had to get an idea at how well this NUC would build FreeBSD. I don't expect this to be a build machine, but having spent a while shopping for build machines recently (I settled on a VM in hetzner) it is the only benchmark I really care about. make -j4 buildworld buildkernel took 2:45, that isn't the fastest in the world, but it is about an 1:15 faster than my x270 with its 2015 i5-6300U. The difference 5ish years makes.

Graphics in this 10th Gen Intel processor wasn't supported by drm-current-kmod and I had to install drm-devel-kmod. With the devel kmod the NUC is happy to push all the pixels of the 4k TV I have here and even drive additional second monitor connected with USB-C->HDMI adapter at the same time.

Audio works through the front 3.5mm jack and after changing hw.snd.default_unit from HDMI too. There is an issue where when the display is blanked (turned off), audio will stop, but won't come back when the display does. manu@ suggested setting hw.i915kms.disable_power_well=0 in loader.conf, this resolved the problem for me.

The Intel 9462 WiFi in the NUC is not supported (yet), I stuck a cheap rtwn based USB WiFi dongle in the front port. External USB WiFi is cumbersome on a laptop, but on a machine that will rarely move it is fine. The onboard Ethernet is supported by em and is happy to do 940ishMbit/s with iperf3 with TCP and UDP.

The NUC has 5 USB ports, 3 USB-A and two USB-C. The USB-C port on the back has a little lightning bolt next to it and I imagine that means it is the one with thunderbolt support. I did a disk speed test to a Sandisk Extreme Portable SSD. The front port managed 200MB/s or so while the rear thunderbolt one only got to about 160MB/s for read and write. My MacBook Pro manages the advertised 500MB/s read speeds, so I need to dig into whether this is a hardware problem or a FreeBSD problem. Luckily I have two so installing Linux to test performance won't be a bother.

I quite like these little boxes, you can get more hardware for your money with a tower pc, but not in this form factor. I wouldn't be surprised if the i7 versions of these were quite good at building FreeBSD. The fans are a bit loud, but that can be fixed for me by playing blaring techno.

I think the NUC in this configuration is going to make a great test node in my satellite testbed or a nice little desktop box for doing video tasks.

Hardware Support

Storage         Yes     M.2 is fine, I didn't have a drive to test SATA
Graphics        Yes     drm-devel-kmod (on r362612)
Audio           Yes     hda(4)
Ethernet        Yes     Supported by em(4)
Wireless        No      Intel AC 9462 not supported :(
USB             Yes     All ports work, 
Suspend/resume  Yes     suspend is fine, resume required drm-devel-kmod
SD slot         Yes     Comes up as a mmcsdX device
Thunderbolt     N/A     I have no idea what the implications of this are or how to test

June 26, 2020 12:00 AM

June 13, 2020

Tom Jones


Capturing a screen sub section with ffmpeg

June 13, 2020 12:00 AM


Last updated:
October 24, 2020 04:34 PM
All times are UTC.

Powered by: