The firmware contract

This is the whole of what passes between sim-mesh and a firmware. A firmware that keeps it runs on sim-mesh with nothing of it in sim-mesh; sim-mesh that keeps it runs any such firmware. Must, must not and may are requirements; everything else describes. It is also sim-mesh’s README, The firmware contract.

project ── its own script ──► <base>_<arch>_<version>.zip ──► sim firmware add ──► firmware/<name>/
script  ── nodes().firmware("<base>_latest") ──► simd: resolve, import the zip's driver.py, DRIVER(firmware)
simd    ── start the executable: SIM_MESH_* env, LD_LIBRARY_PATH (the radio), LD_PRELOAD (the time shim)
station ── libsimradio-sx1262.so ── UDP, JSON ──► the ether          hello, state, tx, idle …
simd    ── driver.wait_up / verbs / flush ──► the station          over its console, a tool, a file …
station ── console lines ──► driver.console_line ──► station.report(event) ──► run/events.jsonl

• 1. The firmware zip
• 2. node.yaml
• 3. What a station is given
• 4. The process
• 5. Time
• 6. The driver
• 7. The reticulum category
• 8. The ether’s protocol
• 9. The virtual radio
• 10. Building against it
• 11. A whole example

1. The firmware zip

A firmware is one station build for one architecture: a zip archive named

<base>_<arch>_<version>.zip

• base: lower-case letters a–z, digits, - and ., starting with a letter or digit; never _. Two builds that differ in anything sim-mesh does not read, the virtual radio they drive among it, are two bases; by custom the base ends with the radio (relay-sx1262).
• arch: the architecture it runs on, as uname -m spells it on Linux: aarch64, x86_64.
• version: either a build stamp, the UTC time of the build as YYYYMMDDhhmmss (fourteen digits), or a semantic version MAJOR.MINOR.PATCH with an optional -pre-release. A base must keep to one of the two.

The name splits at its first and its last _. Installed, the firmware is the directory firmware/<base>_<arch>_<version>/, the zip unpacked, so a zip and its installed directory have one name. <base>_latest means the newest installed firmware of exactly that base for the machine: by stamp, or by semantic version (a pre-release before its release).

Members sit at the archive’s root, with forward-slash paths that must stay inside it (no absolute path, no ..). It holds node.yaml, the executable, the driver, and whatever else the station runs. A member’s Unix permission bits, where the archive records them, are the file’s; the executable is made executable whatever the archive says.

A zip is refused when it is built for another architecture than the machine’s, when its base already holds firmware versioned the other way, when a firmware of its name is installed already, and when its node.yaml, or anything it names, is missing.

2. node.yaml

A YAML mapping at the archive’s root.

Other keys are kept and not read.

3. What a station is given

sim-mesh starts the executable with its argument list from the driver (§6), its working directory its own (below), its stdin and stdout one pty (§4), and this environment: the host’s, then these, then node.yaml’s env, then the driver’s env.

A station reads its identity from these and from nowhere else, so two stations on one host never collide. A station never learns where it stands: its position is the ether’s and the loss table’s alone.

What a station may count on. sim-mesh runs everything in its own image, the same on every machine: Ubuntu 24.04 of the host’s own architecture. An executable may count on its C library and C++ runtime (libc, libm, libstdc++, libgcc_s), on the radio library node.yaml names, on the time shim, and on the image’s python3, CPython 3.12 with its standard library (its driver runs in it, with aiohttp and pyyaml beside sim_mesh). Anything else it loads must be in its zip: a shared library under lib/ with env: {LD_LIBRARY_PATH: ./lib}, a Python package under a directory it puts on PYTHONPATH, built for CPython 3.12; a firmware written in Python may use the image’s interpreter and need bring none. A zip must not carry a radio library.

4. The process

• stdin and stdout are the console: a pty, text, shown in the map’s console window and appended to log in its directory, and handed to its driver line by line (console_line). The one binary thing that may cross it is a framed RPC frame, which sim-mesh takes out of the stream before anything else sees it (§6). A driver may ask for a pipe each way instead (console_tty).
• Exit to reboot. sim-mesh starts the executable again, on the same directory and address, half a second later (of T, in a virtual-time run).
• State is its directory. A new simulation starts it with an empty state/, or with one a snapshot kept; a factory reset empties it; a reset leaves it.
• Ports are its own, on its own address. A web UI is on the port its driver names (web_port), reached through sim-mesh’s proxy as <node>.<simulation>.sim.localhost:8800.
• It may be several processes. A process it starts that waits on time joins a virtual-time run as a station of its own, under the id its driver names for it (sids), with the same address, the time shim and SIM_MESH_IDLE=threads: it opens its link (simradio_station_open) and, with no radio, no chip. Which of them reads the console is the driver’s to name (console_sid).

5. Time

In a real-time run a station keeps the host’s time. In a virtual-time run the ether owns conductor time T and moves it only when every station is idle (§8), and a station keeps four promises:

• It reads time only through the C library or the radio library. The time shim answers clock_gettime, gettimeofday, time, the sleeps, setitimer and the timeouts of poll, select, epoll_wait, pthread_cond_timedwait and sem_timedwait/sem_clockwait in node time. A raw rdtsc, a clock_gettime made by system call, or a wait none of those is does not move with the run.
• It opens its link early, before anything in it waits on time: simradio_station_open is where its clock starts and the shim attaches; until the ether’s welcome the monotonic clocks read 0 and the wall clocks the run’s epoch.
• It says when it is idle, and until when. Idle is every thread blocked; the until it reports is the earliest wake anything in it holds (simradio_wake_at). Either the host calls simradio_idle() itself, a scheduler’s idle with a wake at its next due tick and timer, or its driver sets SIM_MESH_IDLE=threads and the shim keeps a census of its threads. A station that says neither is reported idle by the radio library’s watchdog, 20 ms of wall time after every message once none of its threads is on the CPU: it runs, but crawls.
• It reads its console and talks TCP to other stations through the C library: read on descriptor 0, and read/recv/send/write on its sockets. The shim counts those bytes for the ether, which holds T until a station has read what it was sent. Input taken another way holds T a second of wall time each time.

In return, nothing the ether says reaches the host piecemeal. The radio library applies a datagram whole, the chip’s own timers running as T moves, before the host is told of any of it: the host’s waits that fall due, simradio_on_advance and DIO1 come after, so a thread woken at T finds all of T.

6. The driver

The driver is a Python module in the zip, imported by sim-mesh from the installed firmware’s directory under a module name of its own. It must define DRIVER, a subclass of its category’s driver class (§7), which subclasses sim_mesh.driver.Driver. It must import from sim-mesh only sim_mesh.driver and its category’s module; anything else in sim-mesh may change under it. sim-mesh makes one DRIVER(firmware) per installed firmware a run uses, shared by its stations.

What a driver is given. self.firmware: dir, exec, fixed, env (paths made absolute), driver, name/firmware, base, arch, version, category, radio, title, hardware. Every call names a station, which offers:

And from sim_mesh.driver: CommandError (anything that could not be done; its text is shown as it is), run_tool(argv, timeout, env=None) (a helper program to completion, env added to sim-mesh’s environment), chip_dbm(board, connector_dbm) (the chip power that puts that much at the connector through the board’s front end), parse_setting, UP, and the framed RPC constants.

What a driver implements.

Its helpers: await self.pause(station, s), a wait on the run’s clock; await self.joined(station, timeout), True once the station has joined the ether, at the T of its hello (at once in a real-time run), so what the driver does next it does at that T; with self.tool_turn(station): around a tool run against the station’s host door, during which T stands while the tool has the floor and runs while the station works on what it read; await self.rpc_ready(station, timeout, marker_wait_s) and await self.rpc_query(station, line, timeout=None), framed RPC.

Framed RPC on the console.

station → sim-mesh   "… framed rpc v1"                        once, early in boot, as log text
sim-mesh → station   F5 53 47 01 <id> <len:2> <a command line>
station → sim-mesh   F5 53 47 01 <id> <len:2> <what it printed>

A firmware may multiplex a framed side channel onto its console: a frame is never echoed and never enters its line editor, and sim-mesh takes each reply frame out of the stream, so the log and the console window never see one. One frame is in flight per station; a command line is at most 4096 bytes; the id is a hash of the command, so a retry carries the one it had (Framed RPC on the console).

7. The reticulum category

A firmware of category reticulum is a Reticulum node. Its DRIVER subclasses sim_mesh.reticulum.driver.ReticulumDriver and implements these verbs, each taking the station first: every firmware’s (name, radio, radio_up, tx_power, diagnostics, from sim_mesh.driver.Driver), then the category’s. A verb it cannot do raises CommandError (self.cannot(verb)), which the default does. A script reaches the category’s as <selection>.reticulum.<verb>, and they are nothing to a station of another category.

A verb with a dot is the method with an underscore: lxmf.send is lxmf_send.

Events. For every message lxmf.send was given, the driver must report how it ended, under mid, from console_line or however else it learns it:

self.lxmf_status(station, mid, "delivered")
self.lxmf_status(station, mid, "failed", why=<text>)

and may report the statuses between (pending, sent); the last one reported is what a traffic report shows for a message that never arrived. Each is the event lxmf.message.status with mid, status and why. A firmware’s own id for a message is coupled to mid with self.lxmf_couple(station, mid, its_id) once the driver learns it, and what the station says under its own id is given to self.lxmf_native(station, its_id, status, why), which reports it under mid, holding it until the coupling is made; what goes wrong before the firmware has an id is reported under mid directly. sim-mesh writes each event as a line of the run’s events.jsonl: {"t": <T in µs>, "node": …, "event": …, …fields}.

8. The ether’s protocol

The ether is the medium between stations: it decides who hears each frame, from the run’s loss tables, and in a virtual-time run it owns time. A station speaks to it over UDP, one JSON object per datagram, from a socket bound to its own address; payloads are base64, times in microseconds. The virtual radio (§9) speaks it for a firmware; a firmware may speak it itself (the ether’s README has the whole of it).

A real-time run:

station → ether   hello {sid, slots}
ether → station   welcome {t, mode: "real", rate: 1, epoch, seed}
station → ether   state {slot, mode, mod, freq, bw, sf, cr, sync, hdr, crc, pre}   on every change
station → ether   tx {slot, id, t0, t_pre, t_hdr, t_end, power_dbm, mod, freq, bw, sf, …, payload}
ether → station   rx_begin {slot, id, t0, t_pre, t_hdr, t_end, level[, cad]}    each receiver it reaches
ether → station   rx_end {slot, id, verdict, payload, rssi, snr}               at the frame's end

A virtual-time run is the same conversation with the ether as conductor:

station → ether   hello {sid, slots}
ether → station   welcome {t, mode: "virtual", rate, epoch, seed, seq: 1}
station → ether   idle {seq: 1, until: 25000}            nothing to do before T 25 000
ether → station   run {t: 25000, seq: 2}                 every station idle; T moves to 25 000
station → ether   state {…}  tx {t0: 25000, …}           what it did at 25 000
station → ether   idle {seq: 2, until: 30000}
ether → station   rx_begin {t: 25000, seq: 7, …}         to each receiver, at the same T
receiver → ether  idle {seq: 7, until: …}

• mod is the modulation; a receiver hears only a frame in its own. The ether models lora (with bw, sf, cr, sync, hdr, crc, pre); a state or tx naming another is refused. It does not model preamble length.
• The id in a tx is the transmitter’s own count; in rx_begin and rx_end it is the ether’s, unique across stations.
• Virtual time. Every message the ether sends carries t and seq; the station applies it at that T and answers with an idle for that seq, and messages are applied strictly in seq order, never twice. T moves only when every station is idle, to the earliest of their untils and the air’s next instant. An idle is said again every 250 ms of wall time until something comes back; the ether answers an idle for an older seq said twice by sending what came after it again. A station that says hello again has restarted.
• Unknown message types are ignored, on both sides.

9. The virtual radio

A virtual radio is a model of one radio chip on a virtual SPI bus, and the station’s link to the ether, as a shared library sim-mesh provides: libsimradio-sx1262.so is the SX1262. A firmware is compiled against its header, radio/include/simradio.h, and linked with it by name (-lsimradio-sx1262), and must not carry it: sim-mesh puts its own on the station’s LD_LIBRARY_PATH, so a firmware keeps working when the model or the ether’s protocol changes, and only a change to simradio.h itself needs it rebuilt. It makes the simulation and the hardware do the same thing: the firmware’s own driver talks SPI to it frame by frame, exactly as to the chip on a board.

int  simradio_station_open(int sid, const char *bind_addr, const char *ether_addr);  /* once, early */
simradio_t *simradio_open(int slot, void (*on_pin)(void *ctx, int pin, int level), void *ctx);
void simradio_transfer(simradio_t *, const uint8_t *out, size_t len, uint8_t *in);    /* one NSS cycle */
void simradio_reset(simradio_t *);                                                    /* RST's rising edge */
int  simradio_pin(simradio_t *, int pin);                    /* SIMRADIO_PIN_DIO1, SIMRADIO_PIN_BUSY */
void simradio_set_services(const struct simradio_services *); /* a host's own, before anything else */

int     simradio_virtual(void);          int64_t simradio_node_us(void);
int     simradio_joined(void);           int64_t simradio_node_at_join(void);
int64_t simradio_epoch_us(void);         int64_t simradio_node_to_conductor(int64_t node_us);
int     simradio_wake_create(void (*due)(void *), void *arg);
void    simradio_wake_at(int wake, int64_t node_us);           /* INT64_MAX clears it */
void    simradio_idle(void);
void    simradio_on_advance(void (*moved)(void));

• Open the link once, early: simradio_station_open(SIM_MESH_NODE_ID, SIM_MESH_BIND_ADDR, SIM_MESH_ETHER), before anything waits on time (§5); then simradio_open(slot, …) per radio.
• One whole frame per NSS cycle. A bus adapter hands the model everything the driver put on the bus between NSS going low and going high as one frame: writes inside a frame are appended, never passed on one by one; a frame containing a read is complete at the read; the adapter never adds a NOP of its own. A reassembly one byte off makes GetIrqStatus read the status byte as the IRQ word’s high byte, so every flag looks set.
• Interrupts. on_pin runs on whatever thread moved the line. A pin shim whose interrupt is level-triggered must fire at once when the interrupt is enabled while the line is asserted: a driver that disables its interrupt, drains, and re-enables relies on a line still high re-firing.
• The host’s services. The library needs a clock, one-shot timers, a recursive lock, a UDP socket and a thread to read it, and a log (struct simradio_services in the header). Its own are a plain process’s threads; a host with a scheduler of its own (FreeRTOS on ESP-IDF’s Linux target) must hand it its own with simradio_set_services before any other call (a constructor is the place), so model callbacks run as that host’s tasks.
• The front end is the library’s: what the chip radiates goes through SIM_MESH_BOARD’s transmit curve before the ether is told its power, and every level the chip reads is the front end’s receive gain above the connector’s. A firmware that drives a front end converts with the same figures.

10. Building against it

Start sim-mesh once first (sim, which builds its radio and leaves radio/build/libsimradio-sx1262.so), then build the station for Linux:

# C or C++
cc station.c -I sim-mesh/radio/include -L sim-mesh/radio/build -lsimradio-sx1262 -lpthread -o station

# Rust: link it by name from a build script
println!("cargo:rustc-link-search=native={}/build", radio_dir);
println!("cargo:rustc-link-lib=dylib=simradio-sx1262");

# PlatformIO on Portduino: sim-mesh's radio/portduino library, in lib_deps as
#   symlink://<path to sim-mesh>/radio/portduino
# links the radio by name and stands in for spidev and libgpiod

Then make the zip: the executable, the driver, anything else it runs, the shared libraries beyond the C library and C++ runtime under lib/, and node.yaml. Check what the executable loads with ldd: everything but libc, libm, libstdc++, libgcc_s, the loader and libsimradio-* belongs in lib/. Then sim firmware add it, and run a script with it. A firmware that publishes its zips puts them on the pre-built list with sim-mesh’s tools/deploy-firmware.

11. A whole example

node.yaml:

base: relay-sx1262
arch: aarch64
version: '20261001120000'
category: reticulum
radio: sx1262
title: Relay (dev)
hardware: ESP32-S3
exec: relay
driver: driver.py
fixed: fixed
env:
  LD_LIBRARY_PATH: ./lib

driver.py, for a firmware that answers framed RPC on its console:

import os
import re

from sim_mesh.reticulum.driver import ReticulumDriver

QUEUED = re.compile(r"queued (\S+)")
MID = re.compile(r"mid=(\S+)")


class Relay(ReticulumDriver):
    def configured(self, station):
        return os.path.exists(os.path.join(station.dir, "state", "boot"))

    async def wait_up(self, station, timeout):
        return await self.rpc_ready(station, timeout, 20.0)

    async def run(self, station, line, timeout=None):
        return await self.rpc_query(station, line, timeout)

    async def name(self, station, name):
        await self.run(station, "hostname %s" % name)

    async def lxmf_create(self, station, name):
        return None                 # one identity, there from its start

    async def lxmf_identities(self, station):
        address = (await self.run(station, "address")).strip()
        return [(station.name, address)] if address else []

    async def lxmf_announce(self, station, name=None):
        await self.run(station, "announce")

    async def lxmf_send(self, station, dest, text, mid, sender=None):
        found = QUEUED.search(await self.run(station, "send %s %s" % (dest, text)))
        if not found:
            self.lxmf_status(station, mid, "failed", "not queued")
            return
        self.lxmf_couple(station, mid, found.group(1))

    def console_line(self, station, line):
        found = MID.search(line)
        if found and "delivered" in line:
            self.lxmf_native(station, found.group(1), "delivered")
        elif found and "failed" in line:
            self.lxmf_native(station, found.group(1), "failed", line)


DRIVER = Relay