Engineering
From RTL to user space, and why the order matters
Five layers of stack on a single board. The engineering lesson was not about hardware; it was about the discipline of bringing up one layer at a time.
In short
- Bring up one layer at a time and verify each before adding the next. Layers have their own tools and failure modes, and debugging two at once produces bugs you cannot localize.
- A well-built driver makes the hardware indistinguishable from a file. That boundary is what lets application authors and hardware authors work without knowing each other’s problems.
- Do not hand-roll a bus interface that a vendor template already implements correctly. The upside is zero and the downside is weeks.
- Polling against interrupts is a latency-versus-CPU-cost decision that depends on the operation’s duration, not a matter of taste.
- Understanding the full path from gates to system calls turns most embedded and integration work from mystery into a remix of pieces you have already built.
On a system-on-chip with a processor and programmable logic on the same die, a custom peripheral is the hardware equivalent of a driver: it lives on the bus, exposes registers at known addresses, and does something the processor is comparatively bad at. Our engineers have built one end to end, from the register-transfer-level description through the bus interface, the embedded Linux build, a kernel driver, and a user-space program that talks to it.
The peripheral itself is deliberately unimpressive: a multiplier. Against a software loop the speedup is modest. That is the point. A trivial function keeps every one of the five layers visible instead of hiding four of them behind a vendor block, and the layers are the lesson.
The five layers, and what each one owes the next¶
| Layer | What it owns | The check that has to pass first |
|---|---|---|
| The logic | A small state machine with three registers, two operands and a result, plus a control register carrying a start bit software sets and a done bit hardware sets. | Simulate it before synthesizing it. |
| The bus interface | A handshaking protocol with separate channels for read address, read data, write address, write data and write response, so the processor can reach those registers as memory. | Read and write the registers from a bare-metal program, before an operating system exists. |
| The build | Synthesis turns the description into a bitstream that configures the fabric at boot; the software-side tooling produces the register map and the device description the kernel needs. | Boot the image and confirm the device is enumerated, before any driver is loaded. |
| The kernel | A driver that maps the register window, exposes a character device, and translates reads, writes and control calls into register accesses. | Reach the same registers through the driver, before an application exists. |
| User space | Open the device, write the operands, read the result, close. | Nothing left underneath it to blame. |
Every one of those boundaries is a contract, and every one of them is the same kind of contract we draw in a distributed system or an integration layer. What differs is only how unforgiving the enforcement is.
Do not rebuild what the vendor got right¶
The bus interface is a textbook exercise that everybody gets wrong the first time, because the specification permits a master to interleave channel transactions in ways that are technically legal and rare enough in practice that a hand-rolled implementation appears to work until it does not. The vendor toolchain ships a correct template for exactly this reason.
Use it. The wins available from writing your own are zero and the losses are measured in weeks. We say the same thing to clients about authentication, payment handling, cryptography and date arithmetic, and we say it with the same confidence, because in each case the interesting work is above the layer, not inside it.
The driver is where the abstraction gets made¶
With the driver loaded, the user-space program is six lines: open, write two operands, read a result, close. The exact same shape as talking to any other character device. From user space, a hardware multiplier running on programmable logic is indistinguishable from a file.
/* Five layers sit under this file, each brought up and verified before the next: Verilog RTL, AXI4-Lite bus attachment, device tree, kernel character driver, and then this. */int fd = open("/dev/axi_acc0", O_RDWR);write(fd, &(uint32_t){ 42 }, sizeof(uint32_t)); uint32_t out;read(fd, &out, sizeof(uint32_t));printf("%u\n", out); /* 84 */close(fd);That is the whole value of the layer. The application author does not need to know the accelerator exists. The hardware author does not need to know what the result is used for. Neither can break the other by changing something local. When people talk about a good interface, this is the concrete version of what they mean, and it is worth having built one where the boundary is enforced by silicon rather than by convention.
The driver is short, and the interesting content is three decisions rather than the code. Polling against interrupts is a real trade: an operation that completes in tens of nanoseconds wants polling, because interrupt overhead dominates, and a longer operation flips it, so a driver worth keeping supports both. Mapping the register window directly into a process skips the system call and gives up isolation, which is a per-peripheral judgement rather than a default. And the driver matches hardware by a compatibility string, which is the single most common reason a freshly built kernel does not see a freshly built peripheral.
The order is the method¶
The instinct on a five-layer bringup is to work on several layers at once, because each has idle time while another builds. The instinct is wrong, and expensively so. Each layer has its own tools, its own failure modes and its own vocabulary. A bug that could be in the logic, the bus interface, the device description or the driver is a bug you cannot localize, and localization is most of debugging.
Simulate the logic before synthesizing it. Confirm register access from a bare-metal program before booting an operating system. Boot the operating system and confirm the device is enumerated before loading a driver. Load the driver and confirm register access through it before writing an application. Each step has a check that either passes or fails on its own terms, and the first build takes an hour while every subsequent one takes minutes.
Why a software firm keeps this on the shelf¶
Most of our work is several levels above a bus transaction. But manufacturing and utility clients hand us problems that sit right on this boundary constantly: reading a programmable controller, talking to a sensor over a serial line, understanding why a gateway drops packets under load, judging whether a vendor’s integration claim is plausible.
Engineers who have built the whole path from gates to system calls treat those as tractable problems with known shapes. Engineers who have only ever worked above the operating system treat them as somebody else’s department. On a plant floor, somebody else’s department is where projects go to stall.
Related reading
- Systems6 min read
Consensus is not the hard part. Reconfiguration is.
Every distributed-systems reading list ends where the real engineering starts. What breaks in production is the day the topology changes.
- Engineering5 min read
The shortest path is usually the wrong one
A routing problem taught us more about requirements gathering than about geometry. The algorithm was never the hard part; the objective was.
Next step
Tell us what’s breaking.
Forty-five minutes, no charge, no deck. We’ll tell you what we’d do, what it would likely cost, and whether you should be building this at all.