From 187 Tests to Silicon

In software, test-driven development is unremarkable. Write the failing test, write the code that makes it pass, refactor. Millions of developers do it every day.

In hardware it is rare, and the reason is structural: the feedback loop is long and the last iteration is expensive. A compile error costs seconds. A synthesis error costs minutes. A bug found on a fabricated chip costs a shuttle slot and several months. So the discipline that software gets to treat as a style preference is, in silicon, the only affordable way to work.

The CLI-CPU is a processor that runs .NET CIL bytecode natively, with no JIT and no interpreter. It was built test-first from the beginning. Here is what that actually looks like, measured on the repository as it stands today.

A test suite is not a quality ritual. On a chip project it is the only mechanism that tells you the RTL still means what the specification meant.

The specification is an executable program

Before a single line of Verilog existed, the CIL-T0 instruction set had a reference implementation: CilCpu.Sim, a C# simulator of the Nano core. It is not a documentation artifact. It is the canonical definition of what every opcode does to the evaluation stack, the frame pointer, the call depth, and the trap conditions.

That simulator carries 187 green xUnit tests today, across the simulator core, the CIL-T0 linker, the runner, the tracer, and the trap semantics:

$ dotnet test src/CilCpu.Sim.Tests/CilCpu.Sim.Tests.csproj

Passed!  - Failed: 0, Passed: 187, Skipped: 0, Total: 187

The proportions are worth a look. The simulator is 3,684 lines of C#; its test suite is 3,391 lines. Roughly one line of test for every line of implementation, written first.

The golden vector: how the RTL is held to the simulator

Once the Verilog exists, the interesting question is not "do the RTL tests pass". It is "does the RTL mean the same thing as the reference". Those are different claims, and only the second one matters when the design goes to fabrication.

The golden-vector harness answers it directly. The C# simulator emits a JSONL execution trace — one entry per instruction, recorded before the instruction executes. The cocotb testbench then runs the same program on the Verilog core under Verilator, and on every ST_DECODE cycle it compares six architectural signals against the corresponding trace step:

A mismatch fails the test with both sides printed — trace pc/sp/fp/cd/ac/lc against RTL pc/sp/fp/cd/ac/lc, at a named step and cycle. There is no interpretation step, no "looks about right". Either the hardware walked the same path as the reference, or the test names the exact instruction where they diverged.

This is the mechanism that turns a simulator into a specification with teeth. The CALL/RET golden test, for example, follows a call depth transition of 1 → 2 → 1 with an argument count of 0 → 2 → 0 across eight steps. Any RTL change that quietly breaks frame bookkeeping fails there, immediately, with the failing step identified.

What the hardware suite measures today

The RTL side is verified by cocotb against Verilator: 322 test functions across 24 testbench modules, covering the ALU, the decoder, the microcode sequencer, the stack cache, the multiplier, the divider, the QSPI controller, the mailbox, GPIO, the trace MUX, the UART transmitter and receiver, the boot controller, the decimal printer, and the assembled SoC.

The size ratio is the part that surprises people coming from software:

There is 1.6 times more testbench code than RTL. That ratio is not an accident of over-testing; it is what it costs to state, in executable form, what a piece of hardware is supposed to do in every cycle of every corner case.

The same function, three layers deep

The end-to-end test is deliberately unglamorous: a Fibonacci function, written in ordinary C#, in samples/PureMath. What makes it useful is that one source file feeds three independent verification layers, each one closer to physical reality than the last.

C# source (samples/PureMath, Math.cs)
    ↓ Roslyn
.dll (CIL bytecode)
    ↓ TCliCpuLinker
CIL-T0 binary
    ↓
│ layer 1  C# reference simulator     Assert.Equal(6765, result.Result)
│ layer 2  Verilog core / Verilator    return_value == 55  (recursive Fib(10))
└ layer 3  XC7A200T FPGA, real UART    36 37 36 35 0D 0A  =  "6765\r\n"

Layer 1 is RunBinary_Fibonacci20_Returns6765 in the xUnit suite: the iterative Fibonacci binary run through the reference simulator, asserting 6765 for n = 20.

Layer 2 takes the recursive variant, links it with the same toolchain, and boots it on the Verilog core under Verilator, asserting a return value of 55 for n = 10. The recursive form is the deliberate choice here: it is the case that stresses frame bookkeeping hardest — caller evaluation depth preserved across nested CALL/RET, stack cache spill and refill, header reserved fields. A core that handles iteration fine can still be quietly wrong about recursion, and this test is where that shows.

Layer 3 is the physical board: the same class of binary in config flash, printing to a real serial port. The decimal conversion itself has its own RTL test — the printer module turns 6765 into the byte sequence "6765\r\n", verified in cocotb independently of everything above it.

One function, three layers. When a change breaks the semantics, it breaks the cheapest layer first — which is exactly the point.

Why the discipline pays for itself

The economics of a hardware bug are not linear. They are stepwise, and each step is roughly an order of magnitude:

This is why the project's execution order puts FPGA validation (F2.7) before the silicon synthesis flow, and why the rule is stated plainly in the roadmap: no silicon tape-out with a design that hasn't run on FPGA. Bugs found on an FPGA are free. The same bug on a fabricated die is a dead chip and a lost shuttle slot.

The first FPGA run made this concrete: three bugs that every simulation had passed cleanly surfaced within hours on real hardware — a synthesis-only part-select issue, a timing constraint, and a flash chip whose Quad Enable bit ships as zero from the factory. None of them were logic errors. All of them were assumptions. Simulation checks your logic; hardware checks your assumptions; and the test suite is what lets you tell the two apart.

What TDD does and does not buy you

It does not prove the design is correct. No test suite does. What it buys is narrower and more useful: when something breaks, you know within seconds which layer broke it. A failing xUnit test means the specification changed. A failing golden-vector test means the RTL drifted from the specification. A failing FPGA run means an assumption about the physical world was wrong.

For a small team building a processor with an open toolchain and a modest budget, that separation is not a luxury. It is the difference between a design that can be changed and a design that can only be admired.