For four days and two published releases, my own comparison page said the engine loses to Puppeteer on large documents. 500 pages: ~9.7 seconds against ~2.5. 50 pages: ~490ms against ~220. Both numbers were stale. The real figures, sitting in the published benchmark artifact the whole time, had the engine ahead on both.

I didn't catch it, and the reason is the actual subject of this post. Every review instinct I have built into this project points one way: never overclaim. Check that the number is not too good. Re-derive the flattering claim before publishing it. A stale number that flatters the competitor sails through every one of those checks, because it feels safe. It is not lying, it is honest in spirit, and it is still wrong. The discipline that stops you overclaiming will happily let you underclaim forever, because nothing about it forces a re-measure.

Here is how the numbers got measured, fixed, and then left to rot in a losing direction, and what changed so the next drift is loud. The specifics come from the parity artifact and the commits, not from memory, because "from memory" is exactly the failure mode this post is about.

The guess era

Until this month, the published performance numbers were guesses. Not fabrications, but the next worst thing: one-off measurements from one machine, transcribed into prose, never re-run. The landing page said a 6-page report renders in ~26ms. That number was real once. I later learned it was measured on a document doing two full layout passes, one of them wasted, so even the flattering number described the engine doing double work. Nobody knew, because nothing measured.

The harness

So I built a fixed benchmark corpus: six committed HTML documents, from a one-page receipt to a 500-page ledger, each a pure function of constants so regeneration is byte-identical and numbers stay comparable across runs. Everything is HTML on purpose, because HTML is the one input surface shared by every Forme target and by Puppeteer, so engine-versus-engine runs on byte-identical documents.

Each run measures five targets: the native binary, the Node WASM build, the web build in headless Chromium, a real workerd isolate, and Puppeteer as a same-machine control, browser reused across warm iterations the way a production pool would. Two machines are published side by side, a quiet dev machine and a noisy shared CI runner, neither cherry-picked. A document that times out or gets OOM-killed on the shared runner is recorded as that, not silently dropped.

What it found: the engine lost, badly

The first honest run put the losses on the table. On the 500-page ledger, Forme took ~10.0 seconds against Puppeteer's ~2.7 on the same machine, about 3.8x slower. The 50-page invoice: ~534ms against ~232ms. I published those numbers, losses included, on the day I measured them. The comparison page said a warm, pooled Chrome is faster on very large table-heavy documents, which was true, and linked the artifact.

Then I profiled, with a hypothesis: layout must be scaling super-linearly on large documents.

Wrong the first time

The hypothesis was wrong. Phase timing showed layout scaling linearly, at a bad constant. The 3.8x decomposed into two independent linear factors multiplying each other:

First, the ledger crosses page 100, where the page-counter text Page 1 of 500 needs three digits instead of the two the engine had reserved. Correcting the reserved width forces a full second layout pass. Factor of ~2x.

Second, allocation churn. A counting allocator put the ledger at 182.8 million allocations for one render. The profile's guess about where they came from was wrong too, which is why you take backtraces instead of trusting the summary: ~76% of all allocated bytes came from one line. Before laying out each table cell, the engine cloned the entire page cursor, including every element already placed on the page, as a rollback checkpoint in case the cell forced a page break. Then, whenever the cell fit, it discarded the clone unused. On a 500-page table, that is a full-page deep copy per cell, thrown away almost every time. The other top site by count was each positioned glyph owning its own heap-allocated copy of its font family name, millions of one-word strings.

Both fixes are boring, which is the good kind of fix. The checkpoint is skipped when the row provably fits, which is byte-identical by construction, since a checkpoint that would never be restored cannot change output. The font family string became a shared reference-counted string. Allocations fell 74%, bytes allocated fell 88%, and the output was verified byte-identical across the fixture wall and the full corpus.

I had estimated the allocation factor at ~1.9x. It came out at ~4x per layout pass, 7.3ms per page down to 1.8. Directionally right, quantitatively wrong, in the engine's favor. The ledger went from 9997ms to 2284ms, the invoice from 534ms to 147ms, and both flipped from behind the Puppeteer control to ahead of it, with Puppeteer as a stable control moving less than 2%.

Wrong the second time, and worse

The artifact was updated the same day the fixes landed. The prose was not. The comparison table kept its losing numbers through one release, then another, while the artifact it linked to said the opposite. The site was 3.6x wrong about its own engine, in a losing direction, on the page whose whole purpose is that comparison. That is the four days from the top of this post.

The numbers were finally corrected in a project-wide claims audit, the kind where you re-derive every published figure from its source. That audit also found the WASM module size published wrong in three places, transcribed differently each time. Same disease: numbers copied into prose by hand, drifting independently of reality in whatever direction nobody was checking. The checks only ran in one direction, so that direction was the only one that stayed true.

What changed structurally

Correcting the numbers is not the fix. The fix is making the next drift loud:

The parity artifact now carries measured artifact sizes, raw and gzipped bytes of both WASM builds, stat'd from the built files in CI rather than transcribed. The benchmark artifact carries per-run provenance: commit, machine, timestamp, so a run that lags the code being served is labeled as lagging instead of implying freshness.

The site keeps every published number in one module, each with its source and measurement date, and a check script diffs it against the live registry and the artifact. On its first real run it caught a genuine divergence I did not know about: the WASM that CI builds differs from the WASM I publish by 1,777 bytes, because CI's toolchain pin lags the release machine's. That sentence deserves a second read: it means the module CI tests is not, byte for byte, the module users download. Today the gap is a compiler-version artifact, the same source through two toolchain minors, and every behavioral gate runs against both sides of it. But "the tested artifact is not the shipped artifact" is an alarming class of fact whatever its current size, so it is now tracked, the pin alignment is queued, and the check will say so if the gap grows.

And the losses that remain stay published. The ledger still runs two layout passes, because of that digit-width correction; the engine currently leads while doing twice Chrome's layout work, and the artifact says so, along with the record of a scoped fix I attempted and set aside because it silently skipped the correction in an edge case that byte-identity testing caught. Removing that pass would roughly halve the ledger's layout time again. When it happens, the harness will say so. That is the point of having one.

The current numbers, dev machine, warm, per the artifact: receipt 17.3ms vs 48. Six-page report 20.5ms vs 54. Fifty pages, 171ms vs 233. Five hundred pages, 2.74s vs 2.79. Cold start to first PDF byte, ~110ms on Node against ~370ms for a warm-machine browser launch, and seconds against a cold serverless one. If those numbers are stale by the time you read this, the artifact is the one to trust, which is the entire lesson.