Benchmarks / Engine prefill on CUDA vs llama.cpp

Engine prefill on CUDA vs llama.cpp

hanzo-engine against llama.cpp on CUDA, built against the same weights and run on the same box: prefill throughput at the prompt lengths where the whole 95% interval of the ratio lies above parity.

harness

hanzoai/engine · bench/dossier

our paper

Refutation-Driven Performance Engineering

prints the table on this page — transcribed from the committed run files on branch bench/dossier, the pass of 2026-07-21

What is measured

hanzo-engine and llama.cpp, built against the same weights, run on the same box, alternating. Prefill is tokens per second at a given prompt length, and the ratio is ours over theirs, so above 1.000 is a win. The models are Zen-1.7B at Q4_K_M and Zen-4B at Q4_K_M; 7 repetitions per cell, as its run records, with the first discarded as warmup, a Student-t 95% interval on each engine, and the ratio's interval formed by adding the two relative intervals in quadrature.

The verdict is the harness's, not a reading of ours: WIN where the whole interval lies above 1.000 and neither engine's coefficient of variation exceeded 5%, which is how the harness says the box was quiet. Every cell below is one.

Prefill on CUDA, cell by cell

3 cells from 2 runs on branch bench/dossier · engine 1.7.87 · each run pins its engine release, its llama.cpp build and its model digest, and is linked below

The work that produced it

Refutation-Driven Performance Engineering is the campaign that produced these kernels, written as a log of numbered hypotheses, each settled by a cheap decisive experiment. A long prefill is compute-bound, which is where that work landed.

the committed runs, in bench-runs: CUDA Zen-1.7B Q4_K_M · CUDA Zen-4B Q4_K_M