Intelligence, cost and speed, each measured.
Zen 6 prefills ~2,400 tokens a second and decodes ~43 on NVIDIA Grace-Blackwell, measured Sep 23, 2026.
Enso · intelligence per dollar
Enso on the boards it was run on.
The frontier, one benchmark at a time
Accuracy against what the run cost at list prices, on a log axis, for each board where Enso’s router is on the frontier. Every point is a research run, read when the page opens, and its cost is the tokens it records on the whole question set at the price list’s rate for its own model id. The dashed line joins the runs no other run beats on both.
Every system on GPQA Diamond
Kai · decisions
Kai beside Jev, on the same questions.
Decision Suite
Kai against Jev, suite by suite
The suites Kai leads, both models on the same questions, each row dated by its runs. Sort by the lead, switch to calibration error, draw it as a chart or a table; the view is kept in the address, and the rows are read from the research API when the page opens.
Yes or no, read from the state
Production Kai answers a yes/no question with a reader trained for it. The tests below are the ones it leads Jev on, both models given the same questions and the same rows.
| Test | Kai | Jev |
|---|---|---|
| A question and its paraphrase, mean |Δp| over 20 pairs | 0.124 | 0.151 |
| A state silent on the question, answered between 0.35 and 0.65: share of 60 | 81.7% | 38.3% |
| Accuracy over 951 validation rows of 9 yes/no suites, pooled | 88.3% | 87.5% |
Kai is kai-1.2, read 2026-10-08; Jev is typesafe/jev-1.13 through the API, read 2026-10-06. Hanzo-measured; harness private.
Finding the right tool in a large catalog
Kai’s tool-relevance head, in development, ranks a shortlist of 32 tools drawn from the whole catalog. The shortlist is an open embedding model, zen-embedding-0.6B, with a small BM25 term; nothing is trained for it. Here it is scored against BM25 alone on a dev catalog Hanzo assembled from public tool-calling sets and its own API: 2,928 tools, 6,608 requests.
6,608 requests · 2,928 tools · Recall@32: a right tool among the first 32
670 requests · a service scored as the best of its operations
An internal dev set, not a public benchmark: one run, 2026-10-11. Retrieval only, not an agent result.
Verified coding-agent performance
Hanzo Dev, with Kai and Enso Flash on api.hanzo.ai, against native Claude Code on held-out repository repairs. Each task is one fault its visible test fails on, and counts as solved only when a hidden verifier passes. Speed is the paired wall time over the tasks both solved.
3.02×faster on the 42 tasks both solved95% interval 2.06–4.36×.| Tasks both solved, by how Hanzo Dev settled them | Tasks | Speed | 95% interval |
|---|---|---|---|
| Every task both solved | 42 | 3.02× | 2.06–4.36× |
| Settled by search, no model call | 24 | 8.22× | 6.86–9.82× |
| The model’s first edit | 4 | 1.31× | 1.14–1.48× |
| Agent | Median seconds | Model calls a task |
|---|---|---|
| Claude Code, sonnet-5.5 default effort | 9.5 | 3.8 |
| Hanzo Dev, dev flow fix, Kai + enso-flash | 2.3 | 1.0 |
The fastest tasks are settled by a search with no model call: it tries single-token edits of the kinds these faults are drawn from against the visible test alone, and never reads a hidden test or the fault. These results are specific to this benchmark, one attempt a task; they are not a claim about general-purpose coding.
Research runs of coding.repair on split held, ended 2026-10-15, harness commit 8e966be: GET /v1/research/runs?benchmark=coding.repair.
Agent tools
Tool handlers in Hanzo MCP, each run on the same cases as the handler it replaced. Each row says what decides it, rules, BM25 or Kai, as its run records. A rate is shown with its count.
Shell commands an agent should not run unasked
Split hermes: hermes-jev-skills evals/gate/fixtures.jsonl (MIT), 118 made-up commands. The rules were frozen and read once (1 of 65 allowed: a metadata-endpoint fetch); that class and a bare `rm -rf /` were fixed after, so this run is not held out
| Handler | What decides | Allowed, should deny or ask | Ordinary, stopped | Must deny, denied | Median time |
|---|---|---|---|---|---|
| Current | Rules | 0.0% (0 / 65) | 34.0% (18 / 53) | 54.3% (19 / 35) | 0.060 ms |
| Replaced, ae4d07b | Rules, then Kai | 18.5% (12 / 65) | 62.3% (33 / 53) | 14.3% (5 / 35) | – |
Research runs of mcp.command on split hermes, ended 2026-10-15; the current handler at commit 863cb8a: GET /v1/research/runs?org=hanzo&benchmark=mcp.command
Which registered tool a task needs
Split t1dev: supplied tool lists from Kai's T1 tools6 dev split (manifest 48be841f)
| Handler | What decides | Right tool first | MRR | Abstained | Median time |
|---|---|---|---|---|---|
| Current | Rules, then BM25 | 73.3% (220 / 300) | 0.841 | 1 | 0.160 ms |
| Replaced, ae4d07b | Kai | 65.7% (197 / 300) | 0.805 | 43 | – |
Research runs of mcp.select on split t1dev, ended 2026-10-15; the current handler at commit 863cb8a: GET /v1/research/runs?org=hanzo&benchmark=mcp.select
Split hand: eight hand-labelled tasks over eight tools
| Handler | What decides | Right tool first | MRR | Abstained | Median time |
|---|---|---|---|---|---|
| Current | Rules, then BM25 | 50.0% (4 / 8) | 0.590 | 2 | 0.100 ms |
| Replaced, ae4d07b | Kai | 25.0% (2 / 8) | 0.491 | 2 | – |
Research runs of mcp.select on split hand, ended 2026-10-15; the current handler at commit 863cb8a: GET /v1/research/runs?org=hanzo&benchmark=mcp.select
What kind of failure a CI log shows
Split ci60: 60 real CI logs from 48 public repositories, labelled from the excerpt alone without the rules in view; nameable = the 40 whose class the old six categories can name
| Handler | What decides | Right class | Right class, nameable | Unknown | Median time |
|---|---|---|---|---|---|
| Current | Rules | 70.0% (42 / 60) | 70.0% (28 / 40) | 9 | 0.370 ms |
| Replaced, ae4d07b | Kai | – | 50.0% (20 / 40) | – | – |
Research runs of mcp.triage on split ci60, ended 2026-10-15; the current handler at commit 863cb8a: GET /v1/research/runs?org=hanzo&benchmark=mcp.triage
Price
Zen 6 · speed on our own hardware
Zen 6 prefills ~2,400 tokens a second on NVIDIA Grace-Blackwell.
~2,400 tok/sPrefill · Zen 6 on NVIDIA Grace-BlackwellDecode ~43 tok/s · 50.6 tok/s decoding code · 112 tok/s across 8 requests · measured by Hanzo on Sep 23, 2026.the low end of a range is the bar
one request
Agent harness · what the agent remembers and finds
On LoCoMo, Hanzo's context engine finds 2.0× the multi-hop evidence cosine does.
2.0×Multi-hop ALL@20 · LoCoMo37.5% against cosine's 18.8% on the test split: every annotated evidence turn in the top 20.Memory and retrieval
What a memory finds in a long conversation, and what a reader does with it. Each panel is one metric, the baseline and Hanzo on the same data, the same embedder and the same cut-off, drawn on one scale.
substring EM · pooled over the dev haystack and the three test haystacks, and the largest alone · reader: none, embedder zenlm/zen-embedding-0.6b
test split · zen-embedding-0.6b · k=20 · ALL = every annotated evidence turn in the top 20
500 questions · all-minilm vectors for both · ALL@5 = every evidence session in the top 5
Knowledge graph
Point-in-time questions over facts that begin and end, against another temporal store on the same data. On the CronQuestions types the harness covers, the Hanzo graph answers 99.88% and Semantica 99.88%; what differs is the price of a question.
Code
Cross-file retrieval in a repository, scored apart from generation. Typed links are regular expressions over imports, definitions and identifiers; no model reads anything in that row.
Agent platform · what an agent costs
A dormant agent costs 477 bytes.
477 bytesState per dormant agentAgainst a published ~1 MB (2,198×). 1M agents: 455 MB on disk vs ~977 GB.Platform, agents, and sandboxes
What an agent costs at rest, what a live one costs to wake, and what a sandbox costs to start.
A live agent here is a goroutine: it shares one address space with every other, and nothing about it is an isolation boundary. Code that has to be isolated gets a microVM. The goroutine timings are the median of one pass on an Apple M4 Max, 16 cores. Every figure here is transcribed from the harness's README rather than read from a run file, and each page names the command that prints it.
Start-up and cost
Methods
How every number was taken
Methodology
Every result is current. Each build pulls the latest validated runs from the research API: canonical, public, and not flagged inconsistent. When the API cannot be read, or answers with fewer runs than the committed snapshot already holds, the build keeps the snapshot, and the page swaps in what the API holds once it answers in the browser.
Every row is dated. A run is dated by when it ended; the platform bench records its month.
Live, frozen, superseded. Live is the newest run the API serves for a benchmark. Frozen is a committed run file, a snapshot, or a figure transcribed from a harness’s README. Superseded is an earlier run of the same benchmark that a later one replaced; it stays in the library, so a revision’s history can be read.
Where a number comes from. Kai’s and Enso’s rows are research runs, served at the research API and read when this page opens. The memory, graph and code panels read the committed run files of the benchmarks repository. Kai’s yes/no table and the tool shortlist are Hanzo-measured reads the research API does not file: each is its run’s report, copied unchanged, and names its model, date and count.
Measured, not reported. Every frontier point is a run Hanzo filed: the router and each single model on the same questions, at the same tag. No vendor-reported figure is on a board here.
Prices are prices. Kai beside Jev is the input-token rate each is billed, from the price list, not a cost per task. The frontier’s axis is what each run spent on the whole question set. No speed is claimed here without a measured comparison; latency, where this Kai version was timed, is in its tables.
Comparing rows. A frontier is drawn among the rows of one benchmark, never across two, and a leaderboard row compares two models on one suite. Suites measure different tasks, so the mean is a summary, not a score on any of them.
The memory and code panels regenerate from the committed runs on a fresh clone: git clone https://github.com/hanzoai/benchmarks && cd benchmarks/brain && node results.mjs
The research
Every paper the library publishes. A benchmark’s own page names the paper that reports it, where one does. Specifications are the HIPs.
What long-term agent memory has to retrieve, and how it is measured
Useful-Work Mining, the Native AI Coin, and Post-Quantum Omnichain Settlement
Cloud Economics of a Sovereign OSS Stack
papers.hanzo.aiA Regulated Capital-Markets Migration Case Study
One Native Stack for Private, Continuously-Learning AI
papers.hanzo.aiInference at the Bandwidth Wall, On-Device QLoRA, One Engine
Native ROCm Inference on a Consumer RDNA3.5 APU
papers.hanzo.aiReaching llama.cpp Decode Parity via a Unified 1-bit-to-Full Quant Core
Edge LLM Inference Across Commodity Accelerators
papers.hanzo.aiMeasured AMD, NVIDIA, and Apple Performance
Continuously-Learning Private AI
papers.hanzo.aiSelf-Improvement and Privacy in the Hanzo Native Stack
Economical Serving and the hanzo.ai Platform
papers.hanzo.aiA Bandwidth-Optimal Unified Train-and-Serve Runtime
The Unified Sovereign-Tenant Cloud
papers.hanzo.aiLinear Shared-Nothing Scaling via Per-Tenant SQLite and In-Process Composition
Refutation-Driven Performance Engineering
papers.hanzo.aiAn Empirical Study of a Multi-Week GPU Kernel Campaign
Evolutionary Schedule Search in a One-Source Kernel DSL
papers.hanzo.aiAnd the Continual-Improvement Loop It Anchors
Memory-Aware, Local-First LLM Routing with a Learned SLO-Constrained Policy
State, Execution and Isolation Measured Separately at a Fleet of One Million
Hops Do Not Fix Multi-Hop Retrieval
papers.hanzo.aiA Negative Result on Long-Horizon Conversational Memory, and What the Failure Actually Is
A factorial measurement of conversational memory on LoCoMo, and the negative controls that locate the effect
Every benchmark
Every benchmark, every run
One shelf for everything above, by the same five products: each Kai revision on each suite it leads, each Enso tier on the frontiers it holds, each Zen 6 reading, each memory and platform pass. Search it, filter it, pick rows of one benchmark to compare, and open any of them for its history, conditions, hardware, source and the command that reproduces it.
151 of 151 results, 24 benchmarks