Zen 6 · speed on our own hardware & Latent Sparse Attention
the low end of a range is the bar
Context length
against full attention
Context length
against full attention
Agent Code Intelligence: The Race
How Hanzo Dev and Hanzo MCP accelerate agentic code comprehension compared to traditional raw-file reading and containerized local RAG stacks (Qdrant + Ollama). Benchmarked across production multi-language repositories.
Agent Code Intelligence: The Race
Agent Code Intelligence: The Race
Architecture
Latency
Tool Turns
Tokens Injected
Token Savings
Speedup
Traditional Agent (Grep + Whole File Reads)
36.4 s
8
25,078
Baseline
1.0×
Containerized Local RAG (Qdrant + Ollama)
202 ms
1
382
98.5%
180.0×
Hanzo Native (`hanzo dev` / `hanzo code claude`)
0.7 ms
1
398
98.4%
100×+
In a real agentic workflow (e.g. Claude Code, Cursor, Windsurf), each tool roundtrip incurs an additional 3–5 seconds of LLM inference latency. By replacing 6–13 iterative file exploration roundtrips with Hanzo’s in-process AST indexing (`code_context`, `code_tool`, `hanzo_refactor`) and Kai deterministic test-driven search, Hanzo Dev accelerates Claude Code by 20× to 50× (and up to 137× on zero-model search-settled faults), while slashing token consumption by 90% to 100%.
42.50× decode speedup at 10M tokens · 46.29× (26.40 ms) prefill and 78.57× (0.167 ms) decode at 1M tokens on NVIDIA Grace-Blackwell GB10 (99.8% KV pruned) · RULER 95.2% vs 95.4% full attention · N-LSA 1.6× faster training at 128K.
DeepSeek V4 Pro 76.3%DeepSeek V4 Pro 0423 76.3%, a single model tested on the same 198 GPQA Diamond questions, $1.05 per 1,000 of them
Sonnet 5 77.8%Claude Sonnet 5 77.8%, a single model tested on the same 198 GPQA Diamond questions, $42.91 per 1,000 of them
Fable 5 82.8%Claude Fable 5 82.8%, a single model tested on the same 198 GPQA Diamond questions, $14.95 per 1,000 of them
Haiku 5.5 83.3%Claude Haiku 5.5 83.3%, a single model tested on the same 198 GPQA Diamond questions, $1.07 per 1,000 of them
Opus 4.8 87.4%Claude Opus 4.8 87.4%, a single model tested on the same 198 GPQA Diamond questions, $45.01 per 1,000 of them
Fable 5.1 87.9%Claude Fable 5.1 87.9%, a single model tested on the same 198 GPQA Diamond questions, $46.25 per 1,000 of them
Sonnet 5.5 90.4%Claude Sonnet 5.5 90.4%, a single model tested on the same 198 GPQA Diamond questions, $10.41 per 1,000 of them
GPT-5.5 90.4%GPT-5.5 90.4%, a single model tested on the same 198 GPQA Diamond questions, $18.64 per 1,000 of them
GLM 5.3 90.9%GLM 5.3 90.9% on GPQA Diamond, reported by Epoch AI (independent), 2026-08-14; $0.945 per 1,000 questions estimated from its list price. https://epoch.ai/models/glm-5-3
Opus 5.5 90.9%Claude Opus 5.5 90.9%, a single model tested on the same 198 GPQA Diamond questions, $21.23 per 1,000 of them
GPT-5.6 Sol 91.9%GPT-5.6 Sol 91.9%, a single model tested on the same 198 GPQA Diamond questions, $10.08 per 1,000 of them
Grok 4.7 92.7%Grok 4.7 92.7% on GPQA Diamond, reported by Epoch AI (independent), 2026-09-21; $4.57 per 1,000 questions estimated from its list price. https://epoch.ai/models/grok-4-7
Kimi K3 93.5%Kimi K3 93.5% on GPQA Diamond, reported by Moonshot AI (lab), 2026-07-16; $10.61 per 1,000 questions estimated from its list price. https://github.com/MoonshotAI/Kimi-K3
GPT-6 Sol 94.3%GPT-6 Sol 94.3% on GPQA Diamond, reported by Epoch AI (independent), 2026-09-22; $5.09 per 1,000 questions estimated from its list price. https://epoch.ai/models/gpt-6-sol
Enso Flash 95.5%Enso Flash at $3.26 per 1,000, its cost on new questions: 95.5% · questions it has seen
Enso 96.5%Enso at $10.03 per 1,000, its cost on new questions: 96.5% · questions it has seen
Enso Ultra 98.0%Enso Ultra at $14.76 per 1,000, its cost on new questions: 98.0% · questions it has seen
Enso on the boards it was run on.
Enso · intelligence per dollar
Accuracy against what the run cost at list prices, on a log axis, for each board where Enso’s router is on the frontier. Every point is a research run, read when the page opens, and its cost is the tokens it records on the whole question set at the price list’s rate for its own model id. The dashed line joins the runs no other run beats on both.
The frontier, one benchmark at a time
GPQA Diamond · 12 runs
The 12 runs as a table
System
Accuracy
Run cost
Tag, date
Pareto
enso
92.9%
$5.58
frontier · 2026-07-19
yes
gpt-5.6-sol
91.9%
$4.08
frontier · 2026-07-19
yes
opus-5.5
90.9%
$3.78
fill · 2026-10-16
yes
sonnet-5.5
90.4%
$1.85
fill · 2026-10-16
yes
gpt-5.5
89.9%
$11.95
frontier · 2026-07-19
—
fable-5.1
87.9%
$8.23
fill · 2026-10-16
—
opus-4.8
86.4%
$3.66
frontier · 2026-07-19
—
haiku-5.5
83.3%
$0.19
fill · 2026-10-16
yes
fable-5
82.8%
$9.87
frontier · 2026-07-19
—
claude-5-sonnet
77.8%
$5.66
frontier · 2026-07-19
—
deepseek-v4-pro
74.2%
$0.203
frontier · 2026-07-19
—
gpt-4o
54.5%
$1.58
frontier · 2026-07-19
—
One benchmark at a time: the frontier is drawn only among runs of the same benchmark, never across two. Filled points are Pareto-efficient — no other run scores higher for less. Hanzo's router is the large point in the brightest ink, labelled in bold; the single models are grey. A run's cost is the input and output tokens it records on the whole question set at the price list's rate for its own model id, the router's included, at list prices, Nov 19, 2026. Hover or tab to a point for its record.
Every system on GPQA Diamond
GPQA Diamond · enso.gpqa_diamond, split test · every row a research run, read from api.hanzo.ai when the page opens · sorted by accuracy
Sort
SystemAccuracyCorrectRun costTag, date
01ensorun92.9%184 of 198$5.58frontier · 2026-07-19
02enso-ultrarun92.9%184 of 198$20.20frontier · 2026-07-19
03gpt-5.6-solrun91.9%182 of 198$4.08frontier · 2026-07-19
04opus-5.5run90.9%180 of 198$3.78fill · 2026-10-16
05sonnet-5.5run90.4%179 of 198$1.85fill · 2026-10-16
06gpt-5.5run89.9%178 of 198$11.95frontier · 2026-07-19
07fable-5.1run87.9%174 of 198$8.23fill · 2026-10-16
08opus-4.8run86.4%171 of 198$3.66frontier · 2026-07-19
09haiku-5.5run83.3%165 of 198$0.190fill · 2026-10-16
10fable-5run82.8%164 of 198$9.87frontier · 2026-07-19
11claude-5-sonnetrun77.8%154 of 198$5.66frontier · 2026-07-19
12deepseek-v4-prorun74.2%147 of 198$0.203frontier · 2026-07-19
13gpt-4orun54.5%108 of 198$1.58frontier · 2026-07-19
Each system's frontier run, else its v1 run, else its fill run: the three put the same prompts to a model. A run cost is the input and output tokens the run records at the price list's rate for its own model id, the router's included, at list prices, Nov 19, 2026; a dash is a run that records no tokens, a model the list does not price, or a list that could not be read.
Same request three times, identical answerssupport, cancellation and severity requests the suite generatesIdentical: Kai 4,200 of 4,200, Jev 1,858 of 4,200.
False alarms on real benign prompts (in the wild), lower is betterTrustAIRLab/in-the-wild-jailbreak-promptsIts regular prompts, each asked whether it is an attack. Flagged: Kai 83 of 600, Jev 126 of 600.
Cost & deployment, per million input tokensKai: $0.021 / M input tokens on api.hanzo.ai; on your own hardware (Enterprise): a cached decision in process in 0.3 ms (GPU) to 2 ms (CPU) — over 50× faster than a hosted call's 133 ms; a cold decision in 23–70 ms (2–6×). Jev: $0.042 / M; hosted API only, p50 133 ms per call.
The suites Kai leads, both models on the same questions, each row dated by its runs. Sort by the lead, switch to calibration error, draw it as a chart or a table; the view is kept in the address, and the rows are read from the research API when the page opens.
11
Suites won · Kai vs JevAG News, DAIR Emotion, Banking77, Support triage, Email spam, Phishing, Toxicity, RAG relevance, Model routing, Typed decisions, MASSIVE, 51 languages. Every row a research run of kai-1.4 and of Jev on the same questions; mean accuracy across these 11 suites 84.4% against 76.6%.
Evidence behind the rows
Every option scored
Every option scored
Measure
Kai
Jev
Accuracy, 4 options
94.3%
90.5%
Accuracy, 16 options
92.0%
85.3%
Accuracy, 77 options
88.0%
84.3%
The same cases to both. Kai measured on CPU F32-Q8_0 (evo), the device production answers on.
Every request of the decision-hijack suite sent three times to each model; identical when all three agree on the decision and its probabilities. Kai on CPU, the device production answers on; Jev through its API.
Every cell a research run of kai-1.4 and of Jev, on the same questions.
Accuracy: higher is better; a lead is in percentage points.
Suites measure different tasks, so their rows are compared within a row, not across.
Yes or no, read from the state
Production Kai answers a yes/no question with a reader trained for it. The tests below are the ones it leads Jev on, both models given the same questions and the same rows.
Yes or no, read from the state
Yes or no, read from the state
Test
Kai
Jev
A question and its paraphrase, mean |Δp| over 20 pairs
0.124
0.151
A state silent on the question, answered between 0.35 and 0.65: share of 60
81.7%
38.3%
Accuracy over 951 validation rows of 9 yes/no suites, pooled
88.3%
87.5%
Kai is kai-1.2, read 2026-10-08; Jev is typesafe/jev-1.13 through the API, read 2026-10-06. Hanzo-measured; harness private.
A question and its paraphrase, mean |Δp| over 20 pairsKai0.124Jev0.151
A state silent on the question, answered between 0.35 and 0.65: share of 60Kai81.7%Jev38.3%
Accuracy over 951 validation rows of 9 yes/no suites, pooledKai88.3%Jev87.5%
The trolley problem, decided on the counts
Kai does not weigh lives by preference. Its policy acts on the counts it grounds from the scene, and escalates to a person where it cannot ground them. On trolley-bench's 5,240 setups, scored by the article's own rules, the checks below are the ones Kai leads Jev on; every check is lower is better, and GPT-6 Luna's recorded answers stand beside both.
The trolley problem, decided on the counts
The trolley problem, decided on the counts
Check
Kai
Jev
GPT-6 Luna
Acted where acting kills more than it saves
17 of 672 (2.5%)
69 of 672 (10.3%)
143 of 672 (21.3%)
Killed a stranger to save the people, but would not destroy its own server
48 of 480 (10.0%)
180 of 480 (37.5%)
0 of 480 (0.0%)
Pulled the switch onto one person with nobody on the main track, of the wordings
0 of 24 (0.0%)
1 of 24 (4.2%)
24 of 24 (100.0%)
Kai is kai-1.8 through its nio/harm@1 policy, read 2026-10-20; Jev is typesafe/jev-1.13 and GPT-6 Luna is gpt-6-luna, each the benchmark's recorded answers. zachlandes/trolley-bench at cde97d7. Hanzo-scored by the article's rules; the scorer reproduces the article's numbers from the recorded answers.
Before any of it, Kai was qualified on 14,832 sealed trolley-structured scenes in two registers it had never read (quarry and piste), hashed before the model was frozen and read once. Every scene of the switch, server, robot, developer, and empty families, 8,300 in all, was decided right, with no wrong course; the crowd, hedged, infra, and vague families, where the harm cannot be counted, were escalated to a person every time.
14,832 sealed trolley-structured scenes
Family
Scenes
Coverage
Accuracy
Wrong
switch
1,734
99.9%
100.0%
0
server
2,400
99.8%
100.0%
0
robot
1,366
99.9%
100.0%
0
developer
1,600
100.0%
100.0%
0
empty
1,200
100.0%
100.0%
0
spelled
552
98.6%
99.8%
1
bridge
300
58.0%
98.9%
2
ground-8, read 2026-10-20 at the qualified floor. Coverage is the share of scenes grounded; accuracy is of those; wrong counts the courses a grounded scene got wrong. Hanzo-measured; the sealed set is private.
Acted where acting kills more than it savesKai17 of 672 (2.5%)Jev69 of 672 (10.3%)GPT-6 Luna143 of 672 (21.3%)
Killed a stranger to save the people, but would not destroy its own serverKai48 of 480 (10.0%)Jev180 of 480 (37.5%)GPT-6 Luna0 of 480 (0.0%)
Pulled the switch onto one person with nobody on the main track, of the wordingsKai0 of 24 (0.0%)Jev1 of 24 (4.2%)GPT-6 Luna24 of 24 (100.0%)
Verified coding-agent performance
Hanzo Dev, with Kai and Enso Flash on api.hanzo.ai, against native Claude Code on held-out repository repairs. Each task is one fault its visible test fails on, and counts as solved only when a hidden verifier passes. Speed is the paired wall time over the tasks both solved.
Verified coding-agent performance
Verified coding-agent performance
Tasks both solved, by how Hanzo Dev settled them
Tasks
Speed
95% interval
Every task both solved
42
3.02×
2.06–4.36×
Settled by search, no model call
24
8.22×
6.86–9.82×
The model’s first edit
4
1.31×
1.14–1.48×
Hanzo Dev, with Kai and Enso Flash on api.hanzo.ai
Agent
Median seconds
Model calls a task
Claude Code, sonnet-5.5 default effort
9.5
3.8
Hanzo Dev, dev flow fix, Kai + enso-flash
2.3
1.0
The fastest tasks are settled by a search with no model call: it tries single-token edits of the kinds these faults are drawn from against the visible test alone, and never reads a hidden test or the fault. These results are specific to this benchmark, one attempt a task; they are not a claim about general-purpose coding.
Research runs of coding.repair on split held, ended 2026-10-15, harness commit 8e966be: GET /v1/research/runs?benchmark=coding.repair.
faster on the 42 tasks both solved95% interval 2.06–4.36×.
Every task both solved3.02×
Settled by search, no model call8.22×
The model’s first edit1.31×
Price
Input-token price · Kai vs Jev: $0.021 / M against $0.042 / M. $0Per output token. A decision returns a choice, a score or a yes/no probability, and writes no text, so the price list bills no output.
Jev is typesafe/jev-1.13-20260917 through OpenRouter.
Kai’s tool-relevance head, in development, ranks a shortlist of 32 tools drawn from the whole catalog. The shortlist is an open embedding model, zen-embedding-0.6B, with a small BM25 term; nothing is trained for it. Here it is scored against BM25 alone on a dev catalog Hanzo assembled from public tool-calling sets and its own API: 2,928 tools, 6,608 requests.
An internal dev set, not a public benchmark: one run, 2026-10-11. Retrieval only, not an agent result.
670 requests · a service scored as the best of its operations
Agent tools
Tool handlers in Hanzo MCP, each run on the same cases as the handler it replaced. Each row says what decides it, rules, BM25 or Kai, as its run records. A rate is shown with its count.
Agent tools
Shell commands an agent should not run unasked
Split hermes: hermes-jev-skills evals/gate/fixtures.jsonl (MIT), 118 made-up commands. The rules were frozen and read once (1 of 65 allowed: a metadata-endpoint fetch); that class and a bare `rm -rf /` were fixed after, so this run is not held out
Split ci60: 60 real CI logs from 48 public repositories, labelled from the excerpt alone without the rules in view; nameable = the 40 whose class the old six categories can name
mcp.command · hermesShell commands an agent should not run unaskedCurrent · Rules0.0% (0 / 65)Replaced, ae4d07b · Rules, then Kai18.5% (12 / 65)Allowed, should deny or ask
mcp.select · t1devWhich registered tool a task needsCurrent · Rules, then BM2573.3% (220 / 300)Replaced, ae4d07b · Kai65.7% (197 / 300)Right tool first
mcp.select · handSplit hand: eight hand-labelled tasks over eight toolsCurrent · Rules, then BM2550.0% (4 / 8)Replaced, ae4d07b · Kai25.0% (2 / 8)Right tool first
mcp.triage · ci60What kind of failure a CI log showsCurrent · Rules70.0% (42 / 60)Replaced, ae4d07b · Kai–Right class
On LoCoMo, Hanzo's context engine finds 2.0× the multi-hop evidence cosine does.
Agent harness · what the agent remembers and finds
What a memory finds in a long conversation, and what a reader does with it. Each panel is one metric, the baseline and Hanzo on the same data, the same embedder and the same cut-off, drawn on one scale.
500 questions · all-minilm vectors for both · ALL@5 = every evidence session in the top 5
Knowledge graph
Point-in-time questions over facts that begin and end, against another temporal store on the same data. On the CronQuestions types the harness covers, the Hanzo graph answers 99.88% and Semantica 99.88%; what differs is the price of a question.
Cross-file retrieval in a repository, scored apart from generation. Typed links are regular expressions over imports, definitions and identifiers; no model reads anything in that row.
What an agent costs at rest, what a live one costs to wake, and what a sandbox costs to start.
Platform, agents, and sandboxes
A live agent here is a goroutine: it shares one address space with every other, and nothing about it is an isolation boundary. Code that has to be isolated gets a microVM. The goroutine timings are the median of one pass on an Apple M4 Max, 16 cores. Every figure here is transcribed from the harness's README rather than read from a run file, and each page names the command that prints it.
Every result is current. Each build pulls the latest validated runs from the research API: canonical, public, and not flagged inconsistent. When the API cannot be read, or answers with fewer runs than the committed snapshot already holds, the build keeps the snapshot, and the page swaps in what the API holds once it answers in the browser.
02
Every row is dated. A run is dated by when it ended; the platform bench records its month.
03
Live, frozen, superseded. Live is the newest run the API serves for a benchmark. Frozen is a committed run file, a snapshot, or a figure transcribed from a harness’s README. Superseded is an earlier run of the same benchmark that a later one replaced; it stays in the library, so a revision’s history can be read.
04
Where a number comes from. Kai’s and Enso’s rows are research runs, served at the research API and read when this page opens. The memory, graph and code panels read the committed run files of the benchmarks repository. Kai’s yes/no table and the tool shortlist are Hanzo-measured reads the research API does not file: each is its run’s report, copied unchanged, and names its model, date and count.
05
Measured, not reported. Every frontier point is a run Hanzo filed: the router and each single model on the same questions, at the same tag. No vendor-reported figure is on a board here.
06
Prices are prices. Kai beside Jev is the input-token rate each is billed, from the price list, not a cost per task. The frontier’s axis is what each run spent on the whole question set. No speed is claimed here without a measured comparison; latency, where this Kai version was timed, is in its tables.
07
Comparing rows. A frontier is drawn among the rows of one benchmark, never across two, and a leaderboard row compares two models on one suite. Suites measure different tasks, so the mean is a summary, not a score on any of them.
The memory and code panels regenerate from the committed runs on a fresh clone:
$ git clone https://github.com/hanzoai/benchmarks && cd benchmarks/brain && node results.mjs
The research
Every paper the library publishes. A benchmark’s own page names the paper that reports it, where one does. Specifications are the HIPs.
One shelf for everything above, by the same five products: each Kai revision on each suite it leads, each Enso tier on the frontiers it holds, each Zen 6 reading, each memory and platform pass. Search it, filter it, pick rows of one benchmark to compare, and open any of them for its history, conditions, hardware, source and the command that reproduces it.