Hanzo AI
Try Hanzo

Benchmarks / Kai / MASSIVE

Kai · Decision harness

MASSIVE

Kai and Jev on the same 5,100 questions of MASSIVE, one run per Kai revision.

Latest

91.0%Accuracy · kai-1 · 0834a74fJev 89.0% · Win · Oct 3, 2026 · Hanzo-measured
0.060Calibration error (ECE) · kai-1 · 0834a74fJev 0.064 · Win · Oct 3, 2026 · Hanzo-measured

History

10 results across 2 metrics, newest first. Superseded runs stay, so a revision's progress can be read.

88.5%89.9%91.3%09-2709-2810-03Kai a5 · df1c16a2: 88.8%, Sep 27, 2026Kai a8 · 0cfeef05: 90.6%, Sep 28, 2026Kai a7 · 0834a74f: 90.9%, Sep 28, 2026Kai a6 · a211cc70: 91.0%, Sep 28, 2026kai-1 · 0834a74f: 91.0%, Oct 3, 2026
MeasuredSubjectMetricValueBaselineResultStatus
Oct 3, 2026kai-1 · 0834a74fAccuracy91.0%Jev 89.0%WinLive · Hanzo-measured
Oct 3, 2026kai-1 · 0834a74fCalibration error (ECE)0.060Jev 0.064WinLive · Hanzo-measured
Sep 28, 2026Kai a8 · 0cfeef05Accuracy90.6%Jev 89.0%WinSuperseded · Hanzo-measured
Sep 28, 2026Kai a7 · 0834a74fAccuracy90.9%Jev 89.0%WinSuperseded · Hanzo-measured
Sep 28, 2026Kai a6 · a211cc70Accuracy91.0%Jev 89.0%WinSuperseded · Hanzo-measured
Sep 28, 2026Kai a8 · 0cfeef05Calibration error (ECE)0.068Jev 0.064LossSuperseded · Hanzo-measured
Sep 28, 2026Kai a7 · 0834a74fCalibration error (ECE)0.060Jev 0.064WinSuperseded · Hanzo-measured
Sep 28, 2026Kai a6 · a211cc70Calibration error (ECE)0.059Jev 0.064WinSuperseded · Hanzo-measured
Sep 27, 2026Kai a5 · df1c16a2Accuracy88.8%Jev 89.0%LevelSuperseded · Hanzo-measured
Sep 27, 2026Kai a5 · df1c16a2Calibration error (ECE)0.066Jev 0.064LossSuperseded · Hanzo-measured

Conditions

split: harness5,100 questions, 5,100 answereddataset dde140a63fc55 Kai revisions measuredJev: typesafe/jev-1.13-20260917, measured Sep 25, 2026

Hardware

CUDA BF16

Source

research run massive-harness-kai-1.0834a74f-kai

Reproduce

bench score --harness . --kai results/kai/preds.json.gz --out results/scores.json # in hanzoai/benchmarks/decision @ 2c74e2d

More Kai benchmarks

AG News · DAIR Emotion · Banking77 · Support triage · Email spam · Phishing · Jailbreak · Toxicity · RAG relevance · Model routing · Typed decisions · AG News, validation split · Support triage, validation split · Typed decisions, validation split · HarmBench (benign prompts) · Choosing among many options · Joint decisions · Release gate against Jev · Latency of one decision

Every result for MASSIVE on the shelf · How these are measured

Build what’s next.

Try HanzoRead the docs