Hanzo AI
Try Hanzo

Benchmarks / Kai / Email spam

Kai · Decision harness

Email spam

Kai and Jev on the same 400 questions of Email spam, one run per Kai revision.

Latest

99.8%Accuracy · kai-1 · 0834a74fJev 97.8% · Win · Oct 3, 2026 · Hanzo-measured
0.003Calibration error (ECE) · kai-1 · 0834a74fJev 0.060 · Win · Oct 3, 2026 · Hanzo-measured

History

10 results across 2 metrics, newest first. Superseded runs stay, so a revision's progress can be read.

97.5%98.7%100.0%09-2709-2810-03Kai a5 · df1c16a2: 99.5%, Sep 27, 2026Kai a8 · 0cfeef05: 99.5%, Sep 28, 2026Kai a7 · 0834a74f: 99.8%, Sep 28, 2026Kai a6 · a211cc70: 99.8%, Sep 28, 2026kai-1 · 0834a74f: 99.8%, Oct 3, 2026
MeasuredSubjectMetricValueBaselineResultStatus
Oct 3, 2026kai-1 · 0834a74fAccuracy99.8%Jev 97.8%WinLive · Hanzo-measured
Oct 3, 2026kai-1 · 0834a74fCalibration error (ECE)0.003Jev 0.060WinLive · Hanzo-measured
Sep 28, 2026Kai a8 · 0cfeef05Accuracy99.5%Jev 97.8%WinSuperseded · Hanzo-measured
Sep 28, 2026Kai a7 · 0834a74fAccuracy99.8%Jev 97.8%WinSuperseded · Hanzo-measured
Sep 28, 2026Kai a6 · a211cc70Accuracy99.8%Jev 97.8%WinSuperseded · Hanzo-measured
Sep 28, 2026Kai a8 · 0cfeef05Calibration error (ECE)0.004Jev 0.060WinSuperseded · Hanzo-measured
Sep 28, 2026Kai a7 · 0834a74fCalibration error (ECE)0.003Jev 0.060WinSuperseded · Hanzo-measured
Sep 28, 2026Kai a6 · a211cc70Calibration error (ECE)0.003Jev 0.060WinSuperseded · Hanzo-measured
Sep 27, 2026Kai a5 · df1c16a2Accuracy99.5%Jev 97.8%WinSuperseded · Hanzo-measured
Sep 27, 2026Kai a5 · df1c16a2Calibration error (ECE)0.004Jev 0.060WinSuperseded · Hanzo-measured

Conditions

split: harness400 questions, 400 answereddataset dde140a63fc55 Kai revisions measuredJev: typesafe/jev-1.13-20260917, measured Sep 25, 2026

Hardware

CUDA BF16

Source

research run app.email_spam-harness-kai-1.0834a74f-kai

Reproduce

bench score --harness . --kai results/kai/preds.json.gz --out results/scores.json # in hanzoai/benchmarks/decision @ 2c74e2d

More Kai benchmarks

AG News · DAIR Emotion · Banking77 · Support triage · Phishing · Jailbreak · Toxicity · RAG relevance · Model routing · Typed decisions · MASSIVE · AG News, validation split · Support triage, validation split · Typed decisions, validation split · HarmBench (benign prompts) · Choosing among many options · Joint decisions · Release gate against Jev · Latency of one decision

Every result for Email spam on the shelf · How these are measured

Build what’s next.

Try HanzoRead the docs