Model routing
Kai and Jev on the same 399 questions of Model routing, one run per Kai revision.
History
12 results across 2 metrics, newest first. Superseded runs stay, so a revision's progress can be read.
| Measured | Subject | Metric | Value | Baseline | Result | Status |
|---|---|---|---|---|---|---|
| Oct 16, 2026 | kai-1.4 · c1856a77 | Accuracy | 99.8% | Jev 97.7% | Win | Live · Hanzo-measured |
| Oct 16, 2026 | kai-1.4 · c1856a77 | Calibration error (ECE) | 0.002 | Jev 0.014 | Win | Live · Hanzo-measured |
| Oct 3, 2026 | kai-1 · 0834a74f | Accuracy | 100.0% | Jev 97.7% | Win | Superseded · Hanzo-measured |
| Oct 3, 2026 | kai-1 · 0834a74f | Calibration error (ECE) | 0.000 | Jev 0.014 | Win | Superseded · Hanzo-measured |
| Sep 28, 2026 | Kai a8 · 0cfeef05 | Accuracy | 100.0% | Jev 97.7% | Win | Superseded · Hanzo-measured |
| Sep 28, 2026 | Kai a7 · 0834a74f | Accuracy | 100.0% | Jev 97.7% | Win | Superseded · Hanzo-measured |
| Sep 28, 2026 | Kai a6 · a211cc70 | Accuracy | 100.0% | Jev 97.7% | Win | Superseded · Hanzo-measured |
| Sep 28, 2026 | Kai a8 · 0cfeef05 | Calibration error (ECE) | 0.001 | Jev 0.014 | Win | Superseded · Hanzo-measured |
| Sep 28, 2026 | Kai a7 · 0834a74f | Calibration error (ECE) | 0.000 | Jev 0.014 | Win | Superseded · Hanzo-measured |
| Sep 28, 2026 | Kai a6 · a211cc70 | Calibration error (ECE) | 0.000 | Jev 0.014 | Win | Superseded · Hanzo-measured |
| Sep 27, 2026 | Kai a5 · df1c16a2 | Accuracy | 100.0% | Jev 97.7% | Win | Superseded · Hanzo-measured |
| Sep 27, 2026 | Kai a5 · df1c16a2 | Calibration error (ECE) | 0.002 | Jev 0.014 | Win | Superseded · Hanzo-measured |
Conditions
- split: harness
- 399 questions, 399 answered
- dataset dde140a63fc5
- 6 Kai revisions measured
- Jev: typesafe/jev-1.13-20260917, measured Sep 25, 2026
Hardware
CPU AS SERVED
Reproduce
bench score --harness . --kai results/kai/preds.json.gz --out results/scores.json # in hanzoai/benchmarks/decision @ 5f616dd
More Kai benchmarks
GET api.hanzo.ai/v1/research/runs?org=hanzo&id=app.model_routing_domain-harness-kai-1.4.c1856a77-kai
{ "data": [
"app.model_routing_domain-harness-kai-1.4.c1856a77-kai"
] }Kai a5 · df1c16a2 100.0% Kai a8 · 0cfeef05 100.0% Kai a7 · 0834a74f 100.0% Kai a6 · a211cc70 100.0% kai-1 · 0834a74f 100.0% kai-1.4 · c1856a77 99.8%
