Choosing among many options
Accuracy at each number of listed options, every option scored, the same cases put to both models.
History
3 results, newest first. Superseded runs stay, so a revision's progress can be read.
| Measured | Subject | Metric | Value | Baseline | Result | Status |
|---|---|---|---|---|---|---|
| Oct 16, 2026 | kai-1.4 · c1856a77 · 4 options | Accuracy | 94.3% | Jev 90.5% | Win | Live · Hanzo-measured |
| Oct 16, 2026 | kai-1.4 · c1856a77 · 16 options | Accuracy | 92.0% | Jev 85.3% | Win | Live · Hanzo-measured |
| Oct 16, 2026 | kai-1.4 · c1856a77 · 77 options | Accuracy | 88.0% | Jev 84.3% | Win | Live · Hanzo-measured |
Conditions
- choice sizes: 4, 16, 77
- every option scored
- Jev: typesafe/jev-1.13, measured Sep 27, 2026
Hardware
CPU F32-Q8_0 on evo
Reproduce
bench cap --harness . --who kai,jev --kai <checkpoint> # in hanzoai/benchmarks/decision
