Skip to content
Try Hanzo
Language: English (United States)

Choosing among many options

Benchmarks / Kai / Choosing among many options

Kai · Capability

Accuracy at each number of listed options, every option scored, the same cases put to both models.

Accuracy
ValueBaseline
  • kai-1.4 · c1856a77 · 4 options94.3%Jev 90.5%
  • kai-1.4 · c1856a77 · 16 options92.0%Jev 85.3%
  • kai-1.4 · c1856a77 · 77 options88.0%Jev 84.3%

History

3 results, newest first. Superseded runs stay, so a revision's progress can be read.

History
MeasuredSubjectMetricValueBaselineResultStatus
Oct 16, 2026kai-1.4 · c1856a77 · 4 optionsAccuracy94.3%Jev 90.5%WinLive · Hanzo-measured
Oct 16, 2026kai-1.4 · c1856a77 · 16 optionsAccuracy92.0%Jev 85.3%WinLive · Hanzo-measured
Oct 16, 2026kai-1.4 · c1856a77 · 77 optionsAccuracy88.0%Jev 84.3%WinLive · Hanzo-measured

Conditions

  • choice sizes: 4, 16, 77
  • every option scored
  • Jev: typesafe/jev-1.13, measured Sep 27, 2026

Hardware

CPU F32-Q8_0 on evo

Reproduce

bench cap --harness . --who kai,jev --kai <checkpoint> # in hanzoai/benchmarks/decision

GET api.hanzo.ai/v1/research/runs?org=hanzo&id=cap.cardinality-cases-kai-1.4.c1856a77-kai

{ "data": [
  "cap.cardinality-cases-kai-1.4.c1856a77-kai"
] }
kai-1.4 · c1856a77 · 4 options  94.3%
kai-1.4 · c1856a77 · 16 options  92.0%
kai-1.4 · c1856a77 · 77 options  88.0%