Benchmarks / Kai / Typed decisions, validation split
Kai · Validation splitTyped decisions, validation split
Kai on Typed decisions, validation split, one run per revision. No baseline was run on this split.
Latest
History
6 results across 2 metrics, newest first. Superseded runs stay, so a revision's progress can be read.
| Measured | Subject | Metric | Value | Baseline | Result | Status |
|---|---|---|---|---|---|---|
| Sep 29, 2026 | Kai triage+job_30bfc6e5af09a9f7 · 0cac9e85 | Accuracy | 80.6% | – | Unscored | Live · Hanzo-measured |
| Sep 29, 2026 | Kai a7 · 0834a74f | Accuracy | 79.7% | – | Unscored | Superseded · Hanzo-measured |
| Sep 29, 2026 | Kai triage+job_cccd2fcfeae9ecb9 · 57535ef4 | Accuracy | 80.0% | – | Unscored | Superseded · Hanzo-measured |
| Sep 29, 2026 | Kai triage+job_30bfc6e5af09a9f7 · 0cac9e85 | Calibration error (ECE) | 0.182 | – | Unscored | Live · Hanzo-measured |
| Sep 29, 2026 | Kai a7 · 0834a74f | Calibration error (ECE) | 0.179 | – | Unscored | Superseded · Hanzo-measured |
| Sep 29, 2026 | Kai triage+job_cccd2fcfeae9ecb9 · 57535ef4 | Calibration error (ECE) | 0.170 | – | Unscored | Superseded · Hanzo-measured |
Conditions
split: val:0b8b6476822b343c310 questions, 310 answereddataset sha256:0b8b63 Kai revisions measuredHardware
ROCm BF16 on evo
Source
research run kai.typed_decisions-val:0b8b6476822b343c-triage+job_30bfc6e5af09a9f7.0cac9e85-kai
Reproduce
No public command regenerates this result. The source above is the record it is read from.
More Kai benchmarks
AG News · DAIR Emotion · Banking77 · Support triage · Email spam · Phishing · Jailbreak · Toxicity · RAG relevance · Model routing · Typed decisions · MASSIVE · AG News, validation split · Support triage, validation split · HarmBench (benign prompts) · Choosing among many options · Joint decisions · Release gate against Jev · Latency of one decision
Every result for Typed decisions, validation split on the shelf · How these are measured