Benchmarks / Kai / Latency of one decision
Kai · LatencyLatency of one decision
Time to one decision, by serving mode and device. No baseline was timed on the same hardware, so these stand alone.
Latest
History
2 results across 2 metrics, newest first. Superseded runs stay, so a revision's progress can be read.
| Measured | Subject | Metric | Value | Baseline | Result | Status |
|---|---|---|---|---|---|---|
| Sep 28, 2026 | Kai a7 · 0834a74f · cold, ROCM BF16 | p50 latency | 19.8 ms | – | Unscored | Live · Hanzo-measured |
| Sep 28, 2026 | Kai a7 · 0834a74f · cold, ROCM BF16 | p95 latency | 29.0 ms | – | Unscored | Live · Hanzo-measured |
Conditions
cold/rocm-bf16: encoder 768 wide, 22 layersHardware
on evo
Source
research run latency-pool-a7.0834a74f-kai
Reproduce
bench speed --states results/states.json.gz --results capability/results --who kai --kai <checkpoint>
More Kai benchmarks
AG News · DAIR Emotion · Banking77 · Support triage · Email spam · Phishing · Jailbreak · Toxicity · RAG relevance · Model routing · Typed decisions · MASSIVE · AG News, validation split · Support triage, validation split · Typed decisions, validation split · HarmBench (benign prompts) · Choosing among many options · Joint decisions · Release gate against Jev
Every result for Latency of one decision on the shelf · How these are measured