Latency of one decision
Time to one decision, by serving mode and device. No baseline was timed on the same hardware, so these stand alone.
History
2 results across 2 metrics, newest first. Superseded runs stay, so a revision's progress can be read.
| Measured | Subject | Metric | Value | Baseline | Result | Status |
|---|---|---|---|---|---|---|
| Sep 28, 2026 | Kai a7 · 0834a74f · cold, ROCM BF16 | p50 latency | 19.8 ms | – | Unscored | Live · Hanzo-measured |
| Sep 28, 2026 | Kai a7 · 0834a74f · cold, ROCM BF16 | p95 latency | 29.0 ms | – | Unscored | Live · Hanzo-measured |
Conditions
- cold/rocm-bf16: encoder 768 wide, 22 layers
Hardware
on evo
Reproduce
bench speed --states results/states.json.gz --results capability/results --who kai --kai <checkpoint>
