Skip to content
Try Hanzo
Language: English (United States)

Latency of one decision

Benchmarks / Kai / Latency of one decision

Kai · Latency

Time to one decision, by serving mode and device. No baseline was timed on the same hardware, so these stand alone.

History
MetricValue
  • p50 latency19.8 ms
  • p95 latency29.0 ms

History

2 results across 2 metrics, newest first. Superseded runs stay, so a revision's progress can be read.

History
MeasuredSubjectMetricValueBaselineResultStatus
Sep 28, 2026Kai a7 · 0834a74f · cold, ROCM BF16p50 latency19.8 ms–UnscoredLive · Hanzo-measured
Sep 28, 2026Kai a7 · 0834a74f · cold, ROCM BF16p95 latency29.0 ms–UnscoredLive · Hanzo-measured

Conditions

  • cold/rocm-bf16: encoder 768 wide, 22 layers

Hardware

on evo

Reproduce

bench speed --states results/states.json.gz --results capability/results --who kai --kai <checkpoint>

GET api.hanzo.ai/v1/research/runs?org=hanzo&id=latency-pool-a7.0834a74f-kai

{ "data": [
  "latency-pool-a7.0834a74f-kai"
] }
p50 latency  19.8 ms
p95 latency  29.0 ms