# CronQuestions benchmark — Hanzo AI > CronQuestions, 26,168 questions over 327,983 Wikidata facts: the graph API answers 99.9%, question for question the same as Semantica, at 0.06 ms a question against 5.4 s. [Benchmarks](https://hanzo.ai/benchmarks) / CronQuestions measured # CronQuestions Wikidata facts with start and end years, and 26,168 questions about who held what, and when. The graph API and Semantica’s temporal query engine, on the same facts and the same machine, with no model reading anything. dataset [CronQuestions](https://github.com/apoorvumang/CronKGQA) licence see the dataset repository introduced by [Question Answering Over Temporal Knowledge Graphs](https://aclanthology.org/2021.acl-long.520/) — Saxena, Chakrabarti, Talukdar, ACL 2021 harness [hanzoai/benchmarks · cronquestions](https://github.com/hanzoai/benchmarks/tree/main/cronquestions) `bash cronquestions/run.sh && python3 cron.py site test`regenerates every table on this page from the raw runs ## What is measured CronQuestions is 327,983 Wikidata facts — who held a position, played for a team, received an award — each with a start and an end year, and questions about them: who held a position in a given year, when someone held it, who held it before or after someone else, who held it first or last. The questions come annotated with their entities and years, so each is answered by a fixed plan of store calls and no model reads anything. What differs between rows is only the store the plan calls. Both stores load the same facts and run in-process on one machine; nothing crosses a network. The Hanzo rows go through the graph’s own admission and its own resolution — what `POST /v1/graph` and `POST /v1/graph/resolve` do — asking every question at `as_known` now, as any caller does. Semantica is driven through its documented temporal query calls. ## Results test split, 4 question types, Hits@1 with bootstrap 95% intervals · hanzo-inc/cloud 1fffdf408f2e, apps/graph clean · semantica-agi/semantica 3910c33bc88c store all who held it in year T when before / after first / last control: the plan over dictionaries 99.9[99.8, 99.9] 100.0[100.0, 100.0] 100.0[100.0, 100.0] 98.6[98.0, 99.0] 100.0[100.0, 100.0] Semantica 0.7.0 99.9[99.8, 99.9] 100.0[100.0, 100.0] 100.0[100.0, 100.0] 98.6[98.0, 99.0] 100.0[100.0, 100.0] Hanzo graph, through the API 99.9[99.8, 99.9] 100.0[100.0, 100.0] 100.0[100.0, 100.0] 98.6[98.0, 99.0] 100.0[100.0, 100.0] Hanzo graph, replayed write clock 99.9[99.8, 99.9] 100.0[100.0, 100.0] 100.0[100.0, 100.0] 98.6[98.0, 99.0] 100.0[100.0, 100.0] cost of a question, and of loading the facts store query p50 query p99 load load average Semantica 0.7.0 5,357 ms 7,531 ms 0.1 s 11.0 Hanzo graph, through the API 0.063 ms 2.4 ms 116 s 18.6 Hanzo graph, replayed write clock 0.058 ms 2.1 ms 145 s 25.1 ## What it says The graph API answers 99.9% of the questions, and Semantica 99.9% — each the control’s answer, question for question. 99.9% is the plan’s own ceiling: the before/after questions it misses, it misses with the facts in a dictionary too. The difference is the price of a question. A point-in-time question about one entity costs the graph one indexed read of that entity’s relation, 0.063 ms at the median. Semantica’s temporal calls take no entity, so every question about one person reconstructs the whole graph at that year: 5,357 ms. That is about 85 thousand times the cost for the same answer. ## What it found The first run of this benchmark answered 70.0% through the API, and none of the questions asking who held a position in a given year. The graph’s `as_of` bounded when it had learned a fact, not when the fact was so: history filed today counted as learned today, so a question about 1995 found nothing. And ending one holder’s term ended the whole relation, erasing every other holder. The graph now keeps the two instants apart — `as_of` for when the world was so, `as_known` for how much had been heard — a statement carries its own `until`, and a relation holds one value at a time or many, as the organization declares it. The replayed-clock row, which sets each fact’s write time to its own year, now answers the same as the API row: when a fact became known no longer decides what the world was. ## What this table is not It is not question answering from text. The questions arrive annotated, so this measures the store — whether it gives the right facts for a point in time, and at what cost — not a model’s reading of a question. 110 facts that start after today were refused at admission, which rejects a start more than five minutes ahead of the server clock; no covered question asks about them. The 3,832 time-join questions are not covered. Semantica loads in a fraction of a second because it builds no index; the graph’s load is admission, content addressing, its indexes and full-text search. The Hanzo timings were taken on a loaded machine, and each run records the load it had. HANZO AI CLOUD · PRODUCTION READY ### Deploy Frontier Intelligence with 10× Lower Costs Run models, agents, and sandboxes on the world's fastest AI cloud. Instant GPU linking, 1,800+ tok/s speculative throughput, and unified management across all your infrastructure. [Start on Hanzo Cloud →](https://cloud.hanzo.ai)[View Models & Pricing](https://hanzo.ai/pricing) [← PreviousLongMemEval](https://hanzo.ai/benchmarks/longmemeval)[All Benchmarks Board](https://hanzo.ai/benchmarks)[Next →RepoBench-R](https://hanzo.ai/benchmarks/repobench-r) Other benchmarks [LoCoMo](https://hanzo.ai/benchmarks/locomo) · [MemoryAgentBench](https://hanzo.ai/benchmarks/memoryagentbench) · [LongMemEval](https://hanzo.ai/benchmarks/longmemeval) · [RepoBench-R](https://hanzo.ai/benchmarks/repobench-r) · [LoCoMo · subject scope](https://hanzo.ai/benchmarks/locomo-subject-scope) · [LoCoMo-Conv](https://hanzo.ai/benchmarks/locomo-conv) · [Fleet residency](https://hanzo.ai/benchmarks/fleet) · [Live agent footprint](https://hanzo.ai/benchmarks/goroutine) · [Sandbox cold start](https://hanzo.ai/benchmarks/sandbox) · [Inference vs llama.cpp](https://hanzo.ai/benchmarks/inference) · [GPQA-Diamond](https://hanzo.ai/benchmarks/gpqa) raw runs and scoring: [hanzoai/benchmarks · cronquestions](https://github.com/hanzoai/benchmarks/tree/main/cronquestions) · regenerated by `bash cronquestions/run.sh && python3 cron.py site test`