# LoCoMo-Conv benchmark — Hanzo AI

> LoCoMo-Conv is unreleased, so there is no score here. What there is: the paper’s own retrieval protocol reproduced on the questions the rewrites came from, showing the embedder moves R@10 by 16.8 points and the unit of memory by 19.7.

[Benchmarks](https://hanzo.ai/benchmarks) / LoCoMo-Conv

baseline only

# LoCoMo-Conv

The same memories asked the way people ask them — dialog, implicit, counterfactual, composed. The dataset is not released, so what is here is the paper’s retrieval protocol run on the original questions the rewrites came from, and nothing more.

dataset

[LoCoMo-Conv](https://arxiv.org/abs/2609.03467)

licence

unreleased

introduced by

[LoCoMo-Conv](https://arxiv.org/abs/2609.03467) — Chang & Chen, NTU MiuLab, September 2026

harness

[hanzoai/cloud · bench/brain](https://github.com/hanzoai/cloud/tree/main/bench/brain/conv)

`node results.mjs`regenerates every table on this page from the raw runs

## There is no score on this page

LoCoMo-Conv rewrites LoCoMo’s questions the way people actually ask them — dialog, implicit, counterfactual, composed — and it is the benchmark on this list we would most like to report. The data is not released. The paper’s one address answers 404. So no LoCoMo-Conv number appears here, no Hanzo configuration has been run on it, and nothing below is a result on that benchmark.

What is below is the paper’s retrieval protocol — all-MiniLM-L6-v2, k=10, verbatim containment — reproduced on the original LoCoMo questions the rewrites came from. Read as a diagnostic it says something useful about what a LoCoMo-Conv number would be measuring, which is why it is worth the page even though it scores nothing.

## The embedder moves it more than most systems differ

the paper’s protocol · k=10 · verbatim containment · the original LoCoMo questions

embedder

n

R@10, all

R@10, no adversarial

single-hop

multi-hop

temporal

all-MiniLM-L6-v2 (the paper’s)

1977

42.9

45.5

52.3

26.7

49.4

all-MiniLM-L6-v2, sentence-transformers

1977

42.9

45.5

52.3

27.0

49.4

zen-embedding-0.6b

1977

59.7

63.5

70.5

38.1

74.0

Changing nothing but the embedding model moves R@10 from 42.9 to 59.7 — 16.8 points, on identical questions, an identical index unit and an identical k.

## So does the unit of memory

one embedder · k=10 · the same questions · what changes is how the conversation is cut up

memory unit

n

R@10, all

R@10, no adversarial

a single turn

1982

42.8

45.4

a 2-turn window

1982

56.3

54.3

a 3-turn window

1982

62.5

60.7

a 5-turn window

1982

60.3

58.2

a whole session

1982

62.1

61.8

A single turn reaches 42.8; a three-turn window reaches 62.5 at three times the tokens — 19.7 points for a decision nobody reports.

## What this is for

Both of those differences are larger than most published differences between named memory systems. A paper reporting R@10 on a conversational benchmark, ours included, is reporting the sum of three choices — the embedder, the unit and k — and usually names one of them. That is the reason every table on this site carries its embedder, its unit and its k in the line above it, and the reason a single R@10 quoted without them is not a comparison between systems.

For reference, and not comparable to anything here since the set is unreleased: LoCoMo-Conv reports a best retrieval R@10 with query facets and AnchorMem of dialog 0.754, implicit 0.524, counterfactual 0.732, composed 0.432.

## The other benchmarks

[LoCoMo](https://hanzo.ai/benchmarks/locomo) · [MemoryAgentBench](https://hanzo.ai/benchmarks/memoryagentbench) · [LongMemEval](https://hanzo.ai/benchmarks/longmemeval) · [RepoBench-R](https://hanzo.ai/benchmarks/repobench-r) · [LoCoMo · subject scope](https://hanzo.ai/benchmarks/locomo-subject-scope) · [Fleet residency](https://hanzo.ai/benchmarks/fleet) · [Live agent footprint](https://hanzo.ai/benchmarks/goroutine) · [Sandbox cold start](https://hanzo.ai/benchmarks/sandbox) · [Inference vs llama.cpp](https://hanzo.ai/benchmarks/inference) · [GPQA-Diamond](https://hanzo.ai/benchmarks/gpqa) · [all of them, and the head-to-head](https://hanzo.ai/benchmarks)

raw runs and scoring: [hanzoai/cloud · bench/brain](https://github.com/hanzoai/cloud/tree/main/bench/brain/conv) · regenerated by `node results.mjs`