Skip to content
goldenxingxingPublic

About

Cross-session memory for agents — two text files and the standard library

Resources

Stars

3 stars

Watchers

0 watching

Forks

Repository files navigation

English | 中文 · How it works

Carryover

Cross-session memory for agents. Two text files and the standard library.

Carryover keeps what an agent learns about the person it works with, brings the standing parts into every later conversation without being asked, and lets the rest be searched. It is a library, not a service: no daemon, no vector database, no embedding model, nothing to install beyond Python.

Nothing in it is specific to a domain. The examples below come from a coding assistant because that is where it was built, and the comparison further down was measured on LoCoMo — two people talking about their families and their weeks.


Why it exists

Most memory libraries are a vector store and a retrieval call: you ask, it finds. That answers one kind of question well and leaves two unanswered.

Some memories are never asked for. "Never put function names in the external docs", "always ask before emailing a client" — a rule like that is obeyed by not doing something. Nobody types a query that retrieves it, and a system that only answers queries never surfaces it, so the rule sits correctly in the database while the agent breaks it. Measured over 243 real turns from one user: 89% were instructions, where no retrieval happens and only memory that arrives unasked can help. 8% asked for a single stored fact. 2.5% needed several combined.

Something has to decide what is kept. Extraction can notice a fact; it must not be able to write one. Every other library here writes silently. You find out what it recorded by going to look.

Both answers are the design: memory is split by whether it must be present or can be looked up, and nothing enters the store without a person saying so.

How it compares

Ten open-source memory systems were surveyed, five measured. Same corpus, same judge, same content into every one. Configurations and reproduction steps are in benchmarks/README.md.

Blank cells are marked with why they are blank: ✗ the system's design prevents the measurement · — not run.

Where Carryover wins outright

Dimension Carryover Best of the rest The gap
Install footprint nothing beyond the stdlib 360 MB dependency tree (mem0) the others are 0.97–1.4 GB environments
Build over 788 turns 0.0 s 0.1 s (BM25, entirely in memory) Cognee ~20 min, SimpleMem ~2 hours
Write a memory, then search 1.7 ms the two fastest retrievers cannot do it their index is in memory and must be rebuilt whole
Billed tokens over 20 turns 1,262 2,480 (retrieve every turn) half the cost, widening as the conversation runs
Current value after a fact changes 6/6 4/6 (mem0) mem0 also gave 1 superseded answer and 1 unmarked pair
Never-revised rules still present 3/3 1/3 (mem0) the two it lost went with no indication
Chinese recall@8 (non-dense) 58.9% 49.3% (mem0 hybrid) LlamaIndex BM25 scores 2.3%
Writes need a person's approval yes none of them every other system writes silently
Store is plain text you can edit yes none of them the others are vector stores or opaque formats
Memory arrives without being asked yes none of them every other system is query-driven

The last three rows are design choices rather than scores, but they decide whether you can trust it, correct it, and rely on it to govern behaviour. No competitor offers any of the three.

Quality

Recall@8 asks whether the turn holding the answer came back. It is deterministic, over 302 questions. End-to-end accuracy is LLM-judged over 120, reported as a mean with the spread across identical runs, because the same system varies by up to ten points between them.

System Recall@8 EN Recall@8 ZH End-to-end runs spread
Cognee — chunk path 75.4% ✻† — English-only run — see below — —
MemOS 69.5% ✻ — English-only run 47.0% 2 43.3–50.8
mem0 — qdrant hybrid 68.5% 49.3% 52.7% 8 49.2–55.8
Carryover 64.6% 58.9% 49.4% 8 46.7–53.3
LlamaIndex BM25 60.6% 2.3% 48.0% 8 43.3–52.5
BM25 + 5-line CJK tokenizer 60.6% 51.3% — a baseline, not a system — —
mem0 — dense, multilingual 58.3% 54.6% — end-to-end not run — —
txtai — dense + BM25 57.6% 61.9% 46.6% 3 43.3–48.3
SimpleMem ✗ answers, never returns turns ✗ same — see below — —
Carryover, answered by a stronger model — same retrieval — same 54.5% 7 52.5–55.8

✻ 118-question subset, not comparable with the 302-question rows. On that same subset Carryover scores 61.9% against MemOS's 69.5%. † Not the same budget: one Cognee "result" is a twenty-turn document, so eight results are 160 turns against everyone else's eight.

Cognee and SimpleMem have no end-to-end figure here. They run their own model at query time, so the number measures their model and their memory together, and there is no way to hold the model fixed across them. The control row shows how much of that is the model: the same Carryover retrieval, answered by a stronger one, moves five points without a single change to what was stored. Their costs are in the table below, where they are comparable.

Cost

788 turns ingested, 40 queries, one machine.

System Build Query, median Index on disk Add one memory, search again Install
Carryover 0.0 s 0.5 ms 1.0 MB 1.7 ms 0 beyond stdlib
LlamaIndex BM25 0.1 s 0.1 ms 0.7 MB ✗ full rebuild — index is in memory ~970 MB env
txtai 1.5 s 8.5 ms 2.0 MB ✗ full rebuild — index is in memory ~970 MB env
mem0 (extraction off) 30.5 s 39.5 ms 3.7 MB — not run; qdrant writes incrementally, so no rebuild is expected 360 MB tree
MemOS minutes ✗ every query is an LLM call ✗ no local index ✗ no local index ~1.2 GB env
Cognee ~20 min ✗ every query is an LLM call ✗ no local index ✗ no local index ~1.0 GB env
SimpleMem ~2 h ✗ every query is an LLM call ✗ no local index ✗ no local index ~1.4 GB env

The bottom three build inside an LLM, so their wall-clock is a bill rather than a benchmark and is not comparable with the top four. Their query latency and index size are marked ✗ for the same reason: there is no local index to time or measure, and a millisecond figure for a network round trip would be a figure about the network.

Adding one memory and searching again costs 1.7 ms here and a full rebuild in the two fastest retrievers, because they hold their index in memory. Memory is accumulated one entry at a time, so that path runs far more often than any retrieval. Recorded in the other direction too: BM25 answers a query five times faster than Carryover, and both are far below anything perceptible.

Token cost per conversation

The metric this field advertises on. Twenty turns, counted from the usage the provider returns, against a floor of the same conversation with no memory.

Strategy Prompt tokens Of which cached Paid Over floor
No memory (floor) 5,566 5,426 140 n/a — this is the floor
Carryover — preamble written once 45,726 44,464 1,262 +1,122
Retrieve and inject per turn 21,191 18,711 2,480 +2,340

By nominal tokens the preamble costs 2.2× more. By what is charged it costs half as much, and the gap widens with the conversation — a preamble is a stable prefix every later turn reuses, while a retrieval written ahead of each turn rewrites the prefix and invalidates the cache behind it.

This one is worth every memory author's attention: a comparison that reports cost in nominal tokens is reporting the wrong number.

Does memory survive its facts changing

Twelve sessions in order, six of nine facts revised partway through, then asked for the current value. No other benchmark in this space tests this, and it is where mem0 bet the other way in 2026-04, replacing UPDATE/DELETE consolidation with single-pass accumulation.

System Returns current value Returns superseded Both, unmarked Unchanged rules kept
Carryover 6/6 0 0 3/3
mem0 (infer=True) 4/6 1 1 1/3
txtai · LlamaIndex BM25 ✗ retrievers, not memory — no notion of a fact being replaced
Cognee · MemOS · SimpleMem — not run

The last column matters more than the first: of three never-revised rules, mem0 kept one. A standing instruction that stops arriving is a standing instruction that stops being followed, and nothing indicates that it went.

And Carryover does not win this with the mechanism built for it: deduplication fired zero times and by design cannot, since differing numbers are a veto and 5494 against 8721 is exactly that shape. What carries it is that every entry states its date, so both versions sit in the store and a reader can order them.

Properties no benchmark scores

Carryover mem0 Cognee MemOS SimpleMem txtai · BM25
Writes need a person's approval yes no no no no n/a
Store is plain text you can edit yes no — vector db no no no — lancedb no
Runs with no service or model yes needs an embedder needs an LLM needs an embedder needs an LLM needs an embedder
Memory arrives without being asked yes no no no no no

One column is yes on all four rows. The last one especially is why Carryover exists: every other system here is query-driven — ask and it finds. "Never put function names in the external docs" is not something anyone asks for, and on 243 real turns 89% were instructions where nothing is queried at all. A memory that only works when asked is not working for 89% of your time.

Five that could not be measured

Not "worse at memory" — not measured, because something stopped the comparison running. Each is a real cost to an adopter, so they are named rather than left out.

System What blocked it
Graphiti Its embedded backend does not work: ten turns built 2 entities and 0 edges, and full-text search errors out because the driver never creates the index it queries. Working as advertised means running Neo4j and a JVM beside your app.
Zep Ships a client, not a library. No local mode, so nothing can be evaluated offline.
Letta No path for loading memory you already have. Memories form only during live agent turns.
Memori Captures by intercepting LLM calls, so only traffic through its wrapper becomes memory. A transcript cannot be handed to it.
TencentDB Agent Memory Three Node services and a proxy in front of the model's base URL. Infrastructure, not a dependency.

For Letta and Memori this limits how memory gets in, not answer quality. It is still a real constraint on adopting either: memory that cannot be loaded cannot be migrated, evaluated, or seeded.

The split

Every memory has a kind, and the kind decides how it reaches a conversation.

kind holds arrives as
user who the user is, how they work Stated in full, every conversation
feedback a correction or standing instruction Stated in full, every conversation
project a durable fact about a repo, system or decision One line in an index; fetched in full on demand
reference where something you had to find lives One line in an index; fetched in full on demand

The first two are behavioural: they change how the agent works and are never the subject of a question, so they are injected unconditionally.

The last two are lookup: most are irrelevant to any given conversation, and carrying all of them everywhere costs more than fetching the occasional right answer. They appear as a one-line summary, enough to know the fact was recorded, and are fetched in full when wanted.

That is the whole architecture. Everything below exists to make each half work.

How it works, briefly

The mechanisms and the measurements behind them are in DESIGN.md. In short:

A memory gets made by extraction running once per session, where a summary is already being made. It proposes; it cannot write. A candidate sits in a queue shown as undecided and expires if nobody acts, and suggest() and keep() are separate calls so that distinction cannot be erased by a flag. Facts carry the date they were true, which is not the date they were recorded.

Retrieval runs two mechanisms and picks per query by writing system: an FTS5 trigram index leads for scripts that separate words, a weighted substring scan leads for those that do not, because the most meaningful unit in Chinese is two characters and that is below what a trigram index holds. Weighting is by inverse document frequency, so a question's grammar stops outscoring its subject.

A duplicate write merges into the entry it restates and reports what it overwrote. Similarity alone decides nothing: three guards refuse a merge when the two differ in length, in the numbers they contain, or in polarity — the cases where near-identical text means different things.

A full store says so at 75% of the behavioural budget rather than at the limit, because a signal that fires on overflow fires after entries have stopped arriving. Three advisories — superseded, dormant, pressure — and none of them acts on its own.

What it deliberately does not do

Each was built or measured and then rejected. The numbers that decided each of them are in benchmarks/.

No embedding model. A dense retriever, same store, same turns, same judge: 18.1% then 13.7%, against the lexical path's 14.0% then 15.3%. The ranges overlap and the ordering flipped between runs — the difference is smaller than the judge's own variance, for 183 MB, a 150× slower build and a 25× slower query.

No knowledge graph. Synthesising across several facts is a real advantage, and the systems that lead end-to-end earn it that way. But counting the questions actually asked in one user's history, the share needing two or more stored facts combined is around 1% — a large piece of engineering for a question shape that barely occurs.

No per-turn automatic injection. Measured: roughly 85% of what would be injected does not help, paid on every turn. Something relevant exists for 64% of turns, retrieval finds it in a third of those, and no lexical threshold separates them — the misses are semantic, which the paragraph above rules out. It also costs more, for a reason visible only on the bill: writing ahead of each turn rewrites the prompt prefix and invalidates the cache behind it, while a preamble written once is reused. Over twenty turns it used half the nominal tokens and paid twice as much.

No automatic retirement. The detection is here and proposes only. Retiring a rule that still holds changes how an agent behaves with nothing to show it; leaving a stale one costs a line of context. The costs are not symmetric, so the decision is not automatic.


Using it

Python 3.11 or newer. One dependency, which pip will fetch:

pip install git+https://github.com/goldenxingxing/carryover

Not on PyPI yet: nothing is published while the API can still move. Pin a commit if you depend on it. What it is measured by, and the conclusions that were published and later retracted, are in benchmarks/README.md.

The whole integration

Four places to call it. This is a complete host, not a sketch: it was written against a fresh install and run, and it is twenty-three lines because the package does the rest.

import carryover

store = carryover.Store("~/.myagent/memory")  # your path; `~` is expanded


# 1. Opening the conversation. No query — what the agent most needs to know
#    is what nobody would think to ask for.
def opening_context() -> str:
    return carryover.render(store.entries(), store.recent(), store.pending())


# 2. One tool. The operations are validated data, and executing them is
#    included; what stays yours is whether to ask first.
async def memory_tool(payload: dict) -> str:
    try:
        op = carryover.parse_operation(payload)
    except carryover.UnknownOperation as exc:
        return f"error: {exc}"  # names the real operations
    if op.writes and not await ask_the_user(op.describe()):
        return "declined"
    return carryover.execute(store, op).model_dump_json(exclude_defaults=True)


# 3. Ending it. Extraction notices; a person decides.
async def on_session_end(transcript: str, complete) -> None:
    store.suggest(await carryover.propose(complete, transcript))
    store.note_topics(transcript)  # feeds the dormancy ranking
    append_summary(store.directory / "recent.jsonl", summary_of(transcript))


# 4. Whatever your approval is. A dialog, a CLI prompt, a policy — this
#    package has no opinion, and no way to answer for you.
async def ask_the_user(what: str) -> bool: ...

complete is your model. Anything matching async (system, user) -> str works; nothing here imports a client or reads an environment variable.

The operations

parse_operation validates, execute performs. Twelve verbs, each mapping to a Store method, so a host that wires this function needs no schema of its own:

  • search, get, list: reading
  • add: a fact the user stated outright, so there is no approval to ask for
  • promote, dismiss: the queue, and the only path from proposal to memory
  • update, delete: changing or removing one
  • retire, restore: out of the preamble without being lost
  • affirm: this one still holds. The answer that is not retirement
  • consolidate: replace several with one that keeps what all of them said

op.writes says whether performing it changes the store, so a host gating writes does not keep its own list — and cannot fall behind when this package gains a verb. affirm reports False there deliberately: it stores no fact and removes none, it records that a question was answered, and asking someone to approve the storing of their own answer is a loop with nothing at the end.

If your agent reaches these through a tool of its own, say so — the preamble tells it what to call, so the wording has to name something that exists in your host:

carryover.render(entries, recent, pending, actions=carryover.Actions(
    search='`Memory({"op": "search", "query": "<query>"})`',
    promote='`Memory({"op": "promote", "id": "<id>"})`',
    ...
))

The defaults are the operations above, so a host that passes them to execute needs no override at all.

Doing it without the operation layer is the same call:

store.add("project", "Reports go in output/reports/daily/.", key="reports/daily")
store.search("where do daily reports go")  # the question as asked, not keywords
store.get("reports/daily")  # by key, id, or unambiguous id prefix
store.keep(candidate_id)  # only after someone said yes

Keeping it affordable

The store fills and entries stop being true. None of this acts on its own — see How the store stays affordable for what each one means:

fit, held = carryover.pressure(store.entries(), carryover.BEHAVIOURAL_BUDGET_CHARS)
carryover.find_superseded(store.entries())  # a newer entry that replaced an older
carryover.find_dormant(store.entries(), now=time.time())

store.affirm("reports/daily")  # raised, and it still holds
store.consolidate(merged, replacing=["reports/v1", "reports/v2"])
store.retire("api/version")  # out of the preamble, still in the file

Prefer consolidate over retire when a later entry restates an earlier one.

If you are running on a server

Two things default to this process, which is right on a desktop and arbitrary in a request handler:

await carryover.propose(complete, transcript, today=user_local_date)
await carryover.propose(complete, transcript, prompt=your_own_template)

today is what the model resolves "last Tuesday" against, and it is written into a fact that outlives the conversation — a host in UTC serving someone in Shanghai disagrees about the date for several hours out of every twenty-four. prompt is for a host whose domain, languages or rules about what may be recorded are its own; it must take {conversation} and {today}, and copying the function to change one paragraph is how a fork starts.

What if there is nobody to approve

The gate needs a decision by a person, not a user interface. In an agent the conversation is where that happens: render puts pending proposals in the opening context, and near expiry it asks for them to be raised before the conversation ends rather than when they are relevant.

How reliably that works is measured, because it is the assumption the gate rests on. On a task that never touches the queue's subject, the original wording was raised 1% of the time across four runs of thirty; the escalated one, 25% (20–40%). So the conversation is a real path and not a guaranteed one — a host with an interface should put the queue in it too, and the endpoints for that are store.pending(), keep() and dismiss().

There is deliberately no setting that approves for you — not because it cannot be done, but because it is three lines that belong in your code, where a reviewer can see them:

for candidate in await propose(complete, transcript):
    store.suggest([candidate])
    store.keep(candidate.id)  # unattended: no one is going to be asked

As a default it would be the behaviour of everyone who never looked, and the one column this project can claim over every alternative would quietly read the same as theirs.

Nothing accumulates if nobody decides: proposals cap at twelve and expire after a fortnight.

Checking that it is actually working

Every way this fails is quiet. Extraction that finds nothing looks like a conversation with nothing worth keeping — the common and correct answer. A search with no match looks like a store with no match. Nothing raises, on purpose: a memory system must not break what it is attached to. The cost is that you cannot tell working from absent by watching.

python -m carryover check ~/.myagent/memory

Capability is about the machine — the index builds, a probe written to a scratch directory comes back, in a spaced script and an unspaced one. It touches nothing of yours. Evidence is about your store: a session summary means something calls the end-of-session hook, a queued proposal means extraction ran, a topical stamp means the conversation was handed back, an entry means somebody approved one.

It does not guess. "Found nothing" and "never called" leave identical stores, so it reports both readings and names the experiment that separates them — rather than picking one, which is the mistake this whole design exists to avoid.

Design commitments

Constraints, not preferences. Each is something Carryover will give up performance for.

No dependency beyond the standard library sqlite3 ships with Python. Retrieval adds nothing else; pydantic is the one import, for validation at the file boundary.
The store is a text file Greppable, diffable, editable, versionable. The index is a cache that rebuilds.
Nothing is written without approval Extraction produces candidates. Candidates are proposals.
Nothing is destroyed Superseded entries are retired: out of the injected set, still in the file, still searchable, restorable.
The model is yours Carryover never imports an LLM client. A test asserts it.

About

Cross-session memory for agents — two text files and the standard library

Resources

Stars

3 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages