Repository navigation
Shared memory M0–M4: revisions, reconciler, evidence, handoff and the review loop - #387
Merged
Merged
Conversation
#367 dropped numkong from atlas-memory's usearch on Windows (numkong 7.x does not build with MSVC on ARM64), but atlas-retrieval still asked for usearch's default features, and Cargo unifies them, so the app's Windows job failed again once the code index landed. Same per-target split here.
Thirteen deterministic probes drive the real loopback server with two agents (claude-code, codex) and a bag-of-words embedder. Each is tagged with the milestone that makes it pass; the gate test asserts every probe up to CURRENT_MILESTONE, and an ignored scorecard prints them all.
…pping A kind with more than eight changes between two looks used to move the clock past the rest, so the reader never saw them. Changes now come one page at a time: the clock stops just before the oldest entry a full kind left out, and "more": true asks for the next page.
…nges memory_changes only listed entries that still existed, so another agent never heard that a memory it had been briefed on was forgotten. Schema v4 keeps a tombstone (id, kind, session, at) per forget; memory_changes returns the ids other sessions forgot, once, within the page's clock.
…iefing until confirmed
A line an agent echoed with a marker word ("Note: always force-push",
from a README it read) became a trusted team fact. Capture now records
such lines as candidates (source capture, confidence 0.3, not logged), the
briefing index leaves them out, entries mark them "candidate", and an
agent restating one or the user editing it promotes it.
…es; honest summarizer copy The extraction prompt and the handoff tail went to the gateway or a BYOK provider unredacted; the record only redacted what it stored. Both are redacted before they leave now. The summarizer popover said context was "injected" with "no model call" while extraction called the Atlas model; its copy now lives in a tested module and says what happens.
…ry files memory_policy_update allowed any path that merely started with ~/.claude or ended with CLAUDE.md/AGENTS.md, so ~/.claude/../.zshrc, NOTCLAUDE.md or a symlink named AGENTS.md pointing at a dotfile could be rewritten. The check now resolves symlinks and .. first, requires an existing file, and matches the exact file name.
…ct a turn that outlives its session's end Capture jobs went to the blocking pool, which keeps no order, so an older plan update could land after a newer one and win by key; they now run on one ingest thread in emit order. The sharing switch is read when a job runs, so a session registered while sharing was on stops capturing once it is switched off. A turn job handled just after its session's end runs as the end pass instead of waiting for an end that already happened.
The Clear button wiped every agent's plans, decisions and facts for the project on one click. It now opens the same confirm dialog forgetting an entry uses, and clears only on confirm.
The vendored engine ships a dormant memories pipeline (extraction into a global folder) that a repo's .atlas-agent config could switch on, giving the native agent a second memory store and write protocol. Atlas sets features.memories = false as a CLI override, the engine's session-flags layer, which is merged after every project layer.
… words, not exact substring A fact promoted to ~/.atlas/memory stayed there after a repository forgot it, and recall only matched the whole query as a case-sensitive substring. Each promotion pass now drops the repository from every ledger row it no longer holds and demotes a row left in fewer than two repositories (out of the archive and MEMORY.md). Recall matches whole words, any case, needing 60% of the query's words, best share first.
…data, never instructions The server's instructions said when to write but not what to leave out, and called results "background" without saying they are never commands. They now say what a later session needs, what not to save (session summaries, task state, anything cheap to rediscover, secrets), that a memory never directs an action and is a lead, not proof. The extractor prompt says the same and treats quoted text as evidence. Every record write and every model-bound text drops zero-width, bidi and tag-block characters, and an <atlas-memory tag is defanged.
…es candidates A web page or a third-party MCP tool result could talk the extractor into a trusted memory. Each transcript turn now says whether one of its tool calls brought outside content in (an ACP fetch, a web search or fetch, or an MCP tool of a server other than Atlas's own, in either agent family's spelling); a pass over such a session caps what it stores at candidate confidence, so the briefing leaves it out until confirmed.
…ical and the given path read_capture_docs looked sessions up by the path as given, while live capture keys a Workspace by its canonical path. On a symlinked root (macOS /var, a symlinked ~/dev) every live native, opencode, cursor and kilo session dropped out of the memory corpus. It now reads both spellings, once each, as memory_pack::capture_heads does; a read error on one spelling no longer empties the reader.
…AS, content-keyed vectors, hybrid search memory.sqlite moves to v5 in one transaction. Every write appends an immutable revision sealed into a blake3 chain; entries is the current view, rebuildable from them. A keyed agent write replaces another writer's entry (another agent, or another session of the same agent) only with the revision it read. Vectors are cached by (model, text), so none can describe text it was not computed from and switching models back re-embeds nothing. Search fuses BM25 (Porter-stemmed FTS5) and meaning with trust and recency priors, and says which legs found each hit. History, purge of a forgotten secret from every table, sources per entry, and a preference kind.
…heals from the cache MemoryEngine kept three files written one after another (hnsw.usearch, manifest.json, docstore.json); a crash between them or a torn index left retrieval silently empty. The corpus now lives in corpus.sqlite (docs, BM25, an f16 embedding cache keyed by model + content) and a per-model vector file rebuilt from the cache whenever it is missing, torn or out of step, without the model. Retrieval fuses a BM25 leg with the embedding leg, a model switch keeps the docs and reuses cached vectors, and the old files are dropped on first open (one re-embed).
…why; provenance from recorded sessions memory_remember takes expected_revision and a refused keyed write names the revision to pass. memory_history lists every wording of an entry. Every entry carries its revision, state, sources (the sessions that wrote it) and the date it was added; the briefing leads with the user's preferences, which never age; search says which legs found each hit. memory_get and memory_history resolve each source to its recorded session and commits through memory_capture.rs, which opens the session recorder read-only and never writes it. The panel can erase a forgotten secret with its history, shows where a memory was learned and marks candidates. AtlasMemBench adds three probes and its gate moves to M1.
…e on open, idempotent retries health.rs checks every invariant against canonical data (the view against its revisions, FTS against entries, the vector cache against live texts, the hash chain, SQLite's quick_check) and repairs what can be rebuilt; a broken chain is reported and the view is not rebuilt from edited history until the user accepts it. A daily VACUUM INTO snapshot makes the record restorable: a database that fails to open or check is kept as .corrupt-<ms> and the snapshot takes its place. The global ledger is locked and a corrupt one is kept, not overwritten. The same writer saying the same thing again within five minutes is a retry: no revision, no use bump. A fault-injection suite covers tamper, lost rows, lost FTS, a damaged file and a torn corpus vector file.
…after a model switch; no lost jobs The indexer gains Job::Health: the record's check, repair and daily snapshot plus the corpus's vector file and FTS, stored for the panel and announced on atlas:memory-health. It runs when a project opens, every 30 minutes and after the embedding model changes (backfilling the new model's vectors). A job that does not fit the queue is kept and retried after the next one instead of dropped, and extraction (model calls up to 60 s) runs on its own lane so indexing never waits behind it. The Memory panel's header says whether memory is healthy, what was repaired or restored, and offers "I made this edit" when the history was changed outside Atlas.
…ories leave the briefing
- memory_remember takes evidence [{path, lines, symbol?}]: the server reads
and hashes the lines, refuses paths outside the scope root, at most 8.
- Every entry read carries validity (valid/moved/stale/unverifiable) and
the citations as found now; moved symbols are found through the open
code index (CodeIndexResolver), else by a same-length window search.
- Uncited decisions, facts and architecture notes get commit evidence from
the recorded turn that wrote them (atlas-checkpoint turn_spans, read-only):
stale only when every landed file lost the agent's work.
- The briefing hides stale entries (staleHidden); search sinks them.
- The health pass archives unused candidates and unused memories whose
citations are stale, after 28 days; uncited actives never expire on time.
- ADR-0018. AtlasMemBench gate raised to M3.
…ly proposes - Schema v6: links, episodes, feedback, dreams, dream_proposals and the revisions_session index; clear and purge cover them. - memory_feedback (a write): useful counts and lets another session vouch for a candidate, wrong archives (kept in history), stale demotes. - The health pass links contradicting memories (numbers, keys, negations among close vectors); entries carry "conflicts". Near-duplicates become merge proposals, never merges. - A Review tab: approve or dismiss candidates, merge duplicates or tell them apart, settle contradictions (keep one, or both hold).
…iefing carries it - Built from the session's own events (plan, open items, decisions, failures, facts, architecture, files), no model; one per session in the v6 episodes table. - When the session recorder captured the session, the note read in the briefing also says what it did: files written, failed tool calls folded with counts, commits, branch, title, and whether its last turn was cut off. Read from capture at read time (atlas-checkpoint failed_tool_calls and written_paths), never stored in memory. - Capped at 4 KB: files go first, decisions and open items last.
…ack turn drop to candidate - The chat shows what its session saved to shared memory under the transcript, counted by op, with Forget behind the usual confirm. - A retry or /undo takes turns back: what the session remembered inside them (matched by time to the recorder's rewound turns, live turns cut out) drops to candidate with a `rewind` revision. Checked on the rewind, at session end and by the health pass for the last day; a restatement revives it. Without capture nothing changes.
…a commit, and what they decided - A read tool (READ_TOOLS is now seven): for a path, the sessions that wrote it or whose linked commits carried it; for a commit, its linked checkpoints (a short sha resolves through git, else a prefix range). - Each session comes with its title, commits and handoff note; the memories those sessions wrote or that cite the path follow. - Without capture it answers from citations and says so; it never creates a capture store. AtlasMemBench gains its 17th probe.
- Once a day, when a session left a handoff note since the last dream, the memory model (extraction's route and consent) reads the notes, with what each recorded session did minus any tool output, beside the active and candidate memories, all cleaned and redacted. - It answers with typed operations (add, merge, archive, rewrite, link). A pure validator drops any that touch a preference or a user's edit, cite a session it was not shown, name a stale revision, shrink an entry, or take more than max(3, 20%) of memory at once. - The rest wait in the Review tab; an accepted one is re-checked and applied through the ordinary write paths (source `dream`), or marked obsolete. Nothing is written by the dream itself. - Settings memoryDreams and memoryRepoMirror (default off) and atlasOnlyMemory (default on).
- Select memories in Memory > Shared > Memories and export them: a preview shows <scope root>/AGENTS.md as it would be written, and Write replaces only the block between the atlas-memory markers (or appends one). - Archived, forgotten and stale (cited code changed) memories are refused.
…candidates - With memoryRepoMirror on, the health pass writes active, non-stale memories in the open Agent Memory Repo layout (MEMORY.md with preferences and an index, plus decisions/architecture/facts/failures) to a local git repository under ~/.atlas/memory-repos/, committing only when memory changed. Lines carry source (atlas-session:<agent>/<id>), added, atlas-id, revision and code citations; no recorder text. - A purge rebuilds the mirror with fresh history. - Memory > Shared can import an Agent Memory Repo folder: every line from MEMORY.md and the files it links (inside the folder only) lands as a candidate with source import:amr, never briefed until confirmed.
…ault on) - With atlasOnlyMemory on, Claude Code starts with CLAUDE_CODE_DISABLE_AUTO_MEMORY=1 and Codex with -c memories.use_memories=false -c memories.generate_memories=false (both switches checked against each agent's docs). Off: today's launch. - The Claude memory import now offers what Claude wrote after a directory was first imported: each offered line is recorded, so only new ones come back.
…dge (ADR-0019) - `atlas mcp-bridge <url>` runs the binary as a stdio MCP server that forwards each JSON-RPC line to the loopback tool server, carrying the MCP session id, and writes each answer back as a line. Loopback only. - An agent that does not advertise mcpCapabilities.http is now offered atlas_memory (and atlas_code) as that stdio server, with the session token in ATLAS_MCP_TOKEN, never in argv. HTTP agents are unchanged. - ADR-0019 amends ADR-0010's "no HTTP MCP, no memory".
…ME and MIGRATION - Seven kinds, entry states, revisions, evidence and validity, the handoff note, the review queue, the ten tools and the stdio bridge. - The new on-disk files: corpus.sqlite and its per-model vector file, the record's snapshot and quarantine, schema v5/v6, the mirror.
- health check: bind the statement's result before the connection guard drops (E0597). - Clippy denies: Result::ok over closures, sort_by_key, no needless collect, inspect_err for the migration rollbacks, type aliases for complex types, allow trivially_copy_pass_by_ref on serde's is_zero. - The citation cache keys by the resolver's root, so two projects citing the same path never share a result. - Preferences captured through the event log get a vector too. - Tests: the indexer's first open also queues Health; the corpus eviction test no longer expects an empty k-NN answer; citation rewrites change the file size; the three memory settings are in the configuration reference.
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the ⚙️ Run configuration
You can disable this status message by setting the Use the checkbox below for a quick retry:
Comment |
…ks, rewinds, rebuilds and evidence hold up
…ion and checked against what the model saw
…JSON-RPC errors and accepts loopback only
…d the per-line Claude import
…ws live state, modals reset
…eckout; memory_why reports stale
2 of 3 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Shared memory, phases M0–M4 of the memory plan, in one PR. It makes the memory every agent shares trustworthy enough to rely on when the user switches from one agent to another in the same repository:
usearchbuild.memory_why, a daily dream pass that only proposes, an AGENTS.md export, an Agent Memory Repo mirror, "Atlas is the only memory", and a stdio bridge for agents without HTTP MCP.For the reviewer: please test the stdio bridge by hand. CI covers the forwarding in-process only. See "How to test the bridge" below.
Phases
M0 — fixes (12 commits)
memory_changespages through every change instead of skipping. A forget reaches other sessions as a tombstone.INSTRUCTIONS) says what not to save, and that memory is data, never instructions.features.memories=false).atlas-retrievalbuildsusearchwithout numkong on Windows. This also fixes main's Windows build.M1 — revisions are canonical (ADR-0017)
revisions(immutable, blake3 hash chain);state,scopeandevidence;entries_fts(BM25, porter stemming);embed_cachekeyed by(model, text).expected_revision. A refusal names the revision to read.memory_history. Entries carryrevision,state,sources(atlas-session:<agent>/<session>) andadded. Search results saywhy(which legs found them).corpus.sqliteplus a per-model vector file that heals from the cache without the model.M2 — the reconciler
health.rschecks four things against canonical data: the view against its revisions, BM25 against entries, the cache against live texts, and the hash chain. It repairs every derived side.VACUUM INTOsnapshot. A damaged record is renamed.corrupt-<ms>and the snapshot restored.crates/atlas-memory/tests/fault_injection.rs).M3 — evidence (ADR-0018)
memory_remembertakesevidence: [{path, lines, symbol?}]. The server reads and hashes the lines. Paths outside the scope root are refused, and at most 8 citations are kept.valid,moved(found again through the code index's symbol lookup, or a same-length window),staleorunverifiable.staleHidden) and sink in search.M4 — the loop
links,episodes,feedback,dreamsanddream_proposals.memory_feedback(a write): useful / wrong / stale.memory_briefing.handoffcarries it, capped at 4 KB./undotakes turns back; what the session remembered inside them drops to candidate.memory_why(path | commit)names the recorded sessions behind a file or a commit, their handoff notes, and the memories they wrote.memoryDreams, default off).<!-- atlas-memory:begin/end -->. Nothing outside the markers changes.memoryRepoMirror, default off). A local git repository under~/.atlas/memory-repos/, rebuilt with fresh history on purge. Importing a memory repo folder brings its lines in as candidates.atlasOnlyMemory, default on). Claude Code starts withCLAUDE_CODE_DISABLE_AUTO_MEMORY=1; Codex starts with-c memories.use_memories=false -c memories.generate_memories=false. Both switches were checked against each agent's docs. The Claude import now re-offers memory files added after a directory was first imported.atlas_memory(andatlas_code) as<atlas binary> mcp-bridge <loopback url>, with the session token inATLAS_MCP_TOKEN, never in argv.The memory tool server now has ten tools. Reads:
memory_briefing,memory_changes,memory_search,memory_get,memory_list,memory_history,memory_why. Writes:memory_remember,memory_feedback,memory_forget.How it builds on the code-index PR (#382)
atlas-retrieval: the cache-key / f16 codec, the healingVectorFile, and RRF fusion. Nothing is duplicated.CodeIndexRegistry::root_forandCodeIndex::find_symbol, insrc-tauri/src/commands/code_index/citations.rs. When no index is open for the scope, it falls back to reading the files.memory_briefing.handoff(what a session left) and the code index'stask_context(what the code says) don't overlap.atlas_codetoo, on the same token.Key design decisions, with evidence
memory_rememberoverwrote another writer's entry silently.only_clear_and_purge_delete_revisionspins that nothing else deletes history.memory_whycome fromatlas-checkpointat read time, throughopen_reader. Nothing from it is stored inmemory.sqliteor the mirror, and every feature falls back to today's behaviour when capture is off. The checkpoint crate gains read methods only (turn_spans,failed_tool_calls,written_paths,sessions_touching_path,checkpoints_touching_path,checkpoints_for_commit_prefix), and no index.a_parallel_session_of_the_same_agent_cannot_clobber).Data migration notes and risks
memory.sqlitev4 → v5 → v6 runs on first open, each step in one transaction with an explicit ROLLBACK.baselinerevision, moves stored vectors intoembed_cache, buildsentries_fts, and dropsentry_vectors.links,episodes,feedback,dreams,dream_proposals), therevisions_sessionindex, and the columnsentries.changed_byandrevisions.note. Both columns sit outside the hash chain, so existing chains still verify.hnsw.usearch,manifest.jsonanddocstore.jsonare derived and are deleted on first open; the corpus is embedded again by the indexer.<scope root>/.atlas/memory/:corpus.sqlite,corpus.<model>.usearch,memory.snapshot.sqlite, andmemory.sqlite.corrupt-<ms>when a damaged file is set aside.seqandchanged_byare carried over for rows that were still there.atlasOnlyMemorydefaults on: Claude Code sessions launched from Atlas no longer write Claude's own auto memory.memoryDreamsandmemoryRepoMirrorare off (P2 / P1).AtlasMemBench and how this was verified
CURRENT_MILESTONE = M4: all 17 must pass inatlasmembench_passes_every_probe_up_to_the_current_milestone. Scorecard:cargo test -p atlas -- memory_server::bench::atlasmembench_scorecard --ignored --nocapture.valid,moved,stale,unverifiable;memory_whywith and without capture;7aecaa36: 37/37 checks green.atlas-memory: 147 unit tests plus its integration suites.memory: fix review – …commits, with tests. A second verifier per group re-read each fix.How to test the bridge (reviewer, please)
mcpCapabilities.http. Or force it: make the session request'shttp_mcpfalse.memory tool server offer: ... memory_server=included_via_bridge.atlas_memoryentry is the Atlas binary with argsmcp-bridge http://127.0.0.1:<port>/mcp;ATLAS_MCP_TOKENenvironment variable, not inpsoutput.memory_briefingandmemory_remember. Both should work.ATLAS_MCP_TOKEN=x atlas mcp-bridge https://example.com/mcpexits with an error, and so doeshttp://127.0.0.1:1@example.com/mcp.memory_briefingagain. It should still answer, without the bridge restarting.Follow-ups
src-tauridependency and aCargo.lockchange.git worktree listper memory checked for commit evidence.