Skip to content

Fix what live-testing code search and shared memory found - #396

Merged
uzayer merged 3 commits into
0.4.1from
fix/codeindex-memory-live-test
Oct 9, 2026
Merged

uzayer merged 3 commits into
0.4.1from
fix/codeindex-memory-live-test

Conversation

@uzayer

@uzayer uzayer commented Oct 9, 2026 •

Copy link
Copy Markdown
Collaborator

What?

Fixes for what a live test of the code-search (#382) and shared-memory (#387) pipelines found, run against this repo and the installed 0.4.1 app. Three commits:

  1. codeindex: the engine (vendor/atlas-engine, two thirds of the source) is indexed; built-in skips show up in the skipped counts; find_symbol says when nothing matches the name; the resolver types Rust method receivers.
  2. models: an empty model folder is no longer counted as an install; a composer pill offers the code embedding model, which semantic search needs and nothing prompted for.
  3. memory: capture recovers, the briefing is accurate, memory_search output is bounded and filtered by relevance, and the session token no longer reaches an agent's command line (ADR-0020).

Why?

What the live test found, each now covered by a test:

  • Session token visible in ps. The token was in claude --mcp-config {…"Authorization":"Bearer <token>"}, and any local process could read it. Putting it in env instead doesn't help, because the adapter copies env onto the command line too.
  • No durable memory captured since Oct 3. Only in-app sessions were extracted, and the work had moved to terminal sessions.
  • The briefing:
    • showed an Aug plan, every item done, as "active";
    • listed files from other checkouts;
    • had no handoff field.
  • memory_search:
    • returned whole transcripts, about 5–8k tokens for 5 results;
    • ranked unrelated entries at the top when nothing matched.
  • The engine was missing from the code index, so find_symbol AppServerClient returned unrelated hits.
  • find_symbol dressed loose matches up as answers.
  • Semantic search silently ran keyword-only. The model was never downloaded, and looking up its path created an empty folder that looked like an install.

New data flow, behind a setting that is off by default. "Learn from sessions outside Atlas" lets extraction read sessions the recorder imported from the terminal. Their transcripts then go to the extraction model, the same way in-app sessions' do. It is a per-project switch next to the sharing switch, and it is greyed out while sharing is off.

How?

  • Token (ADR-0020, amends 0010, 0015 and 0019).
    • Only the in-process native agent gets the loopback servers over HTTP. Every ACP agent gets them through the stdio bridge, which reads the token from a 0600 file. The decision is made per connection (in_process), never per agent id.
    • Each token file is deleted when its session ends. Files from ended launches are cleared at startup, off the runtime thread. A running launch holds a lock, so it is never cleared.
    • Proven against the live app: driven through the real claude-acp adapter and claude CLI, neither claude nor the bridge has the token in its command line.
  • Capture. Catch-up passes are prompt-sized, at most 3 per job, and skip any session active in the last 15 minutes in any process. Failures log at warn.
  • Briefing.
    • Finished or superseded plans are no longer shown as active.
    • Paths are kept to the project root and its worktrees, with .., ~ and linked-worktree roots resolved.
    • handoff is always present (null when there is none), and one is built for sessions that never recorded their end.
    • Tested on a copy of the live store, before → after:
      • plan: stale → none;
      • out-of-repo paths: 16 → 0;
      • handoff: missing → present;
      • newest durable memory: Oct 2 → Oct 9.
  • memory_search.
    • Results are excerpts inside a max_output_tokens budget (default 4096), with a truncation marker.
    • A relevance floor: words or meaning must support each hit.
    • Tested on a copy of the live store: an off-topic query goes from about 1,390 tokens to an empty result with a note; "engine quarantine" goes from about 8,363 tokens to 324, the right passage only.
    • AtlasMemBench passes unchanged.
  • Code index.
    • .atlasignore re-includes vendor/atlas-engine. A re-include now lifts only the skip it names, so vendor/atlas-engine/target stays pruned.
    • On a clone: 1,557 → 4,626 files, 21k → 69k symbols, and a full build in 6.5 s.
    • EXTRACTOR_VERSION is 2.3, so existing indexes rebuild once.
  • Call graph, measured against rust-analyzer (SCIP) on crates/ and src-tauri/.
    • Calls-only recall goes 0.849 → 0.926, precision 0.972 → 0.980.
    • On Code search for agents: atlas_code tools, SQLite code index, graph, hybrid semantic search, grep index #382's metric, which counts every reference on a call line including types and fields, recall goes 0.661 → 0.700, and that metric can't get far above 0.70. The 0.85 target is met only on the calls-only measure; eval_edges now prints both.
    • The keyword-only MRR drop (0.582 → 0.504) is corpus growth, not a code change: the same code still scores 0.583 on the merge-time snapshot.

Not fixed:

  • clippy.toml's config warnings: it is upstream's file.
  • MemoryRegistry::request_reindex is already called only from tests at the base commit; probably dead since code left the memory corpus.
  • For a query with no real answer, memory_search can still return a couple of loosely related excerpts.

Checklist

  • Targets the current version branch
  • New behaviour has a test; each bug fix has a test that fails without it
  • You've run the app and used the change in a window. Not yet. The download pill and the new toggle were checked in bun run dev (headless Chrome). The token path was proven with a debug binary and the real claude-acp adapter against the installed app. No session has been driven through a debug build of Atlas itself.

bun run ci:local is green from clean builds, including app (src-tauri, Linux) in the container. The Windows compile is CI-only, and launching the stdio bridge on Windows is still the manual check ADR-0019 describes. It now applies to every ACP agent.

Co-authored-by: Ahammad Nafiz ahammadnafiz@outlook.com

@coderabbitai

coderabbitai Bot commented Oct 9, 2026 •

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration
  • Configuration used: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 7e52d0ef-7ece-4510-8e24-67929f859263

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Comment @coderabbitai help to get the list of available commands.

uzayer and others added 3 commits October 10, 2026 00:15
…symbol has no match, type method receivers

- .atlasignore re-includes vendor/atlas-engine (first-party per ADR-0003); a
  re-include now lifts only the skip it names, so target/ and node_modules
  inside it stay pruned. The CI planner classifies it as an input of
  atlas-codeindex's tests.
- files pruned by built-in rules (vendor, .atlasignore) count as skipped
- find_symbol leads with 'no exact match for "X"' when nothing matches the name
- the resolver infers Rust receiver types (let bindings, tuples, fields, method
  chains, Arc/Box/guards, ?/await) and links a method only to that type's impl;
  a name-only match in another crate drops to 0.375 confidence
- eval_edges reports call-only recall beside the historical metric; a gold
  answer gains the moved memory store
- examples get Metal on macOS like the app does

Call recall vs rust-analyzer: 0.849 -> 0.926, precision 0.972 -> 0.980. On the
PR #382 metric (every reference on a call line, types included) 0.661 -> 0.700;
that metric is capped near 0.70. EXTRACTOR_VERSION 2.3: indexes rebuild once.

Co-authored-by: Ahammad Nafiz <ahammadnafiz@outlook.com>
…r offers the code embedding model

- looking up a model's path no longer creates its folder; installed means every
  model file is present, and model_download creates the folder on tokio::fs
- an 'Enable semantic search' pill in the composer starts the download or opens
  Local Models; hidden while agent code tools are off, dismissible per model
  (every open composer hears it), never downloads on its own
- the mock backend keeps memory and code model selections apart and fakes a
  download

Co-authored-by: Ahammad Nafiz <ahammadnafiz@outlook.com>
…s bounded, and no session token reaches an agent's argv

Capture and briefing
- extraction catches up on recorded sessions in prompt-sized passes, at most 3
  per job, skipping anything active in the last 15 minutes (in any process);
  failures log at warn instead of debug
- sessions imported from outside Atlas (terminal agents) are read only when the
  new per-project 'Learn from sessions outside Atlas' setting is on; it is off
  by default because it sends their transcripts to the extraction model
- a plan whose items are all done (or superseded) is no longer the active plan
- filesChanged keeps only paths under the project root or its worktrees, with
  .., ~ and linked-worktree roots resolved; out-of-root paths are not captured
- the briefing always carries handoff (null when there is none) and builds one
  for a session that never recorded its end

memory_search
- documents are excerpts around the matching passage inside a
  max_output_tokens budget (1000-12000, default 4096) with a truncation marker
- a relevance floor: a match must be supported by its words or its meaning, and
  a query nothing supports returns an empty result with a note

MCP credentials (ADR-0020, amends 0010, 0015, 0019)
- only the in-process agent gets the loopback servers over HTTP with a bearer
  header; every ACP agent gets them through the stdio bridge, which reads the
  token from a 0600 file, because adapters copy an entry's headers and env onto
  the agent's command line
- token files are removed when their session ends; ended launches are swept at
  startup off the runtime thread; a live launch holds a lock and is never swept

Tests for each; AtlasMemBench passes unchanged; parity_bench fixtures now carry
a body as the corpus format requires.

Co-authored-by: Ahammad Nafiz <ahammadnafiz@outlook.com>
@uzayer
uzayer force-pushed the fix/codeindex-memory-live-test branch from 71346e3 to c9e7a40 Compare October 9, 2026 18:16
@uzayer
uzayer merged commit 25ae6cf into 0.4.1 Oct 9, 2026
36 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant