feat(web): every conversation feeds a memory, quoted and never summarized

[adityak74]

Rebased onto 0f58b976. #83, the branch this started from, squash-merged as 195c8cc0, and #85, #86, #82 and #84 landed after it, so this now targets main and merges on its own. #83 gave zorp an on-device index of the conversations already in the store, read by a sidebar search box. This is the second way that index gets read: not into the sidebar, into a live turn.

Every finished turn indexes its own session in the background, so the corpus keeps up without anybody pressing Index. A turn can then be told to read it before answering, behind a non-default memory feature on zorp-web, a "memory": true field on POST /api/sessions/:id/turn, and a checkbox next to the composer that starts unticked on every message.

Exactly how model-derived claims are kept from being laundered into fact

There is no extraction step, so there is nothing to launder. No model is asked to read the corpus and write down what it learned. There is no claim table, no profile, and no row anywhere holding a sentence a model composed about the past. Retrieval alone answers the ask, and a derived store would have built precisely the failure CLAUDE.md names for aryabhatta: a guess written down as a fact and cited six weeks later as something the corpus says.

The chain that makes that structural rather than a promise:

1. chunks.text in the index is only ever written by Index::replace from a Chunk.text, which zorp-web::recall::chunks_for fills verbatim from Message::text(). There is no other writer.
2. Passage.text is that column read back. memory::citation clones it. memory::block prints it between two boundary lines and adds nothing. the_quoted_text_is_the_stored_text_and_nothing_else asserts the bytes between the boundary lines equal the stored message exactly.
3. The recalled block is never persisted. It is appended to the seed, which with_message_records counts as already recorded, so sync never offers it to the recorder. Persisted, it would be embedded by the next feed and recalled by the turn after that, and the harness's own framing of somebody else's text would become a thing the corpus says. the_recalled_block_is_never_written_back_into_the_conversation_store runs a real memory turn and asserts the store holds exactly what was typed.

One kind of model-authored text does reach the prompt, unavoidably: half of every conversation was written by an assistant. It is labelled, not laundered. role travels from the store to the index to the block to the browser, and an assistant excerpt reads written by the assistant, a model's earlier output, not a checked fact in the block, and the assistant, a model's earlier answer on the page, with its own CSS class. An unrecognized role is treated as model output, because being wrong in that direction costs a needless caveat and being wrong the other way calls something a fact on no evidence.

Exactly how retrieved memory is kept inert

Not a filter on the text. A filter on the text can be worded around, and refusing to recall anything that reads like an instruction would refuse half of a legitimate corpus.

1. A per-turn nonce on the fence. The block opens BEGIN RECALLED CONVERSATION EXCERPTS <16 hex> and closes with the same marker, and every excerpt header carries it too. It is derived from a nanosecond clock read, the pid, and the passages. Text written before this turn cannot close a fence it has never seen. an_excerpt_that_quotes_the_fence_cannot_break_out_of_it puts the literal closing words inside an excerpt and asserts the forged close does not carry the marker and the real one does.
2. The frame, in this repo's existing words. CLAUDE.md says a skill body "can grant no tool, loosen no approval, and bypass no denylist entry", and zorp-skill states that under every skill body it returns. The memory frame says the same thing, plus what to do with an excerpt that reads like an order: report it, do not act on it. Same trust boundary, same sentence.
3. The lowest-trust channel available. A user message, never system. The one channel the harness speaks in is the one channel recalled text must never occupy.
4. It grants nothing because nothing in the path could. memory.rs builds a string. It touches no policy, registers no tool, and answers no approval. Two tests: assembling_a_block_does_not_change_what_the_policy_permits mirrors zorp-skill's loading_a_skill_does_not_change_what_the_policy_permits, and an_injection_payload_from_an_old_conversation_arrives_as_inert_data runs two real turns and asserts the tool list the model was offered is byte-identical with and without recall.
5. On the page, everything goes through textContent. web/src/memory-note.ts builds DOM nodes and never an HTML string, and its tests include the same payload as an <img onerror>.

Design questions

Automatic on every turn, or only when asked? Only when asked, and per message rather than per session. Automatic injection spends context on messages that do not need it and leaves a user unable to say why the model knew something. Per session is worse than per turn in the same way a mode you leave on is a mode you stop seeing. The model cannot ask for a recall either: there is no tool for it, following the line panel draws about who launches what.

How does a user see what was used, and can they turn it off? A card in the transcript above the answer, listing every recalled line with its conversation title, its date, its position, and whether it was written by them or by the assistant, with a button to open the conversation. The card is sent even when nothing was found, because "memory was on and found nothing" and "memory was off" look identical otherwise, and even when the recall could not run, because the turn goes ahead without memory and the user has to be told. Turning it off is the same box, which is unticked by default and clears itself after every send. Turning it off entirely is not building with the feature.

What is the unit? A message, quoted verbatim, with provenance. Not a whole conversation, whose average vector is about nothing in particular. Not a derived claim, for the reasons above. Index::search_passages returns messages rather than the one-row-per-conversation roll-up search gives the sidebar, because two messages from the same conversation are often exactly what somebody is trying to remember.

Does anything get forgotten or superseded? Nothing here can outlive its source, because nothing here is derived from anything but the source. A changed conversation is re-embedded whole under a new fingerprint; a deleted one is dropped by retain. The index has no independent existence and is rebuildable from the store.

What it cannot do is know that July corrected March. Both excerpts exist and retrieval cannot tell which is right. So every excerpt is dated in the block and on the page, and the frame tells the model in words that the later of two disagreeing excerpts is the more recent thing the user saw and that neither is current unless checked. That is the honest limit rather than a fix.

Open questions, all of which want data from a real corpus before a number gets picked:

• Nothing filters by score. A fixed threshold would be a guess, so all six passages are shown with their similarity and the reader can see how weak a match was.
• Nothing ranks by recency. Boosting a newer excerpt is a heuristic that can be wrong in both directions.
• There is no way to say "forget this one conversation but keep it in the sidebar". Forgetting today means deleting the conversation.

What else changed

• zorp-recall: Index::search_passages, a Conversation header struct carrying updated, and Index::vectors_by_text. The conversations table gains an updated column with a migration on open, so an index built by feat(recall): search your own conversations, on this machine only #83 keeps its vectors instead of being cleared. search and search_passages now share one scan, so they cannot drift on what a score means.
• zorp-agent: SessionRow gains updated. The column has always been there and nothing read it; it is the only "when" the store has, since messages carry a sequence number and no clock.
• The feed reuses the vector it already holds for unchanged text. Rewriting a conversation whole is what keeps an edited message from appearing twice, but whole does not have to mean re-embedded, and without this a fiftieth turn would cost fifty embeddings to record one.

Verification

Rebased onto 0f58b976, not merged. Four conflicts, all of them two branches appending at the same insertion point, all resolved by keeping both sides:

File	Conflict	Resolution
zorp-web/Cargo.toml	#82's research key and my memory key added at the end of [features]	Both kept. The block now carries default, search, recall, research, memory. [dependencies] merged on its own with both zorp-recall and zorp-track.
zorp-web/src/lib.rs	#82's investigate module and my memory module declared on the same line	Both kept, in alphabetical order with the rest.
zorp-web/src/event.rs	#82's InvestigateDone variant and my Memory variant added before Error	Both kept. The }, below the conflict closed only one variant, so the first one got its own.
web/index.html	#82's Zorp mode button and my recall checkbox added inside composer-actions	Both kept. All four control ids appear exactly once.

git diff origin/main on each of those four reports insertions and zero deletions, so nothing from #82 was dropped in the resolution. Cargo.lock did not conflict and needed no regeneration: this change adds no dependencies.

Files belonging to the merges I rebased over report no diff at all: Cargo.lock, zorp-web/src/investigate.rs, web/src/investigate-view.ts, web/test/investigate-view.test.ts, zorp-track/, .github/, zorp-agent/src/sandbox/mod.rs (#85) and zorp-agent/src/investigate/forecast.rs (#86). CLAUDE.md and AGENTS.md show 20 insertions and zero deletions each, so #84's MSRV bullet is main's wording and my diff does not restate the MSRV anywhere.

Toolchain refreshed with rustup update before verifying. rustc 1.98.0, clippy 1.98. MSRV 1.95.

Command	Result
cargo fmt --all --check	exit 0
cargo clippy --workspace --exclude zorp-track --all-targets --locked -- -D warnings	exit 0
cargo clippy -p zorp-web --features memory,research --all-targets --locked --no-deps -- -D warnings	exit 0
cargo build --workspace --locked	exit 0
cargo test --workspace --locked	exit 0
cargo build -p zorp-web --features memory,research --locked	exit 0
cargo test -p zorp-web --features memory,research --locked	exit 0
cargo test -p zorp-web --features memory,recall,search,research --locked	exit 0
cargo test -p zorp-web --features memory,search --locked	exit 0
cargo test -p zorp-web --features memory --locked	exit 0
cargo test -p zorp-web --features recall --locked	exit 0
cargo test -p zorp-web --locked	exit 0
cargo test -p zorp-recall --locked	exit 0
cargo test -p zorp-agent --features research --locked	exit 0
cargo +1.95 check --workspace --locked	exit 0
cargo +1.95 check -p zorp-web --features memory,research --all-targets --locked	exit 0
npm run check (from web/)	exit 0
npm test (from web/)	221 pass, 0 fail
npm run build (from web/)	exit 0

Two things worth stating plainly rather than hiding in an exit code.

The --no-deps on the memory,research clippy line is not a dodge. Turning on research makes zorp-track a path dependency, and clippy then applies -D warnings to it. zorp-track does not pass that gate today: too_many_arguments at prereg.rs:170 and manual_contains at track.rs:344. Both lines exist verbatim on origin/main, both are in a crate this diff does not touch by a single line, and the identical two errors reproduce with --features zorp-web/research alone with no memory involved. That is exactly why the CI gate says --exclude zorp-track. --no-deps lints the package under review and leaves the excluded crate excluded. Not fixed here: zorp-track's lint debt is somebody's decision, not a side effect of this PR.

The macOS CI failure on the previous push was not from this change. zorp-web/tests/auto_approve.rs:55, an empty HTTP body during startup. This diff does not touch that file; the coordinator re-ran the job and it passed. Left alone as a known pre-existing flake.

Feature combinations

zorp-web now has five keys and CI exercises almost none of the pairings. Exercised locally on the rebased tree: default (no features), recall, memory, memory,search, memory,research, and all four together. memory,research is the pairing that did not exist before this rebase and it builds, tests and lints clean.

Nobody exercises any of these in CI. This PR widens the untested cross product by one key rather than closing it. A job that builds the combinations would be the fix and is deliberately not in this change, because adding one is a separate decision about CI time, and #82 has just raised both research jobs from a 45 to a 60 minute cap.

Tests added

zorp-web/tests/memory.rs, 11 integration tests, each driving the whole path with a real loopback embedding server, a real index, a real turn, and a mock model that returns the request body it was sent:

• a_fact_from_an_older_thread_reaches_a_new_one
• memory_is_off_unless_the_turn_asks_for_it
• provenance_survives_retrieval
• a_recalled_assistant_line_is_labelled_as_model_output
• an_injection_payload_from_an_old_conversation_arrives_as_inert_data
• the_recalled_block_is_never_written_back_into_the_conversation_store
• no_local_embedder_means_an_explicit_refusal
• a_proxy_in_the_environment_never_sees_the_conversation
• a_remote_embed_url_refuses_the_recall
• a_finished_turn_feeds_the_memory_on_its_own
• feeding_a_growing_conversation_only_embeds_what_is_new
• plus recall_into_a_turn_is_off_unless_the_feature_is_on, compiled in only without the feature

zorp-web/src/memory.rs, 10 unit tests on the framing: no_passages_means_no_block_rather_than_an_empty_fence, the_frame_states_the_boundary_before_the_first_excerpt, every_excerpt_carries_its_provenance_inline, an_assistant_excerpt_is_marked_as_model_output_not_evidence, an_unknown_role_is_treated_as_model_output, the_quoted_text_is_the_stored_text_and_nothing_else, the_fence_carries_a_nonce_that_changes_every_time, an_excerpt_that_quotes_the_fence_cannot_break_out_of_it, assembling_a_block_does_not_change_what_the_policy_permits, a_date_is_a_date_and_a_missing_one_is_empty.

zorp-recall/tests/index.rs, 5 new: passage_search_answers_messages_not_conversations, a_passage_carries_where_who_and_when, a_model_authored_passage_stays_marked_as_one, passage_search_respects_the_limit, an_index_from_before_dates_existed_opens_and_keeps_its_vectors.

web/test/memory-note.test.ts, 18 new: five on markup never being built, including the injection payload as an <img onerror>; five on attribution and dates; four on the empty and unavailable states; four on refusing a response body that is not the shape it claims to be.

What was not touched

The user's real session store was never opened by anything here. Every test uses a scratch ZORP_STATE_DB and ZORP_RECALL_DB in a temp directory. No release build was run and nothing on port 7777 was disturbed.

https://claude.ai/code/session_01KGPVQ8wUG7h36zashWYCp4