SPE-5054 [S3]: swarm memory through the substrate's governed memory.write (stacked on #5)

[SomeRandmGuyy]

What

ADR 0001 phase S3, swarm side (SPE-5054). Stacked on #5, so its base is that branch; GitHub retargets it to main when #5 merges. Pairs with the substrate PR agent-substrate#12.

The swarm has no memory module today (its vector memory is design only in 04-integration-plan.md), so this is a thin client onto the substrate's governed memory_write, not a migration:

• swarm/memory.py: memory_write maps swarm decision / retro / pattern to substrate decision / fact / fact, and scopes graph / repo / agent / global to graph:<id> / repo:<owner/name> / agent:<surface> / global. The idempotency key is derived from correlation, task, attempt, kind, scope and text and is reused on every retry of one write.
• Outcomes: denied raises E-POLICY (fail-closed, never retried); quarantined and review.required are returned as-is and not retried; version.* conflicts re-read the version and re-submit with expected_version; an accepted-but-unverified write retries with the same key; ok is true only for accepted and verified_persisted; an unreachable substrate is unavailable, never a pretend write.
• memory_query (memory_search) and run_start_context (/brief) are fail-open reads. scripts/swarm_run.py prepends a ## Substrate memory section to the assignment prompt when the substrate answers, and is byte-for-byte unchanged otherwise.
• swarm/substrate_client.py gains mcp_call_json because memory_search answers a JSON array.
• Criterion 7: an AST scan over swarm/, scripts/ and hooks/ proves no swarm-side store feeds an agent's context. The substrate's store-lock.test.ts cannot prove that.

A swarm decision can only be written at graph or repo scope (the gate never grants it at agent: scope), and a global write is a proposal until a reviewer token releases it. Both are documented in docs/substrate-memory.md.

Verification

• pytest: whole suite passes (this branch adds tests/test_swarm_memory.py); build_agents.py --check, build_trae_agents.py --check and bun test tests/ts pass.
• ruff: nothing new; the two E401 findings in tests/test_uplift_harness.py are on origin/main.

Not in this PR: agents calling this through tooling (S4), gate verdicts as memory (S5).

Refs SPE-5054
Refs SPE-7629

Confidence Score: 5/5

The reviewed changes add no new issue that would block merging.

What we checked:

• Refused writes never look saved: A policy denied reply raises E-POLICY. WriteOutcome.ok also requires both accepted and verified_persisted.
• Existing callers keep object replies: mcp_call_json accepts lists, but mcp_call still returns only objects or None.
• Failed reads leave dispatch working: run_start_context returns an empty string for failed replies. Prompt construction also catches exceptions and keeps an empty memory section.

Summary

Adds a thin Substrate memory client and a run-start brief in task prompts.

• Writes require accepted and verified_persisted; reads fail open.
• Adds array replies while preserving object-only mcp_call callers.
• Adds memory tests and checks for a second memory store.
• No new actionable findings.

Greptile automatically discovered a related ticket that helped explain the purpose of this PR: route swarm memory through Substrate’s governed write API.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    Caller[Caller] --> Adapter[memory_write]
    Adapter --> Write[POST /memory]
    Write --> Result{Write result}
    Result -->|accepted and verified| Done[ok = true]
    Result -->|policy denied| Refused[E-POLICY]
    Result -->|version conflict| Read[memory_search]
    Read -->|version found| Adapter
    Result -->|other result| Pending[ok = false]
    Prompt[assignment_prompt] --> Brief[Cached run-start /brief]
    Brief -->|text returned| Prefix[Prepend Substrate memory]
    Brief -->|off or failed| Original[Keep original prompt]

Loading

_{Reviews (4) · Last reviewed commit: "Merge remote-tracking branch 'origin/mai..." · Reviewed by Greptile}

[greptile-apps]

Comments Outside Diff

These findings could not be posted inline.

•  Brief skips configured token swarm/memory.py:285 ▶

  With SUBSTRATE_TOKEN_SWARM_A01_ORCH set and SUBSTRATE_TOKEN unset, run_start_context sends /brief without an Authorization header. lookup_graph_id uses the configured A01 token, but this call does not. An auth refusal becomes '' and is cached for the whole run, so no task gets memory context. Pass surface=ORCH_SURFACE to rest_post.

      got = substrate_client.rest_post("/brief", {"repo": repo, "graph_id": graph_id, "surface": ORCH_SURFACE}, env,
                                       surface=ORCH_SURFACE)
  

•  Dry runs contact Substrate scripts/swarm_run.py:118 ▶

  With SUBSTRATE_URL enabled, swarm_run --dry-run now contacts live Substrate. execute_one builds assignment_prompt before checking args.dry_run, so this prefix triggers graph lookup and /brief during a preview. That breaks the documented promise that dry runs make no Substrate requests and makes preview prompts depend on live replies. Pass the dry-run flag into prompt construction and skip this fetch.

[greptile-apps]

Comments Outside Diff

These findings could not be posted inline.

•  Brief skips configured token swarm/memory.py:285 ▶

  With SUBSTRATE_TOKEN_SWARM_A01_ORCH set and SUBSTRATE_TOKEN unset, run_start_context sends /brief without an Authorization header. lookup_graph_id uses the configured A01 token, but this call does not. An auth refusal becomes '' and is cached for the whole run, so no task gets memory context. Pass surface=ORCH_SURFACE to rest_post.

      got = substrate_client.rest_post("/brief", {"repo": repo, "graph_id": graph_id, "surface": ORCH_SURFACE}, env,
                                       surface=ORCH_SURFACE)
  

•  Dry runs contact Substrate scripts/swarm_run.py:118 ▶

  With SUBSTRATE_URL enabled, swarm_run --dry-run now contacts live Substrate. execute_one builds assignment_prompt before checking args.dry_run, so this prefix triggers graph lookup and /brief during a preview. That breaks the documented promise that dry runs make no Substrate requests and makes preview prompts depend on live replies. Pass the dry-run flag into prompt construction and skip this fetch.

Dismissed because a newer commit was pushed; Greptile will re-review the current head.