Skip to content

Bind the memory security framework at startup and repair the defects that binding exposed - #1829

Merged
seangalliher merged 11 commits into
mainfrom
fix/bf938-ad607-security-binding-20261118
Nov 18, 2026
Merged

seangalliher merged 11 commits into
mainfrom
fix/bf938-ad607-security-binding-20261118

Conversation

@seangalliher

@seangalliher seangalliher commented Nov 18, 2026 •

Copy link
Copy Markdown
Owner

Refs #1680

What changes

AD-607's memory security framework was never bound, and where it would have run it had three defects. Its three EpisodicMemory setters (set_security_config, set_security_gate, set_security_event_emitter) had no production caller, so the recall anomaly gates, the store-time prompt-injection gate and every AD-607 event were inert, and the supported profile's armed enforce_provenance and enforce_store did nothing. The profile manifest's evidence for enforce_provenance compared the flag with itself. Bound, weighted recall would have raised NameError, the provenance check would have flagged reflections, cross-agent recall and seven labels production writes, and hybrid semantic recall would have returned an episode its own gate dropped. Now:

  • Startup binds the three setters (src/probos/startup/cognitive_services.py) next to AD-610's storage gate, with the runtime's emit_event_fn, unwrapped: a failure to bind fails startup, because an armed control that stays unbound is the defect. Every enforce_* default is unchanged, so by default the gates only report; recall and store results change only when a flag is armed.
  • Weighted recall's gate builds its view from the recall's own cues (query_watch_section, intent_type), and no recall path compares the recalling agent's sovereign id with an episode's trigger_agent (a callsign, captain or an author id).
  • KNOWN_MEMORY_SOURCES holds every source ProbOS writes: AD-607's seven labels, every MemorySource value and action_dispatch, captain_chat, crew_collaboration, group_chat_fanout, multi_agent_chat, qualification_test, wardroom_import. An AST guard (tests/test_bf938_episode_source_guard.py) enumerates every Episode( construction under src/probos and fails on a source outside the set, with premises so that it cannot pass by finding nothing.
  • Hybrid fusion passes each sparse-only episode through the same gate (_fuse_dense_sparse hydrated them through an unfiltered read, so with hybrid recall on, which the supported profile arms, an episode the dense loop dropped came back whenever it matched a keyword).
  • The rule that a direct episode carries a non-empty AD-492 correlation id is retired, by the Captain's decision of 2026-11-18 (option A; BF-938: AD-607's recall anomaly gate is never bound in production, and would raise NameError if it were #1680 comment 4). It had never run, it tested only that the string was non-empty, and most direct producers set none (704 of 1,227 episodes on the Captain's store when counted on 2026-10-27; they cannot be backfilled). Under the supported profile it would have dropped 71% of crew recall. The retirement is temporary: AD-1360 (AD-1360: restore AD-607's provenance rule: a direct memory's correlation id must resolve to an AD-492 chain, judged only for episodes written after a cutoff #1828) restores it as a check that the id leads to a real AD-492 chain, for episodes written after a cutoff. The retired rule's comment, the DECISIONS.md entry and BF-938: AD-607's recall anomaly gate is never bound in production, and would raise NameError if it were #1680 link it.
  • The manifest's evidence for the three AD-607 flags (docs/development/config-profiles.yaml) now names tests that cross the startup binding and a real EpisodicMemory, and the two AD-607 tests that only compared the flag with itself call the store's own anomaly check with a bound config.
  • The in-memory test double takes the three setters (src/probos/cognitive/episodic_mock.py, review round 1): runtime boots that install MockEpisodicMemory (21 boots in 10 test files) would otherwise stop at startup, because the binding is unwrapped. The mock stores each value under the real memory's attribute name and applies none, so it still gates and emits nothing. MemorySecurityGate and StoreSecurityDecision leave the dormant-class baseline (docs/development/dormant-class-baseline.yaml): startup uses both now, as AD-1270: platform maturity — exercised capabilities, robust seams, modular owners, fast gates, truthful docs #1324 asks; MemorySecurityRegistry keeps its row.
  • The Captain-facing description of AD-607 names what it does not check yet (DECISIONS.md, PROGRESS.md, commits 359a781f and 51f20905). The coverage measured after the broad gate is narrower than the contract said: seven public recall methods reach the verdict (recall, recall_with_confidence, recall_with_control, recall_weighted, recall_valid_at, recall_by_anchor, recall_by_anchor_scored, plus the hybrid fusion's hydration); recall_for_agent, recall_for_agent_with_confidence, recall_for_agent_scored, recent_for_agent, recent and recall_by_intent do not. The entry says so, with the call-site count, the effects bullet no longer says that every recall runs the validator, and the gap is BF-984: AD-607's recall gate covers only some of the ways to read an episode: per-agent recall, recency and intent recall return a forged episode when enforce_provenance is armed, and the rebuild command stores without the store gate #1830.
  • The UI job no longer downloads an unused CUDA tarball (.github/workflows/ci.yml, commit f3d8f98d). npm ci in ui/ runs onnxruntime-node@1.21.0's install script (a dependency of @huggingface/transformers@3.8.1), which fetches onnxruntime-linux-x64-gpu-1.21.0.tgz from github.com/microsoft/onnxruntime; that URL now returns 404 (jobs 134737919767 and 134747403132; the rerun failed the same way, and the last green ui-tests runs were on 2026-11-17). Nothing under ui/src references onnxruntime-node, so the install step sets ONNXRUNTIME_NODE_INSTALL_CUDA: skip, which the package's own install script reads before any download. This is a blocker repaired inside this change: an external asset returned 404. It is also the smallest repair of the CUDA-download half of Follow-up: residual findings from the #1722 / #1604 / #1723 / #1724 / #1736 / #1741 wave (reserves BF-954) #1754's item CI-F1, named on 2026-11-08 when the same download failed on timeouts; the Install uv retry half of that item is not touched, and this change is noted on Follow-up: residual findings from the #1722 / #1604 / #1723 / #1724 / #1736 / #1741 wave (reserves BF-954) #1754 after it merges. package.json and the lockfile are untouched.

Four new test files: tests/test_bf938_ad607_recall_gate.py (18 cases, a real episodic store), tests/test_bf938_episode_source_guard.py (10), tests/test_bf938_ad607_binding.py (8, the real startup function) and tests/test_bf938_mock_episodic_binding.py (1, startup on the in-memory mock). The rest of the diff is the contract (prompts/bf-938-ad607-security-gate-binding.md), DECISIONS.md, PROGRESS.md, the regenerated ledger pair and the one workflow step.

Commits

Commit Subject
a64cab3e Add the build prompt for binding the memory security framework
0260ccab Complete the memory security checks on the gated recall paths
48e91896 Bind the memory security config, gate and event emitter at startup
56f0f89c Point the profile manifest at tests that cross the startup binding
e82921a7 Record the memory security binding in the trackers
dae1c0cd Accept the memory security binding on the in-memory episodic mock
d9dfb415 Correct the memory security binding contract's startup-caller record
4eeabacf Restore the contract's gate-list sentence and narrow its correction note
f3d8f98d Skip the unused CUDA provider download in the ui-tests install step
359a781f State which recall paths the memory security checks cover
51f20905 Name the recall methods that run the validator at the default

Review and the broad gate

Four review rounds on the gated candidate, one decision by the Architect at the round-three brake, and one delta review of each of the three commits added after it. Every review was by the Diff Reviewer on gpt-6.1-sol, a different model family from the author, and the model of each was read back from the session log's usage records (including the first dispatched model), not taken from the request.

  • Round 1 (the whole candidate, head e82921a7, 11.4 minutes): not ready, one finding. P1, ordinary reach: startup calls the three AD-607 setters unwrapped, and MockEpisodicMemory, the in-memory test double, has none, so every runtime boot that installs it stopped with an AttributeError (21 boots in 10 test files). The orchestrator reproduced it, and found one more thing the review had not: MemorySecurityGate and StoreSecurityDecision were still listed in the dormant-class baseline, which AD-1270: platform maturity — exercised capabilities, robust seams, modular owners, fast gates, truthful docs #1324 says must leave in the commit that makes them live. Repaired in dae1c0cd: the mock stores the three values under the real names and applies none, a new test boots startup on the mock (it fails on the previous head with the exact AttributeError), the two baseline rows are removed by hand, and the contract and trackers record it.
  • Round 2 (the delta, head dae1c0cd, 30.9 minutes): not ready, one finding. P3, ordinary reach, not a regression: the contract's absence record AV9 listed tests/test_bf103_episode_id_mismatch.py:144 as a direct caller of the startup function. That line is an argument of a DreamAdapter, and the file only inspects the function's signature. Every code, test, baseline and tracker property held. Repaired in d9dfb415 (the contract only).
  • Round 3 (the delta, head d9dfb415, 8.9 minutes): not ready, one finding. P3, ordinary reach, a regression of the round-2 repair: it had reworded a sentence of the contract to say that the second milestone's gate list names every test that names the startup function, and one test names it only in a docstring and is not on the list. The orchestrator had seen that line in its own search and dismissed it. Every other property held (an enumeration of the 3,014 Python files found exactly the callers AV9 lists).
  • The round-three brake. A fourth round needs a fresh read-only decision by the Architect, which it made: continue, with a repair that only reverts. The sentence goes back to its earlier text, and the words that said it had been corrected are deleted from the two notes that said so; no new sentence is written. The decision and its reason are on BF-938: AD-607's recall anomaly gate is never bound in production, and would raise NameError if it were #1680. Repaired in 4eeabacf.
  • Round 4 (the delta, head 4eeabacf, 4.8 minutes): ready, no findings. The review ran its checks: every path but the contract has the same blob as on the previous head (6,008 entries compared), the delta is exactly three hunks (lines 5, 177 and 1666, +3 -3) with the file's CRLF endings intact, the restored sentence is byte-equal to the one on dae1c0cd, and the net change over dae1c0cd is exactly the header, AV9's list and the paragraph of section 16. An enumeration of the 3,014 Python files under src and tests found exactly four direct callers of the startup function (runtime.py:3034 and three test files); both test callers that pass an episodic memory are on the second milestone's list, and so is the one that passes None. It also showed that the one test that only names the function does so in a docstring. The ledger check reports the ledger current. It did not run the test suites or touch live data.
  • The three commits added after the gate (f3d8f98d, the CI step; 359a781f, the coverage wording in the two trackers; 51f20905, the one-sentence repair of a finding on the second). Each was reviewed as a delta. They are new scope, not repairs of a finding of the four rounds, so the Architect ruled that the round-three brake does not count them.
    • f3d8f98d, the CI step (18.6 minutes): not ready on one finding, recorded. Six of its seven properties held by execution: only .github/workflows/ci.yml differs (+5 -0, one hunk); the parsed workflow equals the parent's but for the step's env; the published installer reads the variable (line 173) and skip exits before any download, and a controlled npm ci through the real dependency hook exits 0 with no download call with the variable and reaches the download without it; both job logs show the URL and the 404, and nothing tracked under ui/ but the lockfile references the package; tests/test_ci_shards.py passes (231); package.json and the lockfile are unchanged. The seventh, a check for the same exposure elsewhere, is the finding (P2, ordinary reach, not a regression): the documented Linux/x64 UI installs (README.md:138, docs/getting-started/installation.md:39, ui/public/models/wake-word/README.md:21) do not set the variable and reach the same 404. It predates the commit and lies outside its objective, so the Architect decided to record it on Follow-up: residual findings from the #1817/#1796 (BF-982, BF-965), #1803/#1806 (BF-972, BF-975), #1808 (BF-977), #1804 (BF-973), #1809 (BF-978) and #1807 (BF-976) waves #1820 (item bf-938 F1, P2) and not to widen the commit: the commit had no repair and no second review.
    • 359a781f, the coverage wording (10.7 minutes): not ready on one P2 and two recorded P3s. The P2: the effects bullet (DECISIONS.md:2031) still said that at the default every recall runs the validator; the reviewer ran it against a real store: semantic recall called the validator once and each of six readers (recall_for_agent, recall_for_agent_with_confidence, recall_for_agent_scored, recent_for_agent, recent, recall_by_intent) zero times. The two P3s say what the flags are meant to do without their limit (the docstring at episodic.py:1501, the comments at config_models/crew.py:155-156) and the archive's design paragraph (roadmap-era-5-completed.md:5011) states no current coverage. By the Architect's class rule (a documentation statement that is false about current behaviour is repaired at any severity; one that is true as far as it goes, or states design intent, is recorded with its reach on the issue whose repair completes it) the P2 was repaired and the P3s are recorded on BF-984: AD-607's recall gate covers only some of the ways to read an episode: per-agent recall, recency and intent recall return a forged episode when enforce_provenance is armed, and the rebuild command stores without the store gate #1830.
    • 51f20905, the repair (round 2 of that slice; 10.0 minutes): ready, no findings. Executed on real Chroma and FTS stores, with the premises asserted and the default configuration bound as startup binds it: semantic recall (hybrid off and on), weighted recall and anchor recall each ran the validator, and each excluded reader ran it zero times and emitted no event, so none of them is an empty-result shortcut; the entry as the real CodebaseIndex delivers it (16 lines, 7,699 bytes) holds no universal recall claim (the enumeration discriminates: the base holds the old sentence); the delta is one sentence on one line, every other path unchanged (6,008 entries), CRLF intact; the ledger check reports the ledger current.

Size findings: 0 (no size finding in any round or delta review; no round deleted code).

The broad gate. One run of the canonical wrapper on the frozen head 51f209053b69cb37a5335716b0dfc59cf8d4f5ab (tree bf88af4e4f6ac0c8317b1a6291c7892dfdbb9d92), after the last review and with no change since: scripts/run_test_gate.py --label bf938-1680b --receipt logs/gates/bf938-1680b.receipt.json, 3 workers, on a host with nothing else running. Result: 54184 passed, 184 skipped, 243 warnings, 7 subtests passed in 1:36:15, exit 0. The receipt validates (54,368 tests collected and reported on 3 workers) and binds the committed tree; the live-data manifest taken before and after the run is identical (54,107 files, none added, removed or changed). The count equals the first gate's (bf938-1680 on 4eeabacf3e92: 54184 passed, 184 skipped, in 1:32:53): the three commits since change one workflow step and two tracker files, and no test or code file. origin/main was 9f43697f at the start of the run and at the push. That first gate is superseded by this one.

Evidence

  • Tests first, on the base source (each new file ran before the change it tests):
    • tests/test_bf938_ad607_recall_gate.py: 17 of 18 cases fail. The one that passes is the direct case of the provenance-label test, which pins what the base already does. The contract predicted that the dense case of the hybrid-fusion test passes on the base. It fails there too, because the base drops the legitimate reflection episode as unknown_source.
    • tests/test_bf938_episode_source_guard.py: 8 of 10 fail (its two premise cases pass).
    • tests/test_bf938_ad607_binding.py: 7 of 8 fail (the case that checks that every manifest node exists passes).
    • tests/test_bf938_mock_episodic_binding.py (review round 1's repair): fails on round 1's head with AttributeError: 'MockEpisodicMemory' object has no attribute 'set_security_config', raised at startup/cognitive_services.py:337, and passes on the repair.
  • On the head. The Builder's focused runs per milestone: 1,132, 424, 212 and 33 passed. My consolidated focused run on e82921a7 (39 files, one process, probos imported from the worktree): 1,452 passed. After the repair, a 29-file focused run (the new file, the binding tests, the ten files that boot a runtime on the in-memory mock, the fourteen that name it, and the dormant-class, architecture-fitness and shutdown-grace tests): 1,621 passed, 1 skipped (a Windows symlink skip in test_knowledge_store.py), 0 failed.
  • Mutation campaigns. The contract's 21 rows (section 10), run on an identical-code scratch tree: every row killed, every named killer failed. Repair R1's four rows (each setter's assignment replaced by pass, and set_security_config renamed): all four killed.
  • Checks. check_config_profiles.py --check passes (3 profiles, 55 classified flags). check_dormant_classes.py --check reports 0 baseline rows that are no longer dormant. The one new class it lists, admission/tickets.py:97 TestTicket, is listed the same way on the base and is not touched here. The AD ledger is current.
  • The last two commits of the head that passed review round 4 change the contract prompt only (d9dfb415 and 4eeabacf, repairs of review rounds 2 and 3). The three commits after it (f3d8f98d, 359a781f, 51f20905) change one workflow step, one line each of DECISIONS.md and PROGRESS.md, and one more sentence of DECISIONS.md; no code or test file changed. The focused set over the files that read them (tests/test_ci_shards.py and the six that read DECISIONS.md: 412 tests) passed on the author's scratch tree and on the Builder's reproduction, gen_ad_ledger.py --check reports the ledger current, and the broad gate ran on the final head (below).

What this pull request does not do

seangalliher and others added 8 commits November 17, 2026 23:06
The Architect's contract for #1680: bind the three memory security setters at
startup, build the weighted recall gate's view from its own cues, admit every
episode source production writes (an AST guard pins the set), retire the
correlation-id rule temporarily as the Captain decided, and gate the episodes
that hybrid fusion hydrates. Four milestones and 21 mutation rows.

Refs #1680.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Three gaps kept the AD-607 verdicts from doing what the profiles claim:

- Semantic recall with hybrid retrieval gated only the dense loop. An episode
  that sparse retrieval found was rehydrated through an unfiltered read, so,
  once armed, a forged source returned. Hydration now applies the same verdict.
- Weighted recall built its anchor view from names that were never defined, so
  a bound config raised instead of completing. The view now uses the recall's
  own cues (the watch section and the intent type), and no recall path puts the
  recalling agent's id where the episode's driver belongs.
- The known-source set lacked values that production writes. It now holds the
  seven original labels, every MemorySource value and the seven production
  labels, and an AST guard test enumerates every Episode construction so a new
  label cannot slip past it.

The rule that a direct episode must carry a correlation id is retired, as the
Captain decided on 2026-11-18 (option A): 704 stored direct episodes carried
none when counted on 2026-10-27, and they cannot be backfilled. The retirement
is temporary; the follow-up that restores it as a check that the id leads to a
real chain, after a cutoff, is #1828.

New tests: the recall gate (18 cases, real episodic store) and the source guard
(10 cases). One existing test that pinned the retired rule now pins the new
behaviour.

Refs #1680.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
None of the three AD-607 setters had a production caller, so on a started
vessel the framework held no config, no gate and no emitter, and every verdict
was skipped. Startup now binds them on the episodic memory it starts, right
after the utility gate's binding and without a try block: a binding that fails
should stop startup, not leave an unguarded store.

With every enforce flag false the recall and store results are unchanged; the
verdicts only emit their observational events. Under the supported profile the
two armed flags now act through this binding: with enforce_provenance a forged
source is left out of weighted recall, and with enforce_store an episode the
injection check rejects is skipped.

New test file: the binding through the real startup function (8 cases),
including that each setter has a production caller and that every test the
manifest rows name exists.

Refs #1680.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The evidence rows for enforce_store and enforce_provenance named tests that
called the verdict directly, which passes whether or not startup binds
anything, and the row for enforce_recall said that no test exercised it. The
rows now name the binding and recall-gate tests, which run through a real
episodic store. The two AD-607 tests that only compared the flag with itself
call the store's own anomaly check with a bound config.

Refs #1680.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
PROGRESS.md and DECISIONS.md record the binding, the source set and its guard,
the hybrid hydration gate and the retirement of the correlation-id rule as
temporary (follow-up #1828). The AD ledger pair is regenerated.

Refs #1680.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Review of the startup binding found that the unwrapped setters stopped every
runtime boot that installs the in-memory test double: it had none of the three,
so 21 boots in 10 test files raised AttributeError. The mock now stores the
config, gate and emitter under the real memory's attribute names and applies
none of them, so it still gates and emits nothing. The binding stays unwrapped.

The MemorySecurityGate and StoreSecurityDecision rows leave the dormant-class
baseline, since startup now uses both; MemorySecurityRegistry keeps its row.
The contract's startup-caller check is corrected, and the trackers record the
repair.

New test file: startup on the mock (1 case).

Refs #1680.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Review of the repair found that the contract's record of which tests call the
startup function listed a test file that only inspects its signature: the cited
line is an argument of a different constructor. The record now lists the files
that call it, with their lines and the kind of memory each passes, and the gate
list's explanation says which files call it and which only name it.

Only the contract prompt changes; no code, test, baseline or tracker line moves.

Refs #1680.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Review of the previous correction found that it had widened the sentence on
which tests the second milestone's gate lists to every test that names the
startup function, and one test names it only in a docstring. The sentence is
restored to its earlier wording, which holds now that the startup-caller record
is corrected. The header line and the paragraph that recorded the correction no
longer say that this sentence was corrected.

Only the contract prompt changes.

Refs #1680.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
seangalliher and others added 3 commits November 18, 2026 03:55
The install script of onnxruntime-node, a dependency of
@huggingface/transformers, downloads a CUDA provider tarball from a GitHub
release that now answers 404, so the install stopped the job before any UI
test ran. The UI loads onnxruntime-web in the browser and nothing under ui/src
uses the Node binding, so the step sets the package's own variable to skip that
download.

Only that job's install step changes.

Refs #1680.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The Captain-facing entry said that each recalled memory is checked. The checks
run on semantic, weighted and anchor recall and on hybrid fusion's hydration.
Per-agent recall, the recency reads and the intent recall are not checked, and
the agent's own prompt recall uses them for the basic tier and when its
weighted recall returns nothing. The decisions entry and the progress limits
now say so and name the issue that records it. The progress entry also names
the rebuild command as it is, probos rebuild-episodic.

Only the two trackers change.

Refs #1680.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The decisions entry's effects bullet said that at the default every recall
runs the validator. The previous commit added the sentence that lists the
readers that do not (#1830), so the two lines disagreed. A review of that
commit ran it against a real store: semantic recall called the validator once
and each of the six readers called it zero times.

The sentence now names the three recall classes that the coverage bullet
already names (semantic, weighted and anchor recall) and points to that
bullet's list of readers that are not yet covered. It adds no coverage claim
of its own.

Only DECISIONS.md changes, one sentence on one line.

Refs #1680.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant