feat: expire stale candidates and stop charging for a discarded field

[Barsoomx]

Two small, independently-safe changes carved out of a larger attempt that did not survive review (see "What was dropped").

Stop paying for a field we throw away

parse_extraction_output rejected any extraction output whose top-level keys were not exactly {memories, no_signal_observation_ids} — and then, ~18 lines later, recomputed no_signal_observation_ids from the chunk manifest, discarding whatever the model sent:

no_signal = tuple(sorted(chunk_observation_ids - supporting_union))

So omitting that key bought a provider_output_malformed → PROVIDER_TRANSIENT → paid retry, for a value that never reaches anything. Unknown keys are still rejected; a missing memories is still rejected.

One existing test used exactly this omission as its definition of "malformed", so it now uses a genuinely missing key instead — noted because it is a behaviour change, not a test tweak.

Candidate review TTL

candidate_ttl.py was a no-op: it ran the reconciler and returned rejected=0 unconditionally. Nothing ever expired, which is why 5,836 candidates have accumulated in proposed.

It now expires stale proposed candidates with an audit trail (MemoryAutoRejected, reason review_ttl_expired, TTL days, timestamps), skipping anything with an unresolved conflict or active decision work. Rows are preserved — PROPOSED → REJECTED, never deleted.

Two safety properties, both tested:

• Preview by default. The celery task under this name used to be a harmless reconciler; an unchanged caller must not suddenly mass-reject. execute() with no arguments previews.
• ENGRAM_CANDIDATE_TTL_DRY_RUN is a kill switch, not a default. When set, the sweep can only preview, whatever the caller passes. An operator can freeze it without finding every call site.

Operator path: engram_expire_stale_candidates previews and prints what it would expire; --apply performs it.

What was dropped, and why

A sibling attempt precomputed the judge's feasible verdict set to avoid paying for impossible decisions. Two reviewers found it made things worse, and I verified both:

• judge_policy_denied was reclassified to INVALID_INPUT, and work_execution.py:883 makes INVALID_INPUT terminal on the first failure, bypassing the streak. An honest answer would have killed the candidate permanently on attempt 1.
• In ~20% of the policy matrix every admissible verdict asserts the candidate is identical to some target, and merge_evidence routes to _execute_candidate_revision — it writes a new version of the target memory using the candidate's text. Handing the model a menu of only-identity options and ordering it to copy one verbatim would coerce a false merge that overwrites stored memories.

Both are data-integrity failures, not cost bugs. The underlying fix is to invert the contract — the model answers only per-target relations and code derives the outcome — which is a separate, properly-designed slice.

Verification

docker compose -p engram-curation run --rm app pytest -q engram → 3514 passed. ruff check engram → clean.

🤖 Generated with Claude Code

[Barsoomx]

✅ Snyk checks have passed. No issues have been found so far.

Status	Scan Engine	Critical	High	Medium	Low	Total (0)
✅	Open Source Security	0	0	0	0	0 issues

💻 Catch issues earlier using the plugins for VS Code, JetBrains IDEs, Visual Studio, and Eclipse.

[codecov-commenter]

Bundle Report

Bundle size has no change ✅

Affected Assets, Files, and Routes:

view changes for bundle: engram-frontend-client-array-push

Assets Changed:

Asset Name	Size Change	Total Size	Change (%)
server/app/(admin)/model-*.js	105 bytes	19.05kB	0.55%
server/app/(admin)/model-*.js	-105 bytes	18.94kB	-0.55%
static/JYpjgffo5L_O62WBvFJfh/_buildManifest.js (New)	1.51kB	1.51kB	100.0% 🚀
static/JYpjgffo5L_O62WBvFJfh/_ssgManifest.js (New)	77 bytes	77 bytes	100.0% 🚀
static/p1hkOMZIz9PYHSG3tG46y/_buildManifest.js (Deleted)	-1.51kB	0 bytes	-100.0% 🗑️
static/p1hkOMZIz9PYHSG3tG46y/_ssgManifest.js (Deleted)	-77 bytes	0 bytes	-100.0% 🗑️

[codecov-commenter]

Codecov Report

❌ Patch coverage is 92.23301% with 8 lines in your changes missing coverage. Please review.
✅ Project coverage is 91.03%. Comparing base (82f2382) to head (eaa7c30).
✅ All tests successful. No failed tests found.

Files with missing lines	Patch %	Lines
...agement/commands/engram_expire_stale_candidates.py	71.42%	8 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master     #301      +/-   ##
==========================================
- Coverage   91.03%   91.03%   -0.01%     
==========================================
  Files         221      222       +1     
  Lines       23099    23195      +96     
==========================================
+ Hits        21029    21116      +87     
- Misses       2070     2079       +9     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.