Improvement controller lane 3: control journal, fenced dispatch, and the valor-improve CLI - #3315
Conversation
Review: Changes RequestedThree blockers, all verified by reading the code and reproducing the behavior in the review worktree (detached at Mode: Rubric
Pre-Verdict Checklist
Blockers
Tech Debt
Nits
Miscellaneous
Acknowledged Deferrals (verified)
Verification Results
Screenshots
|
Blockers: - journal.transition now seeds the head's `state` field (HSETNX) from the case's own ImprovementCase.state on first write; projection.apply/replay refuse to clobber the projection with an empty state as defense in depth. Previously read_head always returned state="" and apply/replay dropped every case from OPEN_CASE_STATES. - cmd_propose now writes the payload through VerifyingArtifactStore before taking the lease, journals the reference as `artifact_ref` alongside `payload_digest`, keeps it on the refusal evidence row, and export.py populates artifacts.json from journaled references instead of `[]`. - cmd_doctor's per-case read_head/list_intents calls now run inside the same guard as the ORM query, so a control-namespace outage reports "namespace unreachable" with exit 2 instead of a traceback; added the end-to-end break-glass drill test the plan's Success Criterion 2 named. Tech debt: - --action-type flows through transition into the journal entry and the admitted intent (previously always defaulted to "investigate"). - mark_reconciliation_required's slot release and `reason` write now happen inside the same CAS script as the state move, not a second unconditional HDEL after it returns. - dead_letter_exhausted is now the one caller writing a DeadLetter for an exhausted intent, on every branch (bound row or not), always replayable=False. - scheduler_adapter resolves a real working_dir, threads request_digest/ charter_digest into admit(), soft-checks the pinned charter, and the dispatch message names /improve-research with a brief_ref. - Dispatch tests now assert publish-exactly-once, the no-live-worker path never publishes, extra_context_overrides' exact key set, and one case's failure never stops the tick (new isolation test). - export/import round-trip unit-2 window/reservation hashes and the namespace pause hash (previously exported but never restored). - sweep_unsettled_reservations uses the injected clock, not wall time. Nits: deleted dead intent_scan_pattern; dropped the unreachable event ~= "ns_resumed" clause; corrected the shared-prelude and _LUA_MOVE_INTENT ARGV comments; reworded the __init__.py docstring so it no longer trips its own anti-criterion grep; vault_write's default runner sets OP_CACHE=false explicitly; control.html panel gets a heading; struck the unimplemented --acknowledge-unknown claim from the plan's Risk 5. Ticks plan Success Criterion 2 (fault-injection tests, all four races). Verified: targeted suite for every touched file (139 passed), integration suite (5 passed, 1 skipped, matches the review's own baseline), full tests/unit/ (16531 passed, 9 pre-existing failures unrelated to this diff -- none touch a file this commit changes).
77512c4 to
e8dcaf8
Compare
Blockers: - journal.transition now seeds the head's `state` field (HSETNX) from the case's own ImprovementCase.state on first write; projection.apply/replay refuse to clobber the projection with an empty state as defense in depth. Previously read_head always returned state="" and apply/replay dropped every case from OPEN_CASE_STATES. - cmd_propose now writes the payload through VerifyingArtifactStore before taking the lease, journals the reference as `artifact_ref` alongside `payload_digest`, keeps it on the refusal evidence row, and export.py populates artifacts.json from journaled references instead of `[]`. - cmd_doctor's per-case read_head/list_intents calls now run inside the same guard as the ORM query, so a control-namespace outage reports "namespace unreachable" with exit 2 instead of a traceback; added the end-to-end break-glass drill test the plan's Success Criterion 2 named. Tech debt: - --action-type flows through transition into the journal entry and the admitted intent (previously always defaulted to "investigate"). - mark_reconciliation_required's slot release and `reason` write now happen inside the same CAS script as the state move, not a second unconditional HDEL after it returns. - dead_letter_exhausted is now the one caller writing a DeadLetter for an exhausted intent, on every branch (bound row or not), always replayable=False. - scheduler_adapter resolves a real working_dir, threads request_digest/ charter_digest into admit(), soft-checks the pinned charter, and the dispatch message names /improve-research with a brief_ref. - Dispatch tests now assert publish-exactly-once, the no-live-worker path never publishes, extra_context_overrides' exact key set, and one case's failure never stops the tick (new isolation test). - export/import round-trip unit-2 window/reservation hashes and the namespace pause hash (previously exported but never restored). - sweep_unsettled_reservations uses the injected clock, not wall time. Nits: deleted dead intent_scan_pattern; dropped the unreachable event ~= "ns_resumed" clause; corrected the shared-prelude and _LUA_MOVE_INTENT ARGV comments; reworded the __init__.py docstring so it no longer trips its own anti-criterion grep; vault_write's default runner sets OP_CACHE=false explicitly; control.html panel gets a heading; struck the unimplemented --acknowledge-unknown claim from the plan's Risk 5. Ticks plan Success Criterion 2 (fault-injection tests, all four races). Verified: targeted suite for every touched file (139 passed), integration suite (5 passed, 1 skipped, matches the review's own baseline), full tests/unit/ (16531 passed, 9 pre-existing failures unrelated to this diff -- none touch a file this commit changes).
Review (Judge code-quality): Changes RequestedIndependent judge, dispatched as its own subagent with no view of the Round-1 disposition (located by symbol at
|
Review (Judge risk): Changes RequestedIndependent judge, dispatched as its own subagent with no view of the Round-1 disposition (located by symbol at
|
Review: Changes RequestedMode: Verdict: CHANGES REQUESTED. Round 2 at head Parent-reproduced gates (Hard Rule 10): Blockers
Tech Debt
Nits
Miscellaneous
Acknowledged Deferrals (verified)
Review Delta (vs prior review on HEAD bd3e49e)
Verification Results
Screenshots
|
Review: Changes RequestedIdempotent: prior review on HEAD Mode: inherited from the round-2 Idempotency evidence
Carried-forward verdict: CHANGES REQUESTED — 2 blockers, 7 tech debt, 9 nits, exactly as enumerated in the round-2 aggregate review (comment The two blockers, in one line each for routing purposes:
Because blocker 2 is not patchable, the next round should pair the Blockers
Tech Debt
Nits
Miscellaneous
Acknowledged Deferrals (verified)
Review Delta (vs prior review on HEAD e8dcaf8)
Verification Results
Screenshots
|
…head-state contract, import hardening (Refs #3215) Closes the round-2 blocker (B2 artifact path now asserted on the accepted, refused, and export paths) and the seven tech-debt items: propose tests write to a tmp content root, projection.apply runs after every accepted transition, a state_changed journal event is the writer of head state with re-seed on empty, import refuses foreign keys and force replaces history, budget lists unknown-metered receipts, dead-letter comment and docs corrected. Eight nits folded in: settle is one Lua script, ARTIFACT_WRITE_FAILED reason code, brief_ref carries the loadable artifact ref, review-history comments rewritten as invariants. Addresses "Activation publishes publish_session_notify(session) exactly once per activation".
valorengels
left a comment
There was a problem hiding this comment.
Review: Changes Requested
Mode: sequential lenses (Agent tool unavailable: not in tool list). This stage runs inside a supervisor-dispatched Agent and the Agent tool is absent from this context's tool list, so the declared two-judge roster (code-quality, risk) could not be dispatched. Both lenses were applied by one reviewer in sequence. Recorded as a quorum shortfall (judges_run=1, expected_judges=2), not as agreement between judges — per the roster contract in docs/sdlc/do-pr-review.md and #3198.
Head judged: 93def1ecbc715bc74318c90df6e4f57816153aca (resolved via tools/pr_head_resolver.py, not a bare gh read). Base main b13dc8ad3. Preflight: state=OPEN, mergeable=MERGEABLE, mergeStateStatus=CLEAN — no short-circuit.
Round 3 summary. All 18 round-2 findings are closed and independently mutation-verified. The round-2 visual-proof blocker is discharged this round — BYOB was live, the dashboard was started from this worktree, and the control panel was captured in both its empty and its populated branch. It is no longer a carry-forward obstacle.
One new blocker surfaced, and it surfaced precisely because this machine has what the two prior rounds lacked: a working non-interactive op service account. tests/integration/test_vault_write_integration.py skipped in rounds 1 and 2 ("op auth absent by design"), so the vault writer's single contract with the external tool had never actually been exercised. Run here, it fails — and the defect is that the writer cannot create an item at all.
Blockers
- File:
tools/vault_write.py:51(verified: read the file, reproduced the failure and the fix against the realopCLI)
Code:category: str = "API Credential",
Issue: The defaultcategoryis 1Password's display name, butop item create --templaterequires the template's enum form.op item template get "API Credential"returns"category": "API_CREDENTIAL". With the shipped default,oprejects every call:
[ERROR] "API Credential" is an unknown item type: "API Credential" isn't a recognized item type.
sowrite_credential— described in the PR as "the one sanctionedop item createpath" and the whole of plan Task 9 — returnsstate="refused"on every invocation of its default (and only production) path. Reproduced both directions on this machine: the shipped default refuses;category="API_CREDENTIAL"returnsstate="created"and the item was created inm-valorand deleted again. Integration result at this head:tests/integration/test_vault_write_integration.py1 failed (assert 'refused' == 'created'), not "1 skipped".
Nothing caught this because every unit test intests/unit/test_vault_write.pyinjects a fakerunner, so no test ever compares the category againstop's real vocabulary; the integration test is the only real-boundary check and it had never run.
Severity: blocker
Fix: Default to the enum form (category: str = "API_CREDENTIAL"), and pin it with a unit test asserting the template dict'scategoryis the enum spelling so a display-name regression goes red without needingop. Confirm the integration test passes on a machine with thevalor-localservice account before calling Task 9 done.
Tech Debt
-
File:
tools/improvement_resources.py:198-201(verified: read the file)
Code:if _VAULT_WRITER.exists(): return _entry("verified", "the sanctioned vault writer is present")
Issue: The charter §8 resource probe reportsverifiedon file existence alone. With the blocker above live,doctor/probereports the vault-write capability as verified while every write refuses — the exact "counts a capability the system does not have" failure the improvement charter's own framing warns against. A probe that cannot distinguish "present" from "works" is the surface that let the blocker stay invisible for two rounds.
Severity: tech_debt
Fix: Either downgrade the wording topresent(the honest claim for an existence check), or make the probe assert the template category againstop item template getwhenopis authenticated and returnunknownotherwise. -
File:
tests/integration/test_vault_write_integration.py:33-38(verified: read the file, ran it)
Code:pytestmark = pytest.mark.skipif(not _op_available(), reason="op CLI not authenticated non-interactively ...")
Issue: The guard is correct, but nothing in the suite or the PR distinguishes "this test was skipped" from "this test would have failed". The PR's Testing section reports the skip as an expected, benign outcome, which is how a hard failure in the one real-boundary test travelled two review rounds as a green line. This is theop-dependent counterpart of Hard Rule 10: a skip is not evidence.
Severity: tech_debt
Fix: Add a unit-level assertion on the template'scategoryvalue (noopneeded) so the contract is pinned unconditionally, and state in the PR body when the integration row was skipped rather than passed.
Nits
-
File:
tools/improvement.py:20-21(verified: grepped every call site)
Code:def _project_key(args) -> str: return getattr(args, "project_key", None) or PROJECT_KEY
Issue: Dead code. No caller anywhere intools/ortests/, and no--project-keyargument is registered on any subparser — every command hardcodes the module-levelPROJECT_KEY. CLAUDE.md principle 1 (no legacy code tolerance).
Severity: nit
Fix: Delete it, or register the flag and route the commands through it. -
File:
ui/templates/improvement/control.html:9(verified: read the template, compared siblings, confirmed in the captured screenshot)
Code:<h4>Control</h4>
Issue: The panel title is the same heading level as its own five subsections (Lane slots,Intents by state,Paused heads,Reconciliation required,Unit 2: paid inference), so in the rendered page "Control" reads as a peer of "Lane slots" rather than their parent. The sibling panels (goals.html,intervention_burden.html) carry no panel-title heading at all — their first<h4>is already a section heading. Visible in the screenshot below.
Severity: nit
Fix: Drop the title (matching the siblings) or promote it to<h3>so the hierarchy reads. -
File:
tools/improvement_control/export.py:151(verified: read the file)
Code:r.expire(key, KEY_EXPIRY_SECONDS)immediately afterr.hset(keys.assert_control_key(key), mapping=mapping)
Issue: The line above routes through the package's stated guard idiom; this one uses the rawkey. Functionally identical here (the prefix was validated earlier), but the inconsistency invites a future edit that drops the guard entirely.
Severity: nit
Fix:r.expire(keys.assert_control_key(key), KEY_EXPIRY_SECONDS).
Miscellaneous
- None
Acknowledged Deferrals (verified)
--acknowledge-unknowngate — plan Risk 5 reframe as accepted residual scope; thebudgetunknown-receipt block is now its real operator surface (round-2 TD6 closed).dead_letter_stageisimprove_intent— explicit deviation, sound;dead_letter_exhaustedis the one writer on every branch.- Exact-match
ImprovementCase.query.filter()with per-state loops — consistent across adapter, recovery, doctor, dashboard. - Explicit
from_statein_move()— sound; the contradicting comment was corrected this round. - 6 malformed Verification rows — re-verified by hand at this head (values below).
- Unit-2 reserve/release unfenced by the case lease — deliberate scope boundary.
- #3220 hand-off — verified OPEN (
Session execution lease: fence worker ownership with a renewable Redis lease).
Review Delta (vs prior review on HEAD e8dcaf8)
- Resolved (all 18 round-2 findings):
- B1
artifact_refround-trip — closed and mutation-verified. Blankingartifact_refafterstore.savenow goes red on three tests (test_propose_break_glass_without_agent_session_id,test_propose_refused_by_intent_state_keeps_the_payload_as_evidence,test_export_artifacts_index_carries_the_proposal_reference); previously green. File restored, tree clean. - B2 visual-proof gate — discharged this round (screenshots below).
- TD1 propose tests write to a tmp content root (
POPOTO_IMPROVEMENT_CONTENT_PATHset in both the unit and the integration fixture). - TD2
projection.applynow has four production callers (tools/improvement.py:188,240,291,337via_project). - TD3 all seven previously-unreached adapter/recovery/projection fix paths are now asserted (
working_dirabsolute+existing,message_textprefix andbrief_refsuffix,charter_digest_staleskip,session_gone_or_terminalreason,improve_intentDeadLetter, empty-head-state guard). - TD4 import hardening —
FOREIGN_KEYrefusal onproject_keymismatch and on any unit-2 key outside the project prefix,assert_control_keyon restore, TTL re-applied, and--forcedeletes journal/intents/intent hashes before restoring instead of appending. - TD5 head
statecontract —state_changedis a real writer (journal.set_state), empty heads re-seed instead of pinning to""forever, and the contract is documented atjournal.py:43-51anddocs/features/improvement-controller.md:318-327. - TD6
budgetnow listsmetering="unknown"receipts viapaid_inference_meter.unknown_receipts, asserted inTestBudget. - TD7 dead-letter comment and both doc rows corrected.
- N1-N9 all closed:
from_statecomment corrected;git grep "#3315 review" -- tools/is now 0 (was 15);replay'safterno longer misreports on the empty-state branch;ARTIFACT_WRITE_FAILEDreason code added;brief_refcarries the loadable$CF:ref and the skill documents it;settleis one Lua script; publish ordering asserted as(session_id, status); PR body count refreshed to 726 and reproduced.
- B1
- New: the
vault_writecategory blocker and its two tech-debt companions; three nits. - Unchanged: none.
Verification Results (Hard Rule 10 — reproduced in this environment)
ruff check .— clean (exit 0).ruff format --check .— clean, 1597 files.- Plan
## Verificationtable: 28 rows parsed, 22 PASS, 6 parser-FAIL rows all verified passing by hand at this head:- Status trio —
session_lifecycle.py=3 (≥2),ui/data/sdlc.py=1 (≥1). Pass; the expectation is prose, not machine-comparable. - No second general lease — output
0, #3220 OPEN. Pass. - Session carries no generation —
0matches;grep -cexits 1 on zero matches. result_digestone writer — runner reported 2; by handgrep -rl 'HSET.*result_digest' tools/improvement_control/ | wc -l= 1 (journal.py:107is the soleHSET;intents.pyonlyHGETs). The runner's extra match is a stale__pycache__.pyccarrying the embedded Lua source, not a second writer. Pass.- Two counters, two owners — first=7 (>0), second=0. Pass.
- Judge receipt pinned —
record_receipt(attools/cross_vendor_judge.py:275-276carriesproject_key="valor"across a line break, so the single-line grep cannot see it. Pass. - Investigation vocabulary —
grep -cE(alternation needs-E) returns 2. Pass.
- Status trio —
- Lane tests, the plan's own 21-file row: 726 passed in 51.88s — the PR's claim reproduces exactly.
- Integration:
test_improvement_control_cli.py5 passed. (It first skipped all 5 because this worktree's venv had novalor-improveconsole script; afteruv pip install -e . --no-depsthe binary materialized and the tests ran. That is a local venv-sync condition, not a code defect.)test_vault_write_integration.py1 failed — the blocker above. - Documentation gate: all 7 files named in the plan's
## Documentationsection exist and are in the diff. - Plan section compliance:
## Test Impact,## No-Gos,## Update System,## Agent Integration,## Documentation,## Verificationall present. - Bridge/worker impact:
models/session_lifecycle.pyandagent/session_health.pyare imported by both bridge and worker, and two new reflections register on/update. A worker restart on every machine is required after deploy.
Screenshots
Visual proof gate: PASSED (round-2 blocker 2 discharged). BYOB MCP was live; the dashboard was started from this worktree (UI_PORT=8521/8522 python -m ui.app) and both rendering branches of the changed template were captured, then the servers were stopped and the scratch data removed through the ORM.
- Empty branch, against live data —
#improvement-controlrenders the<h4>Control</h4>heading and "Nothing yet, written by lane 3 when a case is admitted." - Populated branch, against an isolated test Redis db (db 14, asserted before any write; never production) seeded with two open cases and four intents —
generated_images/pr-3315/01_control_panel_populated.jpg. Renders: Lane slots: 1 slot(s) in use; Intents by state table withadmitted/admitted/running/reconciliation_requiredrows carrying case id, action id and action type; Paused heads: No paused cases; Reconciliation required with the correctvalor-improve resume --case <id> --forcebreak-glass line; Unit 2: paid inference — Window 2026-09-14: reserved $0.00, settled $0.00. All five subsections render correctly. The only visual issue is the heading-level nit above.
Pre-Verdict Checklist
- 1. All plan acceptance/success criteria validated against diff — FAIL — Task 9's vault writer cannot create an item on its default path.
- 2. No-Gos from plan — none violated — PASS — scope guard clean on the six No-Go paths.
- 3. New
except Exceptionblocks — each has logger/raise/swallow-ok — PASS —vault_write.py:88,:114and theui/data/improvement.pyreader all log; swallow is intentional and documented. - 4. New integration tests — exercise serialization boundary (not in-memory only) — FAIL — the one external-boundary integration test fails at this head.
- 5. Plan internal consistency — spike findings match task steps — PASS.
- 6. No hardcoded secrets or debug artifacts — PASS —
write_credentialnever puts the value in argv; it goes through a 0600 temp file that is unlinked infinally, and only asha256:fingerprint is returned. - 7. New public APIs — docstrings present — PASS.
- 8. Breaking changes — migration path documented — N/A — no schema migration;
admittedis an additive status. - 9. Tests added for new behavior — PASS — 726 lane tests; all round-2 mutation gaps now closed.
- 10. Tests cover the failure path (not just happy path) — FAIL —
vault_write's refusal paths are tested only against a fake runner, never the realopvocabulary. - 11. UI changes (if any) — screenshot captured — PASS — both template branches captured this round.
- 12. Docs updated for user-facing changes — PASS — all 7 documentation targets updated.
Rubric
- 1. Plan vs. implementation match — fail — Task 9's vault writer is non-functional on its default path.
- 2. New code quality — pass — three nits, no structural issues.
- 3. Test coverage — fail — the
optemplate-category contract has no passing test at any level. - 4. Regression risk to existing callers — pass —
admittedis additive;_record_terminal_dead_letter's stage coercion fix is covered. - 5. Data integrity — pass — no schema migration needed; import now refuses foreign keys and replaces history under
--force. - 6. Security — pass — no credential in argv, logs, or the returned result; import refuses out-of-namespace keys.
- 7. Documentation accuracy — pass — the head-state contract and dead-letter rows now match the code.
- 8. PR body accuracy — fail — "Integration suite: 5 passed, 1 skipped" does not reproduce; the vault row fails where
opis authenticated. The 726 test count and the ruff claims do reproduce exactly. - 9. Disclosed deferrals — pass — all 7 sound.
- 10. Follow-up claims verified — pass — #3220 OPEN.
Verdict derivation: Rule 1 — critical items 1 and 3 fail with no matching acknowledgement → CHANGES REQUESTED — Blocker. The quorum shortfall independently forbids APPROVED this round regardless.
Is the carry-forward the sole remaining obstacle? No — and it is no longer an obstacle at all. Round-2 blocker 2 (ui/templates/improvement/control.html:9, the visual-proof gate) is discharged. The only blocker standing between this PR and approval is the new tools/vault_write.py:51 category defect, which /do-patch can close in one line plus a pinning test.
Review (Judge code-quality): Changes RequestedIndependent judge, dispatched as its own subagent (no Verdict: CHANGES REQUESTED (0 blockers, 0 tech debt, 2 nits). Every round-2 and round-3 finding is closed, located by symbol at this head, and the five mutations run all went red. Round-2 dispositions (16, located by symbol at f657d12)
Round-3 dispositions (6)
Mutation log (each restored,
|
Review (Judge risk): Changes RequestedIndependent judge, dispatched as its own subagent (no Verdict: CHANGES REQUESTED. The judge classified its two substantive findings as tech debt; the parent promoted the first to a blocker under rubric item 5 (critical) after verifying the reproduction, so the roster-level counts are 1 blocker, 1 tech debt, 3 nits. Every round-2 and round-3 finding is closed with mutation evidence (19 of 20 mutations red; the one green is an equivalent mutant). Round-2 dispositions (16, located by symbol at f657d12)
Round-3 dispositions (6)
Mutation log (20; every mutation restored,
|
Review: Changes RequestedMode: Head judged: Verdict: CHANGES REQUESTED, 1 blocker, 1 tech debt, 5 nits. Every prior-round finding is closed with mutation evidence from both judges independently (code-quality: 5 mutations, all red; risk: 20 mutations, 19 red, 1 equivalent mutant). The visual-proof gate passed this round. The one blocker is new surface the Parent-reproduced gates (Hard Rule 10): both judges reproduced Blockers
Tech Debt
Nits
Miscellaneous
Acknowledged Deferrals (verified)
Review Delta (vs prior review on HEAD 93def1e)
Verification Results
ScreenshotsVisual proof gate: PASSED. BYOB MCP (real Chrome) was live; the dashboard was started from this worktree at the PR head on spare ports (8531 against live data, 8532 against a claimed test db seeded through the ORM and cleaned through the ORM afterward), both stopped by PID.
Pre-Verdict Checklist (aggregate, any judge FAIL = FAIL)
Rubric (aggregate, any judge fail = fail)
Verdict derivation: Rule 1, critical items 1 and 5 |
Review (Judge code-quality): ApprovedIndependent judge, continued from round 4 (same subagent, own throwaway worktree, no view of the Verdict: APPROVED, zero findings. All seven round-4 dispositions closed, located by symbol; the new export test mutation-verified red. Round-4 dispositions (7, at 8f8c22f)
Mutation log
Blockers
Tech Debt
Nits
Miscellaneous
Acknowledged Deferrals (verified)
Verification results (reproduced in the judge's worktree,
|
Review (Judge risk): ApprovedIndependent judge, continued from round 4 (same subagent, own throwaway worktree, no view of the Verdict: APPROVED, zero findings. All seven round-4 dispositions closed by symbol and by mutation; the judge's own round-4 reproduction of the stranded slot no longer reproduces. Round-4 dispositions (7, at 8f8c22f)
Mutation log (all restored,
|
Review: ApprovedMode: Head judged: Round 5 summary. Round 4 at Parent-reproduced gates (Hard Rule 10): the parent measured the plan's 21-file lane row at 732 passed and Verified
Blockers
Tech Debt
Nits
Miscellaneous
Acknowledged Deferrals (verified)
Review Delta (vs prior review on HEAD f657d12)
Verification Results
ScreenshotsVisual proof gate: PASSED, carried from round 4. The
Pre-Verdict Checklist (aggregate, any judge FAIL = FAIL)
Rubric (aggregate, any judge fail = fail)
Verdict derivation: Rule 4, every rubric item |
…y, ImprovementSettings fields tools/improvement_control/__init__.py: package docstring (namespace, private alias rule, reason-code vocabulary, #3220 hand-off). tools/improvement_control/keys.py: SCHEMA_VERSION, assert_control_key, and a key builder per Decision 2's layout, including the reconcile pass's one sanctioned scan pattern. config/settings.py: ImprovementSettings gains lease_ttl_seconds=90, journal_max_entries=1000, max_dispatch_attempts=3. Recovered from an interrupted prior build attempt on this worktree; verified against the plan and reflowed the keys.py docstring under ruff's line-length limit before committing. journal.py's transition script and its tests are still outstanding (Refs #3215).
…one change (Refs #3215) NON_TERMINAL_STATUSES and RECOVERY_OWNERSHIP (models/session_lifecycle.py) and ACTIVE_STATUSES (ui/data/sdlc.py) all gain "admitted" together, with admitted's owner "reflection" (the improvement-intent-reconcile pass). test_recovery_ownership.py::test_owners_are_known_values gains "reflection". test_session_lifecycle_consolidation.py's total-status count moves 14 -> 15 (10 non-terminal), naming admitted beside paused_budget. test_ui_sdlc_data.py's ACTIVE_STATUSES pin gains admitted. test_session_recovery_drip_budget.py gains a same-shape drip-exclusion case for admitted, since it must never be dripped to pending outside the improvement scheduler adapter's own liveness check (Task 3 of docs/plans/improvement-controller-lane-3-control-journal-fenced-dispatch.md).
…D to the concurrent driver (Refs #3215) tools/improvement_control/journal.py and lease.py landed already via the concurrent lane-3 builder's Task 3 commit (e46130f), which picked up my staged files alongside its own. This commit adds the two test files that were still outstanding: tests/unit/test_improvement_control_journal.py (accept/reject paths, bounded journal, STALE_GENERATION/REVISION_MISMATCH/ PAUSED/INVALID_ARGUMENT/UNAVAILABLE, the session-intent binding compare) and tests/unit/test_improvement_control_lease.py (LeaseProtocol conformance, the improve: prefix restriction, the #3220 retirement test). 27/27 pass via scripts/pytest-clean.sh. Standing down here per lane3-supervisor: a second, independent coordinator is already driving lane 3 BUILD in this worktree. No further edits.
…s (Refs #3215) test_settings.py::TestImprovementSettingsControlJournal covers the three Task 1 fields (lease_ttl_seconds=90, journal_max_entries=1000, max_dispatch_attempts=3) and their IMPROVEMENT__* env overrides, read through the top-level Settings() (ImprovementSettings itself is a plain BaseModel; env_nested_delimiter only applies at the Settings() level).
…ok (Refs #3215) tools/improvement_control/intents.py: the six-state intent machine (Decision 13) with one Lua script per effect (admit, record_materialized, record_running, cancel, mark_reconciliation_required, on_session_terminal), each re-checking the case's generation/revision fence before recording anything and appending its own journal entry. admit walks the case's `intents` set (never KEYS/SCAN) to refuse admission while any intent is reconciliation_required, before the slot count. list_intents is the one Python reader over that set. on_session_terminal releases the lane slot (compare-and-delete against the intent's own created_ts) and settles by outcome in one script (Decision 6): running->settled on a result_digest, settled/no_proposal on a bare completion, or session_terminal_at stamped and left running for the reconcile sweep otherwise. models/session_lifecycle.py::finalize_session gains step 7 exactly as Decision 6: gated on extra_context.action_id, lazy-imported, exception- isolated. _make_session() in test_session_lifecycle.py now defaults extra_context to {} (a bare MagicMock attribute is truthy, which would have silently exercised the real Redis path for every unrelated existing finalize_session test). Fixed a bug caught by tests before commit: _move()'s from_state auto-derivation assumed exactly one predecessor per target state, which is false for reconciliation_required (three valid predecessors) -- it now takes from_state explicitly. dead_letter_exhausted uses the already- reserved "improve_intent" DeadLetter stage (bridge/dead_letters.py STAGES, "Reserved for the improvement control plane") via the canonical bridge.dead_letters.record() helper rather than a raw DeadLetter.create() with an ad hoc "improvement-intent" stage string. 55 new tests in test_improvement_control_intents.py (Race 1, the reconciliation-required admit guard and its intents-set read, the three settle branches, foreign_holder/absent slot outcomes, cancel as the only reconciliation_required exit, the reconcile/materialize race, dead-letter exhaustion, and runtime refusal of several illegal transitions) plus 3 new tests in test_session_lifecycle.py for step 7's exception isolation and provenance gate. 91 + 55 = 146 tests green via scripts/pytest-clean.sh.
projection.py: apply() writes the head's truth onto ImprovementCase
through the ORM without ever reading the projection first; replay()
folds the journal tail as a cross-check (fold_reached_head=False when
LTRIM has trimmed past the fold's ability to verify from revision 1,
reported rather than raised) and always reconciles the projection to
the head.
export.py: export_namespace() enumerates cases through the ORM (never
a keyspace scan) and writes namespace.json + artifacts.json under
lane 7's export-root contract; import_namespace() refuses a schema
mismatch unconditionally and a non-empty *control-plane* namespace
(the improve:{project}:_ns:schema key, not the ORM projection, which
is a separate store) unless force=True.
8 new tests: apply's no-head no-op, replay's correction of a direct
ORM save and its honest non-raising report on a trimmed tail, and the
export/import round trip including the reconciliation_required intent
surviving through the intents-set (a restore that dropped the index
would let a following admit through instead of refusing INTENT_STATE).
The improvement scheduler adapter's liveness gate before it flips an admitted research session to pending: reuses the existing WORKER_REGISTERED_PID_KEY_PREFIX scan and _worker_pid_heartbeat_fresh verbatim (Task 6), true on the first fresh heartbeat. No other change to the module.
…ration (Refs #3215) tools/improvement_control/scheduler_adapter.py::tick(): admits an unadmitted action_proposed, materializes through the create-or-bind seam under asyncio.run, and activates once a worker is alive. _activate is shared between the fresh-dispatch path and the retry path (Race 7): a case with an already-materialized intent is re-activated on every tick before the adapter looks for anything new to admit, branching on the row's freshly re-read status exactly as Data Flow step 8 specifies (admitted -> flip+publish+running; pending -> republish+running, no flip; terminal -> settle via on_session_terminal; any other non-terminal -> record_running only; missing row -> skip, left for reconcile). Per-case exception isolation (Risk 2) and a reconciliation_required skip before the lease acquire. Fixed a real bug found by the tests: ImprovementCase.query.filter() does exact-match on an IndexedField, not IN -- passing OPEN_CASE_STATES as one filter value silently matched nothing. One filter call per state, the same pattern ui/data/improvement.py's goals partial uses. reflections/improvement_controller_tick.py::run_improvement_controller_tick(): gated on ImprovementSettings.enabled like improvement_collect. scripts/update/reflection_register.py gains register_improvement_controller_tick (cadence from controller_tick_seconds) and register_improvement_intent_reconcile (fixed 300s, Task 7's registration landed here alongside it since both share one generalized register path); scripts/update/run.py calls both. 11 new tests in test_improvement_control_dispatch.py (the happy path, no-live-worker, Race 2's crash-after-bind retry, reconciliation_required skip, and all four Race 7 activate-retry branches), 3 in test_session_health_worker_liveness.py for any_worker_alive(), and 6 in test_reflection_register.py for the two new registrations.
tools/improvement_control/recovery.py::reconcile(): a controller itself (Decision 12) -- acquires the case lease per case before touching any of its intents, exactly like the scheduler adapter. admitted/materialized intents past 4x lease_ttl_seconds get HINCRBY stale_sweeps (the pass's own counter; record_materialized's attempts is never read or written here); at max_dispatch_attempts the intent moves to reconciliation_required, its slot releases, and a live bound row is forced abandoned through finalize_session with dead_letter_stage= "improve_intent" -- guarded on the row not already being terminal, since finalize_session raises StatusConflictError otherwise. A running intent whose bound row is missing or terminal is acted on the first sweep past the threshold (the session is gone; no budget applies). A running intent with a live row is untouched regardless of age. Fixed a real bug in shared code: models/session_lifecycle.py's _record_terminal_dead_letter accepted a `stage` parameter but silently coerced anything other than "session_init_hang" to the hardcoded "session_recovery_cap" string, so this lane's forced-abandon dead letters were landing on the wrong stage. Both of its two existing production callers already pass their own exact desired stage explicitly, so passing `stage` straight through (session_init_hang alone stays non-replayable) is a pure bugfix with no behavior change for either -- confirmed by the full test_session_lifecycle.py suite (112 tests) staying green. reflections/improvement_intent_reconcile.py::run_improvement_intent_reconcile(): unconditional (recovery must run against intents an earlier enabled window admitted, even after ImprovementSettings.enabled flips back off). 7 new tests: a fresh intent left alone, the three-pass stale-sweep budget, Race 3's unreleased-slot-on-restart freed on the first qualifying sweep, a running intent with a live row untouched regardless of age, reconciliation_required's one exit staying shut against a further sweep, and the two forced-finalize logging cases (a terminal bound row skips finalize_session entirely; a non-terminal one settles through it with no WARNING from intents.py's own logger, only the ordinary dead-letter one).
…pt (Refs #3215) tools/paid_inference_meter.py mirrors tools/infrastructure_budget.py's shape (reserve-then-check in one Lua EVAL on a plain non-Popoto key, a paired idempotent release, spend_receipt evidence for settlement, window boundaries disclosed on every decision) without sharing code -- units 2 and 3 are charter §8's two separate pools and must never transfer between each other. settle_from_response has exactly two branches (usage.cost when present -> "exact"; otherwise a dated PRICE_TABLE estimate from tokens -> "estimated") and no third: a response with no usage at all leaves the reservation open for the reconcile pass's sweep_unsettled_reservations to receipt as "unknown". Only purpose="rsi" reservations count against the pool; purpose="sdlc_review" receipts (the judge, below) are record-only. _redis() binds through utils.redis_client.text_redis(), never popoto.redis_db.POPOTO_REDIS_DB -- the plan's Verification anti- criterion greps this module by name alongside tools/improvement_control/. tools/cross_vendor_judge.py: two lines after the existing usage log -- one record_receipt(project_key="valor", purpose="sdlc_review", ...) call, exception-isolated. Pinned to "valor" because the judge runs for any repository and holds no project key of its own; the receipt is about the paid-inference pool, which is Valor's. 17 new tests: exact settlement in both response shapes (attribute and mapping), estimated-from-tokens, the no-usage-at-all open-reservation path, two concurrent reservations admitting exactly one, INVALID_AMOUNT on every unforecastable input, window attribution across a UTC midnight, the sdlc_review receipt landing under project_key="valor" without touching any other project's window, and the no-HTTP-client/ no-OpenRouter-URL anti-criterion asserted directly against the module source. 15 existing test_cross_vendor_judge.py tests stay green.
…bulary (Refs #3215) models/improvement_evidence.py: EVIDENCE_KINDS gains "resource_acquired" (8 kinds, exactly DEFAULT_VOCABULARY_MAXIMUM; the closing comment tells the next lane to bring its own VOCABULARY_MAXIMUMS entry rather than a free append). models/improvement_investigation.py: INVESTIGATION_KINDS gains "charter_amendment", INVESTIGATION_STATES gains "awaiting_authorization" (both well under the cap). tools/vault_write.py::write_credential(): the one sanctioned `op item create` path, with an injectable runner (tools/improvement_resources.py's pattern) so unit tests never call op. The value never appears in argv, logs, the result, or the evidence row -- only a title and a sha256:<hex> fingerprint leave the function. Empty title/value refuse before any process spawns; any op failure refuses with stderr's first line. A successful write records ImprovementEvidence(kind= "resource_acquired", source_ref=f"vault:{title}"). render_resource_ acquired_section(rows) is the pure renderer lane 5's digest calls. tests/unit/test_improvement_resources.py gains test_vault_write_probe_reports_verified_once_the_writer_exists: the probe's `_probe_vault_write` absent branch can no longer be reached on main now that tools/vault_write.py exists. 12 new unit tests (no credential byte in the result, logs, or evidence row for both the refused and created paths; empty-title/whitespace- value/missing-binary/non-zero-exit refusals; the digest renderer) plus one integration test (tests/integration/test_vault_write_integration.py) that creates and deletes one real m-valor item, skipped with a named reason when op cannot authenticate non-interactively.
tools/improvement.py: a thin argparse shell over the control package (Decision 9) -- propose, propose-amendment, pause, resume, doctor, case show/explain, budget, export, import, replay-projection, release compare. propose resolves its session through AGENT_SESSION_ID, refuses NOT_A_RESEARCH_SESSION for a plain session, validates the case (ranking_rationale, priority_area, charter_digest match) before touching the lease, and stores a refused session-bound proposal's payload digest as ImprovementEvidence(kind="other", detail="intent_state:<reason>") so nothing is silently lost. resume's body is ordered exactly per Decision 9: the reconciliation_required scan runs before the paused check, so an unpaused wedge (the common shape, since mark_reconciliation_required never writes the head) has a real exit -- "not paused; cancelled N intent(s)" -- instead of "not paused" with no cancellation. case explain composes the operator's answer from the head, the intents list, and the pinned-charter check in one JSON shape. pyproject.toml gains valor-improve = "tools.improvement:main" (Decision 14, venv-only). .claude/skills/improve-research/SKILL.md is the research session's own brief: read the case through `case explain`, research the open web and memory, write only through `propose`, the three nevers (no valor-session create, no direct control-namespace writes, no messages outside propose-amendment). Fixed a real gap caught by the CLI's own test: import_namespace raised FileNotFoundError on a missing archive instead of returning a named refusal; it now returns ARCHIVE_NOT_FOUND. cmd_budget's first draft called infrastructure_budget.status_dict positionally against a keyword-only signature. 17 new unit tests (case show/explain --json shapes, the blocking-intent naming, resume --force clearing an unpaused wedge and enabling a following admit, doctor/budget/export/import smoke coverage, the propose validation refusals, and the NOT_A_RESEARCH_SESSION / break-glass split) plus 5 integration tests against the real installed binary (Path(sys.executable).parent / "valor-improve"): propose end to end under a seeded research session, Race 4b's stale-then-redispatched session end to end (with the refused artifact landing as evidence), doctor on a seeded paused case, and the child-session gate untouched.
ui/data/improvement.py::get_control_status(): intents by state, lane slots, unit-2 spend, paused heads, and reconciliation_required wedges, read through intents.list_intents over each open case's own set -- never a keyspace scan, matching doctor and case explain. Three-state rendering like every other panel in the module (content / "nothing yet, written by lane 3 when a case is admitted" / "unavailable" on a read failure). The getter-list pin (test_ui_app.py) gains get_control_status; the module docstring is corrected to describe the new status quo rather than the lane-3-shaped hole it used to name. ui/templates/improvement/control.html and the inline route in ui/app.py (beside the goals partial); index.html wires the panel in alongside the other three improvement partials. 12 new tests: get_control_status's empty/seeded/reconciliation-required/ unavailable shapes (test_ui_improvement_data.py) and the control partial's empty-namespace, seeded-paused-case, and read-failure renderings plus the updated index-page and getter-list pins (test_ui_app.py).
docs/features/improvement-controller.md: "Control namespace contract", "Dispatch", "Break-glass", and "Dependency on #3183" rewritten with the shipped shapes (key layout, reason-code vocabulary, the two-fence model from Decision 12, the full valor-improve command table, the reconcile pass's stale_sweeps/attempts split). Unit 2 now has a meter alongside unit 3; the unit-3 migration sentence corrected to the recorded No-Gos decision (the counter stays on its own key). Dashboard section updated for the fourth (Control) panel and the getter count. docs/features/session-recovery-mechanisms.md: new mechanism 11 (Improvement Intent Reconcile) in the same table shape as the other ten, and `admitted` added to the RECOVERY_OWNERSHIP table with its `reflection` owner. docs/tools-reference.md: the valor-improve section drops "planned, lane 3" and lists all twelve shipped subcommands. docs/features/adding-reflection-tasks.md: the tracked-registration wrapper list gains register_improvement_controller_tick and register_improvement_intent_reconcile. docs/features/redis-models.md: the control-namespace exception section confirmed shipped (was written predictively by lane 2), cross-linked to keys.py::assert_control_key and the private-alias binding site, and extended to name the unit-2 meter's identical rationale for binding the same way. docs/plans/critiques/recursive-self-improvement-capability-matrix.md: new Lane 3 section in the established four-column shape (every primitive implemented and tested, nothing yet measured against a real research session since none exists until lane 5 ships); the "Not built, by lane" table's lane-3 rows removed now that they are built.
…ring (Refs #3215) tests/unit/test_improvement_control_admitted.py was named in Task 3 but never created -- its assertions had been folded into the four other Test Impact files instead, leaving the plan's own Verification row ("Lane tests pass") referencing a nonexistent path and failing the whole command's collection. Adds it: admitted not in RESUMABLE_STATUSES, and a seeded admitted row is invisible to the worker's pending-status query and a running-status query (the same exact-match AgentSession.query.filter shape worker/__main__.py and _agent_session_health_check both use), with a positive control confirming the row really is admitted. tools/improvement_control/recovery.py's own docstring literally spelled out the regex `HINCRBY.*attempts` it was warning readers about, which matches itself under grep -E (`.` and `*` interpreted as regex against plain text) -- the exact "Two counters, two owners" Verification row this docstring cites. Reworded to describe both counters without the literal adjacency. Full Verification-table run (agent.verification_parser) against this build: 6 rows are malformed in the plan's own table text (unescaped `|` splitting a cell, or basic grep where the pattern needs -E for alternation -- both pre-existing plan-authoring issues, not this commit); of the remaining checks, all but one false-fail are green after this fix (the naive evaluator can't parse a few rows' compound "X and Y" or "empty or 0" expected-value text against grep's own exit-1-on-zero-matches convention, verified by hand instead). One row ("Judge receipt pinned") cannot pass a literal single-line grep because ruff's formatter always wraps cross_vendor_judge.py's multi-kwarg record_receipt(...) call across lines; the call itself is correct and verified by test_paid_inference_meter.py.
Blockers: - journal.transition now seeds the head's `state` field (HSETNX) from the case's own ImprovementCase.state on first write; projection.apply/replay refuse to clobber the projection with an empty state as defense in depth. Previously read_head always returned state="" and apply/replay dropped every case from OPEN_CASE_STATES. - cmd_propose now writes the payload through VerifyingArtifactStore before taking the lease, journals the reference as `artifact_ref` alongside `payload_digest`, keeps it on the refusal evidence row, and export.py populates artifacts.json from journaled references instead of `[]`. - cmd_doctor's per-case read_head/list_intents calls now run inside the same guard as the ORM query, so a control-namespace outage reports "namespace unreachable" with exit 2 instead of a traceback; added the end-to-end break-glass drill test the plan's Success Criterion 2 named. Tech debt: - --action-type flows through transition into the journal entry and the admitted intent (previously always defaulted to "investigate"). - mark_reconciliation_required's slot release and `reason` write now happen inside the same CAS script as the state move, not a second unconditional HDEL after it returns. - dead_letter_exhausted is now the one caller writing a DeadLetter for an exhausted intent, on every branch (bound row or not), always replayable=False. - scheduler_adapter resolves a real working_dir, threads request_digest/ charter_digest into admit(), soft-checks the pinned charter, and the dispatch message names /improve-research with a brief_ref. - Dispatch tests now assert publish-exactly-once, the no-live-worker path never publishes, extra_context_overrides' exact key set, and one case's failure never stops the tick (new isolation test). - export/import round-trip unit-2 window/reservation hashes and the namespace pause hash (previously exported but never restored). - sweep_unsettled_reservations uses the injected clock, not wall time. Nits: deleted dead intent_scan_pattern; dropped the unreachable event ~= "ns_resumed" clause; corrected the shared-prelude and _LUA_MOVE_INTENT ARGV comments; reworded the __init__.py docstring so it no longer trips its own anti-criterion grep; vault_write's default runner sets OP_CACHE=false explicitly; control.html panel gets a heading; struck the unimplemented --acknowledge-unknown claim from the plan's Risk 5. Ticks plan Success Criterion 2 (fault-injection tests, all four races). Verified: targeted suite for every touched file (139 passed), integration suite (5 passed, 1 skipped, matches the review's own baseline), full tests/unit/ (16531 passed, 9 pre-existing failures unrelated to this diff -- none touch a file this commit changes).
…head-state contract, import hardening (Refs #3215) Closes the round-2 blocker (B2 artifact path now asserted on the accepted, refused, and export paths) and the seven tech-debt items: propose tests write to a tmp content root, projection.apply runs after every accepted transition, a state_changed journal event is the writer of head state with re-seed on empty, import refuses foreign keys and force replaces history, budget lists unknown-metered receipts, dead-letter comment and docs corrected. Eight nits folded in: settle is one Lua script, ARTIFACT_WRITE_FAILED reason code, brief_ref carries the loadable artifact ref, review-history comments rewritten as invariants. Addresses "Activation publishes publish_session_notify(session) exactly once per activation".
…3 review nits (Refs #3215) op item create --template rejects 1Password's display name; the enum form API_CREDENTIAL is what op item template get itself emits. Every write_credential call on its default path refused. Pinned by a unit test that needs no op auth, and the real-boundary integration test now runs and passes. _probe_vault_write reported verified on file existence alone; it now abstains with unknown, since an existence check cannot certify that a write is accepted. Also: delete dead _project_key, drop the duplicate-level Control panel heading to match sibling panels, and route export.py's expire through assert_control_key.
…ations, round-4 review nits (Refs #3215) import --force now deletes the namespace slot and pause hashes before restoring, so a slot admitted after the export is dropped with its intent instead of stranded with no release path. Pinned by test_import_force_drops_a_post_export_slot, red without the delete. doctor reads the unit-1 slot hash and the open unit-2 window and prints them under `reservations`, naming each slot's holder; the clean line prints only when paused heads, wedged intents, and reservations are all empty. Both doc sites now describe exactly what doctor prints, and the plan's doctor criterion is ticked. Also: pause carries `reason` and refuses CASE_BUSY on a held case lease; a break-glass propose without --action-id mints one; release is one Lua script under a CAS on state == 'reserved'; _probe_vault_write carries a docstring stating the invariant.
The charter-amendment path, the reconcile pass, and the case lease are described as shipped rather than arriving with lane 3; the evaluation doc's lane-4 plan links follow the plan into docs/archive; the code-registered reflections list names the three improvement reflections; the feature index row covers lanes 1-3. The lane-3 plan's Documentation checklist is ticked and its frontmatter reads docs_complete.
eca0347 to
d527bd6
Compare
Review: ApprovedRe-verification of the round-5 verdict at head
The round-5 code verdict (732 lane tests, ruff clean, 6 delta mutations red, all 7 round-4 dispositions closed) therefore carries to this head unchanged. Preflight at posting: Blockers
Tech Debt
Nits
Miscellaneous
|
Sibling lane 3 (PR #3315, f026832) landed the control journal, fenced dispatch, the unit-2 paid-inference meter, and the `valor-improve` CLI, making this branch DIRTY. Merged rather than rebased: the branch is pushed and review artifacts reference its SHAs. Both lanes' work is kept in every conflict: - ui/data/improvement.py: module docstring unions lane 3's control-status view with lane 6's release lineage; the "not written yet" list shrinks to what neither lane writes (hypotheses, rejected experiments, lane 5). - ui/templates/index.html: both the releases panel and the control panel render; neither replaces the other. - tests/unit/test_ui_app.py: the index-page test asserts both partial URLs, and both lanes' partial test cases are kept. The pinned getter list auto-merged to the correct six-name union. - docs/features/README.md: controller row reads "Partial (lanes 1-4, 6)" and its summary names both lanes' surfaces. - docs/features/improvement-controller.md: "What exists today" covers lanes 1, 2, 3, 4, 6; the dashboard section documents five panels and six getters; lane 3's shipped work is stated as shipped, lane 5 stays open. - docs/tools-reference.md: lane 3's fuller `valor-improve` command block wins as a superset; the evidence-side note points at `valor-improve-release` for the release side. - docs/plans/critiques/recursive-self-improvement-capability-matrix.md: lane-3 rows leave the "not built" table (they shipped); lane 5 and the automated-promotion row stay. Semantic (non-textual) fallout resolved beyond the conflicted hunks: - Six public getters now exist in ui/data/improvement.py, so every "five getters"/"fifth getter" claim in the matrix and the lane-6 plan (including a verification row whose command asserts the exact list) is updated to the six-name union. The command passes as written. - tools/improvement_recursion/budget.py and compare.py claimed no paid-inference meter exists "until lane 3". It now exists, but meters the daily pool by window and reservation with no `arm_run_id` dimension, so `LedgerBudgetReader.unit2_usd` still answers `None` and `BUDGET_UNKNOWN:unit2` still refuses the claim. Behavior is unchanged; the comments now state the real reason. The arm-scoped read is recorded in the matrix as lane 5 work. Refs #3218
Closes #3215
Plan:
docs/plans/improvement-controller-lane-3-control-journal-fenced-dispatch.mdSummary
Ships the control substrate the improvement controller's dispatch decisions run on: a control journal with a fencing lease, durable dispatch intents, an
admittedsession status, a reconcile reflection, a unit-2 paid-inference meter, a vault writer, and thevalor-improveCLI — the only door through which a research session proposes anything.tools/improvement_control/journal.py): one LuaEVALper transition, checking schema, pause,generation >= highest_accepted,expected_revision == revision, and — for a session-bound write — the intent binding, all before recording anything.lease.py):LeaseProtocolmatches Session execution lease: fence worker ownership with a renewable Redis lease #3220's declared interface;CaseLeaseis the interim implementation, deletable in one commit the daymodels/redis_lease.pyexists (hand-off posted on #3220).intents.py): the six-state machine (Decision 13), one script per effect, the lane-slot reservation, andfinalize_sessionstep 7's terminal hook.admittedsession status: added toNON_TERMINAL_STATUSES,RECOVERY_OWNERSHIP, andACTIVE_STATUSESin one commit; inert to the worker, the health check, startup recovery, and the resume drip.scheduler_adapter.py, theimprovement-controller-tickreflection): admits, materializes through the create-or-bind seam, and activates once a worker is alive — Race 7's four branches (admitted/pending/terminal/missing row) all handled on a retry tick.recovery.py,improvement-intent-reconcile, 300s): stale-sweep budget for admitted/materialized intents, first-sweep action for a running intent whose row is gone, forced-terminal via the existingfinalize_session.tools/paid_inference_meter.py): mirrors unit 3's shape; settles exact fromusage.costor estimated from tokens; the cross-vendor judge's review spend is receipted, never gated.tools/vault_write.py): the one sanctionedop item createpath; no credential byte on any path.valor-improveCLI (tools/improvement.py, 12 subcommands) and the research skill (.claude/skills/improve-research/SKILL.md).ui/data/improvement.py::get_control_status).Fault injection (the issue's acceptance criteria)
test_stale_controller_generation_is_refused(Race 4a) andtest_stale_session_intent_is_refused(Race 4b, unit + integration).UNAVAILABLEreason code,doctorreports it, namespace state is untouched and reads correctly once restored.Mutation review (Task 12)
12/12 mutations caught (generation compare, revision compare,
from_statecompare, and compare-and-delete acrossjournal.transition,intents.admit/record_materialized/record_running/cancel/on_session_terminal, andlease.renew/release) — no blockers. Hand-off comment posted on #3220.Deviations from the plan (noted for the record)
dead_letter_stagevalue isimprove_intent(the stage already reserved inbridge/dead_letters.py'sSTAGES), not the plan's literal"improvement-intent"string — fixed a real bug inmodels/session_lifecycle.py::_record_terminal_dead_letteralong the way (it silently coerced any non-session_init_hangstage to"session_recovery_cap", ignoring the caller's own stage).ImprovementCase.query.filter()does exact-match on anIndexedField, notIN— every "iterate open cases" site loops per-state, matchingui/data/improvement.py's existing pattern (this was a real bug caught by the scheduler-adapter tests before commit)._move()'sfrom_stateis always explicit, never auto-derived from_ALLOWED, becausereconciliation_requiredhas three valid predecessors (another real bug caught before commit).|, or a basicgrepneeding-Efor alternation) that are plan-authoring artifacts, not code issues — verified by hand instead; see the Task 14 commit message for the full accounting.tools/paid_inference_meter.py's reserve/release scripts are not generation/revision-fenced by the case lease — unit 2 is a project-level daily pool, independent of any one case's state, so Task 12's mutation matrix for "reserve_unit2/settle_unit2" didn't apply the same way; the pool's own reserve-then-check atomicity is still one LuaEVAL.Review rounds
e8dcaf8ce): headstateseeded on the first accepted write withapply/replayrefusing an empty clobber;proposestores the payload throughVerifyingArtifactStorebefore the lease and journals the$CF:reference;doctorguards every namespace read and the outage break-glass drill is a test;action_typejournaled; slot release andreasonmoved inside the move script;dead_letter_exhaustedon every exhaustion branch; adapter passes a realworking_dir, both digests, and a/improve-researchmessage; publish-count, override-key, and isolation assertions;unit2andns_pauseround-trip on import; sweep honors the injected clock. The plan's Risk 5--acknowledge-unknowngate is reframed as accepted residual scope (charter §8 receipt retained; thebudgetunknown-receipt block below is its visible surface).93def1ecb): the artifact path is asserted on the accepted, refused, and export paths; propose tests write to a tmp content root;projection.applyruns after every accepted CLI transition; astate_changedjournal event is the writer of headstate, re-seeded when empty;importrefuses foreign keys before writing and--forcereplaces history;budgetlistsmetering="unknown"receipts with their window;settleis one Lua script;ARTIFACT_WRITE_FAILEDreason code;brief_refcarries the loadable reference.f657d12ea):tools/vault_write.pydefaults to theopenum categoryAPI_CREDENTIAL(the display name"API Credential"was refused byop item create --template), pinned at unit level byTestTemplateCategoryIsTheEnumSpelling;_probe_vault_writeabstains withunknowninstead of certifying a write from an existence check; three nits: the unused_project_keyhelper deleted, the control panel's<h4>Control</h4>title dropped so the heading hierarchy matches its sibling panels, andimport'sexpirerouted throughassert_control_key.8f8c22f22):import --forcedeletes the namespace slot and pause hashes before restoring, so a slot admitted after the export is dropped with its intent instead of stranded with no release path (test_import_force_drops_a_post_export_slot, red under mutation);doctorreads the unit-1 slot hash and the open unit-2 window for real and prints them underreservations, naming each slot's holder case, with the clean line only when all three views are empty (unit and end-to-end tests assert the seeded slot); five nits:pausecarriesreasonand refusesCASE_BUSYon a held case lease instead of presenting generation 0; a break-glassproposewithout--action-idmints one so the proposal is admittable;releaseis one Lua script under a CAS onstate == 'reserved'(a double release decrements exactly once);_probe_vault_writecarries a docstring stating the invariant; this body refreshed.Testing
scripts/pytest-clean.shon all 21 files the plan's own "Lane tests pass" Verification row names: 732 passed at8f8c22f22.test_improvement_control_cli.py: 5 passed at8f8c22f22, run against the real.venv/bin/valor-improvebinary in a claimed test db.test_vault_write_integration.pyis guarded byOP_CACHE=false op whoamiand skips where thevalor-localservice account is absent (both round-4 judges reported it skipped on their machine); whereopauthenticates non-interactively it runs, creates onetest-lane3-<uuid>item inm-valor, and deletes it in afinally(verified at round 3: no residue, 57 items before and after). The round-3 category fix is pinned bytests/unit/test_vault_write.py::TestTemplateCategoryIsTheEnumSpelling, which needs noopauth and was proven red against the display-name value.ruff check ./ruff format --check .: clean at8f8c22f22(50 changed Python files in the diff).scripts/validate_docs_changed.py): passed, 7 docs changed.