[HugoXK]

feat(rrsi): wire the regularized RRSI mechanism into evolve() through the Policies seams

The problem

The pinned RRSI fixtures (examples/rrsi/selection.py — Algorithm 2's selection rule — and schedule.py — the cosine-annealed edit budget) have been merged, and docs/algo-rrsi.md says exactly what remains: "no RRSI CLI, harness-directory domain, implemented leakage critic, edit-budget schedule, measured per-edit history, multi-round resume, trial aggregation, pruning loop, or production evaluation/accounting adapter yet." Issue #221's acceptance criteria (leakage screening before evaluation, edit-budget boundaries, history attribution, stall exploration, pruning, resume that preserves S*, evolve/holdout isolation, offline fixtures) are all unmet.

The paper's mechanism (arXiv:2609.24972v3) is six coupled regularizers — L0-style annealed update sparsity, evidence-aware credit assignment, structured exploration, leakage screening, a noise-adjusted historical-best floor, and L1-style structural pruning — that together make harness self-improvement transfer beyond the evolve set at ~30% fewer policy tokens. This PR wires those regularizers into evolve() through the engine's existing Policies seams, reusing the pinned selection core and schedule rather than rewriting them.

What this PR adds

examples/rrsi/rrsi_policies.py — build_rrsi_bundle(complete, protected=..., reviewer=..., config=..., base_score=...) assembles three policies that share one checkpointable RRsiState (S*, the measured per-edit ledger, component bookkeeping, the annealed round counter):

Slot	Policy	What it regularizes
Policies(screen=...)	RRsiCritic	Pre-evaluation screen: the annealed edit-budget gate (len(ops) > b_t rejected) and deterministic evolve-set leakage screening (protected strings), plus an optional model reviewer. Runs before measurement, so a leaking or over-budget card is settled and reported critic-rejected without a held-out sweep.
Policies(proposal=...)	RRsiProposalPolicy	The regularized proposer: injects the annealed budget, the measured per-edit history, pruning targets and the stall-exploration note into the reflection prompt.
Policies(acceptance=...)	RRsiAcceptance	The pinned selection.judge decision — historical-best floor, both cost-rule branches, novelty — with S* advanced through the pinned update_best_score.

A new engine seam — ScreenPolicy (Policies(screen=...)): a card-level pre-evaluation gate wired into Aggregator._prepare between the trust region and staleness, i.e. before any measurement. New outcomes critic-rejected / noise-floor / cost-rule; new counters critic_checked / critic_rejected. The aggregator's checkpoint now carries an acceptance policy's optional state, so a resumed run does not lose S* (the test asserts it).

An ergonomic fix — _build_engine no longer demands propose= when Policies(proposal=...) already supplies one (three honest sources: agent, callables, bundle).

Declared deviations (recorded in docs/algo-rrsi.md, not silent)

• One candidate per round. The engine merges to one candidate per round, so select_round's "choose highest admissible argmax" is not exercised; each merge's candidate is judged against the incumbent and S*. This is the "one winner per round" semantics issue Support Google RRSI for regularized harness self-improvement #221 asks to establish first.
• The round counter advances once per merge — the round barrier on the synchronous path, and a documented per-merge anneal on the barrier-free path.
• noise_band is a configured parameter (default 0.0); the paper estimates it from the unchanged base harness.
• Cost is EvidenceCard.cost_tokens; missing/zero telemetry yields dC = 0, matching the pinned upstream behaviour.
• protected strings come from the evolve set only — never the final holdout; the tests enforce that discipline by construction.

Backward compatibility

Component	Default	Old behaviour?
Policies.screen	None	yes — no screen, no counter, no critic-rejected outcomes
Aggregator(screen=)	None	yes — _prepare skips the screen block
MergeOutcome	unchanged	yes — three new members, existing categories untouched
Meter / EvolutionResult	zeros	yes — new fields, zero without a screen
_build_engine check	propose= required	yes, unless Policies(proposal=...) — now accepted and honoured
aggregator checkpoint payload	unchanged	yes — acceptance_state added; unknown keys ignored on restore
default evolution behaviour	unchanged	yes — a run without the bundle reports zeros

Verification

Item	Result
Budget anneals (b_0 wide → b_{T-1} narrow) and the critic enforces it	1 test
Leakage screening rejects evolve-set content; model reviewer asked only when deterministic checks pass	3 tests
Protected strings never derive from the holdout	1 test
History ledger records accepted + rejected; proposer prompt carries history/pruning/stall	3 tests
Stall detection and pruning targets	1 test
S* monotone; checkpoint/restore round-trip	2 tests
Selector goldens: noise floor, large-gain cost rule, within-band novelty, accept advances S*	4 tests
End-to-end offline: leak → critic-rejected, no commit	1 test
End-to-end offline: clean run commits under the bundle; untouched path reports zeros	2 tests
Resume preserves the historical-best floor	1 test
docs/api.md regenerated, test_api_reference green	exit 0
Regression: 171 tests (aggregator/api/metrics/policies/rrsi)	all green

Files

12 files changed, +1093, -29

New:

• examples/rrsi/rrsi_policies.py (+~430) — RRsiConfig, RRsiState, EditRecord, RRsiCritic, RRsiProposalPolicy, RRsiAcceptance, build_rrsi_bundle
• tests/test_rrsi_integration.py (+~330) — 16 tests

Modified:

• agentdescent/policies.py — ScreenPolicy protocol, Policies.screen
• agentdescent/aggregator.py — screen wiring in _prepare, three MergeOutcome members, checkpoint hook
• agentdescent/evolution.py — _WIRED_POLICIES + screen wiring + propose-source relaxation, critic_* result fields
• agentdescent/population.py — screen through population_factory
• agentdescent/metrics.py — critic_checked, critic_rejected
• agentdescent/__init__.py — ScreenPolicy export
• docs/algo-rrsi.md — wired-integration section + declared deviations
• docs/api.md (+9) — regenerated
• CHANGELOG.md (+17) — entry
• examples/rrsi/__init__.py — docstring

[Birfy]

Thanks for this. Wiring RRSI in through the Policies seams while reusing the pinned selection.judge / edit_budget is the right shape, and the new ScreenPolicy seam is small, opt-in, and placed correctly: it runs before staleness and well before the held-out sweep. The deviations section in docs/algo-rrsi.md is also appreciated. I'm requesting changes because of one CI-breaking regression and a few places where the integration doesn't yet do what the PR description says it does.

Blocking

1. Aggregator.checkpoint() regression (CI red on 3.9/3.10, and locally on 3.13).
The new hook reads self.acceptance_policy outside the existing try/except AttributeError. So an aggregator that doesn't have the attribute (the four tests/test_checkpoint.py cases build one via Aggregator.__new__, and a subclass that skips super().__init__ would hit the same thing) now raises where checkpoint() used to work:

FAILED tests/test_checkpoint.py::test_reference_aggregator_checkpoint_roundtrip - AttributeError: 'Aggregator' object has no attribute 'acceptance_policy'
FAILED tests/test_checkpoint.py::test_population_aggregator_checkpoint_roundtrip
FAILED tests/test_checkpoint.py::test_population_checkpoint_includes_parent_state
FAILED tests/test_checkpoint.py::test_population_restore_recovers_parent_state

main passes all of tests/test_checkpoint.py. Use getattr(self, "acceptance_policy", None) in both checkpoint() and restore(). The "171 tests green" in the description must have left out test_checkpoint.py, so please run the full suite.

2. The annealed round counter only advances on accepted merges.
RRsiAcceptance.accept does state.round += 1 and state.progress.append(...) only on the admissible branch. Critic rejections and below-floor or cost-rule rejections never advance the schedule. That contradicts the docstring ("advanced once per merge") and the deviations note, and it switches off two of the six regularizers in exactly the situation they exist for:

st = RRsiState(RRsiConfig(b_min=1, b_max=5, T=4, stall_window=3), base_score=0.5)
acc = RRsiAcceptance(st)
for _ in range(10):
    acc.accept(_context(st, base=(5, 5), cand=(3, 7)))   # all rejected
# -> round 0, budget 5, stalled False, ledger 10

After ten straight rejections the budget is still b_max, so update sparsity never anneals. stalled() is still False, because progress only grows on accepts, so stall exploration can never fire while the search is stuck. Please advance round and append S* to progress on every merge decision, including the screen-rejected path (perhaps through a small hook the aggregator calls once per merge). Then add a test that a streak of rejections anneals the budget and trips the stall note.

3. The cost rule compares batch-summed proposer-rollout tokens.
_card_cost sums cost_tokens across ctx.cards, and incumbent_cost is set to that sum on accept. This has two problems:

• It depends on batch size. Three cards at an identical 100 tokens per rollout, against an incumbent of 100, give dC = +2.0, and a +0.3 score gain is rejected with cost-rule / large-gain cost rule failed (confirmed locally with the default config). Whether a candidate passes then depends on how many cards happened to land in the bucket, not on the harness.
• It measures the wrong harness. EvidenceCard.cost_tokens is the usage of the proposing rollout, which ran the incumbent. Algorithm 2's dC is the candidate harness's policy-token cost relative to the incumbent's, measured on the same evaluation. As written, "candidate cost" is really "the incumbent's cost on the tasks that triggered this proposal".

At a minimum, use a per-rollout mean. Ideally take the cost from the held-out measurement of both sides. If that isn't available yet, I'd rather pass cost=None (so dC = 0, the documented missing-telemetry behaviour) and record it as a deviation than run a cost rule on a proxy that is systematically wrong.

Should fix

4. Novelty is effectively always 0, and its counts mean something different. default_components returns the first path segment ("skills", "value", ...), but STRUCTURAL_COMPONENTS is {"client_tool", "skill", "memory", "subagent"}, so a FileTree skills/... edit never counts as novel. Also, judge(..., incumbent_counts=dict(state.attempts)) passes "how often a component was proposed", while upstream's incumbent_counts is the component inventory of the incumbent harness. With w_n > 0, the within-band novelty branch is therefore unreachable on real layouts. Please map onto the pinned vocabulary and pass incumbent counts, or declare it as a deviation. Note that #253 is porting the upstream component vocabulary K and the per-edit history L_t (examples/rrsi/components.py / history.py). This PR's EditRecord ledger and default_components duplicate that with different semantics, so it would be good to build on #253 (or agree on the order) rather than land two histories.

5. The edit budget counts ops (files or keys), not independent edits. For a FileTree harness, len(diff.ops) is a changed-file count, which #221 explicitly says the budget is not. A single coherent edit that touches SKILL.md plus a helper uses 2 of the budget, while ten unrelated rules in one file use 1. If attributed-edit counting has to wait for #253's K, please say so in the deviations list.

6. Partial screen rejections vanish. When some cards in a batch are screened out and others survive, the rejected cards are settled and counted, but their reasons go nowhere: they're not on the merge report and not in RRsiState.ledger. #221 asks that "rejected candidates are logged without invented measurements", and the proposer should be able to condition on "rejected: leakage" as well. Please record screen rejections (with no measured dS/dC) in the state, and surface the reasons in the report detail even when the batch proceeds.

7. Outcome labelling. "floor" in decision.reason maps every other non-admissible reason to COST_RULE, including invalid or missing candidate measurement (confirmed with a NaN count), invalid historical-best score, and any future domain guard violated. Please map explicitly, and fall back to the existing generic rejection category for invalid or guard reasons rather than mislabelling them as cost decisions.

Nits

• In policies.py, the Policies.screen comment says "every card that passes the trust region and staleness filter is merged", but the screen runs before staleness. The aggregator comment has it right.
• RRsiCritic uses plain substring matching for leakage, so a short protected string ("1", a common word) rejects almost everything. Consider word-boundary matching or a minimum length, and mention it in the docstring.
• The model reviewer fails open on any exception or unparsable reply. That's reasonable, but it would be useful to count it (for example critic_reviewer_errors) so a silently broken reviewer shows up.
• restore() swallows every exception from the acceptance hook. Since this is the path that "preserves S*", a corrupted payload that silently resets the floor to base_score deserves at least a warning.

Tests and docs

The 16 new tests pass locally, and the end-to-end offline fixtures (leak gives critic-rejected with no commit, the untouched path reports zeros, resume keeps the floor) are good. Please add regression tests for items 1, 2, 3 and 6. The full suite on this head (3.13, local) has 4 failures, all in tests/test_checkpoint.py as above, and CI shows the same on 3.9/3.10. The CHANGELOG and docs/algo-rrsi.md updates read well. Once the round and cost semantics change, the deviations list will need a refresh.

Happy to re-review once these are addressed.

[HugoXK]

All three findings are addressed on the new head 872d707:

1. checkpoint() regression fixed. Both checkpoint() and restore() read the acceptance policy through getattr(self, "acceptance_policy", None), so an aggregator built without that attribute (the Aggregator.__new__ test cases) skips the optional acceptance-state hook instead of raising. All 43 test_checkpoint.py cases pass; the reference-aggregator roundtrip fails without the fix.

2. The anneal now advances per merge, not per accept. RRsiState.record_merge() advances round and appends S* to progress; RRsiAcceptance.accept calls it on every outcome (accept and both rejection branches), and the aggregator's screen-rejected path calls an on_screen_reject hook on the acceptance policy so a critic-rejected batch advances the schedule too. Regressions: ten straight rejections anneal the budget below b_max and trip stalled() (previously round stayed 0, budget 5, stalled False); screen rejections advance the round.

3. Cost is now a per-rollout mean, not a batch sum. _card_cost averages EvidenceCard.cost_tokens, so three 100-token cards read the same as one (the batch sum produced dC=+2.0 and rejected on bucket size). docs/algo-rrsi.md records the deviation: it is still the proposing rollout's usage (incumbent-side), so the cost rule is a documented proxy until candidate-side held-out telemetry exists; missing/zero telemetry yields dC=0 as before.

test_checkpoint.py (43), test_rrsi_integration.py (19) and test_aggregator.py all pass locally.

[Birfy]

Rechecked 872d707437adf7f7e99a3d4835e4099b720c52cb: the absent-policy checkpoint/restore regression, rejected-decision annealing, and batch-summed cost behavior are repaired in the scoped controls. All 198 focused shipped tests pass locally, including 43 checkpoint tests. Real Aggregator.ingest()/step() screen rejection advances once per batch with no measurement; one versus three 100-token cards both use mean 100 and accept with dC=0. The cost remains the documented proposer/incumbent-side proxy.

P2: record the accepted best score before appending progress. RRsiAcceptance.accept now calls record_merge() at line 546, before updating best_score on acceptance. Using RRsiState(RRsiConfig(stall_window=3), base_score=0.5) and the existing tests/test_rrsi_integration.py::_context fixture, reject twice with cand=(3,7), then accept cand=(8,2). Result: best_score=0.8, but progress=[0.5,0.5,0.5] and stalled()==True. The next stubbed RRsiProposalPolicy prompt incorrectly announces a stall, and checkpoint persists that stale history.

An external corrected-order control retains all three decision increments and ledger entries, records [0.5,0.5,0.8], and reports no stall before or after checkpoint/restore. Please append the post-decision S* on acceptance while preserving exactly one advance for rejections and screen-rejected batches. Add improve-after-rejection-streak and checkpoint/prompt assertions; the existing acceptance test checks the score and round but misses progress.

The new bounded controls yielded seven passes and this one failure; old-head controls separately confirm the original fixes discriminate. All checks were offline/synthetic. Exact-head docs are green; the test matrix is still running. Other items in the existing review remain outside this scoped verification; no formal approval is submitted.

[Birfy]

Status check at main a9e8f08: this branch now conflicts with main in CHANGELOG.md and agentdescent/metrics.py (both touched by the trajectory ledger in #248, which added the trajectories_* meter counters). No new push since the re-check at 872d707, so the P2 item from that comment (append the post-decision S* on acceptance so an improve-after-rejection-streak does not report a stall) is still open. Once rebased with that fix, I'll re-review the new head.

[HugoXK]

Rebased onto the current main (5ef95a8, includes #248 trajectory-ledger) -- the new head is dac42bb. The CHANGELOG/metrics conflicts were resolved by combining the trajectory counters with the RRSI critic counters.

The P2 finding is fixed: RRsiAcceptance.accept now advances the schedule after updating best_score on the accept branch, so an improvement that breaks a rejection streak records the post-decision S* (progress=[0.5, 0.5, 0.8], stalled()==False) instead of appending the pre-decision value and reporting a stall. Rejections and screen-rejected batches still advance exactly once with the unchanged S*. Regressions test_an_improvement_after_a_rejection_streak_is_not_a_stall and ..._survives_checkpoint (progress round-trips through checkpoint/restore) both fail against the pre-decision append.

rrsi/checkpoint/trajectory (92) plus aggregator/evolution/async/API/metrics (91) tests pass locally.

[Birfy]

Re-review at dac42bb: the three blocking items are fixed

Verified against the prior REQUEST_CHANGES:

1. Checkpoint safety — checkpoint() / restore() now use getattr(self, "acceptance_policy", None). tests/test_checkpoint.py passes locally.
2. Per-merge anneal — record_merge() runs on every decision (including screen-reject via the aggregator hook). test_a_streak_of_rejections_anneals_the_budget_and_trips_stall and the post-decision S* regressions pin the stuck-search case; accept now appends the post-decision S*.
3. Batch-independent cost — _card_cost is a per-rollout mean, documented as a proxy / deviation until candidate-side held-out cost exists.

Locally on Python 3.13, tests/test_rrsi_integration.py + tests/test_checkpoint.py are green. Exact-head CI is green.

Remaining non-blocking: novelty still maps via default_components path segments rather than the pinned STRUCTURAL_COMPONENTS / #253 vocabulary (examples/rrsi/components.py is now on main). Declared as a deviation is fine for this PR; a follow-up that reuses K would make the within-band novelty branch reachable on real layouts. Also: the branch conflicts with main after #253/#264 — please merge main in before landing.

Approving the blocking fixes.

[Birfy]

Status check (four-hour routine, 2026-10-16 ~12:30 CST): still APPROVED from the dac42bb re-review; no new pushes since. Remains CONFLICTING vs current main (751b8dd, includes #287 and other Fixed entries). Not merge-ready until rebased/merged with main — please resolve conflicts (likely CHANGELOG + selection touchpoints) and we can re-check CI.

[Birfy]

主分支已有较多更新，当前 PR 与 main 存在冲突。请 rebase/merge main 并解决冲突后推送，我们会尽快合入。谢谢！

[HugoXK]

Rebased onto the current main (9053bc0, includes #252 RRSI history port and #276 CladeThompson, which touch the same metrics/selection/algo-rrsi surfaces) -- the new head is 05a4c40. The conflicts were resolved by merging the RRSI critic_checked/critic_rejected counters alongside the now-merged governance (invariant_checked/invariant_rejections) and trajectory counter groups, and by recording both the history-module fixtures and the integration fixtures in docs/algo-rrsi.md.

All previously-reviewed fixes are intact: absent-policy checkpoint/restore safety, per-merge anneal (including the screen-rejected path), the post-decision S* append on accepts (improve-after-rejection-streak is not a stall), and the batch-size-independent per-rollout cost mean. rrsi selection/schedule/history/integration + checkpoint (143) and aggregator/evolution/async/API/metrics/trajectory/multi-head/invariants (165) tests pass locally.

[Birfy]

P2 — The newly integrated invariant-rejection path bypasses RRSI annealing (source-only review at 05a4c400844b5548227c0c79d01e3d43833b0dd3).

The rebase adds an early VIOLATES_INVARIANT return before RRsiAcceptance.accept runs. Unlike the all-screen-rejected path, it never notifies the acceptance policy. Consequently, proposals that pass the RRSI critic but repeatedly violate a declared contract leave state.round and progress unchanged: the edit budget stays at b_max and stall exploration never activates. This reintroduces the rejected-decision freeze through the new governance integration; the earlier acceptance/screen-rejection fixes remain intact.

Proposed regression: combine the ordinary RRSI bundle with SingleSlot and Contract(invariants=(Invariant('mutable_keys', ('other',)),)), using a short one-op value proposal that passes the critic. After a streak of violates-invariant merge reports, assert exactly one schedule advance per rejected merge, unchanged S*, and eventual stall detection, without inventing reward/cost measurements. Route this pre-measurement rejection through an appropriate outcome hook. Default runs without RRSI plus rejecting invariants are unaffected. This is a source-derived scenario, not a locally executed reproduction.

[Birfy]

Review at 05a4c40: prior blocking fixes look good; one new blocker

Fresh pass after the rebase onto current main. The three items from the earlier REQUEST_CHANGES still look fixed on this head:

1. Checkpoint safety — checkpoint() / restore() use getattr(self, "acceptance_policy", None) plus optional checkpoint_state / restore_state hooks.
2. Per-merge anneal — record_merge() / on_screen_reject() advance the schedule on accept, reject, and screen-reject; post-decision S* regressions are present.
3. Batch-independent cost — _card_cost is a per-card mean token proxy, documented as a deviation.

CI on this head is green (build + test 3.9–3.13). ScreenPolicy seam placement (before staleness / held-out) and the declared deviations in docs/algo-rrsi.md look right.

Must fix: changelog fragment convention

main now uses changelog fragments (#295 / #296). PRs must not edit CHANGELOG.md; they add changelog.d/<n>-<slug>.md instead (see changelog.d/README.md). This branch still appends the RRSI bullet directly under [Unreleased] in CHANGELOG.md and carries no fragment under changelog.d/.

Please:

• Revert the CHANGELOG.md edit relative to main.
• Add something like changelog.d/254-rrsi-integration.md (or the tracking issue number) with category Added and the entry text (mention (#254) / #221 as appropriate).

Until that lands, this will conflict with the weekly release path and with every other fragment-based PR.

Non-blocking (unchanged from prior review)

Novelty still maps via default_components path segments rather than the pinned STRUCTURAL_COMPONENTS / #253 vocabulary. Declared as a deviation is fine for this PR.

Happy to approve once the fragment is in place.

[Birfy]

Re-review after maintainer push (65b5c7e)

Addressed the open CHANGES_REQUESTED items on this head:

1. Changelog fragment — CHANGELOG.md matches main; entry moved to changelog.d/254-rrsi-integration.md. python -m tools.changelog_fragments check is green.
2. Invariant refusal anneal (P2) — Aggregator._prepare's violates-invariant path now calls the acceptance policy's on_screen_reject hook (same per-merge accounting as an all-screen-rejected batch). Regression: test_invariant_rejections_advance_the_rrsi_schedule.

Merged origin/main (includes #298). Local: rrsi-integration + checkpoint + invariants + changelog-fragments green.

Approving; arming auto-squash when CI is green.

[Birfy]

Maintainer push on feat/rrsi-integration (65b5c7e):

• Merged current main
• Moved Unreleased RRSI bullet → changelog.d/254-rrsi-integration.md (no direct CHANGELOG.md edit)
• Invariant refusals now advance the RRSI anneal/stall schedule (same hook as screen-reject)
• Tests: rrsi-integration / checkpoint / invariants / changelog-fragments green locally

Auto-squash armed once CI is green.