Repository navigation
feat(replay): settled-evidence pool gets a consumer - ReplaySampler + ProposalContext.rejected - #193
Conversation
… ProposalContext.rejected + end-to-end verification ReplaySampler (sampling.py), ProposalContext.rejected (policies.py), ReplayAwareProposal (replay.py), set_settled_consumer + recent_settled (aggregator.py), EvolutionResult.replay, bench/ab_replay.py. Sync and async paths wired. 35 tests. Full suite exit 0.
compare() takes ArmResult, not Workload. _run_arm builds ArmResult with distinct arm names (control/replay) so compare() groups them correctly. Verified end-to-end with --budget-rollouts 4.
Birfy
left a comment
There was a problem hiding this comment.
Blocking: _replay_stats checks only for a settle method, then calls settled_counts unconditionally. A valid custom TaskSampler that also exposes settle but has no settled_counts now raises AttributeError at the end of evolve or async_evolve; I reproduced this with a minimal sampler. Please restrict replay statistics to ReplaySampler (or guard the optional stats API) and add a regression test. The focused sampling and aggregator tests otherwise pass locally (45 tests), and CI is green.
hasattr(sampler, 'settle') matched any custom sampler that also exposed settle but not settled_counts, crashing on the result line of evolve/ async_evolve. Switched to isinstance(sampler, ReplaySampler). Regression test: a custom sampler with settle() but no settled_counts() returns result.replay = None instead of AttributeError.
Birfy
left a comment
There was a problem hiding this comment.
Re-review: the custom sampler regression is fixed by restricting replay statistics to ReplaySampler, and the added regression test covers the original failure. I ran the sampling and aggregator tests locally (46 passed); CI passes on Python 3.9/3.11/3.12. Approved.
Rebuilt cleanly on upstream/main (PR Birfy#193 replay already merged), so this PR now contains ONLY the audit-drain changes: - AggregatorConfig.audit_drain_per_step (default 0 = old behaviour) - _drain_audit_queue: pops highest-priority queued audits after each finish_step, runs full_eval on (base_state, candidate), updates trust - _verifier_can_spend(n): getattr-based, VerifierProtocol-safe - _requeue via AuditScheduler.requeue: preserves original priority - AuditScheduler.requeue: push item back with original priority - EvolutionResult.audit_drained on sync + async paths - _audit submits (artifact_id, base_state, candidate) 3-tuple Verified: requeue preserves priority, meter reuses cas_conflicts. Tests: 4 new drain tests + existing audit tests pass. exit 0.
ReplaySampler + ProposalContext.rejected — make the settled-evidence pool a consumer
The problem
aggregator.py'ssettle()pool collected stale / oversized / CAS-conflict evidence cards. Nobody read it back. The docstring said "a fuller system would re-file the cards into the trajectory pool" and called it "a diagnostic ring, not a queue."ProposalContext's docstring said "'do not re-propose what was just rejected' is inexpressible."Two gaps, one root: the infrastructure existed and was wired but had no consumer.
What this PR adds
ReplaySampler(agentdescent/sampling.py) — aTaskSamplerthat subscribes to the settled pool viaaggregator.set_settled_consumer(sampler.settle)and up-weights tasks whose recent proposals were discarded. The replay bonus is additive and saturating (temperature×min(capped_at, count)), orthogonal toDifficultyWeighted's pass-rate signal: pass rate says "is this task hard?", settled count says "is this task being starved by the parallel scheduler?"temperature=0reproducesDifficultyWeightedidentically. Wired on both sync (evolve) and async (async_evolve) paths.ProposalContext.rejected(agentdescent/policies.py) — a newSequence[EvidenceCard]field carrying the most recent settled cards for the task being rolled out. The engine fills it via a mutable cell (the aggregator is built after the propose adapter;_settled_reader[0] = aggregator.recent_settledresolves the ordering).ReplayAwareProposal(agentdescent/replay.py) — aProposalPolicywrapper that readsctx.rejectedand, when non-empty, renders a summary (paths, base version, delta) into the context'soutputbefore delegating. A model-based inner policy that rendersoutput— the shippedLLMAgent.propose— sees "these were discarded, do not repeat them."Buffer.recent_settled(task, n=3)+Aggregator.recent_settled— filter the pool bytrajectory_refsmatching the task, O(256) per call.EvolutionResult.replay— reports{"settled": int, "picks": int, "replay_affected": int}when aReplaySampleris installed.Noneotherwise.bench/ab_replay.py— a working A/B script comparingDifficultyWeightedvsReplaySamplerat equal budget using thebaselinesharness.Backward compatibility
ReplaySampler(temperature=0)DifficultyWeightedReplaySamplerwith no aggregator wiredaggregatorwith nosettled_consumerNoneProposalContext.rejected()TaskSamplerprotocolEvolutionResult.replayNoneVerification
evolve()ctx.rejectedresult.replaypopulated{"settled": 8, "picks": 8, "replay_affected": 3}result.replay is Nonewithout samplerbench/ab_replay.pyruns end-to-endFiles
14 files changed, +859, -9
New:
agentdescent/replay.py(+73) —ReplayAwareProposalbench/ab_replay.py(+91) — A/B harnessModified:
agentdescent/sampling.py(+98) —ReplaySampleragentdescent/aggregator.py(+56) —set_settled_consumer,recent_settled,Bufferchangesagentdescent/policies.py(+6) —ProposalContext.rejectedagentdescent/evolution.py(+33) —_propose_via_policysettled_reader,_replay_stats,EvolutionResult.replayagentdescent/async_evolve.py(+4) — sampler + result wiringagentdescent/__init__.py(+2) — exportsagentdescent/evolvespec.py(+2) — registrydocs/sampling.md(+50) — ReplaySampler sectiondocs/api.md(+5) — regeneratedCHANGELOG.md(+10)tests/test_sampling.py(+327) — 24 teststests/test_aggregator.py(+32) — 1 test