Skip to content

feat(replay): settled-evidence pool gets a consumer - ReplaySampler + ProposalContext.rejected - #193

Merged
Birfy merged 4 commits into
Birfy:mainfrom
HugoXK:feat/replay-sampler
Oct 7, 2026
Merged

Birfy merged 4 commits into
Birfy:mainfrom
HugoXK:feat/replay-sampler

Conversation

@HugoXK

@HugoXK HugoXK commented Sep 18, 2026

Copy link
Copy Markdown
Collaborator

ReplaySampler + ProposalContext.rejected — make the settled-evidence pool a consumer

The problem

aggregator.py's settle() pool collected stale / oversized / CAS-conflict evidence cards. Nobody read it back. The docstring said "a fuller system would re-file the cards into the trajectory pool" and called it "a diagnostic ring, not a queue." ProposalContext's docstring said "'do not re-propose what was just rejected' is inexpressible."

Two gaps, one root: the infrastructure existed and was wired but had no consumer.

What this PR adds

ReplaySampler (agentdescent/sampling.py) — a TaskSampler that subscribes to the settled pool via aggregator.set_settled_consumer(sampler.settle) and up-weights tasks whose recent proposals were discarded. The replay bonus is additive and saturating (temperature × min(capped_at, count)), orthogonal to DifficultyWeighted's pass-rate signal: pass rate says "is this task hard?", settled count says "is this task being starved by the parallel scheduler?" temperature=0 reproduces DifficultyWeighted identically. Wired on both sync (evolve) and async (async_evolve) paths.

ProposalContext.rejected (agentdescent/policies.py) — a new Sequence[EvidenceCard] field carrying the most recent settled cards for the task being rolled out. The engine fills it via a mutable cell (the aggregator is built after the propose adapter; _settled_reader[0] = aggregator.recent_settled resolves the ordering).

ReplayAwareProposal (agentdescent/replay.py) — a ProposalPolicy wrapper that reads ctx.rejected and, when non-empty, renders a summary (paths, base version, delta) into the context's output before delegating. A model-based inner policy that renders output — the shipped LLMAgent.propose — sees "these were discarded, do not repeat them."

Buffer.recent_settled(task, n=3) + Aggregator.recent_settled — filter the pool by trajectory_refs matching the task, O(256) per call.

EvolutionResult.replay — reports {"settled": int, "picks": int, "replay_affected": int} when a ReplaySampler is installed. None otherwise.

bench/ab_replay.py — a working A/B script comparing DifficultyWeighted vs ReplaySampler at equal budget using the baselines harness.

Backward compatibility

Component Default Old behaviour?
ReplaySampler(temperature=0) temperature=0 yes — identical to DifficultyWeighted
ReplaySampler with no aggregator wired counts stay 0 yes
aggregator with no settled_consumer None yes — settle() unchanged
ProposalContext.rejected () yes — field exists, is empty
TaskSampler protocol unchanged yes
EvolutionResult.replay None yes

Verification

Item Result
ReplaySampler unit tests 18 passed
ReplayAwareProposal tests 4 passed
Aggregator settled_consumer test 1 passed
End-to-end through evolve() 2 passed — oversized diffs → settled → sampler sees them → proposer sees them via ctx.rejected
result.replay populated verified — {"settled": 8, "picks": 8, "replay_affected": 3}
result.replay is None without sampler verified
bench/ab_replay.py runs end-to-end verified
Existing genesis tests 121 passed, unchanged
Core regression (aggregator + API + sampling) 195 passed, 0 failed

Files

14 files changed, +859, -9

New:

  • agentdescent/replay.py (+73) — ReplayAwareProposal
  • bench/ab_replay.py (+91) — A/B harness

Modified:

  • agentdescent/sampling.py (+98) — ReplaySampler
  • agentdescent/aggregator.py (+56) — set_settled_consumer, recent_settled, Buffer changes
  • agentdescent/policies.py (+6) — ProposalContext.rejected
  • agentdescent/evolution.py (+33) — _propose_via_policy settled_reader, _replay_stats, EvolutionResult.replay
  • agentdescent/async_evolve.py (+4) — sampler + result wiring
  • agentdescent/__init__.py (+2) — exports
  • agentdescent/evolvespec.py (+2) — registry
  • docs/sampling.md (+50) — ReplaySampler section
  • docs/api.md (+5) — regenerated
  • CHANGELOG.md (+10)
  • tests/test_sampling.py (+327) — 24 tests
  • tests/test_aggregator.py (+32) — 1 test

xuke added 2 commits September 18, 2026 10:03
… ProposalContext.rejected + end-to-end verification

ReplaySampler (sampling.py), ProposalContext.rejected (policies.py),
ReplayAwareProposal (replay.py), set_settled_consumer + recent_settled
(aggregator.py), EvolutionResult.replay, bench/ab_replay.py.

Sync and async paths wired. 35 tests. Full suite exit 0.
compare() takes ArmResult, not Workload. _run_arm builds ArmResult
with distinct arm names (control/replay) so compare() groups them
correctly. Verified end-to-end with --budget-rollouts 4.

@Birfy Birfy left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blocking: _replay_stats checks only for a settle method, then calls settled_counts unconditionally. A valid custom TaskSampler that also exposes settle but has no settled_counts now raises AttributeError at the end of evolve or async_evolve; I reproduced this with a minimal sampler. Please restrict replay statistics to ReplaySampler (or guard the optional stats API) and add a regression test. The focused sampling and aggregator tests otherwise pass locally (45 tests), and CI is green.

xuke and others added 2 commits September 28, 2026 15:07
hasattr(sampler, 'settle') matched any custom sampler that also exposed
settle but not settled_counts, crashing on the result line of evolve/
async_evolve. Switched to isinstance(sampler, ReplaySampler).

Regression test: a custom sampler with settle() but no settled_counts()
returns result.replay = None instead of AttributeError.

@Birfy Birfy left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-review: the custom sampler regression is fixed by restricting replay statistics to ReplaySampler, and the added regression test covers the original failure. I ran the sampling and aggregator tests locally (46 passed); CI passes on Python 3.9/3.11/3.12. Approved.

@Birfy
Birfy merged commit ad613be into Birfy:main Oct 7, 2026
3 checks passed
HugoXK pushed a commit to HugoXK/agentdescent that referenced this pull request Oct 8, 2026
Rebuilt cleanly on upstream/main (PR Birfy#193 replay already merged), so
this PR now contains ONLY the audit-drain changes:

- AggregatorConfig.audit_drain_per_step (default 0 = old behaviour)
- _drain_audit_queue: pops highest-priority queued audits after each
  finish_step, runs full_eval on (base_state, candidate), updates trust
- _verifier_can_spend(n): getattr-based, VerifierProtocol-safe
- _requeue via AuditScheduler.requeue: preserves original priority
- AuditScheduler.requeue: push item back with original priority
- EvolutionResult.audit_drained on sync + async paths
- _audit submits (artifact_id, base_state, candidate) 3-tuple

Verified: requeue preserves priority, meter reuses cas_conflicts.
Tests: 4 new drain tests + existing audit tests pass. exit 0.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants