Repository navigation
Bind accepted-task cost feedback to complete per-candidate cohorts #722
Description
Summary
Same-run accepted-task cost is reduced to one scalar without selection-lane identity. That value is then used as observed cost for unrelated previous candidates, and it remains available when the cohort contains costly incomplete attempts.
Evidence and reproduction
On main fa0c79b9ae38eca2c59b2b1911becfe1a6b3c9ab:
src/supervise/accepted_task_cost.rs:31-38drops rows with unknown terminal results; lines 49-55 omit incomplete cost buckets from the numeric rollup. Lines 70-77 still return a numeric value from the complete subset.- The existing test at lines 138-148 combines a complete accepted attempt costing 12 with an incomplete accepted attempt containing 550 of known spend and asserts a value of 12. Incompleteness is counted, but that subset value remains available to selection.
src/supervise/scheduler.rs:2655-2661,2680-2706records assignment/attempt/result/costs without candidate/runtime/model/effort/task-shape identity; lines 2743-2772 aggregate all rows together.- Scheduler/selection propagation (
scheduler.rs:684-695,selection_bridge.rs:1126-1128,1312-1348) passes the scalar into role reselection.src/selection/selector.rs:1100-1106assigns it to each role's own previous candidate.
For a two-candidate fixture, distinct actual costs cannot be attributed through the current row type. For the existing 12-plus-550 fixture, the full two-acceptance cohort has a known lower bound of 281 per acceptance, while feedback uses 12 from one complete row.
Acceptance criteria
- Preserve exact task shape, role, candidate, runtime, model and effort in attempt-cost evidence and group feedback by the corresponding selection lane.
- Prevent one lane's observation from replacing another candidate's expected cost.
- Unknown terminal status or missing required buckets must make full-cohort feedback unavailable or explicitly bounded. A complete-subset descriptive average may remain separately labelled.
- Preserve known partial spend and incompleteness in reporting/provenance; do not discard failed/rejected/rework costs.
- Test two-lane isolation, complete-plus-incomplete cohorts, unknown outcomes with known spend, and rejected retries followed by acceptance.
Scope and verification
Source/test-definition inspection only; the existing test was not rerun. #631 concerns empirical qualification, and #695 concerns observed-model pricing attribution; neither preserves candidate cohort identity through this aggregation. No matching open Issue/PR covers this feedback defect.
Activity