feat(workforce): WR2 P5 — grading store, scorecards, quantified promotion gate (D10)#463
Merged
Merged
Conversation
…tion gate (D10)
Implements the locked D10 design (ops/WORKER_RUNTIME_V2_DESIGN_2026-06-11.md):
- Append-only PG grade store: job_grade_events table (idempotent DDL +
drizzle schema), GradeEvent/GradeEventInput types, JobAdapter
recordGrades/listGrades (PG-native per the 2026-06-11 storage lock;
SQLite hard-fails like the rest of workforce, dual delegates to PG).
Assignment/template ids are denormalized from the run at write time so
scorecards query without joins. No update/delete paths — audit first.
- Pure gate math (src/infra/worker-grading.ts): buildScorecard rolls up
per-template and per-component; evaluatePromotionGate implements the
LOCKED numbers — ≥95% correct over ≥50 graded decisions sustained 2
weeks — with exact integer math (correct*100 >= total*95) and a
sustain clock that resets on any dip below the bar. The gate INFORMS
(eligible/passing/reasons); it never auto-promotes — stage changes
remain the operator's jobs.runs.review/assignments.update action.
- Gateway: jobs.grades.record (batch, verdict-validated),
jobs.grades.approveAll (one-click, grades every gradable component
correct — FEEDBACK ONLY, executes nothing, per D10),
jobs.grades.list, jobs.scorecard (scorecard + gate status). All behind
the existing strict-PG workforce guard; audit events emitted.
- Workforce Board: per-run grading panel (✓/±/✗ per decision component,
optional feedback, approve-all button, existing grades shown) and a
template Scorecard section with the promotion-gate chip
(eligible / passing—sustaining / not passing) plus gate reasons.
- Gradable components derive from the run itself (D9 proposed_action
events by tool + report), so grading plugs straight into the simulate
evidence trail; knowledge-retrieval grades ride the same store via
component naming ("knowledge:<doc>").
Verified: 14 gate-math tests (boundaries: 95% exact at 76/80, dip-reset,
sustain window), 6 gateway tests, adapter policy suites green, dashboard
tsc clean, tsc-since 0 net-new (189 baseline).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01T3dkQzz2mSrSNtGBE9cmXr
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Repository UI Review profile: CHILL Plan: Pro Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
…rder (review round) Two confirmed findings from the adversarial review of e2fc392: - CRITICAL: listGrades defaulted to LIMIT 5000 ascending — once a template accrued >5000 grades the scorecard/gate computed over a frozen OLDEST window: new wrongs never lowered the rate, dips never reset the sustain clock, and a frozen-passing window read "eligible" forever by wall-clock alone. listGrades now has NO default limit (the gate's prefix-walk needs the full ordered history); the wire-facing jobs.grades.list gets an explicit 500 default page instead. - MAJOR: recordGrades stamped one shared timestamp per batch with a random-UUID tiebreak, so whether a mixed-verdict batch transiently "dipped below the bar" (resetting the 14-day sustain clock) depended on insert-order randomness. Rows now get per-row +1ms offsets so persisted order equals the operator's batch order — deterministic gate math. Also: board's gradable-component mirror now includes metadata.report (matching gradableComponentsForRun), and the grades-recorded audit event carries a singular runId so eventLinkValues() links it into run traces. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01T3dkQzz2mSrSNtGBE9cmXr
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
WR2 P5 — the locked D10 design, implemented
Resumes the Worker Runtime v2 lane where it stopped at P4 (complete + live-verified 2026-06-14). Design source:
ops/WORKER_RUNTIME_V2_DESIGN_2026-06-11.md§D10 — numbers and storage locked by Jason 2026-06-11.Grade store (append-only, PG-native)
job_grade_eventstable (idempotent DDL in the adapter init + drizzle schema):{run, assignment, template, component, verdict ∈ correct|needs_change|wrong, feedback, grader, created_at}. Assignment/template ids denormalized from the run at write time; no update/delete paths — audit trail first.JobAdapter.recordGrades/listGrades— PG implements, dual delegates, SQLite hard-fails like all workforce paths (PG-canonical policy).Promotion gate (pure math, exact integers)
src/infra/worker-grading.ts:buildScorecard(per-template + per-component rollups) andevaluatePromotionGate— ≥95% correct over ≥50 graded decisions sustained 2 weeks, computed ascorrect*100 >= total*95(no float at the boundary). The sustain clock starts when the passing condition is entered and resets on any dip below the bar.jobs.runs.review/jobs.assignments.updateactions.Gateway
jobs.grades.record(batch, verdict-validated) ·jobs.grades.approveAll(one-click: grades every gradable componentcorrect— feedback only, executes nothing, per D10) ·jobs.grades.list·jobs.scorecard(rollup + gate status). All behind the strict-PG workforce guard, audit events emitted.Workforce Board
proposed_actionevents + report), optional feedback, approve-all, existing grades shown.eligible for next stage/passing — sustaining/not passing) and the gate's own reasons.Verification
tsc-since0 net-new (189 baseline); dashboardtscclean.Not in this PR (per D10 phasing)
Position-ledger retrieval (D11 read path, P5/P6), knowledge-retrieval grading UI (rides the same store via
knowledge:<doc>components), skill self-update proposals (P6). D9 live proof with a write-granting role happens post-deploy.🤖 Generated with Claude Code
https://claude.ai/code/session_01T3dkQzz2mSrSNtGBE9cmXr