Repository navigation
feat(cyborg): implement reward, objective, and outcome projection #18
Description
Activity
🛠️ Picked up by /implement - driver codex, branch
18-reward-outcome-projection, 2026-08-01T17:32:56.088Z.gc workflow phase recorded:
preflight(issue #18). Posted by the MCP server to enforce ordering between workflow steps (issue #794 MVP-2). Do not edit or delete — used by downstream tools to gate phase prerequisites.gc workflow phase recorded:
plan(issue #18). Posted by the MCP server to enforce ordering between workflow steps (issue #794 MVP-2). Do not edit or delete — used by downstream tools to gate phase prerequisites.Implementation plan
-
Bind the native fact boundary and scenario projection
- Update the generated in-repo CybORG scenario payload in
src/raes_adapters/cyborg/scenario.pyto select the pinned Scenario2 Blue and Red reward calculators while keeping Green unscored and retaining the existing no-network-service/non-equivalence boundary. - Extend the verified driver/provisioner transaction so the driver strictly projects finite scalar step rewards and source calculator host components into backend-private facts with portable run, episode, logical-step, action, target, source-row, and derivation joins.
- Stage those facts during the effectful native turn and commit them only after participant, workflow, observation, and time validation succeeds; reset, failure, quarantine, replacement, and cleanup discard uncommitted or prior-episode accumulators.
- Update the generated in-repo CybORG scenario payload in
-
Compose the published RAES evaluator surface
- Add a CybORG evaluator over
EvaluationPlan/EvaluationOp, compiled result/execution contracts,EvaluationResultStateModel,EvaluationHistoryEventModel, andPropositionTruthResultModel. - Emit typed capture specifications, redacted evidence records, and revisioned derived measures. Keep per-step components, cumulative Blue score, proposition/objective satisfaction, terminal state, workflow completion, participant outcomes, conformance, and replication claims distinct.
- Bind supported proposition truth only through admitted source-ledger-backed evidence requirements and portable target coordinates. Missing, stale, unsupported, or non-terminal facts produce unknown/unsupported truth, missing measures, failed checks, and explicit limitations rather than zero/false.
- Record the RAES 2.0.0 run-artifact export gap explicitly; typed evaluator accessors demonstrate closed model construction but will not be represented as a new adapter protocol or metadata escape hatch.
- Add a CybORG evaluator over
-
Declare only implemented capabilities
- Compose the evaluator into the CybORG target, add exact evaluator capability and contract declarations to the manifest, and preserve lazy simulator imports, closed target configuration, one distribution, and one lock.
- Update the README and the preflight-authored guardrail navigation/outcome without changing
raes_adapters.base, RAES schemas, SDL vocabulary, conformance authority, or release metadata.
-
Drive the implementation with regression tests
- First add failing unit/integration tests for calculator selection, source sign, component/total consistency, cumulative ordering, terminal-step inclusion, restore cost, persistent impact, reset carry-over, replay/double-count prevention, objective/evidence joins, unsupported facts, lifecycle contracts, and participant-view leakage.
- Extend
tools/verify_cyborg_qualification.pyand qualification expectations so the clean patched pinned source executes actual CybORG Scenario2 reward paths: compromise, critical impact, a second persistent-impact step, Restore-1, cumulative totals, and the adapter evaluator projection. - Run targeted CybORG tests, the native qualification verifier, and the canonical
uv tool run --from 'nox[uv]==2026.4.10' nox -f noxfile.py -s verifygraph. The issue is requirement-free in Ground Control because its body has no formal## RequirementsUID section; governance will use that recorded issue contract rather than inventing a UID binding.
Acceptance mapping
- Pinned component and cumulative arithmetic: driver projection tests plus actual Scenario2 native qualification smoke.
- Source/run/episode/action/derivation joins: typed evidence and derived-measure tests.
- Unsupported facts and claim weakening: proposition/result failure and limitation tests.
- Sign, aggregation, terminal, Restore, persistence, and reset regressions: focused unit cases and pinned native sequence.
- Participant isolation: participant retrieval/action-result leakage tests.
- Score/conformance/equivalence separation: manifest/result/artifact assertions and guardrail documentation.
-
gc_codex_review — cycle 1 of 1 (pre-push) on issue #18 (branch
18-reward-outcome-projection)
Diff mode: inline — the complete diff was supplied in one promptCore review
Verdict:
ship-with-fixesThe change chooses the right seam: a backend-private, post-turn projection is staged by the provisioner and the evaluator constructs RAES-owned result, truth, and evidence models without exposing native state. However, that commit boundary is not mandatory for every driver, and evaluator reconciliation is not shaped for incremental plans: unchanged assertions are excluded from the assertion inputs while unchanged objectives are still recomputed. Those two gaps violate the intended all-or-nothing and lifecycle semantics.
Blocking findings (2):
- [one-off] Require reward projection before accepting an aggregate turn —
src/raes_adapters/cyborg/provisioner.py:234
The projection hook is optional: a driver withoutproject_evaluationcan successfully execute and commit a portable action, but no evaluation fact is staged. The manifest nevertheless advertises evaluator support, and later evaluation will see an empty history rather than quarantining the effectful turn. Make projection a required capability for this target (or fail and quarantine when it is absent) so action, observation, and evaluator facts retain the same commit boundary. - [one-off] Preserve assertion outcomes when reconciling unchanged evaluation resources —
src/raes_adapters/cyborg/evaluator.py:175
On an incremental evaluation plan,UNCHANGEDassertions are skipped beforeassertion_outcomesis populated, but every non-deleted objective—including unchanged objectives—is recomputed afterward. Consequently, a previously ready objective is overwritten as failed once its assertions arrive unchanged in a later plan. Seed outcomes from the snapshot's existing proposition truth results, or evaluate the needed unchanged assertions before recomputing dependent objectives.
Security review
Verdict:
shipThis change introduces a backend-private, post-commit evaluation projection seam: native reward data is reduced to finite allowlisted facts under the existing session lock, then mapped into typed RAES evaluation and evidence models. It touches evaluator capability declaration, transactional participant-turn completion, and artifact exposure, but adds no network route, authentication boundary, dynamic loading path, or raw native serialization path. The design preserves the important evaluator-versus-participant isolation seam and is shaped to accommodate additional reviewed source mappings without widening the portable boundary.
No blocking findings.
- [one-off] Require reward projection before accepting an aggregate turn —
gc_codex_review pre-push cycle 1 of 1 complete for issue #18 on branch '18-reward-outcome-projection'. Posted by the MCP server to enforce the pre-push hard-cap-1 contract (issues #796, #804, #906). Do not edit or delete — used by the next
gc_codex_review(uncommitted) invocation to count cycles.Review decision record — codex cycle 1 (issue #18)
Reviewer: codex
Cycle: 1Architectural read:
Core reviewer: The change chooses the right seam: a backend-private, post-turn projection is staged by the provisioner and the evaluator constructs RAES-owned result, truth, and evidence models without exposing native state. However, that commit boundary is not mandatory for every driver, and evaluator reconciliation is not shaped for incremental plans: unchanged assertions are excluded from the assertion inputs while unchanged objectives are still recomputed. Those two gaps violate the intended all-or-nothing and lifecycle semantics.
Security reviewer: This change introduces a backend-private, post-commit evaluation projection seam: native reward data is reduced to finite allowlisted facts under the existing session lock, then mapped into typed RAES evaluation and evidence models. It touches evaluator capability declaration, transactional participant-turn completion, and artifact exposure, but adds no network route, authentication boundary, dynamic loading path, or raw native serialization path. The design preserves the important evaluator-versus-participant isolation seam and is shaped to accommodate additional reviewed source mappings without widening the portable boundary.
Blocking findings: 2
Finding 1 —
one-off- ID:
F1 - Title: [core] Require reward projection before accepting an aggregate turn
- Location:
src/raes_adapters/cyborg/provisioner.py:234 - Decision: fix
- Rationale: The projection hook is optional: a driver without
project_evaluationcan successfully execute and commit a portable action, but no evaluation fact is staged. The manifest nevertheless advertises evaluator support, and later evaluation wi…
Finding 2 —
one-off- ID:
F2 - Title: [core] Preserve assertion outcomes when reconciling unchanged evaluation resources
- Location:
src/raes_adapters/cyborg/evaluator.py:175 - Decision: fix
- Rationale: On an incremental evaluation plan,
UNCHANGEDassertions are skipped beforeassertion_outcomesis populated, but every non-deleted objective—including unchanged objectives—is recomputed afterward. Consequently, a previously ready object…
- ID:
gc_test_quality_review cycle 1 of 1 — issue #18
Reviewer: test-quality (claude-sonnet-5 via gc_test_quality_review)
Branch:18-reward-outcome-projection
Cycle: 1 / 1
Findings: 2Finding 1 — [critical]
tests/test_cyborg_execution.py::test_source_driver_rejects_invalid_native_reward_projectionProblem: The parametrized negative-test list has 3 cases (NaN reward, unmapped native hostname, Blue component/total disagreement), all invoked with
external_address=_SLEEPand Red rewards/breakdown left mutually consistent (both zero/empty in every case). Two arithmetic-consistency guards newly added in_project_evaluation_turn— the Red reward-vs-breakdown cross-check (red_difference = totals[_RED] - component_totals[_RED]) and the Restore-specific action-cost check (external_address == _RESTORE and not math.isclose(action_cost, -1.0, ...)) — are never driven into their rejecting branch by any test.
Why it matters: If either guard were deleted, weakened, or had its threshold/sign changed (e.g. comparing against the wrong participant, or checking -2.0 instead of -1.0 for Restore), every currently-passing test would still pass: the one all-valid case (test_source_driver_projects_closed_reward_and_terminal_facts) already satisfies both checks by construction, and the 3 negative cases never touch Red or Restore. This is precisely the 'component/total disagreement' and 'restore charged at the wrong step' failure mode the project's own design doc (docs/decisions/cyborg-evaluation-projection-guardrails.md) says tests must independently catch — a wrong reward/evidence record could reach the evaluator's proposition-truth and evidence-record output undetected.
Fix: Add two more parametrize cases to test_source_driver_rejects_invalid_native_reward_projection (or a sibling test): (1) Red get_rewards() total that disagrees with get_reward_breakdown('Red')'s summed components, and (2) external_address=_RESTORE with an action_cost that isn't exactly -1.0 (e.g. breakdown implies -2.0 or 0.0 Blue cost). Both should assert pytest.raises(ValueError).Finding 2 — [critical]
tests/test_cyborg_execution.py::test_evaluator_preserves_unchanged_assertion_outcomes_during_reconciliationProblem: The test calls evaluator.start() twice with the identical underlying committed facts (action.snapshot's committed_evaluation() never changes between the two calls), marks the assertion/proposition/objective operations ChangeAction.UNCHANGED on the second call, and then only asserts the final objective is ready/passed=True. Because the facts are unchanged, recomputing the assertion from scratch would yield the exact same truth result as preserving the prior snapshot value — the test never inspects reconciled.changed_addresses (which should exclude the UNCHANGED assertion address) or varies the facts to make preservation and recomputation diverge.
Why it matters: If the 'if operation.action == ChangeAction.UNCHANGED: continue' skip in CyborgEvaluator._apply were removed — causing every assertion to be recomputed on every reconciliation regardless of its declared change action — this test would still pass, since recomputing against the same facts produces an identical result. The one behavior the test's name promises (that UNCHANGED entries are preserved rather than recomputed) is exactly what it cannot detect a regression in.
Fix: Either (a) assert that reconciled.changed_addresses does not include the UNCHANGED assertion/proposition/objective addresses (only the new condition-binding), or (b) change the committed facts between the two evaluator.start() calls (e.g. commit a second turn with a different confidentiality value) and assert the UNCHANGED assertion's proposition_truth_result still reflects the original evidence_refs/temporal_context rather than the new facts.gc_test_quality_review cycle 1 of 1 complete for issue #18 on branch '18-reward-outcome-projection'. Posted by the MCP server to enforce the gc_test_quality_review hard-cap-1 contract (issue #884 follow-up, default lowered in #906). Do not edit or delete — used by the next
gc_test_quality_reviewinvocation to count cycles.Review decision record — test-quality cycle 1 (issue #18)
Reviewer: test-quality
Cycle: 1Architectural read:
This PR adds the CybORG evaluator seam (new
evaluator.py, plus the_project_evaluation_turn/commit_evaluationplumbing threaded throughdriver.py,provisioner.py, andparticipant_runtime.py). The shape is correct and matches the repo's established pattern: theSourceInstalledCyborgDriverremains the sole layer that touches native reward surfaces, projection is committed atomically with the rest of the aggregate turn under the existing transaction/quarantine machinery, and the evaluator only ever reads sanitized, allowlisted facts off the provisioner — exactly the seam the accompanying design doc (cyborg-evaluation-projection-guardrails.md) prescribes. TheFakeExecutionDriver/SourceInstalledCyborgDriversplit is the right unit-test boundary (driver-internals tests substitute fakeCybORG.Shared.Actions/CybORG.Sharedmodules via monkeypatchedimport_module, rather than requiring the real native package; qualification-level truth against the real simulator is captured separately inqualification.jsonvia the out-of-bandtools/verify_cyborg_qualification.py). Given the design doc's own explicit acceptance bar — "tests must independently catch sign inversion, ... restore charged at the wrong step, ... component/total disagreement... A total that happens to match while its components or joins are wrong is a failure" — the test suite under-delivers on exactly that bar in two places: the new arithmetic-consistency guards in_project_evaluation_turnare only positively exercised, and the evaluator's UNCHANGED-preservation path is asserted only in a scenario where preservation and blind recomputation are indistinguishable.Blocking findings: 2
Finding 1 —
class(2 instances)- ID:
F1 - Title: (no title)
- Decision: fix
- Rationale: Addressed by next cycle
- Instances:
src/raes_adapters/cyborg/driver.py: red_difference consistency check (Red get_rewards() vs get_reward_breakdown() disagreement) — untested rejection pathsrc/raes_adapters/cyborg/driver.py: Restore-specific action_cost == -1.0 check — untested rejection path
Finding 2 —
one-off- ID:
F2 - Title: (no title)
- Decision: fix
- Rationale: Addressed by next cycle
- ID:
Pre-PR base synchronization
- Source:
refs/remotes/origin/devat36bdb67f888c826a86ac15dd76c585433f5b91db - Outcome:
already_current - Published feature head:
a2c90fc6227123a172a3d0333c13c1594ccee974 - Verified tree:
1a44332bd461557bb774d49f96a789c4ce9d5170
- Source:
Pre-PR base synchronization
- Source:
refs/remotes/origin/devat36bdb67f888c826a86ac15dd76c585433f5b91db - Outcome:
already_current - Published feature head:
03b4480ce5b934fec9adf8c370a793583f6e762c - Verified tree:
d79b06ee83de17efa6d19345fc3d008a49bc77f1
- Source:
Pre-PR base synchronization
- Source:
refs/remotes/origin/devat36bdb67f888c826a86ac15dd76c585433f5b91db - Outcome:
already_current - Published feature head:
fdbd8b29b09636c98c83f1f7260f22a814f23695 - Verified tree:
572abaea0efc20b2c0bcdf0b0c3c19ffac44d3c5
- Source:
Ready for review — issue #18
PR: #64
Plan: #18 (comment)Outcome
CybORG runs now project committed reward facts into evaluator-only, traceable RAES evidence and objective results without exposing native simulator truth to participants. The pinned native qualification proves the source reward behavior and adapter projection against actual CybORG.
Project CybORG reward totals and components into typed RAES evaluation evidence, with real pinned-simulator qualification and fail-closed commit semantics.
Files changed
Added:
docs/decisions/cyborg-evaluation-projection-guardrails.mdsrc/raes_adapters/cyborg/evaluator.py
Modified:
mkdocs.ymlsrc/raes_adapters/cyborg/__init__.pysrc/raes_adapters/cyborg/driver.pysrc/raes_adapters/cyborg/manifest.pysrc/raes_adapters/cyborg/participant_runtime.pysrc/raes_adapters/cyborg/provisioner.pysrc/raes_adapters/cyborg/qualification.jsonsrc/raes_adapters/cyborg/scenario.pysrc/raes_adapters/cyborg/target.pytests/test_cyborg_execution.pytests/test_cyborg_provisioner.pytests/test_cyborg_qualification.pytools/verify_cyborg_qualification.py
Reviews
- codex: Two one-off findings were fixed: reward projection is mandatory after an effectful turn, and unchanged assertion outcomes are preserved during evaluator reconciliation.
- test-quality: Arithmetic guards now independently cover Red breakdown consistency and Restore action cost, and incremental reconciliation asserts the exact changed-address set.
Traceability reconciliation
- Pending — requirement status transition and IMPLEMENTS/TESTS reconciliation run in Phase E once the PR merges.
Status
- CI: ✅ green
- SonarCloud: ✅ passed
- PR ready for user review and merge. Ground Control reconciliation (requirement status + traceability) runs on merge (Phase E).
gc workflow phase recorded:
traceability_reconciled(issue #18). Posted by the MCP server to enforce ordering between workflow steps (issue #794 MVP-2). Do not edit or delete — used by downstream tools to gate phase prerequisites.no in-scope requirements; touched-files audit clean.
Final report — issue #18 complete
PR: #64
Plan: #18 (comment)Outcome
CybORG runs now project committed reward facts into evaluator-only, traceable RAES evidence and objective results without exposing native simulator truth to participants. The pinned native qualification proves the source reward behavior and adapter projection against actual CybORG.
Project CybORG reward totals and components into typed RAES evaluation evidence, with real pinned-simulator qualification and fail-closed commit semantics.
Files changed
Added:
docs/decisions/cyborg-evaluation-projection-guardrails.mdsrc/raes_adapters/cyborg/evaluator.py
Modified:
mkdocs.ymlsrc/raes_adapters/cyborg/__init__.pysrc/raes_adapters/cyborg/driver.pysrc/raes_adapters/cyborg/manifest.pysrc/raes_adapters/cyborg/participant_runtime.pysrc/raes_adapters/cyborg/provisioner.pysrc/raes_adapters/cyborg/qualification.jsonsrc/raes_adapters/cyborg/scenario.pysrc/raes_adapters/cyborg/target.pytests/test_cyborg_execution.pytests/test_cyborg_provisioner.pytests/test_cyborg_qualification.pytools/verify_cyborg_qualification.py
Reviews
- codex: Two one-off findings were fixed: reward projection is mandatory after an effectful turn, and unchanged assertion outcomes are preserved during evaluator reconciliation.
- test-quality: Arithmetic guards now independently cover Red breakdown consistency and Restore action cost, and incremental reconciliation asserts the exact changed-address set.
Traceability reconciliation
- IMPLEMENTS / TESTS / DOCUMENTS added: 0
- Links updated: 0
- Stale links removed: 0
No formal in-scope requirements exist for issue 18. All 15 merged files and the issue artifact were reverse-checked; no pre-existing traceability links were present, so reconciliation required no link changes.
Status
- CI: ✅ green
- SonarCloud: ✅ passed
- PR ready for user review and merge.
Objective
Project CAGE-2 evaluation facts into RAES evidence without collapsing distinct scientific concepts into one simulator score.
Scope
Acceptance criteria
References