Skip to content

test(cyborg): reproduce the public CAGE-2 protocol and emit equivalence evidence #22

Description

@Brad-Edwards

Objective

Demonstrate the adapter with a bounded, independently rerunnable reconstruction of the public CAGE Challenge 2 evaluation protocol.

Scope

  • Reconstruct the evaluation matrix from pinned primary sources: trial lengths 30/50/100; B-line, Meander, and Sleep red policies; random.seed(153); 1,000 episodes per condition; and the upstream-shipped blue baseline agents selected in the frozen run plan.
  • Preserve the upstream evaluator and score calculation as source facts while also emitting RAES objectives, evidence, derived measures, and run provenance.
  • Predeclare baseline agents, exclusions, failure handling, confidence-interval method, tolerances, and criteria for exact, bounded, or failed reproduction.
  • Drive the same authored scenario through the RAES reference backend for the equivalence tiers it genuinely supports. Unsupported runtime facts must weaken the claim rather than be synthesized.
  • Generate immutable inputs, per-run inventories, aggregate tables, validation reports, environment metadata, and a one-command reproduction path into a fresh operator-selected output.
  • Compare published and observed results without treating score similarity as conformance or semantic equivalence.

Acceptance criteria

  • The frozen matrix and all source identities/digests are machine-readable.
  • Every scheduled run is retained as valid, invalid, failed, or excluded; retries use new IDs.
  • Aggregate scores and 95% confidence intervals are recomputable from retained run evidence.
  • Authored-source, contract, execution-control, state/observation, outcome/evaluation, and disclosure tiers each have a pass/fail/weakened result.
  • Reference-backend support and limitations are reported separately from CybORG outcomes.
  • A clean environment can reproduce the aggregate artifact from an operator-supplied immutable export.
  • The report states the exact bounded claim and does not claim deterministic replay, backend identity, or scientific reproducibility beyond the evidence.
  • Tests are updated for the implemented behavior and the relevant targeted/native plus canonical verification commands are actually run and pass before acceptance.

References


Data ownership and capture layout

The adapter owns the apparatus and writes a complete machine-readable result to a fresh explicit output selected by the operator. Generated study data is not checked into the adapter repository, packaged in the distribution, or bound to a downstream repository or filesystem location.

The generated export contains:

  • protocol.json — the frozen matrix/oracle: conditions, agents, seeds, run counts, tolerances, and exact/bounded/failed reproduction criteria, frozen before mediated runs.
  • source-ledger.json — pinned upstream sources with versions and content digests, plus loss and exclusion disclosures.
  • runs/ — per-run inventory with never-reused IDs, terminal state, provenance, and evidence.
  • aggregates.json — aggregate measures with 95% confidence intervals, recomputable from runs/.
  • tiers.json — pass/fail/weakened results for authored-source, contract, execution-control, state/observation, outcome/evaluation, and disclosure, with cited evidence.
  • validation-report.md — clean-environment recomputation and the one-command validation path.
  • environment.json — exact software/source/pack/study identities and stated limitations.

Research projects and archives independently own retention and publication after validating an export. Export only reviewed, non-sensitive material; keep private, restricted, credential, provider-log, and native hidden-truth data out of portable artifacts.

Research purpose

The result supplies apparatus evidence for OpenRAE/research#14 and behavioral baseline outcome-reproduction evidence for OpenRAE/research#20. It is not a proof of submitted-agent identity, state equivalence, or general OpenRAE/CybORG research equivalence.

Activity

  1. added
    in-progressAn agent is actively working this issue via /implement
    on Aug 9, 2026
  2. Brad-Edwards commented on Aug 9, 2026

    @Brad-Edwards
    CollaboratorAuthor

    🛠️ Picked up by /implement - driver codex, branch 22-cage2-protocol-reproduction, 2026-08-09T20:06:09.296Z.

  3. Brad-Edwards commented on Aug 9, 2026

    @Brad-Edwards
    CollaboratorAuthor

    Execution obligation OBL-22-ARCH-PREFLIGHT-TIMEOUT — Opened

    Category: workflow
    Observed state: The first asynchronous architecture-preflight attempt returned job_failed after its hard timeout.
    Impact: The implementation plan cannot be posted or treated as architecture-cleared until the canonical preflight writes its required marker.
    Current obligation: Obtain a successful canonical architecture preflight before finalizing the issue 22 implementation plan.

    Evidence

    • Architecture preflight job job-msm8gqu5-1 exceeded the tool's 1,200,000 ms limit and was terminated without a result.
  4. Brad-Edwards commented on Aug 9, 2026

    @Brad-Edwards
    CollaboratorAuthor

    gc workflow phase recorded: preflight (issue #22). Posted by the MCP server to enforce ordering between workflow steps (issue #794 MVP-2). Do not edit or delete — used by downstream tools to gate phase prerequisites.

  5. Brad-Edwards commented on Aug 9, 2026

    @Brad-Edwards
    CollaboratorAuthor

    Execution obligation OBL-22-ARCH-PREFLIGHT-TIMEOUT — Resolved

    Category: workflow
    Observed state: The initial preflight attempt timed out; the retry completed successfully.
    Impact: The workflow can now proceed to codebase assessment and planning with architecture guidance in force.
    Current obligation: Obtain a successful canonical architecture preflight before finalizing the issue 22 implementation plan.

    Evidence

    • Retry job job-msm97cfa-2 completed successfully and wrote the required preflight phase marker.
    • The preflight returned binding guardrails and identified its durable design note.

    Disposition: fix
    Corrective action: Retried the canonical asynchronous architecture preflight with the same issue, project, and invocation checkout after the timed-out attempt.

    Verification

    • gc_codex_job reported status=done with result.phase_marker.phase=preflight.
    • Preflight summary identified docs/decisions/cyborg-cage2-protocol-reproduction-guardrails.md as the durable guidance.
  6. Brad-Edwards commented on Aug 9, 2026

    @Brad-Edwards
    CollaboratorAuthor

    gc workflow phase recorded: plan (issue #22). Posted by the MCP server to enforce ordering between workflow steps (issue #794 MVP-2). Do not edit or delete — used by downstream tools to gate phase prerequisites.

    Implementation plan

    Freeze the experiment before execution

    • Add a tracked, data-only revision at packages/cyborg_adapter/reproduction/cage2-26ce1c1-ccs-sleeper-v1/; keep all implementation under src/raes_adapters/cyborg and retain the repository's single pyproject.toml / uv.lock.
    • Pin the CAGE-2 commit and selected-file digests, qualified evaluator, public README leaderboard, paper, RAES/adapter/reference-backend versions, full authored pack/scenario, and the existing blue Sleep participant artifacts. Preserve the source disagreement: evaluation.py runs 100 episodes, while the published validation protocol reports 1,000.
    • Select the public CCS Sleeper baseline because it is both published and representable by the reviewed adapter Sleep participant without learned-policy or fallback-training bytes. Freeze the canonical matrix as trial lengths 30/50/100 × B-line/Meander/Sleep red policies × 1,000 episodes (9,000 admitted entries).
    • Record random.seed(153) as a one-time ordered-stream augmentation, not a per-episode reset or deterministic-replay claim. Keep every unbound simulator/policy stream explicit.
    • Predeclare the primary total-score estimand (sum of the nine condition means), descriptive condition means, deterministic decimal arithmetic/rounding, a two-sided normal-approximation 95% interval over independent condition means, eligibility and missing-data handling, no multiplicity correction for the single primary estimand, and exact/bounded/failed rules. Exact requires source-precision agreement; bounded requires the published and observed estimates to fall within each other's declared 95% bounds; otherwise the outcome comparison fails. Score agreement cannot upgrade any equivalence tier.

    Implement the backend-local study and bundle paths

    • Add src/raes_adapters/cyborg/reproduction.py to load bounded strict JSON, validate canonical RAES experiment/study/participant/admitted-plan models and digest joins, schedule deterministic sealed plan partitions with plan_batch_schedule(), and compose the existing CybORG target, participant lifecycle, evaluator, evidence, cleanup, provenance, and atomic-write helpers.
    • Extend src/raes_adapters/cli.py with two closed surfaces: a full native/reference study command that writes only to a fresh mode-0700 staging root, and an offline reproduce command that validates a frozen public bundle and recomputes its aggregate in a fresh output root without CybORG or network access.
    • Keep serial execution unless the sealed plan and live allocator evidence prove isolation. Do not resume, append, overwrite, allocate identities from time/PID, or let scheduling select conditions, identities, exclusions, randomness, or scores.
    • Retain every admitted entry as valid, invalid, failed, or excluded. Preserve failed attempts and cleanup evidence; retries get a new execution-attempt identity and never replace the original run.
    • Run the identical full authored scenario through CybORG and the RAES reference backend as separate apparatus/results. Unsupported reference policies, observations, reward facts, or temporal behavior produce cited weakened tiers, never synthetic facts.
    • Gate scientific execution on the exact reviewed Scenario2 artifact from adapters docs(cyborg): author the validated CAGE-2 (Scenario2) SDL document #77 and immutable participant/source digests. The current two-step smoke pack must never be substituted. A missing prerequisite produces explicit exclusions/weakened evidence rather than simulated results.

    Produce and verify the research bundle

    • Materialize protocol.json, source-ledger.json, partitioned sealed plans, per-run evidence/inventories, aggregates.json, tiers.json, environment.json, validation-report.md, and a final relative integrity inventory. Keep published leaderboard values as separate baseline-result study members.
    • Give authored-source, contract, execution-control, state/observation, outcome/evaluation, and disclosure independent pass/fail/weakened decisions with evidence references and limitations; never publish native objects, action ids, reward vectors, hidden truth, raw logs, environment/argv dumps, credentials, host paths, symlinks/special files, or tracebacks.
    • Update tests/test_cyborg_reproduction.py and tests/test_researcher_cli.py for a bounded injected-driver matrix, strict/duplicate/non-finite/oversized JSON, digest/cardinality/terminal-state mismatches, all four dispositions, retry identity, cleanup/sealing failures, deterministic aggregate regeneration, tier separation, reference-backend limitations, output confinement, and leakage sentinels. The 9,000-run native study remains retained research evidence, not a PR/CI workload.
    • Update the researcher guide, README/index/navigation, architecture note, and clean-install distribution probe so an installed wheel can recompute a copied checkout bundle without packaging packages/.
    • Run targeted tests, then the canonical graph: uv tool run --from 'nox[uv]==2026.4.10' nox -f noxfile.py -s verify (the Makefile/policy path supplies the legitimate requirement-free skip for issue 22). Do not edit CHANGELOG.md or add a changelog fragment.

    Research contribution and dependency boundary

    • OpenRAE/research#14 receives exact apparatus/source/pack/participant identities, supported/unsupported control evidence, leakage and lifecycle probes, a clean short validation path, and a retained readiness/claim matrix. adapters release: publish and smoke-test the first usable CybORG adapter release #24 remains the owner of public-index release qualification; this bundle supplies evidence but does not claim that release gate.
    • OpenRAE/research#20 receives the frozen public protocol/schedules, machine-readable runs and tables, deterministic derivation, claim/evidence mapping, limitations, publication screening, exact identities, and integrity manifest needed for the paper/artifact.
    • adapters docs(cyborg): author the validated CAGE-2 (Scenario2) SDL document #77 is the claim-strength prerequisite for a real full-Scenario2 run. Until its reviewed SDL is available, implementation can build and test the apparatus, but the scientific study must not execute against or relabel the smoke pack.
  7. Brad-Edwards commented on Aug 9, 2026

    @Brad-Edwards
    CollaboratorAuthor

    Execution obligation OBL-22-FULL-SCENARIO2-PREREQUISITE — Opened

    Category: quality
    Observed state: Issue 22 is planned, but its exact authored Scenario2 input is not present in the checkout.
    Impact: Executing the 9,000-run protocol against the current two-step smoke pack would not reconstruct the public CAGE-2 experiment and would contaminate the evidence intended for OpenRAE/research#14 and #20.
    Current obligation: Obtain and bind a reviewed full Scenario2 SDL/pack before scientific execution, or explicitly authorize absorbing the separately owned Scenario2 authoring scope into this implementation.

    Evidence

    • docs(cyborg): author the validated CAGE-2 (Scenario2) SDL document #77 is open and owns authoring the validated full CAGE-2 Scenario2 SDL.
    • src/raes_adapters/cyborg/examples/cage2-research/docs/golden-readiness-checklist.md states that the bounded smoke SDL must be replaced before any full-CAGE or replication claim.
    • The architecture preflight identifies the full authored Scenario2 pack as a claim-strength blocker.
  8. Brad-Edwards commented on Aug 9, 2026

    @Brad-Edwards
    CollaboratorAuthor

    Execution obligation OBL-22-FULL-SCENARIO2-PREREQUISITE — Escalated

    Category: quality
    Observed state: The implementation plan is complete, but scientific execution is blocked on the missing full authored scenario.
    Impact: Proceeding without this decision risks either invalid scientific evidence or an unexpectedly material expansion into separately owned scenario-authoring work.
    Current obligation: Obtain and bind a reviewed full Scenario2 SDL/pack before scientific execution, or explicitly authorize absorbing the separately owned Scenario2 authoring scope into this implementation.

    Evidence

    Pause class: unexpected_material_scope_expansion
    Decision request: Should this issue 22 branch absorb and complete the full Scenario2 authoring work from #77 before implementing/running the reproduction, or should issue 22 remain paused until #77 lands through its own reviewed change?

    This obligation remains open while the decision is pending.

  9. Brad-Edwards commented on Aug 12, 2026

    @Brad-Edwards
    CollaboratorAuthor

    Execution obligation OBL-22-FULL-SCENARIO2-PREREQUISITE — Resolved

    Category: quality
    Observed state: The reviewed full Scenario2 SDL and pack are now present on the issue 22 branch after synchronizing with origin/dev.
    Impact: Scientific protocol implementation can bind the exact reviewed scenario instead of the former two-step placeholder.
    Current obligation: Obtain and bind a reviewed full Scenario2 SDL/pack before scientific execution.

    Evidence

    Disposition: fix
    Corrective action: Fetched origin/dev and fast-forwarded the issue 22 branch to the post-issue-77 merge commit.

    Verification

    • git merge origin/dev completed successfully at d6aea6d.
    • The canonical Scenario2 SDL, RAES lock, realization mapping, and end-to-end tests are present in the checkout.
  10. 25 remaining items

  11. Brad-Edwards commented on Aug 12, 2026

    @Brad-Edwards
    CollaboratorAuthor

    gc_codex_review pre-push cycle 1 of 1 complete for issue #22 on branch '22-cage2-protocol-reproduction'. Posted by the MCP server to enforce the pre-push hard-cap-1 contract (issues #796, #804, #906). Do not edit or delete — used by the next gc_codex_review (uncommitted) invocation to count cycles.

  12. Brad-Edwards commented on Aug 12, 2026

    @Brad-Edwards
    CollaboratorAuthor

    Review decision record — codex cycle 1 (issue #22)

    Reviewer: codex
    Cycle: 1

    Architectural read:

    Core reviewer: The execution seam is mostly well chosen: the study remains CybORG-local while reusing the incumbent target, evaluator, RAES experiment models, artifact writer, and offline aggregation path. However, the change turns issue-local projections into seven versioned, exact-key backend contract families, crossing the repository’s RAES-authority boundary, and the production scheduler does not implement the retry/failure lifecycle its protocol declares. The natural seam is an immutable reproduction selection driving a complete scheduler and RAES-owned evidence records; the current shape forecloses that lifecycle despite already exposing it in the protocol.

    Security reviewer: This is shaped correctly for a local, offline reproduction command: it keeps backend-specific execution within the CybORG module, uses closed protocol and model validation, reserves fresh output roots, suppresses native process output, and seals a bounded portable evidence tree with transitive digest verification. The security-sensitive seams—operator-selected paths, qualified source admission, native-to-portable projection, diagnostics, and offline bundle verification—retain explicit validation and leakage boundaries, and the selection-based design leaves the next admitted baseline or frozen revision as data rather than an arbitrary import or execution path. I found no concrete exploitable security issue introduced by this diff.

    Blocking findings: 2

    Finding 1 — class (7 instances)

    • ID: F1
    • Title: [core] Introduces backend-specific schema families outside RAES authority
    • Location: src/raes_adapters/cyborg/reproduction.py:55
    • Decision: fix
    • Rationale: These constants identify public, versioned protocol, source-ledger, attempt, aggregate, tier, reference, and environment formats. The module then emits them with schema_version and enforces exact field sets, making them backend-specific …
    • Instances:
      • src/raes_adapters/cyborg/reproduction.py:55
      • src/raes_adapters/cyborg/reproduction.py:56
      • src/raes_adapters/cyborg/reproduction.py:57
      • src/raes_adapters/cyborg/reproduction.py:58
      • src/raes_adapters/cyborg/reproduction.py:59
      • src/raes_adapters/cyborg/reproduction.py:60
      • src/raes_adapters/cyborg/reproduction.py:61

    Finding 2 — one-off

    • ID: F2
    • Title: [core] The production scheduler never executes its frozen retry policy
    • Location: src/raes_adapters/cyborg/reproduction.py:130
    • Decision: fix
    • Rationale: The protocol freezes one retry and validates/selects second attempts, but run_full_study only creates attempt-01: _attempt_record hard-codes sequence 1, and no production path creates a predecessor-linked second attempt. On the first…
  13. Brad-Edwards commented on Aug 12, 2026

    @Brad-Edwards
    CollaboratorAuthor

    Execution obligation STATION-OBS-CODEX-REVIEW-C1 — Resolved

    Category: workflow
    Observed state: The originally unobserved Codex station later completed cycle 1 over all 18 changed files and returned two concrete findings.
    Impact: The Codex review station is observed; its two findings were fixed and the final tree passed targeted and canonical verification.
    Current obligation: Observe the Codex review station before the run completes.

    Evidence

    Disposition: fix
    Corrective action: Reran the Codex pre-push station after reducing the change to the apparatus-only diff; the station completed a whole-diff review and posted its findings and decision records.

    Verification

    • Review coverage reports whole-diff, 18 of 18 files, complete.
    • Removed all adapter-defined schema/version identifiers while retaining RAES-owned model validation.
    • Implemented one predecessor-linked retry per failed condition and continued all later conditions after retry exhaustion.
    • Targeted tests passed; canonical nox verify completed successfully on the post-fix tree.
  14. Brad-Edwards commented on Aug 12, 2026

    @Brad-Edwards
    CollaboratorAuthor

    Pre-PR base synchronization

    • Source: refs/remotes/origin/dev at d6aea6d6a751b8e3315103aff1ec31e136c2497e
    • Outcome: already_current
    • Published feature head: ca511818f49931f49565172d086269dfc4298bd7
    • Verified tree: 3884062b106a0dd3a52ee3ec94248761bb741527
  15. Brad-Edwards commented on Aug 12, 2026

    @Brad-Edwards
    CollaboratorAuthor

    Execution obligation OBL-22-SONARCLOUD-NEW-VIOLATIONS — Opened

    Category: quality
    Observed state: SonarCloud analyzed the branch successfully but failed the zero-new-violations gate on 105 maintainability findings, 100 in reproduction.py.
    Impact: PR #91 cannot satisfy the required SonarCloud and aggregate PR gates until every new violation is removed.
    Current obligation: Eliminate the full class of new SonarCloud violations without suppressing rules or excluding the new apparatus from analysis, then rerun verification and CI.

    Evidence

  16. Brad-Edwards commented on Aug 12, 2026

    @Brad-Edwards
    CollaboratorAuthor

    Pre-PR base synchronization

    • Source: refs/remotes/origin/dev at 8db4fbd25467cde51fddba7141098db618d641af
    • Outcome: merged_clean
    • Published feature head: 6177d15cff41debd78692693aff51a204b8ddbbb
    • Verified tree: 3b1eca1d3d83a36e10b899c82dc0fb33faf50d78
  17. Brad-Edwards commented on Aug 12, 2026

    @Brad-Edwards
    CollaboratorAuthor

    Pre-PR base synchronization

    • Source: refs/remotes/origin/dev at 8db4fbd25467cde51fddba7141098db618d641af
    • Outcome: already_current
    • Published feature head: 03d1a7626832503eef0de3ee85a7fd9a4fab1be2
    • Verified tree: 4b3b38ec306c7521217433e5037fa28035089f87
  18. Brad-Edwards commented on Aug 12, 2026

    @Brad-Edwards
    CollaboratorAuthor

    Execution obligation OBL-22-SONARCLOUD-NEW-VIOLATIONS — Resolved

    Category: quality
    Observed state: PR #91 SonarCloud quality gate is OK with zero new violations.
    Impact: The sole remaining merge-readiness quality gate is cleared.
    Current obligation: Eliminate all SonarCloud new-code violations without suppressions or exclusions.

    Evidence

    Disposition: fix
    Corrective action: Refactored the reproduction scheduler, validators, evidence compactor, evaluator measure construction, and CLI dispatch into bounded helpers; centralized repeated literals; added missing function documentation; replaced ambiguous dataclass operations with explicit typed value construction.

    Verification

  19. Brad-Edwards commented on Aug 12, 2026

    @Brad-Edwards
    CollaboratorAuthor

    Execution obligation STATION-OBS-CODEX-REVIEW-C1 — Resolved

    Category: workflow
    Observed state: Codex review cycle 1 completed over the full 18-file Issue 22 diff and produced two actionable findings.
    Impact: The previously unobserved review station is observed, both findings are fixed, and the reviewed final tree is green.
    Current obligation: Observe the Codex review station before the run completes.

    Evidence

    Disposition: fix
    Corrective action: Completed Codex review cycle 1, removed adapter-defined schema/version contract families, implemented predecessor-linked bounded condition retry with continuation, and fixed all resulting quality findings.

    Verification

    • The Codex cycle records whole-diff coverage of 18/18 files.
    • Targeted reproduction, conformance, and qualification tests pass.
    • Canonical verify passed locally and through Ground Control.
    • All PR checks, SonarCloud, and PR Gate pass at commit 03d1a76.
  20. Brad-Edwards commented on Aug 12, 2026

    @Brad-Edwards
    CollaboratorAuthor

    Station observation — codex review cycle 1 — Resolved

    Observed state: The previously unobserved Codex station completed cycle 1 over the whole 18-file diff and produced two actionable findings.

    Corrective action: Removed adapter-defined schema/version contract families and implemented the declared predecessor-linked condition retry with continuation after exhaustion.

    Verification: The cycle record reports complete 18/18-file coverage; targeted and canonical verification pass; all PR checks, SonarCloud, and PR Gate pass at commit 03d1a7626832503eef0de3ee85a7fd9a4fab1be2.

    Evidence:

  21. Brad-Edwards commented on Aug 12, 2026

    @Brad-Edwards
    CollaboratorAuthor

    Station observation — codex review cycle 1 — Re-observed

    Disposition: reobserved
    Corrective action: the existing cycle-1 findings record is now bound to the earlier missing-observation obligation.
    Evidence: #22 (comment)

    Verification

    • The linked findings record is the completed whole-diff verdict for Codex review cycle 1.
    • Its two findings were fixed and independently verified.
    • This record reconciles workflow observation only; it does not run or consume another review cycle.
  22. Brad-Edwards commented on Aug 12, 2026

    @Brad-Edwards
    CollaboratorAuthor

    Ready for review — issue #22

    PR: #91

    Outcome

    Issue 22 now provides an executable, retry-bounded CAGE-2 public-protocol reproduction with portable RAES evidence, deterministic offline verification, and explicit limits on the research claim. The generated study bundle remains in OpenRAE/research, with no dataset path or copy in adapters.

    Implemented and executed the frozen nine-condition CAGE-2 outcome-reproduction protocol; retained portable evidence and deterministic aggregates; fixed all review and Sonar findings; all required PR checks are green.

    Files changed

    Added:

    • docs/decisions/cyborg-cage2-protocol-reproduction-guardrails.md
    • src/raes_adapters/cyborg/reproduction.py
    • tests/test_cyborg_reproduction.py

    Modified:

    • docs/index.md
    • docs/researcher-command.md
    • mkdocs.yml
    • src/raes_adapters/cli.py
    • src/raes_adapters/cyborg/driver.py
    • src/raes_adapters/cyborg/evaluator.py
    • src/raes_adapters/cyborg/orchestrator.py
    • src/raes_adapters/cyborg/participant_runtime.py
    • src/raes_adapters/cyborg/provisioner.py
    • src/raes_adapters/cyborg/researcher.py
    • src/raes_adapters/cyborg/scenario.py
    • src/raes_adapters/cyborg/target.py
    • tests/test_cyborg_execution.py
    • tests/test_cyborg_scenario2.py
    • tests/test_researcher_cli.py

    Reviews

    • codex: Completed whole-diff review; fixed adapter-owned pseudo-contract schema identifiers and incomplete condition retry behavior.
    • test-quality: The recorded response was malformed and contained no actionable finding; no over-cap cycle was run per user direction.

    Traceability reconciliation

    • Pending — requirement status transition and IMPLEMENTS/TESTS reconciliation run in Phase E once the PR merges.

    Status

    • CI: ✅ green
    • SonarCloud: ✅ passed
    • PR ready for user review and merge. Ground Control reconciliation (requirement status + traceability) runs on merge (Phase E).
  23. Brad-Edwards commented on Aug 13, 2026

    @Brad-Edwards
    CollaboratorAuthor

    Final report — issue #22 complete

    PR: #91

    Outcome

    Issue 22 delivers an executable, retry-bounded CAGE-2 public-protocol reproduction with portable RAES evidence, deterministic offline verification, and explicit limits on the research claim. The generated study bundle remains in OpenRAE/research, with no dataset path or copy in adapters.

    Merged the frozen nine-condition CAGE-2 outcome-reproduction apparatus after executing the 9,000-run study, retaining portable evidence and deterministic aggregates, fixing all review and Sonar findings, and passing every required check.

    Files changed

    Added:

    • docs/decisions/cyborg-cage2-protocol-reproduction-guardrails.md
    • src/raes_adapters/cyborg/reproduction.py
    • tests/test_cyborg_reproduction.py

    Modified:

    • docs/index.md
    • docs/researcher-command.md
    • mkdocs.yml
    • src/raes_adapters/cli.py
    • src/raes_adapters/cyborg/driver.py
    • src/raes_adapters/cyborg/evaluator.py
    • src/raes_adapters/cyborg/orchestrator.py
    • src/raes_adapters/cyborg/participant_runtime.py
    • src/raes_adapters/cyborg/provisioner.py
    • src/raes_adapters/cyborg/researcher.py
    • src/raes_adapters/cyborg/scenario.py
    • src/raes_adapters/cyborg/target.py
    • tests/test_cyborg_execution.py
    • tests/test_cyborg_scenario2.py
    • tests/test_researcher_cli.py

    Reviews

    • codex: Completed whole-diff review; fixed adapter-owned pseudo-contract schema identifiers and incomplete condition retry behavior.
    • test-quality: The recorded response was malformed and contained no actionable finding; no over-cap cycle was run per user direction.

    Traceability reconciliation

    • IMPLEMENTS / TESTS / DOCUMENTS added: 0
    • Links updated: 0
    • Stale links removed: 0

    Issue-driven work with no ACES requirement UID. Adapter artifacts refer only to OpenRAE/research issues #14 and #20, not to a dataset path. The result bundle is retained through OpenRAE/research PR #75; the stronger native-versus-OpenRAE equivalence experiment is OpenRAE/research#76.

    Status

    • CI: ✅ green
    • SonarCloud: ✅ passed
    • PR ready for user review and merge.
  24. removed
    in-progressAn agent is actively working this issue via /implement
    on Aug 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions