Repository navigation
docs(primaite): author the RAES scenario and pinned source mapping ledger #40
Description
Objective
Author the portable RAES scenario and pinned mapping evidence for the selected PrimAITE case.
Scope
- Create the example pack's canonical RAES SDL for network laydown, nodes, links, addresses, ports, services, traffic, firewall/routing behavior, red and green behavior, blue actions, per-agent observation/action boundaries, rewards, termination, and multi-agent scheduling.
- Build a pinned source ledger over the selected repository, scenario/configuration, simulator code, agents, evaluator, and public experiment protocol.
- Map every consumed source fact to published RAES SDL, backend, participant, experiment, evidence, or derived-measure surfaces.
- Mark every fact mapped, out of scope, or loss-disclosed; bind losses to the equivalence tier they weaken.
- Keep PrimAITE-native terms inside
raes_adapters.primaitemapping evidence and pack provenance. - Add deterministic validation for source pins/digests, row shape, unique IDs, required categories, RAES targets, and loss coverage.
- Do not add simulator-specific SDL, schemas, vocabularies, profiles, manifest blocks, or semantic metadata conventions.
Acceptance criteria
- The SDL validates and compiles with the pinned RAES release.
- The ledger covers topology/state, identities, participants, actions, observations, controls, rewards/objectives, termination, evaluator facts, and stochastic policy.
- Every source category is mapped, excluded, or loss-disclosed.
- No native action ID, Gym tuple, raw object/state, reward vector, hidden truth, or simulator log becomes portable content.
- Provenance and licensing facts are reusable by the environment pack.
- CI fails on source drift, missing categories, duplicate rows, invalid targets, or undisclosed loss.
References
- https://github.com/Autonomous-Resilient-Cyber-Defence/PrimAITE
- RAES ADR-069 authority and evidence principles
Activity
Brad-Edwards commentedon Aug 2, 2026
🛠️ Picked up by /implement - driver claude-code, branch
40-primaite-scenario-ledger, 2026-08-02T20:30:06.314Z.Brad-Edwards commentedon Aug 2, 2026
gc workflow phase recorded:
preflight(issue #40). Posted by the MCP server to enforce ordering between workflow steps (issue #794 MVP-2). Do not edit or delete — used by downstream tools to gate phase prerequisites.Brad-Edwards commentedon Aug 2, 2026
gc workflow phase recorded:
plan(issue #40). Posted by the MCP server to enforce ordering between workflow steps (issue #794 MVP-2). Do not edit or delete — used by downstream tools to gate phase prerequisites.Plan — author the PrimAITE RAES scenario and pinned source-mapping ledger
Requirement-free run (
--skip-requirement; no in-scope GC requirement — REP-* live upstream). This replicates the CyberBattleSim #26 evidence pattern for the already-qualified PrimAITE v4.0.0data_manipulationcase (issue #39,qualification.json+public-protocol.mdalready pinned), followingdocs/decisions/primaite-scenario-ledger-guardrails.md.Source (already pinned, verified locally)
ARCD PrimAITE
v4.0.0@98617981…;src/primaite/config/_package_data/data_manipulation.yaml(sha25673c93965…) matches the qualification pin. Topology: arouter_1bridging a server LAN192.168.1.0/24(domain_controller/DNSarcd.com, web_server, database_server holdingdatabase/database.db, backup_server/FTP, security_suite spanning both LANs) and a client LAN192.168.10.0/24(client_1, client_2), viaswitch_1/switch_2. REDdata_manipulation_attacker(scripted, starts step 25) corruptsdatabase.db; two GREENprobabilistic-agentusers drive web/database workload; BLUEdefender(proxy-agent, RL) hasDiscrete(78)actions over service/file/folder/node/router-ACL/NIC controls and a flattenedBox(1652)observation. Fixed horizon 128;terminatedalways false, ends by truncation.Artifacts (all package-local under
src/raes_adapters/primaite/)scenario/data-manipulation.sdl.yaml— authored RAES SDL carrying portable topology/objective truth only, mirroring the proven cyberbattlesim section set:nodes,infrastructure,relationships,entities,action_contracts,observation_boundaries,propositions,assertions,objectives,agents,evidence_requirements. Portable role-based names (e.g.database-host,web-host,workstation-a,perimeter-router) — native hostnames/action-map strings stay out of the SDL. Parsed/instantiated/compiled/admitted underraes==3.3.0; its canonical instantiated digest is computed and pinned in the module (PINNED_SCENARIO_DIGEST), so any drift fails CI.experiment/data-manipulation.task.exp.yaml—ExperimentTaskModel: evaluation protocol + metric definitions (database-file-integrity, cumulative-blue-reward, steps-to-truncation, terminal-cause, green-availability), evidence requirements, and aprotocolartifact ref checksum-joined topublic-protocol.md. Carries reward/metric intent SDL excludes.experiment/data-manipulation.spec.exp.yaml—ExperimentSpecModelauthoring input:run_plan.episode_control(fixed-horizon 128 truncation;terminatedalways false; distinct from goal terminal) and descriptive-onlystochastic_controls(gym-reset seam [broken w/o torch], python-random, numpy-global, torch-absent) at public seed20260802, each withexecutable_binding: null(retains the determinism loss).mapping/source-ledger.jsonl— one JSON row per pinned source fact. Every row repeats repo/commit/path/selector/digest (joinsqualification.json.source_files), a requiredcategory, exactly onedisposition(mapped/excluded/loss-disclosed), a resolvableraes_targetwhen mapped (sdl:<section>[.<key>],contract:<Model>.<path>, orresource:<file>), orloss_ref+equivalence_tierwhen lossy, plusmapping_rule/verification. Covers all required categories: topology/state, identities, participants, actions, observations, controls (addresses/ports/services/traffic/firewall-routing), rewards/objectives, termination, evaluator, stochastic, provenance/licensing. Native constructs (Discrete(78) action ids/mask, flattened Box vector,info, traffic payload, reward vector, NIC/MAC hardware identity) areexcludedand source-private.mapping/loss-disclosures.md— four disclosures bound to ADR-069 tiers:loss-no-source-artifact(reproducibility — no index/release wheel),loss-unbound-random-streams(deterministic-replay — broken torch seed seam + uncontrolled GREEN/NumPy RNG, nogame.seed),loss-abstracted-participant-interface(outcome-equivalence — Discrete(78)/Box(1652) projected to portable contracts/boundaries),loss-fixed-horizon-only-termination(outcome-equivalence — no source-native success/failure terminal; scalar reward ≠ objective truth).mapping/README.md— mapping-directory description (fields, required categories, governance), mirroring the cyberbattlesim README pattern; PrimAITE-native, package-data only.scenario_ledger.py— deterministic loaders + validators composingraes.parse_sdl/instantiate_scenario/canonical_instantiated_sdl_digest/compile_scenario_runtime_model/admit_instantiated_scenario, closedExperimentTaskModel/ExperimentSpecModel/parse_experiment_spec,schema_bundle()target resolution, strict JSONL parsing (duplicate-key/non-finite/non-object rejection), row-shape/coverage/source-drift/loss-tier checks, cross-artifact selection joins (scenario/task/spec/protocol digests), and a native-leakage scan grounded in qualification provenance (native participant refs +PrimaiteGymEnventrypoint + object/array/traceback/Discrete(78)reprs). Extension seam = one immutableEvidenceSelectionNamedTuple (no registry). No new schema/DTO/vocabulary/policy-gate/exception hierarchy; not generalized intoraes_adapters.base.Tests (TDD —
tests/test_primaite_scenario_ledger.py)Positive: SDL parses/instantiates/admits and its digest equals the pin; task+spec validate and join; ledger covers every category with unique ids; losses bind recognized tiers;
validate_all() == []. Fail-closed negatives (each proves one corruption is rejected): source-digest/commit drift, unqualified source path, duplicate row id, unknown/missing category, unknown disposition, missing/unknown row field, mapped-without-target / unresolvable target, excluded-with-target, loss-row without loss_ref / wrong / unrecognized tier, orphan disclosure, strict-JSONL duplicate-key/non-finite/non-object, native-leakage marker (incl. provenance-grounding), and cross-artifact join drift. Red-first per behavior.Verification & gates
make verify(hygiene, policy--skip-requirement, lint, tool-tests, typecheck [base + each extra], tests+coverage≥80, distributions, docs). The new tests run in the base env (raes/raes_contracts/raes_processorare base deps); theprimaiteextra stays empty. mypy strict, ruff (SIM/UP/B/C4), MkDocs strict (preflight already added the guardrails nav entry).Constraints honored
Single distribution / one
pyproject.toml+uv.lock, emptyprimaiteextra, noconflictsneeded (plan rules; ADR-003). No CAGE-specific SDL/schema/vocab/profile/manifest;raes_adapters.baseuntouched; no native leakage into portable artifacts; noCHANGELOG.md/changelog.dedits (Release Please owns it; PR title will bedocs(primaite): …). No ADR add/amend, so noadr-index.yamlre-pin. No adapter/backend/runtime/env-pack/persistence/second-CI-workflow/equivalence claim. Each retained limitation is mapped or loss-disclosed at the tier it weakens.Acceptance-criteria coverage
SDL validates+compiles with pinned RAES →
validate_scenario_pipeline; ledger covers all categories →_coverage_problems+ test; every fact mapped/excluded/loss-disclosed → disposition validation; no native leakage → structural closed models + grounded leakage scan; provenance/licensing reusable by an environment pack →resource:qualification.jsonrows + legal facts owned byqualification.json; CI fails on drift/missing-category/dup-rows/invalid-targets/undisclosed-loss → the fail-closed negatives run undermake verify.Brad-Edwards commentedon Aug 2, 2026
gc_codex_review — cycle 1 of 1 (pre-push) on issue #40 (branch
40-primaite-scenario-ledger)Diff mode: inline — the complete diff was supplied in one prompt
Core review
Verdict:
ship-with-fixesThe package-local ownership, reuse of published RAES contracts, and separation of SDL meaning from backend evidence are shaped correctly. However, the current evidence boundary can validate green while losing source semantics or allowing reviewed companion artifacts to drift: several ledger rows classify partial control projections as fully mapped, and the advertised immutable Selection pins only the scenario. These are contained fixes within the chosen design seam, but they should land before shipping.
Blocking findings (2):
src/raes_adapters/primaite/mapping/source-ledger.jsonl:14This row claims that per-host addresses, masks, gateways, and DNS domain mapping reach SDL, but its target carries only subnet CIDRs and gateways; the individual host assignments and DNS mapping are absent. The same pattern occurs for concrete baseline ACL rules and NMNE thresholds, whose targets expose only abstract action or observation surfaces. This violates the guardrail that a partial transformation must be split into a mapped fact and a loss-disclosed fact, and allows validation to pass after behaviorally relevant controls disappear. Map each supported fact through the published RAES contracts and add explicit tiered loss rows for the remainder.
src/raes_adapters/primaite/scenario_ledger.py:67Selection is documented as the immutable evidence boundary, but only the scenario has a canonical digest. The task, spec, ledger, and loss-disclosure resources carry paths or logical IDs without content pins. validate_all() merely parses and validates them, so a semantically valid change to reward definitions, run controls, mapping dispositions, or loss narratives can remain green without updating a reviewed selection identity. Add and verify canonical or byte digests for each companion resource, with mutation tests proving drift is rejected.
Security review
Verdict:
shipThis change is shaped correctly as a package-local, offline evidence boundary: static PrimAITE resources are parsed with safe YAML and strict JSONL handling, joined to immutable qualification records, validated through closed RAES contracts, and checked without native execution, network access, shell invocation, dynamic imports, or secret-bearing inputs. It introduces no request, authentication, authorization, persistence, or tenant boundary, and the explicit Selection seam supports additional reviewed evidence sets without weakening the validation boundary. I found no concrete, exploitable security issue introduced by the diff.
No blocking findings.
Brad-Edwards commentedon Aug 2, 2026
gc_codex_review pre-push cycle 1 of 1 complete for issue #40 on branch '40-primaite-scenario-ledger'. Posted by the MCP server to enforce the pre-push hard-cap-1 contract (issues #796, #804, #906). Do not edit or delete — used by the next
gc_codex_review(uncommitted) invocation to count cycles.Brad-Edwards commentedon Aug 2, 2026
Review decision record — codex cycle 1 (issue #40)
Reviewer: codex
Cycle: 1
Architectural read:
Blocking findings: 2
Finding 1 —
class(3 instances)F1src/raes_adapters/primaite/mapping/source-ledger.jsonl:14src/raes_adapters/primaite/mapping/source-ledger.jsonl:14src/raes_adapters/primaite/mapping/source-ledger.jsonl:16src/raes_adapters/primaite/mapping/source-ledger.jsonl:17Finding 2 —
class(4 instances)F2src/raes_adapters/primaite/scenario_ledger.py:67src/raes_adapters/primaite/scenario_ledger.py:67src/raes_adapters/primaite/scenario_ledger.py:69src/raes_adapters/primaite/scenario_ledger.py:70src/raes_adapters/primaite/scenario_ledger.py:71Brad-Edwards commentedon Aug 2, 2026
gc workflow phase recorded:
traceability_reconciled(issue #40). Posted by the MCP server to enforce ordering between workflow steps (issue #794 MVP-2). Do not edit or delete — used by downstream tools to gate phase prerequisites.no in-scope requirements; touched-files audit clean.
Brad-Edwards commentedon Aug 2, 2026
Final report — issue #40 complete
PR: #70
Outcome
PrimAITE's data_manipulation case now ships a portable, machine-checked RAES evidence set: a scenario describing the network and who is attacking/defending what, companion contracts holding the reward/termination/randomness details, and a pinned ledger tracing every source fact to a portable surface (or disclosing why it can't be carried exactly). CI fails if the evidence drifts, a category is missing, a mapping target breaks, a loss goes undisclosed, or native simulator internals leak into portable content.
Authored the portable RAES scenario + pinned source-mapping ledger for the qualified PrimAITE data_manipulation case (#40), replicating the CyberBattleSim/NASim pattern. Added an SDL scenario (topology/objective truth; parses/instantiates/compiles/admits under raes==3.3.0, digest pinned), companion experiment task/spec contracts, and a source ledger + loss disclosures covering all 11 categories with each un-materialized fact bound to its ADR-069 tier. Bound onto the shared _scenario_ledger.ScenarioLedger (no duplication) and folded into the shared parametrized fail-closed test suite; added opt-in companion-resource content pinning. Requirement-free run.
Files changed
Added:
docs/decisions/primaite-scenario-ledger-guardrails.mdsrc/raes_adapters/primaite/scenario/data-manipulation.sdl.yamlsrc/raes_adapters/primaite/experiment/data-manipulation.task.exp.yamlsrc/raes_adapters/primaite/experiment/data-manipulation.spec.exp.yamlsrc/raes_adapters/primaite/mapping/source-ledger.jsonlsrc/raes_adapters/primaite/mapping/loss-disclosures.mdsrc/raes_adapters/primaite/mapping/README.mdsrc/raes_adapters/primaite/scenario_ledger.pyModified:
src/raes_adapters/_scenario_ledger.pytests/test_scenario_ledger.pydocs/index.mdmkdocs.ymlReviews
Traceability reconciliation
Requirement-free run (no in-scope GC requirement; REP-* live upstream). No IMPLEMENTS/TESTS links to reconcile.
Status