docs(nasim): author the RAES scenario and pinned source mapping ledger

Objective

Author the portable RAES scenario and pinned mapping evidence for the selected NASim case.

Scope

Create the example pack's canonical RAES SDL for subnets, hosts, services, vulnerabilities, credentials, firewalls, discovery, exploits, privilege escalation, host value, attacker observations/actions, rewards, goal conditions, partial observability, and generated-scenario parameters.
Build a pinned source ledger over the selected repository, scenario/configuration, simulator code, agents, evaluator, and public experiment protocol.
Map every consumed source fact to published RAES SDL, backend, participant, experiment, evidence, or derived-measure surfaces.
Mark every fact mapped, out of scope, or loss-disclosed; bind losses to the equivalence tier they weaken.
Keep NASim-native terms inside raes_adapters.nasim mapping evidence and pack provenance.
Add deterministic validation for source pins/digests, row shape, unique IDs, required categories, RAES targets, and loss coverage.
Do not add simulator-specific SDL, schemas, vocabularies, profiles, manifest blocks, or semantic metadata conventions.

Acceptance criteria

The SDL validates and compiles with the pinned RAES release.
The ledger covers topology/state, identities, participants, actions, observations, controls, rewards/objectives, termination, evaluator facts, and stochastic policy.
Every source category is mapped, excluded, or loss-disclosed.
No native action ID, Gym tuple, raw object/state, reward vector, hidden truth, or simulator log becomes portable content.
Provenance and licensing facts are reusable by the environment pack.
CI fails on source drift, missing categories, duplicate rows, invalid targets, or undisclosed loss.

References

https://github.com/Jjschwartz/NetworkAttackSimulator
RAES ADR-069 authority and evidence principles

🛠️ Picked up by /implement - driver claude, branch 33-nasim-scenario-ledger, 2026-08-02T18:58:15.453Z.

gc workflow phase recorded: preflight (issue #33). Posted by the MCP server to enforce ordering between workflow steps (issue #794 MVP-2). Do not edit or delete — used by downstream tools to gate phase prerequisites.

gc workflow phase recorded: plan (issue #33). Posted by the MCP server to enforce ordering between workflow steps (issue #794 MVP-2). Do not edit or delete — used by downstream tools to gate phase prerequisites.

Plan — NASim RAES scenario + pinned source-mapping ledger (#33)

Requirement-free authoring issue (no #### Requirements; runs with --skip-requirement). It mirrors the already-landed CyberBattleSim evidence set (#26) for the qualified NASim tiny case pinned in #32 (nasim==0.12.0 @ 7c732bc). The architecture preflight authored docs/decisions/nasim-scenario-ledger-guardrails.md; this plan honors it.

Key design decision — extract a shared validator (SonarCloud duplication gate)

NASim's validator + tests would be a near-verbatim copy of CyberBattleSim's ~835-line scenario_ledger.py and ~378-line test. The strict SonarCloud gate fails on new_duplicated_lines_density > 3%, and suppression is not an option. The root-cause fix — matching the raes_adapters/_qualification.py precedent (commit 44d988e, "share the qualification-evidence loader across backends") — is to extract the backend-agnostic machinery into a new shared raes_adapters/_scenario_ledger.py and make both backends thin wrappers:

New src/raes_adapters/_scenario_ledger.py — the generic core (moved, not copied, out of cyberbattlesim): JsonValue/JsonObject types, generic EvidenceSelection, LedgerProblem, the issue-local label constants (DISPOSITIONS, REQUIRED_CATEGORIES, RECOGNIZED_EQUIVALENCE_TIERS, row-field sets), strict JSONL/loss parsers, target resolution against parsed SDL + schema_bundle(), full ledger validation, cross-artifact joins, scenario pipeline, and a native_leakage_problems(markers, texts) that takes markers explicitly. A small ScenarioLedger binder holds one backend's (selection, package, load_qualification, read_public_protocol, native_markers_fn) and exposes the resource-reading + orchestration methods. Not raes_adapters.base (base stays semantics-free), not a DTO/schema/registry/policy-gate/exception-hierarchy — just a private shared helper, exactly like _qualification.py. Labels remain issue-local validation labels, not RAES vocabulary.
cyberbattlesim/scenario_ledger.py → thin wrapper (behavior-preserving). Preserves the symbols its backend importers use (backend/conformance.py, backend/evaluator.py need CYBERBATTLE_CHAIN, EvidenceSelection, load_loss_disclosures, validate_all) plus a LEDGER object. Pinned digest and evidence content are unchanged, so nothing drifts.

This yields exactly one copy of the logic → duplication gate passes legitimately.

NASim evidence set — `src/raes_adapters/nasim/`

Per the guardrails, artifacts stay separate; SDL carries objective/topology truth only (RAES 2.0.0 rejects reward/score sections); reward/evaluator/termination/stochastic live in companion experiment contracts; native state stays source-private.

scenario/nasim-tiny.sdl.yaml (name: nasim-tiny). Models the fixed tiny benchmark faithfully (it is static, not generated — so no abstracted-topology loss): 3 subnets / 3 hosts, ssh service, the e_ssh exploit + pe_tomcat privilege-escalation as CWE-classed vulnerabilities, sensitive hosts (2,0)/(3,0) as high asset-value nodes, subnet reachability/firewall as relationships, a bruteforce attacker agent, propositions/assertions/objectives for "all sensitive hosts compromised". Portable action names are hyphenated (service-scan, exploit, privilege-escalation, …) so no native CamelCase leaks. Exact tiny.yaml topology read from the qualified source file during implementation.
experiment/nasim-tiny.task.exp.yaml (ExperimentTaskModel) — metric definitions (steps-to-termination, cumulative-reward, goal-reached, terminal-cause), the bruteforce evaluator intent, and a protocol artifact_ref joined to public-protocol.md (sha256 + size). Reward (action_result.value - action.cost, host value 100) is an evaluator metric here, never an SDL score.
experiment/nasim-tiny.spec.exp.yaml (ExperimentSpecModel) — run_plan.episode_control keeps terminated = goal_reached and truncated = step_limit (1000) distinct (they are computed independently upstream), and stochastic_controls for the NASim streams are descriptive only, no executable binding, reflecting the unbound-action-RNG loss.
mapping/source-ledger.jsonl — one row per consumed source fact, each joining qualification.json source_files (commit + sha256, offline drift check). Covers all 11 required categories using only the 11 qualified paths (tiny.yaml, environment.py, network.py, action.py, bruteforce_agent.py, setup.py, LICENSE.md, README.rst). Native constructs (flat action index / FlatActionSpace, the raw Box(56,) ndarray observation, host state) are excluded and source-private. mapped targets resolve to sdl:<section>[.<key>], contract:<Model>.<path>, or resource:<file>.
mapping/loss-disclosures.md — two losses, each referenced by ≥1 ledger row and bound to its ADR-069 tier:
- loss-unbound-action-rng (deterministic-replay): action success is drawn from the global NumPy RNG (network.py:87); make_benchmark(seed)/reset(seed) are ignored for the static benchmark, so a seed alone is not a replay and the stochastic controls stay descriptive.
- loss-apparatus-reconstruction (reproducibility): the supplied agents require gymnasium==0.26.3 (later releases return numpy.int64 and break them) and importing NASim requires Tk system libraries (render.py), bounding headless apparatus reconstruction.
mapping/README.md — ledger schema + governance doc (mirrors the CyberBattleSim one, NASim-local).

Native-leakage grounding (differs from CyberBattleSim)

CyberBattleSim grounds native identifiers in qualification.runtime.smoke.observation_keys. NASim's selected observation is a keyless flat float32 Box(56,) — it records observation_type/shape/dtype, not keys — so there are no observation keys to enumerate. NASim grounds its scan in the native action-class symbols (servicescan, osscan, subnetscan, processscan, privilegeescalation, flatactionspace) plus the object/array/traceback representation markers (<nasim, ndarray(, array([, object at 0x, traceback…). Documented in the wrapper docstring.

Pinned canonical digest

Compute the instantiated canonical SDL digest via uv run --frozen python against raes==2.0.0, hard-pin it into NASIM_TINY.pinned_scenario_digest and both experiment contracts' ref_digest. Any scenario edit changes the digest and fails CI.

Tests (TDD, zero duplication)

Replace tests/test_cyberbattlesim_scenario_ledger.py with a parametrized tests/test_scenario_ledger.py driving both backends off their LEDGER objects + the shared _scenario_ledger core. Ports every fail-closed negative (source/commit/digest drift, unqualified path, unknown/missing/duplicate field & row id, missing category, bad disposition, unresolvable/mis-placed target, loss ref/tier disagreement, unreferenced disclosure, strict-JSONL, cross-artifact join drift, native leakage). Backend-specific grounding assertions (observation-keys vs fixed-markers) are case-guarded. New code is fully covered by the parametrized suite.

Constraints honored (plan-rules + guardrails)

Canonical nox -s verify graph is the completion gate; no CHANGELOG.md/changelog.d/ edits (Release Please owns it); PR title will be docs(nasim): ….
No new extra/dependency/lockfile change; single distribution preserved (raes==2.0.0 base). No changes to raes_adapters.base, RAES contracts, cyborg, or environment-pack schemas. raes_adapters.nasim imports no simulator stack (only raes*), so the test/typecheck graph never imports NASim.
No adapter/runtime, Gym wrapper, action registry, backend/participant manifest, conformance harness, environment pack, persistence, or equivalence claim.
Documentation coverage: outcome_required: false; the mapping README + preflight guardrails note are the docs. Keep the preflight's mkdocs.yml nav entry.

Whole-repo / security review

Static checked-in package data only — no auth surface, no secrets (synthetic credential/process intent only, .secrets not an input), bounded RAES parser + closed models + strict UTF-8 JSONL, in-process validation over checked-in resources (no upstream fetch/exec/network/home-cache), bounded row-id/field/reason errors (no native payloads/tracebacks), no new store. Extensibility seam: a second NASim case is another EvidenceSelection, not a validator edit.

Files

Add: _scenario_ledger.py; nasim/scenario_ledger.py; nasim/scenario/nasim-tiny.sdl.yaml; nasim/experiment/nasim-tiny.{task,spec}.exp.yaml; nasim/mapping/{source-ledger.jsonl,loss-disclosures.md,README.md}; tests/test_scenario_ledger.py.
Modify: cyberbattlesim/scenario_ledger.py (→ thin wrapper).
Remove: tests/test_cyberbattlesim_scenario_ledger.py (folded into the parametrized suite).
Already added by preflight: docs/decisions/nasim-scenario-ledger-guardrails.md, mkdocs.yml nav.

gc_codex_review — cycle 1 of 1 (pre-push) on issue #33 (branch 33-nasim-scenario-ledger)
Diff mode: inline — the complete diff was supplied in one prompt

Core review

Verdict: ship-with-fixes

Architecture-fit: extracting the identical ledger machinery into a private shared module is the right seam, while each backend retains its evidence selection and native-leakage provenance. Code-quality: the parameterized tests preserve the fail-closed behavior across both backends. However, the advertised second-selection extension seam is incomplete because one of its artifact coordinates is never honored during validation.

Blocking findings (1):

[one-off] Validate the protocol resource from the selected evidence case — src/raes_adapters/_scenario_ledger.py:840
EvidenceSelection.protocol_resource is never used: selection validation always hashes self.read_public_protocol(), which is bound once per backend. A second NASim (or CyberBattleSim) selection with its own protocol artifact will therefore be checked against the default case's protocol and fail checksum/size validation (or validate the wrong artifact). Load the protocol through selection.protocol_resource, or remove that field and narrow the extension contract; as written, the documented selection-based extension seam is not functional.

Security review

Verdict: ship

This change extracts deterministic scenario-ledger validation into a shared, package-local helper and binds it separately for CyberBattleSim and NASim, while adding NASim’s checked-in evidence artifacts. The seam is appropriately the immutable evidence selection plus backend-specific native-leakage markers; it preserves existing fail-closed parsing, digest joins, and closed-model validation, and accommodates the obvious next backend or scenario selection without introducing a runtime or public control-plane surface.

No blocking findings.

gc_codex_review pre-push cycle 1 of 1 complete for issue #33 on branch '33-nasim-scenario-ledger'. Posted by the MCP server to enforce the pre-push hard-cap-1 contract (issues #796, #804, #906). Do not edit or delete — used by the next gc_codex_review (uncommitted) invocation to count cycles.

Review decision record — codex cycle 1 (issue #33)

Reviewer: codex
Cycle: 1

Architectural read:

Core reviewer: Architecture-fit: extracting the identical ledger machinery into a private shared module is the right seam, while each backend retains its evidence selection and native-leakage provenance. Code-quality: the parameterized tests preserve the fail-closed behavior across both backends. However, the advertised second-selection extension seam is incomplete because one of its artifact coordinates is never honored during validation.

Security reviewer: This change extracts deterministic scenario-ledger validation into a shared, package-local helper and binds it separately for CyberBattleSim and NASim, while adding NASim’s checked-in evidence artifacts. The seam is appropriately the immutable evidence selection plus backend-specific native-leakage markers; it preserves existing fail-closed parsing, digest joins, and closed-model validation, and accommodates the obvious next backend or scenario selection without introducing a runtime or public control-plane surface.

Blocking findings: 1

Finding 1 — `one-off`

ID: F1
Title: [core] Validate the protocol resource from the selected evidence case
Location: src/raes_adapters/_scenario_ledger.py:840
Decision: fix
Rationale: EvidenceSelection.protocol_resource is never used: selection validation always hashes self.read_public_protocol(), which is bound once per backend. A second NASim (or CyberBattleSim) selection with its own protocol artifact will theref…

gc_test_quality_review cycle 1 of 1 — issue #33

Reviewer: test-quality (claude-sonnet-5 via gc_test_quality_review)
Branch: 33-nasim-scenario-ledger
Cycle: 1 / 1
Findings: 0 (clean run)

gc_test_quality_review cycle 1 of 1 complete for issue #33 on branch '33-nasim-scenario-ledger'. Posted by the MCP server to enforce the gc_test_quality_review hard-cap-1 contract (issue #884 follow-up, default lowered in #906). Do not edit or delete — used by the next gc_test_quality_review invocation to count cycles.

Review decision record — test-quality cycle 1 (issue #33)

Reviewer: test-quality
Cycle: 1

Architectural read:

This is an extract-shared-module refactor: CyberBattleSim's original ~800-line scenario_ledger.py (loaders + deterministic validators) is generalized into raes_adapters/_scenario_ledger.py as a ScenarioLedger class parameterized by an EvidenceSelection NamedTuple and a native-marker callback, and both cyberbattlesim/scenario_ledger.py and the new nasim/scenario_ledger.py shrink to ~80-line binders that just supply backend-specific identity and native markers. The test suite follows the same generalization: test_cyberbattlesim_scenario_ledger.py's module-level-function suite becomes a Case-parameterized suite in test_scenario_ledger.py, fixture-driven over both backends' bound ScenarioLedger objects. I diffed the deleted file against the new one line-by-line: every positive test, every fail-closed negative test (source drift, missing category, duplicate row, unresolvable target, undisclosed loss, native leakage, JSONL strictness, cross-artifact identity joins, protocol checksum/size drift) survives with an equivalent assertion, and now runs against NASim's tiny scenario too, for free. The extension seam is genuinely data, not code: a third backend needs only a new Case + EvidenceSelection, not new test methods. The negative tests actually drive the enforcement boundary (mutate source_digest, source_commit, disposition, loss_ref, equivalence_tier, then assert the specific rejection) rather than merely asserting a row or rule is present, which is the right bar for an evidence-integrity gate. This is shaped correctly; I would ship it.

Blocking findings: 0 (clean run)

Pre-PR base synchronization

Source: refs/remotes/origin/dev at f05ef0ccdce3063de5159222f7638ef1487353f4
Outcome: merged_clean
Published feature head: 7a1231dca19070f0642009b22e6d1aa7bb8dfc55
Verified tree: 797cf25447d217c1c372f9e1928c2765a3bf4e16

Pre-PR base synchronization

Source: refs/remotes/origin/dev at f05ef0ccdce3063de5159222f7638ef1487353f4
Outcome: already_current
Published feature head: 6caf7ceadd1eef1affbdf1323a6752da66a3399e
Verified tree: 74d1262686ed61440cd042146994079835b155de

Pre-PR base synchronization

Source: refs/remotes/origin/dev at f05ef0ccdce3063de5159222f7638ef1487353f4
Outcome: already_current
Published feature head: 7e7142e074aeb7e5a984933f7d89ebd2230b20c6
Verified tree: 4ba185eceddfe554405af3ccb304f77ef768f992

Ready for review — issue #33

PR: #69

Outcome

NASim now ships a portable, machine-checked RAES evidence set for the qualified tiny benchmark, like CyberBattleSim: an SDL scenario for topology/objective, experiment contracts for reward/evaluator/termination/randomness, and a pinned ledger accounting for every source fact (mapped, excluded, or loss-disclosed). CI fails on source drift, missing categories, malformed rows, unresolvable targets, undisclosed losses, or native-state leakage. Shared validation was extracted so both backends reuse it.

Author NASim's portable RAES scenario + pinned source-mapping ledger, extracting the shared validator both backends reuse.

Files changed

Added:

src/raes_adapters/_scenario_ledger.py
src/raes_adapters/nasim/scenario_ledger.py
src/raes_adapters/nasim/scenario/nasim-tiny.sdl.yaml
src/raes_adapters/nasim/experiment/nasim-tiny.task.exp.yaml
src/raes_adapters/nasim/experiment/nasim-tiny.spec.exp.yaml
src/raes_adapters/nasim/mapping/source-ledger.jsonl
src/raes_adapters/nasim/mapping/loss-disclosures.md
src/raes_adapters/nasim/mapping/README.md
tests/test_scenario_ledger.py
docs/decisions/nasim-scenario-ledger-guardrails.md

Modified:

src/raes_adapters/cyberbattlesim/scenario_ledger.py
mkdocs.yml

Deleted:

tests/test_cyberbattlesim_scenario_ledger.py

Reviews

codex: ship-with-fixes: fixed one finding where the selection join ignored EvidenceSelection.protocol_resource; now reads the protocol via the selection, with a regression test. Security review: ship, no findings.
test-quality: clean, no findings.

Traceability reconciliation

Pending — requirement status transition and IMPLEMENTS/TESTS reconciliation run in Phase E once the PR merges.

Status

CI: ✅ green
SonarCloud: ✅ passed
PR ready for user review and merge. Ground Control reconciliation (requirement status + traceability) runs on merge (Phase E).

gc workflow phase recorded: traceability_reconciled (issue #33). Posted by the MCP server to enforce ordering between workflow steps (issue #794 MVP-2). Do not edit or delete — used by downstream tools to gate phase prerequisites.

no in-scope requirements; touched-files audit clean.

Final report — issue #33 complete

PR: #69

Outcome

NASim now ships a portable, machine-checked RAES evidence set for the qualified tiny benchmark, like CyberBattleSim: an SDL scenario for topology/objective, experiment contracts for reward/evaluator/termination/randomness, and a pinned ledger accounting for every source fact (mapped, excluded, or loss-disclosed). CI fails on source drift, missing categories, malformed rows, unresolvable targets, undisclosed losses, or native-state leakage. Shared validation was extracted so both backends reuse it.

Author NASim's portable RAES scenario + pinned source-mapping ledger, extracting the shared validator both backends reuse.

Files changed

Added:

src/raes_adapters/_scenario_ledger.py
src/raes_adapters/nasim/scenario_ledger.py
src/raes_adapters/nasim/scenario/nasim-tiny.sdl.yaml
src/raes_adapters/nasim/experiment/nasim-tiny.task.exp.yaml
src/raes_adapters/nasim/experiment/nasim-tiny.spec.exp.yaml
src/raes_adapters/nasim/mapping/source-ledger.jsonl
src/raes_adapters/nasim/mapping/loss-disclosures.md
src/raes_adapters/nasim/mapping/README.md
tests/test_scenario_ledger.py
docs/decisions/nasim-scenario-ledger-guardrails.md

Modified:

src/raes_adapters/cyberbattlesim/scenario_ledger.py
mkdocs.yml

Deleted:

tests/test_cyberbattlesim_scenario_ledger.py

Reviews

codex: ship-with-fixes: fixed one finding where the selection join ignored EvidenceSelection.protocol_resource; now reads the protocol via the selection, with a regression test. Security review: ship, no findings.
test-quality: clean, no findings.

Traceability reconciliation

IMPLEMENTS / TESTS / DOCUMENTS added: 0
Links updated: 0
Stale links removed: 0

Status

CI: ✅ green
SonarCloud: ✅ passed
PR ready for user review and merge.

docs(nasim): author the RAES scenario and pinned source mapping ledger #33

Description

Objective

Scope

Acceptance criteria

References

Activity

Brad-Edwards commented on Aug 2, 2026

Brad-Edwards commented on Aug 2, 2026

Brad-Edwards commented on Aug 2, 2026

Plan — NASim RAES scenario + pinned source-mapping ledger (#33)

Key design decision — extract a shared validator (SonarCloud duplication gate)

NASim evidence set — src/raes_adapters/nasim/

Native-leakage grounding (differs from CyberBattleSim)

Pinned canonical digest

Tests (TDD, zero duplication)

Constraints honored (plan-rules + guardrails)

Whole-repo / security review

Files

Brad-Edwards commented on Aug 2, 2026

Core review

Security review

Brad-Edwards commented on Aug 2, 2026

Brad-Edwards commented on Aug 2, 2026

Review decision record — codex cycle 1 (issue #33)

Finding 1 — one-off

Brad-Edwards commented on Aug 2, 2026

gc_test_quality_review cycle 1 of 1 — issue #33

Brad-Edwards commented on Aug 2, 2026

Brad-Edwards commented on Aug 2, 2026

Review decision record — test-quality cycle 1 (issue #33)

Brad-Edwards commented on Aug 2, 2026

Pre-PR base synchronization

Brad-Edwards commented on Aug 2, 2026

Pre-PR base synchronization

Brad-Edwards commented on Aug 2, 2026

Pre-PR base synchronization

Brad-Edwards commented on Aug 2, 2026

Ready for review — issue #33

Outcome

Files changed

Reviews

Traceability reconciliation

Status

Brad-Edwards commented on Aug 2, 2026

Brad-Edwards commented on Aug 2, 2026

Final report — issue #33 complete

Outcome

Files changed

Reviews

Traceability reconciliation

Status

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

NASim evidence set — `src/raes_adapters/nasim/`

Finding 1 — `one-off`