build(nasim): qualify and pin the public source and experiment profile

Objective

Establish one immutable, legally usable, runnable NASim source and experiment profile for the RAES study.

Scope

Select and pin the published NASim package/source, named benchmark scenarios, scenario generator, Gymnasium interface, benchmark scripts, and supplied planning/RL baseline agents.
Record repository/release, package, dependency, Python, scenario, agent, evaluator, seed, and configuration identities and digests.
Verify licenses, redistribution/attribution requirements, external downloads, model weights, and retained-output permissions.
Reconstruct installation in a clean environment and execute reset, representative action/step, observation, reward/result, and termination behavior.
Identify undocumented defaults, incompatible dependencies, known defects, and stochastic entry points.
Choose the exact example scenario and public baseline protocol that downstream issues must implement.
Isolate dependencies behind the nasim extra; use uv conflicts declarations where simulator stacks are mutually incompatible.
Do not patch semantics silently. Any compatibility patch is pinned, explained, and tested against the unmodified source behavior.

Qualification semantics

The maintainer-selected simulator and experiment profile are admitted inputs to the RAES study. Qualification records the exact source and installation route, what RAES can attest or reproduce, stochastic-control coverage, known upstream defects, and limits on claim strength. Packaging gaps, incomplete determinism, compatibility limits, and source defects are disclosed limitations; they do not by themselves veto adapter implementation or require RAES to create behavior the simulator lacks.

Acceptance criteria

One immutable runtime/source profile and one public experiment protocol are selected.
Clean installation and a source-native smoke run pass through a documented supported route.
Exact scenario, agent, evaluator, reset, seed, metric, and termination identities are known.
License, attribution, redistribution, archive/maintenance, known-defect, and patch decisions are explicit.
Base-only and unrelated simulator extras remain independent.
The qualification record grades attainable RAES claims across source identity, configuration/control attestation, run evidence, and outcome reproduction, with every weakened or unsupported claim disclosed without changing the maintainer admission decision.

References

https://github.com/Jjschwartz/NetworkAttackSimulator
RAESystem/research#12

🛠️ Picked up by /implement - driver claude, branch 32-qualify-nasim-source, 2026-08-02T14:59:14.253Z.

gc workflow phase recorded: preflight (issue #32). Posted by the MCP server to enforce ordering between workflow steps (issue #794 MVP-2). Do not edit or delete — used by downstream tools to gate phase prerequisites.

gc workflow phase recorded: plan (issue #32). Posted by the MCP server to enforce ordering between workflow steps (issue #794 MVP-2). Do not edit or delete — used by downstream tools to gate phase prerequisites.

Plan — qualify and pin NASim public source + experiment profile (#32)

Requirement-free maintainer-qualification run, mirroring the CyberBattleSim
profile-only pattern (#25): a backend-local evidence module under
raes_adapters.nasim (loader + qualification.json + public-protocol.md) plus
one nasim extra and a structural test. No adapter, backend manifest,
conformance profile, RAES SDL, or raes_adapters.base change is delivered
(guardrails: docs/decisions/nasim-qualification-guardrails.md). All evidence
below was produced by actually acquiring, installing (clean venv, PYTHONPATH
cleared, safe-path), and running NASim — no value is fabricated.

Selected immutable source

Upstream Jjschwartz/NetworkAttackSimulator, tag v0.12.0, commit
7c732bc4620d20a25b221a782adee29c2a89d800, tree 4eedd4cc3ed9c31b17281baf447e67fcaa826298.
Publishable distribution nasim==0.12.0 (PyPI wheel sha256:4c059e64…,
sdist sha256:c5da8be9…, both verified). MIT (LICENSE.md sha256:1d8084ef…).

Selected qualified runtime

CPython 3.12.3, Linux glibc2.39. gymnasium 0.26.3 + numpy 1.26.4
(both within NASim's declared ranges). This pin is deliberate — see defect 1.

Selected public experiment protocol

Scenario: tiny static benchmark (nasim/scenarios/benchmark/tiny.yaml),
3 hosts / 3 subnets, one exploit e_ssh (prob 0.8), one privesc pe_tomcat
(prob 1.0), step_limit: 1000, sensitive hosts (2,0),(3,0).
Construction: nasim.make_benchmark("tiny", fully_obs=True, flat_actions=True, flat_obs=True)
(the documented bruteforce_agent.py tiny route). Flat action space n=18,
Box obs shape (56,) float32.
Baseline/evaluator: nasim.agents.bruteforce_agent.run_bruteforce_agent
(exhaustive round-robin; no torch, no model weights).
Gymnasium mapping: reset()->(obs,info); step()->(obs, reward, terminated=goal_reached, truncated=step_limit_reached, info).
Seed: np.random.seed(20260802) — the only binding seed (see defect 2).

Observed smoke evidence (real)

Representative step (action 0 = ServiceScan target=(1,0)): reward −1.0,
terminated False, truncated False, step 1.
Termination probe (supplied bruteforce, seed 20260802): steps 47, total
reward 153.0, goal True.
Truncation probe (unproductive action to step limit): steps 1000,
terminated False, truncated True — proving truncated is the step-limit
boundary, distinct from goal termination.

Findings recorded as known defects / limitations

Supplied agents break on gymnasium ≥ 0.27 — FlatActionSpace.get_action
asserts isinstance(action_idx, int) (nasim/envs/action.py:698); gymnasium
≥ 0.27 makes Discrete.n/.sample() return np.int64, so bruteforce/random
and the greedy paths of ql/dqn raise AssertionError. Qualified runtime pins
gymnasium 0.26.3, the last release where Discrete.n is a Python int.
Action-success RNG is unbound by the env seed — success is drawn from the
global np.random (network.py:87), never the gym-seeded self.np_random;
make_benchmark(seed)/reset(seed) do not make runs reproducible. Only an
explicit np.random.seed() does.
Static-benchmark seed ignored — make_benchmark_scenario calls
load_scenario for static YAML benchmarks and ignores seed entirely.
Headless import requires tkinter — nasim/envs/render.py unconditionally
import tkinter + matplotlib.use('TkAgg') at import (pulled in by
environment.py), so import nasim needs the Tk system libraries even for
headless use.
Model weight nasim/agents/policies/dqn_tiny.pt ships in the git repo
(for the dqn baseline) but is excluded from the PyPI wheel; the selected
tiny+bruteforce profile uses no weights.

Packaging (single distribution, one lock — ADR-003)

Add nasim = ["nasim==0.12.0"] to [project.optional-dependencies]; refresh
the single uv.lock. uv.lock currently holds no gymnasium/numpy/etc. and no
other extra constrains them, so no conflicts declaration is needed. Base,
cyberbattlesim, and cyborg extras remain independent (_verification_envs
verifies each alone). raes_adapters.nasim imports none of the simulator
stack, so CI never imports NASim (no tkinter needed in CI).

Files

src/raes_adapters/nasim/__init__.py — loader (load_qualification,
read_public_protocol), mirroring raes_adapters.cyberbattlesim.
src/raes_adapters/nasim/qualification.json — record_version: "nasim-qualification/1"; source/runtime identities + digests, source_files,
import-root trees (git 42-file + wheel 36-file, __pycache__-excluded),
runtime_artifacts (nasim/gymnasium/numpy wheels + trees), protocol selection,
dependencies (19, all permissive), stochastic_sources, legal, known_defects,
patches [], maintenance, admission + claim_strength.
src/raes_adapters/nasim/public-protocol.md — the experiment profile; its
sha256 pinned in the record.
pyproject.toml — nasim extra; uv.lock — refreshed.
tests/test_nasim_qualification.py — structural assertions (source identity,
bounded smoke, import-root attestation, protocol identities + stochastic
streams, legal/patch dispositions, admission + extras["nasim"] == []… i.e.
extras["nasim"] == ["nasim==0.12.0"]).
README.md — nasim section; docs/decisions/nasim-qualification-guardrails.md
- mkdocs.yml nav (already present from preflight) committed.

Claim strength (admission = admitted, maintainer-selection, target research#12)

source_identity attested; protocol_configuration attested;
execution_controls partial (unbound action RNG); run_evidence attestable;
outcome_reproduction stochastic-bounded (exploit prob 0.8 + agents require
gymnasium 0.26.3). Limitations disclosed; they bound claims, not the admission.

Verification

make verify (nox graph: hygiene, policy, lint, tool-tests, typecheck, tests,
distributions, docs). Branch carries no REQ/REP token → policy runs
--skip-requirement (genuine requirement-free maintenance).

gc_codex_review — cycle 1 of 1 (pre-push) on issue #32 (branch 32-qualify-nasim-source)
Diff mode: inline — the complete diff was supplied in one prompt

Core review

Verdict: ship-with-fixes

The change is shaped as backend-local NASim qualification evidence: it keeps NASim out of the shared base layer, records a bounded source-native protocol, and avoids presenting qualification as adapter or RAES-conformance semantics. The decisive seam is the optional extra: the documented qualified runtime must be what the distributable extra actually resolves, otherwise the evidence and the install contract diverge.

Blocking findings (1):

[one-off] Make the NASim extra enforce its qualified runtime pins — pyproject.toml:56
The extra declares only nasim==0.12.0, while the updated lock resolves its transitive dependencies to gymnasium==1.3.0 and numpy==2.5.1. The qualification record and protocol explicitly require gymnasium 0.26.3 / numpy 1.26.4 because later Gymnasium versions make NASim's supplied bruteforce agent fail. Consequently, raes-adapters[nasim] installs a known-incompatible runtime rather than the admitted protocol. Declare the qualified pins in the extra (and regenerate the lock/evidence hashes) so installation honors the published qualification contract.

Security review

Verdict: ship

This change adds backend-local NASim qualification evidence, a narrowly scoped optional dependency extra, and documentation without introducing an HTTP surface, user-controlled input path, credential handling, or execution of untrusted content. The resource loader reads only package-owned static files, while the supply-chain additions are registry-resolved and hash-locked. The design seam is appropriately confined to the NASim package and leaves the existing shared security boundaries unchanged.

No blocking findings.

gc_codex_review pre-push cycle 1 of 1 complete for issue #32 on branch '32-qualify-nasim-source'. Posted by the MCP server to enforce the pre-push hard-cap-1 contract (issues #796, #804, #906). Do not edit or delete — used by the next gc_codex_review (uncommitted) invocation to count cycles.

Review decision record — codex cycle 1 (issue #32)

Reviewer: codex
Cycle: 1

Architectural read:

Core reviewer: The change is shaped as backend-local NASim qualification evidence: it keeps NASim out of the shared base layer, records a bounded source-native protocol, and avoids presenting qualification as adapter or RAES-conformance semantics. The decisive seam is the optional extra: the documented qualified runtime must be what the distributable extra actually resolves, otherwise the evidence and the install contract diverge.

Security reviewer: This change adds backend-local NASim qualification evidence, a narrowly scoped optional dependency extra, and documentation without introducing an HTTP surface, user-controlled input path, credential handling, or execution of untrusted content. The resource loader reads only package-owned static files, while the supply-chain additions are registry-resolved and hash-locked. The design seam is appropriately confined to the NASim package and leaves the existing shared security boundaries unchanged.

Blocking findings: 1

Finding 1 — `one-off`

ID: F1
Title: [core] Make the NASim extra enforce its qualified runtime pins
Location: pyproject.toml:56
Decision: fix
Rationale: The extra declares only nasim==0.12.0, while the updated lock resolves its transitive dependencies to gymnasium==1.3.0 and numpy==2.5.1. The qualification record and protocol explicitly require gymnasium 0.26.3 / numpy 1.26.4 because…

Test-quality review — cycle 1 (pre-push, manual)

Run manually because gc_test_quality_review_cycle was blocked mid-run by the MCP
identity guard (implement_repo_identity_changed) in this linked-worktree /
concurrent-run setup. Filed upstream: autarchy-ai/Ground-Control#1502.

Scope: tests/test_nasim_qualification.py (new, 6 tests) and the digest re-pin
in tests/test_cyborg_qualification.py.

Assessment: The suite follows the accepted CyberBattleSim qualification-test
pattern — it locks the immutable evidence record and adds three genuine
cross-consistency checks that are not tautological:

sha256(public-protocol.md) recomputed == record.protocol.sha256 (guards
doc↔record drift);
record.packaging.extra_declaration == pyproject optional-dependencies.nasim
(guards record↔packaging drift);
record.dependency_resolution.qualified_pins ⊆ the nasim extra (guards that
raes-adapters[nasim] installs exactly the admitted runtime).

No over-mocking, no assertion that merely restates production constants, and the
smoke/termination/truncation facts are the checked-in evidence (re-running NASim in
CI is out of scope per the qualification guardrails). Bounded-output invariants are
asserted (raw_observation/info absent from the record).

Finding fixed this cycle: added the qualified_pins ⊆ extra cross-check,
directly reinforcing the codex finding that the extra must install the qualified
runtime.

Decision: fix (applied). No open test-quality findings.

Pre-merge readiness — ready for review & merge (#32)

Plain-English outcome: RAES now has one pinned, legally-usable, runnable NASim
source and one public experiment profile to build a NASim adapter against.
pip install raes-adapters[nasim] installs the exact qualified runtime
(NASim 0.12.0 + gymnasium 0.26.3 + numpy 1.26.4); the checked-in record fixes the
source, wheel, and protocol identities and grades what RAES can attest. Four
upstream limitations are disclosed (supplied agents break on gymnasium >= 0.27;
action-success RNG unbound by the env seed; static-benchmark seed ignored; tkinter
required to import). No adapter, manifest, or outcome-equivalence claim is made
here — that is downstream.

PR: #68 (base dev), head 44d988e.

Automated gates — all green:

CI: Fast checks, Policy, Tool tests, Typecheck, Tests, Distributions, Docs,
CodeQL, Lint PR title, GitGuardian, PR Gate — pass.
SonarCloud strict gate: OK (new_violations 0, new_coverage 100%, duplication
0.0%, reliability/security/maintainability A).
make verify + make policy completion gate green locally on the final tree.

Pre-push reviews: Codex — 1 finding (the nasim extra must pin the qualified
runtime) fixed; test-quality — run manually, consistency checks added. A follow-up
refactor commit extracts the shared qualification-evidence loader
(raes_adapters._qualification) to remove the SonarCloud new-code duplication the
nasim loader introduced.

Tooling note: the GC MCP identity guard refused this linked worktree mid-run
(implement_repo_identity_changed) because the concurrent issue-#39 run in the
sibling worktree rewrites the shared .git/config origin (RAESystem <-> OpenRAE).
Filed upstream autarchy-ai/Ground-Control#1502; the test-quality review, publish,
PR, CI/Sonar watch, and this readiness record were therefore driven manually.

Next: ready for your review and merge into dev. After merge, re-run
/implement 32 for Phase E (requirement transition — N/A, this is requirement-free;
traceability reconciliation; final report; issue close). Because issue #39
concurrently edits pyproject.toml, uv.lock, and
src/raes_adapters/cyborg/qualification.json, whichever PR merges second will need
a base-sync conflict resolution (re-run uv lock, re-pin the CybORG
uv_lock_sha256).

gc workflow phase recorded: traceability_reconciled (issue #32). Posted by the MCP server to enforce ordering between workflow steps (issue #794 MVP-2). Do not edit or delete — used by downstream tools to gate phase prerequisites.

no in-scope requirements; touched-files audit clean.

Final report — issue #32 complete

PR: #68
Plan: #32 (comment)

Outcome

RAES now has one pinned, legally-usable, runnable NASim source (v0.12.0) and one public tiny+bruteforce experiment profile to build a NASim adapter against. Installing raes-adapters[nasim] reproduces the exact qualified runtime (gymnasium 0.26.3 / numpy 1.26.4); the checked-in record fixes source/wheel/protocol identities and grades what RAES can attest, disclosing four upstream limitations (supplied agents break on gymnasium >= 0.27, action RNG unbound by the env seed, static-benchmark seed ignored, tkinter required to import).

Files changed

Added:

src/raes_adapters/nasim/__init__.py
src/raes_adapters/nasim/qualification.json
src/raes_adapters/nasim/public-protocol.md
src/raes_adapters/_qualification.py
tests/test_nasim_qualification.py
docs/decisions/nasim-qualification-guardrails.md

Modified:

pyproject.toml
uv.lock
mkdocs.yml
README.md
src/raes_adapters/cyborg/qualification.json
src/raes_adapters/cyberbattlesim/__init__.py
tests/test_cyborg_qualification.py

Reviews

codex: One finding: the nasim extra installed a known-broken runtime (gymnasium >= 0.27 breaks NASim's supplied agents); fixed by pinning the qualified runtime in the extra. Security review clean (ship).
test-quality: Run manually (MCP identity guard blocked the cycle). Tests are evidence-locking with md<->record, record<->pyproject, and qualified_pins<->extra consistency checks; added the pins<->extra cross-check.

Traceability reconciliation

IMPLEMENTS / TESTS / DOCUMENTS added: 0
Links updated: 0
Stale links removed: 0

Requirement-free maintainer-qualification run; this Ground Control project holds no requirements, so there are no requirement->code IMPLEMENTS/TESTS links to reconcile.

Status

CI: ✅ green
SonarCloud: ✅ passed
PR ready for user review and merge.

Navigation Menu

Repository navigation

build(nasim): qualify and pin the public source and experiment profile #32

Objective

Scope

Qualification semantics

Acceptance criteria

References

Plan — qualify and pin NASim public source + experiment profile (#32)

Selected immutable source

Selected qualified runtime

Selected public experiment protocol

Observed smoke evidence (real)

Findings recorded as known defects / limitations

Packaging (single distribution, one lock — ADR-003)

Files

Claim strength (admission = admitted, maintainer-selection, target research#12)

Verification

Core review

Security review

Review decision record — codex cycle 1 (issue #32)

Finding 1 — `one-off`

Test-quality review — cycle 1 (pre-push, manual)

Pre-merge readiness — ready for review & merge (#32)

Final report — issue #32 complete

Outcome

Files changed

Reviews

Traceability reconciliation

Status

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

build(nasim): qualify and pin the public source and experiment profile #32

Description

Objective

Scope

Qualification semantics

Acceptance criteria

References

Activity

Brad-Edwards commented on Aug 2, 2026

Brad-Edwards commented on Aug 2, 2026

Brad-Edwards commented on Aug 2, 2026

Plan — qualify and pin NASim public source + experiment profile (#32)

Selected immutable source

Selected qualified runtime

Selected public experiment protocol

Observed smoke evidence (real)

Findings recorded as known defects / limitations

Packaging (single distribution, one lock — ADR-003)

Files

Claim strength (admission = admitted, maintainer-selection, target research#12)

Verification

Brad-Edwards commented on Aug 2, 2026

Core review

Security review

Brad-Edwards commented on Aug 2, 2026

Brad-Edwards commented on Aug 2, 2026

Review decision record — codex cycle 1 (issue #32)

Finding 1 — one-off

Brad-Edwards commented on Aug 2, 2026

Test-quality review — cycle 1 (pre-push, manual)

Brad-Edwards commented on Aug 2, 2026

Pre-merge readiness — ready for review & merge (#32)

Brad-Edwards commented on Aug 2, 2026

Brad-Edwards commented on Aug 2, 2026

Final report — issue #32 complete

Outcome

Files changed

Reviews

Traceability reconciliation

Status

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

Finding 1 — `one-off`