Repository navigation
build(primaite): qualify and pin the public source and experiment profile #39
Description
Activity
🛠️ Picked up by /implement - driver claude-code, branch
39-qualify-primaite-profile, 2026-08-02T14:59:14.953Z.gc workflow phase recorded:
preflight(issue #39). Posted by the MCP server to enforce ordering between workflow steps (issue #794 MVP-2). Do not edit or delete — used by downstream tools to gate phase prerequisites.gc workflow phase recorded:
plan(issue #39). Posted by the MCP server to enforce ordering between workflow steps (issue #794 MVP-2). Do not edit or delete — used by downstream tools to gate phase prerequisites.Plan — qualify and pin the public PrimAITE source and experiment profile
Requirement-free qualification run (no
REP-*in scope; governance via
--skip-requirement). This mirrors the CyberBattleSim (#54) and CybORG (#56)
qualification precedent: a backend-local immutable qualification record plus one
selected public experiment protocol underraes_adapters.primaite. No adapter,
SDL, backend manifest, or conformance claim is implemented here. The Step 2.5
guardrails (docs/decisions/primaite-qualification-guardrails.md) are binding.Selected immutable source profile (already investigated, real evidence)
- Repository
Autonomous-Resilient-Cyber-Defence/PrimAITE, tag v4.0.0
(annotated tagf6513362e9882217fb90595de51e7ca659ee1416), commit
98617981d7f6ae2c3ffd9a8cc39944e05c9a09ea, tree
3928ab452c51206ba8706b3f575e31fe99eb842e. - Distribution
primaite==4.0.0,requires-python >=3.9,<3.13. Not on PyPI
— the README documents wheel-from-GitHub install only (packaging gap, same
class as CyberBattleSim). Import rootprimaitecanonicalized to
sha256 9fe7ba15…over 234 files; the built wheel's tree matches the source
tree exactly. - License: MIT, Crown copyright 2023 DSTL UK. GitHub reports
NOASSERTION
because theLICENSEpreamble is mangled ("MIT License License …"); the
grant/warranty clauses are standard MIT and the string propagates into wheel
metadata (recorded finding).
Selected public experiment protocol
- Shipped use case
data_manipulation.yaml(metadata.version 3.0,
max_episode_length 128), driven through the source-native Gymnasium
entrypointprimaite.session.environment.PrimaiteGymEnv. - Participants: BLUE
defender(proxy-agent, the RL-controlled seat), RED
data_manipulation_attacker(red-database-corrupting-agent, scripted), two
GREENprobabilistic-agentusers. Baseline participant = scriptable policy
over the action space (do-nothing baseline used for the smoke). - Spaces (observed): action
Discrete(78); observation flattened
Box(shape=[1652], int64, 0..1). Reset arity 2 (ndarrayobs + empty info),
step arity 5. Reward is a float. - Termination:
step()hardcodesterminated=False; the episode ends only
bytruncated=Truewhenstep_counter >= max_episode_length(128).
Fixed-horizon, truncation-only — no source-native success/failureterminated
condition in this scenario.
Clean install + source-native smoke (documented supported route, verified)
- Build the wheel from pinned source, install into a fresh venv without the
rlextra (README-documented option), pinsetuptools==75.6.0(upstream
dev pin; supplies thepkg_resourcesmodule the runtime imports), run under an
isolatedHOME, clearedPYTHONPATH,PYTHONSAFEPATH=1, no post-install
network. Qualified on Python 3.11.15 (classifier/README supported range). - Smoke (do-nothing baseline, observed identical across 3 fresh-process runs
despite uncontrolled green RNG): representative first step reward0.65,
terminated=false,truncated=false; full episode 128 steps ending
truncated=true, final step reward-0.8, cumulative-57.05. Recorded as
observed-stable bounded evidence, not a seed-controlled determinism claim.
Recorded findings / limitations (all graded, none vetoing)
- Undeclared runtime dependency on
pkg_resources/setuptools: breaks with
setuptools>=81(module removed) and on default Python-3.12 venvs (no
setuptools seeded). Qualified route pinssetuptools==75.6.0. reset(seed=int)raisesKeyError('torch')—set_random_seeddereferences
sys.modules["torch"]unconditionally, so the public seed seam is unusable
without the heavyrl/torch stack. Seeded replay is therefore not attestable
on the light route.requires-python <3.13(metadata) vs classifiers/README<3.12
inconsistency; qualified on 3.11.typer[all]extra removed upstream (benign warning);primaiteCLI also
imports undeclaredclick, soprimaite setup/CLI is broken on the no-rl
route — the Gym runtime does not need it (import creates app dirs).jupyterlab==3.6.1is a core (non-optional) dependency, so even the
no-rl install resolves 133 packages. Upstream ships no lockfile and uses loose
constraints, so the graph is a dated resolved snapshot (normalized
sha256 b05fa4c0…), not reproducible. All licenses permissive/weak-copyleft
(MIT/BSD/Apache/ISC/PSF/MPL-2.0), no strong copyleft; one transitive dep
(ypy-websocket 0.8.4) declares no license.- Green
ProbabilisticAgent.rngseeds from the uncontrolled global NumPy RNG;
documented as a stochastic source.
Files
src/raes_adapters/primaite/__init__.py—load_qualification()+
read_public_protocol()accessors (mirrorcyberbattlesim).src/raes_adapters/primaite/qualification.json— the immutable record: source
identity, runtime source tree, source-file digests, wheel/sdist evidence,
Python/host runtime, clean-install + bounded smoke, protocol selection,
stochastic sources, dependency resolution + license summary + full resolved
graph, legal, patches (none), maintenance, known defects, packaging decision,
admission + graded claim strength.src/raes_adapters/primaite/public-protocol.md— the selected protocol prose
(itssha256is asserted by the record).tests/test_primaite_qualification.py— assert record identity, spaces,
termination model, smoke bounds, stochastic disclosure, legal/patch
dispositions, admission, andextras["primaite"] == [].pyproject.toml— add dependency-lightprimaite = []extra;uv lockto
register it (no second lockfile; noconflictsneeded while the extra is
empty — the incompatiblegymnasium==0.28.1/numpy~=1.23/pydantic==2.7.0
pins never enter the lock because the source installs separately).README.md— list PrimAITE among qualified backends.- Kept from preflight:
docs/decisions/primaite-qualification-guardrails.md,
docs/index.md,mkdocs.yml.
Bounded-artifact rule honored: no native observations, traffic contents, action
ids, reward vectors, object reprs, paths, or tracebacks enter the record — only
types, shapes, dtypes, booleans, scalar measures, and stable digests.Verification
uv run --frozen ruff+mypy src+ fullnox -s verify(hygiene, policy with
--skip-requirement, lint, tool-tests, typecheck, tests across base + each
extra alone incl.primaite, distributions clean-install/identity, strict
docs). Coverage ≥ 80%. NoCHANGELOG.md/version edits (Release Please owns
them); PR title will be a valid Conventional Commit (build(primaite): …).- Repository
gc_codex_review — cycle 1 of 1 (pre-push) on issue #39 (branch
39-qualify-primaite-profile)
Diff mode: inline — the complete diff was supplied in one promptCore review
Verdict:
shipThis is shaped correctly: it adds backend-local PrimAITE qualification evidence and a source-native protocol without promoting backend facts into RAES or
basesemantics. The empty optional extra accurately avoids asserting installability for an unpublished upstream artifact, while the resource accessors match the existing qualification-evidence seam. The record explicitly bounds reproducibility claims and preserves the obvious next variation—another module-local profile selection—without introducing a registry or premature abstraction.No blocking findings.
Security review
Verdict:
shipThis change adds backend-local PrimAITE qualification evidence, a dependency-light optional extra, and package-resource accessors without introducing a runtime service, untrusted-input boundary, or execution path. The design seam is correctly contained in
raes_adapters.primaite; the record documents the separate upstream-install limitation rather than silently acquiring or executing remote content. I found no concrete, exploitable security regression in the supplied diff.No blocking findings.
gc_codex_review pre-push cycle 1 of 1 complete for issue #39 on branch '39-qualify-primaite-profile'. Posted by the MCP server to enforce the pre-push hard-cap-1 contract (issues #796, #804, #906). Do not edit or delete — used by the next
gc_codex_review(uncommitted) invocation to count cycles.Review decision record — codex cycle 1 (issue #39)
Reviewer: codex
Cycle: 1Architectural read:
Core reviewer: This is shaped correctly: it adds backend-local PrimAITE qualification evidence and a source-native protocol without promoting backend facts into RAES or
basesemantics. The empty optional extra accurately avoids asserting installability for an unpublished upstream artifact, while the resource accessors match the existing qualification-evidence seam. The record explicitly bounds reproducibility claims and preserves the obvious next variation—another module-local profile selection—without introducing a registry or premature abstraction.Security reviewer: This change adds backend-local PrimAITE qualification evidence, a dependency-light optional extra, and package-resource accessors without introducing a runtime service, untrusted-input boundary, or execution path. The design seam is correctly contained in
raes_adapters.primaite; the record documents the separate upstream-install limitation rather than silently acquiring or executing remote content. I found no concrete, exploitable security regression in the supplied diff.Blocking findings: 0 (clean run)
gc_test_quality_review cycle 1 of 1 — issue #39
Reviewer: test-quality (claude-sonnet-5 via gc_test_quality_review)
Branch:39-qualify-primaite-profile
Cycle: 1 / 1
Findings: 0 (clean run)gc_test_quality_review cycle 1 of 1 complete for issue #39 on branch '39-qualify-primaite-profile'. Posted by the MCP server to enforce the gc_test_quality_review hard-cap-1 contract (issue #884 follow-up, default lowered in #906). Do not edit or delete — used by the next
gc_test_quality_reviewinvocation to count cycles.Review decision record — test-quality cycle 1 (issue #39)
Reviewer: test-quality
Cycle: 1Architectural read:
Both files are golden-record tests over qualification-evidence modules whose entire implementation is "read a packaged resource and parse/return it verbatim" — there's no business logic to exercise, so pinning the record's exact content is the correct test shape here, not a shortcut around one. The primaite module deliberately mirrors the cyborg qualification accessor's shape (load_qualification/read_*) while explicitly declining to implement an adapter yet, per the ADR — a sound seam that keeps qualification evidence decoupled from adapter construction and doesn't foreclose the adapter work that will presumably follow (as it did for cyborg). Where independent verification is possible without a full external reproduction run, the tests do it (dependency-digest recomputation, patch-content hashing, pyproject.toml cross-checks) rather than just re-asserting literals pulled from the same JSON. The expensive, real reproduction (git clone, wheel builds, subprocess) is correctly isolated in tools/verify_cyborg_qualification.py, outside the pytest suite, so these aren't integration tests in unit-test clothing. I'd ship this as-is.
Blocking findings: 0 (clean run)
Pre-PR base synchronization
- Source:
refs/remotes/origin/devat0f20e975fb7e3f583f4bd6bee1be30da78919eca - Outcome:
already_current - Published feature head:
b7cf56c05858ed46603ac3b242d7657ab2e2bfa7 - Verified tree:
5bc7da9073c6520bd39a265cfa40bb243a2d8afb
- Source:
- added a commit that references this issue
on Aug 2, 2026 Ready for review — issue #39
PR: #67
Plan: #39 (comment)Outcome
PrimAITE v4.0.0 is qualified and pinned as backend-local evidence under raes_adapters.primaite, with a bounded source-native smoke and graded limitations. Bundled at maintainer request: raes upgraded to 3.3.0 (env-packs 3.6.2) and the active GitHub identity reconciled to OpenRAE/adapters (Sonar key and digest-pinned historical evidence unchanged). Branch is merged current with dev (incl. NASim #68). Base and all four simulator extras stay independently installable; no existing backend's runtime behavior changes.
Files changed
Added:
docs/decisions/primaite-qualification-guardrails.mdsrc/raes_adapters/primaite/__init__.pysrc/raes_adapters/primaite/public-protocol.mdsrc/raes_adapters/primaite/qualification.jsontests/test_primaite_qualification.py
Modified:
.ground-control.yamlREADME.mddocs/decisions/identity-register.yamldocs/index.mddocs/maintainers/project-services.mdmkdocs.ymlpyproject.tomlsrc/raes_adapters/cyborg/qualification.jsontests/test_cyberbattlesim_qualification.pytests/test_cyborg_conformance.pytests/test_cyborg_qualification.pytests/test_nasim_qualification.pytools/check_project_services.pytools/tests/test_identity_policy.pyuv.lock
Reviews
- codex: Cycle 1 clean (0 findings) on the PrimAITE core; the raes 3.x + OpenRAE delta exceeded pre-push cap-1, over-cap re-review declined by maintainer, accepted on green full-verify + identity gates.
- test-quality: Cycle 1 clean (0 findings) on the PrimAITE qualification tests.
Traceability reconciliation
- Pending — requirement status transition and IMPLEMENTS/TESTS reconciliation run in Phase E once the PR merges.
Status
- CI: ✅ green
- SonarCloud: ✅ passed
- PR ready for user review and merge. Ground Control reconciliation (requirement status + traceability) runs on merge (Phase E).
- added a commit that references this issue
on Aug 2, 2026 gc workflow phase recorded:
traceability_reconciled(issue #39). Posted by the MCP server to enforce ordering between workflow steps (issue #794 MVP-2). Do not edit or delete — used by downstream tools to gate phase prerequisites.no in-scope requirements; touched-files audit clean.
Final report — issue #39 complete
PR: #67
Plan: #39 (comment)Outcome
PrimAITE v4.0.0 is qualified and pinned as backend-local evidence under raes_adapters.primaite, with a bounded source-native smoke and graded limitations. Bundled at maintainer request: raes upgraded to 3.3.0 (env-packs 3.6.2) and the active GitHub identity reconciled to OpenRAE/adapters (Sonar key and digest-pinned historical evidence unchanged). Merged to dev in PR #67. Base and all four simulator extras stay independently installable; no existing backend's runtime behavior changes.
Files changed
Added:
docs/decisions/primaite-qualification-guardrails.mdsrc/raes_adapters/primaite/__init__.pysrc/raes_adapters/primaite/public-protocol.mdsrc/raes_adapters/primaite/qualification.jsontests/test_primaite_qualification.py
Modified:
.ground-control.yamlREADME.mddocs/decisions/identity-register.yamldocs/index.mddocs/maintainers/project-services.mdmkdocs.ymlpyproject.tomlsrc/raes_adapters/cyborg/qualification.jsontests/test_cyberbattlesim_qualification.pytests/test_cyborg_conformance.pytests/test_cyborg_qualification.pytests/test_nasim_qualification.pytools/check_project_services.pytools/tests/test_identity_policy.pyuv.lock
Reviews
- codex: Cycle 1 clean (0 findings) on the PrimAITE core; raes 3.x + OpenRAE delta exceeded pre-push cap-1, over-cap re-review declined by maintainer, accepted on green full-verify + identity gates.
- test-quality: Cycle 1 clean (0 findings) on the PrimAITE qualification tests.
Traceability reconciliation
- IMPLEMENTS / TESTS / DOCUMENTS added: 0
- Links updated: 0
- Stale links removed: 0
Requirement-free qualification run (no REP-* in scope; no new structural gate). No IMPLEMENTS/TESTS links to reconcile.
Status
- CI: ✅ green
- SonarCloud: ✅ passed
- PR ready for user review and merge.
Objective
Establish one immutable, legally usable, runnable PrimAITE source and experiment profile for the RAES study.
Scope
primaiteextra; use uv conflicts declarations where simulator stacks are mutually incompatible.Qualification semantics
The maintainer-selected simulator and experiment profile are admitted inputs to the RAES study. Qualification records the exact source and installation route, what RAES can attest or reproduce, stochastic-control coverage, known upstream defects, and limits on claim strength. Packaging gaps, incomplete determinism, compatibility limits, and source defects are disclosed limitations; they do not by themselves veto adapter implementation or require RAES to create behavior the simulator lacks.
Acceptance criteria
References