Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .ground-control.yaml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
schema_version: 1
project: aces-adapters
github_repo: RAESystem/adapters
github_repo: OpenRAE/adapters
workflow:
test_command: make verify
completion_command: make verify
Expand Down
59 changes: 48 additions & 11 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
# raes-adapters

[![Documentation](https://readthedocs.org/projects/raes-adapters/badge/?version=latest)](https://raes-adapters.readthedocs.io/en/latest/)
[![OpenSSF Scorecard](https://api.scorecard.dev/projects/github.com/RAESystem/adapters/badge)](https://scorecard.dev/viewer/?uri=github.com/RAESystem/adapters)
[![OpenSSF Best Practices](https://www.bestpractices.dev/projects?as=badge&url=https%3A%2F%2Fgithub.com%2FRAESystem%2Fadapters)](https://www.bestpractices.dev/projects?as=entry&url=https%3A%2F%2Fgithub.com%2FRAESystem%2Fadapters)
[![OpenSSF Scorecard](https://api.scorecard.dev/projects/github.com/OpenRAE/adapters/badge)](https://scorecard.dev/viewer/?uri=github.com/OpenRAE/adapters)
[![OpenSSF Best Practices](https://www.bestpractices.dev/projects?as=badge&url=https%3A%2F%2Fgithub.com%2FOpenRAE%2Fadapters)](https://www.bestpractices.dev/projects?as=entry&url=https%3A%2F%2Fgithub.com%2FOpenRAE%2Fadapters)

A single distribution, **`raes-adapters`**, that qualifies and realizes
[RAES](https://github.com/RAESystem/rae) scenarios against concrete simulator
Expand Down Expand Up @@ -35,7 +35,7 @@ pip install raes-adapters # shared base plumbing + qualification evidence

The selected CybORG backend is admitted and usable through its documented
source installation. The `cyborg` extra key is dependency-light. Issue
[#12](https://github.com/RAESystem/adapters/issues/12) qualified the official
[#12](https://github.com/OpenRAE/adapters/issues/12) qualified the official
CAGE Challenge 2 source and a packaging-only fix, but the upstream wheel omits
the version and Scenario2 runtime data. The fixed wheel passed a clean Python
3.12 smoke locally, but it is not a governed public artifact and the
Expand All @@ -57,6 +57,16 @@ profile. Because no official index/release artifact exists, users install the
pinned simulator source separately. Unbound random streams and open benchmark
findings remain explicit limits on deterministic-replay and outcome claims.

The `primaite` extra is dependency-light for the same reason. The
[qualification record](src/raes_adapters/primaite/qualification.json) binds
DSTL's MIT-licensed PrimAITE source (tag `v4.0.0`) and admits the selected
`data_manipulation` profile through the source-native `PrimaiteGymEnv`. PrimAITE
publishes no index/release wheel, so users install the pinned source separately
(the qualified route pins `setuptools==75.6.0` to supply `pkg_resources`). A
broken public seed seam, undeclared runtime dependencies, and an unpinned
dependency graph remain explicit limits on deterministic-replay and
reproducibility claims.

The `nasim` extra pins the published `nasim==0.12.0` distribution together with
its qualified runtime (`gymnasium==0.26.3`, `numpy==1.26.4`), so installing
`raes-adapters[nasim]` reproduces the admitted, runnable protocol. The
Expand Down Expand Up @@ -127,6 +137,7 @@ raes-adapters/
cyborg/ # CybORG qualification, patch evidence, and future backend
mapping/ # pinned CAGE-2 → RAES source ledger (REP-003)
profiles/ # conformance profile overrides
primaite/ # immutable qualification + selected public protocol
nasim/ # immutable NASim qualification + selected public protocol
tests/ # pytest suite for the distribution
release-please-config.json # Release Please: versioning + CHANGELOG from main
Expand All @@ -150,7 +161,7 @@ backends land issue by issue:

## CyberBattleSim qualification

Issue [#25](https://github.com/RAESystem/adapters/issues/25) selects the
Issue [#25](https://github.com/OpenRAE/adapters/issues/25) selects the
official Microsoft source at commit
`854d6966607fb68645651f55b0f97221bd293e0d` and one public
`CyberBattleChain-v0` protocol with the credential-cache baseline and basic
Expand All @@ -161,7 +172,7 @@ semantics. The separate
[architecture guardrails](docs/decisions/cyberbattlesim-qualification-guardrails.md)
explain why this evidence is not an adapter manifest or RAES conformance claim.

Issue [#26](https://github.com/RAESystem/adapters/issues/26) authors the
Issue [#26](https://github.com/OpenRAE/adapters/issues/26) authors the
portable evidence set for that case: an authored RAES SDL scenario
(`scenario/cyberbattle-chain.sdl.yaml`) that validates and compiles against
`raes==2.0.0`, companion published experiment contracts
Expand All @@ -181,7 +192,7 @@ fix its boundaries.

## CyberBattleSim backend

Issue [#27](https://github.com/RAESystem/adapters/issues/27) implements a RAES
Issue [#27](https://github.com/OpenRAE/adapters/issues/27) implements a RAES
runtime target for the admitted size-10 `CyberBattleChain-v0` profile:

```python
Expand Down Expand Up @@ -233,7 +244,7 @@ The [backend architecture guardrails](docs/decisions/cyberbattlesim-backend-guar
record the component ownership, failure hygiene, capability claims, and
acceptance-test mapping.

Issue [#28](https://github.com/RAESystem/adapters/issues/28) composes that
Issue [#28](https://github.com/OpenRAE/adapters/issues/28) composes that
runtime target with the published RAES conformance report and adapter-local
source-protocol probes. The backend conformance result remains the exact
`BackendConformanceReport` from RAES and is serialized only through the
Expand All @@ -244,9 +255,35 @@ corpus, report schema, or research-validity claim. The
[conformance-composition guardrails](docs/decisions/cyberbattlesim-conformance-guardrails.md)
fix those boundaries.

## PrimAITE qualification

Issue [#39](https://github.com/OpenRAE/adapters/issues/39) selects the ARCD
PrimAITE source at tag `v4.0.0` (commit
`98617981d7f6ae2c3ffd9a8cc39944e05c9a09ea`) and one public `data_manipulation`
protocol driven through the source-native Gymnasium entrypoint
`primaite.session.environment.PrimaiteGymEnv`. The shipped
[protocol](src/raes_adapters/primaite/public-protocol.md) fixes the exact
scenario, participants (BLUE `proxy-agent`, scripted RED, probabilistic GREEN),
`Discrete(78)` action space, flattened `Box(1652,)` observation, seed
obligations, metrics, and the fixed-horizon truncation semantics (`terminated`
is always false; the episode truncates at `max_episode_length=128`).

The [qualification record](src/raes_adapters/primaite/qualification.json) binds
the source identity, the canonical import-root digest (which matches the built
wheel exactly), the MIT/Crown-copyright legal disposition, the clean-install and
bounded do-nothing smoke, the resolved dependency graph and its permissive
license summary, and the maintainer admission with graded claim strength. It
also records the source's honest limitations: no index/release wheel, an
undeclared `pkg_resources`/setuptools runtime dependency (the qualified route
pins `setuptools==75.6.0`), a public seed seam that raises without the `rl`/torch
stack, a `requires-python` vs classifier inconsistency, and an unpinned upstream
dependency graph. The
[qualification guardrails](docs/decisions/primaite-qualification-guardrails.md)
explain why this evidence is not an adapter manifest or RAES conformance claim.

## CybORG/CAGE-2 runtime qualification

Issue [#12](https://github.com/RAESystem/adapters/issues/12) selects the
Issue [#12](https://github.com/OpenRAE/adapters/issues/12) selects the
official CAGE Challenge 2 repository at commit
`26ce1c1253fa9e2e73f25e6a7f2da32860c11257`, including its bundled CybORG 2.1,
Scenario2, evaluator, wrappers, and baseline agents as one source closure. The
Expand All @@ -256,7 +293,7 @@ and sanitized red/blue/green smoke result. The accompanying
[packaging patch](src/raes_adapters/cyborg/cage2-wheel-package-data.patch) is
qualification evidence only; it is not silently applied or published.

Issue [#15](https://github.com/RAESystem/adapters/issues/15) supplies the
Issue [#15](https://github.com/OpenRAE/adapters/issues/15) supplies the
provisioning path for that backend. `create_cyborg_target()` accepts
admitted RAES provisioning plans and deterministically generates the native
CybORG scenario: RAES switches become subnets, VM multiplicity becomes hosts,
Expand All @@ -266,7 +303,7 @@ configuration-bound realization-envelope identity remain in the RAES snapshot;
native CybORG objects stay private. Unsupported or lossy node facts fail before
construction.

Issue [#16](https://github.com/RAESystem/adapters/issues/16) adds aggregate
Issue [#16](https://github.com/OpenRAE/adapters/issues/16) adds aggregate
logical-turn execution. A validated blue action is translated by exact contract
address and drives one source-native turn; the resulting blue, green, and red
occurrences are recorded in declared source order with shared-state, joint-action,
Expand Down Expand Up @@ -317,7 +354,7 @@ deterministic replay or scientific equivalence. See the

## NASim qualification

Issue [#32](https://github.com/RAESystem/adapters/issues/32) selects Jonathon
Issue [#32](https://github.com/OpenRAE/adapters/issues/32) selects Jonathon
Schwartz's official
[NASim](https://github.com/Jjschwartz/NetworkAttackSimulator) source at tag
`v0.12.0` (commit `7c732bc4620d20a25b221a782adee29c2a89d800`, published as
Expand Down
2 changes: 1 addition & 1 deletion docs/decisions/identity-register.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@
# granting a new exception is a reviewed edit to this file.
entries:
- path: .ground-control.yaml
digest: de1c598e28e28cbc1f336cf23a4549eb7ed214812b1ad04f2b7323741abd8e33
digest: 63d6cedefd87fe724b1d587bfe750d38f7d55610879124b587f1184544a77ee5
record_class: external-identity
owner: ground-control
rationale: >-
Expand Down
109 changes: 109 additions & 0 deletions docs/decisions/primaite-qualification-guardrails.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,109 @@
# PrimAITE qualification guardrails

Issue #39 is the authority for the qualification outcome. This note fixes the
repository and contract boundaries that outcome must respect; it does not select
an upstream revision, define a profile schema, or describe an implementation
plan.

Maintainer selection admits the PrimAITE source and public experiment profile.
Qualification records what RAES can attest or reproduce and explicitly grades
any limitation. It does not veto later adapter work or make RAES invent missing
simulator behavior.

## Keep the artifacts and claims separate

Place qualification evidence under `raes_adapters.primaite`. It is backend-local
package data, distinct from all of the following:

- a public, source-native experiment protocol, which names the chosen use case,
YAML configuration, red/green/blue agents, traffic model, action and
observation spaces, evaluator, rewards, seeds, metrics, and termination;
- a future RAES SDL, published experiment contract, backend manifest, or
conformance profile, each of which is owned by published RAES contracts and
must be supported by later implementation evidence; and
- CybORG and CyberBattleSim qualification records and ledgers, which are
precedents for local ownership rather than schemas to copy or generalize.

The record must bind the official repository, full commit and tree ids, archive
digest, installable distribution/wheel identity, selected files and symbols,
resolved dependency graph, Python and host identity, and every observed or
configured default that changes behavior. A tag, release label, use-case name,
notebook title, package version, or YAML filename alone is not immutable.

Qualification must separately identify the unmodified source and every patch.
For a compatibility patch, record the patch and resulting tree/artifact
digests, purpose, license, semantic effect, and a bounded comparison with the
unmodified source. A dependency override, monkey patch, edited example, copied
file, or setup-time mutation that is not recorded is a prohibited silent patch.

Record license, notices/attribution, redistribution and retained-output rights,
external downloads, datasets, model weights, cache behavior, archive/
maintenance state, known upstream defects, and the evidence supporting each
conclusion. Refer to large upstream artifacts by immutable identity and digest;
do not vendor them unless necessary and explicitly permitted.

## Reuse the canonical boundaries

| Concern | Canonical incumbent | Required use |
| --- | --- | --- |
| Backend-local qualification evidence | `raes_adapters.cyborg.qualification` and `raes_adapters.cyberbattlesim` | Use package resources and small accessors; do not create a shared profile loader, DTO, registry, or schema. |
| Packaging and isolation | `pyproject.toml`, one `uv.lock`, ADR-003, `_extras()` and `_verification_envs()` in `noxfile.py` | Add only `primaite`; keep base and every extra independently resolvable. Add a uv `conflicts` declaration only for a demonstrated, recorded incompatibility. |
| Clean built-artifact proof | `_distributions()` and `tools/probe_installed_identity.py` | Extend the existing wheel/sdist, throwaway-venv proof rather than adding a second install workflow. The native smoke runs outside the checkout with `PYTHONPATH` cleared and safe-path/isolation enabled. |
| Portable experiment artifacts | `ExperimentTaskModel`, `ExperimentSpecModel`, `parse_experiment_spec`, `ContractModel` (`extra="forbid"`), and RAES cross-artifact validation | Use only if machine-readable RAES protocol/evidence is emitted; do not define local YAML/JSON schemas, permissive parsing, or duplicate validation. |
| Artifact/provenance identity | `ExperimentArtifactRefModel`, `ExperimentChecksumModel`, `ExperimentScenarioSnapshotReferenceModel`, and apparatus models | Use published meanings and checksum forms when source identity is referenced in RAES artifacts; keep legal and qualification metadata backend-local. |
| Stochastic control | `ExperimentStochasticControlModel`, `PublicSeedModel`, and `RandomStreamControlBindingModel` | Record simulator, use-case/topology, traffic, scheduler, action/observation-space, red/green/blue policy, evaluator, Python, NumPy and library RNGs separately. A top-level seed is not evidence of complete control. |
| Metrics and outcomes | `ExperimentEvaluationProtocolModel`, `ExperimentMetricDefinitionModel`, `ExperimentEvidenceRecordModel`, and `ExperimentDerivedMeasureModel` | Keep native reward, evaluator output, study metric, evidence, and derived measure distinct. |
| Failures and diagnostics | RAES `Diagnostic`, `DiagnosticModel`, `ApplyResult`, plus `raes_adapters.base.redaction` | Later portable errors use bounded, input-free diagnostics; never carry native exception text, rejected values, raw observations, reward vectors, action ids, object reprs, paths, env dumps, or tracebacks. |
| Durable state and workflow | Checked-in package resources, ephemeral temp directories, and the `policy`, `typecheck`, `tests`, `distributions`, `docs`, and `verify` nox sessions | Do not add a database, cache authority, evidence service, standalone validator, or CI workflow. |

`raes_adapters.base` remains plumbing only. This issue does not justify a
generic simulator protocol, qualification exception hierarchy, backend catalog,
schema registry, or shared persistence layer.

## Native smoke, security, and observability

The source-native smoke precedes adapter normalization. From the documented
supported route it must construct the selected case, reset with the selected
seed, issue one representative valid action/step, and record bounded facts about
the returned observation and reward/result shapes. It must exercise a bounded
path to every selected termination behavior and record their identity,
precedence, and off-by-one semantics. It must also disclose undisclosed
defaults, incompatibilities, runtime downloads/writes, network access,
subprocesses, and stochastic sources.

This is a local library execution: no HTTP, authentication, authorization, or
secret-binding surface is introduced. Public sources only; no credentials,
private downloads, token-bearing argv, environment dumps, or home-cache
evidence. Use argument vectors, explicit temporary/cache paths, isolated
working directories, frozen resolution, and bounded structural output. A later
runtime service must use RAES runtime strict defaults, verified identities,
target/role authorization, request-size guards, denial audit, and its redacted
exception handler; qualification creates no alternate path.

Use ordinary module logging only for bounded local operational facts. Portable
observability belongs to RAES diagnostics and experiment-evidence contracts.
Native state, traffic contents, hidden truth, learned credentials, full action
or observation payloads, raw logs, tracebacks, and full reward vectors remain
source-private. A digest of a native-state dump does not make it portable.

## Extension seam and boundaries

The one extension seam is an explicit module-local immutable selection identity
passed to the source-native smoke runner. Source paths, use case, YAML files,
agents, traffic configuration, evaluator, seed, representative action, metric,
and termination bounds come from that selection, not repeated across tests,
scripts, notebooks, and prose. A second public PrimAITE profile must be
addable as another selection without editing a central registry or altering the
first profile.

Non-goals: adapter semantics, RAES SDL, backend manifest, conformance profile,
participant implementation, persistence, environment-pack content, and any
claim of deterministic replay, scientific validity, or outcome equivalence.

Do not treat a successful import, notebook, one episode, or green CI as proof
of reproducibility, legal usability, public installability, determinism, or
equivalence. Do not conflate source-native `done`/`terminated`/`truncated`,
scenario success/failure, reward thresholds, evaluator cutoffs, max steps, and
participant stop conditions. Do not lower the repository Python boundary, hide
an incompatibility in an environment marker, make PrimAITE a base dependency,
or use another simulator extra to satisfy it transitively.
5 changes: 3 additions & 2 deletions docs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,16 +17,17 @@ semantic and protocol authority.

## Start here

- [Repository overview](https://github.com/RAESystem/adapters#readme)
- [Repository overview](https://github.com/OpenRAE/adapters#readme)
- [Installed researcher command](researcher-command.md)
- [Contribution guide](https://github.com/RAESystem/adapters/blob/dev/CONTRIBUTING.md)
- [Contribution guide](https://github.com/OpenRAE/adapters/blob/dev/CONTRIBUTING.md)
- [Architecture decisions](decisions/adrs/README.md)
- [CybORG/CAGE-2 backend qualification guardrails](decisions/cyborg-cage2-runtime-qualification-guardrails.md)
- [CybORG/CAGE-2 source-ledger guardrails](decisions/cyborg-cage2-source-ledger-guardrails.md)
- [CybORG/CAGE-2 provisioner and backend-manifest guardrails](decisions/cyborg-cage2-provisioner-manifest-guardrails.md)
- [CybORG conformance-composition guardrails](decisions/cyborg-conformance-guardrails.md)
- [CybORG researcher run-and-evidence command guardrails](decisions/cyborg-researcher-command-guardrails.md)
- [CyberBattleSim qualification guardrails](decisions/cyberbattlesim-qualification-guardrails.md)
- [PrimAITE qualification guardrails](decisions/primaite-qualification-guardrails.md)
- [CyberBattleSim scenario and source-ledger guardrails](decisions/cyberbattlesim-scenario-ledger-guardrails.md)
- [CyberBattleSim backend architecture guardrails](decisions/cyberbattlesim-backend-guardrails.md)
- [CyberBattleSim conformance-composition guardrails](decisions/cyberbattlesim-conformance-guardrails.md)
Expand Down
Loading
Loading