Repository navigation
Realization semantics for runtime.forwarding_agents (ADR-050): what does 'realized' mean for a non-executable inventory concern, and how are divergent/exogenous agent postures expressed? #1043
Description
Activity
🛠️ Picked up by /implement - driver codex, branch
1043-forwarding-agent-semantics, 2026-08-01T21:42:30.681Z.gc workflow phase recorded:
plan(issue #1043). Posted by the MCP server to enforce ordering between workflow steps (issue #794 MVP-2). Do not edit or delete — used by downstream tools to gate phase prerequisites.Implementation plan
-
Drive the public semantics with tests first.
- Add issue-1043 tests for the closed forwarding-agent ownership role, including the backward-compatible
system_under_testdefault. - Add semantic-validation tests proving
measurement_apparatusagents require an inboundEvidenceRequirement.source_refsbinding classified as apparatus, and rejecting apparatus-class bindings to system-under-test agents. - Add manifest/planner/runtime tests proving exact
forwarding-agentsdemands fail closed without sufficient concern-scoped corroboration and pass only with a matching safe observation disclosure; preserve configuration comparison and ensure operational forwarding is not inferred. - Add contract round-trip/schema tests for manifest and runtime-snapshot carriers and serializer/store paths.
- Add issue-1043 tests for the closed forwarding-agent ownership role, including the backward-compatible
-
Add the authored role and evidence-plane binding.
- Extend the existing forwarding-agent vocabulary/model with the closed
system_under_test | measurement_apparatusownership role; do not add an execution/health/status field. - Reuse the declaration index and
SemanticValidatorto make scenario- and node-scoped agents targetable evidence sources and enforce the single-directional evidence binding. - Keep host paths, evidence destinations, credentials, and capture payloads solely in existing evidence/capture contracts.
- Extend the existing forwarding-agent vocabulary/model with the closed
-
Add graded realization corroboration on canonical incumbents.
- Reuse the existing
ObservationStrength; add a closedpresence | configurationverification-scope axis and typed concern-keyed capability profiles toRealizationSupportDeclaration/ backend-manifest-v2. - Add a value-free realization-observation disclosure to the existing
RuntimeSnapshot/ runtime-snapshot-v1 carrier and its API/store/conformance serializers. It will identify address, field path, concern kind, scope, and strength only. - Extend the existing realization concern descriptor, planner capability check, and
_call_backend_apply()runtime gate so exact forwarding inventory requires adequate declared and returned corroboration before the safe configuration projection can match. Missing, malformed, or under-scoped disclosure returns the existing fail-closed diagnostic. - Keep
project_forwarding_agentsas the sole value projection, keep role outside backend-observed configuration equality, and reserve behavioral truth for existing ADR-079 propositions/evidence.
- Reuse the existing
-
Publish the decision and contract evolution.
- Amend ADR-050, ADR-056, ADR-066, and SEM-218 formal/spec catalogs; record issue-1043 amendment rows and update ADR-059 pins.
- Regenerate affected SDL, backend-manifest, and runtime-snapshot schemas from the Python authority; update valid/invalid fixtures and every changed contract entry's
last_changesummary/hash. - Preserve concept-authority locations, avoid new authority surfaces, and do not edit
CHANGELOG.mdor the version.
-
Verify at the correct boundaries.
- Run focused issue-1043/model/semantic/manifest/runtime tests during red-green-refactor, then the affected contract and policy tests.
- Before completion run, with
RAES_REQUIREMENT_UID=SEM-218,implementations/python/.venv/bin/python tools/check_repo_policy.py,tools/check_requirement_governance.py, andtools/verify_all.py; Ground Control then runs the configured completion and policy gates. - After merge, reconcile SEM-218 IMPLEMENTS/TESTS/DOCUMENTS traceability to the final code, tests, ADR/spec, and contract artifacts.
Security/whole-repo constraints: parser/model closure and semantic reference validation stay authoritative; compiler code consumes typed models only; backend-return validation precedes persistence/API conversion; safe commitments and ADR-056 redaction remain intact; no native inspect payload, secret, evidence location, command probe, log heuristic, or argv value enters snapshots or diagnostics. Maintainability reuses the existing forwarding model, declaration index, SEM-218 descriptor/projection route, support declaration, snapshot envelope, serializers, and proposition/evidence carriers. Extensibility is the concern-keyed verification profile: another runtime-agent family can reuse the role vocabulary and corroboration seam without a universal agent schema or a second realization taxonomy.
-
Pre-PR base synchronization
- Source:
refs/remotes/origin/devatb3cd98386db6268fb21ccc2500ba91b210790114 - Outcome:
already_current - Published feature head:
4dd8a4641e04f46b83f84055742ba9171d367df7 - Verified tree:
8927c69df98a0f5b4b80187f1c0cfd2f16609a9e
- Source:
Pre-PR base synchronization
- Source:
refs/remotes/origin/devatb3cd98386db6268fb21ccc2500ba91b210790114 - Outcome:
already_current - Published feature head:
1a75a7f1d6aa884dcdda044b70a1a0e2ac8f2ba6 - Verified tree:
c93b3ab12f61c1c77dfafc67ac5b63ce8abff0f7
- Source:
Pre-PR base synchronization
- Source:
refs/remotes/origin/devatb3cd98386db6268fb21ccc2500ba91b210790114 - Outcome:
already_current - Published feature head:
da26374f6b3851dd94c21cff6798c9fde924d824 - Verified tree:
6d55bb0ca255ffebadc62c63236efc09650ba02c
- Source:
Final report — issue #1043 complete
PR: #1047
Plan: #1043 (comment)Outcome
Forwarding-agent declarations now distinguish system-under-test behavior from measurement apparatus and bind apparatus agents to evidence requirements. Exact forwarding-agent realization now requires declared and returned presence or configuration corroboration before it can be accepted.
Phase E reconciliation completed for the merged SEM-218 forwarding-agent realization semantics.
In-scope requirements
SEM-218(Explicitness And Realization Semantics) — ACTIVE
Files changed
Added:
contracts/fixtures/snapshots/runtime-snapshot-v1/invalid/realization-observation-missing-strength.jsondocs/decisions/issue-1043-forwarding-agent-realization-posture-preflight.mdimplementations/python/tests/test_issue_1043_forwarding_agent_posture.pyimplementations/python/tests/test_issue_1043_realization_corroboration.py
Modified:
contracts/fixtures/backend-manifest/backend-manifest-v2/valid/feature-support-bounded.jsoncontracts/fixtures/backend-manifest/backend-manifest-v2/valid/stub.jsoncontracts/fixtures/snapshots/runtime-snapshot-v1/valid/realization-provenance.jsoncontracts/schema-publication/entries/backend-manifest-v2.jsoncontracts/schema-publication/entries/instantiated-scenario-snapshot-v1.jsoncontracts/schema-publication/entries/instantiated-scenario-v1.jsoncontracts/schema-publication/entries/runtime-snapshot-v1.jsoncontracts/schema-publication/entries/scenario-satisfiability-evidence-v1.jsoncontracts/schema-publication/entries/sdl-authoring-input-v1.jsoncontracts/schemas/backend-manifest/backend-manifest-v2.jsoncontracts/schemas/satisfiability/scenario-satisfiability-evidence-v1.jsoncontracts/schemas/sdl/instantiated-scenario-snapshot-v1.jsoncontracts/schemas/sdl/instantiated-scenario-v1.jsoncontracts/schemas/sdl/sdl-authoring-input-v1.jsoncontracts/schemas/snapshots/runtime-snapshot-v1.jsondocs/conformance/libvirt-qemu.provisioning-only.report.jsondocs/decisions/adrs/adr-050-forwarding-agent-runtime-inventory.mddocs/decisions/adrs/adr-056-runtime-observed-values-and-credential-posture.mddocs/decisions/adrs/adr-066-observability-evidence-plane-separation.mddocs/decisions/adrs/adr-index.yamldocs/explain/reference/shared-semantic-integrity.mddocs/research/formal-semantic-validation/bundles/base-v1.2.jsondocs/research/formal-semantic-validation/bundles/retest-v2.jsondocs/research/formal-semantic-validation/bundles/satisfiability-v1.jsondocs/research/formal-semantic-validation/execution-snapshot-v1.2.jsondocs/research/formal-semantic-validation/execution-snapshot-v2.jsonimplementations/python/packages/raes/_declarations.pyimplementations/python/packages/raes/runtime_forwarding_agent.pyimplementations/python/packages/raes/runtime_forwarding_agent_vocab.pyimplementations/python/packages/raes/validator/_evidence_requirements.pyimplementations/python/packages/raes_backend_protocols/manifest.pyimplementations/python/packages/raes_conformance/conformance/snapshot_semantics.pyimplementations/python/packages/raes_contracts/apparatus.pyimplementations/python/packages/raes_contracts/contracts/__init__.pyimplementations/python/packages/raes_contracts/contracts/_exports.pyimplementations/python/packages/raes_contracts/contracts/capabilities.pyimplementations/python/packages/raes_contracts/contracts/realization_plans.pyimplementations/python/packages/raes_contracts/realization_envelope_carrier.pyimplementations/python/packages/raes_contracts/realization_observation.pyimplementations/python/packages/raes_contracts/runtime_state.pyimplementations/python/packages/raes_contracts/vocabulary.pyimplementations/python/packages/raes_processor/compiler/realization_requirements.pyimplementations/python/packages/raes_processor/semantics/realization.pyimplementations/python/packages/raes_processor/semantics/realization_concern_projections.pyimplementations/python/packages/raes_processor/semantics/realization_concerns.pyimplementations/python/packages/raes_processor/semantics/realization_runtime_evaluation.pyimplementations/python/packages/raes_runtime/control_plane_api_models.pyimplementations/python/packages/raes_runtime/control_plane_store.pyspecs/formal/realization/explicitness-and-realization.mdspecs/sdl/observability-and-evidence.mdspecs/sdl/runtime-inventory.mdtools/policy/historical_identity_records.json
Reviews
- codex: No pre-push AI review was run because the user explicitly directed that non-blocking ceremony be skipped; repository completion, policy, CI, CodeQL, GitGuardian, and Sonar checks passed.
Traceability reconciliation
- IMPLEMENTS / TESTS / DOCUMENTS added: 7
- Links updated: 0
- Stale links removed: 0
Existing SEM-218 links to its normative realization specification and core realization evaluator remain valid; Phase E added the merged forwarding-agent implementation and test coverage.
Status
- CI: ✅ green
- SonarCloud: ✅ passed
- PR ready for user review and merge.
Question
What does it mean to realize a
runtime.forwarding_agents(ADR-050) concern,and how should the SDL express an agent's realized posture — including
deliberately-divergent or misconfigured postures — so that both endogenous
(system-under-test) and exogenous (measurement-instrumentation) agents are
honestly expressible and testable?
This is a design question, not a bug report. It surfaced from a downstream
consumer (
Brad-Edwards/aptl#889) but is an ecosystem-semantics question aboutADR-050 / ADR-056 / ADR-066 / SEM-218, so it belongs here.
The contradiction as it stands
forwarding-agentssits across two RAES layers that currently disagree aboutwhat it is:
ADR-050 (SDL surface):
runtime.forwarding_agentsis an observed runtimeinventory that is explicitly "targetable but not executable." Its
Non-Goals exclude "implementing log shipping, IOC ingestion, rule generation,
reload execution," and its Risks warn that "treating a transform's presence
as proof of execution would overclaim what the SDL can validate." So
declaring an agent asserts the node has an agent of this shape/config, not
that shipping runs.
SEM-218 (realization gate): the processor registers
forwarding-agentsasa first-class realization concern
(
raes_processor/semantics/realization_concerns.py), so an author-declaredEXACT forwarding agent goes through the non-approximation gate
(
explicitness-and-realization.md, invariants I1/I2): a backend must"realize R exactly as declared or reject" — no silent approximation.
The undefined middle: for a surface ADR-050 calls non-executable
inventory, SEM-218 demands the backend disclose a realized value — but
nothing defines what evidence discharges "realized" for it.
project_forwarding_agentsprojectsobservedandplannedto the samecanonical shape (no gradation), and unlike ADR-088 service-materialization —
which carries an
ObservationStrength(DAEMON_OBSERVED/GUEST_OBSERVED,raes_backend_protocols/service_materialization.py) — the six ADR-050/chore: back-merge v0.24.0 into dev #876runtime concerns carry no observation-strength disclosure at all. So a
backend has no honest, graded way to say how it corroborated a declared
agent, and is forced into a binary exact-match it cannot ground.
The expressivity we actually need
The realization-honesty question is inseparable from an expressivity question,
because the same concern must carry both exogenous and endogenous agents,
and (per the guardrail against overclaiming) neither can be faked:
1. Exogenous instrumentation (measurement apparatus). A scenario may have to
insert logging purely to collect the data an experiment needs, even though the
instrumentation is exogenous to the phenomenon under study. The author needs to
say: the agent is present, what it collects, and where its output goes —
abstractly (e.g. "to the evidence pack"), not a concrete host path. This is part
of the experimental data-collection apparatus. ADR-066 (scenario-native
observability vs authored evidence-requirement plane) and ADR-064
(capture-spec / evidence-record / derived-measure) clearly own adjacent
concerns, but it is not clear how a
runtime.forwarding_agentsentry that existsas measurement apparatus binds to the evidence plane, nor how its realization
is disclosed without implying it is a scenario variable.
2. Endogenous posture (system-under-test). A scenario may deliberately
contain agents in any combination of postures, because the posture itself is
the independent/dependent variable:
These must all be appropriately expressible and honestly corroborable: a
"present but not forwarding" agent is a legitimate realized scenario state the
backend must be able to disclose as realized-as-declared, not rejected as an
unrealized approximation. The experiment then tests the posture (e.g. "does the
SOC detect that the DB agent stopped shipping?"), which looks like an
observable/proposition question (ADR-079 truth semantics / ADR-054 observable
lifecycle) rather than a realization exact-match question.
3. This generalizes. The same reasoning applies to an EDR sensor, or any
runtime agent with a configured/behavioral posture — not just log forwarders. A
resolution should be a general pattern (or make
forwarding-agentsits firstinstance), not a forwarding-specific patch.
What seems missing (for discussion, not a prescription)
Observation strength for ADR-050/chore: back-merge v0.24.0 into dev #876 runtime realization concerns.
Generalize the ADR-088
ObservationStrengthidea so a backend honestlydiscloses how strongly it corroborated a declared agent — e.g.
presence/image-attested (agent baked into a digest-locked source image),
configuration-observed, behavior-observed — and the SEM-218 gate compares
declared-vs-observed at the disclosed strength. An author who declares
"forwarding to X" as EXACT then requires a backend that can observe
forwarding; a backend that can only attest presence must say so (and either
satisfies a presence-level declaration or is honestly rejected for a
behavior-level one).
Separation of realized presence/config from realized behavioral
posture. Is "is it actually forwarding / to where / correctly" a
realization concern (SEM-218 exact-match on
forwarding-agents), or anobservable/proposition (ADR-079/ADR-054) layered over a
presence/config realization? The latter seems to resolve the ADR-050 tension
cleanly (realization attests the declared agent inventory exists; propositions
test its behavior), and makes the divergent/misconfigured cases first-class —
but this should be decided in the ADRs, not inferred by a backend.
Endogenous vs exogenous role. A defined way to mark a
runtime.forwarding_agentsentry as measurement apparatus (exogenous, boundto the ADR-064/066 evidence plane, not a scenario variable) vs system-under-
test (endogenous, whose posture is an observable / variation factor per
ADR-084). Today the same model carries both with no role marker.
Concrete downstream impact
The current reference-backend/#876 behavior is that any EXACT
forwarding_agentsdeclaration is disclosed as absent and therefore rejectedby the non-approximation gate — on the premise that a backend never materializes
a forwarding agent. That premise does not hold for a backend that materializes
the agent as a digest-locked image sidecar or as an installed unit on a
dynamically-composed node; those agents are realized, and the gate rejects a
node the backend honestly stood up. This blocks a clean realization of any
scenario that declares forwarding agents (e.g. the TechVault operational
scenario: two image-backed Wazuh log-forwarder sidecars plus one dynamically
composed MISP→Suricata content-sync agent).
We would rather resolve this at the semantic/ADR layer than have each backend
invent its own answer to "what does realizing a non-executable inventory concern
mean," which would fragment realization honesty across backends.
Ask
Guidance (and, if warranted, an ADR-050/ADR-056 amendment or a new realization
ADR) on:
should carry a graded observation strength;
realization exact-match or an observable/proposition over a
presence/config realization;
agents are distinguished and how the exogenous case binds to the ADR-064/066
evidence plane.
Downstream tracking:
Brad-Edwards/aptl#889is blocked on the answer.Requirements