Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion docs/requirements/API-404/requirement.md
Original file line number Diff line number Diff line change
Expand Up @@ -213,7 +213,7 @@ identifies those implementation gaps and the retained canonical requirements.
- DOCUMENTS → GITHUB_ISSUE `1188` (CP-8: Served profile alignment)
- DOCUMENTS → GITHUB_ISSUE `1187` (CP-9: Crash and profile conformance suite)
- DOCUMENTS → DOCUMENTATION `docs/decisions/issue-1187-control-plane-conformance-preflight.md` (CP-9 profile, fault-evidence, and security boundaries)
- DOCUMENTS → DOCUMENTATION `docs/research/runtime-control-plane/conformance.md` (Unified suite invocation, finite evidence map, and explicit profile nonclaims)
- DOCUMENTS → DOCUMENTATION `docs/research/runtime-control-plane/conformance.md` (Unified suite invocation, finite evidence map, API-404 clause-to-profile verification map, and explicit profile nonclaims)
- IMPLEMENTS → CODE_FILE `implementations/python/packages/raes_runtime/backend_result_diagnostics.py` (Value-free backend failure diagnostics without provider type names)
- IMPLEMENTS → CODE_FILE `implementations/python/packages/raes_runtime/observation_execution.py` (Observation-adapter failure diagnostics without provider exception names or text)
- TESTS → TEST `implementations/python/tests/test_issue_1212_observation_demand.py` (Observation failures preserve terminal state and redact exception details from operation carriers)
Expand All @@ -238,6 +238,8 @@ identifies those implementation gaps and the retained canonical requirements.
- DOCUMENTS → GITHUB_ISSUE `1185` (CP-11: API-404 requirement update)
- DOCUMENTS → DOCUMENTATION `docs/decisions/issue-1185-api-404-profile-alignment-preflight.md` (CP-11 profile and evidence boundaries)
- TESTS → TEST `implementations/python/tests/test_issue_1185_api_404_profile_alignment.py` (Clause matrix, profile catalog, nonclaims, public documentation, and local-evidence drift checks)
- DOCUMENTS → GITHUB_ISSUE `1435` (API-404 clause-to-profile evidence map and run binding on every profile)
- TESTS → TEST `implementations/python/tests/test_issue_1435_profile_clause_verification.py` (Out-of-run evaluation plans refused on P0, P1, and P2 without record, evaluator effect, or revision change, and without audit unless P2's planner-authorization check refuses an unregistered plan with operations first; P0 run rebinding refused; clause-map profiles checked against profile declarations; citation and link drift checks)
- DOCUMENTS → GITHUB_ISSUE `1186` (CP-12: Recovery runbook and operator tooling)
- DOCUMENTS → DOCUMENTATION `docs/decisions/issue-1186-control-plane-recovery-operations-preflight.md` (CP-12 recovery, maintenance, health, and disclosure boundaries)
- DOCUMENTS → DOCUMENTATION `docs/explain/sdl/control-plane-operations.md` (Operator recovery, health, shutdown, backup, restore, and upgrade runbook)
Expand Down
80 changes: 80 additions & 0 deletions docs/research/runtime-control-plane/conformance.md
Original file line number Diff line number Diff line change
Expand Up @@ -69,3 +69,83 @@ external-effect, tamper-evidence, or universal recovery-proof claim. Payload
digests detect corruption; a writer able to replace both content and digest is
not thereby authenticated. ASGI tests do not verify deployment TLS, trusted-proxy
header stripping, external backend recovery, or a matching backend backup.

## API-404 clause verification

[API-404](../../requirements/API-404/requirement.md) states the clauses and
the profiles each clause binds. `profile_declaration()` holds each profile's
guarantee and nonclaim identifiers. The table maps each clause to the landed
code, conformance cases, and operator guidance behind it. Code paths are
relative to `implementations/python/packages/raes_runtime/`, and test paths to
`implementations/python/tests/`.

| Clause | Profiles | Implementation | Conformance cases | Operator guidance |
| --- | --- | --- | --- | --- |
| `API-404-C1` | P0, P1, P2 | `control_plane.py`, `control_plane_configuration.py`, `control_plane_operation_context.py`, `control_plane_admission.py`, `control_plane_mutation.py`, `control_plane_execution.py`, `control_plane_store.py`, `control_plane_store_revision.py`, `control_plane_store_memory.py`, `control_plane_store_local_snapshot.py` | `test_issue_1187_control_plane_profiles.py`, `test_issue_1187_control_plane_lifecycle_properties.py`, `test_issue_1184_atomic_idempotency_claims.py`, `test_issue_1180_snapshot_revision_cas.py`, `test_issue_1181_unified_control_plane_mutations.py`, `test_issue_1189_control_plane_profile_declarations.py`, `test_issue_1435_profile_clause_verification.py` | [Control-plane operating profiles](../../explain/sdl/runtime-architecture.md#control-plane-operating-profiles) |
| `API-404-C2` | P1, P2 | `control_plane_store_local.py`, `control_plane_store_local_records.py`, `control_plane_store_local_scope.py`, `control_plane_store_local_codec.py`, `control_plane_store_records.py`, `control_plane_store_lease.py`, `control_plane_durability.py`, `control_plane_recovery.py` | `test_issue_1187_control_plane_process_loss.py`, `test_issue_1187_control_plane_durable_carriers.py`, `test_issue_1179_startup_reconciliation.py`, `test_issue_1183_store_ownership_leases.py`, `test_issue_1092_control_plane_crash_consistency.py`, `test_issue_1187_control_plane_profiles.py` | [Control-plane operations](../../explain/sdl/control-plane-operations.md) |
| `API-404-C3` | P2 | `control_plane_api/__init__.py`, `control_plane_api/_auth.py`, `control_plane_api/_offload.py`, `control_plane_api/_operation_routes.py`, `control_plane_api/_responses.py`, `control_plane_api_guards.py`, `control_plane_api_participant_retrieval.py`, `control_plane_security.py` | `test_issue_1187_control_plane_security_conformance.py`, `test_issue_1359_runtime_api_trust_boundary.py`, `test_issue_1093_request_rejection_offload.py`, `test_runtime_control_plane_api.py`, `test_issue_1187_control_plane_profiles.py` | [Serve the runtime control plane](../../public/guides/control-plane.md) |
| `API-404-C4` | P0, P1, P2; P3 is unavailable | `control_plane_profiles.py` | `test_issue_1189_control_plane_profile_declarations.py`, `test_issue_1185_api_404_profile_alignment.py`, `test_issue_1187_control_plane_process_loss.py` | [Control-plane operating profiles](../../explain/sdl/runtime-architecture.md#control-plane-operating-profiles) and the limits above |

Each property #8 names is checked on every profile that guarantees it:

- **Actor binding.** Retry, readback and disclosure stay with the submitting
actor on P0, P1 and P2
(`test_issue_1187_control_plane_profiles.py::test_profile_scoped_retry_status_audit_and_cache_coherence`).
- **Target and run binding.** A selected P0 store refuses another target
(`test_issue_1189_control_plane_profile_declarations.py::test_selected_p0_uses_canonical_declaration_and_binds_one_scope`)
and another run
(`test_issue_1435_profile_clause_verification.py::test_selected_p0_store_cannot_be_rebound_to_another_run`).
A P1 or P2 store refuses both
(`test_issue_1187_control_plane_profiles.py::test_profile_lease_scope_and_restore_keep_receipts_and_claims`).
On every profile, an evaluation plan for another run that carries no
operations or observation demands leaves no record, effect, audit or revision
change. The same plan for the admitted run adds one record, one terminal
audit and one evaluator start in the backend-effect witness, and moves the
snapshot revision from 0 to 1
(`test_issue_1435_profile_clause_verification.py::test_request_for_another_run_changes_nothing`).
On P2, the provisioning, orchestration and evaluation routes check planner
authorization for a plan that carries operations or observation demands,
before the core's run check (`control_plane_api/_operation_routes.py`). So P2
refuses an unregistered evaluation plan for another run that carries an
operation with 403 and the route's `planner-authorization-mismatch` denial
audit. P0 and P1 refuse it on the run check with no audit. No profile adds a
record or effect, and the revision stays 0
(`test_issue_1435_profile_clause_verification.py::test_plan_with_operations_for_another_run_leaves_no_record_or_effect`).
The run refusal itself is not an admission denial, which the
[CP-2 preflight](../../decisions/issue-1181-unified-control-plane-mutations-preflight.md)
defines as an `accepted=False` receipt plus the bounded denial audit and no
stored operation. P0 and P1 raise `ValueError` and P2 answers 409, with no
receipt. The
[CP-5 preflight](../../decisions/issue-1183-store-ownership-lease-preflight.md)
requires a request for another run to fail before claim or disclosure, and
`operation_admission_context()` refuses it before building the operation
context that the core's admission-denial audit records.
- **Atomic state and audit.** Every operation kind commits its terminal state
with exactly one actor-bound audit on P0 and P1
(`test_issue_1187_control_plane_lifecycle_properties.py::test_each_operation_kind_commits_one_immutable_actor_bound_terminal_cut`),
and so does an HTTP submission on P2
(`test_issue_1187_control_plane_profiles.py::test_profile_scoped_retry_status_audit_and_cache_coherence`).
- **Retained idempotency.** P1 and P2 keep receipts and claims across restore
(`test_issue_1187_control_plane_profiles.py::test_profile_lease_scope_and_restore_keep_receipts_and_claims`).
P0 loses them with its process
(`test_issue_1187_control_plane_process_loss.py::test_p0_process_loss_loses_run_and_claim_even_when_external_effect_survives`).
- **Ownership.** A P0 store has one live owner
(`test_issue_1189_control_plane_profile_declarations.py::test_selected_p0_store_has_one_active_owner_and_releases_it_on_close`).
P1 and P2 admit one lease owner and refuse a second
(`test_issue_1187_control_plane_profiles.py::test_profile_lease_scope_and_restore_keep_receipts_and_claims`).
- **Revision checks.** A stale snapshot write fails on every profile
(`test_issue_1187_control_plane_profiles.py::test_profile_scoped_retry_status_audit_and_cache_coherence`).
- **Startup classification.** P1 and P2 classify interrupted work without
replay
(`test_issue_1187_control_plane_process_loss.py::test_process_loss_preserves_atomic_cut_and_classifies_without_replay`).
It does not apply to P0, which keeps no state across process loss.

P2 serves a selected P1 core and adds no store of its own, so its C1 and C2
evidence is the P1 store evidence plus the HTTP cases.
`test_issue_1435_profile_clause_verification.py` derives each clause's profiles
from `profile_declaration()`: the available profiles whose guarantees include
the clause's identifiers, or every available profile for C4. It fails if that
set, this table and an independently written expectation disagree. It also
fails if an Implementation or Conformance cell holds an entry that is not a
cited file, if the section cites a file or test that does not exist, or if a
link names a missing page or heading.
20 changes: 18 additions & 2 deletions implementations/python/tests/control_plane_conformance_fixtures.py
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@
RecoveryObservationRequest,
RecoveryObservationResult,
)
from raes_backend_stubs.stubs import create_stub_target
from raes_backend_stubs.stubs import StubEvaluator, create_stub_target
from raes_contracts.plan_projection import provisioning_plan_model
from raes_contracts.planning import ProvisioningPlan
from raes_contracts.runtime_state import OperationKind, OperationState
Expand Down Expand Up @@ -81,6 +81,18 @@ def apply(self, plan, snapshot):
return result


class WitnessEvaluator(StubEvaluator):
# An evaluator start is a backend effect too, so it joins the same witness.
def __init__(self, witness: Path) -> None:
super().__init__()
self.witness = witness

def start(self, plan, snapshot):
assert plan.operation_id
record_effect(self.witness, "evaluate", plan.operation_id)
return super().start(plan, snapshot)


class WitnessObserver:
def __init__(self, witness: Path) -> None:
self.witness = witness
Expand Down Expand Up @@ -109,7 +121,11 @@ def witness_target(witness: Path, *, observe: bool = False, boundary=None):
# Recovery intentionally refuses to invent a provisioning plan to authorize
# new realization/resource state from an observer's assertion alone.
target = create_stub_target(with_realization_envelope=False)
target = replace(target, provisioner=WitnessProvisioner(target.provisioner, witness, boundary))
target = replace(
target,
provisioner=WitnessProvisioner(target.provisioner, witness, boundary),
evaluator=WitnessEvaluator(witness),
)
if observe:
capability = RecoveryObservationCapabilities(
name="conformance-witness", supported_operation_kinds=frozenset({OperationKind.PROVISIONING})
Expand Down
Loading
Loading