Skip to content

Coordinate mixed participant runtimes fail-closed #1016

Closed
Closed
Coordinate mixed participant runtimes fail-closed#1016
Labels
area:runtimeRuntime and control-plane codeblockedBlocked on an upstream gate or dependencyenhancementNew feature or requestin-progressAn agent is actively working this issue via /implement
Milestone
Participant Information-Flow & Behavioral Equivalence

Description

@Brad-Edwards
Brad-Edwards
opened on Jul 31, 2026

Parent: #813

Bounded outcome

Implement mixed runtime coordination through existing RUN-310/API-423
mediation. Resolve the admitted allocation, capability, controller and policy
cut, clock/order mapping, and topology edge before dispatch; commit the exact
decision before any backend effect or disclosure; append phase, handoff,
delivery, observation, weakening, and failure evidence.

Negative cases

  • A backend adapter bypasses participant control, crossing policy, or atomic
    persistence.
  • HLA ownership, adapter responsibility, or bridge routing changes the acting
    controller.
  • Stale handoff, concurrent intervention, unmapped time, unsupported action,
    failed commit, or unadmitted join causes an external effect.
  • Retraction or rollback erases a prior delivery or participant-visible fact.

Evidence required

  • Red-green tests through RuntimeControlPlane to RuntimeTarget boundaries.
  • Zero prohibited backend calls or disclosures for every denied/stale/
    unsupported/failed-commit case.
  • Append-only histories, replay/idempotency, CAS conflict, phase progression,
    and sanitized failure evidence.
  • Pure, mixed, and staged reference-runtime fixtures.

Explicit nonclaims

  • No backend-native realization from reference coordination alone.
  • No multi-controller, lease, joint/fused-control, IFC, or equivalence claim.

Dependencies

Requirements

  • SEM-234
  • RUN-310
  • RUN-319
  • API-423

Activity

  1. Brad-Edwards
    added this to the Participant Information-Flow & Behavioral Equivalence milestone on Jul 31, 2026
  2. Brad-Edwards
    added
    enhancementNew feature or request
    area:runtimeRuntime and control-plane code
    on Jul 31, 2026
  3. Brad-Edwards
    marked this as blocked by 2 issues on Sep 17, 2026
  4. Brad-Edwards
    marked this as blocking 2 issues on Sep 17, 2026
  5. Brad-Edwards
    added
    blockedBlocked on an upstream gate or dependency
    in-progressAn agent is actively working this issue via /implement
    on Sep 17, 2026

  6. Brad-Edwards commented on Sep 20, 2026

    @Brad-Edwards
    Brad-Edwards
    on Sep 20, 2026
    CollaboratorAuthor

    🛠️ Picked up by /implement - driver codex, branch 1016-mixed-runtime-coordination, 2026-09-20T05:38:47.350Z.

  7. Brad-Edwards
    added
    deferred
    on Sep 20, 2026
  8. Brad-Edwards
    removed this from the Participant Information-Flow & Behavioral Equivalence milestone on Sep 20, 2026
  9. Brad-Edwards
    changed the title [-]Coordinate mixed participant runtimes fail-closed[/-] [+]Deferred: Coordinate mixed participant runtimes fail-closed[/+] on Sep 20, 2026
  10. Brad-Edwards
    added this to the Participant Information-Flow & Behavioral Equivalence milestone on Sep 20, 2026
  11. Brad-Edwards
    removed
    deferred
    on Sep 20, 2026

  12. 7 remaining items

  13. Brad-Edwards commented on Sep 20, 2026

    @Brad-Edwards
    Brad-Edwards
    on Sep 20, 2026
    CollaboratorAuthor

    gc_codex_review pre-push cycle 1 of 1 complete for issue #1016 on branch '1016-mixed-runtime-coordination'. Posted by the MCP server to enforce the pre-push hard-cap-1 contract (issues #796, #804, #906). Do not edit or delete — used by the next gc_codex_review (uncommitted) invocation to count cycles.

  14. Brad-Edwards commented on Sep 20, 2026

    @Brad-Edwards
    Brad-Edwards
    on Sep 20, 2026
    CollaboratorAuthor

    Review decision record — codex cycle 1 (issue #1016)

    Reviewer: codex
    Cycle: 1
    Verdict: ship-with-fixes

    Architectural read:

    The implementation uses the existing single-control-plane authority, immutable admitted component bindings, crossing mediation, snapshot ownership, and exact-target recovery. Two coordination gaps remain before publication: phase changes need an outstanding-operation fence, and mixed lifecycle effects must not bypass the composition decision boundary.

    Blocking findings: 2

    Finding 1 — one-off

    • ID: core-F1
    • Title: Fence phase changes on outstanding mixed operations
    • Location: implementations/python/packages/raes_runtime/mixed_runtime_phase.py:118
    • Decision: fix
    • Rationale: A phase transition must not change provider responsibility while an authorized mixed action may be executing or awaiting terminal commit.

    Finding 2 — one-off

    • ID: core-F2
    • Title: Prevent mixed lifecycle effects outside the composition cut
    • Location: implementations/python/packages/raes_runtime/participant_episode_control.py:208
    • Decision: fix
    • Rationale: Episode lifecycle calls currently select a mixed provider without committing composition decision and outcome facts; they must use a fully coordinated path or fail closed.

  15. Brad-Edwards commented on Sep 20, 2026

    @Brad-Edwards
    Brad-Edwards
    on Sep 20, 2026
    CollaboratorAuthor

    Pre-PR base synchronization

    • Source: refs/remotes/origin/dev at 7128f0cc25d44a37264a660a1b3854e4b9348678
    • Outcome: merged_conflicts_resolved
    • Published feature head: 1d0651c9e240acdd04f38e7664171964674cc368
    • Synchronized tree: 276f3060e8e7efde0f2454bf8473c7c808bf086a

  16. Brad-Edwards commented on Sep 20, 2026

    @Brad-Edwards
    Brad-Edwards
    on Sep 20, 2026
    CollaboratorAuthor

    Pre-PR base synchronization

    • Source: refs/remotes/origin/dev at 7128f0cc25d44a37264a660a1b3854e4b9348678
    • Outcome: already_current
    • Published feature head: 7c895e165af97a36bba052b6c5fc29000af3b7d2
    • Synchronized tree: 5084335402f1f6de1a4a40777130fdab2aff4b2c

  17. Brad-Edwards commented on Sep 20, 2026

    @Brad-Edwards
    Brad-Edwards
    on Sep 20, 2026
    CollaboratorAuthor

    Pre-PR base synchronization

    • Source: refs/remotes/origin/dev at 7128f0cc25d44a37264a660a1b3854e4b9348678
    • Outcome: already_current
    • Published feature head: 6543cc6a1dcd8b04bf7f63c44cf3d692344b124b
    • Synchronized tree: 1d867f510cafa12a7fb0bc0587081fe31e64d299

  18. Brad-Edwards commented on Sep 20, 2026

    @Brad-Edwards
    Brad-Edwards
    on Sep 20, 2026
    CollaboratorAuthor

    Pre-PR base synchronization

    • Source: refs/remotes/origin/dev at f784be2d1dd8714afd9ad60b56b8100c850f8e3e
    • Outcome: merged_conflicts_resolved
    • Published feature head: 83255695f11087f15cd0059bacf9e7c71fd7de20
    • Synchronized tree: e385bca54caed587db21e750e30ca0e673b97f7f

  19. Brad-Edwards commented on Sep 20, 2026

    @Brad-Edwards
    Brad-Edwards
    on Sep 20, 2026
    CollaboratorAuthor

    Pre-PR base synchronization

    • Source: refs/remotes/origin/dev at f784be2d1dd8714afd9ad60b56b8100c850f8e3e
    • Outcome: already_current
    • Published feature head: 06e15eb03e1a84094ebbd1a44070d8c08ca06fed
    • Synchronized tree: 853eee0df4f73ab9c6e36acff236b56a7ca5d4d5

  20. Brad-Edwards commented on Sep 20, 2026

    @Brad-Edwards
    Brad-Edwards
    on Sep 20, 2026
    CollaboratorAuthor

    Pre-PR base synchronization

    • Source: refs/remotes/origin/dev at f784be2d1dd8714afd9ad60b56b8100c850f8e3e
    • Outcome: already_current
    • Published feature head: 363906e762b294f3fcef1840d06f24f9f9cc3041
    • Synchronized tree: e2e3fdd9bac1fa6c99e4484ad0bfbbe7a923eb1d

  21. Brad-Edwards commented on Sep 20, 2026

    @Brad-Edwards
    Brad-Edwards
    on Sep 20, 2026
    CollaboratorAuthor

    Pre-PR base synchronization

    • Source: refs/remotes/origin/dev at f784be2d1dd8714afd9ad60b56b8100c850f8e3e
    • Outcome: already_current
    • Published feature head: eba363538a99582cfed59eeb0d8ae84af4610d30
    • Synchronized tree: 171580b3d247c8645c946798dc46a3a5c794668e

  22. Brad-Edwards commented on Sep 20, 2026

    @Brad-Edwards
    Brad-Edwards
    on Sep 20, 2026
    CollaboratorAuthor

    Pre-PR base synchronization

    • Source: refs/remotes/origin/dev at f784be2d1dd8714afd9ad60b56b8100c850f8e3e
    • Outcome: already_current
    • Published feature head: 4bfd64e75a5f538538a92c21256f22384462eeb1
    • Synchronized tree: 3791cb8194ffd78177c18ae07a62d4bf8ece43c5

  23. Brad-Edwards commented on Sep 20, 2026

    @Brad-Edwards
    Brad-Edwards
    on Sep 20, 2026
    CollaboratorAuthor

    Ready for review — issue #1016

    PR: #1336
    Plan: #1016 (comment)

    Outcome

    Mixed participant workloads can now bind and coordinate admitted providers through durable fail-closed decisions, bounded phase changes, exact lifecycle and action dispatch, and exact recovery routing without changing portable SDL meaning.

    Implemented fail-closed mixed-runtime coordination, exact provider/lifecycle/action routing, quiescent phase transitions, durable recovery evidence, published schemas and documentation, and updated requirement traceability. Focused local checks passed; repository-wide CI is green and SonarCloud reports 0 issues and 0 hotspots.

    In-scope requirements

    Proposed state in this PR — authoritative only after merge.

    • SEM-234 (Mixed Cross-Backend Participant-Control Composition) — ACTIVE (proposed)
    • RUN-310 (Intervention, Handoff, And Supervisory Lifecycle) — ACTIVE (proposed)
    • RUN-319 (Participant Information-Flow Policy Enforcement) — ACTIVE (proposed)
    • API-423 (Participant Crossing Policy And Evidence Contracts) — ACTIVE (proposed)

    Files changed

    Added:

    • docs/decisions/issue-1016-mixed-runtime-coordination-preflight.md
    • docs/governance/requirement-scopes/1016.json
    • docs/research/formal-semantic-validation/analysis-v38.json
    • docs/research/formal-semantic-validation/bundles/retest-v38.json
    • docs/research/formal-semantic-validation/execution-snapshot-v38.json
    • docs/research/specification-coverage/analysis-v38.json
    • docs/research/specification-coverage/bundles/raes-standardized-specification-coverage-issue-1016-v38.json
    • docs/research/specification-coverage/execution-snapshot-v38.json
    • implementations/python/packages/raes_contracts/contracts/mixed_runtime.py
    • implementations/python/packages/raes_contracts/mixed_runtime_history.py
    • implementations/python/packages/raes_runtime/mixed_runtime.py
    • implementations/python/packages/raes_runtime/mixed_runtime_dispatch.py
    • implementations/python/packages/raes_runtime/mixed_runtime_lifecycle.py
    • implementations/python/packages/raes_runtime/mixed_runtime_phase.py
    • implementations/python/packages/raes_runtime/mixed_runtime_state.py
    • implementations/python/tests/test_issue_1016_mixed_runtime_coordination.py

    Modified:

    • contracts/schema-publication/entries/backend-manifest-v2.json
    • contracts/schema-publication/entries/operation-receipt-v1.json
    • contracts/schema-publication/entries/operation-status-v1.json
    • contracts/schema-publication/entries/runtime-snapshot-v1.json
    • contracts/schemas/backend-manifest/backend-manifest-v2.json
    • contracts/schemas/control-plane/operation-receipt-v1.json
    • contracts/schemas/control-plane/operation-status-v1.json
    • contracts/schemas/snapshots/runtime-snapshot-v1.json
    • docs/explain/reference/mixed-participant-composition.md
    • docs/requirements/API-423/requirement.md
    • docs/requirements/RUN-310/requirement.md
    • docs/requirements/RUN-319/requirement.md
    • docs/requirements/SEM-234/requirement.md
    • docs/research/formal-semantic-validation/index.md
    • docs/research/specification-coverage/index.md
    • implementations/python/packages/raes_contracts/_snapshot_updates.py
    • implementations/python/packages/raes_contracts/contracts/__init__.py
    • implementations/python/packages/raes_contracts/contracts/_mixed_composition_exports.py
    • implementations/python/packages/raes_contracts/contracts/realization_plans.py
    • implementations/python/packages/raes_contracts/operation_lifecycle.py
    • implementations/python/packages/raes_contracts/runtime_state.py
    • implementations/python/packages/raes_runtime/__init__.py
    • implementations/python/packages/raes_runtime/backend_snapshot_contracts.py
    • implementations/python/packages/raes_runtime/control_plane.py
    • implementations/python/packages/raes_runtime/control_plane_configuration.py
    • implementations/python/packages/raes_runtime/control_plane_recovery.py
    • implementations/python/packages/raes_runtime/control_plane_store_history.py
    • implementations/python/packages/raes_runtime/control_plane_store_snapshots.py
    • implementations/python/packages/raes_runtime/participant_action_validation.py
    • implementations/python/packages/raes_runtime/participant_control.py
    • implementations/python/packages/raes_runtime/participant_crossing_boundary.py
    • implementations/python/packages/raes_runtime/participant_crossing_policy.py
    • implementations/python/packages/raes_runtime/participant_episode_control.py
    • implementations/python/packages/raes_runtime/participant_result_contracts.py
    • implementations/python/packages/raes_runtime/participant_submission_options.py
    • implementations/python/tests/test_formal_semantic_validation.py
    • implementations/python/tests/test_issue_989_versioned_evidence.py
    • implementations/python/tests/test_participant_concurrent_batch_reservations.py
    • implementations/python/tests/test_specification_coverage.py
    • tools/check_json_artifacts.py
    • tools/check_specification_coverage.py
    • tools/formal_semantic_validation/_baseline.py
    • tools/formal_semantic_validation/_loading.py
    • tools/formal_semantic_validation/_releases.py
    • tools/formal_semantic_validation/_retest.py
    • tools/policy/requirement_order.yaml

    Deleted:

    • tools/verify_all.py

    Reviews

    • codex: Cycle 1 found a missing phase quiescence fence and a lifecycle path bypassing the composition cut. Both were fixed; the user directed proceeding without an over-cap review.

    Traceability reconciliation

    • Pending — requirement status transition and IMPLEMENTS/TESTS reconciliation run in Phase E once the PR merges.

    Status

    • CI: ✅ green
    • SonarCloud: ✅ passed
    • PR ready for user review and merge. Ground Control reconciliation (requirement status + traceability) runs on merge (Phase E).

  24. Brad-Edwards commented on Sep 20, 2026

    @Brad-Edwards
    Brad-Edwards
    on Sep 20, 2026
    CollaboratorAuthor

    Final report — issue #1016 complete

    PR: #1336
    Plan: #1016 (comment)

    Outcome

    Mixed participant workloads can now bind and coordinate admitted providers through durable fail-closed decisions, bounded phase changes, exact lifecycle and action dispatch, and exact recovery routing without changing portable SDL meaning.

    Implemented fail-closed mixed-runtime coordination, exact provider/lifecycle/action routing, quiescent phase transitions, durable recovery evidence, published schemas and documentation, and updated requirement traceability. Focused local checks passed; repository-wide CI is green and SonarCloud reports 0 issues and 0 hotspots.

    In-scope requirements

    Verified at merge revision 24e66529bd23.

    • SEM-234 (Mixed Cross-Backend Participant-Control Composition) — ACTIVE
    • RUN-310 (Intervention, Handoff, And Supervisory Lifecycle) — ACTIVE
    • RUN-319 (Participant Information-Flow Policy Enforcement) — ACTIVE
    • API-423 (Participant Crossing Policy And Evidence Contracts) — ACTIVE

    Files changed

    • (none)

    Reviews

    • codex: Cycle 1 found a missing phase quiescence fence and a lifecycle path bypassing the composition cut. Both were fixed; the user directed proceeding without an over-cap review.

    Traceability reconciliation

    • IMPLEMENTS / TESTS / DOCUMENTS added: 8
    • Links updated: 9
    • Stale links removed: 0

    Requirement descriptors, exact issue scope, ownership roots, executable TESTS links, schema publication, and immutable formal/specification evidence were updated for SEM-234, RUN-310, RUN-319, and API-423.

    Status

    • CI: ✅ green
    • SonarCloud: ✅ passed
    • PR ready for user review and merge.
  25. Brad-Edwards
    closed this as completedon Sep 20, 2026
  26. Brad-Edwards
    unmarked this as blocking 2 issues on Sep 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area:runtimeRuntime and control-plane codeblockedBlocked on an upstream gate or dependencyenhancementNew feature or requestin-progressAn agent is actively working this issue via /implement

    Type

    No type

    Projects

    No projects

      Milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions