Skip to content

fix(runtime): judge and persist realization concerns through portable payloads - #1457

Open
doublewhy wants to merge 4 commits into
devfrom
1451-portable-disclosure-payloads
Open

doublewhy wants to merge 4 commits into
devfrom
1451-portable-disclosure-payloads

Conversation

@doublewhy

@doublewhy doublewhy commented Oct 9, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Snapshot admission accepts tuple, enum, dataclass and model carriers in entry payloads; the SEM-218 disclosure readers and sanitizer that run next read JSON objects and arrays only. Up to dev cf8b4308, a registered concern inside a carrier was misjudged or committed raw:

  • closed runtime-environment excess in a spec wrapped in an Enum member or dataclass committed with success;
  • an open observation in such a spec was committed raw ("value": "production", no provenance row);
  • an exact observation inside a carrier was refused.

Both now read the portable JSON projections of the plan's and result's payloads (as snapshot_values_equal does), so a registered concern is judged and persisted identically as a plain mapping or inside an admitted carrier whose portable serialization keeps every field admission walks under the same name and value. Plain-mapping results commit what they did before.

Requirement UIDs

Related Issues

Closes #1451

Follow-ups: #1502 (model fields the serializer excludes, renames or rewrites), #1518 (credential and prepared-node boundaries).

ADR Impact

  • None. Readers, sanitizer and rules are unchanged.

Changes

  • backend_apply_results.py:_portable_payloads projects payloads via to_jsonable_python; _realization_disclosure_result runs both disclosures over them.
  • _safe_realization_snapshot sanitizes the same projection; an entry keeps the backend's representation when it serializes to its sanitized view (a tuple returned as planned stays a tuple), else the sanitized view is committed.
  • A planned tuple returned as JSON is now admitted, but planner/ordering._entry_matches_resource still re-plans the three domain resources as UPDATE; one returned as planned re-plans as UNCHANGED.
  • Research evidence collected with collect_research_evidence.py on source 71e86070 (dev 348c4903, clean): 6 pinned artifacts through the production pipeline (42 stage results: 38 passed, 4 preregistered non-execution); 10 retained SDL cases replayed, 4 production cases and 14 participant functions run (18 passed); all 20 formal observations as expected. Recorded with python -m tools.research_capture record (templates c9ca186a / 3bad3945): capture-55f3cb3a…, capture-36481077…. Drift: versions.python 3.12.3 (was 3.11.15); formal commands[0] is the retained-case-replay argv that ran.
Issue's Verification Plan bullet (verbatim) Change Test failing without it CI lane Run
the reference provisioner returns the web node's spec with closed runtime-environment excess as a plain dict, an Enum member or a dataclass. All three are refused with runtime.backend-contract-invalid, and the snapshot stays empty. An honest apply is still admitted. _realization_disclosure_result test_closed_runtime_environment_is_refused_inside_a_carrier_that_serializes_it (enum-member, dataclass); test_honest_apply_is_admitted pins test-shard 38498469224
the issue #985 safe-projection case, for an exact and an open runtime environment, with the returned spec as a plain mapping, an Enum member or a dataclass, and with the environment value itself wrapped. Each is admitted, commits the same portable payload and provenance as the plain mapping, and never commits production or the description annotation. _safe_realization_snapshot test_only_the_safe_projection_persists_inside_a_carrier_that_serializes_it (6 carrier cases) test-shard 38498469224
the stub target applies a planned domain-topology tuple that its provisioner returns as planned or as JSON. Both are admitted. _portable_payloads test_planned_tuple_matches_the_same_value_returned_as_json[returned-as-json]; test_tuple_returned_as_planned_is_committed_as_returned_and_replans_unchanged test-shard 38498469224
the carrier cases and the JSON case fail on dev before the fix. all rows 9 cases, dev's file local Test Plan

Test Plan

  • Unit tests pass
  • Integration tests pass if applicable
  • Full completion suite required in CI before merge
  • No coverage regression

pytest <new module>: 15 passed. With dev's backend_apply_results.py: 9 failed (exactly the carrier cases and returned-as-json), 6 passed. With the sanitized view always committed (before the rebase), only the re-plan test failed.

45 SEM-218 modules: 686 passed (+1 integration). 158 raes_runtime modules: 3587 passed, 2 skipped (43 integration). 82 backend and conformance modules: 2075 passed (22 integration). Docker tests: CI integration-docker. tools/check_specification_coverage.py and tools/check_formal_semantic_validation.py exit 0 at the head and on git merge-tree --write-tree origin/dev HEAD. nox -s verify-fast-feedback -- --base-rev origin/dev and nox -s lint passed.

CI on head be2c827a (dev base 348c4903): all 28 checks passed, 1 skipped; canonical run 38498469224 (21 jobs succeeded).

Ground Control Checks

  • Repository policy command passes
  • Pre-push code review and test-quality review completed, or not run for this lane

make policy passed; test quality: the fail-without-change run.

Traceability

  • IMPLEMENTS: SEM-218 ← implementations/python/packages/raes_runtime/backend_apply_results.py
  • TESTS: SEM-218 ← implementations/python/tests/test_issue_1451_portable_disclosure_payloads.py

Checklist

  • Code follows the project coding standards
  • FM2 (portability rule between layers). I1: plain mapping and carrier are judged and persisted identically (differential tests). I2: every committed entry serializes to the judged sanitized view; one that already does is committed unchanged (re-plan test).
  • Published contract schemas regenerated if models changed: no model or schema changed.
  • PR title is a Conventional Commit (release-please derives the version and CHANGELOG.md from it)
  • Architectural docs updated if applicable: not applicable.

@doublewhy
doublewhy force-pushed the 1451-portable-disclosure-payloads branch from 5e39e53 to a156e90 Compare October 9, 2026 14:40
@doublewhy doublewhy changed the title fix(runtime): read admitted payload carriers through their portable projection during realization disclosure fix(runtime): read admitted payload carriers through their portable projection in realization disclosure and safe persistence Oct 9, 2026
@doublewhy
doublewhy marked this pull request as ready for review October 9, 2026 15:40
@doublewhy doublewhy changed the title fix(runtime): read admitted payload carriers through their portable projection in realization disclosure and safe persistence fix(runtime): judge and persist realization concerns through portable payloads Oct 9, 2026
@doublewhy
doublewhy force-pushed the 1451-portable-disclosure-payloads branch from a06450c to 34f1850 Compare October 13, 2026 09:26
@doublewhy
doublewhy marked this pull request as draft October 13, 2026 19:39
…rojection during realization disclosure

Backend snapshot admission accepts tuple, enum, dataclass and model carriers in entry payloads, but the SEM-218 disclosure readers walk only JSON objects and arrays. A closed concern wrapped in an Enum member or a dataclass read as absent and was committed as realized.

The runtime now presents declared operation payloads and returned entry payloads to the disclosure readers as their portable JSON projections, the relation snapshot_values_equal already uses.
…judges

The safe-persistence sanitizer walked plain mappings only. A registered
concern reached through an admitted carrier therefore skipped
sanitization while the disclosure readers, which read the portable
projection, admitted it, and the raw observation was committed.

The sanitizer now reads the same portable projection. An entry keeps the
backend's own representation when that representation, sanitized in
place, serializes to its sanitized view; otherwise the sanitized view is
committed in its place.

Tests cover the issue #985 safe-projection case for an exact and an open
runtime environment, with the returned spec as a plain mapping, an Enum
member or a dataclass, and with the environment value itself wrapped.
They also cover a planned domain-topology tuple returned as the planned
tuple or as a JSON array.
…and re-plans unchanged

_safe_realization_snapshot keeps the backend's own entry when it
serializes to the sanitized view. No test failed when that branch was
removed: honest domain-topology results then committed
controller_addresses as a list, and every later plan scheduled UPDATE
for provision.node.dc, provision.domain-controller.corp.dc and
provision.account.domain-admin. The new test asserts the committed
tuple and an all-UNCHANGED re-plan.

The module docstring and the two carrier test names now limit the
carrier claim to carriers that serialize the wrapped value: a model
field that its serializer excludes, renames or rewrites is not in the
portable projection, so the readers and the sanitizer do not see a
concern placed there, as on dev.
Collected on this branch's source (dev 348c490 plus #1451) with the shared collection tool: 6 pinned artifacts through the production pipeline, 10 retained SDL cases via replay_case, 4 production cases via raes processor, 14 participant functions (18 passed); recorded with python -m tools.research_capture record.
@doublewhy
doublewhy force-pushed the 1451-portable-disclosure-payloads branch from 34f1850 to be2c827 Compare October 14, 2026 01:57
@doublewhy
doublewhy marked this pull request as ready for review October 14, 2026 03:52

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant