Skip to content

Native source enforcement for owned sets - #6300

Open
schickling-assistant wants to merge 1 commit into
mainfrom
schickling-assistant/2026-11-10-source-enforcement
Open

schickling-assistant wants to merge 1 commit into
mainfrom
schickling-assistant/2026-11-10-source-enforcement

Conversation

@schickling-assistant

@schickling-assistant schickling-assistant commented Nov 10, 2026 •

Copy link
Copy Markdown
Contributor

One squashed commit on main. It builds on #6286 (owned-set full tree, merged as 10104dde).

Scope (decided)

  • Revision.policy: Option<SourcePolicy> holds {enforce: off|audit|enforce, publisher, namespaces, unmanaged_allow}. It is carried in owned-set.revised and fenced by --expect-set. A publication that omits the policy keeps the current one. CLI: --source-policy FILE.json or --enforce/--publisher/--namespace/--unmanaged-allow. Namespace globs may claim only agent/, schedule/, subscription/ and mission/. An agent's exec/pty tasks are members carried with their agent (Owned sets carry agent exec/pty tasks and resolve ST_REVISION tokens #6286). exec/RUN/… and pty/RUN/… are mission-run runtimes, so a glob on them is refused with invalid-source-policy.
  • One gate (owned_sets::gate) covers apply, create in a claimed namespace, stop, start, mission publish, mission retire, hold, suspend, resume, and retire. Refusals name the verb (ad hoc stop is refused …).
  • --force --reason records source.debt.recorded in the same transaction as the write.
    • Hold, suspend, resume and agent-retire audit go through ClaimCommitContext.then: Store::append_source_gated / source_follow_up. Their debt names the runtime request.
    • missions retire --force --reason writes through append_source_forced_receipt_claim_tx, which skips only the plain set-managed-subject admission refusal, and records the debt in the same transaction.
    • Forced mission and desired declarations override the set's member reference until the next publication.
  • Publication closes all open debt. Each debt is reported as reverted, adopted or closed, and source.debt.closed is written in the publication transaction. Inside that same transaction, a forced hold still in place is released and a forced suspension still in place is resumed, written as the publisher.
  • Audit mode records source.audited. View the records with fractal source status --audit.
  • Activation gate: a policy or subscription members require features.source_enforcement=1, which is advertised in both feature maps. A publication from anyone outside the publisher list is refused with source-publisher-refused.
  • Top-level subscriptions are set members and retire through their stopped form. Plain apply already refuses a top-level subscription (runtime-outside-mission), so the set is its only writer.
  • set-managed-subject is now a client-v0 error code (schema, templates, regenerated clients). The schema doc is regenerated, and the principal-transition digest test excludes the three new source.* kinds.
  • Docs: docs/st3/owned-sets.md § Source enforcement. The large-value inventory review notes the source_forced admission flag.

Not enforced

These are not enforced: messages, asks and answers, work claim/complete, run cancel, mission run starts, self restart-when-idle, and restart of another agent. startable-by grants are not enforced either. Fractal has no native startable-by grant today, and this PR does not add one.

Test evidence

Focused runs on dev5 (nice). Earlier runs used head content 2e4cf410b0, and the integration owned_sets:: suite was rerun at 95966801ed. After the rebase onto main 5bde948843, these were rerun: fractal-evidence --lib owned_sets_tests 43/43 at 3bdc2d11d1, fractal --test integration owned_sets:: 17/17 at ebe481aeb3, and fractal-schema/fractal-missions/fractal-store --lib 1868/1869 at d3ca5c41d5. The one failure, client_messages::single_send_selector_is_visible_immediately_after_commit, is a wall-clock commit budget that passes when run alone.

  • fractal-schema --lib: 151 passed
  • fractal-evidence --lib owned_sets_tests: 42 passed
  • fractal-missions --lib: 324 passed
  • fractal --test integration owned_sets::: 17 passed at 9596680
  • fractal-store --lib retirements: the producer guard passed
  • scripts/check-sqlite-large-values: passed
  • Clippy on fractal-store and fractal-missions: no warnings on changed lines

Full nextest bursts: at 9596680 https://github.com/compoundingtech/fractal/actions/runs/42878569498; at 3bdc2d1 https://github.com/compoundingtech/fractal/actions/runs/42913064790. The convergence::seed_* failures also occur on main 5bde948843 (https://github.com/compoundingtech/fractal/actions/runs/42907646046: seeds 13 and 179). The test-only sim fix is in #6264.

Fixes since the last draft:

  • The forced mission retire was refused by the generic admission check (append_admitted_claim_tx → refuse_unmanaged), not by the gate. It now writes through the source-forced receipt writer.
  • In the replica-convergence test, the "nothing written" check compared the store index. The first write as person/operator on a signed store mints that person's root and device keys (2 claims), so the check now compares the member's own claims.
  • The subscription test now confirms its retirement preview and expects the plain path's runtime-outside-mission refusal.

Johannes pre-approved queueing high-risk fractal PRs from the factory-cleanup effort under the merge bar: exact-head independent reviewer GO, hosted CI via the merge queue, merge as schickling-assistant.

Companion: schickling/factory#743 now claims only the four accepted namespace kinds.

Review fix: a forced mission creation in a claimed mission/* namespace that the source still omits is now retired through its retired definition. Before, stop "mission/…" failed to parse and every later publication of the set failed. Covered in forced_writes_record_debt_until_the_next_publication_closes_it.
Known, non-blocking: an identical republication (same source sequence and bundle) is a noop and does not close open debt. The applier publishes for each new source.

Rebase onto main after #6286 merged: the conflicts were in the principal-digest test exclusions, the source_enforcement feature-map entry, the regenerated append_admitted_claim_tx large-value flow fingerprint, and the claims-v18 corpus union.

Second review round (exact-head GO at 3bdc2d11d1):

  • Run-owned declarations (owner_run, which only mission materialization sets) skip the namespace-create gate, because source does not route run starts. They still pass refuse_unmanaged, so a run cannot redeclare a set member. Test: a_claimed_namespace_still_admits_run_owned_declarations.
  • The exec/pty tasks of a force-created agent are adopted with it at the next publication, so --adopt is no longer needed.

@schickling
schickling force-pushed the schickling-assistant/2026-11-10-source-enforcement branch 3 times, most recently from aadff94 to 9596680 Compare November 10, 2026 18:46
@schickling-assistant

Copy link
Copy Markdown
Contributor Author

Independent reviewer (Fix6300.Review6300) gave GO at exact head 9596680, after a NO-GO on aadff94 for the orphaned forced mission creation, which is fixed in this head. Johannes pre-approved queueing high-risk fractal PRs from the factory-cleanup effort under the merge bar.

@schickling
schickling force-pushed the schickling-assistant/2026-11-10-source-enforcement branch 6 times, most recently from d3ca5c4 to ebe481a Compare November 10, 2026 22:07
…erges, audit mode

Owned-set revisions carry a source policy (enforce off|audit|enforce, publisher,
namespaces, unmanaged-allow). One gate covers apply, create in a claimed
namespace, stop, start, mission publish/retire, hold, suspend, resume and
retire. --force --reason records source debt in the write's transaction; the
set's next publication closes every debt and reverts forced holds,
suspensions and mission declarations. Audit mode records would-be refusals.
@schickling
schickling force-pushed the schickling-assistant/2026-11-10-source-enforcement branch from ebe481a to 3bdc2d1 Compare November 10, 2026 22:16
@schickling-assistant

Copy link
Copy Markdown
Contributor Author

Independent reviewer (Fix6300b.Review6300b): GO for exact head 3bdc2d11d15724c71288696ef6bdeeb542129b8c. Two findings from the d3ca5c4 round are addressed: run-owned declarations skip the namespace-create gate but still refuse set members, and tasks of a force-created agent are adopted with it. No open findings.

This branch had an error being deployed

1 failed deployment
evidence-signing — 3bdc2d11 Deployed Nov 10, 2026 by schickling-assistant via linux-tests-shard-2 #19701
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants