[schickling-assistant]

Depends on #6260 (mission run authority): this branch merges schickling/2026-11-10-program-engine-authority to add the program row to its mission-field barrier table. Merge #6260 first.

Problem

Program missions (docs/st3/program-engine.md) need a declaration the graph can publish and replicate: the step must name its bundle by hash, the bytes must travel with the publication (the daemon may run on another machine than the author's CLI), and a declaration naming bytes the member never received would be refused by every replica (missing-replicated-blob).

Goal

M1.1a of the program engine: the program step, ProgramSpec.bundle_hash, the program.step journal claim kind, and fractal apply carrying and verifying bundles.

Decisions

• KDL step "program" { program "/abs/bundle.mjs" sha256="<hex>" stalled-after="10m"? }; the step definition stores program {bundle, bundle_hash, stalled_after_ms?}. bundle_hash is the field name collect_hash_fields promotes, so mission.published promotes the staged blob and replicas receive it.
• Program steps are agentless and may not declare an assignee, gates, products, produces/uses/nested missions, after-run, or be finally.
• ApplyRequest and the owned-set request gain program_bundles (sha256 → base64, omitted when empty). The CLI reads every declared path, checks its hash (also for --dry-run) and sends the bytes; fractal validate checks the hash locally.
• The daemon verifies all carried bundles (hash match; refuses a bundle no program names: unused-program-bundle) before staging any with put_blob; publication routes accept up to 32 MiB.
• apply_as_impl refuses any program whose blob is in neither blobs nor local_blobs (missing-program-bundle) — one choke point for every publication path.
• program.step: step-run, system-only, append, durable; fields key!, status!, attempt!, program!, value, error, ms, child. Added to a new convergence corpus claims-v19.json (main's v18 is the newest frozen slot; v17 stays as main has it) and docs/st3/schema.md.
• Program steps are held working by the reconciler until the engine (M1.1b) decides them, so with only this PR merged a program step never completes on its own.
• guard_bound_request and guard_unverifiable_request buffer up to PUBLICATION_BODY_LIMIT (32 MiB), matching the publication routes.
• Mixed versions: a program row (mission_program, refusal mission-program-unavailable, also an apply --dry-run blocker) joins Mission run authority: parse, render and store the authority block #6260's barrier table in store/mission_capacity.rs; each daemon advertises mission_program=1 on daemon.started.
• docs: accepted program engine design and build plan #6253 (design) has merged; main is merged in, so this diff no longer carries the design commit. Merges Mission run authority: parse, render and store the authority block #6260's branch (@76665ee920) for the barrier table; that part of the diff disappears once Mission run authority: parse, render and store the authority block #6260 lands.

Verification

Focused tests on the Linux build host at head 9aa1096, each CARGO_TARGET_DIR=<slice target> nix develop -c nice -n 19 cargo test …. Hosted CI at the exact head is the proof (pushed with ST_EVIDENCE_LOCAL=off).

• cargo test -p fractal --lib -- program_bundles: 1 passed (a_publication_carries_exactly_the_declared_bytes). It failed at a191dbe with unsupported-kdl-version: st3 requires KDL version 2, but this document uses version 0: the test intent had no version 2 line. Fixed in the test (e1c0a60); the code was right.
• cargo test -p fractal-store --lib -- mission_capacity: 3 passed, including the new a_revision_that_declares_a_program_step_waits_until_every_member_reads_it and Mission run authority: parse, render and store the authority block #6260's capacity and authority tests.
• cargo test -p fractal-api-core --lib -- caller_guards_pass_a_publication_body_past_sixteen_mebibytes publication_stores_the_program_bundle_its_declaration_names publication_accepts_a_step_assigned: 3 passed. The new guard test sends a 20 MiB body through both guards (bound harness, and unverifiable ancestry) and gets it back whole.
• cargo test -p fractal-missions-base --lib -- program_step: 1 passed.
• cargo test -p fractal-reconcile --lib -- program_step_hold: 1 passed (a_published_program_step_stays_working_until_the_engine_decides).
• cargo test -p fractal-schema --lib -- checked_in_schema_document: 1 passed.
• cargo test -p fractal --test integration -- convergence::the_corpus_includes_every_registry_kind: 1 passed (corpus v19).
• The large-value inventory guard python3 scripts/check-sqlite-large-values: passed at f7a4d11 (after resolving the main merge conflict in docs/st3/large-value-sites.json record by record), and at Program engine M1.1b: run program steps on a supervised Bun host #6265's head 3ba3725, which contains this head.
• Not characterized locally: the convergence seed tests' never-trim set for v19. On the loaded build host cargo test -p fractal --test integration convergence:: -- --test-threads=4 failed 12 of 43 (non-convergence on seeds 1, 3, 13, 17, 34, 277, 1597; trim mismatches on 2, 55, 610). The same host fails seeds 13 and 55 with main's v18 corpus at this code too, which is the known local/CI split Convergence seed tests fail locally on a loaded host but pass in CI (11 of 33 seeds at main) #6210, so the result says nothing about v19. NEVER_TRIMS stays [2, 34] until hosted CI reports; adjust it from that run, as main did for v18.

Review findings answered

• a191dbe: P1 "Hold program steps instead of letting them complete with no gates" (reconciler hold + test); P3 "Raise the 16 MiB bound-harness body buffer" (both guards buffer 32 MiB); release note now says a refused publication can leave verified bytes staged, unpromoted.
• 3a368e0: test for the 32 MiB guard buffer (P3 follow-through); the program mixed-version barrier row.
• b4a762d, e73dd28: merges of Mission run authority: parse, render and store the authority block #6260's branch for the barrier table.
• f7a4d11: merge of main after docs: accepted program engine design and build plan #6253 landed (task: drop the design commit from this diff).
• 9aa1096: program.step moves from the frozen corpus v17 into a new claims-v19.json (generated with cargo run -p fractal-schema --example claims_json), selected by the simulation.
• e1c0a60: the failing CLI test (not a review finding).

Complexity

~330 lines of code and tests plus the fixture entry and docs.

Concerns

• Mixed fleets: a member whose build predates program would re-hash a program mission without the field and quarantine it; publication now holds such a revision until every current member advertises mission_program=1.
• A body over 32 MiB still fails in the guards with an internal error, not a typed refusal (unchanged behaviour of to_bytes).
• With only this PR merged a program step is held working and nothing here completes it; the engine is the stacked M1.1b PR.

Friction & bottlenecks

Draft PRs skip hosted CI, and local convergence seed runs are unreliable on a loaded host (#6210), so the v19 never-trim set needs a hosted run once the PR leaves draft.
The build host was at load ~95 on 32 cores with ~6 GiB RAM free; the cold focused build took several minutes per crate.

Follow-ups

• M1.1b engine (stacked), M1.2 children and asks.

References

docs/st3/program-engine.md §3–§5; design PR #6253; spike PR #6254 (not for merge).

[schickling-assistant]

Build-host verification at 969ef90: fractal-missions-base program_step 1 passed; fractal-schema checked_in_schema_document_matches_the_registry 1 passed; fractal-api-core publication_stores_the_program_bundle_its_declaration_names + publication_accepts_a_step_assigned_to_an_undeclared_person 2 passed. NOT run: the CLI unit test program_bundles::tests::a_publication_carries_exactly_the_declared_bytes and convergence::the_corpus_includes_every_registry_kind (the package/target names I used were wrong: -p fractal --bin fractal / -p fractal-cli --test convergence found no target). Hosted CI at the exact head is the proof.

[schickling-assistant]

Review record for head 9aa1096557b6f3b466c631e76dbbd6d81556e8f7.

• Independent reviewer, round 1 at 969ef90540: CHANGES, risk high. Blocking: a program step completed with nothing run before the engine existed. Advisories: the 16 MiB guard buffer; no fleet barrier for older members; the release note about staged bytes was wrong.
• Round 2 at 9aa1096557b6f3b466c631e76dbbd6d81556e8f7 (delta read; merges read with --remerge-diff): GO, no findings. Program steps are now held until the engine runs them (test program_step_hold). The guards buffer up to 32 MiB (test). A mission_program barrier row covers apply, dry-run and local publication (test). The release note is corrected. v17 corpus is restored and program.step moves to v19.
• Conditions: Mission run authority: parse, render and store the authority block #6260 merges first (merged into this branch). Hosted CI must be green at this exact head, including convergence::. If NEVER_TRIMS needs a v19 adjustment, that is a test-only change with a short delta read.
• Risk high: a new claim kind, a new apply request field and a larger body limit on publication routes. Lowered by: no program.step writer yet, an omitted-when-empty wire field, unchanged hashes for missions without a program, the fleet barrier, and held program steps. Queue alone.
• The owner pre-approved queueing this effort's high-risk pull requests on an independent GO plus green CI.
• Verdict: GO; queue alone on CI green at this head, after Mission run authority: parse, render and store the authority block #6260.

[schickling-assistant]

Delta review record for head e574d7dafbb26746a3c1720f52431a87f1408214.

• Hosted CI at 004d6c210b (run 42865212282) failed convergence:: (seeds 1, 2, 34, a_seed_runs_the_same_every_time) and one fractal-schema test. Root cause: the convergence simulation's generic claim draw could write the keyed-only kind client.operation-authorized in a fleet with no member keys; every replica holds such an unsigned envelope pending, so checkpoints never agree. Corpus v19's RNG shift exposed it: a burst at Mission run authority: parse, render and store the authority block #6260's head with only corpus v19 selected fails identically (burst 42875448090). program.step is not implicated.
• Fix (test and fixture only, product behaviour unchanged): the simulation still draws the kind but writes nothing when FractalRuntime::requires_keyed_envelope covers it; the failure report names kind, subject and writer; the schema transition test leaves program.step out like message.expired; claims-v19.json regenerated (one line from main); NEVER_TRIMS = [], measured on hosted run 42884923668. Two merges of main (remerge diffs: none, and two trivial conflicts resolved by keeping both sides).
• Independent reviewer delta read at e574d7dafbb26746a3c1720f52431a87f1408214: GO. The round-1 and round-2 fixes are still in place (missing-bundle refusal, program-step hold, 32 MiB limits, mission_program barrier). Hosted Workspace CI is green at this exact head (run 42894301313).
• Advisories (non-blocking): bodies over 32 MiB return an internal 500 rather than a typed refusal; bundle bytes staged by a refused publication are not reclaimed.
• Risk high: a new claim kind and corpus v19, a new apply request field and larger publication bodies. Queue alone.
• The owner pre-approved queueing this effort's high-risk pull requests on an independent GO plus green CI.
• Verdict: GO; queue alone at this exact head.

[schickling-assistant]

Compounding Manager (agent/compoundingtech/compounding-manager, backup merge-queue coordinator): I removed #6264 from the merge queue. GitHub reported its entry as UNMERGEABLE, with no merge_group run, although git merge-tree against main 88d7f54 shows no conflict. #6316 hit the same GitHub defect at the queue head and blocked every entry behind it. Gates are unchanged. To land it, re-enqueue it after the current entries (#6286, #6272, #6255) finish; if the state stays UNMERGEABLE or UNKNOWN, open a hosted copy.

Posted on behalf of @schickling

field	value
agent_identity	agent/compoundingtech/compounding-manager
session	01a1d992
agent_persona	generalist
agent_supervisor	unavailable
agent_tool	OMP
agent_tool_version	18.12.5
agent_runtime	OMP 18.12.5
agent_model	anthropic/claude-opus-5-5
worktree	unknown

[schickling-assistant]

Delta review record for head 771871d53f3e721384778d7d82afbd2b2b1a285f.

• The merge queue dropped Program engine M1.1a: declare program steps and carry their bundles through apply #6264 at e574d7dafb (GitHub reported the entry UNMERGEABLE), and after Mission run authority: decide every write by a run, audited by default #6272 landed it conflicted with main. This head merges main 5bde948843 into e574d7dafb.
• One conflict, in the daemon.started features JSON in run_up: main added "owned_set_tasks":1, this branch "mission_program":1; both are kept, the rest of the line is unchanged.
• Independent reviewer delta read at 771871d53f3e721384778d7d82afbd2b2b1a285f: GO. Schema, retention, convergence fixtures and convergence.rs are unchanged against e574d7dafb (green on hosted CI); main's owned-set changes do not touch the apply handler, and store_program_bundles still runs before apply_owned_set. All earlier fixes are in place.
• Risk high (unchanged). Queue alone; any further head move needs another delta read.
• The owner pre-approved queueing this effort's high-risk pull requests on an independent GO plus green CI.
• Verdict: GO; queue alone on hosted CI green at this exact head.