[schickling-assistant]

Stacked on #6265 (program engine M1.1b). Milestone M1.2 of docs/st3/program-engine.md: §6 linked child runs and asks, §7 serialized sections.

Problem

M1.1b runs program steps, but a program can only run its own effects. The bundle side already offers more (#6266, fractal/effect): Fractal.run(mission, input), worker turns (Fractal.step({ id, success }).pipe(Effect.provide(Fractal.Worker.layer({ mission, input, product })))) and Fractal.ask(...). Their host messages run and ask reach the daemon, and the daemon answers neither, so such a program waits forever. A serialized section (R09, a child run of a concurrent-runs max=1 mission) needs the same child runs.

Goal

A program step can wait on a linked child run or a native person ask. The daemon starts it, learns its end reactively, journals the result under the program's key and resolves the program:

• run: the run's evaluation creates the child with create_child_mission_run (parent_step_run = the program step, idempotency key program:STEP:ATTEMPT:KEY, inputs from the message), or adopts it on resume. When the mission is at capacity, the start waits; the end of a run holding the capacity triggers the retry.
• ask: a native person ask on the program step, with work ask --step semantics (structured requests, deadlines, defaults). The requester is daemon/program. Several asks can be open at once; the step is waiting-person and its program stays loaded.
• When a program step ends, fails, times out or retries while the run goes on, its open child runs and asks are cancelled. A run cancel still cascades through cancel_descendant_mission_runs_tx. A retried attempt starts new children.

Exit shapes, matching #6266:

Waits on	Success	Failure
child run	{ run, status: 'completed', products? }, products keyed by full subject (resource/mission-run/CHILD/…)	{ _tag: 'ChildFailed', run, status, reason }
ask	{ answer?: { type, outcome?, id?, label?, text? }, summary?, resolved_by } (resolved_by: the person or deadline)	{ _tag: 'AskCancelled', ask }

Decisions

• The evaluation owns starting and resolving. The engine's host-reader thread only registers the request and journals a started entry for the key. The parent run's evaluation creates or adopts the child or ask, journals a second started entry with child set, and resolves once it has ended. Its reads (mission-run/CHILD, the ask subject, the runs holding a full mission) are noted, so the store transition that ends them evaluates the parent again (follow_mission_reads) and nothing polls. The started entry is written before the request is registered and the engine's wake then touches the step, so an evaluation that acts on a request is always one the incremental scheduler marked.
• Capacity wait = noted reads of the holders. enforce_mission_run_capacity notes no reads. On mission-run-capacity the evaluation notes mission-run/HOLDER for each active run of the mission (active_mission_runs_for_mission); a holder's terminal claim re-evaluates the parent, which retries the start.
• Program asks go through ask_person with a daemon/program branch, not a second ask path. A program step has no claimant, so for this requester the claimant, incarnation and lease fence is replaced by: the origin is agentless, in ready|working|blocked|waiting-person, its definition in the generation's revision runs a program, and the request's incarnation equals the step's current attempt. Subject hashing, idempotency, deadlines and defaults, projection and person_answers are the agent path's. current() already treats daemon/* requesters as unfenced, so the ask lives while its run lives. PROGRAM_ASKER is a kernel constant shared by store and engine. The legacy work API (POST /v1/work/ask, cancel-ask) takes the actor from the body, so it now refuses daemon/program (reserved-actor); client_v0 already passes the authenticated actor.
• Cancellation reads the journal, not memory. cancel_waits(step, attempt) cancels each child the attempt journaled under a key that has no result yet. Child runs are cancelled with request_mission_run_cancellation; asks with finish_person_step(cancel) as their requester (a stale-fence answer means the ask already ended). It runs from release_left, from stop_if_left, when a program ends (done: children left behind by a lost race) and when a worker is unresponsive. Children already ended and runs already terminal are skipped, so cascades are not repeated.
• A child or ask that starts after its program stopped waiting is still recorded and cancelled. waiting() always journals the started + child entry (the request keeps its bundle hash). If the attempt no longer runs and its step has left the attempt, waiting() cancels it itself. Every path that ends a program sets the step's state before it reads the journal to cancel (done, fail, unresponsive; release_left/stop_if_left act on a step that already left). So either that cancellation sees the new entry, or waiting() sees the step left. A program removed only because its host exited is not cancelled: it resumes and adopts the child.
• Capacity holders are read before the start and again after a refusal, so a holder that ends between the refused start and the listing still evaluates the parent again.
• Store failures are retried, not blamed on the program. A child start or ask that fails with code internal propagates (the evaluation records a fault and the next one tries again, as uses-mission does). Other codes are refusals and fail the step.
• Waiting-person program steps are dispatched. After a daemon restart, the step loop now hands a waiting-person program step to the engine (before the status's early return), so the program replays to the same wait and keeps resolving its other asks and children. Its step timeout stays paused, as for a worker's ask (DQ6).
• A refused request fails the step with the reason (an unknown mission, an input that is not a string, ask deadlines turned off, an ask without a person), like a defect. The protocol gains no new failure semantics.
• The child's starter actor stays in one place. start_program_child is the only call that creates a program's child. Mission-run authority (Mission run authority: parse, render and store the authority block #6260 and its follow-up, create_program_child_mission_run, which writes the creation as mission-run/PARENT) is not on this base yet; when it lands, this one call switches. The requester stays the parent run's requester.

Verification

Focused tests on the Linux build host at head 728faf2150 (#6265 @3ba3725965 merged), nix develop -c nice -n 19 cargo test -p fractal-reconcile --lib -- program:

test reconcile::program::tests::messages_for_another_attempt_or_host_change_nothing ... ok
test reconcile::program::tests::a_revision_runs_the_stored_blob_it_pins ... ok
test reconcile::program::tests::recorded_groups_are_killed_only_when_their_leader_is_verified ... ok
test reconcile::program::tests::replay_returns_this_attempts_results_in_order ... ok
test reconcile::program::tests::a_key_has_one_result_per_attempt ... ok
test reconcile::program::tests::replay_pages_through_long_journals_in_order ... ok
test reconcile::tests::program_children::a_child_started_after_its_step_left_is_journaled_and_cancelled ... ok
test reconcile::tests::program_children::a_child_of_a_full_mission_waits_and_starts_when_the_holder_ends ... ok
test reconcile::tests::program_engine::a_step_that_leaves_its_attempt_stops_its_program_and_effects_at_once ... ok
test reconcile::tests::program_children::asks_are_answered_defaulted_or_withdrawn_and_their_answers_replay ... ok
test reconcile::tests::program_children::a_child_run_is_adopted_on_resume_and_its_result_is_journaled_and_replayed ... ok
test reconcile::tests::program_engine::a_program_resumes_from_its_journal_after_its_host_is_killed ... ok
test reconcile::tests::program_children::children_are_cancelled_with_their_run_or_their_step_and_a_retry_starts_a_new_one ... ok
test reconcile::tests::program_engine::an_unresponsive_worker_fails_only_its_own_step ... ok
test result: ok. 14 passed; 0 failed

Mutation checks (each applied alone, the named test re-run, then reverted):

Mutation	Test	Result
waiting() never cancels a late child	a_child_started_after_its_step_left_is_journaled_and_cancelled	FAILED
no noted reads of capacity holders	a_child_of_a_full_mission_waits_and_starts_when_the_holder_ends	FAILED
release_left cancels no waits	children_are_cancelled_with_their_run_or_their_step_and_a_retry_starts_a_new_one	FAILED

New integration tests (reconcile/tests/program_children.rs) run a real Bun host with a small hand-written bundle against HostContext v1:

• a_child_run_is_adopted_on_resume_and_its_result_is_journaled_and_replayed: killing the host while the child runs leads to the same request, and the daemon adopts the same child; exactly one child is linked to the step. The completed child's value is pinned, products included. A host killed after the result was journaled replays it without a new request.
• a_child_of_a_full_mission_waits_and_starts_when_the_holder_ends: with a concurrent-runs max=1 mission held by another run, the start waits. Cancelling the holder and then one evaluate_changed_mission_runs() (no pass) starts the child (reactive wake); the serialized section then completes.
• children_are_cancelled_with_their_run_or_their_step_and_a_retry_starts_a_new_one: a parent run cancel cascades to its child. A program step failing with an attempt left requests its child's cancellation in that evaluation while the run keeps running. Attempt 2 starts a new child. A failed child pins the exact ChildFailed {_tag, run, status, reason} shape.
• a_child_started_after_its_step_left_is_journaled_and_cancelled: an evaluation holding a request from before the step left starts the child afterwards; the child is journaled and cancelled.
• asks_are_answered_defaulted_or_withdrawn_and_their_answers_replay: three concurrent daemon/program asks; waiting-person keeps the program loaded; one answer resolves while the others stay open; a replaced host replays the answer and adopts the open asks; withdrawal gives AskCancelled { ask }; the deadline applies the default (resolved_by: deadline); every exit is pinned in the journal and in $result.
• messages_for_another_attempt_or_host_change_nothing now also covers run and ask.

Not covered by a test: the internal error retry (a store failure cannot be injected through the public store), the pre-start holder read race, and the daemon/program bound to program steps and its HTTP refusal. fractal-api-core was compiled (cargo check -p fractal-api-core --lib); the large-value inventory check passes.

Hosted CI at the exact head is the merge proof. Pushes skipped the local pre-push evidence (ST_EVIDENCE_LOCAL=off) because the authoring machine is overloaded.

Complexity

About 230 lines in the engine (request registry, waiting/resolve/fail, journal-based cancel_waits), 190 in the reconciler (settle hook, child exit and products), 20 in the store ask path, 430 in tests and docs. The engine and the reconciler are split by thread: only the evaluation can note reads, so starting and resolving live there and the engine keeps what the host asked for. No new abstraction beyond Request/Wait.

Concerns

• Above the ~300-line guide: the engine, the reconciler hook and the ask fence cannot be tested apart from each other with the real host, and the tests are about half the diff.
• Request state is in memory. After a daemon restart the program re-sends its requests (replay), which adopts children and asks by key. A step that left while the daemon was down has its children cancelled only by a later run end. [INFERENCE] That window is rare: the step would have to fail or time out with no daemon running.
• Products are read when the child completes. A product subject that changes later is not re-read; the journaled value stands (R03).
• An ask whose run ends is never resolved; the program ended with it. An ask withdrawn by anyone other than the daemon is rare: only its requester may cancel it.
• A child's cancellation is written as the store's default actor until mission-run authority lands.

Friction & bottlenecks

• branchy new branched from main, which carried main commits Program engine M1.1b: run program steps on a supervised Bun host #6265 lacks. The stacked diff was then unreadable, so the change moved to this clean branch from Program engine M1.1b: run program steps on a supervised Bun host #6265 (schickling/2026-11-10-program-engine-child-runs). The canonical worktree guard on the build host blocks switching a worktree's branch, so the build host needed its own worktree for it.
• The build host was serialized across five slices; this slice waited for two other cold builds before its own.

Follow-ups

• Switch start_program_child to create_program_child_mission_run once the mission-run authority change lands (the creation is then written as mission-run/PARENT).
• M1.3 shows the child of each open key in missions show / waiting-on.

References

• Design: docs/st3/program-engine.md §2, §5, §6, §7, §11 (updated here)
• Depends on Program engine M1.1b: run program steps on a supervised Bun host #6265 (engine), Program engine M1.1a: declare program steps and carry their bundles through apply #6264 (declaration); bundle side fractal: program missions on effect/workflow (fractal/effect) #6266

[schickling-assistant]

Review record for head 728faf21502688d2263483e4f5bfbde4d2123dc7 (stacked on #6265 @3ba3725965).

• Independent reviewer round 1 at 450b819d82: CHANGES. Findings: a child or ask started after its program stopped waiting was orphaned; transient store errors failed the step; capacity holders were noted only after a refusal; the daemon/program ask relaxation was not bounded to program steps.
• Round 2 at 728faf2150 (delta read: fix commit a7385565ac plus merges, remerge diffs empty): GO, no findings, all four answered, with a new test for the late child and mutation checks in the PR body.
• Risk high: the ask_person requester relaxation is a permission change, bounded to steps whose revision runs a program; it also adds child-run creation by the engine. Queue after Program engine M1.1b: run program steps on a supervised Bun host #6265.
• The owner pre-approved queueing this effort's high-risk pull requests on an independent GO plus green CI.
• Verdict: GO; queue once Program engine M1.1b: run program steps on a supervised Bun host #6265 has merged and hosted CI is green at the head.