[schickling-assistant]

Problem

Decision D6 of the program engine says the data DSL compiles into programs, so there is one execution model (R12). Milestone M3 ships that, with byte and behaviour parity for existing data-DSL missions. Before any code, the compilation, its parity proof and its slicing need a design the owner can accept. The first sketch in §9 (worker steps as child-run worker turns, gates read through the host contract) breaks ${ST_MISSION_RUN} interpolation in products and gates, splits run-scoped agents per step, and needs a graph-read and wait API the host contract does not have.

Goal

A timeless design, docs/st3/program-engine-data-dsl.md, that specifies:

• the compiled form: an orchestrated mission made of the native steps exactly as the data DSL renders them, one generated program step orchestrator (a reserved ID), and orchestrated-by "orchestrator". The program releases native steps; the reconciler keeps the step lifecycle, the normal-phase transition, mission products and gates, the completion frontier, the final phase and the outcome;
• the explicit released set (top-level declared steps and loops only; the orchestrator, finally steps, the synthetic --after step and nested or round steps are admitted natively), the release/resolved host messages, HostContextV1.release, step:PATH journal keys, and the admission rules (the guard, the orchestrator excluded from frontiers, exhaustion and the outcome, no carry-forward of the orchestrator, settling unreleased nodes);
• the full construct mapping: dependencies and parallelism (Effect.all), failed/terminal dependencies, gates, human gates, onFailure/finally, worker steps, uses-mission, inputs and products as Schemas, retries and timeouts, loops, declarations, --after runs;
• in-body constructs that keep their native node: settled local execs (settled-by "program" + settle), program gates over typed products (gate { program } + evaluate), approval checkpoints as asks, values into steps;
• latency, published-mission cutover and state classes, parity tiers, slices, risks and open questions.

§9 of the engine design now summarises this and links to it, the M3 build-plan row links to it, and docs/st3/README.md indexes it.

Decisions

• Released native steps (option B) instead of worker turns as child runs (A) or a daemon-shipped interpreter (C). B keeps native semantics wherever the program has nothing to decide, so step definitions, step-run subjects, work claims, gates, products and run-scoped agents stay as they are. An in-flight native run can be revised onto the compiled revision and carry its completed steps forward. This refines §9 and needs the owner's acceptance (Q1).
• depends-on stays declared as a guard: the daemon refuses a release whose step dependencies do not hold.
• The run's outcome stays native. The orchestrator is not normal work: frontiers and all-steps-exhausted ignore it, the end of the normal phase settles it (completed, or cancelled after a failure), and a cancelled orchestrator never fails a run. Only its own defect fails the run. A released step's final failure ends the normal phase as for any step, so work retry and revision reopen runs exactly as for the native rendering.
• finally and onFailure keep their native final-phase lowering: compensation also runs on cancel, which a program cannot outlive.
• In-body constructs never remove a node. A settled step keeps its ID, depends-on, timeout and place, so dependents, frontiers and onFailure guards never dangle and keep their definition hashes. Only local, restart "never", exit_code-gated execs are settled, and only in missions that declare authority, so the compile never changes who writes.
• settled-by and gate { program } exist only on released nodes; the loader and apply refuse them on final, nested and round steps, loop until and mission gates, where no orchestrator attempt can answer. Revising a native run onto the compiled revision restarts each settled step and its transitive dependents (native revision rule); the tier-2 revision scenario states that trace.
• The capability is advertised only once admission and release both ship (slice 2b); slice 1 refuses orchestrated-by on every member, and the compiler (3b) is based on 2b.
• Parity runs in two tiers. Tier 1 is a TS oracle of the reconciler's admission rules plus property tests over generated graphs on the in-memory host; it only proves agreement with the model. Tier 2 is golden runs of each fixture rendered both ways on scratch daemons, with scenarios for frontier + finally, --after, nested missions, loops, settled steps, program gates, kills and revision. A rendering-parity test checks that the compiled KDL is the native KDL plus two nodes, with each settled step's body as the one named difference.

Verification

Documentation only. The construct semantics were checked against the code they cite: StepSchema/MissionSchema/GateSchema in packages/fractal/src/mod.ts; step_dependencies_hold, mission_completion_selected and the normal-phase transition in crates/fractal-reconcile/src/reconcile.rs; the default completion in crates/fractal-missions-base/src/mission.rs; the mission_program=1 capability refusal in M1.1a (#6264); program.ts (step, ask, run, Worker) from #6266; and the loader in #6304. No build or test was run. Hosted CI at the exact head is the proof for the release note and the docs.

Complexity

One new 448-line document, a 24-line rewrite of §9 and the build-plan row, a two-line index entry and a release note. The planned implementation is ten stacked slices of about 150–350 lines each (see "Slices" in the document).

Concerns

• This proposal changes the accepted §9 mapping (worker steps, gates, uses-mission, finally). Merge it only with the owner's answer to Q1.
• Every data-DSL run's orchestration then depends on the program host (T02), and each run holds a Bun worker for its whole life (Q2). Slice 4b measures worker memory before the cutover.
• The oracle may misread native edges (unreleased nodes, failed/terminal dependencies under the first-failure rule, frontiers, --after, nested missions, loops, reopen). The tier-2 lane validates the oracle, and a mismatch blocks the cutover.
• Each fractal package upgrade revises every compiled mission once (Q4).
• Released steps are about as fast as native admission now that agentless handoff takes ~0.2 s; the latency gain is the in-body path only.

Friction & bottlenecks

• The data-DSL API the owner decided (after handles, onFailure, Fractal.sequence) is not yet in packages/fractal, which has dependsOn with completed/failed/terminal and finally. The mapping covers both forms.
• Native semantics are spread across a 52k-line reconcile.rs. The oracle in slice 3a is the place to write them down once.

Follow-ups

• Owner answers to Q1–Q6 (Q1 before merge; Q4 before slice 3b; Q2 before slice 5; Q3 before slice 7).
• Slices 1 → 2a → 2b (daemon) and 3a (package) in parallel, then 3b, then 4a → 4b (golden harness), then 5 (cutover), then 6 (settled execs) and 7 (decisions over held values).

References

• Design: docs/st3/program-engine.md (docs: accepted program engine design and build plan #6253); stack: Mission run authority: parse, render and store the authority block #6260, Program engine M1.1a: declare program steps and carry their bundles through apply #6264, Program engine M1.1b: run program steps on a supervised Bun host #6265, Program engine M1.2: linked child runs and person asks for program steps #6277, Mission run authority: bind run credentials to mission-run/RUN #6263, Mission run authority: decide every write by a run, audited by default #6272, Mission run authority: enforce by default #6273, Program engine: attempts act as their run (credential in effects, journal as the run) #6283, Show program step journals in missions show and waiting-on #6261, Show program step journals and faults in the run view #6262, Raise program/stalled and program/host faults on program steps #6297; program API fractal: program missions on effect/workflow (fractal/effect) #6266; loader fractal: bundle and render program missions in the tree loader #6304.
• docs/st3/mission-graph-runtime.md (runtime sequence, gates, dependencies, reopen and revision rules).