diff --git a/docs/st3/program-engine.md b/docs/st3/program-engine.md index 6f012f80e31..d8327ea507c 100644 --- a/docs/st3/program-engine.md +++ b/docs/st3/program-engine.md @@ -389,15 +389,17 @@ command (`Fractal.command(argv, { priority: 'background' })`); this is the progr ### 11. Program API -The author-facing API is the `Fractal` namespace of the `fractal` package (D7), on `effect/workflow`: +The author-facing API is the `Fractal` namespace of the `fractal` package's `fractal/effect` entry +point (D7), on `effect/workflow`: | `Fractal` | `effect/workflow` | Engine | |---|---|---| | `Fractal.mission(intent, program)` | `Workflow.make` with `intent.input` as payload and `intent.output` as success schema | one `program` step; the bundle's entry runs the workflow body for the step run's current attempt | | `Fractal.step({ id, success, error }, effect)` | `Activity.make` named by the step's key (`id`, `id#2`, …) | a `program.step` entry under that key, replayed by the key | +| `Fractal.step({ id, success }).pipe(Effect.provide(Fractal.Worker.layer({ mission, input, product })))` | `DurableDeferred.make` named by the step's key | a worker turn: a linked child run of `mission` (§6); the value is its product `resource//` | | `Fractal.ask({ id, for, request, deadline, default })` | `DurableDeferred.make` named by the ask's key | a native person ask on the program step; the answer is journaled under the key | -| `Fractal.run(mission, input)` | a child workflow execution | a linked child run (§6), its terminal result journaled under the key | -| — | `WorkflowEngine` | the host's engine: `activityExecute` returns a journaled result or runs the activity and records it through the host protocol; `deferredResult` reads the journal | +| `Fractal.run(mission, input)` | `DurableDeferred.make` named by the run's key | a linked child run (§6), its terminal result journaled under the key | +| — | `WorkflowEngine` | the host's engine: `activityExecute` returns a journaled result or runs the activity and records it through the host protocol; `deferredResult` reads the journal, else waits for the host's `ctx.child` or `ctx.ask` | Inputs and outputs are Effect Schemas. `intent.input` is a Struct: each field renders as a native mission `input`, `missions start` checks the inputs against it, and the program receives the decoded @@ -407,6 +409,15 @@ value. The success value is encoded with `intent.output` and journaled under the A program waiting on an ask or a child run stays loaded in the host. A replaced host replays the program to the same wait. +**Publication.** The `fractal` package's tree loader (`loadSubjectTree({ root, bundleDir })`) +bundles each `/mission.ts` that declares a program with `bun build --target=bun`, from the tree +root so the bytes do not depend on where the tree is checked out, and writes it to +`/.mjs` after checking that its default export carries the entry. The mission +loads as its declaration: the header from the intent (`timeout`, `concurrentRuns`, `authority`), +one text `input` per field of `intent.input`, and one step `program` pinning the bundle by path and +sha256 (`stalledAfter` becomes its `stalled-after`). Rendering it and running `fractal apply` on the +KDL publishes the mission and carries the bundle (§4). + ## Design questions - **DQ1 Bundle distribution:** decided: replicated blobs stored at `apply` (§4). diff --git a/packages/fractal/COVERAGE.md b/packages/fractal/COVERAGE.md index 8c4a318b000..5d9cd6dd963 100644 --- a/packages/fractal/COVERAGE.md +++ b/packages/fractal/COVERAGE.md @@ -30,6 +30,7 @@ Authoritative grammar: `compoundingtech/smalltalk`, `crates/st3/src/graph.rs` an | `resource` | Partial | vcs.repository, filesystem.file, vcs.pull-request and vcs.ref. Additional native resource kinds are not modeled. | | `doc`, `observer`, `subscription` | Partial | Immutable document bindings and step pins, named/pinned document gates, github.ref observers selecting head/ancestors, and message subscriptions with field conditions. Mission/step/round declarations preserve native run scoping and alias rewriting. Other observer providers and subscription delivery modes are not modeled. | | mission `lane` | Covered | Unique run-owned names, optional slash-terminated `entries` prefixes and `person/` approvers. Lane blocks precede execution blocks and omit absent children. Root lane declarations are not modeled. | +| step `program`, mission `authority`, `concurrent-runs` | Covered | Absolute bundle path, lowercase 64-hex `sha256` and optional `stalled-after`; a program step has no assignee, gates, products, exec or produced/used mission and is never a finally step. Authority rules `publish`, `start`, `revise` (mission IDs) and `write` (subjects), exact or `/*`, rendered in canonical order; the native parser checks `write` subjects against the schema registry. `cancel` is not offered. | | `objective` | Covered | `Fractal.objective`, `indicator.*` and `response.fault` check the native reader's rules (`fractal_kernel::objective`) and `objectiveNode` lowers every indicator, target, window and breach response. A subject tree declares interval objectives only; native telemetry objectives are rendered by `slo/objectives.kdl.genie.ts`. | | `account`, `pty`, `host`, `person`, `mission-run`, `planning-session`, `message`, `repair`, `stop` | Not covered | Additional root grammar nodes omitted. | | `version 2` | Covered | All emitted documents start with this directive. | @@ -38,7 +39,7 @@ Schedule work accepts either a pinned `mission@revision` or an unpinned mission ## Subject trees (`fractal/tree`, Node only) -`loadSubjectTree({ root, context })` discovers `agent.ts|kdl`, `mission.ts|kdl`, `schedule.ts|kdl` and `objective.ts|kdl` declarations recursively. IDs come from directories relative to the root. A seat and mission may share a directory; mission and schedule cannot. Symlink aliases load once under their canonical path, and declarations outside the tree are rejected. +`loadSubjectTree({ root, context, bundleDir? })` discovers `agent.ts|kdl`, `mission.ts|kdl`, `schedule.ts|kdl` and `objective.ts|kdl` declarations recursively. IDs come from directories relative to the root. A seat and mission may share a directory; mission and schedule cannot. Symlink aliases load once under their canonical path, and declarations outside the tree are rejected. A `mission.ts` declaring a program (`fractal/effect`) is bundled with `bun build --target=bun` into the absolute `bundleDir` as `.mjs` and loads as a mission with one `program` step pinning it; the loader refuses a program tree without `bundleDir`, a declared id other than the directory, and a bundle whose default export lacks the program entry. `src/program-loader.unit.test.ts` covers rendering against `fixtures/land-program.kdl`, determinism and refusals; with `ST_BIN` it also runs `fractal validate` on the rendering, which checks the bundle's sha256. TS modules export an intent, a context factory, or `defineAgent|defineMission|defineSchedule({ meta: import.meta, intent })`. Typed agent refs are accepted for `under.target`, required mission `reportTo`, mission/step `assignedTo`, and loop/final workers; they lower to validated `{ id }` agent references, never legacy assignee strings. Typed mission refs lower to unpinned schedule work IDs. Plain imported declarations and attributed people retain the authoring schemas' contracts. diff --git a/packages/fractal/README.md b/packages/fractal/README.md index 0aae80eec57..ba3f72ac9e6 100644 --- a/packages/fractal/README.md +++ b/packages/fractal/README.md @@ -5,7 +5,7 @@ Typed authoring for Fractal declarations: agents, missions, schedules and object | Entrypoint | Runtime | Contents | |---|---|---| | `fractal` | isomorphic | The `Fractal` authoring namespace; declaration schemas and the KDL lowering (`agent`, `mission`, `schedule`, `objectiveNode` nodes, `emit`, `fractalKdl`) as named exports | -| `fractal/tree` | Node | `loadSubjectTree` plus the define forms and reference types | +| `fractal/tree` | Node | `loadSubjectTree`, `programMission`, the define forms and reference types | | `fractal/testing` | Node | `scratchDaemonLayer`, an isolated daemon for conformance tests | | `fractal/effect` | Bun program host, isomorphic | The `Fractal` namespace with the program forms of `mission` and `step`, plus `ask`, `run`, `command`, `env`, `cwd` and `Worker`; the host contract (`ProgramEntry`, `HostContextV1`) as named exports | @@ -75,7 +75,7 @@ The namespace also holds `step`, `gate`, `person`, `input`, `product`, `t`, `doc ## Loading and rendering -`loadSubjectTree({ root, context })` imports every declaration module below `root`, lowers typed references to IDs without copying authored step and gate handles, assembles missions with `missionWire`, decodes the result with the strict schemas, and checks objectives with `Fractal.objective`. Render a loaded subject with the named exports: +`loadSubjectTree({ root, context, bundleDir? })` imports every declaration module below `root`, lowers typed references to IDs without copying authored step and gate handles, assembles missions with `missionWire`, decodes the result with the strict schemas, and checks objectives with `Fractal.objective`. Program missions are bundled into `bundleDir` (see [Publishing a program](#publishing-a-program)). Render a loaded subject with the named exports: ```ts import { agent, emit, mission, objectiveNode } from 'fractal' @@ -131,7 +131,32 @@ export default Fractal.mission( Every `input` field must be required and encode to a string, since each one is a native `input "name" kind="text"`; a payload that does not decode refuses the run with `InvalidInput`. Without `error`, a step or program journals any tagged error as `_tag` plus its fields and the step fails with that plain object (typed `Journaled`), live and on replay, so `catchTag` matches; an untagged error is a defect. With `error`, the schema rebuilds the error as its class, which a program's `error` class needs. Only one fiber may use a step id: concurrent branches would take `id#n` keys in completion order, which replay does not reproduce, so a second branch using the id is a defect. A journal entry the daemon refuses ends the attempt with a defect that the body cannot catch. -A program mission's default export carries `[ProgramEntry]` (`Symbol.for('fractal.program.v1')`), the entry the program host calls with a `HostContextV1`; `fixtures/programs/factory/land-program` is the factory's landing program, and `src/program-bundle.unit.test.ts` runs its `bun build --target=bun` bundle. Bundling a program and rendering its `program` step declaration are not part of `loadSubjectTree` yet. +A program mission's default export carries `[ProgramEntry]` (`Symbol.for('fractal.program.v1')`), the entry the program host calls with a `HostContextV1`. `fixtures/programs/factory/land-program` is the factory's landing program; `src/program-bundle.unit.test.ts` runs its bundle against an in-memory host. + +### Publishing a program + +`loadSubjectTree({ root, context, bundleDir })` bundles every program mission it loads with `bun build --target=bun` (`bun` on `PATH`, or the `bun` option) and writes the bundle to `/.mjs`; `bundleDir` must be absolute, and a tree with a program mission and no `bundleDir` is refused. Bun names inlined modules by their path from the tree root, so the same tree and lockfile give the same bytes wherever they are checked out. Before it pins the bundle the loader checks that its default export carries the entry. The mission then loads like any other, so `emit([mission(intent)])` renders it: + +```kdl +mission "factory/land-program" report-to="agent/factory" state="ready" timeout="2h" { + concurrent-runs max=8 + input "change" kind="text" + input "reviewer" kind="text" + goal "Land one change on main and prove it live, or leave main as it was." + authority { + publish "scratch/*" + start "factory/step/review" + write "resource/factory/*" + } + step "program" timeout="2h" { + program "/bundles/.mjs" sha256="" stalled-after="10m" + } +} +``` + +The intent's `timeout` bounds the mission and its program step, `stalledAfter` becomes the program's `stalled-after`, `concurrentRuns` becomes `concurrent-runs max=N`, and `authority` (`publish`, `start`, `revise` and `write` rules, rendered in canonical order) becomes the run authority block. `fractal validate` checks the KDL and that each bundle still has its pinned sha256; `fractal apply` publishes the mission and carries the bundle to the daemon. `fixtures/land-program.kdl` is the landing program's rendering; after an intended change to the program or to code its bundle inlines, `FRACTAL_UPDATE_GOLDENS=1 pnpm --filter fractal test` rewrites it. + +A loader that walks its own tree renders one program mission the same way with `programMission({ root, file, definition, bundleDir, bun? })`: `file` is a `mission.ts` below the tree `root`, `definition` its default export, and the subject ID its real directory below `root`'s real path. The result is `{ kdl, bundle, sha256 }`, the bytes `loadSubjectTree` would render for that subject. ## Checks diff --git a/packages/fractal/fixtures/land-program.kdl b/packages/fractal/fixtures/land-program.kdl new file mode 100644 index 00000000000..3e32008b389 --- /dev/null +++ b/packages/fractal/fixtures/land-program.kdl @@ -0,0 +1,15 @@ +version 2 +mission "factory/land-program" report-to="agent/factory" state="ready" timeout="2h" { + concurrent-runs max=8 + input "change" kind="text" + input "reviewer" kind="text" + goal "Land one change on main and prove it live, or leave main as it was." + authority { + publish "scratch/*" + start "factory/step/review" + write "resource/factory/*" + } + step "program" timeout="2h" { + program "/bundles/1fbec08c0ddb3fb320a50696b91a1bdb33f429b6bd6a0aab5b1c1f5b8e5dff7d.mjs" sha256="1fbec08c0ddb3fb320a50696b91a1bdb33f429b6bd6a0aab5b1c1f5b8e5dff7d" stalled-after="10m" + } +} diff --git a/packages/fractal/fixtures/program-refusals/id-mismatch/factory/agent.ts b/packages/fractal/fixtures/program-refusals/id-mismatch/factory/agent.ts new file mode 100644 index 00000000000..0849f2f746a --- /dev/null +++ b/packages/fractal/fixtures/program-refusals/id-mismatch/factory/agent.ts @@ -0,0 +1,3 @@ +import { Fractal } from '../../../../src/index.ts' + +export default Fractal.agent({ meta: import.meta, workspace: '/work/factory', restart: 'always' }) diff --git a/packages/fractal/fixtures/program-refusals/id-mismatch/factory/land/mission.ts b/packages/fractal/fixtures/program-refusals/id-mismatch/factory/land/mission.ts new file mode 100644 index 00000000000..8eebef82779 --- /dev/null +++ b/packages/fractal/fixtures/program-refusals/id-mismatch/factory/land/mission.ts @@ -0,0 +1,16 @@ +import { Effect, Schema } from 'effect' + +import { Fractal } from '../../../../../src/effect.ts' +import factory from '../agent.ts' + +/** Declares an id that is not its directory. */ +export default Fractal.mission( + { + meta: import.meta, + id: 'factory/elsewhere', + goal: 'Land.', + reportTo: factory, + input: Schema.Struct({ change: Schema.String }), + }, + () => Effect.void, +) diff --git a/packages/fractal/fixtures/program-refusals/number-input/factory/agent.ts b/packages/fractal/fixtures/program-refusals/number-input/factory/agent.ts new file mode 100644 index 00000000000..0849f2f746a --- /dev/null +++ b/packages/fractal/fixtures/program-refusals/number-input/factory/agent.ts @@ -0,0 +1,3 @@ +import { Fractal } from '../../../../src/index.ts' + +export default Fractal.agent({ meta: import.meta, workspace: '/work/factory', restart: 'always' }) diff --git a/packages/fractal/fixtures/program-refusals/number-input/factory/land/mission.ts b/packages/fractal/fixtures/program-refusals/number-input/factory/land/mission.ts new file mode 100644 index 00000000000..d8aaf00c395 --- /dev/null +++ b/packages/fractal/fixtures/program-refusals/number-input/factory/land/mission.ts @@ -0,0 +1,17 @@ +import { Effect, Schema } from 'effect' + +import { Fractal } from '../../../../../src/effect.ts' +import factory from '../agent.ts' + +/** A number input does not encode to a native text input. */ +export default Fractal.mission( + { + meta: import.meta, + id: 'factory/land', + goal: 'Land.', + reportTo: factory, + // @ts-expect-error a number does not encode to a native text input + input: Schema.Struct({ attempts: Schema.Number }), + }, + () => Effect.void, +) diff --git a/packages/fractal/fixtures/programs/factory/land-program/mission.ts b/packages/fractal/fixtures/programs/factory/land-program/mission.ts index 75bd6449ac0..bc5539b38f4 100644 --- a/packages/fractal/fixtures/programs/factory/land-program/mission.ts +++ b/packages/fractal/fixtures/programs/factory/land-program/mission.ts @@ -45,6 +45,13 @@ export default Fractal.mission( goal: 'Land one change on main and prove it live, or leave main as it was.', reportTo: factory, timeout: '2h', + stalledAfter: '10m', + concurrentRuns: 8, + authority: { + publish: ['scratch/*'], + start: ['factory/step/review'], + write: ['resource/factory/*'], + }, input: Schema.Struct({ change: Schema.String, reviewer: Schema.Literals(['llm', 'scripted']) }), output: Schema.Struct({ class: steps.Class, merged: Schema.String, head: Schema.String }), }, diff --git a/packages/fractal/src/mod.ts b/packages/fractal/src/mod.ts index 6e9dd7c0bf7..833926843b1 100644 --- a/packages/fractal/src/mod.ts +++ b/packages/fractal/src/mod.ts @@ -685,6 +685,82 @@ const OwnedDeclarationFields = { ), } +/** + * `program "/abs/bundle.mjs" sha256="…" stalled-after="…"?`: the step runs a program bundle the + * daemon pins by its sha256 (docs/st3/program-engine.md §3, §4). + */ +export const ProgramSpecSchema = Schema.Struct({ + bundle: Schema.String.pipe( + Schema.refine((value): value is string => value.startsWith('/'), { + message: 'a program bundle path must be absolute', + }), + ), + sha256: Schema.String.pipe( + Schema.refine((value): value is string => /^[0-9a-f]{64}$/u.test(value), { + message: 'a program sha256 is 64 lowercase hex digits', + }), + ), + stalledAfter: Schema.optionalKey(Duration), +}).annotate({ identifier: 'Fractal.ProgramSpec' }) + +/** + * An authority pattern is exact or a terminal `/*` namespace. Mission verbs take mission IDs + * without `mission/`; `write` takes a subject, which the native parser checks against the schema + * registry. + */ +const authorityPattern = (target: 'mission' | 'subject') => + Schema.String.pipe( + Schema.refine( + (pattern): pattern is string => { + const exact = pattern.endsWith('/*') ? pattern.slice(0, -2) : pattern + return ( + exact.length > 0 && + exact.includes('*') === false && + /\s/u.test(exact) === false && + exact.split('/').includes('') === false && + (target === 'subject' || exact.startsWith('mission/') === false) + ) + }, + { + message: `an authority ${target === 'mission' ? 'mission ID' : 'subject'} is exact or ends in a /* namespace`, + }, + ), + ) +const authorityRules = (target: 'mission' | 'subject') => + Schema.optionalKey( + Schema.NonEmptyArray(authorityPattern(target)).pipe( + Schema.refine((values): values is typeof values => new Set(values).size === values.length, { + message: 'an authority rule is declared once', + }), + ), + ) +/** The verbs of a mission's `authority` block, in their canonical order. */ +export const authorityVerbs = ['publish', 'start', 'revise', 'write'] as const +/** What runs of a top-level mission may do as `mission-run/RUN` (docs/st3/program-engine.md §3). */ +export const AuthoritySchema = Schema.Struct({ + publish: authorityRules('mission'), + start: authorityRules('mission'), + revise: authorityRules('mission'), + write: authorityRules('subject'), +}).pipe( + Schema.refine( + (value): value is typeof value => authorityVerbs.some((verb) => value[verb] !== undefined), + { message: 'an authority block needs at least one rule' }, + ), + Schema.annotate({ identifier: 'Fractal.Authority' }), +) +/** A mission's `authority` block: rules by verb. */ +export type Authority = typeof AuthoritySchema.Encoded + +/** Renders rules in the parser's canonical order, so reordering them is no revision. */ +const authorityNode = (authority: typeof AuthoritySchema.Type): Node => + block({ + name: 'authority', + children: authorityVerbs.flatMap((verb) => + (authority[verb] ?? []).toSorted().map((pattern) => child({ name: verb, value: pattern })), + ), + }) + const UsedMissionSchema = Schema.Union([ Schema.Struct({ kind: Schema.Literal('revision'), revision: Revision }), Schema.Struct({ kind: Schema.Literal('output'), outputOf: LocalId }), @@ -714,6 +790,7 @@ export const StepSchema = Schema.Struct({ ), producesMission: Schema.optionalKey(MissionId), usesMission: Schema.optionalKey(UsedMissionSchema), + program: Schema.optionalKey(ProgramSpecSchema), }).pipe( Schema.refine( (s): s is typeof s => (s.agentless === true && s.assignedTo !== undefined) === false, @@ -731,6 +808,20 @@ export const StepSchema = Schema.Struct({ message: 'feedback human gates require a worker step', }, ), + Schema.refine( + (s): s is typeof s => + s.program === undefined || + (s.assignedTo === undefined && + s.gates === undefined && + s.produces === undefined && + s.producesMission === undefined && + s.usesMission === undefined && + s.exec === undefined), + { + message: + 'a program step alone decides its outcome: no assignee, gates, products, exec, or produced or used mission', + }, + ), Schema.annotate({ identifier: 'Fractal.Step' }), ) /** All exhausted work, or an explicit completion frontier of normal steps. */ @@ -873,6 +964,14 @@ export const MissionSchema = Schema.Struct({ state: Schema.Literal('ready'), reportTo: ReportToSchema, assignedTo: Schema.optionalKey(Schema.Union([AgentReferenceSchema, PersonReferenceSchema])), + concurrentRuns: Schema.optionalKey( + Schema.Int.pipe( + Schema.refine((value): value is number => value > 0 && value <= 0xffff_ffff, { + message: 'concurrent-runs max must be a positive integer', + }), + ), + ), + authority: Schema.optionalKey(AuthoritySchema), timeout: Schema.optionalKey(Duration), stalledAfter: Schema.optionalKey(Duration), reportStalls: Schema.optionalKey(Schema.Boolean), @@ -916,7 +1015,13 @@ export const MissionSchema = Schema.Struct({ // Try loops first: plain steps accept an id alone and would strip loop fields. steps: Schema.Array(Schema.Union([LoopSchema, StepSchema])), completion: Schema.optionalKey(CompletionSchema), - finally: Schema.optionalKey(Schema.NonEmptyArray(StepSchema)), + finally: Schema.optionalKey( + Schema.NonEmptyArray(StepSchema).pipe( + Schema.refine((steps): steps is typeof steps => steps.every((s) => s.program === undefined), { + message: 'a program step cannot be a finally step; compensate inside the program', + }), + ), + ), schedule: Schema.optionalKey(ScheduleSchema), }).pipe( Schema.refine((m): m is typeof m => validGraph(m), { @@ -1576,6 +1681,19 @@ const lowerStep = (input: typeof StepSchema.Encoded): Node => { ) } for (const goal of s.goal ?? []) children.push(child({ name: 'goal', value: goal })) + if (s.program !== undefined) + children.push( + node({ + name: 'program', + args: [s.program.bundle], + props: { + sha256: s.program.sha256, + ...(s.program.stalledAfter === undefined + ? {} + : { 'stalled-after': lowerDuration(s.program.stalledAfter) }), + }, + }), + ) for (const document of s.documents ?? []) children.push(child({ name: 'document', value: document })) children.push(...ownedDeclarationNodes(s)) @@ -2334,6 +2452,9 @@ export const mission = (input: MissionInput node({ name: 'input', args: [value.name], props: { kind: value.kind } }), ), @@ -2350,6 +2471,7 @@ export const mission = (input: MissionInput child({ name: 'constraint', value: constraint }), ), + ...(m.authority === undefined ? [] : [authorityNode(m.authority)]), ...ownedDeclarationNodes(m), ...(m.schedule === undefined ? [] : [schedule(m.schedule)]), ...(m.completion === undefined ? [] : [completion(m.completion)]), diff --git a/packages/fractal/src/program-bundler.ts b/packages/fractal/src/program-bundler.ts new file mode 100644 index 00000000000..5f7bb0d2b23 --- /dev/null +++ b/packages/fractal/src/program-bundler.ts @@ -0,0 +1,88 @@ +/** + * Builds the bundle a program mission's `program` step pins (docs/st3/program-engine.md §4): one + * self-contained ES module with Effect and the `fractal/effect` runtime inside, named by its + * sha256. Node-only; it runs Bun. + */ +import { spawn } from 'node:child_process' +import { createHash, randomUUID } from 'node:crypto' +import { join } from 'node:path' + +import { Data, Effect, FileSystem } from 'effect' + +/** The program bundle of `module` could not be built or is not a program. */ +export class ProgramBundleError extends Data.TaggedError('ProgramBundleError')<{ + readonly message: string +}> {} + +const run = ({ bun, args, cwd }: { bun: string; args: readonly string[]; cwd: string }) => + Effect.callback<{ readonly status: number | null; readonly output: string }, ProgramBundleError>( + (resume) => { + const child = spawn(bun, args, { cwd, stdio: ['ignore', 'pipe', 'pipe'] }) + let output = '' + child.stdout.setEncoding('utf8').on('data', (chunk: string) => (output += chunk)) + child.stderr.setEncoding('utf8').on('data', (chunk: string) => (output += chunk)) + child.once('error', (cause) => + resume(Effect.fail(new ProgramBundleError({ message: `${bun}: ${cause.message}` }))), + ) + child.once('close', (status) => resume(Effect.succeed({ status, output }))) + return Effect.sync(() => child.kill('SIGKILL')) + }, + ) + +/** Loads the bundle as the program host does and checks its default export is a program. */ +const entryCheck = (bundle: string) => + `const m = await import(${JSON.stringify(bundle)}); ` + + `if (typeof m.default?.[Symbol.for('fractal.program.v1')] !== 'function') ` + + `{ console.error('its default export does not carry Symbol.for("fractal.program.v1")'); process.exit(1) }` + +/** + * Bundles `module` with `bun build --target=bun` and writes it to `/.mjs`. + * Bun names inlined modules by their path relative to `cwd`, so building from the tree root keeps + * the bytes, and the hash, independent of where the tree is checked out. Only a bundle that is a + * program reaches `bundleDir`, and it appears there whole: it is renamed into place, and a bundle + * already present under its hash is never rewritten. + */ +export const bundleProgram = Effect.fnUntraced(function* ({ + module, + cwd, + bundleDir, + bun, +}: { + readonly module: string + readonly cwd: string + readonly bundleDir: string + readonly bun: string +}) { + const fs = yield* FileSystem.FileSystem + const failed = (message: string) => (cause: unknown) => + new ProgramBundleError({ message: `${message}: ${String(cause)}` }) + const scratch = yield* fs + .makeTempDirectoryScoped({ prefix: 'fractal-program-' }) + .pipe(Effect.mapError(failed('cannot create a build directory'))) + const built = join(scratch, 'bundle.mjs') + const build = yield* run({ + bun, + cwd, + args: ['build', module, '--target=bun', '--format=esm', `--outfile=${built}`], + }) + if (build.status !== 0) + return yield* new ProgramBundleError({ message: `bun build failed:\n${build.output.trim()}` }) + const check = yield* run({ bun, cwd, args: ['--eval', entryCheck(built)] }) + if (check.status !== 0) + return yield* new ProgramBundleError({ + message: `${module} is not a program: ${check.output.trim()}`, + }) + const bytes = yield* fs.readFile(built).pipe(Effect.mapError(failed('cannot read the bundle'))) + const sha256 = createHash('sha256').update(bytes).digest('hex') + const bundle = join(bundleDir, `${sha256}.mjs`) + const present = yield* fs.exists(bundle).pipe(Effect.mapError(failed(`cannot read ${bundle}`))) + if (!present) { + const staged = join(bundleDir, `.${sha256}.${randomUUID()}.tmp`) + yield* fs.makeDirectory(bundleDir, { recursive: true }).pipe( + Effect.andThen(fs.writeFile(staged, bytes)), + Effect.andThen(fs.rename(staged, bundle)), + Effect.mapError(failed(`cannot write ${bundle}`)), + ) + } + return { bundle, sha256 } +}, Effect.scoped) diff --git a/packages/fractal/src/program-loader.unit.test.ts b/packages/fractal/src/program-loader.unit.test.ts new file mode 100644 index 00000000000..74c6224684a --- /dev/null +++ b/packages/fractal/src/program-loader.unit.test.ts @@ -0,0 +1,287 @@ +import { spawnSync } from 'node:child_process' +import { createHash } from 'node:crypto' +import { mkdtempSync, readFileSync, symlinkSync, writeFileSync } from 'node:fs' +import { tmpdir } from 'node:os' +import { join } from 'node:path' +import { fileURLToPath, pathToFileURL } from 'node:url' + +import { NodeFileSystem } from '@effect/platform-node' +import { describe, expect, it } from '@effect/vitest' +import { Effect } from 'effect' + +import { emit, mission, type MissionInput } from './mod.ts' +import { bundleProgram } from './program-bundler.ts' +import { loadSubjectTree, programMission } from './tree.ts' + +const reportTo = { id: 'factory' } +const sha256 = 'a'.repeat(64) +const programStep = { id: 'program', program: { bundle: '/bundles/land.mjs', sha256 } } +const render = (input: Partial>) => + emit([ + mission({ + id: 'factory/land', + state: 'ready', + reportTo, + goal: 'Land.', + steps: [programStep], + ...input, + }), + ]) + +describe('program declarations as data', () => { + it('render the program step, run authority in canonical order and concurrent runs', () => { + expect( + render({ + concurrentRuns: 2, + authority: { write: ['resource/z', 'resource/a/*'], revise: ['factory/*'], publish: ['x'] }, + steps: [ + { + ...programStep, + timeout: '1h', + program: { ...programStep.program, stalledAfter: '5m' }, + }, + ], + }), + ).toBe( + [ + 'version 2', + 'mission "factory/land" report-to="agent/factory" state="ready" {', + ' concurrent-runs max=2', + ' goal "Land."', + ' authority {', + ' publish "x"', + ' revise "factory/*"', + ' write "resource/a/*"', + ' write "resource/z"', + ' }', + ' step "program" timeout="1h" {', + ` program "/bundles/land.mjs" sha256="${sha256}" stalled-after="5m"`, + ' }', + '}', + '', + ].join('\n'), + ) + }) + + it('refuse a program step that does not alone decide its outcome', () => { + for (const extra of [ + { assignedTo: { id: 'factory' } }, + { producesMission: 'factory/child' }, + { usesMission: { kind: 'revision', revision: `factory/x@${'0'.repeat(64)}` } }, + ] as const) + expect(() => render({ steps: [{ ...programStep, ...extra }] })).toThrow( + 'a program step alone decides its outcome', + ) + expect(() => + render({ steps: [{ id: 'work', agentless: true }], finally: [programStep] }), + ).toThrow('a program step cannot be a finally step') + expect(() => + render({ steps: [{ ...programStep, program: { bundle: 'land.mjs', sha256 } }] }), + ).toThrow('a program bundle path must be absolute') + expect(() => + render({ + steps: [{ ...programStep, program: { bundle: '/land.mjs', sha256: 'A'.repeat(64) } }], + }), + ).toThrow('a program sha256 is 64 lowercase hex digits') + }) + + it('refuse malformed authority and concurrent runs', () => { + expect(() => render({ authority: {} })).toThrow('an authority block needs at least one rule') + expect(() => render({ authority: { start: ['mission/factory/x'] } })).toThrow( + 'an authority mission ID is exact or ends in a /* namespace', + ) + expect(() => render({ authority: { write: ['resource/*/x'] } })).toThrow( + 'an authority subject is exact or ends in a /* namespace', + ) + expect(() => render({ authority: { publish: ['x', 'x'] } })).toThrow( + 'an authority rule is declared once', + ) + expect(() => render({ concurrentRuns: 0 })).toThrow( + 'concurrent-runs max must be a positive integer', + ) + }) +}) +const fixtures = fileURLToPath(new URL('../fixtures', import.meta.url)) +const programs = join(fixtures, 'programs') +const golden = join(fixtures, 'land-program.kdl') + +/** Renders the landing program with its bundle directory shown as `/bundles`. */ +const renderLanding = (bundleDir: string) => + Effect.gen(function* () { + const subjects = yield* loadSubjectTree({ root: programs, context: undefined, bundleDir }) + const landing = subjects.find((subject) => subject.id === 'factory/land-program') + if (landing?.declaration._tag !== 'Module') + return yield* Effect.die('the landing program did not load') + const kdl = emit([mission(landing.declaration.intent as MissionInput)]) + const sha256 = /sha256="([0-9a-f]{64})"/u.exec(kdl)?.[1] + if (sha256 === undefined) return yield* Effect.die(`no program step in\n${kdl}`) + return { kdl, sha256, shown: kdl.replaceAll(bundleDir, '/bundles') } + }).pipe(Effect.provide(NodeFileSystem.layer)) + +const refusal = (root: string, bundleDir: string | undefined) => + loadSubjectTree({ + root, + context: undefined, + ...(bundleDir === undefined ? {} : { bundleDir }), + }).pipe(Effect.flip, Effect.provide(NodeFileSystem.layer)) + +const scratch = () => mkdtempSync(join(tmpdir(), 'fractal-bundles-')) + +describe('program missions', () => { + it.effect( + 'render to a pinned bundle and the KDL that runs it', + () => + Effect.gen(function* () { + const bundleDir = scratch() + const { kdl, sha256, shown } = yield* renderLanding(bundleDir) + // `FRACTAL_UPDATE_GOLDENS=1` rewrites the golden after an intended change to the program + // or to code its bundle inlines. + if (process.env.FRACTAL_UPDATE_GOLDENS === '1') writeFileSync(golden, shown) + expect(shown).toBe(readFileSync(golden, 'utf8')) + const bundle = readFileSync(join(bundleDir, `${sha256}.mjs`)) + expect(createHash('sha256').update(bundle).digest('hex')).toBe(sha256) + expect(kdl).toContain(`program "${join(bundleDir, `${sha256}.mjs`)}"`) + }), + 120_000, + ) + + it.effect( + 'bundle deterministically', + () => + Effect.gen(function* () { + const first = scratch() + const second = scratch() + const a = yield* renderLanding(first) + const b = yield* renderLanding(second) + expect(b.sha256).toBe(a.sha256) + expect(readFileSync(join(second, `${b.sha256}.mjs`))).toEqual( + readFileSync(join(first, `${a.sha256}.mjs`)), + ) + }), + 120_000, + ) + + it.effect( + 'render the same through programMission for a loader that walks its own tree', + () => + Effect.gen(function* () { + const bundleDir = scratch() + const { kdl, sha256 } = yield* renderLanding(bundleDir) + // Reached through a symlink, the tree still names its subjects by their real directories. + const root = join(scratch(), 'tree') + symlinkSync(programs, root) + const file = join(root, 'factory/land-program/mission.ts') + const real = pathToFileURL(join(programs, 'factory/land-program/mission.ts')).href + const module = yield* Effect.promise( + (): Promise<{ readonly default: unknown }> => import(real), + ) + const rendered = yield* programMission({ root, file, bundleDir, definition: module.default }) + expect(rendered).toEqual({ kdl, bundle: join(bundleDir, `${sha256}.mjs`), sha256 }) + const refused = yield* programMission({ root, file, bundleDir, definition: {} }).pipe( + Effect.flip, + ) + expect(refused.message).toBe('The default export is not a program mission') + const elsewhere = join(fixtures, 'program-refusals/id-mismatch/factory/land/mission.ts') + const foreign = yield* programMission({ + root, + file: elsewhere, + bundleDir, + definition: module.default, + }).pipe(Effect.flip) + expect(foreign.message).toContain('Module must export its own mission definition') + }).pipe(Effect.provide(NodeFileSystem.layer)), + 120_000, + ) + + it.effect('need an absolute bundle directory', () => + Effect.gen(function* () { + const missing = yield* refusal(programs, undefined) + expect(missing.path).toBe('factory/land-program/mission.ts') + expect(missing.message).toContain('needs loadSubjectTree({ bundleDir })') + const relative = yield* refusal(programs, 'bundles') + expect(relative.message).toBe('bundleDir must be an absolute directory') + }), + ) + + it.effect('declare their own directory as their id', () => + Effect.gen(function* () { + const error = yield* refusal(join(fixtures, 'program-refusals/id-mismatch'), scratch()) + expect(error.path).toBe('factory/land/mission.ts') + expect(error.message).toBe( + 'Declared id factory/elsewhere must equal the subject directory factory/land', + ) + }), + ) + + it.effect('have inputs that encode to strings', () => + Effect.gen(function* () { + const error = yield* refusal(join(fixtures, 'program-refusals/number-input'), scratch()) + expect(error.path).toBe('factory/land/mission.ts') + expect(String(error.cause)).toContain( + 'Program input attempts must be required and encode to a string', + ) + }), + ) +}) + +describe('program bundles', () => { + const build = (source: string, bun = 'bun') => + Effect.gen(function* () { + const dir = scratch() + writeFileSync(join(dir, 'mission.ts'), source) + return yield* bundleProgram({ + module: join(dir, 'mission.ts'), + cwd: dir, + bundleDir: dir, + bun, + }) + }).pipe(Effect.flip, Effect.provide(NodeFileSystem.layer)) + + it.effect('refuse a module whose default export is not a program', () => + Effect.gen(function* () { + const error = yield* build('export default { name: "not a program" }\n') + expect(error.message).toContain('is not a program') + expect(error.message).toContain('Symbol.for("fractal.program.v1")') + }), + ) + + it.effect('refuse a module bun cannot build', () => + Effect.gen(function* () { + const error = yield* build("export { default } from 'no-such-package'\n") + expect(error.message).toContain('bun build failed') + expect(error.message).toContain('no-such-package') + }), + ) + + it.effect('refuse a missing bun', () => + Effect.gen(function* () { + const error = yield* build('export default {}\n', '/nonexistent/bun') + expect(error.message).toContain('/nonexistent/bun') + }), + ) +}) + +// Native conformance is opt-in, as elsewhere in this package: `ST_BIN` is the `fractal` binary. +const stBin = process.env.ST_BIN +const testWithSt = stBin !== undefined && stBin !== '' ? it.effect : it.effect.skip + +describe('program declarations', () => { + testWithSt( + 'validate natively, including the bundle hash', + () => + Effect.gen(function* () { + const bundleDir = scratch() + const { kdl, sha256 } = yield* renderLanding(bundleDir) + const file = join(bundleDir, 'land-program.kdl') + writeFileSync(file, kdl) + const valid = spawnSync(stBin!, ['validate', file], { encoding: 'utf8' }) + expect(valid.status, `${valid.stderr}\n${valid.stdout}`).toBe(0) + // The native parser reads the bundle and refuses one whose bytes are not the pinned ones. + writeFileSync(join(bundleDir, `${sha256}.mjs`), '// changed after rendering\n') + const stale = spawnSync(stBin!, ['validate', file], { encoding: 'utf8' }) + expect(stale.status).not.toBe(0) + expect(stale.stderr).toContain(`the mission declares ${sha256}`) + }), + 120_000, + ) +}) diff --git a/packages/fractal/src/program.ts b/packages/fractal/src/program.ts index 2a1dfcd37fb..a14e5419cc1 100644 --- a/packages/fractal/src/program.ts +++ b/packages/fractal/src/program.ts @@ -8,7 +8,7 @@ import { Context, Effect, Layer, Schedule, Schema, SchemaAST } from 'effect' import { Activity, DurableDeferred, type Workflow } from 'effect/workflow' import { mission as dataMission, type MissionIntent } from './define.ts' -import type { PersonReference } from './mod.ts' +import type { Authority, PersonReference } from './mod.ts' import { keyed, Program, @@ -76,7 +76,14 @@ export interface ProgramIntent< readonly id: string readonly goal: string readonly reportTo: MissionIntent['reportTo'] + /** The mission's and its program step's timeout. */ readonly timeout?: string + /** How long the program may go without a journal entry before its run reports it stalled. */ + readonly stalledAfter?: string + /** At most this many runs at once (`concurrent-runs max=N`). */ + readonly concurrentRuns?: number + /** What the mission's runs may do as `mission-run/RUN` (docs/st3/program-engine.md §3). */ + readonly authority?: Authority readonly input: I readonly output?: O readonly error?: ES @@ -95,7 +102,6 @@ export interface ProgramMission< type Meta = { readonly meta: { readonly url: string } } -/** Each input is one native `input "name" kind="text"`, and a run supplies every one. */ const encodesToString = (ast: SchemaAST.AST): boolean => SchemaAST.isString(ast) || SchemaAST.isTemplateLiteral(ast) || @@ -103,6 +109,20 @@ const encodesToString = (ast: SchemaAST.AST): boolean => (SchemaAST.isEnum(ast) && ast.enums.every(([, value]) => typeof value === 'string')) || (SchemaAST.isUnion(ast) && ast.types.every(encodesToString)) +/** + * The native inputs of a program mission: each field of `input` is one `input "name" kind="text"`, + * and a run supplies every one, so a field must be required and encode to a string. + */ +export const programInputNames = (input: Workflow.AnyStructSchema): readonly string[] => + Object.entries(input.fields).map(([name, field]) => { + if ( + SchemaAST.isOptional(field.ast) || + encodesToString(SchemaAST.toEncoded(field.ast)) === false + ) + throw new TypeError(`Program input ${name} must be required and encode to a string`) + return name + }) + /** * Declares a mission. With flat fields it is a data mission (see `Fractal.mission` in `fractal`). * With `(intent, program)` it is a program: `program` receives the decoded `intent.input`, its @@ -134,12 +154,7 @@ export function mission( ): SubjectDefinition<'mission', unknown, never> { if (program === undefined) return dataMission(intent as Meta & MissionIntent) const { meta, ...rest } = intent as Meta & ProgramIntent - for (const [name, field] of Object.entries(rest.input.fields)) - if ( - SchemaAST.isOptional(field.ast) || - encodesToString(SchemaAST.toEncoded(field.ast)) === false - ) - throw new TypeError(`Program input ${name} must be required and encode to a string`) + programInputNames(rest.input) const definition: ProgramMission = { _tag: 'SubjectRef', kind: 'mission', diff --git a/packages/fractal/src/tree.ts b/packages/fractal/src/tree.ts index ea3f73b871f..fadf8bab46d 100644 --- a/packages/fractal/src/tree.ts +++ b/packages/fractal/src/tree.ts @@ -1,13 +1,24 @@ -import { basename, dirname, join, relative, sep } from 'node:path' +import { basename, dirname, isAbsolute, join, relative, sep } from 'node:path' import { fileURLToPath, pathToFileURL } from 'node:url' import { Effect, FileSystem, Schema } from 'effect' import { parse } from '@overeng/kdl' -import { AgentSchema, MissionSchema, missionWire, type MissionInput, SubjectId } from './mod.ts' import type { PinnedMissionRef } from './define.ts' +import { + AgentSchema, + emit, + mission as missionNode, + MissionSchema, + missionWire, + type MissionInput, + SubjectId, +} from './mod.ts' import { objective, validName, type Objective, type ObjectiveInput } from './objective.ts' +import { bundleProgram } from './program-bundler.ts' +import { ProgramEntry } from './program-engine.ts' +import { programInputNames, type InputSchema, type ProgramIntent } from './program.ts' import type { SubjectKind, SubjectRef } from './subject-ref.ts' export { defineAgent, @@ -278,6 +289,109 @@ const lowerReferences: (args: { }, ) +/** Where the loader writes program bundles, and the Bun that builds them. */ +interface ProgramOptions { + readonly bundleDir: string | undefined + readonly bun: string +} + +/** + * The declaration of a program mission (docs/st3/program-engine.md §3): its native inputs, one + * per field of `input`, and one `program` step pinning the module's bundle by sha256. + */ +const programDeclaration = Effect.fnUntraced(function* ({ + root, + file, + path, + id, + intent, + programs, +}: { + readonly root: string + readonly file: string + readonly path: string + readonly id: SubjectId + readonly intent: ProgramIntent, undefined> + readonly programs: ProgramOptions +}) { + if (intent.id !== id) { + return yield* new SubjectTreeError({ + path, + message: `Declared id ${intent.id} must equal the subject directory ${id}`, + }) + } + const inputs = yield* Effect.try({ + try: () => programInputNames(intent.input), + catch: (cause) => new SubjectTreeError({ path, message: 'Invalid program inputs', cause }), + }) + if (programs.bundleDir === undefined) { + return yield* new SubjectTreeError({ + path, + message: + 'A program mission needs loadSubjectTree({ bundleDir }), the absolute directory its bundle is written to', + }) + } + const { bundle, sha256 } = yield* bundleProgram({ + module: file, + cwd: root, + bundleDir: programs.bundleDir, + bun: programs.bun, + }).pipe( + Effect.mapError( + (cause) => new SubjectTreeError({ path, message: 'Cannot bundle program', cause }), + ), + ) + const timeout = intent.timeout === undefined ? {} : { timeout: intent.timeout } + const declaration = { + id, + state: 'ready', + reportTo: intent.reportTo, + goal: intent.goal, + ...timeout, + ...(intent.concurrentRuns === undefined ? {} : { concurrentRuns: intent.concurrentRuns }), + ...(intent.authority === undefined ? {} : { authority: intent.authority }), + inputs: inputs.map((name) => ({ name, kind: 'text' })), + steps: [ + { + id: 'program', + ...timeout, + program: { + bundle, + sha256, + ...(intent.stalledAfter === undefined ? {} : { stalledAfter: intent.stalledAfter }), + }, + }, + ], + } + return { declaration, bundle, sha256 } +}) + +/** Lowers a mission intent's references and decodes the native mission it declares. */ +const decodeMission = Effect.fnUntraced(function* ({ + root, + path, + kind, + intent, +}: { + readonly root: string + readonly path: string + readonly kind: Exclude + readonly intent: unknown +}) { + const invalid = (cause: unknown) => + new SubjectTreeError({ path, message: `Invalid ${kind} declaration`, cause }) + const lowered = yield* lowerReferences({ root, path, intent }) + const wire = yield* Effect.try({ + // Trusted module authoring is validated by missionWire's strict schemas; the + // boundary assertion preserves templates/handles until assembly, not a wire cast. + try: () => missionWire(lowered as MissionInput), + catch: invalid, + }) + return yield* Schema.decodeEffect(MissionSchema, { onExcessProperty: 'error' })(wire).pipe( + Effect.mapError(invalid), + ) +}) + const loadModule = Effect.fnUntraced(function* ({ root, file, @@ -285,6 +399,7 @@ const loadModule = Effect.fnUntraced(function* ({ kind, id, context, + programs, }: { readonly root: string /** The module's realpath, which is what Node reports as its `import.meta.url`. */ @@ -293,6 +408,7 @@ const loadModule = Effect.fnUntraced(function* ({ readonly kind: SubjectKind readonly id: SubjectId readonly context: C + readonly programs: ProgramOptions }) { const url = pathToFileURL(file).href const exports = yield* Effect.tryPromise({ @@ -312,12 +428,25 @@ const loadModule = Effect.fnUntraced(function* ({ message: `Module must export its own ${kind} definition, got ${definition.kind} from ${definition.url}`, }) } + const program = definition !== undefined && ProgramEntry in definition ? definition : undefined const source = definition === undefined ? value : 'intent' in definition ? definition.intent : undefined - const intent = yield* Effect.try({ - try: (): unknown => (typeof source === 'function' ? source(context) : source), - catch: (cause) => new SubjectTreeError({ path, message: 'Subject factory failed', cause }), - }) + const intent = + program === undefined + ? yield* Effect.try({ + try: (): unknown => (typeof source === 'function' ? source(context) : source), + catch: (cause) => + new SubjectTreeError({ path, message: 'Subject factory failed', cause }), + }) + : (yield* programDeclaration({ + root, + file, + path, + id, + // A program definition's intent is the one `Fractal.mission(intent, program)` checked. + intent: source as ProgramIntent, undefined>, + programs, + })).declaration if (kind === 'objective') return yield* loadObjective({ root, path, id, intent, exports }) // Definitions take their ID from the path; an explicit `id` in the intent must still agree. const identified = @@ -335,13 +464,15 @@ const loadModule = Effect.fnUntraced(function* ({ message: `Declared id ${String(identified.id)} must equal the subject directory ${id}`, }) } - const invalid = (cause: unknown) => - new SubjectTreeError({ path, message: `Invalid ${kind} declaration`, cause }) - const lowered = yield* lowerReferences({ root, path, intent: identified }) if (kind === 'agent') { + const lowered = yield* lowerReferences({ root, path, intent: identified }) const agent = yield* Schema.decodeUnknownEffect(AgentSchema, { onExcessProperty: 'error' })( lowered, - ).pipe(Effect.mapError(invalid)) + ).pipe( + Effect.mapError( + (cause) => new SubjectTreeError({ path, message: `Invalid ${kind} declaration`, cause }), + ), + ) const subject: Subject = { kind, id, @@ -350,15 +481,7 @@ const loadModule = Effect.fnUntraced(function* ({ } return subject } - const wire = yield* Effect.try({ - // Trusted module authoring is validated by missionWire's strict schemas; the - // boundary assertion preserves templates/handles until assembly, not a wire cast. - try: () => missionWire(lowered as MissionInput), - catch: invalid, - }) - const mission = yield* Schema.decodeEffect(MissionSchema, { onExcessProperty: 'error' })( - wire, - ).pipe(Effect.mapError(invalid)) + const mission = yield* decodeMission({ root, path, kind, intent: identified }) if ((mission.schedule !== undefined) !== (kind === 'schedule')) { return yield* new SubjectTreeError({ path, message: scheduleMismatch[kind] }) } @@ -371,6 +494,76 @@ const loadModule = Effect.fnUntraced(function* ({ return subject }) +/** + * Bundles one program mission (`Fractal.mission(intent, program)` from `fractal/effect`) and + * renders its native declaration exactly as `loadSubjectTree` loads that subject, for a loader + * that walks its own tree. `file` is a `mission.ts` below the tree `root` and `definition` its + * default export; the subject ID is the file's directory below `root`. The bundle is built from + * `root` and written to `bundleDir` as `.mjs`. + */ +export const programMission = Effect.fn('fractal.programMission')(function* ({ + root, + file, + definition, + bundleDir, + bun = 'bun', +}: { + readonly root: string + readonly file: string + readonly definition: unknown + readonly bundleDir: string + readonly bun?: string +}) { + if (isAbsolute(bundleDir) === false) { + return yield* new SubjectTreeError({ + path: bundleDir, + message: 'bundleDir must be an absolute directory', + }) + } + const fs = yield* FileSystem.FileSystem + const real = (target: string, message: string) => + fs + .realPath(target) + .pipe(Effect.mapError((cause) => new SubjectTreeError({ path: target, message, cause }))) + const tree = yield* real(root, 'Cannot read subject tree') + const module = yield* real(file, 'Cannot resolve declaration') + const path = relative(tree, module).split(sep).join('/') + if (basename(module) !== 'mission.ts') { + return yield* new SubjectTreeError({ path, message: 'A program mission is a mission.ts' }) + } + if ( + typeof definition !== 'object' || + definition === null || + ProgramEntry in definition === false || + 'intent' in definition === false || + 'url' in definition === false || + typeof definition.url !== 'string' + ) { + return yield* new SubjectTreeError({ + path, + message: 'The default export is not a program mission', + }) + } + const defined = yield* real(fileURLToPath(definition.url), 'Cannot resolve declaration') + if (defined !== module) { + return yield* new SubjectTreeError({ + path, + message: `Module must export its own mission definition, got one from ${definition.url}`, + }) + } + const id = yield* subjectId({ root: tree, file: module, path }) + const { declaration, bundle, sha256 } = yield* programDeclaration({ + root: tree, + file: module, + path, + id, + intent: definition.intent as ProgramIntent, undefined>, + programs: { bundleDir, bun }, + }) + const intent = yield* decodeMission({ root: tree, path, kind: 'mission', intent: declaration }) + return { kdl: emit([missionNode(intent as MissionInput)]), bundle, sha256 } +}) + /** * Checks an objective module's intent: its `name`, if given, is the path ID; `onBreach` agent * references resolve to `agent/`; and it is an interval over claims named `NAMESPACE/NAME`, @@ -494,14 +687,30 @@ const loadKdl = Effect.fnUntraced(function* ({ * a factory; factories receive `context`. Typed references are lowered, then intents are decoded * with `AgentSchema` or `MissionSchema`, or checked by `Fractal.objective`. A declared objective is an * interval over claims. Subject modules are trusted code and are imported. + * + * A program mission (`Fractal.mission(intent, program)` from `fractal/effect`) is bundled with + * `bun build --target=bun` from the tree root into `bundleDir`, and loads as the mission that runs + * it: one text input per field of `intent.input` and one `program` step pinning the bundle's sha256. */ export const loadSubjectTree = Effect.fn('fractal.loadSubjectTree')(function* ({ root, context, + bundleDir, + bun = 'bun', }: { readonly root: string readonly context: C + /** Absolute directory program bundles are written to as `.mjs`. */ + readonly bundleDir?: string + /** The Bun that builds program bundles; default `bun` on PATH. */ + readonly bun?: string }) { + if (bundleDir !== undefined && isAbsolute(bundleDir) === false) { + return yield* new SubjectTreeError({ + path: bundleDir, + message: 'bundleDir must be an absolute directory', + }) + } const fs = yield* FileSystem.FileSystem const tree = yield* fs.realPath(root).pipe( Effect.flatMap((real) => @@ -513,6 +722,7 @@ export const loadSubjectTree = Effect.fn('fractal.loadSubjectTree')(function* new SubjectTreeError({ path: root, message: 'Cannot read subject tree', cause }), ), ) + const programs: ProgramOptions = { bundleDir, bun } const seen = new Set() const loaded = new Set() const subjects: Subject[] = [] @@ -544,7 +754,7 @@ export const loadSubjectTree = Effect.fn('fractal.loadSubjectTree')(function* (a.path < b.path ? -1 : a.path > b.path ? 1 : 0)) diff --git a/release-notes/program-mission-loader.json b/release-notes/program-mission-loader.json new file mode 100644 index 00000000000..07403e27edf --- /dev/null +++ b/release-notes/program-mission-loader.json @@ -0,0 +1,11 @@ +{ + "version": 1, + "summary": "The private `fractal` package's tree loader publishes program missions: `loadSubjectTree({ root, bundleDir })` bundles each `Fractal.mission(intent, program)` with `bun build --target=bun` into `/.mjs`, and the mission renders as KDL with its text inputs, `concurrent-runs`, run `authority` block and one `program` step pinning the bundle", + "replay": {"status": "none", "detail": "No daemon, claim or projection change. The rendered declaration uses the `program` step and `authority` grammar the daemon parses."}, + "schema": {"status": "none", "detail": "No table, index, migration or claim kind change."}, + "rules": {"status": "none", "detail": "No rule change. The authoring schemas gain the mission `authority` block (publish, start, revise and write rules); what it grants is enforced by the daemon."}, + "compatibility": {"status": "changed", "detail": "Additive authoring surface: `StepSchema.program`, `MissionSchema.authority` and `MissionSchema.concurrentRuns`, and the `bundleDir` and `bun` options of `loadSubjectTree`. A tree that contains a program mission now needs `bundleDir` and a `bun` on PATH; trees without program missions load and render exactly as before. Rendered `program` steps and `authority` blocks need a daemon whose parser knows them."}, + "downtime": {"status": "none", "detail": "No deploy-time effect; the package is private and not published to npm."}, + "manual": {"status": "none", "detail": "No operator action is required."}, + "recovery": {"status": "none", "detail": "Bundles are content-addressed files in the caller's bundle directory; rebuilding the same tree writes the same file."} +}