Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
56 changes: 56 additions & 0 deletions .changeset/flow-function-declared-effect.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
---
"@objectstack/spec": minor
"@objectstack/objectql": minor
"@objectstack/runtime": minor
"@objectstack/service-automation": minor
"@objectstack/cli": minor
---

feat(automation): a `script` node's purity contract is declared, and a function that writes can say so (#4396)

The `script` executor's contract — *the named function returns a value; data I/O
stays on the flow graph* — existed only as a comment inside the executor, while
#4354's run summary depended on it. That summary reports no record metrics for a
`script` step precisely because a pure function's writes are downstream
`create_record` / `update_record` nodes counting themselves. A function that
wrote anyway made its run report `selected: 30, acted: 0` — indistinguishable
from the broken sweep the counters exist to detect, recorded permanently on
`sys_automation_run`.

**The rule is now visible.** `ActionDescriptor` carries
`handlerContract: 'none' | 'pure'`, and the `script` descriptor publishes
`'pure'`, so the action catalog, the designer palette and the reference docs
state the rule an author has to follow instead of an executor holding it
privately.

**And a legitimate writer can opt out honestly.** A `defineStack({ functions })`
entry may declare what it does, in either shape:

```ts
defineStack({
functions: {
scoreLead: (ctx) => ({ score: 42 }), // pure — the default
syncBilling: { handler: syncBilling, effect: 'writes' }, // declared writer
},
});
```

A step calling a declared writer reports `unmeasuredEffect`, so the run's
`unmeasured` tally keeps the broken-sweep query
(`selected > 0 AND acted = 0 AND unmeasured = 0`) off that flow — and only that
flow. Marking *every* `script` step unmeasured was rejected: it would blind the
detector on every flow that calls any function in order to cover the few that
break the rule.

Nothing here is retired or renamed: a bare `functions: { fn }` entry is
unchanged and means `effect: 'pure'`. The declaration is carried end to end —
`ObjectQL.registerFunction` accepts `{ packageId, effect }` alongside the
existing `packageId` string and exposes `resolveFunctionEntry(name)`,
`objectstack build` lowers a declared entry without dropping it, and the
artifact loader re-attaches the module's callable to the declaration the JSON
carried.

**Also fixed:** `bindHooksToEngine` returned before registering a bundle's
functions when the stack declared no hooks, so a flow-only app's
`defineStack({ functions })` reached the engine as nothing and every `script`
node calling one failed with "no function named 'x' is registered".
48 changes: 47 additions & 1 deletion content/docs/automation/flows.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -245,6 +245,44 @@ function fails the step loudly rather than passing silently.
}
```

<Callout type="info" title="A flow function is a pure compute step">

It takes its `inputs`, **returns** a value, and a later **declarative** node
persists that value. Data I/O stays on the flow graph — the function itself does
no writes, and the runtime hands it nothing to write with (`ctx` carries
`input` / `variables` / `automation` / `logger`, no data engine). The `script`
action publishes this as `handlerContract: 'pure'` on its
[action descriptor](/docs/references/automation/node-executor).

The rule is load-bearing, not stylistic. [Run summaries](#run-summaries) report
what a run did to the data, and a `script` step reports **no** record metrics
precisely because of it: every write a pure function causes is a downstream
`create_record` / `update_record` that counts itself. A function that writes
behind the platform's back makes its run claim it wrote nothing — the durable
`sys_automation_run` row then reads exactly like a broken sweep, permanently.

If a function genuinely must write (an upstream billing system, a legacy API),
**declare it** so the run stays honest:

```typescript
defineStack({
functions: {
scoreLead: (ctx) => ({ score: 42 }), // pure — the default
syncBilling: { handler: syncBilling, effect: 'writes' }, // declared writer
},
});
```

A step that calls a declared writer is counted as an effect the platform cannot
measure (`unmeasured`), never as zero — so the broken-sweep query
`selected > 0 AND acted = 0 AND unmeasured = 0` stops firing on that flow, and
keeps working on every other flow that calls a function. Declaring changes what
is *reported*, not what is *allowed*: an undeclared writer is still counted as
having written nothing, and no runtime check can catch it — a function is
ordinary host code and may close over a client at module scope.

</Callout>

**Screen (flat fields):**

The default shape. Each field is collected as a **bare flow variable**, so a
Expand Down Expand Up @@ -623,7 +661,15 @@ instead:
| `http`, mutating method, rejected / timed out | `unmeasured` — a 500 can arrive after the write landed |
| `http`, `durable: true` | `acted: 1` — the outbox row is a real, durable effect |
| `connector_action` | `unmeasured` |
| `script` | nothing — a registered function is **contractually pure**: data I/O stays on the flow graph, so every write it causes is a downstream node that counts itself |
| `script`, function declared pure (the default) | nothing — a registered function is **contractually pure**: data I/O stays on the flow graph, so every write it causes is a downstream node that counts itself |
| `script`, function declared `effect: 'writes'` | `unmeasured` — the function said it writes where the platform cannot see, so the run says the count is incomplete |

The `script` row is a contract, not a measurement: nothing stops a registered
function from writing, so an **undeclared** writer still makes its run report
`acted: 0`. That is why the declaration exists and why it is worth using — see
[the purity callout](#node-examples). It is also why the reverse fix was
rejected: marking *every* `script` step `unmeasured` would blind the detector on
every flow that calls any function, to cover the few that break the rule.

`unmeasured` propagates through `subflow` and `map` roll-ups, so a parent whose
child dispatched an uncountable effect knows its own `acted` is incomplete.
Expand Down
109 changes: 109 additions & 0 deletions content/docs/references/automation/flow-function.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,109 @@
---
title: Flow Function
description: Flow Function protocol schemas
---

{/* ⚠️ AUTO-GENERATED — DO NOT EDIT. Run build-docs.ts to regenerate. Hand-written docs live in the module folders under content/docs/. */}

@module automation/flow-function

The contract for a **named handler function a `script` node invokes** —

contributed by `defineStack(\{ functions \})` and resolved by name at execute

time (#1870).

## The rule, and why it lives here instead of in a comment

A flow function is a PURE compute step: it receives its mapped `input`,

RETURNS a value, and the node's `outputVariable` exposes that value as a flow

variable so a later DECLARATIVE node persists it (`update_record fields: \{

ai_category: '\{aiResult.ai_category\}' \}`). Data I/O stays on the flow graph.

That is not style advice. #4354's per-run summary reports what a run did to

the data, and the `script` node reports NO record metrics *because* of this

rule: every write a script causes is a downstream `create_record` /

`update_record` that counts itself, so "this node touched no records" is the

accurate answer rather than a guess. A function that writes anyway makes its

run under-report — `selected: 30, acted: 0` on a run that wrote 30 invoices,

which reads exactly like the broken sweep #4354 exists to detect, and the

durable `sys_automation_run` row says so permanently.

Until #4396 that rule lived ONLY in a comment inside the executor, so neither

an author, a lint, nor the runtime could see the contract the summary was

relying on. It is now declared in two halves:

1. `ActionDescriptor.handlerContract` — `script` publishes `'pure'`, so the

action catalog and the designer palette carry the rule an author reads.

2. `FlowFunctionEffectSchema` — a function that legitimately writes

DECLARES it, and its step then reports `unmeasuredEffect`, so the run

says "cannot count" instead of claiming it wrote nothing.

## What is deliberately not here

A blanket `unmeasuredEffect` on every `script` step (the escape hatch #4354

gave `connector_action`) was rejected: it would suppress the broken-sweep

signal on every flow that calls any function, in order to accommodate the

flows that break the rule — paying for a rule-breaker with everyone else's

signal, and fossilizing the violation as supported behaviour.

Nor is this *enforcement*. The runtime hands a function no data reach —

`FlowFunctionContext` in `@objectstack/service-automation` carries

`input` / `variables` / `automation` / `logger` and no engine handle — but a

function is ordinary host code and can close over a client at module scope.

What the declaration buys is that the honest case is now *expressible*, and

the platform's own counters stop being wrong for it.

<Callout type="info">
**Source:** `packages/spec/src/automation/flow-function.zod.ts`
</Callout>

## TypeScript Usage

```typescript
import { FlowFunctionEffect } from '@objectstack/spec/automation';
import type { FlowFunctionEffect } from '@objectstack/spec/automation';

// Validate data
const result = FlowFunctionEffect.parse(data);
```

---

## FlowFunctionEffect

What a script-node function does to data: 'pure' (computes and returns — the contract) or 'writes' (performs uncountable writes/effects, reported as unmeasured)

### Allowed Values

* `pure`
* `writes`


---

1 change: 1 addition & 0 deletions content/docs/references/automation/index.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ This section contains all protocol schemas for the automation layer of ObjectSta
<Card href="/docs/references/automation/etl" title="Etl" description="Source: packages/spec/src/automation/etl.zod.ts" />
<Card href="/docs/references/automation/execution" title="Execution" description="Source: packages/spec/src/automation/execution.zod.ts" />
<Card href="/docs/references/automation/flow" title="Flow" description="Source: packages/spec/src/automation/flow.zod.ts" />
<Card href="/docs/references/automation/flow-function" title="Flow Function" description="Source: packages/spec/src/automation/flow-function.zod.ts" />
<Card href="/docs/references/automation/io-node-config" title="Io Node Config" description="Source: packages/spec/src/automation/io-node-config.zod.ts" />
<Card href="/docs/references/automation/node-executor" title="Node Executor" description="Source: packages/spec/src/automation/node-executor.zod.ts" />
<Card href="/docs/references/automation/schemaless-node-config" title="Schemaless Node Config" description="Source: packages/spec/src/automation/schemaless-node-config.zod.ts" />
Expand Down
1 change: 1 addition & 0 deletions content/docs/references/automation/meta.json
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@
"job",
"---More---",
"builtin-node-config",
"flow-function",
"io-node-config",
"schemaless-node-config"
]
Expand Down
1 change: 1 addition & 0 deletions content/docs/references/automation/node-executor.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -76,6 +76,7 @@ Canonical cross-paradigm action/node descriptor (ADR-0018)
| **supportsRetry** | `boolean` | ✅ | Supports retry on failure |
| **needsOutbox** | `boolean` | ✅ | Dispatch via service-messaging outbox (retry/idempotency/dead-letter) |
| **isAsync** | `boolean` | ✅ | Suspends the flow awaiting an external reply |
| **handlerContract** | `Enum<'none' \| 'pure'>` | ✅ | Effect contract for author-supplied code this action invokes: 'none' (invokes none) or 'pure' (must not write — it returns a value and the flow graph persists it) |
| **resumeAuthority** | `Enum<'any' \| 'service'>` | ✅ | Who may resume a run this node suspended: 'any' (the generic resume route) or 'service' (only the owning service, e.g. approvals) |
| **maturity** | `Enum<'ga' \| 'beta' \| 'reserved'>` | ✅ | Runtime maturity: ga (shipped), beta, or reserved (contract only — designers grey this out) |
| **source** | `Enum<'builtin' \| 'plugin'>` | ✅ | builtin = platform baseline; plugin = third-party contributed |
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -145,7 +145,7 @@ const result = DecisionCondition.parse(data);
| Property | Type | Required | Description |
| :--- | :--- | :--- | :--- |
| **actionType** | `string` | optional | How this step runs: a built-in side effect ('email' \| 'slack'), the 'invoke_function' marker, or shorthand for a registered-function name |
| **function** | `string` | optional | Registered function to call (defineStack(`{ functions }`)); takes precedence over actionType |
| **function** | `string` | optional | Registered function to call (defineStack(`{ functions }`)); takes precedence over actionType. Contractually pure — it returns a value a later declarative node persists |
| **inputs** | `Record<string, any>` | optional | Inputs passed to the function (values interpolate `{token}` templates) |
| **outputVariable** | `string` | optional | Flow variable the function's return value is bound to |
| **template** | `string` | optional | Built-in side effects only: message template id |
Expand Down
3 changes: 2 additions & 1 deletion docs/audits/2026-07-unknown-key-strictness-ledger.md
Original file line number Diff line number Diff line change
Expand Up @@ -165,7 +165,7 @@ tightening (the #4001 "sharing-rule lesson": candidates, not verdicts).
| `external-catalog.zod.ts` | 4 | wire (p) | |
| `field-value.zod.ts` / `seed.zod.ts` / `validation.zod.ts` | 1 ea | mixed (p) | |

### `automation/` — 98 sites
### `automation/` — 99 sites

| File | Sites | Class | Note |
|---|---|---|---|
Expand All @@ -182,6 +182,7 @@ tightening (the #4001 "sharing-rule lesson": candidates, not verdicts).
| `builtin-node-config.zod.ts` | 8 | authorable | Same family (#4045): the CRUD quartet, `screen`, `map`. Written from what the executors read rather than from the descriptors' `configSchema` literals, and reconciled bidirectionally by `builtin-node-form-zod-ledger.test.ts` — so unlike most rows here, this one already has a drift check of its own. Same candidacy note as `io-node-config` |
| `schemaless-node-config.zod.ts` | 4 | authorable | Same family, third panel (#4278): `script` / `subflow` / `decision` (+ the decision branch item) — the descriptor-schemaless nodes whose form lives in objectui's hand-written table. Written from the executors; the drift check is objectui's `flow-node-config.spec-reconciliation` test (cross-repo, via the published exports). Contract exports only — nothing parses node config with them yet, so strictness candidacy follows `io-node-config` |
| `webhook.zod.ts` | 1 | authorable (p) | spec-only (#3461) |
| `flow-function.zod.ts` | 1 | authorable | `FlowFunctionDeclarationSchema` (#4396) — the `{ handler, effect }` form of a `defineStack({ functions })` entry. Authored, but note what an undeclared key here would be: a sibling of a **live function**, not data. `defineStack`'s union already rejects a record whose `handler` is not callable, and the boot-path reader is the hand-written `normalizeFlowFunctionEntry` rather than a `.parse()` (re-validating a live handler every boot buys nothing), so strictness would bind at authoring only. Candidate on the same verify-first rule as its `*-node-config` neighbours |

### `security/` — 20 sites

Expand Down
7 changes: 5 additions & 2 deletions packages/cli/src/utils/build-runtime.ts
Original file line number Diff line number Diff line change
Expand Up @@ -76,10 +76,13 @@ function collect(cfg) {
}
}
}
// Top-level functions map
// Top-level functions map — a value is the handler, or a declaration
// record stating its effect (\`{ handler, effect: 'writes' }\`).
if (cfg.functions && !Array.isArray(cfg.functions) && typeof cfg.functions === 'object') {
for (const [k, v] of Object.entries(cfg.functions)) {
if (typeof v === 'function' && REFS.has(k)) out[k] = v;
if (!REFS.has(k)) continue;
if (typeof v === 'function') out[k] = v;
else if (v && typeof v === 'object' && typeof v.handler === 'function') out[k] = v.handler;
}
}
// Top-level functions array
Expand Down
37 changes: 37 additions & 0 deletions packages/cli/src/utils/lower-callables.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -87,3 +87,40 @@ describe('lowerCallables — only `target` binds a handler (#3855)', () => {
expect(result.functions[ref]()).toBe('preferred');
});
});

// ── #4396: a function's DECLARATION survives lowering ───────────────────────
//
// `functions: { syncBilling: { handler, effect: 'writes' } }` is how a function
// that legitimately writes keeps its run's metrics honest. Lowering is the
// first place a built artifact could lose that: the map branch bound only bare
// callables, so a declared entry was dropped entirely — the function then went
// missing from `objectstack.json` AND from the runtime bundle, and every
// `script` node calling it failed on a built deployment while working from
// source.
describe('lowerCallables — declared `functions` entries (#4396)', () => {
const functionsOf = (result: { lowered: Record<string, unknown> }) =>
(result.lowered as { functions: Record<string, unknown> }).functions;

it('lowers a bare handler to its ref, unchanged', () => {
const result = lowerCallables({ functions: { scoreLead: () => 'scored' } });
expect(functionsOf(result).scoreLead).toBe('scoreLead');
expect((result.functions.scoreLead as () => string)()).toBe('scored');
});

it('lowers a declared entry and keeps what it declared', () => {
const result = lowerCallables({
functions: { syncBilling: { handler: () => 'synced', effect: 'writes' } },
});
expect(functionsOf(result).syncBilling).toEqual({ handler: 'syncBilling', effect: 'writes' });
expect((result.functions.syncBilling as () => string)()).toBe('synced');
expect(result.count).toBe(1);
});

it('keeps `effect` on the array form too', () => {
const result = lowerCallables({
functions: [{ name: 'syncBilling', handler: () => 'synced', effect: 'writes' }],
});
const [entry] = functionsOf(result) as unknown as Array<Record<string, unknown>>;
expect(entry).toEqual({ name: 'syncBilling', handler: 'syncBilling', effect: 'writes' });
});
});
12 changes: 11 additions & 1 deletion packages/cli/src/utils/lower-callables.ts
Original file line number Diff line number Diff line change
Expand Up @@ -148,13 +148,23 @@ export function lowerCallables(input: Record<string, unknown>): LoweringResult {
});
(lowered as Record<string, unknown>).functions = arr;
} else if (isPlainObject(fnsField)) {
const out: Record<string, string> = {};
const out: Record<string, unknown> = {};
for (const [key, value] of Object.entries(fnsField)) {
if (typeof value === 'function') {
const ref = uniqueName(key, taken);
taken.add(ref);
functions[ref] = value as AnyFn;
out[ref] = ref;
} else if (isPlainObject(value) && typeof value.handler === 'function') {
// A DECLARED entry (`{ handler, effect: 'writes' }`, #4396). Lower the
// callable exactly like the bare form and keep the declaration beside
// it, so what the function said about itself survives into the
// artifact — dropping it here would silently un-declare the function
// on every built deployment while it kept working from source.
const ref = uniqueName(key, taken);
taken.add(ref);
functions[ref] = value.handler as AnyFn;
out[ref] = { ...value, handler: ref };
}
}
// Preserve any pre-existing string entries (legacy bundles).
Expand Down
Loading
Loading