diff --git a/.changeset/docs-accuracy-audit-4212-scope.md b/.changeset/docs-accuracy-audit-4212-scope.md
new file mode 100644
index 0000000000..9ee3f8ad55
--- /dev/null
+++ b/.changeset/docs-accuracy-audit-4212-scope.md
@@ -0,0 +1,6 @@
+---
+---
+
+docs: implementation-accuracy audit of the #4212-family affected docs. Fixes
+fabricated APIs, wrong manifest filenames, unresolvable version pins and
+credential-ref formats. Releases nothing.
diff --git a/content/docs/concepts/metadata-lifecycle.mdx b/content/docs/concepts/metadata-lifecycle.mdx
index 4bcadaedce..abb5deab70 100644
--- a/content/docs/concepts/metadata-lifecycle.mdx
+++ b/content/docs/concepts/metadata-lifecycle.mdx
@@ -8,7 +8,7 @@ description: How metadata flows through Repository → Change Log → Cache →
This page documents the metadata data path introduced by [ADR-0008](https://github.com/objectstack-ai/objectstack/blob/main/docs/adr/0008-metadata-repository-and-change-log.md) and refined by [ADR-0005](https://github.com/objectstack-ai/objectstack/blob/main/docs/adr/0005-metadata-customization-overlay.md). It is the canonical event stream that powers Studio Hot Module Replacement (HMR), REST writes, and future cloud editing.
-**TL;DR** — Every write goes through a single `MetadataRepository.put()` call. The repository appends to a change log, emits a watch event with a monotonic `seq`, and broadcasts over SSE to the Studio UI. The Studio's `useMetadataHmr` hook displays the latest `seq` in the status badge and re-fetches affected views.
+**TL;DR** — Every write goes through a single `MetadataRepository.put()` call. The repository appends to a change log, emits a watch event with a monotonic `seq`, and broadcasts over SSE to the console UI. In dev, the console's `MetadataHmrReloader` subscribes to that stream and triggers a debounced full page reload.
---
@@ -17,8 +17,8 @@ This page documents the metadata data path introduced by [ADR-0008](https://gith
```
┌─────────────────────────────────────────────┐
- │ Studio UI │
- │ (useMetadataHmr / HmrStatusBadge) │
+ │ Console UI (incl. Studio) │
+ │ (MetadataHmrReloader) │
└────────────────────┬────────────────────────┘
│ SSE /api/v1/dev/metadata-events
▼
@@ -29,7 +29,7 @@ This page documents the metadata data path introduced by [ADR-0008](https://gith
│ ChangeEvent{kind, metadataType, name, seq?}
▼
┌─────────────────────────────────────────────┐
- │ LayeredRepository │
+ │ Overlay precedence (protocol.ts) │
│ ┌─────────────────────────────────────┐ │
│ │ Top: SysMetadataRepository (org) │ │ ← writable overlay
│ │ Bottom: FS / InMemory (artifact) │ │ ← read-only baseline
@@ -43,15 +43,26 @@ This page documents the metadata data path introduced by [ADR-0008](https://gith
Reads walk top-to-bottom: the first non-null layer wins. Writes always route to the topmost writable layer (the overlay). Deletes from the layered API only remove the overlay row — the artifact baseline survives.
+
+Today this precedence is implemented **inside the protocol layer**, not by
+`LayeredRepository`. `ObjectStackProtocolImplementation` lazily builds one
+`SysMetadataRepository` per org (`getOverlayRepo()`) and merges it against the
+in-memory code registry — `getMetaItemLayered()` returns `code` / `overlay` /
+`effective` separately. `LayeredRepository` ships in `@objectstack/metadata-core`
+and implements exactly the semantics above, but it is **not yet composed** into
+any production path: every `new LayeredRepository(...)` call site is a test
+(`metadata-core`'s own suite plus one `objectql` integration test).
+
+
---
## The four primitives
| Primitive | Package | Purpose |
| :--- | :--- | :--- |
-| **Repository** | `@objectstack/metadata-core` | CRUD + watch interface over a single metadata source. `InMemoryRepository` and `LayeredRepository` ship from `@objectstack/metadata-core`; `FileSystemRepository` from `@objectstack/metadata-fs`; `SysMetadataRepository` from `@objectstack/metadata-protocol` (re-exported by `@objectstack/objectql` for back-compat, per [ADR-0076](https://github.com/objectstack-ai/objectstack/blob/main/docs/adr/0076-objectql-core-tiering.md)). |
+| **Repository** | `@objectstack/metadata-core` | CRUD + watch interface over a single metadata source. `InMemoryRepository` and `LayeredRepository` ship from `@objectstack/metadata-core`; `FileSystemRepository` from `@objectstack/metadata-fs`; `SysMetadataRepository` from `@objectstack/metadata-protocol` (relocated out of `@objectstack/objectql` by [ADR-0076](https://github.com/objectstack-ai/objectstack/blob/main/docs/adr/0076-objectql-core-tiering.md) — import it from `@objectstack/metadata-protocol`; `@objectstack/objectql` no longer re-exports it). |
| **Change Log** | `@objectstack/metadata-core` | Append-only log of every mutation, tagged with a monotonic `seq`. Watchers can replay from any `since`. |
-| **Cache** | `@objectstack/metadata-core` | In-memory snapshot of the registry, keyed by `MetaRef`. Invalidated by change-log events. |
+| **Cache** | `@objectstack/metadata-core` | `MetadataCache` — a **bounded LRU** in front of a repository, keyed by `refKey(ref)` and capped by `maxEntries` / `maxBytes`. Lazily filled on read miss (no bulk preload), invalidated by `repo.watch()` events, and negative-caches misses. |
| **Registry** | *(per consumer)* | Not a single package — each consumer owns a typed projection over the cache (e.g. `SchemaRegistry` in `@objectstack/objectql`), per [ADR-0008 §2.9](https://github.com/objectstack-ai/objectstack/blob/main/docs/adr/0008-metadata-repository-and-change-log.md#29-registry-layer). |
`MetaRef = (type, name, org)`. As of [ADR-0008 §0 amendment (2026-04-13)](https://github.com/objectstack-ai/objectstack/blob/main/docs/adr/0008-metadata-repository-and-change-log.md#0-2026-04-13-amendment--drop-project-and-branch-from-metaref), `project` and `branch` are removed from the runtime tuple. Project survives only as an artifact-packaging concept on the `objectstack.json` envelope; branching is left to Git.
@@ -63,16 +74,16 @@ Reads walk top-to-bottom: the first non-null layer wins. Writes always route to
When a user edits a view in Studio:
1. Studio calls `PUT /api/v1/meta/view/case_grid` (REST).
-2. `protocol.ts:saveMetaItem()` validates against `MetadataTypeRegistryEntry.allowOrgOverride` (see [overlay whitelist](#overlay-whitelist)).
+2. `protocol.ts:saveMetaItem()` runs the two-tier gate (see [overlay whitelist](#overlay-whitelist)): an item that already exists as a packaged artifact requires `MetadataTypeRegistryEntry.allowOrgOverride`; a brand-new item requires `allowRuntimeCreate` **or** `allowOrgOverride`.
3. If allowed, the call lands on `MetadataRepository.put(ref, body, { parentVersion, actor })`.
4. The repository:
- Verifies `parentVersion` matches the current head (`ConflictError` on mismatch).
- Writes the new row + appends a change-log entry with the next `seq`.
- - Emits a `MetadataEvent { op: 'create' | 'update' | 'delete', ref, seq, source }`.
+ - Emits a `MetadataEvent { op, ref, hash, parentHash, actor, seq, ts, source }`, where `op` is one of `create | update | delete | rename | publish | revert`.
5. The `MetadataManager` bridge forwards the event over the SSE channel `/api/v1/dev/metadata-events`. The wire payload is **not** the internal `MetadataEvent` — it is a `ChangeEvent { kind: 'metadata-change', type, metadataType, name, path?, timestamp, seq? }` (emitted as `event: metadata-change`). `seq` is the canonical repository sequence and is **absent** for FS-watcher (chokidar) dev events.
-6. The Studio's `useMetadataHmr` hook receives the event, updates `lastSeq`, and triggers a refetch of the affected view.
+6. In dev, the console's `MetadataHmrReloader` receives the `metadata-change` event and schedules a debounced `location.reload()` (default `debounceMs: 400`, plus a 150 ms toast delay).
-The `seq` is the single source of truth for ordering. The Studio status badge (`HmrStatusBadge`) shows `Repo seq: #N` in its tooltip.
+The `seq` is the single source of truth for ordering on the server. Note that the shipped console client does **not** read `seq` — it reloads the whole page rather than invalidating per-item caches. There is no granular-refetch hook or seq status badge in the current UI.
---
@@ -100,10 +111,15 @@ In shared-database multi-tenancy, **most metadata types must not be per-org cust
| `flow` | ✅ | Per-org overlays are allowed for automation definitions. |
| `agent` | ❌ | Agents are platform-owned and closed to third parties (ADR-0063 §2) — no per-org agent fork. |
| `permission`, `position` | ✅ | Per-org overlays are allowed; tenant-level controls layer on top. |
-| `object`, `field` | ❌ | Defines the table schema. Overriding would break existing data. |
-| `datasource` | ❌ | Connection strings; multi-tenant isolation is enforced at a higher layer. |
+| `object`, `field` | ❌ | Defines the table schema. Overriding a packaged object/field would break existing data — but both set `allowRuntimeCreate: true`, so tenants *can* author brand-new objects and fields. |
+| `datasource` | ❌ | Connection strings; multi-tenant isolation is enforced at a higher layer. (`allowRuntimeCreate: true` — the datasource wizard persists `origin: 'runtime'` rows.) |
+
+There is no `workflow` metadata type (per [ADR-0020](https://github.com/objectstack-ai/objectstack/blob/main/docs/adr/0020-state-machine-converge-and-enforce.md), record state machines are a `state_machine` validation). The runtime gate is implemented in `OVERLAY_ALLOWED_TYPES` (derived from the registry) and enforced by `SysMetadataRepository.put()`.
+
+The gate is **two-tier** — `allowOrgOverride: false` is not the same as "no runtime writes":
-There is no `workflow` metadata type (per [ADR-0020](https://github.com/objectstack-ai/objectstack/blob/main/docs/adr/0020-state-machine-converge-and-enforce.md), record state machines are a `state_machine` validation). The runtime gate is implemented in `OVERLAY_ALLOWED_TYPES` (derived from the registry) and enforced by `SysMetadataRepository.put()`. Denied types return `403 not_overridable`.
+- Overwriting an item that ships from a code package requires `allowOrgOverride`. Denied → `403 not_overridable`.
+- Creating a brand-new item that has no artifact backing requires `allowRuntimeCreate` (or `allowOrgOverride`). Denied → `403 not_creatable`.
See [ADR-0005](https://github.com/objectstack-ai/objectstack/blob/main/docs/adr/0005-metadata-customization-overlay.md) for the full design and amendments.
@@ -111,7 +127,7 @@ See [ADR-0005](https://github.com/objectstack-ai/objectstack/blob/main/docs/adr/
## Conflict handling
-Every `put()` requires a `parentVersion` (or `null` for fresh creates). If the head has moved since the caller read it, `ConflictError(ref, expectedParent, actualHead)` is thrown. Studio surfaces this as a "metadata moved" toast and offers reload-and-retry.
+Every `put()` requires a `parentVersion` (or `null` for fresh creates). If the head has moved since the caller read it, the repository throws `ConflictError(ref, expectedParent, actualHead)` with code `METADATA_CONFLICT`. `protocol.ts` catches it at every write entry point (save, publish, rollback, overlay-delete) and re-throws an API error with code `metadata_conflict`, **HTTP 409 Conflict**, and the `expectedParent` / `actualHead` pair attached. Clients are expected to re-read the item and retry; the console does not yet surface a dedicated conflict-resolution UI.
The hash is `sha256:` + 64-hex of a canonical (sorted-keys, no-undefined) JSON serialization of the body. Identical bodies produce identical hashes — `put()` short-circuits on no-op writes.
@@ -119,9 +135,9 @@ The hash is `sha256:` + 64-hex of a canonical (sorted-keys, no-undefined) JSON s
## HMR end-to-end (latency & ordering)
-- **Latency.** A typical write-to-render round-trip is **< 100ms** on localhost (REST `PUT` → DB write → SSE flush → Studio refetch → React re-render).
-- **Ordering.** The `seq` is monotonic per repository. The repository-level `watch()` API supports replay from a `since` cursor, but the dev HMR SSE endpoint does **not** replay on reconnect — it registers a fresh listener, emits a `ready` event, and then streams only live events. A tab that disconnects and reconnects will miss any events that occurred while it was offline; it should refetch the affected metadata on reconnect.
-- **Multi-tab.** Each tab has its own `seq` counter. Out-of-order delivery between tabs is impossible because they share the server change log.
+- **Latency.** The server half (REST `PUT` → DB write → SSE flush) is fast on localhost, but the client half dominates: `MetadataHmrReloader` debounces for 400 ms, waits another 150 ms so its toast can paint, and then does a **full page reload**. Budget roughly half a second plus a cold React mount, not a sub-100 ms in-place re-render.
+- **Ordering.** The `seq` is monotonic per repository. The repository-level `watch()` API supports replay from a `since` cursor, but the dev HMR SSE endpoint does **not** replay on reconnect — it registers a fresh listener, emits a `ready` event, and then streams only live events. A tab that disconnects and reconnects will miss any events that occurred while it was offline; it should refetch the affected metadata on reconnect. (The stream also emits `: ping` heartbeat comments every 15 s.)
+- **Multi-tab.** Every open tab registers its own listener on the same in-process broadcast hub, so all tabs receive the same events in the same order. No tab tracks `seq` client-side — the shipped reloader ignores it entirely.
---
@@ -143,19 +159,28 @@ The hash is `sha256:` + 64-hex of a canonical (sorted-keys, no-undefined) JSON s
| `FileSystemRepository` | ✅ Shipped (`@objectstack/metadata-fs`) |
| Change log + `seq` (per-org, monotonic) | ✅ Shipped |
| SSE bridge (`/api/v1/dev/metadata-events`, `event: metadata-change`) | ✅ Shipped |
-| Studio `useMetadataHmr` + `HmrStatusBadge` | ✅ Shipped |
-| Console dev-mode HMR reloader (`MetadataHmrReloader`) | ✅ Shipped |
-| `SysMetadataRepository` (overlay over `sys_metadata`) | ✅ Shipped (`@objectstack/metadata-protocol`, re-exported by `@objectstack/objectql`) |
-| `LayeredRepository(SysMeta + artifact)` composition | ✅ Shipped |
-| `protocol.ts:saveMetaItem` routed through `SysMetadataRepository.put` | ✅ Shipped (PR-10d.6, flag removed) |
-| `sys_metadata_history` table (durable, org-keyed change log) | ✅ Shipped (`@objectstack/metadata-protocol`) |
+| Granular client-side HMR hook + seq status badge | ❌ Not implemented — no such hook or badge exists in the console today |
+| Console dev-mode HMR reloader (`MetadataHmrReloader`) | ✅ Shipped (console app, mounted app-wide, dev-only) |
+| `SysMetadataRepository` (overlay over `sys_metadata`) | ✅ Shipped (`@objectstack/metadata-protocol`) |
+| `LayeredRepository(SysMeta + artifact)` composition | ⏳ Not wired — the class ships, but nothing composes it outside tests |
+| `protocol.ts:saveMetaItem` routed through `SysMetadataRepository.put` | ✅ Shipped (via the per-org `getOverlayRepo()` cache) |
+| `sys_metadata_history` table (durable, org-keyed change log) | ✅ Shipped (object definition in `@objectstack/metadata-core`; written by `SysMetadataRepository`) |
| Cache + Registry refactor against `MetadataRepository` | ⏳ Post-M0 |
-> **Cross-replica overlay sync is out of scope.** Single-instance deployments
-> are the supported topology for the metadata overlay. If you run multiple
-> replicas and need them to converge on the same overlay state, point them at
-> the same database and rely on each replica re-reading on demand — there is
-> no `LISTEN/NOTIFY` (or equivalent) push mechanism on the roadmap.
+> **Cross-replica sync happens above the repository, not inside it.**
+> `SysMetadataRepository.watch()` is deliberately scoped to the local instance —
+> it has no `LISTEN/NOTIFY` or pub/sub of its own, and never will. Cross-node
+> propagation is instead a **cluster** concern: `@objectstack/runtime` registers
+> `ClusterServicePlugin` and `MetadataClusterBridgePlugin` by default (opt out with
+> `cluster: false`), and at `kernel:ready` the bridge calls
+> `MetadataManager.attachClusterPubSub()`, fanning every local watch event out over
+> the `metadata.changed` channel (loopback-suppressed by `originNode`) so peer nodes
+> invalidate their caches. Note this replays the *invalidation event*, not the
+> overlay row — peers re-read from the shared database. The default cluster driver
+> is `memory`, which is in-process, so real cross-replica fan-out still requires
+> configuring a distributed driver; and if no `cluster` (or `metadata`) service is
+> registered the bridge logs and skips, leaving each replica seeing only its own
+> writes.
---
@@ -167,7 +192,7 @@ The hash is `sha256:` + 64-hex of a canonical (sorted-keys, no-undefined) JSON s
|:---|:---|:---|
| `defineView(...)` / `defineFlow(...)` / any source file → compiled into `dist/objectstack.json` | ❌ Never. Loaded into the in-memory registry on boot; refreshed via HMR in dev. | ❌ The artifact's own version history *is* Git. The metadata layer does not duplicate it. |
| Editing a `.json` under `//.json` (FS overlay, e.g. `/view/case_grid.json`) | ❌ FS layer is independent of DB. | ✅ Appended to the change log at `/.objectstack/.log/main.jsonl` by `FileSystemRepository`. |
-| Studio inline edit, or `PUT /api/v1/meta/...` (REST) on an `allowOrgOverride: true` type | ✅ Written by `SysMetadataRepository.put()` as an **overlay row** scoped to `organization_id`. | ✅ Appended to `sys_metadata_history` (per-org `event_seq`) in the **same transaction** as the `sys_metadata` write (single-instance scope; no cross-replica push). |
+| Studio inline edit, or `PUT /api/v1/meta/...` (REST) on an `allowOrgOverride: true` type | ✅ Written by `SysMetadataRepository.put()` as an **overlay row** scoped to `organization_id`. | ✅ Appended to `sys_metadata_history` (per-org `event_seq`) in the **same transaction** as the `sys_metadata` write. No-op puts (identical hash) skip the history row entirely. |
| Deploying a new build (new `dist/objectstack.json`) | ❌ The artifact is loaded into memory, not synced into `sys_metadata`. | ❌ Use Git tags / your deployment platform's release log; that's where artifact "version history" lives. |
### Why artifact never enters the database
diff --git a/content/docs/data-modeling/external-datasources.mdx b/content/docs/data-modeling/external-datasources.mdx
index bfe819fd43..d2cdc4ae38 100644
--- a/content/docs/data-modeling/external-datasources.mdx
+++ b/content/docs/data-modeling/external-datasources.mdx
@@ -37,8 +37,8 @@ export const Warehouse = defineDatasource({
schemaMode: 'external', // ObjectStack never runs DDL here
config: { host: 'db.internal', port: 5432, database: 'analytics', user: 'readonly' },
external: {
- allowWrites: false, // read-only (the default)
- credentialsRef: 'secret:warehouse/password', // resolved from the secret store
+ allowWrites: false, // read-only (the default)
+ credentialsRef: 'sys_secret:9f2c…', // opaque handle minted by the secret store
validation: { onMismatch: 'fail', checkOnBoot: true },
},
active: true,
@@ -90,11 +90,12 @@ That's it. `GET /api/v1/data/ext_customer` now returns live rows from the remote
`customers` table; filters (`?region=EU`) push down to the remote query.
-**Do not set `field.columnName` on an external object.** For federated objects the
-driver's query pipeline ignores `field.columnName`; `external.columnMap`
-(`remoteColumn → localField`) is the single, authoritative column mapping.
-`os build` / `os validate` rejects `field.columnName` on an external object with a
-clear error (ADR-0062 D7). Managed objects are unaffected.
+**`field.columnName` no longer exists.** It was removed from `FieldSchema` in the
+16.x line (#2377, ADR-0049) — the SQL driver always used the field key as the
+physical column, so a custom column name was silently ignored — and the
+ADR-0062 D7 lint that rejected it on external objects was removed with it.
+`external.columnMap` (`remoteColumn → localField`) is the single, authoritative
+column mapping for a federated object.
## 3. Auto-connect — no `onEnable` needed
@@ -121,9 +122,10 @@ from external configuration. Auto-connect is idempotent with it (whichever
registers the datasource name first wins), so the two never conflict.
-A connected external datasource is also visible in **Setup → Datasources** (stamped
-`origin: code`, read-only in the UI) and via `GET /api/v1/datasources` and
-`GET /api/v1/meta/datasource`, where an admin can run the "Sync objects" wizard.
+A connected external datasource is also visible in **Setup → Integrations →
+Datasources** (stamped `origin: code`, read-only in the UI) and via
+`GET /api/v1/datasources` and `GET /api/v1/meta/datasource`, where an admin can
+run the "Sync objects" wizard.
### When auto-connect fails
@@ -160,8 +162,8 @@ Every connect attempt's verdict is retained, so a datasource that is down no
longer has to be diagnosed by restarting the server and re-reading boot logs
([#3827](https://github.com/objectstack-ai/objectstack/issues/3827)).
-**Setup → Datasources** and `GET /api/v1/datasources` report a `status` per
-datasource:
+**Setup → Integrations → Datasources** and `GET /api/v1/datasources` report a
+`status` per datasource:
| `status` | Meaning |
|:---|:---|
@@ -202,7 +204,7 @@ const policy: DatasourceConnectPolicy = {
? { allow: true }
: {
allow: false,
- // operator-facing: logs + Setup → Datasources only
+ // operator-facing: logs + the Setup datasource list only
reason: `egress allow-list miss for ${ds.name} (${tenant.id}, plan=${tenant.plan})`,
// tenant-facing: appended to the query-time error
publicReason: 'External datasources require the Scale plan.',
@@ -217,6 +219,12 @@ secret in the secret store (the same `SecretBinder` / `ICryptoProvider` the
runtime-admin "Add Datasource" wizard uses). The credential is resolved to
cleartext **at connect, before the driver is built**.
+The shipped binder encrypts the cleartext into a `sys_secret` row and mints an
+**opaque** handle — `sys_secret:` — as the `credentialsRef`; that is the only
+form it resolves, so a hand-written path like `secret:warehouse/password` will
+not dereference. A host that wants a different scheme (a vault path, say)
+injects its own resolver via `new DatasourceAdminServicePlugin({ secrets })`.
+
Resolution is **fail-closed**: if a `credentialsRef` is declared but no secret store
is configured, or the secret cannot be resolved/decrypted, the datasource is left
**unconnected with a clear error** — never connected without the credential. An
diff --git a/content/docs/data-modeling/formulas.mdx b/content/docs/data-modeling/formulas.mdx
index 35c936a2df..b0fb6f30da 100644
--- a/content/docs/data-modeling/formulas.mdx
+++ b/content/docs/data-modeling/formulas.mdx
@@ -39,7 +39,7 @@ Every expression in metadata is persisted as the same envelope:
type Expression = {
dialect: 'cel' | 'cron' | 'template';
source?: string; // surface syntax
- ast?: unknown; // parsed AST (filled by `objectstack compile`)
+ ast?: unknown; // parsed AST (filled by `os compile`)
meta?: { rationale?: string; generatedBy?: string };
};
```
@@ -66,11 +66,11 @@ to relearn the syntax across surfaces.
most common mistake (and the root cause of issue #1491) is a condition like
`{record.rating} >= 4`: in CEL, `{…}` is a **map literal**, so it is a parse
error. Write bare CEL: `record.rating >= 4`. Braces belong only in `{{ … }}`
-text templates. As of 7.6 a malformed expression no longer silently does
-nothing — it fails `objectstack build` and throws at runtime.
+text templates. A malformed expression does not silently do nothing — it fails
+`os build` and throws at runtime.
-**Template formatting (7.6).** A `template` hole is a field path with an optional
+**Template formatting.** A `template` hole is a field path with an optional
whitelisted formatter — `{{ path | formatter[:arg] }}` — with defined value→string
semantics (no arbitrary logic; put logic in a CEL field):
@@ -121,9 +121,14 @@ evaluates (`objectql` computes a formula virtual field from its `expression`
only). Do **not** pass `type: 'currency' | 'number' | …` to `Field.formula` — it
is rejected by the typed `FieldInput` and, untyped, would override `type:'formula'`
so the field silently never computes. To declare what the formula returns, set
-**`returnType`** (`'number' | 'text' | 'boolean' | 'date'`); it is inferred and
-stamped automatically by the AI build path, and consumers (dashboard measures,
-formatting) read it instead of re-parsing the expression. The CEL source is the
+**`returnType`** (`'number' | 'text' | 'boolean' | 'date'`). You declare it
+yourself — nothing back-fills it — but the agent-callable `validate_expression`
+tool reports the type it infers from the expression (`inferredType`) so an AI
+author can stamp the matching value. Consumers read the declared `returnType`
+instead of re-parsing the expression (record-title eligibility, for one: a
+formula is title-eligible only when its `returnType` is `'text'`, which is what
+lets it be *derived* as the record title — an explicit `nameField` pointer is
+honored either way). The CEL source is the
**`expression`** key — it is the only key the schema and runtime read for a
formula's source; there is no separate `formula` source key.
@@ -161,7 +166,7 @@ Cheat-sheet of what you'll write daily:
Registered automatically into every `Environment` by `@objectstack/formula`.
All functions are pure given a pinned `now`, which is what makes
-`objectstack build` artifacts byte-stable across runs.
+`os build` artifacts byte-stable across runs.
| Function | Returns | Description |
|:---|:---|:---|
@@ -197,7 +202,8 @@ Keep them pure, dependency-free, and AI-readable.
| `record` | the row being evaluated | formulas, validation, sharing, visibility |
| `previous` | row before update | hooks, validation on update |
| `input` | hook payload | hooks |
-| `os.user` | install / request user | seed, predicates with identity |
+| `current_user` | the authenticated subject — the canonical binding (ADR-0068). `user`, `ctx.user` and `os.user` are aliases of the **same** object | predicates with identity |
+| `os.user` | alias of `current_user` | seed, predicates with identity |
| `os.org` | active organization | seed, predicates |
| `os.env` | install env (`prod`, `dev`, `test`) | seed, predicates |
@@ -205,12 +211,12 @@ Keep them pure, dependency-free, and AI-readable.
## Build-time validation
-`objectstack build` type-checks every expression against the object schema, so
+`os build` type-checks every expression against the object schema, so
a mistake that would silently evaluate to `null` at runtime is caught before it
ships. The same shared validator also runs when a flow is registered
(`registerFlow`), so a flow authored dynamically gets the same verdict. Findings
come at two severities: **errors** fail the build; **warnings** are advisory
-(surfaced, not fatal — `objectstack validate --strict` promotes them to errors).
+(surfaced, not fatal — `os validate --strict` promotes them to errors).
| Check | Example | Severity |
|:---|:---|:---|
@@ -330,10 +336,19 @@ invert the test (here `amount <= 0` rejects non-positive amounts).
object: 'case',
events: ['afterUpdate'],
condition: P`previous.status != 'escalated' && record.status == 'escalated'`,
- body: { language: 'js', source: 'await ctx.notifyOpsTeam(record)' },
+ body: {
+ language: 'js',
+ source: "ctx.log.warn('case escalated', { id: ctx.input.id });",
+ capabilities: ['log'],
+ },
}
```
+The `condition` is the CEL part — it binds `record` / `previous` directly. The
+JS `body` is a different surface: it is wrapped as `new AsyncFunction('ctx', source)`,
+so inside it the record is `ctx.input` (there is no bare `record`), and every
+`ctx` API it touches must be covered by a declared capability.
+
---
## Formula patterns
@@ -375,24 +390,32 @@ Field.formula({
Field.formula({
label: 'Days to Close',
returnType: 'number',
- expression: 'daysBetween(record.close_date, today())',
+ expression: 'daysBetween(today(), record.close_date)',
})
// Contract end in 30 days
Field.formula({
label: 'Expiring Soon',
returnType: 'boolean',
- expression: 'record.status == "active" && daysBetween(record.end_date, today()) <= 30',
+ expression: 'record.status == "active" && daysBetween(today(), record.end_date) <= 30',
})
// Age in days
Field.formula({
label: 'Age (Days)',
returnType: 'number',
- expression: 'daysBetween(today(), record.created_date)',
+ expression: 'daysBetween(record.created_date, today())',
})
```
+
+**Argument order matters.** `daysBetween(a, b)` is `b − a`, so it counts
+*forward from `a` to `b`* and goes negative when `b` is earlier. "Days
+remaining" is `daysBetween(today(), record.due_date)`; "age so far" is
+`daysBetween(record.created_date, today())`. Swapping the two operands is the
+easiest way to ship a formula whose sign is inverted on every row.
+
+
**`dateField == today()` now matches (#3183).** A `date` field reads back as a
`YYYY-MM-DD` string, and CEL treats a string and a timestamp as unequal — so the
@@ -479,18 +502,20 @@ Field.formula({
❌ **DON'T:**
- Create circular references
- Use formulas for frequently changing data
-- Nest too many IF statements
+- Nest ternaries so deeply the expression stops being readable
- Ignore null handling
---
## Determinism contract
-Two consecutive `objectstack build` runs with no source changes must produce
+Two consecutive `os build` runs with no source changes must produce
**byte-identical** `dist/objectstack.json`. CEL plus pinned `now` is what
makes this possible — there are zero compile-time-evaluated `Date.now()`
-calls in seed metadata. CI enforces this via SHA-1 comparison; if you add a
-new dialect or stdlib helper, ensure it preserves determinism.
+calls in seed metadata: a CEL seed value travels into the artifact as
+unevaluated source and only resolves at install. This is a design contract
+held by review, not a gate — no CI job currently diffs two builds — so if you
+add a new dialect or stdlib helper, ensure it preserves determinism.
---
@@ -524,7 +549,7 @@ Salesforce semantics rewrite their formulas in CEL.
```ts
import { ExpressionEngine } from '@objectstack/formula';
-// Compile + cache
+// Compile — parse + type-check, returning the engine-native AST as `value`
const compiled = ExpressionEngine.compile({ dialect: 'cel', source: 'record.x * 2' });
// Evaluate
@@ -536,11 +561,11 @@ const result = ExpressionEngine.evaluate(
```
The low-level engine never throws — `evaluate()` returns `{ ok: false, error }`.
-But **call sites must not silently swallow that** (ADR-0032): `objectstack build`
+But **call sites must not silently swallow that** (ADR-0032): `os build`
**fails** on an invalid expression (with a located, schema-aware message), and at
runtime the flow/rule engines **throw** a loud, attributed error instead of
treating a bad expression as `false`/`null`. The same `validateExpression`
-validator backs `objectstack build` and metadata registration (and the
+validator backs `os build` and metadata registration (and the
agent-callable `validate_expression` tool), so an expression is checked before it ships.
---
diff --git a/content/docs/deployment/migration-from-objectql.mdx b/content/docs/deployment/migration-from-objectql.mdx
index a711717993..980708f473 100644
--- a/content/docs/deployment/migration-from-objectql.mdx
+++ b/content/docs/deployment/migration-from-objectql.mdx
@@ -5,7 +5,7 @@ description: Step-by-step guide for migrating from @objectql/core to @objectstac
# Migrating from `@objectql/core` to `@objectstack/objectql`
-Starting with `@objectstack/objectql` v3.1, all core functionality previously provided by `@objectql/core` is now available upstream. This guide walks you through migrating your project so that `@objectql/core` can be removed.
+The core functionality previously provided by `@objectql/core` — the introspection types, the utility functions, and the kernel factory — was upstreamed into `@objectstack/objectql` in **v3.0.4** and has shipped in every release since (the current release line is **v16.x**). This guide walks you through migrating your project so that `@objectql/core` can be removed.
## Why Migrate?
@@ -26,7 +26,7 @@ Starting with `@objectstack/objectql` v3.1, all core functionality previously pr
"dependencies": {
- "@objectql/core": "^4.x",
- "@objectql/types": "^4.x",
-+ "@objectstack/objectql": "^3.1.0"
++ "@objectstack/objectql": "^16.1.0"
}
}
```
diff --git a/content/docs/deployment/vercel.mdx b/content/docs/deployment/vercel.mdx
index 239ad49292..ed2e051ed7 100644
--- a/content/docs/deployment/vercel.mdx
+++ b/content/docs/deployment/vercel.mdx
@@ -5,7 +5,7 @@ description: Deploy ObjectStack applications to Vercel with the Hono server adap
# Deploy to Vercel
-ObjectStack 11 deploys to Vercel in **server mode** — serverless functions running the **Hono** adapter (`@objectstack/hono`). The published ObjectStack Studio/console ships as a static Vite SPA that points at a separate ObjectStack server (via `VITE_SERVER_URL`).
+ObjectStack deploys to Vercel in **server mode** — serverless functions running the **Hono** adapter (`@objectstack/hono`). The Console SPA can also be deployed on its own as a static Vite build that points at a separate ObjectStack server (via `VITE_SERVER_URL`) — but that build comes from the Console source repo ([objectui](https://github.com/objectstack-ai/objectui)), not from the prebuilt `@objectstack/console` package.
**Removed in 11.** The in-browser MSW mode (`@objectstack/plugin-msw`) and the Next.js adapter (`@objectstack/nextjs`) are no longer published — use the Hono server path below.
@@ -43,7 +43,7 @@ In Server mode, ObjectStack runs inside Vercel Serverless Functions. API request
This is a valid self-contained pattern when you want to ship a Vite SPA *and* its API from a single Vercel project: the SPA is served as static assets, and a Hono-based serverless function handles `/api/*` requests.
-The published ObjectStack Studio/console (`@objectstack/console`) does **not** use this topology. It deploys as a pure static SPA with no `api/` function — it is meant to be embedded in, or pointed at, a separate ObjectStack server via `VITE_SERVER_URL`. For that separate server, prefer the CLI (`objectstack serve`) over a hand-rolled Hono function.
+The framework-vendored Console (`@objectstack/console`) does **not** use this topology. It is a **prebuilt, dist-only** package — no source and no build step, just static assets — that an ObjectStack server mounts at `/_console` (`os serve --ui`, on by default). Its bundle is baked same-origin, and `VITE_SERVER_URL` is a build-time Vite flag, so aiming a Console at a *different* origin means building it yourself from the Console source repo ([objectui](https://github.com/objectstack-ai/objectui)). For the server itself, prefer the CLI (`os serve`) over a hand-rolled Hono function.
**1. Create the kernel singleton** (`api/_kernel.ts` — prefixed with `_` to prevent Vercel from creating a route):
@@ -82,7 +82,10 @@ export async function ensureApp(): Promise {
`InMemoryDriver` keeps this snippet self-contained, but on serverless **all data
is lost on every cold start** — fine for a demo, wrong for anything real. Before
going past a proof of concept, swap the driver for a real database (see
-[Database Drivers](/docs/data-modeling/drivers)) and point `OS_DATABASE_URL` at it.
+[Database Drivers](/docs/data-modeling/drivers)). A hand-built kernel does **not**
+resolve `OS_DATABASE_URL` on its own — that precedence lives in the CLI and in
+`createStandaloneStack()` — so read the connection string yourself, e.g.
+`new SqlDriver({ client: 'pg', connection: process.env.OS_DATABASE_URL })`.
**2. Create the API entrypoint** (`api/index.ts`):
@@ -134,7 +137,7 @@ Setting `VITE_SERVER_URL` to empty string tells the client SDK to use same-origi
Configure these in Vercel Project Settings → Environment Variables:
-### Studio / SPA build
+### Console / SPA build
| Variable | Description |
| :--- | :--- |
@@ -179,7 +182,7 @@ storage for artifacts.
- [ ] `api/_kernel.ts` boots the kernel with the correct driver
- [ ] `vercel.json` sets `VITE_USE_MOCK_SERVER=false` and `VITE_SERVER_URL=` (empty)
- [ ] Rewrite rule routes `/api/*` to `/api` and excludes `/api/` from SPA rewrite
-- [ ] `OS_DATABASE_URL` is configured in Vercel environment variables (for production drivers)
+- [ ] The production connection string is set in Vercel environment variables **and** read explicitly by the driver you construct in `api/_kernel.ts` (`OS_DATABASE_URL` is resolved by the CLI / `createStandaloneStack()`, not by a hand-built kernel)
- [ ] CORS is configured if frontend and API are on different origins
---
diff --git a/content/docs/kernel/cluster.mdx b/content/docs/kernel/cluster.mdx
index 34dd10cdf7..b23e17009e 100644
--- a/content/docs/kernel/cluster.mdx
+++ b/content/docs/kernel/cluster.mdx
@@ -83,19 +83,32 @@ These four are the minimum sufficient set. Higher-level facilities
- **No native job queue primitive.** A job queue is `KV + Lock + PubSub`
composed; expressing it directly in the cluster layer would couple the
- protocol to a specific queue shape. `service-queue` builds on the primitives
- instead.
+ protocol to a specific queue shape. `service-queue` is expected to build on
+ the primitives instead — today it ships memory and DB adapters only and does
+ not consume the cluster service.
- **No native cache primitive.** Caches are `KV + PubSub` (read the value,
- subscribe to the invalidation channel). `service-cache` builds on the
- primitives instead.
+ subscribe to the invalidation channel). `service-cache` is expected to build
+ on the primitives instead — today it ships a memory adapter plus a Redis
+ adapter skeleton that throws `RedisCacheAdapter not yet implemented`, and it
+ does not consume the cluster service either.
- **No native stream/Kafka primitive.** Streaming is intentionally out of
scope for v1. It can be added as a fifth primitive later without changing
the four above.
## 4. Event scope and delivery semantics
-The single biggest source of cluster bugs is "I called `eventBus.emit(...)`
-and assumed everyone would hear it". Every emit must answer three questions:
+The single biggest source of cluster bugs is "I published an event and assumed
+everyone would hear it". Every emit must answer three questions:
+
+> **Status: protocol only.** The three fields below exist as
+> `EventMetadata.cluster` (`EventClusterOptionsSchema` in
+> `kernel/cluster.zod.ts`, wired into `kernel/events/core.zod.ts`), but **no
+> runtime consumes them** — there is no `eventBus` object in the codebase at
+> all. The kernel's plugin event API is `ctx.hook(name, handler)` /
+> `ctx.trigger(name, ...args)`, neither of which takes cluster metadata. The
+> shipped cross-node path today is a direct `cluster.pubsub.publish(...)` call
+> (§6.2, §7.3). §4.1-§4.4 define what these enum values are *supposed* to mean
+> once a bus reads them.
### 4.1 Scope — who receives this event?
@@ -133,6 +146,11 @@ deliverySemantics: 'best-effort' | 'at-least-once' | 'exactly-once'
returns. Survives node crash. Handlers must be **idempotent** —
duplicates are possible during retry. Use for: webhook outbox, audit
shipping, async job enqueue.
+ **No shipped driver provides this yet.** The `redis` driver publishes over
+ plain Redis pub/sub, which is *at-most-once* — fire-and-forget, no
+ persistence, no replay for a node that was down at publish time. Only route
+ cache-invalidation hints through it; anything that must survive a crash needs
+ a durable outbox of its own.
- **`exactly-once`** — Reserved keyword; **not implemented in v1**. The
protocol accepts the enum value so future runtimes can add it without a
breaking change. Startup rejection of this value is *intended* behaviour
@@ -153,29 +171,56 @@ partition keys: record id, tenant id, conversation id.
When unset, no ordering is guaranteed — handlers may run in parallel and
out of emit order.
+> **Status: accepted but ignored.** `PublishOptions.partitionKey` is part of
+> the `IPubSub` contract and callers already pass it (the metadata manager
+> keys on `` `${type}:${name}` ``), but both shipped drivers take the options
+> bag as an unused `_opts` parameter. The memory driver's synchronous
+> single-process fan-out preserves emit order anyway; the redis driver gives
+> you whatever ordering Redis pub/sub does. Do not rely on cross-key ordering
+> guarantees yet.
+
### 4.4 The combined contract
+The three fields compose into one `EventMetadata.cluster` block:
+
```ts
-eventBus.emit('account.updated', payload, {
+// target shape — parses today, nothing reads it yet
+const metadata = {
+ source: 'my-plugin',
+ timestamp: new Date().toISOString(),
cluster: {
scope: 'cluster',
deliverySemantics: 'at-least-once',
partitionKey: payload.id,
},
-})
+}
```
-This is the **only** correct way to invalidate a per-record cache entry
-across a cluster: the `cluster` scope reaches every node, `at-least-once`
-guarantees no node misses the invalidation across a crash, and
-`partitionKey` ensures that two rapid updates to the same record are
-applied in order.
+Once a bus honours it, that combination is the intended way to invalidate a
+per-record cache entry across a cluster: the `cluster` scope reaches every
+node, `at-least-once` guarantees no node misses the invalidation across a
+crash, and `partitionKey` ensures that two rapid updates to the same record
+are applied in order.
+
+Until then, the working equivalent is a direct publish on the cluster
+service — it accepts `partitionKey` (see the §4.3 caveat) and gives
+best-effort delivery only:
+
+```ts
+import type { IClusterService } from '@objectstack/spec/contracts'
+
+const cluster = ctx.getService('cluster')
+await cluster.pubsub.publish('account.updated', payload, {
+ partitionKey: payload.id,
+})
+```
## 5. Service scope and leader election
-Every service registered with the kernel must declare its **cluster scope**.
-This is independent of DI lifecycle scope (`singleton` / `transient` /
-`scoped`); it answers a different question.
+Every service registered with the kernel has a **cluster scope**. The
+annotation is optional — omitting it is equivalent to
+`{ clusterScope: 'node' }`. This is independent of DI lifecycle scope
+(`singleton` / `transient` / `scoped`); it answers a different question.
| `clusterScope` | Meaning |
|----------------|---------|
@@ -236,12 +281,16 @@ registration (`ServiceMetadata.cluster` / `ServiceFactoryRegistration.cluster`
When a service declares `clusterScope: 'cluster'`, the runtime will
automatically:
-1. Wrap `onEnable` so the work only starts after lock acquisition (for
- `leader-elected`).
+1. Wrap the plugin's `start(ctx)` so the work only begins after lock
+ acquisition (for `leader-elected`).
2. Refresh the lock TTL via heartbeat while the node is healthy.
-3. Release the lock on `onDisable` or graceful shutdown.
+3. Release the lock on `destroy()` / the `kernel:shutdown` hook.
4. Emit a leadership-change event so dependent services can react.
+(The kernel's plugin contract is `init(ctx)` / `start(ctx)` / `destroy()`. The
+`onEnable` / `onDisable` family was retired from the protocol — it never had
+an invocation site — so do not reach for it here.)
+
A plugin author writing a cron scheduler will **not** write any lock code —
they declare scope and strategy, and the runtime does the rest.
@@ -278,19 +327,25 @@ invalidation evicts a "newer" value the node has already learned about.
### 6.2 The metadata change event
**Current behaviour.** Cross-node metadata invalidation already works, but
-it does **not** go through `eventBus.emit`. The metadata manager fans out
-over the cluster PubSub channel `metadata.changed` (note: dot, not colon)
-with the payload shape `ClusterMetadataChangedPayload`:
+not through any event bus (there isn't one — see §4). The metadata manager
+fans out over the cluster PubSub channel `metadata.changed` (note: dot, not
+colon) with the payload shape `ClusterMetadataChangedPayload`:
```ts
-// packages/metadata/src/metadata-manager.ts
-ctx.cluster.pubsub.publish('metadata.changed', {
- originNode, // used for loopback suppression
- type, // 'object' | 'view' | 'dashboard' | …
- event, // the local watch event, replayed verbatim on peers
-})
+// packages/metadata/src/metadata-manager.ts — notifyWatchers()
+this.clusterPubSub.publish('metadata.changed', {
+ originNode: this.clusterNodeId, // used for loopback suppression
+ type, // 'object' | 'view' | 'dashboard' | …
+ event, // the local watch event, replayed verbatim on peers
+}, { partitionKey: `${type}:${event.name ?? ''}` })
```
+The transport is handed to the manager by `MetadataClusterBridgePlugin`
+(`@objectstack/service-cluster`), which calls
+`metadata.attachClusterPubSub(cluster.pubsub, cluster.nodeId)` on
+`kernel:ready`. `Runtime` registers that bridge automatically alongside the
+cluster service.
+
Peers suppress their own messages by `originNode` and replay the watch
event locally — there is currently **no** `version` / `name` / `tenantId` /
`operation` field and **no** version comparison.
@@ -336,7 +391,8 @@ in §4-§6; this is the cheat sheet.
### 7.1 "I want to fire an event"
-Pick scope first, then delivery, then partition key:
+Target design only — no runtime reads `EventMetadata.cluster` today (§4). Pick
+scope first, then delivery, then partition key:
| You want to… | scope | delivery | partitionKey |
|---------------------------------------------|-----------|-----------------|---------------|
@@ -360,18 +416,36 @@ Ask: **if I deploy 5 nodes, should there be 5 of these or 1?**
- "Pulls from a shared queue" → `partitioned`
- "Reacts to events with idempotent writes" → `idempotent-broadcast`
+Declaring `cluster` does not yet *do* anything (§5.1) — until Phase 4 lands
+you still have to hold the invariant yourself, either by wrapping the work in
+`cluster.lock.withLock(key, fn)` or by acquiring and releasing by hand the way
+`service-job`'s cron scheduler does (`cluster.lock.acquire(key, { waitMs: 0 })`,
+bail out when it returns `null`, release in a `finally`).
+
### 7.3 "I want to read metadata in a long-lived cache"
-There is **no** `ctx.cluster.cache(...)` factory. The cluster service
-exposes only the four primitives (`ctx.cluster.{pubsub, lock, kv, counter}`,
-see §3) — caches are *derived* from `KV + PubSub` by `service-cache`, they
-are not a built-in cluster method.
+There is **no** `cluster.cache(...)` factory, and there is no `ctx.cluster`
+property either — `PluginContext` exposes the service-registry family
+(`registerService` / `registerServiceFactory` / `getService` /
+`getServiceScoped` / `getServices` / `replaceService`), `hook` / `trigger`,
+`logger` and `getKernel`. No cluster handle is among them, so resolve the
+cluster service by name:
+
+```ts
+import type { IClusterService } from '@objectstack/spec/contracts'
+
+const cluster = ctx.getService('cluster')
+```
+
+It exposes only the four primitives (`cluster.{pubsub, lock, kv, counter}`,
+see §3) plus `nodeId` / `driver` / `close()`. Caches are meant to be *derived*
+from `KV + PubSub`; they are not a built-in cluster method.
To keep an in-process metadata cache coherent today, subscribe to the
metadata change channel via PubSub and invalidate on each notification:
```ts
-ctx.cluster.pubsub.subscribe('metadata.changed', (msg) => {
+cluster.pubsub.subscribe('metadata.changed', (msg) => {
const payload = msg.payload // { originNode?, type, event }
if (payload.type === 'object') {
myObjectCache.invalidate(payload.event)
@@ -386,29 +460,31 @@ the subscribe call a no-op-equivalent local fan-out on a single node.
### 7.4 What you must never do
-- Never call `setInterval` for cluster-wide periodic work — declare a
- `cluster`-scoped service with `leader-elected`.
+- Never call `setInterval` for cluster-wide periodic work — gate each tick on
+ `cluster.lock` (and declare `clusterScope: 'cluster'` + `leader-elected` so
+ Phase 4 can take the wiring over from you).
- Never `SELECT … WHERE status = 'pending' LIMIT 1` without a row lock —
use `SELECT … FOR UPDATE SKIP LOCKED` (Postgres) or
`UPDATE … RETURNING` with a `claimed_by` column.
- Never write into a process-local `Map` and assume other nodes see it —
- use `ctx.cluster.kv` for shared state.
-- Never `import Redis from 'redis'` directly — go through
- `ctx.cluster.{pubsub, lock, kv, counter}`.
+ use `cluster.kv` for shared state.
+- Never `import Redis from 'ioredis'` directly — go through
+ `cluster.{pubsub, lock, kv, counter}`.
## 8. Configuration surface
-Cluster behaviour is configured once, at the stack level. The protocol
-ships a single optional field on `defineStack()`:
+Cluster behaviour is configured once, at the runtime level. The shape is
+`ClusterCapabilityConfig` (`kernel/cluster.zod.ts`), and it is passed on the
+`Runtime` constructor:
```ts
-defineStack({
- // …existing fields…
+import { Runtime } from '@objectstack/runtime'
+
+const runtime = new Runtime({
cluster: {
driver: 'memory', // default — single-process
- // driver: 'redis', url: process.env.REDIS_URL,
- // driver: 'postgres', useExistingPool: true,
- // driver: 'nats', url: process.env.NATS_URL,
+ // driver: 'redis', url: process.env.OS_REDIS_URL,
+ // driver: 'custom', // resolved from registerClusterDriver()
nodeId: process.env.NODE_ID, // optional, auto-generated if absent
heartbeatMs: 5000, // leader-election heartbeat
@@ -420,32 +496,67 @@ defineStack({
})
```
+
+ `defineStack()` has **no** `cluster` field — `stack.zod.ts` never gained one.
+ Putting a `cluster` key in a stack definition is silently dropped. Cluster
+ config belongs to the runtime, not the metadata package.
+
+
+The same object can be passed lower down when you are wiring the kernel by
+hand, or used to build a bare service:
+
+```ts
+import { ClusterServicePlugin, defineCluster } from '@objectstack/service-cluster'
+
+kernel.use(new ClusterServicePlugin({ config: { driver: 'memory', nodeId: 'web-1' } }))
+
+// …or standalone, outside a kernel
+const cluster = defineCluster({ driver: 'memory' })
+```
+
+Under `os serve` the driver is selected from the environment instead:
+`OS_CLUSTER_DRIVER` (anything other than `memory` triggers a dynamic import of
+`@objectstack/service-cluster-`) and `OS_REDIS_URL` for the connection
+string. A distribution may register a multi-node gate; when it denies,
+`os serve` logs a warning and downgrades to the single-node in-memory cluster
+rather than failing.
+
Every cluster primitive selects its implementation from `cluster.driver`.
-When the field is absent, the kernel silently auto-registers the in-memory,
-single-node driver. Emitting a production warning in this case (so operators
-notice that horizontal scale will be incorrect until `cluster.driver` is
-configured) is *intended* but **not yet implemented** — no such warning is
-logged today.
+When the field is absent, `Runtime` auto-registers the in-memory, single-node
+driver (pass `cluster: false` to skip registration entirely). No production
+warning is emitted for that case. There *is* a split-brain guard: if the
+operator declares a multi-node topology via `OS_EXPECT_MULTI_NODE=true` or
+`OS_CLUSTER_REPLICAS` > 1 while the resolved driver is `memory`, startup
+**throws** — override with `OS_ALLOW_MEMORY_CLUSTER_MULTINODE=true`.
### 8.1 Driver matrix
-| Driver | PubSub | Lock | KV | Counter |
-|--------------|-----------------------|----------------------------|---------------------|----------------|
-| `memory` | EventEmitter | in-proc mutex | Map | int |
-| `redis` | Redis Pub/Sub Streams | SETNX + TTL + Lua release | GET/SET | INCR |
-| `postgres` | `LISTEN/NOTIFY` | advisory locks | dedicated KV table | sequence |
-| `nats` | NATS subjects + JetStream | KV bucket lock | KV bucket | KV INCR |
+| Driver | PubSub | Lock | KV | Counter | Ships today |
+|--------------|-----------------------|----------------------------|---------------------|----------------|-------------|
+| `memory` | in-process fan-out | per-key FIFO queue + TTL | Map | Map of bigints | ✅ `@objectstack/service-cluster` |
+| `redis` | `PUBLISH`/`SUBSCRIBE` (at-most-once) | `SET … NX PX` + Lua release/renew | `WATCH`/`MULTI` | `INCRBY` | ✅ `@objectstack/service-cluster-redis` |
+| `postgres` | `LISTEN/NOTIFY` | advisory locks | dedicated KV table | sequence | ❌ not built |
+| `nats` | NATS subjects + JetStream | KV bucket lock | KV bucket | KV INCR | ❌ not built |
-The `redis` driver is the recommended starting point for production
-ObjectStack deployments: it shares its connection pool with the planned
-`service-cache` Redis adapter (no extra infrastructure once cache is on
-Redis), gives native semantics for every primitive (SETNX/Lua/INCR/PUBSUB),
-and decouples cluster transport from the choice of database driver
-(SQL / Turso / Postgres / SQLite all work the same).
+Only `memory` and `redis` are implemented. `postgres` and `nats` are accepted
+by `ClusterDriverSchema` but no package provides them — `defineCluster()`
+throws `Cluster driver "" is not registered` for either. The `custom`
+driver value resolves whatever a plugin registered via
+`registerClusterDriver(name, factory)`.
-The `postgres` driver remains a viable "single binary, single dependency"
+The `redis` driver is the recommended starting point for production
+ObjectStack deployments: it accepts a pre-built ioredis client via
+`driverOptions.client`, so it can share one connection pool with any other
+Redis consumer (the `service-cache` Redis adapter is still a skeleton, so
+there is nothing to share with yet); it gives native semantics for every
+primitive (SET NX/Lua/INCRBY/PUBLISH); and it decouples cluster transport from
+the choice of database driver (SQL / Turso / Postgres / SQLite all work the
+same).
+
+A `postgres` driver would remain a viable "single binary, single dependency"
choice for small deployments that already run Postgres and want to avoid
-a Redis dependency. It is deferred to a community/optional driver.
+a Redis dependency, but nobody has built one — it is deferred to a
+community/optional driver against the same `registerClusterDriver()` SPI.
## 9. Industry alignment
@@ -459,7 +570,8 @@ ecosystem; none are novel.
with invalidation broadcast; leader-elected schedulers. Maps to §6 and
`leader-elected` strategy.
- **Odoo `bus.bus`** — Postgres `LISTEN/NOTIFY` for cross-worker fan-out.
- Validates our `postgres` driver as the right default starting point.
+ Validates `LISTEN/NOTIFY` as a sound transport for the deferred `postgres`
+ driver (§8.1, Phase 5) — ObjectStack shipped `redis` first instead.
- **Notion / Linear** — Redis pub/sub for realtime fan-out with
per-document partitioning. Validates `partitionKey` ordering as the
right primitive.
@@ -482,24 +594,30 @@ v4 consumers.
3. Extend `kernel/service-registry.zod.ts` `ServiceMetadataSchema` and
`ServiceFactoryRegistrationSchema` with an optional nested `cluster`
object (`clusterScope`, `leaderStrategy`).
-4. Add optional `cluster` field to `stack.zod.ts`.
+4. ~~Add optional `cluster` field to `stack.zod.ts`.~~ **Dropped** — cluster
+ config landed on `RuntimeConfig` instead (see §8); `stack.zod.ts` has no
+ `cluster` field.
-No runtime changes. After this phase the protocol *describes* cluster
-semantics; nothing *enforces* them.
+Steps 1-3 shipped. No runtime changes in this phase: the protocol *describes*
+cluster semantics; nothing *enforces* them.
-### Phase 2 — In-memory runtime (~2 days)
+### Phase 2 — In-memory runtime (~2 days) ✅
Implement `service-cluster` with the `memory` driver only. Every cluster
primitive works; behaviour on one node is identical to today. This phase
-proves the API shape against real callers (EventBus, service-cache,
-service-job) before committing to a wire format.
+proves the API shape against real callers before committing to a wire format.
+In practice the only callers wired so far are `MetadataClusterBridgePlugin`,
+`service-job`'s cron scheduler, and `service-messaging`'s dispatchers —
+`service-cache` and `service-queue` still do not consume the cluster service.
### Phase 3 — Redis driver (~2 days) ✅
Implement `redis` driver via `@objectstack/service-cluster-redis`
-(PUBSUB + SETNX/Lua locks + WATCH/MULTI KV + INCR counter). Ships as a
-sibling package so adopters opt in by importing it. Shares its ioredis
-client pool with `service-cache` to avoid double connections.
+(PUBLISH/SUBSCRIBE + `SET NX PX`/Lua locks + WATCH/MULTI KV + INCRBY counter).
+Ships as a sibling package so adopters opt in by importing it — the import
+self-registers the driver via `registerClusterDriver('redis', …)`. It can
+share an ioredis client with other consumers by passing
+`driverOptions.client`, though nothing shares one with it today.
> **Note**: this was switched from "Postgres first" — see commit history.
> Rationale: Postgres isn't a fixed dependency in the ObjectStack stack,
@@ -508,10 +626,18 @@ client pool with `service-cache` to avoid double connections.
### Phase 4 — Existing services migrate (~3 days, opportunistic)
+Not started. `service-job` already leader-elects by calling
+`cluster.lock.acquire('job:')` directly, but that is hand-rolled — none
+of the declarative wiring below exists yet.
+
- `service-job` declares `clusterScope: 'cluster' / 'leader-elected'`.
-- `service-cache` switches to `ctx.cluster.cache(...)` factory.
+- `service-cache` gains a cluster-backed cache derived from `KV + PubSub`.
- `service-realtime` adds cross-node fan-out via PubSub.
-- Webhook dispatcher (planned) uses `partitioned` strategy from day one.
+- Webhook dispatcher declares the `partitioned` strategy. (The behaviour
+ already ships hand-rolled: webhook deliveries drain through
+ `service-messaging`'s `HttpDispatcher`, which walks `partitionCount`
+ partitions behind per-partition `cluster.lock` keys — only the declarative
+ annotation is missing.)
### Phase 5 — Postgres / NATS drivers (deferred, optional)
diff --git a/content/docs/kernel/events.mdx b/content/docs/kernel/events.mdx
index 3b928ac812..5819dcba64 100644
--- a/content/docs/kernel/events.mdx
+++ b/content/docs/kernel/events.mdx
@@ -19,7 +19,7 @@ Triggered by the Kernel during bootstrap and shutdown:
| Event | Description |
| :--- | :--- |
| `kernel:ready` | All plugins have successfully started. System is live. |
-| `kernel:bootstrapped` | Fired after every `kernel:ready` handler has settled, before `kernel:listening`. The "all bootstrap + seed data is ready" anchor — use it for reconcile/backfill work that consumes data a later-starting plugin produces during `kernel:ready`. |
+| `kernel:bootstrapped` | Fired after every `kernel:ready` handler has settled, before `kernel:listening`. The "all **synchronous** bootstrap has settled" anchor — use it for reconcile/backfill work that consumes data a later-starting plugin produces during `kernel:ready`. It does **not** guarantee app *seed* data has settled: an inline seed that overruns `OS_INLINE_SEED_BUDGET_MS` (default 8s) keeps seeding in the background and can outlast even `kernel:listening`. Subscribe to `app:seeded` for the true per-app seed-settle point. |
| `kernel:listening` | Fired after every `kernel:ready` and `kernel:bootstrapped` handler has completed (e.g. the HTTP server is accepting connections). |
| `kernel:shutdown` | Shutdown signal received. Plugins should clean up resources. |
@@ -61,7 +61,7 @@ The Data Engine runs hooks around record reads and mutations. Write operations f
| `beforeUpdate` / `afterUpdate` | Around record update — single-id **and** bulk (`multi: true`). |
| `beforeDelete` / `afterDelete` | Around record deletion — single-id **and** bulk (`multi: true`). |
-Reads fire `beforeFind` / `afterFind`, for **both** `find` and `findOne` (one read event covers every read shape). There are no per-method (`findOne`/`count`/`aggregate`) or `*Many` events: read authorization and row filtering are handled by RLS/permission rules, field masking by field-level metadata, and bulk writes by the same `before*`/`after*` write events (with the row-scoping predicate in `ctx.input.ast`).
+Reads fire `beforeFind` / `afterFind`, for **both** `find` and `findOne` (one read event covers every read shape). There are no per-method (`findOne`/`count`/`aggregate`) or `*Many` events: read authorization and row filtering are handled by RLS/permission rules, field masking by field-level metadata, and bulk writes by the same `before*`/`after*` write events (a bulk write leaves `ctx.input.id` undefined and carries the caller's predicate in `ctx.input.options.where`; the middleware-composed row-scoping AST stays on the engine's internal operation context and is **not** exposed on `ctx.input`).
### The HookContext
@@ -71,10 +71,10 @@ Every data hook is a single-argument handler `(ctx: HookContext) => void | Promi
| :--- | :--- |
| `ctx.object` | Target object name (immutable). |
| `ctx.event` | Current lifecycle event, e.g. `'beforeInsert'` (immutable). |
-| `ctx.input` | **Mutable** input. Shapes: insert `{ data }`, update `{ id, data }`, delete `{ id }`. Modify this to change the operation. |
+| `ctx.input` | **Mutable** input. Shapes: find `{ ast, options }`, insert `{ data, options }`, update `{ id, data, options }`, delete `{ id, options }`. Modify this to change the operation. |
| `ctx.result` | Operation result, available in `after*` events (mutable). |
| `ctx.previous` | Record state before the operation (update/delete). |
-| `ctx.session` | Auth/tenancy info (`userId`, `organizationId`, `roles`, …). |
+| `ctx.session` | Auth/tenancy info (`userId`, `organizationId`, `positions`, `accessToken`, …). Absent entirely when the call carried no identity envelope. |
| `ctx.api` | Scoped cross-object data access. |
### Registering Data Hooks
@@ -95,19 +95,21 @@ engine.registerHook('afterInsert', async (ctx) => {
### Error Handling
-What happens when a data hook throws is governed by the hook's **`onError`** policy (`'abort'` | `'log'`, default `'abort'`):
-
-- `onError: 'abort'` (default) — the error rolls back the transaction (when the hook is blocking), cancelling the operation.
-- `onError: 'log'` — the error is logged and execution continues; the operation is **not** cancelled.
+A hook registered in code with `engine.registerHook()` always propagates: the engine awaits each handler in turn and never catches, so a throw aborts the operation.
```typescript
engine.registerHook('beforeInsert', async (ctx) => {
if (ctx.object === 'invoice' && !ctx.input.data.customer_id) {
- throw new Error('Invoice must have a customer'); // Aborts the insert (default onError)
+ throw new Error('Invoice must have a customer'); // Aborts the insert
}
});
```
+Hooks declared as **`Hook` metadata** additionally honour an **`onError`** policy (`'abort'` | `'log'`, default `'abort'`). It is a metadata field — `registerHook()` has no `onError` option:
+
+- `onError: 'abort'` (default) — the error propagates, rolling back the transaction (when the hook is blocking) and cancelling the operation.
+- `onError: 'log'` — the error is logged and swallowed; the operation is **not** cancelled.
+
### Ordering
Data hooks accept a `priority` option (**default `100`**). They are sorted so that **lower priority values run first**:
diff --git a/content/docs/kernel/services-checklist.mdx b/content/docs/kernel/services-checklist.mdx
index b00e238652..67f18463a2 100644
--- a/content/docs/kernel/services-checklist.mdx
+++ b/content/docs/kernel/services-checklist.mdx
@@ -6,24 +6,30 @@ description: Complete inventory of ObjectStack kernel services with protocol met
# Kernel Services Checklist
-**This checklist is out of date.** It predates several packages that have since
-shipped — SQL and MongoDB drivers, automation, realtime, storage, and more (see
-[Plugins & Packages](/docs/plugins/packages) for the current catalog). Keep it
-for the protocol-method inventory; treat the per-service status columns as
-historical until this page is regenerated.
+**This page is hand-maintained.** The per-slot Provider column mirrors
+`CORE_SERVICE_PROVIDER` (`packages/spec/src/system/core-services.zod.ts`), which
+is the CI-guarded source of truth; the method inventories mirror the per-domain
+contracts in `packages/spec/src/api/protocol.zod.ts`. When the two disagree,
+the code wins. See [Plugins & Packages](/docs/plugins/packages) for the full
+package catalog.
The ObjectStack protocol defines **16 kernel services** registered via the `CoreServiceName` enum (the never-implemented `graphql` entry was removed in v17). Each service maps to a set of protocol methods governed by its per-domain contract (`DataProtocol`, `MetadataProtocol`, ...) — the transitional `ObjectStackProtocol` composition alias was dissolved in v17 (ADR-0076 D9); capability availability comes from the runtime discovery `services` registry.
-**Key architecture principle**: The kernel itself only provides **data** and **metadata** (framework). Everything else — including **auth** and **automation** — is delivered by **plugins**. The current `@objectstack/objectql` package is an example kernel implementation to get the basic API running; production kernels will be rebuilt as new plugins.
+**Key architecture principle**: the kernel guarantees only **data** and **metadata**, and even those are filled by packages (`@objectstack/objectql`, `@objectstack/metadata`) rather than baked in — the kernel's own contribution is an in-memory fallback for the `core` slots that have one (`metadata`, `cache`, `queue`, `job`, `i18n` — **not** `auth`). Everything else — including **auth** and **automation** — is delivered by plugins. `@objectstack/objectql` is an example kernel implementation to get the basic API running; production kernels will be rebuilt as separate plugins.
-**Last Updated**: February 2026
-
-- ✅ Implemented — 18 protocol methods (kernel-provided)
-- ⚠️ Framework — metadata (in-memory registry, DB persistence pending)
-- 🟡 In Development — auth (plugin structure complete, logic planned)
-- ❌ Plugin Required — 38 protocol methods (to be delivered by plugins)
+**Legend**
+
+- ✅ Implemented — the 17 kernel-provided protocol methods (`DataProtocol` 9 + `MetadataProtocol` 8)
+- ⚠️ Framework — the slot is filled by the kernel's in-memory fallback: real reads and
+ writes, no persistence (self-declares `degraded`, ADR-0076 D12)
+- ❌ Plugin Required — the remaining 36 methods declared across the other per-domain
+ contracts (`analytics` 2, `automation` 1, `packages` 6, `views` 5, `permissions` 3,
+ `workflow` 3, `realtime` 6, `notification` 7, `i18n` 3). *Declared* is not *routed*:
+ `packages` is answered kernel-side by the `/packages` dispatcher domain over the
+ ObjectQL registry, `i18n` has a kernel in-memory fallback, and `views` / `permissions` /
+ `workflow` have no implementation anywhere — the rest wait on whatever fills the slot
---
@@ -34,10 +40,10 @@ The ObjectStack protocol defines **16 kernel services** registered via the `Core
┌─────────────────────────────────────────────────────────┐
│ Kernel Layer │
│ ┌──────────────────┐ ┌──────────────────────────────┐ │
-│ │ metadata (⚠️) │ │ data (✅) │ │
-│ │ In-memory only │ │ ObjectQL example kernel │ │
-│ │ DB persistence │ │ Will be rebuilt as plugins │ │
-│ │ pending │ │ │ │
+│ │ metadata (✅) │ │ data (✅) │ │
+│ │ MetadataPlugin │ │ ObjectQL example kernel │ │
+│ │ → sys_metadata │ │ Will be rebuilt as plugins │ │
+│ │ (in-mem fallback)│ │ │ │
│ └──────────────────┘ └──────────────────────────────┘ │
├─────────────────────────────────────────────────────────┤
│ Plugin Layer │
@@ -45,8 +51,9 @@ The ObjectStack protocol defines **16 kernel services** registered via the `Core
│ workflow, ui, realtime, notification, ai, i18n, │
│ search, file-storage, cache, queue, job │
│ │
-│ Discovery API reports: enabled/unavailable/degraded │
-│ per service so clients adapt their UI accordingly │
+│ Discovery API reports availability per service │
+│ (available / degraded / stub / unavailable) so │
+│ clients adapt their UI accordingly │
└─────────────────────────────────────────────────────────┘
```
@@ -56,23 +63,50 @@ The ObjectStack protocol defines **16 kernel services** registered via the `Core
| # | Service Name | Criticality | Methods | Status | Provider |
|:--|:-------------|:------------|:--------|:-------|:---------|
-| 1 | **metadata** | `core` | 8 | ⚠️ Framework | Kernel (in-memory) |
+| 1 | **metadata** | `core` | 8 | ✅ Implemented (kernel fallback is ⚠️ in-memory) | `@objectstack/metadata` |
| 2 | **data** | `required` | 9 | ✅ Implemented | `@objectstack/objectql` |
| 3 | **analytics** | `optional` | 2 | ❌ Plugin Required | `@objectstack/service-analytics` |
-| 4 | **auth** | `core` | — | 🟡 In Development | `@objectstack/plugin-auth` |
-| 5 | **ui** | `optional` | 5 | ❌ Plugin Required | TBD plugin |
-| 6 | **workflow** | `optional` | 3 | ❌ Plugin Required | TBD plugin |
-| 7 | **automation** | `optional` | 1 | ❌ Plugin Required | TBD plugin |
-| 8 | **realtime** | `optional` | 6 | ❌ Plugin Required | TBD plugin |
-| 9 | **notification** | `optional` | 7 | ❌ Plugin Required | TBD plugin |
-| 10 | **ai** | `optional` | 3 | ❌ Plugin Required | TBD plugin |
+| 4 | **auth** | `core` | — | ✅ Implemented | `@objectstack/plugin-auth` |
+| 5 | **ui** | `optional` | 5 | ❌ Nothing fills this slot | `@objectstack/metadata-protocol` — `/ui/view` is served by its `protocol` service, not by a `ui` service |
+| 6 | **workflow** | `optional` | 3 | ❌ Nothing ships | — |
+| 7 | **automation** | `optional` | 1 | ❌ Plugin Required | `@objectstack/service-automation` |
+| 8 | **realtime** | `optional` | 6 | ❌ Plugin Required (in-process only — no HTTP/WS route is mounted) | `@objectstack/service-realtime` |
+| 9 | **notification** | `optional` | 7 | ❌ Plugin Required | `@objectstack/service-messaging` |
+| 10 | **ai** | `optional` | — | ❌ Nothing ships in this repo | `@objectstack/service-ai` (Cloud/EE — not installable, so the table entry is `null`) |
| 11 | **i18n** | `core` | 3 | ✅ Built-in (in-memory fallback) | `@objectstack/service-i18n` |
-| 12 | **file-storage** | `optional` | — | ❌ Plugin Required | TBD plugin |
-| 13 | **search** | `optional` | — | ❌ Plugin Required | TBD plugin |
+| 12 | **file-storage** | `optional` | — | ❌ Plugin Required | `@objectstack/service-storage` |
+| 13 | **search** | `optional` | — | ❌ Nothing ships | — |
| 14 | **cache** | `core` | — | ✅ Built-in (in-memory fallback) | `@objectstack/service-cache` |
| 15 | **queue** | `core` | — | ✅ Built-in (in-memory fallback) | `@objectstack/service-queue` |
| 16 | **job** | `core` | — | ✅ Built-in (in-memory fallback) | `@objectstack/service-job` |
+
+The Provider column mirrors `CORE_SERVICE_PROVIDER` in
+`@objectstack/spec/system` — the single table discovery reads when it tells a
+caller what to install for an empty slot, and the value it reports as
+`provider`. Two caveats. The table carries **no entry for
+`metadata` or `data`**: those are filled by the engine itself, never by an
+installable optional package, so they need no remedy (`NO_REMEDY_SLOTS` in the
+guard script). And `null` there does **not** always mean "nothing ships" — it
+means "no name belongs in an `Install X` sentence", which covers two cases:
+
+- **Nothing provides the slot at all** — `search`, `workflow` (and the retired
+ `graphql`). Discovery says exactly that rather than naming a plausible
+ package: `No implementation ships for the '' slot — register a service
+ under it to enable`.
+- **A provider exists but cannot be installed** — `ai`. `@objectstack/service-ai`
+ registers the slot in `objectstack-ai/cloud` and is `private: true`.
+
+Two slots therefore override the generic line with a hand-written
+`REMEDY_DETAIL` sentence: `ui`, because the package named does not fill the
+slot (`Served by the protocol service — register MetadataPlugin
+(@objectstack/metadata-protocol) to enable`), and `ai` (`Provided by
+@objectstack/service-ai in ObjectStack Cloud/Enterprise — no implementation
+ships in the open framework`). `scripts/check-service-providers.mjs` fails CI
+if a name in the table is not a real workspace package, or if a
+`CoreServiceName` slot other than `data`/`metadata` is missing from it.
+
+
**Criticality Levels**
- **required**: System cannot start without this service
@@ -82,10 +116,12 @@ The ObjectStack protocol defines **16 kernel services** registered via the `Core
---
-## 1. metadata Service ⚠️ Framework
+## 1. metadata Service ✅ Implemented
**Service Name**: `metadata` · **Criticality**: `core`
-**Implementation**: Kernel — `SchemaRegistry` (in-memory)
+**Implementation**: `MetadataPlugin` (`@objectstack/metadata`) — persisted to `sys_metadata`.
+Fallbacks: the kernel's in-memory `createMemoryMetadata` (`@objectstack/core`) when nothing
+registers the slot, over ObjectQL's in-memory `SchemaRegistry` of artifact-loaded metadata.
**Route Mounts**: `/api/v1/meta`, `/api/v1`, `/api/v1/packages`
### Protocol Methods
@@ -101,16 +137,18 @@ The ObjectStack protocol defines **16 kernel services** registered via the `Core
| `getMetaItemCached` | `GetMetaItemCachedRequest → GetMetaItemCachedResponse` | ✅ |
| `getUiView` | `GetUiViewRequest → GetUiViewResponse` | ✅ |
-### Current Limitations
+### Former Gaps — all now closed
-The metadata service is a **framework** — it works with an in-memory `SchemaRegistry` loaded from YAML/TS config files at startup.
+Every gap this section used to list has shipped. Only the kernel's own
+in-memory fallback is still a "framework": it serves real reads and writes but
+never reaches disk, which is exactly what it self-declares (`degraded`).
-| Gap | Description | Priority |
-|:----|:------------|:--------:|
-| **DB Persistence** | Shipped: `MetadataPlugin` (`@objectstack/metadata`) persists to `sys_metadata`; only the kernel's in-memory fallback is non-persistent | Shipped |
-| **Multi-instance Sync** | No mechanism to share metadata changes across instances | P1 |
-| **Migration / Versioning** | No schema diff or migration tooling | P1 |
-| **Hot Reload** | Metadata changes require restart | P2 |
+| Former gap | Where it landed |
+|:----|:------------|
+| **DB Persistence** | `MetadataPlugin` (`@objectstack/metadata`) persists to `sys_metadata`; only the kernel's in-memory fallback is non-persistent |
+| **Multi-instance Sync** | `MetadataClusterBridgePlugin` (`@objectstack/service-cluster`) bridges the `metadata.changed` channel onto the cluster pub/sub bus, so a mutation on one node invalidates peer registry caches. Needs a `cluster` service and a metadata service exposing `attachClusterPubSub()` |
+| **Migration / Versioning** | `os diff ` compares two configurations and flags breaking changes; `os migrate plan` / `os migrate apply` reconcile the physical database against metadata |
+| **Hot Reload** | `MetadataPlugin` watches its sources (`watch`) and the compiled artifact (`artifactWatch`) with chokidar; `os dev` rebuilds and the server reloads without a restart |
### Discovery: Per-Service Status
@@ -243,62 +281,104 @@ empty slot does. The implementations that really do the work in memory declare
---
-## 4. auth Service 🟡 In Development
+## 4. auth Service ✅ Implemented
**Service Name**: `auth` · **Criticality**: `core`
+**Implementation**: `AuthPlugin` → `AuthManager` (`@objectstack/plugin-auth`, built on better-auth)
**Route Mount**: `/api/v1/auth` — only exposed when a plugin registers the auth service
**The kernel does NOT handle auth.** Install an auth plugin to enable authentication. Without it, the discovery response shows `auth: { enabled: false, status: "unavailable" }`.
-### Plugin Requirements
+### Where each capability lives
-The `auth` service covers both **authentication** (identity) and **authorization** (permissions). There is no separate `permission` service in `CoreServiceName`.
+The `auth` service covers **authentication** (identity). **Authorization** is not a
+`CoreServiceName` slot at all — there is no `permission` service; the evaluators are
+registered under their own names (`security.permissions`, `security.rls`,
+`security.fieldMasker`) by `@objectstack/plugin-security`.
-| Module | Area | Description | Priority |
-|:-------|:-----|:------------|:--------:|
-| **Identity Provider** | Authentication | JWT issuance/verification, session management | P0 |
-| **User CRUD** | Authentication | Create / read / update / delete users | P0 |
-| **Role Management** | Authentication | Role definitions, role-user associations | P0 |
-| **Permission Engine** | Authorization | Object-level and field-level permissions | P0 |
-| **OAuth2 / OIDC** | Authentication | Third-party login (Google, GitHub, etc.) | P1 |
-| **Sharing Rules** | Authorization | Record-level sharing and visibility | P1 |
-| **Row-Level Security** | Authorization | Automatic query filtering by user context | P1 |
-| **Multi-tenancy** | Authentication | Space / Tenant isolation | P1 |
-| **Territory Management** | Authorization | Data territory assignment and access | P2 |
-| **SCIM** | Authentication | Enterprise user provisioning protocol | P2 |
+| Module | Area | Ships in |
+|:-------|:-----|:---------|
+| **Identity Provider** | Authentication | `@objectstack/plugin-auth` — better-auth sessions + `jose` JWTs (`auth-manager.ts`) |
+| **User CRUD** | Authentication | `@objectstack/plugin-auth` — `admin-user-endpoints.ts`, `admin-import-users.ts` |
+| **Role Management** | Authorization | `@objectstack/plugin-security` — owns role / permission-set / user-permission-set / role-permission-set objects (ADR-0029 K2) |
+| **Permission Engine** | Authorization | `@objectstack/plugin-security` — `security.permissions` (object) + `security.fieldMasker` (field) |
+| **OAuth2 / OIDC** | Authentication | `@objectstack/plugin-auth` — `@better-auth/oauth-provider`, `@better-auth/sso` |
+| **Sharing Rules** | Authorization | `@objectstack/plugin-sharing` — `sharing`, `sharingRules`, `shareLinks` services |
+| **Row-Level Security** | Authorization | `@objectstack/plugin-security` — `security.rls` (`rls-compiler.ts`) |
+| **Multi-tenancy** | Authentication | `@objectstack/plugin-auth` — the `tenancy` service (`tenancy-service.ts`, ADR-0093/ADR-0105) |
+| **Territory access** | Authorization | Not a separate module: expressed as an RLS dynamic-membership predicate (`current_user.territory_account_ids`, staged in `ExecutionContext.rlsMembership`) |
+| **SCIM** | Authentication | `@objectstack/plugin-auth` — `@better-auth/scim`, gated by `OS_SCIM_ENABLED` |
-### Spec Files: `identity.zod.ts`, `permission.zod.ts`, `organization.zod.ts`, `auth-config.zod.ts`, `tenant.zod.ts`
+### Spec Files: `identity/identity.zod.ts`, `identity/organization.zod.ts`, `security/permission.zod.ts`, `system/auth-config.zod.ts`, `system/tenant.zod.ts` (all under `packages/spec/src/`)
---
-## 5–7. Business Services ❌ Plugin Required
+## 5–7. Business Services
-### 5. ui Service — 5 methods
-`listViews`, `getView`, `createView`, `updateView`, `deleteView`
-View CRUD, layout engine, action registry, permission filtering.
+### 5. ui Service — 5 declared methods, none routed ❌
+`listViews`, `getView`, `createView`, `updateView`, `deleteView`
-### 6. workflow Service — 3 methods
+
+These five are **optional members of `ViewProtocol` that nothing implements and no
+route reaches** — `view` is a metadata type, so view CRUD goes through the metadata
+API (`/api/v1/meta`), not through them. Nothing anywhere registers the `ui` slot
+either (#4093 / #4146), so `CORE_SERVICE_PROVIDER.ui` names
+`@objectstack/metadata-protocol` rather than a `ui` plugin: the one route the `/ui`
+domain serves is `GET /api/v1/ui/view/:object[/:type]`, which calls `getUiView` on
+the **`protocol`** service that `assembleMetadataProtocol()` registers (invoked by
+`ObjectQLPlugin`, or by the standalone `createMetadataProtocolPlugin()`). Without it
+the domain answers **501** with that remedy spelled out, not a generic "install a ui
+plugin".
+
+
+### 6. workflow Service — 3 methods ❌ Nothing ships
`getWorkflowConfig`, `getWorkflowState`, `workflowTransition`
-State machine transitions. Approve/reject are no longer workflow methods — per ADR-0019 they moved to the request-id-based approvals API under `/api/v1/approvals`.
+State machine transitions. No package registers the `workflow` slot
+(`CORE_SERVICE_PROVIDER.workflow` is `null`). Approve/reject are not workflow
+methods — per ADR-0019 they moved to the request-id-based approvals API under
+`/api/v1/approvals` (`POST /requests/:id/{approve,reject,recall}`, served by
+`@objectstack/plugin-approvals`).
-### 7. automation Service — 1 method
+### 7. automation Service — 1 method ✅ `@objectstack/service-automation`
`triggerAutomation`
Trigger engine, event triggers from ObjectQL hooks, flow executor, scheduled triggers.
+The `/automation` dispatcher domain gates on `isServiceServeable`, so a slot filled by
+a self-declared stub answers as an empty one.
---
-## 8–11. Communication Services ❌ Plugin Required
+## 8–11. Communication Services
-### 8. realtime — 6 methods
+### 8. realtime — 6 methods · `@objectstack/service-realtime`
`realtimeConnect`, `realtimeDisconnect`, `realtimeSubscribe`, `realtimeUnsubscribe`, `setPresence`, `getPresence`
-### 9. notification — 7 methods
+
+`service-realtime` is an **in-process pub/sub bus**, not an HTTP/WS surface. The
+dispatcher has no `/realtime` branch and no plugin mounts one, so `routes.realtime`
+is **never advertised** — an advertised route would 404 (ADR-0076 D12, #2462), and
+`features.websockets` is hardcoded `false` for the same reason. These six
+`RealtimeProtocol` members are declared and unrouted; when the service is registered
+both discovery builders report the slot `degraded` with a message saying the bus is
+in-process only and no HTTP/WS surface is mounted. Re-advertising waits on a real
+transport.
+
+
+### 9. notification — 7 methods · `@objectstack/service-messaging`
`registerDevice`, `unregisterDevice`, `getNotificationPreferences`, `updateNotificationPreferences`, `listNotifications`, `markNotificationsRead`, `markAllNotificationsRead`
-### 10. ai — 3 methods
-`aiNlq`, `aiSuggest`, `aiInsights`
+The slot name is `notification` (singular) and the package that fills it shares no word
+with it — which is why the remedy sentence is looked up in `CORE_SERVICE_PROVIDER`
+rather than derived from the slot name. The `/notifications` domain currently routes
+the inbox subset: `GET /notifications` → `listInbox`, `POST /notifications/read` →
+`markRead`, `POST /notifications/read/all` → `markAllRead`. All three are **optional**
+on `INotificationService` — a send-only provider (SMTP, Twilio, a webhook) fills the
+slot legitimately without an inbox, and each route probes its own method and answers
+501 when absent.
+
+### 10. ai — contract removed ❌
+~~`aiNlq`, `aiSuggest`, `aiInsights`~~
Removed. These three were **optional** protocol methods (`aiNlq?` …) that no
@@ -309,6 +389,8 @@ methods that called them were deleted outright (#3718). The AI service that
does exist (`service-ai`, Cloud/EE) serves a different surface — chat,
complete, models, conversations, agents — and `client.ai` was rebuilt against
those real routes, so this entry describes a contract that no longer exists.
+The `ai` slot still exists in `CoreServiceName`, but nothing in this repo fills
+it (`CORE_SERVICE_PROVIDER.ai` is `null`).
### 11. i18n — 3 methods
@@ -324,7 +406,7 @@ those real routes, so this entry describes a contract that no longer exists.
| Environment | Provider | Registration |
|:------------|:---------|:-------------|
| **Production** | `I18nServicePlugin` | File-based `FileI18nAdapter` loads JSON locale files from disk |
-| **In-memory fallback** | `@objectstack/core` (`createMemoryI18n`) | Registered by `AppPlugin` when the stack declares translation bundles and no i18n service is present; self-describes as `degraded` (ADR-0076 D12) |
+| **In-memory fallback** | `@objectstack/core` (`createMemoryI18n`) | Two paths, same factory: the kernel pre-injects it for any unfilled `core` slot (`CORE_FALLBACK_FACTORIES`), and `AppPlugin` registers it during `start` when the stack declares translation bundles and no i18n service is present. Self-describes as `degraded` (ADR-0076 D12) |
| **Development** | `DevPlugin` | Auto-wires `I18nServicePlugin` when the stack declares translations; otherwise the AppPlugin fallback above applies. DevPlugin registers no stub of its own (ADR-0115) |
```typescript
@@ -382,45 +464,48 @@ AppPlugin will:
| Service | Description |
|:--------|:------------|
-| **file-storage** | Unified upload/download/delete. Drivers: local FS, S3, MinIO. |
-| **search** | Full-text search. Drivers: in-memory, Elasticsearch, Meilisearch. |
-| **cache** | General-purpose cache. In-memory fallback; Redis via `@objectstack/service-cache`. |
-| **queue** | Message queue. In-memory fallback; durable DB-backed adapter via `@objectstack/service-queue` (no BullMQ/Redis adapter is shipped). |
-| **job** | Scheduled task execution. In-memory fallback; cron-based, concurrency control. |
+| **file-storage** | Unified upload/download/delete via `@objectstack/service-storage`, which mounts `/api/v1/storage` itself. Adapters: local FS and S3 (the S3 adapter's `endpoint` + path-style options cover S3-compatible services such as MinIO and R2). |
+| **search** | **Nothing ships.** `ISearchService` and the engine enum (`elasticsearch`, `meilisearch`, …) exist in `@objectstack/spec`, but no package implements the contract or registers the `search` slot, so `CORE_SERVICE_PROVIDER.search` is `null`. |
+| **cache** | General-purpose cache. In-memory fallback; memory or Redis adapter via `@objectstack/service-cache`. |
+| **queue** | Message queue. In-memory fallback; durable DB-backed adapter (`sys_job_queue`) via `@objectstack/service-queue` (no BullMQ/Redis adapter is shipped). |
+| **job** | Scheduled task execution via `@objectstack/service-job`. In-memory fallback; interval, cron, and DB-backed adapters with concurrency policy. |
---
-## Recommended Development Phases
-
-### Phase 1: Foundation (P0)
+## Roadmap Status
-| Plugin | Description |
-|:-------|:------------|
-| **plugin-auth** | Identity engine — JWT, sessions, user CRUD, roles |
-| **plugin-ui** | View CRUD — core low-code capability |
-| **plugin-cache** | General-purpose cache — performance infrastructure |
-| **Metadata DB** | Solve metadata persistence (SchemaRegistry → database) |
+The three-phase build-out this page used to propose has landed, under package names
+that are `service-*` more often than `plugin-*`. Use the real names — a remedy naming
+a package that cannot be installed is a dead end, which is why
+`CORE_SERVICE_PROVIDER` is CI-checked against the workspace.
-### Phase 2: Business Engine (P1)
+### Shipped
-| Plugin | Description |
+| Capability | Package |
|:-------|:------------|
-| **service-automation** | Flow orchestration, state-machine adjacent execution, approval-node pauses |
-| **plugin-automation** | Triggers + flow orchestration |
-| **plugin-i18n** | Internationalization |
-| **plugin-storage** | File storage (local + S3) |
-| **DB drivers** | PostgreSQL, MySQL, MongoDB, SQLite |
-
-### Phase 3: Advanced Capabilities (P2)
-
-| Plugin | Description |
+| Identity engine — sessions, JWT, user CRUD, OAuth2/OIDC, SCIM | `@objectstack/plugin-auth` |
+| RBAC, field-level permissions, RLS | `@objectstack/plugin-security` |
+| Record sharing + share links | `@objectstack/plugin-sharing` |
+| Metadata persistence (`sys_metadata`), HMR, cluster cache invalidation | `@objectstack/metadata` · `@objectstack/service-cluster` |
+| Cache — memory + Redis adapters | `@objectstack/service-cache` |
+| Flow orchestration, triggers, approval-node pauses | `@objectstack/service-automation` |
+| Internationalization | `@objectstack/service-i18n` |
+| File storage — local FS + S3 | `@objectstack/service-storage` |
+| DB drivers — PostgreSQL / MySQL / SQLite (Knex), MongoDB, SQLite-WASM, in-memory | `@objectstack/driver-sql` · `driver-mongodb` · `driver-sqlite-wasm` · `driver-memory` |
+| Notification engine + inbox | `@objectstack/service-messaging` |
+| Scheduled tasks — interval, cron, DB-backed | `@objectstack/service-job` |
+| Message queue — memory + durable DB-backed | `@objectstack/service-queue` |
+| Realtime pub/sub (in-process; **no HTTP/WS surface yet**) | `@objectstack/service-realtime` |
+
+### Still open
+
+| Slot | State |
|:-------|:------------|
-| **plugin-realtime** | WebSocket / SSE real-time push |
-| **plugin-notification** | Notification engine |
-| **plugin-ai** | Chat, completion, models, conversations (`service-ai`, Cloud/EE) |
-| **plugin-search** | Full-text search |
-| **plugin-job** | Scheduled tasks |
-| **plugin-queue** | Message queue |
+| **ui** | Nothing registers the slot. `ViewProtocol`'s five methods are declared and unrouted; view CRUD runs through `/api/v1/meta`, and `/api/v1/ui/view/:object` is served by the `protocol` service. |
+| **workflow** | Nothing ships. `WorkflowProtocol`'s three methods have no implementation and no consumer. |
+| **search** | Nothing ships. Contract and engine enum exist in `@objectstack/spec` only. |
+| **ai** | Nothing in this repo — `service-ai` (chat, completion, models, conversations) is Cloud/EE. |
+| **realtime transport** | The service exists but no WebSocket/SSE route is mounted, so `routes.realtime` is deliberately never advertised. |
---
@@ -428,11 +513,19 @@ AppPlugin will:
```typescript
import type { Plugin } from '@objectstack/core';
+import type { IDataEngine } from '@objectstack/spec/contracts';
let authService: AuthServiceImpl;
export const authPlugin: Plugin = {
- name: 'plugin-auth',
+ // Plugin names are reverse-domain (e.g. 'com.objectstack.auth'),
+ // not the npm package name.
+ name: 'com.example.auth',
+
+ // ADR-0116 — declare what init() needs and what it unconditionally
+ // registers, so a misordering is a named boot error instead of a crash.
+ requiresServices: ['data'],
+ providesServices: ['auth'],
async init(ctx) {
// Register the 'auth' CoreServiceName value
@@ -442,8 +535,11 @@ export const authPlugin: Plugin = {
},
async start(ctx) {
- const auth = ctx.getService('auth');
- await auth.onReady();
+ // Late-bind anything that must wait for every plugin to be up.
+ // (There is no `onReady()` on IAuthService — that is your own type's API.)
+ ctx.hook('kernel:ready', async () => {
+ await authService.connect();
+ });
},
// destroy() takes no arguments — capture what you need in module scope
@@ -454,9 +550,13 @@ export const authPlugin: Plugin = {
```
When a plugin registers a service, the discovery endpoint automatically updates:
-- `services.auth.enabled` → `true`, `status` → `"available"`, `route` → `"/api/v1/auth"`
+- `services.auth.enabled` → `true`, `status` → `"available"`, `handlerReady` → `true`,
+ `route` → `"/api/v1/auth"` (unless the instance self-declares `stub`/`degraded` via
+ `__serviceInfo`, which is reported verbatim instead)
- `routes.auth` → `"/api/v1/auth"` appears in routes
-- `features` capability flags update accordingly
+- `features` flags follow for the slots that have one — `search`, `files`,
+ `analytics`, `ai`, `workflow`, `notifications`, `i18n` (`websockets` is hardcoded
+ `false`; there is no `features.auth`)
---
diff --git a/content/docs/kernel/services.mdx b/content/docs/kernel/services.mdx
index 822e9c9d9c..60efcd826f 100644
--- a/content/docs/kernel/services.mdx
+++ b/content/docs/kernel/services.mdx
@@ -25,14 +25,23 @@ export const myPlugin: Plugin = {
async init(ctx) {
// Register a service with a concrete implementation
- const settings = ctx.getService('settings');
ctx.registerService('cache', new RedisCacheProvider({
- url: settings.get('redis.url'),
+ url: 'redis://localhost:6379',
}));
},
};
```
+Take init-time configuration from the plugin's own options (as
+`CacheServicePlugin` and `createApiRegistryPlugin()` do) rather than from the
+`settings` service: that service is an async, namespaced resolver —
+`await settings.get(namespace, key)` returns a `{ value, source, locked, … }`
+envelope, not a synchronous config bag keyed by dotted paths — and it is only
+guaranteed to be registered during your `init` if you declare
+`requiresServices: ['settings']`. Settings-driven config usually belongs in
+`start` instead, the way `EmailServicePlugin` binds the `mail` namespace there
+and re-applies it on every change.
+
### Factory Registration (Lazy)
Use `registerServiceFactory` when the service requires async initialization.
@@ -44,8 +53,8 @@ context (and an optional scope id) and is wrapped in lifecycle management:
import { ServiceLifecycle } from '@objectstack/core';
ctx.registerServiceFactory('data', async (ctx) => {
- const settings = ctx.getService('settings');
- const pool = await createPool(settings.get('database'));
+ const pool = await createPool({ url: 'postgres://localhost:5432/app' });
+ ctx.logger.info('Postgres pool ready');
return new PostgresDriver(pool);
}, ServiceLifecycle.SINGLETON);
```
@@ -79,28 +88,36 @@ if (ctx.getServices().has('search')) {
}
```
-**Presence is not capability.** A registered service may be a stub or a degraded
-fallback — dev-mode fakes and the kernel's own in-memory fallbacks both occupy
-real slots. They say so with the `__serviceInfo` descriptor (ADR-0076 D12), and
-consumers are expected to read it rather than trust mere registration:
+**Presence is not capability.** A registered service may be a degraded fallback
+or an outright stub — the kernel auto-injects in-memory fallbacks into the
+`metadata`, `cache`, `queue`, `job`, and `i18n` slots when no plugin provides
+them, and those occupy the real slot. They say so with the `__serviceInfo`
+descriptor (ADR-0076 D12), and consumers are expected to read it rather than
+trust mere registration:
```typescript
import { readServiceSelfInfo } from '@objectstack/spec/api';
-const ai = ctx.getServices().get('ai');
-const self = readServiceSelfInfo(ai); // undefined ⇒ a real implementation
+const cache = ctx.getServices().get('cache');
+const self = readServiceSelfInfo(cache); // undefined ⇒ a real implementation
-if (ai && self?.status !== 'stub') {
+if (cache && self?.status !== 'stub') {
// Safe: a real implementation, or a `degraded` one that genuinely works.
} else if (self) {
- ctx.logger.info(`ai is a stub — ${self.message}`);
+ ctx.logger.info(`cache is a stub — ${self.message}`);
}
```
-`status: 'degraded'` means real work with reduced capability (the in-memory
-cache, the in-memory search index); `status: 'stub'` means the answer is
-fabricated and must not be used for real work. `handlerReady: false`
-additionally means no HTTP handler serves it.
+`status: 'degraded'` means real work with reduced capability (all five kernel
+fallbacks above declare it); `status: 'stub'` means the answer is fabricated and
+must not be used for real work. `handlerReady: false` additionally means no HTTP
+handler serves it.
+
+Dev mode adds no fakes here: `@objectstack/plugin-dev` registers no
+implementations of its own — it composes the same real plugins (ObjectQL, the
+in-memory driver, auth, security, the HTTP server, REST) you would compose in
+production — so a slot no plugin fills is empty in dev exactly as in production
+(ADR-0115).
## Standard Services
@@ -112,7 +129,7 @@ The core ecosystem defines several standard service contracts:
| `data` | `IDataEngine` | `@objectstack/objectql` (drivers implement `IDataDriver`) |
| `auth` | `IAuthService` | `plugin-auth` |
| `api-registry` | `ApiRegistry` | `@objectstack/core` |
-| `cache` | `ICacheService` | Redis, Memcached, or in-memory |
+| `cache` | `ICacheService` | `@objectstack/service-cache` (memory adapter; its Redis adapter is still a skeleton that throws) — otherwise the kernel's in-memory fallback |
| `lifecycle` | `LifecycleService` (`@objectstack/objectql`) | Registered by `ObjectQLPlugin` — enforces object `lifecycle` declarations (ADR-0057 retention/rotation/archival); call `sweep()` for an on-demand pass |
The logger is not a registered service — it is exposed directly as `ctx.logger`
@@ -148,21 +165,25 @@ Services follow the plugin lifecycle:
3. **`destroy`** — Clean up resources (close connections, flush buffers)
```typescript
+let pool: Pool | undefined;
+
export const dbPlugin: Plugin = {
name: 'database',
async init(ctx) {
- const settings = ctx.getService('settings');
- const pool = await createPool(settings.get('database'));
+ pool = await createPool({ url: 'postgres://localhost:5432/app' });
ctx.registerService('data', new PostgresDriver(pool));
- this.db = pool;
},
async destroy() {
- await this.db.close(); // Clean shutdown
+ await pool?.close(); // Clean shutdown
},
};
```
+`destroy()` receives no context, so keep any handle you need to close in the
+plugin's own scope — a class field, or a module-level binding as above. `Plugin`
+declares no state slot, so stashing it on `this` does not typecheck.
+
The `Plugin` interface defines `init`, `start?`, and `destroy?` — there is no
`stop` hook, and `destroy()` takes no arguments.
diff --git a/content/docs/permissions/authentication.mdx b/content/docs/permissions/authentication.mdx
index 223ffb3430..ae8d9f2655 100644
--- a/content/docs/permissions/authentication.mdx
+++ b/content/docs/permissions/authentication.mdx
@@ -91,7 +91,8 @@ Install the plugin in your ObjectStack project:
pnpm add @objectstack/plugin-auth
```
-The plugin requires Better-Auth as a peer dependency, which will be automatically installed.
+Better-Auth is a direct runtime dependency of the plugin, not a peer dependency,
+so the command above installs it for you — you never add `better-auth` yourself.
---
@@ -376,8 +377,10 @@ once (never overwriting your edits) and can be changed under Setup →
Notification Templates. The locale follows the deployment default
(`localization.locale` setting) with a `zh-CN → zh → en` fallback chain;
holes are `{{code}}`, `{{appName}}`, `{{minutes}}` (OTP) and `{{appName}}`,
-`{{baseUrl}}` (invitation). Template lookups are best-effort — an outage
-falls back to the built-in text and never blocks an OTP send.
+`{{loginUrl}}` (invitation — `{{baseUrl}}`, the bare origin, is still
+interpolated for tenant templates written against the older text). Template
+lookups are best-effort — an outage falls back to the built-in text and never
+blocks an OTP send.
---
@@ -799,9 +802,9 @@ every row through better-auth so the accounts are login-capable:
link, or a password-reset link) and the Console detects the missing password
(`hasLocalPassword()`) and offers set-initial-password.
- `mode: 'insert' | 'upsert'` with `matchBy: 'email' | 'phone'`. Upsert
- updates only touch profile fields (`name`, `phone_number`, `role`) — a
- re-imported file can never modify an existing user's email or reset their
- password.
+ updates only touch profile fields (`name`, `image`, `phone_number`, `role`)
+ — a re-imported file can never modify an existing user's email or reset
+ their password.
- Synchronous only, at most 500 rows per request (password hashing is
CPU-bound); split larger files into batches. Imports are not undoable.
@@ -1050,7 +1053,7 @@ AuthPlugin is designed to work in **both** server and MSW/mock (browser-only) en
### How It Works
- **Server mode** (HonoServerPlugin active): AuthPlugin registers HTTP routes at `/api/v1/auth/*` and forwards all requests to better-auth.
-- **MSW/mock mode** (no HTTP server): AuthPlugin gracefully skips route registration but still registers the `auth` service. The `HttpDispatcher` provides mock fallback responses for core auth endpoints.
+- **MSW/mock mode** (no HTTP server): AuthPlugin gracefully skips route registration but still registers the `auth` service. The dispatcher's `/auth` domain then bridges straight to that service's `handleRequest()` — it has no mock fallback of its own, and answers `501` when the `auth` slot is empty (see below).
### Minimal Configuration for Mock Mode
diff --git a/content/docs/plugins/index.mdx b/content/docs/plugins/index.mdx
index ea65d3ae3b..a694603244 100644
--- a/content/docs/plugins/index.mdx
+++ b/content/docs/plugins/index.mdx
@@ -122,7 +122,7 @@ export default defineStack({
myPlugin,
],
- // Development-only plugins (loaded only with `os dev`)
+ // Development-only plugins (loaded only with `os dev` / `os serve --dev`)
devPlugins: [
],
});
@@ -199,8 +199,8 @@ Security features including field-level and row-level security.
- Middleware-based security
### `@objectstack/driver-memory`
-In-memory data driver for development and testing.
-- Auto-registered in dev mode if no driver is configured
+In-memory (mingo) data driver for development and testing.
+- Opt in explicitly with `OS_DATABASE_DRIVER=memory`, `--database-driver memory`, or a `memory://` URL — when no driver is configured the dev default is SQLite, not this driver
---
@@ -211,11 +211,23 @@ The `os serve` and `os dev` commands automatically detect and load plugins:
| Condition | Auto-loaded Plugin |
|:----------|:-------------------|
| `objects` defined, no ObjectQL | `@objectstack/objectql` |
-| Dev mode, no driver, has objects | `@objectstack/driver-memory` |
-| `objects`, `manifest`, or `apps` defined | `AppPlugin` (runtime) |
+| `objects` defined, no driver | `DefaultDatasourcePlugin` (runtime) — driver kind from `OS_DATABASE_DRIVER` / `OS_DATABASE_URL`, defaulting to SQLite |
+| `objects`, `manifest`, `apps`, `flows`, or `apis` defined | `AppPlugin` (runtime) |
| Server not disabled | `@objectstack/plugin-hono-server` |
| Server enabled | `@objectstack/rest` (REST API) |
+
+**Which SQLite default you get depends on the boot path.** A bare `defineStack()`
+config — one whose `plugins` array instantiates nothing — boots through
+`createStandaloneStack`, which anchors a SQLite *file* under the project
+(`.objectstack/data/standalone.db`) in development **and** production. A host config
+that already instantiates plugins takes `serve`'s own fallback instead: with no
+`OS_DATABASE_URL` / `OS_DATABASE_DRIVER` set it declares SQLite `:memory:` in dev and
+registers **no** datasource in production, so a missing driver surfaces loudly
+downstream. `os dev` sidesteps both defaults by exporting
+`OS_DATABASE_URL=file:/.objectstack/data/dev.db`.
+
+
This means a minimal config like this already works:
```typescript
@@ -225,7 +237,9 @@ export default defineStack({
});
```
-Running `os dev` will auto-register ObjectQL, MemoryDriver, AppPlugin, HonoServer, and REST API.
+Running `os dev` will auto-register ObjectQL, the default datasource (a
+project-anchored SQLite file at `.objectstack/data/dev.db`), AppPlugin,
+HonoServer, and the REST API.
---
@@ -312,7 +326,7 @@ const slowPlugin: PluginMetadata = {
Plugins can extend the ObjectStack CLI (`os`) with custom commands. This enables third-party packages — such as marketplace tools, deployment utilities, or domain-specific workflows — to register new top-level subcommands.
-The CLI is built on [oclif](https://oclif.io). Command extension is handled entirely by oclif's plugin system: a plugin ships oclif `Command` classes, declares oclif config in its `package.json`, and is installed into the CLI with `os plugins install `. The host project's `objectstack.config.ts` does **not** determine CLI command availability.
+The CLI is built on [oclif](https://oclif.io). Command extension is handled entirely by oclif's plugin system: a plugin ships oclif `Command` classes, declares oclif config in its `package.json`, and is loaded by the `os` binary as an oclif plugin. The host project's `objectstack.config.ts` does **not** determine CLI command availability.
#### Step 1: Add oclif Config to `package.json`
@@ -360,17 +374,21 @@ export default class MarketplaceSearch extends Command {
}
```
-#### Step 3: Install the Plugin into the CLI
+#### Step 3: Load the Plugin into the CLI
-Users add the plugin to their `os` installation with oclif's plugin manager:
-
-```bash
-os plugins install @acme/plugin-marketplace
-```
+
+**`os plugins install` is not available today.** `@objectstack/cli`'s `package.json`
+lists `@oclif/plugin-plugins` under `oclif.plugins`, but the package sits in
+`devDependencies` — and oclif's core-plugin loader only matches names that appear in
+`dependencies`. The result is that neither `os plugins …` nor `os help` is a
+registered command; `os --help` shows no `plugins` topic. Until that is fixed, load a
+CLI extension by building an `os` distribution that lists the package in **both**
+`oclif.plugins` and `dependencies`.
+
#### Using the Extended CLI
-Once installed, the new commands appear in `os --help` and can be invoked directly:
+Once loaded, the new commands appear in `os --help` and can be invoked directly:
```bash
# List available commands (includes plugin commands)
@@ -380,12 +398,6 @@ os --help
os marketplace search "crm"
```
-To remove a plugin command, uninstall it:
-
-```bash
-os plugins uninstall @acme/plugin-marketplace
-```
-
---
## Directory Structure Convention
@@ -407,12 +419,15 @@ plugin-my-feature/
│ ├── feature.view.ts # View definition
│ └── index.ts # Barrel export
└── services/
- └── feature.service.ts # Service implementation
+ └── feature-service.ts # Service implementation (plain module)
```
**Naming Rules**:
- Domain directories: `snake_case`
-- File suffixes: `.object.ts`, `.view.ts`, `.flow.ts`, `.service.ts`, etc.
+- Semantic metadata suffixes: `.object.ts`, `.field.ts`, `.view.ts`, `.page.ts`,
+ `.dashboard.ts`, `.flow.ts`, `.app.ts`. ADR-0088 retired `.trigger.ts`,
+ `.router.ts`, `.function.ts`, and `.service.ts` — plain source modules
+ (services, helpers) simply carry no semantic suffix.
- Entry point: `index.ts` in every module
---
diff --git a/content/docs/plugins/packages.mdx b/content/docs/plugins/packages.mdx
index b62b0884d1..cfe69561a2 100644
--- a/content/docs/plugins/packages.mdx
+++ b/content/docs/plugins/packages.mdx
@@ -26,7 +26,7 @@ ObjectStack is organized into **72 package manifests** across multiple categorie
**The Constitution** — Protocol schemas, types, and constants for the entire ObjectStack ecosystem.
- **Purpose**: Zod-first schema definitions for all 15 protocol domains
-- **Exports**: Builder functions (`defineStack`, `defineView`, `defineApp`, `defineFlow`, `defineAgent`, `defineTool`, `defineSkill`) plus `ObjectSchema.create()` for objects. Protocol namespaces (Data, UI, System, Automation, AI, API, Identity, Security, Kernel, Cloud, QA, Contracts, Integration, Studio, Shared) are not re-exported from the top-level entry for tree-shaking reasons — import them from subpaths such as `@objectstack/spec/data` and `@objectstack/spec/ui`.
+- **Exports**: Builder functions (`defineStack`, `defineView`, `defineApp`, `defineFlow`, `defineAgent`, `defineTool`, `defineSkill`) from the root entry, plus `ObjectSchema.create()` for objects from the `@objectstack/spec/data` subpath. Protocol namespaces (Data, UI, System, Automation, AI, API, Identity, Security, Kernel, Cloud, QA, Contracts, Integration, Studio, Shared) are not re-exported from the top-level entry for tree-shaking reasons — import them from subpaths such as `@objectstack/spec/data` and `@objectstack/spec/ui`.
- **When to use**: Import types, schemas, and builder functions when authoring metadata.
- **Documentation**: [Protocol Reference](/docs/references)
@@ -98,9 +98,10 @@ const kernel = new ObjectKernel();
### @objectstack/platform-objects
-**Platform Objects Library** — The canonical set of `sys_*` objects shipped with every ObjectStack runtime (users, sessions, approvals, sharing, audit, …).
+**Platform Objects Library** — The canonical set of `sys_*` objects shipped with every ObjectStack runtime (users, sessions, organizations, teams, jobs, notifications, email, settings, secrets, …).
-- **Purpose**: Standard system tables and their metadata, so apps don't redefine identity, audit, or approvals
+- **Purpose**: Standard system tables and their metadata, so apps don't redefine identity, jobs, or settings
+- **Not here**: per ADR-0029 each domain plugin owns its own objects — `sys_audit_log` / `sys_activity` / `sys_comment` in `plugin-audit`, `sys_record_share` in `plugin-sharing`, `sys_approval_request` / `sys_approval_action` in `plugin-approvals`, the RBAC objects in `plugin-security`, `sys_presence` in `service-realtime`, and `sys_metadata*` in `@objectstack/metadata-core`
- **When to use**: Always — bundled into the runtime
---
diff --git a/content/docs/protocol/kernel/index.mdx b/content/docs/protocol/kernel/index.mdx
index 47bc2c5855..1321400e59 100644
--- a/content/docs/protocol/kernel/index.mdx
+++ b/content/docs/protocol/kernel/index.mdx
@@ -176,12 +176,16 @@ const apiKey =
**Kernel Approach:**
```typescript
-// Declarative merge strategy
-const config = context.config.resolve('stripe.apiKey', {
- sources: ['environment', 'tenant', 'plugin', 'default'],
- required: true
+// One resolver, one precedence order — the `settings` service
+// (SettingsService, registered by SettingsServicePlugin)
+const settings = ctx.getService('settings');
+const { value, source, locked } = await settings.get('stripe', 'apiKey', {
+ tenantId,
+ userId,
});
-// Kernel merges in precedence order, validates, throws clear error if missing
+// Cascade: OS_STRIPE_APIKEY (env) > global > tenant > user > registered default.
+// `source` names the winning scope; `locked: true` means a higher scope pinned it
+// (a write to a locked key throws SettingsLockedError).
```
**Business Value:** Configuration errors dropped 90% after adopting Kernel. New engineers can understand config logic in 5 minutes instead of 5 hours.
@@ -214,7 +218,7 @@ const config = context.config.resolve('stripe.apiKey', {
**Kernel Solution:**
- **Package Manifest:** The `manifest` block in `objectstack.config.ts` (compiled to `objectstack.json`) declares what a package provides and needs
- **Dependency Resolution:** Automatic validation that plugin versions are compatible with core platform
-- **Lifecycle:** every plugin implements `init` / `start` / `destroy`, driven by `kernel.bootstrap()` and shutdown
+- **Lifecycle:** every plugin implements a **required** `init` hook plus optional `start` / `destroy` hooks, driven by `kernel.bootstrap()` and shutdown
- **Sandboxing:** Plugins can't access each other's data or crash each other
**Results:**
@@ -234,7 +238,7 @@ const config = context.config.resolve('stripe.apiKey', {
- **i18n Standard:** Translation bundles with fallback chains (`de-AT` → `de` → `en`)
- **Region Config:** Configuration profiles per region (defaults + region overrides)
- **Plugin System:** Regional plugins (SEPA plugin for EU, ACH plugin for US)
-- **Locale Resolution:** Automatic detection from user preferences, IP geolocation, or explicit selection
+- **Locale Resolution:** Automatic detection from the user's localization settings or the request's `Accept-Language` header, with explicit selection as an override
**Results:**
- Launched in 12 countries in 4 months (previously 18-month estimate)
@@ -265,57 +269,57 @@ await kernel.use(new ObjectQLPlugin());
// Phase 3 — application metadata (objects, views, apps, flows, agents…)
await kernel.use(new AppPlugin(stack));
-// Phase 4 — host (HTTP / Studio / MCP server, optional)
+// Phase 4 — host (HTTP via HonoServerPlugin, MCP via MCPServerPlugin; optional)
// await kernel.use(new HonoServerPlugin({ port: 3000 }));
// Phase 5 — start everything
await kernel.bootstrap();
```
-The kernel validates each plugin at `use()`, then `bootstrap()` runs every
-plugin's `init` in registration order followed by every plugin's `start`, and
-exposes the resulting services through DI.
+The kernel validates each plugin's structure and version compatibility at
+`use()`, then `bootstrap()` topologically sorts the registry — each plugin's
+declared `dependencies` are hoisted ahead of it, registration order breaking
+ties — and runs every plugin's `init` (kernel Phase 1) followed by every
+plugin's `start` (kernel Phase 2), exposing the resulting services through DI.
+It then fires `kernel:ready`, `kernel:bootstrapped`, and `kernel:listening` in
+that order.
### 2. Request Handling
```typescript
// Incoming API request: GET /api/v1/data/account/123
-async function handleRequest(req) {
- // Kernel provides context
- const context = await Kernel.createContext({
- tenantId: req.headers['x-tenant-id'],
- userId: req.user.id,
- locale: req.headers['accept-language'],
- });
-
- // Kernel resolves configuration
- const config = context.config.resolve();
-
- // ObjectQL executes query
- const account = await ObjectQL.findById('account', '123', {
- context, // Kernel injects permissions, audit, etc.
- });
-
- // ObjectUI renders response
- const view = await ObjectUI.render('account_detail', {
- record: account,
- locale: context.locale, // Kernel provides i18n
- });
-
- // Kernel logs audit trail
- context.audit.log('account.view', { accountId: '123' });
-
- return view;
-}
+//
+// The transport plugin (HonoServerPlugin) hands the raw request to the REST
+// layer, which resolves ONE identity envelope and threads it into the engine:
+
+// 1. Identity — resolveExecutionContext() reads the better-auth session (or
+// API key), aggregates positions/permission sets/RLS membership, and layers
+// locale + timezone on top. It always resolves; anonymous yields
+// `{ isSystem: false, positions: [], permissions: [] }`.
+const context = await resolveExecutionContext({ getService, getQl, request });
+// → ExecutionContext { userId, tenantId, locale, timezone, positions,
+// permissions, isSystem, ... }
+
+// 2. Data — the context rides along as `context`, on the query itself or on
+// the engine's trailing options argument (the engine merges both). It is
+// what permissions and RLS are enforced against.
+const ql = ctx.getService('objectql');
+const account = await ql.findOne('account', { where: { id: '123' }, context });
```
+Auditing is not called by hand at the request boundary: `AuditPlugin` subscribes
+to the engine's data hooks and writes audit records as a side effect of the
+operation.
+
### 3. Package Installation
```typescript
-// Install a package: os package install @vendor/salesforce-sync
+// Install a package: os package install com.vendor.salesforce-sync
+// (the argument is a reverse-domain manifest id, or a path to a compiled
+// artifact JSON for an air-gapped install)
//
// A package is METADATA — no package code executes at install time.
// What actually happens:
//
-// 1. Validate — manifest shape, engines.protocol compatibility (ADR-0087),
+// 1. Validate — manifest shape, engines.protocol compatibility (ADR-0025 §3.2),
// artifact signature.
// 2. Register — registry.installPackage() gates the namespace and records
// the InstalledPackage; the durable copy lands in `sys_packages` and is
@@ -344,10 +348,11 @@ permissions.grant('admin', 'accounts', 'read');
**Good (Declarative):**
```yaml
# objectstack.config.ts — declared via defineStack({ ... })
+# `objects` is a list; each object's `fields` is a MAP keyed by snake_case name
objects:
- name: account
fields:
- - name: name
+ name:
type: text
# PermissionSet grants use allow* flags (not role/object/access)
permissions:
@@ -388,18 +393,37 @@ packages/plugins/plugin-slack-integration/
README.md
```
+
+ The scaffold still emits an `initialize` method. The kernel's plugin contract
+ only invokes `init` / `start` / `destroy`, and `init` is **required** — rename
+ `initialize` to `init` in the generated `src/index.ts` or `kernel.use()`
+ rejects the plugin outright with
+ `Failed to load plugin: slack-integration - Plugin init function is required`.
+
+
### Configuration Management
```typescript
-// Define plugin configuration schema
-export const configSchema = z.object({
- apiKey: z.string().describe('Slack API Key'),
- channel: z.string().default('#general'),
- enabled: z.boolean().default(true),
-});
-
-// Access in plugin code
-const config = context.config.get('slack', configSchema);
-// Kernel validates against schema, throws clear error if invalid
+// A plugin object may carry a Zod `configSchema` describing its settings.
+// NOTE: the kernel RECORDS the schema but does not enforce it yet — `use()`
+// takes no config argument, so the loader has nothing to parse and logs
+// "config validation postponed" instead of running the schema. Treat
+// `configSchema` as a declaration of shape, and validate values you actually
+// depend on yourself.
+export const slackPlugin: Plugin = {
+ name: 'slack-integration',
+ version: '0.1.0',
+ configSchema: z.object({
+ apiKey: z.string().describe('Slack API Key'),
+ channel: z.string().default('#general'),
+ enabled: z.boolean().default(true),
+ }),
+ async init(ctx) {
+ // Runtime-resolved values come from the `settings` service, not from a
+ // `config` object on the context — there is no `ctx.config`.
+ const settings = ctx.getService('settings');
+ const { value: channel } = await settings.get('slack', 'channel', {});
+ },
+};
```
### Internationalization
@@ -417,9 +441,12 @@ const config = context.config.get('slack', configSchema);
"slack.error.invalid_channel": "Kanal {{channel}} nicht gefunden"
}
-// Use in code
-const message = context.i18n.t('slack.button.send');
-// Kernel automatically uses user's locale
+// Use in code — the i18n service is resolved from the registry, and the
+// locale is an explicit argument (take it from the ExecutionContext).
+const i18n = ctx.getService('i18n');
+const message = i18n.t('slack.button.send', context.locale);
+// t(key, locale, params?) interpolates {{params}} and falls back to the
+// configured fallback locale, returning the key itself when nothing matches.
```
## Best Practices
@@ -429,7 +456,7 @@ const message = context.i18n.t('slack.button.send');
**Why:** Keeps core stable. Plugins evolve independently. Customers pay for only what they use.
-**Example:** Don't build email sending into Kernel core. Create `@objectstack/email` plugin. Customers who don't send emails don't load the plugin (smaller memory footprint, faster boot).
+**Example:** Don't build email sending into Kernel core. Ship it as the `@objectstack/plugin-email` package. Customers who don't send emails don't load the plugin (smaller memory footprint, faster boot).
### 2. Configuration Over Code
**Principle:** Behavior should be configurable without code changes.
@@ -443,7 +470,7 @@ const message = context.i18n.t('slack.button.send');
**Why:** Easier to add multi-tenancy later = rebuild entire system.
-**Example:** Don't store config in global variables. Use `context.config.get()` which automatically scopes to current tenant.
+**Example:** Don't store config in global variables. Read through the `settings` service — `settings.get(namespace, key, { tenantId, userId })` walks the global → tenant → user cascade for you.
### 4. Version Everything
**Principle:** All artifacts (plugins, schemas, configs) have semantic versions.
@@ -457,7 +484,14 @@ const message = context.i18n.t('slack.button.send');
**Why:** Catch errors before customers see them.
-**Example:** Plugin requires `apiKey` config. If missing, Kernel throws error during boot with clear message: "Plugin @vendor/slack requires config key 'slack.apiKey' (not found in environment, tenant, or default config)".
+**Example:** Wiring, not config, is what the kernel currently enforces at boot. A plugin that declares a dependency the kernel never received fails `bootstrap()` outright — `[Kernel] Dependency 'com.objectstack.engine.objectql' not found for plugin 'com.objectstack.audit'` — and a dependency cycle throws `[Kernel] Circular dependency detected: `. Boot stops there instead of a request failing later.
+
+
+ Plugin `configSchema` is **not** part of this fail-fast path yet. The loader
+ stores the schema and postpones the check, so a missing or malformed value in
+ a plugin's config will not stop boot today. Validate config you depend on in
+ your own `init`.
+
## Comparison: Kernel vs Alternatives
diff --git a/content/docs/protocol/kernel/plugin-spec.mdx b/content/docs/protocol/kernel/plugin-spec.mdx
index 1c420e7d3b..2ef4d30c9c 100644
--- a/content/docs/protocol/kernel/plugin-spec.mdx
+++ b/content/docs/protocol/kernel/plugin-spec.mdx
@@ -9,8 +9,8 @@ import { Package, FileCode, Folder, Box, Link2, Shield, Settings } from 'lucide-
**Protocol spec.** This page describes the **target** plugin packaging
-specification — `plugin.manifest.ts`, semantic-version dependency resolution,
-provider/consumer contracts, lifecycle hooks. The current runtime implements
+specification — an ergonomic manifest wrapper, semantic-version dependency
+resolution, provider/consumer contracts. The current runtime implements
plugins as classes/objects matching the `Plugin` interface from
`@objectstack/core`, and manifests as plain objects matching
`ManifestSchema` from `@objectstack/spec/kernel`. There is no `definePlugin()`
@@ -28,17 +28,23 @@ Every plugin **must** have a manifest file that declares its identity, dependenc
```
my-plugin/
- plugin.manifest.ts ← TypeScript (recommended)
- plugin.manifest.yml ← YAML (alternative)
- plugin.manifest.json ← JSON (alternative)
+ src/manifest.ts ← TypeScript source: ManifestSchema.parse({ … })
+ objectstack.config.ts ← stack entry: defineStack({ manifest, objects, … })
+ objectstack.plugin.json ← distributable manifest read by `os plugin build`
```
-**Recommendation:** Use TypeScript manifest for **type safety** and **validation**. ObjectStack auto-generates JSON schema from TypeScript.
+There is no `plugin.manifest.ts` / `.yml` / `.json` — that filename appears nowhere in
+the implementation. The shape is always `ManifestSchema` from `@objectstack/spec/kernel`.
+`os plugin build` reads the JSON form (`objectstack.plugin.json`, exported as
+`MANIFEST_FILENAME` from `packages/cli/src/utils/osplugin.ts`) and validates it with
+`ManifestSchema.safeParse`; TypeScript authoring runs the same object through
+`ManifestSchema.parse(...)` and hands it to `defineStack({ manifest })`.
### Manifest Schema
```typescript
-// plugin.manifest.ts
+// src/manifest.ts — proposed authoring wrapper (see the callout below for the
+// fields ManifestSchema actually declares today)
import { definePlugin } from '@objectstack/core';
export default definePlugin({
@@ -54,7 +60,7 @@ export default definePlugin({
// DEPENDENCIES
dependencies: {
'@objectstack/core': '^2.0.0',
- '@mycompany/base': '>=1.0.0 <2.0.0',
+ '@mycompany/base': '1.0.0 - 2.0.0',
},
// OPTIONAL DEPENDENCIES (plugin works without these)
@@ -102,16 +108,14 @@ export default definePlugin({
// never declared it and nothing ever loaded those files (#4212). Runtime
// behaviour lives on the plugin class (`init`/`start`/`destroy`, below).
- // PERMISSIONS
+ // PERMISSIONS — proposed shape only. The real block has no `system` or
+ // `objects` key; see "Permissions" below for what ManifestSchema accepts.
permissions: {
- // System capabilities plugin requires
system: [
- 'network.http', // Make HTTP requests
- 'storage.database', // Database access
- 'storage.cache', // Redis/cache access
+ 'network.http',
+ 'storage.database',
+ 'storage.cache',
],
-
- // Objects plugin creates/manages
objects: [
'account',
'contact',
@@ -163,13 +167,32 @@ export default definePlugin({
});
```
+
+**What `ManifestSchema` actually declares today**
+(`packages/spec/src/kernel/manifest.zod.ts`): `id`, `version`, `type` and `name` are
+required; the optional fields are `namespace`, `defaultDatasource`, `scope`,
+`description`, `permissions`, `objects`, `datasources`, `dependencies`,
+`configuration`, `contributes`, `data`, `capabilities`, `extensions`,
+`navigationContributions`, `loading`, `engine`, `engines`, `runtime`, `packaging`
+and `integrity`.
+
+The `displayName` / `author` / `license` / `homepage` / `optionalDependencies` /
+`peerDependencies` / `metadata` / `config` / `marketplace` keys above are
+**proposal-only** — the schema declares none of them. Notably: compatibility ranges
+live in `engines: { platform, protocol }` (or the legacy `engine: { objectstack }`),
+not `marketplace.compatibility`; config defaults live in
+`configuration: { title, properties }` (a simplified JSON-Schema map with a per-key
+`secret` flag), not `config.defaults` / `config.secrets`; and metadata globs are the
+top-level `objects` / `datasources` arrays, not a `metadata` block.
+
+
## Directory Structure
A well-organized plugin follows this **standard structure**:
```
@mycompany/crm/
-├── plugin.manifest.ts # Plugin manifest (required)
+├── objectstack.plugin.json # Plugin manifest — ManifestSchema (required)
├── package.json # NPM package metadata
├── README.md # Documentation
├── CHANGELOG.md # Version history
@@ -202,12 +225,9 @@ A well-organized plugin follows this **standard structure**:
│ ├── jobs/ # Background jobs
│ │ └── cleanup.job.ts
│ │
-│ ├── config.schema.ts # Configuration schema (Zod)
-│ │
-│ └── lifecycle/ # Lifecycle hooks
-│ ├── install.ts
-│ ├── boot.ts
-│ └── upgrade.ts
+│ └── config.schema.ts # Configuration schema (Zod)
+│ # (no src/lifecycle/ — the install/boot/upgrade
+│ # hook family was retired, #4212)
│
├── i18n/ # Translations
│ ├── en.json
@@ -397,13 +417,19 @@ export type PluginConfig = z.infer;
### Dependency Types
+
+Only `dependencies` exists in `ManifestSchema` today (a
+`Record`). `optionalDependencies` and `peerDependencies`
+below are **proposal-only** — the schema declares neither, so nothing resolves them.
+
+
#### 1. **Dependencies (Required)**
Plugin **cannot function** without these.
```typescript
dependencies: {
'@objectstack/core': '^2.0.0',
- '@mycompany/base': '>=1.0.0 <2.0.0',
+ '@mycompany/base': '1.0.0 - 2.0.0',
}
```
@@ -415,8 +441,10 @@ optionalDependencies: {
'@vendor/email': '^3.0.0',
}
-// In plugin code
-if (context.plugins.isInstalled('@vendor/email')) {
+// In plugin code — `PluginContext` (packages/core/src/types.ts) has no
+// `plugins` namespace and no `isInstalled()`. Probe the service registry:
+if (ctx.getServices().has('email')) {
+ const email = ctx.getService('email');
await email.send({ to: contact.email, subject: 'Welcome' });
}
```
@@ -434,7 +462,8 @@ Use for large dependencies (React, Vue) that should be shared across plugins.
### Version Constraints
-ObjectStack uses **semantic versioning (semver)** with these operators:
+ObjectStack uses **semantic versioning (semver)**. `SemanticVersionManager.satisfies()`
+(`packages/core/src/dependency-resolver.ts`) accepts exactly these forms:
```typescript
dependencies: {
@@ -447,11 +476,17 @@ dependencies: {
// Minor updates allowed (1.x.x)
'plugin-c': '^1.0.0',
- // Version range
- 'plugin-d': '>=1.0.0 <2.0.0',
+ // Single bound: >= , > , <= , <
+ 'plugin-d': '>=1.0.0',
+
+ // Inclusive range — the hyphen form is the ONLY two-bound form the
+ // resolver parses. A space-separated `>=1.0.0 <2.0.0` is not supported:
+ // `parse()` anchors its semver regex, so the trailing bound throws
+ // "Invalid semantic version".
+ 'plugin-e': '1.0.0 - 2.0.0',
- // Latest version
- 'plugin-e': '*', // ⚠️ Not recommended for production
+ // Latest version (`*` and `latest` both match everything)
+ 'plugin-f': '*', // ⚠️ Not recommended for production
}
```
@@ -472,20 +507,30 @@ ObjectStack builds a **dependency graph** and loads plugins in **topological ord
**Load Order:** core → base → crm → sales
**Conflict Resolution:**
-If two plugins require incompatible versions of the same dependency, installation fails with error:
+When two plugins require incompatible versions of the same dependency,
+`DependencyResolver.detectConflicts()` reports it as a structured
+`DependencyConflict` record rather than a formatted CLI banner:
+```typescript
+{
+ type: 'version-mismatch', // | missing-dependency | circular-dependency
+ // | incompatible-versions | conflicting-interfaces
+ severity: 'error',
+ description: 'Version mismatch for @objectstack/core: detected 1 unsatisfied requirements',
+ plugins: [
+ { pluginId: '@objectstack/core', version: '2.4.0' },
+ { pluginId: '@vendor/analytics', version: '^3.0.0' },
+ ],
+ resolutions: [
+ // strategy: upgrade | downgrade | replace | disable | manual
+ { strategy: 'upgrade', description: 'Upgrade @objectstack/core to satisfy all constraints' },
+ ],
+}
```
-Error: Dependency conflict
- @mycompany/sales requires @objectstack/core@^2.0.0
- @vendor/analytics requires @objectstack/core@^3.0.0
-
-Cannot satisfy both constraints.
-Solutions:
- 1. Upgrade @mycompany/sales to version compatible with core@3.x
- 2. Downgrade @vendor/analytics to version compatible with core@2.x
- 3. Contact plugin authors to update dependencies
-```
+Circular dependencies are reported separately with `severity: 'critical'`; the kernel
+also throws on them directly during `bootstrap()` (`[Kernel] Circular dependency
+detected: …`).
## Runtime Contract
@@ -497,8 +542,8 @@ export class CRMPlugin implements Plugin {
name = 'plugin.crm';
version = '2.0.0';
- // Phase 1 — sequential, registration order. Register services, schemas,
- // routes. Other plugins' services may not exist yet.
+ // Phase 1 — sequential, dependency-topological order. Register services,
+ // schemas, routes. Other plugins' services may not exist yet.
async init(ctx: PluginContext) {
ctx.registerService('crm', new CRMService());
}
@@ -524,8 +569,10 @@ A plugin may also expose `healthCheck()`, which the kernel invokes on demand
During **boot** (`kernel.bootstrap()`):
```
1. use(plugin) — validate structure + version, store (no code runs)
-2. init — every plugin, sequentially, registration order
-3. start — every plugin that defines it
+2. init — every plugin, sequentially, in dependency-topological
+ order (kernel.resolveDependencies() walks each plugin's
+ `dependencies`; registration order when none declared)
+3. start — same order, every plugin that defines it
4. kernel:ready → kernel:bootstrapped → kernel:listening events
```
@@ -546,72 +593,75 @@ boot — not at install.
Earlier revisions of this page documented an `onInstall` / `onEnable` /
`onDisable` / `onUninstall` / `onUpgrade` / `onBoot` hook family loaded from a
-manifest `lifecycle` file-map. **None of it existed** — ManifestSchema never
-declared the map and the kernel never called the hooks, so code written
-against them silently never ran. The family was retired from the protocol
+manifest `lifecycle` file-map. **That file-map never existed** — `ManifestSchema`
+has no `lifecycle` key and the kernel never invoked any of these hooks, so code
+written against them silently never ran. `PluginLifecycleSchema`
+(`packages/spec/src/kernel/plugin.zod.ts`) still *declares* `onInstall`,
+`onEnable`, `onDisable`, `onUninstall` and `onUpgrade` as optional functions, but
+nothing calls them; `onBoot` appears nowhere in the implementation at all. The
+single live exception is the **module-level** `onEnable` export of an app bundle
+described above, which `AppPlugin.start()` calls at boot — that is a stack-entry
+export, not a manifest hook. The family was retired from the protocol
(#4212, ADR-0049 enforce-or-remove).
## Permissions
-Plugins must **declare permissions** they require. ObjectStack enforces these at runtime.
+Plugins **declare permissions** in the manifest. The declaration is the install-time
+consent request; what the runtime enforces is the *granted* set.
-### System Permissions
+### Declared Permissions
+
+`ManifestSchema.permissions` accepts either the legacy flat `string[]` or the structured
+`PluginPermissionsSchema` block — four keys, and no `system` list
+(`packages/spec/src/kernel/manifest.zod.ts`):
```typescript
permissions: {
- system: [
- // Network access
- 'network.http', // Make HTTP requests
- 'network.websocket', // Use WebSockets
-
- // Storage access
- 'storage.database', // Query database
- 'storage.cache', // Use Redis/cache
- 'storage.filesystem', // Read/write files
-
- // System capabilities
- 'system.cron', // Schedule jobs
- 'system.email', // Send emails
- 'system.events', // Publish/subscribe events
-
- // Security
- 'security.encrypt', // Encrypt data
- 'security.sign', // Sign JWTs
- ],
+ services: ['object', 'http'], // services the plugin may resolve
+ hooks: ['record.beforeInsert'], // lifecycle hooks it may register
+ network: ['api.acme.com'], // hosts it may reach
+ fs: [], // filesystem paths it may access
}
```
-If plugin attempts operation without permission, ObjectStack throws error:
+Dotted capability strings like `network.http`, `network.websocket` and
+`storage.database` are values of `ResourceTypeSchema` in the plugin-sandbox permission
+*descriptor* (`packages/spec/src/kernel/plugin-security-advanced.zod.ts`) — they are not
+manifest keys. `storage.cache`, `storage.filesystem`, `system.cron`, `system.email`,
+`system.events`, `security.encrypt` and `security.sign` do not exist anywhere in the
+implementation.
-```javascript
-// Plugin tries to make HTTP request without permission
-await fetch('https://api.example.com/data');
-
-// Error: PermissionDenied
-// Plugin @mycompany/crm does not have permission 'network.http'
-// Add to plugin.manifest.ts:
-// permissions: { system: ['network.http'] }
-```
+
+**Enforcement is narrower than a sandbox.** `PluginPermissionEnforcer`
+(`packages/core/src/security/plugin-permission-enforcer.ts`) derives
+`canAccessService` / `canTriggerHook` / `canReadFile` / `canWriteFile` /
+`canNetworkRequest` from the granted set, but only **service access** is wired into a
+live code path: `SecurePluginContext.getService()` and `.replaceService()` call
+`enforceServiceAccess`. `enforceNetworkRequest`, `enforceFileRead` and `enforceFileWrite`
+have no call sites outside the enforcer module itself, and nothing patches global
+`fetch` — a plugin that calls
+`fetch()` directly is **not** intercepted today. Treat `network` / `fs` declarations as
+consent metadata, not a runtime jail.
+
-### Object Permissions
+### Declaring Objects
-Declare which objects plugin creates and manages:
+A package declares the objects it ships through the manifest's top-level `objects` glob
+list — `ManifestSchema` has no `permissions.objects` key:
```typescript
-permissions: {
- objects: [
- 'account', // Full control
- 'contact',
- 'opportunity',
- ],
-}
+objects: [
+ './src/objects/*.object.ts',
+]
```
-ObjectStack uses this for:
-- **Dependency tracking:** Can't uninstall plugin if other plugins reference its objects
-- **Security:** Plugin can only modify its own objects (not objects from other plugins)
-- **Cleanup:** Uninstalling plugin can optionally remove its objects
+Ownership is tracked by the package registry rather than by a permission declaration:
+install records the package and its metadata (`registry.installPackage()` →
+`sys_packages`), and uninstall unregisters it and runs the data-plane cleanups domain
+plugins registered via `protocol.registerUninstallCleanup(name, cleanup)` — that is how
+plugin-security revokes its package-owned permission sets. There is no runtime rule
+preventing one plugin from writing to another plugin's objects.
## Distribution Format
@@ -859,8 +909,9 @@ Always include migration guide in CHANGELOG.md for major versions.
- `account.owner` field (use `account.owner_id` instead)
### Migration
-Upgrading to v2.0.0 applies the metadata migration that renames the field on
-existing records (ADR-0087). No plugin code runs at upgrade.
+Ship a metadata migration with the upgrade (ADR-0087) that renames the field on
+existing records. There is no `onUpgrade` hook to hang this on: the schema
+declares one (`PluginLifecycleSchema`) but nothing in the kernel calls it.
```
### 3. Pin Core Dependencies
@@ -873,39 +924,47 @@ dependencies: {
```
### 4. Validate Configuration Early
-Use Zod schema to validate config during boot, not at runtime:
+Parse config with your Zod schema in `init()` — the first phase that runs — so a bad
+value fails the boot instead of the first request. There is no `onBoot` hook (the name
+appears nowhere in the implementation), and `PluginContext` has no `config` member:
```typescript
-export async function onBoot({ context }) {
- const config = context.config.get('crm', configSchema);
- // Throws clear error if config is invalid
+export class CRMPlugin implements Plugin {
+ name = 'plugin.crm';
+ constructor(private readonly options: unknown) {}
+
+ async init(ctx: PluginContext) {
+ // Throws a clear error before anything starts, not on the first request
+ const config = configSchema.parse(this.options);
+ ctx.registerService('crm', new CRMService(config));
+ }
}
```
-### 5. Clean Up on Uninstall
-Always provide `onUninstall` hook to clean up resources:
+### 5. Release Resources in `destroy()`
+`destroy()` is the only cleanup seam the kernel invokes — on shutdown (reverse
+registration order) and on rollback when a later plugin's `start()` fails. There is no
+`onUninstall` hook: `PluginLifecycleSchema` declares one but nothing calls it, and
+`PluginContext` exposes no `db` or `scheduler` member.
```typescript
-export async function onUninstall({ context, transaction }) {
- // Stop jobs
- await context.scheduler.delete('crm-sync', { transaction });
-
- // Archive data (don't delete!)
- await context.db.update('account',
- { plugin: 'crm' },
- { archived: true },
- { transaction }
- );
+async destroy() {
+ clearInterval(this.syncTimer);
+ await this.connection.close();
}
```
+For data-plane cleanup at **package**-uninstall time, register a named cleanup with the
+metadata protocol instead — `protocol.registerUninstallCleanup('crm-archive', fn)`, which
+`protocol.deletePackage()` runs and whose outcome rides on the response.
+
## Summary
Plugin packages in ObjectStack:
-- **Manifest-driven:** `plugin.manifest.ts` is the source of truth
+- **Manifest-driven:** `ManifestSchema` is the source of truth (`objectstack.plugin.json` inside a built artifact)
- **Self-contained:** Bundle objects, views, logic, and config
- **Dependency-managed:** Semantic versioning with conflict detection
-- **Lifecycle-aware:** Hooks for install, upgrade, boot, and uninstall
+- **Lifecycle-aware:** `init` → `start` → `destroy` on the plugin class — no install/upgrade/boot/uninstall hooks
- **NPM-compatible:** Distribute via NPM or `.osplugin` archives
**Next:** Learn how configuration is resolved in [Configuration Resolution](/docs/protocol/kernel/config-resolution).
diff --git a/content/docs/protocol/objectql/query-syntax.mdx b/content/docs/protocol/objectql/query-syntax.mdx
index 8fca01fc28..6cc39b5b03 100644
--- a/content/docs/protocol/objectql/query-syntax.mdx
+++ b/content/docs/protocol/objectql/query-syntax.mdx
@@ -539,9 +539,10 @@ The engine resolves `expand` via batch `$in` queries (driver-agnostic) with a de
### Filtered Expand
-Expansion follows **`lookup`**, **`master_detail`**, and **`user`** fields — i.e. the
-foreign key lives on the object you are querying. The nested `QueryAST` can **filter**
-(`where`) and **select** (`fields`) the related records:
+Expansion follows the same reference field types as above — **`lookup`**,
+**`master_detail`**, **`user`** and **`tree`** — i.e. the foreign key lives on the object
+you are querying. The nested `QueryAST` can **filter** (`where`) and **select**
+(`fields`) the related records:
```typescript
const query: QueryAST = {
@@ -921,9 +922,9 @@ const sorted = monthlyRevenue.sort((a, b) =>
### Unknown Fields Are Tolerated
-Unknown field **names** are not rejected — a projected field that doesn't exist on the
-object is **silently dropped** (matching OData / `SELECT *` tolerance), so a stale field
-reference never fails the whole query:
+Unknown field **names** are not rejected by `engine.find()` — a projected field that
+doesn't exist on the object is **silently dropped** (matching OData / `SELECT *`
+tolerance), so a stale field reference never fails the whole query:
```typescript
const rows = await engine.find('customer', {
@@ -932,6 +933,15 @@ const rows = await engine.find('customer', {
// Returns each row with `name`; `nonexistent` is omitted — no error thrown.
```
+
+This tolerance does **not** extend to the REST/protocol ingress: a `select` / `fields`
+naming a column the object does not have is `400 INVALID_FIELD` there, because dropping
+it silently answered a *narrower* projection with a *wider* one — a projection left with
+no known field falls all the way back to every field. The engine's tolerance guards
+internal callers (hooks, flows, expand sub-reads, registry-less hosts) that never pass
+through that ingress.
+
+
### Security Violations
Row-level scoping is applied by *narrowing* the query — the security middleware
@@ -979,8 +989,7 @@ Similarly, the following legacy field names should not be used in new code:
|:-------|:----------|:------|
| `filters` (array of tuples) | `where` (FilterCondition object) | Use `parseFilterAST()` to convert |
| `sort` | `orderBy` | Array of `{ field, order }` objects |
-| `group_by` | `groupBy` | Array of field name strings |
-| `aggregate` (object map) | `aggregations` (array) | Array of `AggregationNode` objects |
+| `aggregate` | `aggregations` | Same `AggregationNode[]`; the SQL and MongoDB drivers read either key, but `engine.aggregate()` forwards only `aggregations` |
| `expand` (string array) | `expand` (Record) | Map of field name → nested QueryAST |
| `skip` | `offset` | Number |
| `select` | `fields` | Array of FieldNode (field-name strings) |
diff --git a/content/docs/protocol/objectql/state-machine.mdx b/content/docs/protocol/objectql/state-machine.mdx
index 9d96d87bec..8a370b43a5 100644
--- a/content/docs/protocol/objectql/state-machine.mdx
+++ b/content/docs/protocol/objectql/state-machine.mdx
@@ -8,7 +8,7 @@ description: Define strict business logic constraints prevents AI hallucinations
The **State Machine** (`state_machine` validation rule) lets you define the "Constitution" of a record's lifecycle: the legal `status` transitions a record may take. It is a **flat, textbook finite-state-machine** transition table that the write path enforces.
- Per [ADR-0020](https://github.com/objectstack-ai/objectstack/blob/main/docs/adr/0020-state-machine-converge-and-enforce.md), a state machine is **one of the object's `validations`** — a rule with `type: 'state_machine'`. There is no top-level `stateMachine`/`stateMachines` property on an object, and the older XState-style shape (hierarchical states, `on`/`cond`/`actions`, `meta.aiInstructions`) was retired. A flat `{ from: [to] }` transition table is the only enforced shape.
+ Per [ADR-0020](https://github.com/objectstack-ai/objectstack/blob/main/docs/adr/0020-state-machine-converge-and-enforce.md), a state machine is **one of the object's `validations`** — a rule with `type: 'state_machine'`. There is no top-level `stateMachine`/`stateMachines` property on an object, and the older XState-style shape (hierarchical states, `on`/`cond`/`actions`, `meta.aiInstructions`) was retired **as a record-lifecycle declaration** — nothing on the write path reads it. A flat `{ from: [to] }` transition table is the only enforced shape.
## Why State Machines?
@@ -56,13 +56,19 @@ export const PurchaseRequest = ObjectSchema.create({
// below). `initialStates` — the optional FSM entry point that locks
// which states a record may be CREATED in (#3165) — is checked on
// INSERT, so to use it here you'd also add 'insert' to `events`.
- // Otherwise the field's `default: true` option is the starting value.
+ // Otherwise INSERT is unchecked by this rule and ANY declared option is
+ // a legal starting value. (An option's `default: true` is an authoring
+ // hint — only a field-level `defaultValue` fills an omitted field.)
events: ['update'],
message: 'Invalid purchase status transition.',
transitions: {
draft: ['pending'],
pending: ['approved', 'rejected'],
- // `approved` and `rejected` are dead-end states (no outgoing edges).
+ // Terminal states need an EXPLICIT empty array to be enforced: a
+ // state with no key at all is one the checker cannot reason about,
+ // so it does not block moves out of it (see "Enforcement semantics").
+ approved: [],
+ rejected: [],
},
},
],
@@ -85,25 +91,27 @@ A `state_machine` rule shares the common validation-rule fields (`name`, `label`
### Transitions
-`transitions` is the whole legal graph. The keys are current state values; each value is the list of states the record may move *to*. A state with no key (or an empty array) is a dead-end with no outgoing edges.
+`transitions` is the whole legal graph. The keys are current state values; each value is the list of states the record may move *to*. An **explicit empty array** is an enforced dead-end — every move to another state is rejected. A state with **no key at all** is different: the checker has nothing to reason about there and lets the write through (see [Enforcement semantics](#enforcement-semantics)), so declare `[]` when you mean "terminal".
```typescript
transitions: {
draft: ['pending'],
pending: ['approved', 'rejected'],
+ approved: [], // terminal — enforced
+ rejected: [], // terminal — enforced
}
```
### Enforcement semantics
-- **On insert**, the `transitions` table is not consulted — there is no prior state to transition from. If the rule declares `initialStates`, the created value must be one of them, or the write is rejected with `invalid_initial_state` (the FSM entry point). Without `initialStates`, insert is a no-op and the starting value is constrained only by the field-level `select` check — typically the option marked `default: true`.
-- **On update**, if the state field changed and the new value is **not** in `transitions[oldValue]`, the write is rejected.
-- The check is **lenient where it cannot reason**: if the prior state is not described by the table (e.g. legacy or externally-written data), it does not block.
+- **On insert**, the `transitions` table is not consulted — there is no prior state to transition from. If the rule declares `initialStates`, the created value must be one of them, or the write is rejected with `invalid_initial_state` (the FSM entry point). Without `initialStates`, insert is a no-op and the starting value is constrained only by the field-level `select` option-membership check (`invalid_option`), so any declared option is a legal start.
+- **On update**, if the state field changed and the new value is **not** in `transitions[oldValue]`, the write is rejected. Clearing the field (writing `null`) is exempt.
+- The check is **lenient where it cannot reason**: if the prior state has no key in `transitions` (legacy or externally-written data, or a state you simply forgot to declare), it does not block. Only an explicit `[]` makes a state a hard dead-end.
- Only a rule with `severity: 'error'` (the default) blocks the write; `warning`/`info` are logged.
- **Seed writes are exempt** (#3433). Curated seed data — package bootstrap fixtures, marketplace templates, per-org replay, all loaded by `SeedLoaderService` — is a snapshot of established facts, not a record walking its lifecycle, so it bypasses the `state_machine` rule entirely: a seed may be born mid-lifecycle (a `completed` project, a `closed_won` opportunity) and neither `initialStates` (insert) nor `transitions` (update) is enforced. Every *other* validation still runs, so a seed must still satisfy field shape, `format`, `script`, and the rest. `os lint` warns when a seeded value is not a state the machine declares, so a typo is still caught before boot.
- **A "historical" data import is exempt too** (#3479). Migrating established facts — a batch of already-`closed` tickets, `closed_won` deals — is the same "snapshot, not a lifecycle event" situation. Set `treatAsHistorical: true` on the import request (default **off**) and the runner puts `skipStateMachine` on the write context, so `initialStates` doesn't reject those mid-lifecycle rows. A normal import leaves it off and still walks the FSM — the strict behavior is the default, so the exemption is always an explicit opt-in.
- **`treatAsHistorical` also preserves the original audit timeline** (#3493). Skipping the FSM is only half of migrating established facts; the other half is keeping *when* they happened and *who* did them. Under the same flag the write context also carries `preserveAudit`, which (1) makes `updated_at` / `updated_by` **client-preferred** — a supplied historical last-modified survives instead of being stamped with the import instant, symmetric with how `created_at` / `created_by` already behave on insert — and (2) admits a **whitelist** through the static-`readonly` write strip: the audit/timestamp family plus author-declared business `readonly` fields (`closed_at`, `resolved_by`, …), so an `upsert` refresh no longer silently drops them. Platform-managed `system` columns outside that family (`organization_id` and other tenancy/generated columns) stay stripped — a historical import reinstates facts, it does not forge tenancy. Like the FSM exemption this is opt-in: a normal write still auto-stamps `updated_at`/`updated_by` and strips `readonly` exactly as before, and permissions / RLS / field-level security are unchanged.
-- **Undoing a historical import is symmetric** (#3549 / #3556). The import undo (`POST /data/import/jobs/:jobId/undo`) logically rolls back a finished job — deleting the rows it created and restoring the captured pre-import snapshot on the rows it updated. That restore write now carries `preserveAudit` too, but **only** when the job was flagged `treatAsHistorical`, so the snapshotted `updated_at` / `updated_by` and business `readonly` fields (`closed_at`, …) are reinstated verbatim instead of being re-stamped to the undo instant. Without it the undo would silently overwrite the very timeline the historical import preserved; a normal (non-historical) import's undo keeps the default stamp/strip.
+- **Undoing a historical import is symmetric** (#3549 / #3556). The import undo (`POST /api/v1/data/import/jobs/:jobId/undo`) logically rolls back a finished job — deleting the rows it created and restoring the captured pre-import snapshot on the rows it updated. That restore write now carries `preserveAudit` too, but **only** when the job was flagged `treatAsHistorical`, so the snapshotted `updated_at` / `updated_by` and business `readonly` fields (`closed_at`, …) are reinstated verbatim instead of being re-stamped to the undo instant. Without it the undo would silently overwrite the very timeline the historical import preserved; a normal (non-historical) import's undo keeps the default stamp/strip.
### Conditional transitions
@@ -113,7 +121,7 @@ A `state_machine` rule has no per-transition guard. To gate a transition on a pr
Because the transition table is data, both UIs and Agents can ask "from this state, what's legal next?" instead of parsing a formula.
-- **In code**, `legalNextStates(objectSchema, field, currentState)` from `@objectstack/objectql` returns the declared next states, `[]` for a known dead-end, or `null` when no `state_machine` rule governs the field.
+- **In code**, `legalNextStates(objectSchema, field, currentState)` from `@objectstack/objectql` returns the declared next states, `[]` when the state has no outgoing edges (an explicit `[]` **or** a state the table never mentions — introspection does not distinguish the two, enforcement does), or `null` when no `state_machine` rule governs the field.
- **Over HTTP**, `GET /api/v1/meta/objects/:name/state/:field?from=:state` returns `{ object, field, from, next }`, where `next` is the legal-next list (or `null`).
diff --git a/content/docs/releases/implementation-status.mdx b/content/docs/releases/implementation-status.mdx
index 0ffa1eaef9..11af8ae3d4 100644
--- a/content/docs/releases/implementation-status.mdx
+++ b/content/docs/releases/implementation-status.mdx
@@ -19,6 +19,7 @@ This matrix is generated from actual codebase analysis and represents the curren
|:------:|:-------|:------------|
| ✅ | **Fully Implemented** | Production-ready implementation with all core features |
| ⚠️ | **Partially Implemented** | Basic implementation, missing advanced features |
+| 🟡 | **Partially Delivered** | Shipping implementation; full protocol parity still in progress |
| 🚧 | **In Progress** | Currently being developed |
| 📋 | **Planned** | Scheduled for future implementation |
| ❌ | **Not Implemented** | Protocol defined but not yet implemented |
@@ -159,14 +160,14 @@ This matrix is generated from actual codebase analysis and represents the curren
| **HTTP Server** | ❌ | ✅ | ✅ | ❌ | ✅ Full |
| **Endpoint** | ✅ | ❌ | ✅ | ❌ | ✅ Full |
| **Router** | ✅ | ✅ | ✅ | ❌ | ✅ Full |
-| **Discovery** | ❌ | ✅ | ✅ | ✅ | ✅ Full |
+| **Discovery** | ✅ | ✅ | ✅ | ✅ | ✅ Full |
| **Contract** | ✅ | ✅ | ❌ | ❌ | ⚠️ Partial |
| **Protocol** | ❌ | ✅ | ❌ | ✅ | ✅ Full |
| **Errors** | ✅ | ✅ | ✅ | ✅ | ✅ Full |
| **HTTP Cache** | ✅ | ✅ | ✅ | ✅ | ✅ Full |
| **Batch** | ✅ | ✅ | ✅ | ✅ | ✅ Full |
-The data (`/api/v1/data`), metadata (`/api/v1/meta`), and batch endpoints are implemented in `@objectstack/rest` (`rest-server.ts`); `@objectstack/runtime` re-exports `RestServer` from that package and provides the underlying HTTP server/dispatcher.
+The data (`/api/v1/data`), metadata (`/api/v1/meta`), discovery (`/api/v1/discovery`), and batch endpoints are implemented in `@objectstack/rest` (`rest-server.ts` — `registerDiscoveryEndpoints`); `@objectstack/runtime` re-exports `RestServer` from that package and provides the underlying HTTP server/dispatcher.
**REST Endpoints Implemented:**
@@ -196,8 +197,8 @@ Every route carries the `/api/v1` prefix. When project scoping is enabled each r
|:---------|:-------------:|:------:|:------|
| **Analytics** | ✅ | ✅ | ObjectQL aggregation plus `@objectstack/service-analytics` dataset execution; analytics read scope auto-bridges to `security.getReadFilter` for RLS-aware dashboards/reports |
| **OData** | ❌ | 📋 | Protocol defined, not implemented |
-| **Realtime** | @objectstack/service-realtime | ⚠️ | Realtime service with in-memory adapter shipped; production adapters in progress |
-| **WebSocket** | @objectstack/service-realtime | ⚠️ | Transport provided via the realtime service in-memory adapter |
+| **Realtime** | @objectstack/service-realtime | ⚠️ | Realtime service shipped, but only the process-local `'memory'` adapter (`in-memory-realtime-adapter.ts`); a Redis-backed HA adapter is a post-GA fast-follow |
+| **WebSocket** | ❌ | 📋 | Protocol defined (`spec/api/websocket.zod.ts`); no WebSocket transport ships in any package — the realtime service is in-process pub/sub, not a socket transport |
---
@@ -309,21 +310,26 @@ describe that separate runtime.
|:---------|:-----------------:|:-------------:|:------:|
| **Agent** | ✅ | ObjectOS runtime | ✅ Agent runtime |
| **Model Registry** | ✅ | ObjectOS runtime | ✅ Model registry |
-| **RAG Pipeline** | ✅ | @objectstack/service-knowledge | ⚠️ Knowledge service + `knowledge-memory` / `knowledge-ragflow` / `embedder-openai` plugins |
-| **NLQ** | ✅ | ObjectOS runtime | ⚠️ Data-query tools |
+| **RAG Pipeline** | ⚠️ | @objectstack/service-knowledge | ⚠️ Knowledge service + `knowledge-memory` / `knowledge-ragflow` / `embedder-openai` plugins; spec defines the retrieval binding (`ai/knowledge-source.zod.ts`, `ai/knowledge-document.zod.ts`), not a pipeline DSL |
+| **NLQ** | ❌ | ObjectOS runtime | ⚠️ Data-query tools |
| **Conversation** | ✅ | ObjectOS runtime | ✅ In-memory + ObjectQL conversation services |
-| **Agent Action** | ✅ | ObjectOS runtime | ⚠️ Action/data/knowledge tools |
-| **Cost** | ✅ | ❌ | ❌ Spec only |
-| **Predictive** | ✅ | ❌ | ❌ Spec only |
-| **Orchestration** | ✅ | ObjectOS runtime | ⚠️ Agent tool orchestration |
-| **Feedback Loop** | ✅ | ObjectOS runtime | ⚠️ Eval harness |
-| **DevOps Agent** | ✅ | ❌ | ❌ Spec only |
+| **Agent Action** | ✅ | ObjectOS runtime | ⚠️ Action/data/knowledge tools (`tool.zod.ts` + `action_` tools) |
+| **Cost** | ✅ | ❌ | ❌ Spec only — `usage.zod.ts` accounts tokens/per-call cost; no budget enforcement |
+| **Predictive** | ❌ | ❌ | ❌ Not defined |
+| **Orchestration** | ❌ | ObjectOS runtime | ⚠️ Agent tool orchestration |
+| **Feedback Loop** | ❌ | ObjectOS runtime | ⚠️ Eval harness |
+| **DevOps Agent** | ❌ | ❌ | ❌ Not defined |
**Notes:**
-- Complete AI protocol suite defined
+- The `@objectstack/spec/ai` surface is deliberately narrow — Agent, Skill, Tool,
+ Conversation, Model Registry, Embedding, Usage, MCP, plus the knowledge-source /
+ knowledge-document and solution-blueprint schemas. Application-level protocols
+ (DevOps agents, predictive pipelines, AIOps, orchestration plans, NLQ services,
+ RAG-pipeline DSLs, budget enforcement) were **removed in v1** and have no schema
+ today — see the scope note in `packages/spec/src/ai/index.ts`
- The ObjectOS runtime ships agents, model registry, conversation, tools, skills, and an eval harness
- RAG/embedding is provided by `@objectstack/service-knowledge` plus the `knowledge-memory`, `knowledge-ragflow`, and `embedder-openai` plugins (all open)
-- Cost tracking, predictive, and the DevOps agent protocols remain spec-only
+- Token/cost accounting has a schema (`ai/usage.zod.ts`) but no shipping enforcement; the predictive and DevOps-agent protocols are not defined in spec at all
---
@@ -431,7 +437,7 @@ There is no MSW package in this repo — browser API mocking is a devDependency
- [x] Multi-tenancy — verified cross-organization isolation on `pnpm dev:crm` (Alice@OrgAlpha vs. Bob@OrgBeta only see their own records across `sys_organization`, `sys_member`, `sys_user`, and `sys_*_permission_set` link tables)
- [x] Legacy `objectql.registerTenantMiddleware` removed — SecurityPlugin is now the sole tenant-isolation authority
- [x] Organization-Wide Defaults / sharing model — `private`, `public_read`, `public_read_write`, and `controlled_by_parent` enforced via `plugin-sharing` + `plugin-security`, proven by dogfood over the real HTTP stack (ADR-0056). Canonical vocabulary only — legacy aliases removed from the enum (ADR-0090 D4); unset custom-object OWD resolves to `private` (ADR-0090 D1)
-- [x] Sharing Rule evaluator — criteria rules re-evaluated on `afterInsert` / `afterUpdate` (`plugin-sharing/rule-hooks.ts`); enforced recipients are user / position / `unit_and_subordinates` (business-unit-subtree widening, ADR-0057 D5 / ADR-0090 D3); owner-type rules and group/guest recipients remain `[experimental — not enforced]`
+- [x] Sharing Rule evaluator — criteria rules re-evaluated on `afterInsert` / `afterUpdate` (`plugin-sharing/rule-hooks.ts`); every authorable recipient maps 1:1 onto an enforced `expandRecipient` branch (`plugin-sharing/sharing-rule-service.ts`) — `user`, `team`, `position`, `business_unit`, and `unit_and_subordinates` (business-unit-subtree widening, ADR-0057 D5 / ADR-0090 D3). Under ADR-0078 enforce-or-remove, `criteria` is now the only rule *type* (owner-type rules were removed from the authoring surface because the static materialiser cannot track live membership), the `group` recipient was renamed to `team`, `guest` was removed, and `queue` stays reserved in the runtime contract but deliberately non-authorable
- [x] Everyone-baseline suggestion — a permission set may set `isDefault: true` as the install-time suggestion to bind it to the built-in `everyone` position; resolved per-request as an additive baseline, no fallback cliff (ADR-0090 D5)
- [x] Default-deny for anonymous traffic — the global default-deny landed (ADR-0056 D2) and the `api.requireAuth` opt-out was then **removed** in `@objectstack/spec` 18 (#3963): the key is tombstoned and rejected at parse time, the deny decision is centralised in `@objectstack/core` `security/anonymous-deny.ts`, and public forms self-authorize via `publicFormGrant` (Option A)
- [ ] Studio RLS visual editor
@@ -455,11 +461,11 @@ There is no MSW package in this repo — browser API mocking is a devDependency
|:---------|:---------------:|:-------|
| **Data** | 16 | Core modeling, hooks, and query engine fully implemented; document/mapping/external-lookup still pending |
| **UI** | 10 | Studio/ObjectUI render most authored surfaces (🟡); full cross-surface renderer parity in progress |
-| **API** | 14 | REST/HTTP/discovery/batch fully implemented; OData pending; GraphQL removed from the plan |
+| **API** | 14 | REST/HTTP/discovery/batch fully implemented; OData and the WebSocket transport pending; GraphQL removed from the plan |
| **System** | 39 | Logging, audit, job, translation, metrics, cache, notification implemented; encryption partial (local crypto provider); several governance services pending |
| **Auth** (plugin) | 10 | Identity/organizations live (`plugin-auth`), permission + RLS live (`plugin-security`), OWD + sharing rules live (`plugin-sharing`); SCIM is opt-in |
| **Automation** (plugin) | 7 | Flow, workflow, approval, webhook, and triggers implemented; ETL/Sync pending |
-| **AI** | 12 | Agents, model registry, conversation, tools, RAG implemented (ObjectOS runtime + open knowledge plugins); cost/predictive/DevOps-agent pending |
+| **AI** | 12 | Agents, model registry, conversation, tools, RAG implemented (ObjectOS runtime + open knowledge plugins); cost accounting has a schema but no enforcement, and the predictive / DevOps-agent protocols are not defined in spec |
| **Integration** | 7 | REST/OpenAPI/MCP/Slack connectors, file storage, and queue implemented; GitHub/Vercel connectors pending |
| **QA** | 1 | Fully implemented |
diff --git a/content/docs/releases/v9.mdx b/content/docs/releases/v9.mdx
index 6dbaf4c579..dfd6ef61f5 100644
--- a/content/docs/releases/v9.mdx
+++ b/content/docs/releases/v9.mdx
@@ -17,19 +17,31 @@ variables, rename them to `OS__`. Everything else is additive.
### 1. Analytics single-form cutover — datasets are now required (ADR-0021)
-The inline analytics author surface is removed from `@objectstack/spec`. Every
-dashboard widget, report, and list chart must now bind a semantic **dataset**
-and select dimensions/measures **by name**. Define a metric once, reference it
-everywhere.
+The inline analytics author surface is removed from the dashboard-widget,
+report, and list-chart schemas in `@objectstack/spec`. Every dashboard widget,
+report, and list chart must now bind a semantic **dataset** and select
+dimensions/measures **by name**. Define a metric once, reference it
+everywhere. (An inline `object`/`objectName` + `aggregate` binding still exists
+elsewhere in the spec — the React `` block and the page
+`element:number` component — just not on these three schemas.)
Removed from the spec:
| Schema | Removed properties | Now required |
|:---|:---|:---|
| `DashboardWidget` | `object`, `categoryField`, `categoryGranularity`, `valueField`, `aggregate`, `measures` (and the `WidgetMeasure` type) | `dataset` + `values` |
-| `Report` | top-level and joined-block `objectName`, `columns`, `groupingsDown`, `groupingsAcross`, `filter` | `dataset` + `values` (`rows` are the dimensions) |
+| `Report` | top-level and joined-block `objectName`, `columns`, `groupingsDown`, `groupingsAcross`, `filter` | `dataset` + `values` (`rows` are the dimensions; scope filtering moves to `runtimeFilter`) |
| `ListChart` | `xAxisField`, `yAxisFields`, `aggregation`, `groupByField` | `dataset` + `values` |
+> Note: `Report.columns` came back one release later (9.1.0) with a different
+> meaning — on both the report and a joined block it now names the dataset
+> **dimensions across** a `matrix` report, not raw report columns. A later
+> release also flipped `DashboardWidget` to reject *any* undeclared top-level
+> key instead of silently stripping it, so a leftover inline-analytics key is
+> now a hard parse error; the error map also names the pivot `rowField` /
+> `columnField` and list-chart `xAxisField` / `yAxisFields` / `aggregation`
+> spellings.
+
**Migration:**
1. For each inline query, create a dataset with `defineDataset(...)` declaring
@@ -64,6 +76,12 @@ single-value performance family (`metric` / `kpi` / `gauge` / `solid-gauge` /
`bullet`). Removed variants can return behind an opt-in renderer once a real
renderer and data model back them.
+> Note: stacking has since come back as a property of the *series* rather than
+> a chart family — the renderer now honors `ChartSeries.stack` (series sharing
+> a group id stack, otherwise they group), so a stacked bar or area today is
+> the `bar` / `area` family plus a shared `stack` group id on the series. The
+> taxonomy above is unchanged.
+
### 3. Settings env overrides: canonical `OS__` only
`@objectstack/service-settings` no longer accepts unprefixed aliases for
@@ -124,16 +142,20 @@ Two reliability upgrades to `@objectstack/service-automation`'s durable pause
Whether a `seed` draft's rows actually materialized used to depend on which
route you published through — the per-ref publish (used by the Studio home
banner) never applied them, yielding "Published!" with empty tables. Seed
-application now lives in the protocol itself (`@objectstack/objectql`), so
-every publish path converges on it and reports the outcome under
-`seedApplied`. A seed problem never fails the publish.
+application now lives in the metadata protocol itself (today
+`@objectstack/metadata-protocol`), so every publish path converges on it and
+reports the outcome under `seedApplied`. A seed problem never fails the
+publish — callers must check `seedApplied.success` instead of assuming the data
+went live.
### AI service: tools stream progress while they run
-`ToolExecutionContext.onProgress(part)` lets a long-running tool report
+`ToolExecutionContext.onProgress?.(part)` lets a long-running tool report
progress before it returns, emitted as custom `data-*` parts on the UI message
stream (this is what powers the live build tree in Console's Build-with-AI).
-Existing tools that never emit behave exactly as before.
+It is set only on the streaming path (`streamChatWithTools`) and is `undefined`
+for non-streaming/system calls, so tools must call it optionally. Existing
+tools that never emit behave exactly as before.
### Auth: Google sign-in and an extension hook
@@ -195,9 +217,11 @@ release lands.
The deployment story for cloud-bound apps is now **publish, then install**:
`os package publish` puts a package version in the cloud catalog, and you
install it into an environment from that environment's **Marketplace** in
-Console — installing is an explicit step, no longer a side effect of a publish
-flag. The environment's **Installed** view reflects what Cloud has installed,
-so packages installed via the CLI show up there too.
+Console — installing is a separate, explicit step rather than something a
+publish does on its own (a publish installs only when you opt in with
+`os package publish --env --install`). The environment's **Installed**
+view reflects what Cloud has installed, so packages installed via the CLI show
+up there too.
Coming in the next framework release: published packages default to
**organization visibility**, so they appear in your own environments'
diff --git a/scripts/role-word-baseline.json b/scripts/role-word-baseline.json
index ad6ee3a2ae..d507524505 100644
--- a/scripts/role-word-baseline.json
+++ b/scripts/role-word-baseline.json
@@ -12,8 +12,7 @@
"content/docs/getting-started/index.mdx": 1,
"content/docs/getting-started/quick-start.mdx": 1,
"content/docs/kernel/contracts/data-engine.mdx": 1,
- "content/docs/kernel/events.mdx": 1,
- "content/docs/kernel/services-checklist.mdx": 4,
+ "content/docs/kernel/services-checklist.mdx": 3,
"content/docs/permissions/authentication.mdx": 4,
"content/docs/permissions/authorization.mdx": 3,
"content/docs/permissions/delegated-administration.mdx": 11,