From 240b287e623591a014f76aa0b2818c8bf7c4ac09 Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 31 Jul 2026 04:55:04 +0000 Subject: [PATCH 1/5] docs: checkpoint the docs-accuracy audit pass over 18 hand-written docs MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit WIP checkpoint — the adversarial verification pass is still running and will add repairs on top of this commit. Recorded now so the work survives. An audit agent read each doc, located the backing implementation under packages/, and applied evidence-backed fixes in place (158 across 18 docs). Every fix carries file:line evidence in the run's fix log. The heaviest: - kernel/cluster.mdx (22): documented an `eventBus.emit(...)` API that exists nowhere in the codebase, a `ctx.cluster` PluginContext property that does not exist, `defineStack({ cluster })` that stack.zod.ts silently strips, fabricated `postgres`/`nats` drivers, the wrong env var (real one is OS_CLUSTER_DRIVER), and an at-least-once delivery promise neither shipped driver provides. - protocol/kernel/plugin-spec.mdx (20): `plugin.manifest.ts` as the manifest filename — grep-empty across the implementation, the real one is objectstack.plugin.json; `context.plugins.isInstalled()` which does not exist; compound semver ranges ('>=1.0.0 <2.0.0') that make the resolver's anchored parse throw; init described as registration order when bootstrap() runs the dependency-topological order. - data-modeling/external-datasources.mdx (3): credentialsRef samples used a `secret:` prefix the shipped SecretBinder cannot dereference (`sys_secret:`), so copying the sample fails closed at connect. - deployment/migration-from-objectql.mdx (2): a `^3.1.0` pin that can never resolve to the current 16.1.0 line. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_01HrRNgrWaRtggzmrHpbomyh --- content/docs/concepts/metadata-lifecycle.mdx | 79 +++-- .../data-modeling/external-datasources.mdx | 21 +- content/docs/data-modeling/formulas.mdx | 67 ++-- .../deployment/migration-from-objectql.mdx | 4 +- content/docs/deployment/vercel.mdx | 13 +- content/docs/kernel/cluster.mdx | 274 ++++++++++++----- content/docs/kernel/events.mdx | 16 +- content/docs/kernel/services-checklist.mdx | 289 +++++++++++------- content/docs/kernel/services.mdx | 57 ++-- content/docs/permissions/authentication.mdx | 18 +- content/docs/plugins/index.mdx | 47 +-- content/docs/plugins/packages.mdx | 7 +- content/docs/protocol/kernel/index.mdx | 134 ++++---- content/docs/protocol/kernel/plugin-spec.mdx | 270 +++++++++------- .../docs/protocol/objectql/query-syntax.mdx | 25 +- .../docs/protocol/objectql/state-machine.mdx | 26 +- .../docs/releases/implementation-status.mdx | 40 +-- content/docs/releases/v9.mdx | 61 ++-- 18 files changed, 924 insertions(+), 524 deletions(-) diff --git a/content/docs/concepts/metadata-lifecycle.mdx b/content/docs/concepts/metadata-lifecycle.mdx index 4bcadaedce..84ec26230b 100644 --- a/content/docs/concepts/metadata-lifecycle.mdx +++ b/content/docs/concepts/metadata-lifecycle.mdx @@ -8,7 +8,7 @@ description: How metadata flows through Repository → Change Log → Cache → This page documents the metadata data path introduced by [ADR-0008](https://github.com/objectstack-ai/objectstack/blob/main/docs/adr/0008-metadata-repository-and-change-log.md) and refined by [ADR-0005](https://github.com/objectstack-ai/objectstack/blob/main/docs/adr/0005-metadata-customization-overlay.md). It is the canonical event stream that powers Studio Hot Module Replacement (HMR), REST writes, and future cloud editing. -**TL;DR** — Every write goes through a single `MetadataRepository.put()` call. The repository appends to a change log, emits a watch event with a monotonic `seq`, and broadcasts over SSE to the Studio UI. The Studio's `useMetadataHmr` hook displays the latest `seq` in the status badge and re-fetches affected views. +**TL;DR** — Every write goes through a single `MetadataRepository.put()` call. The repository appends to a change log, emits a watch event with a monotonic `seq`, and broadcasts over SSE to the console UI. In dev, the console's `MetadataHmrReloader` subscribes to that stream and triggers a debounced full page reload. --- @@ -17,8 +17,8 @@ This page documents the metadata data path introduced by [ADR-0008](https://gith ``` ┌─────────────────────────────────────────────┐ - │ Studio UI │ - │ (useMetadataHmr / HmrStatusBadge) │ + │ Console UI (incl. Studio) │ + │ (MetadataHmrReloader) │ └────────────────────┬────────────────────────┘ │ SSE /api/v1/dev/metadata-events ▼ @@ -29,7 +29,7 @@ This page documents the metadata data path introduced by [ADR-0008](https://gith │ ChangeEvent{kind, metadataType, name, seq?} ▼ ┌─────────────────────────────────────────────┐ - │ LayeredRepository │ + │ Overlay precedence (protocol.ts) │ │ ┌─────────────────────────────────────┐ │ │ │ Top: SysMetadataRepository (org) │ │ ← writable overlay │ │ Bottom: FS / InMemory (artifact) │ │ ← read-only baseline @@ -43,15 +43,26 @@ This page documents the metadata data path introduced by [ADR-0008](https://gith Reads walk top-to-bottom: the first non-null layer wins. Writes always route to the topmost writable layer (the overlay). Deletes from the layered API only remove the overlay row — the artifact baseline survives. + +Today this precedence is implemented **inside the protocol layer**, not by +`LayeredRepository`. `ObjectStackProtocolImplementation` lazily builds one +`SysMetadataRepository` per org (`getOverlayRepo()`) and merges it against the +in-memory code registry — `getMetaItemLayered()` returns `code` / `overlay` / +`effective` separately. `LayeredRepository` ships in `@objectstack/metadata-core` +and implements exactly the semantics above, but it is **not yet composed** into +any production path: the only `new LayeredRepository(...)` call sites are its own +tests. + + --- ## The four primitives | Primitive | Package | Purpose | | :--- | :--- | :--- | -| **Repository** | `@objectstack/metadata-core` | CRUD + watch interface over a single metadata source. `InMemoryRepository` and `LayeredRepository` ship from `@objectstack/metadata-core`; `FileSystemRepository` from `@objectstack/metadata-fs`; `SysMetadataRepository` from `@objectstack/metadata-protocol` (re-exported by `@objectstack/objectql` for back-compat, per [ADR-0076](https://github.com/objectstack-ai/objectstack/blob/main/docs/adr/0076-objectql-core-tiering.md)). | +| **Repository** | `@objectstack/metadata-core` | CRUD + watch interface over a single metadata source. `InMemoryRepository` and `LayeredRepository` ship from `@objectstack/metadata-core`; `FileSystemRepository` from `@objectstack/metadata-fs`; `SysMetadataRepository` from `@objectstack/metadata-protocol` (relocated out of `@objectstack/objectql` by [ADR-0076](https://github.com/objectstack-ai/objectstack/blob/main/docs/adr/0076-objectql-core-tiering.md) — import it from `@objectstack/metadata-protocol`; `@objectstack/objectql` no longer re-exports it). | | **Change Log** | `@objectstack/metadata-core` | Append-only log of every mutation, tagged with a monotonic `seq`. Watchers can replay from any `since`. | -| **Cache** | `@objectstack/metadata-core` | In-memory snapshot of the registry, keyed by `MetaRef`. Invalidated by change-log events. | +| **Cache** | `@objectstack/metadata-core` | `MetadataCache` — a **bounded LRU** in front of a repository, keyed by `refKey(ref)` and capped by `maxEntries` / `maxBytes`. Lazily filled on read miss (no bulk preload), invalidated by `repo.watch()` events, and negative-caches misses. | | **Registry** | *(per consumer)* | Not a single package — each consumer owns a typed projection over the cache (e.g. `SchemaRegistry` in `@objectstack/objectql`), per [ADR-0008 §2.9](https://github.com/objectstack-ai/objectstack/blob/main/docs/adr/0008-metadata-repository-and-change-log.md#29-registry-layer). | `MetaRef = (type, name, org)`. As of [ADR-0008 §0 amendment (2026-04-13)](https://github.com/objectstack-ai/objectstack/blob/main/docs/adr/0008-metadata-repository-and-change-log.md#0-2026-04-13-amendment--drop-project-and-branch-from-metaref), `project` and `branch` are removed from the runtime tuple. Project survives only as an artifact-packaging concept on the `objectstack.json` envelope; branching is left to Git. @@ -63,16 +74,16 @@ Reads walk top-to-bottom: the first non-null layer wins. Writes always route to When a user edits a view in Studio: 1. Studio calls `PUT /api/v1/meta/view/case_grid` (REST). -2. `protocol.ts:saveMetaItem()` validates against `MetadataTypeRegistryEntry.allowOrgOverride` (see [overlay whitelist](#overlay-whitelist)). +2. `protocol.ts:saveMetaItem()` runs the two-tier gate (see [overlay whitelist](#overlay-whitelist)): an item that already exists as a packaged artifact requires `MetadataTypeRegistryEntry.allowOrgOverride`; a brand-new item requires `allowRuntimeCreate` **or** `allowOrgOverride`. 3. If allowed, the call lands on `MetadataRepository.put(ref, body, { parentVersion, actor })`. 4. The repository: - Verifies `parentVersion` matches the current head (`ConflictError` on mismatch). - Writes the new row + appends a change-log entry with the next `seq`. - - Emits a `MetadataEvent { op: 'create' | 'update' | 'delete', ref, seq, source }`. + - Emits a `MetadataEvent { op, ref, hash, parentHash, actor, seq, ts, source }`, where `op` is one of `create | update | delete | rename | publish | revert`. 5. The `MetadataManager` bridge forwards the event over the SSE channel `/api/v1/dev/metadata-events`. The wire payload is **not** the internal `MetadataEvent` — it is a `ChangeEvent { kind: 'metadata-change', type, metadataType, name, path?, timestamp, seq? }` (emitted as `event: metadata-change`). `seq` is the canonical repository sequence and is **absent** for FS-watcher (chokidar) dev events. -6. The Studio's `useMetadataHmr` hook receives the event, updates `lastSeq`, and triggers a refetch of the affected view. +6. In dev, the console's `MetadataHmrReloader` receives the `metadata-change` event and schedules a debounced `location.reload()` (default `debounceMs: 400`, plus a 150 ms toast delay). -The `seq` is the single source of truth for ordering. The Studio status badge (`HmrStatusBadge`) shows `Repo seq: #N` in its tooltip. +The `seq` is the single source of truth for ordering on the server. Note that the shipped console client does **not** read `seq` — it reloads the whole page rather than invalidating per-item caches. There is no granular-refetch hook or seq status badge in the current UI. --- @@ -100,10 +111,15 @@ In shared-database multi-tenancy, **most metadata types must not be per-org cust | `flow` | ✅ | Per-org overlays are allowed for automation definitions. | | `agent` | ❌ | Agents are platform-owned and closed to third parties (ADR-0063 §2) — no per-org agent fork. | | `permission`, `position` | ✅ | Per-org overlays are allowed; tenant-level controls layer on top. | -| `object`, `field` | ❌ | Defines the table schema. Overriding would break existing data. | -| `datasource` | ❌ | Connection strings; multi-tenant isolation is enforced at a higher layer. | +| `object`, `field` | ❌ | Defines the table schema. Overriding a packaged object/field would break existing data — but both set `allowRuntimeCreate: true`, so tenants *can* author brand-new objects and fields. | +| `datasource` | ❌ | Connection strings; multi-tenant isolation is enforced at a higher layer. (`allowRuntimeCreate: true` — the datasource wizard persists `origin: 'runtime'` rows.) | + +There is no `workflow` metadata type (per [ADR-0020](https://github.com/objectstack-ai/objectstack/blob/main/docs/adr/0020-state-machine-converge-and-enforce.md), record state machines are a `state_machine` validation). The runtime gate is implemented in `OVERLAY_ALLOWED_TYPES` (derived from the registry) and enforced by `SysMetadataRepository.put()`. + +The gate is **two-tier** — `allowOrgOverride: false` is not the same as "no runtime writes": -There is no `workflow` metadata type (per [ADR-0020](https://github.com/objectstack-ai/objectstack/blob/main/docs/adr/0020-state-machine-converge-and-enforce.md), record state machines are a `state_machine` validation). The runtime gate is implemented in `OVERLAY_ALLOWED_TYPES` (derived from the registry) and enforced by `SysMetadataRepository.put()`. Denied types return `403 not_overridable`. +- Overwriting an item that ships from a code package requires `allowOrgOverride`. Denied → `403 not_overridable`. +- Creating a brand-new item that has no artifact backing requires `allowRuntimeCreate` (or `allowOrgOverride`). Denied → `403 not_creatable`. See [ADR-0005](https://github.com/objectstack-ai/objectstack/blob/main/docs/adr/0005-metadata-customization-overlay.md) for the full design and amendments. @@ -111,7 +127,7 @@ See [ADR-0005](https://github.com/objectstack-ai/objectstack/blob/main/docs/adr/ ## Conflict handling -Every `put()` requires a `parentVersion` (or `null` for fresh creates). If the head has moved since the caller read it, `ConflictError(ref, expectedParent, actualHead)` is thrown. Studio surfaces this as a "metadata moved" toast and offers reload-and-retry. +Every `put()` requires a `parentVersion` (or `null` for fresh creates). If the head has moved since the caller read it, `ConflictError(ref, expectedParent, actualHead)` is thrown — code `METADATA_CONFLICT`, mapped to HTTP 412 Precondition Failed. Clients are expected to re-read the item and retry; the console does not yet surface a dedicated conflict-resolution UI. The hash is `sha256:` + 64-hex of a canonical (sorted-keys, no-undefined) JSON serialization of the body. Identical bodies produce identical hashes — `put()` short-circuits on no-op writes. @@ -119,9 +135,9 @@ The hash is `sha256:` + 64-hex of a canonical (sorted-keys, no-undefined) JSON s ## HMR end-to-end (latency & ordering) -- **Latency.** A typical write-to-render round-trip is **< 100ms** on localhost (REST `PUT` → DB write → SSE flush → Studio refetch → React re-render). -- **Ordering.** The `seq` is monotonic per repository. The repository-level `watch()` API supports replay from a `since` cursor, but the dev HMR SSE endpoint does **not** replay on reconnect — it registers a fresh listener, emits a `ready` event, and then streams only live events. A tab that disconnects and reconnects will miss any events that occurred while it was offline; it should refetch the affected metadata on reconnect. -- **Multi-tab.** Each tab has its own `seq` counter. Out-of-order delivery between tabs is impossible because they share the server change log. +- **Latency.** The server half (REST `PUT` → DB write → SSE flush) is fast on localhost, but the client half dominates: `MetadataHmrReloader` debounces for 400 ms, waits another 150 ms so its toast can paint, and then does a **full page reload**. Budget roughly half a second plus a cold React mount, not a sub-100 ms in-place re-render. +- **Ordering.** The `seq` is monotonic per repository. The repository-level `watch()` API supports replay from a `since` cursor, but the dev HMR SSE endpoint does **not** replay on reconnect — it registers a fresh listener, emits a `ready` event, and then streams only live events. A tab that disconnects and reconnects will miss any events that occurred while it was offline; it should refetch the affected metadata on reconnect. (The stream also emits `: ping` heartbeat comments every 15 s.) +- **Multi-tab.** Every open tab registers its own listener on the same in-process broadcast hub, so all tabs receive the same events in the same order. No tab tracks `seq` client-side — the shipped reloader ignores it entirely. --- @@ -143,19 +159,24 @@ The hash is `sha256:` + 64-hex of a canonical (sorted-keys, no-undefined) JSON s | `FileSystemRepository` | ✅ Shipped (`@objectstack/metadata-fs`) | | Change log + `seq` (per-org, monotonic) | ✅ Shipped | | SSE bridge (`/api/v1/dev/metadata-events`, `event: metadata-change`) | ✅ Shipped | -| Studio `useMetadataHmr` + `HmrStatusBadge` | ✅ Shipped | -| Console dev-mode HMR reloader (`MetadataHmrReloader`) | ✅ Shipped | -| `SysMetadataRepository` (overlay over `sys_metadata`) | ✅ Shipped (`@objectstack/metadata-protocol`, re-exported by `@objectstack/objectql`) | -| `LayeredRepository(SysMeta + artifact)` composition | ✅ Shipped | -| `protocol.ts:saveMetaItem` routed through `SysMetadataRepository.put` | ✅ Shipped (PR-10d.6, flag removed) | -| `sys_metadata_history` table (durable, org-keyed change log) | ✅ Shipped (`@objectstack/metadata-protocol`) | +| Granular client-side HMR hook + seq status badge | ❌ Not implemented — no such hook or badge exists in the console today | +| Console dev-mode HMR reloader (`MetadataHmrReloader`) | ✅ Shipped (console app, mounted app-wide, dev-only) | +| `SysMetadataRepository` (overlay over `sys_metadata`) | ✅ Shipped (`@objectstack/metadata-protocol`) | +| `LayeredRepository(SysMeta + artifact)` composition | ⏳ Not wired — the class ships, but nothing composes it outside tests | +| `protocol.ts:saveMetaItem` routed through `SysMetadataRepository.put` | ✅ Shipped (via the per-org `getOverlayRepo()` cache) | +| `sys_metadata_history` table (durable, org-keyed change log) | ✅ Shipped (object definition in `@objectstack/metadata-core`; written by `SysMetadataRepository`) | | Cache + Registry refactor against `MetadataRepository` | ⏳ Post-M0 | -> **Cross-replica overlay sync is out of scope.** Single-instance deployments -> are the supported topology for the metadata overlay. If you run multiple -> replicas and need them to converge on the same overlay state, point them at -> the same database and rely on each replica re-reading on demand — there is -> no `LISTEN/NOTIFY` (or equivalent) push mechanism on the roadmap. +> **Cross-replica sync happens above the repository, not inside it.** +> `SysMetadataRepository.watch()` is deliberately scoped to the local instance — +> it has no `LISTEN/NOTIFY` or pub/sub of its own, and never will. Cross-node +> propagation is instead a **cluster** concern: install `@objectstack/service-cluster` +> and its `MetadataClusterBridgePlugin` calls `MetadataManager.attachClusterPubSub()` +> at `kernel:ready`, fanning every local watch event out over the `metadata.changed` +> channel (loopback-suppressed by `originNode`) so peer nodes invalidate their +> caches. Note this replays the *invalidation event*, not the overlay row — peers +> re-read from the shared database. Without the cluster service registered, the +> bridge logs and skips, and each replica only sees its own writes. --- @@ -167,7 +188,7 @@ The hash is `sha256:` + 64-hex of a canonical (sorted-keys, no-undefined) JSON s |:---|:---|:---| | `defineView(...)` / `defineFlow(...)` / any source file → compiled into `dist/objectstack.json` | ❌ Never. Loaded into the in-memory registry on boot; refreshed via HMR in dev. | ❌ The artifact's own version history *is* Git. The metadata layer does not duplicate it. | | Editing a `.json` under `//.json` (FS overlay, e.g. `/view/case_grid.json`) | ❌ FS layer is independent of DB. | ✅ Appended to the change log at `/.objectstack/.log/main.jsonl` by `FileSystemRepository`. | -| Studio inline edit, or `PUT /api/v1/meta/...` (REST) on an `allowOrgOverride: true` type | ✅ Written by `SysMetadataRepository.put()` as an **overlay row** scoped to `organization_id`. | ✅ Appended to `sys_metadata_history` (per-org `event_seq`) in the **same transaction** as the `sys_metadata` write (single-instance scope; no cross-replica push). | +| Studio inline edit, or `PUT /api/v1/meta/...` (REST) on an `allowOrgOverride: true` type | ✅ Written by `SysMetadataRepository.put()` as an **overlay row** scoped to `organization_id`. | ✅ Appended to `sys_metadata_history` (per-org `event_seq`) in the **same transaction** as the `sys_metadata` write. No-op puts (identical hash) skip the history row entirely. | | Deploying a new build (new `dist/objectstack.json`) | ❌ The artifact is loaded into memory, not synced into `sys_metadata`. | ❌ Use Git tags / your deployment platform's release log; that's where artifact "version history" lives. | ### Why artifact never enters the database diff --git a/content/docs/data-modeling/external-datasources.mdx b/content/docs/data-modeling/external-datasources.mdx index bfe819fd43..f040cce659 100644 --- a/content/docs/data-modeling/external-datasources.mdx +++ b/content/docs/data-modeling/external-datasources.mdx @@ -37,8 +37,8 @@ export const Warehouse = defineDatasource({ schemaMode: 'external', // ObjectStack never runs DDL here config: { host: 'db.internal', port: 5432, database: 'analytics', user: 'readonly' }, external: { - allowWrites: false, // read-only (the default) - credentialsRef: 'secret:warehouse/password', // resolved from the secret store + allowWrites: false, // read-only (the default) + credentialsRef: 'sys_secret:9f2c…', // opaque handle minted by the secret store validation: { onMismatch: 'fail', checkOnBoot: true }, }, active: true, @@ -90,11 +90,12 @@ That's it. `GET /api/v1/data/ext_customer` now returns live rows from the remote `customers` table; filters (`?region=EU`) push down to the remote query. -**Do not set `field.columnName` on an external object.** For federated objects the -driver's query pipeline ignores `field.columnName`; `external.columnMap` -(`remoteColumn → localField`) is the single, authoritative column mapping. -`os build` / `os validate` rejects `field.columnName` on an external object with a -clear error (ADR-0062 D7). Managed objects are unaffected. +**`field.columnName` no longer exists.** It was removed from `FieldSchema` in the +16.x line (#2377, ADR-0049) — the SQL driver always used the field key as the +physical column, so a custom column name was silently ignored — and the +ADR-0062 D7 lint that rejected it on external objects was removed with it. +`external.columnMap` (`remoteColumn → localField`) is the single, authoritative +column mapping for a federated object. ## 3. Auto-connect — no `onEnable` needed @@ -217,6 +218,12 @@ secret in the secret store (the same `SecretBinder` / `ICryptoProvider` the runtime-admin "Add Datasource" wizard uses). The credential is resolved to cleartext **at connect, before the driver is built**. +The shipped binder encrypts the cleartext into a `sys_secret` row and mints an +**opaque** handle — `sys_secret:` — as the `credentialsRef`; that is the only +form it resolves, so a hand-written path like `secret:warehouse/password` will +not dereference. A host that wants a different scheme (a vault path, say) +injects its own resolver via `new DatasourceAdminServicePlugin({ secrets })`. + Resolution is **fail-closed**: if a `credentialsRef` is declared but no secret store is configured, or the secret cannot be resolved/decrypted, the datasource is left **unconnected with a clear error** — never connected without the credential. An diff --git a/content/docs/data-modeling/formulas.mdx b/content/docs/data-modeling/formulas.mdx index 35c936a2df..9874717bd6 100644 --- a/content/docs/data-modeling/formulas.mdx +++ b/content/docs/data-modeling/formulas.mdx @@ -39,7 +39,7 @@ Every expression in metadata is persisted as the same envelope: type Expression = { dialect: 'cel' | 'cron' | 'template'; source?: string; // surface syntax - ast?: unknown; // parsed AST (filled by `objectstack compile`) + ast?: unknown; // parsed AST (filled by `os compile`) meta?: { rationale?: string; generatedBy?: string }; }; ``` @@ -66,11 +66,11 @@ to relearn the syntax across surfaces. most common mistake (and the root cause of issue #1491) is a condition like `{record.rating} >= 4`: in CEL, `{…}` is a **map literal**, so it is a parse error. Write bare CEL: `record.rating >= 4`. Braces belong only in `{{ … }}` -text templates. As of 7.6 a malformed expression no longer silently does -nothing — it fails `objectstack build` and throws at runtime. +text templates. A malformed expression does not silently do nothing — it fails +`os build` and throws at runtime. -**Template formatting (7.6).** A `template` hole is a field path with an optional +**Template formatting.** A `template` hole is a field path with an optional whitelisted formatter — `{{ path | formatter[:arg] }}` — with defined value→string semantics (no arbitrary logic; put logic in a CEL field): @@ -121,9 +121,12 @@ evaluates (`objectql` computes a formula virtual field from its `expression` only). Do **not** pass `type: 'currency' | 'number' | …` to `Field.formula` — it is rejected by the typed `FieldInput` and, untyped, would override `type:'formula'` so the field silently never computes. To declare what the formula returns, set -**`returnType`** (`'number' | 'text' | 'boolean' | 'date'`); it is inferred and -stamped automatically by the AI build path, and consumers (dashboard measures, -formatting) read it instead of re-parsing the expression. The CEL source is the +**`returnType`** (`'number' | 'text' | 'boolean' | 'date'`). You declare it +yourself — nothing back-fills it — but the agent-callable `validate_expression` +tool reports the type it infers from the expression (`inferredType`) so an AI +author can stamp the matching value. Consumers read the declared `returnType` +instead of re-parsing the expression (record-title eligibility, for one: only a +`returnType: 'text'` formula may be a `nameField`). The CEL source is the **`expression`** key — it is the only key the schema and runtime read for a formula's source; there is no separate `formula` source key. @@ -161,7 +164,7 @@ Cheat-sheet of what you'll write daily: Registered automatically into every `Environment` by `@objectstack/formula`. All functions are pure given a pinned `now`, which is what makes -`objectstack build` artifacts byte-stable across runs. +`os build` artifacts byte-stable across runs. | Function | Returns | Description | |:---|:---|:---| @@ -197,7 +200,8 @@ Keep them pure, dependency-free, and AI-readable. | `record` | the row being evaluated | formulas, validation, sharing, visibility | | `previous` | row before update | hooks, validation on update | | `input` | hook payload | hooks | -| `os.user` | install / request user | seed, predicates with identity | +| `current_user` | the authenticated subject — the canonical binding (ADR-0068). `user`, `ctx.user` and `os.user` are aliases of the **same** object | predicates with identity | +| `os.user` | alias of `current_user` | seed, predicates with identity | | `os.org` | active organization | seed, predicates | | `os.env` | install env (`prod`, `dev`, `test`) | seed, predicates | @@ -205,12 +209,12 @@ Keep them pure, dependency-free, and AI-readable. ## Build-time validation -`objectstack build` type-checks every expression against the object schema, so +`os build` type-checks every expression against the object schema, so a mistake that would silently evaluate to `null` at runtime is caught before it ships. The same shared validator also runs when a flow is registered (`registerFlow`), so a flow authored dynamically gets the same verdict. Findings come at two severities: **errors** fail the build; **warnings** are advisory -(surfaced, not fatal — `objectstack validate --strict` promotes them to errors). +(surfaced, not fatal — `os validate --strict` promotes them to errors). | Check | Example | Severity | |:---|:---|:---| @@ -330,10 +334,19 @@ invert the test (here `amount <= 0` rejects non-positive amounts). object: 'case', events: ['afterUpdate'], condition: P`previous.status != 'escalated' && record.status == 'escalated'`, - body: { language: 'js', source: 'await ctx.notifyOpsTeam(record)' }, + body: { + language: 'js', + source: "ctx.log.warn('case escalated', { id: ctx.input.id });", + capabilities: ['log'], + }, } ``` +The `condition` is the CEL part — it binds `record` / `previous` directly. The +JS `body` is a different surface: it is wrapped as `new AsyncFunction('ctx', source)`, +so inside it the record is `ctx.input` (there is no bare `record`), and every +`ctx` API it touches must be covered by a declared capability. + --- ## Formula patterns @@ -375,24 +388,32 @@ Field.formula({ Field.formula({ label: 'Days to Close', returnType: 'number', - expression: 'daysBetween(record.close_date, today())', + expression: 'daysBetween(today(), record.close_date)', }) // Contract end in 30 days Field.formula({ label: 'Expiring Soon', returnType: 'boolean', - expression: 'record.status == "active" && daysBetween(record.end_date, today()) <= 30', + expression: 'record.status == "active" && daysBetween(today(), record.end_date) <= 30', }) // Age in days Field.formula({ label: 'Age (Days)', returnType: 'number', - expression: 'daysBetween(today(), record.created_date)', + expression: 'daysBetween(record.created_date, today())', }) ``` + +**Argument order matters.** `daysBetween(a, b)` is `b − a`, so it counts +*forward from `a` to `b`* and goes negative when `b` is earlier. "Days +remaining" is `daysBetween(today(), record.due_date)`; "age so far" is +`daysBetween(record.created_date, today())`. Swapping the two operands is the +easiest way to ship a formula whose sign is inverted on every row. + + **`dateField == today()` now matches (#3183).** A `date` field reads back as a `YYYY-MM-DD` string, and CEL treats a string and a timestamp as unequal — so the @@ -479,18 +500,20 @@ Field.formula({ ❌ **DON'T:** - Create circular references - Use formulas for frequently changing data -- Nest too many IF statements +- Nest ternaries so deeply the expression stops being readable - Ignore null handling --- ## Determinism contract -Two consecutive `objectstack build` runs with no source changes must produce +Two consecutive `os build` runs with no source changes must produce **byte-identical** `dist/objectstack.json`. CEL plus pinned `now` is what makes this possible — there are zero compile-time-evaluated `Date.now()` -calls in seed metadata. CI enforces this via SHA-1 comparison; if you add a -new dialect or stdlib helper, ensure it preserves determinism. +calls in seed metadata: a CEL seed value travels into the artifact as +unevaluated source and only resolves at install. This is a design contract +held by review, not a gate — no CI job currently diffs two builds — so if you +add a new dialect or stdlib helper, ensure it preserves determinism. --- @@ -524,7 +547,7 @@ Salesforce semantics rewrite their formulas in CEL. ```ts import { ExpressionEngine } from '@objectstack/formula'; -// Compile + cache +// Compile — parse + type-check, returning the engine-native AST as `value` const compiled = ExpressionEngine.compile({ dialect: 'cel', source: 'record.x * 2' }); // Evaluate @@ -536,11 +559,11 @@ const result = ExpressionEngine.evaluate( ``` The low-level engine never throws — `evaluate()` returns `{ ok: false, error }`. -But **call sites must not silently swallow that** (ADR-0032): `objectstack build` +But **call sites must not silently swallow that** (ADR-0032): `os build` **fails** on an invalid expression (with a located, schema-aware message), and at runtime the flow/rule engines **throw** a loud, attributed error instead of treating a bad expression as `false`/`null`. The same `validateExpression` -validator backs `objectstack build` and metadata registration (and the +validator backs `os build` and metadata registration (and the agent-callable `validate_expression` tool), so an expression is checked before it ships. --- diff --git a/content/docs/deployment/migration-from-objectql.mdx b/content/docs/deployment/migration-from-objectql.mdx index a711717993..980708f473 100644 --- a/content/docs/deployment/migration-from-objectql.mdx +++ b/content/docs/deployment/migration-from-objectql.mdx @@ -5,7 +5,7 @@ description: Step-by-step guide for migrating from @objectql/core to @objectstac # Migrating from `@objectql/core` to `@objectstack/objectql` -Starting with `@objectstack/objectql` v3.1, all core functionality previously provided by `@objectql/core` is now available upstream. This guide walks you through migrating your project so that `@objectql/core` can be removed. +The core functionality previously provided by `@objectql/core` — the introspection types, the utility functions, and the kernel factory — was upstreamed into `@objectstack/objectql` in **v3.0.4** and has shipped in every release since (the current release line is **v16.x**). This guide walks you through migrating your project so that `@objectql/core` can be removed. ## Why Migrate? @@ -26,7 +26,7 @@ Starting with `@objectstack/objectql` v3.1, all core functionality previously pr "dependencies": { - "@objectql/core": "^4.x", - "@objectql/types": "^4.x", -+ "@objectstack/objectql": "^3.1.0" ++ "@objectstack/objectql": "^16.1.0" } } ``` diff --git a/content/docs/deployment/vercel.mdx b/content/docs/deployment/vercel.mdx index 239ad49292..ed2e051ed7 100644 --- a/content/docs/deployment/vercel.mdx +++ b/content/docs/deployment/vercel.mdx @@ -5,7 +5,7 @@ description: Deploy ObjectStack applications to Vercel with the Hono server adap # Deploy to Vercel -ObjectStack 11 deploys to Vercel in **server mode** — serverless functions running the **Hono** adapter (`@objectstack/hono`). The published ObjectStack Studio/console ships as a static Vite SPA that points at a separate ObjectStack server (via `VITE_SERVER_URL`). +ObjectStack deploys to Vercel in **server mode** — serverless functions running the **Hono** adapter (`@objectstack/hono`). The Console SPA can also be deployed on its own as a static Vite build that points at a separate ObjectStack server (via `VITE_SERVER_URL`) — but that build comes from the Console source repo ([objectui](https://github.com/objectstack-ai/objectui)), not from the prebuilt `@objectstack/console` package. **Removed in 11.** The in-browser MSW mode (`@objectstack/plugin-msw`) and the Next.js adapter (`@objectstack/nextjs`) are no longer published — use the Hono server path below. @@ -43,7 +43,7 @@ In Server mode, ObjectStack runs inside Vercel Serverless Functions. API request This is a valid self-contained pattern when you want to ship a Vite SPA *and* its API from a single Vercel project: the SPA is served as static assets, and a Hono-based serverless function handles `/api/*` requests. -The published ObjectStack Studio/console (`@objectstack/console`) does **not** use this topology. It deploys as a pure static SPA with no `api/` function — it is meant to be embedded in, or pointed at, a separate ObjectStack server via `VITE_SERVER_URL`. For that separate server, prefer the CLI (`objectstack serve`) over a hand-rolled Hono function. +The framework-vendored Console (`@objectstack/console`) does **not** use this topology. It is a **prebuilt, dist-only** package — no source and no build step, just static assets — that an ObjectStack server mounts at `/_console` (`os serve --ui`, on by default). Its bundle is baked same-origin, and `VITE_SERVER_URL` is a build-time Vite flag, so aiming a Console at a *different* origin means building it yourself from the Console source repo ([objectui](https://github.com/objectstack-ai/objectui)). For the server itself, prefer the CLI (`os serve`) over a hand-rolled Hono function. **1. Create the kernel singleton** (`api/_kernel.ts` — prefixed with `_` to prevent Vercel from creating a route): @@ -82,7 +82,10 @@ export async function ensureApp(): Promise { `InMemoryDriver` keeps this snippet self-contained, but on serverless **all data is lost on every cold start** — fine for a demo, wrong for anything real. Before going past a proof of concept, swap the driver for a real database (see -[Database Drivers](/docs/data-modeling/drivers)) and point `OS_DATABASE_URL` at it. +[Database Drivers](/docs/data-modeling/drivers)). A hand-built kernel does **not** +resolve `OS_DATABASE_URL` on its own — that precedence lives in the CLI and in +`createStandaloneStack()` — so read the connection string yourself, e.g. +`new SqlDriver({ client: 'pg', connection: process.env.OS_DATABASE_URL })`. **2. Create the API entrypoint** (`api/index.ts`): @@ -134,7 +137,7 @@ Setting `VITE_SERVER_URL` to empty string tells the client SDK to use same-origi Configure these in Vercel Project Settings → Environment Variables: -### Studio / SPA build +### Console / SPA build | Variable | Description | | :--- | :--- | @@ -179,7 +182,7 @@ storage for artifacts. - [ ] `api/_kernel.ts` boots the kernel with the correct driver - [ ] `vercel.json` sets `VITE_USE_MOCK_SERVER=false` and `VITE_SERVER_URL=` (empty) - [ ] Rewrite rule routes `/api/*` to `/api` and excludes `/api/` from SPA rewrite -- [ ] `OS_DATABASE_URL` is configured in Vercel environment variables (for production drivers) +- [ ] The production connection string is set in Vercel environment variables **and** read explicitly by the driver you construct in `api/_kernel.ts` (`OS_DATABASE_URL` is resolved by the CLI / `createStandaloneStack()`, not by a hand-built kernel) - [ ] CORS is configured if frontend and API are on different origins --- diff --git a/content/docs/kernel/cluster.mdx b/content/docs/kernel/cluster.mdx index 34dd10cdf7..356a239978 100644 --- a/content/docs/kernel/cluster.mdx +++ b/content/docs/kernel/cluster.mdx @@ -83,19 +83,32 @@ These four are the minimum sufficient set. Higher-level facilities - **No native job queue primitive.** A job queue is `KV + Lock + PubSub` composed; expressing it directly in the cluster layer would couple the - protocol to a specific queue shape. `service-queue` builds on the primitives - instead. + protocol to a specific queue shape. `service-queue` is expected to build on + the primitives instead — today it ships memory and DB adapters only and does + not consume the cluster service. - **No native cache primitive.** Caches are `KV + PubSub` (read the value, - subscribe to the invalidation channel). `service-cache` builds on the - primitives instead. + subscribe to the invalidation channel). `service-cache` is expected to build + on the primitives instead — today it ships a memory adapter plus a Redis + adapter skeleton that throws `RedisCacheAdapter not yet implemented`, and it + does not consume the cluster service either. - **No native stream/Kafka primitive.** Streaming is intentionally out of scope for v1. It can be added as a fifth primitive later without changing the four above. ## 4. Event scope and delivery semantics -The single biggest source of cluster bugs is "I called `eventBus.emit(...)` -and assumed everyone would hear it". Every emit must answer three questions: +The single biggest source of cluster bugs is "I published an event and assumed +everyone would hear it". Every emit must answer three questions: + +> **Status: protocol only.** The three fields below exist as +> `EventMetadata.cluster` (`EventClusterOptionsSchema` in +> `kernel/cluster.zod.ts`, wired into `kernel/events/core.zod.ts`), but **no +> runtime consumes them** — there is no `eventBus` object in the codebase at +> all. The kernel's plugin event API is `ctx.hook(name, handler)` / +> `ctx.trigger(name, ...args)`, neither of which takes cluster metadata. The +> shipped cross-node path today is a direct `cluster.pubsub.publish(...)` call +> (§6.2, §7.3). §4.1-§4.4 define what these enum values are *supposed* to mean +> once a bus reads them. ### 4.1 Scope — who receives this event? @@ -133,6 +146,11 @@ deliverySemantics: 'best-effort' | 'at-least-once' | 'exactly-once' returns. Survives node crash. Handlers must be **idempotent** — duplicates are possible during retry. Use for: webhook outbox, audit shipping, async job enqueue. + **No shipped driver provides this yet.** The `redis` driver publishes over + plain Redis pub/sub, which is *at-most-once* — fire-and-forget, no + persistence, no replay for a node that was down at publish time. Only route + cache-invalidation hints through it; anything that must survive a crash needs + a durable outbox of its own. - **`exactly-once`** — Reserved keyword; **not implemented in v1**. The protocol accepts the enum value so future runtimes can add it without a breaking change. Startup rejection of this value is *intended* behaviour @@ -153,29 +171,56 @@ partition keys: record id, tenant id, conversation id. When unset, no ordering is guaranteed — handlers may run in parallel and out of emit order. +> **Status: accepted but ignored.** `PublishOptions.partitionKey` is part of +> the `IPubSub` contract and callers already pass it (the metadata manager +> keys on `` `${type}:${name}` ``), but both shipped drivers take the options +> bag as an unused `_opts` parameter. The memory driver's synchronous +> single-process fan-out preserves emit order anyway; the redis driver gives +> you whatever ordering Redis pub/sub does. Do not rely on cross-key ordering +> guarantees yet. + ### 4.4 The combined contract +The three fields compose into one `EventMetadata.cluster` block: + ```ts -eventBus.emit('account.updated', payload, { +// target shape — parses today, nothing reads it yet +const metadata = { + source: 'my-plugin', + timestamp: new Date().toISOString(), cluster: { scope: 'cluster', deliverySemantics: 'at-least-once', partitionKey: payload.id, }, -}) +} ``` -This is the **only** correct way to invalidate a per-record cache entry -across a cluster: the `cluster` scope reaches every node, `at-least-once` -guarantees no node misses the invalidation across a crash, and -`partitionKey` ensures that two rapid updates to the same record are -applied in order. +Once a bus honours it, that combination is the intended way to invalidate a +per-record cache entry across a cluster: the `cluster` scope reaches every +node, `at-least-once` guarantees no node misses the invalidation across a +crash, and `partitionKey` ensures that two rapid updates to the same record +are applied in order. + +Until then, the working equivalent is a direct publish on the cluster +service — it accepts `partitionKey` (see the §4.3 caveat) and gives +best-effort delivery only: + +```ts +import type { IClusterService } from '@objectstack/spec/contracts' + +const cluster = ctx.getService('cluster') +await cluster.pubsub.publish('account.updated', payload, { + partitionKey: payload.id, +}) +``` ## 5. Service scope and leader election -Every service registered with the kernel must declare its **cluster scope**. -This is independent of DI lifecycle scope (`singleton` / `transient` / -`scoped`); it answers a different question. +Every service registered with the kernel has a **cluster scope**. The +annotation is optional — omitting it is equivalent to +`{ clusterScope: 'node' }`. This is independent of DI lifecycle scope +(`singleton` / `transient` / `scoped`); it answers a different question. | `clusterScope` | Meaning | |----------------|---------| @@ -236,12 +281,16 @@ registration (`ServiceMetadata.cluster` / `ServiceFactoryRegistration.cluster` When a service declares `clusterScope: 'cluster'`, the runtime will automatically: -1. Wrap `onEnable` so the work only starts after lock acquisition (for - `leader-elected`). +1. Wrap the plugin's `start(ctx)` so the work only begins after lock + acquisition (for `leader-elected`). 2. Refresh the lock TTL via heartbeat while the node is healthy. -3. Release the lock on `onDisable` or graceful shutdown. +3. Release the lock on `destroy()` / the `kernel:shutdown` hook. 4. Emit a leadership-change event so dependent services can react. +(The kernel's plugin contract is `init(ctx)` / `start(ctx)` / `destroy()`. The +`onEnable` / `onDisable` family was retired from the protocol — it never had +an invocation site — so do not reach for it here.) + A plugin author writing a cron scheduler will **not** write any lock code — they declare scope and strategy, and the runtime does the rest. @@ -278,19 +327,25 @@ invalidation evicts a "newer" value the node has already learned about. ### 6.2 The metadata change event **Current behaviour.** Cross-node metadata invalidation already works, but -it does **not** go through `eventBus.emit`. The metadata manager fans out -over the cluster PubSub channel `metadata.changed` (note: dot, not colon) -with the payload shape `ClusterMetadataChangedPayload`: +not through any event bus (there isn't one — see §4). The metadata manager +fans out over the cluster PubSub channel `metadata.changed` (note: dot, not +colon) with the payload shape `ClusterMetadataChangedPayload`: ```ts -// packages/metadata/src/metadata-manager.ts -ctx.cluster.pubsub.publish('metadata.changed', { - originNode, // used for loopback suppression - type, // 'object' | 'view' | 'dashboard' | … - event, // the local watch event, replayed verbatim on peers -}) +// packages/metadata/src/metadata-manager.ts — notifyWatchers() +this.clusterPubSub.publish('metadata.changed', { + originNode: this.clusterNodeId, // used for loopback suppression + type, // 'object' | 'view' | 'dashboard' | … + event, // the local watch event, replayed verbatim on peers +}, { partitionKey: `${type}:${event.name ?? ''}` }) ``` +The transport is handed to the manager by `MetadataClusterBridgePlugin` +(`@objectstack/service-cluster`), which calls +`metadata.attachClusterPubSub(cluster.pubsub, cluster.nodeId)` on +`kernel:ready`. `Runtime` registers that bridge automatically alongside the +cluster service. + Peers suppress their own messages by `originNode` and replay the watch event locally — there is currently **no** `version` / `name` / `tenantId` / `operation` field and **no** version comparison. @@ -336,7 +391,8 @@ in §4-§6; this is the cheat sheet. ### 7.1 "I want to fire an event" -Pick scope first, then delivery, then partition key: +Target design only — no runtime reads `EventMetadata.cluster` today (§4). Pick +scope first, then delivery, then partition key: | You want to… | scope | delivery | partitionKey | |---------------------------------------------|-----------|-----------------|---------------| @@ -360,18 +416,34 @@ Ask: **if I deploy 5 nodes, should there be 5 of these or 1?** - "Pulls from a shared queue" → `partitioned` - "Reacts to events with idempotent writes" → `idempotent-broadcast` +Declaring `cluster` does not yet *do* anything (§5.1) — until Phase 4 lands +you still have to hold the invariant yourself, e.g. by wrapping the work in +`cluster.lock.withLock(key, fn)` the way `service-job`'s cron scheduler does. + ### 7.3 "I want to read metadata in a long-lived cache" -There is **no** `ctx.cluster.cache(...)` factory. The cluster service -exposes only the four primitives (`ctx.cluster.{pubsub, lock, kv, counter}`, -see §3) — caches are *derived* from `KV + PubSub` by `service-cache`, they -are not a built-in cluster method. +There is **no** `cluster.cache(...)` factory, and there is no `ctx.cluster` +property either — `PluginContext` exposes the service-registry family +(`registerService` / `registerServiceFactory` / `getService` / +`getServiceScoped` / `getServices` / `replaceService`), `hook` / `trigger`, +`logger` and `getKernel`. No cluster handle is among them, so resolve the +cluster service by name: + +```ts +import type { IClusterService } from '@objectstack/spec/contracts' + +const cluster = ctx.getService('cluster') +``` + +It exposes only the four primitives (`cluster.{pubsub, lock, kv, counter}`, +see §3) plus `nodeId` / `driver` / `close()`. Caches are meant to be *derived* +from `KV + PubSub`; they are not a built-in cluster method. To keep an in-process metadata cache coherent today, subscribe to the metadata change channel via PubSub and invalidate on each notification: ```ts -ctx.cluster.pubsub.subscribe('metadata.changed', (msg) => { +cluster.pubsub.subscribe('metadata.changed', (msg) => { const payload = msg.payload // { originNode?, type, event } if (payload.type === 'object') { myObjectCache.invalidate(payload.event) @@ -386,29 +458,31 @@ the subscribe call a no-op-equivalent local fan-out on a single node. ### 7.4 What you must never do -- Never call `setInterval` for cluster-wide periodic work — declare a - `cluster`-scoped service with `leader-elected`. +- Never call `setInterval` for cluster-wide periodic work — gate each tick on + `cluster.lock` (and declare `clusterScope: 'cluster'` + `leader-elected` so + Phase 4 can take the wiring over from you). - Never `SELECT … WHERE status = 'pending' LIMIT 1` without a row lock — use `SELECT … FOR UPDATE SKIP LOCKED` (Postgres) or `UPDATE … RETURNING` with a `claimed_by` column. - Never write into a process-local `Map` and assume other nodes see it — - use `ctx.cluster.kv` for shared state. -- Never `import Redis from 'redis'` directly — go through - `ctx.cluster.{pubsub, lock, kv, counter}`. + use `cluster.kv` for shared state. +- Never `import Redis from 'ioredis'` directly — go through + `cluster.{pubsub, lock, kv, counter}`. ## 8. Configuration surface -Cluster behaviour is configured once, at the stack level. The protocol -ships a single optional field on `defineStack()`: +Cluster behaviour is configured once, at the runtime level. The shape is +`ClusterCapabilityConfig` (`kernel/cluster.zod.ts`), and it is passed on the +`Runtime` constructor: ```ts -defineStack({ - // …existing fields… +import { Runtime } from '@objectstack/runtime' + +const runtime = new Runtime({ cluster: { driver: 'memory', // default — single-process - // driver: 'redis', url: process.env.REDIS_URL, - // driver: 'postgres', useExistingPool: true, - // driver: 'nats', url: process.env.NATS_URL, + // driver: 'redis', url: process.env.OS_REDIS_URL, + // driver: 'custom', // resolved from registerClusterDriver() nodeId: process.env.NODE_ID, // optional, auto-generated if absent heartbeatMs: 5000, // leader-election heartbeat @@ -420,32 +494,67 @@ defineStack({ }) ``` + + `defineStack()` has **no** `cluster` field — `stack.zod.ts` never gained one. + Putting a `cluster` key in a stack definition is silently dropped. Cluster + config belongs to the runtime, not the metadata package. + + +The same object can be passed lower down when you are wiring the kernel by +hand, or used to build a bare service: + +```ts +import { ClusterServicePlugin, defineCluster } from '@objectstack/service-cluster' + +kernel.use(new ClusterServicePlugin({ config: { driver: 'memory', nodeId: 'web-1' } })) + +// …or standalone, outside a kernel +const cluster = defineCluster({ driver: 'memory' }) +``` + +Under `os serve` the driver is selected from the environment instead: +`OS_CLUSTER_DRIVER` (anything other than `memory` triggers a dynamic import of +`@objectstack/service-cluster-`) and `OS_REDIS_URL` for the connection +string. A distribution may register a multi-node gate; when it denies, +`os serve` logs a warning and downgrades to the single-node in-memory cluster +rather than failing. + Every cluster primitive selects its implementation from `cluster.driver`. -When the field is absent, the kernel silently auto-registers the in-memory, -single-node driver. Emitting a production warning in this case (so operators -notice that horizontal scale will be incorrect until `cluster.driver` is -configured) is *intended* but **not yet implemented** — no such warning is -logged today. +When the field is absent, `Runtime` auto-registers the in-memory, single-node +driver (pass `cluster: false` to skip registration entirely). No production +warning is emitted for that case. There *is* a split-brain guard: if the +operator declares a multi-node topology via `OS_EXPECT_MULTI_NODE=true` or +`OS_CLUSTER_REPLICAS` > 1 while the resolved driver is `memory`, startup +**throws** — override with `OS_ALLOW_MEMORY_CLUSTER_MULTINODE=true`. ### 8.1 Driver matrix -| Driver | PubSub | Lock | KV | Counter | -|--------------|-----------------------|----------------------------|---------------------|----------------| -| `memory` | EventEmitter | in-proc mutex | Map | int | -| `redis` | Redis Pub/Sub Streams | SETNX + TTL + Lua release | GET/SET | INCR | -| `postgres` | `LISTEN/NOTIFY` | advisory locks | dedicated KV table | sequence | -| `nats` | NATS subjects + JetStream | KV bucket lock | KV bucket | KV INCR | +| Driver | PubSub | Lock | KV | Counter | Ships today | +|--------------|-----------------------|----------------------------|---------------------|----------------|-------------| +| `memory` | in-process fan-out | per-key FIFO queue + TTL | Map | Map of bigints | ✅ `@objectstack/service-cluster` | +| `redis` | `PUBLISH`/`SUBSCRIBE` (at-most-once) | `SET … NX PX` + Lua release/renew | `WATCH`/`MULTI` | `INCRBY` | ✅ `@objectstack/service-cluster-redis` | +| `postgres` | `LISTEN/NOTIFY` | advisory locks | dedicated KV table | sequence | ❌ not built | +| `nats` | NATS subjects + JetStream | KV bucket lock | KV bucket | KV INCR | ❌ not built | -The `redis` driver is the recommended starting point for production -ObjectStack deployments: it shares its connection pool with the planned -`service-cache` Redis adapter (no extra infrastructure once cache is on -Redis), gives native semantics for every primitive (SETNX/Lua/INCR/PUBSUB), -and decouples cluster transport from the choice of database driver -(SQL / Turso / Postgres / SQLite all work the same). +Only `memory` and `redis` are implemented. `postgres` and `nats` are accepted +by `ClusterDriverSchema` but no package provides them — `defineCluster()` +throws `Cluster driver "" is not registered` for either. The `custom` +driver value resolves whatever a plugin registered via +`registerClusterDriver(name, factory)`. -The `postgres` driver remains a viable "single binary, single dependency" +The `redis` driver is the recommended starting point for production +ObjectStack deployments: it accepts a pre-built ioredis client via +`driverOptions.client`, so it can share one connection pool with any other +Redis consumer (the `service-cache` Redis adapter is still a skeleton, so +there is nothing to share with yet); it gives native semantics for every +primitive (SET NX/Lua/INCRBY/PUBLISH); and it decouples cluster transport from +the choice of database driver (SQL / Turso / Postgres / SQLite all work the +same). + +A `postgres` driver would remain a viable "single binary, single dependency" choice for small deployments that already run Postgres and want to avoid -a Redis dependency. It is deferred to a community/optional driver. +a Redis dependency, but nobody has built one — it is deferred to a +community/optional driver against the same `registerClusterDriver()` SPI. ## 9. Industry alignment @@ -459,7 +568,8 @@ ecosystem; none are novel. with invalidation broadcast; leader-elected schedulers. Maps to §6 and `leader-elected` strategy. - **Odoo `bus.bus`** — Postgres `LISTEN/NOTIFY` for cross-worker fan-out. - Validates our `postgres` driver as the right default starting point. + Validates `LISTEN/NOTIFY` as a sound transport for the deferred `postgres` + driver (§8.1, Phase 5) — ObjectStack shipped `redis` first instead. - **Notion / Linear** — Redis pub/sub for realtime fan-out with per-document partitioning. Validates `partitionKey` ordering as the right primitive. @@ -482,24 +592,30 @@ v4 consumers. 3. Extend `kernel/service-registry.zod.ts` `ServiceMetadataSchema` and `ServiceFactoryRegistrationSchema` with an optional nested `cluster` object (`clusterScope`, `leaderStrategy`). -4. Add optional `cluster` field to `stack.zod.ts`. +4. ~~Add optional `cluster` field to `stack.zod.ts`.~~ **Dropped** — cluster + config landed on `RuntimeConfig` instead (see §8); `stack.zod.ts` has no + `cluster` field. -No runtime changes. After this phase the protocol *describes* cluster -semantics; nothing *enforces* them. +Steps 1-3 shipped. No runtime changes in this phase: the protocol *describes* +cluster semantics; nothing *enforces* them. -### Phase 2 — In-memory runtime (~2 days) +### Phase 2 — In-memory runtime (~2 days) ✅ Implement `service-cluster` with the `memory` driver only. Every cluster primitive works; behaviour on one node is identical to today. This phase -proves the API shape against real callers (EventBus, service-cache, -service-job) before committing to a wire format. +proves the API shape against real callers before committing to a wire format. +In practice the only callers wired so far are `MetadataClusterBridgePlugin`, +`service-job`'s cron scheduler, and `service-messaging`'s dispatchers — +`service-cache` and `service-queue` still do not consume the cluster service. ### Phase 3 — Redis driver (~2 days) ✅ Implement `redis` driver via `@objectstack/service-cluster-redis` -(PUBSUB + SETNX/Lua locks + WATCH/MULTI KV + INCR counter). Ships as a -sibling package so adopters opt in by importing it. Shares its ioredis -client pool with `service-cache` to avoid double connections. +(PUBLISH/SUBSCRIBE + `SET NX PX`/Lua locks + WATCH/MULTI KV + INCRBY counter). +Ships as a sibling package so adopters opt in by importing it — the import +self-registers the driver via `registerClusterDriver('redis', …)`. It can +share an ioredis client with other consumers by passing +`driverOptions.client`, though nothing shares one with it today. > **Note**: this was switched from "Postgres first" — see commit history. > Rationale: Postgres isn't a fixed dependency in the ObjectStack stack, @@ -508,10 +624,14 @@ client pool with `service-cache` to avoid double connections. ### Phase 4 — Existing services migrate (~3 days, opportunistic) +Not started. `service-job` already leader-elects by calling +`cluster.lock.acquire('job:')` directly, but that is hand-rolled — none +of the declarative wiring below exists yet. + - `service-job` declares `clusterScope: 'cluster' / 'leader-elected'`. -- `service-cache` switches to `ctx.cluster.cache(...)` factory. +- `service-cache` gains a cluster-backed cache derived from `KV + PubSub`. - `service-realtime` adds cross-node fan-out via PubSub. -- Webhook dispatcher (planned) uses `partitioned` strategy from day one. +- Webhook dispatcher uses `partitioned` strategy from day one. ### Phase 5 — Postgres / NATS drivers (deferred, optional) diff --git a/content/docs/kernel/events.mdx b/content/docs/kernel/events.mdx index 3b928ac812..b2b9ff4460 100644 --- a/content/docs/kernel/events.mdx +++ b/content/docs/kernel/events.mdx @@ -61,7 +61,7 @@ The Data Engine runs hooks around record reads and mutations. Write operations f | `beforeUpdate` / `afterUpdate` | Around record update — single-id **and** bulk (`multi: true`). | | `beforeDelete` / `afterDelete` | Around record deletion — single-id **and** bulk (`multi: true`). | -Reads fire `beforeFind` / `afterFind`, for **both** `find` and `findOne` (one read event covers every read shape). There are no per-method (`findOne`/`count`/`aggregate`) or `*Many` events: read authorization and row filtering are handled by RLS/permission rules, field masking by field-level metadata, and bulk writes by the same `before*`/`after*` write events (with the row-scoping predicate in `ctx.input.ast`). +Reads fire `beforeFind` / `afterFind`, for **both** `find` and `findOne` (one read event covers every read shape). There are no per-method (`findOne`/`count`/`aggregate`) or `*Many` events: read authorization and row filtering are handled by RLS/permission rules, field masking by field-level metadata, and bulk writes by the same `before*`/`after*` write events (a bulk write leaves `ctx.input.id` undefined and carries the caller's predicate in `ctx.input.options.where`; the middleware-composed row-scoping AST stays on the engine's internal operation context and is **not** exposed on `ctx.input`). ### The HookContext @@ -71,7 +71,7 @@ Every data hook is a single-argument handler `(ctx: HookContext) => void | Promi | :--- | :--- | | `ctx.object` | Target object name (immutable). | | `ctx.event` | Current lifecycle event, e.g. `'beforeInsert'` (immutable). | -| `ctx.input` | **Mutable** input. Shapes: insert `{ data }`, update `{ id, data }`, delete `{ id }`. Modify this to change the operation. | +| `ctx.input` | **Mutable** input. Shapes: find `{ ast, options }`, insert `{ data, options }`, update `{ id, data, options }`, delete `{ id, options }`. Modify this to change the operation. | | `ctx.result` | Operation result, available in `after*` events (mutable). | | `ctx.previous` | Record state before the operation (update/delete). | | `ctx.session` | Auth/tenancy info (`userId`, `organizationId`, `roles`, …). | @@ -95,19 +95,21 @@ engine.registerHook('afterInsert', async (ctx) => { ### Error Handling -What happens when a data hook throws is governed by the hook's **`onError`** policy (`'abort'` | `'log'`, default `'abort'`): - -- `onError: 'abort'` (default) — the error rolls back the transaction (when the hook is blocking), cancelling the operation. -- `onError: 'log'` — the error is logged and execution continues; the operation is **not** cancelled. +A hook registered in code with `engine.registerHook()` always propagates: the engine awaits each handler in turn and never catches, so a throw aborts the operation. ```typescript engine.registerHook('beforeInsert', async (ctx) => { if (ctx.object === 'invoice' && !ctx.input.data.customer_id) { - throw new Error('Invoice must have a customer'); // Aborts the insert (default onError) + throw new Error('Invoice must have a customer'); // Aborts the insert } }); ``` +Hooks declared as **`Hook` metadata** additionally honour an **`onError`** policy (`'abort'` | `'log'`, default `'abort'`). It is a metadata field — `registerHook()` has no `onError` option: + +- `onError: 'abort'` (default) — the error propagates, rolling back the transaction (when the hook is blocking) and cancelling the operation. +- `onError: 'log'` — the error is logged and swallowed; the operation is **not** cancelled. + ### Ordering Data hooks accept a `priority` option (**default `100`**). They are sorted so that **lower priority values run first**: diff --git a/content/docs/kernel/services-checklist.mdx b/content/docs/kernel/services-checklist.mdx index b00e238652..413b1d5818 100644 --- a/content/docs/kernel/services-checklist.mdx +++ b/content/docs/kernel/services-checklist.mdx @@ -6,24 +6,27 @@ description: Complete inventory of ObjectStack kernel services with protocol met # Kernel Services Checklist -**This checklist is out of date.** It predates several packages that have since -shipped — SQL and MongoDB drivers, automation, realtime, storage, and more (see -[Plugins & Packages](/docs/plugins/packages) for the current catalog). Keep it -for the protocol-method inventory; treat the per-service status columns as -historical until this page is regenerated. +**This page is hand-maintained.** The per-slot Provider column mirrors +`CORE_SERVICE_PROVIDER` (`packages/spec/src/system/core-services.zod.ts`), which +is the CI-guarded source of truth; the method inventories mirror the per-domain +contracts in `packages/spec/src/api/protocol.zod.ts`. When the two disagree, +the code wins. See [Plugins & Packages](/docs/plugins/packages) for the full +package catalog. The ObjectStack protocol defines **16 kernel services** registered via the `CoreServiceName` enum (the never-implemented `graphql` entry was removed in v17). Each service maps to a set of protocol methods governed by its per-domain contract (`DataProtocol`, `MetadataProtocol`, ...) — the transitional `ObjectStackProtocol` composition alias was dissolved in v17 (ADR-0076 D9); capability availability comes from the runtime discovery `services` registry. -**Key architecture principle**: The kernel itself only provides **data** and **metadata** (framework). Everything else — including **auth** and **automation** — is delivered by **plugins**. The current `@objectstack/objectql` package is an example kernel implementation to get the basic API running; production kernels will be rebuilt as new plugins. +**Key architecture principle**: the kernel guarantees only **data** and **metadata**, and even those are filled by packages (`@objectstack/objectql`, `@objectstack/metadata`) rather than baked in — the kernel's own contribution is an in-memory fallback for each unfilled `core` slot. Everything else — including **auth** and **automation** — is delivered by plugins. `@objectstack/objectql` is an example kernel implementation to get the basic API running; production kernels will be rebuilt as separate plugins. -**Last Updated**: February 2026 - -- ✅ Implemented — 18 protocol methods (kernel-provided) -- ⚠️ Framework — metadata (in-memory registry, DB persistence pending) -- 🟡 In Development — auth (plugin structure complete, logic planned) -- ❌ Plugin Required — 38 protocol methods (to be delivered by plugins) +**Legend** + +- ✅ Implemented — the 17 kernel-provided protocol methods (`DataProtocol` 9 + `MetadataProtocol` 8) +- ⚠️ Framework — the slot is filled by the kernel's in-memory fallback: real reads and + writes, no persistence (self-declares `degraded`, ADR-0076 D12) +- ❌ Plugin Required — the remaining 36 protocol methods (`analytics` 2, `automation` 1, + `packages` 6, `views` 5, `permissions` 3, `workflow` 3, `realtime` 6, `notification` 7, + `i18n` 3), delivered by whatever fills the slot --- @@ -34,10 +37,10 @@ The ObjectStack protocol defines **16 kernel services** registered via the `Core ┌─────────────────────────────────────────────────────────┐ │ Kernel Layer │ │ ┌──────────────────┐ ┌──────────────────────────────┐ │ -│ │ metadata (⚠️) │ │ data (✅) │ │ -│ │ In-memory only │ │ ObjectQL example kernel │ │ -│ │ DB persistence │ │ Will be rebuilt as plugins │ │ -│ │ pending │ │ │ │ +│ │ metadata (✅) │ │ data (✅) │ │ +│ │ MetadataPlugin │ │ ObjectQL example kernel │ │ +│ │ → sys_metadata │ │ Will be rebuilt as plugins │ │ +│ │ (in-mem fallback)│ │ │ │ │ └──────────────────┘ └──────────────────────────────┘ │ ├─────────────────────────────────────────────────────────┤ │ Plugin Layer │ @@ -45,8 +48,9 @@ The ObjectStack protocol defines **16 kernel services** registered via the `Core │ workflow, ui, realtime, notification, ai, i18n, │ │ search, file-storage, cache, queue, job │ │ │ -│ Discovery API reports: enabled/unavailable/degraded │ -│ per service so clients adapt their UI accordingly │ +│ Discovery API reports availability per service │ +│ (available / degraded / stub / unavailable) so │ +│ clients adapt their UI accordingly │ └─────────────────────────────────────────────────────────┘ ``` @@ -56,23 +60,36 @@ The ObjectStack protocol defines **16 kernel services** registered via the `Core | # | Service Name | Criticality | Methods | Status | Provider | |:--|:-------------|:------------|:--------|:-------|:---------| -| 1 | **metadata** | `core` | 8 | ⚠️ Framework | Kernel (in-memory) | +| 1 | **metadata** | `core` | 8 | ✅ Implemented (kernel fallback is ⚠️ in-memory) | `@objectstack/metadata` | | 2 | **data** | `required` | 9 | ✅ Implemented | `@objectstack/objectql` | | 3 | **analytics** | `optional` | 2 | ❌ Plugin Required | `@objectstack/service-analytics` | -| 4 | **auth** | `core` | — | 🟡 In Development | `@objectstack/plugin-auth` | -| 5 | **ui** | `optional` | 5 | ❌ Plugin Required | TBD plugin | -| 6 | **workflow** | `optional` | 3 | ❌ Plugin Required | TBD plugin | -| 7 | **automation** | `optional` | 1 | ❌ Plugin Required | TBD plugin | -| 8 | **realtime** | `optional` | 6 | ❌ Plugin Required | TBD plugin | -| 9 | **notification** | `optional` | 7 | ❌ Plugin Required | TBD plugin | -| 10 | **ai** | `optional` | 3 | ❌ Plugin Required | TBD plugin | +| 4 | **auth** | `core` | — | ✅ Implemented | `@objectstack/plugin-auth` | +| 5 | **ui** | `optional` | 5 | ❌ Nothing fills this slot | `@objectstack/metadata-protocol` — `/ui/view` is served by its `protocol` service, not by a `ui` service | +| 6 | **workflow** | `optional` | 3 | ❌ Nothing ships | — | +| 7 | **automation** | `optional` | 1 | ❌ Plugin Required | `@objectstack/service-automation` | +| 8 | **realtime** | `optional` | 6 | ❌ Plugin Required (in-process only — no HTTP/WS route is mounted) | `@objectstack/service-realtime` | +| 9 | **notification** | `optional` | 7 | ❌ Plugin Required | `@objectstack/service-messaging` | +| 10 | **ai** | `optional` | — | ❌ Nothing ships in this repo | `service-ai` (Cloud/EE) | | 11 | **i18n** | `core` | 3 | ✅ Built-in (in-memory fallback) | `@objectstack/service-i18n` | -| 12 | **file-storage** | `optional` | — | ❌ Plugin Required | TBD plugin | -| 13 | **search** | `optional` | — | ❌ Plugin Required | TBD plugin | +| 12 | **file-storage** | `optional` | — | ❌ Plugin Required | `@objectstack/service-storage` | +| 13 | **search** | `optional` | — | ❌ Nothing ships | — | | 14 | **cache** | `core` | — | ✅ Built-in (in-memory fallback) | `@objectstack/service-cache` | | 15 | **queue** | `core` | — | ✅ Built-in (in-memory fallback) | `@objectstack/service-queue` | | 16 | **job** | `core` | — | ✅ Built-in (in-memory fallback) | `@objectstack/service-job` | + +The Provider column is not prose — it is `CORE_SERVICE_PROVIDER` in +`@objectstack/spec/system`, the single table discovery reads when it tells a +caller what to install for an empty slot. `null` there (`ai`, `search`, +`workflow`) means **nothing ships**, and discovery says so rather than naming a +plausible package: `No implementation ships for the '' slot — register a +service under it to enable`. `ui` is the one slot with a hand-written remedy, +because the package named does not fill the slot: `Served by the protocol +service — register MetadataPlugin (@objectstack/metadata-protocol) to enable`. +`scripts/check-service-providers.mjs` fails CI if a name in that table is not a +real workspace package. + + **Criticality Levels** - **required**: System cannot start without this service @@ -82,10 +99,12 @@ The ObjectStack protocol defines **16 kernel services** registered via the `Core --- -## 1. metadata Service ⚠️ Framework +## 1. metadata Service ✅ Implemented **Service Name**: `metadata` · **Criticality**: `core` -**Implementation**: Kernel — `SchemaRegistry` (in-memory) +**Implementation**: `MetadataPlugin` (`@objectstack/metadata`) — persisted to `sys_metadata`. +Fallbacks: the kernel's in-memory `createMemoryMetadata` (`@objectstack/core`) when nothing +registers the slot, over ObjectQL's in-memory `SchemaRegistry` of artifact-loaded metadata. **Route Mounts**: `/api/v1/meta`, `/api/v1`, `/api/v1/packages` ### Protocol Methods @@ -101,16 +120,18 @@ The ObjectStack protocol defines **16 kernel services** registered via the `Core | `getMetaItemCached` | `GetMetaItemCachedRequest → GetMetaItemCachedResponse` | ✅ | | `getUiView` | `GetUiViewRequest → GetUiViewResponse` | ✅ | -### Current Limitations +### Former Gaps — all now closed -The metadata service is a **framework** — it works with an in-memory `SchemaRegistry` loaded from YAML/TS config files at startup. +Every gap this section used to list has shipped. Only the kernel's own +in-memory fallback is still a "framework": it serves real reads and writes but +never reaches disk, which is exactly what it self-declares (`degraded`). -| Gap | Description | Priority | -|:----|:------------|:--------:| -| **DB Persistence** | Shipped: `MetadataPlugin` (`@objectstack/metadata`) persists to `sys_metadata`; only the kernel's in-memory fallback is non-persistent | Shipped | -| **Multi-instance Sync** | No mechanism to share metadata changes across instances | P1 | -| **Migration / Versioning** | No schema diff or migration tooling | P1 | -| **Hot Reload** | Metadata changes require restart | P2 | +| Former gap | Where it landed | +|:----|:------------| +| **DB Persistence** | `MetadataPlugin` (`@objectstack/metadata`) persists to `sys_metadata`; only the kernel's in-memory fallback is non-persistent | +| **Multi-instance Sync** | `MetadataClusterBridgePlugin` (`@objectstack/service-cluster`) bridges the `metadata.changed` channel onto the cluster pub/sub bus, so a mutation on one node invalidates peer registry caches. Needs a `cluster` service and a metadata service exposing `attachClusterPubSub()` | +| **Migration / Versioning** | `os diff ` compares two configurations and flags breaking changes; `os migrate plan` / `os migrate apply` reconcile the physical database against metadata | +| **Hot Reload** | `MetadataPlugin` watches its sources (`watch`) and the compiled artifact (`artifactWatch`) with chokidar; `os dev` rebuilds and the server reloads without a restart | ### Discovery: Per-Service Status @@ -243,62 +264,104 @@ empty slot does. The implementations that really do the work in memory declare --- -## 4. auth Service 🟡 In Development +## 4. auth Service ✅ Implemented **Service Name**: `auth` · **Criticality**: `core` +**Implementation**: `AuthPlugin` → `AuthManager` (`@objectstack/plugin-auth`, built on better-auth) **Route Mount**: `/api/v1/auth` — only exposed when a plugin registers the auth service **The kernel does NOT handle auth.** Install an auth plugin to enable authentication. Without it, the discovery response shows `auth: { enabled: false, status: "unavailable" }`. -### Plugin Requirements +### Where each capability lives -The `auth` service covers both **authentication** (identity) and **authorization** (permissions). There is no separate `permission` service in `CoreServiceName`. +The `auth` service covers **authentication** (identity). **Authorization** is not a +`CoreServiceName` slot at all — there is no `permission` service; the evaluators are +registered under their own names (`security.permissions`, `security.rls`, +`security.fieldMasker`) by `@objectstack/plugin-security`. -| Module | Area | Description | Priority | -|:-------|:-----|:------------|:--------:| -| **Identity Provider** | Authentication | JWT issuance/verification, session management | P0 | -| **User CRUD** | Authentication | Create / read / update / delete users | P0 | -| **Role Management** | Authentication | Role definitions, role-user associations | P0 | -| **Permission Engine** | Authorization | Object-level and field-level permissions | P0 | -| **OAuth2 / OIDC** | Authentication | Third-party login (Google, GitHub, etc.) | P1 | -| **Sharing Rules** | Authorization | Record-level sharing and visibility | P1 | -| **Row-Level Security** | Authorization | Automatic query filtering by user context | P1 | -| **Multi-tenancy** | Authentication | Space / Tenant isolation | P1 | -| **Territory Management** | Authorization | Data territory assignment and access | P2 | -| **SCIM** | Authentication | Enterprise user provisioning protocol | P2 | +| Module | Area | Ships in | +|:-------|:-----|:---------| +| **Identity Provider** | Authentication | `@objectstack/plugin-auth` — better-auth sessions + `jose` JWTs (`auth-manager.ts`) | +| **User CRUD** | Authentication | `@objectstack/plugin-auth` — `admin-user-endpoints.ts`, `admin-import-users.ts` | +| **Role Management** | Authorization | `@objectstack/plugin-security` — owns role / permission-set / user-permission-set / role-permission-set objects (ADR-0029 K2) | +| **Permission Engine** | Authorization | `@objectstack/plugin-security` — `security.permissions` (object) + `security.fieldMasker` (field) | +| **OAuth2 / OIDC** | Authentication | `@objectstack/plugin-auth` — `@better-auth/oauth-provider`, `@better-auth/sso` | +| **Sharing Rules** | Authorization | `@objectstack/plugin-sharing` — `sharing`, `sharingRules`, `shareLinks` services | +| **Row-Level Security** | Authorization | `@objectstack/plugin-security` — `security.rls` (`rls-compiler.ts`) | +| **Multi-tenancy** | Authentication | `@objectstack/plugin-auth` — the `tenancy` service (`tenancy-service.ts`, ADR-0093/ADR-0105) | +| **Territory access** | Authorization | Not a separate module: expressed as an RLS dynamic-membership predicate (`current_user.territory_account_ids`, staged in `ExecutionContext.rlsMembership`) | +| **SCIM** | Authentication | `@objectstack/plugin-auth` — `@better-auth/scim`, gated by `OS_SCIM_ENABLED` | -### Spec Files: `identity.zod.ts`, `permission.zod.ts`, `organization.zod.ts`, `auth-config.zod.ts`, `tenant.zod.ts` +### Spec Files: `identity/identity.zod.ts`, `identity/organization.zod.ts`, `security/permission.zod.ts`, `system/auth-config.zod.ts`, `system/tenant.zod.ts` (all under `packages/spec/src/`) --- -## 5–7. Business Services ❌ Plugin Required +## 5–7. Business Services -### 5. ui Service — 5 methods -`listViews`, `getView`, `createView`, `updateView`, `deleteView` -View CRUD, layout engine, action registry, permission filtering. +### 5. ui Service — 5 declared methods, none routed ❌ +`listViews`, `getView`, `createView`, `updateView`, `deleteView` -### 6. workflow Service — 3 methods + +These five are **optional members of `ViewProtocol` that nothing implements and no +route reaches** — `view` is a metadata type, so view CRUD goes through the metadata +API (`/api/v1/meta`), not through them. Nothing anywhere registers the `ui` slot +either (#4093 / #4146), so `CORE_SERVICE_PROVIDER.ui` names +`@objectstack/metadata-protocol` rather than a `ui` plugin: the one route the `/ui` +domain serves is `GET /api/v1/ui/view/:object[/:type]`, which calls `getUiView` on +the **`protocol`** service that `assembleMetadataProtocol()` registers (invoked by +`ObjectQLPlugin`, or by the standalone `createMetadataProtocolPlugin()`). Without it +the domain answers **501** with that remedy spelled out, not a generic "install a ui +plugin". + + +### 6. workflow Service — 3 methods ❌ Nothing ships `getWorkflowConfig`, `getWorkflowState`, `workflowTransition` -State machine transitions. Approve/reject are no longer workflow methods — per ADR-0019 they moved to the request-id-based approvals API under `/api/v1/approvals`. +State machine transitions. No package registers the `workflow` slot +(`CORE_SERVICE_PROVIDER.workflow` is `null`). Approve/reject are not workflow +methods — per ADR-0019 they moved to the request-id-based approvals API under +`/api/v1/approvals` (`POST /requests/:id/{approve,reject,recall}`, served by +`@objectstack/plugin-approvals`). -### 7. automation Service — 1 method +### 7. automation Service — 1 method ✅ `@objectstack/service-automation` `triggerAutomation` Trigger engine, event triggers from ObjectQL hooks, flow executor, scheduled triggers. +The `/automation` dispatcher domain gates on `isServiceServeable`, so a slot filled by +a self-declared stub answers as an empty one. --- -## 8–11. Communication Services ❌ Plugin Required +## 8–11. Communication Services -### 8. realtime — 6 methods +### 8. realtime — 6 methods · `@objectstack/service-realtime` `realtimeConnect`, `realtimeDisconnect`, `realtimeSubscribe`, `realtimeUnsubscribe`, `setPresence`, `getPresence` -### 9. notification — 7 methods + +`service-realtime` is an **in-process pub/sub bus**, not an HTTP/WS surface. The +dispatcher has no `/realtime` branch and no plugin mounts one, so `routes.realtime` +is **never advertised** — an advertised route would 404 (ADR-0076 D12, #2462), and +`features.websockets` is hardcoded `false` for the same reason. These six +`RealtimeProtocol` members are declared and unrouted; when the service is registered +both discovery builders report the slot `degraded` with a message saying the bus is +in-process only and no HTTP/WS surface is mounted. Re-advertising waits on a real +transport. + + +### 9. notification — 7 methods · `@objectstack/service-messaging` `registerDevice`, `unregisterDevice`, `getNotificationPreferences`, `updateNotificationPreferences`, `listNotifications`, `markNotificationsRead`, `markAllNotificationsRead` -### 10. ai — 3 methods -`aiNlq`, `aiSuggest`, `aiInsights` +The slot name is `notification` (singular) and the package that fills it shares no word +with it — which is why the remedy sentence is looked up in `CORE_SERVICE_PROVIDER` +rather than derived from the slot name. The `/notifications` domain currently routes +the inbox subset: `GET /notifications` → `listInbox`, `POST /notifications/read` → +`markRead`, `POST /notifications/read/all` → `markAllRead`. All three are **optional** +on `INotificationService` — a send-only provider (SMTP, Twilio, a webhook) fills the +slot legitimately without an inbox, and each route probes its own method and answers +501 when absent. + +### 10. ai — contract removed ❌ +~~`aiNlq`, `aiSuggest`, `aiInsights`~~ Removed. These three were **optional** protocol methods (`aiNlq?` …) that no @@ -309,6 +372,8 @@ methods that called them were deleted outright (#3718). The AI service that does exist (`service-ai`, Cloud/EE) serves a different surface — chat, complete, models, conversations, agents — and `client.ai` was rebuilt against those real routes, so this entry describes a contract that no longer exists. +The `ai` slot still exists in `CoreServiceName`, but nothing in this repo fills +it (`CORE_SERVICE_PROVIDER.ai` is `null`). ### 11. i18n — 3 methods @@ -324,7 +389,7 @@ those real routes, so this entry describes a contract that no longer exists. | Environment | Provider | Registration | |:------------|:---------|:-------------| | **Production** | `I18nServicePlugin` | File-based `FileI18nAdapter` loads JSON locale files from disk | -| **In-memory fallback** | `@objectstack/core` (`createMemoryI18n`) | Registered by `AppPlugin` when the stack declares translation bundles and no i18n service is present; self-describes as `degraded` (ADR-0076 D12) | +| **In-memory fallback** | `@objectstack/core` (`createMemoryI18n`) | Two paths, same factory: the kernel pre-injects it for any unfilled `core` slot (`CORE_FALLBACK_FACTORIES`), and `AppPlugin` registers it during `start` when the stack declares translation bundles and no i18n service is present. Self-describes as `degraded` (ADR-0076 D12) | | **Development** | `DevPlugin` | Auto-wires `I18nServicePlugin` when the stack declares translations; otherwise the AppPlugin fallback above applies. DevPlugin registers no stub of its own (ADR-0115) | ```typescript @@ -382,45 +447,48 @@ AppPlugin will: | Service | Description | |:--------|:------------| -| **file-storage** | Unified upload/download/delete. Drivers: local FS, S3, MinIO. | -| **search** | Full-text search. Drivers: in-memory, Elasticsearch, Meilisearch. | -| **cache** | General-purpose cache. In-memory fallback; Redis via `@objectstack/service-cache`. | -| **queue** | Message queue. In-memory fallback; durable DB-backed adapter via `@objectstack/service-queue` (no BullMQ/Redis adapter is shipped). | -| **job** | Scheduled task execution. In-memory fallback; cron-based, concurrency control. | +| **file-storage** | Unified upload/download/delete via `@objectstack/service-storage`, which mounts `/api/v1/storage` itself. Adapters: local FS and S3 (the S3 adapter's `endpoint` + path-style options cover S3-compatible services such as MinIO and R2). | +| **search** | **Nothing ships.** `ISearchService` and the engine enum (`elasticsearch`, `meilisearch`, …) exist in `@objectstack/spec`, but no package implements the contract or registers the `search` slot, so `CORE_SERVICE_PROVIDER.search` is `null`. | +| **cache** | General-purpose cache. In-memory fallback; memory or Redis adapter via `@objectstack/service-cache`. | +| **queue** | Message queue. In-memory fallback; durable DB-backed adapter (`sys_job_queue`) via `@objectstack/service-queue` (no BullMQ/Redis adapter is shipped). | +| **job** | Scheduled task execution via `@objectstack/service-job`. In-memory fallback; interval, cron, and DB-backed adapters with concurrency policy. | --- -## Recommended Development Phases - -### Phase 1: Foundation (P0) +## Roadmap Status -| Plugin | Description | -|:-------|:------------| -| **plugin-auth** | Identity engine — JWT, sessions, user CRUD, roles | -| **plugin-ui** | View CRUD — core low-code capability | -| **plugin-cache** | General-purpose cache — performance infrastructure | -| **Metadata DB** | Solve metadata persistence (SchemaRegistry → database) | +The three-phase build-out this page used to propose has landed, under package names +that are `service-*` more often than `plugin-*`. Use the real names — a remedy naming +a package that cannot be installed is a dead end, which is why +`CORE_SERVICE_PROVIDER` is CI-checked against the workspace. -### Phase 2: Business Engine (P1) +### Shipped -| Plugin | Description | +| Capability | Package | |:-------|:------------| -| **service-automation** | Flow orchestration, state-machine adjacent execution, approval-node pauses | -| **plugin-automation** | Triggers + flow orchestration | -| **plugin-i18n** | Internationalization | -| **plugin-storage** | File storage (local + S3) | -| **DB drivers** | PostgreSQL, MySQL, MongoDB, SQLite | - -### Phase 3: Advanced Capabilities (P2) - -| Plugin | Description | +| Identity engine — sessions, JWT, user CRUD, OAuth2/OIDC, SCIM | `@objectstack/plugin-auth` | +| RBAC, field-level permissions, RLS | `@objectstack/plugin-security` | +| Record sharing + share links | `@objectstack/plugin-sharing` | +| Metadata persistence (`sys_metadata`), HMR, cluster cache invalidation | `@objectstack/metadata` · `@objectstack/service-cluster` | +| Cache — memory + Redis adapters | `@objectstack/service-cache` | +| Flow orchestration, triggers, approval-node pauses | `@objectstack/service-automation` | +| Internationalization | `@objectstack/service-i18n` | +| File storage — local FS + S3 | `@objectstack/service-storage` | +| DB drivers — PostgreSQL / MySQL / SQLite (Knex), MongoDB, SQLite-WASM, in-memory | `@objectstack/driver-sql` · `driver-mongodb` · `driver-sqlite-wasm` · `driver-memory` | +| Notification engine + inbox | `@objectstack/service-messaging` | +| Scheduled tasks — interval, cron, DB-backed | `@objectstack/service-job` | +| Message queue — memory + durable DB-backed | `@objectstack/service-queue` | +| Realtime pub/sub (in-process; **no HTTP/WS surface yet**) | `@objectstack/service-realtime` | + +### Still open + +| Slot | State | |:-------|:------------| -| **plugin-realtime** | WebSocket / SSE real-time push | -| **plugin-notification** | Notification engine | -| **plugin-ai** | Chat, completion, models, conversations (`service-ai`, Cloud/EE) | -| **plugin-search** | Full-text search | -| **plugin-job** | Scheduled tasks | -| **plugin-queue** | Message queue | +| **ui** | Nothing registers the slot. `ViewProtocol`'s five methods are declared and unrouted; view CRUD runs through `/api/v1/meta`, and `/api/v1/ui/view/:object` is served by the `protocol` service. | +| **workflow** | Nothing ships. `WorkflowProtocol`'s three methods have no implementation and no consumer. | +| **search** | Nothing ships. Contract and engine enum exist in `@objectstack/spec` only. | +| **ai** | Nothing in this repo — `service-ai` (chat, completion, models, conversations) is Cloud/EE. | +| **realtime transport** | The service exists but no WebSocket/SSE route is mounted, so `routes.realtime` is deliberately never advertised. | --- @@ -428,11 +496,19 @@ AppPlugin will: ```typescript import type { Plugin } from '@objectstack/core'; +import type { IDataEngine } from '@objectstack/spec/contracts'; let authService: AuthServiceImpl; export const authPlugin: Plugin = { - name: 'plugin-auth', + // Plugin names are reverse-domain (e.g. 'com.objectstack.auth'), + // not the npm package name. + name: 'com.example.auth', + + // ADR-0116 — declare what init() needs and what it unconditionally + // registers, so a misordering is a named boot error instead of a crash. + requiresServices: ['data'], + providesServices: ['auth'], async init(ctx) { // Register the 'auth' CoreServiceName value @@ -442,8 +518,11 @@ export const authPlugin: Plugin = { }, async start(ctx) { - const auth = ctx.getService('auth'); - await auth.onReady(); + // Late-bind anything that must wait for every plugin to be up. + // (There is no `onReady()` on IAuthService — that is your own type's API.) + ctx.hook('kernel:ready', async () => { + await authService.connect(); + }); }, // destroy() takes no arguments — capture what you need in module scope @@ -454,9 +533,13 @@ export const authPlugin: Plugin = { ``` When a plugin registers a service, the discovery endpoint automatically updates: -- `services.auth.enabled` → `true`, `status` → `"available"`, `route` → `"/api/v1/auth"` +- `services.auth.enabled` → `true`, `status` → `"available"`, `handlerReady` → `true`, + `route` → `"/api/v1/auth"` (unless the instance self-declares `stub`/`degraded` via + `__serviceInfo`, which is reported verbatim instead) - `routes.auth` → `"/api/v1/auth"` appears in routes -- `features` capability flags update accordingly +- `features` flags follow for the slots that have one — `search`, `files`, + `analytics`, `ai`, `workflow`, `notifications`, `i18n` (`websockets` is hardcoded + `false`; there is no `features.auth`) --- diff --git a/content/docs/kernel/services.mdx b/content/docs/kernel/services.mdx index 822e9c9d9c..67306d9385 100644 --- a/content/docs/kernel/services.mdx +++ b/content/docs/kernel/services.mdx @@ -25,14 +25,19 @@ export const myPlugin: Plugin = { async init(ctx) { // Register a service with a concrete implementation - const settings = ctx.getService('settings'); ctx.registerService('cache', new RedisCacheProvider({ - url: settings.get('redis.url'), + url: 'redis://localhost:6379', })); }, }; ``` +Take configuration from the plugin's own options (as `CacheServicePlugin` and +`createApiRegistryPlugin()` do) rather than from the `settings` service: that +service is an async, namespaced resolver — `await settings.get(namespace, key)` +returns a `{ value, source, locked, … }` envelope — not a synchronous config bag +keyed by dotted paths. + ### Factory Registration (Lazy) Use `registerServiceFactory` when the service requires async initialization. @@ -44,8 +49,8 @@ context (and an optional scope id) and is wrapped in lifecycle management: import { ServiceLifecycle } from '@objectstack/core'; ctx.registerServiceFactory('data', async (ctx) => { - const settings = ctx.getService('settings'); - const pool = await createPool(settings.get('database')); + const pool = await createPool({ url: 'postgres://localhost:5432/app' }); + ctx.logger.info('Postgres pool ready'); return new PostgresDriver(pool); }, ServiceLifecycle.SINGLETON); ``` @@ -79,28 +84,34 @@ if (ctx.getServices().has('search')) { } ``` -**Presence is not capability.** A registered service may be a stub or a degraded -fallback — dev-mode fakes and the kernel's own in-memory fallbacks both occupy -real slots. They say so with the `__serviceInfo` descriptor (ADR-0076 D12), and -consumers are expected to read it rather than trust mere registration: +**Presence is not capability.** A registered service may be a degraded fallback +or an outright stub — the kernel auto-injects in-memory fallbacks into the +`metadata`, `cache`, `queue`, `job`, and `i18n` slots when no plugin provides +them, and those occupy the real slot. They say so with the `__serviceInfo` +descriptor (ADR-0076 D12), and consumers are expected to read it rather than +trust mere registration: ```typescript import { readServiceSelfInfo } from '@objectstack/spec/api'; -const ai = ctx.getServices().get('ai'); -const self = readServiceSelfInfo(ai); // undefined ⇒ a real implementation +const cache = ctx.getServices().get('cache'); +const self = readServiceSelfInfo(cache); // undefined ⇒ a real implementation -if (ai && self?.status !== 'stub') { +if (cache && self?.status !== 'stub') { // Safe: a real implementation, or a `degraded` one that genuinely works. } else if (self) { - ctx.logger.info(`ai is a stub — ${self.message}`); + ctx.logger.info(`cache is a stub — ${self.message}`); } ``` -`status: 'degraded'` means real work with reduced capability (the in-memory -cache, the in-memory search index); `status: 'stub'` means the answer is -fabricated and must not be used for real work. `handlerReady: false` -additionally means no HTTP handler serves it. +`status: 'degraded'` means real work with reduced capability (all five kernel +fallbacks above declare it); `status: 'stub'` means the answer is fabricated and +must not be used for real work. `handlerReady: false` additionally means no HTTP +handler serves it. + +Dev mode adds nothing here: `@objectstack/plugin-dev` registers no +implementations of its own, so a slot no plugin fills is empty in dev exactly as +in production (ADR-0115). ## Standard Services @@ -112,7 +123,7 @@ The core ecosystem defines several standard service contracts: | `data` | `IDataEngine` | `@objectstack/objectql` (drivers implement `IDataDriver`) | | `auth` | `IAuthService` | `plugin-auth` | | `api-registry` | `ApiRegistry` | `@objectstack/core` | -| `cache` | `ICacheService` | Redis, Memcached, or in-memory | +| `cache` | `ICacheService` | `@objectstack/service-cache` (memory adapter; its Redis adapter is still a skeleton that throws) — otherwise the kernel's in-memory fallback | | `lifecycle` | `LifecycleService` (`@objectstack/objectql`) | Registered by `ObjectQLPlugin` — enforces object `lifecycle` declarations (ADR-0057 retention/rotation/archival); call `sweep()` for an on-demand pass | The logger is not a registered service — it is exposed directly as `ctx.logger` @@ -148,21 +159,25 @@ Services follow the plugin lifecycle: 3. **`destroy`** — Clean up resources (close connections, flush buffers) ```typescript +let pool: Pool | undefined; + export const dbPlugin: Plugin = { name: 'database', async init(ctx) { - const settings = ctx.getService('settings'); - const pool = await createPool(settings.get('database')); + pool = await createPool({ url: 'postgres://localhost:5432/app' }); ctx.registerService('data', new PostgresDriver(pool)); - this.db = pool; }, async destroy() { - await this.db.close(); // Clean shutdown + await pool?.close(); // Clean shutdown }, }; ``` +`destroy()` receives no context, so keep any handle you need to close in the +plugin's own scope — a class field, or a module-level binding as above. `Plugin` +declares no state slot, so stashing it on `this` does not typecheck. + The `Plugin` interface defines `init`, `start?`, and `destroy?` — there is no `stop` hook, and `destroy()` takes no arguments. diff --git a/content/docs/permissions/authentication.mdx b/content/docs/permissions/authentication.mdx index 223ffb3430..221315a21e 100644 --- a/content/docs/permissions/authentication.mdx +++ b/content/docs/permissions/authentication.mdx @@ -91,7 +91,9 @@ Install the plugin in your ObjectStack project: pnpm add @objectstack/plugin-auth ``` -The plugin requires Better-Auth as a peer dependency, which will be automatically installed. +Better-Auth is a direct runtime dependency of the plugin (`better-auth` in +`packages/plugins/plugin-auth/package.json` — there is no `peerDependencies` +block), so it is installed with the plugin; you do not add it yourself. --- @@ -376,8 +378,10 @@ once (never overwriting your edits) and can be changed under Setup → Notification Templates. The locale follows the deployment default (`localization.locale` setting) with a `zh-CN → zh → en` fallback chain; holes are `{{code}}`, `{{appName}}`, `{{minutes}}` (OTP) and `{{appName}}`, -`{{baseUrl}}` (invitation). Template lookups are best-effort — an outage -falls back to the built-in text and never blocks an OTP send. +`{{loginUrl}}` (invitation — `{{baseUrl}}`, the bare origin, is still +interpolated for tenant templates written against the older text). Template +lookups are best-effort — an outage falls back to the built-in text and never +blocks an OTP send. --- @@ -799,9 +803,9 @@ every row through better-auth so the accounts are login-capable: link, or a password-reset link) and the Console detects the missing password (`hasLocalPassword()`) and offers set-initial-password. - `mode: 'insert' | 'upsert'` with `matchBy: 'email' | 'phone'`. Upsert - updates only touch profile fields (`name`, `phone_number`, `role`) — a - re-imported file can never modify an existing user's email or reset their - password. + updates only touch profile fields (`name`, `image`, `phone_number`, `role`) + — a re-imported file can never modify an existing user's email or reset + their password. - Synchronous only, at most 500 rows per request (password hashing is CPU-bound); split larger files into batches. Imports are not undoable. @@ -1050,7 +1054,7 @@ AuthPlugin is designed to work in **both** server and MSW/mock (browser-only) en ### How It Works - **Server mode** (HonoServerPlugin active): AuthPlugin registers HTTP routes at `/api/v1/auth/*` and forwards all requests to better-auth. -- **MSW/mock mode** (no HTTP server): AuthPlugin gracefully skips route registration but still registers the `auth` service. The `HttpDispatcher` provides mock fallback responses for core auth endpoints. +- **MSW/mock mode** (no HTTP server): AuthPlugin gracefully skips route registration but still registers the `auth` service. The dispatcher's `/auth` domain then bridges straight to that service's `handleRequest()` — it has no mock fallback of its own, and answers `501` when the `auth` slot is empty (see below). ### Minimal Configuration for Mock Mode diff --git a/content/docs/plugins/index.mdx b/content/docs/plugins/index.mdx index ea65d3ae3b..05c192eaa7 100644 --- a/content/docs/plugins/index.mdx +++ b/content/docs/plugins/index.mdx @@ -122,7 +122,7 @@ export default defineStack({ myPlugin, ], - // Development-only plugins (loaded only with `os dev`) + // Development-only plugins (loaded only with `os dev` / `os serve --dev`) devPlugins: [ ], }); @@ -199,8 +199,8 @@ Security features including field-level and row-level security. - Middleware-based security ### `@objectstack/driver-memory` -In-memory data driver for development and testing. -- Auto-registered in dev mode if no driver is configured +In-memory (mingo) data driver for development and testing. +- Opt in explicitly with `OS_DATABASE_DRIVER=memory`, `--database-driver memory`, or a `memory://` URL — when no driver is configured the dev default is SQLite, not this driver --- @@ -211,8 +211,8 @@ The `os serve` and `os dev` commands automatically detect and load plugins: | Condition | Auto-loaded Plugin | |:----------|:-------------------| | `objects` defined, no ObjectQL | `@objectstack/objectql` | -| Dev mode, no driver, has objects | `@objectstack/driver-memory` | -| `objects`, `manifest`, or `apps` defined | `AppPlugin` (runtime) | +| `objects` defined, no driver | `DefaultDatasourcePlugin` (runtime) — driver kind from `OS_DATABASE_DRIVER` / `OS_DATABASE_URL`, defaulting to SQLite in dev and to nothing in production | +| `objects`, `manifest`, `apps`, `flows`, or `apis` defined | `AppPlugin` (runtime) | | Server not disabled | `@objectstack/plugin-hono-server` | | Server enabled | `@objectstack/rest` (REST API) | @@ -225,7 +225,9 @@ export default defineStack({ }); ``` -Running `os dev` will auto-register ObjectQL, MemoryDriver, AppPlugin, HonoServer, and REST API. +Running `os dev` will auto-register ObjectQL, the default datasource (a +project-anchored SQLite file at `.objectstack/data/dev.db`), AppPlugin, +HonoServer, and the REST API. --- @@ -312,7 +314,7 @@ const slowPlugin: PluginMetadata = { Plugins can extend the ObjectStack CLI (`os`) with custom commands. This enables third-party packages — such as marketplace tools, deployment utilities, or domain-specific workflows — to register new top-level subcommands. -The CLI is built on [oclif](https://oclif.io). Command extension is handled entirely by oclif's plugin system: a plugin ships oclif `Command` classes, declares oclif config in its `package.json`, and is installed into the CLI with `os plugins install `. The host project's `objectstack.config.ts` does **not** determine CLI command availability. +The CLI is built on [oclif](https://oclif.io). Command extension is handled entirely by oclif's plugin system: a plugin ships oclif `Command` classes, declares oclif config in its `package.json`, and is loaded by the `os` binary as an oclif plugin. The host project's `objectstack.config.ts` does **not** determine CLI command availability. #### Step 1: Add oclif Config to `package.json` @@ -360,17 +362,21 @@ export default class MarketplaceSearch extends Command { } ``` -#### Step 3: Install the Plugin into the CLI +#### Step 3: Load the Plugin into the CLI -Users add the plugin to their `os` installation with oclif's plugin manager: - -```bash -os plugins install @acme/plugin-marketplace -``` + +**`os plugins install` is not available today.** `@objectstack/cli`'s `package.json` +lists `@oclif/plugin-plugins` under `oclif.plugins`, but the package sits in +`devDependencies` — and oclif's core-plugin loader only matches names that appear in +`dependencies`. The result is that neither `os plugins …` nor `os help` is a +registered command; `os --help` shows no `plugins` topic. Until that is fixed, load a +CLI extension by building an `os` distribution that lists the package in **both** +`oclif.plugins` and `dependencies`. + #### Using the Extended CLI -Once installed, the new commands appear in `os --help` and can be invoked directly: +Once loaded, the new commands appear in `os --help` and can be invoked directly: ```bash # List available commands (includes plugin commands) @@ -380,12 +386,6 @@ os --help os marketplace search "crm" ``` -To remove a plugin command, uninstall it: - -```bash -os plugins uninstall @acme/plugin-marketplace -``` - --- ## Directory Structure Convention @@ -407,12 +407,15 @@ plugin-my-feature/ │ ├── feature.view.ts # View definition │ └── index.ts # Barrel export └── services/ - └── feature.service.ts # Service implementation + └── feature-service.ts # Service implementation (plain module) ``` **Naming Rules**: - Domain directories: `snake_case` -- File suffixes: `.object.ts`, `.view.ts`, `.flow.ts`, `.service.ts`, etc. +- Semantic metadata suffixes: `.object.ts`, `.field.ts`, `.view.ts`, `.page.ts`, + `.dashboard.ts`, `.flow.ts`, `.app.ts`. ADR-0088 retired `.trigger.ts`, + `.router.ts`, `.function.ts`, and `.service.ts` — plain source modules + (services, helpers) simply carry no semantic suffix. - Entry point: `index.ts` in every module --- diff --git a/content/docs/plugins/packages.mdx b/content/docs/plugins/packages.mdx index b62b0884d1..cfe69561a2 100644 --- a/content/docs/plugins/packages.mdx +++ b/content/docs/plugins/packages.mdx @@ -26,7 +26,7 @@ ObjectStack is organized into **72 package manifests** across multiple categorie **The Constitution** — Protocol schemas, types, and constants for the entire ObjectStack ecosystem. - **Purpose**: Zod-first schema definitions for all 15 protocol domains -- **Exports**: Builder functions (`defineStack`, `defineView`, `defineApp`, `defineFlow`, `defineAgent`, `defineTool`, `defineSkill`) plus `ObjectSchema.create()` for objects. Protocol namespaces (Data, UI, System, Automation, AI, API, Identity, Security, Kernel, Cloud, QA, Contracts, Integration, Studio, Shared) are not re-exported from the top-level entry for tree-shaking reasons — import them from subpaths such as `@objectstack/spec/data` and `@objectstack/spec/ui`. +- **Exports**: Builder functions (`defineStack`, `defineView`, `defineApp`, `defineFlow`, `defineAgent`, `defineTool`, `defineSkill`) from the root entry, plus `ObjectSchema.create()` for objects from the `@objectstack/spec/data` subpath. Protocol namespaces (Data, UI, System, Automation, AI, API, Identity, Security, Kernel, Cloud, QA, Contracts, Integration, Studio, Shared) are not re-exported from the top-level entry for tree-shaking reasons — import them from subpaths such as `@objectstack/spec/data` and `@objectstack/spec/ui`. - **When to use**: Import types, schemas, and builder functions when authoring metadata. - **Documentation**: [Protocol Reference](/docs/references) @@ -98,9 +98,10 @@ const kernel = new ObjectKernel(); ### @objectstack/platform-objects -**Platform Objects Library** — The canonical set of `sys_*` objects shipped with every ObjectStack runtime (users, sessions, approvals, sharing, audit, …). +**Platform Objects Library** — The canonical set of `sys_*` objects shipped with every ObjectStack runtime (users, sessions, organizations, teams, jobs, notifications, email, settings, secrets, …). -- **Purpose**: Standard system tables and their metadata, so apps don't redefine identity, audit, or approvals +- **Purpose**: Standard system tables and their metadata, so apps don't redefine identity, jobs, or settings +- **Not here**: per ADR-0029 each domain plugin owns its own objects — `sys_audit_log` / `sys_activity` / `sys_comment` in `plugin-audit`, `sys_record_share` in `plugin-sharing`, `sys_approval_request` / `sys_approval_action` in `plugin-approvals`, the RBAC objects in `plugin-security`, `sys_presence` in `service-realtime`, and `sys_metadata*` in `@objectstack/metadata-core` - **When to use**: Always — bundled into the runtime --- diff --git a/content/docs/protocol/kernel/index.mdx b/content/docs/protocol/kernel/index.mdx index 47bc2c5855..abcdcccbb7 100644 --- a/content/docs/protocol/kernel/index.mdx +++ b/content/docs/protocol/kernel/index.mdx @@ -176,12 +176,16 @@ const apiKey = **Kernel Approach:** ```typescript -// Declarative merge strategy -const config = context.config.resolve('stripe.apiKey', { - sources: ['environment', 'tenant', 'plugin', 'default'], - required: true +// One resolver, one precedence order — the `settings` service +// (SettingsService, registered by SettingsServicePlugin) +const settings = ctx.getService('settings'); +const { value, source, locked } = await settings.get('stripe', 'apiKey', { + tenantId, + userId, }); -// Kernel merges in precedence order, validates, throws clear error if missing +// Cascade: OS_STRIPE_APIKEY (env) > global > tenant > user > registered default. +// `source` names the winning scope; `locked: true` means a higher scope pinned it +// (a write to a locked key throws SettingsLockedError). ``` **Business Value:** Configuration errors dropped 90% after adopting Kernel. New engineers can understand config logic in 5 minutes instead of 5 hours. @@ -214,7 +218,7 @@ const config = context.config.resolve('stripe.apiKey', { **Kernel Solution:** - **Package Manifest:** The `manifest` block in `objectstack.config.ts` (compiled to `objectstack.json`) declares what a package provides and needs - **Dependency Resolution:** Automatic validation that plugin versions are compatible with core platform -- **Lifecycle:** every plugin implements `init` / `start` / `destroy`, driven by `kernel.bootstrap()` and shutdown +- **Lifecycle:** plugins implement the optional `init` / `start` / `destroy` hooks, driven by `kernel.bootstrap()` and shutdown - **Sandboxing:** Plugins can't access each other's data or crash each other **Results:** @@ -234,7 +238,7 @@ const config = context.config.resolve('stripe.apiKey', { - **i18n Standard:** Translation bundles with fallback chains (`de-AT` → `de` → `en`) - **Region Config:** Configuration profiles per region (defaults + region overrides) - **Plugin System:** Regional plugins (SEPA plugin for EU, ACH plugin for US) -- **Locale Resolution:** Automatic detection from user preferences, IP geolocation, or explicit selection +- **Locale Resolution:** Automatic detection from the user's localization settings or the request's `Accept-Language` header, with explicit selection as an override **Results:** - Launched in 12 countries in 4 months (previously 18-month estimate) @@ -265,57 +269,55 @@ await kernel.use(new ObjectQLPlugin()); // Phase 3 — application metadata (objects, views, apps, flows, agents…) await kernel.use(new AppPlugin(stack)); -// Phase 4 — host (HTTP / Studio / MCP server, optional) +// Phase 4 — host (HTTP via HonoServerPlugin, MCP via MCPServerPlugin; optional) // await kernel.use(new HonoServerPlugin({ port: 3000 })); // Phase 5 — start everything await kernel.bootstrap(); ``` -The kernel validates each plugin at `use()`, then `bootstrap()` runs every -plugin's `init` in registration order followed by every plugin's `start`, and -exposes the resulting services through DI. +The kernel validates each plugin at `use()`, then `bootstrap()` topologically +sorts the registry — declared `dependencies` / `optionalDependencies` are +hoisted ahead of their dependents, registration order breaking ties — and runs +every plugin's `init` (Phase 1) followed by every plugin's `start` (Phase 2), +exposing the resulting services through DI. It then fires `kernel:ready`, +`kernel:bootstrapped`, and `kernel:listening` in that order. ### 2. Request Handling ```typescript // Incoming API request: GET /api/v1/data/account/123 -async function handleRequest(req) { - // Kernel provides context - const context = await Kernel.createContext({ - tenantId: req.headers['x-tenant-id'], - userId: req.user.id, - locale: req.headers['accept-language'], - }); - - // Kernel resolves configuration - const config = context.config.resolve(); - - // ObjectQL executes query - const account = await ObjectQL.findById('account', '123', { - context, // Kernel injects permissions, audit, etc. - }); - - // ObjectUI renders response - const view = await ObjectUI.render('account_detail', { - record: account, - locale: context.locale, // Kernel provides i18n - }); - - // Kernel logs audit trail - context.audit.log('account.view', { accountId: '123' }); - - return view; -} +// +// The transport plugin (HonoServerPlugin) hands the raw request to the REST +// layer, which resolves ONE identity envelope and threads it into the engine: + +// 1. Identity — resolveExecutionContext() reads the better-auth session (or +// API key), aggregates positions/permission sets/RLS membership, and layers +// locale + timezone on top. It always resolves; anonymous yields +// `{ isSystem: false, positions: [], permissions: [] }`. +const context = await resolveExecutionContext({ getService, getQl, request }); +// → ExecutionContext { userId, tenantId, locale, timezone, positions, +// permissions, isSystem, ... } + +// 2. Data — the context travels as `options.context`, which is what the engine +// enforces permissions and RLS against. +const ql = ctx.getService('objectql'); +const account = await ql.findOne('account', { where: { id: '123' }, context }); ``` +Auditing is not called by hand at the request boundary: `AuditPlugin` subscribes +to the engine's data hooks and writes audit records as a side effect of the +operation. + ### 3. Package Installation ```typescript -// Install a package: os package install @vendor/salesforce-sync +// Install a package: os package install com.vendor.salesforce-sync +// (the argument is a reverse-domain manifest id, or a path to a compiled +// artifact JSON for an air-gapped install) // // A package is METADATA — no package code executes at install time. // What actually happens: // -// 1. Validate — manifest shape, engines.protocol compatibility (ADR-0087), +// 1. Validate — manifest shape, engines.protocol compatibility (ADR-0025 §3.2), // artifact signature. // 2. Register — registry.installPackage() gates the namespace and records // the InstalledPackage; the durable copy lands in `sys_packages` and is @@ -344,10 +346,11 @@ permissions.grant('admin', 'accounts', 'read'); **Good (Declarative):** ```yaml # objectstack.config.ts — declared via defineStack({ ... }) +# `objects` is a list; each object's `fields` is a MAP keyed by snake_case name objects: - name: account fields: - - name: name + name: type: text # PermissionSet grants use allow* flags (not role/object/access) permissions: @@ -388,18 +391,32 @@ packages/plugins/plugin-slack-integration/ README.md ``` + + The scaffold still emits an `initialize` method. The kernel's plugin contract + only invokes `init` / `start` / `destroy` — rename `initialize` to `init` in + the generated `src/index.ts` or the hook never runs. + + ### Configuration Management ```typescript -// Define plugin configuration schema -export const configSchema = z.object({ - apiKey: z.string().describe('Slack API Key'), - channel: z.string().default('#general'), - enabled: z.boolean().default(true), -}); - -// Access in plugin code -const config = context.config.get('slack', configSchema); -// Kernel validates against schema, throws clear error if invalid +// Declare a Zod `configSchema` on the plugin object — the kernel's plugin +// loader validates the plugin's config against it at `use()` time and refuses +// to load the plugin when it fails. +export const slackPlugin: Plugin = { + name: 'slack-integration', + version: '0.1.0', + configSchema: z.object({ + apiKey: z.string().describe('Slack API Key'), + channel: z.string().default('#general'), + enabled: z.boolean().default(true), + }), + async init(ctx) { + // Runtime-resolved values come from the `settings` service, not from a + // `config` object on the context — there is no `ctx.config`. + const settings = ctx.getService('settings'); + const { value: channel } = await settings.get('slack', 'channel', {}); + }, +}; ``` ### Internationalization @@ -417,9 +434,12 @@ const config = context.config.get('slack', configSchema); "slack.error.invalid_channel": "Kanal {{channel}} nicht gefunden" } -// Use in code -const message = context.i18n.t('slack.button.send'); -// Kernel automatically uses user's locale +// Use in code — the i18n service is resolved from the registry, and the +// locale is an explicit argument (take it from the ExecutionContext). +const i18n = ctx.getService('i18n'); +const message = i18n.t('slack.button.send', context.locale); +// t(key, locale, params?) interpolates {{params}} and falls back to the +// configured fallback locale, returning the key itself when nothing matches. ``` ## Best Practices @@ -429,7 +449,7 @@ const message = context.i18n.t('slack.button.send'); **Why:** Keeps core stable. Plugins evolve independently. Customers pay for only what they use. -**Example:** Don't build email sending into Kernel core. Create `@objectstack/email` plugin. Customers who don't send emails don't load the plugin (smaller memory footprint, faster boot). +**Example:** Don't build email sending into Kernel core. Ship it as the `@objectstack/plugin-email` package. Customers who don't send emails don't load the plugin (smaller memory footprint, faster boot). ### 2. Configuration Over Code **Principle:** Behavior should be configurable without code changes. @@ -443,7 +463,7 @@ const message = context.i18n.t('slack.button.send'); **Why:** Easier to add multi-tenancy later = rebuild entire system. -**Example:** Don't store config in global variables. Use `context.config.get()` which automatically scopes to current tenant. +**Example:** Don't store config in global variables. Read through the `settings` service — `settings.get(namespace, key, { tenantId, userId })` walks the global → tenant → user cascade for you. ### 4. Version Everything **Principle:** All artifacts (plugins, schemas, configs) have semantic versions. @@ -457,7 +477,7 @@ const message = context.i18n.t('slack.button.send'); **Why:** Catch errors before customers see them. -**Example:** Plugin requires `apiKey` config. If missing, Kernel throws error during boot with clear message: "Plugin @vendor/slack requires config key 'slack.apiKey' (not found in environment, tenant, or default config)". +**Example:** A plugin declares `apiKey` in its `configSchema`. If it is missing, the plugin loader fails the load with the Zod issues spelled out per key — `Plugin slack-integration configuration validation failed:` followed by ` - apiKey: Required` — so boot stops instead of a request failing later. ## Comparison: Kernel vs Alternatives diff --git a/content/docs/protocol/kernel/plugin-spec.mdx b/content/docs/protocol/kernel/plugin-spec.mdx index 1c420e7d3b..90f20e9e7b 100644 --- a/content/docs/protocol/kernel/plugin-spec.mdx +++ b/content/docs/protocol/kernel/plugin-spec.mdx @@ -9,8 +9,8 @@ import { Package, FileCode, Folder, Box, Link2, Shield, Settings } from 'lucide- **Protocol spec.** This page describes the **target** plugin packaging -specification — `plugin.manifest.ts`, semantic-version dependency resolution, -provider/consumer contracts, lifecycle hooks. The current runtime implements +specification — an ergonomic manifest wrapper, semantic-version dependency +resolution, provider/consumer contracts. The current runtime implements plugins as classes/objects matching the `Plugin` interface from `@objectstack/core`, and manifests as plain objects matching `ManifestSchema` from `@objectstack/spec/kernel`. There is no `definePlugin()` @@ -28,17 +28,23 @@ Every plugin **must** have a manifest file that declares its identity, dependenc ``` my-plugin/ - plugin.manifest.ts ← TypeScript (recommended) - plugin.manifest.yml ← YAML (alternative) - plugin.manifest.json ← JSON (alternative) + src/manifest.ts ← TypeScript source: ManifestSchema.parse({ … }) + objectstack.config.ts ← stack entry: defineStack({ manifest, objects, … }) + objectstack.plugin.json ← distributable manifest read by `os plugin build` ``` -**Recommendation:** Use TypeScript manifest for **type safety** and **validation**. ObjectStack auto-generates JSON schema from TypeScript. +There is no `plugin.manifest.ts` / `.yml` / `.json` — that filename appears nowhere in +the implementation. The shape is always `ManifestSchema` from `@objectstack/spec/kernel`. +`os plugin build` reads the JSON form (`objectstack.plugin.json`, exported as +`MANIFEST_FILENAME` from `packages/cli/src/utils/osplugin.ts`) and validates it with +`ManifestSchema.safeParse`; TypeScript authoring runs the same object through +`ManifestSchema.parse(...)` and hands it to `defineStack({ manifest })`. ### Manifest Schema ```typescript -// plugin.manifest.ts +// src/manifest.ts — proposed authoring wrapper (see the callout below for the +// fields ManifestSchema actually declares today) import { definePlugin } from '@objectstack/core'; export default definePlugin({ @@ -54,7 +60,7 @@ export default definePlugin({ // DEPENDENCIES dependencies: { '@objectstack/core': '^2.0.0', - '@mycompany/base': '>=1.0.0 <2.0.0', + '@mycompany/base': '1.0.0 - 2.0.0', }, // OPTIONAL DEPENDENCIES (plugin works without these) @@ -102,16 +108,14 @@ export default definePlugin({ // never declared it and nothing ever loaded those files (#4212). Runtime // behaviour lives on the plugin class (`init`/`start`/`destroy`, below). - // PERMISSIONS + // PERMISSIONS — proposed shape only. The real block has no `system` or + // `objects` key; see "Permissions" below for what ManifestSchema accepts. permissions: { - // System capabilities plugin requires system: [ - 'network.http', // Make HTTP requests - 'storage.database', // Database access - 'storage.cache', // Redis/cache access + 'network.http', + 'storage.database', + 'storage.cache', ], - - // Objects plugin creates/manages objects: [ 'account', 'contact', @@ -163,13 +167,32 @@ export default definePlugin({ }); ``` + +**What `ManifestSchema` actually declares today** +(`packages/spec/src/kernel/manifest.zod.ts`): `id`, `version`, `type` and `name` are +required; the optional fields are `namespace`, `defaultDatasource`, `scope`, +`description`, `permissions`, `objects`, `datasources`, `dependencies`, +`configuration`, `contributes`, `data`, `capabilities`, `extensions`, +`navigationContributions`, `loading`, `engine`, `engines`, `runtime`, `packaging` +and `integrity`. + +The `displayName` / `author` / `license` / `homepage` / `optionalDependencies` / +`peerDependencies` / `metadata` / `config` / `marketplace` keys above are +**proposal-only** — the schema declares none of them. Notably: compatibility ranges +live in `engines: { platform, protocol }` (or the legacy `engine: { objectstack }`), +not `marketplace.compatibility`; config defaults live in +`configuration: { title, properties }` (a simplified JSON-Schema map with a per-key +`secret` flag), not `config.defaults` / `config.secrets`; and metadata globs are the +top-level `objects` / `datasources` arrays, not a `metadata` block. + + ## Directory Structure A well-organized plugin follows this **standard structure**: ``` @mycompany/crm/ -├── plugin.manifest.ts # Plugin manifest (required) +├── objectstack.plugin.json # Plugin manifest — ManifestSchema (required) ├── package.json # NPM package metadata ├── README.md # Documentation ├── CHANGELOG.md # Version history @@ -202,12 +225,9 @@ A well-organized plugin follows this **standard structure**: │ ├── jobs/ # Background jobs │ │ └── cleanup.job.ts │ │ -│ ├── config.schema.ts # Configuration schema (Zod) -│ │ -│ └── lifecycle/ # Lifecycle hooks -│ ├── install.ts -│ ├── boot.ts -│ └── upgrade.ts +│ └── config.schema.ts # Configuration schema (Zod) +│ # (no src/lifecycle/ — the install/boot/upgrade +│ # hook family was retired, #4212) │ ├── i18n/ # Translations │ ├── en.json @@ -397,13 +417,19 @@ export type PluginConfig = z.infer; ### Dependency Types + +Only `dependencies` exists in `ManifestSchema` today (a +`Record`). `optionalDependencies` and `peerDependencies` +below are **proposal-only** — the schema declares neither, so nothing resolves them. + + #### 1. **Dependencies (Required)** Plugin **cannot function** without these. ```typescript dependencies: { '@objectstack/core': '^2.0.0', - '@mycompany/base': '>=1.0.0 <2.0.0', + '@mycompany/base': '1.0.0 - 2.0.0', } ``` @@ -415,8 +441,10 @@ optionalDependencies: { '@vendor/email': '^3.0.0', } -// In plugin code -if (context.plugins.isInstalled('@vendor/email')) { +// In plugin code — `PluginContext` (packages/core/src/types.ts) has no +// `plugins` namespace and no `isInstalled()`. Probe the service registry: +if (ctx.getServices().has('email')) { + const email = ctx.getService('email'); await email.send({ to: contact.email, subject: 'Welcome' }); } ``` @@ -434,7 +462,8 @@ Use for large dependencies (React, Vue) that should be shared across plugins. ### Version Constraints -ObjectStack uses **semantic versioning (semver)** with these operators: +ObjectStack uses **semantic versioning (semver)**. `SemanticVersionManager.satisfies()` +(`packages/core/src/dependency-resolver.ts`) accepts exactly these forms: ```typescript dependencies: { @@ -447,11 +476,17 @@ dependencies: { // Minor updates allowed (1.x.x) 'plugin-c': '^1.0.0', - // Version range - 'plugin-d': '>=1.0.0 <2.0.0', + // Single bound: >= , > , <= , < + 'plugin-d': '>=1.0.0', + + // Inclusive range — the hyphen form is the ONLY two-bound form the + // resolver parses. A space-separated `>=1.0.0 <2.0.0` is not supported: + // `parse()` anchors its semver regex, so the trailing bound throws + // "Invalid semantic version". + 'plugin-e': '1.0.0 - 2.0.0', - // Latest version - 'plugin-e': '*', // ⚠️ Not recommended for production + // Latest version (`*` and `latest` both match everything) + 'plugin-f': '*', // ⚠️ Not recommended for production } ``` @@ -472,20 +507,30 @@ ObjectStack builds a **dependency graph** and loads plugins in **topological ord **Load Order:** core → base → crm → sales **Conflict Resolution:** -If two plugins require incompatible versions of the same dependency, installation fails with error: +When two plugins require incompatible versions of the same dependency, +`DependencyResolver.detectConflicts()` reports it as a structured +`DependencyConflict` record rather than a formatted CLI banner: +```typescript +{ + type: 'version-mismatch', // | missing-dependency | circular-dependency + // | incompatible-versions | conflicting-interfaces + severity: 'error', + description: 'Version mismatch for @objectstack/core: detected 1 unsatisfied requirements', + plugins: [ + { pluginId: '@objectstack/core', version: '2.4.0' }, + { pluginId: '@vendor/analytics', version: '^3.0.0' }, + ], + resolutions: [ + // strategy: upgrade | downgrade | replace | disable | manual + { strategy: 'upgrade', description: 'Upgrade @objectstack/core to satisfy all constraints' }, + ], +} ``` -Error: Dependency conflict - @mycompany/sales requires @objectstack/core@^2.0.0 - @vendor/analytics requires @objectstack/core@^3.0.0 - -Cannot satisfy both constraints. -Solutions: - 1. Upgrade @mycompany/sales to version compatible with core@3.x - 2. Downgrade @vendor/analytics to version compatible with core@2.x - 3. Contact plugin authors to update dependencies -``` +Circular dependencies are reported separately with `severity: 'critical'`; the kernel +also throws on them directly during `bootstrap()` (`[Kernel] Circular dependency +detected: …`). ## Runtime Contract @@ -497,8 +542,8 @@ export class CRMPlugin implements Plugin { name = 'plugin.crm'; version = '2.0.0'; - // Phase 1 — sequential, registration order. Register services, schemas, - // routes. Other plugins' services may not exist yet. + // Phase 1 — sequential, dependency-topological order. Register services, + // schemas, routes. Other plugins' services may not exist yet. async init(ctx: PluginContext) { ctx.registerService('crm', new CRMService()); } @@ -524,8 +569,10 @@ A plugin may also expose `healthCheck()`, which the kernel invokes on demand During **boot** (`kernel.bootstrap()`): ``` 1. use(plugin) — validate structure + version, store (no code runs) -2. init — every plugin, sequentially, registration order -3. start — every plugin that defines it +2. init — every plugin, sequentially, in dependency-topological + order (kernel.resolveDependencies() walks each plugin's + `dependencies`; registration order when none declared) +3. start — same order, every plugin that defines it 4. kernel:ready → kernel:bootstrapped → kernel:listening events ``` @@ -554,64 +601,60 @@ against them silently never ran. The family was retired from the protocol ## Permissions -Plugins must **declare permissions** they require. ObjectStack enforces these at runtime. +Plugins **declare permissions** in the manifest. The declaration is the install-time +consent request; what the runtime enforces is the *granted* set. -### System Permissions +### Declared Permissions + +`ManifestSchema.permissions` accepts either the legacy flat `string[]` or the structured +`PluginPermissionsSchema` block — four keys, and no `system` list +(`packages/spec/src/kernel/manifest.zod.ts`): ```typescript permissions: { - system: [ - // Network access - 'network.http', // Make HTTP requests - 'network.websocket', // Use WebSockets - - // Storage access - 'storage.database', // Query database - 'storage.cache', // Use Redis/cache - 'storage.filesystem', // Read/write files - - // System capabilities - 'system.cron', // Schedule jobs - 'system.email', // Send emails - 'system.events', // Publish/subscribe events - - // Security - 'security.encrypt', // Encrypt data - 'security.sign', // Sign JWTs - ], + services: ['object', 'http'], // services the plugin may resolve + hooks: ['record.beforeInsert'], // lifecycle hooks it may register + network: ['api.acme.com'], // hosts it may reach + fs: [], // filesystem paths it may access } ``` -If plugin attempts operation without permission, ObjectStack throws error: +Dotted capability strings like `network.http`, `network.websocket` and +`storage.database` are values of `ResourceTypeSchema` in the plugin-sandbox permission +*descriptor* (`packages/spec/src/kernel/plugin-security-advanced.zod.ts`) — they are not +manifest keys. `storage.cache`, `storage.filesystem`, `system.cron`, `system.email`, +`system.events`, `security.encrypt` and `security.sign` do not exist anywhere in the +implementation. -```javascript -// Plugin tries to make HTTP request without permission -await fetch('https://api.example.com/data'); - -// Error: PermissionDenied -// Plugin @mycompany/crm does not have permission 'network.http' -// Add to plugin.manifest.ts: -// permissions: { system: ['network.http'] } -``` + +**Enforcement is narrower than a sandbox.** `PluginPermissionEnforcer` +(`packages/core/src/security/plugin-permission-enforcer.ts`) derives +`canAccessService` / `canTriggerHook` / `canReadFile` / `canWriteFile` / +`canNetworkRequest` from the granted set, but only **service access** is wired into a +live code path: `SecurePluginContext.getService()` and `.replaceService()` call +`enforceServiceAccess`. `enforceNetworkAccess`, `enforceFileRead` and `enforceFileWrite` +have no non-test call sites, and nothing patches global `fetch` — a plugin that calls +`fetch()` directly is **not** intercepted today. Treat `network` / `fs` declarations as +consent metadata, not a runtime jail. + -### Object Permissions +### Declaring Objects -Declare which objects plugin creates and manages: +A package declares the objects it ships through the manifest's top-level `objects` glob +list — `ManifestSchema` has no `permissions.objects` key: ```typescript -permissions: { - objects: [ - 'account', // Full control - 'contact', - 'opportunity', - ], -} +objects: [ + './src/objects/*.object.ts', +] ``` -ObjectStack uses this for: -- **Dependency tracking:** Can't uninstall plugin if other plugins reference its objects -- **Security:** Plugin can only modify its own objects (not objects from other plugins) -- **Cleanup:** Uninstalling plugin can optionally remove its objects +Ownership is tracked by the package registry rather than by a permission declaration: +install records the package and its metadata (`registry.installPackage()` → +`sys_packages`), and uninstall unregisters it and runs the data-plane cleanups domain +plugins registered via `protocol.registerUninstallCleanup(name, cleanup)` — that is how +plugin-security revokes its package-owned permission sets. There is no runtime rule +preventing one plugin from writing to another plugin's objects. ## Distribution Format @@ -859,8 +902,9 @@ Always include migration guide in CHANGELOG.md for major versions. - `account.owner` field (use `account.owner_id` instead) ### Migration -Upgrading to v2.0.0 applies the metadata migration that renames the field on -existing records (ADR-0087). No plugin code runs at upgrade. +Ship a metadata migration with the upgrade (ADR-0087) that renames the field on +existing records. There is no `onUpgrade` hook to hang this on: the schema +declares one (`PluginLifecycleSchema`) but nothing in the kernel calls it. ``` ### 3. Pin Core Dependencies @@ -873,39 +917,47 @@ dependencies: { ``` ### 4. Validate Configuration Early -Use Zod schema to validate config during boot, not at runtime: +Parse config with your Zod schema in `init()` — the first phase that runs — so a bad +value fails the boot instead of the first request. There is no `onBoot` hook (the name +appears nowhere in the implementation), and `PluginContext` has no `config` member: ```typescript -export async function onBoot({ context }) { - const config = context.config.get('crm', configSchema); - // Throws clear error if config is invalid +export class CRMPlugin implements Plugin { + name = 'plugin.crm'; + constructor(private readonly options: unknown) {} + + async init(ctx: PluginContext) { + // Throws a clear error before anything starts, not on the first request + const config = configSchema.parse(this.options); + ctx.registerService('crm', new CRMService(config)); + } } ``` -### 5. Clean Up on Uninstall -Always provide `onUninstall` hook to clean up resources: +### 5. Release Resources in `destroy()` +`destroy()` is the only cleanup seam the kernel invokes — on shutdown (reverse +registration order) and on rollback when a later plugin's `start()` fails. There is no +`onUninstall` hook: `PluginLifecycleSchema` declares one but nothing calls it, and +`PluginContext` exposes no `db` or `scheduler` member. ```typescript -export async function onUninstall({ context, transaction }) { - // Stop jobs - await context.scheduler.delete('crm-sync', { transaction }); - - // Archive data (don't delete!) - await context.db.update('account', - { plugin: 'crm' }, - { archived: true }, - { transaction } - ); +async destroy() { + clearInterval(this.syncTimer); + await this.connection.close(); } ``` +For data-plane cleanup at **package**-uninstall time, register a named cleanup with the +metadata protocol instead — `protocol.registerUninstallCleanup('crm-archive', fn)`, which +`protocol.deletePackage()` runs and whose outcome rides on the response. + ## Summary Plugin packages in ObjectStack: -- **Manifest-driven:** `plugin.manifest.ts` is the source of truth +- **Manifest-driven:** `ManifestSchema` is the source of truth (`objectstack.plugin.json` inside a built artifact) - **Self-contained:** Bundle objects, views, logic, and config - **Dependency-managed:** Semantic versioning with conflict detection -- **Lifecycle-aware:** Hooks for install, upgrade, boot, and uninstall +- **Lifecycle-aware:** `init` → `start` → `destroy` on the plugin class — no install/upgrade/boot/uninstall hooks - **NPM-compatible:** Distribute via NPM or `.osplugin` archives **Next:** Learn how configuration is resolved in [Configuration Resolution](/docs/protocol/kernel/config-resolution). diff --git a/content/docs/protocol/objectql/query-syntax.mdx b/content/docs/protocol/objectql/query-syntax.mdx index 8fca01fc28..15d9caba1e 100644 --- a/content/docs/protocol/objectql/query-syntax.mdx +++ b/content/docs/protocol/objectql/query-syntax.mdx @@ -539,9 +539,10 @@ The engine resolves `expand` via batch `$in` queries (driver-agnostic) with a de ### Filtered Expand -Expansion follows **`lookup`**, **`master_detail`**, and **`user`** fields — i.e. the -foreign key lives on the object you are querying. The nested `QueryAST` can **filter** -(`where`) and **select** (`fields`) the related records: +Expansion follows the same reference field types as above — **`lookup`**, +**`master_detail`**, **`user`** and **`tree`** — i.e. the foreign key lives on the object +you are querying. The nested `QueryAST` can **filter** (`where`) and **select** +(`fields`) the related records: ```typescript const query: QueryAST = { @@ -921,9 +922,9 @@ const sorted = monthlyRevenue.sort((a, b) => ### Unknown Fields Are Tolerated -Unknown field **names** are not rejected — a projected field that doesn't exist on the -object is **silently dropped** (matching OData / `SELECT *` tolerance), so a stale field -reference never fails the whole query: +Unknown field **names** are not rejected by `engine.find()` — a projected field that +doesn't exist on the object is **silently dropped** (matching OData / `SELECT *` +tolerance), so a stale field reference never fails the whole query: ```typescript const rows = await engine.find('customer', { @@ -932,6 +933,15 @@ const rows = await engine.find('customer', { // Returns each row with `name`; `nonexistent` is omitted — no error thrown. ``` + +This tolerance does **not** extend to the REST/protocol ingress: a `select` / `fields` +naming a column the object does not have is `400 INVALID_FIELD` there, because dropping +it silently answered a *narrower* projection with a *wider* one — a projection left with +no known field falls all the way back to every field. The engine's tolerance guards +internal callers (hooks, flows, expand sub-reads, registry-less hosts) that never pass +through that ingress. + + ### Security Violations Row-level scoping is applied by *narrowing* the query — the security middleware @@ -979,8 +989,7 @@ Similarly, the following legacy field names should not be used in new code: |:-------|:----------|:------| | `filters` (array of tuples) | `where` (FilterCondition object) | Use `parseFilterAST()` to convert | | `sort` | `orderBy` | Array of `{ field, order }` objects | -| `group_by` | `groupBy` | Array of field name strings | -| `aggregate` (object map) | `aggregations` (array) | Array of `AggregationNode` objects | +| `aggregate` | `aggregations` | Same `AggregationNode[]`; the SQL and MongoDB drivers read either key | | `expand` (string array) | `expand` (Record) | Map of field name → nested QueryAST | | `skip` | `offset` | Number | | `select` | `fields` | Array of FieldNode (field-name strings) | diff --git a/content/docs/protocol/objectql/state-machine.mdx b/content/docs/protocol/objectql/state-machine.mdx index 9d96d87bec..e6a9f30ae5 100644 --- a/content/docs/protocol/objectql/state-machine.mdx +++ b/content/docs/protocol/objectql/state-machine.mdx @@ -8,7 +8,7 @@ description: Define strict business logic constraints prevents AI hallucinations The **State Machine** (`state_machine` validation rule) lets you define the "Constitution" of a record's lifecycle: the legal `status` transitions a record may take. It is a **flat, textbook finite-state-machine** transition table that the write path enforces. - Per [ADR-0020](https://github.com/objectstack-ai/objectstack/blob/main/docs/adr/0020-state-machine-converge-and-enforce.md), a state machine is **one of the object's `validations`** — a rule with `type: 'state_machine'`. There is no top-level `stateMachine`/`stateMachines` property on an object, and the older XState-style shape (hierarchical states, `on`/`cond`/`actions`, `meta.aiInstructions`) was retired. A flat `{ from: [to] }` transition table is the only enforced shape. + Per [ADR-0020](https://github.com/objectstack-ai/objectstack/blob/main/docs/adr/0020-state-machine-converge-and-enforce.md), a state machine is **one of the object's `validations`** — a rule with `type: 'state_machine'`. There is no top-level `stateMachine`/`stateMachines` property on an object, and the older XState-style shape (hierarchical states, `on`/`cond`/`actions`, `meta.aiInstructions`) was retired **as a record-lifecycle declaration** — nothing on the write path reads it. A flat `{ from: [to] }` transition table is the only enforced shape. ## Why State Machines? @@ -56,13 +56,19 @@ export const PurchaseRequest = ObjectSchema.create({ // below). `initialStates` — the optional FSM entry point that locks // which states a record may be CREATED in (#3165) — is checked on // INSERT, so to use it here you'd also add 'insert' to `events`. - // Otherwise the field's `default: true` option is the starting value. + // Otherwise INSERT is unchecked by this rule and ANY declared option is + // a legal starting value. (An option's `default: true` is an authoring + // hint — only a field-level `defaultValue` fills an omitted field.) events: ['update'], message: 'Invalid purchase status transition.', transitions: { draft: ['pending'], pending: ['approved', 'rejected'], - // `approved` and `rejected` are dead-end states (no outgoing edges). + // Terminal states need an EXPLICIT empty array to be enforced: a + // state with no key at all is one the checker cannot reason about, + // so it does not block moves out of it (see "Enforcement semantics"). + approved: [], + rejected: [], }, }, ], @@ -85,25 +91,27 @@ A `state_machine` rule shares the common validation-rule fields (`name`, `label` ### Transitions -`transitions` is the whole legal graph. The keys are current state values; each value is the list of states the record may move *to*. A state with no key (or an empty array) is a dead-end with no outgoing edges. +`transitions` is the whole legal graph. The keys are current state values; each value is the list of states the record may move *to*. An **explicit empty array** is an enforced dead-end — every move out of it is rejected. A state with **no key at all** is different: the checker has nothing to reason about there and lets the write through (see [Enforcement semantics](#enforcement-semantics)), so declare `[]` when you mean "terminal". ```typescript transitions: { draft: ['pending'], pending: ['approved', 'rejected'], + approved: [], // terminal — enforced + rejected: [], // terminal — enforced } ``` ### Enforcement semantics -- **On insert**, the `transitions` table is not consulted — there is no prior state to transition from. If the rule declares `initialStates`, the created value must be one of them, or the write is rejected with `invalid_initial_state` (the FSM entry point). Without `initialStates`, insert is a no-op and the starting value is constrained only by the field-level `select` check — typically the option marked `default: true`. -- **On update**, if the state field changed and the new value is **not** in `transitions[oldValue]`, the write is rejected. -- The check is **lenient where it cannot reason**: if the prior state is not described by the table (e.g. legacy or externally-written data), it does not block. +- **On insert**, the `transitions` table is not consulted — there is no prior state to transition from. If the rule declares `initialStates`, the created value must be one of them, or the write is rejected with `invalid_initial_state` (the FSM entry point). Without `initialStates`, insert is a no-op and the starting value is constrained only by the field-level `select` option-membership check (`invalid_option`), so any declared option is a legal start. +- **On update**, if the state field changed and the new value is **not** in `transitions[oldValue]`, the write is rejected. Clearing the field (writing `null`) is exempt. +- The check is **lenient where it cannot reason**: if the prior state has no key in `transitions` (legacy or externally-written data, or a state you simply forgot to declare), it does not block. Only an explicit `[]` makes a state a hard dead-end. - Only a rule with `severity: 'error'` (the default) blocks the write; `warning`/`info` are logged. - **Seed writes are exempt** (#3433). Curated seed data — package bootstrap fixtures, marketplace templates, per-org replay, all loaded by `SeedLoaderService` — is a snapshot of established facts, not a record walking its lifecycle, so it bypasses the `state_machine` rule entirely: a seed may be born mid-lifecycle (a `completed` project, a `closed_won` opportunity) and neither `initialStates` (insert) nor `transitions` (update) is enforced. Every *other* validation still runs, so a seed must still satisfy field shape, `format`, `script`, and the rest. `os lint` warns when a seeded value is not a state the machine declares, so a typo is still caught before boot. - **A "historical" data import is exempt too** (#3479). Migrating established facts — a batch of already-`closed` tickets, `closed_won` deals — is the same "snapshot, not a lifecycle event" situation. Set `treatAsHistorical: true` on the import request (default **off**) and the runner puts `skipStateMachine` on the write context, so `initialStates` doesn't reject those mid-lifecycle rows. A normal import leaves it off and still walks the FSM — the strict behavior is the default, so the exemption is always an explicit opt-in. - **`treatAsHistorical` also preserves the original audit timeline** (#3493). Skipping the FSM is only half of migrating established facts; the other half is keeping *when* they happened and *who* did them. Under the same flag the write context also carries `preserveAudit`, which (1) makes `updated_at` / `updated_by` **client-preferred** — a supplied historical last-modified survives instead of being stamped with the import instant, symmetric with how `created_at` / `created_by` already behave on insert — and (2) admits a **whitelist** through the static-`readonly` write strip: the audit/timestamp family plus author-declared business `readonly` fields (`closed_at`, `resolved_by`, …), so an `upsert` refresh no longer silently drops them. Platform-managed `system` columns outside that family (`organization_id` and other tenancy/generated columns) stay stripped — a historical import reinstates facts, it does not forge tenancy. Like the FSM exemption this is opt-in: a normal write still auto-stamps `updated_at`/`updated_by` and strips `readonly` exactly as before, and permissions / RLS / field-level security are unchanged. -- **Undoing a historical import is symmetric** (#3549 / #3556). The import undo (`POST /data/import/jobs/:jobId/undo`) logically rolls back a finished job — deleting the rows it created and restoring the captured pre-import snapshot on the rows it updated. That restore write now carries `preserveAudit` too, but **only** when the job was flagged `treatAsHistorical`, so the snapshotted `updated_at` / `updated_by` and business `readonly` fields (`closed_at`, …) are reinstated verbatim instead of being re-stamped to the undo instant. Without it the undo would silently overwrite the very timeline the historical import preserved; a normal (non-historical) import's undo keeps the default stamp/strip. +- **Undoing a historical import is symmetric** (#3549 / #3556). The import undo (`POST /api/v1/data/import/jobs/:jobId/undo`) logically rolls back a finished job — deleting the rows it created and restoring the captured pre-import snapshot on the rows it updated. That restore write now carries `preserveAudit` too, but **only** when the job was flagged `treatAsHistorical`, so the snapshotted `updated_at` / `updated_by` and business `readonly` fields (`closed_at`, …) are reinstated verbatim instead of being re-stamped to the undo instant. Without it the undo would silently overwrite the very timeline the historical import preserved; a normal (non-historical) import's undo keeps the default stamp/strip. ### Conditional transitions @@ -113,7 +121,7 @@ A `state_machine` rule has no per-transition guard. To gate a transition on a pr Because the transition table is data, both UIs and Agents can ask "from this state, what's legal next?" instead of parsing a formula. -- **In code**, `legalNextStates(objectSchema, field, currentState)` from `@objectstack/objectql` returns the declared next states, `[]` for a known dead-end, or `null` when no `state_machine` rule governs the field. +- **In code**, `legalNextStates(objectSchema, field, currentState)` from `@objectstack/objectql` returns the declared next states, `[]` when the state has no outgoing edges (an explicit `[]` **or** a state the table never mentions — introspection does not distinguish the two, enforcement does), or `null` when no `state_machine` rule governs the field. - **Over HTTP**, `GET /api/v1/meta/objects/:name/state/:field?from=:state` returns `{ object, field, from, next }`, where `next` is the legal-next list (or `null`). diff --git a/content/docs/releases/implementation-status.mdx b/content/docs/releases/implementation-status.mdx index 0ffa1eaef9..d78bf33606 100644 --- a/content/docs/releases/implementation-status.mdx +++ b/content/docs/releases/implementation-status.mdx @@ -19,6 +19,7 @@ This matrix is generated from actual codebase analysis and represents the curren |:------:|:-------|:------------| | ✅ | **Fully Implemented** | Production-ready implementation with all core features | | ⚠️ | **Partially Implemented** | Basic implementation, missing advanced features | +| 🟡 | **Partially Delivered** | Shipping implementation; full protocol parity still in progress | | 🚧 | **In Progress** | Currently being developed | | 📋 | **Planned** | Scheduled for future implementation | | ❌ | **Not Implemented** | Protocol defined but not yet implemented | @@ -159,14 +160,14 @@ This matrix is generated from actual codebase analysis and represents the curren | **HTTP Server** | ❌ | ✅ | ✅ | ❌ | ✅ Full | | **Endpoint** | ✅ | ❌ | ✅ | ❌ | ✅ Full | | **Router** | ✅ | ✅ | ✅ | ❌ | ✅ Full | -| **Discovery** | ❌ | ✅ | ✅ | ✅ | ✅ Full | +| **Discovery** | ✅ | ✅ | ✅ | ✅ | ✅ Full | | **Contract** | ✅ | ✅ | ❌ | ❌ | ⚠️ Partial | | **Protocol** | ❌ | ✅ | ❌ | ✅ | ✅ Full | | **Errors** | ✅ | ✅ | ✅ | ✅ | ✅ Full | | **HTTP Cache** | ✅ | ✅ | ✅ | ✅ | ✅ Full | | **Batch** | ✅ | ✅ | ✅ | ✅ | ✅ Full | -The data (`/api/v1/data`), metadata (`/api/v1/meta`), and batch endpoints are implemented in `@objectstack/rest` (`rest-server.ts`); `@objectstack/runtime` re-exports `RestServer` from that package and provides the underlying HTTP server/dispatcher. +The data (`/api/v1/data`), metadata (`/api/v1/meta`), discovery (`/api/v1/discovery`), and batch endpoints are implemented in `@objectstack/rest` (`rest-server.ts` — `registerDiscoveryEndpoints`); `@objectstack/runtime` re-exports `RestServer` from that package and provides the underlying HTTP server/dispatcher. **REST Endpoints Implemented:** @@ -196,8 +197,8 @@ Every route carries the `/api/v1` prefix. When project scoping is enabled each r |:---------|:-------------:|:------:|:------| | **Analytics** | ✅ | ✅ | ObjectQL aggregation plus `@objectstack/service-analytics` dataset execution; analytics read scope auto-bridges to `security.getReadFilter` for RLS-aware dashboards/reports | | **OData** | ❌ | 📋 | Protocol defined, not implemented | -| **Realtime** | @objectstack/service-realtime | ⚠️ | Realtime service with in-memory adapter shipped; production adapters in progress | -| **WebSocket** | @objectstack/service-realtime | ⚠️ | Transport provided via the realtime service in-memory adapter | +| **Realtime** | @objectstack/service-realtime | ⚠️ | Realtime service shipped, but only the process-local `'memory'` adapter (`in-memory-realtime-adapter.ts`); a Redis-backed HA adapter is a post-GA fast-follow | +| **WebSocket** | ❌ | 📋 | Protocol defined (`spec/api/websocket.zod.ts`); no WebSocket transport ships in any package — the realtime service is in-process pub/sub, not a socket transport | --- @@ -309,21 +310,26 @@ describe that separate runtime. |:---------|:-----------------:|:-------------:|:------:| | **Agent** | ✅ | ObjectOS runtime | ✅ Agent runtime | | **Model Registry** | ✅ | ObjectOS runtime | ✅ Model registry | -| **RAG Pipeline** | ✅ | @objectstack/service-knowledge | ⚠️ Knowledge service + `knowledge-memory` / `knowledge-ragflow` / `embedder-openai` plugins | -| **NLQ** | ✅ | ObjectOS runtime | ⚠️ Data-query tools | +| **RAG Pipeline** | ❌ | @objectstack/service-knowledge | ⚠️ Knowledge service + `knowledge-memory` / `knowledge-ragflow` / `embedder-openai` plugins | +| **NLQ** | ❌ | ObjectOS runtime | ⚠️ Data-query tools | | **Conversation** | ✅ | ObjectOS runtime | ✅ In-memory + ObjectQL conversation services | -| **Agent Action** | ✅ | ObjectOS runtime | ⚠️ Action/data/knowledge tools | -| **Cost** | ✅ | ❌ | ❌ Spec only | -| **Predictive** | ✅ | ❌ | ❌ Spec only | -| **Orchestration** | ✅ | ObjectOS runtime | ⚠️ Agent tool orchestration | -| **Feedback Loop** | ✅ | ObjectOS runtime | ⚠️ Eval harness | -| **DevOps Agent** | ✅ | ❌ | ❌ Spec only | +| **Agent Action** | ✅ | ObjectOS runtime | ⚠️ Action/data/knowledge tools (`tool.zod.ts` + `action_` tools) | +| **Cost** | ✅ | ❌ | ❌ Spec only — `usage.zod.ts` accounts tokens/per-call cost; no budget enforcement | +| **Predictive** | ❌ | ❌ | ❌ Not defined | +| **Orchestration** | ❌ | ObjectOS runtime | ⚠️ Agent tool orchestration | +| **Feedback Loop** | ❌ | ObjectOS runtime | ⚠️ Eval harness | +| **DevOps Agent** | ❌ | ❌ | ❌ Not defined | **Notes:** -- Complete AI protocol suite defined +- The `@objectstack/spec/ai` surface is deliberately narrow — Agent, Skill, Tool, + Conversation, Model Registry, Embedding, Usage, MCP, plus the knowledge-source / + knowledge-document and solution-blueprint schemas. Application-level protocols + (DevOps agents, predictive pipelines, AIOps, orchestration plans, NLQ services, + RAG-pipeline DSLs, budget enforcement) were **removed in v1** and have no schema + today — see the scope note in `packages/spec/src/ai/index.ts` - The ObjectOS runtime ships agents, model registry, conversation, tools, skills, and an eval harness - RAG/embedding is provided by `@objectstack/service-knowledge` plus the `knowledge-memory`, `knowledge-ragflow`, and `embedder-openai` plugins (all open) -- Cost tracking, predictive, and the DevOps agent protocols remain spec-only +- Token/cost accounting has a schema (`ai/usage.zod.ts`) but no shipping enforcement; the predictive and DevOps-agent protocols are not defined in spec at all --- @@ -431,7 +437,7 @@ There is no MSW package in this repo — browser API mocking is a devDependency - [x] Multi-tenancy — verified cross-organization isolation on `pnpm dev:crm` (Alice@OrgAlpha vs. Bob@OrgBeta only see their own records across `sys_organization`, `sys_member`, `sys_user`, and `sys_*_permission_set` link tables) - [x] Legacy `objectql.registerTenantMiddleware` removed — SecurityPlugin is now the sole tenant-isolation authority - [x] Organization-Wide Defaults / sharing model — `private`, `public_read`, `public_read_write`, and `controlled_by_parent` enforced via `plugin-sharing` + `plugin-security`, proven by dogfood over the real HTTP stack (ADR-0056). Canonical vocabulary only — legacy aliases removed from the enum (ADR-0090 D4); unset custom-object OWD resolves to `private` (ADR-0090 D1) -- [x] Sharing Rule evaluator — criteria rules re-evaluated on `afterInsert` / `afterUpdate` (`plugin-sharing/rule-hooks.ts`); enforced recipients are user / position / `unit_and_subordinates` (business-unit-subtree widening, ADR-0057 D5 / ADR-0090 D3); owner-type rules and group/guest recipients remain `[experimental — not enforced]` +- [x] Sharing Rule evaluator — criteria rules re-evaluated on `afterInsert` / `afterUpdate` (`plugin-sharing/rule-hooks.ts`); every authorable recipient maps 1:1 onto an enforced `expandRecipient` branch (`plugin-sharing/sharing-rule-service.ts`) — `user`, `team`, `position`, `business_unit`, and `unit_and_subordinates` (business-unit-subtree widening, ADR-0057 D5 / ADR-0090 D3). Under ADR-0078 enforce-or-remove, `criteria` is now the only rule *type* (owner-type rules were removed from the authoring surface because the static materialiser cannot track live membership), the `group` recipient was renamed to `team`, `guest` was removed, and `queue` stays reserved in the runtime contract but deliberately non-authorable - [x] Everyone-baseline suggestion — a permission set may set `isDefault: true` as the install-time suggestion to bind it to the built-in `everyone` position; resolved per-request as an additive baseline, no fallback cliff (ADR-0090 D5) - [x] Default-deny for anonymous traffic — the global default-deny landed (ADR-0056 D2) and the `api.requireAuth` opt-out was then **removed** in `@objectstack/spec` 18 (#3963): the key is tombstoned and rejected at parse time, the deny decision is centralised in `@objectstack/core` `security/anonymous-deny.ts`, and public forms self-authorize via `publicFormGrant` (Option A) - [ ] Studio RLS visual editor @@ -455,11 +461,11 @@ There is no MSW package in this repo — browser API mocking is a devDependency |:---------|:---------------:|:-------| | **Data** | 16 | Core modeling, hooks, and query engine fully implemented; document/mapping/external-lookup still pending | | **UI** | 10 | Studio/ObjectUI render most authored surfaces (🟡); full cross-surface renderer parity in progress | -| **API** | 14 | REST/HTTP/discovery/batch fully implemented; OData pending; GraphQL removed from the plan | +| **API** | 14 | REST/HTTP/discovery/batch fully implemented; OData and the WebSocket transport pending; GraphQL removed from the plan | | **System** | 39 | Logging, audit, job, translation, metrics, cache, notification implemented; encryption partial (local crypto provider); several governance services pending | | **Auth** (plugin) | 10 | Identity/organizations live (`plugin-auth`), permission + RLS live (`plugin-security`), OWD + sharing rules live (`plugin-sharing`); SCIM is opt-in | | **Automation** (plugin) | 7 | Flow, workflow, approval, webhook, and triggers implemented; ETL/Sync pending | -| **AI** | 12 | Agents, model registry, conversation, tools, RAG implemented (ObjectOS runtime + open knowledge plugins); cost/predictive/DevOps-agent pending | +| **AI** | 12 | Agents, model registry, conversation, tools, RAG implemented (ObjectOS runtime + open knowledge plugins); cost accounting has a schema but no enforcement, and the predictive / DevOps-agent protocols are not defined in spec | | **Integration** | 7 | REST/OpenAPI/MCP/Slack connectors, file storage, and queue implemented; GitHub/Vercel connectors pending | | **QA** | 1 | Fully implemented | diff --git a/content/docs/releases/v9.mdx b/content/docs/releases/v9.mdx index 6dbaf4c579..73a85bca3a 100644 --- a/content/docs/releases/v9.mdx +++ b/content/docs/releases/v9.mdx @@ -17,19 +17,26 @@ variables, rename them to `OS__`. Everything else is additive. ### 1. Analytics single-form cutover — datasets are now required (ADR-0021) -The inline analytics author surface is removed from `@objectstack/spec`. Every -dashboard widget, report, and list chart must now bind a semantic **dataset** -and select dimensions/measures **by name**. Define a metric once, reference it -everywhere. +The inline analytics author surface is removed from the dashboard-widget, +report, and list-chart schemas in `@objectstack/spec`. Every dashboard widget, +report, and list chart must now bind a semantic **dataset** and select +dimensions/measures **by name**. Define a metric once, reference it +everywhere. (An inline `object`/`objectName` + `aggregate` binding still exists +elsewhere in the spec — the React `` block and the page +`element:number` component — just not on these three schemas.) Removed from the spec: | Schema | Removed properties | Now required | |:---|:---|:---| -| `DashboardWidget` | `object`, `categoryField`, `categoryGranularity`, `valueField`, `aggregate`, `measures` (and the `WidgetMeasure` type) | `dataset` + `values` | -| `Report` | top-level and joined-block `objectName`, `columns`, `groupingsDown`, `groupingsAcross`, `filter` | `dataset` + `values` (`rows` are the dimensions) | +| `DashboardWidget` | `object`, `categoryField`, `categoryGranularity`, `valueField`, `aggregate`, `aggregation`, `rowField`, `columnField`, `xAxisField`, `yAxisFields`, `measures` (and the `WidgetMeasure` type) | `dataset` + `values` | +| `Report` | top-level and joined-block `objectName`, `columns`, `groupingsDown`, `groupingsAcross`, `filter` | `dataset` + `values` (`rows` are the dimensions; scope filtering moves to `runtimeFilter`) | | `ListChart` | `xAxisField`, `yAxisFields`, `aggregation`, `groupByField` | `dataset` + `values` | +`Report.columns` is repurposed rather than simply deleted: on both the report +and a joined block the key now names the dataset **dimensions across** a +`matrix` report, not raw report columns. + **Migration:** 1. For each inline query, create a dataset with `defineDataset(...)` declaring @@ -46,14 +53,19 @@ Removed from the spec: ### 2. `ChartTypeSchema` trimmed to distinctly-renderable families -Eight chart types that only ever rendered as their base chart are removed, so -the taxonomy no longer advertises variants the renderer doesn't draw. Update -any widget or series using a removed type to its base: +Eight chart types are removed, so the taxonomy no longer advertises variants +the renderer doesn't draw. Five of them (`bi-polar-bar`, `step-line`, `spline`, +`pyramid`, `bubble`) only ever rendered as their base chart. The other three +are dropped for a different reason: stacking is not a chart family but a +property of the series — `ChartSeries.stack`, where series sharing a group id +stack and otherwise group — so one `bar` / `area` family plus a series-level +stack group expresses all three. Update any widget or series using a removed +type: | Removed type | Use instead | |:---|:---| -| `grouped-bar`, `stacked-bar`, `bi-polar-bar` | `bar` | -| `stacked-area` | `area` | +| `grouped-bar`, `stacked-bar`, `bi-polar-bar` | `bar` (to stack, give the series a shared `stack` group id) | +| `stacked-area` | `area` (with a shared `series[].stack` group id) | | `step-line`, `spline` | `line` | | `pyramid` | `funnel` | | `bubble` | `scatter` | @@ -64,6 +76,11 @@ single-value performance family (`metric` / `kpi` / `gauge` / `solid-gauge` / `bullet`). Removed variants can return behind an opt-in renderer once a real renderer and data model back them. +> Note: a later release added one family the taxonomy had never named — +> `combo`, for mixed bar/line/area series on shared dual axes, which the +> renderer does draw distinctly (configured with `ChartSeries.type` + +> `ChartSeries.yAxis`). + ### 3. Settings env overrides: canonical `OS__` only `@objectstack/service-settings` no longer accepts unprefixed aliases for @@ -124,16 +141,20 @@ Two reliability upgrades to `@objectstack/service-automation`'s durable pause Whether a `seed` draft's rows actually materialized used to depend on which route you published through — the per-ref publish (used by the Studio home banner) never applied them, yielding "Published!" with empty tables. Seed -application now lives in the protocol itself (`@objectstack/objectql`), so -every publish path converges on it and reports the outcome under -`seedApplied`. A seed problem never fails the publish. +application now lives in the metadata protocol itself (today +`@objectstack/metadata-protocol`), so every publish path converges on it and +reports the outcome under `seedApplied`. A seed problem never fails the +publish — callers must check `seedApplied.success` instead of assuming the data +went live. ### AI service: tools stream progress while they run -`ToolExecutionContext.onProgress(part)` lets a long-running tool report +`ToolExecutionContext.onProgress?.(part)` lets a long-running tool report progress before it returns, emitted as custom `data-*` parts on the UI message stream (this is what powers the live build tree in Console's Build-with-AI). -Existing tools that never emit behave exactly as before. +It is set only on the streaming path (`streamChatWithTools`) and is `undefined` +for non-streaming/system calls, so tools must call it optionally. Existing +tools that never emit behave exactly as before. ### Auth: Google sign-in and an extension hook @@ -195,9 +216,11 @@ release lands. The deployment story for cloud-bound apps is now **publish, then install**: `os package publish` puts a package version in the cloud catalog, and you install it into an environment from that environment's **Marketplace** in -Console — installing is an explicit step, no longer a side effect of a publish -flag. The environment's **Installed** view reflects what Cloud has installed, -so packages installed via the CLI show up there too. +Console — installing is a separate, explicit step rather than something a +publish does on its own (a publish installs only when you opt in with +`os package publish --env --install`). The environment's **Installed** +view reflects what Cloud has installed, so packages installed via the CLI show +up there too. Coming in the next framework release: published packages default to **organization visibility**, so they appear in your own environments' From c5d6ec26dc060133e1e1175b70f01c23e9341f4c Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 31 Jul 2026 05:33:10 +0000 Subject: [PATCH 2/5] docs: implementation-accuracy audit of the #4212-family affected docs MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Ran the repo's docs-accuracy-audit workflow scoped to the drift-check advisory from #4242 (14 docs) plus 4 the retired-surface grep implicated. 158 evidence-backed fixes across 18 docs, each cited to file:line. The largest classes: - kernel/cluster.mdx (22): a documented `eventBus.emit(...)` that exists nowhere in packages/, a `ctx.cluster` PluginContext property that does not exist, `defineStack({ cluster })` that stack.zod.ts silently strips, `postgres`/`nats` listed as shipped drivers when neither package exists, `REDIS_URL` for the real `OS_CLUSTER_DRIVER`, and an at-least-once delivery promise no shipped driver provides. - protocol/kernel/plugin-spec.mdx (20): `plugin.manifest.ts` as the manifest filename — grep-empty across packages/, apps/ and examples/; the real one is `objectstack.plugin.json`. Plus `context.plugins.isInstalled()` (no such member), compound semver ranges (`'>=1.0.0 <2.0.0'`) that make the resolver throw, and init ordering described as registration order when bootstrap() runs a dependency topological sort. - data-modeling/external-datasources.mdx: `credentialsRef: 'secret:…'` — the shipped binder only resolves `sys_secret:`, so the sample as written fails closed at connect. - deployment/migration-from-objectql.mdx: a `^3.1.0` pin that can never resolve to the current 16.1.0 line. VERIFICATION STATUS — 7 of 18 docs carry an adversarial-verifier pass; the other 11 do not (the run hit the session limit mid-phase). The verifier earned its keep on the 7 it reached: 8 repairs, including a fabricated `enforceNetworkAccess` (real: `enforceNetworkRequest`, plugin-permission-enforcer.ts:212) and an over-broad retirement claim that would have contradicted the live app-bundle `onEnable`. The remaining 11 are committed unverified and MUST NOT ship until their verifier pass runs — the PR stays draft until then. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_01HrRNgrWaRtggzmrHpbomyh --- .changeset/docs-accuracy-audit-4212-scope.md | 6 ++++ .../data-modeling/external-datasources.mdx | 13 +++---- content/docs/kernel/cluster.mdx | 12 +++++-- content/docs/kernel/events.mdx | 4 +-- content/docs/plugins/index.mdx | 14 +++++++- content/docs/protocol/kernel/index.mdx | 36 ++++++++++++------- content/docs/protocol/kernel/plugin-spec.mdx | 17 ++++++--- 7 files changed, 73 insertions(+), 29 deletions(-) create mode 100644 .changeset/docs-accuracy-audit-4212-scope.md diff --git a/.changeset/docs-accuracy-audit-4212-scope.md b/.changeset/docs-accuracy-audit-4212-scope.md new file mode 100644 index 0000000000..9ee3f8ad55 --- /dev/null +++ b/.changeset/docs-accuracy-audit-4212-scope.md @@ -0,0 +1,6 @@ +--- +--- + +docs: implementation-accuracy audit of the #4212-family affected docs. Fixes +fabricated APIs, wrong manifest filenames, unresolvable version pins and +credential-ref formats. Releases nothing. diff --git a/content/docs/data-modeling/external-datasources.mdx b/content/docs/data-modeling/external-datasources.mdx index f040cce659..d2cdc4ae38 100644 --- a/content/docs/data-modeling/external-datasources.mdx +++ b/content/docs/data-modeling/external-datasources.mdx @@ -122,9 +122,10 @@ from external configuration. Auto-connect is idempotent with it (whichever registers the datasource name first wins), so the two never conflict. -A connected external datasource is also visible in **Setup → Datasources** (stamped -`origin: code`, read-only in the UI) and via `GET /api/v1/datasources` and -`GET /api/v1/meta/datasource`, where an admin can run the "Sync objects" wizard. +A connected external datasource is also visible in **Setup → Integrations → +Datasources** (stamped `origin: code`, read-only in the UI) and via +`GET /api/v1/datasources` and `GET /api/v1/meta/datasource`, where an admin can +run the "Sync objects" wizard. ### When auto-connect fails @@ -161,8 +162,8 @@ Every connect attempt's verdict is retained, so a datasource that is down no longer has to be diagnosed by restarting the server and re-reading boot logs ([#3827](https://github.com/objectstack-ai/objectstack/issues/3827)). -**Setup → Datasources** and `GET /api/v1/datasources` report a `status` per -datasource: +**Setup → Integrations → Datasources** and `GET /api/v1/datasources` report a +`status` per datasource: | `status` | Meaning | |:---|:---| @@ -203,7 +204,7 @@ const policy: DatasourceConnectPolicy = { ? { allow: true } : { allow: false, - // operator-facing: logs + Setup → Datasources only + // operator-facing: logs + the Setup datasource list only reason: `egress allow-list miss for ${ds.name} (${tenant.id}, plan=${tenant.plan})`, // tenant-facing: appended to the query-time error publicReason: 'External datasources require the Scale plan.', diff --git a/content/docs/kernel/cluster.mdx b/content/docs/kernel/cluster.mdx index 356a239978..b23e17009e 100644 --- a/content/docs/kernel/cluster.mdx +++ b/content/docs/kernel/cluster.mdx @@ -417,8 +417,10 @@ Ask: **if I deploy 5 nodes, should there be 5 of these or 1?** - "Reacts to events with idempotent writes" → `idempotent-broadcast` Declaring `cluster` does not yet *do* anything (§5.1) — until Phase 4 lands -you still have to hold the invariant yourself, e.g. by wrapping the work in -`cluster.lock.withLock(key, fn)` the way `service-job`'s cron scheduler does. +you still have to hold the invariant yourself, either by wrapping the work in +`cluster.lock.withLock(key, fn)` or by acquiring and releasing by hand the way +`service-job`'s cron scheduler does (`cluster.lock.acquire(key, { waitMs: 0 })`, +bail out when it returns `null`, release in a `finally`). ### 7.3 "I want to read metadata in a long-lived cache" @@ -631,7 +633,11 @@ of the declarative wiring below exists yet. - `service-job` declares `clusterScope: 'cluster' / 'leader-elected'`. - `service-cache` gains a cluster-backed cache derived from `KV + PubSub`. - `service-realtime` adds cross-node fan-out via PubSub. -- Webhook dispatcher uses `partitioned` strategy from day one. +- Webhook dispatcher declares the `partitioned` strategy. (The behaviour + already ships hand-rolled: webhook deliveries drain through + `service-messaging`'s `HttpDispatcher`, which walks `partitionCount` + partitions behind per-partition `cluster.lock` keys — only the declarative + annotation is missing.) ### Phase 5 — Postgres / NATS drivers (deferred, optional) diff --git a/content/docs/kernel/events.mdx b/content/docs/kernel/events.mdx index b2b9ff4460..5819dcba64 100644 --- a/content/docs/kernel/events.mdx +++ b/content/docs/kernel/events.mdx @@ -19,7 +19,7 @@ Triggered by the Kernel during bootstrap and shutdown: | Event | Description | | :--- | :--- | | `kernel:ready` | All plugins have successfully started. System is live. | -| `kernel:bootstrapped` | Fired after every `kernel:ready` handler has settled, before `kernel:listening`. The "all bootstrap + seed data is ready" anchor — use it for reconcile/backfill work that consumes data a later-starting plugin produces during `kernel:ready`. | +| `kernel:bootstrapped` | Fired after every `kernel:ready` handler has settled, before `kernel:listening`. The "all **synchronous** bootstrap has settled" anchor — use it for reconcile/backfill work that consumes data a later-starting plugin produces during `kernel:ready`. It does **not** guarantee app *seed* data has settled: an inline seed that overruns `OS_INLINE_SEED_BUDGET_MS` (default 8s) keeps seeding in the background and can outlast even `kernel:listening`. Subscribe to `app:seeded` for the true per-app seed-settle point. | | `kernel:listening` | Fired after every `kernel:ready` and `kernel:bootstrapped` handler has completed (e.g. the HTTP server is accepting connections). | | `kernel:shutdown` | Shutdown signal received. Plugins should clean up resources. | @@ -74,7 +74,7 @@ Every data hook is a single-argument handler `(ctx: HookContext) => void | Promi | `ctx.input` | **Mutable** input. Shapes: find `{ ast, options }`, insert `{ data, options }`, update `{ id, data, options }`, delete `{ id, options }`. Modify this to change the operation. | | `ctx.result` | Operation result, available in `after*` events (mutable). | | `ctx.previous` | Record state before the operation (update/delete). | -| `ctx.session` | Auth/tenancy info (`userId`, `organizationId`, `roles`, …). | +| `ctx.session` | Auth/tenancy info (`userId`, `organizationId`, `positions`, `accessToken`, …). Absent entirely when the call carried no identity envelope. | | `ctx.api` | Scoped cross-object data access. | ### Registering Data Hooks diff --git a/content/docs/plugins/index.mdx b/content/docs/plugins/index.mdx index 05c192eaa7..a694603244 100644 --- a/content/docs/plugins/index.mdx +++ b/content/docs/plugins/index.mdx @@ -211,11 +211,23 @@ The `os serve` and `os dev` commands automatically detect and load plugins: | Condition | Auto-loaded Plugin | |:----------|:-------------------| | `objects` defined, no ObjectQL | `@objectstack/objectql` | -| `objects` defined, no driver | `DefaultDatasourcePlugin` (runtime) — driver kind from `OS_DATABASE_DRIVER` / `OS_DATABASE_URL`, defaulting to SQLite in dev and to nothing in production | +| `objects` defined, no driver | `DefaultDatasourcePlugin` (runtime) — driver kind from `OS_DATABASE_DRIVER` / `OS_DATABASE_URL`, defaulting to SQLite | | `objects`, `manifest`, `apps`, `flows`, or `apis` defined | `AppPlugin` (runtime) | | Server not disabled | `@objectstack/plugin-hono-server` | | Server enabled | `@objectstack/rest` (REST API) | + +**Which SQLite default you get depends on the boot path.** A bare `defineStack()` +config — one whose `plugins` array instantiates nothing — boots through +`createStandaloneStack`, which anchors a SQLite *file* under the project +(`.objectstack/data/standalone.db`) in development **and** production. A host config +that already instantiates plugins takes `serve`'s own fallback instead: with no +`OS_DATABASE_URL` / `OS_DATABASE_DRIVER` set it declares SQLite `:memory:` in dev and +registers **no** datasource in production, so a missing driver surfaces loudly +downstream. `os dev` sidesteps both defaults by exporting +`OS_DATABASE_URL=file:/.objectstack/data/dev.db`. + + This means a minimal config like this already works: ```typescript diff --git a/content/docs/protocol/kernel/index.mdx b/content/docs/protocol/kernel/index.mdx index abcdcccbb7..906378ba46 100644 --- a/content/docs/protocol/kernel/index.mdx +++ b/content/docs/protocol/kernel/index.mdx @@ -276,12 +276,13 @@ await kernel.use(new AppPlugin(stack)); await kernel.bootstrap(); ``` -The kernel validates each plugin at `use()`, then `bootstrap()` topologically -sorts the registry — declared `dependencies` / `optionalDependencies` are -hoisted ahead of their dependents, registration order breaking ties — and runs -every plugin's `init` (Phase 1) followed by every plugin's `start` (Phase 2), -exposing the resulting services through DI. It then fires `kernel:ready`, -`kernel:bootstrapped`, and `kernel:listening` in that order. +The kernel validates each plugin's structure and version compatibility at +`use()`, then `bootstrap()` topologically sorts the registry — each plugin's +declared `dependencies` are hoisted ahead of it, registration order breaking +ties — and runs every plugin's `init` (kernel Phase 1) followed by every +plugin's `start` (kernel Phase 2), exposing the resulting services through DI. +It then fires `kernel:ready`, `kernel:bootstrapped`, and `kernel:listening` in +that order. ### 2. Request Handling ```typescript @@ -298,8 +299,9 @@ const context = await resolveExecutionContext({ getService, getQl, request }); // → ExecutionContext { userId, tenantId, locale, timezone, positions, // permissions, isSystem, ... } -// 2. Data — the context travels as `options.context`, which is what the engine -// enforces permissions and RLS against. +// 2. Data — the context rides along as `context`, on the query itself or on +// the engine's trailing options argument (the engine merges both). It is +// what permissions and RLS are enforced against. const ql = ctx.getService('objectql'); const account = await ql.findOne('account', { where: { id: '123' }, context }); ``` @@ -399,9 +401,12 @@ packages/plugins/plugin-slack-integration/ ### Configuration Management ```typescript -// Declare a Zod `configSchema` on the plugin object — the kernel's plugin -// loader validates the plugin's config against it at `use()` time and refuses -// to load the plugin when it fails. +// A plugin object may carry a Zod `configSchema` describing its settings. +// NOTE: the kernel RECORDS the schema but does not enforce it yet — `use()` +// takes no config argument, so the loader has nothing to parse and logs +// "config validation postponed" instead of running the schema. Treat +// `configSchema` as a declaration of shape, and validate values you actually +// depend on yourself. export const slackPlugin: Plugin = { name: 'slack-integration', version: '0.1.0', @@ -477,7 +482,14 @@ const message = i18n.t('slack.button.send', context.locale); **Why:** Catch errors before customers see them. -**Example:** A plugin declares `apiKey` in its `configSchema`. If it is missing, the plugin loader fails the load with the Zod issues spelled out per key — `Plugin slack-integration configuration validation failed:` followed by ` - apiKey: Required` — so boot stops instead of a request failing later. +**Example:** Wiring, not config, is what the kernel currently enforces at boot. A plugin that declares a dependency the kernel never received fails `bootstrap()` outright — `[Kernel] Dependency 'com.objectstack.engine.objectql' not found for plugin 'com.objectstack.audit'` — and a dependency cycle throws `[Kernel] Circular dependency detected: `. Boot stops there instead of a request failing later. + + + Plugin `configSchema` is **not** part of this fail-fast path yet. The loader + stores the schema and postpones the check, so a missing or malformed value in + a plugin's config will not stop boot today. Validate config you depend on in + your own `init`. + ## Comparison: Kernel vs Alternatives diff --git a/content/docs/protocol/kernel/plugin-spec.mdx b/content/docs/protocol/kernel/plugin-spec.mdx index 90f20e9e7b..2ef4d30c9c 100644 --- a/content/docs/protocol/kernel/plugin-spec.mdx +++ b/content/docs/protocol/kernel/plugin-spec.mdx @@ -593,9 +593,15 @@ boot — not at install. Earlier revisions of this page documented an `onInstall` / `onEnable` / `onDisable` / `onUninstall` / `onUpgrade` / `onBoot` hook family loaded from a -manifest `lifecycle` file-map. **None of it existed** — ManifestSchema never -declared the map and the kernel never called the hooks, so code written -against them silently never ran. The family was retired from the protocol +manifest `lifecycle` file-map. **That file-map never existed** — `ManifestSchema` +has no `lifecycle` key and the kernel never invoked any of these hooks, so code +written against them silently never ran. `PluginLifecycleSchema` +(`packages/spec/src/kernel/plugin.zod.ts`) still *declares* `onInstall`, +`onEnable`, `onDisable`, `onUninstall` and `onUpgrade` as optional functions, but +nothing calls them; `onBoot` appears nowhere in the implementation at all. The +single live exception is the **module-level** `onEnable` export of an app bundle +described above, which `AppPlugin.start()` calls at boot — that is a stack-entry +export, not a manifest hook. The family was retired from the protocol (#4212, ADR-0049 enforce-or-remove). @@ -632,8 +638,9 @@ implementation. `canAccessService` / `canTriggerHook` / `canReadFile` / `canWriteFile` / `canNetworkRequest` from the granted set, but only **service access** is wired into a live code path: `SecurePluginContext.getService()` and `.replaceService()` call -`enforceServiceAccess`. `enforceNetworkAccess`, `enforceFileRead` and `enforceFileWrite` -have no non-test call sites, and nothing patches global `fetch` — a plugin that calls +`enforceServiceAccess`. `enforceNetworkRequest`, `enforceFileRead` and `enforceFileWrite` +have no call sites outside the enforcer module itself, and nothing patches global +`fetch` — a plugin that calls `fetch()` directly is **not** intercepted today. Treat `network` / `fs` declarations as consent metadata, not a runtime jail. From 2383192e5d06fd6ffdf85ebcc9a92143cebfdd37 Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 31 Jul 2026 05:41:14 +0000 Subject: [PATCH 3/5] chore(lint): ratchet the role-word baseline down after the docs audit MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The audit pass removed a "role" occurrence from kernel/events.mdx (now clean) and one from kernel/services-checklist.mdx (4 → 3). check-role-word is a ratchet, so an improvement fails the gate until the baseline records it. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_01HrRNgrWaRtggzmrHpbomyh --- scripts/role-word-baseline.json | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/scripts/role-word-baseline.json b/scripts/role-word-baseline.json index ad6ee3a2ae..d507524505 100644 --- a/scripts/role-word-baseline.json +++ b/scripts/role-word-baseline.json @@ -12,8 +12,7 @@ "content/docs/getting-started/index.mdx": 1, "content/docs/getting-started/quick-start.mdx": 1, "content/docs/kernel/contracts/data-engine.mdx": 1, - "content/docs/kernel/events.mdx": 1, - "content/docs/kernel/services-checklist.mdx": 4, + "content/docs/kernel/services-checklist.mdx": 3, "content/docs/permissions/authentication.mdx": 4, "content/docs/permissions/authorization.mdx": 3, "content/docs/permissions/delegated-administration.mdx": 11, From 647930116c0e143ad45ad9471543ba9750f2c3ea Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 31 Jul 2026 05:51:01 +0000 Subject: [PATCH 4/5] docs: apply adversarial-verifier repairs (metadata-lifecycle, protocol/kernel/index) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Verifier pass for two more of the 18 audited docs completed: 27 of the audit's fixes confirmed against code, 5 repaired. Checkpoint commit — the remaining verifier passes are still running and will land separately. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_01HrRNgrWaRtggzmrHpbomyh --- content/docs/concepts/metadata-lifecycle.mdx | 24 ++++++++++++-------- content/docs/protocol/kernel/index.mdx | 8 ++++--- 2 files changed, 19 insertions(+), 13 deletions(-) diff --git a/content/docs/concepts/metadata-lifecycle.mdx b/content/docs/concepts/metadata-lifecycle.mdx index 84ec26230b..abb5deab70 100644 --- a/content/docs/concepts/metadata-lifecycle.mdx +++ b/content/docs/concepts/metadata-lifecycle.mdx @@ -50,8 +50,8 @@ Today this precedence is implemented **inside the protocol layer**, not by in-memory code registry — `getMetaItemLayered()` returns `code` / `overlay` / `effective` separately. `LayeredRepository` ships in `@objectstack/metadata-core` and implements exactly the semantics above, but it is **not yet composed** into -any production path: the only `new LayeredRepository(...)` call sites are its own -tests. +any production path: every `new LayeredRepository(...)` call site is a test +(`metadata-core`'s own suite plus one `objectql` integration test). --- @@ -127,7 +127,7 @@ See [ADR-0005](https://github.com/objectstack-ai/objectstack/blob/main/docs/adr/ ## Conflict handling -Every `put()` requires a `parentVersion` (or `null` for fresh creates). If the head has moved since the caller read it, `ConflictError(ref, expectedParent, actualHead)` is thrown — code `METADATA_CONFLICT`, mapped to HTTP 412 Precondition Failed. Clients are expected to re-read the item and retry; the console does not yet surface a dedicated conflict-resolution UI. +Every `put()` requires a `parentVersion` (or `null` for fresh creates). If the head has moved since the caller read it, the repository throws `ConflictError(ref, expectedParent, actualHead)` with code `METADATA_CONFLICT`. `protocol.ts` catches it at every write entry point (save, publish, rollback, overlay-delete) and re-throws an API error with code `metadata_conflict`, **HTTP 409 Conflict**, and the `expectedParent` / `actualHead` pair attached. Clients are expected to re-read the item and retry; the console does not yet surface a dedicated conflict-resolution UI. The hash is `sha256:` + 64-hex of a canonical (sorted-keys, no-undefined) JSON serialization of the body. Identical bodies produce identical hashes — `put()` short-circuits on no-op writes. @@ -170,13 +170,17 @@ The hash is `sha256:` + 64-hex of a canonical (sorted-keys, no-undefined) JSON s > **Cross-replica sync happens above the repository, not inside it.** > `SysMetadataRepository.watch()` is deliberately scoped to the local instance — > it has no `LISTEN/NOTIFY` or pub/sub of its own, and never will. Cross-node -> propagation is instead a **cluster** concern: install `@objectstack/service-cluster` -> and its `MetadataClusterBridgePlugin` calls `MetadataManager.attachClusterPubSub()` -> at `kernel:ready`, fanning every local watch event out over the `metadata.changed` -> channel (loopback-suppressed by `originNode`) so peer nodes invalidate their -> caches. Note this replays the *invalidation event*, not the overlay row — peers -> re-read from the shared database. Without the cluster service registered, the -> bridge logs and skips, and each replica only sees its own writes. +> propagation is instead a **cluster** concern: `@objectstack/runtime` registers +> `ClusterServicePlugin` and `MetadataClusterBridgePlugin` by default (opt out with +> `cluster: false`), and at `kernel:ready` the bridge calls +> `MetadataManager.attachClusterPubSub()`, fanning every local watch event out over +> the `metadata.changed` channel (loopback-suppressed by `originNode`) so peer nodes +> invalidate their caches. Note this replays the *invalidation event*, not the +> overlay row — peers re-read from the shared database. The default cluster driver +> is `memory`, which is in-process, so real cross-replica fan-out still requires +> configuring a distributed driver; and if no `cluster` (or `metadata`) service is +> registered the bridge logs and skips, leaving each replica seeing only its own +> writes. --- diff --git a/content/docs/protocol/kernel/index.mdx b/content/docs/protocol/kernel/index.mdx index 906378ba46..1321400e59 100644 --- a/content/docs/protocol/kernel/index.mdx +++ b/content/docs/protocol/kernel/index.mdx @@ -218,7 +218,7 @@ const { value, source, locked } = await settings.get('stripe', 'apiKey', { **Kernel Solution:** - **Package Manifest:** The `manifest` block in `objectstack.config.ts` (compiled to `objectstack.json`) declares what a package provides and needs - **Dependency Resolution:** Automatic validation that plugin versions are compatible with core platform -- **Lifecycle:** plugins implement the optional `init` / `start` / `destroy` hooks, driven by `kernel.bootstrap()` and shutdown +- **Lifecycle:** every plugin implements a **required** `init` hook plus optional `start` / `destroy` hooks, driven by `kernel.bootstrap()` and shutdown - **Sandboxing:** Plugins can't access each other's data or crash each other **Results:** @@ -395,8 +395,10 @@ packages/plugins/plugin-slack-integration/ The scaffold still emits an `initialize` method. The kernel's plugin contract - only invokes `init` / `start` / `destroy` — rename `initialize` to `init` in - the generated `src/index.ts` or the hook never runs. + only invokes `init` / `start` / `destroy`, and `init` is **required** — rename + `initialize` to `init` in the generated `src/index.ts` or `kernel.use()` + rejects the plugin outright with + `Failed to load plugin: slack-integration - Plugin init function is required`. ### Configuration Management From c59060ab66db59c7973e459f482b0e5a9540acf8 Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 31 Jul 2026 06:19:18 +0000 Subject: [PATCH 5/5] docs: apply the remaining adversarial-verifier repairs MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Completes the verifier phase for all 18 audited docs (36/36 agents, 0 errors). Across the run: 158 audit fixes, 29 verifier repairs, 91 residual items recorded rather than guessed at. The repairs are why this phase exists — the audit agents were confident and sometimes wrong. Representative catches: - `enforceNetworkAccess` invented for the real `enforceNetworkRequest` (plugin-permission-enforcer.ts:212). - A retirement callout widened to "None of it existed" over a list containing `onEnable`, which would have contradicted the live app-bundle contract AppPlugin.start() dispatches (app-plugin.ts:574). - `cluster.lock.withLock(key, fn)` attributed to service-job's cron scheduler, which actually does `lock.acquire(...)` + manual release (cron-job-adapter.ts:139-147). - A "(planned)" marker dropped from the webhook dispatcher, which filed an already-shipping partitioned dispatcher under "Not started". Gates: check-doc-authoring 215 files clean; check-role-word OK (44 baselined); docs build compiles all pages. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_01HrRNgrWaRtggzmrHpbomyh --- content/docs/data-modeling/formulas.mdx | 6 ++- content/docs/kernel/services-checklist.mdx | 47 +++++++++++++------ content/docs/kernel/services.mdx | 22 +++++---- content/docs/permissions/authentication.mdx | 5 +- .../docs/protocol/objectql/query-syntax.mdx | 2 +- .../docs/protocol/objectql/state-machine.mdx | 2 +- .../docs/releases/implementation-status.mdx | 2 +- content/docs/releases/v9.mdx | 37 ++++++++------- 8 files changed, 74 insertions(+), 49 deletions(-) diff --git a/content/docs/data-modeling/formulas.mdx b/content/docs/data-modeling/formulas.mdx index 9874717bd6..b0fb6f30da 100644 --- a/content/docs/data-modeling/formulas.mdx +++ b/content/docs/data-modeling/formulas.mdx @@ -125,8 +125,10 @@ so the field silently never computes. To declare what the formula returns, set yourself — nothing back-fills it — but the agent-callable `validate_expression` tool reports the type it infers from the expression (`inferredType`) so an AI author can stamp the matching value. Consumers read the declared `returnType` -instead of re-parsing the expression (record-title eligibility, for one: only a -`returnType: 'text'` formula may be a `nameField`). The CEL source is the +instead of re-parsing the expression (record-title eligibility, for one: a +formula is title-eligible only when its `returnType` is `'text'`, which is what +lets it be *derived* as the record title — an explicit `nameField` pointer is +honored either way). The CEL source is the **`expression`** key — it is the only key the schema and runtime read for a formula's source; there is no separate `formula` source key. diff --git a/content/docs/kernel/services-checklist.mdx b/content/docs/kernel/services-checklist.mdx index 413b1d5818..67f18463a2 100644 --- a/content/docs/kernel/services-checklist.mdx +++ b/content/docs/kernel/services-checklist.mdx @@ -16,7 +16,7 @@ package catalog. The ObjectStack protocol defines **16 kernel services** registered via the `CoreServiceName` enum (the never-implemented `graphql` entry was removed in v17). Each service maps to a set of protocol methods governed by its per-domain contract (`DataProtocol`, `MetadataProtocol`, ...) — the transitional `ObjectStackProtocol` composition alias was dissolved in v17 (ADR-0076 D9); capability availability comes from the runtime discovery `services` registry. -**Key architecture principle**: the kernel guarantees only **data** and **metadata**, and even those are filled by packages (`@objectstack/objectql`, `@objectstack/metadata`) rather than baked in — the kernel's own contribution is an in-memory fallback for each unfilled `core` slot. Everything else — including **auth** and **automation** — is delivered by plugins. `@objectstack/objectql` is an example kernel implementation to get the basic API running; production kernels will be rebuilt as separate plugins. +**Key architecture principle**: the kernel guarantees only **data** and **metadata**, and even those are filled by packages (`@objectstack/objectql`, `@objectstack/metadata`) rather than baked in — the kernel's own contribution is an in-memory fallback for the `core` slots that have one (`metadata`, `cache`, `queue`, `job`, `i18n` — **not** `auth`). Everything else — including **auth** and **automation** — is delivered by plugins. `@objectstack/objectql` is an example kernel implementation to get the basic API running; production kernels will be rebuilt as separate plugins. **Legend** @@ -24,9 +24,12 @@ The ObjectStack protocol defines **16 kernel services** registered via the `Core - ✅ Implemented — the 17 kernel-provided protocol methods (`DataProtocol` 9 + `MetadataProtocol` 8) - ⚠️ Framework — the slot is filled by the kernel's in-memory fallback: real reads and writes, no persistence (self-declares `degraded`, ADR-0076 D12) -- ❌ Plugin Required — the remaining 36 protocol methods (`analytics` 2, `automation` 1, - `packages` 6, `views` 5, `permissions` 3, `workflow` 3, `realtime` 6, `notification` 7, - `i18n` 3), delivered by whatever fills the slot +- ❌ Plugin Required — the remaining 36 methods declared across the other per-domain + contracts (`analytics` 2, `automation` 1, `packages` 6, `views` 5, `permissions` 3, + `workflow` 3, `realtime` 6, `notification` 7, `i18n` 3). *Declared* is not *routed*: + `packages` is answered kernel-side by the `/packages` dispatcher domain over the + ObjectQL registry, `i18n` has a kernel in-memory fallback, and `views` / `permissions` / + `workflow` have no implementation anywhere — the rest wait on whatever fills the slot --- @@ -69,7 +72,7 @@ The ObjectStack protocol defines **16 kernel services** registered via the `Core | 7 | **automation** | `optional` | 1 | ❌ Plugin Required | `@objectstack/service-automation` | | 8 | **realtime** | `optional` | 6 | ❌ Plugin Required (in-process only — no HTTP/WS route is mounted) | `@objectstack/service-realtime` | | 9 | **notification** | `optional` | 7 | ❌ Plugin Required | `@objectstack/service-messaging` | -| 10 | **ai** | `optional` | — | ❌ Nothing ships in this repo | `service-ai` (Cloud/EE) | +| 10 | **ai** | `optional` | — | ❌ Nothing ships in this repo | `@objectstack/service-ai` (Cloud/EE — not installable, so the table entry is `null`) | | 11 | **i18n** | `core` | 3 | ✅ Built-in (in-memory fallback) | `@objectstack/service-i18n` | | 12 | **file-storage** | `optional` | — | ❌ Plugin Required | `@objectstack/service-storage` | | 13 | **search** | `optional` | — | ❌ Nothing ships | — | @@ -78,16 +81,30 @@ The ObjectStack protocol defines **16 kernel services** registered via the `Core | 16 | **job** | `core` | — | ✅ Built-in (in-memory fallback) | `@objectstack/service-job` | -The Provider column is not prose — it is `CORE_SERVICE_PROVIDER` in -`@objectstack/spec/system`, the single table discovery reads when it tells a -caller what to install for an empty slot. `null` there (`ai`, `search`, -`workflow`) means **nothing ships**, and discovery says so rather than naming a -plausible package: `No implementation ships for the '' slot — register a -service under it to enable`. `ui` is the one slot with a hand-written remedy, -because the package named does not fill the slot: `Served by the protocol -service — register MetadataPlugin (@objectstack/metadata-protocol) to enable`. -`scripts/check-service-providers.mjs` fails CI if a name in that table is not a -real workspace package. +The Provider column mirrors `CORE_SERVICE_PROVIDER` in +`@objectstack/spec/system` — the single table discovery reads when it tells a +caller what to install for an empty slot, and the value it reports as +`provider`. Two caveats. The table carries **no entry for +`metadata` or `data`**: those are filled by the engine itself, never by an +installable optional package, so they need no remedy (`NO_REMEDY_SLOTS` in the +guard script). And `null` there does **not** always mean "nothing ships" — it +means "no name belongs in an `Install X` sentence", which covers two cases: + +- **Nothing provides the slot at all** — `search`, `workflow` (and the retired + `graphql`). Discovery says exactly that rather than naming a plausible + package: `No implementation ships for the '' slot — register a service + under it to enable`. +- **A provider exists but cannot be installed** — `ai`. `@objectstack/service-ai` + registers the slot in `objectstack-ai/cloud` and is `private: true`. + +Two slots therefore override the generic line with a hand-written +`REMEDY_DETAIL` sentence: `ui`, because the package named does not fill the +slot (`Served by the protocol service — register MetadataPlugin +(@objectstack/metadata-protocol) to enable`), and `ai` (`Provided by +@objectstack/service-ai in ObjectStack Cloud/Enterprise — no implementation +ships in the open framework`). `scripts/check-service-providers.mjs` fails CI +if a name in the table is not a real workspace package, or if a +`CoreServiceName` slot other than `data`/`metadata` is missing from it. diff --git a/content/docs/kernel/services.mdx b/content/docs/kernel/services.mdx index 67306d9385..60efcd826f 100644 --- a/content/docs/kernel/services.mdx +++ b/content/docs/kernel/services.mdx @@ -32,11 +32,15 @@ export const myPlugin: Plugin = { }; ``` -Take configuration from the plugin's own options (as `CacheServicePlugin` and -`createApiRegistryPlugin()` do) rather than from the `settings` service: that -service is an async, namespaced resolver — `await settings.get(namespace, key)` -returns a `{ value, source, locked, … }` envelope — not a synchronous config bag -keyed by dotted paths. +Take init-time configuration from the plugin's own options (as +`CacheServicePlugin` and `createApiRegistryPlugin()` do) rather than from the +`settings` service: that service is an async, namespaced resolver — +`await settings.get(namespace, key)` returns a `{ value, source, locked, … }` +envelope, not a synchronous config bag keyed by dotted paths — and it is only +guaranteed to be registered during your `init` if you declare +`requiresServices: ['settings']`. Settings-driven config usually belongs in +`start` instead, the way `EmailServicePlugin` binds the `mail` namespace there +and re-applies it on every change. ### Factory Registration (Lazy) @@ -109,9 +113,11 @@ fallbacks above declare it); `status: 'stub'` means the answer is fabricated and must not be used for real work. `handlerReady: false` additionally means no HTTP handler serves it. -Dev mode adds nothing here: `@objectstack/plugin-dev` registers no -implementations of its own, so a slot no plugin fills is empty in dev exactly as -in production (ADR-0115). +Dev mode adds no fakes here: `@objectstack/plugin-dev` registers no +implementations of its own — it composes the same real plugins (ObjectQL, the +in-memory driver, auth, security, the HTTP server, REST) you would compose in +production — so a slot no plugin fills is empty in dev exactly as in production +(ADR-0115). ## Standard Services diff --git a/content/docs/permissions/authentication.mdx b/content/docs/permissions/authentication.mdx index 221315a21e..ae8d9f2655 100644 --- a/content/docs/permissions/authentication.mdx +++ b/content/docs/permissions/authentication.mdx @@ -91,9 +91,8 @@ Install the plugin in your ObjectStack project: pnpm add @objectstack/plugin-auth ``` -Better-Auth is a direct runtime dependency of the plugin (`better-auth` in -`packages/plugins/plugin-auth/package.json` — there is no `peerDependencies` -block), so it is installed with the plugin; you do not add it yourself. +Better-Auth is a direct runtime dependency of the plugin, not a peer dependency, +so the command above installs it for you — you never add `better-auth` yourself. --- diff --git a/content/docs/protocol/objectql/query-syntax.mdx b/content/docs/protocol/objectql/query-syntax.mdx index 15d9caba1e..6cc39b5b03 100644 --- a/content/docs/protocol/objectql/query-syntax.mdx +++ b/content/docs/protocol/objectql/query-syntax.mdx @@ -989,7 +989,7 @@ Similarly, the following legacy field names should not be used in new code: |:-------|:----------|:------| | `filters` (array of tuples) | `where` (FilterCondition object) | Use `parseFilterAST()` to convert | | `sort` | `orderBy` | Array of `{ field, order }` objects | -| `aggregate` | `aggregations` | Same `AggregationNode[]`; the SQL and MongoDB drivers read either key | +| `aggregate` | `aggregations` | Same `AggregationNode[]`; the SQL and MongoDB drivers read either key, but `engine.aggregate()` forwards only `aggregations` | | `expand` (string array) | `expand` (Record) | Map of field name → nested QueryAST | | `skip` | `offset` | Number | | `select` | `fields` | Array of FieldNode (field-name strings) | diff --git a/content/docs/protocol/objectql/state-machine.mdx b/content/docs/protocol/objectql/state-machine.mdx index e6a9f30ae5..8a370b43a5 100644 --- a/content/docs/protocol/objectql/state-machine.mdx +++ b/content/docs/protocol/objectql/state-machine.mdx @@ -91,7 +91,7 @@ A `state_machine` rule shares the common validation-rule fields (`name`, `label` ### Transitions -`transitions` is the whole legal graph. The keys are current state values; each value is the list of states the record may move *to*. An **explicit empty array** is an enforced dead-end — every move out of it is rejected. A state with **no key at all** is different: the checker has nothing to reason about there and lets the write through (see [Enforcement semantics](#enforcement-semantics)), so declare `[]` when you mean "terminal". +`transitions` is the whole legal graph. The keys are current state values; each value is the list of states the record may move *to*. An **explicit empty array** is an enforced dead-end — every move to another state is rejected. A state with **no key at all** is different: the checker has nothing to reason about there and lets the write through (see [Enforcement semantics](#enforcement-semantics)), so declare `[]` when you mean "terminal". ```typescript transitions: { diff --git a/content/docs/releases/implementation-status.mdx b/content/docs/releases/implementation-status.mdx index d78bf33606..11af8ae3d4 100644 --- a/content/docs/releases/implementation-status.mdx +++ b/content/docs/releases/implementation-status.mdx @@ -310,7 +310,7 @@ describe that separate runtime. |:---------|:-----------------:|:-------------:|:------:| | **Agent** | ✅ | ObjectOS runtime | ✅ Agent runtime | | **Model Registry** | ✅ | ObjectOS runtime | ✅ Model registry | -| **RAG Pipeline** | ❌ | @objectstack/service-knowledge | ⚠️ Knowledge service + `knowledge-memory` / `knowledge-ragflow` / `embedder-openai` plugins | +| **RAG Pipeline** | ⚠️ | @objectstack/service-knowledge | ⚠️ Knowledge service + `knowledge-memory` / `knowledge-ragflow` / `embedder-openai` plugins; spec defines the retrieval binding (`ai/knowledge-source.zod.ts`, `ai/knowledge-document.zod.ts`), not a pipeline DSL | | **NLQ** | ❌ | ObjectOS runtime | ⚠️ Data-query tools | | **Conversation** | ✅ | ObjectOS runtime | ✅ In-memory + ObjectQL conversation services | | **Agent Action** | ✅ | ObjectOS runtime | ⚠️ Action/data/knowledge tools (`tool.zod.ts` + `action_` tools) | diff --git a/content/docs/releases/v9.mdx b/content/docs/releases/v9.mdx index 73a85bca3a..dfd6ef61f5 100644 --- a/content/docs/releases/v9.mdx +++ b/content/docs/releases/v9.mdx @@ -29,13 +29,18 @@ Removed from the spec: | Schema | Removed properties | Now required | |:---|:---|:---| -| `DashboardWidget` | `object`, `categoryField`, `categoryGranularity`, `valueField`, `aggregate`, `aggregation`, `rowField`, `columnField`, `xAxisField`, `yAxisFields`, `measures` (and the `WidgetMeasure` type) | `dataset` + `values` | +| `DashboardWidget` | `object`, `categoryField`, `categoryGranularity`, `valueField`, `aggregate`, `measures` (and the `WidgetMeasure` type) | `dataset` + `values` | | `Report` | top-level and joined-block `objectName`, `columns`, `groupingsDown`, `groupingsAcross`, `filter` | `dataset` + `values` (`rows` are the dimensions; scope filtering moves to `runtimeFilter`) | | `ListChart` | `xAxisField`, `yAxisFields`, `aggregation`, `groupByField` | `dataset` + `values` | -`Report.columns` is repurposed rather than simply deleted: on both the report -and a joined block the key now names the dataset **dimensions across** a -`matrix` report, not raw report columns. +> Note: `Report.columns` came back one release later (9.1.0) with a different +> meaning — on both the report and a joined block it now names the dataset +> **dimensions across** a `matrix` report, not raw report columns. A later +> release also flipped `DashboardWidget` to reject *any* undeclared top-level +> key instead of silently stripping it, so a leftover inline-analytics key is +> now a hard parse error; the error map also names the pivot `rowField` / +> `columnField` and list-chart `xAxisField` / `yAxisFields` / `aggregation` +> spellings. **Migration:** @@ -53,19 +58,14 @@ and a joined block the key now names the dataset **dimensions across** a ### 2. `ChartTypeSchema` trimmed to distinctly-renderable families -Eight chart types are removed, so the taxonomy no longer advertises variants -the renderer doesn't draw. Five of them (`bi-polar-bar`, `step-line`, `spline`, -`pyramid`, `bubble`) only ever rendered as their base chart. The other three -are dropped for a different reason: stacking is not a chart family but a -property of the series — `ChartSeries.stack`, where series sharing a group id -stack and otherwise group — so one `bar` / `area` family plus a series-level -stack group expresses all three. Update any widget or series using a removed -type: +Eight chart types that only ever rendered as their base chart are removed, so +the taxonomy no longer advertises variants the renderer doesn't draw. Update +any widget or series using a removed type to its base: | Removed type | Use instead | |:---|:---| -| `grouped-bar`, `stacked-bar`, `bi-polar-bar` | `bar` (to stack, give the series a shared `stack` group id) | -| `stacked-area` | `area` (with a shared `series[].stack` group id) | +| `grouped-bar`, `stacked-bar`, `bi-polar-bar` | `bar` | +| `stacked-area` | `area` | | `step-line`, `spline` | `line` | | `pyramid` | `funnel` | | `bubble` | `scatter` | @@ -76,10 +76,11 @@ single-value performance family (`metric` / `kpi` / `gauge` / `solid-gauge` / `bullet`). Removed variants can return behind an opt-in renderer once a real renderer and data model back them. -> Note: a later release added one family the taxonomy had never named — -> `combo`, for mixed bar/line/area series on shared dual axes, which the -> renderer does draw distinctly (configured with `ChartSeries.type` + -> `ChartSeries.yAxis`). +> Note: stacking has since come back as a property of the *series* rather than +> a chart family — the renderer now honors `ChartSeries.stack` (series sharing +> a group id stack, otherwise they group), so a stacked bar or area today is +> the `bar` / `area` family plus a shared `stack` group id on the series. The +> taxonomy above is unchanged. ### 3. Settings env overrides: canonical `OS__` only