Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 33 additions & 0 deletions .changeset/retire-inert-driver-plugin-options.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
---
"@objectstack/runtime": major
"@objectstack/cli": patch
---

feat(runtime)!: retire the inert `DriverPluginOptions` — `DriverPlugin` takes `(driver, driverName?)` (#4320)

`new DriverPlugin(driver, { datasourceName, registerAsDefault })` never did
what it promised: both options configured a datasource-registration block in
`start()` gated on `metadata.addDatasource`, a method **no metadata service
implements** — so the block early-returned on every boot since inception and
the options were dead weight (found while typing service lookups for #4251).

**Migration** — delete the options argument; nothing changes at runtime
because nothing ever happened:

- FROM `new DriverPlugin(driver, { datasourceName: 'x', registerAsDefault: false })`
TO `new DriverPlugin(driver)`
- FROM `new DriverPlugin(driver, 'name', options)` TO `new DriverPlugin(driver, 'name')`
- The string second argument (`new DriverPlugin(driver, 'memory')`) is unchanged.

If you passed `datasourceName` expecting routing to a named auxiliary driver:
that routing never came from the option. It keys off the **driver name** —
`DriverPlugin.init()` registers `driver.<name>`, ObjectQL's discovery loop
adopts it, and the engine's lifecycle/datasource resolution looks the name up
(see the telemetry provision in `os serve` for the pattern: stamp
`driver.name`, register the plugin, done). For Setup → Datasources visibility,
declare the datasource through `DatasourceConnectionService` /
`registerInMemory('datasource', …)` (ADR-0062).

The `DriverPluginOptions` interface was module-local (never exported from the
package root), so the only public break is the constructor's second/third
argument shape.
32 changes: 32 additions & 0 deletions .changeset/slot-lookup-type-argument-ratchet.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
---
"@objectstack/spec": patch
"@objectstack/runtime": patch
---

fix(spec,runtime): the service-lookup `any` guard now sees the type-argument form, and its scope stops at nothing under `packages/` (#4251)

The #4127/#4214 rule banned `: any` and `as any` on a service-lookup result but
not `getService<any>('data')` — the form the codebase actually used (80 sites,
zero matches), erasing the slot contract identically. And the rule's `files`
covered only `packages/runtime`, leaving the composition roots (rest,
plugins/*, services/*) that hold most lookups unlinted. Both gaps closed: a
third AST selector catches the type-argument form, the scope is now all of
`packages/`, and the 40 not-yet-swept files are grandfathered in a visible,
shrinking ratchet list (`SLOT_LOOKUP_UNSWEPT`) — enumerated at 180 sites by
running the widened rule with the list emptied. `http.server` joins
`UNCONTRACTED_SLOTS` (three providers, no written contract).

Typing the three in-scope runtime sites surfaced its first yield: both
`addDatasource` datasource-registration branches (DefaultDatasourcePlugin,
DriverPlugin) probed a method **no metadata service implements**, so they had
never run on any boot — deleted rather than typed against a phantom shape. The
inert `DriverPluginOptions` they configured are tracked in #4320.
`registerInMemory('datasource', …)` is the actual visibility path (#3827).

Contract members declared from evidence, both optional: `IDataEngine` gains
`getDefaultDriverName?()` / `getDriverByName?()` (ObjectQL's driver registry —
the surface `os migrate` and serve's storage detection reach through
`driver.<name>` services), `IMetadataService` gains `registerInMemory?()`
(MetadataManager's boot-time seeding primitive). Callers that supplied `<any>`
to these lookups should pass the slot's contract type instead — or nothing:
an unmapped slot deliberately resolves to `unknown`, not `any`.
19 changes: 19 additions & 0 deletions content/docs/kernel/contracts/data-engine.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,10 @@ export interface IDataEngine {

// Raw Command Escape Hatch (optional)
execute?(command: any, options?: Record<string, any>): Promise<any>;

// Driver Registry (optional — engines that own named drivers)
getDefaultDriverName?(): string | undefined;
getDriverByName?(name: string): IDataDriver | undefined;
}
```

Expand Down Expand Up @@ -365,6 +369,21 @@ const result = await engine.execute?.(
);
```

### getDefaultDriverName / getDriverByName (Driver Registry)

Look up the engine's registered drivers. Only engines that own a named-driver
registry implement these (ObjectQL does; test fakes and remote/virtual engines
need not) — always probe with `?.`:

```typescript
const driverName = engine.getDefaultDriverName?.();
const driver = driverName ? engine.getDriverByName?.(driverName) : undefined;
```

This is the surface the runtime uses to re-register the default driver as a
`driver.<name>` kernel service, which is where `os migrate` and serve's storage
detection locate drivers.

---

## Error Codes
Expand Down
21 changes: 21 additions & 0 deletions content/docs/kernel/contracts/metadata-service.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@ definition for a type.
export interface IMetadataService {
// Core CRUD (by type + name)
register(type: string, name: string, data: unknown): Promise<void>;
registerInMemory?(type: string, name: string, data: unknown): void;
get(type: string, name: string): Promise<unknown | undefined>;
list(type: string): Promise<unknown[]>;
unregister(type: string, name: string): Promise<void>;
Expand Down Expand Up @@ -166,6 +167,26 @@ await metadataService.unregister('object', 'project');
To inspect what would break before removing an item, call `getDependents('object', 'project')` — it returns the items that reference this one.
</Callout>

### registerInMemory

Optional. Seeds an item into the in-memory registry **only** — never persisted
to the DB store, never announced to `watch` subscribers. This is the boot-time
path for source-control-owned artefacts that must be *listable* without
creating DB drift — e.g. code-defined datasources (`origin: 'code'`), which is
how the default datasource appears in Setup → Datasources:

```typescript
metadataService.registerInMemory?.('datasource', 'default', {
name: 'default',
label: 'Default',
driver: 'sqlite',
origin: 'code',
});
```

Callers that mutate metadata mid-run want `register`, which persists and
announces.

---

## Bulk Operations
Expand Down
100 changes: 92 additions & 8 deletions eslint.config.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -59,16 +59,82 @@ const SLOT_LOOKUPS = ['resolveService', 'getService', 'getRequestKernelService']
// eslint-disable comments on purpose: exceptions belong in one reviewable
// place, not sprinkled through the code. Deleting a name from this list is how
// the exemption ends once that contract gets written.
const UNCONTRACTED_SLOTS = ['protocol', 'mcp', 'kernel-resolver', 'scope-manager'].join('|');
//
// Entries are spliced into a regex, so escape metacharacters (`http\\.server`).
// `http.server` is served by three providers (plugin-hono-server, runtime's
// config.server path, qa's node-plugin) and no IHttpServer contract exists;
// callers read only `getPort()`.
const UNCONTRACTED_SLOTS = ['protocol', 'mcp', 'kernel-resolver', 'scope-manager', 'http\\.server'].join('|');

const SLOT_LOOKUP_ANY_MESSAGE =
'Do not annotate a service-lookup result as `any` — the lookup already returns ' +
'the slot\'s contract (#4168/#4176/#4202), and this switches that checking off ' +
'Do not erase a service-lookup result to `any` (`: any`, `as any`, or a ' +
'`getService<any>(…)` type argument) — the lookup already returns the slot\'s ' +
'contract (#4168/#4176/#4202), and this switches that checking off ' +
'for the call site while looking identical to code that has it. Every such ' +
'annotation found so far was hiding a real gap, including a project-membership ' +
'gate that silently stopped gating. If the slot genuinely has no contract, add ' +
'its name to UNCONTRACTED_SLOTS in eslint.config.mjs with a note, so the ' +
'exemption is reviewed once and visible in one place — see issue #4127.';
'gate that silently stopped gating and two datasource-registration branches ' +
'probing a method no metadata service has (#4251). Pass the slot\'s contract ' +
'type instead (`getService<IDataEngine>(\'data\')`). If the slot genuinely has ' +
'no contract, add its name to UNCONTRACTED_SLOTS in eslint.config.mjs with a ' +
'note, so the exemption is reviewed once and visible in one place — see ' +
'issues #4127 and #4251.';

// [#4251] The sweep ratchet. These files hold pre-existing lookup-erasure
// sites — `getService<any>(…)`, `: any`, or `as any` — that predate the rule
// reaching them: the rule's scope was packages/runtime only until #4251
// widened it, and the type-argument selector did not exist. Enumerated by
// running this config with this list emptied: 180 sites in 44 files (the
// issue's 80 was the non-test `<any>` form alone; the annotation forms and
// test files the old selectors would have caught under a wider scope roughly
// double it). They are grandfathered BY FILE, here, for the same reason
// UNCONTRACTED_SLOTS is central: `--no-inline-config` means the escape must
// live in config, and a shrinking list in one place is the ratchet made
// visible. Batches remove entries as they sweep (see #4214 for the batch
// pattern and its yield — these sites are where the erased contracts live).
// NEVER add an entry: a new file starts covered, and a new violation in a
// listed file rides an existing entry only until its batch.
const SLOT_LOOKUP_UNSWEPT = [
'packages/cli/src/commands/migrate/files-to-references.ts',
'packages/cli/src/commands/migrate/value-shapes.ts',
'packages/cli/src/commands/serve.ts',
'packages/client/src/client.hono.test.ts',
'packages/cloud-connection/src/cloud-connection-plugin.ts',
'packages/cloud-connection/src/marketplace-install-local-plugin.ts',
'packages/core/examples/kernel-features-example.ts',
'packages/metadata-protocol/src/plugin.ts',
'packages/metadata/src/plugin.ts',
'packages/objectql/src/plugin.integration.test.ts',
'packages/objectql/src/plugin.ts',
'packages/plugins/plugin-approvals/src/approvals-plugin.ts',
'packages/plugins/plugin-approvals/src/status-mirror-cascade.integration.test.ts',
'packages/plugins/plugin-audit/src/audit-plugin.ts',
'packages/plugins/plugin-auth/src/auth-plugin.ts',
'packages/plugins/plugin-email/src/email-plugin.ts',
'packages/plugins/plugin-hono-server/src/current-user-endpoints.ts',
'packages/plugins/plugin-pinyin-search/src/pinyin-search-plugin.ts',
'packages/plugins/plugin-reports/src/reports-plugin.ts',
'packages/plugins/plugin-security/src/security-plugin.ts',
'packages/plugins/plugin-sharing/src/sharing-plugin.ts',
'packages/plugins/plugin-webhooks/src/webhook-outbox-plugin.ts',
'packages/qa/dogfood/test/showcase-agent-intersection.dogfood.test.ts',
'packages/qa/dogfood/test/showcase-bu-hierarchy-sharing.dogfood.test.ts',
'packages/qa/dogfood/test/showcase-d3-d4-capabilities.dogfood.test.ts',
'packages/qa/dogfood/test/showcase-permission-zoo.dogfood.test.ts',
'packages/rest/src/external-datasource-routes.ts',
'packages/rest/src/rest-api-plugin.ts',
'packages/services/service-datasource/src/admin-routes.ts',
'packages/services/service-job/src/job-service-plugin.ts',
'packages/services/service-messaging/src/messaging-service-plugin.test.ts',
'packages/services/service-messaging/src/messaging-service-plugin.ts',
'packages/services/service-queue/src/queue-service-plugin.ts',
'packages/services/service-realtime/src/realtime-service-plugin.ts',
'packages/services/service-settings/src/settings-service-plugin.ts',
'packages/services/service-sms/src/sms-plugin.ts',
'packages/services/service-storage/src/storage-service-plugin.ts',
'packages/triggers/trigger-record-change/src/formula-context.test.ts',
'packages/triggers/trigger-record-change/src/multilookup-context.test.ts',
'packages/triggers/trigger-record-change/src/record-change-integration.test.ts',
];

export default [
{
Expand Down Expand Up @@ -193,15 +259,22 @@ export default [
// having — a deliberate gap is a reviewed line in this file, a careless one
// is a build failure, and the two stop looking identical in the code.
//
// [#4251] Scope is all of packages/ — the rule shipped scoped to
// packages/runtime while the composition roots (rest, plugins/*, services/*)
// held 77 of the 80 known sites, an unlinted majority that looked covered.
// Per-package curation would recreate that gap one package at a time, so the
// scope is total and the not-yet-swept files are grandfathered individually
// in SLOT_LOOKUP_UNSWEPT above — a shrinking list, not a silent boundary.
//
// KNOWN RESIDUAL: a wrapper whose own return type is annotated
// (`const getEngine = async (): Promise<any> => …resolveService(…)`) erases
// the slot type just as effectively, and this selector cannot see it — the
// annotation is on the enclosing function, not on the call. One such site
// existed (share-links `getEngine`, fixed in batch 4). Catching that shape
// needs type information, so it belongs to a typed-lint pass, not here.
{
files: ['packages/runtime/**/*.{ts,mts,cts}'],
ignores: ['**/node_modules/**', '**/dist/**'],
files: ['packages/**/*.{ts,tsx,mts,cts}'],
ignores: ['**/node_modules/**', '**/dist/**', ...SLOT_LOOKUP_UNSWEPT],
languageOptions: {
parser: tsParser,
parserOptions: { ecmaVersion: 'latest', sourceType: 'module' },
Expand All @@ -224,6 +297,17 @@ export default [
`:not(:has(Literal[value=/^(${UNCONTRACTED_SLOTS})$/])))`,
message: SLOT_LOOKUP_ANY_MESSAGE,
},
{
// `ctx.getService<any>('data')` — the type-argument form (#4251).
// No annotation, no `as`, and the contract is erased all the same;
// this is the shape 80 sites actually used while the two selectors
// above matched zero of them.
selector:
`CallExpression[callee.property.name=/^(${SLOT_LOOKUPS})$/]` +
'[typeArguments.params.0.type="TSAnyKeyword"]' +
`:not(:has(Literal[value=/^(${UNCONTRACTED_SLOTS})$/]))`,
message: SLOT_LOOKUP_ANY_MESSAGE,
},
],
},
},
Expand Down
10 changes: 8 additions & 2 deletions packages/cli/src/commands/serve.ts
Original file line number Diff line number Diff line change
Expand Up @@ -1003,9 +1003,15 @@ export default class Serve extends Command {
});
if (telemetry.engine !== 'memory') {
// The engine keys datasources by driver name — the
// lifecycle router looks this exact name up.
// lifecycle router looks this exact name up. The driver
// name is the WHOLE wiring: DriverPlugin.init registers
// `driver.telemetry`, ObjectQL's discovery loop adopts
// it, and lifecycle-classed objects route to it. (An
// options bag once also asked for `datasourceName:
// 'telemetry'` metadata registration — inert since
// inception, retired in #4320.)
Object.defineProperty(telemetry.driver, 'name', { value: 'telemetry' });
await kernel.use(new DriverPlugin(telemetry.driver, { datasourceName: 'telemetry', registerAsDefault: false }));
await kernel.use(new DriverPlugin(telemetry.driver));
trackPlugin('TelemetryDatasource');
console.log(chalk.dim(` telemetry datasource: ${telemetryPath} (lifecycle-classed system data; OS_TELEMETRY_DB=0 to disable)`));
}
Expand Down
23 changes: 13 additions & 10 deletions packages/runtime/src/default-datasource-plugin.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@
// the real kernel (init-all → start-all) with the real driver factory.

import { describe, it, expect, beforeEach, afterEach } from 'vitest';
import type { IDataEngine } from '@objectstack/spec/contracts';
import { Runtime } from './runtime.js';
import { DefaultDatasourcePlugin } from './default-datasource-plugin.js';
import { AppPlugin } from './app-plugin.js';
Expand Down Expand Up @@ -64,10 +65,10 @@ describe('DefaultDatasourcePlugin — the default datasource as a declaration (#
});
try {
await kernel.bootstrap();
const engine = kernel.getService<any>('data');
const engine = kernel.getService<IDataEngine>('data');
// The driver keeps its NATURAL name (no 'default' stamping) — routing to
// `default` goes through the engine's default-driver fallback.
expect(engine.getDriverByName('default')).toBeUndefined();
expect(engine.getDriverByName?.('default')).toBeUndefined();
await engine.insert('note', { title: 'through-the-default' });
const rows = await engine.find('note');
expect(rows.map((r: any) => r.title)).toContain('through-the-default');
Expand All @@ -93,7 +94,7 @@ describe('DefaultDatasourcePlugin — the default datasource as a declaration (#
const kernel = await assemble({ withAdminPlugin: false });
try {
await kernel.bootstrap();
const engine = kernel.getService<any>('data');
const engine = kernel.getService<IDataEngine>('data');
await engine.insert('sys_metadata', undefined as never).catch(() => { /* shape probe only */ });
// The default driver exists and the engine can answer a trivial query path.
expect(typeof engine.find).toBe('function');
Expand Down Expand Up @@ -133,8 +134,8 @@ describe('DefaultDatasourcePlugin — the default datasource as a declaration (#
});
try {
await expect(kernel.bootstrap()).resolves.not.toThrow();
const engine = kernel.getService<any>('data');
expect(engine.getDefaultDriverName()).toBeDefined();
const engine = kernel.getService<IDataEngine>('data');
expect(engine.getDefaultDriverName?.()).toBeDefined();
} finally {
try { await (kernel as any)?.stop?.(); } catch { /* noop */ }
}
Expand All @@ -158,11 +159,11 @@ describe('DefaultDatasourcePlugin — the default datasource as a declaration (#
});
try {
await kernel.bootstrap();
const engine = kernel.getService<any>('data');
const defaultName = engine.getDefaultDriverName();
const engine = kernel.getService<IDataEngine>('data');
const defaultName = engine.getDefaultDriverName?.();
expect(defaultName).toBeDefined();
// Identity, not equivalence: the engine's default driver IS the host's instance.
expect(engine.getDriverByName(defaultName)).toBe(hostBuilt);
expect(engine.getDriverByName?.(defaultName!)).toBe(hostBuilt);
await engine.insert('note', { title: 'through-the-adopted-default' });
const rows = await engine.find('note');
expect(rows.map((r: any) => r.title)).toContain('through-the-adopted-default');
Expand Down Expand Up @@ -197,8 +198,10 @@ describe('DefaultDatasourcePlugin — the default datasource as a declaration (#
it('kernel teardown disconnects an OWNED (factory-built) default through the one service (#3993)', async () => {
const kernel = await assemble({});
await kernel.bootstrap();
const engine = kernel.getService<any>('data');
const drv = engine.getDriverByName(engine.getDefaultDriverName());
const engine = kernel.getService<IDataEngine>('data');
// The real engine carries the driver registry; `!` asserts exactly that,
// and a missing surface fails the test rather than sliding past it.
const drv = engine.getDriverByName!(engine.getDefaultDriverName!()!)!;
let disconnects = 0;
const orig = drv.disconnect?.bind(drv);
drv.disconnect = async () => { disconnects += 1; return orig?.(); };
Expand Down
Loading
Loading