From 82b161f1c8e3b2c76a0f62f6edf895af01647481 Mon Sep 17 00:00:00 2001 From: Claude Date: Thu, 30 Jul 2026 09:10:20 +0000 Subject: [PATCH 1/5] =?UTF-8?q?fix(driver-memory,spec):=20persistence=20is?= =?UTF-8?q?=20opt-in=20again=20=E2=80=94=20a=20bare=20InMemoryDriver=20is?= =?UTF-8?q?=20pure=20in-memory=20(#4065)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `InMemoryDriverConfig.persistence` defaulted to `'auto'`, and on Node.js `'auto'` resolves to FILE persistence. So `new InMemoryDriver()` — the shape every caller in this repo uses — silently wrote `.objectstack/data/memory-driver.json` into the process CWD and reloaded it on the next boot. The default is now `false`. This restores the design #815 accepted rather than replacing it: its requirement #1 reads "默认情况下不启用持久化（纯内存，行为不变）", and its own config examples list `new InMemoryDriver()` under "纯内存". `'auto'` was drift. Symptom: `datasource-autoconnect.test.ts` seeds two rows with fixed ids and asserts the exact set. Run 1 passed and wrote them to disk; run 2 loaded them back, appended two more and failed with four; run N had 2N. CI never saw it — every job is a fresh clone, so every CI run is run 1 — but `pnpm test` twice in one working tree could only ever go green once. What let the drift survive is not "there was no test". `MemoryConfigSchema` pinned the default and asserted `'auto'`; the driver honoured `'auto'`; the pair agreed and looked verified. Nothing checked that the agreed value was #815's. The driver's own persistence tests could not have caught it either — every case passed `persistence` explicitly, leaving the omitted-value path untested. Both sides are now covered: three behavioural tests (no CWD write, no cross-instance row carry-over, opt-in still persists) plus the flipped schema assertion. Verified they fail against the old default before landing the fix. Hosts that genuinely want durability now say so instead of inheriting it: - `resolveSqliteDriver`'s last-resort rung mirrors the sqlite target it stands in for — `persistence: 'file'` for a file database, `false` for `:memory:` — so a dev whose native and wasm SQLite both failed keeps the durability they would have had. - `createDefaultDatasourceDriverFactory`'s `memory` branch is ephemeral unless the datasource DEFINITION declares `config.persistence` (Prime Directive #12: a contract-level knob, not a silent host default). - `DevPlugin`'s driver is explicitly `persistence: false`, matching the cache, queue, job, i18n, storage and search stubs it ships beside — it was the one piece of that stack that quietly outlived the process. Also stops this suite writing into its own package dir: the shorthand `'file'`/`'auto'` cases take the adapter's default RELATIVE path, so they now run inside a scratch CWD. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01L5q3BZMpdiAueHf2emhDY9 --- .../memory-driver-opt-in-persistence.md | 67 +++++++++++++ .../driver-memory/src/memory-driver.ts | 31 ++++-- .../src/persistence/persistence.test.ts | 94 +++++++++++++++++-- packages/plugins/plugin-dev/src/dev-plugin.ts | 7 +- .../src/default-datasource-driver-factory.ts | 12 ++- .../src/sqlite-driver-fallback.ts | 13 ++- packages/spec/src/data/driver/memory.test.ts | 17 +++- packages/spec/src/data/driver/memory.zod.ts | 31 +++--- 8 files changed, 232 insertions(+), 40 deletions(-) create mode 100644 .changeset/memory-driver-opt-in-persistence.md diff --git a/.changeset/memory-driver-opt-in-persistence.md b/.changeset/memory-driver-opt-in-persistence.md new file mode 100644 index 0000000000..f7b0bb028a --- /dev/null +++ b/.changeset/memory-driver-opt-in-persistence.md @@ -0,0 +1,67 @@ +--- +"@objectstack/driver-memory": major +"@objectstack/spec": major +"@objectstack/service-datasource": patch +"@objectstack/plugin-dev": patch +--- + +fix(driver-memory,spec): persistence is opt-in again — `new InMemoryDriver()` is pure in-memory (#4065) + +`InMemoryDriverConfig.persistence` defaulted to `'auto'`, and in Node.js `'auto'` +means **file**. So a bare `new InMemoryDriver()` — the shape every caller in this +repo used — silently wrote `.objectstack/data/memory-driver.json` into the process +CWD and reloaded it on the next boot. The default is now `false`. + +**This restores the accepted design rather than replacing it.** #815, the issue +that introduced the persistence capability, specified it as opt-in in requirement +\#1 — "默认情况下不启用持久化（纯内存，行为不变）" — and listed +`new InMemoryDriver()` under "纯内存" in its own config examples. The `'auto'` +default was a drift from that spec. + +What let the drift survive is worth naming, because it is not "there was no +test". `MemoryConfigSchema` *did* pin the default, and asserted `'auto'`; the +driver honoured `'auto'`; so spec and implementation agreed, and the pair looked +verified. What nothing checked was whether the value they agreed on was the one +#815 accepted. The driver's own `persistence.test.ts` could not have caught it +either — every case there passes `persistence` explicitly, so the omitted-value +path was untested on the implementation side. Both sides are now covered: three +behavioural tests in `persistence.test.ts` (no CWD write, no cross-instance row +carry-over, opt-in still persists) and the flipped schema assertion. + +**The symptom this fixes.** `packages/runtime/src/datasource-autoconnect.test.ts` +seeds two rows with fixed ids and asserts the exact set. Run 1 passed and wrote +the rows to disk; run 2 loaded them back, appended two more, and failed with four +rows; run N had 2N. CI never saw it — every job is a fresh clone, so every CI run +is run 1 — but `pnpm test` twice in one working tree could only ever go green +once. The persisted file's `created_at` values, one pair per run, were the proof. + +The blast radius was wider than that one suite: **every** bare +`new InMemoryDriver()` inherited the default, including +`createDefaultDatasourceDriverFactory`'s `memory` branch, so any test using a +`memory` datasource wrote to its package directory. Unit tests should not have +write side effects on the CWD at all. + +**Migrating.** Callers that want durability now ask for it: + +```ts +new InMemoryDriver() // pure in-memory (new default) +new InMemoryDriver({ persistence: 'file' }) // Node.js, durable across restarts +new InMemoryDriver({ persistence: 'local' }) // browser, durable across reloads +new InMemoryDriver({ persistence: 'auto' }) // previous default behaviour +``` + +The `'auto'` / `'file'` / `'local'` / custom-adapter paths are unchanged; only +the value used when `persistence` is omitted moved. Two in-tree hosts that +genuinely wanted durability now state it, rather than inheriting it: + +- **`resolveSqliteDriver`'s last-resort rung** (`service-datasource`) mirrors the + sqlite target it stands in for — `persistence: 'file'` when the caller asked + for a file database, `false` for `:memory:`. A dev whose native and wasm SQLite + both failed to load keeps the durability they would have had. +- **`createDefaultDatasourceDriverFactory`'s `memory` branch** is ephemeral unless + the datasource *definition* declares `config.persistence`. Per Prime Directive + \#12 that is a contract-level knob on the definition, not a silent host default. + +`DevPlugin`'s driver is now explicitly `persistence: false`, matching the cache, +queue, job, i18n, storage and search stubs it ships beside — it was the one piece +of that stack that quietly outlived the process. diff --git a/packages/plugins/driver-memory/src/memory-driver.ts b/packages/plugins/driver-memory/src/memory-driver.ts index f59f26cf78..bae76bde6e 100644 --- a/packages/plugins/driver-memory/src/memory-driver.ts +++ b/packages/plugins/driver-memory/src/memory-driver.ts @@ -38,19 +38,28 @@ export interface InMemoryDriverConfig { /** Optional: Logger instance */ logger?: Logger; /** - * Persistence configuration. Defaults to `'auto'`. - * - `'auto'` (default) — Auto-detect environment (browser → localStorage, Node.js → file, serverless → disabled) + * Persistence configuration. **Defaults to `false` — pure in-memory.** + * - `false` (default) — No persistence. `new InMemoryDriver()` keeps nothing + * across process exits, which is what the driver's name promises. + * - `'auto'` — Auto-detect environment (browser → localStorage, Node.js → file, serverless → disabled) * - `'file'` — File-system persistence with defaults (Node.js only) * - `'local'` — localStorage persistence with defaults (Browser only) * - `{ type: 'file', path?: string, autoSaveInterval?: number }` — File-system with options * - `{ type: 'local', key?: string }` — localStorage with options * - `{ type: 'auto', path?: string, key?: string, autoSaveInterval?: number }` — Auto-detect with options * - `{ adapter: PersistenceAdapterInterface }` — Custom adapter - * - `false` — Disable persistence (pure in-memory) + * + * Durability is **opt-in**, as #815 specified ("默认情况下不启用持久化（纯内存，行为不变）", + * requirement #1, with `new InMemoryDriver()` listed as the pure-memory example). + * The `'auto'` default this driver shipped with was a drift from that accepted + * design, and it silently wrote `.objectstack/data/memory-driver.json` into the + * process CWD on every Node.js run — which made any suite seeded with fixed ids + * pass once and fail on every rerun (#4065). A host that wants durability now + * says so: `persistence: 'file'` / `'local'` / `'auto'`. * * ⚠️ In serverless environments (Vercel, AWS Lambda, Netlify, etc.), * auto mode disables file persistence to prevent silent data loss. - * Use `persistence: false` or supply a custom adapter for serverless deployments. + * Supply a custom adapter for serverless deployments. */ persistence?: string | false | { type?: 'file' | 'local' | 'auto'; @@ -1323,15 +1332,17 @@ export class InMemoryDriver implements IDataDriver { /** * Initialize the persistence adapter based on configuration. - * Defaults to 'auto' when persistence is not specified. - * Use `persistence: false` to explicitly disable persistence. * - * In serverless environments (Vercel, AWS Lambda, etc.), auto mode disables - * file-system persistence and emits a warning. Use `persistence: false` or - * supply a custom adapter for serverless-safe operation. + * **Persistence is opt-in.** An omitted `persistence` means none — the driver + * is pure in-memory, matching its name and #815's requirement #1. Ask for + * durability explicitly with `'auto'` / `'file'` / `'local'` / a custom adapter. + * + * In serverless environments (Vercel, AWS Lambda, etc.), `'auto'` disables + * file-system persistence and emits a warning; supply a custom adapter for + * serverless-safe durability. */ private async initPersistence(): Promise { - const persistence = this.config.persistence === undefined ? 'auto' : this.config.persistence; + const persistence = this.config.persistence ?? false; if (persistence === false) return; if (typeof persistence === 'string') { diff --git a/packages/plugins/driver-memory/src/persistence/persistence.test.ts b/packages/plugins/driver-memory/src/persistence/persistence.test.ts index 073ebd3775..280b22bc52 100644 --- a/packages/plugins/driver-memory/src/persistence/persistence.test.ts +++ b/packages/plugins/driver-memory/src/persistence/persistence.test.ts @@ -6,6 +6,26 @@ import * as path from 'node:path'; const TEST_DATA_DIR = path.join('/tmp', 'objectstack-test-persistence'); const TEST_FILE_PATH = path.join(TEST_DATA_DIR, 'test-db.json'); +/** + * Run `fn` with the process CWD moved to a scratch dir, then restore it. + * + * The shorthand forms (`persistence: 'file'` / `'auto'`) deliberately take the + * adapter's DEFAULT path, which is `.objectstack/data/memory-driver.json` + * *relative to the CWD* — so without this the suite wrote that file into + * `packages/plugins/driver-memory/` and the next run loaded it back. A unit + * test must not have write side effects on the package directory (#4065). + */ +async function inScratchCwd(fn: () => Promise): Promise { + const prev = process.cwd(); + fs.mkdirSync(TEST_DATA_DIR, { recursive: true }); + process.chdir(TEST_DATA_DIR); + try { + await fn(); + } finally { + process.chdir(prev); + } +} + describe('InMemoryDriver Persistence', () => { beforeEach(() => { // Clean up test directory @@ -52,11 +72,14 @@ describe('InMemoryDriver Persistence', () => { }); it('should support shorthand "file" persistence string', async () => { - // Use shorthand — just verifies no error is thrown with 'file' - const driver = new InMemoryDriver({ persistence: 'file' }); - await driver.connect(); - await driver.create('items', { id: '1', name: 'Widget' }); - await driver.disconnect(); + // Use shorthand — just verifies no error is thrown with 'file'. + // Scratch CWD: the shorthand takes the adapter's default relative path. + await inScratchCwd(async () => { + const driver = new InMemoryDriver({ persistence: 'file' }); + await driver.connect(); + await driver.create('items', { id: '1', name: 'Widget' }); + await driver.disconnect(); + }); }); it('should persist updates and deletes', async () => { @@ -157,11 +180,15 @@ describe('InMemoryDriver Persistence', () => { describe('Auto Persistence', () => { it('should auto-detect Node.js environment and use file persistence with shorthand', async () => { - // In Node.js, 'auto' should select file persistence - const driver = new InMemoryDriver({ persistence: 'auto' }); - await driver.connect(); - await driver.create('items', { id: '1', name: 'Widget' }); - await driver.disconnect(); + // In Node.js, 'auto' should select file persistence. + // Scratch CWD: the shorthand takes the adapter's default relative path. + await inScratchCwd(async () => { + const driver = new InMemoryDriver({ persistence: 'auto' }); + await driver.connect(); + await driver.create('items', { id: '1', name: 'Widget' }); + await driver.disconnect(); + expect(fs.existsSync(path.join('.objectstack', 'data', 'memory-driver.json'))).toBe(true); + }); }); it('should auto-detect Node.js environment and use file persistence with object config', async () => { @@ -213,6 +240,53 @@ describe('InMemoryDriver Persistence', () => { }); }); + // The default is the whole bug in #4065, and nothing pinned it — every test + // in this file passes `persistence` explicitly, so the driver was free to + // drift from #815's requirement #1 ("默认情况下不启用持久化（纯内存，行为不变）") + // to a silent `'auto'` → file write without a single assertion turning red. + describe('Default persistence (opt-in)', () => { + it('writes nothing to the CWD when persistence is not configured', async () => { + await inScratchCwd(async () => { + const driver = new InMemoryDriver(); + await driver.connect(); + await driver.create('items', { id: 'a', name: 'Widget' }); + await driver.disconnect(); + // The path the file adapter defaults to, relative to the CWD. + expect(fs.existsSync(path.join('.objectstack', 'data', 'memory-driver.json'))).toBe(false); + }); + }); + + it('does not carry rows across driver instances (the #4065 rerun shape)', async () => { + await inScratchCwd(async () => { + // Two instances seeded with the SAME fixed ids, exactly as a suite that + // runs twice in one working tree does. Under the old `'auto'` default + // the second instance loaded the first's rows and `find` returned four. + for (const _run of [1, 2]) { + const driver = new InMemoryDriver(); + await driver.connect(); + await driver.bulkCreate('ext_note', [ + { id: 'n1', title: 'first' }, + { id: 'n2', title: 'second' }, + ]); + const rows = await driver.find('ext_note', { object: 'ext_note' }); + expect(rows.map((r: any) => r.title).sort()).toEqual(['first', 'second']); + await driver.disconnect(); + } + }); + }); + + it('still persists when a host opts in explicitly', async () => { + const filePath = path.join(TEST_DATA_DIR, 'opt-in.json'); + const driver = new InMemoryDriver({ + persistence: { type: 'file', path: filePath, autoSaveInterval: 100 }, + }); + await driver.connect(); + await driver.create('items', { id: 'a', name: 'Widget' }); + await driver.disconnect(); + expect(fs.existsSync(filePath)).toBe(true); + }); + }); + describe('Serverless Environment Detection', () => { const serverlessEnvVars = [ 'VERCEL', diff --git a/packages/plugins/plugin-dev/src/dev-plugin.ts b/packages/plugins/plugin-dev/src/dev-plugin.ts index 57d88d9781..a1122884fa 100644 --- a/packages/plugins/plugin-dev/src/dev-plugin.ts +++ b/packages/plugins/plugin-dev/src/dev-plugin.ts @@ -469,7 +469,12 @@ export class DevPlugin implements Plugin { try { const { DriverPlugin } = await import('@objectstack/runtime') as any; const { InMemoryDriver } = await import('@objectstack/driver-memory') as any; - const driver = new InMemoryDriver(); + // Ephemeral, like every other service this plugin stubs (cache, queue, + // job, i18n, storage, search are all in-memory). Stated explicitly + // rather than inherited: the driver used to default to writing + // `.objectstack/data/memory-driver.json` into the CWD, which made this + // stack the one piece of DevPlugin that quietly outlived the process. + const driver = new InMemoryDriver({ persistence: false }); const driverPlugin = new DriverPlugin(driver, 'memory'); this.childPlugins.push(driverPlugin); ctx.logger.info(' ✔ InMemoryDriver enabled'); diff --git a/packages/services/service-datasource/src/default-datasource-driver-factory.ts b/packages/services/service-datasource/src/default-datasource-driver-factory.ts index d7cbc3e619..abc1572569 100644 --- a/packages/services/service-datasource/src/default-datasource-driver-factory.ts +++ b/packages/services/service-datasource/src/default-datasource-driver-factory.ts @@ -257,8 +257,18 @@ export function createDefaultDatasourceDriverFactory( } // memory + // + // Ephemeral unless the DEFINITION asks for durability. The driver's own + // default is `false` (#815 requirement #1, restored in #4065), and a + // datasource that wants its rows to survive a restart declares it — + // `config.persistence: 'file' | 'local' | 'auto'` — rather than inheriting + // a silent CWD write nobody requested. `persist` is accepted as the same + // spelling the sqlite-wasm branch above uses for its durability knob. const { InMemoryDriver } = await import('@objectstack/driver-memory'); - return toHandle(new InMemoryDriver()); + const memoryPersistence = cfg.persistence ?? persistOverride; + return toHandle( + new InMemoryDriver({ persistence: memoryPersistence ?? false } as never), + ); }, }; } diff --git a/packages/services/service-datasource/src/sqlite-driver-fallback.ts b/packages/services/service-datasource/src/sqlite-driver-fallback.ts index aba044b993..98d39b6170 100644 --- a/packages/services/service-datasource/src/sqlite-driver-fallback.ts +++ b/packages/services/service-datasource/src/sqlite-driver-fallback.ts @@ -188,8 +188,17 @@ export async function resolveSqliteDriver( return { driver: wasmDriver, engine: 'sqlite-wasm', label: 'SqliteWasmDriver' }; } - // 3. In-memory (mingo) — dev-only last resort. Not real SQL, not persistent. + // 3. In-memory (mingo) — dev-only last resort. Not real SQL. + // + // Durability mirrors the sqlite target this rung is standing in for: the + // caller asked for a FILE, so a dev's data still survives a restart; an + // ephemeral `:memory:` stays ephemeral. Stated explicitly because the driver + // no longer persists by default (#4065 / #815 requirement #1) — the step-down + // must not silently downgrade a file-backed dev database to a volatile one. const { InMemoryDriver } = await import('@objectstack/driver-memory'); warn(NATIVE_SQLITE_MEMORY_FALLBACK_WARNING); - return { driver: new InMemoryDriver(), engine: 'memory', label: 'InMemoryDriver' }; + const driver = new InMemoryDriver( + isEphemeralFilename(filename) ? { persistence: false } : { persistence: 'file' }, + ); + return { driver, engine: 'memory', label: 'InMemoryDriver' }; } diff --git a/packages/spec/src/data/driver/memory.test.ts b/packages/spec/src/data/driver/memory.test.ts index a85320a6c3..b5098ea798 100644 --- a/packages/spec/src/data/driver/memory.test.ts +++ b/packages/spec/src/data/driver/memory.test.ts @@ -12,16 +12,29 @@ import { } from './memory.zod'; describe('MemoryConfigSchema', () => { - it('should default persistence to "auto" when empty config', () => { + // Persistence is OPT-IN (#815 requirement #1, restored in #4065). This + // schema previously defaulted to `'auto'`, which on Node.js resolves to file + // persistence — so an unconfigured memory driver wrote + // `.objectstack/data/memory-driver.json` into the process CWD and reloaded it + // on the next boot. The spec and the driver agreed on `'auto'`, which is why + // the drift survived; what neither checked was whether `'auto'` was the + // default #815 accepted. It was not. + it('should default persistence to false (pure in-memory) when empty config', () => { const config = MemoryConfigSchema.parse({}); expect(config.strictMode).toBe(false); expect(config.initialData).toBeUndefined(); - expect(config.persistence).toBe('auto'); + expect(config.persistence).toBe(false); expect(config.indexes).toBeUndefined(); expect(config.maxRecordsPerObject).toBeUndefined(); }); + it('still accepts "auto" as an explicit opt-in to the previous behaviour', () => { + const config = MemoryConfigSchema.parse({ persistence: 'auto' }); + + expect(config.persistence).toBe('auto'); + }); + it('should accept persistence: false to disable persistence', () => { const config = MemoryConfigSchema.parse({ persistence: false, diff --git a/packages/spec/src/data/driver/memory.zod.ts b/packages/spec/src/data/driver/memory.zod.ts index cd70ec19e3..4f72a550cc 100644 --- a/packages/spec/src/data/driver/memory.zod.ts +++ b/packages/spec/src/data/driver/memory.zod.ts @@ -187,37 +187,40 @@ export const MemoryConfigSchema = lazySchema(() => z.object({ strictMode: z.boolean().default(false).describe('Throw on missing records instead of returning null'), /** - * Persistence configuration. - * Defaults to `'auto'` — the memory store automatically detects the environment - * and saves/restores data using the best available strategy. + * Persistence configuration. **Defaults to `false` — pure in-memory.** * - * - `'auto'` (default): Auto-detect environment (browser → localStorage, Node.js → file) + * Durability is opt-in, per #815's requirement #1 ("默认情况下不启用持久化（纯内存， + * 行为不变）"). A driver named "in-memory" that silently wrote to the process CWD + * surprised every caller that never asked for it, and made fixed-id fixtures pass + * on the first run and fail on every rerun (#4065). + * + * - `false` (default): No persistence — pure in-memory, data lost on disconnect + * - `'auto'`: Auto-detect environment (browser → localStorage, Node.js → file) * - `'file'`: Persist to disk file (Node.js only, default path: `.objectstack/data/memory-driver.json`) * - `'local'`: Persist to localStorage (Browser only, default key: `objectstack:memory-db`) * - `{ type: 'auto', path?: string, key?: string }`: Auto-detect with overrides * - `{ type: 'file', path?: string }`: File-system with custom path * - `{ type: 'local', key?: string }`: localStorage with custom key * - `{ adapter: PersistenceAdapter }`: Custom persistence adapter - * - `false`: Disable persistence (pure in-memory, data lost on disconnect) * * **⚠️ Serverless / Edge environments (Vercel, AWS Lambda, Netlify, etc.):** - * Auto mode detects serverless runtimes and disables file persistence to prevent - * silent data loss. Set `persistence: false` to opt-in to pure in-memory mode, - * or supply a custom adapter (e.g. Upstash Redis, Vercel KV) for durable storage. + * `'auto'` detects serverless runtimes and disables file persistence to prevent + * silent data loss. Supply a custom adapter (e.g. Upstash Redis, Vercel KV) for + * durable storage there. * * @example - * // Auto-detect environment (default) + * // Pure memory — the default * new InMemoryDriver() - * // Node.js + * // Node.js, durable across restarts * new InMemoryDriver({ persistence: 'file' }) - * // Browser + * // Browser, durable across reloads * new InMemoryDriver({ persistence: 'local' }) - * // Pure memory (no persistence) - * new InMemoryDriver({ persistence: false }) + * // Auto-detect the environment's best strategy + * new InMemoryDriver({ persistence: 'auto' }) * // Custom adapter for serverless * new InMemoryDriver({ persistence: { adapter: upstashAdapter } }) */ - persistence: MemoryPersistenceConfigSchema.or(z.literal(false)).default('auto').describe('Persistence configuration (defaults to auto-detect)'), + persistence: MemoryPersistenceConfigSchema.or(z.literal(false)).default(false).describe('Persistence configuration (opt-in; defaults to pure in-memory)'), /** * Fields to index for faster lookups. From a7e932f14d701876f0f556de4ca43062bbb5193d Mon Sep 17 00:00:00 2001 From: Claude Date: Thu, 30 Jul 2026 09:26:59 +0000 Subject: [PATCH 2/5] test(runtime,client,metadata): back the remaining suites with in-memory SQLite (#4065) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Ten test files used `InMemoryDriver` as a convenience backing store — somewhere for rows to go while the suite proved something else (REST routing, datasource auto-connect, the batch `$ref` contract, metadata history). They now run on `SqliteWasmDriver` at `:memory:`, the engine `@objectstack/verify`'s `bootStack` already gives the dogfood gate: pure JS (no native build, CI-safe) and real SQL. The point is fidelity, not tidiness. Production runs SQL, and mingo differs from it in ways that let a suite pass while the behaviour it stands for is broken. Every failure this migration produced was a fixture defect the memory driver had been absorbing: - Tables were never created. `create()` on the memory driver is a bare `table.push()` onto an auto-vivified array, so an object registered AFTER `kernel.bootstrap()` — which misses the boot-time schema sync — looked fine. On SQL the first write fails with `no such table`, which the REST error mapper turns into a 404 OBJECT_NOT_FOUND: a routing-shaped symptom for a DDL-shaped cause. Four suites needed an explicit `syncObjectSchema`; the `schemaMode: 'external'` datasource in datasource-autoconnect needed an `initObjects` standing in for the DDL that already ran on the remote side — the step a real external datasource genuinely requires. - A missing object declaration read as working. notifications.hono.integration writes `sys_notification`, which MessagingServicePlugin does not declare (it is a platform object, and that lean kernel never boots platform-objects). Auto-vivification hid it. The suite now registers the REAL SysNotification rather than a hand-copied stand-in, keeping one schema (Prime Directive #12). - `connect()` was optional. The memory driver needs none; a SQL driver does. Deliberately NOT moved: read-coercion-conformance keeps its two-driver matrix (proving a stored value reads back as its declared type on BOTH engines is the point of that gate), and the suites whose subject IS the memory driver or its wiring — standalone-stack (`memory://`), sqlite-driver-fallback (the dev step-down), the CLI driver-label tests, and driver-memory's own suite. No new coverage is claimed: each suite asserts exactly what it did before, against a more faithful store. 2,375 tests green across the eight affected packages. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01L5q3BZMpdiAueHf2emhDY9 --- .changeset/tests-off-memory-driver.md | 46 +++++++++++++++++++ packages/client/package.json | 2 +- .../src/client.batch-transaction.test.ts | 21 +++++++-- .../src/client.environment-scoping.test.ts | 9 +++- packages/client/src/client.hono.test.ts | 9 +++- packages/metadata/package.json | 1 + .../metadata/src/metadata-history.test.ts | 15 ++++-- packages/metadata/vitest.config.ts | 1 - packages/qa/http-conformance/package.json | 4 +- .../src/conformance.integration.test.ts | 18 +++++++- packages/runtime/package.json | 1 + .../src/datasource-autoconnect.test.ts | 45 ++++++++++++------ .../runtime/src/datasource-visibility.test.ts | 4 +- .../src/default-datasource-plugin.test.ts | 8 ++-- .../notifications.hono.integration.test.ts | 22 +++++++-- packages/runtime/vitest.config.ts | 4 ++ pnpm-lock.yaml | 14 ++++-- 17 files changed, 176 insertions(+), 48 deletions(-) create mode 100644 .changeset/tests-off-memory-driver.md diff --git a/.changeset/tests-off-memory-driver.md b/.changeset/tests-off-memory-driver.md new file mode 100644 index 0000000000..b3b93c0268 --- /dev/null +++ b/.changeset/tests-off-memory-driver.md @@ -0,0 +1,46 @@ +--- +"@objectstack/runtime": patch +"@objectstack/client": patch +"@objectstack/metadata": patch +--- + +test(runtime,client,metadata): back the remaining suites with in-memory SQLite instead of the mingo driver (#4065) + +Ten test files used `InMemoryDriver` as a convenience backing store — somewhere +for rows to go while the suite proved something else (REST routing, datasource +auto-connect, the batch `$ref` contract, metadata history). They now run on +`SqliteWasmDriver` at `:memory:`, the same engine `@objectstack/verify`'s +`bootStack` already gives the dogfood gate: pure JS (no native build, CI-safe on +any runner) and real SQL semantics. + +The point is fidelity, not tidiness. Production runs SQL, and mingo differs from +it in ways that let a suite pass while the behaviour it stands for is broken. +Every failure this migration produced was a fixture defect the memory driver had +been absorbing: + +- **Tables were never created.** `driver.create()` on the memory driver is a + bare `table.push()` onto an auto-vivified array, so an object registered + *after* `kernel.bootstrap()` — which misses the boot-time schema sync — looked + fine. On SQL the first write fails with `no such table`, which the REST error + mapper turns into a **404 `OBJECT_NOT_FOUND`**: a routing-shaped symptom for a + DDL-shaped cause. Four suites needed an explicit `syncObjectSchema` (or, for + the `schemaMode: 'external'` datasource in `datasource-autoconnect`, an + `initObjects` standing in for the DDL that already ran on the remote side — + the step a real external datasource genuinely requires). +- **A missing object declaration read as working.** `notifications.hono.integration` + writes `sys_notification`, which `MessagingServicePlugin` does not declare — + it is a platform object, and that lean kernel never booted `platform-objects`. + Auto-vivification hid the omission entirely. The suite now registers the real + `SysNotification` rather than a hand-copied stand-in, so there is still exactly + one schema for it (Prime Directive #12). +- **`connect()` was optional.** The memory driver needs none; a SQL driver does. + +What deliberately did NOT move: `read-coercion-conformance` keeps its two-driver +matrix (proving a stored value reads back as its declared type on *both* engines +is the entire point of that gate), and the suites whose subject IS the memory +driver or its wiring — `standalone-stack` (`memory://` scheme), +`sqlite-driver-fallback` (the dev step-down), the CLI's driver-label tests, and +driver-memory's own suite. + +No new coverage is claimed here: each suite asserts exactly what it asserted +before, against a more faithful store. diff --git a/packages/client/package.json b/packages/client/package.json index d1edfa1d91..3593ab06cb 100644 --- a/packages/client/package.json +++ b/packages/client/package.json @@ -23,7 +23,7 @@ }, "devDependencies": { "@hono/node-server": "^2.0.12", - "@objectstack/driver-memory": "workspace:*", + "@objectstack/driver-sqlite-wasm": "workspace:*", "@objectstack/hono": "workspace:*", "@objectstack/objectql": "workspace:*", "@objectstack/plugin-hono-server": "workspace:*", diff --git a/packages/client/src/client.batch-transaction.test.ts b/packages/client/src/client.batch-transaction.test.ts index 2fec232c9e..00a3ab76f5 100644 --- a/packages/client/src/client.batch-transaction.test.ts +++ b/packages/client/src/client.batch-transaction.test.ts @@ -15,16 +15,21 @@ * 4. `atomic: false` is rejected with 400 BATCH_NOT_ATOMIC — the contract * reason the SDK method exposes no `atomic` flag. * - * NOTE on atomicity: the InMemoryDriver's `transaction()` is a passthrough - * (no rollback), so all-or-nothing semantics are NOT asserted here — they are + * NOTE on atomicity: all-or-nothing semantics are NOT asserted here — they are * covered by objectql/src/engine-ambient-transaction.test.ts and - * rest/src/rest-batch-endpoint.test.ts against transactional drivers. + * rest/src/rest-batch-endpoint.test.ts. This suite's four proofs are about the + * `$ref` wiring, the environment-scoped mirror and the error contract, not + * rollback. (It ran on the mingo driver until #4065, whose `transaction()` is a + * passthrough and could not have asserted rollback even if it wanted to; the + * in-memory SQLite it now runs on does have real transactions, but adding a + * rollback proof here would duplicate the two suites above rather than close a + * gap.) */ import { describe, it, expect, beforeAll, afterAll } from 'vitest'; import { LiteKernel } from '@objectstack/core'; import { ObjectQL, ObjectQLPlugin } from '@objectstack/objectql'; -import { InMemoryDriver } from '@objectstack/driver-memory'; +import { SqliteWasmDriver } from '@objectstack/driver-sqlite-wasm'; import { HonoServerPlugin } from '@objectstack/plugin-hono-server'; import { createRestApiPlugin } from '@objectstack/runtime'; import { ObjectStackClient } from './index'; @@ -75,7 +80,7 @@ describe('data.batchTransaction (live Hono, #1604)', () => { await kernel.bootstrap(); const ql = kernel.getService('objectql'); - ql.registerDriver(new InMemoryDriver(), true); + ql.registerDriver(new SqliteWasmDriver({ filename: ':memory:' }) as never, true); ql.registerObject({ name: 'project', @@ -92,6 +97,12 @@ describe('data.batchTransaction (live Hono, #1604)', () => { project: { type: 'lookup', reference_to: 'project', label: 'Project' }, }, }); + // Objects registered AFTER bootstrap miss the boot-time schema sync, so + // nothing has issued their DDL. The mingo driver this suite used before + // #4065 created a table on first touch and hid that; on SQL the first + // write fails with `no such table`. + await ql.syncObjectSchema('project'); + await ql.syncObjectSchema('task'); const httpServer = kernel.getService('http.server'); baseUrl = `http://localhost:${httpServer.getPort()}`; diff --git a/packages/client/src/client.environment-scoping.test.ts b/packages/client/src/client.environment-scoping.test.ts index 7fb99a3e56..c44193f500 100644 --- a/packages/client/src/client.environment-scoping.test.ts +++ b/packages/client/src/client.environment-scoping.test.ts @@ -16,7 +16,7 @@ import { describe, it, expect, beforeAll, afterAll } from 'vitest'; import { LiteKernel } from '@objectstack/core'; import { ObjectQL, ObjectQLPlugin } from '@objectstack/objectql'; -import { InMemoryDriver } from '@objectstack/driver-memory'; +import { SqliteWasmDriver } from '@objectstack/driver-sqlite-wasm'; import { HonoServerPlugin } from '@objectstack/plugin-hono-server'; import { createRestApiPlugin } from '@objectstack/runtime'; import { ObjectStackClient } from './index'; @@ -68,7 +68,7 @@ describe('Project-scoped REST routing (live Hono)', () => { await kernel.bootstrap(); const ql = kernel.getService('objectql'); - ql.registerDriver(new InMemoryDriver(), true); + ql.registerDriver(new SqliteWasmDriver({ filename: ':memory:' }) as never, true); ql.registerObject({ name: 'task', @@ -77,6 +77,11 @@ describe('Project-scoped REST routing (live Hono)', () => { title: { type: 'text', label: 'Title' }, }, }); + // Objects registered AFTER bootstrap miss the boot-time schema sync, so + // nothing has issued their DDL. The mingo driver this suite used before + // #4065 created a table on first touch and hid that; on SQL the first + // write fails with `no such table`. + await ql.syncObjectSchema('task'); const httpServer = kernel.getService('http.server'); const port = httpServer.getPort(); diff --git a/packages/client/src/client.hono.test.ts b/packages/client/src/client.hono.test.ts index 185998928d..2a467567d1 100644 --- a/packages/client/src/client.hono.test.ts +++ b/packages/client/src/client.hono.test.ts @@ -1,7 +1,7 @@ import { describe, it, expect, beforeAll, afterAll } from 'vitest'; import { LiteKernel } from '@objectstack/core'; import { ObjectQL, ObjectQLPlugin } from '@objectstack/objectql'; -import { InMemoryDriver } from '@objectstack/driver-memory'; +import { SqliteWasmDriver } from '@objectstack/driver-sqlite-wasm'; import { HonoServerPlugin } from '@objectstack/plugin-hono-server'; import { ObjectStackClient } from './index'; @@ -108,7 +108,7 @@ describe('ObjectStackClient (with Hono Server)', () => { // 3. Setup Driver const ql = kernel.getService('objectql'); - ql.registerDriver(new InMemoryDriver(), true); + ql.registerDriver(new SqliteWasmDriver({ filename: ':memory:' }) as never, true); // 4. Load Metadata (Schema) ql.registerObject({ @@ -119,6 +119,11 @@ describe('ObjectStackClient (with Hono Server)', () => { email: { type: 'text', label: 'Email' } } }); + // Objects registered AFTER bootstrap miss the boot-time schema sync, so + // nothing has issued their DDL. The mingo driver this suite used before + // #4065 created a table on first touch and hid that; on SQL the first + // write fails with `no such table`. + await ql.syncObjectSchema('customer'); // 5. Get Port from Service const httpServer = kernel.getService('http.server'); diff --git a/packages/metadata/package.json b/packages/metadata/package.json index c88185368e..882efe6ec5 100644 --- a/packages/metadata/package.json +++ b/packages/metadata/package.json @@ -55,6 +55,7 @@ "zod": "^4.4.3" }, "devDependencies": { + "@objectstack/driver-sqlite-wasm": "workspace:*", "@types/js-yaml": "^4.0.9", "@types/node": "^26.1.1", "typescript": "^6.0.3", diff --git a/packages/metadata/src/metadata-history.test.ts b/packages/metadata/src/metadata-history.test.ts index 33328f7bd1..193ef648d8 100644 --- a/packages/metadata/src/metadata-history.test.ts +++ b/packages/metadata/src/metadata-history.test.ts @@ -3,15 +3,20 @@ import { describe, it, expect, beforeEach } from 'vitest'; import { MetadataManager } from './metadata-manager'; import { DatabaseLoader } from './loaders/database-loader'; -import { InMemoryDriver } from '@objectstack/driver-memory'; +import { SqliteWasmDriver } from '@objectstack/driver-sqlite-wasm'; describe('Metadata History', () => { let manager: MetadataManager; - let driver: InMemoryDriver; + let driver: SqliteWasmDriver; beforeEach(async () => { - // Create a fresh in-memory driver and database loader - driver = new InMemoryDriver({}); + // A fresh in-memory SQLite database per test + the database loader. + // `DatabaseLoader` provisions its own two tables via `driver.syncSchema`, + // so it needs a CONNECTED driver — the mingo driver this used before #4065 + // needed no connect and created tables on first touch, so neither step was + // visible here. + driver = new SqliteWasmDriver({ filename: ':memory:' }); + await driver.connect(); const dbLoader = new DatabaseLoader({ driver, @@ -21,7 +26,7 @@ describe('Metadata History', () => { }); manager = new MetadataManager({ - datasource: 'memory', + datasource: 'sqlite', loaders: [dbLoader], }); }); diff --git a/packages/metadata/vitest.config.ts b/packages/metadata/vitest.config.ts index 10915ef3c0..df7bb9b54a 100644 --- a/packages/metadata/vitest.config.ts +++ b/packages/metadata/vitest.config.ts @@ -7,7 +7,6 @@ export default defineConfig({ resolve: { alias: { '@objectstack/core': path.resolve(__dirname, '../core/src/index.ts'), - '@objectstack/driver-memory': path.resolve(__dirname, '../plugins/driver-memory/src/index.ts'), '@objectstack/spec/api': path.resolve(__dirname, '../spec/src/api/index.ts'), '@objectstack/spec/cloud': path.resolve(__dirname, '../spec/src/cloud/index.ts'), '@objectstack/spec/contracts': path.resolve(__dirname, '../spec/src/contracts/index.ts'), diff --git a/packages/qa/http-conformance/package.json b/packages/qa/http-conformance/package.json index 5898f53ffb..714a2562a2 100644 --- a/packages/qa/http-conformance/package.json +++ b/packages/qa/http-conformance/package.json @@ -3,7 +3,7 @@ "version": "0.0.6-rc.0", "private": true, "license": "Apache-2.0", - "description": "HTTP transport-port conformance gate (ADR-0076 D11/OQ#10, #2462) — a zero-dependency node:http reference implementation of IHttpServer plus a cross-adapter suite that boots the dispatcher bridge and REST generator on it AND on plugin-hono-server, pinning that the port stays free of framework-isms. Not published; validation instrument, not a product server.", + "description": "HTTP transport-port conformance gate (ADR-0076 D11/OQ#10, #2462) \u2014 a zero-dependency node:http reference implementation of IHttpServer plus a cross-adapter suite that boots the dispatcher bridge and REST generator on it AND on plugin-hono-server, pinning that the port stays free of framework-isms. Not published; validation instrument, not a product server.", "type": "module", "scripts": { "test": "vitest run" @@ -12,7 +12,7 @@ "@objectstack/core": "workspace:*" }, "devDependencies": { - "@objectstack/driver-memory": "workspace:*", + "@objectstack/driver-sqlite-wasm": "workspace:*", "@objectstack/objectql": "workspace:*", "@objectstack/plugin-hono-server": "workspace:*", "@objectstack/runtime": "workspace:*", diff --git a/packages/qa/http-conformance/src/conformance.integration.test.ts b/packages/qa/http-conformance/src/conformance.integration.test.ts index 876c28eba6..614227b935 100644 --- a/packages/qa/http-conformance/src/conformance.integration.test.ts +++ b/packages/qa/http-conformance/src/conformance.integration.test.ts @@ -18,7 +18,7 @@ import { describe, it, expect, beforeAll, afterAll } from 'vitest'; import { LiteKernel } from '@objectstack/core'; import { ObjectQL, ObjectQLPlugin } from '@objectstack/objectql'; -import { InMemoryDriver } from '@objectstack/driver-memory'; +import { SqliteWasmDriver } from '@objectstack/driver-sqlite-wasm'; import { HonoServerPlugin } from '@objectstack/plugin-hono-server'; import { createDispatcherPlugin, createRestApiPlugin } from '@objectstack/runtime'; import { NodeServerPlugin } from './node-plugin.js'; @@ -74,7 +74,15 @@ async function bootStack(makePlugin: () => any, opts: { withAnalytics?: boolean await kernel.bootstrap(); const ql = kernel.getService('objectql'); - ql.registerDriver(new InMemoryDriver(), true); + // In-memory SQLite (pure-JS WASM, no native build) — the same engine + // `@objectstack/verify`'s bootStack runs the dogfood gate on. `connect()` + // is NOT optional the way it was for the mingo driver this used before + // #4065: an unconnected datasource is an UNAVAILABLE one, and every data + // entry point then reports the object as unregistered (a 404 that looks + // like a routing bug and is really a boot-order one). + const driver = new SqliteWasmDriver({ filename: ':memory:' }); + await driver.connect(); + ql.registerDriver(driver as never, true); ql.registerObject({ name: 'task', label: 'Task', @@ -82,6 +90,12 @@ async function bootStack(makePlugin: () => any, opts: { withAnalytics?: boolean title: { type: 'text', label: 'Title' }, }, }); + // Registering an object after bootstrap misses the boot-time schema sync, + // so nothing has issued this table's DDL. The mingo driver this fixture used + // before #4065 created a table on first touch and hid that; on SQL the + // insert fails with `no such table`, which the REST error mapper turns into + // a 404 OBJECT_NOT_FOUND — a routing-shaped symptom for a DDL-shaped cause. + await ql.syncObjectSchema('task'); const httpServer = kernel.getService('http.server'); return { kernel, base: `http://127.0.0.1:${httpServer.getPort()}` }; diff --git a/packages/runtime/package.json b/packages/runtime/package.json index 743d04332e..cc2e39d3ae 100644 --- a/packages/runtime/package.json +++ b/packages/runtime/package.json @@ -44,6 +44,7 @@ "@objectstack/driver-mongodb": "workspace:*" }, "devDependencies": { + "@objectstack/platform-objects": "workspace:*", "@objectstack/plugin-hono-server": "workspace:*", "@objectstack/service-datasource": "workspace:*", "@objectstack/service-messaging": "workspace:*", diff --git a/packages/runtime/src/datasource-autoconnect.test.ts b/packages/runtime/src/datasource-autoconnect.test.ts index 99505205f8..4080758280 100644 --- a/packages/runtime/src/datasource-autoconnect.test.ts +++ b/packages/runtime/src/datasource-autoconnect.test.ts @@ -8,8 +8,15 @@ // This boots the host-config shape (instantiated plugins, no MetadataPlugin — // the same shape `examples/app-showcase` runs under `os dev`) with the REAL // driver factory (`createDefaultDatasourceDriverFactory`) building an in-memory -// driver, so the full AppPlugin → `datasource-connection` → engine path runs -// without any native driver dependency. +// SQLite database, so the full AppPlugin → `datasource-connection` → engine path +// runs without any native driver dependency. +// +// The engine is the pure-JS WASM SQLite driver at `:memory:` — the same choice +// `@objectstack/verify`'s `bootStack` makes for the dogfood gate, and for the +// same two reasons: no native build (so it is CI-safe on any runner) and REAL +// SQL semantics, so a federated read proved here behaves the way it will +// against the Postgres/MySQL an external datasource actually points at. This +// suite used `driver: 'memory'` until #4065; mingo is neither of those things. import { describe, it, expect, beforeAll, afterAll, beforeEach, afterEach } from 'vitest'; import { Runtime } from './runtime.js'; @@ -39,11 +46,11 @@ function artifact() { datasources: [ { name: 'autoconn_ext', - label: 'External (in-memory)', - driver: 'memory', + label: 'External (in-memory SQLite)', + driver: 'sqlite-wasm', schemaMode: 'external', origin: 'code', - config: {}, + config: { filename: ':memory:' }, external: { allowWrites: false, validation: { onMismatch: 'warn', checkOnBoot: false } }, active: true, }, @@ -52,10 +59,10 @@ function artifact() { { name: 'decorative', label: 'Decorative (unrouted)', - driver: 'memory', + driver: 'sqlite-wasm', schemaMode: 'managed', origin: 'code', - config: {}, + config: { filename: ':memory:' }, active: true, }, ], @@ -64,14 +71,14 @@ function artifact() { async function boot(opts: { connectPolicy?: DatasourceConnectPolicy } = {}) { const { ObjectQLPlugin } = await import('@objectstack/objectql'); - const { InMemoryDriver } = await import('@objectstack/driver-memory'); + const { SqliteWasmDriver } = await import('@objectstack/driver-sqlite-wasm'); const { DatasourceAdminServicePlugin, createDefaultDatasourceDriverFactory } = await import( '@objectstack/service-datasource' ); const runtime = new Runtime({ cluster: false }); const kernel = runtime.getKernel(); - await kernel.use(new DriverPlugin(new InMemoryDriver())); // default driver + await kernel.use(new DriverPlugin(new SqliteWasmDriver({ filename: ':memory:' }))); // default driver await kernel.use(new ObjectQLPlugin()); await kernel.use(new AppPlugin(artifact())); await kernel.use( @@ -122,6 +129,14 @@ describe('ADR-0062 declared-datasource auto-connect', () => { // Seed the live external driver directly (bypassing the read-only write gate, // exactly as a real remote DB would already hold the rows). const driver = engine.getDriverByName('autoconn_ext'); + // `schemaMode: 'external'` means ObjectStack does NOT own this table — the + // remote database does — so nothing in the boot path creates it. Materialize + // it here, standing in for the DDL that already ran on the other side. The + // memory driver this suite used before #4065 auto-created a table on first + // touch, which quietly hid the step a real external datasource requires. + await driver.initObjects([ + { name: 'ext_note', fields: { id: { type: 'text' }, title: { type: 'text' } } }, + ]); await driver.bulkCreate('ext_note', [ { id: 'n1', title: 'first' }, { id: 'n2', title: 'second' }, @@ -142,10 +157,10 @@ describe('ADR-0062 credentials fail-closed (D3)', () => { datasources: [ { name: 'needs_secret', - driver: 'memory', + driver: 'sqlite-wasm', schemaMode: 'external', origin: 'code', - config: {}, + config: { filename: ':memory:' }, external: { allowWrites: false, credentialsRef: 'sys_secret:does-not-exist', @@ -159,13 +174,13 @@ describe('ADR-0062 credentials fail-closed (D3)', () => { it('bricks boot with a clear message when a required credential cannot be resolved', async () => { const { ObjectQLPlugin } = await import('@objectstack/objectql'); - const { InMemoryDriver } = await import('@objectstack/driver-memory'); + const { SqliteWasmDriver } = await import('@objectstack/driver-sqlite-wasm'); const { DatasourceAdminServicePlugin, createDefaultDatasourceDriverFactory } = await import( '@objectstack/service-datasource' ); const runtime = new Runtime({ cluster: false }); const kernel = runtime.getKernel(); - await kernel.use(new DriverPlugin(new InMemoryDriver())); + await kernel.use(new DriverPlugin(new SqliteWasmDriver({ filename: ':memory:' }))); await kernel.use(new ObjectQLPlugin()); await kernel.use(new AppPlugin(credArtifact())); await kernel.use( @@ -211,13 +226,13 @@ describe('ADR-0062 D5 — an explicitly-bound datasource that cannot connect bri async function bootBound() { const { ObjectQLPlugin } = await import('@objectstack/objectql'); - const { InMemoryDriver } = await import('@objectstack/driver-memory'); + const { SqliteWasmDriver } = await import('@objectstack/driver-sqlite-wasm'); const { DatasourceAdminServicePlugin, createDefaultDatasourceDriverFactory } = await import( '@objectstack/service-datasource' ); const runtime = new Runtime({ cluster: false }); const kernel = runtime.getKernel(); - await kernel.use(new DriverPlugin(new InMemoryDriver())); + await kernel.use(new DriverPlugin(new SqliteWasmDriver({ filename: ':memory:' }))); await kernel.use(new ObjectQLPlugin()); await kernel.use(new AppPlugin(boundArtifact())); await kernel.use( diff --git a/packages/runtime/src/datasource-visibility.test.ts b/packages/runtime/src/datasource-visibility.test.ts index 7e2520a83d..24db33a550 100644 --- a/packages/runtime/src/datasource-visibility.test.ts +++ b/packages/runtime/src/datasource-visibility.test.ts @@ -52,14 +52,14 @@ describe('code-defined datasource visibility (ADR-0015 §18)', () => { beforeAll(async () => { const { ObjectQLPlugin } = await import('@objectstack/objectql'); - const { InMemoryDriver } = await import('@objectstack/driver-memory'); + const { SqliteWasmDriver } = await import('@objectstack/driver-sqlite-wasm'); const { DatasourceAdminServicePlugin } = await import('@objectstack/service-datasource'); // Host-config shape: NO MetadataPlugin — the kernel auto-injects its // in-memory `metadata` fallback (CORE_FALLBACK_FACTORIES.metadata). const runtime = new Runtime({ cluster: false }); kernel = runtime.getKernel(); - await kernel.use(new DriverPlugin(new InMemoryDriver())); + await kernel.use(new DriverPlugin(new SqliteWasmDriver({ filename: ':memory:' }))); await kernel.use(new ObjectQLPlugin()); await kernel.use(new AppPlugin(ARTIFACT)); await kernel.use(new DatasourceAdminServicePlugin({})); diff --git a/packages/runtime/src/default-datasource-plugin.test.ts b/packages/runtime/src/default-datasource-plugin.test.ts index 042bd42a52..f4ecfd03ae 100644 --- a/packages/runtime/src/default-datasource-plugin.test.ts +++ b/packages/runtime/src/default-datasource-plugin.test.ts @@ -146,8 +146,8 @@ describe('DefaultDatasourcePlugin — the default datasource as a declaration (# // rebuild. createPrebuiltDriverFactory wraps it; the plugin's connect // orchestration must register THAT instance — never a rebuilt copy. const { createPrebuiltDriverFactory } = await import('@objectstack/service-datasource'); - const { InMemoryDriver } = await import('@objectstack/driver-memory'); - const hostBuilt = new InMemoryDriver(); + const { SqliteWasmDriver } = await import('@objectstack/driver-sqlite-wasm'); + const hostBuilt = new SqliteWasmDriver({ filename: ':memory:' }); const kernel = await assemble({ driver: 'turso', // a kind the SHARED factory does not support — proves dispatch factory: createPrebuiltDriverFactory(hostBuilt, { driverId: 'turso' }), @@ -211,8 +211,8 @@ describe('DefaultDatasourcePlugin — the default datasource as a declaration (# // registry cache). An LRU eviction shutting the kernel down must not pull // the pool from under every other consumer. const { createPrebuiltDriverFactory } = await import('@objectstack/service-datasource'); - const { InMemoryDriver } = await import('@objectstack/driver-memory'); - const hostBuilt = new InMemoryDriver() as any; + const { SqliteWasmDriver } = await import('@objectstack/driver-sqlite-wasm'); + const hostBuilt = new SqliteWasmDriver({ filename: ':memory:' }) as any; let disconnects = 0; const orig = hostBuilt.disconnect?.bind(hostBuilt); hostBuilt.disconnect = async () => { disconnects += 1; return orig?.(); }; diff --git a/packages/runtime/src/notifications.hono.integration.test.ts b/packages/runtime/src/notifications.hono.integration.test.ts index d19fb4a782..30acccaa6d 100644 --- a/packages/runtime/src/notifications.hono.integration.test.ts +++ b/packages/runtime/src/notifications.hono.integration.test.ts @@ -4,11 +4,13 @@ import { describe, it, expect, beforeAll, afterAll } from 'vitest'; import { ObjectKernel, Plugin, PluginContext } from '@objectstack/core'; import { HonoServerPlugin } from '@objectstack/plugin-hono-server'; import { ObjectQLPlugin } from '@objectstack/objectql'; -import { InMemoryDriver } from '@objectstack/driver-memory'; +import { SqliteWasmDriver } from '@objectstack/driver-sqlite-wasm'; import { MessagingServicePlugin, MessagingService } from '@objectstack/service-messaging'; +import { SysNotification } from '@objectstack/platform-objects'; import { createDispatcherPlugin } from './dispatcher-plugin.js'; import { DriverPlugin } from './driver-plugin.js'; +import { AppPlugin } from './app-plugin.js'; /** * End-to-end regression for framework #3362 (`#3354 not effective on hono`). @@ -67,13 +69,27 @@ describe('in-app notifications over a real hono server (integration, #3362)', () beforeAll(async () => { kernel = new ObjectKernel({ logLevel: 'silent' }); - // An in-memory driver backs persistence; ObjectQL (registered after the + // In-memory SQLite backs persistence; ObjectQL (registered after the // driver so it discovers it) provides `objectql` + `data` + `manifest`; // MessagingServicePlugin registers the `notification` service the dispatcher // resolves and owns the inbox tables. Inline delivery (reliableDelivery:false) // writes the inbox row synchronously so `emit()` is observable immediately. - await kernel.use(new DriverPlugin(new InMemoryDriver())); + await kernel.use(new DriverPlugin(new SqliteWasmDriver({ filename: ':memory:' }))); await kernel.use(new ObjectQLPlugin()); + // The L2 event object `sys_notification` is a PLATFORM object, declared in + // `@objectstack/platform-objects` — MessagingServicePlugin writes it but + // does not declare it, so this lean kernel (no platform-objects boot) has to + // register it or the engine has no schema to issue DDL from. Under the + // mingo driver this suite used before #4065 the omission was invisible: it + // auto-creates a table on first touch, so a missing declaration read as + // working. Registering the REAL object rather than a hand-copied stand-in + // keeps one schema (Prime Directive #12). + await kernel.use( + new AppPlugin({ + manifest: { id: 'com.test.notifications-e2e', name: 'Notifications E2E', version: '1.0.0' }, + objects: [SysNotification], + } as never), + ); await kernel.use(new MessagingServicePlugin({ reliableDelivery: false })); await kernel.use(fakeAuthPlugin()); // port 0 → OS-assigned free port; resolved via getPort() after listening. diff --git a/packages/runtime/vitest.config.ts b/packages/runtime/vitest.config.ts index 8bd526c2d9..5eb4520ac4 100644 --- a/packages/runtime/vitest.config.ts +++ b/packages/runtime/vitest.config.ts @@ -17,6 +17,10 @@ export default defineConfig({ '@objectstack/spec/api': path.resolve(__dirname, '../spec/src/api/index.ts'), '@objectstack/spec/contracts': path.resolve(__dirname, '../spec/src/contracts/index.ts'), '@objectstack/spec/data': path.resolve(__dirname, '../spec/src/data/index.ts'), + // Reached via `@objectstack/platform-objects` (sys-user.object.ts), which + // notifications.hono.integration.test.ts pulls in for the real + // `sys_notification` declaration. + '@objectstack/spec/identity': path.resolve(__dirname, '../spec/src/identity/index.ts'), '@objectstack/spec/kernel': path.resolve(__dirname, '../spec/src/kernel/index.ts'), '@objectstack/spec/shared': path.resolve(__dirname, '../spec/src/shared/index.ts'), '@objectstack/spec/system': path.resolve(__dirname, '../spec/src/system/index.ts'), diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml index 1e60504dd9..768571bd73 100644 --- a/pnpm-lock.yaml +++ b/pnpm-lock.yaml @@ -573,9 +573,9 @@ importers: '@hono/node-server': specifier: ^2.0.12 version: 2.0.12(hono@4.12.32) - '@objectstack/driver-memory': + '@objectstack/driver-sqlite-wasm': specifier: workspace:* - version: link:../plugins/driver-memory + version: link:../plugins/driver-sqlite-wasm '@objectstack/hono': specifier: workspace:* version: link:../adapters/hono @@ -887,6 +887,9 @@ importers: specifier: ^4.4.3 version: 4.4.3 devDependencies: + '@objectstack/driver-sqlite-wasm': + specifier: workspace:* + version: link:../plugins/driver-sqlite-wasm '@types/js-yaml': specifier: ^4.0.9 version: 4.0.9 @@ -1680,9 +1683,9 @@ importers: specifier: workspace:* version: link:../../core devDependencies: - '@objectstack/driver-memory': + '@objectstack/driver-sqlite-wasm': specifier: workspace:* - version: link:../../plugins/driver-memory + version: link:../../plugins/driver-sqlite-wasm '@objectstack/objectql': specifier: workspace:* version: link:../../objectql @@ -1808,6 +1811,9 @@ importers: specifier: ^4.4.3 version: 4.4.3 devDependencies: + '@objectstack/platform-objects': + specifier: workspace:* + version: link:../platform-objects '@objectstack/plugin-hono-server': specifier: workspace:* version: link:../plugins/plugin-hono-server From 988910a8ec2277b86955e88af2bfc3860b332d6c Mon Sep 17 00:00:00 2001 From: Claude Date: Thu, 30 Jul 2026 09:30:34 +0000 Subject: [PATCH 3/5] chore(cli,create-objectstack,docs): scaffolds no longer name a driver (#4065) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `os init` and the `create-objectstack` blank template both listed `@objectstack/driver-memory` in the generated `dependencies`. It was the only driver named, which read as an endorsement — "this is the driver your app runs on" — when it is the LAST-RESORT rung of the dev step-down (native better-sqlite3 → WASM SQLite → mingo). A new project's first impression of the data layer should not be the engine that enforces no primary keys, no uniqueness, no NOT NULL and no column types. It was also redundant: `@objectstack/runtime` already depends on driver-sql, driver-sqlite-wasm and driver-memory, and every script in both scaffolds runs through the CLI, which carries all four. Removing the line changes nothing a generated project can do — `objectstack dev` still resolves SQLite by default and OS_DATABASE_URL still selects Postgres / MySQL / MongoDB. Docs updated to match: - *Your first project*'s package table dropped its driver row and now says where drivers actually come from. - *Database Drivers* § Memory Driver claimed "Data is lost when the process exits" — false while `'auto'` was the default, which wrote a file into the working directory. It now documents the opt-in persistence default, carries a migration callout for callers who relied on the old behaviour, and points test authors at in-memory SQLite with the reason (mingo enforces no constraints, so a green run against it is weaker evidence than it looks). The Vercel deployment snippet is left as-is: it builds an InMemoryDriver deliberately to stay self-contained and already warns that serverless loses all data on cold start — still accurate, and now true by construction rather than by the serverless detection in `'auto'`. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01L5q3BZMpdiAueHf2emhDY9 --- .changeset/scaffolds-drop-memory-driver.md | 28 +++++++++++++ content/docs/data-modeling/drivers.mdx | 42 ++++++++++++++++--- .../getting-started/your-first-project.mdx | 8 +++- packages/cli/src/commands/init.ts | 8 +++- .../src/templates/blank/package.json | 1 - 5 files changed, 77 insertions(+), 10 deletions(-) create mode 100644 .changeset/scaffolds-drop-memory-driver.md diff --git a/.changeset/scaffolds-drop-memory-driver.md b/.changeset/scaffolds-drop-memory-driver.md new file mode 100644 index 0000000000..e418b84a13 --- /dev/null +++ b/.changeset/scaffolds-drop-memory-driver.md @@ -0,0 +1,28 @@ +--- +"@objectstack/cli": patch +"create-objectstack": patch +--- + +chore(cli,create-objectstack): scaffolds no longer name a driver (#4065) + +`os init` and the `create-objectstack` blank template both listed +`@objectstack/driver-memory` in the generated `dependencies`. It was the only +driver named, which read as an endorsement — "this is the driver your app runs +on" — when it is in fact the **last-resort rung** of the dev step-down (native +`better-sqlite3` → WASM SQLite → mingo). A new project's first impression of the +data layer should not be the engine that enforces no primary keys, no +uniqueness, no `NOT NULL` and no column types. + +It was also redundant: `@objectstack/runtime` already depends on `driver-sql`, +`driver-sqlite-wasm` and `driver-memory`, and every script in both scaffolds runs +through the CLI, which carries all four. Removing the line changes nothing a +generated project can do — `objectstack dev` still resolves SQLite by default, +and `OS_DATABASE_URL` still selects Postgres / MySQL / MongoDB. + +Docs updated to match: the "packages you depend on" table in *Your first project* +no longer lists a driver row (it now says where drivers come from), and the +Memory Driver section of *Database Drivers* documents the opt-in persistence +default, carries a migration callout for the old `'auto'` behaviour, and points +test authors at in-memory SQLite. That section also claimed "Data is lost when +the process exits", which was simply false while `'auto'` was the default — it +wrote a file into the working directory. diff --git a/content/docs/data-modeling/drivers.mdx b/content/docs/data-modeling/drivers.mdx index 2955712ff6..d9cd16500f 100644 --- a/content/docs/data-modeling/drivers.mdx +++ b/content/docs/data-modeling/drivers.mdx @@ -353,11 +353,12 @@ equivalent age-based reap. ## Memory Driver The in-memory driver keeps records in plain in-process objects (queried via -[`mingo`](https://github.com/kofrasa/mingo)). Data is lost when the process exits. -It is the **last-resort fallback** in dev mode: `objectstack dev` prefers native -SQLite (`better-sqlite3`), falls back to the pure-JS WASM SQLite driver if the -native binary is unavailable, and only drops to the in-memory driver if WASM also -fails to load. Set `OS_DATABASE_DRIVER=memory` to select it explicitly. +[`mingo`](https://github.com/kofrasa/mingo)). By default data is lost when the +process exits. It is the **last-resort fallback** in dev mode: `objectstack dev` +prefers native SQLite (`better-sqlite3`), falls back to the pure-JS WASM SQLite +driver if the native binary is unavailable, and only drops to the in-memory +driver if WASM also fails to load. Set `OS_DATABASE_DRIVER=memory` to select it +explicitly. ```typescript import { InMemoryDriver } from '@objectstack/driver-memory'; @@ -365,8 +366,37 @@ import { InMemoryDriver } from '@objectstack/driver-memory'; new InMemoryDriver(); ``` +### Persistence is opt-in + +A bare `new InMemoryDriver()` persists nothing. Durability is requested +explicitly: + +```typescript +new InMemoryDriver({ persistence: 'file' }) // Node.js — .objectstack/data/memory-driver.json +new InMemoryDriver({ persistence: 'local' }) // browser — localStorage +new InMemoryDriver({ persistence: 'auto' }) // pick per environment (file / localStorage / off) +``` + +`'auto'` chooses localStorage in a browser, a file under Node.js, and **disables** +persistence in serverless/edge runtimes (Vercel, Lambda, Netlify, Cloud Run, Deno +Deploy) where a local write would be silently discarded — supply a custom adapter +there instead. + + +Before v17 the default was `'auto'`, which on Node.js meant **file** — so a bare +`new InMemoryDriver()` silently wrote `.objectstack/data/memory-driver.json` into +the working directory and reloaded it on the next boot. If you relied on that, +pass `persistence: 'auto'` (or `'file'`) explicitly. See +[#4065](https://github.com/objectstack-ai/objectstack/issues/4065). + + -Use the memory driver for unit tests. It requires no setup and runs instantly. +For tests, prefer in-memory **SQLite** — `SqlDriver` with +`connection: { filename: ':memory:' }`, or `SqliteWasmDriver({ filename: ':memory:' })` +when you want no native build. Both give the SQL semantics production runs on; +mingo does not enforce primary keys, uniqueness, `NOT NULL` or column types, so a +green run against the memory driver is weaker evidence than it looks. The +framework's own dogfood gate boots on WASM SQLite at `:memory:` for this reason. ## Local Environment Runtime diff --git a/content/docs/getting-started/your-first-project.mdx b/content/docs/getting-started/your-first-project.mdx index 889b07c592..03593e236e 100644 --- a/content/docs/getting-started/your-first-project.mdx +++ b/content/docs/getting-started/your-first-project.mdx @@ -164,12 +164,16 @@ framework surface you install: | Package | What it does | |:---|:---| | `@objectstack/spec` | The protocol: `defineStack`, `ObjectSchema`, `Field.*` builders, all Zod schemas. Your metadata imports only this. | -| `@objectstack/runtime` | The kernel that interprets metadata at runtime. | -| `@objectstack/driver-memory` | In-memory database driver for development. Swap for SQLite / Postgres / MongoDB in production — [no code changes](/docs/data-modeling/drivers). | +| `@objectstack/runtime` | The kernel that interprets metadata at runtime. Brings the database drivers with it — you do not install one separately. | | `@objectstack/plugin-hono-server` | The HTTP server that mounts the generated REST API. | | `@objectstack/connector-rest` · `-openapi` · `-mcp` | The three generic connector executors — register the `rest` / `openapi` / `mcp` provider factories so declarative `connectors:` entries materialize ([ADR-0097](/docs/automation/flows)). | | `@objectstack/cli` *(dev)* | The `os` / `objectstack` CLI: `dev`, `validate`, `build`, `start`. | +No driver package is listed because none is installed directly: `@objectstack/runtime` +depends on `driver-sql`, `driver-sqlite-wasm` and `driver-memory`, and the CLI carries +them too. `objectstack dev` resolves **SQLite** by default; point `OS_DATABASE_URL` at +Postgres / MySQL / MongoDB for production — [no code changes](/docs/data-modeling/drivers). + ## 3. Run it ```bash diff --git a/packages/cli/src/commands/init.ts b/packages/cli/src/commands/init.ts index 0298fe6352..7c0c8669eb 100644 --- a/packages/cli/src/commands/init.ts +++ b/packages/cli/src/commands/init.ts @@ -103,11 +103,17 @@ export const TEMPLATES: Record Date: Thu, 30 Jul 2026 09:43:27 +0000 Subject: [PATCH 4/5] docs(driver-memory): stop calling this production-ready; say what it does not enforce (#4065) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The class docstring opened with "A production-ready implementation of the ObjectStack Driver Protocol". The driver stores no constraints of any kind: `create()` is a `table.push()` and `syncSchema()` only allocates an array and indexes temporal fields, so there is no primary key, no uniqueness, no NOT NULL, no foreign key and no column typing. `bulkCreate` lands two rows with the same id where a SQL driver raises a constraint violation, and a read returns both — the second finding in #4065, and the mechanism behind the 2N row growth there. Per Prime Directive #10 the options for `declared ≠ enforced` are implement it, trim the claim, or file it. With this driver moving to maintenance-only — kept for the last-resort rung of the dev step-down, browser/edge runtimes with no SQLite build, and the read-coercion parity gate — the claim is what goes. The docstring now states the missing constraints plainly, names the driver as a WEAK oracle for tests, and points at in-memory SQLite instead. No behaviour change; 197 driver-memory tests unchanged and green. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01L5q3BZMpdiAueHf2emhDY9 --- .../memory-driver-opt-in-persistence.md | 10 +++++++ .../driver-memory/src/memory-driver.ts | 30 +++++++++++++++---- 2 files changed, 35 insertions(+), 5 deletions(-) diff --git a/.changeset/memory-driver-opt-in-persistence.md b/.changeset/memory-driver-opt-in-persistence.md index f7b0bb028a..23862ded3a 100644 --- a/.changeset/memory-driver-opt-in-persistence.md +++ b/.changeset/memory-driver-opt-in-persistence.md @@ -65,3 +65,13 @@ genuinely wanted durability now state it, rather than inheriting it: `DevPlugin`'s driver is now explicitly `persistence: false`, matching the cache, queue, job, i18n, storage and search stubs it ships beside — it was the one piece of that stack that quietly outlived the process. + +**One claim trimmed, no behaviour attached.** The class docstring called this a +"production-ready implementation of the ObjectStack Driver Protocol". It stores +no constraints at all — `create()` is a `table.push()` and `syncSchema()` only +allocates an array — so there is no primary key, uniqueness, `NOT NULL`, foreign +key or column typing, and `bulkCreate` lands duplicate ids where a SQL driver +raises a violation (the second finding in #4065). The docstring now says so, and +points test authors at in-memory SQLite. Per Prime Directive #10 the fix for +`declared ≠ enforced` is to implement it, trim the claim, or file it; with this +driver moving to maintenance-only the claim is what goes. diff --git a/packages/plugins/driver-memory/src/memory-driver.ts b/packages/plugins/driver-memory/src/memory-driver.ts index bae76bde6e..cdd52bccb7 100644 --- a/packages/plugins/driver-memory/src/memory-driver.ts +++ b/packages/plugins/driver-memory/src/memory-driver.ts @@ -80,10 +80,10 @@ interface MemoryTransaction { /** * In-Memory Driver for ObjectStack - * - * A production-ready implementation of the ObjectStack Driver Protocol - * powered by Mingo — a MongoDB-compatible query and aggregation engine. - * + * + * An implementation of the ObjectStack Driver Protocol powered by Mingo — a + * MongoDB-compatible query and aggregation engine. + * * Features: * - MongoDB-compatible query engine (Mingo) for filtering, projection, aggregation * - Full CRUD and bulk operations @@ -91,7 +91,27 @@ interface MemoryTransaction { * - Snapshot-based transactions (begin/commit/rollback) * - Field projection and distinct values * - Strict mode and initial data loading - * + * + * ## What this driver does NOT enforce + * + * It stores no constraints of any kind. {@link create} is a `table.push()` and + * {@link syncSchema} only allocates an array and indexes temporal fields, so + * there is no primary key, no uniqueness, no `NOT NULL`, no foreign key and no + * column typing. `bulkCreate` will happily land two rows with the same `id` + * where a SQL driver raises a constraint violation, and a read returns both. + * + * That makes it a WEAK oracle: code green against this driver can still be + * broken against the SQL engines production runs on. Prefer in-memory SQLite + * for tests — `SqlDriver` with `connection: { filename: ':memory:' }`, or + * `SqliteWasmDriver({ filename: ':memory:' })` where no native build is wanted. + * This driver's remaining roles are the last-resort rung of the dev step-down + * (native better-sqlite3 → WASM SQLite → here), browser/edge runtimes where no + * SQLite build is available, and the cross-driver read-coercion parity gate. + * + * The docstring said "production-ready" until #4065; the constraints above were + * true then too, and saying so is the fix (Prime Directive #10 — never advertise + * a capability the runtime does not deliver). + * * Reference: objectql/packages/drivers/memory */ export class InMemoryDriver implements IDataDriver { From e7d31c6d91f1350bf69ee49d94d39e2fc9767508 Mon Sep 17 00:00:00 2001 From: Claude Date: Thu, 30 Jul 2026 11:06:49 +0000 Subject: [PATCH 5/5] fix(runtime,cli): thread projectRoot to the metadata repo; stop compiling tests into dist (#4065) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Two defects behind the last stray `.objectstack/` directory — the one under `packages/cli/`. Neither turned out to be cosmetic. 1. `projectRoot` only got half the stack. `createStandaloneStack`'s `projectRoot` is documented as scoping a boot's on-disk state to the project folder, and it did redirect the default sqlite database — but it was never passed to MetadataPlugin, whose FileSystemRepository kept rooting at `process.cwd()`. One "project root" therefore meant two directories: a boot pointed at project A wrote `A/.objectstack/data/` and `/.objectstack/metadata/`. It now forwards `rootDir`, and `bootSchemaStack` takes a `projectRoot` to pass down (default `process.cwd()` — right for every real `os migrate`, which runs from the project dir). The two migrate integration suites now scope their boots to the tempdir fixture they already build. 2. The CLI compiled its own tests into dist — and vitest ran them. `tsconfig.build.json` included all of `src` with no exclude, so every `src/**/*.test.ts` was emitted as `dist/**/*.test.js`. `files: ["dist"]` published them, and — this package has no vitest config — `vitest run` collected the compiled copies alongside the sources: 81 test files / 849 tests where the sources hold 58 / 581. That silently defeats edits. A fix to a source test appeared not to work because the run was still executing the pre-fix compiled duplicate; that is how this residue survived a correct fix long enough to look like a different bug. It also lets a source test be edited to pass while its stale twin keeps asserting the old behaviour, with neither obviously wrong. Tests are now excluded from the build. No other package is affected — the rest build with tsup, which emits only declared entry points. Verified by scanning every packages/*/dist for `*.test.js`; the CLI was the only hit. Verified: full forced suite INCLUDING dogfood green (132/132 tasks), CLI suite run twice back-to-back leaves no `.objectstack` behind, lint clean. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01L5q3BZMpdiAueHf2emhDY9 --- .../cli-stale-dist-tests-and-project-root.md | 42 +++++++++++++++++++ ...a-migrate.deferred-ddl.integration.test.ts | 8 ++-- .../utils/schema-migrate.integration.test.ts | 4 +- packages/cli/src/utils/schema-migrate.ts | 14 ++++++- packages/cli/tsconfig.build.json | 11 ++++- packages/runtime/src/standalone-stack.ts | 18 +++++++- 6 files changed, 87 insertions(+), 10 deletions(-) create mode 100644 .changeset/cli-stale-dist-tests-and-project-root.md diff --git a/.changeset/cli-stale-dist-tests-and-project-root.md b/.changeset/cli-stale-dist-tests-and-project-root.md new file mode 100644 index 0000000000..1fd12e0b5e --- /dev/null +++ b/.changeset/cli-stale-dist-tests-and-project-root.md @@ -0,0 +1,42 @@ +--- +"@objectstack/runtime": patch +"@objectstack/cli": patch +--- + +fix(runtime,cli): `projectRoot` reaches the metadata repository; stop compiling tests into the CLI's dist (#4065) + +Two defects behind the last of #4065's stray `.objectstack/` directories — the +one under `packages/cli/`. Neither is cosmetic. + +**1. `projectRoot` only got half the stack.** `createStandaloneStack`'s +`projectRoot` is documented as scoping a boot's on-disk state to the project +folder "so different examples / apps don't share a single database by accident", +and it did redirect the default sqlite database. But it was never passed to +`MetadataPlugin`, whose `FileSystemRepository` kept rooting at `process.cwd()`. +So one "project root" meant two different directories: a boot pointed at project +A wrote `A/.objectstack/data/` and `/.objectstack/metadata/`. It now +forwards `rootDir`, and `bootSchemaStack` accepts a `projectRoot` to pass down +(defaulting to `process.cwd()`, which is right for every real `os migrate` — the +CLI runs from the project directory). The two migrate integration suites, which +build a fixture project in a tempdir, now scope their boots to it. + +**2. The CLI compiled its own tests into `dist/` — and vitest ran them.** +`tsconfig.build.json` included all of `src` with no exclude, so every +`src/**/*.test.ts` was emitted as `dist/**/*.test.js`. Two consequences: + +- `files: ["dist"]` **published** them. +- This package has no vitest config, so `vitest run` collected the compiled + copies alongside the sources: **81 test files and 849 tests where the sources + hold 58 and 581**. Every `src/` test also ran as a stale `dist/` twin built + from whatever the source said at the last build. + +That is not just noise — it silently defeats edits. A fix to a source test +appeared not to work, because the run was still executing the pre-fix compiled +duplicate; that is exactly how the `.objectstack` residue survived a correct +fix long enough to look like a different bug. It also means a source test could +be edited to pass while its stale twin kept asserting the old behaviour, and +neither would be obviously wrong. Test files are now excluded from the build. + +No other package is affected: the rest build with `tsup`, which emits only +declared entry points. Verified by scanning every `packages/*/dist` for +`*.test.js` — the CLI was the only hit. diff --git a/packages/cli/src/utils/schema-migrate.deferred-ddl.integration.test.ts b/packages/cli/src/utils/schema-migrate.deferred-ddl.integration.test.ts index 8ca50501a7..404534112f 100644 --- a/packages/cli/src/utils/schema-migrate.deferred-ddl.integration.test.ts +++ b/packages/cli/src/utils/schema-migrate.deferred-ddl.integration.test.ts @@ -100,7 +100,7 @@ describe('bootSchemaStack({ deferSchemaDdl }) — the boot writes nothing (#3917 expect(before.tables).toEqual(['defer_widget']); expect(before.widgetColumns).toEqual(['created_at', 'id', 'name', 'updated_at']); - const stack = await bootSchemaStack({ databaseUrl: `file:${dbFile}`, deferSchemaDdl: true }); + const stack = await bootSchemaStack({ databaseUrl: `file:${dbFile}`, deferSchemaDdl: true, projectRoot: dir }); try { const after = await inspect(); // The core assertion: boot created no table, added no column, wrote no @@ -128,7 +128,7 @@ describe('bootSchemaStack({ deferSchemaDdl }) — the boot writes nothing (#3917 }, 60_000); it('flushSchemaDdl performs exactly the work that was reported', async () => { - const stack = await bootSchemaStack({ databaseUrl: `file:${dbFile}`, deferSchemaDdl: true }); + const stack = await bootSchemaStack({ databaseUrl: `file:${dbFile}`, deferSchemaDdl: true, projectRoot: dir }); try { const pending = stack.pendingSchemaWork; const performed = await stack.flushSchemaDdl(); @@ -142,7 +142,7 @@ describe('bootSchemaStack({ deferSchemaDdl }) — the boot writes nothing (#3917 // A second boot finds nothing left to do. await stack.shutdown(); - const again = await bootSchemaStack({ databaseUrl: `file:${dbFile}`, deferSchemaDdl: true }); + const again = await bootSchemaStack({ databaseUrl: `file:${dbFile}`, deferSchemaDdl: true, projectRoot: dir }); try { expect(again.pendingSchemaWork).toEqual([]); } finally { @@ -155,7 +155,7 @@ describe('bootSchemaStack({ deferSchemaDdl }) — the boot writes nothing (#3917 }, 60_000); it('without the flag, the boot syncs as it always did', async () => { - const stack = await bootSchemaStack({ databaseUrl: `file:${dbFile}` }); + const stack = await bootSchemaStack({ databaseUrl: `file:${dbFile}`, projectRoot: dir }); try { expect(stack.pendingSchemaWork).toEqual([]); const after = await inspect(); diff --git a/packages/cli/src/utils/schema-migrate.integration.test.ts b/packages/cli/src/utils/schema-migrate.integration.test.ts index 5b28462f0f..8873fa61d9 100644 --- a/packages/cli/src/utils/schema-migrate.integration.test.ts +++ b/packages/cli/src/utils/schema-migrate.integration.test.ts @@ -78,7 +78,7 @@ describe('bootSchemaStack + migrate engine (integration)', () => { }); it('detects the NOT NULL + legacy-unique drift, applies both, and self-verifies in-sync', async () => { - const stack = await bootSchemaStack({ databaseUrl: `file:${dbFile}` }); + const stack = await bootSchemaStack({ databaseUrl: `file:${dbFile}`, projectRoot: dir }); try { expect(stack.driver).toBeTruthy(); expect(stack.managedTableCount).toBeGreaterThan(0); @@ -198,7 +198,7 @@ describe('bootSchemaStack — dev-provisioned __search companions are not orphan }); it('the migrate boot provisions the companion in metadata, so plan reports no __search drift', async () => { - const stack = await bootSchemaStack({ databaseUrl: `file:${dbFile}` }); + const stack = await bootSchemaStack({ databaseUrl: `file:${dbFile}`, projectRoot: dir }); try { expect(stack.driver).toBeTruthy(); diff --git a/packages/cli/src/utils/schema-migrate.ts b/packages/cli/src/utils/schema-migrate.ts index 4d5ab18487..4c65c8090b 100644 --- a/packages/cli/src/utils/schema-migrate.ts +++ b/packages/cli/src/utils/schema-migrate.ts @@ -157,13 +157,25 @@ export async function bootSchemaStack( * tables to exist. */ deferSchemaDdl?: boolean; + /** + * Project root the booted stack scopes its on-disk state to — the default + * sqlite database and the metadata FileSystemRepository + * (`/.objectstack/…`). Defaults to `process.cwd()`, which is + * correct for every real `os migrate` invocation: the CLI runs from the + * project directory. + * + * Tests that assemble a fixture project in a tempdir must pass it, or the + * boot scopes its database to the tempdir while writing metadata into + * whatever directory the test runner happens to be standing in (#4065). + */ + projectRoot?: string; } = {}, ): Promise { const { createStandaloneStack, Runtime } = await import('@objectstack/runtime'); const defer = opts.deferSchemaDdl === true; const stack = await createStandaloneStack({ - projectRoot: process.cwd(), + projectRoot: opts.projectRoot ?? process.cwd(), ...(opts.databaseUrl ? { databaseUrl: opts.databaseUrl } : {}), ...(defer ? { skipSeedData: true } : {}), }); diff --git a/packages/cli/tsconfig.build.json b/packages/cli/tsconfig.build.json index f21a6d1a4c..a628977864 100644 --- a/packages/cli/tsconfig.build.json +++ b/packages/cli/tsconfig.build.json @@ -16,5 +16,14 @@ "sourceMap": true, "types": ["node"] }, - "include": ["src"] + "include": ["src"], + // Tests are not part of the shipped CLI. Compiling them had two costs beyond + // tarball weight: `files: ["dist"]` published them, and — because this package + // has no vitest config — `vitest run` COLLECTED the compiled copies too, so + // every `src/**/*.test.ts` also ran as a stale `dist/**/*.test.js` built from + // whatever the source said at the last build. That is how #4065's + // `packages/cli/.objectstack` residue survived a fix to the source: the run + // was still executing the pre-fix compiled duplicate. It also means a source + // test could be edited to pass while its stale twin asserted the old thing. + "exclude": ["src/**/*.test.ts", "src/**/*.spec.ts", "src/**/__tests__/**"] } diff --git a/packages/runtime/src/standalone-stack.ts b/packages/runtime/src/standalone-stack.ts index ee2ac06d27..84529b980e 100644 --- a/packages/runtime/src/standalone-stack.ts +++ b/packages/runtime/src/standalone-stack.ts @@ -68,10 +68,16 @@ export const StandaloneStackConfigSchema = z.object({ * Project root directory. When set (typically by the CLI after locating * `objectstack.config.ts`), the default sqlite database is placed under * `/.objectstack/data/standalone.db` instead of the global - * `~/.objectstack/data/standalone.db`. This keeps per-project data - * scoped to the project folder so different examples / apps don't + * `~/.objectstack/data/standalone.db`, and the metadata FileSystemRepository + * roots at `/.objectstack/metadata`. This keeps per-project + * data scoped to the project folder so different examples / apps don't * share a single database by accident. * + * Both halves matter: until #4065 only the database honoured it while the + * metadata repository still used `process.cwd()`, so a boot whose + * projectRoot pointed elsewhere silently wrote `.objectstack/metadata/` + * into the current directory. + * * Explicit `databaseUrl` / `OS_DATABASE_URL` / `OS_HOME` still take * precedence over this default. */ @@ -326,6 +332,14 @@ export async function createStandaloneStack(config?: StandaloneStackConfig): Pro artifactWatch: process.env.NODE_ENV !== 'production', environmentId, artifactSource: { mode: 'local-file', path: artifactPath }, + // `projectRoot` has to reach the metadata repository too. It already + // redirects the default sqlite database; without this line the + // FileSystemRepository still rooted at `process.cwd()`, so one + // "project root" meant two different directories and a boot pointed + // at some other project wrote `.objectstack/metadata/` into whatever + // directory the process happened to be standing in (#4065). Omitted + // when unset so the plugin keeps its own cwd default. + ...(cfg.projectRoot ? { rootDir: cfg.projectRoot } : {}), }), new ObjectQLPlugin({ environmentId }), ];