diff --git a/docs/guide/database/d1.md b/docs/guide/database/d1.md index cb3836df1..dad23a353 100644 --- a/docs/guide/database/d1.md +++ b/docs/guide/database/d1.md @@ -41,9 +41,47 @@ await db.batch([ ## Read replication -Void enables [D1 read replication](https://developers.cloudflare.com/d1/best-practices/read-replication/) when it provisions a database. To enable it on an existing database, run `void deploy --platform cloudflare --provision`. Queries through `db` from `void/db` use a D1 session for each Void request, so reads can run near the request and later queries in that request see at least the same data. Framework handlers without Void's request scope start each database operation on the primary. `createDb` starts a replica-capable session for the binding you pass it. +::: warning Previously deployed databases +Void **0.21.3** introduced automatic D1 read replication. If you deployed with that version or a later version before replication became opt-in, your database may already have replicas enabled. Replicas can serve stale data between requests. -Replicas can lag between requests. Authentication checks use the primary so a lagging replica can't accept a revoked session. For other reads that need the latest primary data, create a client inside the request: +Unless your app intentionally uses replicas with the consistency handling described below, we recommend upgrading to a Void version that supports `database.readReplication`, setting it to `false` in `void.config.ts`, and running `void deploy`: + +```ts +import { defineConfig } from 'void/config'; + +export default defineConfig({ + database: { readReplication: false }, +}); +``` + +If you self-host the Void platform, upgrade it as well before deploying. Upgrading alone or omitting the option does **not** disable existing replication. +::: + +[D1 read replication](https://developers.cloudflare.com/d1/best-practices/read-replication/) is opt-in. Enable it in `void.config.ts`, then run `void deploy`: + +```ts +import { defineConfig } from 'void/config'; + +export default defineConfig({ + database: { readReplication: true }, +}); +``` + +This enables replication for new and existing application D1 databases on both direct Cloudflare and Void platform deployments. Set `readReplication: false` and deploy again to disable replication. + +When you omit `readReplication`, deployment leaves the database's replication setting unchanged. New databases default to replication off. Removing the option from a database that already has replication enabled keeps it enabled. + +Queries through `db` from `void/db` use a D1 session for each Void request. With replication enabled, reads can run near the request and later queries in that request see at least the same data. Framework handlers without Void's request scope start each database operation on the primary. `createDb` starts a replica-capable session for the binding you pass it. + +::: warning +Replicas can lag between requests. [Carry a bookmark](#cross-request-bookmarks) to preserve previously observed database state, or use a [primary-first session](#reading-from-the-primary) when you need the latest primary data. +::: + +For raw D1 calls or another query library, use `withSession()` to read from replicas. A bare D1 binding reads from the primary. + +### Avoid stale reads with a primary-first session {#reading-from-the-primary} + +Authentication checks use the primary so a lagging replica can't accept a revoked session. For other reads that need the latest primary data, create a client inside the request: ```ts import { users } from '@schema'; @@ -59,7 +97,249 @@ export const GET = defineHandler(async () => { With `session: 'first-primary'`, the first query reads from the primary and later queries see at least the same data. The default, `'first-unconstrained'`, allows the first query to use a replica. -For raw D1 calls or another query library, use `withSession()` to read from replicas. A bare D1 binding reads from the primary. +### See your writes in subsequent requests {#cross-request-bookmarks} + +To make a read after a save see that save in a later request, carry a [D1 bookmark](https://developers.cloudflare.com/d1/best-practices/read-replication/#start-a-session-from-previous-context-bookmark) between requests. A bookmark allows D1 to use a replica that has caught up to at least the previously observed database state. + +Keep saved bookmarks scoped to the database that issued them. Use separate cookie names or browser-storage keys for different databases and environments, and clear saved bookmarks when switching to a different database. Clearing a bookmark starts a new consistency chain; subsequent requests no longer carry the previous bookmark's guarantee. Treat bookmarks as opaque strings rather than parsing or constructing them. + +::: warning +Bookmarks constrain database queries, not cached responses. A cached response can predate your save without running a query. Disable ISR for participating pages (`export const revalidate = 0` in the page's `.server.ts` file) and bypass any existing response caches for these routes. Setting response headers alone does not bypass an ISR cache hit. +::: + +Use middleware to set the default `db` session before your handlers or Pages loaders run, and return its bookmark after their queries. For a browser app, a cookie carries it through redirects and navigation: + +This example opts `/profile` and its subpaths into bookmark handling and `Cache-Control: private, no-store`, preventing browser and shared response-cache storage. Change the path to cover your participating reads, saves, and redirect destinations. Routes outside this scope keep their existing cache policies. + +Local server-side requests through `void/client` share that session. If middleware runs again, repeating an already satisfied session requirement preserves the session and its database progress. For an independent primary-first read or a newer bookmark after using `db`, use `createDb` instead. + +Await consistency-critical database work before returning from your handler or loader. Deferred or background queries that finish after middleware captures the response bookmark are not guaranteed to be reflected in it. + +::: warning +Concurrent responses can overwrite this cookie with an older bookmark. The [header-based recipe](#api-requests-with-concurrent-clients) retains the newest bookmark across concurrent API responses. Clearing or expiring bookmark storage starts a new bookmark chain. +::: + +::: note +To use cookies and API headers together, see [Supporting both transports](#supporting-both-transports). +::: + +```ts +// middleware/01.database.ts +import { getCookie, setCookie } from 'hono/cookie'; +import { defineMiddleware } from 'void'; +import { getBookmark, setSession } from 'void/db'; + +export default defineMiddleware(async (c, next) => { + const path = c.req.path; + if (path !== '/profile' && !path.startsWith('/profile/')) { + await next(); + return; + } + + const incoming = getCookie(c, 'db-bookmark'); + setSession(incoming === undefined ? 'first-unconstrained' : { bookmark: incoming }); + await next(); + + const bookmark = getBookmark(); + if (bookmark) { + setCookie(c, 'db-bookmark', bookmark, { + httpOnly: true, + secure: new URL(c.req.url).protocol === 'https:', + sameSite: 'Lax', + path: '/', + }); + } + c.header('Cache-Control', 'private, no-store'); +}); +``` + +#### API requests with concurrent clients + +For browser API requests, exchange an `X-DB-Bookmark` header and retain the newest bookmark. [D1 bookmarks are lexicographically sortable](https://developers.cloudflare.com/d1/reference/time-travel/#bookmarks), so an older response must not replace a newer bookmark from the same database. + +For requests made through the browser fetch wrapper below, exchange bookmarks using this middleware. It opts only `/api/profile` and its subpaths into bookmark handling and uncached responses; adjust the path to match your participating API routes: + +::: warning +This recipe carries bookmarks only through the fetch wrapper below, not ordinary navigation, form submissions, or Pages requests. It does not share bookmarks with the cookie-based recipe. +::: + +::: note +To use cookies and API headers together, see [Supporting both transports](#supporting-both-transports). +::: + +```ts +// middleware/01.database.ts +import { defineMiddleware } from 'void'; +import { getBookmark, setSession } from 'void/db'; + +export default defineMiddleware(async (c, next) => { + const path = c.req.path; + if (path !== '/api/profile' && !path.startsWith('/api/profile/')) { + await next(); + return; + } + + const incoming = c.req.header('X-DB-Bookmark'); + setSession(incoming === undefined ? 'first-unconstrained' : { bookmark: incoming }); + await next(); + const bookmark = getBookmark(); + if (bookmark) { + c.header('X-DB-Bookmark', bookmark); + } + c.header('Cache-Control', 'private, no-store'); +}); +``` + +On the browser, send the saved bookmark and atomically retain the newest returned value using [Web Locks](https://developer.mozilla.org/en-US/docs/Web/API/Web_Locks_API). This requires HTTPS (or localhost), browser storage, and Web Locks support. Use a scope unique to the database and signed-in account; same-origin tabs with the same scope share progress. + +```ts +// src/bookmark-fetch.ts +export function createBookmarkFetch(scope: string) { + const key = `d1-bookmark:${scope}`; + + return async function bookmarkFetch(path: string, init: RequestInit = {}): Promise { + const url = new URL(path, location.origin); + if (url.origin !== location.origin || !url.pathname.startsWith('/api/')) { + throw new Error('bookmarkFetch requires a same-origin /api/ URL.'); + } + + const bookmark = localStorage.getItem(key); + const headers = new Headers(init.headers); + headers.delete('X-DB-Bookmark'); + if (bookmark) headers.set('X-DB-Bookmark', bookmark); + + const response = await fetch(url, { + ...init, + headers, + credentials: 'same-origin', + cache: 'no-store', + redirect: 'error', + }); + + const returned = response.headers.get('X-DB-Bookmark'); + if (returned) { + await navigator.locks.request(key, () => { + const current = localStorage.getItem(key); + if (current === null || returned > current) { + localStorage.setItem(key, returned); + } + }); + } + + return response; + }; +} +``` + +Use the wrapper for each participating API request. For example, with an existing profile API: + +```ts +import { createBookmarkFetch } from './bookmark-fetch'; + +export async function saveAndReadProfile(accountId: string, name: string) { + const api = createBookmarkFetch(`app-db:${accountId}`); + const saved = await api('/api/profile', { + method: 'POST', + headers: { 'Content-Type': 'text/plain' }, + body: name, + }); + if (!saved.ok) throw new Error(`Save failed: ${saved.status}`); + + // Sends at least the completed save's bookmark. + const response = await api('/api/profile'); + if (!response.ok) throw new Error(`Read failed: ${response.status}`); + return response.json(); +} +``` + +The wrapper saves the newest bookmark before resolving, so subsequent requests inherit its progress even if concurrent responses arrive out of order. + +Use API endpoints that return data, not redirects. + +#### Supporting both transports + +Use this single middleware with the fetch wrapper above to support API requests alongside browser navigation and forms. It selects the newest incoming bookmark for the same database and returns it through both transports. + +Only `/profile`, `/api/profile`, and their subpaths participate and receive `Cache-Control: private, no-store`. Adjust this list for your app, disable ISR on participating pages, and leave unrelated routes' cache policies unchanged. + +::: warning +The fetch wrapper retains the newest bookmark across concurrent responses, but cookies can still be overwritten out of order. For a guaranteed fresh read on a redirect's destination page, use a primary-first session. +::: + +```ts +// middleware/01.database.ts +import { getCookie, setCookie } from 'hono/cookie'; +import { defineMiddleware } from 'void'; +import { getBookmark, setSession } from 'void/db'; + +export default defineMiddleware(async (c, next) => { + const path = c.req.path; + const participates = ['/profile', '/api/profile'].some( + (base) => path === base || path.startsWith(`${base}/`), + ); + if (!participates) { + await next(); + return; + } + + const incoming: string[] = []; + + // Cookie input: navigation, forms, and same-origin fetch requests. + // Remove for header-only use. + const cookieBookmark = getCookie(c, 'db-bookmark'); + if (cookieBookmark) incoming.push(cookieBookmark); + + // Header input: the fetch wrapper's saved progress. + // Remove for cookie-only use. + const headerBookmark = c.req.header('X-DB-Bookmark'); + if (headerBookmark) incoming.push(headerBookmark); + + // Bookmarks from the same database sort oldest to newest. + const newest = incoming.sort().at(-1); + setSession(newest === undefined ? 'first-unconstrained' : { bookmark: newest }); + await next(); + + const bookmark = getBookmark(); + if (bookmark) { + // Cookie output: progress for subsequent navigation and forms. + // Remove for header-only use. + setCookie(c, 'db-bookmark', bookmark, { + httpOnly: true, + secure: new URL(c.req.url).protocol === 'https:', + sameSite: 'Lax', + path: '/', + }); + + // Header output: progress for the fetch wrapper. + // Remove for cookie-only use. + c.header('X-DB-Bookmark', bookmark); + } + + c.header('Cache-Control', 'private, no-store'); +}); +``` + +#### Using `createDb` + +For another D1 binding or a framework-owned endpoint, such as an Astro API route, create a client with the incoming bookmark. After querying, return `getBookmark(client)` in the response header: + +```ts +import { users } from '@schema'; +import { createDb, getBookmark } from 'void/db'; + +export async function readUsers(request: Request, binding: D1Database) { + const incoming = request.headers.get('X-DB-Bookmark'); + const client = createDb(binding, { + session: incoming === null ? 'first-unconstrained' : { bookmark: incoming }, + }); + const result = await client.select().from(users).all(); + + const headers = new Headers({ 'Cache-Control': 'private, no-store' }); + const bookmark = getBookmark(client); + if (bookmark) headers.set('X-DB-Bookmark', bookmark); + return Response.json(result, { headers }); +} +``` ## Alternative Query Layers diff --git a/docs/reference/api.md b/docs/reference/api.md index b6a6a8010..05a09b43d 100644 --- a/docs/reference/api.md +++ b/docs/reference/api.md @@ -58,7 +58,7 @@ Import Void's app-facing APIs from these paths: | `void/schema-pg` | Re-exports [`drizzle-orm/pg-core`](https://orm.drizzle.team/docs/column-types/pg) for PostgreSQL table and column builders | | `void/schema-mysql` | Re-exports `drizzle-orm/mysql-core` for MySQL table and column builders | | `void/database-provider` | `defineDatabaseProvider` and adapter types for `void db connect --provider` | -| `void/db` | `db` (Drizzle D1, PostgreSQL, or MySQL instance for the active dialect), `createDb`, query operators (`eq`, `and`, `or`, `desc`, `like`, `inArray`, etc.) | +| `void/db` | `db` (Drizzle D1, PostgreSQL, or MySQL instance for the active dialect), `createDb`, D1 session helpers (`setSession`, `getBookmark`), query operators (`eq`, `and`, `or`, `desc`, `like`, `inArray`, etc.) | | `void/seed` | `defineSeed`, `SeedContext`, `SeedFn` for programmatic `void db seed` modules | | `void/kv` | `kv`, `createKV`, typed KV client/map types | | `void/storage` | `storage`, `createStorage` for the inferred R2 binding or a specific `R2Bucket` | diff --git a/docs/reference/api/database.md b/docs/reference/api/database.md index 5ad4c163b..3d1859b5a 100644 --- a/docs/reference/api/database.md +++ b/docs/reference/api/database.md @@ -37,7 +37,9 @@ import { createDb } from 'void/db'; const db = createDb(env.MY_OTHER_DB); ``` -For a fresh D1 read, pass `{ session: 'first-primary' }` and create the client inside the request. The default, `'first-unconstrained'`, allows the first query to use a replica. Both keep your schema and query types. See [Read replication](../../guide/database/d1.md#read-replication). +For a fresh D1 read, pass `{ session: 'first-primary' }` and create the client inside the request. The default, `'first-unconstrained'`, allows the first query to use a replica. Both keep your schema and query types. See [Avoid stale reads with a primary-first session](../../guide/database/d1.md#reading-from-the-primary). + +The `session` option also accepts `{ bookmark }` to resume from a D1 bookmark from a previous request. Use `getBookmark(client)` after queries to obtain the bookmark string to return to your client. See [Using `createDb`](../../guide/database/d1.md#using-createdb) for an example. For PostgreSQL and MySQL projects, pass a connection string. @@ -51,9 +53,11 @@ const db = createDb('postgres://user:password@host:5432/app'); ```ts // D1 +type DbSessionStart = 'first-primary' | 'first-unconstrained' | { bookmark: string }; + function createDb( d1: D1Database, - options?: { session?: D1SessionConstraint }, + options?: { session?: DbSessionStart }, ): DrizzleD1Database; // PostgreSQL @@ -63,6 +67,42 @@ function createDb(connectionString: string): NodePgDatabase; function createDb(connectionString: string): MySql2Database; ``` +## `setSession(start)` {#setsession} + +D1 only. Set the starting bookmark or constraint for the default request's `db`. Call it in middleware before the first use of `db`. It does not change explicitly created clients or later requests. + +Pass `'first-primary'`, `'first-unconstrained'`, or `{ bookmark: string }`. Wrap bookmark strings in an object, including strings returned by `getBookmark()`. If no incoming bookmark is available, pass `'first-unconstrained'`. + +Use bookmarks only with the database that issued them. Keep client-side storage database-specific and clear saved bookmarks when switching databases; clearing them starts a new consistency chain. + +The session opens when you read a database property, including extracting a method or accessing `$client`; it does not wait for a query to execute. Awaiting or inspecting the `db` object does not open it. + +Passing `'first-unconstrained'` preserves any bookmark or primary-first requirement already configured, even before the first use of `db`. Other starting values replace the configuration until the session opens. + +After the session opens, repeated calls preserve it when the requirement is already satisfied: the original starting value, `'first-unconstrained'`, or a bookmark no newer than the session's current bookmark. This lets middleware run again during a local server-side fetch without resetting database progress. Repeating `'first-primary'` does not force another primary read. + +A newer bookmark or changing an opened session to `'first-primary'` throws. Configure these before using `db`, or use `createDb(binding, { session })` for an independent session. + +```ts +function setSession(start: DbSessionStart): void; +``` + +## `getBookmark(client?)` {#getbookmark} + +D1 only. Return the default request session's bookmark, or pass a client returned by `createDb()` to read that client's bookmark. Returns `null` when no bookmark is available. Reading the default bookmark does not open a database session. + +Await consistency-critical database work before capturing the bookmark you send to the client. Deferred or background queries that finish after capture are not guaranteed to be reflected in that bookmark. + +```ts +function getBookmark>( + client?: DrizzleD1Database, +): D1SessionBookmark | null; +``` + +`setSession()` and the no-argument `getBookmark()` require Void's request scope. For a framework-owned endpoint, such as an Astro API route, use an [explicit client](../../guide/database/d1.md#using-createdb) instead. + +For cookie and header middleware, a browser fetch wrapper, and guidance on combining both transports, see [See your writes in subsequent requests](../../guide/database/d1.md#cross-request-bookmarks). + ## Query Operators {#query-operators} `void/db` re-exports commonly used [Drizzle operators](https://orm.drizzle.team/docs/operators) so you never need to depend on `drizzle-orm` directly: diff --git a/docs/reference/cli/deploy.md b/docs/reference/cli/deploy.md index 609481a14..ed923e1bf 100644 --- a/docs/reference/cli/deploy.md +++ b/docs/reference/cli/deploy.md @@ -28,18 +28,18 @@ For an existing Void Worker, `void deploy` can link the Worker and migrate its r For Drizzle projects, deploy performs a read-only schema drift check. If a new migration would be generated, deploy stops and tells you to run `void db generate`, review the migration, commit it yourself, and rerun `void deploy`. -| Flag | Purpose | -| ------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `--platform ` | Override the platform stored in `.void/project.json` | -| `--project ` | Target a specific Void project by slug; not supported with `--platform cloudflare` | -| `--dir ` | Deploy a pre-built static directory (skips build) | -| `--spa` | Use SPA mode instead of SSG for static deploys | -| `--skip-build` | Skip the build step; on Cloudflare this is supported for static/SPA/SSG deploys only | -| `--skip-migrations` | Deploy without applying application or authentication SQL migrations; supported on Cloudflare and Void platforms | -| `--require-email` | Fail when email cannot be set up instead of deploying without it; requires `--platform cloudflare` | -| `--provision` | Run resource provisioning again even when every resource exists, for example to enable D1 read replication on an existing database; requires `--platform cloudflare` | -| `--atomic` | Publish an existing Durable Object Worker directly; readiness is checked after traffic changes | -| `--debug` | Mirror the structured deploy log to stderr (also written to `~/.void/logs/`) | +| Flag | Purpose | +| ------------------------------- | ---------------------------------------------------------------------------------------------------------------- | +| `--platform ` | Override the platform stored in `.void/project.json` | +| `--project ` | Target a specific Void project by slug; not supported with `--platform cloudflare` | +| `--dir ` | Deploy a pre-built static directory (skips build) | +| `--spa` | Use SPA mode instead of SSG for static deploys | +| `--skip-build` | Skip the build step; on Cloudflare this is supported for static/SPA/SSG deploys only | +| `--skip-migrations` | Deploy without applying application or authentication SQL migrations; supported on Cloudflare and Void platforms | +| `--require-email` | Fail when email cannot be set up instead of deploying without it; requires `--platform cloudflare` | +| `--provision` | Run resource provisioning again even when every resource exists; requires `--platform cloudflare` | +| `--atomic` | Publish an existing Durable Object Worker directly; readiness is checked after traffic changes | +| `--debug` | Mirror the structured deploy log to stderr (also written to `~/.void/logs/`) | `--atomic` applies to one deployment. For an existing Durable Object Worker that consistently cannot stage, set `deploy: { cloudflare: { mode: 'atomic' } }` in `void.config.ts` so plain `void deploy` uses atomic publication. The default is `staged`. diff --git a/docs/reference/config.md b/docs/reference/config.md index 9eedc55f6..2b3c012ce 100644 --- a/docs/reference/config.md +++ b/docs/reference/config.md @@ -152,10 +152,11 @@ Use an object to also set how Drizzle names columns declared without a name: } ``` -| Key | Values | Description | -| --------- | --------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `dialect` | `"sqlite"` (default), `"pg"`, `"mysql"` | Database backend. `"sqlite"` is D1 | -| `casing` | `"snake_case"`, `"camelCase"` | [Drizzle casing](https://orm.drizzle.team/docs/sql-schema-declaration#camel-and-snake-casing) for `void/db` queries and `void db` commands. Omit to use each key as its column name | +| Key | Values | Description | +| ----------------- | --------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | +| `dialect` | `"sqlite"` (default), `"pg"`, `"mysql"` | Database backend. `"sqlite"` is D1 | +| `casing` | `"snake_case"`, `"camelCase"` | [Drizzle casing](https://orm.drizzle.team/docs/sql-schema-declaration#camel-and-snake-casing) for `void/db` queries and `void db` commands. Omit to use each key as its column name | +| `readReplication` | `boolean` (optional) | D1 only. Enable (`true`) or disable (`false`) [read replication](../guide/database/d1.md#read-replication) on deployment. Omit to preserve existing settings; new databases default to off | When using an external database: diff --git a/packages/deploy-core/src/index.ts b/packages/deploy-core/src/index.ts index 971c8245d..ffd100644 100644 --- a/packages/deploy-core/src/index.ts +++ b/packages/deploy-core/src/index.ts @@ -247,6 +247,8 @@ export type DurableObjectManifestMeta = { export type DeployManifest = { version: number; + /** D1 provisioning preferences. Omitted preserves existing replication; false disables it. */ + database?: { readReplication?: boolean }; /** R2 creation preferences. Requires manifest version 5. */ storage?: StorageConfig; assetManifestVersion?: 1 | 2; diff --git a/packages/deploy-core/src/resource-plan.ts b/packages/deploy-core/src/resource-plan.ts index 58b65bfb8..cdbc0427e 100644 --- a/packages/deploy-core/src/resource-plan.ts +++ b/packages/deploy-core/src/resource-plan.ts @@ -1,7 +1,7 @@ import type { DeployManifest, QueueMeta } from './index.ts'; export type DeployResource = - | { type: 'd1'; bindingName: string } + | { type: 'd1'; bindingName: string; readReplication?: boolean } | { type: 'kv'; bindingName: string } | { type: 'r2'; bindingName: string } | { type: 'queue'; bindingName: string; resourceName: string; consumer: QueueMeta } @@ -28,7 +28,13 @@ export function createDeployResourcePlan(manifest: DeployManifest): DeployResour const resources: Array = []; for (const bindingName of manifest.bindings.d1 ?? []) { - resources.push({ type: 'd1', bindingName }); + resources.push({ + type: 'd1', + bindingName, + ...(manifest.database?.readReplication !== undefined && { + readReplication: manifest.database.readReplication, + }), + }); } for (const bindingName of manifest.bindings.kv ?? []) { resources.push({ type: 'kv', bindingName }); diff --git a/packages/deploy-core/src/validate.ts b/packages/deploy-core/src/validate.ts index e630d15a6..da338a20c 100644 --- a/packages/deploy-core/src/validate.ts +++ b/packages/deploy-core/src/validate.ts @@ -156,6 +156,19 @@ export function validateManifestShape(manifest: unknown): string | null { return 'invalid manifest shape: bindings must be an object'; } const bindings = manifest.bindings; + if (manifest.database !== undefined) { + if ( + !isRecord(manifest.database) || + Object.keys(manifest.database).some((key) => key !== 'readReplication') || + (manifest.database.readReplication !== undefined && + typeof manifest.database.readReplication !== 'boolean') + ) { + return 'invalid manifest shape: database.readReplication must be a boolean'; + } + if (!Array.isArray(bindings.d1) || bindings.d1.length === 0) { + return 'invalid manifest shape: database requires a D1 binding'; + } + } for (const key of ['d1', 'kv', 'r2', 'hyperdrive'] as const) { const bindingValue = bindings[key]; if (bindingValue === undefined) { diff --git a/packages/deploy-core/test/unit/resource-plan.test.ts b/packages/deploy-core/test/unit/resource-plan.test.ts index 935763b15..31a620655 100644 --- a/packages/deploy-core/test/unit/resource-plan.test.ts +++ b/packages/deploy-core/test/unit/resource-plan.test.ts @@ -2,6 +2,23 @@ import { describe, expect, it } from 'vite-plus/test'; import { createDeployResourcePlan, type DeployManifest } from '../../src/index.ts'; describe('createDeployResourcePlan', () => { + it.each([undefined, false, true])( + 'carries explicit D1 replication setting %s', + (readReplication) => { + const plan = createDeployResourcePlan({ + version: 3, + bindings: { d1: ['APP_DB'] }, + database: { readReplication }, + }); + expect(plan.resources).toEqual([ + { + type: 'd1', + bindingName: 'APP_DB', + ...(readReplication !== undefined && { readReplication }), + }, + ]); + }, + ); it('normalizes provisionable manifest resources without provider details', () => { const manifest: DeployManifest = { version: 3, diff --git a/packages/deploy-core/test/unit/validate.test.ts b/packages/deploy-core/test/unit/validate.test.ts index ae41d00b1..6cf0e869d 100644 --- a/packages/deploy-core/test/unit/validate.test.ts +++ b/packages/deploy-core/test/unit/validate.test.ts @@ -1,6 +1,39 @@ import { describe, expect, it } from 'vite-plus/test'; import { validateManifestShape, type DeployManifest } from '../../src/index.ts'; +describe('D1 read replication', () => { + it.each([undefined, false, true])('accepts readReplication %s', (readReplication) => { + expect( + validateManifestShape({ + version: 3, + bindings: { d1: ['DB'] }, + database: { readReplication }, + }), + ).toBeNull(); + }); + it.each([null, true, { readReplication: 'true' }, { readReplication: 1 }, { replicas: true }])( + 'rejects invalid database preferences %j', + (database) => { + expect( + validateManifestShape({ + version: 3, + bindings: { d1: ['DB'] }, + database, + }), + ).toContain('database.readReplication'); + }, + ); + it('rejects database preferences without a D1 binding', () => { + expect( + validateManifestShape({ + version: 3, + bindings: { hyperdrive: ['HYPERDRIVE'] }, + database: { readReplication: true }, + }), + ).toContain('requires a D1 binding'); + }); +}); + describe('queue concurrency', () => { it.each([undefined, 1, 2, 250])('accepts maxConcurrency = %s', (maxConcurrency) => { expect( diff --git a/packages/void/AGENTS.md b/packages/void/AGENTS.md index 9ab870aec..c8641f6ac 100644 --- a/packages/void/AGENTS.md +++ b/packages/void/AGENTS.md @@ -138,6 +138,7 @@ generated entries must preserve the module-boundary rules below. - **Edge-routing ownership**: managed builds set `VOID_DEPLOY_PROJECT_ID` and omit the generated redirects, rewrites, headers, and fallbacks because platform dispatch applies them. Native Cloudflare, local dev, and non-CF builds compile the same ordered rules into the application runtime. Only managed bundles may trust dispatch-authored `X-Void-Original-URL` metadata. - **Direct Sandbox entitlement invariant**: `void/sandbox` is opt-in and is the only public Void feature that currently requires Workers Paid on the native Cloudflare path. When source inference or `void.config.ts` enables it, deploy must verify Containers with a read-only account probe after authentication and before provisioning/building, then explain both the Workers Paid signup and Containers/Cloudchamber token permissions on failure. A non-Sandbox app must emit no Container and perform no Containers probe. - **Local D1 identity**: `local-d1.ts` selects the configured D1 binding from root JSON/JSONC config, preferring `preview_database_id` over `database_id` for local development. Development, schema checks, auth migrations, and every local database command use this same identity and persistence directory. `voidPlugin({ persistTo })` records its canonical path in `.void/local-d1.json`; CLI commands reread binding IDs and reject stale Vite-config state until `void dev` refreshes it. +- **D1 read replication**: preserve all three `database.readReplication` states through native and hosted deployment. True enables replication, false disables it, and omitted skips replication configuration (new databases default to off). Native deploy reconciles either explicit boolean on already-provisioned databases too. - **Migration DB path**: Miniflare stores D1 data at a deterministic path computed from the `database_id` using HMAC-SHA256. The `getMiniflareDatabasePath()` function in `migrations/runner.ts` replicates this algorithm so migrations target `/v3/d1/miniflare-D1DatabaseObject/.sqlite` (for example `.void/v3/...` in default/class-A mode and `.wrangler/state/v3/...` in class-B/C framework mode). - **Drizzle-first database**: Schema is defined in `db/schema.ts` (Drizzle table definitions). `void/db` exports a dialect-aware Drizzle instance for D1, PostgreSQL, or MySQL. `@schema` is an auto-configured alias for `db/schema.ts`. Migrations live in `db/migrations/`. CLI commands: `void db push` (prototype), `void db generate` (create migration files), `void db migrate` (apply locally). When auth is enabled, `void db generate` adds `.void/better-auth-schema.ts` to Drizzle Kit's schema inputs. Direct Cloudflare deploy applies PostgreSQL or MySQL migrations from the shell-provided production `DATABASE_URL`; PostgreSQL is transactional, while MySQL DDL is not. `drizzle-orm` and `drizzle-kit` ship with void. For PostgreSQL and MySQL, `void/db` is one drizzle instance whose client looks up the current request's pool on every call. workerd ties a socket to the request that opened it, so a query built or prepared at module scope must run on the pool of the request that executes it, and building one must not create a pool (global scope has no request to own it). mysql2 starts its idle timer only when it builds a pool with `maxIdle` below `connectionLimit`, and that timer runs until the pool ends, so the MySQL pools set neither: each connection closes from its pool `release` event after 1 second idle (60 seconds for `createDb`). Inside a request, reading `db` builds that request's pool first, so a missing connection string throws its own error rather than drizzle's "Failed query". The client answers drizzle's pool checks (`instanceof Pool`, `'getConnection' in`) through `getPrototypeOf` and `has`, cached per isolate, so a transaction still holds one connection. Astro's adapter entry runs inside `withRuntimeEnv` for these dialects, like the Class A wrapper. - **Database provisioning adapters**: `void db connect ` saves an external database URL to `.env` and syncs a linked Void project; `--local-only` skips the sync. `void db connect --provider neon` provisions through the built-in Neon adapter using `NEON_API_KEY`. Third-party packages or local modules can export a `DatabaseProviderAdapter` built with `void/database-provider`; runtime database code remains provider-neutral. diff --git a/packages/void/package.json b/packages/void/package.json index f3106e33e..040acf123 100644 --- a/packages/void/package.json +++ b/packages/void/package.json @@ -28,6 +28,7 @@ "type": "module", "imports": { "#self/_env": "./dist/runtime/env/runtime.mjs", + "#self/runtime/db-session": "./dist/runtime/db/session.mjs", "#self/email": "./dist/runtime/email/index.mjs", "#self/env": "./dist/runtime/env/public.mjs", "#self/client": "./dist/runtime/client/index.mjs", diff --git a/packages/void/schema.json b/packages/void/schema.json index 5edac0669..05c3da654 100644 --- a/packages/void/schema.json +++ b/packages/void/schema.json @@ -415,7 +415,7 @@ "additionalProperties": false }, "database": { - "description": "Database backend. Omit for D1 (SQLite, default). Set to \"pg\" for PostgreSQL or \"mysql\" for MySQL via Hyperdrive, or use an object to also set the column casing.", + "description": "Database backend. Omit for D1 (SQLite, default). Set to \"pg\" for PostgreSQL or \"mysql\" for MySQL via Hyperdrive, or use an object for database options.", "oneOf": [ { "type": "string", @@ -433,8 +433,14 @@ "type": "string", "enum": ["snake_case", "camelCase"], "description": "Drizzle casing: how columns declared without a name are named in the database, for queries and generated migrations." + }, + "readReplication": { + "type": "boolean", + "description": "Enable or disable D1 read replication during deployment. Omit to preserve existing settings; new databases default to replication off." } }, + "if": { "required": ["readReplication"] }, + "then": { "properties": { "dialect": { "const": "sqlite" } } }, "additionalProperties": false } ] diff --git a/packages/void/skills/void/SKILL.md b/packages/void/skills/void/SKILL.md index 24550f2c0..f36ee0b6e 100644 --- a/packages/void/skills/void/SKILL.md +++ b/packages/void/skills/void/SKILL.md @@ -44,6 +44,7 @@ Use `void deploy --skip-migrations` when database migrations run separately; it | Server/API routing and middleware | `docs/guide/server-routing.md`, `docs/integrations/hono.md` | | Pages mode, loader/action, forms, layouts | `docs/guide/pages-routing/*.md`, `docs/guide/type-safety.md` | | Database and migrations | `docs/guide/database.md`, `docs/guide/type-safety.md` | +| D1 replicas, bookmarks, read-after-write | `docs/guide/database/d1.md`, `docs/reference/api/database.md` | | Typed fetch and end-to-end typing | `docs/guide/typed-fetch.md`, `docs/guide/type-safety.md` | | Authentication | `docs/guide/auth.md`, `docs/reference/api/auth.md`, `docs/guide/env-vars.md` | | Cloudflare runtime bindings and config | `docs/integrations/cloudflare.md`, `docs/reference/config.md`, `docs/guide/env-vars.md` | diff --git a/packages/void/src/cli/deploy-cloudflare-preflight.ts b/packages/void/src/cli/deploy-cloudflare-preflight.ts index 5a73c97bf..26528c33b 100644 --- a/packages/void/src/cli/deploy-cloudflare-preflight.ts +++ b/packages/void/src/cli/deploy-cloudflare-preflight.ts @@ -624,6 +624,7 @@ export async function preflightCloudflareWorker(input: { options.provision || initiallyMissing.length > 0 || d1NeedsConfigReconciliation || + (!existingWorker && desired.d1?.readReplication !== undefined) || absentQueues.length > 0 ) { await provisionAndVerify(); diff --git a/packages/void/src/cli/deploy-managed-framework.ts b/packages/void/src/cli/deploy-managed-framework.ts index 1f5bc3de6..26353a95c 100644 --- a/packages/void/src/cli/deploy-managed-framework.ts +++ b/packages/void/src/cli/deploy-managed-framework.ts @@ -201,6 +201,7 @@ export async function runFrameworkDeploy( frameworkName: detected.name, alwaysUseHttps: voidConfig.routing?.alwaysUseHttps, storage: voidConfig.storage, + database: typeof voidConfig.database === 'object' ? voidConfig.database : undefined, workerMain: actualWorkerMain, bindings, migrations: validatedMigrations, diff --git a/packages/void/src/cli/deploy-managed-full.ts b/packages/void/src/cli/deploy-managed-full.ts index dfc171252..890792d33 100644 --- a/packages/void/src/cli/deploy-managed-full.ts +++ b/packages/void/src/cli/deploy-managed-full.ts @@ -422,6 +422,7 @@ export async function runFullDeploy( bindings: liveConfig ? { ...bindings, needsLive: true } : bindings, alwaysUseHttps: voidConfig.routing?.alwaysUseHttps, storage: voidConfig.storage, + database: typeof voidConfig.database === 'object' ? voidConfig.database : undefined, hasEmailHandlers: !isFrameworkMode && routeScan.email !== undefined && hasEmailHandlers(routeScan.email), migrations: validatedMigrations, diff --git a/packages/void/src/cli/deploy-resources.ts b/packages/void/src/cli/deploy-resources.ts index 43e951ac1..aa56855aa 100644 --- a/packages/void/src/cli/deploy-resources.ts +++ b/packages/void/src/cli/deploy-resources.ts @@ -242,6 +242,12 @@ export async function computeDesiredCloudflareResources( : undefined) || suffixedResourceName(workerName, 'db'); desired.d1 = { binding, name, migrationsDir: migrationsDirRelative }; + if ( + typeof voidConfig.database === 'object' && + voidConfig.database.readReplication !== undefined + ) { + desired.d1.readReplication = voidConfig.database.readReplication; + } reasons.set(binding, databaseReason); } diff --git a/packages/void/src/cli/help.ts b/packages/void/src/cli/help.ts index 7ee35a038..b028c9000 100644 --- a/packages/void/src/cli/help.ts +++ b/packages/void/src/cli/help.ts @@ -183,7 +183,7 @@ const HELP_TREE: HelpTree = { { command: '--provision', description: - 'Run resource provisioning again, for example to enable D1 read replication (with --platform cloudflare)', + 'Run resource provisioning again even when every resource exists (with --platform cloudflare)', }, { command: '--atomic', diff --git a/packages/void/src/cli/package.ts b/packages/void/src/cli/package.ts index 115df656d..6dd20d33b 100644 --- a/packages/void/src/cli/package.ts +++ b/packages/void/src/cli/package.ts @@ -384,12 +384,16 @@ function applyResourceBindings( migrationCount: number, dialect: 'sqlite' | 'postgresql' | 'mysql', bindingNames: PackageOptions['bindingNames'], + database: DeployManifest['database'], ): void { if (needsApplicationDatabase(bindings, migrationCount)) { if (dialect !== 'sqlite') { manifest.bindings.hyperdrive = ['HYPERDRIVE']; } else { manifest.bindings.d1 = [bindingNames?.d1 ?? 'DB']; + if (database?.readReplication !== undefined) { + manifest.database = { readReplication: database.readReplication }; + } } } if (bindings.needsKV) { @@ -448,6 +452,7 @@ export function buildVoidManifest(opts: { prerender?: Array; assetConfig?: AssetConfig; options?: PackageOptions; + database?: DeployManifest['database']; hashedAssetsPrefix?: string; headerRules?: Array; alwaysUseHttps?: boolean; @@ -511,6 +516,7 @@ export function buildVoidManifest(opts: { migrations?.length ?? 0, resolvedDialect, options?.bindingNames, + opts.database, ); if (bindings.needsSandbox) { if (!options?.sandbox) { @@ -748,6 +754,7 @@ export function buildFrameworkManifest(opts: { vars?: Record; assetConfig?: AssetConfig; packageOptions?: Omit; + database?: DeployManifest['database']; hashedAssetsPrefix?: string; headerRules?: Array; alwaysUseHttps?: boolean; @@ -773,6 +780,7 @@ export function buildFrameworkManifest(opts: { opts.migrations?.length ?? 0, dialect, po?.bindingNames, + opts.database, ); if (opts.migrations && opts.migrations.length > 0) { diff --git a/packages/void/src/cli/provision.ts b/packages/void/src/cli/provision.ts index 99cea1054..dd643f78d 100644 --- a/packages/void/src/cli/provision.ts +++ b/packages/void/src/cli/provision.ts @@ -40,7 +40,7 @@ import { /** The Cloudflare resources an app needs, computed by the deploy orchestrator. */ export interface DesiredResourceSet { /** `migrationsDir` is config-relative and defaults to `./db/migrations`. */ - d1?: { binding: string; name: string; migrationsDir?: string }; + d1?: { binding: string; name: string; migrationsDir?: string; readReplication?: boolean }; /** KV namespaces (app KV + `ISR_CACHE`). Titles are NOT account-unique. */ kvNamespaces: Array<{ binding: string; title: string }>; /** R2 bucket — bucket names ARE account-unique. */ @@ -2530,7 +2530,7 @@ export interface ProvisionOptions { accountId?: string; apiToken?: string; /** Test seam for the D1 database-level read-replication update. */ - enableD1ReadReplication?: (context: D1ReplicationContext) => Promise; + setD1ReadReplication?: (context: D1ReplicationContext) => Promise; listHyperdriveConfigs?: ListHyperdriveConfigs; log?: ProvisionLogger; /** Test-only override of the `proper-lockfile` stale/update/retries/fs knobs. */ @@ -2543,9 +2543,10 @@ export type D1ReplicationContext = { databaseId: string; accountId: string; apiBaseUrl: string; + readReplication: boolean; }; -export async function enableD1ReadReplicationViaApi( +export async function setD1ReadReplicationViaApi( context: D1ReplicationContext, token: string, fetchImpl: typeof fetch = fetch, @@ -2555,14 +2556,16 @@ export async function enableD1ReadReplicationViaApi( { method: 'PUT', headers: { Authorization: `Bearer ${token}`, 'Content-Type': 'application/json' }, - body: JSON.stringify({ read_replication: { mode: 'auto' } }), + body: JSON.stringify({ + read_replication: { mode: context.readReplication ? 'auto' : 'disabled' }, + }), signal: AbortSignal.timeout(15_000), }, ); const envelope = (await response.json().catch(() => null)) as { success?: unknown } | null; if (!response.ok || envelope?.success !== true) { throw new Error( - `provision: could not enable D1 read replication for database ${context.databaseId} ` + + `provision: could not ${context.readReplication ? 'enable' : 'disable'} D1 read replication for database ${context.databaseId} ` + `(Cloudflare API status ${response.status}). The credential needs D1 Edit permission.`, ); } @@ -2699,8 +2702,11 @@ async function reconcileD1( } } - // The database-level setting is separate from the Worker binding. A reused - // database may predate this feature, so converge it on every provision run. + // Omitted never changes existing replication or acquires an API bearer. + if (desired.d1.readReplication === undefined) { + return; + } + // The database-level setting is separate from the Worker binding. const databaseId = readBindingEntry(configPath, 'd1_databases', binding)?.database_id; if (!isRealD1Id(databaseId)) { throw new Error(`provision: D1 binding ${binding} has no real database id after setup.`); @@ -2709,9 +2715,10 @@ async function reconcileD1( databaseId, accountId: transportAccountId, apiBaseUrl: transportApiBaseUrl, + readReplication: desired.d1.readReplication, }; - if (opts?.enableD1ReadReplication) { - await opts.enableD1ReadReplication(context); + if (opts?.setD1ReadReplication) { + await opts.setD1ReadReplication(context); } else { let token = opts?.apiToken ?? process.env.CLOUDFLARE_API_TOKEN; if (!token) { @@ -2730,7 +2737,7 @@ async function reconcileD1( 'Run `void cloudflare login`, or set CLOUDFLARE_API_TOKEN, then retry.', ); } - await enableD1ReadReplicationViaApi(context, token); + await setD1ReadReplicationViaApi(context, token); } } } diff --git a/packages/void/src/codegen/db-types.ts b/packages/void/src/codegen/db-types.ts index acc99f7dc..fb1deacdc 100644 --- a/packages/void/src/codegen/db-types.ts +++ b/packages/void/src/codegen/db-types.ts @@ -11,7 +11,7 @@ const DIALECT_DB_TYPES = { sqlite: { type: 'DrizzleD1Database', from: 'void/_db', - arg: 'd1: D1Database, options?: { session?: D1SessionConstraint }', + arg: 'd1: D1Database, options?: { session?: DbSessionStart }', }, postgresql: { type: 'NodePgDatabase', from: 'void/_db-pg', arg: 'connectionString: string' }, mysql: { type: 'MySql2Database', from: 'void/_db-mysql', arg: 'connectionString: string' }, @@ -42,6 +42,14 @@ export function generateDbDts( ...(schema ? ['import type * as schema from "@schema";', 'type Schema = typeof schema;'] : []), `export declare const db: ${type}<${schemaType}>;`, `export declare function createDb(${arg}): ${type}<${schemaType}>;`, + ...(dialect === 'sqlite' + ? [ + 'import type { DbSessionStart } from "void/_db";', + 'export type { DbSessionStart } from "void/_db";', + 'export declare function setSession(start: DbSessionStart): void;', + `export declare function getBookmark>(client?: ${type}): D1SessionBookmark | null;`, + ] + : []), `export { ${DRIZZLE_OPERATORS} } from "void/_db";`, `export type { ${DRIZZLE_TYPES} } from "void/_db";`, `export type { ${type} } from "${from}";`, diff --git a/packages/void/src/config.ts b/packages/void/src/config.ts index a46a58bdd..aaf4e8b9d 100644 --- a/packages/void/src/config.ts +++ b/packages/void/src/config.ts @@ -33,6 +33,8 @@ export type DatabaseConfig = { dialect?: 'sqlite' | 'pg' | 'mysql'; /** Column naming for keys without an explicit column name, as in Drizzle's `casing` option. */ casing?: DatabaseCasing; + /** Enable or disable D1 read replication during deployment. Omit to preserve existing settings. */ + readReplication?: boolean; }; /** Curated Worker fields accepted by the `worker` config field. */ @@ -971,9 +973,9 @@ function validateConfigObject(obj: unknown): VoidConfig { if (isRecord(obj.database)) { const databaseCfg = obj.database; for (const key of Object.keys(databaseCfg)) { - if (key !== 'dialect' && key !== 'casing') { + if (key !== 'dialect' && key !== 'casing' && key !== 'readReplication') { throw new Error( - `config: Unknown key 'database.${key}' in ${CONFIG_FILE}. Valid: dialect, casing.`, + `config: Unknown key 'database.${key}' in ${CONFIG_FILE}. Valid: dialect, casing, readReplication.`, ); } } @@ -994,9 +996,21 @@ function validateConfigObject(obj: unknown): VoidConfig { `config: "database.casing" in ${CONFIG_FILE} must be "snake_case" or "camelCase".`, ); } + if (databaseCfg.readReplication !== undefined) { + if (typeof databaseCfg.readReplication !== 'boolean') { + throw new Error( + `config: "database.readReplication" in ${CONFIG_FILE} must be a boolean.`, + ); + } + if (databaseCfg.dialect !== undefined && databaseCfg.dialect !== 'sqlite') { + throw new Error( + `config: "database.readReplication" in ${CONFIG_FILE} is only supported for D1 (sqlite).`, + ); + } + } } else if (typeof obj.database !== 'string' || !VALID_DATABASE_VALUES.has(obj.database)) { throw new Error( - `config: "database" in ${CONFIG_FILE} must be "pg", "mysql" or an object with "dialect" and "casing".`, + `config: "database" in ${CONFIG_FILE} must be "pg", "mysql" or an object with "dialect", "casing" and "readReplication".`, ); } } diff --git a/packages/void/src/deploy.ts b/packages/void/src/deploy.ts index f9606fb5a..e83780b52 100644 --- a/packages/void/src/deploy.ts +++ b/packages/void/src/deploy.ts @@ -160,6 +160,7 @@ export async function prepareDeployArtifact( bindings, alwaysUseHttps: config.routing?.alwaysUseHttps, storage: config.storage, + database: typeof config.database === 'object' ? config.database : undefined, vars: config.worker?.vars, hasEmailHandlers: scan.email !== undefined && hasEmailHandlers(scan.email), migrations, diff --git a/packages/void/src/plugin-drizzle.ts b/packages/void/src/plugin-drizzle.ts index 86b364044..22e373f5e 100644 --- a/packages/void/src/plugin-drizzle.ts +++ b/packages/void/src/plugin-drizzle.ts @@ -22,6 +22,7 @@ import { readProjectPaths, resolveProjectPaths, type ProjectPaths } from './proj const drizzleD1Resolved = fileURLToPath(import.meta.resolve('drizzle-orm/d1')); const drizzleOrmResolved = fileURLToPath(import.meta.resolve('drizzle-orm')); const envModuleResolved = fileURLToPath(import.meta.resolve('#self/_env')); +const dbSessionResolved = fileURLToPath(import.meta.resolve('#self/runtime/db-session')); const drizzleNodePgResolved = fileURLToPath(import.meta.resolve('drizzle-orm/node-postgres')); const drizzleMysql2Resolved = fileURLToPath(import.meta.resolve('drizzle-orm/mysql2')); @@ -300,29 +301,48 @@ export function createDb(connectionString) { return ` export { ${DRIZZLE_OPERATORS} } from ${JSON.stringify(drizzleOrmResolved)}; import { drizzle } from ${JSON.stringify(drizzleD1Resolved)}; -import { getRuntimeRequestResource, requireRuntimeBinding } from ${JSON.stringify(envModuleResolved)}; +import { getRuntimeRequestResource } from ${JSON.stringify(envModuleResolved)}; +import { createD1SessionAccess, resolveD1SessionStart } from ${JSON.stringify(dbSessionResolved)}; ${schemaImport} const _requestResourceKey = {}; +const sessions = createD1SessionAccess(${JSON.stringify(d1BindingName)}); + +export function setSession(start) { + sessions.setSession(start); +} + +export function getBookmark(client) { + return sessions.getBookmark(client === db ? undefined : client); +} export const db = new Proxy({}, { get(_, prop) { + // Promise assimilation and inspection are not database use. + if ( + prop === 'then' || + prop === 'toJSON' || + prop === Symbol.toStringTag || + prop === Symbol.for('nodejs.util.inspect.custom') + ) { + return undefined; + } const instance = getRuntimeRequestResource( _requestResourceKey, - () => createDb(requireRuntimeBinding(${JSON.stringify(d1BindingName)})), - () => createPrimaryDb(requireRuntimeBinding(${JSON.stringify(d1BindingName)})) + () => drizzle(sessions.getSession()${drizzleConfig}), + () => drizzle(sessions.getSession()${drizzleConfig}) ); return Reflect.get(instance, prop); }, }); export function createDb(d1, options) { - return drizzle(d1.withSession(options?.session ?? "first-unconstrained")${drizzleConfig}); + return drizzle(d1.withSession(resolveD1SessionStart( + options?.session === undefined ? "first-unconstrained" : options.session, + "createDb" + ))${drizzleConfig}); } -function createPrimaryDb(d1) { - return drizzle(d1.withSession("first-primary")${drizzleConfig}); -} `; }, }, diff --git a/packages/void/src/runtime/db/index.ts b/packages/void/src/runtime/db/index.ts index 94e3396f5..51fc48dec 100644 --- a/packages/void/src/runtime/db/index.ts +++ b/packages/void/src/runtime/db/index.ts @@ -11,6 +11,7 @@ import { getRuntimeRequestResource, requireRuntimeBinding, } from '../env/runtime.ts'; +import { createD1SessionAccess, resolveD1SessionStart, type DbSessionStart } from './session.ts'; // Re-export commonly used Drizzle utilities so consumers never need // to depend on drizzle-orm directly. @@ -48,21 +49,40 @@ export type { AnyColumn, SQL, SQLWrapper } from 'drizzle-orm'; export type { BatchItem } from 'drizzle-orm/batch'; export type { DrizzleD1Database } from 'drizzle-orm/d1'; +export type { DbSessionStart }; + +const DEFAULT_BINDING = 'DB'; + /** * Create a Drizzle D1 instance from a custom D1 binding. * Use this when you have multiple D1 databases or need a non-default binding. */ export function createDb( d1: D1Database, - options?: { session?: D1SessionConstraint }, + options?: { session?: DbSessionStart }, ): DrizzleD1Database { return _drizzle( - d1.withSession(options?.session ?? 'first-unconstrained') as unknown as D1Database, + d1.withSession( + resolveD1SessionStart( + options?.session === undefined ? 'first-unconstrained' : options.session, + 'createDb', + ), + ) as unknown as D1Database, ); } -function createPrimaryDb(d1: D1Database): DrizzleD1Database { - return _drizzle(d1.withSession('first-primary') as unknown as D1Database); +const sessions = createD1SessionAccess(DEFAULT_BINDING); + +/** Set the default D1 session's starting bookmark or constraint before using db. */ +export function setSession(start: DbSessionStart): void { + sessions.setSession(start); +} + +/** Read the default request session's bookmark, or that of an explicit D1 client. */ +export function getBookmark>( + client?: DrizzleD1Database, +): D1SessionBookmark | null { + return sessions.getBookmark((client as object | undefined) === db ? undefined : client); } const requestDbKey = {}; @@ -73,14 +93,19 @@ function requireD1(): D1Database { // an installed package only when the package is an ES module that depends on void and the // app lists it or reaches it through such packages. if ( - getRuntimeBinding('DB') == null && + getRuntimeBinding(DEFAULT_BINDING) == null && (getRuntimeBinding('HYPERDRIVE') != null || getRuntimeBinding('DATABASE_URL') != null) ) { throw new Error( - "db: Missing Cloudflare binding 'DB'. This app connects through HYPERDRIVE or DATABASE_URL, but its server bundle loaded the D1 client. In Nuxt or Analog, add 'void/nitro' to the Nitro modules. An installed package that imports void/db must be an ES module that lists void in its dependencies or peerDependencies, and the app must list it or reach it through such packages.", + `db: Missing Cloudflare binding '${DEFAULT_BINDING}'. This app connects through HYPERDRIVE or DATABASE_URL, but its server bundle loaded the D1 client. In Nuxt or Analog, add 'void/nitro' to the Nitro modules. An installed package that imports void/db must be an ES module that lists void in its dependencies or peerDependencies, and the app must list it or reach it through such packages.`, ); } - return requireRuntimeBinding('DB'); + return requireRuntimeBinding(DEFAULT_BINDING); +} + +function d1Init() { + requireD1(); + return _drizzle(sessions.getSession()); } /** @@ -90,12 +115,20 @@ function requireD1(): D1Database { */ export const db: DrizzleD1Database = new Proxy({} as DrizzleD1Database, { get(_, prop) { + // Promise assimilation and inspection are not database use. + if ( + prop === 'then' || + prop === 'toJSON' || + prop === Symbol.toStringTag || + prop === Symbol.for('nodejs.util.inspect.custom') + ) { + return undefined; + } const instance = getRuntimeRequestResource( requestDbKey, - () => createDb(requireD1()), - // Framework adapters do not always enter Void's request scope. Without a - // shared session, each operation starts on primary to preserve fresh reads. - () => createPrimaryDb(requireD1()), + d1Init, + // Outside Void's request scope, create an uncached client. + d1Init, ); return Reflect.get(instance, prop); }, diff --git a/packages/void/src/runtime/db/session.ts b/packages/void/src/runtime/db/session.ts new file mode 100644 index 000000000..4c6db0137 --- /dev/null +++ b/packages/void/src/runtime/db/session.ts @@ -0,0 +1,106 @@ +import type { DrizzleD1Database } from 'drizzle-orm/d1'; +import { getRuntimeRequestResource, requireRuntimeBinding } from '../env/runtime.ts'; + +export type DbSessionStart = D1SessionConstraint | { bookmark: D1SessionBookmark }; + +/** @internal Validate Void's input shape without interpreting opaque D1 bookmarks. */ +export function resolveD1SessionStart( + start: DbSessionStart, + helper: 'setSession' | 'createDb', +): D1SessionConstraint | D1SessionBookmark { + if (start === 'first-primary' || start === 'first-unconstrained') { + return start; + } + if (typeof start === 'object' && start !== null && typeof start.bookmark === 'string') { + return start.bookmark; + } + throw new Error( + `db: ${helper}() requires 'first-primary', 'first-unconstrained', or { bookmark: string } for its session start.`, + ); +} + +export interface DbRequestSession { + start: DbSessionStart; + session?: D1DatabaseSession; +} + +export interface DbSessionAccess { + getSession(): D1Database; + setSession(start: DbSessionStart): void; + getBookmark>( + client?: DrizzleD1Database | undefined, + ): D1SessionBookmark | null; +} + +/** @internal Shared by the published D1 client and schema-aware generated clients. */ +export function createD1SessionAccess(bindingName: string): DbSessionAccess { + const key = {}; + + function state(helper: 'setSession' | 'getBookmark'): DbRequestSession { + return getRuntimeRequestResource( + key, + () => ({ start: 'first-unconstrained' }), + () => { + throw new Error( + `db: ${helper}() requires Void's request scope. In framework-owned handlers, use createDb(binding, { session }) and getBookmark(client).`, + ); + }, + ); + } + + return { + getSession() { + const current = getRuntimeRequestResource( + key, + () => ({ start: 'first-unconstrained' }), + // Outside Void's request scope, sessions are not shared between operations. + // Start each session on primary to preserve fresh reads. + () => ({ start: 'first-primary' }), + ); + return (current.session ??= requireRuntimeBinding(bindingName).withSession( + typeof current.start === 'object' ? current.start.bookmark : current.start, + )) as unknown as D1Database; + }, + + setSession(start) { + const current = state('setSession'); + const value = resolveD1SessionStart(start, 'setSession'); + // Nested middleware may have no bookmark of its own. Its fallback must + // not discard the outer requirement, even before the first DB access. + if (start === 'first-unconstrained') { + return; + } + if (current.session) { + // Middleware runs again on local server-side fetches. Repeated inputs + // must preserve the existing session and any progress made by queries. + const original = typeof current.start === 'object' ? current.start.bookmark : current.start; + if (typeof start === typeof current.start && value === original) { + return; + } + const bookmark = current.session.getBookmark(); + if (typeof start === 'object' && bookmark !== null && value <= bookmark) { + return; + } + throw new Error( + 'db: setSession() cannot strengthen an opened session. Call it before the first use of db, or use createDb(binding, { session }) with getBookmark(client) for an independent session.', + ); + } + current.start = typeof start === 'object' ? { bookmark: value } : start; + }, + + getBookmark>( + client?: DrizzleD1Database | undefined, + ) { + if (client) { + const session = Reflect.get(client, '$client'); + if (!session?.getBookmark) { + throw new Error('db: getBookmark(client) requires a D1 session-backed client.'); + } + + return session.getBookmark(); + } + // Inspect only: middleware on a route without database queries must not open a session. + return state('getBookmark').session?.getBookmark() ?? null; + }, + }; +} diff --git a/packages/void/test/integration/db-session.test.ts b/packages/void/test/integration/db-session.test.ts new file mode 100644 index 000000000..50319f2cd --- /dev/null +++ b/packages/void/test/integration/db-session.test.ts @@ -0,0 +1,300 @@ +import { mkdirSync, mkdtempSync, rmSync, writeFileSync } from 'node:fs'; +import { tmpdir } from 'node:os'; +import { join } from 'node:path'; +import { fileURLToPath } from 'node:url'; +import { convertV4MiniflareOptions, Miniflare } from 'miniflare'; +import { build } from 'vite'; +import { afterEach, describe, expect, it } from 'vite-plus/test'; +import { analyzeApplication } from '../../src/application-routing.ts'; +import { readConfig } from '../../src/config.ts'; +import { drizzlePlugin } from '../../src/plugin-drizzle.ts'; +import { + compileRouteTable, + compileWorkerClient, + compileWorkerEntry, +} from '../../src/router/compile.ts'; + +const source = (file: string) => + fileURLToPath(new URL(`../../src/runtime/${file}`, import.meta.url)); +let directory: string; +let runtime: Miniflare | undefined; +afterEach(async () => { + await runtime?.dispose(); + runtime = undefined; + if (directory) { + rmSync(directory, { recursive: true, force: true }); + } +}); + +describe('D1 bookmarks in generated Workers', () => { + // Local D1 has no replica lag. The unit reproduction controls lag; this verifies + // real D1 bookmarks, middleware, redirects, bundling and request scope in workerd. + it.each( + [ + { module: 'runtime', bindingName: 'DB' }, + { module: 'generated', bindingName: 'DB' }, + { module: 'generated', bindingName: 'APP_DB' }, + ].flatMap((client) => + ['header', 'cookie'].flatMap((transport) => + ['direct', 'server-fetch', 'before-use', 'nested-write', 'concurrent'].map((dispatch) => ({ + ...client, + transport, + dispatch, + })), + ), + ), + )( + '$module / $bindingName carries a bookmark through $transport middleware ($dispatch)', + async ({ module, bindingName, transport, dispatch }) => { + directory = mkdtempSync(join(tmpdir(), 'void-db-bookmark-worker-')); + const routes = join(directory, 'routes'); + const middleware = join(directory, 'middleware'); + const output = join(directory, 'output'); + mkdirSync(routes); + mkdirSync(middleware); + writeFileSync( + join(middleware, '01.database.ts'), + ` +import { db, setSession, getBookmark } from 'void/db'; +import { getCookie, setCookie } from 'hono/cookie'; +export default async (c, next) => { + // Incidental probes must not open the session before middleware configures it. + await Promise.resolve(db); + JSON.stringify(db); + Object.prototype.toString.call(db); + Reflect.get(db, Symbol.for('nodejs.util.inspect.custom')); + const incoming = ${transport === 'cookie' ? "getCookie(c, 'db-bookmark')" : "c.req.header('X-DB-Bookmark')"}; + setSession(incoming === undefined ? 'first-unconstrained' : { bookmark: incoming }); + await next(); + const bookmark = getBookmark(); + if (bookmark) { + c.header('X-DB-Bookmark', bookmark); + ${transport === 'cookie' ? "setCookie(c, 'db-bookmark', bookmark, { httpOnly: true, sameSite: 'Lax', path: '/' });" : ''} + } +}; +`, + ); + writeFileSync( + join(routes, 'profile.ts'), + ` +import { db, createDb, getBookmark, sql } from 'void/db'; +export async function POST(c) { + await db.run(sql\`UPDATE profile SET name = 'Alice' WHERE id = 1\`); + return c.redirect('/profile', 303); +} +export async function GET(c) { + const name = await db.get(sql\`SELECT name FROM profile WHERE id = 1\`); + const bookmark = getBookmark(); + const client = createDb(c.env[${JSON.stringify(bindingName)}], { + session: bookmark === null ? 'first-unconstrained' : { bookmark }, + }); + const explicit = await client.get(sql\`SELECT name FROM profile WHERE id = 1\`); + return c.json({ name, explicit, bookmark: getBookmark(client) }); +} +export async function PATCH(c) { + await db.run(sql\`UPDATE profile SET name = 'Nested name' WHERE id = 1\`); + return c.json({ bookmark: getBookmark() }); +} +`, + ); + writeFileSync(join(routes, 'unused.ts'), 'export function GET() { return "unused"; }'); + if (dispatch !== 'direct') { + writeFileSync( + join(routes, 'nested.ts'), + ` +import { fetch } from 'void/client'; +import { db, sql } from 'void/db'; +export async function GET(c) { + ${dispatch === 'before-use' ? '' : 'await db.get(sql`SELECT name FROM profile WHERE id = 1`);'} + ${ + dispatch === 'nested-write' + ? `await fetch('/profile', { method: 'PATCH', retry: 0 }); + const outerName = await db.get(sql\`SELECT name FROM profile WHERE id = 1\`); + return c.json({ ...await fetch('/profile', { retry: 0 }), outerName });` + : dispatch === 'concurrent' + ? `const results = await Promise.all([ + fetch('/profile', { retry: 0 }), + fetch('/profile', { retry: 0 }), + ]); + return c.json({ ...results[0], concurrent: results });` + : "return c.json(await fetch('/profile', { retry: 0 }));" + } +} +`, + ); + } + const { scan } = await analyzeApplication(directory, { config: readConfig(directory) }); + const routeTable = join(directory, 'generated-routes.ts'); + const entry = join(directory, 'generated-worker.ts'); + writeFileSync( + routeTable, + compileRouteTable( + scan, + routes, + middleware, + join(directory, 'crons'), + join(directory, 'queues'), + null, + { + honoPath: fileURLToPath(import.meta.resolve('hono')), + honoRoutePath: fileURLToPath(import.meta.resolve('hono/route')), + runtimeEnvPath: source('env/runtime.ts'), + projectRoot: directory, + homeDir: tmpdir(), + auth: null, + dev: false, + reactRefresh: false, + }, + ), + ); + writeFileSync(entry, compileWorkerEntry({ runtimeEnvPath: source('env/runtime.ts') })); + const instrumentedEntry = join(directory, 'instrumented-worker.ts'); + writeFileSync( + instrumentedEntry, + ` +import worker from './generated-worker.ts'; +export default { + async fetch(request, env, ctx) { + const starts = []; + const bindingName = ${JSON.stringify(bindingName)}; + const binding = env[bindingName]; + const tracked = new Proxy(binding, { + get(target, prop) { + if (prop === 'withSession') return (start) => { + starts.push(start); + return target.withSession(start); + }; + const value = Reflect.get(target, prop, target); + return typeof value === 'function' ? value.bind(target) : value; + }, + }); + const response = await new worker(ctx, { ...env, [bindingName]: tracked }).fetch(request); + const result = new Response(response.body, response); + result.headers.set('X-Test-Session-Starts', JSON.stringify(starts)); + return result; + }, +}; +`, + ); + await build({ + configFile: false, + logLevel: 'silent', + plugins: [ + { + name: 'bookmark-worker-fixture', + enforce: 'pre', + resolveId(id) { + if (id === 'virtual:void-routes') { + return routeTable; + } + if (id === 'hono/cookie') { + return fileURLToPath(import.meta.resolve('hono/cookie')); + } + if (id.endsWith('/dist/runtime/db/session.mjs')) { + return source('db/session.ts'); + } + if (id.endsWith('/dist/runtime/routing/index.mjs')) { + return source('routing/index.ts'); + } + if (id === 'void/db' && module === 'runtime') { + return source('db/index.ts'); + } + if (id === 'void/client') { + return '\0bookmark-worker-client'; + } + if (id.endsWith('/dist/runtime/client/fetch-server.mjs')) { + return source('client/fetch-server.ts'); + } + }, + load(id) { + if (id === '\0bookmark-worker-client') { + return compileWorkerClient(source('auth/client.ts')); + } + }, + }, + ...(module === 'generated' + ? [ + drizzlePlugin(directory, false, 'sqlite', { + d1BindingName: bindingName, + forceVirtualDb: true, + }), + ] + : []), + ], + ssr: { noExternal: true }, + build: { + outDir: output, + ssr: instrumentedEntry, + target: 'esnext', + rollupOptions: { + external: [/^cloudflare:/, /^node:/], + output: { entryFileNames: 'worker.mjs', codeSplitting: false }, + }, + }, + }); + runtime = new Miniflare( + convertV4MiniflareOptions({ + compatibilityDate: '2026-07-01', + compatibilityFlags: ['nodejs_compat'], + modules: true, + modulesRoot: output, + scriptPath: join(output, 'worker.mjs'), + d1Databases: [bindingName], + }), + ); + const binding = await runtime.getD1Database(bindingName); + await binding.prepare('CREATE TABLE profile (id INTEGER PRIMARY KEY, name TEXT)').run(); + await binding.prepare("INSERT INTO profile VALUES (1, 'Old name')").run(); + const saved = await runtime.dispatchFetch('https://app.test/profile', { + method: 'POST', + redirect: 'manual', + }); + expect(saved.status, await saved.clone().text()).toBe(303); + expect(saved.headers.get('Location')).toBe('/profile'); + const bookmark = saved.headers.get('X-DB-Bookmark'); + expect(bookmark).toBeTruthy(); + const loaded = await runtime.dispatchFetch( + `https://app.test/${dispatch === 'direct' ? 'profile' : 'nested'}`, + { + headers: + transport === 'cookie' + ? { Cookie: `db-bookmark=${encodeURIComponent(bookmark!)}` } + : { 'X-DB-Bookmark': bookmark! }, + }, + ); + expect(loaded.status, await loaded.clone().text()).toBe(200); + // Local D1 has no replica lag, so explicitly assert that middleware + // preserved the outer bookmark even when nested dispatch happens first. + const starts = JSON.parse(loaded.headers.get('X-Test-Session-Starts')!) as Array; + expect(starts[0]).toBe(bookmark); + // Direct GET also creates an explicit client from its wrapped env. + // Local fetch receives cloudflare:workers env for explicit clients, + // while its default db still shares the outer runtime session. + expect(starts).toHaveLength(dispatch === 'direct' ? 2 : 1); + const name = dispatch === 'nested-write' ? 'Nested name' : 'Alice'; + const profile = { + name: { name }, + explicit: { name }, + bookmark: expect.any(String), + }; + expect(await loaded.json()).toEqual({ + ...profile, + ...(dispatch === 'nested-write' ? { outerName: { name } } : {}), + ...(dispatch === 'concurrent' ? { concurrent: [profile, profile] } : {}), + }); + expect(loaded.headers.get('X-DB-Bookmark')).toBeTruthy(); + if (transport === 'cookie') { + expect(loaded.headers.get('Set-Cookie')).toContain('db-bookmark='); + } + const unused = await runtime.dispatchFetch('https://app.test/unused', { + headers: + transport === 'cookie' + ? { Cookie: `db-bookmark=${encodeURIComponent(bookmark!)}` } + : { 'X-DB-Bookmark': bookmark! }, + }); + expect(await unused.text()).toBe('unused'); + expect(unused.headers.has('X-DB-Bookmark')).toBe(false); + }, + 30_000, + ); +}); diff --git a/packages/void/test/unit/application-analysis-consumers.test.ts b/packages/void/test/unit/application-analysis-consumers.test.ts index 082497761..2258e5389 100644 --- a/packages/void/test/unit/application-analysis-consumers.test.ts +++ b/packages/void/test/unit/application-analysis-consumers.test.ts @@ -62,6 +62,22 @@ afterEach(() => { }); describe('application analysis consumer parity', () => { + it.each([undefined, false, true])( + 'carries D1 replication opt-in to native resources: %s', + async (readReplication) => { + const root = project( + { database: { readReplication }, inference: { bindings: { db: 'APP_DB' } } }, + {}, + ); + const result = await computeDesiredCloudflareResources( + root, + readConfig(root), + join(root, 'wrangler.jsonc'), + ); + expect(result.desired.d1?.binding).toBe('APP_DB'); + expect(result.desired.d1?.readReplication).toBe(readReplication); + }, + ); it.each(['CACHE', false] as const)( 'retains storage, auth and live with a partial KV override of %s', async (kv) => { diff --git a/packages/void/test/unit/cloudflare-deploy.test.ts b/packages/void/test/unit/cloudflare-deploy.test.ts index 604a17bae..bcc4656e2 100644 --- a/packages/void/test/unit/cloudflare-deploy.test.ts +++ b/packages/void/test/unit/cloudflare-deploy.test.ts @@ -2987,7 +2987,6 @@ describe('computeDesiredCloudflareResources — provisioned resources are never calls.push(args); return { stdout: '', stderr: '', status: 0 }; }), - enableD1ReadReplication: async () => {}, log: { info: () => {}, warn: () => {} }, }); // Read-first everywhere: no list, no create — the recorded resources are kept. @@ -4248,6 +4247,20 @@ describe('runCloudflareDeploy — preflight', () => { }); describe('runCloudflareDeploy — resource provisioning', () => { + it.each([undefined, false, true])( + 'reconciles an existing D1 only when replication is explicitly configured: %s', + async (readReplication) => { + const root = d1Project(PROVISIONED_D1); + writeFileSync(join(root, 'void.json'), JSON.stringify({ database: { readReplication } })); + const { deps, trace } = harness(); + await captureDeployOutput(() => + runCloudflareDeploy(root, { platform: 'cloudflare' }, silentCliLog, deps), + ); + expect(trace.includes('provision')).toBe(readReplication !== undefined); + expect(trace).toContain('build'); + }, + ); + it('automatically provisions a missing D1 database and continues the deploy', async () => { const root = d1Project({ account_id: 'acc123', name: 'myapp' }); let capturedDesired: Parameters>[2] | undefined; @@ -4293,17 +4306,17 @@ describe('runCloudflareDeploy — resource provisioning', () => { expect(output.match(/Successfully deployed!/g)).toHaveLength(1); }); - it('reconciles D1 replication before build, gate, migrations and deploy when fully provisioned', async () => { + it('reconciles opted-in D1 replication before build, gate, migrations and deploy when fully provisioned', async () => { const root = makeProject({ 'package.json': JSON.stringify({ name: 'myapp' }), - 'void.json': JSON.stringify({}), + 'void.json': JSON.stringify({ database: { readReplication: true } }), 'routes/index.ts': `export const GET = (c) => c.env.DB.prepare('SELECT 1').first();`, 'db/migrations/0000_init.sql': `CREATE TABLE notes (id integer primary key);`, 'wrangler.jsonc': JSON.stringify(PROVISIONED_D1, null, 2), }); const { deps, trace, wranglerArgs } = harness(); const output = await captureDeployOutput(() => - runCloudflareDeploy(root, { platform: 'cloudflare', provision: true }, silentCliLog, deps), + runCloudflareDeploy(root, { platform: 'cloudflare' }, silentCliLog, deps), ); expect(output).toContain('D1 migrations up to date'); expect(output).not.toMatch(/\d+ migrations? applied/); diff --git a/packages/void/test/unit/config.test.ts b/packages/void/test/unit/config.test.ts index 174e167d4..8c36961e3 100644 --- a/packages/void/test/unit/config.test.ts +++ b/packages/void/test/unit/config.test.ts @@ -1605,6 +1605,9 @@ describe('database config', () => { [{ casing: 'snake_case' }, 'sqlite', 'snake_case'], [{ dialect: 'pg' }, 'postgresql', undefined], [{}, 'sqlite', undefined], + [{ readReplication: true }, 'sqlite', undefined], + [{ readReplication: false }, 'sqlite', undefined], + [{ dialect: 'sqlite', readReplication: true }, 'sqlite', undefined], ])('accepts the object form %j', (database, dialect, casing) => { setup(JSON.stringify({ database })); const config = readConfig(tmpDir); @@ -1625,9 +1628,13 @@ describe('database config', () => { ], [{ casing: 'snake' }, '"database.casing" in void.json must be "snake_case" or "camelCase"'], [{ casing: 'preserve' }, '"database.casing" in void.json must be "snake_case" or "camelCase"'], + [{ readReplication: 'true' }, '"database.readReplication" in void.json must be a boolean'], + [{ readReplication: null }, '"database.readReplication" in void.json must be a boolean'], + [{ dialect: 'pg', readReplication: true }, 'only supported for D1 (sqlite)'], + [{ dialect: 'mysql', readReplication: false }, 'only supported for D1 (sqlite)'], [ { dialect: 'pg', url: 'x' }, - "Unknown key 'database.url' in void.json. Valid: dialect, casing.", + "Unknown key 'database.url' in void.json. Valid: dialect, casing, readReplication.", ], ])('rejects %j', (database, message) => { setup(JSON.stringify({ database })); @@ -1650,7 +1657,11 @@ describe('database config', () => { const schema = JSON.parse(readFileSync(join(import.meta.dirname, '../../schema.json'), 'utf8')); const [shorthand, object] = schema.properties.database.oneOf; expect(shorthand.enum).toEqual(['pg', 'mysql']); - expect(Object.keys(object.properties)).toEqual(['dialect', 'casing']); + expect(Object.keys(object.properties)).toEqual(['dialect', 'casing', 'readReplication']); + expect(object.properties.readReplication).toMatchObject({ type: 'boolean' }); + expect(object.properties.readReplication).not.toHaveProperty('default'); + expect(object.if).toEqual({ required: ['readReplication'] }); + expect(object.then).toEqual({ properties: { dialect: { const: 'sqlite' } } }); expect(object.properties.dialect.enum).toEqual(['sqlite', 'pg', 'mysql']); expect(object.properties.casing.enum).toEqual(['snake_case', 'camelCase']); expect(object.additionalProperties).toBe(false); diff --git a/packages/void/test/unit/db-session.test.ts b/packages/void/test/unit/db-session.test.ts new file mode 100644 index 000000000..c9a061dec --- /dev/null +++ b/packages/void/test/unit/db-session.test.ts @@ -0,0 +1,513 @@ +import { mkdtempSync, rmSync, writeFileSync } from 'node:fs'; +import { tmpdir } from 'node:os'; +import { join } from 'node:path'; +import { fileURLToPath, pathToFileURL } from 'node:url'; +import { inspect } from 'node:util'; +import { drizzle } from 'drizzle-orm/d1'; +import { integer, sqliteTable, text } from 'drizzle-orm/sqlite-core'; +import { Hono } from 'hono'; +import { convertV4MiniflareOptions, Miniflare } from 'miniflare'; +import { afterEach, describe, expect, it, vi } from 'vite-plus/test'; +import { drizzlePlugin } from '../../src/plugin-drizzle.ts'; +import * as runtimeDb from '../../src/runtime/db/index.ts'; +import { envContext } from '../../src/runtime/env/raw.ts'; +import { withRuntimeEnv } from '../../src/runtime/env/runtime.ts'; +import { getHookHandler } from '../helpers/vite-hooks.ts'; + +const roots: Array = []; +const profile = sqliteTable('profile', { + id: integer('id').primaryKey(), + name: text('name').notNull(), +}); + +afterEach(() => { + for (const root of roots.splice(0)) { + rmSync(root, { recursive: true, force: true }); + } +}); + +async function loadGeneratedDb(bindingName: string): Promise { + const root = mkdtempSync(join(tmpdir(), 'void-db-session-')); + roots.push(root); + const plugin = drizzlePlugin(root, false, 'sqlite', { + d1BindingName: bindingName, + forceVirtualDb: true, + }); + const load = getHookHandler(plugin.load) as (id: string) => string; + const path = join(root, 'db.mjs'); + writeFileSync( + path, + load('\0virtual:void-db') + .replace( + JSON.stringify(fileURLToPath(import.meta.resolve('#self/_env'))), + JSON.stringify(new URL('../../src/runtime/env/runtime.ts', import.meta.url).href), + ) + .replace( + JSON.stringify(fileURLToPath(import.meta.resolve('#self/runtime/db-session'))), + JSON.stringify(new URL('../../src/runtime/db/session.ts', import.meta.url).href), + ), + ); + return import(pathToFileURL(path).href); +} + +// Miniflare executes real D1 SQL but does not provide controllable replica lag. +// Model only the session boundary here, as in auth-d1-database.test.ts: +// PRIMARY advances after a write, REPLICA stays at the original profile. +// These opaque tokens belong to this fixture, not to Miniflare's Sessions API. +const BEFORE_SAVE = 'test-bookmark-001-before-save'; +const AFTER_SAVE = 'test-bookmark-002-after-save'; + +function laggingBinding(primary: D1Database, replica: D1Database) { + const withSession = vi.fn((start: string = 'first-unconstrained') => { + if (!['first-unconstrained', 'first-primary', BEFORE_SAVE, AFTER_SAVE].includes(start)) { + throw new Error(`Unexpected test session start: ${start}`); + } + let fresh = start === 'first-primary' || start === AFTER_SAVE; + let bookmark: string | null = start === BEFORE_SAVE || start === AFTER_SAVE ? start : null; + const session = { + getBookmark: () => bookmark, + prepare(sql: string) { + const write = /^update\b/i.test(sql); + if (!write && !/^select\b/i.test(sql)) { + throw new Error(`Unexpected test query: ${sql}`); + } + function statement(params: Array) { + // Choose at execution time, so a session's reads observe its previous writes. + async function execute(method: 'run' | 'raw') { + const database = write || fresh ? primary : replica; + const prepared = database.prepare(sql).bind(...params); + const result = await prepared[method](); + if (write) { + fresh = true; + } + bookmark = fresh ? AFTER_SAVE : BEFORE_SAVE; + return result; + } + return { + bind: (...values: Array) => statement(values), + run: () => execute('run'), + raw: () => execute('raw'), + }; + } + return statement([]); + }, + } as unknown as D1DatabaseSession; + return session; + }); + // Only the D1 operations used by this reproduction are implemented. + return { binding: { withSession } as unknown as D1Database, withSession }; +} + +const clients = [ + { module: 'runtime', bindingName: 'DB' }, + { module: 'generated', bindingName: 'DB' }, + { module: 'generated', bindingName: 'APP_DB' }, +]; + +describe('D1 cross-request bookmark reproduction (#783)', () => { + it.each(clients)( + '$module / $bindingName: preserves read-your-writes at the next request', + async (testCase) => { + const miniflare = new Miniflare( + convertV4MiniflareOptions({ + modules: true, + script: 'export default { fetch() { return new Response("ok") } }', + d1Databases: ['PRIMARY', 'REPLICA'], + }), + ); + try { + const primary = await miniflare.getD1Database('PRIMARY'); + const replica = await miniflare.getD1Database('REPLICA'); + for (const database of [primary, replica]) { + await database + .prepare('CREATE TABLE profile (id INTEGER PRIMARY KEY, name TEXT NOT NULL)') + .run(); + await database.prepare("INSERT INTO profile VALUES (1, 'Old name')").run(); + } + const { binding, withSession } = laggingBinding( + primary as unknown as D1Database, + replica as unknown as D1Database, + ); + const { db, createDb, setSession, getBookmark } = + testCase.module === 'runtime' ? runtimeDb : await loadGeneratedDb(testCase.bindingName); + const env = { [testCase.bindingName]: binding }; + const readProfile = () => db.select().from(profile); + + const app = new Hono(); + app.use(async (c, next) => { + const incoming = c.req.header('X-DB-Bookmark'); + setSession(incoming === undefined ? 'first-unconstrained' : { bookmark: incoming }); + await next(); + const bookmark = getBookmark(); + if (bookmark) { + c.header('X-DB-Bookmark', bookmark); + } + }); + // Request A: POST /profile. The write and subsequent read share a session. + app.post('/profile', async (c) => { + expect(await readProfile()).toEqual([{ id: 1, name: 'Old name' }]); + await db.update(profile).set({ name: 'Alice' }).run(); + expect(await readProfile()).toEqual([{ id: 1, name: 'Alice' }]); + return c.redirect('/profile', 303); + }); + app.get('/profile', async (c) => c.json(await readProfile())); + const saved = await withRuntimeEnv(env, () => app.request('/profile', { method: 'POST' })); + expect(saved.status).toBe(303); + expect(saved.headers.get('Location')).toBe('/profile'); + expect(withSession).toHaveBeenCalledExactlyOnceWith('first-unconstrained'); + const bookmark = saved.headers.get('X-DB-Bookmark'); + expect(bookmark).toBe(AFTER_SAVE); + + // Request B: GET /profile after the redirect should observe the saved profile. + const loaded = await withRuntimeEnv(env, () => + app.request('/profile', { + headers: { 'X-DB-Bookmark': bookmark! }, + }), + ); + expect(await loaded.json()).toEqual([{ id: 1, name: 'Alice' }]); + expect(withSession.mock.calls.map(([start]) => start)).toEqual([ + 'first-unconstrained', + AFTER_SAVE, + ]); + + // Control: starting on primary avoids the stale read. + expect( + await createDb(binding, { session: 'first-primary' }).select().from(profile), + ).toEqual([{ id: 1, name: 'Alice' }]); + + // Explicit clients accept the same carried bookmark without casts. + const explicit = createDb(binding, { session: { bookmark: bookmark! } }); + expect(getBookmark(explicit)).toBe(AFTER_SAVE); + expect(await explicit.select().from(profile)).toEqual([{ id: 1, name: 'Alice' }]); + expect(withSession).toHaveBeenLastCalledWith(AFTER_SAVE); + + // Neither control secretly caught up the stale replica. + expect(await withRuntimeEnv(env, readProfile)).toEqual([{ id: 1, name: 'Old name' }]); + } finally { + await miniflare.dispose(); + } + }, + 30_000, + ); +}); + +describe.each(clients)('$module / $bindingName session lifecycle', (testCase) => { + async function loadClient() { + return testCase.module === 'runtime' ? runtimeDb : loadGeneratedDb(testCase.bindingName); + } + + function mockBinding() { + const withSession = vi.fn((start: string) => ({ + getBookmark: vi.fn((): string | null => + start === 'first-unconstrained' || start === 'first-primary' ? null : start, + ), + })); + return { withSession, env: { [testCase.bindingName]: { withSession } } }; + } + + it.each([ + 'first-primray', + AFTER_SAVE, + null, + 123, + {}, + { bookmark: null }, + { bookmark: undefined }, + { bookmark: 123 }, + ])('rejects invalid session input %j before opening a session', async (input) => { + const { createDb, setSession } = await loadClient(); + const { withSession, env } = mockBinding(); + const start = input as runtimeDb.DbSessionStart; + withRuntimeEnv(env, () => { + expect(() => setSession(start)).toThrow('db: setSession()'); + expect(() => + createDb(env[testCase.bindingName] as unknown as D1Database, { session: start }), + ).toThrow('db: createDb()'); + }); + expect(withSession).not.toHaveBeenCalled(); + }); + + it('accepts opaque bookmark objects and snapshots their value', async () => { + const { db, createDb, getBookmark, setSession } = await loadClient(); + const { withSession, env } = mockBinding(); + const start = { bookmark: 'first-future-opaque-bookmark' }; + withRuntimeEnv(env, () => { + setSession(start); + start.bookmark = AFTER_SAVE; + void db.select; + setSession({ bookmark: 'first-future-opaque-bookmark' }); + expect(getBookmark()).toBe('first-future-opaque-bookmark'); + }); + expect(withSession).toHaveBeenCalledExactlyOnceWith('first-future-opaque-bookmark'); + createDb(env[testCase.bindingName] as unknown as D1Database, { session: { bookmark: '' } }); + expect(withSession).toHaveBeenLastCalledWith(''); + }); + + it('does not open a D1 session when middleware configures or inspects an unused db', async () => { + const { db, getBookmark, setSession } = await loadClient(); + const { withSession, env } = mockBinding(); + withRuntimeEnv(env, () => { + expect(getBookmark()).toBeNull(); + expect(getBookmark(db)).toBeNull(); + setSession({ bookmark: AFTER_SAVE }); + expect(getBookmark()).toBeNull(); + }); + expect(withSession).not.toHaveBeenCalled(); + expect(withRuntimeEnv(env, getBookmark)).toBeNull(); + }); + + const probes = [ + { name: 'Promise.resolve', read: (db: typeof runtimeDb.db) => Promise.resolve(db) }, + { + name: 'await', + read: async (db: typeof runtimeDb.db) => { + await db; + }, + }, + { + name: 'JSON.stringify', + read: (db: typeof runtimeDb.db) => { + JSON.stringify(db); + }, + }, + { + name: 'Object.prototype.toString', + read: (db: typeof runtimeDb.db) => { + Object.prototype.toString.call(db); + }, + }, + { + name: 'inspect', + read: (db: typeof runtimeDb.db) => { + inspect(db); + }, + }, + { + name: 'custom inspection hook', + read: (db: typeof runtimeDb.db) => { + Reflect.get(db, inspect.custom); + }, + }, + ]; + + it.each(probes)('keeps $name probes from opening or locking the session', async ({ read }) => { + const { db, getBookmark, setSession } = await loadClient(); + const { withSession, env } = mockBinding(); + await withRuntimeEnv(env, async () => { + setSession({ bookmark: BEFORE_SAVE }); + await read(db); + expect(withSession).not.toHaveBeenCalled(); + expect(getBookmark()).toBeNull(); + setSession({ bookmark: AFTER_SAVE }); + void db.select; + await read(db); + expect(getBookmark()).toBe(AFTER_SAVE); + }); + expect(withSession).toHaveBeenCalledExactlyOnceWith(AFTER_SAVE); + }); + + it('allows promise and inspection probes without request bindings', async () => { + const { db } = await loadClient(); + for (const { read } of probes) { + await read(db); + } + }); + + it.each(['select', 'batch', '$client'])( + 'still opens the session when reading %s', + async (prop) => { + const { db, setSession } = await loadClient(); + const { withSession, env } = mockBinding(); + withRuntimeEnv(env, () => { + void Reflect.get(db, prop); + expect(() => setSession({ bookmark: AFTER_SAVE })).toThrow('createDb'); + }); + expect(withSession).toHaveBeenCalledExactlyOnceWith('first-unconstrained'); + }, + ); + + it.each([AFTER_SAVE, 'first-primary'] as const)( + 'preserves configured %s when nested middleware supplies no bookmark before db use', + async (start) => { + const { db, setSession } = await loadClient(); + const { withSession, env } = mockBinding(); + withRuntimeEnv(env, () => { + setSession(start === 'first-primary' ? start : { bookmark: start }); + setSession('first-unconstrained'); + void db.select(); + }); + expect(withSession).toHaveBeenCalledExactlyOnceWith(start); + }, + ); + + it.each([ + [AFTER_SAVE, BEFORE_SAVE], + [BEFORE_SAVE, AFTER_SAVE], + [AFTER_SAVE, 'first-primary'], + ['first-primary', AFTER_SAVE], + ] as const)('still replaces unopened %s with %s', async (start, replacement) => { + const { db, setSession } = await loadClient(); + const { withSession, env } = mockBinding(); + withRuntimeEnv(env, () => { + setSession(start === 'first-primary' ? start : { bookmark: start }); + setSession(replacement === 'first-primary' ? replacement : { bookmark: replacement }); + void db.select(); + }); + expect(withSession).toHaveBeenCalledExactlyOnceWith(replacement); + }); + + it('rejects stronger requirements after db is used and keeps the original session', async () => { + const { db, getBookmark, setSession } = await loadClient(); + const { withSession, env } = mockBinding(); + withRuntimeEnv(env, () => { + setSession({ bookmark: BEFORE_SAVE }); + void db.select(); + expect(() => setSession({ bookmark: AFTER_SAVE })).toThrow('createDb'); + expect(() => setSession('first-primary')).toThrow('createDb'); + expect(getBookmark()).toBe(BEFORE_SAVE); + expect(getBookmark(db)).toBe(BEFORE_SAVE); + withSession.mock.results[0]!.value.getBookmark.mockReturnValue(AFTER_SAVE); + expect(() => setSession({ bookmark: 'test-bookmark-003-newer' })).toThrow('createDb'); + expect(() => setSession('first-primary')).toThrow('createDb'); + expect(getBookmark()).toBe(AFTER_SAVE); + }); + expect(withSession).toHaveBeenCalledExactlyOnceWith(BEFORE_SAVE); + }); + + it.each(['first-unconstrained', 'first-primary', BEFORE_SAVE] as const)( + 'preserves an opened %s session when repeated middleware requirements are satisfied', + async (start) => { + const { db, getBookmark, setSession } = await loadClient(); + const { withSession, env } = mockBinding(); + withRuntimeEnv(env, () => { + setSession(start === BEFORE_SAVE ? { bookmark: start } : start); + void db.select(); + setSession(start === BEFORE_SAVE ? { bookmark: start } : start); + setSession('first-unconstrained'); + const session = withSession.mock.results[0]!.value; + // D1 advances the bookmark as queries complete. Compare against that + // progress, not just the session's original starting value. + session.getBookmark.mockReturnValue(AFTER_SAVE); + setSession({ bookmark: BEFORE_SAVE }); + setSession({ bookmark: AFTER_SAVE }); + setSession('first-unconstrained'); + setSession(start === BEFORE_SAVE ? { bookmark: start } : start); + expect(getBookmark()).toBe(AFTER_SAVE); + expect(getBookmark(db)).toBe(AFTER_SAVE); + void db.select(); + }); + expect(withSession).toHaveBeenCalledExactlyOnceWith(start); + }, + ); + + it('rejects a bookmark requirement on an opened session with no observed bookmark', async () => { + const { db, setSession, getBookmark } = await loadClient(); + const { withSession, env } = mockBinding(); + withRuntimeEnv(env, () => { + void db.select(); + setSession('first-unconstrained'); + expect(() => setSession({ bookmark: BEFORE_SAVE })).toThrow('createDb'); + expect(() => setSession('first-primary')).toThrow('createDb'); + expect(getBookmark()).toBeNull(); + }); + expect(withSession).toHaveBeenCalledExactlyOnceWith('first-unconstrained'); + }); + + it('keeps overlapping requests and explicit clients independent', async () => { + const { db, createDb, getBookmark, setSession } = await loadClient(); + const { withSession, env } = mockBinding(); + let resume!: () => void; + const resumed = new Promise((resolve) => { + resume = resolve; + }); + const first = withRuntimeEnv(env, async () => { + setSession({ bookmark: AFTER_SAVE }); + void db.select(); + await resumed; + expect(getBookmark()).toBe(AFTER_SAVE); + setSession({ bookmark: BEFORE_SAVE }); + expect(getBookmark()).toBe(AFTER_SAVE); + }); + try { + await withRuntimeEnv(env, async () => { + setSession({ bookmark: BEFORE_SAVE }); + const explicit = createDb(env[testCase.bindingName] as unknown as D1Database, { + session: { bookmark: AFTER_SAVE }, + }); + await Promise.resolve(); + expect(getBookmark(explicit)).toBe(AFTER_SAVE); + expect(getBookmark()).toBeNull(); + void db.select(); + expect(getBookmark()).toBe(BEFORE_SAVE); + }); + } finally { + resume(); + await first; + } + withRuntimeEnv(env, () => { + expect(getBookmark()).toBeNull(); + void db.select(); + expect(getBookmark()).toBeNull(); + }); + expect(withSession.mock.calls.map(([start]) => start)).toEqual([ + AFTER_SAVE, + AFTER_SAVE, + BEFORE_SAVE, + 'first-unconstrained', + ]); + }); + + it.each([false, true])( + 'explains default-session helpers outside request scope (bindings available: %s)', + async (withBindings) => { + const { db, getBookmark, setSession } = await loadClient(); + const { withSession, env } = mockBinding(); + const guidance = + 'In framework-owned handlers, use createDb(binding, { session }) and getBookmark(client).'; + const check = () => { + for (const start of [{ bookmark: AFTER_SAVE }, 'first-unconstrained'] as const) { + expect(() => setSession(start)).toThrow( + `db: setSession() requires Void's request scope. ${guidance}`, + ); + } + for (const inspect of [() => getBookmark(), () => getBookmark(db)]) { + expect(inspect).toThrow(`db: getBookmark() requires Void's request scope. ${guidance}`); + } + }; + if (withBindings) { + envContext.run(env, check); + } else { + check(); + } + expect(withSession).not.toHaveBeenCalled(); + }, + ); + + it('reads an explicit client bookmark without a Void request scope', async () => { + const { createDb, getBookmark } = await loadClient(); + const { withSession, env } = mockBinding(); + const client = createDb(env[testCase.bindingName] as unknown as D1Database, { + session: { bookmark: AFTER_SAVE }, + }); + expect(getBookmark(client)).toBe(AFTER_SAVE); + expect(withSession).toHaveBeenCalledExactlyOnceWith(AFTER_SAVE); + }); + + it('rejects bookmark inspection of a client backed by a bare D1 binding', async () => { + const { getBookmark } = await loadClient(); + const { env } = mockBinding(); + const client = drizzle(env[testCase.bindingName] as unknown as D1Database); + expect(() => getBookmark(client)).toThrow('requires a D1 session-backed client'); + }); +}); + +it('starts each published db access on primary outside a Void request scope', () => { + const withSession = vi.fn(() => ({ getBookmark: () => null })); + envContext.run({ DB: { withSession } }, () => { + void runtimeDb.db.select; + void runtimeDb.db.select; + void runtimeDb.db.select; + }); + expect(withSession.mock.calls).toEqual([['first-primary'], ['first-primary'], ['first-primary']]); + expect(new Set(withSession.mock.results.map((result) => result.value)).size).toBe(3); +}); diff --git a/packages/void/test/unit/db-session.typecheck.ts b/packages/void/test/unit/db-session.typecheck.ts index 6a02f6c92..af43f5b8b 100644 --- a/packages/void/test/unit/db-session.typecheck.ts +++ b/packages/void/test/unit/db-session.typecheck.ts @@ -1,5 +1,11 @@ +import { integer, sqliteTable } from 'drizzle-orm/sqlite-core'; import { expectTypeOf } from 'vite-plus/test'; -import { createDb, type DrizzleD1Database } from '../../src/runtime/db/index.ts'; +import { + createDb, + getBookmark, + setSession, + type DrizzleD1Database, +} from '../../src/runtime/db/index.ts'; declare const binding: D1Database; @@ -7,7 +13,34 @@ expectTypeOf(createDb(binding, { session: 'first-primary' })).toEqualTypeOf(); +expectTypeOf(getBookmark(createDb(binding))).toEqualTypeOf(); + +const schema = { profile: sqliteTable('profile', { id: integer('id').primaryKey() }) }; +declare const schemaClient: DrizzleD1Database; +expectTypeOf(getBookmark(schemaClient)).toEqualTypeOf(); + +// @ts-expect-error Session starts must be constraints or bookmark objects. +createDb(binding, { session: 123 }); +// @ts-expect-error Unknown constraint literals are not bookmarks. +setSession('first-primray'); +// @ts-expect-error Unknown constraint literals are not bookmarks. +createDb(binding, { session: 'first-primray' }); +// @ts-expect-error Bare bookmark strings must be wrapped. +setSession(bookmark); +// @ts-expect-error Bare bookmark strings must be wrapped. +createDb(binding, { session: bookmark }); +// @ts-expect-error Missing bookmarks need an explicit fallback. +setSession({ bookmark: undefined }); +// @ts-expect-error Null is not a bookmark. +createDb(binding, { session: { bookmark: null } }); // @ts-expect-error Session options do not accept arbitrary settings. createDb(binding, { primary: true }); diff --git a/packages/void/test/unit/deploy-artifact-preparation.test.ts b/packages/void/test/unit/deploy-artifact-preparation.test.ts index 522101d00..a12759bb7 100644 --- a/packages/void/test/unit/deploy-artifact-preparation.test.ts +++ b/packages/void/test/unit/deploy-artifact-preparation.test.ts @@ -95,6 +95,21 @@ it('omits empty configured Worker vars', async () => { expect(result.artifact.manifest).not.toHaveProperty('vars'); expect(fetch).not.toHaveBeenCalled(); }); +it.each([undefined, false, true])( + 'packages the configured D1 replication opt-in %s', + async (readReplication) => { + writeFileSync( + join(root, 'void.config.ts'), + `export default { database: { readReplication: ${readReplication}, casing: 'snake_case' }, inference: { bindings: { db: true } } };`, + ); + const { artifact } = await prepareDeployArtifact(root); + expect(artifact.manifest.bindings.d1).toEqual(['DB']); + expect(artifact.manifest.database).toEqual( + readReplication !== undefined ? { readReplication } : undefined, + ); + expect(fetch).not.toHaveBeenCalled(); + }, +); it('fails when Worker build output is missing', async () => { rmSync(join(root, 'dist/server'), { recursive: true }); await expect(prepareDeployArtifact(root)).rejects.toThrow('Worker output not found'); diff --git a/packages/void/test/unit/package.test.ts b/packages/void/test/unit/package.test.ts index b671a1259..15700e77c 100644 --- a/packages/void/test/unit/package.test.ts +++ b/packages/void/test/unit/package.test.ts @@ -38,6 +38,32 @@ afterEach(() => { }); describe('worker packaging', () => { + it.each([undefined, false, true])( + 'publishes explicit D1 replication settings: %s', + (readReplication) => { + for (const dialect of ['sqlite', 'postgresql', 'mysql'] as const) { + for (const needsD1 of [false, true]) { + const bindings = makeBindings({ needsD1 }); + for (const manifest of [ + buildVoidManifest({ bindings, dialect, database: { readReplication } }), + buildFrameworkManifest({ + frameworkName: 'nuxt', + workerMain: 'index.mjs', + bindings, + dialect, + database: { readReplication }, + }), + ]) { + if (needsD1 && dialect === 'sqlite' && readReplication !== undefined) { + expect(manifest.database).toEqual({ readReplication }); + } else { + expect(manifest).not.toHaveProperty('database'); + } + } + } + } + }, + ); it.each([true, false])('only includes the storage hint when R2 is used: %s', (needsR2) => { const bindings = makeBindings({ needsR2 }); for (const manifest of [ diff --git a/packages/void/test/unit/plugin-drizzle.test.ts b/packages/void/test/unit/plugin-drizzle.test.ts index 05bc5b0b5..9de55332b 100644 --- a/packages/void/test/unit/plugin-drizzle.test.ts +++ b/packages/void/test/unit/plugin-drizzle.test.ts @@ -1,7 +1,7 @@ -import { mkdtempSync, mkdirSync, readFileSync, rmSync, writeFileSync } from 'node:fs'; +import { mkdtempSync, mkdirSync, readFileSync, rmSync, symlinkSync, writeFileSync } from 'node:fs'; import { tmpdir } from 'node:os'; import { dirname, join } from 'node:path'; -import { pathToFileURL } from 'node:url'; +import { fileURLToPath, pathToFileURL } from 'node:url'; import { int, mysqlTable } from 'drizzle-orm/mysql-core'; import { pgTable, serial } from 'drizzle-orm/pg-core'; import { afterEach, beforeEach, describe, expect, it, vi } from 'vite-plus/test'; @@ -183,6 +183,11 @@ export function drizzle(config) { /^import \{ ((?:getRuntimeBinding|getRuntimeRequestResource|requireRuntimeBinding)[^}]*) \} from .*;$/m, (_match, imports: string) => `import { ${imports} } from ${JSON.stringify(envUrl)};`, ); + code = code.replace( + /^import \{ (createD1SessionAccess[^}]*) \} from .*;$/m, + (_match, imports: string) => + `import { ${imports} } from ${JSON.stringify(new URL('../../src/runtime/db/session.ts', import.meta.url).href)};`, + ); writeFileSync(join(root, 'virtual-db.mjs'), code); writeFileSync( join(root, 'harness.mjs'), @@ -425,11 +430,110 @@ describe('drizzlePlugin db typing generation', () => { const dts = readFileSync(join(root, '.void', 'db.d.ts'), 'utf8'); expect(dts).toContain('import type { DrizzleD1Database } from "void/_db";'); - expect(dts).toContain('options?: { session?: D1SessionConstraint }'); + expect(dts).toContain('options?: { session?: DbSessionStart }'); + expect(dts).toContain('export type { DbSessionStart } from "void/_db";'); + expect(dts).toContain('export declare function setSession('); + expect(dts).toContain('export declare function getBookmark;'); expect(dts).not.toContain('void/_db-pg'); }); + it.each([false, true])('typechecks generated D1 session helpers (schema: %s)', (schema) => { + const root = createProjectRoot({ schema }); + const sqliteCore = JSON.stringify( + fileURLToPath(import.meta.resolve('drizzle-orm/sqlite-core')), + ); + if (schema) { + writeFileSync( + join(root, 'db/schema.ts'), + `import { integer, sqliteTable } from ${sqliteCore}; +export const profile = sqliteTable('profile', { id: integer('id').primaryKey() }); +`, + ); + } + drizzlePlugin(root, false, 'sqlite', { forceVirtualDb: true }); + symlinkSync( + fileURLToPath(new URL('../../../../node_modules', import.meta.url)), + join(root, 'node_modules'), + ); + writeFileSync(join(root, 'package.json'), '{"type":"module"}'); + const fixture = join(root, 'session-inputs.ts'); + writeFileSync( + fixture, + ` +import { integer, sqliteTable } from ${sqliteCore}; +import { db, createDb, setSession, getBookmark, type DbSessionStart, type DrizzleD1Database } from 'void/db'; +declare const binding: D1Database; +declare const bookmark: string; +const start: DbSessionStart = { bookmark }; +setSession(start); +setSession('first-primary'); +setSession('first-unconstrained'); +createDb(binding, { session: start }); +createDb(binding, { session: 'first-primary' }); +createDb(binding, { session: 'first-unconstrained' }); +const returned: string | null = getBookmark(createDb(binding)); +void returned; +const otherSchema = { audit: sqliteTable('audit', { sequence: integer('sequence').primaryKey() }) }; +declare const otherClient: DrizzleD1Database; +declare const schemalessClient: DrizzleD1Database; +const bookmarks: Array = [ + getBookmark(), + getBookmark(undefined), + getBookmark(db), + getBookmark(schemalessClient), + getBookmark(otherClient), +]; +void bookmarks; +// @ts-expect-error A raw D1 binding is not a Drizzle client. +getBookmark(binding); +// @ts-expect-error Constraint typos must not compile. +setSession('first-primray'); +// @ts-expect-error Constraint typos must not compile. +createDb(binding, { session: 'first-primray' }); +// @ts-expect-error Bookmark strings require a wrapper. +setSession(bookmark); +// @ts-expect-error Bookmark strings require a wrapper. +createDb(binding, { session: bookmark }); +// @ts-expect-error Missing bookmarks require an explicit fallback. +setSession({ bookmark: undefined }); +// @ts-expect-error Null is not a bookmark. +createDb(binding, { session: { bookmark: null } }); +`, + ); + writeFileSync( + join(root, 'tsconfig.json'), + JSON.stringify({ + compilerOptions: { + noEmit: true, + strict: true, + skipLibCheck: true, + target: 'ES2022', + module: 'NodeNext', + moduleResolution: 'NodeNext', + allowImportingTsExtensions: true, + types: ['node', '@cloudflare/workers-types'], + typeRoots: [join(root, 'node_modules/@types'), join(root, 'node_modules')], + paths: { + 'void/db': [join(root, '.void/db.d.ts')], + 'void/_db': [fileURLToPath(new URL('../../src/runtime/db/index.ts', import.meta.url))], + '@schema': [join(root, 'db/schema.ts')], + }, + }, + files: [fixture], + }), + ); + const tsc = join( + dirname(fileURLToPath(import.meta.resolve('typescript/package.json'))), + 'bin/tsc', + ); + const result = spawnSync(process.execPath, [tsc, '--project', join(root, 'tsconfig.json')], { + encoding: 'utf8', + timeout: 30_000, + }); + expect(result.status, `${result.error ?? ''}${result.stdout}${result.stderr}`).toBe(0); + }); + it('keeps postgres-only types off the shared db runtime entry', () => { const root = createProjectRoot(); @@ -915,8 +1019,8 @@ export const requestPoolOptions = (env) => expect(resolved).toBe('\0virtual:void-db'); const code = load.call({}, '\0virtual:void-db'); - expect(code).toContain('requireRuntimeBinding("myapp_db")'); - expect(code).not.toContain('requireRuntimeBinding("DB")'); + expect(code).toContain('createD1SessionAccess("myapp_db")'); + expect(code).not.toContain('createD1SessionAccess("DB")'); }); it('can keep void/db off tsconfig paths for metaframeworks that materialize aliases', () => { @@ -991,3 +1095,4 @@ export const requestPoolOptions = (env) => expect(warnMock).not.toHaveBeenCalled(); }); }); +import { spawnSync } from 'node:child_process'; diff --git a/packages/void/test/unit/provision-fixtures.ts b/packages/void/test/unit/provision-fixtures.ts index 478b0480a..addcca93b 100644 --- a/packages/void/test/unit/provision-fixtures.ts +++ b/packages/void/test/unit/provision-fixtures.ts @@ -17,10 +17,8 @@ import { toSlash } from 'pathslash'; import { afterEach, beforeEach, vi } from 'vite-plus/test'; import { operationFixture } from '../../../deploy-cloudflare/test/operation-fixture.ts'; import { - provisionCloudflareResources as provisionCloudflareResourcesImpl, type DesiredResourceSet, type ProvisionLogger, - type ProvisionOptions, type RunWrangler, type RunWranglerResult, type WranglerCallContext, @@ -176,21 +174,7 @@ export function uuid(seed: string): string { export const emptyDesired: DesiredResourceSet = { kvNamespaces: [], queues: [] }; -/** - * `provisionCloudflareResources` with the D1 replication update stubbed whenever the runner is - * injected. Tests of the real API path call the implementation directly. - */ -export function provisionCloudflareResources( - ...args: Parameters -): ReturnType { - const [root, configPath, desired, opts] = args; - const replicationStub: ProvisionOptions['enableD1ReadReplication'] = async () => {}; - return provisionCloudflareResourcesImpl(root, configPath, desired, { - ...opts, - enableD1ReadReplication: - opts?.enableD1ReadReplication ?? (opts?.runWrangler ? replicationStub : undefined), - }); -} +export { provisionCloudflareResources } from '../../src/cli/provision.ts'; /** Poll until `pid` no longer exists, or `timeoutMs` passes. */ export async function waitForDead(pid: number, timeoutMs: number): Promise { diff --git a/packages/void/test/unit/provision-resources.test.ts b/packages/void/test/unit/provision-resources.test.ts index 23c288a08..307ece4d8 100644 --- a/packages/void/test/unit/provision-resources.test.ts +++ b/packages/void/test/unit/provision-resources.test.ts @@ -8,7 +8,7 @@ import { encodeCloudflareOperation } from '../../src/cli/cloudflare-process.ts'; import { assertNoTransportRedirectInProjectDotenv, checkCloudflareProvisionDrift, - enableD1ReadReplicationViaApi, + setD1ReadReplicationViaApi, normalizeRunWranglerResult, provisionCloudflareResources as provisionCloudflareResourcesImpl, provisionEnv, @@ -39,46 +39,58 @@ const { tmpDir, wranglerPath, setup, readConfig, simulateUpdateConfig } = useProvisionTestDir('resources'); describe('D1 read-replication API', () => { - it('uses the pinned account and endpoint with an authenticated update', async () => { - const requests: Array<{ url: string; init: RequestInit }> = []; - await enableD1ReadReplicationViaApi( - { - accountId: 'account-one', - apiBaseUrl: 'https://api.example.test/client/v4', - databaseId: uuid('db1'), - }, - 'token-one', - async (url, init) => { - requests.push({ url: String(url), init: init! }); - return Response.json({ success: true }); - }, - ); + it.each([true, false])( + 'uses the pinned account and endpoint to set replication to %s', + async (readReplication) => { + const requests: Array<{ url: string; init: RequestInit }> = []; + await setD1ReadReplicationViaApi( + { + accountId: 'account-one', + apiBaseUrl: 'https://api.example.test/client/v4', + databaseId: uuid('db1'), + readReplication, + }, + 'token-one', + async (url, init) => { + requests.push({ url: String(url), init: init! }); + return Response.json({ success: true }); + }, + ); - expect(requests).toHaveLength(1); - expect(requests[0].url).toBe( - `https://api.example.test/client/v4/accounts/account-one/d1/database/${uuid('db1')}`, - ); - expect(requests[0].init.method).toBe('PUT'); - expect(requests[0].init.headers).toEqual({ - Authorization: 'Bearer token-one', - 'Content-Type': 'application/json', - }); - expect(requests[0].init.body).toBe(JSON.stringify({ read_replication: { mode: 'auto' } })); - }); + expect(requests).toHaveLength(1); + expect(requests[0].url).toBe( + `https://api.example.test/client/v4/accounts/account-one/d1/database/${uuid('db1')}`, + ); + expect(requests[0].init.method).toBe('PUT'); + expect(requests[0].init.headers).toEqual({ + Authorization: 'Bearer token-one', + 'Content-Type': 'application/json', + }); + expect(requests[0].init.body).toBe( + JSON.stringify({ + read_replication: { mode: readReplication ? 'auto' : 'disabled' }, + }), + ); + }, + ); - it('fails closed when Cloudflare rejects the replication update', async () => { - const update = enableD1ReadReplicationViaApi( - { - accountId: 'account-one', - apiBaseUrl: 'https://api.example.test/client/v4', - databaseId: uuid('db1'), - }, - 'secret-token', - async () => Response.json({ success: false }, { status: 403 }), - ); + it.each([true, false])( + 'fails closed when Cloudflare rejects replication %s', + async (readReplication) => { + const update = setD1ReadReplicationViaApi( + { + accountId: 'account-one', + apiBaseUrl: 'https://api.example.test/client/v4', + databaseId: uuid('db1'), + readReplication, + }, + 'secret-token', + async () => Response.json({ success: false }, { status: 403 }), + ); - await expect(update).rejects.toThrow(/D1 Edit permission/); - }); + await expect(update).rejects.toThrow(/D1 Edit permission/); + }, + ); }); describe('provisionCloudflareResources — typed operations', () => { @@ -189,6 +201,70 @@ describe('provisionCloudflareResources — typed operations', () => { }); describe('provisionCloudflareResources — read-first', () => { + for (const state of ['created', 'reused', 'adopted', 'collision'] as const) { + it.each([undefined, false, true])( + `${state} D1 applies only explicitly configured replication: %s`, + async (readReplication) => { + const databaseId = uuid('db1'); + setup( + JSON.stringify({ + d1_databases: + state === 'reused' + ? [{ binding: 'DB', database_name: 'myapp-db', database_id: databaseId }] + : [], + }), + ); + vi.stubEnv('CLOUDFLARE_API_TOKEN', undefined); + const fetchMock = vi.fn(); + vi.stubGlobal('fetch', fetchMock); + const setD1ReadReplication = vi.fn(async () => {}); + const operations: Array = []; + const run: RunWrangler = (operation) => { + operations.push(operation); + switch (operation.type) { + case 'd1.list': + return { + ...ok(), + data: state === 'adopted' ? [{ name: 'myapp-db', uuid: databaseId }] : [], + }; + case 'd1.create': + if (state === 'collision') { + return { ...fail('', 'already exists'), errorCode: 7502 }; + } + simulateUpdateConfig('d1_databases', { + binding: 'DB', + database_name: 'myapp-db', + database_id: databaseId, + }); + return ok(); + case 'd1.info': + return { ...ok(), data: { name: 'myapp-db', uuid: databaseId } }; + default: + throw new Error(`Unexpected operation: ${operation.type}`); + } + }; + const result = await provisionCloudflareResourcesImpl( + tmpDir, + wranglerPath, + { ...emptyDesired, d1: { binding: 'DB', name: 'myapp-db', readReplication } }, + { runWrangler: run, log: silentLog, setD1ReadReplication }, + ); + expect(result.resources[0].action).toBe(state === 'collision' ? 'adopted' : state); + expect(setD1ReadReplication).toHaveBeenCalledTimes(readReplication !== undefined ? 1 : 0); + if (readReplication !== undefined) { + expect(setD1ReadReplication).toHaveBeenCalledWith({ + databaseId, + accountId: AMBIENT_ACCOUNT, + apiBaseUrl: 'https://api.cloudflare.com/client/v4', + readReplication, + }); + } + expect(operations.some((operation) => operation.type === 'auth.token')).toBe(false); + expect(fetchMock).not.toHaveBeenCalled(); + }, + ); + } + it('enables D1 replication for a reused database before continuing the deploy', async () => { setup( JSON.stringify({ @@ -201,11 +277,11 @@ describe('provisionCloudflareResources — read-first', () => { await provisionCloudflareResources( tmpDir, wranglerPath, - { ...emptyDesired, d1: { binding: 'DB', name: 'myapp-db' } }, + { ...emptyDesired, d1: { binding: 'DB', name: 'myapp-db', readReplication: true } }, { runWrangler: run, log: silentLog, - enableD1ReadReplication: async (context: unknown) => { + setD1ReadReplication: async (context: unknown) => { enabled.push(context); }, }, @@ -216,6 +292,7 @@ describe('provisionCloudflareResources — read-first', () => { databaseId: uuid('db1'), accountId: AMBIENT_ACCOUNT, apiBaseUrl: 'https://api.cloudflare.com/client/v4', + readReplication: true, }, ]); }); @@ -235,7 +312,7 @@ describe('provisionCloudflareResources — read-first', () => { await provisionCloudflareResourcesImpl( tmpDir, wranglerPath, - { ...emptyDesired, d1: { binding: 'DB', name: 'myapp-db' } }, + { ...emptyDesired, d1: { binding: 'DB', name: 'myapp-db', readReplication: true } }, { runWrangler: run, apiToken: 'test-token', log: silentLog }, ); expect(requests).toEqual([ @@ -254,7 +331,7 @@ describe('provisionCloudflareResources — read-first', () => { const error = await provisionCloudflareResourcesImpl( tmpDir, wranglerPath, - { ...emptyDesired, d1: { binding: 'DB', name: 'myapp-db' } }, + { ...emptyDesired, d1: { binding: 'DB', name: 'myapp-db', readReplication: true } }, { runWrangler: run, log: silentLog }, ).catch((e: unknown) => e as Error); diff --git a/packages/void/vite.config.ts b/packages/void/vite.config.ts index faab5bdce..a1773edb5 100644 --- a/packages/void/vite.config.ts +++ b/packages/void/vite.config.ts @@ -26,6 +26,7 @@ function injectWorkersTypesReferencePlugin() { // Patch declaration entrypoints whose source uses Cloudflare globals. const targets = new Set([ 'runtime/handler/index.d.mts', + 'runtime/db/index.d.mts', 'runtime/env/public.d.mts', 'runtime/sandbox/index.d.mts', 'runtime/sandbox/container.d.mts', @@ -99,6 +100,7 @@ export default defineConfig({ 'src/runtime/env/public.ts', 'src/runtime/env/public-client.ts', 'src/runtime/db/index.ts', + 'src/runtime/db/session.ts', 'src/runtime/db/postgres.ts', 'src/runtime/db/mysql.ts', 'src/runtime/durable/index.ts', diff --git a/platform/packages/api/README.md b/platform/packages/api/README.md index 00598c3ca..e78bcfb38 100644 --- a/platform/packages/api/README.md +++ b/platform/packages/api/README.md @@ -9,6 +9,7 @@ It owns platform auth, projects, deployments, domains, secrets, provisioning, ro - **Platform DB**: PostgreSQL via `HYPERDRIVE.connectionString` - **Public API**: CLI and dashboard-facing JSON routes - **Internal API**: resource resolution and revalidation used by other platform workers +- **Application D1 replication**: provisioning applies explicit `database.readReplication` values from the deploy manifest: true enables replicas and false disables them. Omitted leaves existing replication unchanged; new databases default to off. - **Admin UI**: React pages under `/admin/*`, server-rendered for first load and backed by a private Fate endpoint - **Lifecycle status**: `/admin/status` and its GET JSON endpoint project display fields from the installer-owned PostgreSQL checkpoint. Maintenance admits these routes and administrator sign-in through normal Access and admin authentication. Keep the exception narrow, retain existing Sandbox drain operator access, and keep the Fate POST endpoint and administrator writes gated. - **Self-hosted owner setup**: dashboard OAuth and normal signup share the first-user transaction and configured GitHub identity check. Dashboard login can create the first owner without a CLI login; it never promotes existing users. Void Cloud keeps its existing admin-only login behavior. diff --git a/platform/packages/api/src/provisioning/cloudflare-api.ts b/platform/packages/api/src/provisioning/cloudflare-api.ts index 0dcb9c9a4..f270b61a0 100644 --- a/platform/packages/api/src/provisioning/cloudflare-api.ts +++ b/platform/packages/api/src/provisioning/cloudflare-api.ts @@ -93,21 +93,28 @@ export async function createD1Database( accountId: string, apiToken: string, name: string, + options: { readReplication?: boolean } = {}, ): Promise { return cfFetch(`${CF_API_BASE}/${accountId}/d1/database`, apiToken, { method: 'POST', - body: JSON.stringify({ name, read_replication: { mode: 'auto' } }), + body: JSON.stringify({ + name, + ...(options.readReplication !== undefined && { + read_replication: { mode: options.readReplication ? 'auto' : 'disabled' }, + }), + }), }); } -export async function enableD1ReadReplication( +export async function setD1ReadReplication( accountId: string, apiToken: string, databaseId: string, + readReplication: boolean, ): Promise { await cfFetch(`${CF_API_BASE}/${accountId}/d1/database/${databaseId}`, apiToken, { method: 'PUT', - body: JSON.stringify({ read_replication: { mode: 'auto' } }), + body: JSON.stringify({ read_replication: { mode: readReplication ? 'auto' : 'disabled' } }), }); } diff --git a/platform/packages/api/src/provisioning/provision.ts b/platform/packages/api/src/provisioning/provision.ts index 436b24b61..3e8f5096f 100644 --- a/platform/packages/api/src/provisioning/provision.ts +++ b/platform/packages/api/src/provisioning/provision.ts @@ -11,7 +11,7 @@ import { deleteKVNamespace, deleteQueue, deleteR2Bucket, - enableD1ReadReplication, + setD1ReadReplication, ensureHyperdriveCachingDisabled, getD1DatabaseByName, getKVNamespaceByTitle, @@ -82,15 +82,9 @@ export async function provisionBindings( switch (resource.type) { case 'd1': tasks.push( - ensureResource( - db, - accountId, - apiToken, - projectId, - 'd1', - resource.bindingName, - existing, - ).then((r): WorkerBinding => ({ type: 'd1', name: resource.bindingName, id: r.cf_id })), + ensureResource(db, accountId, apiToken, projectId, 'd1', resource.bindingName, existing, { + readReplication: resource.readReplication, + }).then((r): WorkerBinding => ({ type: 'd1', name: resource.bindingName, id: r.cf_id })), ); break; case 'kv': @@ -112,16 +106,9 @@ export async function provisionBindings( break; case 'r2': tasks.push( - ensureResource( - db, - accountId, - apiToken, - projectId, - 'r2', - resource.bindingName, - existing, + ensureResource(db, accountId, apiToken, projectId, 'r2', resource.bindingName, existing, { locationHint, - ).then((r): WorkerBinding => ({ + }).then((r): WorkerBinding => ({ type: 'r2_bucket', name: resource.bindingName, bucket_name: r.cf_id, @@ -214,15 +201,15 @@ async function ensureResource( type: Resource['type'], bindingName: string, existing: ReadonlyArray, - locationHint?: R2LocationHint, + options: { locationHint?: R2LocationHint; readReplication?: boolean } = {}, ): Promise { // Normalize: R2 bucket names only allow lowercase alphanumeric + hyphens const resourceName = `${projectId}-${bindingName}`.toLowerCase().replaceAll('_', '-'); const found = existing.find((r) => r.type === type && r.binding_name === bindingName); if (found) { - if (type === 'd1') { - await enableD1ReadReplication(accountId, apiToken, found.cf_id); + if (type === 'd1' && options.readReplication !== undefined) { + await setD1ReadReplication(accountId, apiToken, found.cf_id, options.readReplication); } return found; } @@ -233,13 +220,13 @@ async function ensureResource( let createdByUs = false; try { - cfId = await createCfResource(type, accountId, apiToken, resourceName, locationHint); + cfId = await createCfResource(type, accountId, apiToken, resourceName, options); createdByUs = true; } catch (err) { if (isAlreadyExistsError(err)) { cfId = await lookupCfResource(type, accountId, apiToken, resourceName); - if (type === 'd1') { - await enableD1ReadReplication(accountId, apiToken, cfId); + if (type === 'd1' && options.readReplication !== undefined) { + await setD1ReadReplication(accountId, apiToken, cfId, options.readReplication); } } else { throw err; @@ -294,11 +281,13 @@ async function createCfResource( accountId: string, apiToken: string, resourceName: string, - locationHint?: R2LocationHint, + options: { locationHint?: R2LocationHint; readReplication?: boolean }, ): Promise { switch (type) { case 'd1': { - const result = await createD1Database(accountId, apiToken, resourceName); + const result = await createD1Database(accountId, apiToken, resourceName, { + readReplication: options.readReplication, + }); return result.uuid; } case 'kv': { @@ -306,7 +295,7 @@ async function createCfResource( return result.id; } case 'r2': { - const result = await createR2Bucket(accountId, apiToken, resourceName, locationHint); + const result = await createR2Bucket(accountId, apiToken, resourceName, options.locationHint); return result.name; } case 'queue': { diff --git a/platform/packages/api/test/unit/cloudflare-api.test.ts b/platform/packages/api/test/unit/cloudflare-api.test.ts index ffb138068..f305f3c3d 100644 --- a/platform/packages/api/test/unit/cloudflare-api.test.ts +++ b/platform/packages/api/test/unit/cloudflare-api.test.ts @@ -4,7 +4,7 @@ import { cfFetch, createD1Database, createR2Bucket, - enableD1ReadReplication, + setD1ReadReplication, createHyperdrive, ensureHyperdriveCachingDisabled, getDispatchWorkerProxyToken, @@ -133,35 +133,48 @@ describe('Cloudflare API deadlines', () => { describe('createD1Database', () => { afterEach(() => vi.unstubAllGlobals()); - it('enables automatic read replication when creating a database', async () => { - const fetchMock = vi.fn().mockResolvedValue(cfSuccess({ uuid: 'db-uuid', name: 'app-db' })); - vi.stubGlobal('fetch', fetchMock); + it.each([undefined, false, true])( + 'creates a database with readReplication %s', + async (readReplication) => { + const fetchMock = vi.fn().mockResolvedValue(cfSuccess({ uuid: 'db-uuid', name: 'app-db' })); + vi.stubGlobal('fetch', fetchMock); - await createD1Database(ACCOUNT, TOKEN, 'app-db'); + await createD1Database(ACCOUNT, TOKEN, 'app-db', { readReplication }); - expect(fetchMock).toHaveBeenCalledWith( - `https://api.cloudflare.com/client/v4/accounts/${ACCOUNT}/d1/database`, - expect.objectContaining({ - method: 'POST', - body: JSON.stringify({ name: 'app-db', read_replication: { mode: 'auto' } }), - }), - ); - }); + expect(fetchMock).toHaveBeenCalledWith( + `https://api.cloudflare.com/client/v4/accounts/${ACCOUNT}/d1/database`, + expect.objectContaining({ + method: 'POST', + body: JSON.stringify({ + name: 'app-db', + ...(readReplication !== undefined && { + read_replication: { mode: readReplication ? 'auto' : 'disabled' }, + }), + }), + }), + ); + }, + ); - it('enables read replication on an existing database', async () => { - const fetchMock = vi.fn().mockResolvedValue(cfSuccess({})); - vi.stubGlobal('fetch', fetchMock); + it.each([true, false])( + 'sets read replication to %s on an existing database', + async (readReplication) => { + const fetchMock = vi.fn().mockResolvedValue(cfSuccess({})); + vi.stubGlobal('fetch', fetchMock); - await enableD1ReadReplication(ACCOUNT, TOKEN, 'db-uuid'); + await setD1ReadReplication(ACCOUNT, TOKEN, 'db-uuid', readReplication); - expect(fetchMock).toHaveBeenCalledWith( - `https://api.cloudflare.com/client/v4/accounts/${ACCOUNT}/d1/database/db-uuid`, - expect.objectContaining({ - method: 'PUT', - body: JSON.stringify({ read_replication: { mode: 'auto' } }), - }), - ); - }); + expect(fetchMock).toHaveBeenCalledWith( + `https://api.cloudflare.com/client/v4/accounts/${ACCOUNT}/d1/database/db-uuid`, + expect.objectContaining({ + method: 'PUT', + body: JSON.stringify({ + read_replication: { mode: readReplication ? 'auto' : 'disabled' }, + }), + }), + ); + }, + ); }); describe('addQueueConsumer', () => { diff --git a/platform/packages/api/test/unit/provision.test.ts b/platform/packages/api/test/unit/provision.test.ts index deea5595b..3828f37c4 100644 --- a/platform/packages/api/test/unit/provision.test.ts +++ b/platform/packages/api/test/unit/provision.test.ts @@ -5,7 +5,7 @@ import type { DeployManifest, Resource } from '../../src/types.ts'; vi.mock('../../src/provisioning/cloudflare-api', () => ({ createD1Database: vi.fn().mockResolvedValue({ uuid: 'd1-uuid-123', name: 'test-DB' }), - enableD1ReadReplication: vi.fn().mockResolvedValue(undefined), + setD1ReadReplication: vi.fn().mockResolvedValue(undefined), ensureHyperdriveCachingDisabled: vi.fn().mockResolvedValue(undefined), createKVNamespace: vi.fn().mockResolvedValue({ id: 'kv-id-456', title: 'test-KV' }), createR2Bucket: vi.fn().mockResolvedValue({ name: 'test-STORAGE' }), @@ -39,7 +39,7 @@ vi.mock('../../src/db', () => ({ import { getProjectResources, createResource } from '../../src/db.ts'; import { createD1Database, - enableD1ReadReplication, + setD1ReadReplication, ensureHyperdriveCachingDisabled, createKVNamespace, createR2Bucket, @@ -122,7 +122,9 @@ describe('provisionBindings', () => { { projectSlug: PROJECT_SLUG, proxyToken: PROXY_TOKEN }, ); - expect(createD1Database).toHaveBeenCalledWith(ACCOUNT_ID, API_TOKEN, 'proj-abc123-db'); + expect(createD1Database).toHaveBeenCalledWith(ACCOUNT_ID, API_TOKEN, 'proj-abc123-db', { + readReplication: undefined, + }); expect(createResource).toHaveBeenCalledWith(mockDb, PROJECT_ID, 'd1', 'd1-uuid-123', 'DB'); expect(bindings[0]).toEqual({ type: 'd1', name: 'DB', id: 'd1-uuid-123' }); // + 4 always-injected platform bindings @@ -144,7 +146,9 @@ describe('provisionBindings', () => { ).rejects.toThrow('insert failed'); // CF created it... - expect(createD1Database).toHaveBeenCalledWith(ACCOUNT_ID, API_TOKEN, 'proj-abc123-db'); + expect(createD1Database).toHaveBeenCalledWith(ACCOUNT_ID, API_TOKEN, 'proj-abc123-db', { + readReplication: undefined, + }); // ...and the rollback deleted exactly that object. expect(deleteD1Database).toHaveBeenCalledWith(ACCOUNT_ID, API_TOKEN, 'd1-uuid-123'); }); @@ -280,7 +284,7 @@ describe('provisionBindings', () => { ); expect(createD1Database).not.toHaveBeenCalled(); - expect(enableD1ReadReplication).toHaveBeenCalledWith(ACCOUNT_ID, API_TOKEN, 'existing-d1-id'); + expect(setD1ReadReplication).not.toHaveBeenCalled(); expect(createResource).not.toHaveBeenCalled(); expect(bindings[0]).toEqual({ type: 'd1', name: 'DB', id: 'existing-d1-id' }); expect(bindings).toHaveLength(5); @@ -575,6 +579,49 @@ describe('provisionBindings', () => { }); describe('ensureResource — already exists fallback', () => { + for (const state of ['created', 'reused', 'adopted'] as const) { + it.each([undefined, false, true])( + `${state} D1 applies only explicitly configured replication: %s`, + async (readReplication) => { + if (state === 'reused') { + vi.mocked(getProjectResources).mockResolvedValue([ + { type: 'd1', binding_name: 'DB', cf_id: 'existing-d1-id' } as Resource, + ]); + } else if (state === 'adopted') { + vi.mocked(createD1Database).mockRejectedValueOnce(new Error('already exists')); + } + await provisionBindings( + mockDb, + ACCOUNT_ID, + API_TOKEN, + PROJECT_ID, + { + version: 3, + bindings: { d1: ['DB'] }, + ...(readReplication !== undefined && { database: { readReplication } }), + }, + 'void-proxy', + { projectSlug: PROJECT_SLUG, proxyToken: PROXY_TOKEN }, + ); + if (state !== 'reused') { + expect(createD1Database).toHaveBeenCalledWith(ACCOUNT_ID, API_TOKEN, 'proj-abc123-db', { + readReplication, + }); + } + if (state !== 'created' && readReplication !== undefined) { + expect(setD1ReadReplication).toHaveBeenCalledExactlyOnceWith( + ACCOUNT_ID, + API_TOKEN, + state === 'reused' ? 'existing-d1-id' : 'd1-looked-up', + readReplication, + ); + } else { + expect(setD1ReadReplication).not.toHaveBeenCalled(); + } + }, + ); + } + it("falls back to lookup when D1 create returns 'already exists'", async () => { vi.mocked(createD1Database).mockRejectedValueOnce( new Error('Cloudflare API error: A database with that name already exists'), @@ -593,7 +640,7 @@ describe('ensureResource — already exists fallback', () => { expect(createD1Database).toHaveBeenCalledOnce(); expect(getD1DatabaseByName).toHaveBeenCalledWith(ACCOUNT_ID, API_TOKEN, 'proj-abc123-db'); - expect(enableD1ReadReplication).toHaveBeenCalledWith(ACCOUNT_ID, API_TOKEN, 'd1-looked-up'); + expect(setD1ReadReplication).not.toHaveBeenCalled(); expect(createResource).toHaveBeenCalledWith(mockDb, PROJECT_ID, 'd1', 'd1-looked-up', 'DB'); expect(bindings[0]).toEqual({ type: 'd1', name: 'DB', id: 'd1-looked-up' }); });