diff --git a/.agents/skills/void-deps-updates/SKILL.md b/.agents/skills/void-deps-updates/SKILL.md index 96306db6e..d448737cf 100644 --- a/.agents/skills/void-deps-updates/SKILL.md +++ b/.agents/skills/void-deps-updates/SKILL.md @@ -12,7 +12,7 @@ Work in the Void maintainer checkout. Read its current `AGENTS.md`, `packages/vo - Capture a fresh production-install baseline **before editing**. Build and pack the complete public release cohort, including the platform runtime. Use [the measurement helper](#measure). - Update direct and transitive dependencies recursively through Vite+. Preserve explicitly held TypeScript/Svelte versions and their framework-specific catalogs; inspect current compatibility constraints rather than freezing historical versions forever. - Coordinate `cf`, Cloudflare Vite plugin, Wrangler, Miniflare/workerd, and Cloudflare support packages using their actual registry ranges and dist-tags. Dated prereleases need explicit alignment; stable `latest` can be an older major than an intended beta. -- Update the exact pins for `cf`, `@cloudflare/vite-plugin`, and `miniflare` together. Their releases must resolve to the same Miniflare and workerd. Preserve other compatible published ranges, `workspace:*`, strict catalogs, intentional overrides, and package extensions. Avoid forcing incompatible runtime versions. Run `vp dedupe` and inspect remaining outdated packages. +- Update the exact pins for `cf`, `@cloudflare/vite-plugin`, `miniflare`, `@cloudflare/config`, and `@cloudflare/build-output-utils` together. Their releases must resolve to the same Miniflare and workerd. Pin `@cloudflare/config` and `@cloudflare/build-output-utils` to the exact versions that `cf` and the Vite plugin depend on; check with `npm view cf@ dependencies` and `npm view @cloudflare/vite-plugin@ dependencies`. Preserve other compatible published ranges, `workspace:*`, strict catalogs, intentional overrides, and package extensions. Avoid forcing incompatible runtime versions. Run `vp dedupe` and inspect remaining outdated packages. - Update `void init`/Vite+ scaffold and bootstrap expectations. Match the React compiler dependency to the installed React plugin's peer range. Keep Sandbox SDK, Docker helper, and verified helper layer aligned. - Keep prerender architecture changes separate from routine dependency alignment. diff --git a/.github/scripts/packed-consumer.mjs b/.github/scripts/packed-consumer.mjs index 4efb22e56..d3e04d8d2 100644 --- a/.github/scripts/packed-consumer.mjs +++ b/.github/scripts/packed-consumer.mjs @@ -389,6 +389,9 @@ for (const name of ['cf', '@cloudflare/vite-plugin']) { assert.equal(manifest('miniflare', tool), miniflare, name + ' installs a second Miniflare'); assert.equal(manifest('miniflare', types), miniflare, name + ' runtime types install a second Miniflare'); assert.equal(manifest('workerd', types), workerd, name + ' runtime types install a second workerd'); + for (const shared of ['@cloudflare/config', '@cloudflare/build-output-utils']) { + assert.equal(manifest(shared, tool), manifest(shared, voidEntry), name + ' installs a second ' + shared); + } } const manifestPath = findPackageJSON('@void/platform', voidEntry); const entry = JSON.parse(readFileSync(manifestPath, 'utf8')).exports['./installer'].import; diff --git a/AGENTS.md b/AGENTS.md index 8c87c4c75..347f6f731 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -7,7 +7,7 @@ Full-stack framework for Cloudflare Workers, powered by Vite. - Documentation source lives in `docs/`. - Docs is a Vitepress project using our custom theme. When asked to "link the theme", temporarily edit `docs/package.json` so that `@voidzero-dev/vitepress-theme` point to `link:../../../theme`. Rerun `vp install` to link it. Revert these when asked to "unlink the theme". - Packages publish to npm as `void` and `@void/*`. Consumer-facing code, docs examples, playground source, and package manifests should use only those public package names. -- Published external runtime dependencies use compatible semver ranges. Pin `cf`, `@cloudflare/vite-plugin`, and `miniflare` as an exception: update them together so fresh installs share one Miniflare and workerd. Preserve `workspace:*` for dependencies and peers between Void packages. The lockfile fixes the versions tested in this repository. Keep the Sandbox SDK version aligned with its packaged Docker helper and verified platform helper layer. +- Published external runtime dependencies use compatible semver ranges. Pin `cf`, `@cloudflare/vite-plugin`, `miniflare`, `@cloudflare/config`, and `@cloudflare/build-output-utils` as an exception: update them together so fresh installs share one Miniflare and workerd. Pin `@cloudflare/config` and `@cloudflare/build-output-utils` to the exact versions `cf` and the Vite plugin depend on, so fresh installs share one copy of each. Preserve `workspace:*` for dependencies and peers between Void packages. The lockfile fixes the versions tested in this repository. Keep the Sandbox SDK version aligned with its packaged Docker helper and verified platform helper layer. ## Important Note for Release Workflows diff --git a/docs/guide/ai.md b/docs/guide/ai.md index 604657fa8..38063efa5 100644 --- a/docs/guide/ai.md +++ b/docs/guide/ai.md @@ -167,6 +167,8 @@ const response = await ai.provider('openai').fetch('/chat/completions', { }); ``` +If the signal aborts while you read a streaming response, reading it rejects with the signal's reason instead of ending with a `void:error` event. + ### OpenAI ```ts diff --git a/docs/guide/database/d1.md b/docs/guide/database/d1.md index dad23a353..65e8f45f0 100644 --- a/docs/guide/database/d1.md +++ b/docs/guide/database/d1.md @@ -54,7 +54,7 @@ export default defineConfig({ }); ``` -If you self-host the Void platform, upgrade it as well before deploying. Upgrading alone or omitting the option does **not** disable existing replication. +If you self-host the Void platform, upgrade it as well before deploying. When the platform cannot apply `database.readReplication`, `void deploy` stops with an error before uploading anything. Upgrading alone or omitting the option does **not** disable existing replication. ::: [D1 read replication](https://developers.cloudflare.com/d1/best-practices/read-replication/) is opt-in. Enable it in `void.config.ts`, then run `void deploy`: diff --git a/docs/guide/email/domains.md b/docs/guide/email/domains.md index 532d8c04e..299130aee 100644 --- a/docs/guide/email/domains.md +++ b/docs/guide/email/domains.md @@ -83,13 +83,13 @@ Setup enables the zone's plus addressing so `support+T-42@mail.acme.com` reaches ### Sending {#sending} -On Workers Free, `sendEmail()` can send to verified destinations listed under **Email Routing → Destination addresses** in Cloudflare. +On Workers Free, apps with `email/` handlers can send to verified destinations listed under **Email Routing → Destination addresses** in Cloudflare. Apps without `email/` handlers send through Email Sending, which requires [Workers Paid](https://dash.cloudflare.com/?to=/:account/workers/plans). Configure your sender domain's SPF, DKIM, and DMARC records using [Cloudflare's email authentication guidance](https://developers.cloudflare.com/email-service/concepts/email-authentication/), then check the authentication results in a received message's original headers. A successful `sendEmail()` result confirms provider acceptance; it does not confirm delivery or authentication readiness. -Sending to arbitrary recipients requires [Workers Paid](https://dash.cloudflare.com/?to=/:account/workers/plans) and Email Sending onboarding. After upgrading, run `void email setup --platform cloudflare`. Void doesn't retry onboarding during ordinary deploys. +Sending to arbitrary recipients requires [Workers Paid](https://dash.cloudflare.com/?to=/:account/workers/plans) and Email Sending onboarding. After upgrading, run `void email setup --platform cloudflare`. Once setup is saved, deploys don't retry onboarding. -Existing DMARC or bounce-domain records can block onboarding; the checklist identifies conflicts to resolve. Inbound mail remains available. +Existing DMARC or bounce-domain records can block onboarding; the checklist identifies conflicts to resolve. Apps with `email/` handlers keep inbound mail. The `void email usage`, `logs`, `destinations`, `allow`, and `disallow` commands apply to Void platforms. @@ -118,4 +118,4 @@ Commit `void.lock.json`, then deploy in CI with: void deploy --platform cloudflare --require-email ``` -CI cannot answer setup prompts. Without `--require-email`, an unconfigured app can deploy without email. If setup is saved but DNS hasn't propagated, deployment waits for a later retry. For apps with inbound handlers, Workers Free setup for verified recipients satisfies `--require-email`. Sending-only apps must have an onboarded sending domain and a configured send binding. +CI cannot answer setup prompts. Without `--require-email`, an unconfigured app can deploy without email. If setup is saved but DNS hasn't propagated, deployment waits for a later retry. For apps with inbound handlers, Workers Free setup for verified recipients satisfies `--require-email`. Sending-only apps need an onboarded sending domain (Workers Paid) and a configured send binding. diff --git a/docs/integrations/cloudflare.md b/docs/integrations/cloudflare.md index a8ab0039c..a10634769 100644 --- a/docs/integrations/cloudflare.md +++ b/docs/integrations/cloudflare.md @@ -143,7 +143,7 @@ Set these CI secrets and variables as needed: | `CF_ACCESS_CLIENT_ID`, `CF_ACCESS_CLIENT_SECRET` | When Cloudflare Access protects the readiness URL. | | `CF_ACCESS_READINESS_ORIGIN` | When Access protects a route or Custom Domain and `workers_dev` is `false`. See [Cloudflare Access](#cloudflare-access). | -For email, run `void email setup --platform cloudflare` locally, commit `void.lock.json`, and pass `--require-email` in CI. The token also needs Email Routing Edit and Email Sending Edit. +For email, run `void email setup --platform cloudflare` locally, commit `void.lock.json`, and pass `--require-email` in CI. The token also needs Zone Read and Email Sending Edit, plus Email Routing Edit for apps with `email/` handlers. #### Cloudflare Access @@ -208,7 +208,7 @@ Void deploys native apps, static sites, SPAs, supported SSGs, and Cloudflare bui - `--skip-build` works for existing static output. Worker apps require a fresh build. - Named Cloudflare environments aren't supported. Use a separate Void project config for each target. - Sandbox requires Workers Paid and Docker. See [Sandbox requirements](../guide/sandboxes.md#deployment) for plan and token permissions. -- Email to arbitrary recipients requires Workers Paid; verified recipients and inbound mail work on Workers Free. See [Email](../guide/email/domains.md#your-own-cloudflare-account). +- Email to arbitrary recipients, and all sending from apps without `email/` handlers, requires Workers Paid. Apps with handlers keep inbound mail and verified recipients on Workers Free. See [Email](../guide/email/domains.md#your-own-cloudflare-account). - Node.js, Bun, and Deno use their [own deployment path](./nodejs-bun-deno.md). Commit Durable Object migration history in `void.lock.json`. Do not delete or reorder deployed steps. diff --git a/docs/reference/cli/database.md b/docs/reference/cli/database.md index 61d0fd813..53535aced 100644 --- a/docs/reference/cli/database.md +++ b/docs/reference/cli/database.md @@ -52,7 +52,8 @@ Programmatic seed modules must export either a default function or a named `seed ``` void db execute void db execute --file -void db execute --remote +void db execute --remote [--yes] +void db execute --remote --file [--yes] ``` Run ad-hoc SQL against the database. Provide SQL inline or from a file. A statement that returns rows prints them as a formatted table, and statements that change data print nothing. SQL with several statements runs as a script without printed results. @@ -64,9 +65,9 @@ By default, targets the local database. Pass `--remote` to run against the deplo - **Hosted PostgreSQL and MySQL projects**: fetches the stored connection string from the platform and connects directly. - **Direct Cloudflare PostgreSQL and MySQL projects**: uses `DATABASE_URL` from the current shell; Cloudflare cannot return the password from Hyperdrive. -On direct Cloudflare D1 deployments, `--file` uses bulk import. Void waits for the import to finish and reports failures; bulk imports do not return individual query rows. Pass SQL inline to return query rows. +On direct Cloudflare D1 deployments, a `--file` larger than 100 KB runs as a D1 import: Void waits for it to finish, the database cannot serve queries until then, and query rows are not printed. Smaller files print rows like inline SQL. -For destructive statements (`DELETE`, `UPDATE`, `DROP`, etc.) when running in a TTY, you will be prompted to confirm before the query is sent to the deployed database. Non-TTY environments (CI) skip the prompt. +For destructive statements (`DELETE`, `UPDATE`, `DROP`, etc.) when running in a TTY, you will be prompted to confirm before the query is sent to the deployed database. A TTY also asks before a D1 import, whatever the file contains. Pass `--yes` to skip the prompt. Non-TTY environments (CI) never prompt, so `--yes` is optional there. ## `void db migrate` {#void-db-migrate} diff --git a/docs/reference/cli/email.md b/docs/reference/cli/email.md index ba02475a6..aa81bf935 100644 --- a/docs/reference/cli/email.md +++ b/docs/reference/cli/email.md @@ -142,4 +142,4 @@ Set up email before deploying from CI. Requires `email.from` in `void.config.ts` Void shows the planned changes and asks before applying them. Apps without `email/` handlers configure sending only; routing and plus addressing are left alone. Commit `void.lock.json`, then run `void deploy --platform cloudflare --require-email` in CI. -On Workers Free, inbound mail and sends to verified recipients remain available. Run this command again after upgrading to Workers Paid to enable arbitrary recipients. See [Email setup](../../guide/email/domains.md#your-own-cloudflare-account). +On Workers Free, apps with `email/` handlers keep inbound mail and sends to verified recipients. Sending-only apps need Email Sending, which requires Workers Paid. Run this command again after upgrading to Workers Paid to enable arbitrary recipients. See [Email setup](../../guide/email/domains.md#your-own-cloudflare-account). diff --git a/packages/void/AGENTS.md b/packages/void/AGENTS.md index 4fc06c4c0..65b11b079 100644 --- a/packages/void/AGENTS.md +++ b/packages/void/AGENTS.md @@ -4,7 +4,7 @@ Core Vite plugin — routing, inference, migrations, CLI. User-facing CLI output and documentation describe Cloudflare operations through Void. Never ask users to install, authenticate, or invoke Wrangler directly. Keep `wrangler.jsonc` / `wrangler.json` and `WRANGLER_*` names only where users must recognize an actual file or environment variable; implementation details may remain in source comments, tests, and debug logs. -Completed CLI actions use `printSuccess` from `src/cli/output.ts`: a blank line, a green checkmark, and green text. Use `log.step` for intermediate work. Keep listings, no-op notices, cancellations, warnings, and checkpointed or pending operations out of the success renderer. `void init` records its work in `InitReport` (`src/cli/init-report.ts`) instead of logging each step, and prints a summary before `printSuccess`: `printDone` (green ✓) for finished work and `printNextStep` (blue →) for what the user should do next. A run that stops early, on a failed step or a cancelled prompt, prints the same record under a warning, so record each change right where init makes it. Preserve JSON, token, and SQL stdout contracts. +Completed CLI actions use `printSuccess` from `src/cli/output.ts`: a blank line, a green checkmark, and green text. Use `log.step` for intermediate work. Keep listings, no-op notices, cancellations, warnings, and checkpointed or pending operations out of the success renderer. `void init` records its work in `InitReport` (`src/cli/init-report.ts`) instead of logging each step, and prints a summary before `printSuccess`: `printDone` (green ✓) for finished work and `printNextStep` (blue →) for what the user should do next. A new starter prints one line instead of its files; files outside the app and later changes to files that existed before init are listed under it. A run that stops early, on a failed step or a cancelled prompt, prints the same record under a warning, so record each change right where init makes it. Preserve JSON, token, and SQL stdout contracts. Human-facing URLs use `formatUrl` or `formatLink` from `src/cli/output.ts` for terminal hyperlinks with readable plain-text fallbacks. Use the shared `note` renderer for URL box titles; it pads the URL with spaces and adds the hyperlink after layout. Keep machine-readable output and copyable commands free of terminal links. @@ -125,7 +125,7 @@ generated entries must preserve the module-boundary rules below. - **Plugin entry imports**: Every `vp dev` and `vp build` imports the plugin entry before it starts. Import dependencies that only some apps or commands use (feature codegen, CLI-only helpers) where they are used, so other apps don't load them at startup. `test/unit/plugin-entry-imports.test.ts` guards this; add a mock there when you move a heavy import out. - **pnpm scaffold build policy**: Keep `create-void` and `void init` aligned with the repository: `better-sqlite3: false` uses version 13's bundled Node-API binaries; `esbuild`, `sharp`, and `workerd` have approved builds. Do not require an unnecessary node-gyp rebuild in fresh applications. - **Scaffold dependency installation**: Vite+ projects install through local/global `vp`, bootstrapping with `npx` when neither is available. Preserve existing package-manager pins and lockfiles; fresh Vite+ projects select pnpm through the build-policy YAML without adding a version pin. Plain Vite projects use their selected package manager. Do not run a global package manager directly for Vite+ installs. -- **Vite+ starter versions**: Bundle the build-time `vite-plus/package.json` metadata into the CLI and scaffold its exact CLI/core pairing, including package-manager overrides. Preserve existing Vite+ pairings and catalogs. Bootstrap with the project's selected version; upgrades belong to `vp migrate`, which coordinates the toolchain. +- **Vite+ starter versions**: Bundle the build-time `vite-plus/package.json` metadata into the CLI and scaffold its exact CLI/core pairing, including package-manager overrides. Preserve existing Vite+ pairings and catalogs. Bootstrap with the project's selected version (a pnpm, Bun or Yarn catalog entry, where `catalog:` and `catalog:default` name the same catalog), or the CLI's own Vite+ version when that entry cannot be read; upgrades belong to `vp migrate`, which coordinates the toolchain. - **Prerender discovery and environment validation**: Only `POST /__void/prerender-paths` bypasses whole-schema boot validation, because path discovery runs without remote production secrets. Discovery answers only when the build-time runner marks the request (the `__VOID_PRERENDER` Miniflare binding on Cloudflare, the Hono env argument on Node/Bun/Deno), never on a deployed app or dev server. Keep validation on readiness, application requests, page rendering, and event handlers. - **Pre-resolved paths in codegen**: Generated virtual module code uses absolute paths for `hono` and `void/response` (resolved via `import.meta.resolve()`). Virtual modules have no filesystem location, so bare specifiers can't resolve from them. void/db never imports its pg or mysql2 driver itself: drizzle builds the pool from `connection`, and the worker prebundle serves drizzle's bare driver import. Bundles that never run Void's Vite plugins, Nitro (through `void/nitro`) and the deploy cron/queue handler build, get the same module from `voidDbBundlerPlugin`, which resolves `@schema` through the bundler so a schema folder works. Their Worker builds import the driver that `bundleWorkerSqlDriver` builds like the migration handler: Nitro's CommonJS handling and Node stubs break pg and mysql2 in workerd. Each `pg`/`mysql2` import gets its own build of the copy the bundler resolves for it, looked up as a `require` so Rollup's CommonJS plugin neither wraps nor loads the raw driver, so Void's drizzle-orm and an app's own driver import keep their own versions. An import of `mysql2/promise` gets that copy's promise API from the same build, so it shares the driver's modules as in Node. A pg older than 8.11 has no `pg-cloudflare` socket and a mysql2 older than 3.13 has no `disableEval` option, so the lookup fails the build with an error that names the driver version and the importer. In an npm, Yarn or Bun install, Void's hoisted drizzle-orm loads the hoisted driver, which is the app's own copy when the app lists one. For pg and MySQL, `void/nitro` also runs each request in `withRuntimeEnv` (as the Class A wrapper does) and keeps Void's `node:async_hooks` native in workerd builds, because Nitro's unenv AsyncLocalStorage drops the store after the first await. Nitro turns on `cloudflare.nodeCompat` only from a `wrangler.*` file, so `void/nitro` sets it for pg/MySQL Worker builds (every deploy adds `nodejs_compat` for them) and for D1 only when the flags a deploy gives the Worker list `nodejs_compat` (the Cloudflare config's, which a managed deploy replaces with `worker.compatibility_flags` when set): a Worker built that way does not start without the flag. Nuxt and Analog also build server code with Vite (Analog's page server loads, Nuxt's app renderer), and Nitro bundles that output: Void's Vite plugin leaves a pg/MySQL `void/db` import there external, so Nitro serves one module and one driver copy, or the package's D1 client and its `void/nitro` hint when the module is missing. The deploy handler build likewise runs each cron event and queue batch in `withRuntimeEnv`. - **CF plugin API**: Worker config (`main`, `d1_databases`, `kv_namespaces`, `r2_buckets`) goes inside the `config` option of `cloudflare()`, NOT as top-level properties. `persistState` takes `{ path: string }`, not a raw string. The `config` option uses a **function form** `(resolved: WorkerConfig) => Partial` so that bindings from the user's `wrangler.json` (if present) take precedence over inferred placeholder bindings. diff --git a/packages/void/src/cli/cf-d1-import.ts b/packages/void/src/cli/cf-d1-import.ts index d456b0e8c..b585443d7 100644 --- a/packages/void/src/cli/cf-d1-import.ts +++ b/packages/void/src/cli/cf-d1-import.ts @@ -5,21 +5,70 @@ import { sanitizeTerminalLogText } from './terminal-log-text.ts'; type Json = Record; +// Bounds one D1 API request. Like wrangler, the import as a whole has no +// deadline: a large file may take a long time to upload and ingest. +const D1_IMPORT_REQUEST_TIMEOUT_MS = 2 * 60_000; + +// The presigned upload gets the request timeout plus the time to send the file at 1 Mbit/s, +// so a stalled PUT fails, while a normal CI link sends a 25 MB file well within its 5 minutes. +const D1_IMPORT_UPLOAD_MIN_BYTES_PER_SECOND = 128 * 1024; + +export function d1ImportUploadTimeoutMs(bytes: number): number { + return ( + D1_IMPORT_REQUEST_TIMEOUT_MS + Math.ceil(bytes / D1_IMPORT_UPLOAD_MIN_BYTES_PER_SECOND) * 1000 + ); +} + +/** D1 reported that the import failed, so the database is back where it started. */ +class D1ImportStatusError extends Error {} + +/** The D1 API answered a request with a client error, so it did not act on that request. */ +class D1ImportRefusedError extends Error {} + function object(value: unknown): Json { return value && typeof value === 'object' && !Array.isArray(value) ? (value as Json) : {}; } +function strings(value: unknown): Array { + if (!Array.isArray(value)) { + return []; + } + const out: Array = []; + for (const item of value) { + const text = typeof item === 'string' ? item : object(item).message; + if (typeof text === 'string' && text) { + out.push(text); + } + } + return out; +} + +/** + * A failed ingest reports the SQL error in `errors` (and progress in + * `messages`) with `status: 'error'`; `error` is set only when the import + * itself was refused. Show all of them, as wrangler does. + */ +function importFailureDetail(state: Json): string { + const errors = strings(state.errors); + if (typeof state.error === 'string' && state.error) { + errors.unshift(state.error); + } + const lines = [...errors, ...strings(state.messages)]; + return lines.length > 0 ? sanitizeTerminalLogText(lines.join('\n')) : 'Unknown import error.'; +} + /** Upload SQL separately from process arguments and wait for D1 to finish ingesting it. */ export async function importD1SqlFile({ endpoint, headers, file, + pollIntervalMs = 1000, }: { endpoint: string; headers: Record; file: string; + pollIntervalMs?: number; }): Promise { - const signal = AbortSignal.timeout(10 * 60_000); const blob = await openAsBlob(file); const hash = createHash('md5'); for await (const chunk of blob.stream()) { @@ -33,36 +82,70 @@ export async function importD1SqlFile({ headers: { ...headers, 'Content-Type': 'application/json' }, body: JSON.stringify(body), redirect: 'error', - signal, + signal: AbortSignal.timeout(D1_IMPORT_REQUEST_TIMEOUT_MS), }); - const data = object(await response.json()); + let data: Json; + try { + data = object(await response.json()); + } catch (error) { + // A 4xx refusal is still a refusal when a proxy answers it with HTML. + if (response.ok || response.status >= 500) { + throw error; + } + data = {}; + } if (!response.ok || data.success !== true) { const errors = Array.isArray(data.errors) ? data.errors : []; const detail = errors .map((error) => String(object(error).message ?? '')) .filter(Boolean) .join('; '); - throw new Error( - `D1 import failed (HTTP ${response.status})${detail ? ': ' + sanitizeTerminalLogText(detail) : '.'}`, - ); + const message = `D1 import failed (HTTP ${response.status})${detail ? ': ' + sanitizeTerminalLogText(detail) : '.'}`; + // A server error does not prove that the request was not acted on. + throw response.status < 500 ? new D1ImportRefusedError(message) : new Error(message); } const state = object(data.result); if (state.status === 'error' || state.success === false) { - throw new Error( - `D1 import failed: ${sanitizeTerminalLogText(String(state.error ?? 'Unknown import error.'))}`, - ); + throw new D1ImportStatusError(`D1 import failed: ${importFailureDetail(state)}`); } return state; } - const initialized = await request({ action: 'init', etag }); + function unknownOutcome(error: unknown, bookmark: string | undefined): Error { + const message = error instanceof Error ? error.message : String(error); + const cause = message.endsWith('.') ? message : `${message}.`; + return new Error( + sanitizeTerminalLogText( + `D1 did not report whether the import finished: ${cause} ` + + `The import may still be running on Cloudflare${bookmark ? ` (bookmark ${bookmark})` : ''}. ` + + 'Check the database before running the file again.', + ), + { cause: error }, + ); + } + + // When D1 already has a file with this etag, init itself starts the import, so only an + // answer from D1 proves that nothing started. + let initialized: Json; + try { + initialized = await request({ action: 'init', etag }); + } catch (error) { + if (error instanceof D1ImportStatusError || error instanceof D1ImportRefusedError) { + throw error; + } + throw unknownOutcome(error, undefined); + } let state = initialized; if (!initialized.at_bookmark) { - if ( - typeof initialized.filename !== 'string' || - !initialized.filename || - typeof initialized.upload_url !== 'string' - ) { + // Like wrangler, a reply without an upload URL means D1 already has the file. + if (typeof initialized.upload_url !== 'string') { + throw unknownOutcome( + new Error('Cloudflare returned an invalid D1 import upload.'), + undefined, + ); + } + // D1 asked for the file, so it has not started; failures until ingest are plain. + if (typeof initialized.filename !== 'string' || !initialized.filename) { throw new Error('Cloudflare returned an invalid D1 import upload.'); } let uploadUrl: URL; @@ -83,28 +166,53 @@ export async function importD1SqlFile({ ) { throw new Error('Cloudflare returned an insecure D1 import upload URL.'); } - // The presigned URL authorizes this request; never forward the account's credentials. - const upload = await fetch(uploadUrl, { - method: 'PUT', - body: blob, - redirect: 'error', - signal, - }); - await upload.body?.cancel(); + const uploadTimeoutMs = d1ImportUploadTimeoutMs(blob.size); + let upload: Response; + try { + // The presigned URL authorizes this request; never forward the account's credentials. + upload = await fetch(uploadUrl, { + method: 'PUT', + body: blob, + redirect: 'error', + signal: AbortSignal.timeout(uploadTimeoutMs), + }); + await upload.body?.cancel(); + } catch (error) { + if (error instanceof DOMException && error.name === 'TimeoutError') { + throw new Error( + `D1 import upload did not finish within ${Math.round(uploadTimeoutMs / 1000)} seconds.`, + { cause: error }, + ); + } + throw error; + } if (!upload.ok || upload.headers.get('etag')?.replaceAll('"', '') !== etag) { throw new Error(`D1 import upload failed verification (HTTP ${upload.status}).`); } - state = await request({ action: 'ingest', etag, filename: initialized.filename }); } - for (;;) { - if (state.status === 'complete' && state.success === true) { - return; + // From here D1 may be ingesting the file. Only a status that D1 reports + // tells whether it finished; any other failure leaves the outcome unknown. + let bookmark: string | undefined; + try { + if (!initialized.at_bookmark) { + state = await request({ action: 'ingest', etag, filename: initialized.filename }); + } + for (;;) { + if (state.status === 'complete' && state.success === true) { + return; + } + if (typeof state.at_bookmark !== 'string' || !state.at_bookmark) { + throw new Error('Cloudflare returned an invalid D1 import status.'); + } + bookmark = state.at_bookmark; + await setTimeout(pollIntervalMs); + state = await request({ action: 'poll', current_bookmark: bookmark }); } - if (typeof state.at_bookmark !== 'string' || !state.at_bookmark) { - throw new Error('Cloudflare returned an invalid D1 import status.'); + } catch (error) { + if (error instanceof D1ImportStatusError) { + throw error; } - await setTimeout(1000, undefined, { signal }); - state = await request({ action: 'poll', current_bookmark: state.at_bookmark }); + throw unknownOutcome(error, bookmark); } } diff --git a/packages/void/src/cli/client.ts b/packages/void/src/cli/client.ts index 846dec98e..8a447d425 100644 --- a/packages/void/src/cli/client.ts +++ b/packages/void/src/cli/client.ts @@ -737,6 +737,28 @@ export class PlatformClient { } } + /** Platforms that predate the flag ignore the setting and keep replication on. */ + async requireD1ReadReplicationSupport(): Promise { + const response = await this.request(`${this.baseUrl}/capabilities`, { + signal: AbortSignal.timeout(15_000), + }); + if (!response.ok && response.status !== 404) { + await throwPlatformApiError( + 'Failed to check D1 read replication support', + response, + 'deploy', + ); + } + const capabilities = response.ok + ? ((await response.json()) as { d1ReadReplication?: unknown }) + : null; + if (capabilities?.d1ReadReplication !== true) { + throw new Error( + 'deploy: This Void platform cannot apply `database.readReplication` from void.config.ts. Ask its administrator to upgrade with `void platform upgrade`.', + ); + } + } + /** Reads the platform version from public discovery, without credentials. */ private async probePlatformVersion(): Promise { let response: Response; diff --git a/packages/void/src/cli/cloudflare-operation-process.ts b/packages/void/src/cli/cloudflare-operation-process.ts index 9f1fb0461..e1e69a384 100644 --- a/packages/void/src/cli/cloudflare-operation-process.ts +++ b/packages/void/src/cli/cloudflare-operation-process.ts @@ -30,6 +30,7 @@ import { printCloudflareLoginLink, printCloudflareLoginPrompt, } from './cloudflare-login-output.ts'; +import { normalizeD1SqlLineEndings } from './d1-sql-line-endings.ts'; import { platformWorkerUploadAttribution } from './platform-worker-user-agent.ts'; import { sanitizeTerminalLogText } from './terminal-log-text.ts'; @@ -740,8 +741,10 @@ async function resources(): Promise { } case 'd1.execute': { const database = d1Database(operation.database); - const sql = + const source = operation.file !== undefined ? readFileSync(operation.file, 'utf8') : operation.sql; + // cf sends the body as is, and the query API fails on CRLF inside a trigger body. + const sql = normalizeD1SqlLineEndings(source); return withSecretFile(JSON.stringify({ sql }), (path) => runCf(['d1', 'query', database.id, '--body', '@' + path]), ); diff --git a/packages/void/src/cli/d1-sql-line-endings.ts b/packages/void/src/cli/d1-sql-line-endings.ts new file mode 100644 index 000000000..7d525fd79 --- /dev/null +++ b/packages/void/src/cli/d1-sql-line-endings.ts @@ -0,0 +1,60 @@ +/** + * Turn structural CRLF line endings into LF before SQL goes to the D1 query API. That API fails + * on CRLF inside a compound statement such as a `CREATE TRIGGER ... BEGIN ... END;` body + * (`incomplete input: SQLITE_ERROR [7500]`), while the import API accepts the same bytes. Line + * endings inside quoted values and identifiers ('...', "...", `...`, [...]) are kept, so stored + * data does not change. Comments are skipped so a quote inside one does not start a string. + * Matches wrangler's `normalizeSqlLineEndings` (cloudflare/workers-sdk#15044), except that a + * bracket identifier ends at the first `]`, as in SQLite's tokenizer. + */ +export function normalizeD1SqlLineEndings(sql: string): string { + if (!sql.includes('\r\n')) { + return sql; + } + let out = ''; + let index = 0; + const crlfAt = (at: number) => sql[at] === '\r' && sql[at + 1] === '\n'; + while (index < sql.length) { + const char = sql[index]!; + const next = sql[index + 1]; + if (char === "'" || char === '"' || char === '`' || char === '[') { + // Copy the quoted text as is. A doubled quote is an escape, except in [brackets]. + const close = char === '[' ? ']' : char; + let end = index + 1; + for (;;) { + const found = sql.indexOf(close, end); + if (found === -1) { + end = sql.length; + break; + } + end = found + 1; + if (close === ']' || sql[end] !== close) { + break; + } + end++; + } + out += sql.slice(index, end); + index = end; + } else if (char === '-' && next === '-') { + // A line comment runs to the end of the line; its line ending is structural. + let end = index + 2; + while (end < sql.length && sql[end] !== '\n' && !crlfAt(end)) { + end++; + } + out += sql.slice(index, end); + index = end; + } else if (char === '/' && next === '*') { + const found = sql.indexOf('*/', index + 2); + const end = found === -1 ? sql.length : found + 2; + out += sql.slice(index, end).replaceAll('\r\n', '\n'); + index = end; + } else if (crlfAt(index)) { + out += '\n'; + index += 2; + } else { + out += char; + index++; + } + } + return out; +} diff --git a/packages/void/src/cli/db-args.ts b/packages/void/src/cli/db-args.ts index 841951c4a..5fb1cdaaf 100644 --- a/packages/void/src/cli/db-args.ts +++ b/packages/void/src/cli/db-args.ts @@ -16,7 +16,10 @@ export const dbSubcommands: Array = [ name: 'execute', arg: '', description: 'Run SQL against the database (use --remote for the deployed database)', - flags: [{ name: '--remote', description: 'Run against the deployed database' }], + flags: [ + { name: '--remote', description: 'Run against the deployed database' }, + { name: '--yes', description: 'Skip the confirmation prompt' }, + ], }, { name: 'migrate', @@ -82,7 +85,7 @@ export type DbArgs = | { action: 'status' } | { action: 'reset'; yes?: boolean } | { action: 'seed'; file?: string; yes?: boolean } - | { action: 'execute'; command?: string; file?: string; remote?: boolean } + | { action: 'execute'; command?: string; file?: string; remote?: boolean; yes?: boolean } | { action: 'migrate'; remote?: boolean } | { action: 'studio'; remote?: boolean } | { @@ -134,7 +137,7 @@ const ACTION_FLAGS: Record> = { status: [], reset: ['yes'], seed: ['file', 'yes'], - execute: ['file', 'remote'], + execute: ['file', 'remote', 'yes'], migrate: ['remote'], studio: ['remote'], 'set-url': [], @@ -235,7 +238,7 @@ export function parseDbArgs(args: Array): DbArgs { if (command && values.file) { throw new Error('db: Provide either SQL inline or --file, not both.'); } - return { action, command, file: values.file, remote: values.remote }; + return { action, command, file: values.file, remote: values.remote, yes: values.yes }; } case 'migrate': case 'studio': diff --git a/packages/void/src/cli/db.ts b/packages/void/src/cli/db.ts index 7a5058214..5c5b31294 100644 --- a/packages/void/src/cli/db.ts +++ b/packages/void/src/cli/db.ts @@ -53,6 +53,7 @@ import { quoteSqlString, } from './db-export-sequences.ts'; import { formatTimestamp } from './gen.ts'; +import { Buffer } from './node-buffer.ts'; import { printSuccess, dim, formatCount, formatUrl, getErrorMessage, pc } from './output.ts'; import { readDeploymentPlatform, readProjectConfig } from './project.ts'; import { drizzleKitBin } from './yarn-pnp.ts'; @@ -205,7 +206,7 @@ export async function runDbCommand(root: string, args: DbArgs): Promise { } else if (args.action === 'seed') { await runSeed(root, args.file, dialect, args.yes); } else if (args.action === 'execute') { - await runExecute(root, args.command, args.file, dialect, args.remote); + await runExecute(root, args.command, args.file, dialect, args.remote, args.yes); } else if (args.action === 'migrate') { await runMigrate(root, dialect, args.remote); } else if (args.action === 'studio') { @@ -921,10 +922,14 @@ async function executeExternalRemoteSql( /** * Prompt the user to confirm a destructive remote query when running in a TTY. - * Returns true if execution should proceed, false if it was cancelled. + * `always` asks even for read-only SQL, and `yes` (`--yes`) never asks. Returns true if + * execution should proceed, false if it was cancelled. */ -async function confirmDestructiveRemote(sql: string): Promise { - if (isSafeStatement(sql)) { +async function confirmDestructiveRemote( + sql: string, + { always = false, yes = false }: { always?: boolean; yes?: boolean } = {}, +): Promise { + if (yes || (!always && isSafeStatement(sql))) { return true; } if (!process.stdin.isTTY) { @@ -939,19 +944,33 @@ async function confirmDestructiveRemote(sql: string): Promise { return true; } +// D1's maximum SQL statement length. A direct D1 `--file` of this size or smaller cannot hit +// that limit on the query API, which prints rows; a larger dump runs as a D1 import (up to 5 GB). +const D1_MAX_SQL_STATEMENT_BYTES = 100_000; + async function runExecute( root: string, command: string | undefined, file: string | undefined, dialect: DatabaseDialect, remote?: boolean, + yes?: boolean, ): Promise { if (remote) { const directCloudflare = readDeploymentPlatform(root) === 'cloudflare'; const sql = file ? readFileSync(resolve(root, file), 'utf-8') : command!; if (directCloudflare) { - const shouldProceed = await confirmDestructiveRemote(sql); + const bulkImport = + dialect === 'sqlite' && + file !== undefined && + Buffer.byteLength(sql) > D1_MAX_SQL_STATEMENT_BYTES; + if (bulkImport) { + log.warn( + `${file} is larger than 100 KB, so Void runs it as a D1 import. The database cannot serve queries until the import finishes, and query rows are not printed.`, + ); + } + const shouldProceed = await confirmDestructiveRemote(sql, { always: bulkImport, yes }); if (!shouldProceed) { log.info('Cancelled.'); process.exit(0); @@ -960,7 +979,7 @@ async function runExecute( const { executeCloudflareD1Sql, importCloudflareD1Sql, resolveCloudflareD1Context } = await import('./cloudflare-operations.ts'); const context = resolveCloudflareD1Context(root, configuredD1Binding(root)); - if (file) { + if (bulkImport) { await importCloudflareD1Sql(context, sql); } else { const rows = await executeCloudflareD1Sql(context, sql); @@ -994,7 +1013,7 @@ async function runExecute( process.exit(1); } - const shouldProceed = await confirmDestructiveRemote(sql); + const shouldProceed = await confirmDestructiveRemote(sql, { yes }); if (!shouldProceed) { log.info('Cancelled.'); process.exit(0); diff --git a/packages/void/src/cli/deploy-cloudflare-email.ts b/packages/void/src/cli/deploy-cloudflare-email.ts index f7d628027..5d5af1c2b 100644 --- a/packages/void/src/cli/deploy-cloudflare-email.ts +++ b/packages/void/src/cli/deploy-cloudflare-email.ts @@ -10,6 +10,8 @@ import { applyEmailSetup, EMAIL_FROM_VAR, emailConfigInputs, + formatSendingOnlyPaidHint, + isAwaitingSendingOnboarding, preflightEmail, SEND_EMAIL_BINDING, renderEmailChecklist, @@ -127,6 +129,7 @@ export async function runCloudflareEmailStep( routes: pre.derived.routes, addresses, sendingReady, + sendingOnly: pre.usage.handlers.length === 0, creating, }); // Pin `configPath` so every step writes the same file. A refused write (e.g. a comment @@ -141,11 +144,17 @@ export async function runCloudflareEmailStep( } }; - // Ready: no prompt and no account call; a sending onboarding recorded as refused is never - // retried here. The config rows are still rewritten, because a stale `addresses` entry - // becomes a plan conflict and a stale `__VOID_EMAIL_FROM` ships the old sender. `null` - // removes the key when every derived address is routed elsewhere. - if (pre.ready) { + // A saved setup is never prompted or retried here: a ready one, and a sending-only one that + // still waits for Email Sending onboarding (Workers Free cannot be told apart from "not + // onboarded yet", so asking on every deploy would loop). `void email setup` owns that retry. + // The config rows are still rewritten, because a stale `addresses` entry becomes a plan + // conflict and a stale `__VOID_EMAIL_FROM` ships the old sender. `null` removes the key + // when every derived address is routed elsewhere. + const awaitingOnboarding = isAwaitingSendingOnboarding(pre); + if (pre.ready || awaitingOnboarding) { + if (awaitingOnboarding) { + log.warn(formatSendingOnlyPaidHint(pre.domain).join('\n')); + } const patch: ProvisionedWranglerPatch = {}; const rows: Array = []; if (pre.addressesStale) { @@ -178,6 +187,10 @@ export async function runCloudflareEmailStep( configWritten = true; log.message(` ✔ config ${configName} updated (${rows.join(', ')})`); } + if (awaitingOnboarding) { + log.warn(`deploy: deploying with the email setup ${configName} already carries.`); + return { satisfied: false, configWritten, map: mapFor(pre.addresses, false) }; + } return { satisfied: true, configWritten, diff --git a/packages/void/src/cli/deploy-managed-events.ts b/packages/void/src/cli/deploy-managed-events.ts index 3c3740b02..8f62953d3 100644 --- a/packages/void/src/cli/deploy-managed-events.ts +++ b/packages/void/src/cli/deploy-managed-events.ts @@ -299,6 +299,11 @@ export async function streamDirectR2Deploy({ spinner: SpinnerResult; cliLog?: CliLogger; }): Promise { + // Check before any upload: an older platform would accept the manifest and + // leave replication on. + if (manifest.database?.readReplication !== undefined) { + await client.requireD1ReadReplicationSupport(); + } const workerInput = workerFiles && workerFiles.size > 0 && (await client.supportsDirectWorkerUpload()) ? await uploadWorkerModules(client, projectId, workerFiles, s, cliLog) diff --git a/packages/void/src/cli/email-cloudflare.ts b/packages/void/src/cli/email-cloudflare.ts index 7ee08240f..b094df5bb 100644 --- a/packages/void/src/cli/email-cloudflare.ts +++ b/packages/void/src/cli/email-cloudflare.ts @@ -38,6 +38,36 @@ export function subdomainDashboardFallback(apex: string, domain: string): string export const SENDING_FREE_PLAN_HINT = 'Workers Paid needed for arbitrary recipients. Inbound works; sendEmail() to verified destinations only.'; +const CLOUDFLARE_WORKERS_PAID_URL = 'https://dash.cloudflare.com/?to=/:account/workers/plans'; + +/** + * Why a sending-only app (no `email/` handlers) has no working sender: it sends only through + * Email Sending, which Workers Free does not offer. Names the plan, its link and the token + * permissions a sending-only setup needs. + */ +export function formatSendingOnlyPaidHint(domain: string): Array { + return [ + `Email Sending is not onboarded on ${domain}. Apps without email/ handlers send through Cloudflare Email Sending, which requires the Workers Paid plan.`, + `Enable Workers Paid: ${CLOUDFLARE_WORKERS_PAID_URL}`, + 'After upgrading, run `void email setup --platform cloudflare`. An API token needs Zone Read + Email Sending Edit.', + ]; +} + +/** + * A sending-only app whose saved setup (the committed binding) still waits for Email Sending + * onboarding, with nothing else in the way. Preflight cannot tell Workers Free from a domain + * never onboarded, so deploys leave the retry to `void email setup` instead of asking again. + */ +export function isAwaitingSendingOnboarding(pre: EmailPreflight): boolean { + return ( + !pre.ready && + pre.usage.handlers.length === 0 && + pre.binding.state === 'present' && + pre.sendingRetryable && + !pre.blockers.some((b) => b.scope === 'all') + ); +} + // ── Injected seams ───────────────────────────────────────────────────────────── /** Provider results are structured at the subprocess boundary. */ @@ -403,6 +433,30 @@ export function readSendingSettings(data: unknown): SendingSettings | null { : null; } +/** + * The `email.sending.settings` result for `domain`, built from a bearer read of + * `GET /zones/{z}/email/sending/subdomains`: the matching row, `data: null` when the domain + * is not listed (not onboarded), or a failure when the list could not be read. + */ +function sendingSettingsFromList( + response: SessionBearerResponse | null, + domain: string, +): EmailWranglerRunResult { + if (!response?.ok || !Array.isArray(response.result)) { + const first = response?.errors[0]; + return { + status: 1, + stdout: '', + stderr: first?.message ?? '', + ...(first?.code ? { errorCode: first.code } : {}), + }; + } + const row = (response.result as Array).find( + (item) => isRecord(item) && typeof item.name === 'string' && item.name.toLowerCase() === domain, + ); + return { status: 0, stdout: '', stderr: '', data: row ?? null }; +} + function providerErrorCode(result: EmailWranglerRunResult): number | null { return result.errorCode ?? result.errors?.find((error) => error.code !== undefined)?.code ?? null; } @@ -708,10 +762,14 @@ export async function preflightEmail(inputs: EmailCloudflareInputs): Promise null); @@ -783,6 +841,16 @@ export async function preflightEmail(inputs: EmailCloudflareInputs): Promise !foreign.has(a)); // 7. Sending. Not-found is a clean "not onboarded"; anything else unparsable is unknown. - const sendingResult = await run({ type: 'email.sending.settings', domain }); + // Sending-only apps reach this point only after the bearer resolved the zone above. + const sendingResult: EmailWranglerRunResult = receivesEmail + ? await run({ type: 'email.sending.settings', domain }) + : sendingSettingsFromList(sendingRead, domain); if (isHardFailure(sendingResult)) { pre.sending.state = 'unknown'; } else if (sendingResult.status === 0 && sendingResult.data === null) { @@ -1103,7 +1174,9 @@ export function renderEmailChecklist(pre: EmailPreflight): string { : pre.sending.state === 'unknown' ? 'unknown — could not verify; retry void email status --platform cloudflare' : pre.sendingRetryable && pre.binding.state === 'present' - ? 'not onboarded — verified destinations only; after upgrading to Workers Paid run void email setup --platform cloudflare' + ? receivesEmail + ? 'not onboarded — verified destinations only; after upgrading to Workers Paid run void email setup --platform cloudflare' + : 'not onboarded — needs Workers Paid; after upgrading run void email setup --platform cloudflare' : 'not onboarded'; row('sending', `${pre.domain} ${sendingWord}`); if (receivesEmail && pre.derived.addresses.length === 0) { @@ -1306,8 +1379,10 @@ export interface EmailApplyResult { * routing apex → `email routing enable ` * subdomain → bearer POST /zones/{z}/email/routing/dns { name } * subaddr bearer PATCH /zones/{z}/email/routing { support_subaddress: true } - * sending `email sending enable `; code 2040 (already onboarded) is ok; - * any other failure prints a hint and continues without sending + * sending inbound → `email sending enable ` + * sending-only → bearer POST /zones/{z}/email/sending/subdomains { name } + * code 2040 (already onboarded) is ok; any other failure prints a hint and + * continues without sending * * A failed bearer call prints the dashboard fallback and withholds `addresses` this run; * it never fails the deploy. An `all` blocker applies nothing and returns an empty patch. @@ -1458,30 +1533,63 @@ export async function applyEmailSetup( })`, ); } - bearer = null; // 3. Sending. 2040 = already onboarded. Anything else: hint, continue without it. + // Sending-only apps onboard through the bearer on the zone preflight resolved, since + // the provider command first reads Email Routing settings (Zone Settings Read). + const enableSending = async (): Promise<{ + ok: boolean; + code: number | null; + noBearer?: boolean; + }> => { + if (receivesEmail) { + const enable = await run({ type: 'email.sending.enable', domain: pre.domain }); + const code = isHardFailure(enable) ? null : providerErrorCode(enable); + return { ok: !isHardFailure(enable) && (enable.status === 0 || code === 2040), code }; + } + const token = await borrow(); + if (!token) { + return { ok: false, code: null, noBearer: true }; + } + const response = await fetchWithSessionBearer( + inputs.fetchImpl, + token, + 'POST', + `/zones/${pre.zone.zoneId}/email/sending/subdomains`, + { name: pre.domain }, + apiBaseUrl, + ); + const code = response.errors.find((error) => error.code !== 0)?.code ?? null; + return { ok: response.ok || code === 2040, code }; + }; const sendingBlocked = pre.blockers.filter((b) => b.scope === 'sending'); if (!result.sendingReady && sendingBlocked.length > 0) { for (const b of sendingBlocked) { lines.push(` ! sending ${b.message}`); } } else if (!result.sendingReady) { - const enable = await run({ type: 'email.sending.enable', domain: pre.domain }); - const code = isHardFailure(enable) ? null : providerErrorCode(enable); - if (!isHardFailure(enable) && (enable.status === 0 || code === 2040)) { + const { ok, code, noBearer } = await enableSending(); + if (ok) { result.sendingReady = true; lines.push(` ✔ sending ${pre.domain} onboarded`); + } else if (noBearer) { + // Nothing reached Cloudflare, so the plan is not the reason. + lines.push( + ` ! sending ${pre.domain} not onboarded (session bearer unavailable); rerun void email setup --platform cloudflare`, + ); } else { const known = code ? CF_EMAIL_ERROR_CODES[code] : undefined; lines.push( ` ! sending ${pre.domain} not onboarded${code ? ` [code: ${code}]` : ''}${known ? ` — ${known}` : ''}`, ); - lines.push( - ` ${receivesEmail ? SENDING_FREE_PLAN_HINT : 'Workers Paid needed for arbitrary recipients. Email Sending is not configured; verified destinations only with an existing send binding.'}`, - ); + for (const hint of receivesEmail + ? [SENDING_FREE_PLAN_HINT] + : formatSendingOnlyPaidHint(pre.domain)) { + lines.push(` ${hint}`); + } } } + bearer = null; // 4. The patch. The binding is written only when routing is ready or sending is // onboarded (a bare account never carries it). `addresses` is written only when @@ -1526,6 +1634,11 @@ export function renderAddressMap(map: { sendingReady: boolean; /** The pending marker; `void email status` passes one that does not name a deploy. */ pendingLabel?: string; + /** + * An app without `email/` handlers. Void never enables Email Routing for it, so before + * onboarding the map cannot promise the verified-destination fallback. + */ + sendingOnly?: boolean; /** Written addresses whose rule does not exist yet — `wrangler deploy` creates it. */ creating?: Array; }): string { @@ -1542,7 +1655,7 @@ export function renderAddressMap(map: { lines.push(` inbound ${pad(route.address, width + 2)}→ ${route.handler}${marker}`); } lines.push( - ` outbound sendEmail() from ${map.from}${map.sendingReady ? '' : ' (verified destinations only)'}`, + ` outbound sendEmail() from ${map.from}${map.sendingReady ? '' : map.sendingOnly ? ' (Email Sending not onboarded)' : ' (verified destinations only)'}`, ); return lines.join('\n'); } diff --git a/packages/void/src/cli/email.ts b/packages/void/src/cli/email.ts index b9aab3e3a..51b982411 100644 --- a/packages/void/src/cli/email.ts +++ b/packages/void/src/cli/email.ts @@ -29,6 +29,8 @@ import { applyEmailSetup, createNodeEmailDns, emailConfigInputs, + formatSendingOnlyPaidHint, + isAwaitingSendingOnboarding, preflightEmail, probeMx, renderAddressMap, @@ -1132,6 +1134,8 @@ async function runEmailCloudflareCommandInConfig( return fail(`email: ${err instanceof Error ? err.message : String(err)}`); } print(renderEmailChecklist(pre)); + const sendingOnly = pre.usage.handlers.length === 0; + const awaitingOnboarding = isAwaitingSendingOnboarding(pre); if (args.subcommand === 'status') { print( @@ -1140,6 +1144,7 @@ async function runEmailCloudflareCommandInConfig( routes: pre.derived.routes, addresses: pre.rules.mine, sendingReady: pre.sending.state === 'ready', + sendingOnly, pendingLabel: '(no rule yet)', }), ); @@ -1147,6 +1152,10 @@ async function runEmailCloudflareCommandInConfig( log.info(`Email is set up on ${pre.domain}.`); return; } + if (awaitingOnboarding) { + log.warn(formatSendingOnlyPaidHint(pre.domain).join('\n')); + return exit(1); + } log.warn( `Email is not set up on ${pre.domain} yet — run \`void email setup --platform cloudflare\`.`, ); @@ -1154,7 +1163,8 @@ async function runEmailCloudflareCommandInConfig( } // 4. Confirm. A ready preflight needs no prompt, except a sending onboarding the committed - // binding records as refused: apply would retry it, so it gets its own prompt. + // binding records as refused: apply would retry it, so it gets its own prompt. So does + // a saved sending-only setup still waiting for onboarding (deploys never retry it). if (pre.blockers.some((b) => b.scope === 'all')) { return fail('email: nothing was changed — fix the ✘ rows above and rerun.'); } @@ -1173,7 +1183,9 @@ async function runEmailCloudflareCommandInConfig( }); const question = pre.ready ? `Onboard ${pre.domain} for Email Sending? Inbound already works; onboarding needs Workers Paid.` - : `Set up email on ${pre.domain}?`; + : awaitingOnboarding + ? `Onboard ${pre.domain} for Email Sending? Sending-only apps need Workers Paid.` + : `Set up email on ${pre.domain}?`; if (!(await ask(question))) { log.info('Nothing changed.'); return; @@ -1199,14 +1211,18 @@ async function runEmailCloudflareCommandInConfig( routes: pre.derived.routes, addresses: out.patch.addresses ?? [], sendingReady: out.sendingReady, + sendingOnly, pendingLabel: '(no rule yet)', }), ); // Match deploy's capability checks. Apps receiving mail can retain their Workers Free // setup when sending onboarding fails; sending-only apps must verify sending itself. - if (!out.ready && pre.usage.handlers.length === 0) { + // A refused onboarding has already printed the Workers Paid steps with the apply rows. + if (!out.ready && sendingOnly) { log.warn( - `Email Sending is not configured on ${pre.domain}. Check the sending status above, then retry \`void email setup --platform cloudflare\`.`, + pre.sendingRetryable + ? `Email Sending is not onboarded on ${pre.domain}; sendEmail() needs it in apps without email/ handlers. Follow the Workers Paid steps above.` + : `Email Sending is not configured on ${pre.domain}. Check the sending status above, then retry \`void email setup --platform cloudflare\`.`, ); return exit(1); } diff --git a/packages/void/src/cli/help.ts b/packages/void/src/cli/help.ts index b028c9000..27a9f8449 100644 --- a/packages/void/src/cli/help.ts +++ b/packages/void/src/cli/help.ts @@ -1284,6 +1284,7 @@ const HELP_TREE: HelpTree = { description: 'Run against the deployed database (D1 via proxy; PostgreSQL via direct connection)', }, + { command: '--yes', description: 'Skip the confirmation prompt for --remote' }, ], examples: [ { command: 'void db execute "select * from users"', description: 'Run inline SQL' }, diff --git a/packages/void/src/cli/init-report.ts b/packages/void/src/cli/init-report.ts index f6b6b08f4..1b21bb688 100644 --- a/packages/void/src/cli/init-report.ts +++ b/packages/void/src/cli/init-report.ts @@ -19,13 +19,23 @@ export class InitReport { readonly completed: Array = []; /** What the user should do next. */ readonly nextSteps: Array = []; + /** The new starter in one line. See `setStarter`. */ + private starter: string | undefined; + /** Files recorded by the time the starter was created. */ + private starterFiles = new Set(); + private printed = false; + /** - * A new starter in one line, such as "Created a React Pages starter with D1 - * (SQLite)". The summary shows it alone, since listing the starter's files - * would add little. + * Record a new starter in one line, such as "Created a React Pages starter + * with D1 (SQLite)". The summary shows it instead of the files recorded so + * far, since listing the starter's files would add little. It still lists + * files outside the new app (a workspace root's pnpm-workspace.yaml) and + * later changes to files that existed before init. */ - starter: string | undefined; - private printed = false; + setStarter(summary: string): void { + this.starter = summary; + this.starterFiles = new Set(this.files.map((file) => file.path)); + } /** Record a new file. Later changes to it in the same run keep it a new file. */ created(path: string): void { @@ -66,9 +76,9 @@ export class InitReport { } /** One line per file: created files first, then updated files with their details aligned. */ - fileLines(): Array { - const created = this.files.filter((file) => file.kind === 'created'); - const updated = this.files.filter((file) => file.kind === 'updated'); + fileLines(files: ReadonlyArray = this.files): Array { + const created = files.filter((file) => file.kind === 'created'); + const updated = files.filter((file) => file.kind === 'updated'); const width = Math.max(0, ...updated.map((file) => file.path.length)); return [ ...created.map((file) => `+ ${file.path}`), @@ -76,6 +86,23 @@ export class InitReport { ]; } + /** + * The files the summary lists. Under a starter, these are the files outside + * the new app and the later changes to files that existed before init. + * Report paths are relative to the app and use `/`. + */ + private shownFiles(): Array { + if (!this.starter) { + return this.files; + } + return this.files.filter( + (file) => + file.path === '..' || + file.path.startsWith('../') || + (file.kind === 'updated' && !this.starterFiles.has(file.path)), + ); + } + /** * Print the summary of a finished run: the new starter, or else the changed * files. A `targeted` run (`--tsconfig`, `--github`) checks only part of the @@ -83,15 +110,13 @@ export class InitReport { */ print(targeted = false): void { this.printed = true; + const lines = this.fileLines(this.shownFiles()); if (this.starter) { - printDone(this.starter); + printDone(lines.length > 0 ? [this.starter, ...lines] : this.starter); + } else if (lines.length > 0) { + printDone(['Configured Void', ...lines]); } else { - const lines = this.fileLines(); - if (lines.length > 0) { - printDone(['Configured Void', ...lines]); - } else { - printDone(targeted ? 'No files changed.' : 'Void is already set up. No files changed.'); - } + printDone(targeted ? 'No files changed.' : 'Void is already set up. No files changed.'); } this.printOtherWork(); } @@ -121,17 +146,17 @@ export class InitReport { return; } this.printed = true; - const lines = this.starter ? [] : this.fileLines(); - if (lines.length > 0) { + const lines = this.fileLines(this.shownFiles()); + if (this.starter) { + log.warn('Setup did not finish. Done so far:'); + printDone(lines.length > 0 ? [this.starter, ...lines] : this.starter); + } else if (lines.length > 0) { log.warn(['Setup did not finish. Files changed so far:', ...lines].join('\n')); - } else if (this.starter || this.completed.length > 0 || this.nextSteps.length > 0) { + } else if (this.completed.length > 0 || this.nextSteps.length > 0) { log.warn('Setup did not finish. Done so far:'); } else { return; } - if (this.starter) { - printDone(this.starter); - } this.printOtherWork(); } diff --git a/packages/void/src/cli/init-vite-plus.ts b/packages/void/src/cli/init-vite-plus.ts index 181d9a358..e66dd2fce 100644 --- a/packages/void/src/cli/init-vite-plus.ts +++ b/packages/void/src/cli/init-vite-plus.ts @@ -6,26 +6,30 @@ import { SCAFFOLD_VITE_PLUS_DEV_DEPENDENCIES } from './template.ts'; type JsonObject = Record; -/** Resolve a project's catalog reference before invoking npx outside its workspace. */ +/** + * Resolve a project's catalog reference before invoking npx outside its + * workspace. pnpm, Bun and Yarn name the default catalog `catalog:` or + * `catalog:default`, and write it as `catalog` or `catalogs.default`. + */ export function resolveVitePlusBootstrapSpec(root: string, pkg: JsonObject): string { const spec = getPackageDependencies(pkg)['vite-plus']; if (spec && !spec.startsWith('catalog:')) { return spec; } if (spec) { - const catalog = spec.slice('catalog:'.length); + const catalog = spec.slice('catalog:'.length) || 'default'; let directory = root; while (true) { const workspace = join(directory, 'pnpm-workspace.yaml'); - const config = existsSync(workspace) - ? parseDocument(readFileSync(workspace, 'utf8')).toJS() - : readPackageJson(directory); - const entries = catalog - ? (config?.catalogs?.[catalog] ?? config?.workspaces?.catalogs?.[catalog]) - : (config?.catalog ?? config?.workspaces?.catalog); - const version = entries?.['vite-plus']; - if (typeof version === 'string' && version && !version.startsWith('catalog:')) { - return version; + const yarnrc = join(directory, '.yarnrc.yml'); + const config = existsSync(workspace) ? readYaml(workspace) : readPackageJson(directory); + // Yarn keeps its catalogs in .yarnrc.yml, next to the root package.json. + const configs = existsSync(yarnrc) ? [config, readYaml(yarnrc)] : [config]; + for (const entry of configs) { + const version = catalogEntries(entry, catalog)?.['vite-plus']; + if (typeof version === 'string' && version && !version.startsWith('catalog:')) { + return version; + } } if (existsSync(workspace) || config?.workspaces || dirname(directory) === directory) { break; @@ -33,9 +37,29 @@ export function resolveVitePlusBootstrapSpec(root: string, pkg: JsonObject): str directory = dirname(directory); } } - throw new Error( - 'Could not resolve the project’s Vite+ version. Install its dependencies before running void init.', - ); + // `vp install` then installs the project's own Vite+, so this version only + // picks the vp that runs that install. + return SCAFFOLD_VITE_PLUS_DEV_DEPENDENCIES['vite-plus']; +} + +/** A named catalog in a workspace config, or in its `workspaces` field (Bun). */ +function catalogEntries(config: JsonObject | null, name: string): JsonObject | undefined { + for (const scope of [config, config?.workspaces]) { + if (!scope || typeof scope !== 'object') { + continue; + } + const { catalog, catalogs } = scope as { catalog?: unknown; catalogs?: unknown }; + const named = (catalogs as JsonObject | undefined)?.[name]; + const entries = name === 'default' ? (catalog ?? named) : named; + if (entries && typeof entries === 'object') { + return entries as JsonObject; + } + } + return undefined; +} + +function readYaml(path: string): JsonObject | null { + return parseDocument(readFileSync(path, 'utf8')).toJS() as JsonObject | null; } /** Keep every Vite consumer on the core used by the selected Vite+ toolchain. */ diff --git a/packages/void/src/cli/init.ts b/packages/void/src/cli/init.ts index 20f2f8423..c2e381c34 100644 --- a/packages/void/src/cli/init.ts +++ b/packages/void/src/cli/init.ts @@ -2868,10 +2868,12 @@ async function runInitSteps(root: string, report: InitReport, initArgs?: InitArg scaffoldToolchain, report, ); - report.starter = describeStarter( - scaffoldedPagesFramework, - starterMode, - root === requestedRoot ? undefined : relative(requestedRoot, root), + report.setStarter( + describeStarter( + scaffoldedPagesFramework, + starterMode, + root === requestedRoot ? undefined : relative(requestedRoot, root), + ), ); // A workspace member installs from the workspace root: a leaf install would diff --git a/packages/void/src/runtime/ai/index.ts b/packages/void/src/runtime/ai/index.ts index 0cd41a381..e4bd1dbfc 100644 --- a/packages/void/src/runtime/ai/index.ts +++ b/packages/void/src/runtime/ai/index.ts @@ -611,8 +611,15 @@ export type VoidAi = { stream(model: string, inputs: Record, options?: AiOptions): Promise; }; -/** Normalize provider 429 responses and binding quota failures before they reach callers. */ -async function aiOperation(execute: () => Promise, rawStream = false): Promise { +/** + * Normalize provider 429 responses and binding quota failures before they reach callers. + * `signal` is the caller's own cancellation of a provider request; see `streamResponse`. + */ +async function aiOperation( + execute: () => Promise, + rawStream = false, + signal?: AbortSignal | null, +): Promise { let value: T; try { value = await execute(); @@ -630,7 +637,7 @@ async function aiOperation(execute: () => Promise, rawStream = false): Pro value.body && value.headers.get('content-type')?.includes('text/event-stream') ) { - return new Response(streamResponse(value.body).body, value) as T; + return new Response(streamResponse(value.body, signal).body, value) as T; } if (rawStream && value instanceof ReadableStream) { return streamResponse(value).body as T; @@ -656,8 +663,12 @@ function classifyAiLimit(error: unknown): UsageLimit | null { : null; } -/** Streaming failures after headers are sent end with an explicit terminal SSE event. */ -function streamResponse(body: ReadableStream): Response { +/** + * Streaming failures after headers are sent end with an explicit terminal SSE + * event. A provider request the caller cancelled through `signal` instead fails + * with the caller's reason, as a plain fetch does. + */ +function streamResponse(body: ReadableStream, signal?: AbortSignal | null): Response { const reader = body.getReader(); return new Response( new ReadableStream({ @@ -670,6 +681,10 @@ function streamResponse(body: ReadableStream): Response { controller.enqueue(chunk.value); } } catch (error) { + if (signal?.aborted) { + controller.error(signal.reason); + return; + } const limit = classifyAiLimit(error); let data: unknown = { message: 'Generation was interrupted. Please try again.' }; if (limit) { @@ -706,7 +721,11 @@ export const ai: VoidAi = new Proxy({} as VoidAi, { if (prop === 'provider') { return (provider: string, options?: AiProviderOptions) => ({ fetch: (path: string, init?: AiProviderFetchInit) => - aiOperation(() => backend().providerFetch(provider, path, init, options)), + aiOperation( + () => backend().providerFetch(provider, path, init, options), + false, + init?.signal, + ), }); } if (prop === 'run' || prop === 'image' || prop === 'models' || prop === 'toMarkdown') { diff --git a/packages/void/test/integration/init-install.test.ts b/packages/void/test/integration/init-install.test.ts index d3d0f1a4f..96b728765 100644 --- a/packages/void/test/integration/init-install.test.ts +++ b/packages/void/test/integration/init-install.test.ts @@ -3,6 +3,7 @@ import { tmpdir } from 'node:os'; import { join } from 'node:path'; import { afterEach, beforeEach, describe, expect, it, vi } from 'vite-plus/test'; import { installProjectDependencies } from '../../src/cli/init.ts'; +import { SCAFFOLD_VITE_PLUS_DEV_DEPENDENCIES } from '../../src/cli/template.ts'; let root: string; let bin: string; @@ -100,41 +101,44 @@ describe('init dependency installation (real subprocesses)', () => { ); it.each([ - ['pnpm', ''], - ['pnpm', 'tools'], - ['bun', ''], - ['bun', 'tools'], + ['pnpm', 'catalog:', 'catalog'], + ['pnpm', 'catalog:', 'catalogs.default'], + ['pnpm', 'catalog:default', 'catalog'], + ['pnpm', 'catalog:default', 'catalogs.default'], + ['pnpm', 'catalog:tools', 'catalogs.tools'], + ['bun', 'catalog:', 'catalog'], + ['bun', 'catalog:default', 'catalogs.default'], + ['bun', 'catalog:tools', 'catalogs.tools'], + ['yarn', 'catalog:', 'catalog'], + ['yarn', 'catalog:tools', 'catalogs.tools'], ] as const)( - 'bootstraps the Vite+ version from the %s workspace catalog (%s)', - (manager, catalog) => { + 'bootstraps the Vite+ version from the %s workspace catalog (%s in %s)', + (manager, spec, location) => { const member = join(root, 'packages/app'); mkdirSync(member, { recursive: true }); writeFileSync( join(member, 'package.json'), - JSON.stringify({ - devDependencies: { 'vite-plus': 'catalog:' + catalog }, - }), + JSON.stringify({ devDependencies: { 'vite-plus': spec } }), ); + const entries = { 'vite-plus': '1.6.0' }; + const catalogs = + location === 'catalog' + ? { catalog: entries } + : { catalogs: { [location.slice('catalogs.'.length)]: entries } }; + // YAML accepts JSON, so the YAML files can be written as JSON. if (manager === 'pnpm') { writeFileSync( join(root, 'pnpm-workspace.yaml'), - 'packages: [packages/*]\n' + - (catalog - ? 'catalogs: { tools: { vite-plus: "1.6.0" } }\n' - : 'catalog: { vite-plus: "1.6.0" }\n'), + JSON.stringify({ packages: ['packages/*'], ...catalogs }), ); - } else { + } else if (manager === 'bun') { writeFileSync( join(root, 'package.json'), - JSON.stringify({ - workspaces: { - packages: ['packages/*'], - ...(catalog - ? { catalogs: { tools: { 'vite-plus': '1.6.0' } } } - : { catalog: { 'vite-plus': '1.6.0' } }), - }, - }), + JSON.stringify({ workspaces: { packages: ['packages/*'], ...catalogs } }), ); + } else { + writeFileSync(join(root, 'package.json'), JSON.stringify({ workspaces: ['packages/*'] })); + writeFileSync(join(root, '.yarnrc.yml'), JSON.stringify(catalogs)); } executable('npx'); installProjectDependencies(member, manager); @@ -148,6 +152,34 @@ describe('init dependency installation (real subprocesses)', () => { }, ); + it('bootstraps with the CLI’s own Vite+ version when the catalog entry is missing', () => { + const member = join(root, 'packages/app'); + mkdirSync(member, { recursive: true }); + writeFileSync( + join(member, 'package.json'), + JSON.stringify({ devDependencies: { 'vite-plus': 'catalog:missing' } }), + ); + writeFileSync( + join(root, 'pnpm-workspace.yaml'), + 'packages: [packages/*]\ncatalog: { vite-plus: "1.6.0" }\n', + ); + executable('npx'); + installProjectDependencies(member, 'pnpm'); + expect(calls()).toEqual([ + { + command: 'npx', + args: [ + '--yes', + `--package=vite-plus@${SCAFFOLD_VITE_PLUS_DEV_DEPENDENCIES['vite-plus']}`, + 'vp', + 'install', + '--no-frozen-lockfile', + ], + cwd: member, + }, + ]); + }); + it('reports a Vite+ install failure without retrying through another installer', () => { executable('vp', 7); executable('npx'); diff --git a/packages/void/test/unit/ai.test.ts b/packages/void/test/unit/ai.test.ts index 74828e759..e69601a05 100644 --- a/packages/void/test/unit/ai.test.ts +++ b/packages/void/test/unit/ai.test.ts @@ -18,9 +18,16 @@ function makeDirectAi( } describe('runtime ai', () => { - it.each(['direct', 'http', 'service'] as const)( - 'cancels provider response streams on the %s transport without serializing the signal', - async (transport) => { + it.each([ + ['direct', 'application/json'], + ['http', 'application/json'], + ['service', 'application/json'], + ['direct', 'text/event-stream'], + ['http', 'text/event-stream'], + ['service', 'text/event-stream'], + ] as const)( + 'cancels provider response streams on the %s transport (%s) without serializing the signal', + async (transport, contentType) => { vi.resetModules(); const { envContext } = await import('../../src/runtime/env/raw.ts'); const { ai } = await import('../../src/runtime/ai/index.ts'); @@ -30,9 +37,11 @@ describe('runtime ai', () => { new Response( new ReadableStream({ start(stream) { + stream.enqueue(new TextEncoder().encode('data: {"a":1}\n\n')); init.signal?.addEventListener('abort', () => stream.error(init.signal!.reason)); }, }), + { headers: { 'content-type': contentType } }, ), ); vi.stubGlobal('fetch', request); @@ -67,6 +76,46 @@ describe('runtime ai', () => { }, ); + it('ends a provider SSE stream that fails without a cancellation with a void:error event', async () => { + vi.resetModules(); + const { envContext } = await import('../../src/runtime/env/raw.ts'); + const { ai } = await import('../../src/runtime/ai/index.ts'); + const controller = new AbortController(); + let pulls = 0; + vi.stubGlobal( + 'fetch', + vi.fn( + async () => + new Response( + new ReadableStream({ + pull(stream) { + if (pulls++ === 0) { + stream.enqueue(new TextEncoder().encode('data: {"a":1}\n\n')); + } else { + stream.error(new Error('upstream reset')); + } + }, + }), + { headers: { 'content-type': 'text/event-stream' } }, + ), + ), + ); + const env = { + AI: makeDirectAi().AI, + __VOID_AI_GATEWAY: 'gateway', + OPENAI_API_KEY: 'synthetic-provider-key', + }; + await envContext.run(env, async () => { + const response = await ai.provider('openai').fetch('/chat/completions', { + body: { messages: [], stream: true }, + signal: controller.signal, + }); + const text = await response.text(); + expect(text).toContain('data: {"a":1}'); + expect(text).toContain('event: void:error'); + }); + }); + it.each(['http', 'service'] as const)( 'keeps cancellation on the %s transport instead of serializing it as a binding option', async (transport) => { diff --git a/packages/void/test/unit/cf-d1-import.test.ts b/packages/void/test/unit/cf-d1-import.test.ts index 50b8a43ba..fbb0528e2 100644 --- a/packages/void/test/unit/cf-d1-import.test.ts +++ b/packages/void/test/unit/cf-d1-import.test.ts @@ -2,7 +2,7 @@ import { createHash } from 'node:crypto'; import { writeFileSync } from 'node:fs'; import { join } from 'node:path'; import { afterEach, expect, it, vi } from 'vite-plus/test'; -import { importD1SqlFile } from '../../src/cli/cf-d1-import.ts'; +import { d1ImportUploadTimeoutMs, importD1SqlFile } from '../../src/cli/cf-d1-import.ts'; import { temporaryDirectory } from '../helpers/cloudflare-operation-process.ts'; const sql = "INSERT INTO messages VALUES ('synthetic');\n"; @@ -13,7 +13,7 @@ const endpoint = 'https://api.example.com/import'; function fixture() { const file = join(temporaryDirectory('void-d1-import-'), 'input.sql'); writeFileSync(file, sql); - return { file, headers, endpoint }; + return { file, headers, endpoint, pollIntervalMs: 0 }; } function response(result: unknown) { return Response.json({ success: true, result }); @@ -104,3 +104,245 @@ it('rejects non-HTTPS upload URLs without sending SQL or credentials', async () await expect(importD1SqlFile(fixture())).rejects.toThrow('insecure'); expect(fetch).toHaveBeenCalledTimes(1); }); + +async function importError(): Promise { + const error = await importD1SqlFile(fixture()).then( + () => undefined, + (reason: unknown) => reason, + ); + expect(error).toBeInstanceOf(Error); + return error as Error; +} + +it('shows the SQL errors and messages D1 reports for a failed ingest', async () => { + vi.stubGlobal( + 'fetch', + vi + .fn() + .mockResolvedValueOnce(response({ at_bookmark: 'current', status: 'active' })) + .mockResolvedValueOnce( + response({ + success: true, + status: 'error', + at_bookmark: 'current', + errors: ['no such table: nope: SQLITE_ERROR', { message: 'second error' }], + messages: ['Processed 1 queries.'], + }), + ), + ); + expect((await importError()).message).toBe( + 'D1 import failed: no such table: nope: SQLITE_ERROR\nsecond error\nProcessed 1 queries.', + ); +}); + +it('removes terminal escape sequences from D1 import errors', async () => { + vi.stubGlobal( + 'fetch', + vi.fn().mockResolvedValueOnce( + response({ + status: 'error', + errors: ['\u001b]8;;https://x.example\u0007bad\u001b]8;;\u0007 table\u001b[2K'], + }), + ), + ); + expect((await importError()).message).toBe('D1 import failed: bad table'); +}); + +const uploaded = () => [ + response({ filename: 'input.sql', upload_url: 'https://uploads.example.com/sql' }), + new Response('', { headers: { etag } }), +]; +const unknownOutcome = 'The import may still be running on Cloudflare'; + +it('bounds each D1 request and the upload instead of the whole import', async () => { + const fetch = vi.fn(); + for (const value of [ + ...uploaded(), + response({ at_bookmark: 'current', status: 'active' }), + response({ status: 'complete', success: true }), + ]) { + fetch.mockResolvedValueOnce(value); + } + vi.stubGlobal('fetch', fetch); + const timeout = vi.spyOn(AbortSignal, 'timeout'); + await importD1SqlFile(fixture()); + const signals = fetch.mock.calls.map(([, init]) => init?.signal); + for (const signal of signals) { + expect(signal).toBeInstanceOf(AbortSignal); + } + expect(new Set(signals).size).toBe(4); + expect(timeout.mock.calls.map(([ms]) => ms)).toEqual([ + 120_000, + d1ImportUploadTimeoutMs(Buffer.byteLength(sql)), + 120_000, + 120_000, + ]); +}); + +it('gives the upload time for a 25 MB file on a slow link', () => { + // 25 MB at 1 Mbit/s takes 200 seconds, on top of the 2-minute request bound. + expect(d1ImportUploadTimeoutMs(25 * 1024 * 1024)).toBe(320_000); + expect(d1ImportUploadTimeoutMs(0)).toBe(120_000); +}); + +it('keeps a stalled upload a plain failure that names its deadline', async () => { + vi.stubGlobal( + 'fetch', + vi + .fn() + .mockResolvedValueOnce(uploaded()[0]!) + .mockRejectedValueOnce( + new DOMException('The operation was aborted due to timeout', 'TimeoutError'), + ), + ); + const error = await importError(); + expect(error.message).toBe( + `D1 import upload did not finish within ${d1ImportUploadTimeoutMs(Buffer.byteLength(sql)) / 1000} seconds.`, + ); + expect(error.cause).toBeInstanceOf(DOMException); +}); + +it.each([ + { + name: 'times out', + reply: () => Promise.reject(new DOMException('The operation was aborted', 'TimeoutError')), + cause: 'The operation was aborted.', + }, + { + name: 'loses the connection', + reply: () => Promise.reject(new TypeError('fetch failed')), + cause: 'fetch failed.', + }, + { + name: 'answers with a server error', + reply: () => + Promise.resolve( + Response.json({ success: false, errors: [{ message: 'Internal' }] }, { status: 502 }), + ), + cause: 'D1 import failed (HTTP 502): Internal.', + }, + { + name: 'answers with something other than JSON', + reply: () => Promise.resolve(new Response('Bad gateway', { status: 502 })), + cause: 'is not valid JSON', + }, +])( + 'reports an unknown outcome when init $name, since init starts a cached upload', + async ({ reply, cause }) => { + const fetch = vi.fn().mockImplementationOnce(reply); + vi.stubGlobal('fetch', fetch); + const { message } = await importError(); + expect(message).toContain(cause); + expect(message).toContain(unknownOutcome + '.'); + expect(fetch).toHaveBeenCalledTimes(1); + }, +); + +it('reports an unknown outcome with the bookmark when a cached import fails after init', async () => { + vi.stubGlobal( + 'fetch', + vi + .fn() + .mockResolvedValueOnce(response({ at_bookmark: 'cached', status: 'active' })) + .mockRejectedValueOnce(new TypeError('fetch failed')), + ); + expect((await importError()).message).toContain(`${unknownOutcome} (bookmark cached).`); +}); + +it('keeps an init that D1 refused a plain failure', async () => { + vi.stubGlobal( + 'fetch', + vi + .fn() + .mockResolvedValueOnce( + Response.json( + { success: false, errors: [{ message: 'Permission denied' }] }, + { status: 403 }, + ), + ), + ); + expect((await importError()).message).toBe('D1 import failed (HTTP 403): Permission denied'); +}); + +it('keeps an init refused with a non-JSON 4xx a plain failure', async () => { + vi.stubGlobal( + 'fetch', + vi + .fn() + .mockResolvedValueOnce(new Response('Forbidden', { status: 403 })), + ); + expect((await importError()).message).toBe('D1 import failed (HTTP 403).'); +}); + +it('reports an unknown outcome when init asks for no upload but gives no bookmark', async () => { + vi.stubGlobal( + 'fetch', + vi.fn().mockResolvedValueOnce(response({ status: 'active' })), + ); + const { message } = await importError(); + expect(message).toContain('invalid D1 import upload.'); + expect(message).toContain(unknownOutcome + '.'); +}); + +it('keeps an upload request without a filename a plain failure', async () => { + vi.stubGlobal( + 'fetch', + vi + .fn() + .mockResolvedValueOnce(response({ upload_url: 'https://uploads.example.com/sql' })), + ); + expect((await importError()).message).toBe('Cloudflare returned an invalid D1 import upload.'); +}); + +it('reports an unknown outcome with the bookmark when a poll times out', async () => { + vi.stubGlobal( + 'fetch', + vi + .fn() + .mockResolvedValueOnce(response({ at_bookmark: 'current', status: 'active' })) + .mockRejectedValueOnce( + new DOMException('The operation was aborted due to timeout', 'TimeoutError'), + ), + ); + const error = await importError(); + expect(error.message).toBe( + 'D1 did not report whether the import finished: The operation was aborted due to timeout. ' + + `${unknownOutcome} (bookmark current). Check the database before running the file again.`, + ); + expect(error.cause).toBeInstanceOf(DOMException); +}); + +it('reports an unknown outcome when the ingest request fails', async () => { + const fetch = vi.fn(); + for (const value of uploaded()) { + fetch.mockResolvedValueOnce(value); + } + fetch.mockRejectedValueOnce(new TypeError('fetch failed')); + vi.stubGlobal('fetch', fetch); + const { message } = await importError(); + expect(message).toContain('fetch failed.'); + expect(message).toContain(unknownOutcome + '.'); + expect(message).not.toContain('bookmark'); +}); + +it('keeps a D1-reported poll failure a plain failure', async () => { + vi.stubGlobal( + 'fetch', + vi + .fn() + .mockResolvedValueOnce(response({ at_bookmark: 'current', status: 'active' })) + .mockResolvedValueOnce(response({ status: 'error', errors: ['UNIQUE constraint failed'] })), + ); + expect((await importError()).message).toBe('D1 import failed: UNIQUE constraint failed'); +}); + +it('keeps an upload failure a plain failure', async () => { + vi.stubGlobal( + 'fetch', + vi + .fn() + .mockResolvedValueOnce(uploaded()[0]!) + .mockRejectedValueOnce(new TypeError('fetch failed')), + ); + expect((await importError()).message).toBe('fetch failed'); +}); diff --git a/packages/void/test/unit/client-platform-version.test.ts b/packages/void/test/unit/client-platform-version.test.ts index 4a816dfc7..926755d9e 100644 --- a/packages/void/test/unit/client-platform-version.test.ts +++ b/packages/void/test/unit/client-platform-version.test.ts @@ -73,6 +73,37 @@ describe('minimum platform version', () => { }, ); + it.each([true, false, undefined])( + 'requires explicit platform support before applying D1 read replication (advertised: %s)', + async (d1ReadReplication) => { + fetchMock.mockImplementation(async (input: string) => { + if (isPlatformDiscoveryRequest(input)) { + return platformDiscoveryResponse(input, 'hosted'); + } + return Response.json({ skipMigrations: true, d1ReadReplication }); + }); + const result = client().requireD1ReadReplicationSupport(); + if (d1ReadReplication === true) { + await expect(result).resolves.toBeUndefined(); + } else { + await expect(result).rejects.toThrow('cannot apply `database.readReplication`'); + } + expect(requestedPaths()).toEqual(['/.well-known/void-platform', '/capabilities']); + }, + ); + + it('treats a platform without a capabilities route as lacking D1 read replication', async () => { + fetchMock.mockImplementation(async (input: string) => { + if (isPlatformDiscoveryRequest(input)) { + return platformDiscoveryResponse(input, 'hosted'); + } + return Response.json({ error: 'not found' }, { status: 404 }); + }); + await expect(client().requireD1ReadReplicationSupport()).rejects.toThrow( + 'Ask its administrator to upgrade with `void platform upgrade`.', + ); + }); + it.each(['1.10.0', '1.10.1', '1.11.0', '2.0.0', 'hosted'])( 'accepts a platform reporting %s', async (version) => { diff --git a/packages/void/test/unit/cloudflare-deploy.test.ts b/packages/void/test/unit/cloudflare-deploy.test.ts index bcc4656e2..784195ccd 100644 --- a/packages/void/test/unit/cloudflare-deploy.test.ts +++ b/packages/void/test/unit/cloudflare-deploy.test.ts @@ -11053,6 +11053,14 @@ describe('runCloudflareDeploy — native Cloudflare email step', () => { }), [`POST /zones/${ZONE_ID}/email/routing/dns`]: ok({ enabled: true }), [`PATCH /zones/${ZONE_ID}/email/routing`]: ok({ support_subaddress: true }), + // Sending-only apps read and onboard Email Sending through the bearer: not onboarded + // yet, and onboarding succeeds (the inbound runner defaults above say the same). + [`GET /zones/${ZONE_ID}/email/sending/subdomains`]: ok([]), + [`POST /zones/${ZONE_ID}/email/sending/subdomains`]: ok({ + name: 'mail.acme.com', + enabled: true, + tag: '19af4fdbee924a92916afea0de5a8fdd', + }), ...overrides, }; return (async (input: RequestInfo | URL, init?: RequestInit) => { @@ -11188,10 +11196,18 @@ describe('runCloudflareDeploy — native Cloudflare email step', () => { rmSync(join(root, 'email'), { recursive: true }); const { deps, spawns, fetchCalls, exitCalls, trace } = emailHarness({ isCI, - runner: { 'email sending settings mail.acme.com': sendingEnabled }, fetchOverrides: { 'GET /zones?name=mail.acme.com': () => new Response(JSON.stringify({ success: true, result: [] })), + [`GET /zones/${ZONE_ID}/email/sending/subdomains`]: () => + new Response( + JSON.stringify({ + success: true, + result: [ + { name: 'mail.acme.com', enabled: true, tag: '19af4fdbee924a92916afea0de5a8fdd' }, + ], + }), + ), }, confirm: async () => { throw new Error('No inbound setup should be offered'); @@ -11228,16 +11244,18 @@ describe('runCloudflareDeploy — native Cloudflare email step', () => { }); rmSync(join(root, 'email'), { recursive: true }); const { deps, spawns, fetchCalls, confirms, exitCalls, trace } = emailHarness({ - runner: { - 'email sending enable mail.acme.com': enabled - ? cleanExit - : { - ...cleanExit, - status: 1, - errorCode: 10105, - stderr: 'Email Sending is unavailable on this plan [code: 10105]', - }, - }, + fetchOverrides: enabled + ? {} + : { + [`POST /zones/${ZONE_ID}/email/sending/subdomains`]: () => + new Response( + JSON.stringify({ + success: false, + errors: [{ code: 10105, message: 'Email Sending is unavailable on this plan' }], + }), + { status: 403 }, + ), + }, }); const output = captureOutput(); try { @@ -11258,11 +11276,96 @@ describe('runCloudflareDeploy — native Cloudflare email step', () => { expect(confirms).toEqual(['Set up email on mail.acme.com?']); expect(exitCalls).toEqual(enabled ? [] : [1]); expect(trace.includes('build')).toBe(enabled); - expect(spawns.some(({ args }) => args[0] === 'email' && args[1] === 'routing')).toBe(false); - expect(fetchCalls.every(({ method }) => method === 'GET')).toBe(true); + expect(spawns.some(({ args }) => args[0] === 'email')).toBe(false); + expect( + fetchCalls + .filter(({ method }) => method !== 'GET') + .map(({ method, path }) => `${method} ${path}`), + ).toEqual([`POST /zones/${ZONE_ID}/email/sending/subdomains`]); expect(output.lines.join('\n')).toContain('inbound not required'); expect(output.lines.join('\n')).not.toContain('Inbound works'); expect(readRootConfig(root).addresses).toBeUndefined(); + if (!enabled) { + expect(output.lines.join('\n')).toContain( + 'Enable Workers Paid: https://dash.cloudflare.com/?to=/:account/workers/plans', + ); + expect(output.lines.join('\n')).toContain('Zone Read + Email Sending Edit'); + } + }, + ); + + // A sending-only app set up before (binding committed) on an account that cannot onboard + // Email Sending: each deploy says why once, never asks again, and never retries onboarding. + it.each([ + [false, false], + [false, true], + [true, false], + [true, true], + ])( + 'sending-only app with a saved binding is not re-prompted on Workers Free (CI: %s, --require-email: %s)', + async (isCI, requireEmail) => { + const root = emailProject( + { + ...PROVISIONED_D1, + send_email: [{ name: 'SEND_EMAIL' }], + vars: { __VOID_EMAIL_FROM: 'old@mail.acme.com' }, + }, + { + 'routes/send.ts': `import { sendEmail } from 'void/email'; export const POST = () => sendEmail({to: 'a@example.com', subject: 'Hello', text: 'Hello'});`, + }, + ); + rmSync(join(root, 'email'), { recursive: true }); + const { deps, spawns, fetchCalls, confirms, exitCalls, trace } = emailHarness({ + isCI, + // Retrying onboarding would answer 10105; the deploy must not try. + fetchOverrides: { + [`POST /zones/${ZONE_ID}/email/sending/subdomains`]: () => { + throw new Error('a saved sending-only setup must not retry onboarding'); + }, + }, + // Record a prompt instead of answering it, so the assertion names the failure. + confirm: undefined, + }); + const output = captureOutput(); + try { + for (let run = 0; run < 2; run += 1) { + const deploy = runCloudflareDeploy( + root, + { platform: 'cloudflare', requireEmail }, + silentCliLog, + deps, + ); + if (requireEmail) { + await expect(deploy).rejects.toBeInstanceOf(ExitError); + } else { + await deploy; + } + } + } finally { + output.restore(); + } + const printed = output.lines.join('\n'); + expect(confirms).toEqual([]); + expect(spawns.some((s) => s.args[0] === 'email')).toBe(false); + expect(fetchCalls.every(({ method }) => method === 'GET')).toBe(true); + expect(printed).toContain( + 'Enable Workers Paid: https://dash.cloudflare.com/?to=/:account/workers/plans', + ); + expect(printed).toContain('Zone Read + Email Sending Edit'); + expect(printed).not.toContain('cannot ask'); + expect(printed).not.toContain('This changes YOUR Cloudflare account'); + expect(exitCalls).toEqual(requireEmail ? [1, 1] : []); + expect(trace.filter((t) => t === 'build')).toHaveLength(requireEmail ? 0 : 2); + if (!requireEmail) { + expect(printed).toContain( + 'outbound sendEmail() from noreply@mail.acme.com (Email Sending not onboarded)', + ); + } + // The saved setup ships as-is, with the sender brought up to date. + const written = readRootConfig(root); + expect(written.send_email).toEqual([{ name: 'SEND_EMAIL' }]); + expect(written.vars).toEqual({ __VOID_EMAIL_FROM: 'noreply@mail.acme.com' }); + expect(written.addresses).toBeUndefined(); }, ); @@ -12269,7 +12372,7 @@ describe('runCloudflareDeploy — native Cloudflare email step', () => { ...INIT_MIGRATION, }); const snapshots: Array = []; - const { deps, spawns, confirms, exitCalls } = emailHarness({ + const { deps, spawns, fetchCalls, confirms, exitCalls } = emailHarness({ ...readyRows, runner: { ...readyRows.runner, 'email routing rules list acme.com': myRule }, deps: { @@ -12303,8 +12406,8 @@ describe('runCloudflareDeploy — native Cloudflare email step', () => { expect(exitCalls).toEqual([]); expect(confirms).toEqual([]); // Sending-only preflight still flags stale configured addresses without probing routing. - expect(spawns.some((s) => s.args[0] === 'email' && s.args[1] === 'routing')).toBe(false); - expect(spawns.some((s) => s.args[0] === 'email' && s.args[1] === 'sending')).toBe(true); + expect(spawns.some((s) => s.args[0] === 'email')).toBe(false); + expect(fetchCalls.some(({ path }) => path.endsWith('/email/sending/subdomains'))).toBe(true); expect(readRootConfig(root).addresses).toEqual(['support@mail.acme.com']); const printed = output.lines.join('\n'); expect(printed).toContain( diff --git a/packages/void/test/unit/cloudflare-operation-process.test.ts b/packages/void/test/unit/cloudflare-operation-process.test.ts index be72b235d..53aeefcb5 100644 --- a/packages/void/test/unit/cloudflare-operation-process.test.ts +++ b/packages/void/test/unit/cloudflare-operation-process.test.ts @@ -877,6 +877,46 @@ it('sends D1 SQL that starts with a comment through the pinned cf', async () => expect(bodies).toEqual([{ sql }]); }, 30_000); +it('sends a CRLF D1 SQL file to the query API with LF outside quoted values', async () => { + const directory = temporaryDirectory('void-cf-d1-crlf-sql-'); + const bodies: Array = []; + const api = await listen( + createServer(async (request, response) => { + let body = ''; + for await (const chunk of request) { + body += chunk; + } + bodies.push(JSON.parse(body)); + response.setHeader('Content-Type', 'application/json'); + response.end(JSON.stringify({ success: true, result: [{ results: [], success: true }] })); + }), + ); + const configPath = join(directory, 'wrangler.json'); + writeFileSync( + configPath, + JSON.stringify({ + account_id: 'test-account', + d1_databases: [{ binding: 'DB', database_id: 'test-database' }], + }), + ); + const file = join(directory, 'trigger.sql'); + writeFileSync( + file, + "CREATE TRIGGER trg BEFORE DELETE ON probe_z\r\nBEGIN\r\n SELECT RAISE(ABORT, 'no\r\nchange');\r\nEND;\r\n", + ); + const output = await runOperation( + directory, + { type: 'd1.execute', database: 'DB', file, configPath }, + tokenEnv(api), + ); + expect(output.status, output.stderr).toBe(0); + expect(bodies).toEqual([ + { + sql: "CREATE TRIGGER trg BEFORE DELETE ON probe_z\nBEGIN\n SELECT RAISE(ABORT, 'no\r\nchange');\nEND;\n", + }, + ]); +}, 30_000); + it('preserves rows from large D1 queries without importing or exceeding argv limits', async () => { const directory = temporaryDirectory('void-cf-d1-large-query-'); const sql = '/*' + 'synthetic comment '.repeat(150_000) + '*/ SELECT 42 AS answer;'; diff --git a/packages/void/test/unit/d1-sql-line-endings.test.ts b/packages/void/test/unit/d1-sql-line-endings.test.ts new file mode 100644 index 000000000..275a24517 --- /dev/null +++ b/packages/void/test/unit/d1-sql-line-endings.test.ts @@ -0,0 +1,38 @@ +import { expect, it } from 'vite-plus/test'; +import { normalizeD1SqlLineEndings } from '../../src/cli/d1-sql-line-endings.ts'; + +it('turns CRLF in a trigger body into LF and keeps it inside a string', () => { + expect( + normalizeD1SqlLineEndings( + "CREATE TRIGGER trg BEFORE DELETE ON probe_z\r\nBEGIN\r\n SELECT RAISE(ABORT, 'no\r\nchange');\r\nEND;", + ), + ).toBe( + "CREATE TRIGGER trg BEFORE DELETE ON probe_z\nBEGIN\n SELECT RAISE(ABORT, 'no\r\nchange');\nEND;", + ); +}); + +it('keeps CRLF inside quoted values and identifiers, and normalizes comments', () => { + const sql = + "SELECT 'single''quote\r\nvalue', \"double\r\nquote\", `backtick\r\nquote`, [bracket\r\nquote]; -- don't stop scanning\r\n/* block\r\ncomment */\r\nSELECT 1;"; + expect(normalizeD1SqlLineEndings(sql)).toBe( + "SELECT 'single''quote\r\nvalue', \"double\r\nquote\", `backtick\r\nquote`, [bracket\r\nquote]; -- don't stop scanning\n/* block\ncomment */\nSELECT 1;", + ); +}); + +it.each([ + // A bracket identifier ends at the first `]`; `]]` is not an escape in SQLite. + ['SELECT [a]]\r\nFROM t', 'SELECT [a]]\nFROM t'], + // A doubled quote stays inside the string. + ["SELECT 'it''s\r\n'\r\n", "SELECT 'it''s\r\n'\n"], + // A quote inside a comment does not start a string. + ["-- it's\r\nSELECT 1\r\n", "-- it's\nSELECT 1\n"], + ["/* it's */\r\nSELECT 1", "/* it's */\nSELECT 1"], + // Unterminated text is copied as is. + ["SELECT 'open\r\n", "SELECT 'open\r\n"], + ['/* open\r\n', '/* open\n'], + // A lone CR or LF is left alone. + ['SELECT 1\rSELECT 2\n', 'SELECT 1\rSELECT 2\n'], + ['SELECT 1 - -1\r\n', 'SELECT 1 - -1\n'], +])('normalizes %j', (sql, expected) => { + expect(normalizeD1SqlLineEndings(sql)).toBe(expected); +}); diff --git a/packages/void/test/unit/db-args.test.ts b/packages/void/test/unit/db-args.test.ts index 71413858c..e9f84c185 100644 --- a/packages/void/test/unit/db-args.test.ts +++ b/packages/void/test/unit/db-args.test.ts @@ -53,6 +53,16 @@ describe('parseDbArgs', () => { }); }); + it('parses execute with --yes', () => { + expect(parseDbArgs(['execute', '--remote', '--file', 'catalog.sql', '--yes'])).toEqual({ + action: 'execute', + command: undefined, + file: 'catalog.sql', + remote: true, + yes: true, + }); + }); + it('errors on execute with no SQL and no --file', () => { expect(() => parseDbArgs(['execute'])).toThrow( 'db: Usage: void db execute or void db execute --file .', diff --git a/packages/void/test/unit/db-execute-d1-large-file.test.ts b/packages/void/test/unit/db-execute-d1-large-file.test.ts new file mode 100644 index 000000000..99358033d --- /dev/null +++ b/packages/void/test/unit/db-execute-d1-large-file.test.ts @@ -0,0 +1,157 @@ +import { spawn } from 'node:child_process'; +import { createHash } from 'node:crypto'; +import { mkdirSync, writeFileSync } from 'node:fs'; +import { join } from 'node:path'; +import { createInterface } from 'node:readline'; +import { afterEach, expect, it, vi } from 'vite-plus/test'; +import { cleanups, temporaryDirectory } from '../helpers/cloudflare-operation-process.ts'; + +// Record what the CLI hands to each child process, without changing what runs. +const spawned: Array<{ argvBytes: number; largestEnvValueBytes: number }> = []; +vi.mock('node:child_process', async (importOriginal) => { + const actual = await importOriginal(); + return { + ...actual, + spawnSync: ((command: string, args: Array, options: { env?: NodeJS.ProcessEnv }) => { + spawned.push({ + argvBytes: [command, ...args].reduce((total, arg) => total + Buffer.byteLength(arg), 0), + largestEnvValueBytes: Math.max( + 0, + ...Object.values(options.env ?? {}).map((value) => Buffer.byteLength(value ?? '')), + ), + }); + return actual.spawnSync(command, args, options as never); + }) as typeof actual.spawnSync, + }; +}); + +const mockConfirm = vi.fn(); +const mockWarn = vi.fn(); +vi.mock('@voidzero-dev/vite-plus-prompts', async (importOriginal) => ({ + ...(await importOriginal()), + confirm: (...args: Array) => mockConfirm(...args), + log: { error: vi.fn(), info: vi.fn(), step: vi.fn(), success: vi.fn(), warn: mockWarn }, +})); + +// The CLI waits on spawnSync, which blocks this process, so the fake D1 API runs in its own. +const fakeD1Api = ` +import { createHash } from 'node:crypto'; +import { createServer } from 'node:http'; +const report = (entry) => process.stdout.write(JSON.stringify(entry) + '\\n'); +const server = createServer(async (request, response) => { + const chunks = []; + for await (const chunk of request) chunks.push(chunk); + const body = Buffer.concat(chunks); + const origin = 'http://127.0.0.1:' + server.address().port; + if (request.method === 'PUT') { + const etag = createHash('md5').update(body).digest('hex'); + report({ upload: body.length, etag, authorization: request.headers.authorization ?? null }); + response.setHeader('etag', '"' + etag + '"'); + response.end(); + return; + } + const input = JSON.parse(body.toString('utf8')); + report({ url: request.url, input, authorization: request.headers.authorization }); + const result = + input.action === 'init' + ? { filename: 'catalog.sql', upload_url: origin + '/upload' } + : input.action === 'ingest' + ? { success: true, status: 'active', at_bookmark: 'bookmark-1' } + : { success: true, status: 'complete', result: { num_queries: 1 } }; + response.setHeader('Content-Type', 'application/json'); + response.end(JSON.stringify({ success: true, result })); +}); +server.listen(0, '127.0.0.1', () => report({ port: server.address().port })); +`; + +async function startFakeD1Api(directory: string) { + const script = join(directory, 'fake-d1-api.mjs'); + writeFileSync(script, fakeD1Api); + const child = spawn(process.execPath, [script], { stdio: ['ignore', 'pipe', 'inherit'] }); + cleanups.push(() => child.kill()); + const entries: Array> = []; + const port = await new Promise((resolve, reject) => { + child.once('exit', (code) => reject(new Error(`fake D1 API exited with ${code}`))); + createInterface({ input: child.stdout! }).on('line', (line) => { + const entry = JSON.parse(line) as Record; + if (typeof entry.port === 'number') { + resolve(entry.port); + } else { + entries.push(entry); + } + }); + }); + // The last report can still be in the pipe when the CLI returns. + const settled = async (count: number) => { + for (let attempt = 0; entries.length < count && attempt < 100; attempt++) { + await new Promise((resolve) => setTimeout(resolve, 20)); + } + return entries; + }; + return { origin: `http://127.0.0.1:${port}`, settled }; +} + +let stdinIsTTY: PropertyDescriptor | undefined; +afterEach(() => { + vi.unstubAllEnvs(); + vi.restoreAllMocks(); + if (stdinIsTTY) { + Object.defineProperty(process.stdin, 'isTTY', stdinIsTTY); + } else { + delete (process.stdin as { isTTY?: boolean }).isTTY; + } +}); + +it('imports a 25 MB SQL file into direct Cloudflare D1 in CI without a prompt or a large argv', async () => { + const root = temporaryDirectory('void-d1-large-file-'); + writeFileSync(join(root, 'void.json'), '{}'); + mkdirSync(join(root, '.void')); + writeFileSync(join(root, '.void', 'project.json'), JSON.stringify({ platform: 'cloudflare' })); + writeFileSync( + join(root, 'wrangler.json'), + JSON.stringify({ + name: 'catalog-worker', + account_id: 'test-account', + d1_databases: [{ binding: 'DB', database_name: 'catalog', database_id: 'test-database' }], + }), + ); + const row = "INSERT INTO cards (id, name) VALUES (1, 'synthetic card name');\n"; + const sql = row.repeat(Math.ceil((25 * 1024 * 1024) / row.length)); + writeFileSync(join(root, 'catalog.sql'), sql); + const etag = createHash('md5').update(sql).digest('hex'); + + const api = await startFakeD1Api(root); + vi.stubEnv('CLOUDFLARE_API_TOKEN', 'synthetic-token'); + vi.stubEnv('CLOUDFLARE_API_BASE_URL', `${api.origin}/client/v4`); + vi.stubEnv('CI', 'true'); + stdinIsTTY = Object.getOwnPropertyDescriptor(process.stdin, 'isTTY'); + Object.defineProperty(process.stdin, 'isTTY', { value: undefined, configurable: true }); + vi.spyOn(console, 'log').mockImplementation(() => {}); + + const { runDbCommand } = await import('../../src/cli/db.ts'); + await runDbCommand(root, { action: 'execute', file: 'catalog.sql', remote: true }); + + expect(mockConfirm).not.toHaveBeenCalled(); + expect(mockWarn.mock.calls.flat().join(' ')).toContain('runs it as a D1 import'); + const importUrl = '/client/v4/accounts/test-account/d1/database/test-database/import'; + expect(await api.settled(4)).toEqual([ + { url: importUrl, input: { action: 'init', etag }, authorization: 'Bearer synthetic-token' }, + { upload: Buffer.byteLength(sql), etag, authorization: null }, + { + url: importUrl, + input: { action: 'ingest', etag, filename: 'catalog.sql' }, + authorization: 'Bearer synthetic-token', + }, + { + url: importUrl, + input: { action: 'poll', current_bookmark: 'bookmark-1' }, + authorization: 'Bearer synthetic-token', + }, + ]); + // Nothing proportional to the file reaches a child's argv or environment. + expect(spawned.length).toBeGreaterThan(0); + for (const child of spawned) { + expect(child.argvBytes).toBeLessThan(16 * 1024); + expect(child.largestEnvValueBytes).toBeLessThan(16 * 1024); + } +}, 60_000); diff --git a/packages/void/test/unit/db-execute-remote.test.ts b/packages/void/test/unit/db-execute-remote.test.ts index 669c3bb9a..f113723a4 100644 --- a/packages/void/test/unit/db-execute-remote.test.ts +++ b/packages/void/test/unit/db-execute-remote.test.ts @@ -165,21 +165,72 @@ describe('void db execute --remote (D1)', () => { }, ); - it('uses bulk import only for direct D1 file execution', async () => { + // D1's maximum SQL statement length decides between the query API and a D1 import. + const D1_STATEMENT_LIMIT = 100_000; + function directD1File(sql: string, tty: boolean): string { mockReadDeploymentPlatform.mockReturnValue('cloudflare'); mockImportD1.mockResolvedValue(undefined); + mockExecuteD1.mockResolvedValue([{ answer: 42 }]); + Object.defineProperty(process.stdin, 'isTTY', { value: tty, configurable: true }); const root = createProjectRoot(); - const sql = 'INSERT INTO samples VALUES (1);'; - writeFileSync(join(root, 'seed.sql'), sql); - Object.defineProperty(process.stdin, 'isTTY', { value: true, configurable: true }); - mockConfirm.mockResolvedValue(true); - await runDbCommand(root, { action: 'execute', file: 'seed.sql', remote: true }); + writeFileSync(join(root, 'input.sql'), sql); + return root; + } + // A read-only SELECT padded with a leading comment to exactly `bytes` UTF-8 bytes. + function selectOfSize(bytes: number): string { + const fixed = '-- \nSELECT 42 AS answer'.length; + const padding = 'é'.repeat(Math.floor((bytes - fixed) / 2)) + 'x'.repeat((bytes - fixed) % 2); + return `-- ${padding}\nSELECT 42 AS answer`; + } + const printed = () => vi.mocked(console.log).mock.calls.flat().join(' '); + + it.each([31, D1_STATEMENT_LIMIT])( + 'runs a direct D1 --file of %i bytes as a query and prints rows', + async (bytes) => { + const sql = selectOfSize(bytes); + expect(Buffer.byteLength(sql)).toBe(bytes); + const root = directD1File(sql, true); + await runDbCommand(root, { action: 'execute', file: 'input.sql', remote: true }); + expect(mockExecuteD1).toHaveBeenCalledWith(mockD1Context, sql); + expect(mockImportD1).not.toHaveBeenCalled(); + expect(mockConfirm).not.toHaveBeenCalled(); + expect(printed()).toContain('42'); + expect(fetchMock).not.toHaveBeenCalled(); + }, + ); + + it('imports a direct D1 --file over the statement limit after a warning and a TTY prompt', async () => { + const { log } = await import('@voidzero-dev/vite-plus-prompts'); + const sql = selectOfSize(D1_STATEMENT_LIMIT + 1); + const root = directD1File(sql, true); + await runDbCommand(root, { action: 'execute', file: 'input.sql', remote: true }); + expect(vi.mocked(log.warn).mock.calls.flat().join(' ')).toContain('cannot serve queries'); + expect(mockConfirm).toHaveBeenCalledTimes(1); expect(mockImportD1).toHaveBeenCalledWith(mockD1Context, sql); expect(mockExecuteD1).not.toHaveBeenCalled(); - expect(mockConfirm).toHaveBeenCalledTimes(1); expect(fetchMock).not.toHaveBeenCalled(); }); + it('cancels a large direct D1 --file import when the prompt is declined', async () => { + const root = directD1File(selectOfSize(D1_STATEMENT_LIMIT + 1), true); + mockConfirm.mockResolvedValue(false); + await expect( + runDbCommand(root, { action: 'execute', file: 'input.sql', remote: true }), + ).rejects.toThrow('process.exit:0'); + expect(mockImportD1).not.toHaveBeenCalled(); + expect(mockExecuteD1).not.toHaveBeenCalled(); + }); + + it('warns and imports a large direct D1 --file without a prompt outside a TTY', async () => { + const { log } = await import('@voidzero-dev/vite-plus-prompts'); + const sql = 'INSERT INTO samples VALUES (1);\n'.repeat(4000); + const root = directD1File(sql, false); + await runDbCommand(root, { action: 'execute', file: 'input.sql', remote: true }); + expect(vi.mocked(log.warn).mock.calls.flat().join(' ')).toContain('cannot serve queries'); + expect(mockConfirm).not.toHaveBeenCalled(); + expect(mockImportD1).toHaveBeenCalledWith(mockD1Context, sql); + }); + it('POSTs to /d1/query with correct URL, body, and auth headers', async () => { const root = createProjectRoot(); fetchMock.mockResolvedValueOnce(jsonResponse({ results: [], success: true })); @@ -500,6 +551,30 @@ describe('void db execute --remote (D1)', () => { expect(mockConfirm).not.toHaveBeenCalled(); }); + it('skips the destructive prompt in a TTY with --yes', async () => { + const root = createProjectRoot(); + Object.defineProperty(process.stdin, 'isTTY', { value: true, configurable: true }); + fetchMock.mockResolvedValueOnce(jsonResponse({ results: [], success: true })); + + await runDbCommand(root, { + action: 'execute', + command: 'DELETE FROM users', + remote: true, + yes: true, + }); + + expect(mockConfirm).not.toHaveBeenCalled(); + expect(fetchMock).toHaveBeenCalledTimes(1); + }); + + it('imports a large direct D1 --file in a TTY without a prompt with --yes', async () => { + const sql = selectOfSize(D1_STATEMENT_LIMIT + 1); + const root = directD1File(sql, true); + await runDbCommand(root, { action: 'execute', file: 'input.sql', remote: true, yes: true }); + expect(mockConfirm).not.toHaveBeenCalled(); + expect(mockImportD1).toHaveBeenCalledWith(mockD1Context, sql); + }); + it('runs execute without --remote against the local database, not the proxy', async () => { const root = createProjectRoot(); // No local database exists, so the local path exits. diff --git a/packages/void/test/unit/deploy-cloudflare-email.test.ts b/packages/void/test/unit/deploy-cloudflare-email.test.ts new file mode 100644 index 000000000..90529400b --- /dev/null +++ b/packages/void/test/unit/deploy-cloudflare-email.test.ts @@ -0,0 +1,220 @@ +import { mkdtempSync, rmSync, writeFileSync } from 'node:fs'; +import { tmpdir } from 'node:os'; +import { join } from 'node:path'; +import type { CloudflareOperation } from '@void/deploy-cloudflare'; +import { log } from '@voidzero-dev/vite-plus-prompts'; +import { afterEach, describe, expect, it, vi } from 'vite-plus/test'; +import { runCloudflareEmailStep } from '../../src/cli/deploy-cloudflare-email.ts'; +import type { DeployWranglerRunResult } from '../../src/cli/deploy-types.ts'; +import { renderAddressMap, type EmailDns } from '../../src/cli/email-cloudflare.ts'; + +/** + * A sending-only app (no `email/` handlers) deployed from CI with an API token, shaped like + * the pocket-folio report: `email.from` on `mail.pkmn.nakazawa.tech`, a subdomain of the + * `nakazawa.tech` zone, Email Sending onboarded, Email Routing never enabled, and the + * `SEND_EMAIL` binding plus `__VOID_EMAIL_FROM` already committed. + */ +const ACCOUNT = '004b3cab004b3cab004b3cab004b3cab'; +const ZONE_ID = '5b66f5cb84fd7dcfdd0170515e622181'; +const BEARER = 'cfat-SECRET-API-TOKEN-0123456789abcdef'; +const DOMAIN = 'mail.pkmn.nakazawa.tech'; +const FROM = `Pocket Folio `; + +const clean = { status: 0, stdout: '', stderr: '' } as const; + +type FetchCall = { method: string; path: string; auth: string }; + +function cpojerAccount() { + const operations: Array = []; + const fetchCalls: Array = []; + const runWrangler = (operation: CloudflareOperation): DeployWranglerRunResult => { + operations.push(operation.type); + switch (operation.type) { + case 'auth.whoami': + return { + ...clean, + data: { + loggedIn: true, + authType: 'API Token', + accounts: [{ id: ACCOUNT, name: 'Nakazawa Tech KK' }], + }, + }; + case 'auth.token': + return { ...clean, data: { type: 'api_token', token: BEARER } }; + case 'email.routing.settings': + case 'email.sending.settings': { + // The documented token (Zone Read + Email Sending Edit) has no Zone Settings Read, + // so the provider's Email Routing settings read, which also locates the zone for its + // sending read, is refused. + const message = 'Cloudflare could not resolve the Email zone.'; + return { status: 1, stdout: '', stderr: message, errors: [{ message }] }; + } + default: + throw new Error(`unexpected Cloudflare operation: ${operation.type}`); + } + }; + const zones: Record> = { + [DOMAIN]: [], + 'pkmn.nakazawa.tech': [], + 'nakazawa.tech': [ + { id: ZONE_ID, name: 'nakazawa.tech', type: 'full', account: { id: ACCOUNT } }, + ], + }; + const fetchImpl = (async (input: RequestInfo | URL, init?: RequestInit) => { + const url = typeof input === 'string' ? input : input instanceof URL ? input.href : input.url; + const path = url.slice(url.indexOf('/zones')); + const method = init?.method ?? 'GET'; + const headers = (init?.headers ?? {}) as Record; + fetchCalls.push({ method, path, auth: headers.Authorization ?? '' }); + const prefix = '/zones?name='; + if (method === 'GET' && path.startsWith(prefix)) { + const name = decodeURIComponent(path.slice(prefix.length)); + if (name in zones) { + return new Response(JSON.stringify({ success: true, result: zones[name] })); + } + } + if (method === 'GET' && path === `/zones/${ZONE_ID}/email/sending/subdomains`) { + return new Response( + JSON.stringify({ + success: true, + errors: [], + messages: [], + result: [ + { + tag: '19af4fdbee924a92916afea0de5a8fdd', + name: DOMAIN, + enabled: true, + created: '2026-11-01T00:00:00Z', + modified: '2026-11-01T00:00:00Z', + dkim_selector: 'cf-bounce', + return_path_domain: `cf-bounce.${DOMAIN}`, + }, + ], + result_info: { count: 1, page: 1, per_page: 20, total_count: 1, total_pages: 1 }, + }), + ); + } + throw new Error(`unexpected fetch: ${method} ${path}`); + }) as typeof fetch; + // Onboarding wrote its own `_dmarc` and `cf-bounce` records; inbound MX is never read. + const dns: EmailDns = { + resolveMx: async (name) => { + throw new Error(`a sending-only app must not read inbound MX (${name})`); + }, + resolveTxt: async (name) => { + if (name === `_dmarc.${DOMAIN}`) { + return [['v=DMARC1; p=reject']]; + } + if (name === `cf-bounce.${DOMAIN}`) { + return [['v=spf1 include:_spf.mx.cloudflare.net ~all']]; + } + throw Object.assign(new Error('ENODATA'), { code: 'ENODATA' }); + }, + }; + return { operations, fetchCalls, runWrangler, fetchImpl, dns }; +} + +/** Everything the step prints: every clack level plus raw stdout/stderr. */ +function captureOutput(): { lines: Array; restore: () => void } { + const lines: Array = []; + const spies = (['message', 'info', 'warn', 'error', 'success', 'step'] as const).map((level) => + vi.spyOn(log, level).mockImplementation(((message: unknown) => { + lines.push(`${level}: ${String(message)}`); + }) as never), + ); + const out = vi.spyOn(process.stdout, 'write').mockImplementation(((chunk: unknown) => { + lines.push(`stdout: ${String(chunk)}`); + return true; + }) as never); + const err = vi.spyOn(process.stderr, 'write').mockImplementation(((chunk: unknown) => { + lines.push(`stderr: ${String(chunk)}`); + return true; + }) as never); + return { + lines, + restore: () => { + for (const spy of spies) { + spy.mockRestore(); + } + out.mockRestore(); + err.mockRestore(); + }, + }; +} + +const roots: Array = []; +afterEach(() => { + for (const root of roots.splice(0)) { + rmSync(root, { recursive: true, force: true }); + } +}); + +function project(rootConfig: Record): { root: string; configPath: string } { + const root = mkdtempSync(join(tmpdir(), 'void-email-step-')); + roots.push(root); + const configPath = join(root, '.void-wrangler.jsonc'); + writeFileSync(configPath, JSON.stringify(rootConfig, null, 2)); + return { root, configPath }; +} + +describe('runCloudflareEmailStep — sending-only app in CI', () => { + it('prints nothing for an onboarded sending-only app, then one neutral outbound line', async () => { + const rootConfig = { + name: 'pkmn-binder', + send_email: [{ name: 'SEND_EMAIL' }], + vars: { __VOID_EMAIL_FROM: FROM }, + }; + const { root, configPath } = project(rootConfig); + const account = cpojerAccount(); + const output = captureOutput(); + let result: Awaited>; + try { + result = await runCloudflareEmailStep({ + root, + configPath, + voidConfig: { email: { from: FROM } }, + rootConfig, + handlers: { addresses: [] }, + workerName: 'pkmn-binder', + accountId: ACCOUNT, + apiBaseUrl: 'https://api.cloudflare.com/client/v4', + runWrangler: account.runWrangler, + nonInteractive: true, + confirm: async () => { + throw new Error('CI must never be asked'); + }, + fetchImpl: account.fetchImpl, + dns: account.dns, + }); + } finally { + output.restore(); + } + + expect(output.lines).toEqual([]); + expect(result).toEqual({ + satisfied: true, + configWritten: false, + map: { + from: FROM, + routes: [], + addresses: [], + sendingReady: true, + sendingOnly: true, + creating: [], + }, + }); + // The post-deploy block is the only email output of the whole deploy. + expect(renderAddressMap(result.map!)).toBe(` outbound sendEmail() from ${FROM}`); + // Nothing touches Email Routing: with a token scoped Zone Read + Email Sending Edit, + // sending is read on the zone the bearer resolved, so --require-email passes. The + // bearer stays in the Authorization header. + expect(account.operations).toEqual(['auth.whoami', 'auth.token']); + expect(account.fetchCalls.map(({ method, path }) => `${method} ${path}`)).toEqual([ + `GET /zones?name=${DOMAIN}`, + 'GET /zones?name=pkmn.nakazawa.tech', + 'GET /zones?name=nakazawa.tech', + `GET /zones/${ZONE_ID}/email/sending/subdomains`, + ]); + expect(account.fetchCalls.every(({ auth }) => auth === `Bearer ${BEARER}`)).toBe(true); + }); +}); diff --git a/packages/void/test/unit/direct-r2-deploy.test.ts b/packages/void/test/unit/direct-r2-deploy.test.ts index 0aa73a166..bbb0beb26 100644 --- a/packages/void/test/unit/direct-r2-deploy.test.ts +++ b/packages/void/test/unit/direct-r2-deploy.test.ts @@ -125,6 +125,7 @@ function installPlatformMocks(opts?: { r2PutBehavior?: (url: string) => Response | Promise; workerUploadUrls?: () => Response; finalizeEvents?: Array; + capabilities?: Record; }): MockState { const state: MockState = { preflightCalls: [], @@ -139,6 +140,10 @@ function installPlatformMocks(opts?: { const reqInit = init ?? {}; const method = reqInit.method ?? 'GET'; + if (url.endsWith('/capabilities') && method === 'GET' && opts?.capabilities) { + return Response.json(opts.capabilities); + } + if (url.endsWith('/deploy/preflight') && method === 'POST') { state.preflightCalls.push(JSON.parse(reqInit.body as string)); return new Response( @@ -692,6 +697,79 @@ describe('Worker module uploads', () => { }); }); +describe('D1 read replication', () => { + const manifest: DeployManifest = { + version: 3, + assetManifestVersion: 2, + bindings: { d1: ['DB'] }, + type: 'void', + ssr: true, + workerMain: 'index.js', + }; + + async function deployWith(origin: string, deployManifest: DeployManifest): Promise { + const { assetManifest, assetFiles } = await collectAndHashAssets(assetsDir); + await streamDirectR2Deploy({ + client: new PlatformClient('test-token', { apiUrl: origin }), + projectId: 'proj_t', + manifest: deployManifest, + assetManifest, + assetFiles, + needed: ['/index.html'], + preflightUsed: false, + spinner: noopSpinner as unknown as SpinnerResult, + }); + } + + function requestedPaths(): Array { + return fetchMock.mock.calls.map(([url]) => new URL(String(url)).pathname); + } + + it.each([true, false])( + 'stops before any upload when the platform cannot apply readReplication: %s', + async (readReplication) => { + // What a 1.10.0 platform advertises: it ignores the setting. + const state = installPlatformMocks({ capabilities: { skipMigrations: true } }); + + await expect( + deployWith(`https://api-rr-old-${readReplication}.test`, { + ...manifest, + database: { readReplication }, + }), + ).rejects.toThrow( + 'This Void platform cannot apply `database.readReplication` from void.config.ts. Ask its administrator to upgrade with `void platform upgrade`.', + ); + + expect(requestedPaths()).toEqual(['/capabilities']); + expect(state.uploadUrlsCalls).toHaveLength(0); + expect(state.r2PutCalls).toHaveLength(0); + expect(state.finalizeCalls).toHaveLength(0); + }, + ); + + it('sends the setting when the platform advertises support', async () => { + const state = installPlatformMocks({ capabilities: { d1ReadReplication: true } }); + + await deployWith('https://api-rr-new.test', { + ...manifest, + database: { readReplication: false }, + }); + + expect(requestedPaths()[0]).toBe('/capabilities'); + const body = JSON.parse(state.finalizeCalls[0].init.body as string); + expect(body.manifest.database).toEqual({ readReplication: false }); + }); + + it('does not ask for capabilities when the app leaves the setting out', async () => { + const state = installPlatformMocks(); + + await deployWith('https://api-rr-unset.test', manifest); + + expect(requestedPaths()).not.toContain('/capabilities'); + expect(state.finalizeCalls).toHaveLength(1); + }); +}); + function escapeRegex(s: string): string { return s.replace(/[.*+?^${}()|[\]\\]/g, '\\$&'); } diff --git a/packages/void/test/unit/email-cli-cloudflare.test.ts b/packages/void/test/unit/email-cli-cloudflare.test.ts index c4892fb2e..cbef75238 100644 --- a/packages/void/test/unit/email-cli-cloudflare.test.ts +++ b/packages/void/test/unit/email-cli-cloudflare.test.ts @@ -342,6 +342,19 @@ const sendingOnboarded: EmailWranglerRunResult = { data: { name: 'mail.acme.com', enabled: true, tag: ZONE_ID }, stderr: BETA_WARNING, }; +/** Sending-only apps read and onboard Email Sending through the bearer on the resolved zone. */ +const SENDING_SUBDOMAINS = `/zones/${ZONE_ID}/email/sending/subdomains`; +const subdomainsListed = (names: Array) => () => + jsonResponse(200, { + success: true, + result: names.map((name) => ({ name, enabled: true, tag: '19af4fdbee924a92916afea0de5a8fdd' })), + }); +const sendingRefusedByApi = () => + jsonResponse(403, { + success: false, + errors: [{ code: 10105, message: 'email.sending.error.authentication.not_entitled' }], + result: null, + }); describe('sending-only email setup and status', () => { it.each(['setup', 'status'] as const)( @@ -358,9 +371,9 @@ describe('sending-only email setup and status', () => { roots.push(root); rmSync(join(root, 'email'), { recursive: true }); const { deps, calls, fetchCalls, printed } = transcriptDeps({ - runnerTable: { 'email sending settings mail.acme.com': sendingOnboarded }, fetchTable: { 'GET /zones?name=mail.acme.com': () => jsonResponse(200, { success: true, result: [] }), + [`GET ${SENDING_SUBDOMAINS}`]: subdomainsListed(['mail.acme.com']), }, }); const code = await run(root, command, { @@ -373,17 +386,105 @@ describe('sending-only email setup and status', () => { expect(code).toBe(0); expect(printed.join('\n')).toContain('inbound not required'); expect(printed.join('\n')).not.toContain('enable Email Routing'); - expect(calls.some(({ operation }) => operation.type.startsWith('email.routing.'))).toBe( - false, - ); + expect(calls.some(({ operation }) => operation.type.startsWith('email.'))).toBe(false); expect( - fetchCalls.every(({ method, url }) => method === 'GET' && url.includes('/zones?name=')), + fetchCalls.every(({ method, url }) => method === 'GET' && !url.includes('/email/routing')), ).toBe(true); const config = readJsonc(join(root, 'wrangler.jsonc')); expect(config.addresses).toBeUndefined(); expect(config.send_email).toEqual([{ name: 'SEND_EMAIL' }]); }, ); + + /** A sending-only app set up earlier (binding saved) whose domain is not onboarded. */ + function savedSendingOnly(extra: { fetchTable?: Record Response> } = {}) { + const root = makeProject({ + wrangler: JSON.stringify({ + account_id: ACCOUNT, + name: 'acme-support', + send_email: [{ name: 'SEND_EMAIL' }], + vars: { __VOID_EMAIL_FROM: 'noreply@mail.acme.com' }, + }), + }); + roots.push(root); + rmSync(join(root, 'email'), { recursive: true }); + const fakes = transcriptDeps({ + fetchTable: { + 'GET /zones?name=mail.acme.com': () => jsonResponse(200, { success: true, result: [] }), + [`GET ${SENDING_SUBDOMAINS}`]: subdomainsListed([]), + ...extra.fetchTable, + }, + }); + return { root, configPath: join(root, 'wrangler.jsonc'), ...fakes }; + } + + it('status names Workers Paid, its link and the token permissions, and exits 1', async () => { + const { root, deps, calls, fetchCalls, printed } = savedSendingOnly(); + const logged = captureLog(); + let code: number; + try { + code = await run(root, 'status', { ...deps, isInteractive: false }); + } finally { + logged.restore(); + } + expect(code).toBe(1); + expect(printed[0]).toContain( + 'sending mail.acme.com not onboarded — needs Workers Paid; after upgrading run void email setup --platform cloudflare', + ); + expect(printed.at(-1)).toBe( + ' outbound sendEmail() from noreply@mail.acme.com (Email Sending not onboarded)', + ); + const warned = logged.lines.join('\n'); + expect(warned).toContain( + 'Enable Workers Paid: https://dash.cloudflare.com/?to=/:account/workers/plans', + ); + expect(warned).toContain('Zone Read + Email Sending Edit'); + expect(warned).not.toContain('Email is not set up'); + expect(calls.some(({ operation }) => operation.type.startsWith('email.'))).toBe(false); + expect(fetchCalls.some(({ method }) => method === 'POST')).toBe(false); + }); + + it('setup asks to onboard Email Sending; a refusal prints the Workers Paid steps and exits 1', async () => { + const { root, configPath, deps, calls, fetchCalls, printed } = savedSendingOnly({ + fetchTable: { [`POST ${SENDING_SUBDOMAINS}`]: sendingRefusedByApi }, + }); + const before = readFileSync(configPath, 'utf-8'); + const confirms: Array = []; + const logged = captureLog(); + let code: number; + try { + code = await run(root, 'setup', { + ...deps, + isInteractive: true, + confirm: async (message) => { + confirms.push(message); + return true; + }, + }); + } finally { + logged.restore(); + } + expect(code).toBe(1); + expect(confirms).toEqual([ + 'Onboard mail.acme.com for Email Sending? Sending-only apps need Workers Paid.', + ]); + expect(calls.some(({ operation }) => operation.type.startsWith('email.'))).toBe(false); + expect( + fetchCalls + .filter(({ method }) => method === 'POST') + .map(({ url, body }) => [url.slice(url.indexOf('/zones')), body]), + ).toEqual([[SENDING_SUBDOMAINS, JSON.stringify({ name: 'mail.acme.com' })]]); + const output = printed.join('\n'); + expect(output).toContain('[code: 10105]'); + expect(output).toContain( + 'Enable Workers Paid: https://dash.cloudflare.com/?to=/:account/workers/plans', + ); + expect(output).toContain('Zone Read + Email Sending Edit'); + expect(output).not.toContain('verified destinations only'); + expect(logged.lines.join('\n')).toContain('Follow the Workers Paid steps above.'); + // The saved binding and sender stay as they were. + expect(readFileSync(configPath, 'utf-8')).toBe(before); + }); }); describe('void email setup --platform cloudflare', () => { diff --git a/packages/void/test/unit/email-cloudflare.test.ts b/packages/void/test/unit/email-cloudflare.test.ts index b5ee35365..58a9dca0b 100644 --- a/packages/void/test/unit/email-cloudflare.test.ts +++ b/packages/void/test/unit/email-cloudflare.test.ts @@ -6,6 +6,8 @@ import { emailFromDomain, fetchWithSessionBearer, findEmailConfigConflicts, + formatSendingOnlyPaidHint, + isAwaitingSendingOnboarding, readRoutingRules, readRoutingSettings, readSendingSettings, @@ -1302,32 +1304,39 @@ describe('preflightEmail', () => { }); describe('sending-only Cloudflare email', () => { - function sendingOnly(domain = 'mail.acme.com') { - return transcriptInputs({ + const SUBDOMAINS = `/zones/${ZONE_ID}/email/sending/subdomains`; + const listed = (domain: string) => () => + jsonResponse(200, { + success: true, + result: [{ name: domain, enabled: true, tag: '19af4fdbee924a92916afea0de5a8fdd' }], + }); + const notListed = () => jsonResponse(200, { success: true, result: [] }); + const refused = (code: number) => () => + jsonResponse(403, { success: false, errors: [{ code, message: 'Forbidden' }], result: null }); + + /** + * Email Sending is read and onboarded through the bearer on the zone preflight resolved. + * The runner knows no Email operation, so any provider Email call fails the test. + */ + function sendingOnly( + domain = 'mail.acme.com', + account: { list?: () => Response; enable?: () => Response } = {}, + ) { + const fixture = transcriptInputs({ handlers: scan([]), from: `Pocket Folio `, existingSendEmail: [{ name: 'SEND_EMAIL' }], existingVars: { __VOID_EMAIL_FROM: `Pocket Folio ` }, - runnerTable: { - 'whoami --json': { - status: 0, - stdout: '', - stderr: '', - data: { - loggedIn: true, - authType: 'OAuth Token', - tokenPermissions: ['email_sending:write'], - }, - }, - [`email sending settings ${domain}`]: { - status: 0, - stdout: '', - stderr: '', - data: { name: domain, enabled: true, tag: '19af4fdbee924a92916afea0de5a8fdd' }, - }, - }, fetchTable: { 'GET /zones?name=mail.acme.com': () => jsonResponse(200, { success: true, result: [] }), + [`GET ${SUBDOMAINS}`]: account.list ?? listed(domain), + [`POST ${SUBDOMAINS}`]: + account.enable ?? + (() => + jsonResponse(200, { + success: true, + result: { name: domain, enabled: true, tag: '19af4fdbee924a92916afea0de5a8fdd' }, + })), }, dns: { resolveMx: async () => { @@ -1336,7 +1345,17 @@ describe('sending-only Cloudflare email', () => { resolveTxt: async () => [], }, }); + const whoami: EmailWranglerRunResult = { + status: 0, + stdout: '', + stderr: '', + data: { loggedIn: true, authType: 'OAuth Token', tokenPermissions: ['email_sending:write'] }, + }; + fixture.inputs.runWrangler = fakeRunner({ 'whoami --json': whoami }, fixture.calls); + return fixture; } + const paths = (fetchCalls: Array) => + fetchCalls.map(({ method, url }) => `${method} ${url.slice(url.indexOf('/zones'))}`); it.each(['acme.com', 'mail.acme.com'])( 'accepts onboarded sending on %s without routing scopes or probes', @@ -1347,11 +1366,12 @@ describe('sending-only Cloudflare email', () => { expect(pre.blockers).toEqual([]); expect(pre.session.missingScopes).toEqual([]); expect(pre.sender.state).toBe('ready'); - expect(calls.map(({ operation }) => operation.type)).toEqual([ - 'auth.whoami', - 'email.sending.settings', + expect(calls.map(({ operation }) => operation.type)).toEqual(['auth.whoami']); + expect(paths(fetchCalls)).toEqual([ + ...(domain === 'acme.com' ? [] : ['GET /zones?name=mail.acme.com']), + 'GET /zones?name=acme.com', + `GET ${SUBDOMAINS}`, ]); - expect(fetchCalls.every(({ url }) => url.includes('/zones?name='))).toBe(true); const printed = renderEmailChecklist(pre); expect(printed).toContain('sending only'); expect(printed).toContain('inbound not required'); @@ -1360,6 +1380,13 @@ describe('sending-only Cloudflare email', () => { }, ); + it('matches the listed subdomain by name and treats an absent one as not onboarded', async () => { + const other = sendingOnly('mail.acme.com', { list: listed('news.acme.com') }); + const pre = await preflightEmail(other.inputs); + expect(pre.sending.state).toBe('pending'); + expect(pre.ready).toBe(false); + }); + it('writes the binding and sender without touching inbound configuration', async () => { const { inputs, calls, fetchCalls } = sendingOnly(); inputs.existingSendEmail = undefined; @@ -1380,43 +1407,48 @@ describe('sending-only Cloudflare email', () => { expect(fetchCalls).toEqual([]); }); - it('onboards sending only, and never enables routing or subaddressing', async () => { - const { inputs, calls, fetchCalls } = sendingOnly(); - const original = inputs.runWrangler; - inputs.runWrangler = (operation, options) => { - if (operation.type === 'email.sending.settings') { - return sendingNotFound('mail.acme.com'); - } - if (operation.type === 'email.sending.enable') { - calls.push({ operation, args: operationWords(operation), capture: options.capture }); - return { status: 0, stdout: '', stderr: '' }; - } - return original(operation, options); - }; + it('onboards through the bearer on the resolved zone, and never enables routing or subaddressing', async () => { + const { inputs, calls, fetchCalls } = sendingOnly('mail.acme.com', { list: notListed }); const pre = await preflightEmail(inputs); expect(pre.ready).toBe(false); calls.length = 0; fetchCalls.length = 0; const out = await applyEmailSetup(pre, inputs); expect(out.ready).toBe(true); - expect(calls.map(({ operation }) => operation.type)).toEqual(['email.sending.enable']); + expect(out.lines).toContain(' ✔ sending mail.acme.com onboarded'); + expect(calls).toEqual([]); + expect(paths(fetchCalls)).toEqual([`POST ${SUBDOMAINS}`]); + expect(fetchCalls[0]!.body).toBe(JSON.stringify({ name: 'mail.acme.com' })); + expect(fetchCalls[0]!.headers.Authorization).toBe(`Bearer ${BEARER}`); + }); + + it('names a missing session bearer instead of the Workers Paid plan when onboarding cannot start', async () => { + const { inputs, fetchCalls } = sendingOnly('mail.acme.com', { list: notListed }); + const pre = await preflightEmail(inputs); + fetchCalls.length = 0; + inputs.getSessionBearer = async () => { + throw new Error('session expired'); + }; + const out = await applyEmailSetup(pre, inputs); + expect(out.ready).toBe(false); + expect(out.lines).toContain( + ' ! sending mail.acme.com not onboarded (session bearer unavailable); rerun void email setup --platform cloudflare', + ); + expect(out.lines.some((line) => line.includes('Workers Paid'))).toBe(false); expect(fetchCalls).toEqual([]); }); - it.each([sendingNotFound('mail.acme.com'), apiFailure(10000, 'sending settings')])( + it('treats code 2040 (already onboarded) as onboarded', async () => { + const { inputs } = sendingOnly('mail.acme.com', { list: notListed, enable: refused(2040) }); + const out = await applyEmailSetup(await preflightEmail(inputs), inputs); + expect(out.sendingReady).toBe(true); + expect(out.ready).toBe(true); + }); + + it.each([notListed, refused(10000)])( 'does not mistake a binding for verified sending', - async (sending) => { - const { inputs } = sendingOnly(); - const original = inputs.runWrangler; - inputs.runWrangler = (operation, options) => { - if (operation.type === 'email.sending.settings') { - return sending; - } - if (operation.type === 'email.sending.enable') { - return apiFailure(10105, 'sending'); - } - return original(operation, options); - }; + async (list) => { + const { inputs } = sendingOnly('mail.acme.com', { list, enable: refused(10105) }); const pre = await preflightEmail(inputs); expect(pre.ready).toBe(false); const out = await applyEmailSetup(pre, inputs); @@ -1426,6 +1458,38 @@ describe('sending-only Cloudflare email', () => { }, ); + it('names Workers Paid, its link and the token permissions when onboarding is refused', async () => { + const { inputs } = sendingOnly('mail.acme.com', { list: notListed, enable: refused(10105) }); + const pre = await preflightEmail(inputs); + expect(isAwaitingSendingOnboarding(pre)).toBe(true); + const checklist = renderEmailChecklist(pre); + expect(checklist).toContain( + 'sending mail.acme.com not onboarded — needs Workers Paid; after upgrading run void email setup --platform cloudflare', + ); + expect(checklist).not.toContain('verified destinations only'); + const out = await applyEmailSetup(pre, inputs); + expect(out.ready).toBe(false); + expect(out.patch).toEqual({ addresses: null }); + const printed = out.lines.join('\n'); + expect(printed).toContain('[code: 10105]'); + for (const line of formatSendingOnlyPaidHint('mail.acme.com')) { + expect(out.lines).toContain(` ${line}`); + } + expect(printed).toContain('https://dash.cloudflare.com/?to=/:account/workers/plans'); + expect(printed).toContain('Zone Read + Email Sending Edit'); + expect(printed).not.toContain('verified destinations only'); + }); + + it('does not treat an unreadable sending row as waiting for onboarding', async () => { + const { inputs } = sendingOnly('mail.acme.com', { list: refused(10000) }); + const pre = await preflightEmail(inputs); + expect(isAwaitingSendingOnboarding(pre)).toBe(false); + // Without a saved binding the app was never set up: deploy offers setup as usual. + const fresh = sendingOnly('mail.acme.com', { list: notListed }); + fresh.inputs.existingSendEmail = undefined; + expect(isAwaitingSendingOnboarding(await preflightEmail(fresh.inputs))).toBe(false); + }); + it('reports an unreadable zone as unknown without proposing routing setup', async () => { const { inputs } = sendingOnly(); inputs.fetchImpl = (async () => jsonResponse(403, { success: false })) as typeof fetch; @@ -2104,6 +2168,18 @@ describe('renderAddressMap', () => { ); }); + it('does not promise verified destinations to a sending-only app that is not onboarded', () => { + expect( + renderAddressMap({ + from: 'noreply@acme.com', + routes: [], + addresses: [], + sendingReady: false, + sendingOnly: true, + }), + ).toBe(' outbound sendEmail() from noreply@acme.com (Email Sending not onboarded)'); + }); + it('marks a written address whose rule this deploy creates', () => { const text = renderAddressMap({ from: 'noreply@acme.com', diff --git a/packages/void/test/unit/init-report.test.ts b/packages/void/test/unit/init-report.test.ts index 2d4a68e69..8357908ca 100644 --- a/packages/void/test/unit/init-report.test.ts +++ b/packages/void/test/unit/init-report.test.ts @@ -90,7 +90,7 @@ describe('InitReport', () => { it('describes a new starter in one line instead of its files', () => { const report = new InitReport(); report.created('pages/index.tsx'); - report.starter = 'Created a React Pages starter with D1 (SQLite)'; + report.setStarter('Created a React Pages starter with D1 (SQLite)'); report.print(); @@ -99,6 +99,29 @@ describe('InitReport', () => { ]); }); + it('lists files outside the new app and later changes to existing files under the starter', () => { + const report = new InitReport(); + report.created('pages/index.tsx'); + report.updated('package.json', 'added starter scripts and dependencies'); + report.updated('../../pnpm-workspace.yaml', 'approved builds'); + report.setStarter('Created a React Pages starter with D1 (SQLite)'); + report.created('AGENTS.md'); + report.updated('package.json', 'removed typescript'); + report.updated('tsconfig.json', 'extends .void/tsconfig.json'); + + report.print(); + + expect(vi.mocked(printDone).mock.calls).toEqual([ + [ + [ + 'Created a React Pages starter with D1 (SQLite)', + '~ ../../pnpm-workspace.yaml approved builds', + '~ tsconfig.json extends .void/tsconfig.json', + ], + ], + ]); + }); + it('says when init changed no files', () => { new InitReport().print(); @@ -145,7 +168,7 @@ describe('an init run that stops early', () => { let finish = (): void => {}; const run = report.printIfStopped(async () => { report.created('pages/index.tsx'); - report.starter = 'Created a React Pages starter with D1 (SQLite)'; + report.setStarter('Created a React Pages starter with D1 (SQLite)'); await new Promise((resolve) => { finish = resolve; }); @@ -166,6 +189,25 @@ describe('an init run that stops early', () => { expect(process.listeners('exit')).toEqual(listeners); }); + it('lists files outside the new app under the starter when a step fails', async () => { + const report = new InitReport(); + const failure = new Error('pnpm install failed with exit code 1'); + + await expect( + report.printIfStopped(async () => { + report.created('pages/index.tsx'); + report.updated('../../pnpm-workspace.yaml', 'approved builds'); + report.setStarter('Created a static React Pages starter'); + throw failure; + }), + ).rejects.toBe(failure); + + expect(vi.mocked(log.warn).mock.calls).toEqual([['Setup did not finish. Done so far:']]); + expect(vi.mocked(printDone).mock.calls).toEqual([ + [['Created a static React Pages starter', '~ ../../pnpm-workspace.yaml approved builds']], + ]); + }); + it('prints nothing when it stops before it changes anything', async () => { await expect( new InitReport().printIfStopped(async () => { diff --git a/packages/void/test/unit/init.test.ts b/packages/void/test/unit/init.test.ts index 3f3a8fb54..5cdb8a6a1 100644 --- a/packages/void/test/unit/init.test.ts +++ b/packages/void/test/unit/init.test.ts @@ -4332,6 +4332,11 @@ export default config; expect(body).toMatch(/Scaffolded the Pages starter/); expect(body).toMatch(/void db generate/); expect(body).toContain(toSlash(wsroot)); + // The starter's own files stay hidden, but not the edit to the workspace root. + expect(summaryMessages('✓')[0]).toEqual([ + 'Created a Vue Pages starter with D1 (SQLite)', + '~ ../../pnpm-workspace.yaml approved builds', + ]); } finally { rmSync(wsroot, { recursive: true, force: true }); } diff --git a/platform/packages/api/src/index.ts b/platform/packages/api/src/index.ts index 9106534ab..b84654e71 100644 --- a/platform/packages/api/src/index.ts +++ b/platform/packages/api/src/index.ts @@ -527,6 +527,7 @@ const app = new Hono() context.json({ acceptsExecutionPlans: true, skipMigrations: true, + d1ReadReplication: true, migrationProtocolVersion: 2, supportedClassifierVersions: [1], }), diff --git a/platform/packages/api/test/integration/api-deploy.test.ts b/platform/packages/api/test/integration/api-deploy.test.ts index 0f8ec99b2..48b62dce8 100644 --- a/platform/packages/api/test/integration/api-deploy.test.ts +++ b/platform/packages/api/test/integration/api-deploy.test.ts @@ -1551,6 +1551,11 @@ describe('POST /projects/:id/deploy', () => { expect(await response.json()).toMatchObject({ skipMigrations: true }); }); + it('advertises D1 read replication support', async () => { + const response = await app.fetch(new Request('https://api.test/capabilities'), env); + expect(await response.json()).toMatchObject({ d1ReadReplication: true }); + }); + it('runs migrations via worker after upload (v3 flow)', async () => { const user = await createTestUser(env.db); const project = await createTestProject(env.db, user.id, 'migrate-app'); diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml index 9625ae931..865197b26 100644 --- a/pnpm-lock.yaml +++ b/pnpm-lock.yaml @@ -223,13 +223,13 @@ catalogs: specifier: 0.7.16 version: 0.7.16 '@cloudflare/build-output-utils': - specifier: ^1.0.0-rc.0 + specifier: 1.0.0-rc.0 version: 1.0.0-rc.0 '@cloudflare/codemods': specifier: ^0.8.0 version: 0.8.0 '@cloudflare/config': - specifier: ^1.0.0-rc.0 + specifier: 1.0.0-rc.0 version: 1.0.0-rc.0 '@cloudflare/sandbox': specifier: ^1.0.3 diff --git a/pnpm-workspace.yaml b/pnpm-workspace.yaml index 3576af205..3b181bd3d 100644 --- a/pnpm-workspace.yaml +++ b/pnpm-workspace.yaml @@ -49,9 +49,9 @@ catalog: '@stylexjs/stylex': 0.19.2 '@angular/build': ^22.2.7 '@astrojs/cloudflare': ^14.4.8 - '@cloudflare/build-output-utils': ^1.0.0-rc.0 + '@cloudflare/build-output-utils': 1.0.0-rc.0 '@cloudflare/codemods': ^0.8.0 - '@cloudflare/config': ^1.0.0-rc.0 + '@cloudflare/config': 1.0.0-rc.0 '@cloudflare/vite-plugin': 2.0.0-rc.0 '@cloudflare/workers-auth': ^0.18.3 '@nkzw/fate': 2.2.3