diff --git a/packages/cli/CHANGELOG.md b/packages/cli/CHANGELOG.md index e60f3abcfd..b2dfd9d5f7 100644 --- a/packages/cli/CHANGELOG.md +++ b/packages/cli/CHANGELOG.md @@ -1,5 +1,155 @@ # Changelog +## [0.12.0] + +šŸ“¦ **NPM:** https://www.npmjs.com/package/@qvac/cli/v/0.12.0 + +This release closes the gap between installing the CLI and having a working server. `qvac configure` writes a valid `qvac.config.json` for you, a new catalog endpoint lets you browse the models the SDK provides, and `qvac serve openai` finally honours `preload: false` by loading on first use instead of failing forever. `qvac doctor --deep` can now prove the installed SDK actually starts. + +## New Commands + +### `qvac configure` builds your config for you + +Getting from a fresh install to a working `qvac serve openai` used to mean hand-writing `serve.models` and knowing model constant names. `qvac configure` does it interactively: add a model by capability or search everything, preview the entry it will write, edit it in `$EDITOR` if you want, then merge and save. + +```bash +qvac configure # interactive +qvac configure --yes # chat + transcription starter +qvac configure --modality chat --modality image # pick specific capabilities +``` + +Search matches on name, role, addon, and quantization, with id matches ranked first. Aliases are derived from the model name (`QWEN3_600M_INST_Q4` ā†’ `qwen3-600m-inst-q4`) and deduped. For llamacpp chat and embedding entries the prompts are schema-driven ā€” type hints, field descriptions, and per-field validation come from the SDK's own config schemas. + +Writes are safe: the config is written atomically, an existing `qvac.config.json` is merged rather than replaced, and the command refuses to shadow a non-JSON config (`.js`/`.ts`), printing guidance instead. Re-running is idempotent per model; `--force` overwrites an existing entry in place. `Esc` steps back one menu and `Ctrl+C` aborts without writing anything. + +Chat, embedding, transcription, and image presets are runnable as written. TTS is an example template carrying a `referenceAudioSrc` placeholder and a link to the addon docs, because a voice reference cannot be guessed ā€” the command is honest about where you have to finish the job by hand. + +This is the actionable end of the catalog's `not_configured` hint: browse a model with `GET /v1/models/catalog`, then run `qvac configure` to make it callable. + +## New APIs + +### Browse available models by capability + +`GET /v1/models` only ever described models you had already configured, so there was no way to find out what else the SDK could run. `GET /v1/models/catalog` now lists configured models alongside the SDK's in-process constant catalog, filterable by capability: + +```bash +# Chat-capable models, 20 at a time +curl 'http://localhost:11434/v1/models/catalog?role=chat&limit=20' + +# Free-text search on the model id +curl 'http://localhost:11434/v1/models/catalog?search=qwen' + +# A single entry +curl 'http://localhost:11434/v1/models/catalog/QWEN3_600M_INST_Q4' +``` + +Filters cover `search`, `role`, `addon` (or `type`), `quantization`, `engine`, and `configured`, with `limit`/`offset` pagination and a `has_more` flag. + +Entries are deliberately **not** OpenAI `model` objects ā€” they are `model_catalog_entry` rows, because a catalog model that is absent from `serve.models` cannot be called on this server: + +```json +{ + "object": "model_catalog_entry", + "id": "QWEN3_600M_INST_Q4", + "configured": false, + "usable": false, + "state": "not_configured", + "role": "chat", + "addon": "llm", + "quantization": "q4", + "params": "600M", + "size": 382156480, + "hint": "ā€¦" +} +``` + +Every row carries `configured`, `usable`, and a `state` that includes a `not_configured` value for catalog-only models, plus a `hint` pointing at how to configure it. `GET /v1/models` remains the single authoritative list of callable models, so a browsable model can never be mistaken for a ready one. + +Browsing is fully in-process: it triggers no SDK call, no model load, and no download. Sizes, parameter counts, quantizations, and roles come from the constants, while configured models report their live registry state. + +## New Flags + +### `qvac doctor --deep` proves the SDK actually runs + +The static `qvac doctor` checks could pass on an install whose SDK worker could not start, finish its heartbeat, or shut down cleanly. `--deep` exercises the installed `@qvac/sdk` in an isolated child process ā€” import, worker heartbeat, and shutdown ā€” without loading a model: + +```bash +qvac doctor --deep +qvac doctor --deep --verbose # include probe diagnostics +qvac doctor --deep --json # machine-readable result +``` + +It requires a structured IPC result and a matching process exit code, so a probe that dies quietly is a failure rather than a pass. Common CPU, native-library, Visual C++ runtime, Vulkan, Bare, and worker-handshake failures are classified rather than reported as one generic error. Plain `qvac doctor` behaviour is unchanged unless `--deep` is passed. + +### Lazy loading is tunable, and can be turned off + +Lazy loading is on by default. Four new `qvac serve openai` flags control it: + +```bash +# Refuse to load on demand ā€” an unloaded model returns 503 model_not_loaded +qvac serve openai --no-lazy-load + +# Allow two models to load at once (default: 1) +qvac serve openai --load-concurrency 2 + +# Give up on a cold start after 5 minutes (default: unbounded) +qvac serve openai --load-timeout 300000 + +# Finish a load even if the client that triggered it disconnects +qvac serve openai --no-cancel-load-on-disconnect +``` + +The same settings are available in the config file under a new `serve.load` block, which the flags override: + +```json +{ + "serve": { + "load": { + "lazy": true, + "concurrency": 1, + "timeoutMs": null, + "cancelOnDisconnect": true + } + } +} +``` + +`concurrency` and `timeoutMs` must be positive integers; `timeoutMs: null` means no timeout. A config value that is the wrong type or out of range fails startup with the offending path named, rather than being silently ignored. + +## Bug Fixes + +### `preload: false` now lazy-loads instead of failing forever + +A model configured with `preload: false` was registered but never loaded, so every request naming it returned `503 model_not_ready` indefinitely ā€” despite the documented promise of a lazy cold start. Such a model now loads the first time a request names it. + +The load is concurrency-safe: simultaneous first requests share a single load rather than starting several, and a failed cold start surfaces as `503 model_load_failed` and is retried on the next request instead of poisoning the alias. + +```bash +# First request loads the model (and blocks while it does); later requests are fast +curl -X POST http://localhost:11434/v1/chat/completions \ + -H 'content-type: application/json' \ + -d '{"model":"my-llm","messages":[{"role":"user","content":"hi"}]}' +``` + +Preload defaults are unchanged: constant entries still default to `true`, explicit `{ src, type }` entries to `false`, and `--model ` still forces a warm start. + +### Unloading a model is reversible + +`DELETE /v1/models/{id}` used to remove the alias from the registry with no way back. The model stayed resolvable from config but was permanently unavailable until the server was restarted. It now resets the alias to `IDLE` and drops the SDK handle, freeing the resources while leaving the alias intact, so the next request simply reloads it: + +```bash +# Frees resources but keeps the alias; the next request reloads it +curl -X DELETE http://localhost:11434/v1/models/my-llm +``` + +Listing follows the same principle. `GET /v1/models` reports every configured model whether or not it is loaded, and `GET /v1/models/{id}` resolves any configured alias, so loading stays transparent to the client. All inference gates ā€” the model requirement check, audio speech, and vector-store embedding ā€” now share one readiness helper, so they agree on when a model is usable. + +## Requirements + +This release requires `@qvac/sdk@^0.18.1`. `qvac configure` reads its llamacpp field descriptions and validation from the `@qvac/sdk/schemas` subpath, which 0.18.1 is the first SDK release to export. + +It also adds one third-party dependency, `@inquirer/prompts` (pinned to `8.5.2`), for the interactive prompts. + ## [0.11.0] šŸ“¦ **NPM:** https://www.npmjs.com/package/@qvac/cli/v/0.11.0 diff --git a/packages/cli/README.md b/packages/cli/README.md index 6fb8954916..b773393f60 100644 --- a/packages/cli/README.md +++ b/packages/cli/README.md @@ -9,6 +9,7 @@ This package is published to npm as **`@qvac/cli`** and lives in the QVAC monore - [Installation](#installation) - [Command Reference](#command-reference) - [`doctor`](#doctor) + - [`configure`](#configure) - [`bundle sdk`](#bundle-sdk) - [`verify deps`](#verify-deps) - [`verify bundle`](#verify-bundle) @@ -70,11 +71,12 @@ qvac doctor [options] **Options:** -| Flag | Description | -| --------------- | ----------------------------------------- | -| `--json` | Output the report as JSON. | -| `-q, --quiet` | Suppress stdout ā€” only set the exit code. | -| `-v, --verbose` | Detailed output. | +| Flag | Description | +| --------------- | ------------------------------------------------------------- | +| `--deep` | Start the installed SDK worker and verify its heartbeat. | +| `--json` | Output the report as JSON. | +| `-q, --quiet` | Suppress stdout ā€” only set the exit code. | +| `-v, --verbose` | Include bounded worker stdout/stderr when a deep check fails. | **What it checks:** @@ -91,6 +93,11 @@ qvac doctor [options] Bun. - **Project** ā€” whether `@qvac/sdk` is resolvable from the current working directory (works for hoisted monorepo installs too). +- **SDK runtime (`--deep`)** ā€” starts the installed SDK in an isolated Node.js + process, performs a worker heartbeat, and closes it. The probe is bounded to + 45 seconds and classifies common Bare, native library, CPU instruction, + Vulkan, and worker-handshake failures. When `--deep` is requested, a missing + SDK, failed heartbeat, or failed cleanup causes exit code `1`. See [`system-requirements.md`](./system-requirements.md) for the full list of thresholds and rationale. @@ -101,6 +108,9 @@ thresholds and rationale. # Human-readable report qvac doctor +# Exercise SDK worker startup without loading a model +qvac doctor --deep + # JSON for CI / scripts qvac doctor --json @@ -108,6 +118,37 @@ qvac doctor --json qvac doctor --quiet || exit 1 ``` +### `configure` + +Interactively build a `qvac.config.json` with a starter `serve.models`, so you can go +straight to `qvac serve openai`. It searches the models the SDK provides ā€” by name or by +capability (role, addon, quantization) ā€” and on a wide terminal previews, for the +highlighted result, the exact `serve.models` entry it would produce. Pick a model, rename +its alias, set config parameters (guided by the SDK's config schema ā€” each field shows its +type and description and is validated on entry, for model types the SDK exposes a schema for; +currently llama.cpp chat + embedding), and (with `$EDITOR`) tweak the entry and review the +result before adding it. Press `Esc` (or choose `Back`) to step back one menu; `Ctrl+C` +aborts without writing. Existing entries are preserved; re-running is idempotent per model. + +```bash +qvac configure # interactive +qvac configure --yes # non-interactive: write a chat + transcription starter +qvac configure --modality chat --modality image +``` + +| Flag | Description | +| --------------------- | ----------------------------------------------------------------------------------------------------------- | +| `-c, --config ` | Config file to write (default: `./qvac.config.json`). JSON only. | +| `-y, --yes` | Non-interactive: write a sensible default starter (chat + transcription). | +| `--modality ` | Non-interactive: add a modality (repeatable) ā€” `chat` / `embedding` / `transcription` / `speech` / `image`. | +| `--force` | Re-add a model that is already configured (overwrites its existing entry in place). | +| `-q, --quiet` | Suppress output. | + +Single-artifact modalities (chat, embedding, transcription, image) are runnable as written. +Text-to-speech is emitted as a best-effort example with a `referenceAudioSrc` placeholder ā€” +set it to a real `.wav` and see the linked TTS docs to finish. Runs in a terminal; for +non-TTY use `--yes` / `--modality`. + ### `bundle sdk` Generate a tree-shaken Bare worker bundle containing the plugins you select (defaults to all built-in plugins). diff --git a/packages/cli/changelog/0.12.0/CHANGELOG.md b/packages/cli/changelog/0.12.0/CHANGELOG.md new file mode 100644 index 0000000000..af1ebd8d91 --- /dev/null +++ b/packages/cli/changelog/0.12.0/CHANGELOG.md @@ -0,0 +1,17 @@ +# Changelog v0.12.0 + +Release Date: 2026-08-21 + +## āœØ Features + +- Add deep SDK runtime checks to qvac doctor. (see PR [#3492](https://github.com/tetherto/qvac/pull/3492)) +- Add `qvac configure` interactive config generator. (see PR [#3953](https://github.com/tetherto/qvac/pull/3953)) + +## šŸ”Œ API + +- Honor preload:false via lazy-load, keep DELETE reversible. (see PR [#3906](https://github.com/tetherto/qvac/pull/3906)) - See [API changes](./api.md) +- Browse models by capability (serve catalog). (see PR [#3932](https://github.com/tetherto/qvac/pull/3932)) - See [API changes](./api.md) + +## āš™ļø Infrastructure + +- Revert incident-era macOS runner switches ā€” back to qvac-macos26-arm64-gpu. (see PR [#3859](https://github.com/tetherto/qvac/pull/3859)) diff --git a/packages/cli/changelog/0.12.0/CHANGELOG_LLM.md b/packages/cli/changelog/0.12.0/CHANGELOG_LLM.md new file mode 100644 index 0000000000..909c5b451d --- /dev/null +++ b/packages/cli/changelog/0.12.0/CHANGELOG_LLM.md @@ -0,0 +1,149 @@ +# QVAC CLI v0.12.0 Release Notes + +šŸ“¦ **NPM:** https://www.npmjs.com/package/@qvac/cli/v/0.12.0 + +This release closes the gap between installing the CLI and having a working server. `qvac configure` writes a valid `qvac.config.json` for you, a new catalog endpoint lets you browse the models the SDK provides, and `qvac serve openai` finally honours `preload: false` by loading on first use instead of failing forever. `qvac doctor --deep` can now prove the installed SDK actually starts. + +## New Commands + +### `qvac configure` builds your config for you + +Getting from a fresh install to a working `qvac serve openai` used to mean hand-writing `serve.models` and knowing model constant names. `qvac configure` does it interactively: add a model by capability or search everything, preview the entry it will write, edit it in `$EDITOR` if you want, then merge and save. + +```bash +qvac configure # interactive +qvac configure --yes # chat + transcription starter +qvac configure --modality chat --modality image # pick specific capabilities +``` + +Search matches on name, role, addon, and quantization, with id matches ranked first. Aliases are derived from the model name (`QWEN3_600M_INST_Q4` ā†’ `qwen3-600m-inst-q4`) and deduped. For llamacpp chat and embedding entries the prompts are schema-driven ā€” type hints, field descriptions, and per-field validation come from the SDK's own config schemas. + +Writes are safe: the config is written atomically, an existing `qvac.config.json` is merged rather than replaced, and the command refuses to shadow a non-JSON config (`.js`/`.ts`), printing guidance instead. Re-running is idempotent per model; `--force` overwrites an existing entry in place. `Esc` steps back one menu and `Ctrl+C` aborts without writing anything. + +Chat, embedding, transcription, and image presets are runnable as written. TTS is an example template carrying a `referenceAudioSrc` placeholder and a link to the addon docs, because a voice reference cannot be guessed ā€” the command is honest about where you have to finish the job by hand. + +This is the actionable end of the catalog's `not_configured` hint: browse a model with `GET /v1/models/catalog`, then run `qvac configure` to make it callable. + +## New APIs + +### Browse available models by capability + +`GET /v1/models` only ever described models you had already configured, so there was no way to find out what else the SDK could run. `GET /v1/models/catalog` now lists configured models alongside the SDK's in-process constant catalog, filterable by capability: + +```bash +# Chat-capable models, 20 at a time +curl 'http://localhost:11434/v1/models/catalog?role=chat&limit=20' + +# Free-text search on the model id +curl 'http://localhost:11434/v1/models/catalog?search=qwen' + +# A single entry +curl 'http://localhost:11434/v1/models/catalog/QWEN3_600M_INST_Q4' +``` + +Filters cover `search`, `role`, `addon` (or `type`), `quantization`, `engine`, and `configured`, with `limit`/`offset` pagination and a `has_more` flag. + +Entries are deliberately **not** OpenAI `model` objects ā€” they are `model_catalog_entry` rows, because a catalog model that is absent from `serve.models` cannot be called on this server: + +```json +{ + "object": "model_catalog_entry", + "id": "QWEN3_600M_INST_Q4", + "configured": false, + "usable": false, + "state": "not_configured", + "role": "chat", + "addon": "llm", + "quantization": "q4", + "params": "600M", + "size": 382156480, + "hint": "ā€¦" +} +``` + +Every row carries `configured`, `usable`, and a `state` that includes a `not_configured` value for catalog-only models, plus a `hint` pointing at how to configure it. `GET /v1/models` remains the single authoritative list of callable models, so a browsable model can never be mistaken for a ready one. + +Browsing is fully in-process: it triggers no SDK call, no model load, and no download. Sizes, parameter counts, quantizations, and roles come from the constants, while configured models report their live registry state. + +## New Flags + +### `qvac doctor --deep` proves the SDK actually runs + +The static `qvac doctor` checks could pass on an install whose SDK worker could not start, finish its heartbeat, or shut down cleanly. `--deep` exercises the installed `@qvac/sdk` in an isolated child process ā€” import, worker heartbeat, and shutdown ā€” without loading a model: + +```bash +qvac doctor --deep +qvac doctor --deep --verbose # include probe diagnostics +qvac doctor --deep --json # machine-readable result +``` + +It requires a structured IPC result and a matching process exit code, so a probe that dies quietly is a failure rather than a pass. Common CPU, native-library, Visual C++ runtime, Vulkan, Bare, and worker-handshake failures are classified rather than reported as one generic error. Plain `qvac doctor` behaviour is unchanged unless `--deep` is passed. + +### Lazy loading is tunable, and can be turned off + +Lazy loading is on by default. Four new `qvac serve openai` flags control it: + +```bash +# Refuse to load on demand ā€” an unloaded model returns 503 model_not_loaded +qvac serve openai --no-lazy-load + +# Allow two models to load at once (default: 1) +qvac serve openai --load-concurrency 2 + +# Give up on a cold start after 5 minutes (default: unbounded) +qvac serve openai --load-timeout 300000 + +# Finish a load even if the client that triggered it disconnects +qvac serve openai --no-cancel-load-on-disconnect +``` + +The same settings are available in the config file under a new `serve.load` block, which the flags override: + +```json +{ + "serve": { + "load": { + "lazy": true, + "concurrency": 1, + "timeoutMs": null, + "cancelOnDisconnect": true + } + } +} +``` + +`concurrency` and `timeoutMs` must be positive integers; `timeoutMs: null` means no timeout. A config value that is the wrong type or out of range fails startup with the offending path named, rather than being silently ignored. + +## Bug Fixes + +### `preload: false` now lazy-loads instead of failing forever + +A model configured with `preload: false` was registered but never loaded, so every request naming it returned `503 model_not_ready` indefinitely ā€” despite the documented promise of a lazy cold start. Such a model now loads the first time a request names it. + +The load is concurrency-safe: simultaneous first requests share a single load rather than starting several, and a failed cold start surfaces as `503 model_load_failed` and is retried on the next request instead of poisoning the alias. + +```bash +# First request loads the model (and blocks while it does); later requests are fast +curl -X POST http://localhost:11434/v1/chat/completions \ + -H 'content-type: application/json' \ + -d '{"model":"my-llm","messages":[{"role":"user","content":"hi"}]}' +``` + +Preload defaults are unchanged: constant entries still default to `true`, explicit `{ src, type }` entries to `false`, and `--model ` still forces a warm start. + +### Unloading a model is reversible + +`DELETE /v1/models/{id}` used to remove the alias from the registry with no way back. The model stayed resolvable from config but was permanently unavailable until the server was restarted. It now resets the alias to `IDLE` and drops the SDK handle, freeing the resources while leaving the alias intact, so the next request simply reloads it: + +```bash +# Frees resources but keeps the alias; the next request reloads it +curl -X DELETE http://localhost:11434/v1/models/my-llm +``` + +Listing follows the same principle. `GET /v1/models` reports every configured model whether or not it is loaded, and `GET /v1/models/{id}` resolves any configured alias, so loading stays transparent to the client. All inference gates ā€” the model requirement check, audio speech, and vector-store embedding ā€” now share one readiness helper, so they agree on when a model is usable. + +## Requirements + +This release requires `@qvac/sdk@^0.18.1`. `qvac configure` reads its llamacpp field descriptions and validation from the `@qvac/sdk/schemas` subpath, which 0.18.1 is the first SDK release to export. + +It also adds one third-party dependency, `@inquirer/prompts` (pinned to `8.5.2`), for the interactive prompts. diff --git a/packages/cli/changelog/0.12.0/api.md b/packages/cli/changelog/0.12.0/api.md new file mode 100644 index 0000000000..d6f3c8273c --- /dev/null +++ b/packages/cli/changelog/0.12.0/api.md @@ -0,0 +1,57 @@ +# šŸ”Œ API Changes v0.12.0 + +## Honor preload:false via lazy-load, keep DELETE reversible + +PR: [#3906](https://github.com/tetherto/qvac/pull/3906) + +```bash +# preload:false model ā€” first request lazy-loads it (blocks), later requests are fast +curl -X POST http://localhost:11434/v1/chat/completions \ + -H 'content-type: application/json' \ + -d '{"model":"my-llm","messages":[{"role":"user","content":"hi"}]}' + +# unload frees resources but keeps the alias; the next request reloads it +curl -X DELETE http://localhost:11434/v1/models/my-llm + +# lists every configured model, loaded or not +curl http://localhost:11434/v1/models +``` + +New `qvac serve openai` flags controlling the lazy load: + +```bash +qvac serve openai --no-lazy-load # 503 model_not_loaded instead of loading +qvac serve openai --load-concurrency 2 # max simultaneous loads (default: 1) +qvac serve openai --load-timeout 300000 # per-load timeout in ms (default: unbounded) +qvac serve openai --no-cancel-load-on-disconnect # keep loading if the client disconnects +``` + +Equivalent config, which the flags override: + +```json +{ + "serve": { + "load": { "lazy": true, "concurrency": 1, "timeoutMs": null, "cancelOnDisconnect": true } + } +} +``` + +--- + +## Browse models by capability (serve catalog) + +PR: [#3932](https://github.com/tetherto/qvac/pull/3932) + +```bash +# Browse chat-capable models the SDK provides (not configured ā†’ not callable yet) +curl 'http://localhost:11434/v1/models/catalog?role=chat&limit=20' +# ā†’ { "object":"list", "data":[ { "object":"model_catalog_entry", "id":"QWEN3_600M_INST_Q4", +# "configured":false, "usable":false, "state":"not_configured", "role":"chat", +# "addon":"llm", "quantization":"q4", "params":"600M", "size":382156480, "hint":"ā€¦" } ], +# "has_more": true } + +curl 'http://localhost:11434/v1/models/catalog?search=qwen' +curl 'http://localhost:11434/v1/models/catalog/QWEN3_600M_INST_Q4' +``` + +--- diff --git a/packages/cli/docs/serve-openai.md b/packages/cli/docs/serve-openai.md index 947ff3d7da..00edd63c80 100644 --- a/packages/cli/docs/serve-openai.md +++ b/packages/cli/docs/serve-openai.md @@ -140,7 +140,7 @@ Rows are **catalog entries**, not usable `model` objects: "quantization": "q4", "params": "600M", "size": 382156480, - "hint": "Not in serve.models ā€” add it there to make it usable (a `qvac configure` command is planned)." + "hint": "Not in serve.models ā€” run `qvac configure` (or add it there by hand) to make it usable." } ``` diff --git a/packages/cli/package.json b/packages/cli/package.json index 529fc42dcb..6267bb57ce 100644 --- a/packages/cli/package.json +++ b/packages/cli/package.json @@ -1,6 +1,6 @@ { "name": "@qvac/cli", - "version": "0.11.0", + "version": "0.12.0", "description": "Command-line interface for the QVAC ecosystem", "author": "Tether", "license": "Apache-2.0", @@ -60,7 +60,8 @@ "@fastify/multipart": "^9.0.0", "@fastify/swagger": "^9.0.0", "@fastify/swagger-ui": "^5.0.0", - "@qvac/sdk": "^0.17.0", + "@inquirer/prompts": "8.5.2", + "@qvac/sdk": "^0.18.1", "close-with-grace": "^2.1.0", "commander": "^14.0.3", "fastify": "^5.0.0", diff --git a/packages/cli/src/configure/docs-links.ts b/packages/cli/src/configure/docs-links.ts new file mode 100644 index 0000000000..2c613f9ba4 --- /dev/null +++ b/packages/cli/src/configure/docs-links.ts @@ -0,0 +1,48 @@ +// Curated map from an SDK `addon` value to its docs page on +// https://docs.qvac.tether.io. The slug is NOT derivable from the addon/engine +// name, and only some addons have a page (and a config anchor), so this is +// hand-maintained ā€” seeded from the docs sidebar (docs/website/src/lib/custom-tree.ts). +// Trailing slash matters (the site uses trailingSlash: true). + +const BASE = 'https://docs.qvac.tether.io' + +/** Generic fallback when an addon has no dedicated docs page. */ +export const CONFIG_DOCS_URL = `${BASE}/configuration/` + +interface AddonDocs { + url: string + /** Section anchor for model config, where the page has one (varies per page). */ + configAnchor?: string +} + +const ADDON_DOCS: Record = { + llm: { url: `${BASE}/addons/llm-llamacpp/`, configAnchor: '#4-create-the-config-obj' }, + embeddings: { url: `${BASE}/addons/embed-llamacpp/`, configAnchor: '#4-create-config' }, + whisper: { + url: `${BASE}/addons/transcription-whispercpp/`, + configAnchor: '#2-configure-transcription-parameters' + }, + parakeet: { + url: `${BASE}/addons/transcription-parakeet/`, + configAnchor: '#2-configure-parakeet-parameters' + }, + nmt: { + url: `${BASE}/addons/translation-nmtcpp/`, + configAnchor: '#3-create-the-config-object' + }, + diffusion: { + url: `${BASE}/addons/diffusion-cpp/`, + configAnchor: '#3-configure-the-native-backend-argsconfig' + }, + // No config anchor on these pages. + tts: { url: `${BASE}/addons/tts-ggml/` }, + audiogen: { url: `${BASE}/addons/audiogen-ggml/` } +} + +/** Real docs URL for an addon (deep-linked to its config section when one + * exists), or the generic configuration page when the addon has no page. */ +export function docsUrlForAddon(addon: string | null | undefined): string { + const docs = addon ? ADDON_DOCS[addon] : undefined + if (!docs) return CONFIG_DOCS_URL + return docs.configAnchor ? `${docs.url}${docs.configAnchor}` : docs.url +} diff --git a/packages/cli/src/configure/index.ts b/packages/cli/src/configure/index.ts new file mode 100644 index 0000000000..b64bde83f2 --- /dev/null +++ b/packages/cli/src/configure/index.ts @@ -0,0 +1,155 @@ +import { existsSync } from 'node:fs' +import { join, resolve } from 'node:path' +import { loadModelConstants } from '../serve/sdk-constants.js' +import { buildBuiltinCatalog } from '../serve/core/model-catalog.js' +import { + DEFAULT_STARTER, + MODALITIES, + TTS_VOICE_PLACEHOLDER, + buildAdditions, + type AddedEntry, + type Modality +} from './presets.js' +import { CONFIG_DOCS_URL, docsUrlForAddon } from './docs-links.js' +import { + existingAliasesByModel, + existingModelIdentities, + foreignConfigPath, + loadJsonConfig, + mergeServeModels, + writeConfigAtomically +} from './write-config.js' + +export interface ConfigureOptions { + projectRoot?: string + config?: string | undefined + yes?: boolean | undefined + modality?: string[] | undefined + force?: boolean | undefined + quiet?: boolean | undefined +} + +const VALID_MODALITIES = new Set(MODALITIES.map((m) => m.id)) + +function resolveModalities(input: string[] | undefined): Modality[] { + if (!input || input.length === 0) return DEFAULT_STARTER + return input.map((raw) => { + const m = raw.trim().toLowerCase() + if (!VALID_MODALITIES.has(m)) { + throw new Error(`Unknown modality "${raw}". Valid: ${[...VALID_MODALITIES].join(', ')}`) + } + return m as Modality + }) +} + +export async function runConfigure(options: ConfigureOptions): Promise { + const projectRoot = options.projectRoot ?? process.cwd() + const targetPath = options.config + ? resolve(projectRoot, options.config) + : join(projectRoot, 'qvac.config.json') + const print = (msg = ''): void => { + if (!options.quiet) process.stdout.write(`${msg}\n`) + } + + if (!targetPath.endsWith('.json')) { + throw new Error('configure writes JSON only ā€” target a qvac.config.json path.') + } + + // A non-JSON config already owns this project ā€” don't create a competing .json. + if (!existsSync(targetPath)) { + const foreign = foreignConfigPath(projectRoot) + if (foreign) { + print(`Found ${foreign}, which configure can't rewrite safely.`) + print(`Add models to its serve.models by hand ā€” see ${CONFIG_DOCS_URL}`) + return + } + } + + const nonInteractive = options.yes === true || (options.modality?.length ?? 0) > 0 + if (!nonInteractive && process.stdin.isTTY !== true) { + throw new Error('configure is interactive; run it in a terminal, or pass --yes / --modality.') + } + + const existing = loadJsonConfig(targetPath) + const existingAliases = new Set(Object.keys(existing.serve?.models ?? {})) + + let additions: AddedEntry[] + if (nonInteractive) { + const selections = resolveModalities(options.modality).map((modality) => ({ modality })) + additions = buildAdditions(selections, new Set(existingAliases)) + } else { + const { runInteractive } = await import('./prompts.js') + const catalog = buildBuiltinCatalog(loadModelConstants()) + try { + additions = await runInteractive(catalog, existingAliases, process.stdout.isTTY === true) + } catch (err) { + // Ctrl+C in a prompt throws ExitPromptError ā€” treat it as a clean abort: + // "Done" is the only path that writes, so nothing is persisted here. + if (err instanceof Error && err.name === 'ExitPromptError') { + print('\nCancelled ā€” nothing written.') + return + } + throw err + } + } + + if (additions.length === 0) { + print('Nothing to add.') + return + } + + // Idempotent per model: skip a constant that's already configured. With + // --force, re-add it ā€” overwriting the existing alias in place rather than + // minting a deduped `-2` (aliasFor already uniqued against existing). + const configuredIds = existingModelIdentities(existing) + const aliasByModel = options.force === true ? existingAliasesByModel(existing) : null + const fresh: AddedEntry[] = [] + const alreadyConfigured: string[] = [] + for (const a of additions) { + const id = a.entry.model ?? a.entry.src + if (id !== undefined && configuredIds.has(id)) { + if (options.force !== true) { + alreadyConfigured.push(id) + continue + } + const existingAlias = aliasByModel?.get(id) + if (existingAlias) a.alias = existingAlias + } + fresh.push(a) + } + + if (fresh.length === 0) { + print( + alreadyConfigured.length + ? `Already configured: ${alreadyConfigured.join(', ')} (use --force to re-add).` + : 'No changes.' + ) + return + } + + const additionsMap = Object.fromEntries(fresh.map((a) => [a.alias, a.entry])) + const { config, added } = mergeServeModels(existing, additionsMap, options.force === true) + + writeConfigAtomically(targetPath, config) + + const total = Object.keys(config.serve?.models ?? {}).length + const quote = (a: string): string => `"${a}"` + const updated = added.filter((a) => existingAliases.has(a)) + const newlyAdded = added.filter((a) => !existingAliases.has(a)) + const parts: string[] = [] + if (newlyAdded.length) parts.push(`added ${newlyAdded.map(quote).join(', ')}`) + if (updated.length) parts.push(`updated ${updated.map(quote).join(', ')}`) + print(`\nāœ… ${parts.join('; ')} ā€” ${targetPath} now has ${total} model${total === 1 ? '' : 's'}.`) + if (alreadyConfigured.length) { + print(` Already configured (skipped): ${alreadyConfigured.join(', ')}`) + } + for (const a of fresh) { + if (JSON.stringify(a.entry).includes(TTS_VOICE_PLACEHOLDER)) { + print( + ` ā€¢ ${a.alias}: set config.referenceAudioSrc to a real .wav ā€” ${docsUrlForAddon(a.addon)}` + ) + } + } + print('\n Run: qvac serve openai') + print(` Docs: ${CONFIG_DOCS_URL}`) +} diff --git a/packages/cli/src/configure/param-schemas.ts b/packages/cli/src/configure/param-schemas.ts new file mode 100644 index 0000000000..41cc95e022 --- /dev/null +++ b/packages/cli/src/configure/param-schemas.ts @@ -0,0 +1,94 @@ +// Bridges the SDK's exported modelConfig Zod schemas into configure's parameter +// editor: enumerate a model type's config fields with type hints + descriptions, +// and validate user input against the real schema. Only the schemas the SDK +// exports today (llamacpp completion + embedding) are wired; extend the map as +// the SDK exposes more (whisper, diffusion, tts, ...). +import { z } from 'zod' +import { llamacppCompletionConfigSchema, llamacppEmbeddingConfigSchema } from '@qvac/sdk/schemas' + +type ConfigSchema = z.ZodObject + +const SCHEMA_BY_ADDON: Record = { + llm: llamacppCompletionConfigSchema as ConfigSchema, + embeddings: llamacppEmbeddingConfigSchema as ConfigSchema +} + +export function configSchemaForAddon(addon: string | null | undefined): ConfigSchema | undefined { + if (!addon) return undefined + return SCHEMA_BY_ADDON[addon] +} + +export interface ParamField { + name: string + type: string + description: string + schema: z.ZodType +} + +type JsonNode = { + type?: unknown + enum?: unknown + anyOf?: unknown + const?: unknown + items?: unknown + minimum?: unknown + maximum?: unknown + description?: unknown +} + +function typeLabel(node: JsonNode | undefined): string { + if (!node) return 'value' + // Render enum/const values bare (`causal | non-causal`, not `"causal" | ...`) + // so the hint matches what the user types; coerceParam accepts bare, single-, + // or double-quoted forms. + if (Array.isArray(node.enum)) return node.enum.map((v) => String(v)).join(' | ') + if (Array.isArray(node.anyOf)) return node.anyOf.map((n) => typeLabel(n as JsonNode)).join(' | ') + if (node.const !== undefined) return String(node.const) + if (node.type === 'array') return `${typeLabel(node.items as JsonNode)}[]` + const base = typeof node.type === 'string' ? node.type : 'value' + const bounds: string[] = [] + if (typeof node.minimum === 'number') bounds.push(`>= ${node.minimum}`) + if (typeof node.maximum === 'number') bounds.push(`<= ${node.maximum}`) + return bounds.length ? `${base} (${bounds.join(', ')})` : base +} + +export function paramFields(schema: ConfigSchema): ParamField[] { + const json = z.toJSONSchema(schema, { unrepresentable: 'any' }) as { + properties?: Record + } + const props = json.properties ?? {} + return Object.entries(schema.shape).map(([name, field]) => { + const node = props[name] + return { + name, + type: typeLabel(node), + description: typeof node?.description === 'string' ? node.description : '', + schema: field as z.ZodType + } + }) +} + +// Coerce a raw string into the value the field expects. JSON handles +// numbers/booleans/arrays/double-quoted strings; if that fails, a value wrapped +// in single quotes (as field hints render enum values, e.g. 'causal') is +// unwrapped, otherwise the bare string is used. So `causal`, `'causal'`, and +// `"causal"` all coerce to the same value. +export function coerceParam(raw: string): unknown { + const t = raw.trim() + if (t === '') return undefined + try { + return JSON.parse(t) + } catch { + const singleQuoted = t.match(/^'(.*)'$/) + return singleQuoted ? singleQuoted[1] : t + } +} + +// Empty input clears the field; otherwise the coerced value must pass the field +// schema. Returns true or a message for @inquirer's validate. +export function validateParam(field: ParamField, raw: string): true | string { + if (raw.trim() === '') return true + const result = field.schema.safeParse(coerceParam(raw)) + if (result.success) return true + return result.error.issues[0]?.message ?? 'Invalid value' +} diff --git a/packages/cli/src/configure/presets.ts b/packages/cli/src/configure/presets.ts new file mode 100644 index 0000000000..bfcc24f09f --- /dev/null +++ b/packages/cli/src/configure/presets.ts @@ -0,0 +1,145 @@ +// Curated modality metadata, recommended defaults, and the pure functions that +// turn a chosen model (or a template) into a serve.models entry. No prompts and +// no SDK RPCs here ā€” this is the testable core the interactive layer (and, later, +// an LLM advisor) feeds. + +export type Modality = 'chat' | 'embedding' | 'transcription' | 'speech' | 'image' + +export interface ServeModelEntry { + model?: string + src?: string + type?: string + preload?: boolean + config?: Record +} + +export interface BuiltEntry { + /** Basis for the alias (a constant name); the caller kebabs + dedupes it. */ + aliasBase: string + entry: ServeModelEntry + /** SDK addon, for resolving the docs link. */ + addon: string +} + +export interface ModalityInfo { + id: Modality + label: string + /** endpointCategory used to filter the catalog. */ + role: string + addon: string + /** true ā†’ user searches/picks a real constant; false ā†’ fixed template. */ + pick: boolean +} + +export const MODALITIES: ModalityInfo[] = [ + { id: 'chat', label: 'Chat (LLM)', role: 'chat', addon: 'llm', pick: true }, + { id: 'embedding', label: 'Embedding', role: 'embedding', addon: 'embeddings', pick: true }, + { + id: 'transcription', + label: 'Speech-to-text (transcription)', + role: 'transcription', + addon: 'whisper', + pick: true + }, + { id: 'image', label: 'Image generation', role: 'image', addon: 'diffusion', pick: true }, + { id: 'speech', label: 'Text-to-speech', role: 'speech', addon: 'tts', pick: false } +] + +export function modalityInfo(id: Modality): ModalityInfo { + const info = MODALITIES.find((m) => m.id === id) + if (!info) throw new Error(`Unknown modality "${id}"`) + return info +} + +/** Curated recommended constant per pick-able modality (smallest well-known). */ +export const RECOMMENDED: Partial> = { + chat: 'QWEN3_600M_INST_Q4', + embedding: 'EMBEDDINGGEMMA_300M_Q4_0', + transcription: 'WHISPER_TINY_Q8_0', + image: 'SD_V2_1_1B_Q8_0' +} + +/** Non-interactive `--yes` default: ā‰„2 modalities, both fully runnable. */ +export const DEFAULT_STARTER: Modality[] = ['chat', 'transcription'] + +// A best-effort TTS example. TTS is an assembly (no single-artifact constant), +// and the voice is a user asset ā€” so this ships with a placeholder voice path +// and relies on the docs link to finish. Not guaranteed to run untouched. +export const TTS_VOICE_PLACEHOLDER = '/path/to/voice.wav' + +function ttsTemplate(): BuiltEntry { + return { + aliasBase: 'TTS_T3_TURBO_EN_CHATTERBOX_Q8_0', + addon: 'tts', + entry: { + type: 'tts', + src: 'TTS_T3_TURBO_EN_CHATTERBOX_Q8_0', + preload: false, + config: { + ttsEngine: 'chatterbox', + language: 'en', + s3genModelSrc: 'TTS_S3GEN_EN_CHATTERBOX', + referenceAudioSrc: TTS_VOICE_PLACEHOLDER + } + } + } +} + +/** Build a serve.models entry for a modality. For pick-able modalities pass the + * chosen constant name; for `speech` the constant is ignored (fixed template). */ +export function buildEntry(modality: Modality, constantName?: string): BuiltEntry { + const info = modalityInfo(modality) + if (!info.pick) return ttsTemplate() + + const name = constantName ?? RECOMMENDED[modality] + if (!name) throw new Error(`No model chosen and no recommended default for "${modality}"`) + + const entry: ServeModelEntry = { model: name, preload: false } + if (modality === 'image') entry.config = { prediction: 'v' } + return { aliasBase: name, entry, addon: info.addon } +} + +/** Generic entry for a model picked via "search all" (no modality-specific + * config). `addon` drives the docs link. */ +export function buildGenericEntry(constantName: string, addon: string | null): BuiltEntry { + return { + aliasBase: constantName, + entry: { model: constantName, preload: false }, + addon: addon ?? '' + } +} + +export interface AddedEntry { + alias: string + addon: string + entry: ServeModelEntry +} + +/** Turn modality selections into aliased serve.models additions, deduping + * aliases against `taken` (mutated as it goes). Used by the non-interactive path. */ +export function buildAdditions( + selections: Array<{ modality: Modality; constantName?: string }>, + taken: Set +): AddedEntry[] { + const out: AddedEntry[] = [] + for (const sel of selections) { + const built = buildEntry(sel.modality, sel.constantName) + const alias = aliasFor(built.aliasBase, taken) + taken.add(alias) + out.push({ alias, addon: built.addon, entry: built.entry }) + } + return out +} + +/** Kebab-case a constant name into an alias, deduped against `taken`. */ +export function aliasFor(base: string, taken: Set): string { + const slug = + base + .toLowerCase() + .replace(/[^a-z0-9]+/g, '-') + .replace(/^-+|-+$/g, '') || 'model' + if (!taken.has(slug)) return slug + let n = 2 + while (taken.has(`${slug}-${n}`)) n++ + return `${slug}-${n}` +} diff --git a/packages/cli/src/configure/prompts.ts b/packages/cli/src/configure/prompts.ts new file mode 100644 index 0000000000..0ad3f423bc --- /dev/null +++ b/packages/cli/src/configure/prompts.ts @@ -0,0 +1,403 @@ +import { emitKeypressEvents } from 'node:readline' +import { select, search, editor, confirm, input } from '@inquirer/prompts' +import type { ModelCatalogEntry } from '../serve/core/model-catalog.js' +import { parseServeConfig } from '../serve/config.js' +import { + MODALITIES, + RECOMMENDED, + aliasFor, + buildEntry, + buildGenericEntry, + modalityInfo, + type AddedEntry, + type BuiltEntry, + type Modality, + type ServeModelEntry +} from './presets.js' +import { docsUrlForAddon } from './docs-links.js' +import { + configSchemaForAddon, + coerceParam, + paramFields, + validateParam, + type ParamField +} from './param-schemas.js' + +// Sentinel a prompt resolves to when the user backs out (Esc, or a "Back" +// choice). The delimiters can't occur in a model id or a modality value. +const BACK = '::back::' + +// Run an @inquirer prompt with Esc bound to "go back one step": a keypress +// listener aborts the prompt's signal, which rejects with AbortPromptError; +// that is caught and surfaced as BACK. Ctrl+C still throws ExitPromptError and +// is handled one level up (a full abort, not a step back). +function askWithBack( + run: (ctx: { signal: AbortSignal }) => Promise +): Promise { + const controller = new AbortController() + const stdin = process.stdin + const onKeypress = (_str: string | undefined, key: { name?: string } | undefined): void => { + if (key?.name === 'escape') controller.abort() + } + emitKeypressEvents(stdin) + stdin.on('keypress', onKeypress) + return run({ signal: controller.signal }) + .catch((err: unknown): typeof BACK => { + if (err instanceof Error && err.name === 'AbortPromptError') return BACK + throw err + }) + .finally(() => { + stdin.off('keypress', onKeypress) + }) +} + +function fmtSize(bytes: number | null): string { + if (bytes === null) return '' + const mb = bytes / 1_000_000 + return mb >= 1000 ? `${(mb / 1000).toFixed(1)} GB` : `${Math.round(mb)} MB` +} + +function fmtRow(e: ModelCatalogEntry): string { + const meta = [e.params, e.quantization, fmtSize(e.size)].filter(Boolean).join(' - ') + return meta ? `${e.id} ${meta}` : e.id +} + +// Score a term against a model; 0 = no match. All words must appear somewhere in +// id/role/addon/quantization/params so "diffusion" or "q8" finds models by +// capability, not only by name. `engine` is deliberately excluded: its backend +// name (e.g. "llamacpp-completion") contains "llama"/"cpp" and would match every +// model of that backend, drowning out a name search. Matches in the id score +// higher, so typing "llama" surfaces the LLAMA_* models first. +function matchScore(e: ModelCatalogEntry, words: string[]): number { + const id = e.id.toLowerCase() + const haystack = [e.id, e.role, e.addon, e.quantization, e.params] + .filter(Boolean) + .join(' ') + .toLowerCase() + if (!words.every((word) => haystack.includes(word))) return 0 + return 1 + words.filter((word) => id.includes(word)).length +} + +function clamp(n: number, lo: number, hi: number): number { + return Math.max(lo, Math.min(n, hi)) +} + +interface PickOptions { + recommended?: string | undefined + // Builds the serve.models entry a given model would produce, so the picker can + // preview it for the highlighted row (shown only when the terminal is wide). + previewEntry?: ((id: string, addon: string | null) => ServeModelEntry) | undefined +} + +// The @inquirer picker shows the highlighted choice's `description` below the +// list. On a wide terminal, show the concrete serve.models entry the model would +// produce (a real config example); otherwise just the docs link. +function describeChoice( + e: ModelCatalogEntry, + wide: boolean, + previewEntry: PickOptions['previewEntry'] +): string { + const docs = `Docs: ${docsUrlForAddon(e.addon)}` + if (!wide || !previewEntry) return docs + const alias = aliasFor(e.id, new Set()) + const json = JSON.stringify({ [alias]: previewEntry(e.id, e.addon) }, null, 2) + return `${json}\n\n${docs}` +} + +// Returns a chosen constant id, or BACK if the user backs out (Esc / "Back"). +function pickModel( + pool: ModelCatalogEntry[], + message: string, + opts: PickOptions = {} +): Promise { + const { recommended, previewEntry } = opts + const ordered = recommended + ? [...pool].sort((a, b) => (a.id === recommended ? -1 : b.id === recommended ? 1 : 0)) + : pool + const cols = process.stdout.columns ?? 80 + const rows = process.stdout.rows ?? 0 + const wide = cols >= 90 && previewEntry !== undefined + // Fill the terminal with results, leaving headroom for the message, the + // description (taller when it carries a config example), and the shell prompt. + const reserve = wide ? 16 : 4 + const pageSize = rows > 0 ? clamp(rows - reserve, 7, 30) : wide ? 8 : 12 + return askWithBack((ctx) => + search( + { + message: `${message} (Esc to go back)`, + pageSize, + source: (term) => { + const t = term?.toLowerCase().trim() + let list: ModelCatalogEntry[] + if (t) { + const words = t.split(/\s+/) + list = ordered + .map((e) => ({ e, score: matchScore(e, words) })) + .filter((x) => x.score > 0) + .sort((a, b) => b.score - a.score) + .map((x) => x.e) + } else { + list = ordered + } + const choices = list.slice(0, 200).map((e) => ({ + name: e.id === recommended ? `${fmtRow(e)} * recommended` : fmtRow(e), + value: e.id, + description: describeChoice(e, wide, previewEntry) + })) + return [ + ...choices, + { name: '<- Back', value: BACK, description: 'Return to the previous menu' } + ] + } + }, + ctx + ) + ) +} + +function previewText(alias: string, entry: ServeModelEntry, addon: string): string { + const json = JSON.stringify({ [alias]: entry }, null, 2) + return `\n${json}\n\n Docs: ${docsUrlForAddon(addon)}\n` +} + +function validateAlias(value: string, current: string, taken: Set): string | true { + const s = value.trim() + if (!s) return 'Alias cannot be empty' + if (!/^[a-zA-Z0-9._-]+$/.test(s)) return 'Use letters, numbers, dot, dash or underscore' + if (s !== current && taken.has(s)) return `Alias "${s}" is already used` + return true +} + +// Open the entry's JSON in $EDITOR; re-open until it parses and validates. +async function editEntry(alias: string, entry: ServeModelEntry): Promise { + let current = JSON.stringify(entry, null, 2) + for (;;) { + const edited = await editor({ message: `Edit "${alias}"`, default: current, postfix: '.json' }) + try { + const parsed = JSON.parse(edited) as ServeModelEntry + // Structural validation only; the entry shape is a valid serve.models value. + parseServeConfig( + { serve: { models: { [alias]: parsed } } } as Parameters[0], + {} + ) + return parsed + } catch (err) { + const message = err instanceof Error ? err.message : String(err) + const retry = await confirm({ + message: `Invalid entry (${message}). Edit again?`, + default: true + }) + if (!retry) return entry + current = edited + } + } +} + +// Guided, schema-driven editing of a model's config params: each field carries +// its type hint and description from the SDK schema, and input is validated +// against the real field schema. Esc / "Done" returns the updated entry. +async function configureParams( + entry: ServeModelEntry, + fields: ParamField[] +): Promise { + const config: Record = { ...(entry.config ?? {}) } + for (;;) { + const pick = await askWithBack((ctx) => + search( + { + message: 'Set a parameter (Esc when done)', + source: (term) => { + const t = term?.toLowerCase().trim() + const list = t + ? fields.filter( + (f) => f.name.toLowerCase().includes(t) || f.description.toLowerCase().includes(t) + ) + : fields + const rows = list.slice(0, 100).map((f) => ({ + name: + config[f.name] !== undefined + ? `${f.name} = ${JSON.stringify(config[f.name])} [${f.type}]` + : `${f.name} [${f.type}]`, + value: f.name, + description: f.description + })) + return [ + ...rows, + { name: '<- Done', value: BACK, description: 'Finish setting parameters' } + ] + } + }, + ctx + ) + ) + if (pick === BACK) break + const field = fields.find((f) => f.name === pick) + if (!field) continue + const current = config[field.name] + const raw = await askWithBack((ctx) => + input( + { + message: `${field.name} [${field.type}]${field.description ? ` - ${field.description}` : ''}`, + default: current !== undefined ? JSON.stringify(current) : '', + validate: (v) => validateParam(field, v) + }, + ctx + ) + ) + if (raw === BACK) continue + const coerced = coerceParam(raw) + if (coerced === undefined) delete config[field.name] + else config[field.name] = coerced + } + if (Object.keys(config).length === 0) { + const next = { ...entry } + delete next.config + return next + } + return { ...entry, config } +} + +// Preview the entry, then Add / Rename / Set params / Edit / Back. The alias and +// config are editable before and after an $EDITOR pass; after any change the +// preview re-renders with the result, so the user sees the final entry before +// adding. Returns the confirmed addition, or BACK to return to the previous step. +async function confirmEntry( + built: BuiltEntry, + taken: Set, + canEdit: boolean +): Promise { + let alias = aliasFor(built.aliasBase, taken) + let entry = built.entry + const schema = configSchemaForAddon(built.addon) + const fields = schema ? paramFields(schema) : null + for (;;) { + const proceed = await askWithBack((ctx) => + select( + { + message: `${previewText(alias, entry, built.addon)}Proceed? (Esc to go back)`, + choices: [ + { name: `Add it (alias: ${alias})`, value: 'add' }, + { name: 'Rename alias...', value: 'alias' }, + ...(fields ? [{ name: 'Set config parameters...', value: 'params' }] : []), + ...(canEdit ? [{ name: 'Edit in $EDITOR...', value: 'edit' }] : []), + { name: '<- Back', value: 'back' } + ] + }, + ctx + ) + ) + if (proceed === BACK || proceed === 'back') return BACK + if (proceed === 'params' && fields) { + entry = await configureParams(entry, fields) + continue + } + if (proceed === 'alias') { + const next = await askWithBack((ctx) => + input( + { + message: 'Alias', + default: alias, + validate: (v) => validateAlias(v, alias, taken) + }, + ctx + ) + ) + if (next !== BACK) alias = next.trim() + continue + } + if (proceed === 'edit') { + entry = await editEntry(alias, entry) + continue + } + taken.add(alias) + return { alias, addon: built.addon, entry } + } +} + +async function addByCapability( + catalog: ModelCatalogEntry[], + taken: Set, + canEdit: boolean +): Promise { + for (;;) { + const modality = await askWithBack((ctx) => + select( + { + message: 'Capability? (Esc to go back)', + choices: [ + ...MODALITIES.map((m) => ({ name: m.label, value: m.id })), + { name: '<- Back', value: BACK } + ] + }, + ctx + ) + ) + if (modality === BACK) return BACK + + const info = modalityInfo(modality) + let constantName: string | undefined + if (info.pick) { + const pool = catalog.filter((e) => e.role === info.role) + const picked = await pickModel(pool, `Pick a ${info.label} model (type to search)`, { + recommended: RECOMMENDED[modality], + previewEntry: (id) => buildEntry(modality, id).entry + }) + if (picked === BACK) continue + constantName = picked + } + + const res = await confirmEntry(buildEntry(modality, constantName), taken, canEdit) + if (res === BACK) continue + return res + } +} + +async function addBySearch( + catalog: ModelCatalogEntry[], + taken: Set, + canEdit: boolean +): Promise { + for (;;) { + const picked = await pickModel(catalog, 'Search all models (type to search)', { + previewEntry: (id, addon) => buildGenericEntry(id, addon).entry + }) + if (picked === BACK) return BACK + const found = catalog.find((e) => e.id === picked) + const res = await confirmEntry(buildGenericEntry(picked, found?.addon ?? null), taken, canEdit) + if (res === BACK) continue + return res + } +} + +/** Interactive menu loop. Returns the aliased additions the user confirmed. */ +export async function runInteractive( + catalog: ModelCatalogEntry[], + existingAliases: Set, + canEdit: boolean +): Promise { + const taken = new Set(existingAliases) + const added: AddedEntry[] = [] + + for (;;) { + const action = await select({ + message: added.length ? `What next? (${added.length} queued)` : 'What do you want to do?', + choices: [ + { name: 'Add a model by capability', value: 'capability' }, + { name: 'Search all models', value: 'search' }, + { name: added.length ? 'Done - write config' : 'Done', value: 'done' } + ] + }) + if (action === 'done') break + + const res = + action === 'capability' + ? await addByCapability(catalog, taken, canEdit) + : await addBySearch(catalog, taken, canEdit) + if (res === BACK) continue + + added.push(res) + process.stdout.write(` + queued "${res.alias}"${res.addon ? ` (${res.addon})` : ''}\n`) + } + + return added +} diff --git a/packages/cli/src/configure/write-config.ts b/packages/cli/src/configure/write-config.ts new file mode 100644 index 0000000000..d8b6bda6c7 --- /dev/null +++ b/packages/cli/src/configure/write-config.ts @@ -0,0 +1,113 @@ +import { + existsSync, + mkdirSync, + mkdtempSync, + readFileSync, + renameSync, + rmSync, + writeFileSync +} from 'node:fs' +import { dirname, join } from 'node:path' +import { CONFIG_CANDIDATES } from '../config.js' +import type { ServeModelEntry } from './presets.js' + +export interface QvacConfig { + serve?: { models?: Record; [k: string]: unknown } + [k: string]: unknown +} + +/** A non-JSON config (qvac.config.{js,mjs,ts}) in `dir`, if any ā€” we can't safely + * rewrite those, so the command emits guidance instead of clobbering them. */ +export function foreignConfigPath(dir: string): string | null { + for (const candidate of CONFIG_CANDIDATES) { + if (candidate.endsWith('.json')) continue + const full = join(dir, candidate) + if (existsSync(full)) return full + } + return null +} + +/** The set of model constants/sources already configured (an entry's `model` + * or `src`), so re-running configure is idempotent per model. */ +export function existingModelIdentities(config: QvacConfig): Set { + const ids = new Set() + for (const value of Object.values(config.serve?.models ?? {})) { + if (typeof value === 'string') { + ids.add(value) + continue + } + if (value.model) ids.add(value.model) + if (value.src) ids.add(value.src) + } + return ids +} + +/** Map each configured model id (its `model`/`src`) to the alias that holds it ā€” + * first alias wins. Lets `--force` overwrite the existing entry for a model in + * place instead of minting a new deduped alias. */ +export function existingAliasesByModel(config: QvacConfig): Map { + const map = new Map() + for (const [alias, value] of Object.entries(config.serve?.models ?? {})) { + const id = typeof value === 'string' ? value : (value.model ?? value.src) + if (id !== undefined && !map.has(id)) map.set(id, alias) + } + return map +} + +export function loadJsonConfig(path: string): QvacConfig { + if (!existsSync(path)) return {} + const raw = readFileSync(path, 'utf8').trim() + if (raw === '') return {} + return JSON.parse(raw) as QvacConfig +} + +export interface MergeResult { + config: QvacConfig + added: string[] + conflicts: string[] +} + +/** Merge new aliases into `serve.models`, preserving existing keys. Without + * `force`, an alias that already exists is reported as a conflict and skipped. */ +export function mergeServeModels( + existing: QvacConfig, + additions: Record, + force: boolean +): MergeResult { + const config: QvacConfig = { ...existing } + const serve = { ...(config.serve ?? {}) } + const models = { ...((serve.models ?? {}) as Record) } + const added: string[] = [] + const conflicts: string[] = [] + + for (const [alias, entry] of Object.entries(additions)) { + if (alias in models && !force) { + conflicts.push(alias) + continue + } + models[alias] = entry + added.push(alias) + } + + serve.models = models + config.serve = serve + return { config, added, conflicts } +} + +export function serializeConfig(config: QvacConfig): string { + return `${JSON.stringify(config, null, 2)}\n` +} + +/** Write via a temp file + rename so a crash can't leave a half-written config. */ +export function writeConfigAtomically(path: string, config: QvacConfig): void { + const dir = dirname(path) + mkdirSync(dir, { recursive: true }) + const tempDir = mkdtempSync(join(dir, '.qvac-config-')) + const tempPath = join(tempDir, 'qvac.config.json') + try { + writeFileSync(tempPath, serializeConfig(config), 'utf8') + renameSync(tempPath, path) + } finally { + rmSync(tempDir, { recursive: true, force: true }) + } +} diff --git a/packages/cli/src/doctor/checks/project.ts b/packages/cli/src/doctor/checks/project.ts index 38310f2e28..48a7ddd09b 100644 --- a/packages/cli/src/doctor/checks/project.ts +++ b/packages/cli/src/doctor/checks/project.ts @@ -23,6 +23,11 @@ function resolveSdkPackageJson(projectRoot: string): string | null { } } +export function resolveSdkEntrypoint(projectRoot: string): string { + const req = createRequire(path.join(projectRoot, 'package.json')) + return req.resolve(DEFAULT_SDK_NAME) +} + export const checkSdkInstalled: Check = (ctx) => { const projectRoot = ctx.projectRoot const pkgPath = resolveSdkPackageJson(projectRoot) diff --git a/packages/cli/src/doctor/deep-probe-child.ts b/packages/cli/src/doctor/deep-probe-child.ts new file mode 100644 index 0000000000..29f31fd371 --- /dev/null +++ b/packages/cli/src/doctor/deep-probe-child.ts @@ -0,0 +1,155 @@ +import { + DEEP_PROBE_MESSAGE_KIND, + DEEP_PROBE_PROTOCOL_VERSION, + type DeepProbeFailureMessage, + type DeepProbeMessage, + type DeepProbePhase, + type SerializedProbeError +} from './deep-protocol.js' + +const ERROR_MESSAGE_CHARS = 2_048 +const ERROR_STACK_CHARS = 8_192 +const MAX_CAUSE_DEPTH = 5 +const CLEANUP_TIMEOUT_MS = 2_000 + +interface SdkProbeApi { + heartbeat: () => Promise + close: () => Promise +} + +function clip(value: string, limit: number): string { + return value.length <= limit ? value : `${value.slice(0, limit)}\n[truncated]` +} + +function serializeError(error: unknown, depth: number = 0): SerializedProbeError { + if (!(error instanceof Error)) { + return { name: 'NonErrorThrown', message: clip(String(error), ERROR_MESSAGE_CHARS) } + } + + const extended = error as Error & { + code?: unknown + exitCode?: unknown + exitSignal?: unknown + } + const result: SerializedProbeError = { + name: error.name, + message: clip(error.message, ERROR_MESSAGE_CHARS) + } + if (error.stack) result.stack = clip(error.stack, ERROR_STACK_CHARS) + if (typeof extended.code === 'string' || typeof extended.code === 'number') { + result.code = extended.code + } + if (typeof extended.exitCode === 'number' || extended.exitCode === null) { + result.exitCode = extended.exitCode + } + if (typeof extended.exitSignal === 'string' || extended.exitSignal === null) { + result.exitSignal = extended.exitSignal + } + if (error.cause !== undefined && depth < MAX_CAUSE_DEPTH) { + result.cause = serializeError(error.cause, depth + 1) + } + return result +} + +function sendResult(message: DeepProbeMessage): Promise { + return new Promise((resolve) => { + if (typeof process.send !== 'function') { + process.stderr.write('QVAC doctor probe was started without an IPC channel.\n') + resolve() + return + } + process.send(message, () => resolve()) + }) +} + +function flushStream(stream: NodeJS.WriteStream): Promise { + if (!stream.writable) return Promise.resolve() + return new Promise((resolve) => stream.write('', () => resolve())) +} + +async function finish(message: DeepProbeMessage, exitCode: number): Promise { + await sendResult(message) + await Promise.all([flushStream(process.stdout), flushStream(process.stderr)]) + process.exit(exitCode) +} + +async function closeWithTimeout( + close: () => Promise +): Promise { + let timer: NodeJS.Timeout | undefined + try { + await Promise.race([ + close(), + new Promise((_resolve, reject) => { + timer = setTimeout(() => { + const error = new Error(`SDK cleanup timed out after ${CLEANUP_TIMEOUT_MS} ms.`) + error.name = 'CleanupTimeoutError' + reject(error) + }, CLEANUP_TIMEOUT_MS) + }) + ]) + return undefined + } catch (error) { + return serializeError(error) + } finally { + if (timer !== undefined) clearTimeout(timer) + } +} + +async function run(): Promise { + const entrypoint = process.argv[2] + let phase: DeepProbePhase = 'import' + let sdk: SdkProbeApi | undefined + + try { + if (!entrypoint) throw new Error('Missing @qvac/sdk entrypoint argument.') + const imported = (await import(entrypoint)) as Partial + if (typeof imported.heartbeat !== 'function' || typeof imported.close !== 'function') { + throw new Error('The installed @qvac/sdk does not export heartbeat() and close().') + } + sdk = { heartbeat: imported.heartbeat, close: imported.close } + + phase = 'heartbeat' + await sdk.heartbeat() + + phase = 'close' + const closeError = await closeWithTimeout(sdk.close) + if (closeError !== undefined) { + await finish( + { + kind: DEEP_PROBE_MESSAGE_KIND, + version: DEEP_PROBE_PROTOCOL_VERSION, + ok: false, + phase, + error: closeError + }, + 1 + ) + } + + await finish( + { + kind: DEEP_PROBE_MESSAGE_KIND, + version: DEEP_PROBE_PROTOCOL_VERSION, + ok: true, + phase + }, + 0 + ) + } catch (error) { + const failure: DeepProbeFailureMessage = { + kind: DEEP_PROBE_MESSAGE_KIND, + version: DEEP_PROBE_PROTOCOL_VERSION, + ok: false, + phase, + error: serializeError(error) + } + if (phase !== 'close' && sdk !== undefined) { + const cleanupError = await closeWithTimeout(sdk.close) + if (cleanupError !== undefined) failure.cleanupError = cleanupError + } + await finish(failure, 1) + } +} + +await run() diff --git a/packages/cli/src/doctor/deep-protocol.ts b/packages/cli/src/doctor/deep-protocol.ts new file mode 100644 index 0000000000..1a2f9d9ccf --- /dev/null +++ b/packages/cli/src/doctor/deep-protocol.ts @@ -0,0 +1,86 @@ +export const DEEP_PROBE_PROTOCOL_VERSION = 1 +export const DEEP_PROBE_MESSAGE_KIND = 'qvac-doctor-deep-result' + +export type DeepProbePhase = 'import' | 'heartbeat' | 'close' + +export interface SerializedProbeError { + name: string + message: string + stack?: string | undefined + code?: string | number | undefined + exitCode?: number | null | undefined + exitSignal?: string | null | undefined + cause?: SerializedProbeError | undefined +} + +interface DeepProbeMessageBase { + kind: typeof DEEP_PROBE_MESSAGE_KIND + version: typeof DEEP_PROBE_PROTOCOL_VERSION + phase: DeepProbePhase +} + +export interface DeepProbeSuccessMessage extends DeepProbeMessageBase { + ok: true +} + +export interface DeepProbeFailureMessage extends DeepProbeMessageBase { + ok: false + error: SerializedProbeError + cleanupError?: SerializedProbeError | undefined +} + +export type DeepProbeMessage = DeepProbeSuccessMessage | DeepProbeFailureMessage + +function isRecord(value: unknown): value is Record { + return typeof value === 'object' && value !== null +} + +function isPhase(value: unknown): value is DeepProbePhase { + return value === 'import' || value === 'heartbeat' || value === 'close' +} + +function isSerializedError(value: unknown, depth: number = 0): value is SerializedProbeError { + if (!isRecord(value) || depth > 5) return false + if (typeof value['name'] !== 'string' || typeof value['message'] !== 'string') return false + if (value['stack'] !== undefined && typeof value['stack'] !== 'string') return false + if ( + value['code'] !== undefined && + typeof value['code'] !== 'string' && + typeof value['code'] !== 'number' + ) { + return false + } + if ( + value['exitCode'] !== undefined && + value['exitCode'] !== null && + typeof value['exitCode'] !== 'number' + ) { + return false + } + if ( + value['exitSignal'] !== undefined && + value['exitSignal'] !== null && + typeof value['exitSignal'] !== 'string' + ) { + return false + } + return value['cause'] === undefined || isSerializedError(value['cause'], depth + 1) +} + +export function isDeepProbeMessage(value: unknown): value is DeepProbeMessage { + if (!isRecord(value)) return false + if (value['kind'] !== DEEP_PROBE_MESSAGE_KIND) return false + if (value['version'] !== DEEP_PROBE_PROTOCOL_VERSION || !isPhase(value['phase'])) return false + if (value['ok'] === true) { + return value['error'] === undefined && value['cleanupError'] === undefined + } + return ( + value['ok'] === false && + isSerializedError(value['error']) && + (value['cleanupError'] === undefined || isSerializedError(value['cleanupError'])) + ) +} + +export function isDeepProbeProtocolCandidate(value: unknown): boolean { + return isRecord(value) && value['kind'] === DEEP_PROBE_MESSAGE_KIND +} diff --git a/packages/cli/src/doctor/deep.ts b/packages/cli/src/doctor/deep.ts new file mode 100644 index 0000000000..797062b81d --- /dev/null +++ b/packages/cli/src/doctor/deep.ts @@ -0,0 +1,398 @@ +import { fork, spawn } from 'node:child_process' +import fs from 'node:fs' +import { fileURLToPath, pathToFileURL } from 'node:url' +import { resolveSdkEntrypoint } from './checks/project.js' +import { + isDeepProbeMessage, + isDeepProbeProtocolCandidate, + type DeepProbeMessage, + type DeepProbePhase, + type SerializedProbeError +} from './deep-protocol.js' +import type { CheckResult, CheckSection } from './types.js' + +const DEFAULT_TIMEOUT_MS = 45_000 +const MAX_OUTPUT_CHARS = 16_384 +const TERMINATION_GRACE_MS = 2_000 + +export interface SdkRuntimeProbeResult { + outcome: 'pass' | 'fail' | 'timeout' | 'spawn-error' | 'protocol-error' + durationMs: number + stdout: string + stderr: string + exitCode: number | null + signal: NodeJS.Signals | null + phase?: DeepProbePhase | undefined + probeMessage?: DeepProbeMessage | undefined + error?: string | undefined +} + +export interface SdkRuntimeProbeOptions { + timeoutMs?: number | undefined + maxOutputChars?: number | undefined + nodePath?: string | undefined + childModulePath?: string | undefined +} + +export type SdkRuntimeFailureId = + | 'cpu-instruction' + | 'libstdcxx' + | 'visual-cpp-runtime' + | 'vulkan' + | 'shared-library' + | 'bare-runtime' + | 'worker-handshake-timeout' + | 'spawn-error' + | 'protocol-error' + | 'import-failed' + | 'cleanup-failed' + | 'heartbeat-failed' + +export interface SdkRuntimeFailureClassification { + id: SdkRuntimeFailureId + hint: string +} + +interface FailureRule extends SdkRuntimeFailureClassification { + matches: (result: SdkRuntimeProbeResult, diagnostics: string) => boolean +} + +function appendBounded(current: string, chunk: string, limit: number): string { + const next = current + chunk + return next.length <= limit ? next : next.slice(next.length - limit) +} + +function defaultChildModulePath(): string { + const compiledPath = fileURLToPath(new URL('./deep-probe-child.js', import.meta.url)) + if (fs.existsSync(compiledPath)) return compiledPath + return fileURLToPath(new URL('./deep-probe-child.ts', import.meta.url)) +} + +function spawnErrorResult(startedAt: number, error: unknown): SdkRuntimeProbeResult { + return { + outcome: 'spawn-error', + durationMs: Date.now() - startedAt, + stdout: '', + stderr: '', + exitCode: null, + signal: null, + error: error instanceof Error ? error.message : String(error) + } +} + +function signalProbeTree(child: ReturnType, signal: NodeJS.Signals): void { + const pid = child.pid + if (pid === undefined) return + + if (process.platform === 'win32') { + const killChildIfRunning = (): void => { + if (child.exitCode === null && child.signalCode === null) child.kill(signal) + } + const killer = spawn('taskkill', ['/PID', String(pid), '/T', '/F'], { + stdio: 'ignore', + windowsHide: true + }) + killer.once('error', killChildIfRunning) + killer.once('close', (exitCode) => { + if (exitCode !== 0) killChildIfRunning() + }) + return + } + + try { + process.kill(-pid, signal) + } catch (error) { + const code = (error as NodeJS.ErrnoException).code + if (code !== 'ESRCH') child.kill(signal) + } +} + +export function probeSdkRuntime( + entrypoint: string, + projectRoot: string, + options: SdkRuntimeProbeOptions = {} +): Promise { + const timeoutMs = options.timeoutMs ?? DEFAULT_TIMEOUT_MS + const maxOutputChars = options.maxOutputChars ?? MAX_OUTPUT_CHARS + const startedAt = Date.now() + let child: ReturnType + try { + child = fork( + options.childModulePath ?? defaultChildModulePath(), + [pathToFileURL(entrypoint).href], + { + cwd: projectRoot, + detached: process.platform !== 'win32', + env: { ...process.env, FORCE_COLOR: '0', NO_COLOR: '1' }, + execPath: options.nodePath ?? process.execPath, + stdio: ['ignore', 'pipe', 'pipe', 'ipc'] + } + ) + } catch (error) { + return Promise.resolve(spawnErrorResult(startedAt, error)) + } + + let stdout = '' + let stderr = '' + let timedOut = false + let spawnError: string | undefined + let protocolMessage: DeepProbeMessage | undefined + let protocolCandidateCount = 0 + child.stdout?.on('data', (chunk: Buffer) => { + stdout = appendBounded(stdout, chunk.toString('utf8'), maxOutputChars) + }) + child.stderr?.on('data', (chunk: Buffer) => { + stderr = appendBounded(stderr, chunk.toString('utf8'), maxOutputChars) + }) + child.on('message', (message: unknown) => { + if (!isDeepProbeProtocolCandidate(message)) return + protocolCandidateCount += 1 + if (isDeepProbeMessage(message) && protocolMessage === undefined) protocolMessage = message + }) + + return new Promise((resolve) => { + let terminationTimer: NodeJS.Timeout | undefined + const timeoutTimer = setTimeout(() => { + timedOut = true + signalProbeTree(child, 'SIGTERM') + terminationTimer = setTimeout(() => signalProbeTree(child, 'SIGKILL'), TERMINATION_GRACE_MS) + terminationTimer.unref() + }, timeoutMs) + timeoutTimer.unref() + + child.once('error', (error) => { + spawnError = error.message + }) + child.once('close', (exitCode, signal) => { + clearTimeout(timeoutTimer) + if (terminationTimer !== undefined) clearTimeout(terminationTimer) + // A failed Unix probe may have spawned descendants even when it did not + // time out. Its process group remains addressable after the leader exits. + // Windows has no equivalent group handle here, and reusing the exited + // leader's PID with taskkill could target an unrelated process tree. + if (process.platform !== 'win32' && (timedOut || exitCode !== 0 || signal !== null)) { + signalProbeTree(child, 'SIGKILL') + } + + let outcome: SdkRuntimeProbeResult['outcome'] + let error = spawnError + if (spawnError !== undefined) { + outcome = 'spawn-error' + } else if (timedOut) { + outcome = 'timeout' + } else if (protocolCandidateCount !== 1 || protocolMessage === undefined) { + outcome = 'protocol-error' + error = + protocolCandidateCount === 0 + ? 'Probe exited without a result message.' + : 'Probe emitted an invalid or duplicate result message.' + } else if (protocolMessage.ok && exitCode === 0) { + outcome = 'pass' + } else if (!protocolMessage.ok && exitCode !== 0) { + outcome = 'fail' + } else { + outcome = 'protocol-error' + error = 'Probe result message did not agree with its exit code.' + } + + resolve({ + outcome, + durationMs: Date.now() - startedAt, + stdout: stdout.trim(), + stderr: stderr.trim(), + exitCode, + signal, + ...(protocolMessage !== undefined + ? { phase: protocolMessage.phase, probeMessage: protocolMessage } + : {}), + ...(error !== undefined ? { error } : {}) + }) + }) + }) +} + +function formatSerializedError(error: SerializedProbeError, label: string): string { + const attributes = [ + error.code !== undefined ? `code=${String(error.code)}` : '', + error.exitCode !== undefined ? `exitCode=${String(error.exitCode)}` : '', + error.exitSignal !== undefined ? `exitSignal=${String(error.exitSignal)}` : '' + ].filter(Boolean) + const heading = attributes.length > 0 ? `${label} (${attributes.join(', ')}):` : `${label}:` + const current = `${heading}\n${error.stack ?? `${error.name}: ${error.message}`}` + return error.cause === undefined + ? current + : `${current}\n${formatSerializedError(error.cause, 'Caused by')}` +} + +function formatDiagnostics(result: SdkRuntimeProbeResult): string | undefined { + const failure = result.probeMessage?.ok === false ? result.probeMessage : undefined + const parts = [ + result.error ? `Probe error:\n${result.error}` : '', + failure ? formatSerializedError(failure.error, `Failure during ${failure.phase}`) : '', + failure?.cleanupError ? formatSerializedError(failure.cleanupError, 'Cleanup failure') : '', + result.stderr ? `stderr:\n${result.stderr}` : '', + result.stdout ? `stdout:\n${result.stdout}` : '' + ].filter(Boolean) + return parts.length > 0 ? parts.join('\n\n') : undefined +} + +function diagnosticText(result: SdkRuntimeProbeResult): string { + return formatDiagnostics(result) ?? '' +} + +const FAILURE_RULES: readonly FailureRule[] = [ + { + id: 'cpu-instruction', + matches: (result, diagnostics) => + result.signal === 'SIGILL' || + /exitSignal=SIGILL|signal SIGILL|illegal instruction|\bSIGILL\b/i.test(diagnostics), + hint: 'A native addon used an instruction unsupported by this CPU. Use a compatible addon build or a worker bundle that excludes the affected plugin.' + }, + { + id: 'libstdcxx', + matches: (_result, diagnostics) => + /GLIBCXX_[\d.]+.*not found|version [`\'"]GLIBCXX_/i.test(diagnostics), + hint: 'The host libstdc++ may be missing or older than a native addon requires. Install or update libstdc++, or use an addon build compatible with this distribution.' + }, + { + id: 'visual-cpp-runtime', + matches: (_result, diagnostics) => + /VCRUNTIME\d*\.dll|MSVCP\d*\.dll|Visual C\+\+.*Redistributable/i.test(diagnostics), + hint: 'A Microsoft Visual C++ runtime dependency may be missing. Install the current Visual C++ Redistributable and retry.' + }, + { + id: 'vulkan', + matches: (_result, diagnostics) => + /VK_ERROR_|vkCreateInstance|vkEnumerateInstance|(?:lib)?vulkan[^\n]*(failed|error|not found|unsupported|version|cannot open)/i.test( + diagnostics + ), + hint: 'A Vulkan dependency may have failed to load or initialize. Install or update the Vulkan loader and GPU driver to versions providing Vulkan 1.4 or newer.' + }, + { + id: 'shared-library', + matches: (_result, diagnostics) => + /error while loading shared libraries|cannot open shared object file|Library not loaded|The specified module could not be found/i.test( + diagnostics + ), + hint: 'A native shared-library dependency may not have loaded. Re-run with --verbose to identify the missing or incompatible library.' + }, + { + id: 'bare-runtime', + matches: (_result, diagnostics) => + /BARE_RUNTIME_BINARY_NOT_FOUND|BareRuntimeBinaryNotFoundError|Bare runtime binary.*not found/i.test( + diagnostics + ), + hint: 'The Bare runtime binary appears to be missing. Reinstall @qvac/sdk with lifecycle scripts enabled for this host.' + }, + { + id: 'worker-handshake-timeout', + matches: (result, diagnostics) => + result.outcome === 'timeout' || + /RPC_INIT_TIMEOUT|RPCInitTimeoutError|RPC initialization timed out/i.test(diagnostics), + hint: 'The SDK worker did not complete its startup handshake. Re-run with --verbose to inspect the bounded worker output.' + }, + { + id: 'spawn-error', + matches: (result) => result.outcome === 'spawn-error', + hint: 'The isolated Node.js probe could not be started. Re-run with --verbose for the operating-system error.' + }, + { + id: 'protocol-error', + matches: (result) => result.outcome === 'protocol-error', + hint: 'The isolated probe exited without a valid result. Re-run with --verbose to inspect its bounded output.' + }, + { + id: 'import-failed', + matches: (result) => result.phase === 'import', + hint: 'The installed @qvac/sdk could not be imported or initialized. Re-run with --verbose to inspect the error.' + }, + { + id: 'cleanup-failed', + matches: (result) => result.phase === 'close', + hint: 'The SDK worker responded, but its cleanup failed. Re-run with --verbose to inspect the cleanup error.' + } +] + +const DEFAULT_FAILURE_RULE: SdkRuntimeFailureClassification = { + id: 'heartbeat-failed', + hint: 'The SDK worker failed its heartbeat. Re-run with --verbose to inspect the bounded worker output.' +} + +const WINDOWS_WORKER_WARNING = + 'On Windows, a Bare worker process may still be running after a failed deep check; terminate it manually if needed.' + +export function classifySdkRuntimeFailure( + result: SdkRuntimeProbeResult, + platform: NodeJS.Platform = process.platform +): SdkRuntimeFailureClassification { + const diagnostics = diagnosticText(result) + const rule = FAILURE_RULES.find((candidate) => candidate.matches(result, diagnostics)) + const classification = rule ?? DEFAULT_FAILURE_RULE + const warnAboutWindowsWorker = platform === 'win32' && result.outcome !== 'spawn-error' + return { + id: classification.id, + hint: warnAboutWindowsWorker + ? `${classification.hint} ${WINDOWS_WORKER_WARNING}` + : classification.hint + } +} + +function formatFailureValue(result: SdkRuntimeProbeResult): string { + if (result.outcome === 'timeout') return `timed out after ${result.durationMs} ms` + if (result.signal !== null) return `terminated by ${result.signal}` + if (result.outcome === 'spawn-error') return 'probe could not start' + if (result.outcome === 'protocol-error') return 'invalid probe result' + if (result.phase !== undefined) return `${result.phase} failed` + return `exited with code ${result.exitCode ?? 'unknown'}` +} + +export async function checkSdkRuntime(projectRoot: string): Promise { + let entrypoint: string + try { + entrypoint = resolveSdkEntrypoint(projectRoot) + } catch (error) { + const detail = error instanceof Error ? (error.stack ?? error.message) : String(error) + return { + id: 'sdk-runtime', + label: '@qvac/sdk worker heartbeat', + status: 'fail', + severity: 'required', + code: 'sdk-not-found', + value: 'SDK entrypoint not found', + hint: 'Install @qvac/sdk in this project, or repair the installation, before running --deep.', + detail: `SDK resolution error:\n${detail}` + } + } + + const result = await probeSdkRuntime(entrypoint, projectRoot) + if (result.outcome === 'pass') { + return { + id: 'sdk-runtime', + label: '@qvac/sdk worker heartbeat', + status: 'pass', + severity: 'required', + value: `${result.durationMs} ms` + } + } + + const detail = formatDiagnostics(result) + const classification = classifySdkRuntimeFailure(result) + return { + id: 'sdk-runtime', + label: '@qvac/sdk worker heartbeat', + status: 'fail', + severity: 'required', + code: classification.id, + value: formatFailureValue(result), + hint: classification.hint, + ...(detail !== undefined ? { detail } : {}) + } +} + +export async function collectDeepCheckSection(projectRoot: string): Promise { + return { + id: 'deep', + title: 'SDK runtime (deep)', + checks: [await checkSdkRuntime(projectRoot)] + } +} diff --git a/packages/cli/src/doctor/format.ts b/packages/cli/src/doctor/format.ts index 74e9902609..00f6e62a17 100644 --- a/packages/cli/src/doctor/format.ts +++ b/packages/cli/src/doctor/format.ts @@ -14,7 +14,10 @@ function formatCheckLine(check: CheckResult): string { return ` ${icon} ${check.label}${value}` } -export function formatReport(report: DoctorReport): string { +export function formatReport( + report: DoctorReport, + options: { verbose?: boolean | undefined } = {} +): string { const lines: string[] = [] lines.push('šŸ©ŗ QVAC doctor') lines.push('') @@ -28,6 +31,9 @@ export function formatReport(report: DoctorReport): string { if (check.status !== 'pass' && check.hint) { lines.push(` ${check.hint}`) } + if (options.verbose && check.detail) { + for (const line of check.detail.split('\n')) lines.push(` ${line}`) + } } lines.push('') } diff --git a/packages/cli/src/doctor/index.ts b/packages/cli/src/doctor/index.ts index 2a7a9846e7..8ae32f55d4 100644 --- a/packages/cli/src/doctor/index.ts +++ b/packages/cli/src/doctor/index.ts @@ -1,4 +1,5 @@ import { collectCheckSections, isReportOk } from './checks/index.js' +import { collectDeepCheckSection } from './deep.js' import { formatJsonReport, formatReport } from './format.js' import type { DoctorReport, RunDoctorOptions } from './types.js' @@ -6,6 +7,7 @@ import type { DoctorReport, RunDoctorOptions } from './types.js' export async function runDoctor(options: RunDoctorOptions = {}): Promise { const projectRoot = options.projectRoot ?? process.cwd() const sections = collectCheckSections({ projectRoot }) + if (options.deep) sections.push(await collectDeepCheckSection(projectRoot)) const report: DoctorReport = { ok: isReportOk(sections), @@ -18,7 +20,7 @@ export async function runDoctor(options: RunDoctorOptions = {}): Promise { - try { - const { runDoctor } = await import('./doctor/index.js') - const report = await runDoctor({ - projectRoot: process.cwd(), - json: options.json, - quiet: options.quiet, - verbose: options.verbose - }) - if (!report.ok) process.exit(1) - } catch (error: unknown) { - handleError(error) - process.exit(1) + .action( + async (options: { deep?: boolean; json?: boolean; quiet?: boolean; verbose?: boolean }) => { + try { + const { runDoctor } = await import('./doctor/index.js') + const report = await runDoctor({ + projectRoot: process.cwd(), + deep: options.deep, + json: options.json, + quiet: options.quiet, + verbose: options.verbose + }) + if (!report.ok) process.exit(1) + } catch (error: unknown) { + handleError(error) + process.exit(1) + } } - }) + ) + + program + .command('configure') + .description('Interactively build a qvac.config.json (serve.models) for local models') + .option('-c, --config ', 'Config file to write (default: ./qvac.config.json)') + .option('-y, --yes', 'Non-interactive: write a sensible default starter (chat + transcription)') + .option( + '--modality ', + 'Non-interactive: add a modality (repeatable) ā€” chat|embedding|transcription|speech|image', + collect, + [] + ) + .option('--force', 'Re-add a model that is already configured, overwriting its existing entry') + .option('-q, --quiet', 'Suppress output') + .action( + async (options: { + config?: string + yes?: boolean + modality: string[] + force?: boolean + quiet?: boolean + }) => { + try { + const { runConfigure } = await import('./configure/index.js') + await runConfigure({ + projectRoot: process.cwd(), + config: options.config, + yes: options.yes, + modality: options.modality.length > 0 ? options.modality : undefined, + force: options.force, + quiet: options.quiet + }) + } catch (error: unknown) { + handleError(error) + process.exit(1) + } + } + ) const verifyCmd = program .command('verify') diff --git a/packages/cli/src/serve/core/model-catalog.ts b/packages/cli/src/serve/core/model-catalog.ts index ec4d9a7bf7..82e9f32a26 100644 --- a/packages/cli/src/serve/core/model-catalog.ts +++ b/packages/cli/src/serve/core/model-catalog.ts @@ -32,7 +32,7 @@ export interface CatalogQuery { } const CATALOG_HINT = - 'Not in serve.models ā€” add it there to make it usable (a `qvac configure` command is planned).' + 'Not in serve.models ā€” run `qvac configure` (or add it there by hand) to make it usable.' export function roleForAddon(addon: string): string { return normalizeEndpointCategory(addon) @@ -78,6 +78,19 @@ export function buildCatalog( }) } + entries.push(...buildBuiltinCatalog(constants)) + + // Configured first, then builtin; each group by id for a stable, paginable order. + entries.sort((a, b) => + a.source === b.source ? a.id.localeCompare(b.id) : a.source === 'config' ? -1 : 1 + ) + return entries +} + +// The catalog rows for every baked-in SDK constant, independent of any server +// context ā€” usable by CLI commands (e.g. `configure`) that have no ServeConfig. +export function buildBuiltinCatalog(constants: Map): ModelCatalogEntry[] { + const entries: ModelCatalogEntry[] = [] for (const [name, model] of constants) { entries.push({ object: 'model_catalog_entry', @@ -95,11 +108,6 @@ export function buildCatalog( hint: CATALOG_HINT }) } - - // Configured first, then builtin; each group by id for a stable, paginable order. - entries.sort((a, b) => - a.source === b.source ? a.id.localeCompare(b.id) : a.source === 'config' ? -1 : 1 - ) return entries } diff --git a/packages/cli/system-requirements.md b/packages/cli/system-requirements.md index 76bbd26880..db815cf713 100644 --- a/packages/cli/system-requirements.md +++ b/packages/cli/system-requirements.md @@ -7,8 +7,9 @@ validate your environment against this list with: qvac doctor ``` -Use `--json` for machine-readable output and `--quiet` to only set the exit -code (`0` when all required checks pass, `1` otherwise). +Use `--deep` to start the project SDK worker and verify a heartbeat plus clean +shutdown. Use `--json` for machine-readable output and `--quiet` to only set +the exit code (`0` when all required checks pass, `1` otherwise). ## Scope: CLI host vs. SDK deploy targets @@ -31,6 +32,10 @@ and iOS. `qvac doctor` reports both, in two distinct sections of its output: | Supported CLI host | `darwin-arm64`, `darwin-x64`, `linux-arm64`, `linux-x64`, `win32-x64`. The `qvac` CLI cannot run on mobile; those are deploy targets only. | | Total RAM `>= 2 GB` (recommended `>= 4 GB`) | Below 4 GB, most LLMs will fail to load. | +When `--deep` is requested, the project SDK must resolve and its isolated +worker probe must import the SDK, complete `heartbeat()`, and complete +`close()`. Failure in any phase is a required-check failure. + ## Recommended | Requirement | When it is needed | @@ -69,13 +74,32 @@ they are missing but does not fail. | ----------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `@qvac/sdk` resolvable from project | Resolved with `require.resolve('@qvac/sdk/package.json')` rooted at the working directory, so hoisted installs (monorepos, Yarn/Bun workspaces) are correctly detected. | +## Deep SDK runtime probe + +`qvac doctor --deep` runs the installed project SDK in an isolated Node.js +child process. The child reports its import, heartbeat, and cleanup result over +a private structured channel, while stdout and stderr are captured separately +and bounded. A valid success result and exit code `0` are both required. + +The outer probe timeout is 45 seconds, and SDK cleanup is bounded separately to +two seconds. On Unix, a timeout requests graceful termination of the probe +process tree and then forces termination after a two-second grace period. On +Windows, the CLI requests termination of the live tree with `taskkill /T /F` +when the probe times out. After a failed Windows probe exits, the CLI cannot +reliably address descendants through the exited parent's PID, so the report +warns that a Bare worker may still require manual termination. The report +prioritizes concrete CPU, native-library, Visual C++ runtime, Vulkan, and Bare +errors over generic lifecycle failures. These classifications are diagnostic +guidance; native loader messages that lack structured SDK error codes are +necessarily matched heuristically. + ## Exit codes - `0` ā€” all required checks passed. Warnings, skips, and informational rows may still be present. - `1` ā€” one or more required checks failed (unsupported Node version, - unsupported CLI host, insufficient total RAM, ā€¦). See the printed hints - for remediation steps. + unsupported CLI host, insufficient total RAM, or a requested deep probe + could not run or complete). See the printed hints for remediation steps. ## JSON schema @@ -86,13 +110,14 @@ interface DoctorReport { arch: string // e.g. "arm64" nodeVersion: string // e.g. "20.19.5" sections: Array<{ - id: 'runtime' | 'hardware' | 'targets' | 'tools' | 'project' + id: 'runtime' | 'hardware' | 'targets' | 'tools' | 'project' | 'deep' title: string checks: Array<{ id: string label: string status: 'pass' | 'warn' | 'fail' | 'skip' | 'info' severity: 'required' | 'recommended' | 'informational' + code?: string // stable machine-readable failure classification value?: string detail?: string hint?: string // typically present for any non-pass result diff --git a/packages/cli/test/configure.test.ts b/packages/cli/test/configure.test.ts new file mode 100644 index 0000000000..f373cf0126 --- /dev/null +++ b/packages/cli/test/configure.test.ts @@ -0,0 +1,147 @@ +import { describe, it } from 'node:test' +import assert from 'node:assert/strict' +import { mkdtempSync, writeFileSync, rmSync } from 'node:fs' +import { tmpdir } from 'node:os' +import { join } from 'node:path' +import { + DEFAULT_STARTER, + TTS_VOICE_PLACEHOLDER, + aliasFor, + buildAdditions, + buildEntry, + buildGenericEntry +} from '../src/configure/presets.js' +import { CONFIG_DOCS_URL, docsUrlForAddon } from '../src/configure/docs-links.js' +import { + foreignConfigPath, + loadJsonConfig, + mergeServeModels, + serializeConfig, + writeConfigAtomically +} from '../src/configure/write-config.js' + +describe('configure: presets / buildEntry', () => { + it('builds a bare {model, preload:false} for chat with the recommended default', () => { + const b = buildEntry('chat') + assert.equal(b.addon, 'llm') + assert.equal(b.aliasBase, 'QWEN3_600M_INST_Q4') + assert.deepEqual(b.entry, { model: 'QWEN3_600M_INST_Q4', preload: false }) + }) + + it('honors an explicit model choice', () => { + assert.equal(buildEntry('chat', 'LLAMA3_2_1B_INST_Q4').entry.model, 'LLAMA3_2_1B_INST_Q4') + }) + + it('adds prediction config for image', () => { + const b = buildEntry('image') + assert.equal(b.entry.model, 'SD_V2_1_1B_Q8_0') + assert.deepEqual(b.entry.config, { prediction: 'v' }) + }) + + it('emits a TTS template with a placeholder voice', () => { + const b = buildEntry('speech') + assert.equal(b.addon, 'tts') + assert.equal(b.entry.type, 'tts') + assert.ok(b.entry.src) + const cfg = b.entry.config as Record + assert.equal(cfg['referenceAudioSrc'], TTS_VOICE_PLACEHOLDER) + assert.ok(cfg['s3genModelSrc']) + }) + + it('buildGenericEntry is a bare model entry', () => { + assert.deepEqual(buildGenericEntry('FOO', 'llm').entry, { model: 'FOO', preload: false }) + }) +}) + +describe('configure: aliasFor', () => { + it('kebab-cases a constant name', () => { + assert.equal(aliasFor('QWEN3_600M_INST_Q4', new Set()), 'qwen3-600m-inst-q4') + }) + it('dedupes on collision', () => { + const taken = new Set(['qwen3-600m-inst-q4']) + assert.equal(aliasFor('QWEN3_600M_INST_Q4', taken), 'qwen3-600m-inst-q4-2') + }) +}) + +describe('configure: buildAdditions (non-interactive)', () => { + it('builds the default starter (chat + transcription)', () => { + const added = buildAdditions( + DEFAULT_STARTER.map((modality) => ({ modality })), + new Set() + ) + assert.deepEqual( + added.map((a) => a.alias), + ['qwen3-600m-inst-q4', 'whisper-tiny-q8-0'] + ) + assert.equal(added[0]!.entry.preload, false) + }) +}) + +describe('configure: docs-links', () => { + it('deep-links known addons to real docs pages', () => { + assert.equal( + docsUrlForAddon('llm'), + 'https://docs.qvac.tether.io/addons/llm-llamacpp/#4-create-the-config-obj' + ) + assert.equal(docsUrlForAddon('tts'), 'https://docs.qvac.tether.io/addons/tts-ggml/') + assert.equal( + docsUrlForAddon('diffusion'), + 'https://docs.qvac.tether.io/addons/diffusion-cpp/#3-configure-the-native-backend-argsconfig' + ) + }) + it('falls back to the configuration page for pageless / unknown addons', () => { + assert.equal(docsUrlForAddon('ocr'), CONFIG_DOCS_URL) + assert.equal(docsUrlForAddon(null), CONFIG_DOCS_URL) + }) +}) + +describe('configure: mergeServeModels', () => { + it('adds new aliases and preserves existing config', () => { + const existing = { serve: { models: { keep: { model: 'X' } }, publicBaseUrl: 'https://x' } } + const { config, added, conflicts } = mergeServeModels( + existing, + { chat: { model: 'Y', preload: false } }, + false + ) + assert.deepEqual(added, ['chat']) + assert.deepEqual(conflicts, []) + assert.deepEqual(config.serve!.models, { + keep: { model: 'X' }, + chat: { model: 'Y', preload: false } + }) + assert.equal(config.serve!['publicBaseUrl'], 'https://x') + }) + it('skips a conflicting alias without force, overwrites with force', () => { + const existing = { serve: { models: { chat: { model: 'OLD' } } } } + const skip = mergeServeModels(existing, { chat: { model: 'NEW' } }, false) + assert.deepEqual(skip.conflicts, ['chat']) + assert.deepEqual(skip.added, []) + assert.equal(skip.config.serve!.models!['chat']!.model, 'OLD') + const over = mergeServeModels(existing, { chat: { model: 'NEW' } }, true) + assert.deepEqual(over.added, ['chat']) + assert.equal(over.config.serve!.models!['chat']!.model, 'NEW') + }) +}) + +describe('configure: write-config fs', () => { + it('serializes with 2-space indent + trailing newline', () => { + const s = serializeConfig({ serve: { models: {} } }) + assert.ok(s.endsWith('}\n')) + assert.ok(s.includes('\n "serve"')) + }) + + it('writes atomically and round-trips through loadJsonConfig; detects foreign config', () => { + const dir = mkdtempSync(join(tmpdir(), 'qvac-cfg-')) + try { + const path = join(dir, 'qvac.config.json') + writeConfigAtomically(path, { serve: { models: { chat: { model: 'X', preload: false } } } }) + const back = loadJsonConfig(path) + assert.equal(back.serve!.models!['chat']!.model, 'X') + assert.equal(foreignConfigPath(dir), null) + writeFileSync(join(dir, 'qvac.config.ts'), 'export default {}\n') + assert.equal(foreignConfigPath(dir), join(dir, 'qvac.config.ts')) + } finally { + rmSync(dir, { recursive: true, force: true }) + } + }) +}) diff --git a/packages/cli/test/doctor.test.ts b/packages/cli/test/doctor.test.ts index da1be24f3f..26775e30bd 100644 --- a/packages/cli/test/doctor.test.ts +++ b/packages/cli/test/doctor.test.ts @@ -22,6 +22,18 @@ import { isReportOk } from '../src/doctor/checks/index.js' import type { CheckContext } from '../src/doctor/checks/index.js' +import { runDoctor } from '../src/doctor/index.js' +import { + checkSdkRuntime, + classifySdkRuntimeFailure, + probeSdkRuntime, + type SdkRuntimeProbeResult +} from '../src/doctor/deep.js' +import { + DEEP_PROBE_MESSAGE_KIND, + DEEP_PROBE_PROTOCOL_VERSION, + isDeepProbeMessage +} from '../src/doctor/deep-protocol.js' // Build a CheckContext with a minimal, deterministic baseline and spread // per-test overrides on top. Keeps each test assertion about a single @@ -39,6 +51,54 @@ function makeCtx(overrides: Partial = {}): CheckContext { } } +function createSdkFixture(source: string): { entrypoint: string; projectRoot: string } { + const projectRoot = fs.mkdtempSync(path.join(os.tmpdir(), 'qvac-deep-check-')) + const sdkDir = path.join(projectRoot, 'node_modules', '@qvac', 'sdk') + fs.mkdirSync(sdkDir, { recursive: true }) + fs.writeFileSync( + path.join(sdkDir, 'package.json'), + JSON.stringify({ + name: '@qvac/sdk', + version: '0.0.0-test', + type: 'module', + exports: { '.': './index.js', './package': './package.json' } + }) + ) + const entrypoint = path.join(sdkDir, 'index.js') + fs.writeFileSync(entrypoint, source) + return { entrypoint, projectRoot } +} + +function failedProbe(overrides: Partial = {}): SdkRuntimeProbeResult { + return { + outcome: 'fail', + durationMs: 10, + stdout: '', + stderr: '', + exitCode: 1, + signal: null, + ...overrides + } +} + +function isProcessAlive(pid: number): boolean { + try { + process.kill(pid, 0) + return true + } catch (error) { + return (error as NodeJS.ErrnoException).code !== 'ESRCH' + } +} + +async function waitForProcessExit(pid: number, timeoutMs: number): Promise { + const deadline = Date.now() + timeoutMs + while (Date.now() < deadline) { + if (!isProcessAlive(pid)) return true + await new Promise((resolve) => setTimeout(resolve, 25)) + } + return !isProcessAlive(pid) +} + describe('checkNodeVersion', () => { it('fails on Node < 18', () => { const r = checkNodeVersion(makeCtx({ nodeVersion: '16.20.0' })) @@ -350,6 +410,419 @@ describe('checkSdkInstalled', () => { }) }) +describe('deep SDK runtime probe', () => { + it('passes after an isolated heartbeat and clean close', async () => { + const fixture = createSdkFixture(` + export async function heartbeat() {} + export async function close() {} + `) + try { + const result = await probeSdkRuntime(fixture.entrypoint, fixture.projectRoot, { + timeoutMs: 2_000 + }) + assert.equal(result.outcome, 'pass') + assert.equal(result.exitCode, 0) + assert.equal(result.phase, 'close') + assert.equal(result.probeMessage?.ok, true) + } finally { + fs.rmSync(fixture.projectRoot, { recursive: true, force: true }) + } + }) + + it('captures and classifies a native library failure', async () => { + const fixture = createSdkFixture(` + export async function heartbeat() { + throw new Error("version 'GLIBCXX_3.4.30' not found") + } + export async function close() {} + `) + try { + const result = await probeSdkRuntime(fixture.entrypoint, fixture.projectRoot, { + timeoutMs: 2_000 + }) + assert.equal(result.outcome, 'fail') + assert.equal(result.phase, 'heartbeat') + assert.equal(result.probeMessage?.ok, false) + if (result.probeMessage?.ok === false) { + assert.match(result.probeMessage.error.message, /GLIBCXX_3\.4\.30/) + } + const classification = classifySdkRuntimeFailure(result) + assert.equal(classification.id, 'libstdcxx') + assert.match(classification.hint, /may be missing or older/i) + + const check = await checkSdkRuntime(fixture.projectRoot) + assert.equal(check.code, 'libstdcxx') + assert.match(check.hint ?? '', /may be missing or older/i) + } finally { + fs.rmSync(fixture.projectRoot, { recursive: true, force: true }) + } + }) + + it('reports a secondary cleanup failure without replacing the heartbeat error', async () => { + const fixture = createSdkFixture(` + export async function heartbeat() { throw new Error('heartbeat failed') } + export async function close() { throw new Error('cleanup failed') } + `) + try { + const result = await probeSdkRuntime(fixture.entrypoint, fixture.projectRoot, { + timeoutMs: 2_000 + }) + assert.equal(result.outcome, 'fail') + assert.equal(result.phase, 'heartbeat') + assert.equal(result.probeMessage?.ok, false) + if (result.probeMessage?.ok === false) { + assert.match(result.probeMessage.error.message, /heartbeat failed/) + assert.match(result.probeMessage.cleanupError?.message ?? '', /cleanup failed/) + } + } finally { + fs.rmSync(fixture.projectRoot, { recursive: true, force: true }) + } + }) + + it('terminates a hung heartbeat at the configured timeout', async () => { + const fixture = createSdkFixture(` + export async function heartbeat() { + await new Promise(() => setInterval(() => {}, 1_000)) + } + export async function close() {} + `) + try { + const result = await probeSdkRuntime(fixture.entrypoint, fixture.projectRoot, { + timeoutMs: 50 + }) + assert.equal(result.outcome, 'timeout') + const classification = classifySdkRuntimeFailure(result) + assert.equal(classification.id, 'worker-handshake-timeout') + assert.match(classification.hint, /startup handshake/i) + } finally { + fs.rmSync(fixture.projectRoot, { recursive: true, force: true }) + } + }) + + it('bounds a hung close with the cleanup timeout', async () => { + const fixture = createSdkFixture(` + export async function heartbeat() {} + export async function close() { + await new Promise(() => setInterval(() => {}, 1_000)) + } + `) + try { + const result = await probeSdkRuntime(fixture.entrypoint, fixture.projectRoot, { + timeoutMs: 5_000 + }) + assert.equal(result.outcome, 'fail') + assert.equal(result.phase, 'close') + assert.equal(result.probeMessage?.ok, false) + if (result.probeMessage?.ok === false) { + assert.equal(result.probeMessage.error.name, 'CleanupTimeoutError') + assert.match(result.probeMessage.error.message, /2_?000|2000/) + } + assert.ok(result.durationMs < 4_500, `close took ${result.durationMs} ms`) + } finally { + fs.rmSync(fixture.projectRoot, { recursive: true, force: true }) + } + }) + + it('terminates descendants when a timed-out probe is forced down', async () => { + const fixture = createSdkFixture(` + import { spawn } from 'node:child_process' + import { writeFileSync } from 'node:fs' + import { join } from 'node:path' + + export async function heartbeat() { + const descendant = spawn( + process.execPath, + ['-e', "process.on('SIGTERM', () => {}); setInterval(() => {}, 1_000)"], + { stdio: 'ignore' } + ) + writeFileSync(join(process.cwd(), 'descendant.pid'), String(descendant.pid)) + await new Promise(() => setInterval(() => {}, 1_000)) + } + export async function close() {} + `) + let descendantPid: number | undefined + try { + const result = await probeSdkRuntime(fixture.entrypoint, fixture.projectRoot, { + timeoutMs: 1_000 + }) + assert.equal(result.outcome, 'timeout') + descendantPid = Number( + fs.readFileSync(path.join(fixture.projectRoot, 'descendant.pid'), 'utf8') + ) + assert.ok(Number.isSafeInteger(descendantPid) && descendantPid > 0) + assert.equal( + await waitForProcessExit(descendantPid, 2_000), + true, + `descendant ${descendantPid} survived probe termination` + ) + } finally { + if (descendantPid !== undefined && isProcessAlive(descendantPid)) { + process.kill(descendantPid, 'SIGKILL') + } + fs.rmSync(fixture.projectRoot, { recursive: true, force: true }) + } + }) + + it('bounds captured output to its tail', async () => { + const fixture = createSdkFixture(` + export async function heartbeat() { + process.stderr.write('x'.repeat(1_000)) + throw new Error('tail marker') + } + export async function close() {} + `) + try { + const result = await probeSdkRuntime(fixture.entrypoint, fixture.projectRoot, { + timeoutMs: 2_000, + maxOutputChars: 512 + }) + assert.ok(result.stderr.length <= 512) + assert.equal(result.probeMessage?.ok, false) + if (result.probeMessage?.ok === false) { + assert.match(result.probeMessage.error.message, /tail marker/) + } + } finally { + fs.rmSync(fixture.projectRoot, { recursive: true, force: true }) + } + }) + + it('classifies common signal, Bare, Windows runtime, and Vulkan failures', () => { + assert.match( + classifySdkRuntimeFailure(failedProbe({ signal: 'SIGILL' })).hint, + /unsupported by this CPU/i + ) + assert.match( + classifySdkRuntimeFailure(failedProbe({ stderr: 'BareRuntimeBinaryNotFoundError' })).hint, + /Bare runtime binary appears to be missing/i + ) + assert.match( + classifySdkRuntimeFailure(failedProbe({ stderr: 'VCRUNTIME140.dll was not found' })).hint, + /Visual C\+\+ runtime dependency/i + ) + assert.match( + classifySdkRuntimeFailure( + failedProbe({ stderr: 'libnative.so: cannot open shared object file' }) + ).hint, + /shared-library dependency/i + ) + assert.match( + classifySdkRuntimeFailure(failedProbe({ stderr: 'vkCreateInstance failed' })).hint, + /Vulkan dependency/i + ) + assert.match( + classifySdkRuntimeFailure( + failedProbe({ stderr: 'libvulkan.so.1: cannot open shared object file' }) + ).hint, + /Vulkan dependency/i + ) + }) + + it('returns stable failure ids in explicit priority order', () => { + const cases: Array<[SdkRuntimeProbeResult, string]> = [ + [failedProbe({ signal: 'SIGILL' }), 'cpu-instruction'], + [failedProbe({ stderr: "version 'GLIBCXX_3.4.30' not found" }), 'libstdcxx'], + [failedProbe({ stderr: 'VCRUNTIME140.dll was not found' }), 'visual-cpp-runtime'], + [failedProbe({ stderr: 'vkCreateInstance failed' }), 'vulkan'], + [failedProbe({ stderr: 'libnative.so: cannot open shared object file' }), 'shared-library'], + [failedProbe({ stderr: 'BareRuntimeBinaryNotFoundError' }), 'bare-runtime'], + [failedProbe({ outcome: 'timeout' }), 'worker-handshake-timeout'], + [failedProbe({ outcome: 'spawn-error' }), 'spawn-error'], + [failedProbe({ outcome: 'protocol-error' }), 'protocol-error'], + [failedProbe({ phase: 'import' }), 'import-failed'], + [failedProbe({ phase: 'close' }), 'cleanup-failed'], + [failedProbe({ phase: 'heartbeat' }), 'heartbeat-failed'] + ] + + for (const [result, expectedId] of cases) { + assert.equal(classifySdkRuntimeFailure(result).id, expectedId) + } + }) + + it('classifies import failures separately from heartbeat failures', () => { + const classification = classifySdkRuntimeFailure(failedProbe({ phase: 'import' })) + assert.equal(classification.id, 'import-failed') + assert.match(classification.hint, /could not be imported or initialized/i) + }) + + it('reports a real SDK import-surface failure with the import classification', async () => { + const fixture = createSdkFixture(` + export async function heartbeat() {} + `) + try { + const result = await probeSdkRuntime(fixture.entrypoint, fixture.projectRoot, { + timeoutMs: 2_000 + }) + assert.equal(result.outcome, 'fail') + assert.equal(result.phase, 'import') + const classification = classifySdkRuntimeFailure(result) + assert.equal(classification.id, 'import-failed') + assert.match(classification.hint, /could not be imported or initialized/i) + } finally { + fs.rmSync(fixture.projectRoot, { recursive: true, force: true }) + } + }) + + it('warns Windows users that a failed deep check may leave a Bare worker', () => { + const windows = classifySdkRuntimeFailure(failedProbe(), 'win32') + assert.match(windows.hint, /Bare worker process may still be running/i) + assert.match(windows.hint, /terminate it manually/i) + + const linux = classifySdkRuntimeFailure(failedProbe(), 'linux') + assert.doesNotMatch(linux.hint, /may still be running/i) + }) + + it('rejects a protocol failure with a malformed cleanup error', () => { + assert.equal( + isDeepProbeMessage({ + kind: DEEP_PROBE_MESSAGE_KIND, + version: DEEP_PROBE_PROTOCOL_VERSION, + ok: false, + phase: 'heartbeat', + error: { name: 'Error', message: 'heartbeat failed' }, + cleanupError: { name: 'Error', message: 42 } + }), + false + ) + assert.equal( + isDeepProbeMessage({ + kind: DEEP_PROBE_MESSAGE_KIND, + version: DEEP_PROBE_PROTOCOL_VERSION, + ok: true, + phase: 'close', + cleanupError: { name: 'Error', message: 'unexpected' } + }), + false + ) + }) + + it('prioritizes a concrete SIGILL cause over an RPC timeout wrapper', () => { + assert.match( + classifySdkRuntimeFailure( + failedProbe({ stderr: 'RPCInitTimeoutError: RPC initialization timed out\nsignal SIGILL' }) + ).hint, + /unsupported by this CPU/i + ) + assert.equal( + classifySdkRuntimeFailure( + failedProbe({ stderr: 'RPCInitTimeoutError: RPC initialization timed out\nsignal SIGILL' }) + ).id, + 'cpu-instruction' + ) + }) + + it('classifies a child-process spawn error', async () => { + const fixture = createSdkFixture('export async function heartbeat() {}') + try { + const result = await probeSdkRuntime(fixture.entrypoint, fixture.projectRoot, { + nodePath: path.join(fixture.projectRoot, 'missing-node'), + timeoutMs: 2_000 + }) + assert.equal(result.outcome, 'spawn-error') + assert.match(classifySdkRuntimeFailure(result).hint, /could not be started/i) + } finally { + fs.rmSync(fixture.projectRoot, { recursive: true, force: true }) + } + }) + + it('adds the deep section to the doctor report', async () => { + const fixture = createSdkFixture(` + export async function heartbeat() {} + export async function close() {} + `) + try { + const report = await runDoctor({ projectRoot: fixture.projectRoot, deep: true, quiet: true }) + const section = report.sections.at(-1) + assert.equal(section?.id, 'deep') + assert.equal(section?.checks[0]?.status, 'pass') + } finally { + fs.rmSync(fixture.projectRoot, { recursive: true, force: true }) + } + }) + + it('fails when --deep cannot resolve an SDK entrypoint', async () => { + const projectRoot = fs.mkdtempSync(path.join(os.tmpdir(), 'qvac-deep-missing-')) + try { + const result = await checkSdkRuntime(projectRoot) + assert.equal(result.status, 'fail') + assert.equal(result.severity, 'required') + assert.equal(result.code, 'sdk-not-found') + assert.match(result.value ?? '', /not found/i) + } finally { + fs.rmSync(projectRoot, { recursive: true, force: true }) + } + }) + + it('rejects exit code zero without a valid success message', async () => { + const fixture = createSdkFixture('process.exit(0)') + try { + const result = await probeSdkRuntime(fixture.entrypoint, fixture.projectRoot, { + timeoutMs: 2_000 + }) + assert.equal(result.outcome, 'protocol-error') + assert.match(classifySdkRuntimeFailure(result).hint, /without a valid result/i) + } finally { + fs.rmSync(fixture.projectRoot, { recursive: true, force: true }) + } + }) + + it('rejects a success message when the probe exits unsuccessfully', async () => { + const fixture = createSdkFixture(` + export async function heartbeat() { + process.send?.({ kind: '${DEEP_PROBE_MESSAGE_KIND}', version: ${DEEP_PROBE_PROTOCOL_VERSION}, ok: true, phase: 'heartbeat' }) + process.exit(1) + } + export async function close() {} + `) + try { + const result = await probeSdkRuntime(fixture.entrypoint, fixture.projectRoot, { + timeoutMs: 2_000 + }) + assert.equal(result.outcome, 'protocol-error') + assert.match(result.error ?? '', /did not agree/i) + } finally { + fs.rmSync(fixture.projectRoot, { recursive: true, force: true }) + } + }) + + it('rejects duplicate protocol result messages', async () => { + const fixture = createSdkFixture(` + export async function heartbeat() { + const result = { kind: '${DEEP_PROBE_MESSAGE_KIND}', version: ${DEEP_PROBE_PROTOCOL_VERSION}, ok: true, phase: 'heartbeat' } + process.send?.(result) + process.send?.(result) + process.exit(0) + } + export async function close() {} + `) + try { + const result = await probeSdkRuntime(fixture.entrypoint, fixture.projectRoot, { + timeoutMs: 2_000 + }) + assert.equal(result.outcome, 'protocol-error') + assert.match(result.error ?? '', /duplicate/i) + } finally { + fs.rmSync(fixture.projectRoot, { recursive: true, force: true }) + } + }) + + it('reports close failures separately from heartbeat failures', async () => { + const fixture = createSdkFixture(` + export async function heartbeat() {} + export async function close() { throw new Error('close failed') } + `) + try { + const result = await probeSdkRuntime(fixture.entrypoint, fixture.projectRoot, { + timeoutMs: 2_000 + }) + assert.equal(result.outcome, 'fail') + assert.equal(result.phase, 'close') + assert.match(classifySdkRuntimeFailure(result).hint, /cleanup failed/i) + } finally { + fs.rmSync(fixture.projectRoot, { recursive: true, force: true }) + } + }) +}) + describe('collectCheckSections + isReportOk', () => { it('returns the expected section order and ids', () => { const sections = collectCheckSections({ projectRoot: process.cwd() }) diff --git a/packages/cli/test/e2e/cli/commands.test.ts b/packages/cli/test/e2e/cli/commands.test.ts index f3378a0978..f69f131e95 100644 --- a/packages/cli/test/e2e/cli/commands.test.ts +++ b/packages/cli/test/e2e/cli/commands.test.ts @@ -11,6 +11,21 @@ async function tmpProject(t: TestContext): Promise { return tempDir(t, 'qvac-cli-cmd-') } +async function installFakeSdk(projectRoot: string, source: string): Promise { + const sdkDir = join(projectRoot, 'node_modules', '@qvac', 'sdk') + await mkdir(sdkDir, { recursive: true }) + await writeFile( + join(sdkDir, 'package.json'), + JSON.stringify({ + name: '@qvac/sdk', + version: '0.0.0-test', + type: 'module', + exports: { '.': './index.js', './package': './package.json' } + }) + ) + await writeFile(join(sdkDir, 'index.js'), source) +} + describe('cli: version & help', () => { it('--version prints semver', async () => { const r = await runCli(['--version']) @@ -165,7 +180,11 @@ describe('cli: doctor', () => { it('--help shows options', async () => { const r = await runCli(['doctor', '--help']) assert.equal(r.code, 0) - assert.ok(r.output.includes('--json') && r.output.includes('QVAC SDK system requirements')) + assert.ok( + r.output.includes('--deep') && + r.output.includes('--json') && + r.output.includes('QVAC SDK system requirements') + ) }) it('--json emits valid JSON with ok boolean', async () => { @@ -175,6 +194,65 @@ describe('cli: doctor', () => { assert.equal(typeof doc.ok, 'boolean') assert.ok(Array.isArray(doc.sections) && doc.sections.length >= 1) }) + + it('--deep fails when the project SDK is missing', async (t) => { + const dir = await tmpProject(t) + const r = await runCli(['doctor', '--deep', '--json'], { cwd: dir }) + assert.equal(r.code, 1) + const doc = JSON.parse(r.stdout) as { + ok: boolean + sections: Array<{ id: string; checks: Array<{ status: string }> }> + } + assert.equal(doc.ok, false) + assert.equal(doc.sections.find((section) => section.id === 'deep')?.checks[0]?.status, 'fail') + }) + + it('--deep accepts a structured heartbeat result without corrupting JSON', async (t) => { + const dir = await tmpProject(t) + await installFakeSdk( + dir, + ` + export async function heartbeat() { console.log('fixture worker log') } + export async function close() {} + ` + ) + const r = await runCli(['doctor', '--deep', '--json'], { cwd: dir }) + assert.equal(r.code, 0) + const doc = JSON.parse(r.stdout) as { + ok: boolean + sections: Array<{ id: string; checks: Array<{ status: string }> }> + } + assert.equal(doc.ok, true) + assert.equal(doc.sections.find((section) => section.id === 'deep')?.checks[0]?.status, 'pass') + }) + + it('--deep --quiet returns failure without output', async (t) => { + const dir = await tmpProject(t) + await installFakeSdk( + dir, + ` + export async function heartbeat() { throw new Error('fixture heartbeat failure') } + export async function close() {} + ` + ) + const r = await runCli(['doctor', '--deep', '--quiet'], { cwd: dir }) + assert.equal(r.code, 1) + assert.equal(r.stdout, '') + }) + + it('--deep --verbose includes bounded failure diagnostics', async (t) => { + const dir = await tmpProject(t) + await installFakeSdk( + dir, + ` + export async function heartbeat() { throw new Error('fixture heartbeat failure') } + export async function close() {} + ` + ) + const r = await runCli(['doctor', '--deep', '--verbose'], { cwd: dir }) + assert.equal(r.code, 1) + assert.match(r.stdout, /fixture heartbeat failure/) + }) }) describe('cli: config errors', () => { diff --git a/packages/cli/test/e2e/cli/configure.test.ts b/packages/cli/test/e2e/cli/configure.test.ts new file mode 100644 index 0000000000..213c2e2103 --- /dev/null +++ b/packages/cli/test/e2e/cli/configure.test.ts @@ -0,0 +1,94 @@ +import { describe, it } from 'node:test' +import assert from 'node:assert/strict' +import { mkdtemp, rm, readFile, writeFile } from 'node:fs/promises' +import { existsSync } from 'node:fs' +import { tmpdir } from 'node:os' +import { join } from 'node:path' +import { runCli } from '../helpers/cli.js' +import { parseServeConfig } from '../../../src/serve/config.js' + +function tmp(): Promise { + return mkdtemp(join(tmpdir(), 'qvac-configure-')) +} + +async function readConfig(dir: string): Promise<{ serve: { models: Record } }> { + const raw = await readFile(join(dir, 'qvac.config.json'), 'utf8') + return JSON.parse(raw) as { serve: { models: Record } } +} + +describe('cli: configure', () => { + it('--yes writes a valid default starter (chat + transcription)', async () => { + const dir = await tmp() + try { + const res = await runCli(['configure', '--yes'], { cwd: dir }) + assert.equal(res.code, 0, res.output) + const cfg = await readConfig(dir) + assert.deepEqual(Object.keys(cfg.serve.models).sort(), [ + 'qwen3-600m-inst-q4', + 'whisper-tiny-q8-0' + ]) + // Structural validity: it must parse through the real serve config parser. + parseServeConfig(cfg as Parameters[0], {}) + assert.match(res.output, /docs\.qvac\.tether\.io/) + } finally { + await rm(dir, { recursive: true, force: true }) + } + }) + + it('--modality selects specific modalities (image gets prediction config)', async () => { + const dir = await tmp() + try { + const res = await runCli(['configure', '--modality', 'chat', '--modality', 'image'], { + cwd: dir + }) + assert.equal(res.code, 0, res.output) + const cfg = await readConfig(dir) + const keys = Object.keys(cfg.serve.models).sort() + assert.deepEqual(keys, ['qwen3-600m-inst-q4', 'sd-v2-1-1b-q8-0']) + const image = cfg.serve.models['sd-v2-1-1b-q8-0'] as { config?: { prediction?: string } } + assert.equal(image.config?.prediction, 'v') + parseServeConfig(cfg as Parameters[0], {}) + } finally { + await rm(dir, { recursive: true, force: true }) + } + }) + + it('preserves existing entries and skips conflicts unless --force', async () => { + const dir = await tmp() + try { + await runCli(['configure', '--modality', 'chat'], { cwd: dir }) + const second = await runCli(['configure', '--modality', 'chat'], { cwd: dir }) + assert.match(second.output, /already configured/i) + const cfg = await readConfig(dir) + assert.deepEqual(Object.keys(cfg.serve.models), ['qwen3-600m-inst-q4']) + const forced = await runCli(['configure', '--modality', 'chat', '--force'], { cwd: dir }) + assert.equal(forced.code, 0, forced.output) + // --force overwrites the existing alias in place; it must not mint a + // deduped `qwen3-600m-inst-q4-2`. + const afterForce = await readConfig(dir) + assert.deepEqual(Object.keys(afterForce.serve.models), ['qwen3-600m-inst-q4']) + assert.match(forced.output, /updated/i) + } finally { + await rm(dir, { recursive: true, force: true }) + } + }) + + it('refuses to shadow a non-JSON config and points at the docs', async () => { + const dir = await tmp() + try { + await writeFile(join(dir, 'qvac.config.ts'), 'export default {}\n') + const res = await runCli(['configure', '--yes'], { cwd: dir }) + assert.equal(res.code, 0, res.output) + assert.match(res.output, /qvac\.config\.ts/) + assert.equal(existsSync(join(dir, 'qvac.config.json')), false) + } finally { + await rm(dir, { recursive: true, force: true }) + } + }) + + it('rejects an unknown modality', async () => { + const res = await runCli(['configure', '--modality', 'bogus'], { timeoutMs: 5000 }) + assert.equal(res.code, 1) + assert.match(res.output, /Unknown modality/i) + }) +}) diff --git a/packages/cli/test/param-schemas.test.ts b/packages/cli/test/param-schemas.test.ts new file mode 100644 index 0000000000..0a5a5536f6 --- /dev/null +++ b/packages/cli/test/param-schemas.test.ts @@ -0,0 +1,78 @@ +import { describe, it } from 'node:test' +import assert from 'node:assert/strict' +import { + configSchemaForAddon, + paramFields, + coerceParam, + validateParam +} from '../src/configure/param-schemas.js' + +describe('configure: param-schemas', () => { + it('resolves schemas only for addons the SDK exposes', () => { + assert.ok(configSchemaForAddon('llm')) + assert.ok(configSchemaForAddon('embeddings')) + assert.equal(configSchemaForAddon('tts'), undefined) + assert.equal(configSchemaForAddon('diffusion'), undefined) + assert.equal(configSchemaForAddon(null), undefined) + assert.equal(configSchemaForAddon(undefined), undefined) + }) + + it('enumerates llamacpp fields with type hints and descriptions', () => { + const schema = configSchemaForAddon('llm') + assert.ok(schema) + const fields = paramFields(schema) + assert.ok(fields.length > 10) + + const ctx = fields.find((f) => f.name === 'ctx_size') + assert.ok(ctx) + assert.equal(ctx.type, 'number') + assert.match(ctx.description, /context window/i) + + const temp = fields.find((f) => f.name === 'temp') + assert.ok(temp) + assert.match(temp.type, /<= 2/) + + // every field carries a schema for validation + for (const f of fields) assert.ok(f.schema) + }) + + it('coerces raw input to the right JSON type, blank clears', () => { + assert.equal(coerceParam('1024'), 1024) + assert.equal(coerceParam('true'), true) + assert.equal(coerceParam(''), undefined) + assert.equal(coerceParam(' '), undefined) + assert.equal(coerceParam('gpu'), 'gpu') + assert.deepEqual(coerceParam('["stop"]'), ['stop']) + }) + + it('renders enum values bare and accepts bare/single/double-quoted input', () => { + const schema = configSchemaForAddon('embeddings') + assert.ok(schema) + const attention = paramFields(schema).find((f) => f.name === 'attention') + assert.ok(attention) + // hint shows bare values, matching how they're typed (no surrounding quotes) + assert.equal(attention.type, 'causal | non-causal') + // all three forms the user might type (incl. the single-quoted form the + // description renders) coerce and validate the same + assert.equal(coerceParam("'causal'"), 'causal') + assert.equal(validateParam(attention, 'causal'), true) + assert.equal(validateParam(attention, "'causal'"), true) + assert.equal(validateParam(attention, '"causal"'), true) + assert.equal(validateParam(attention, "'non-causal'"), true) + }) + + it('validates input against the real field schema', () => { + const schema = configSchemaForAddon('llm') + assert.ok(schema) + const fields = paramFields(schema) + const temp = fields.find((f) => f.name === 'temp') + const ctx = fields.find((f) => f.name === 'ctx_size') + assert.ok(temp && ctx) + + assert.equal(validateParam(temp, '0.8'), true) + assert.equal(validateParam(temp, ''), true) // blank = clear + assert.notEqual(validateParam(temp, '3'), true) // above max 2 + assert.equal(validateParam(ctx, '2048'), true) + assert.notEqual(validateParam(ctx, 'abc'), true) + }) +})