Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion docs/repository-layout.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,8 @@ Legend:
| Package | Description | Category |
| :--- | :--- | :--- |
| sdk | Main entry point to develop AI applications with QVAC | SDK |
| bare-sdk | Bare-targeted slim assembly of the SDK; consumers install only the addons they need and register plugins explicitly | SDK |
| inference | Bare-only in-process engine. Use this instead of `@qvac/bare-sdk` | SDK |
| bare-sdk | Deprecated (last release 0.18.2). Use `@qvac/inference` for in-process Bare | SDK |
| ai-sdk-provider | Vercel AI SDK provider exposing the QVAC runtime (chat, embeddings, transcription, translation, speech, OCR, image) | SDK |
| bci-whispercpp | Brain-Computer Interface (BCI) neural-signal transcription addon powered by whisper.cpp | Addon |
| classification-ggml | Image classification addon (MobileNetV3-Small) on the GGML backend | Addon |
Expand Down
10 changes: 5 additions & 5 deletions docs/website/content/docs/about/how-it-works.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ schemaType: TechArticle

## Overview

The SDK supports multiple JS runtimes, but its [underlying components](https://github.com/tetherto/qvac/tree/main/packages) run only on [Bare](https://bare.pears.com). When the SDK runs in a runtime other than Bare, it spawns a Bare worker where all AI operations will take place. The worker is started lazily on the first RPC call and can be explicitly shut down with `close()`.
The SDK supports multiple JS runtimes, but its [underlying components](https://github.com/tetherto/qvac/tree/main/packages) run only on [Bare](https://bare.pears.com). The SDK spawns a Bare worker where all AI operations take place, talking to it over `bare-rpc`. The worker is started lazily on the first RPC call and can be explicitly shut down with `close()`. For in-process Bare with no worker, use [`@qvac/inference`](https://github.com/tetherto/qvac/tree/main/packages/inference).

## Phase 1: initialization

Expand All @@ -19,13 +19,13 @@ sequenceDiagram

App->>SDK: Call loadModel() (or any API)
SDK->>RPC: getRPC() (create runtime-specific RPC client)
RPC->>Worker: Spawn worker (Node/Expo) or connect in-process (Bare)
RPC->>Worker: Spawn worker
RPC->>Worker: Send __init_config (first call only)
Worker->>Worker: Store config in memory
Worker->>RPC: Init ack
```

The first time you call `loadModel()` (or any function other than `close()`), the SDK performs a complete initialization sequence. It initializes a runtime-specific RPC client and sends configuration to the worker via the internal `__init_config` message. The worker process is spawned once and reused for subsequent calls until you explicitly close it. In Bare runtime, no separate worker process is spawned; requests are handled in-process.
The first time you call `loadModel()` (or any function other than `close()`), the SDK performs a complete initialization sequence. It initializes a runtime-specific RPC client and sends configuration to the worker via the internal `__init_config` message. The worker process is spawned once and reused for subsequent calls until you explicitly close it.

## Phase 2: model loading

Expand Down Expand Up @@ -81,12 +81,12 @@ sequenceDiagram

App->>SDK: Call close()
SDK->>RPC: Close RPC client
RPC->>Worker: Terminate worker process (Node/Expo)
RPC->>Worker: Terminate worker process
Worker->>Worker: Cleanup (unload models, stop swarms, close RAG)
Worker->>RPC: Exit
```

`close()` explicitly shuts down the worker and releases the RPC connection. In Node/Expo, this terminates the worker process; in Bare, the call is a no-op since there is no separate worker process. After `close()`, the next SDK call will reinitialize the RPC client and spawn a fresh worker.
`close()` explicitly shuts down the worker and releases the RPC connection. This terminates the worker process. After `close()`, the next SDK call will reinitialize the RPC client and spawn a fresh worker.

<Callout title="Tip" type="success">
`unloadModel()` will automatically close the RPC connection when there are no active models or providers, but `close()` is the explicit way to shut down the SDK instance.
Expand Down
12 changes: 6 additions & 6 deletions docs/website/content/docs/configuration/plugins/index.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -175,19 +175,19 @@ The Python SDK exposes plugin invocation through the same worker contract:

The `plugins` array in `qvac.config.*` is **bundle-time** configuration — it controls which addons get packed into your worker. That is separate from **runtime registration**, which determines the plugins live in the worker process when an SDK call runs.

On Node.js and Expo the SDK spawns a worker that auto-registers the full built-in set, so you never register manually. **Bare runs in-process with no spawned worker**, so nothing auto-registers — register the plugins you use before the first SDK call:
On Node.js and Expo the SDK spawns a worker that auto-registers the full built-in set, so you never register manually. In-process Bare uses `@qvac/inference` with no spawned worker — register the plugins you use before the first call:

```js
import { plugins } from "@qvac/bare-sdk";
import { llmPlugin } from "@qvac/bare-sdk/llamacpp-completion/plugin";
import { plugins } from "@qvac/inference";
import { llmPlugin } from "@qvac/inference/llamacpp-completion/plugin";

const sdk = plugins([llmPlugin]); // or registerPlugin(llmPlugin) from "@qvac/bare-sdk/plugins"
const sdk = plugins([llmPlugin]); // or registerPlugin(llmPlugin) from "@qvac/inference/plugins"
```

Calls made before any plugin is registered raise `WorkerPluginsNotRegisteredError`.
Calls made before any plugin is registered raise `PluginsNotRegisteredError`.

<Callout type="info">
For direct Bare usage we recommend [`@qvac/bare-sdk`](https://github.com/tetherto/qvac/tree/main/packages/bare-sdk) — the slim distribution built for explicit assembly. `@qvac/sdk` also runs on Bare; import the same plugins from `@qvac/sdk/<capability>/plugin` instead.
In-process Bare uses [`@qvac/inference`](https://github.com/tetherto/qvac/tree/main/packages/inference). `@qvac/bare-sdk` is deprecated; last release is 0.18.2.
</Callout>

## Notes
Expand Down
39 changes: 7 additions & 32 deletions docs/website/content/docs/js-ts-sdk.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ import { TrackCopy } from '@/components/track-copy'
## Overview

The JS/TS SDK (`@qvac/sdk` on npm) is the JavaScript/TypeScript client for QVAC.
It runs the worker in-process (there is no separate transport), and drives every
It drives a Bare worker over `bare-rpc`, and exposes every
QVAC capability through a unified, type-safe API.

## Requirements
Expand Down Expand Up @@ -137,7 +137,7 @@ Run the quickstart script:
QVAC_CONFIG_PATH=./qvac.config.json node quickstart.js
```

Or with the [Bare](https://bare.pears.com) runtime. Running on Bare needs a little setup — a `process` global and plugin registration — so see [Running on Bare](#running-on-bare) below.
Or on [Bare](https://bare.pears.com) with `@qvac/inference` — see [Running on Bare](#running-on-bare).
</Step>

</Steps>
Expand All @@ -146,7 +146,7 @@ Or with the [Bare](https://bare.pears.com) runtime. Running on Bare needs a litt

Follow these instructions to run any example in this documentation:
- All examples are self-contained, runnable JavaScript scripts. Use the `qvac-examples` workspace created in this quickstart to store and run them as you explore this documentation.
- Run each example with the indicated compatible JavaScript environment. QVAC supports multiple environments (Node.js, Bare, and Expo). The examples are written in Node style and run on Node.js or Bun directly; to run them on Bare, see [Running on Bare](#running-on-bare).
- Run each example with the indicated compatible JavaScript environment. QVAC supports Node.js, Expo, and in-process Bare via `@qvac/inference`. The examples are written in Node style and run on Node.js or Bun directly; to run them on Bare, see [Running on Bare](#running-on-bare).
- More examples can be found in the [SDK examples directory](https://github.com/tetherto/qvac/tree/main/packages/sdk/examples).
- Some examples need companion files — sample audio, an image, or a config — that aren't part of the embedded code. These can be found in the same examples directory.
- Some examples also provide a TypeScript version. If you want to run TS directly, install the required dev dependencies:
Expand Down Expand Up @@ -266,44 +266,19 @@ export default config;

To run an example on Bare:

1. Provide a `process` global. Install `bare-process` and set it before using the SDK:
1. Provide a `process` global. Install `bare-process` and set it before using `@qvac/inference`:
```js
import process from "bare-process";
globalThis.process = process;
```
2. Register the plugins the example uses — Bare runs in-process and nothing auto-registers. See [Runtime registration on Bare](/configuration/plugins#runtime-registration-on-bare).

Here is the quickstart adapted for Bare:
Here is the SDK quickstart adapted for Bare with `@qvac/inference`:

```ts title="quickstart.bare.ts" lineNumbers
// The Bare quickstart. Bare has no `process` global and does not spawn a worker,
// so two setup steps come first: install bare-process as the `process` global,
// then register the plugins this example uses via `plugins([...])`.

import bareProcess from "bare-process";
import { plugins, LLAMA_3_2_1B_INST_Q4_0 } from "@qvac/sdk";
import { llmPlugin } from "@qvac/sdk/llamacpp-completion/plugin";

(globalThis as unknown as { process: typeof bareProcess }).process =
bareProcess;

const { loadModel, completion, unloadModel } = plugins([llmPlugin]);

// From here it is the same as the Node quickstart.
const modelId = await loadModel({ modelSrc: LLAMA_3_2_1B_INST_Q4_0 });

const history = [
{ role: "user", content: "Explain quantum computing in one sentence" },
];
const result = completion({ modelId, history, stream: true });
for await (const token of result.tokenStream) {
process.stdout.write(token);
}

await unloadModel({ modelId, autoClose: true });
```ts file=<rootDir>/packages/inference/examples/quickstart-adapted.ts title="quickstart-adapted.ts" lineNumbers
```

For running QVAC on Bare in production, we recommend **[@qvac/bare-sdk](https://github.com/tetherto/qvac/tree/main/packages/bare-sdk)** — a slim, Bare-only assembly where you select addons and register plugins explicitly. It also sets up the bare module shims that examples using `fs` and similar modules rely on.
For running QVAC on Bare in production, use **[@qvac/inference](https://github.com/tetherto/qvac/tree/main/packages/inference)** — the Bare-only in-process engine. Select addons and register plugins explicitly. `@qvac/bare-sdk` is deprecated; last release is 0.18.2.

## API reference

Expand Down
121 changes: 13 additions & 108 deletions packages/bare-sdk/README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# @qvac/bare-sdk

Bare-targeted slim distribution of the QVAC SDK. Designed for consumers assembling their own worker entry on the [Bare runtime](https://bare.pears.com) (Pear apps, bare-expo mobile apps, direct Bare scripts).
> **Deprecated.** `@qvac/bare-sdk` is discontinued. Use [`@qvac/inference`](../inference) for in-process Bare. Last release is **0.18.2**. There will not be a 0.19.0 of this package.

> *Part of **QVAC** ecosystem*
>
Expand All @@ -9,121 +9,26 @@ Bare-targeted slim distribution of the QVAC SDK. Designed for consumers assembli
> [Support](https://discord.com/channels/1425125849346216029/1445400675189264516)  • 
> [Discord](https://discord.com/invite/tetherdev)

## Why this exists
## Migrate to `@qvac/inference`

The default `@qvac/sdk` package ships with the full set of built-in plugin addons so Node and Expo consumers can call any capability out of the box. `@qvac/bare-sdk` exposes the same SDK surface with no built-in addon dependencies, so consumers install only the addons their worker actually registers. Paired with `bare-pack`'s static bundle analysis, the resulting native binary scales with the plugins explicitly assembled in the worker entry.

## Install

```bash
npm install @qvac/bare-sdk @qvac/translation-nmtcpp
```

Replace `@qvac/translation-nmtcpp` with whichever addon packages match the plugins you want to register.

## Worker entry example (NMT-only)

```js
import { plugins } from "@qvac/bare-sdk";
import { nmtPlugin } from "@qvac/bare-sdk/nmtcpp-translation/plugin";

const sdk = plugins([nmtPlugin]);

const result = await sdk.translate({
modelId: "my-model",
text: "Hello world",
sourceLang: "en",
targetLang: "fr",
});
```

## Capability to addon package


| Plugin subpath | Addon package |
| ------------------------------------------------ | -------------------------------- |
| `@qvac/bare-sdk/llamacpp-completion/plugin` | `@qvac/llm-llamacpp` |
| `@qvac/bare-sdk/llamacpp-embedding/plugin` | `@qvac/embed-llamacpp` |
| `@qvac/bare-sdk/whispercpp-transcription/plugin` | `@qvac/asr-ggml` |
| `@qvac/bare-sdk/parakeet-transcription/plugin` | `@qvac/asr-ggml` |
| `@qvac/bare-sdk/nmtcpp-translation/plugin` | `@qvac/translation-nmtcpp` |
| `@qvac/bare-sdk/tts-ggml/plugin` | `@qvac/tts-ggml` |
| `@qvac/bare-sdk/ggml-ocr/plugin` | `@qvac/ocr-ggml` |
| `@qvac/bare-sdk/sdcpp-generation/plugin` | `@qvac/diffusion-cpp` |
| `@qvac/bare-sdk/audiogen-ggml/plugin` | `@qvac/audiogen-ggml` |
| `@qvac/bare-sdk/ggml-vla/plugin` | `@qvac/vla-ggml` |


## Connection lifecycle

`unloadModel` does not close the SDK's connections. The swarm, registry client, and corestore stay up so long-lived workers survive a routine unload across load/unload cycles. Close explicitly when you're done:

```js
import { close } from "@qvac/bare-sdk";

await unloadModel({ modelId });
await close(); // tear down swarm + registry client so the process can exit
```

Or opt into auto-close on the final unload:

```js
await unloadModel({ modelId, autoClose: true });
```

## Relationship to `@qvac/inference` and `@qvac/sdk`

`@qvac/bare-sdk` is a thin re-export of [`@qvac/inference`](../inference), the Bare-only in-process engine. It forwards the same surface (`.`) and per-plugin subpaths, adding only the `@qvac/bare-sdk` package identity and its slim dependency profile — `@qvac/inference` is the sole dependency and does all the work. There is no code of its own beyond the re-exports.

Use `@qvac/sdk` for Node and Expo apps that want the full default worker over RPC. Use `@qvac/bare-sdk` (or `@qvac/inference` directly) when you assemble your own plugins in-process on Bare.

## Release history

`@qvac/bare-sdk` releases in lockstep with `@qvac/inference`. For release notes and version history, see the [`@qvac/inference` changelog](../inference/CHANGELOG.md).

## Migrating from `@qvac/sdk`

Existing Bare consumers running a custom worker entry can switch packages without changing call sites. Two edits:

**1. Swap the dependency** in the package that owns your worker:
`@qvac/inference` is the Bare-only in-process engine. Plugin subpaths match 1:1 except `./commands` and `./worker-core`, which have no inference equivalent (`npx qvac bundle` lives on `@qvac/sdk`).

```diff
-"@qvac/sdk": "^0.11.0",
+"@qvac/bare-sdk": "^0.11.0",
-import { plugins } from "@qvac/bare-sdk";
-import { nmtPlugin } from "@qvac/bare-sdk/nmtcpp-translation/plugin";
+import { plugins } from "@qvac/inference";
+import { nmtPlugin } from "@qvac/inference/nmtcpp-translation/plugin";
```

**2. Rewrite worker imports** — every `@qvac/sdk/...` subpath maps to the same path under `@qvac/bare-sdk`:

```diff
-import { registerPlugin } from "@qvac/sdk/plugins";
-import { nmtPlugin } from "@qvac/sdk/nmtcpp-translation/plugin";
+import { registerPlugin } from "@qvac/bare-sdk/plugins";
+import { nmtPlugin } from "@qvac/bare-sdk/nmtcpp-translation/plugin";
-"@qvac/bare-sdk": "^0.18.2",
+"@qvac/inference": "^0.18.2",
```

Two subpaths have no `@qvac/bare-sdk` equivalent. `@qvac/sdk/commands` (the bundler) has no counterpart — bundle from `@qvac/sdk` with `npx qvac bundle`. `@qvac/sdk/worker-core` has no counterpart — bare-sdk has no worker, so register plugins explicitly as shown above. Importing `@qvac/bare-sdk/commands` or `@qvac/bare-sdk/worker-core` throws with this guidance.
See the [`@qvac/inference` README](../inference/README.md) for install, plugin assembly, and the capability-to-addon table.

If your worker previously relied on the default plugin set (i.e. it never called `registerPlugin`), enumerate the plugins it uses via `plugins([...])` or `registerPlugin(...)` — see [Worker entry example](#worker-entry-example-nmt-only). bare-sdk has no implicit defaults.
Use `@qvac/sdk` for Node, Electron, and Expo apps that want the default worker over RPC.

## Behavior differences vs `@qvac/sdk`

### Explicit plugin assembly

Consumers register plugins via `plugins([...])` or `registerPlugin(...)`. SDK calls made before any plugin is registered raise `WorkerPluginsNotRegisteredError` with guidance to the assembly API.

### Pear pre-hook

`@qvac/sdk` ships a `pear-pre` script that auto-generates `qvac/worker.pear.entry.mjs` from `qvac.config.{json,mjs}`. `@qvac/bare-sdk` follows the explicit-assembly model, so Pear apps using bare-sdk author the entry file directly.

**Fix:** create `qvac/worker.pear.entry.mjs` in your app root:

```js
import { registerPlugin } from "@qvac/bare-sdk/plugins";
import { nmtPlugin } from "@qvac/bare-sdk/nmtcpp-translation/plugin";

registerPlugin(nmtPlugin);

await import("../worker.js");
```
## Release history

Then add `"/qvac/worker.pear.entry.mjs"` to `pear.stage.entrypoints` in your `package.json`. A bare-sdk-aware pre-hook is on the roadmap.
Historical notes live in the [`@qvac/sdk` changelog](../sdk/CHANGELOG.md). New Bare engine releases are documented in the [`@qvac/inference` changelog](../inference/CHANGELOG.md).
2 changes: 1 addition & 1 deletion packages/inference/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ The Bare-only engine of the QVAC SDK. It runs inference directly on the [Bare ru

## Why this exists

`@qvac/inference` is the pure-Bare layer of the SDK, written in TypeScript: the client API, the request engine, and the plugin system, all running in one Bare process. `@qvac/sdk` builds on top of it to reach Node, Electron, Expo, and Pear by launching this engine as a worker; on Bare you use it directly.
`@qvac/inference` is the pure-Bare layer of the SDK, written in TypeScript: the client API, the request engine, and the plugin system, all running in one Bare process. `@qvac/sdk` builds on top of it to reach Node, Electron, Expo, and Pear by launching this engine as a worker; on Bare you use it directly. It replaces the deprecated `@qvac/bare-sdk` package (last release 0.18.2).

`@qvac/inference` ships no plugins by default and no addon dependencies. You install only the addon packages your app registers, so the resulting binary scales with the engines you actually assemble.

Expand Down
4 changes: 2 additions & 2 deletions packages/sdk/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,13 +10,13 @@
> <a href="https://discord.com/channels/1425125849346216029/1445400675189264516" >Support</a> &nbsp;•&nbsp;
> <a href="https://discord.com/invite/tetherdev" >Discord</a>

**QVAC SDK** is the main entry point for developing applications with QVAC. It is type-safe and exposes all QVAC capabilities through a unified interface. It runs on Node.js, [Bare runtime](https://bare.pears.com), and [Expo](https://expo.dev).
**QVAC SDK** is the main entry point for developing applications with QVAC. It is type-safe and exposes all QVAC capabilities through a unified interface. It runs on Node.js and [Expo](https://expo.dev).

See [https://docs.qvac.tether.io/sdk/getting-started](https://docs.qvac.tether.io/sdk/getting-started) for the comprehensive QVAC documentation.

For AI/LLM tools, use [https://docs.qvac.tether.io/llms-full.txt](https://docs.qvac.tether.io/llms-full.txt) as the consolidated plaintext documentation export.

> **Running on Bare directly?** `@qvac/sdk` runs on Bare, but you must register the plugins you use explicitly before the first SDK call (Node and Expo do this automatically). For direct Bare usage we recommend [`@qvac/bare-sdk`](../bare-sdk/README.md) — the same SDK surface with no built-in plugin addons, designed for consumers wiring their own worker entry (Pear apps, bare-expo apps, direct Bare scripts).
> **In-process Bare:** use [`@qvac/inference`](../inference/README.md). `@qvac/bare-sdk` is deprecated; last release is 0.18.2.

## Supported environments and installation

Expand Down
Loading
Loading