Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 5 additions & 3 deletions .env.example
Original file line number Diff line number Diff line change
Expand Up @@ -39,9 +39,11 @@ WORKER_HOST=127.0.0.1
# AGENT_TOKEN=
# OpenBot's Intelligence runtime is not this raw AG-UI URL. See its adapter docs.

# Optional CopilotKit Rich Threads. Server-only project key; enables managed
# conversation persistence, native thread switching and event replay.
# Leave unset for the open-source local conversation store.
# CopilotKit Intelligence is required when WORKSPACE_MODE=live. Create or select
# a project with the commands below, then keep the generated key server-only.
# npx copilotkit@latest login
# npx copilotkit@latest project select
# Sample mode can leave this unset and use the local conversation store.
CPK_INTELLIGENCE_API_KEY=

# Optional private Docker Linux computer (separate from the browser).
Expand Down
17 changes: 9 additions & 8 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,13 +55,13 @@ The computer combines **persistent Chromium and an optional Linux workspace**. T
| **Finance** | Import transaction CSV to create a spending summary with categories, transactions, and a savings-goal action. |
| **Gmail & Calendar** | Google OAuth adapters, complete mail threads, drafts/attachments, calendar discovery, and reviewed event creation/update/deletion. Live credentials required. |
| **Personal context** | Editable name, tone, avatar, and memories. Background-update preferences and durable in-app notifications. |
| **Rich Threads** | Optional CopilotKit Intelligence persistence with a stable main conversation, side chats, renaming, archiving, restoring, and replay. A project key is required; live acceptance is pending. |
| **Rich Threads** | CopilotKit Intelligence persistence for live deployments, with a stable main conversation, side chats, renaming, archiving, restoring, and replay. A server-only project key is required in live mode; sample mode uses local history. |

The [feature inventory](docs/FEATURES.md) describes implemented capabilities and planned extensions. Health/bank/social connectors, device push, voice, generated executable tools, and automatic reservations/payments are on the [roadmap](ROADMAP.md).

## Quick start

**Requirements:** Node 24 LTS and pnpm 11.19.0. The local app needs no model, Google account, Docker, or Intelligence subscription.
**Requirements:** Node 24 LTS and pnpm 11.19.0. The local sample app needs no model, Google account, Docker, or Intelligence subscription.

```sh
git clone https://github.com/CopilotKit/OpenMuse.git openmuse
Expand Down Expand Up @@ -93,9 +93,10 @@ For iOS or Android, use `pnpm --dir apps/mobile ios` or `pnpm --dir apps/mobile
Copy the commented settings in [.env.example](.env.example) into your private `.env`:

1. Set `AGENT_BACKEND=model`, `MODEL=provider/model-id`, and the matching provider key. CopilotKit supports the configured OpenAI, Anthropic or Google provider. Fictional data can still be used with a real model. Provider keys stay on the server.
2. For personal mail/calendar, set `WORKSPACE_MODE=live`, a random `OPENMUSE_ACCESS_KEY` of at least 24 characters, and `TOKEN_ENCRYPTION_KEY` containing 32 random bytes encoded as base64. Restart the API.
3. Configure a Google OAuth web client with Gmail and Calendar APIs enabled. Set `GOOGLE_CLIENT_ID` and `GOOGLE_CLIENT_SECRET`; register `${PUBLIC_API_URL}/api/google/callback` as its redirect URI. Configure consent/test-user access in your Google project.
4. Open **Apps → Gmail** (or **Google Calendar**), connect read access, and grant write access when needed. Every send or calendar change still requires its own stored review. Changing/disconnecting the account invalidates pending connection-bound work.
2. Create or select a CopilotKit Intelligence project with `npx copilotkit@latest login` and `npx copilotkit@latest project select`. Keep the generated `CPK_INTELLIGENCE_API_KEY` on the server.
3. For personal mail/calendar, set `WORKSPACE_MODE=live`, the generated `CPK_INTELLIGENCE_API_KEY`, a random `OPENMUSE_ACCESS_KEY` of at least 24 characters, and `TOKEN_ENCRYPTION_KEY` containing 32 random bytes encoded as base64. Restart the API.
4. Configure a Google OAuth web client with Gmail and Calendar APIs enabled. Set `GOOGLE_CLIENT_ID` and `GOOGLE_CLIENT_SECRET`; register `${PUBLIC_API_URL}/api/google/callback` as its redirect URI. Configure consent/test-user access in your Google project.
5. Open **Apps → Gmail** (or **Google Calendar**), connect read access, and grant write access when needed. Every send or calendar change still requires its own stored review. Changing/disconnecting the account invalidates pending connection-bound work.

Google credentials are encrypted at rest. File URLs and browser consoles use short-lived signatures. This deployment uses one owner protected by a shared access key; it is not a multi-tenant authentication system. Use HTTPS and restricted network access for a remote host. Keep the default local-data mode on loopback.

Expand Down Expand Up @@ -133,17 +134,17 @@ No hidden retry occurs after an uncertain external write. Review its provider ou

## CopilotKit Rich Threads

Set `CPK_INTELLIGENCE_API_KEY` on the server and restart it to use CopilotKit Intelligence for conversation persistence and replay. The native menu uses `useThreads`; rich tool results link back to saved tasks, documents, and browser sessions. The default keeps one conversation in your local database.
Live deployments require `CPK_INTELLIGENCE_API_KEY` on the API server for CopilotKit Intelligence conversation persistence and replay. Create or select a project with `npx copilotkit@latest login` and `npx copilotkit@latest project select`, set the generated server-only key, and restart the API. The native menu uses `useThreads`; rich tool results link back to saved tasks, documents, and browser sessions.

Intelligence is a separate service and is not included in this repository's MIT license. No project key is shipped. [Configuration and validation boundaries](docs/RICH-THREADS.md).
Sample mode can leave the key unset and keeps one conversation in the local database. Intelligence is a separate service and is not included in this repository's MIT license. No project key is shipped. [Configuration and validation boundaries](docs/RICH-THREADS.md).

## Architecture

```mermaid
flowchart TD
Client[Expo / React Native / Web] -->|AG-UI and authenticated API| API[Hono + CopilotKit runtime]
API --> Tasks[Durable task worker]
API --> Threads[Optional CopilotKit Intelligence]
API --> Threads[CopilotKit Intelligence required in live mode]
API --> Store[(PGlite or PostgreSQL)]
Tasks --> Store
Tasks --> Review[Stored action review]
Expand Down
13 changes: 13 additions & 0 deletions apps/server/src/config.ts
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,19 @@ export interface Config {
computerDeploymentId?: string;
allowedOrigins: string[];
}

const missingIntelligenceKeyMessage =
"Live mode requires CPK_INTELLIGENCE_API_KEY for durable Rich Threads. " +
"Run `npx copilotkit@latest login` and `npx copilotkit@latest project select`, " +
"then set the generated server-only key. " +
"See https://docs.copilotkit.ai/intelligence/connect-your-runtime";

export function assertApiDeploymentConfig(config: Config): void {
if (config.mode === "live" && !config.intelligenceApiKey?.trim()) {
throw new Error(missingIntelligenceKeyMessage);
}
}

export function readConfig(): Config {
const mode = process.env.WORKSPACE_MODE ?? "sample";
if (mode !== "sample" && mode !== "live")
Expand Down
3 changes: 2 additions & 1 deletion apps/server/src/index.ts
Original file line number Diff line number Diff line change
@@ -1,9 +1,10 @@
import { serve } from "@hono/node-server";
import { createApp } from "./app.ts";
import { readConfig } from "./config.ts";
import { assertApiDeploymentConfig, readConfig } from "./config.ts";
import { createStore } from "./db.ts";

const config = readConfig();
assertApiDeploymentConfig(config);
const db = await createStore({
dataDir: `${config.dataDir}/postgres`,
databaseUrl: config.databaseUrl,
Expand Down
13 changes: 10 additions & 3 deletions docs/RICH-THREADS.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,21 +4,28 @@ OpenMuse uses `@copilotkit/react-native/headless` for its custom native and web

## Rich Threads

Set the following **server-only** environment variable and restart the API:
`WORKSPACE_MODE=live` requires a CopilotKit Intelligence project key before the API will start. Create or select a project:

```sh
npx copilotkit@latest login
npx copilotkit@latest project select
```

Keep the generated key in the API server environment and restart the API:

```dotenv
CPK_INTELLIGENCE_API_KEY=your-project-key
```

Obtain the project key using [CopilotKit's runtime setup](https://docs.copilotkit.ai/intelligence/connect-your-runtime). Never put it in an `EXPO_PUBLIC_` variable. The server constructs `CopilotKitIntelligence`, and `identifyUser` resolves the owner from the verified OpenMuse session. The existing deployment is a single-user workspace; its access key must not be shared as a multi-user login.
Never put the key in an `EXPO_PUBLIC_`, `NEXT_PUBLIC_`, or `VITE_` variable. The server constructs `CopilotKitIntelligence`, and `identifyUser` resolves the owner from the verified OpenMuse session. The existing deployment is a single-user workspace; its access key must not be shared as a multi-user login. See [CopilotKit's runtime setup](https://docs.copilotkit.ai/intelligence/connect-your-runtime) for project-key configuration.

With this configuration, the conversation menu uses CopilotKit's native `useThreads` hook to list, rename, archive, restore and paginate conversations. The server saves an owner-bound main thread ID and provisions it through Intelligence before the first message, so reloading during the first run retains the same conversation. Side chats create a fresh client thread ID and persist on their first run. Selecting a saved conversation mounts a private `useAgent({ agentId, runtimeAgentId, threadId })` instance and calls `copilotkit.connectAgent` to replay the thread. Visited chats remain mounted during navigation, preserving their drafts and queues. Stop explicitly requests `copilotkit.stopAgent`.

The composer stays editable during replies. Follow-ups enter a visible, removable queue and run in order after the current reply and its persistence finish. Stopping or failing a reply pauses the queue; SDK errors emitted without rejecting the run promise still stop subsequent sends. The queue lives in the open app, not a durable server inbox. Delegated tasks remain separate server work. See the [interaction design](EXPERIENCE.md).

Rich tool messages retain task IDs. The renderer fetches current task status, browser previews, PDF links and structured artifacts from the authenticated task endpoint. Expiring file/preview URLs are generated by the server rather than stored in thread messages. New Intelligence conversations never load or overwrite `/api/conversation`.

When the key is unset, OpenMuse keeps its existing local conversation store. The menu explicitly identifies that mode. This is the entirely self-hosted open-source default; it does not emulate CopilotKit Intelligence or claim to provide its cloud sync. Existing local history remains available when returning to local mode. Historical local messages are not automatically uploaded to Intelligence.
When the key is unset in sample mode, OpenMuse keeps its existing local conversation store and the menu identifies that mode. Sample history remains available after returning from live mode, but it is not automatically uploaded to Intelligence. Live mode does not fall back to this store: the API fails at startup with the project-selection commands when the key is missing.

Automatic thread naming is disabled; conversations can be renamed in the menu. Intelligence is a separately configured service, not bundled with this MIT-licensed application. See [headless threads](https://docs.copilotkit.ai/headless-threads) for the platform lifecycle and hosting options.

Expand Down
46 changes: 46 additions & 0 deletions tests/config.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
import assert from "node:assert/strict";
import { test } from "node:test";
import { assertApiDeploymentConfig, type Config } from "../apps/server/src/config.ts";

const sampleConfig: Config = {
mode: "sample",
port: 8787,
host: "127.0.0.1",
publicUrl: "http://localhost:8787",
dataDir: ".openmuse",
agentBackend: "sample",
googleRedirectUri: "http://localhost:8787/api/google/callback",
allowedOrigins: ["http://localhost:8081"],
};

function liveConfig(intelligenceApiKey?: string): Config {
return {
...sampleConfig,
mode: "live",
agentBackend: "model",
intelligenceApiKey,
};
}

const missingKeyMessage =
"Live mode requires CPK_INTELLIGENCE_API_KEY for durable Rich Threads. " +
"Run `npx copilotkit@latest login` and `npx copilotkit@latest project select`, " +
"then set the generated server-only key. " +
"See https://docs.copilotkit.ai/intelligence/connect-your-runtime";

test("live API configuration rejects a missing or blank Intelligence key", () => {
for (const key of [undefined, "", " \t\n"]) {
assert.throws(() => assertApiDeploymentConfig(liveConfig(key)), {
name: "Error",
message: missingKeyMessage,
});
}
});

test("live API configuration accepts a non-empty Intelligence key", () => {
assert.doesNotThrow(() => assertApiDeploymentConfig(liveConfig("test-project-key-never-sent")));
});

test("sample API configuration remains key-free", () => {
assert.doesNotThrow(() => assertApiDeploymentConfig(sampleConfig));
});
Loading