Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions .env.example
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
# Copy to .env. Sample mode stays on localhost and uses fictional data.
# Every mode requires a server-only CopilotKit Intelligence project key.
WORKSPACE_MODE=sample
AGENT_BACKEND=sample
PORT=8787
Expand Down Expand Up @@ -39,11 +40,10 @@ WORKER_HOST=127.0.0.1
# AGENT_TOKEN=
# OpenBot's Intelligence runtime is not this raw AG-UI URL. See its adapter docs.

# CopilotKit Intelligence is required when WORKSPACE_MODE=live. Create or select
# a project with the commands below, then keep the generated key server-only.
# CopilotKit Intelligence is required in every mode. Create or select a project
# with the commands below, then keep the generated key server-only.
# npx copilotkit@latest login
# npx copilotkit@latest project select
# Sample mode can leave this unset and use the local conversation store.
CPK_INTELLIGENCE_API_KEY=

# Optional private Docker Linux computer (separate from the browser).
Expand Down
2 changes: 1 addition & 1 deletion CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ OpenMuse is an MIT-licensed alpha. Contributions should make delegated work reli
## Local development

1. Fork and clone the repository. Use Node 24 LTS and pnpm 11.19.0.
2. Run `pnpm install --frozen-lockfile` and copy `.env.example` to `.env`.
2. Run `pnpm install --frozen-lockfile`, copy `.env.example` to `.env`, and use `npx copilotkit@latest login` then `npx copilotkit@latest project select` to set the required Intelligence key.
3. Run `pnpm dev` and, in another terminal, `pnpm dev:web`.
4. Use the fictional sample workspace for development and recordings. See [native setup](apps/mobile/README.md) for simulator/emulator builds.

Expand Down
15 changes: 9 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,19 +55,22 @@ The computer combines **persistent Chromium and an optional Linux workspace**. T
| **Finance** | Import transaction CSV to create a spending summary with categories, transactions, and a savings-goal action. |
| **Gmail & Calendar** | Google OAuth adapters, complete mail threads, drafts/attachments, calendar discovery, and reviewed event creation/update/deletion. Live credentials required. |
| **Personal context** | Editable name, tone, avatar, and memories. Background-update preferences and durable in-app notifications. |
| **Rich Threads** | CopilotKit Intelligence persistence for live deployments, with a stable main conversation, side chats, renaming, archiving, restoring, and replay. A server-only project key is required in live mode; sample mode uses local history. |
| **Rich Threads** | CopilotKit Intelligence persistence in every mode, with a stable main conversation, side chats, renaming, archiving, restoring, and replay. A server-only project key is required. |

The [feature inventory](docs/FEATURES.md) describes implemented capabilities and planned extensions. Health/bank/social connectors, device push, voice, generated executable tools, and automatic reservations/payments are on the [roadmap](ROADMAP.md).

## Quick start

**Requirements:** Node 24 LTS and pnpm 11.19.0. The local sample app needs no model, Google account, Docker, or Intelligence subscription.
**Requirements:** Node 24 LTS, pnpm 11.19.0, and a CopilotKit Intelligence project key. The local sample app needs no model, Google account, or Docker.

```sh
git clone https://github.com/CopilotKit/OpenMuse.git openmuse
cd openmuse
pnpm install --frozen-lockfile
cp .env.example .env
npx copilotkit@latest login
npx copilotkit@latest project select
# Set CPK_INTELLIGENCE_API_KEY in .env to the generated server-only project key.
pnpm dev
```

Expand All @@ -84,7 +87,7 @@ Open [localhost:8081](http://localhost:8081). The API runs at [localhost:8787/ap
1. In Chat, send **“Complete the permission slip”**. Open the task, supply fictional form values, inspect the saved PDF, and review the prepared reply. This writes only to the local mailbox.
2. In **Goals → Track**, create a built-in availability watch, then change the built-in test page to trigger an alert.
3. In **Menu → Delegate task → Finance**, use **Try example transactions** to create an interactive spending tracker.
4. Start the [browser worker](#browser-worker) and configure a model, then ask **“Check out Hacker News for cool stuff”** or **“Summarize copilotkit.ai”**. Follow the browser inline and use **Take control** to open its session. For a key-free version of this flow, follow the [AI Mock demo setup](docs/DEMO.md#run-the-agent-browser-demo).
4. Start the [browser worker](#browser-worker) and configure a model, then ask **“Check out Hacker News for cool stuff”** or **“Summarize copilotkit.ai”**. Follow the browser inline and use **Take control** to open its session. For a model-free version of this flow, follow the [AI Mock demo setup](docs/DEMO.md#run-the-agent-browser-demo).

For iOS or Android, use `pnpm --dir apps/mobile ios` or `pnpm --dir apps/mobile android`. Xcode or Android tooling is required. The PDF reader needs an Expo development build; use [native setup](apps/mobile/README.md).

Expand Down Expand Up @@ -134,17 +137,17 @@ No hidden retry occurs after an uncertain external write. Review its provider ou

## CopilotKit Rich Threads

Live deployments require `CPK_INTELLIGENCE_API_KEY` on the API server for CopilotKit Intelligence conversation persistence and replay. Create or select a project with `npx copilotkit@latest login` and `npx copilotkit@latest project select`, set the generated server-only key, and restart the API. The native menu uses `useThreads`; rich tool results link back to saved tasks, documents, and browser sessions.
Every deployment requires `CPK_INTELLIGENCE_API_KEY` on the API server for CopilotKit Intelligence conversation persistence and replay. Create or select a project with `npx copilotkit@latest login` and `npx copilotkit@latest project select`, set the generated server-only key, and restart the API. The native menu uses `useThreads`; rich tool results link back to saved tasks, documents, and browser sessions.

Sample mode can leave the key unset and keeps one conversation in the local database. Intelligence is a separate service and is not included in this repository's MIT license. No project key is shipped. [Configuration and validation boundaries](docs/RICH-THREADS.md).
Intelligence is a separate service and is not included in this repository's MIT license. No project key is shipped. [Configuration and validation boundaries](docs/RICH-THREADS.md).

## Architecture

```mermaid
flowchart TD
Client[Expo / React Native / Web] -->|AG-UI and authenticated API| API[Hono + CopilotKit runtime]
API --> Tasks[Durable task worker]
API --> Threads[CopilotKit Intelligence required in live mode]
API --> Threads[CopilotKit Intelligence required in every mode]
API --> Store[(PGlite or PostgreSQL)]
Tasks --> Store
Tasks --> Review[Stored action review]
Expand Down
2 changes: 1 addition & 1 deletion SECURITY.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,4 +28,4 @@ Docker shares its host kernel and does not provide a full VM or a hostile-tenant

A proposal is bound to the account, reviewed content, and applicable provider version. The server requires a recorded approval before dispatching a send or calendar change. An uncertain network outcome is retained for reconciliation. Cancellation stops later task steps; a provider request already in flight may still finish.

No provider keys, personal data, or third-party logins are needed for the sample walkthrough or CI. CopilotKit Intelligence and any configured model/provider operate under their own terms and data policies.
A server-only CopilotKit Intelligence project key is needed for the sample walkthrough. CI uses synthetic keys and mocked Intelligence boundaries. No provider keys, personal data, or third-party logins are needed for CI. CopilotKit Intelligence and any configured model/provider operate under their own terms and data policies.
1 change: 1 addition & 0 deletions apps/computer/smoke.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ test("real isolated computer executes commands, persists files, bridges PDFs and
publicUrl: "http://localhost:8787",
dataDir: directory,
agentBackend: "sample",
intelligenceApiKey: "test-project-key-never-sent",
googleRedirectUri: "http://localhost/callback",
allowedOrigins: [],
computerEnabled: true,
Expand Down
22 changes: 10 additions & 12 deletions apps/server/src/agent.ts
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ export function makeRuntime(
config: Config,
service: AgentService,
auth: Auth,
intelligence?: CopilotKitIntelligence,
intelligence: CopilotKitIntelligence,
) {
const agents: AgentsFactory = async ({ request }) => ({
default:
Expand All @@ -49,16 +49,14 @@ export function makeRuntime(
await auth.owner(request.headers.get("authorization") ?? undefined),
),
});
const runtime = intelligence
? new CopilotRuntime({
agents,
intelligence,
identifyUser: async (request) => ({
id: await auth.owner(request.headers.get("authorization") ?? undefined),
name: "OpenMuse user",
}),
generateThreadNames: false,
})
: new CopilotRuntime({ agents });
const runtime = new CopilotRuntime({
agents,
intelligence,
identifyUser: async (request) => ({
id: await auth.owner(request.headers.get("authorization") ?? undefined),
name: "OpenMuse user",
}),
generateThreadNames: false,
});
return createCopilotHonoHandler({ runtime, basePath: "/api/copilotkit" });
}
33 changes: 15 additions & 18 deletions apps/server/src/app.ts
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ import { createAuth } from "./auth.ts";
import { BrowserService } from "./browser.ts";
import { ComputerService, type DockerRunner } from "./computer.ts";
import { computerRoutes } from "./computer-routes.ts";
import type { Config } from "./config.ts";
import { assertApiDeploymentConfig, type Config } from "./config.ts";
import type { Store } from "./db.ts";
import { agentRoutes } from "./engine/routes.ts";
import { AgentService } from "./engine/service.ts";
Expand All @@ -26,6 +26,7 @@ export async function createApp(
config: Config,
options: { docker?: DockerRunner } = {},
) {
assertApiDeploymentConfig(config);
const auth = await createAuth(db, config),
files = new Files(db, config, auth),
google = new GoogleAuth(db, config),
Expand All @@ -40,9 +41,7 @@ export async function createApp(
const browser = new BrowserService(db, config, auth, files);
const computer = new ComputerService(db, config, options.docker);
const agent = new AgentService(db, config, workspace, files, actions, browser, computer);
const intelligence = config.intelligenceApiKey
? new CopilotKitIntelligence({ apiKey: config.intelligenceApiKey })
: undefined;
const intelligence = new CopilotKitIntelligence({ apiKey: config.intelligenceApiKey });
const runtime = makeRuntime(config, agent, auth, intelligence);
const app = new Hono<{ Variables: { owner: string } }>();
const origins = new Set([...config.allowedOrigins, new URL(config.publicUrl).origin]);
Expand Down Expand Up @@ -203,21 +202,19 @@ export async function createApp(
});
const main = await db.get<{ threadId: string }>(owner, "conversation-settings", "main");
if (!main) throw new AppError("Main conversation could not be loaded", 503);
if (intelligence) {
try {
await intelligence.getOrCreateThread({
threadId: main.threadId,
userId: owner,
agentId: "default",
});
} catch {
throw new AppError(
"Main conversation is unavailable. Check the Rich Threads connection and try again.",
502,
);
}
try {
await intelligence.getOrCreateThread({
threadId: main.threadId,
userId: owner,
agentId: "default",
});
} catch {
throw new AppError(
"Main conversation is unavailable. Check the Rich Threads connection and try again.",
502,
);
}
return c.json({ threadId: main.threadId, existing: Boolean(intelligence) });
return c.json({ threadId: main.threadId, existing: true });
});
app.get("/api/conversation", async (c) =>
c.json((await db.get(c.get("owner"), "conversations", "default")) ?? { messages: [] }),
Expand Down
23 changes: 16 additions & 7 deletions apps/server/src/config.ts
Original file line number Diff line number Diff line change
Expand Up @@ -31,16 +31,25 @@ export interface Config {
allowedOrigins: string[];
}

const missingIntelligenceKeyMessage =
"Live mode requires CPK_INTELLIGENCE_API_KEY for durable Rich Threads. " +
export const intelligenceKeyRequiredMessage =
"OpenMuse requires CPK_INTELLIGENCE_API_KEY. " +
"Run `npx copilotkit@latest login` and `npx copilotkit@latest project select`, " +
"then set the generated server-only key. " +
"See https://docs.copilotkit.ai/intelligence/connect-your-runtime";

export function assertApiDeploymentConfig(config: Config): void {
if (config.mode === "live" && !config.intelligenceApiKey?.trim()) {
throw new Error(missingIntelligenceKeyMessage);
}
export function required(name: string, message: string, value = process.env[name]): string {
if (!value?.trim()) throw new Error(message);
return value.trim();
}

export function assertApiDeploymentConfig(
config: Config,
): asserts config is Config & { intelligenceApiKey: string } {
required(
"CPK_INTELLIGENCE_API_KEY",
intelligenceKeyRequiredMessage,
config.intelligenceApiKey ?? "",
);
}

export function readConfig(): Config {
Expand All @@ -67,7 +76,7 @@ export function readConfig(): Config {
agentBackend: backend,
agentUrl: process.env.AGENT_URL,
agentToken: process.env.AGENT_TOKEN,
intelligenceApiKey: process.env.CPK_INTELLIGENCE_API_KEY,
intelligenceApiKey: required("CPK_INTELLIGENCE_API_KEY", intelligenceKeyRequiredMessage),
googleClientId: process.env.GOOGLE_CLIENT_ID,
googleClientSecret: process.env.GOOGLE_CLIENT_SECRET,
googleRedirectUri: `${publicUrl}/api/google/callback`,
Expand Down
3 changes: 3 additions & 0 deletions apps/server/src/demo/entry.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,10 +3,12 @@ import { createHash } from "node:crypto";
import { mkdir, mkdtemp, rm } from "node:fs/promises";
import { join } from "node:path";
import { fileURLToPath } from "node:url";
import { intelligenceKeyRequiredMessage, required } from "../config.ts";
import { createStore } from "../db.ts";
import { createDemoModel, demoModel } from "./model.ts";

const root = fileURLToPath(new URL("../../../../", import.meta.url));
const intelligenceApiKey = required("CPK_INTELLIGENCE_API_KEY", intelligenceKeyRequiredMessage);
const dataDir = join(root, "artifacts", "demo", "api");
const runtimeDir = join(root, "artifacts", "demo", "runtime");
const port = Number(process.env.DEMO_API_PORT ?? "8788");
Expand Down Expand Up @@ -84,6 +86,7 @@ const api = spawn(
process.env.DEMO_ALLOWED_ORIGINS ?? "http://localhost:8081,http://127.0.0.1:8081",
DO_NOT_TRACK: "1",
COPILOTKIT_TELEMETRY_DISABLED: "true",
CPK_INTELLIGENCE_API_KEY: intelligenceApiKey,
},
},
);
Expand Down
3 changes: 1 addition & 2 deletions apps/server/src/index.ts
Original file line number Diff line number Diff line change
@@ -1,10 +1,9 @@
import { serve } from "@hono/node-server";
import { createApp } from "./app.ts";
import { assertApiDeploymentConfig, readConfig } from "./config.ts";
import { readConfig } from "./config.ts";
import { createStore } from "./db.ts";

const config = readConfig();
assertApiDeploymentConfig(config);
const db = await createStore({
dataDir: `${config.dataDir}/postgres`,
databaseUrl: config.databaseUrl,
Expand Down
2 changes: 1 addition & 1 deletion apps/server/src/workspace.ts
Original file line number Diff line number Diff line change
Expand Up @@ -323,7 +323,7 @@ export class WorkspaceService {
provider: this.config.agentBackend === "sample" ? "sample" : "model",
configured: agentConfigured(this.config),
openbotConfigured: false,
richThreads: Boolean(this.config.intelligenceApiKey),
richThreads: true,
},
};
}
Expand Down
4 changes: 3 additions & 1 deletion docs/DEMO.md
Original file line number Diff line number Diff line change
Expand Up @@ -62,11 +62,13 @@ From the repository root:

```sh
pnpm install --frozen-lockfile
npx copilotkit@latest login
npx copilotkit@latest project select
pnpm --dir apps/worker exec playwright install chromium
pnpm dev:demo
```

This starts AI Mock, the normal OpenMuse API on port **8788**, and a separate real browser worker on **8791**. Demo files and profiles stay in ignored `artifacts/demo/`. The runner supplies an explicit local environment and does not load the project's private `.env` or provider credentials. The Linux computer is disabled for this focused browser recording.
This starts AI Mock, the normal OpenMuse API on port **8788**, and a separate real browser worker on **8791**. Demo files and profiles stay in ignored `artifacts/demo/`. The runner reads only the Intelligence key from the project's private `.env` and passes it to its isolated API process; it does not pass provider or Google credentials. The Linux computer is disabled for this focused browser recording.

Start the app in another terminal:

Expand Down
4 changes: 2 additions & 2 deletions docs/RICH-THREADS.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ OpenMuse uses `@copilotkit/react-native/headless` for its custom native and web

## Rich Threads

`WORKSPACE_MODE=live` requires a CopilotKit Intelligence project key before the API will start. Create or select a project:
Every workspace mode requires a CopilotKit Intelligence project key before the API will start. Create or select a project:

```sh
npx copilotkit@latest login
Expand All @@ -25,7 +25,7 @@ The composer stays editable during replies. Follow-ups enter a visible, removabl

Rich tool messages retain task IDs. The renderer fetches current task status, browser previews, PDF links and structured artifacts from the authenticated task endpoint. Expiring file/preview URLs are generated by the server rather than stored in thread messages. New Intelligence conversations never load or overwrite `/api/conversation`.

When the key is unset in sample mode, OpenMuse keeps its existing local conversation store and the menu identifies that mode. Sample history remains available after returning from live mode, but it is not automatically uploaded to Intelligence. Live mode does not fall back to this store: the API fails at startup with the project-selection commands when the key is missing.
The API fails at startup with a missing-key error when the key is unset. Existing local sample history is not automatically uploaded to Intelligence.

Automatic thread naming is disabled; conversations can be renamed in the menu. Intelligence is a separately configured service, not bundled with this MIT-licensed application. See [headless threads](https://docs.copilotkit.ai/headless-threads) for the platform lifecycle and hosting options.

Expand Down
15 changes: 12 additions & 3 deletions tests/agent-api.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@ import { mkdtemp, rm } from "node:fs/promises";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { after, before, test } from "node:test";
import { CopilotKitIntelligence } from "@copilotkit/runtime/v2";
import { createApp } from "../apps/server/src/app.ts";
import type { Config } from "../apps/server/src/config.ts";
import { createStore, type Store } from "../apps/server/src/db.ts";
Expand Down Expand Up @@ -41,6 +42,7 @@ before(async () => {
publicUrl: "http://localhost:8787",
dataDir: directory,
agentBackend: "model",
intelligenceApiKey: "test-project-key-never-sent",
googleRedirectUri: "http://localhost:8787/api/google/callback",
allowedOrigins: ["http://localhost:8081"],
};
Expand Down Expand Up @@ -77,19 +79,26 @@ test("agent API requires a session and reports the actual worker state", async (
assert.equal(workspace.identity.tone, "warm");
});

test("the main Rich Thread survives reopening and concurrent initialization", async () => {
test("the main Rich Thread survives reopening and concurrent initialization", async (t) => {
t.mock.method(
CopilotKitIntelligence.prototype,
"getOrCreateThread",
async (input: Parameters<CopilotKitIntelligence["getOrCreateThread"]>[0]) => ({
id: input.threadId,
}),
);
assert.equal((await server.app.request("/api/main-thread")).status, 401);
const responses = await Promise.all(
Array.from({ length: 3 }, () => server.app.request("/api/main-thread", { headers: headers() })),
);
const threads = await Promise.all(responses.map((response) => response.json()));
assert.ok(threads.every((thread) => thread.threadId === threads[0].threadId));
assert.equal(threads[0].existing, false);
assert.equal(threads[0].existing, true);
const reopened = await (
await server.app.request("/api/main-thread", { headers: headers() })
).json();
assert.equal(reopened.threadId, threads[0].threadId);
assert.equal(reopened.existing, false);
assert.equal(reopened.existing, true);
assert.equal(await db.get("other-user", "conversation-settings", "main"), null);
});

Expand Down
Loading
Loading