Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -61,6 +61,8 @@ bun run kill # Kill processes on ports 3000, 3002, 8081

Use `wss://your-domain.com` for secure WebSocket connections. For local development without SSL, use `ws://localhost:8081`.

The client reaches the server through a `UseAITransport`. `SocketIOTransport` is the default. `WebSocketTransport` carries AG-UI events as JSON frames over a plain WebSocket. The server serves one or the other, chosen by `transport: 'socketio' | 'websocket'` (default `socketio`). The framing is documented in `docs/websocket-protocol.md`.

## Core Architecture

### Data Flow
Expand Down
33 changes: 33 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ A React client/framework for easily enabling AI to control your users frontend.
- [Features](#features)
- [General](#general)
- [AG-UI Protocol](#ag-ui-protocol)
- [Transports](#transports)
- [Client](#client)
- [`useAI` hook](#useai-hook)
- [`UseAIProvider`](#useaiprovider)
Expand Down Expand Up @@ -297,6 +298,35 @@ There are some minor extensions to the protocol:
**Message Types**:
- `run_workflow`: Trigger headless workflow (use-ai extension) [see `@meetsmore-oss/use-ai-plugin-workflows`]

### Transports

The client reaches the server through a `UseAITransport`. Two transports ship with the library.

| Transport | Wire | Bundled server setting |
| -------------------- | ------------------------------------------------- | ------------------------ |
| `SocketIOTransport` | Socket.IO, over polling and WebSocket | `transport: 'socketio'` |
| `WebSocketTransport` | AG-UI events as JSON text, over a plain WebSocket | `transport: 'websocket'` |

Give `UseAIProvider` either `serverUrl` or `transport`. `serverUrl` builds a `SocketIOTransport`, so nothing changes if you use the bundled server with its defaults.

Pass `WebSocketTransport` to reach a server that does not serve Socket.IO. Such a server does not have to be Node. It must accept a WebSocket connection. It must then exchange the documented frames.

```tsx
import { UseAIProvider, WebSocketTransport } from '@meetsmore-oss/use-ai-client';

root.render(
<UseAIProvider transport={new WebSocketTransport('wss://your-server.com')}>
<App />
</UseAIProvider>
);
```

The bundled server serves one transport. Set `transport: 'websocket'` on `UseAIServer`, or `TRANSPORT=websocket` on the Docker image, to serve a plain WebSocket at `/` instead of Socket.IO.

To carry the same messages over something else, implement `UseAITransport` yourself. It opens and closes a connection, sends client messages, and delivers AG-UI events.

See [docs/websocket-protocol.md](docs/websocket-protocol.md) for the frames, the turn sequence, and the reconnection behaviour.

## Client

### `useAI` hook
Expand Down Expand Up @@ -353,6 +383,8 @@ root.render(
);
```

Pass `transport` instead of `serverUrl` to reach a server over something other than Socket.IO. See [Transports](#transports).

### Component State via `prompt`

When you call `useAI`, you can provide a prompt that is used to tell the LLM the state of the component in a text-friendly way.
Expand Down Expand Up @@ -1048,6 +1080,7 @@ const server = new UseAIServer({
})
},
defaultAgent: 'claude',
transport: 'socketio', // or 'websocket', see 'Transports'
rateLimitMaxRequests: 1_000,
rateLimitWindowMs: 60_000,
plugins: [ // see 'Plugins'
Expand Down
3 changes: 3 additions & 0 deletions apps/use-ai-server-app/src/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ const maxHttpBufferSize = process.env.MAX_HTTP_BUFFER_SIZE
const corsOrigin = process.env.CORS_ORIGIN;
// Runtime adapter: 'auto' (default), 'bun', or 'node'
const runtime = (process.env.RUNTIME as 'auto' | 'bun' | 'node') || 'auto';
const transport = (process.env.TRANSPORT as 'socketio' | 'websocket') || 'socketio';

/**
* Create agents based on available API keys.
Expand Down Expand Up @@ -389,6 +390,7 @@ logger.info('Starting UseAI server', { logFormat });
}
: undefined,
runtime,
transport,
});

// Initialize MCP endpoints
Expand All @@ -401,6 +403,7 @@ logger.info('Starting UseAI server', { logFormat });
console.log(`βœ“ UseAI server is running on port ${port}`);
console.log(` WebSocket URL: ws://localhost:${port}`);
console.log(` Runtime: ${runtime} (set RUNTIME=bun or RUNTIME=node to change)`);
console.log(` Transport: ${transport} (set TRANSPORT=websocket for a plain WebSocket)`);
console.log(` Log format: ${logFormat} (set LOG_FORMAT=json for structured logs)`);
console.log(' Press Ctrl+C to stop');
}
Expand Down
10 changes: 10 additions & 0 deletions bun.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

140 changes: 140 additions & 0 deletions docs/websocket-protocol.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,140 @@
# Plain WebSocket protocol
Comment thread
mm-zacharydavison marked this conversation as resolved.

The client reaches the server through a `UseAITransport`. `SocketIOTransport` is the default. `WebSocketTransport` is the alternative. It carries AG-UI events as JSON text frames over a plain WebSocket.

Use `WebSocketTransport` to connect the `use-ai` chat UI and hooks to your own server. Your server does not have to be Node. It does not have to serve Socket.IO. It must accept a WebSocket connection. It must then exchange the frames that this document defines.

## Client

```tsx
import { UseAIProvider, WebSocketTransport } from '@meetsmore-oss/use-ai-client';

root.render(
<UseAIProvider transport={new WebSocketTransport('wss://your-server.com')}>
<App />
</UseAIProvider>
);
```

Give the provider `serverUrl` or `transport`, not both. `serverUrl` connects over Socket.IO. `transport` connects over the transport that you pass.

The provider reads `transport` once, on the first render. An inline object therefore does not reconnect the client on each render. To change transports, remount the provider.

## Bundled server

The bundled server serves one transport. The default is Socket.IO.

```typescript
const server = new UseAIServer({
agents: { claude },
defaultAgent: 'claude',
transport: 'websocket', // default: 'socketio'
});
```

With `transport: 'websocket'`, the server accepts WebSocket upgrades at `/`. It does not serve Socket.IO. The `/health` endpoint is unchanged.

The Docker image reads the same setting from the `TRANSPORT` environment variable.

## Encoding

Each frame is one JSON object, as a text frame. JSON is the encoding that AG-UI, MCP and the OpenAI Realtime API use on their wires. AG-UI also defines a protobuf encoding for bandwidth. This protocol does not use it.

## Upstream frames

The client sends each `UseAIClientMessage` as one frame, with nothing around it.

```json
{ "type": "run_agent", "data": { "threadId": "...", "runId": "...", "messages": [], "tools": [], "state": null, "forwardedProps": {} } }
```

The message types are:

- `run_agent`
- `tool_result`
- `tool_approval_response`
- `abort_run`
- `message_feedback`

Plugins add more. See `UseAIClientMessage` in `@meetsmore-oss/use-ai-core` for each payload.

## Downstream frames

The server sends one AG-UI event per frame. Each event has a `type` field.

```json
Comment thread
mm-zacharydavison marked this conversation as resolved.
{ "type": "RUN_STARTED", "threadId": "...", "runId": "...", "timestamp": 1700000000000 }
{ "type": "TEXT_MESSAGE_CONTENT", "messageId": "...", "delta": "Hello" }
{ "type": "RUN_FINISHED", "threadId": "...", "runId": "..." }
```

Two payloads are not AG-UI events. The server sends them as AG-UI `CUSTOM` events, once, after the connection opens.

```json
{ "type": "CUSTOM", "name": "agents", "value": { "agents": [{ "id": "claude", "name": "Claude" }], "defaultAgent": "claude" } }
{ "type": "CUSTOM", "name": "config", "value": { "langfuseEnabled": true } }
```

| Name | Value | Required |
| -------- | -------------------------------------------------- | -------- |
| `agents` | The agent list and the default agent id | Yes |
| `config` | Capability flags, such as `langfuseEnabled` | No |

The client ignores an event type that it does not handle. It also ignores a `CUSTOM` name that it does not know. A server can therefore add events without a change to older clients.

The client handles these event types:

- `RUN_STARTED`, `RUN_FINISHED`, `RUN_ERROR`
- `STEP_STARTED`, `STEP_FINISHED`
- `TEXT_MESSAGE_START`, `TEXT_MESSAGE_CONTENT`, `TEXT_MESSAGE_END`
- `TOOL_CALL_START`, `TOOL_CALL_ARGS`, `TOOL_CALL_END`, `TOOL_CALL_RESULT`
- `REASONING_MESSAGE_START`, `REASONING_MESSAGE_CONTENT`, `REASONING_MESSAGE_END`, `REASONING_ENCRYPTED_VALUE`
- `TOOL_APPROVAL_REQUEST`, a `use-ai` extension

See the [AG-UI protocol](https://docs.ag-ui.com/introduction) for each event. See `packages/core/src/types.ts` for the types that this library uses.

## One turn, step by step

1. The client opens the connection.
2. The server sends `agents`. It then sends `config`.
3. The client sends `run_agent` with the prompt, the tool definitions and the app state.
4. The server sends `RUN_STARTED`. It then streams the model output.
5. For a client-side tool, the server sends `TOOL_CALL_START`, `TOOL_CALL_ARGS` and `TOOL_CALL_END`.
6. The client runs the tool. It then sends `tool_result` with the output.
7. The server resumes the model. It then streams the rest of the output.
8. The server sends `RUN_FINISHED`.

## Reconnection

`WebSocketTransport` reconnects through [partysocket](https://github.com/partykit/partykit/tree/main/packages/partysocket). It retries indefinitely. The delay doubles after each attempt, from one second up to ten seconds. The limits match `SocketIOTransport`, so a mobile app in the background, or a device in airplane mode, recovers without frequent retries.

Set both delays in the options:

```typescript
new WebSocketTransport('wss://your-server.com', {
reconnectionDelay: 1000, // first retry, in milliseconds
reconnectionDelayMax: 10000, // upper bound, in milliseconds
});
```

The server destroys the session when the connection closes. A reconnected client therefore starts a new session. The client sends its conversation history with the next `run_agent`.

## Your own transport

`UseAITransport` has seven members:

- `url`
- `connected`
- `connect`
- `disconnect`
- `send`
- `onEvent`, which delivers AG-UI events
- `onConnectionChange`

Implement `UseAITransport` to carry the same messages over something else. Deliver the agent list and the server config as `CUSTOM` events, as the section above describes.

```typescript
import { UseAIClient, type UseAITransport } from '@meetsmore-oss/use-ai-client';

const client = new UseAIClient(myTransport);
```
1 change: 1 addition & 0 deletions packages/client/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,7 @@
},
"dependencies": {
"@meetsmore-oss/use-ai-core": "workspace:^",
"partysocket": "^1.3.0",
"react-markdown": "^8.0.0",
"remark-gfm": "3",
"socket.io-client": "^4.8.1",
Expand Down
Loading
Loading