Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
85 changes: 79 additions & 6 deletions apps/docs/content/3.core-concepts/11.ai-sdk.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
title: AI SDK Integration
description: Capture token usage, tool calls, model info, and streaming metrics from the Vercel AI SDK into wide events. Wrap your model and get full AI observability.
navigation:
icon: i-lucide-scan-eye
icon: i-simple-icons-vercel
links:
- label: Wide Events
icon: i-lucide-layers
Expand Down Expand Up @@ -112,15 +112,47 @@ Your wide event now includes:

## How It Works

`createAILogger(log)` returns an `AILogger` with two methods:
`createAILogger(log, options?)` returns an `AILogger` with two methods:

| Method | Description |
|--------|-------------|
| `wrap(model)` | Wraps a language model with middleware. Accepts a model string (e.g. `'anthropic/claude-sonnet-4.6'`) or a `LanguageModelV3` object. Works with `generateText`, `streamText`, `generateObject`, `streamObject`, and `ToolLoopAgent`. |
| `wrap(model)` | Wraps a language model with middleware. Accepts a model string (e.g. `'anthropic/claude-sonnet-4.6'`) or a `LanguageModelV3` object. Works with `generateText`, `streamText`, `generateObject`, `streamObject`, and `ToolLoopAgent`. Also works with pre-wrapped models (e.g. from supermemory). |
| `captureEmbed(result)` | Manually captures token usage from `embed()` or `embedMany()` results (embedding models use a different type). |

The middleware intercepts calls at the provider level. It does not touch your callbacks, prompts, or responses. Captured data flows through the normal evlog pipeline (sampling, enrichers, drains) and ends up in Axiom, Better Stack, or wherever you drain to.

### Options

| Option | Type | Default | Description |
|--------|------|---------|-------------|
| `toolInputs` | `boolean \| ToolInputsOptions` | `false` | When enabled, `toolCalls` contains `{ name, input }` objects instead of plain strings. Opt-in because inputs can be large and may contain sensitive data. |

Pass `true` to capture all inputs as-is, or an options object for fine-grained control:

| Sub-option | Type | Description |
|------------|------|-------------|
| `maxLength` | `number` | Truncate stringified inputs exceeding this character length (appends `…`) |
| `transform` | `(input, toolName) => unknown` | Custom transform applied before `maxLength`. Use to redact fields or reshape data. |

```typescript
// Capture everything
const ai = createAILogger(log, { toolInputs: true })

// Truncate long inputs (e.g. SQL queries)
const ai = createAILogger(log, { toolInputs: { maxLength: 200 } })

// Redact sensitive tool inputs
const ai = createAILogger(log, {
toolInputs: {
maxLength: 500,
transform: (input, toolName) => {
if (toolName === 'queryDB') return { sql: '***' }
return input
},
},
})
```

## Usage Patterns

### streamText
Expand Down Expand Up @@ -182,7 +214,9 @@ import { createAILogger } from 'evlog/ai'

export default defineEventHandler(async (event) => {
const log = useLogger(event)
const ai = createAILogger(log)
const ai = createAILogger(log, {
toolInputs: { maxLength: 500 },
})

const agent = new ToolLoopAgent({
model: ai.wrap('anthropic/claude-sonnet-4.6'),
Expand Down Expand Up @@ -210,7 +244,17 @@ Wide event after a 3-step agent run:
"outputTokens": 1200,
"totalTokens": 5700,
"finishReason": "stop",
"toolCalls": ["searchWeb", "queryDatabase", "searchWeb"],
"toolCalls": [
{ "name": "searchWeb", "input": { "query": "TypeScript 6.0 features" } },
{ "name": "queryDatabase", "input": { "sql": "SELECT * FROM docs WHERE topic = 'typescript'" } },
{ "name": "searchWeb", "input": { "query": "TypeScript 6.0 release date" } }
],
"responseId": "msg_01XFDUDYJgAACzvnptvVoYEL",
"stepsUsage": [
{ "model": "claude-sonnet-4.6", "inputTokens": 1200, "outputTokens": 300, "toolCalls": ["searchWeb"] },
{ "model": "claude-sonnet-4.6", "inputTokens": 1500, "outputTokens": 400, "toolCalls": ["queryDatabase", "searchWeb"] },
{ "model": "claude-sonnet-4.6", "inputTokens": 1800, "outputTokens": 500 }
],
"msToFirstChunk": 312,
"msToFinish": 8200,
"tokensPerSecond": 146
Expand Down Expand Up @@ -302,13 +346,42 @@ const model = ai.wrap(anthropic('claude-sonnet-4.6'))
| `ai.cacheWriteTokens` | `usage.inputTokens.cacheWrite` | Tokens written to prompt cache |
| `ai.reasoningTokens` | `usage.outputTokens.reasoning` | Reasoning tokens (extended thinking) |
| `ai.finishReason` | `finishReason.unified` | Why generation ended (`stop`, `tool-calls`, etc.) |
| `ai.toolCalls` | Content / stream chunks | List of tool names called |
| `ai.toolCalls` | Content / stream chunks | `string[]` of tool names by default, or `Array<{ name, input }>` when `toolInputs` is enabled |
| `ai.responseId` | `response.id` | Provider-assigned response ID (e.g. Anthropic's `msg_...`) |
| `ai.steps` | Step count | Number of LLM calls (only when > 1) |
| `ai.stepsUsage` | Per-step accumulation | Per-step token and tool call breakdown (only when > 1 step) |
| `ai.msToFirstChunk` | Stream timing | Time to first text chunk (streaming only) |
| `ai.msToFinish` | Stream timing | Total stream duration (streaming only) |
| `ai.tokensPerSecond` | Computed | Output tokens per second (streaming only) |
| `ai.error` | Error capture | Error message if a model call fails |

## Composability

`ai.wrap()` works with models that are already wrapped by other tools. If you use supermemory, guardrails middleware, or any other model wrapper, pass the wrapped model to `ai.wrap()`:

```typescript
import { createAILogger } from 'evlog/ai'
import { withSupermemory } from '@supermemory/tools/ai-sdk'

const ai = createAILogger(log)
const base = gateway('anthropic/claude-sonnet-4.6')
const model = ai.wrap(withSupermemory(base, orgId, { mode: 'full' }))
```

For explicit middleware composition, use `createAIMiddleware` to get the raw middleware and compose it yourself via `wrapLanguageModel`:

```typescript
import { createAIMiddleware } from 'evlog/ai'
import { wrapLanguageModel } from 'ai'

const model = wrapLanguageModel({
model: base,
middleware: [createAIMiddleware(log, { toolInputs: true }), otherMiddleware],
})
```

`createAIMiddleware` returns the same middleware that `createAILogger` uses internally. The difference: `createAIMiddleware` does not include `captureEmbed` (embedding models don't use middleware). Use `createAILogger` for the full API, `createAIMiddleware` when you need explicit middleware ordering.

## Error Handling

If a model call fails, the middleware captures the error into the wide event before re-throwing:
Expand Down
5 changes: 4 additions & 1 deletion apps/nuxthub-playground/app/components/LogGenerator.vue
Original file line number Diff line number Diff line change
Expand Up @@ -32,11 +32,14 @@ async function fireAll() {
<button @click="fire('/api/test/warn')">
Slow Request
</button>
<button @click="fire('/api/test/ai-wrap')">
AI Wrap Composition
</button>
<button @click="fireAll">
Fire All (x3)
</button>
</div>
<p v-if="lastResult" style="margin-top: 0.5rem; color: #666; font-size: 0.85rem;">
<p v-if="lastResult" style="margin-top: 0.5rem; color: #666; font-size: 0.85rem; word-break: break-all; overflow-wrap: anywhere;">
{{ lastResult }}
</p>
</section>
Expand Down
2 changes: 1 addition & 1 deletion apps/nuxthub-playground/server/api/chat.post.ts
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,7 @@ export default defineEventHandler(async (event) => {

logger.set({ action: 'chat', messagesCount: messages.length })

const ai = createAILogger(logger)
const ai = createAILogger(logger, { toolInputs: true })

try {
const agent = new ToolLoopAgent({
Expand Down
45 changes: 45 additions & 0 deletions apps/nuxthub-playground/server/api/test/ai-wrap.get.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
import { gateway, generateText, wrapLanguageModel } from 'ai'
import type { LanguageModelV3Middleware } from '@ai-sdk/provider'
import { createAILogger } from 'evlog/ai'

/**
* Simulates an external middleware (supermemory, guardrails, etc.)
* that injects a system message — proves the middleware actually ran in the chain.
*/
const externalMiddleware: LanguageModelV3Middleware = {
specificationVersion: 'v3',
transformParams({ params }) {
return Promise.resolve({
...params,
prompt: [
{ role: 'system' as const, content: 'Always start your answer with "MIDDLEWARE_OK:"' },
...params.prompt,
],
})
},
}

export default defineEventHandler(async (event) => {
const logger = useLogger(event)
logger.set({ action: 'test-ai-wrap-composition' })

const ai = createAILogger(logger, { toolInputs: true })

const base = gateway('google/gemini-3-flash')
const preWrapped = wrapLanguageModel({ model: base, middleware: externalMiddleware })
const model = ai.wrap(preWrapped)

const result = await generateText({
model,
prompt: 'Say hello.',
maxOutputTokens: 200,
})

const middlewareRan = result.text.startsWith('MIDDLEWARE_OK:')

return {
status: 'ok',
middlewareRan,
text: result.text,
}
})
2 changes: 1 addition & 1 deletion apps/playground/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -12,9 +12,9 @@
"typecheck": "vue-tsc --noEmit"
},
"dependencies": {
"@nuxt/ui": "^4.5.1",
"evlog": "workspace:*",
"nuxt": "^4.4.2",
"@nuxt/ui": "^4.5.1",
"tailwindcss": "^4.2.1"
}
}
Loading
Loading