Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions .release-intents/20260825-cli-vercel-instrumentation-fix.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
{
"summary": "Fix CLI tracing onboarding so Vercel AI SDK projects install and activate the explicit Vercel instrumentation, enable AI SDK telemetry, and retain the behavior in the packaged skill.",
"packages": {
"@respan/cli": "patch"
}
}
66 changes: 52 additions & 14 deletions claude-plugin/skills/respan/references/tracing.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,20 +30,20 @@ The API key is stored in `.env` as `RESPAN_API_KEY`.

**Before analyzing anything, ask the user which setup they want.** This is the first question — the two branches diverge immediately.

1. **Auto** — the fastest path. Install the core SDK and add `Respan()` (one line). Every LLM call from a supported direct SDK — OpenAI, Anthropic, Azure OpenAI, Bedrock, Vertex, Cohere, Together, Gemini, LiteLLM — is automatically captured as a flat span. **No decorators and no framework instrumentor, even if a framework is detected.** Best for a quick start or a live demo: traces flowing in under two minutes.
1. **Auto** — the fastest working path. For a supported direct SDK, install the core SDK and add `Respan()` (one line) to capture flat LLM spans. The Vercel AI SDK is an explicit-only exception: Auto must also add `@respan/instrumentation-vercel` and `VercelAIInstrumentor`, but still adds no decorators or manual workflow wrappers. Best for a quick start or a live demo: traces flowing in under two minutes.
2. **Full** — structured setup. Adds framework-specific instrumentation and/or workflow structure on top:
- **Explicit instrumentor** — for agent frameworks (LangChain, CrewAI, OpenAI Agents, Claude Agent SDK, LlamaIndex, Haystack, …) so the framework's agent / tool / chain structure is captured.
- **Decorators** — wrap the user's own functions with `@workflow` / `@task` for nested spans.

Follow only the branch the user picks; skip the other one.

> **Frameworks under Auto:** Auto never installs a framework instrumentor. If the app uses a framework (LangChain, CrewAI, an agents SDK) and the user picks Auto, they get only the flat LLM spans the underlying direct SDK emits — no agent / tool / chain structure. Capturing framework structure requires Full.
> **Frameworks under Auto:** Auto normally does not install a framework instrumentor. If the app uses a framework (LangChain, CrewAI, an agents SDK), Auto captures only spans from a supported underlying direct SDK. **Direct Vercel AI SDK usage is different:** the Respan facade deliberately does not auto-discover it, and AI SDK calls are not covered by core-only setup. If the project directly calls the `ai` package and no higher-level framework owns its AI SDK telemetry, follow the Vercel branch below in both Auto and Full.

---

### Auto path

**A1. Detect language and package manager** — `package.json` (JS/TS) or `pyproject.toml` / `requirements.txt` (Python).
**A1. Detect language, package manager, and integration** — inspect `package.json` and source imports for JS/TS, or `pyproject.toml` / `requirements.txt` for Python. Before choosing core-only initialization, check whether the JS/TS project directly imports and calls the `ai` package (Vercel AI SDK). Apply the priority rule first: if a higher-level integration owns AI SDK telemetry, use that integration instead of also adding Vercel instrumentation. For example, an Eve app must use `EveInstrumentor`; do not activate `VercelAIInstrumentor` or another `@ai-sdk/otel` bridge alongside it.

**A2. Install the core SDK** (pin the exact current version):
```bash
Expand All @@ -54,6 +54,12 @@ pip install respan-ai
npm install @respan/respan
```

If the project uses Vercel AI SDK directly, fetch and follow its [integration doc](https://respan.ai/docs/integrations/vercel-ai-sdk.md), and pin exact compatible versions of:

- `@respan/respan`
- `@respan/instrumentation-vercel`
- `@ai-sdk/otel` when the installed `ai` major version is 7 (AI SDK 4 through 6 do not require this adapter)

**A3. Add initialization code** at the top of the entrypoint, before any LLM client is created:
```python
# Python
Expand All @@ -67,7 +73,33 @@ const respan = new Respan();
await respan.initialize();
```

Do **not** add decorators or a framework instrumentor. Go straight to **Verify**.
For a Vercel AI SDK project, use its required explicit instrumentor instead of the core-only TypeScript initialization above:

```typescript
import { Respan } from "@respan/respan";
import { VercelAIInstrumentor } from "@respan/instrumentation-vercel";

const respan = new Respan({
instrumentations: [new VercelAIInstrumentor()],
});
await respan.initialize();
```

If the project already has a `Respan` instance, merge `VercelAIInstrumentor` into its existing `instrumentations` list instead of creating a second instance. Preserve other explicit instrumentors that cover genuinely separate direct-SDK calls; do not add a provider instrumentor solely for calls made through Vercel AI SDK.

If the application already registers its own compatible `@ai-sdk/otel` integration, construct `VercelAIInstrumentor({ autoRegisterAISDKTelemetry: false })` so Respan translates the spans without claiming a second registration. For Next.js, follow the integration doc's root `instrumentation.ts` `register()` placement and `serverExternalPackages` configuration; keep tracing on the server runtime, not an Edge or browser path.

Also enable Vercel telemetry on every AI SDK operation. Use the option that matches the installed AI SDK major version:

```typescript
// AI SDK 7
telemetry: { isEnabled: true }

// AI SDK 4 through 6
experimental_telemetry: { isEnabled: true }
```

Do **not** add decorators. For non-Vercel Auto setups, do not add a framework instrumentor. For Vercel, keep the required `VercelAIInstrumentor` and go straight to **Verify**.

---

Expand Down Expand Up @@ -103,23 +135,29 @@ Check higher-priority categories first. If a match is found, use that instrument

If a Priority 1 framework is found, use its instrumentation. Do NOT also add Priority 2 instrumentation for the same provider.

For direct Vercel AI SDK usage, the explicit `VercelAIInstrumentor` and per-operation telemetry option are required in both Auto and Full. Never replace them with core-only `new Respan()` setup, and never stack them with a higher-level integration that already owns AI SDK telemetry.

**LangChain / LangGraph setup differs by language** (one package covers both frameworks: `respan-instrumentation-langchain` / `@respan/instrumentation-langchain`). In **Python**, pass `LangChainInstrumentor()` to `Respan(instrumentations=[...])` — it patches the LangChain and LangGraph callback managers globally on init, so every chain, graph, node, LLM, tool, and retriever run is traced **automatically with no per-call setup**. `add_respan_callback(config=...)` is optional and only labels a run with a name, tags, or metadata. In **TypeScript**, create a `LangChainInstrumentor`, pass it to `new Respan({ instrumentations: [...] })`, **and** attach `instrumentor.addCallback(...)` to the **outermost** chain or graph invocation (`.invoke()` / `.stream()`, not inside a node — LangChain/LangGraph propagate it to nested runs); the TypeScript instrumentor does **not** patch globally, so without the callback no spans are emitted. (The other Priority-1 frameworks use the standard `Respan(instrumentations=[XInstrumentor()])` plugin pattern.)

**Priority 2 — Direct LLM SDKs** (only if no P1 framework covers this provider):

These are **auto-instrumented** — just `Respan()` / `new Respan()`, no extra packages needed:
The integrations listed for each language are **auto-instrumented** with `Respan()` / `new Respan()`. The application still provides the provider SDK; the matching first-party instrumentation adapter is bundled by the Respan facade. A dash means that language currently requires an explicit instrumentation package and instrumentor.

| Library | Python package | JS/TS package | Docs |
|---------|---------------|---------------|------|
| Library | Python SDK (auto) | JS/TS SDK (auto) | Docs |
|---------|-------------------|------------------|------|
| OpenAI SDK | `openai` | `openai` | [docs](https://respan.ai/docs/integrations/openai-sdk.md) |
| Anthropic SDK | `anthropic` | `@anthropic-ai/sdk` | [docs](https://respan.ai/docs/integrations/anthropic.md) |
| Azure OpenAI | `openai` (azure config) | `openai` | [docs](https://respan.ai/docs/integrations/providers/azure.md) |
| Google Vertex AI | `google-cloud-aiplatform` | — | [docs](https://respan.ai/docs/integrations/vertex-ai.md) |
| AWS Bedrock | `boto3` | — | [docs](https://respan.ai/docs/integrations/aws-bedrock.md) |
| Cohere | `cohere` | — | [docs](https://respan.ai/docs/integrations/providers/cohere.md) |
| Together AI | `together` | — | [docs](https://respan.ai/docs/integrations/together-ai.md) |

**Note:** LiteLLM in JS uses the OpenAI-compatible API, so the OpenAI auto-instrument covers it. For Python LiteLLM, see [LiteLLM guide](https://respan.ai/docs/integrations/litellm.md). For Google GenAI (`@google/genai`), see [Google GenAI guide](https://respan.ai/docs/integrations/google-genai.md).
| Azure OpenAI | `openai` (Azure config) | `openai` (Azure client) | [docs](https://respan.ai/docs/integrations/providers/azure.md) |
| Google Vertex AI | `google-cloud-aiplatform` | `@google-cloud/vertexai` | [docs](https://respan.ai/docs/integrations/vertex-ai.md) |
| Google GenAI | `google-genai` | — | [docs](https://respan.ai/docs/integrations/google-genai.md) |
| AWS Bedrock | `boto3` | `@aws-sdk/client-bedrock-runtime` | [docs](https://respan.ai/docs/integrations/aws-bedrock.md) |
| Cohere | — | `cohere-ai` | [docs](https://respan.ai/docs/integrations/providers/cohere.md) |
| Together AI | `together` | `together-ai` | [docs](https://respan.ai/docs/integrations/together-ai.md) |
| OpenRouter | — | `@openrouter/sdk` | [docs](https://respan.ai/docs/integrations/providers/openrouter.md) |
| Writer | — | `writer-sdk` | [docs](https://respan.ai/docs/integrations/writer.md) |
| Ollama | `ollama` | — | [docs](https://respan.ai/docs/integrations/ollama.md) |

**Note:** LLM wrappers such as Python LiteLLM stay explicit-only to avoid overlapping provider spans. See the [LiteLLM guide](https://respan.ai/docs/integrations/litellm.md) for its explicit setup.

**1c. Read the actual code and understand the workflow:**

Expand Down
66 changes: 52 additions & 14 deletions cursor-plugin/skills/respan/references/tracing.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,20 +30,20 @@ The API key is stored in `.env` as `RESPAN_API_KEY`.

**Before analyzing anything, ask the user which setup they want.** This is the first question — the two branches diverge immediately.

1. **Auto** — the fastest path. Install the core SDK and add `Respan()` (one line). Every LLM call from a supported direct SDK — OpenAI, Anthropic, Azure OpenAI, Bedrock, Vertex, Cohere, Together, Gemini, LiteLLM — is automatically captured as a flat span. **No decorators and no framework instrumentor, even if a framework is detected.** Best for a quick start or a live demo: traces flowing in under two minutes.
1. **Auto** — the fastest working path. For a supported direct SDK, install the core SDK and add `Respan()` (one line) to capture flat LLM spans. The Vercel AI SDK is an explicit-only exception: Auto must also add `@respan/instrumentation-vercel` and `VercelAIInstrumentor`, but still adds no decorators or manual workflow wrappers. Best for a quick start or a live demo: traces flowing in under two minutes.
2. **Full** — structured setup. Adds framework-specific instrumentation and/or workflow structure on top:
- **Explicit instrumentor** — for agent frameworks (LangChain, CrewAI, OpenAI Agents, Claude Agent SDK, LlamaIndex, Haystack, …) so the framework's agent / tool / chain structure is captured.
- **Decorators** — wrap the user's own functions with `@workflow` / `@task` for nested spans.

Follow only the branch the user picks; skip the other one.

> **Frameworks under Auto:** Auto never installs a framework instrumentor. If the app uses a framework (LangChain, CrewAI, an agents SDK) and the user picks Auto, they get only the flat LLM spans the underlying direct SDK emits — no agent / tool / chain structure. Capturing framework structure requires Full.
> **Frameworks under Auto:** Auto normally does not install a framework instrumentor. If the app uses a framework (LangChain, CrewAI, an agents SDK), Auto captures only spans from a supported underlying direct SDK. **Direct Vercel AI SDK usage is different:** the Respan facade deliberately does not auto-discover it, and AI SDK calls are not covered by core-only setup. If the project directly calls the `ai` package and no higher-level framework owns its AI SDK telemetry, follow the Vercel branch below in both Auto and Full.

---

### Auto path

**A1. Detect language and package manager** — `package.json` (JS/TS) or `pyproject.toml` / `requirements.txt` (Python).
**A1. Detect language, package manager, and integration** — inspect `package.json` and source imports for JS/TS, or `pyproject.toml` / `requirements.txt` for Python. Before choosing core-only initialization, check whether the JS/TS project directly imports and calls the `ai` package (Vercel AI SDK). Apply the priority rule first: if a higher-level integration owns AI SDK telemetry, use that integration instead of also adding Vercel instrumentation. For example, an Eve app must use `EveInstrumentor`; do not activate `VercelAIInstrumentor` or another `@ai-sdk/otel` bridge alongside it.

**A2. Install the core SDK** (pin the exact current version):
```bash
Expand All @@ -54,6 +54,12 @@ pip install respan-ai
npm install @respan/respan
```

If the project uses Vercel AI SDK directly, fetch and follow its [integration doc](https://respan.ai/docs/integrations/vercel-ai-sdk.md), and pin exact compatible versions of:

- `@respan/respan`
- `@respan/instrumentation-vercel`
- `@ai-sdk/otel` when the installed `ai` major version is 7 (AI SDK 4 through 6 do not require this adapter)

**A3. Add initialization code** at the top of the entrypoint, before any LLM client is created:
```python
# Python
Expand All @@ -67,7 +73,33 @@ const respan = new Respan();
await respan.initialize();
```

Do **not** add decorators or a framework instrumentor. Go straight to **Verify**.
For a Vercel AI SDK project, use its required explicit instrumentor instead of the core-only TypeScript initialization above:

```typescript
import { Respan } from "@respan/respan";
import { VercelAIInstrumentor } from "@respan/instrumentation-vercel";

const respan = new Respan({
instrumentations: [new VercelAIInstrumentor()],
});
await respan.initialize();
```

If the project already has a `Respan` instance, merge `VercelAIInstrumentor` into its existing `instrumentations` list instead of creating a second instance. Preserve other explicit instrumentors that cover genuinely separate direct-SDK calls; do not add a provider instrumentor solely for calls made through Vercel AI SDK.

If the application already registers its own compatible `@ai-sdk/otel` integration, construct `VercelAIInstrumentor({ autoRegisterAISDKTelemetry: false })` so Respan translates the spans without claiming a second registration. For Next.js, follow the integration doc's root `instrumentation.ts` `register()` placement and `serverExternalPackages` configuration; keep tracing on the server runtime, not an Edge or browser path.

Also enable Vercel telemetry on every AI SDK operation. Use the option that matches the installed AI SDK major version:

```typescript
// AI SDK 7
telemetry: { isEnabled: true }

// AI SDK 4 through 6
experimental_telemetry: { isEnabled: true }
```

Do **not** add decorators. For non-Vercel Auto setups, do not add a framework instrumentor. For Vercel, keep the required `VercelAIInstrumentor` and go straight to **Verify**.

---

Expand Down Expand Up @@ -103,23 +135,29 @@ Check higher-priority categories first. If a match is found, use that instrument

If a Priority 1 framework is found, use its instrumentation. Do NOT also add Priority 2 instrumentation for the same provider.

For direct Vercel AI SDK usage, the explicit `VercelAIInstrumentor` and per-operation telemetry option are required in both Auto and Full. Never replace them with core-only `new Respan()` setup, and never stack them with a higher-level integration that already owns AI SDK telemetry.

**LangChain / LangGraph setup differs by language** (one package covers both frameworks: `respan-instrumentation-langchain` / `@respan/instrumentation-langchain`). In **Python**, pass `LangChainInstrumentor()` to `Respan(instrumentations=[...])` — it patches the LangChain and LangGraph callback managers globally on init, so every chain, graph, node, LLM, tool, and retriever run is traced **automatically with no per-call setup**. `add_respan_callback(config=...)` is optional and only labels a run with a name, tags, or metadata. In **TypeScript**, create a `LangChainInstrumentor`, pass it to `new Respan({ instrumentations: [...] })`, **and** attach `instrumentor.addCallback(...)` to the **outermost** chain or graph invocation (`.invoke()` / `.stream()`, not inside a node — LangChain/LangGraph propagate it to nested runs); the TypeScript instrumentor does **not** patch globally, so without the callback no spans are emitted. (The other Priority-1 frameworks use the standard `Respan(instrumentations=[XInstrumentor()])` plugin pattern.)

**Priority 2 — Direct LLM SDKs** (only if no P1 framework covers this provider):

These are **auto-instrumented** — just `Respan()` / `new Respan()`, no extra packages needed:
The integrations listed for each language are **auto-instrumented** with `Respan()` / `new Respan()`. The application still provides the provider SDK; the matching first-party instrumentation adapter is bundled by the Respan facade. A dash means that language currently requires an explicit instrumentation package and instrumentor.

| Library | Python package | JS/TS package | Docs |
|---------|---------------|---------------|------|
| Library | Python SDK (auto) | JS/TS SDK (auto) | Docs |
|---------|-------------------|------------------|------|
| OpenAI SDK | `openai` | `openai` | [docs](https://respan.ai/docs/integrations/openai-sdk.md) |
| Anthropic SDK | `anthropic` | `@anthropic-ai/sdk` | [docs](https://respan.ai/docs/integrations/anthropic.md) |
| Azure OpenAI | `openai` (azure config) | `openai` | [docs](https://respan.ai/docs/integrations/providers/azure.md) |
| Google Vertex AI | `google-cloud-aiplatform` | — | [docs](https://respan.ai/docs/integrations/vertex-ai.md) |
| AWS Bedrock | `boto3` | — | [docs](https://respan.ai/docs/integrations/aws-bedrock.md) |
| Cohere | `cohere` | — | [docs](https://respan.ai/docs/integrations/providers/cohere.md) |
| Together AI | `together` | — | [docs](https://respan.ai/docs/integrations/together-ai.md) |

**Note:** LiteLLM in JS uses the OpenAI-compatible API, so the OpenAI auto-instrument covers it. For Python LiteLLM, see [LiteLLM guide](https://respan.ai/docs/integrations/litellm.md). For Google GenAI (`@google/genai`), see [Google GenAI guide](https://respan.ai/docs/integrations/google-genai.md).
| Azure OpenAI | `openai` (Azure config) | `openai` (Azure client) | [docs](https://respan.ai/docs/integrations/providers/azure.md) |
| Google Vertex AI | `google-cloud-aiplatform` | `@google-cloud/vertexai` | [docs](https://respan.ai/docs/integrations/vertex-ai.md) |
| Google GenAI | `google-genai` | — | [docs](https://respan.ai/docs/integrations/google-genai.md) |
| AWS Bedrock | `boto3` | `@aws-sdk/client-bedrock-runtime` | [docs](https://respan.ai/docs/integrations/aws-bedrock.md) |
| Cohere | — | `cohere-ai` | [docs](https://respan.ai/docs/integrations/providers/cohere.md) |
| Together AI | `together` | `together-ai` | [docs](https://respan.ai/docs/integrations/together-ai.md) |
| OpenRouter | — | `@openrouter/sdk` | [docs](https://respan.ai/docs/integrations/providers/openrouter.md) |
| Writer | — | `writer-sdk` | [docs](https://respan.ai/docs/integrations/writer.md) |
| Ollama | `ollama` | — | [docs](https://respan.ai/docs/integrations/ollama.md) |

**Note:** LLM wrappers such as Python LiteLLM stay explicit-only to avoid overlapping provider spans. See the [LiteLLM guide](https://respan.ai/docs/integrations/litellm.md) for its explicit setup.

**1c. Read the actual code and understand the workflow:**

Expand Down
Loading
Loading