From 437b2425ff5da56fb136d5150b23871e75a50c40 Mon Sep 17 00:00:00 2001 From: Nightingalelyy Date: Tue, 25 Aug 2026 01:50:05 +0800 Subject: [PATCH] fix(cli): configure Vercel AI SDK instrumentation in Auto setup --- ...260825-cli-vercel-instrumentation-fix.json | 6 ++ .../skills/respan/references/tracing.md | 66 +++++++++++++++---- .../skills/respan/references/tracing.md | 66 +++++++++++++++---- .../src/lib/skill-refs.generated.ts | 66 +++++++++++++++---- .../respan-cli/tests/tracing-skill.test.mjs | 18 +++++ skills/references/tracing.md | 42 ++++++++++-- 6 files changed, 218 insertions(+), 46 deletions(-) create mode 100644 .release-intents/20260825-cli-vercel-instrumentation-fix.json create mode 100644 javascript-sdks/respan-cli/tests/tracing-skill.test.mjs diff --git a/.release-intents/20260825-cli-vercel-instrumentation-fix.json b/.release-intents/20260825-cli-vercel-instrumentation-fix.json new file mode 100644 index 00000000..038eff95 --- /dev/null +++ b/.release-intents/20260825-cli-vercel-instrumentation-fix.json @@ -0,0 +1,6 @@ +{ + "summary": "Fix CLI tracing onboarding so Vercel AI SDK projects install and activate the explicit Vercel instrumentation, enable AI SDK telemetry, and retain the behavior in the packaged skill.", + "packages": { + "@respan/cli": "patch" + } +} diff --git a/claude-plugin/skills/respan/references/tracing.md b/claude-plugin/skills/respan/references/tracing.md index d32c535b..173500da 100644 --- a/claude-plugin/skills/respan/references/tracing.md +++ b/claude-plugin/skills/respan/references/tracing.md @@ -30,20 +30,20 @@ The API key is stored in `.env` as `RESPAN_API_KEY`. **Before analyzing anything, ask the user which setup they want.** This is the first question — the two branches diverge immediately. -1. **Auto** — the fastest path. Install the core SDK and add `Respan()` (one line). Every LLM call from a supported direct SDK — OpenAI, Anthropic, Azure OpenAI, Bedrock, Vertex, Cohere, Together, Gemini, LiteLLM — is automatically captured as a flat span. **No decorators and no framework instrumentor, even if a framework is detected.** Best for a quick start or a live demo: traces flowing in under two minutes. +1. **Auto** — the fastest working path. For a supported direct SDK, install the core SDK and add `Respan()` (one line) to capture flat LLM spans. The Vercel AI SDK is an explicit-only exception: Auto must also add `@respan/instrumentation-vercel` and `VercelAIInstrumentor`, but still adds no decorators or manual workflow wrappers. Best for a quick start or a live demo: traces flowing in under two minutes. 2. **Full** — structured setup. Adds framework-specific instrumentation and/or workflow structure on top: - **Explicit instrumentor** — for agent frameworks (LangChain, CrewAI, OpenAI Agents, Claude Agent SDK, LlamaIndex, Haystack, …) so the framework's agent / tool / chain structure is captured. - **Decorators** — wrap the user's own functions with `@workflow` / `@task` for nested spans. Follow only the branch the user picks; skip the other one. -> **Frameworks under Auto:** Auto never installs a framework instrumentor. If the app uses a framework (LangChain, CrewAI, an agents SDK) and the user picks Auto, they get only the flat LLM spans the underlying direct SDK emits — no agent / tool / chain structure. Capturing framework structure requires Full. +> **Frameworks under Auto:** Auto normally does not install a framework instrumentor. If the app uses a framework (LangChain, CrewAI, an agents SDK), Auto captures only spans from a supported underlying direct SDK. **Direct Vercel AI SDK usage is different:** the Respan facade deliberately does not auto-discover it, and AI SDK calls are not covered by core-only setup. If the project directly calls the `ai` package and no higher-level framework owns its AI SDK telemetry, follow the Vercel branch below in both Auto and Full. --- ### Auto path -**A1. Detect language and package manager** — `package.json` (JS/TS) or `pyproject.toml` / `requirements.txt` (Python). +**A1. Detect language, package manager, and integration** — inspect `package.json` and source imports for JS/TS, or `pyproject.toml` / `requirements.txt` for Python. Before choosing core-only initialization, check whether the JS/TS project directly imports and calls the `ai` package (Vercel AI SDK). Apply the priority rule first: if a higher-level integration owns AI SDK telemetry, use that integration instead of also adding Vercel instrumentation. For example, an Eve app must use `EveInstrumentor`; do not activate `VercelAIInstrumentor` or another `@ai-sdk/otel` bridge alongside it. **A2. Install the core SDK** (pin the exact current version): ```bash @@ -54,6 +54,12 @@ pip install respan-ai npm install @respan/respan ``` +If the project uses Vercel AI SDK directly, fetch and follow its [integration doc](https://respan.ai/docs/integrations/vercel-ai-sdk.md), and pin exact compatible versions of: + +- `@respan/respan` +- `@respan/instrumentation-vercel` +- `@ai-sdk/otel` when the installed `ai` major version is 7 (AI SDK 4 through 6 do not require this adapter) + **A3. Add initialization code** at the top of the entrypoint, before any LLM client is created: ```python # Python @@ -67,7 +73,33 @@ const respan = new Respan(); await respan.initialize(); ``` -Do **not** add decorators or a framework instrumentor. Go straight to **Verify**. +For a Vercel AI SDK project, use its required explicit instrumentor instead of the core-only TypeScript initialization above: + +```typescript +import { Respan } from "@respan/respan"; +import { VercelAIInstrumentor } from "@respan/instrumentation-vercel"; + +const respan = new Respan({ + instrumentations: [new VercelAIInstrumentor()], +}); +await respan.initialize(); +``` + +If the project already has a `Respan` instance, merge `VercelAIInstrumentor` into its existing `instrumentations` list instead of creating a second instance. Preserve other explicit instrumentors that cover genuinely separate direct-SDK calls; do not add a provider instrumentor solely for calls made through Vercel AI SDK. + +If the application already registers its own compatible `@ai-sdk/otel` integration, construct `VercelAIInstrumentor({ autoRegisterAISDKTelemetry: false })` so Respan translates the spans without claiming a second registration. For Next.js, follow the integration doc's root `instrumentation.ts` `register()` placement and `serverExternalPackages` configuration; keep tracing on the server runtime, not an Edge or browser path. + +Also enable Vercel telemetry on every AI SDK operation. Use the option that matches the installed AI SDK major version: + +```typescript +// AI SDK 7 +telemetry: { isEnabled: true } + +// AI SDK 4 through 6 +experimental_telemetry: { isEnabled: true } +``` + +Do **not** add decorators. For non-Vercel Auto setups, do not add a framework instrumentor. For Vercel, keep the required `VercelAIInstrumentor` and go straight to **Verify**. --- @@ -103,23 +135,29 @@ Check higher-priority categories first. If a match is found, use that instrument If a Priority 1 framework is found, use its instrumentation. Do NOT also add Priority 2 instrumentation for the same provider. +For direct Vercel AI SDK usage, the explicit `VercelAIInstrumentor` and per-operation telemetry option are required in both Auto and Full. Never replace them with core-only `new Respan()` setup, and never stack them with a higher-level integration that already owns AI SDK telemetry. + **LangChain / LangGraph setup differs by language** (one package covers both frameworks: `respan-instrumentation-langchain` / `@respan/instrumentation-langchain`). In **Python**, pass `LangChainInstrumentor()` to `Respan(instrumentations=[...])` — it patches the LangChain and LangGraph callback managers globally on init, so every chain, graph, node, LLM, tool, and retriever run is traced **automatically with no per-call setup**. `add_respan_callback(config=...)` is optional and only labels a run with a name, tags, or metadata. In **TypeScript**, create a `LangChainInstrumentor`, pass it to `new Respan({ instrumentations: [...] })`, **and** attach `instrumentor.addCallback(...)` to the **outermost** chain or graph invocation (`.invoke()` / `.stream()`, not inside a node — LangChain/LangGraph propagate it to nested runs); the TypeScript instrumentor does **not** patch globally, so without the callback no spans are emitted. (The other Priority-1 frameworks use the standard `Respan(instrumentations=[XInstrumentor()])` plugin pattern.) **Priority 2 — Direct LLM SDKs** (only if no P1 framework covers this provider): -These are **auto-instrumented** — just `Respan()` / `new Respan()`, no extra packages needed: +The integrations listed for each language are **auto-instrumented** with `Respan()` / `new Respan()`. The application still provides the provider SDK; the matching first-party instrumentation adapter is bundled by the Respan facade. A dash means that language currently requires an explicit instrumentation package and instrumentor. -| Library | Python package | JS/TS package | Docs | -|---------|---------------|---------------|------| +| Library | Python SDK (auto) | JS/TS SDK (auto) | Docs | +|---------|-------------------|------------------|------| | OpenAI SDK | `openai` | `openai` | [docs](https://respan.ai/docs/integrations/openai-sdk.md) | | Anthropic SDK | `anthropic` | `@anthropic-ai/sdk` | [docs](https://respan.ai/docs/integrations/anthropic.md) | -| Azure OpenAI | `openai` (azure config) | `openai` | [docs](https://respan.ai/docs/integrations/providers/azure.md) | -| Google Vertex AI | `google-cloud-aiplatform` | — | [docs](https://respan.ai/docs/integrations/vertex-ai.md) | -| AWS Bedrock | `boto3` | — | [docs](https://respan.ai/docs/integrations/aws-bedrock.md) | -| Cohere | `cohere` | — | [docs](https://respan.ai/docs/integrations/providers/cohere.md) | -| Together AI | `together` | — | [docs](https://respan.ai/docs/integrations/together-ai.md) | - -**Note:** LiteLLM in JS uses the OpenAI-compatible API, so the OpenAI auto-instrument covers it. For Python LiteLLM, see [LiteLLM guide](https://respan.ai/docs/integrations/litellm.md). For Google GenAI (`@google/genai`), see [Google GenAI guide](https://respan.ai/docs/integrations/google-genai.md). +| Azure OpenAI | `openai` (Azure config) | `openai` (Azure client) | [docs](https://respan.ai/docs/integrations/providers/azure.md) | +| Google Vertex AI | `google-cloud-aiplatform` | `@google-cloud/vertexai` | [docs](https://respan.ai/docs/integrations/vertex-ai.md) | +| Google GenAI | `google-genai` | — | [docs](https://respan.ai/docs/integrations/google-genai.md) | +| AWS Bedrock | `boto3` | `@aws-sdk/client-bedrock-runtime` | [docs](https://respan.ai/docs/integrations/aws-bedrock.md) | +| Cohere | — | `cohere-ai` | [docs](https://respan.ai/docs/integrations/providers/cohere.md) | +| Together AI | `together` | `together-ai` | [docs](https://respan.ai/docs/integrations/together-ai.md) | +| OpenRouter | — | `@openrouter/sdk` | [docs](https://respan.ai/docs/integrations/providers/openrouter.md) | +| Writer | — | `writer-sdk` | [docs](https://respan.ai/docs/integrations/writer.md) | +| Ollama | `ollama` | — | [docs](https://respan.ai/docs/integrations/ollama.md) | + +**Note:** LLM wrappers such as Python LiteLLM stay explicit-only to avoid overlapping provider spans. See the [LiteLLM guide](https://respan.ai/docs/integrations/litellm.md) for its explicit setup. **1c. Read the actual code and understand the workflow:** diff --git a/cursor-plugin/skills/respan/references/tracing.md b/cursor-plugin/skills/respan/references/tracing.md index d32c535b..173500da 100644 --- a/cursor-plugin/skills/respan/references/tracing.md +++ b/cursor-plugin/skills/respan/references/tracing.md @@ -30,20 +30,20 @@ The API key is stored in `.env` as `RESPAN_API_KEY`. **Before analyzing anything, ask the user which setup they want.** This is the first question — the two branches diverge immediately. -1. **Auto** — the fastest path. Install the core SDK and add `Respan()` (one line). Every LLM call from a supported direct SDK — OpenAI, Anthropic, Azure OpenAI, Bedrock, Vertex, Cohere, Together, Gemini, LiteLLM — is automatically captured as a flat span. **No decorators and no framework instrumentor, even if a framework is detected.** Best for a quick start or a live demo: traces flowing in under two minutes. +1. **Auto** — the fastest working path. For a supported direct SDK, install the core SDK and add `Respan()` (one line) to capture flat LLM spans. The Vercel AI SDK is an explicit-only exception: Auto must also add `@respan/instrumentation-vercel` and `VercelAIInstrumentor`, but still adds no decorators or manual workflow wrappers. Best for a quick start or a live demo: traces flowing in under two minutes. 2. **Full** — structured setup. Adds framework-specific instrumentation and/or workflow structure on top: - **Explicit instrumentor** — for agent frameworks (LangChain, CrewAI, OpenAI Agents, Claude Agent SDK, LlamaIndex, Haystack, …) so the framework's agent / tool / chain structure is captured. - **Decorators** — wrap the user's own functions with `@workflow` / `@task` for nested spans. Follow only the branch the user picks; skip the other one. -> **Frameworks under Auto:** Auto never installs a framework instrumentor. If the app uses a framework (LangChain, CrewAI, an agents SDK) and the user picks Auto, they get only the flat LLM spans the underlying direct SDK emits — no agent / tool / chain structure. Capturing framework structure requires Full. +> **Frameworks under Auto:** Auto normally does not install a framework instrumentor. If the app uses a framework (LangChain, CrewAI, an agents SDK), Auto captures only spans from a supported underlying direct SDK. **Direct Vercel AI SDK usage is different:** the Respan facade deliberately does not auto-discover it, and AI SDK calls are not covered by core-only setup. If the project directly calls the `ai` package and no higher-level framework owns its AI SDK telemetry, follow the Vercel branch below in both Auto and Full. --- ### Auto path -**A1. Detect language and package manager** — `package.json` (JS/TS) or `pyproject.toml` / `requirements.txt` (Python). +**A1. Detect language, package manager, and integration** — inspect `package.json` and source imports for JS/TS, or `pyproject.toml` / `requirements.txt` for Python. Before choosing core-only initialization, check whether the JS/TS project directly imports and calls the `ai` package (Vercel AI SDK). Apply the priority rule first: if a higher-level integration owns AI SDK telemetry, use that integration instead of also adding Vercel instrumentation. For example, an Eve app must use `EveInstrumentor`; do not activate `VercelAIInstrumentor` or another `@ai-sdk/otel` bridge alongside it. **A2. Install the core SDK** (pin the exact current version): ```bash @@ -54,6 +54,12 @@ pip install respan-ai npm install @respan/respan ``` +If the project uses Vercel AI SDK directly, fetch and follow its [integration doc](https://respan.ai/docs/integrations/vercel-ai-sdk.md), and pin exact compatible versions of: + +- `@respan/respan` +- `@respan/instrumentation-vercel` +- `@ai-sdk/otel` when the installed `ai` major version is 7 (AI SDK 4 through 6 do not require this adapter) + **A3. Add initialization code** at the top of the entrypoint, before any LLM client is created: ```python # Python @@ -67,7 +73,33 @@ const respan = new Respan(); await respan.initialize(); ``` -Do **not** add decorators or a framework instrumentor. Go straight to **Verify**. +For a Vercel AI SDK project, use its required explicit instrumentor instead of the core-only TypeScript initialization above: + +```typescript +import { Respan } from "@respan/respan"; +import { VercelAIInstrumentor } from "@respan/instrumentation-vercel"; + +const respan = new Respan({ + instrumentations: [new VercelAIInstrumentor()], +}); +await respan.initialize(); +``` + +If the project already has a `Respan` instance, merge `VercelAIInstrumentor` into its existing `instrumentations` list instead of creating a second instance. Preserve other explicit instrumentors that cover genuinely separate direct-SDK calls; do not add a provider instrumentor solely for calls made through Vercel AI SDK. + +If the application already registers its own compatible `@ai-sdk/otel` integration, construct `VercelAIInstrumentor({ autoRegisterAISDKTelemetry: false })` so Respan translates the spans without claiming a second registration. For Next.js, follow the integration doc's root `instrumentation.ts` `register()` placement and `serverExternalPackages` configuration; keep tracing on the server runtime, not an Edge or browser path. + +Also enable Vercel telemetry on every AI SDK operation. Use the option that matches the installed AI SDK major version: + +```typescript +// AI SDK 7 +telemetry: { isEnabled: true } + +// AI SDK 4 through 6 +experimental_telemetry: { isEnabled: true } +``` + +Do **not** add decorators. For non-Vercel Auto setups, do not add a framework instrumentor. For Vercel, keep the required `VercelAIInstrumentor` and go straight to **Verify**. --- @@ -103,23 +135,29 @@ Check higher-priority categories first. If a match is found, use that instrument If a Priority 1 framework is found, use its instrumentation. Do NOT also add Priority 2 instrumentation for the same provider. +For direct Vercel AI SDK usage, the explicit `VercelAIInstrumentor` and per-operation telemetry option are required in both Auto and Full. Never replace them with core-only `new Respan()` setup, and never stack them with a higher-level integration that already owns AI SDK telemetry. + **LangChain / LangGraph setup differs by language** (one package covers both frameworks: `respan-instrumentation-langchain` / `@respan/instrumentation-langchain`). In **Python**, pass `LangChainInstrumentor()` to `Respan(instrumentations=[...])` — it patches the LangChain and LangGraph callback managers globally on init, so every chain, graph, node, LLM, tool, and retriever run is traced **automatically with no per-call setup**. `add_respan_callback(config=...)` is optional and only labels a run with a name, tags, or metadata. In **TypeScript**, create a `LangChainInstrumentor`, pass it to `new Respan({ instrumentations: [...] })`, **and** attach `instrumentor.addCallback(...)` to the **outermost** chain or graph invocation (`.invoke()` / `.stream()`, not inside a node — LangChain/LangGraph propagate it to nested runs); the TypeScript instrumentor does **not** patch globally, so without the callback no spans are emitted. (The other Priority-1 frameworks use the standard `Respan(instrumentations=[XInstrumentor()])` plugin pattern.) **Priority 2 — Direct LLM SDKs** (only if no P1 framework covers this provider): -These are **auto-instrumented** — just `Respan()` / `new Respan()`, no extra packages needed: +The integrations listed for each language are **auto-instrumented** with `Respan()` / `new Respan()`. The application still provides the provider SDK; the matching first-party instrumentation adapter is bundled by the Respan facade. A dash means that language currently requires an explicit instrumentation package and instrumentor. -| Library | Python package | JS/TS package | Docs | -|---------|---------------|---------------|------| +| Library | Python SDK (auto) | JS/TS SDK (auto) | Docs | +|---------|-------------------|------------------|------| | OpenAI SDK | `openai` | `openai` | [docs](https://respan.ai/docs/integrations/openai-sdk.md) | | Anthropic SDK | `anthropic` | `@anthropic-ai/sdk` | [docs](https://respan.ai/docs/integrations/anthropic.md) | -| Azure OpenAI | `openai` (azure config) | `openai` | [docs](https://respan.ai/docs/integrations/providers/azure.md) | -| Google Vertex AI | `google-cloud-aiplatform` | — | [docs](https://respan.ai/docs/integrations/vertex-ai.md) | -| AWS Bedrock | `boto3` | — | [docs](https://respan.ai/docs/integrations/aws-bedrock.md) | -| Cohere | `cohere` | — | [docs](https://respan.ai/docs/integrations/providers/cohere.md) | -| Together AI | `together` | — | [docs](https://respan.ai/docs/integrations/together-ai.md) | - -**Note:** LiteLLM in JS uses the OpenAI-compatible API, so the OpenAI auto-instrument covers it. For Python LiteLLM, see [LiteLLM guide](https://respan.ai/docs/integrations/litellm.md). For Google GenAI (`@google/genai`), see [Google GenAI guide](https://respan.ai/docs/integrations/google-genai.md). +| Azure OpenAI | `openai` (Azure config) | `openai` (Azure client) | [docs](https://respan.ai/docs/integrations/providers/azure.md) | +| Google Vertex AI | `google-cloud-aiplatform` | `@google-cloud/vertexai` | [docs](https://respan.ai/docs/integrations/vertex-ai.md) | +| Google GenAI | `google-genai` | — | [docs](https://respan.ai/docs/integrations/google-genai.md) | +| AWS Bedrock | `boto3` | `@aws-sdk/client-bedrock-runtime` | [docs](https://respan.ai/docs/integrations/aws-bedrock.md) | +| Cohere | — | `cohere-ai` | [docs](https://respan.ai/docs/integrations/providers/cohere.md) | +| Together AI | `together` | `together-ai` | [docs](https://respan.ai/docs/integrations/together-ai.md) | +| OpenRouter | — | `@openrouter/sdk` | [docs](https://respan.ai/docs/integrations/providers/openrouter.md) | +| Writer | — | `writer-sdk` | [docs](https://respan.ai/docs/integrations/writer.md) | +| Ollama | `ollama` | — | [docs](https://respan.ai/docs/integrations/ollama.md) | + +**Note:** LLM wrappers such as Python LiteLLM stay explicit-only to avoid overlapping provider spans. See the [LiteLLM guide](https://respan.ai/docs/integrations/litellm.md) for its explicit setup. **1c. Read the actual code and understand the workflow:** diff --git a/javascript-sdks/respan-cli/src/lib/skill-refs.generated.ts b/javascript-sdks/respan-cli/src/lib/skill-refs.generated.ts index cb8807e6..52387fd1 100644 --- a/javascript-sdks/respan-cli/src/lib/skill-refs.generated.ts +++ b/javascript-sdks/respan-cli/src/lib/skill-refs.generated.ts @@ -79,20 +79,20 @@ The API key is stored in \`.env\` as \`RESPAN_API_KEY\`. **Before analyzing anything, ask the user which setup they want.** This is the first question — the two branches diverge immediately. -1. **Auto** — the fastest path. Install the core SDK and add \`Respan()\` (one line). Every LLM call from a supported direct SDK — OpenAI, Anthropic, Azure OpenAI, Bedrock, Vertex, Cohere, Together, Gemini, LiteLLM — is automatically captured as a flat span. **No decorators and no framework instrumentor, even if a framework is detected.** Best for a quick start or a live demo: traces flowing in under two minutes. +1. **Auto** — the fastest working path. For a supported direct SDK, install the core SDK and add \`Respan()\` (one line) to capture flat LLM spans. The Vercel AI SDK is an explicit-only exception: Auto must also add \`@respan/instrumentation-vercel\` and \`VercelAIInstrumentor\`, but still adds no decorators or manual workflow wrappers. Best for a quick start or a live demo: traces flowing in under two minutes. 2. **Full** — structured setup. Adds framework-specific instrumentation and/or workflow structure on top: - **Explicit instrumentor** — for agent frameworks (LangChain, CrewAI, OpenAI Agents, Claude Agent SDK, LlamaIndex, Haystack, …) so the framework's agent / tool / chain structure is captured. - **Decorators** — wrap the user's own functions with \`@workflow\` / \`@task\` for nested spans. Follow only the branch the user picks; skip the other one. -> **Frameworks under Auto:** Auto never installs a framework instrumentor. If the app uses a framework (LangChain, CrewAI, an agents SDK) and the user picks Auto, they get only the flat LLM spans the underlying direct SDK emits — no agent / tool / chain structure. Capturing framework structure requires Full. +> **Frameworks under Auto:** Auto normally does not install a framework instrumentor. If the app uses a framework (LangChain, CrewAI, an agents SDK), Auto captures only spans from a supported underlying direct SDK. **Direct Vercel AI SDK usage is different:** the Respan facade deliberately does not auto-discover it, and AI SDK calls are not covered by core-only setup. If the project directly calls the \`ai\` package and no higher-level framework owns its AI SDK telemetry, follow the Vercel branch below in both Auto and Full. --- ### Auto path -**A1. Detect language and package manager** — \`package.json\` (JS/TS) or \`pyproject.toml\` / \`requirements.txt\` (Python). +**A1. Detect language, package manager, and integration** — inspect \`package.json\` and source imports for JS/TS, or \`pyproject.toml\` / \`requirements.txt\` for Python. Before choosing core-only initialization, check whether the JS/TS project directly imports and calls the \`ai\` package (Vercel AI SDK). Apply the priority rule first: if a higher-level integration owns AI SDK telemetry, use that integration instead of also adding Vercel instrumentation. For example, an Eve app must use \`EveInstrumentor\`; do not activate \`VercelAIInstrumentor\` or another \`@ai-sdk/otel\` bridge alongside it. **A2. Install the core SDK** (pin the exact current version): \`\`\`bash @@ -103,6 +103,12 @@ pip install respan-ai npm install @respan/respan \`\`\` +If the project uses Vercel AI SDK directly, fetch and follow its [integration doc](https://respan.ai/docs/integrations/vercel-ai-sdk.md), and pin exact compatible versions of: + +- \`@respan/respan\` +- \`@respan/instrumentation-vercel\` +- \`@ai-sdk/otel\` when the installed \`ai\` major version is 7 (AI SDK 4 through 6 do not require this adapter) + **A3. Add initialization code** at the top of the entrypoint, before any LLM client is created: \`\`\`python # Python @@ -116,7 +122,33 @@ const respan = new Respan(); await respan.initialize(); \`\`\` -Do **not** add decorators or a framework instrumentor. Go straight to **Verify**. +For a Vercel AI SDK project, use its required explicit instrumentor instead of the core-only TypeScript initialization above: + +\`\`\`typescript +import { Respan } from "@respan/respan"; +import { VercelAIInstrumentor } from "@respan/instrumentation-vercel"; + +const respan = new Respan({ + instrumentations: [new VercelAIInstrumentor()], +}); +await respan.initialize(); +\`\`\` + +If the project already has a \`Respan\` instance, merge \`VercelAIInstrumentor\` into its existing \`instrumentations\` list instead of creating a second instance. Preserve other explicit instrumentors that cover genuinely separate direct-SDK calls; do not add a provider instrumentor solely for calls made through Vercel AI SDK. + +If the application already registers its own compatible \`@ai-sdk/otel\` integration, construct \`VercelAIInstrumentor({ autoRegisterAISDKTelemetry: false })\` so Respan translates the spans without claiming a second registration. For Next.js, follow the integration doc's root \`instrumentation.ts\` \`register()\` placement and \`serverExternalPackages\` configuration; keep tracing on the server runtime, not an Edge or browser path. + +Also enable Vercel telemetry on every AI SDK operation. Use the option that matches the installed AI SDK major version: + +\`\`\`typescript +// AI SDK 7 +telemetry: { isEnabled: true } + +// AI SDK 4 through 6 +experimental_telemetry: { isEnabled: true } +\`\`\` + +Do **not** add decorators. For non-Vercel Auto setups, do not add a framework instrumentor. For Vercel, keep the required \`VercelAIInstrumentor\` and go straight to **Verify**. --- @@ -152,23 +184,29 @@ Check higher-priority categories first. If a match is found, use that instrument If a Priority 1 framework is found, use its instrumentation. Do NOT also add Priority 2 instrumentation for the same provider. +For direct Vercel AI SDK usage, the explicit \`VercelAIInstrumentor\` and per-operation telemetry option are required in both Auto and Full. Never replace them with core-only \`new Respan()\` setup, and never stack them with a higher-level integration that already owns AI SDK telemetry. + **LangChain / LangGraph setup differs by language** (one package covers both frameworks: \`respan-instrumentation-langchain\` / \`@respan/instrumentation-langchain\`). In **Python**, pass \`LangChainInstrumentor()\` to \`Respan(instrumentations=[...])\` — it patches the LangChain and LangGraph callback managers globally on init, so every chain, graph, node, LLM, tool, and retriever run is traced **automatically with no per-call setup**. \`add_respan_callback(config=...)\` is optional and only labels a run with a name, tags, or metadata. In **TypeScript**, create a \`LangChainInstrumentor\`, pass it to \`new Respan({ instrumentations: [...] })\`, **and** attach \`instrumentor.addCallback(...)\` to the **outermost** chain or graph invocation (\`.invoke()\` / \`.stream()\`, not inside a node — LangChain/LangGraph propagate it to nested runs); the TypeScript instrumentor does **not** patch globally, so without the callback no spans are emitted. (The other Priority-1 frameworks use the standard \`Respan(instrumentations=[XInstrumentor()])\` plugin pattern.) **Priority 2 — Direct LLM SDKs** (only if no P1 framework covers this provider): -These are **auto-instrumented** — just \`Respan()\` / \`new Respan()\`, no extra packages needed: +The integrations listed for each language are **auto-instrumented** with \`Respan()\` / \`new Respan()\`. The application still provides the provider SDK; the matching first-party instrumentation adapter is bundled by the Respan facade. A dash means that language currently requires an explicit instrumentation package and instrumentor. -| Library | Python package | JS/TS package | Docs | -|---------|---------------|---------------|------| +| Library | Python SDK (auto) | JS/TS SDK (auto) | Docs | +|---------|-------------------|------------------|------| | OpenAI SDK | \`openai\` | \`openai\` | [docs](https://respan.ai/docs/integrations/openai-sdk.md) | | Anthropic SDK | \`anthropic\` | \`@anthropic-ai/sdk\` | [docs](https://respan.ai/docs/integrations/anthropic.md) | -| Azure OpenAI | \`openai\` (azure config) | \`openai\` | [docs](https://respan.ai/docs/integrations/providers/azure.md) | -| Google Vertex AI | \`google-cloud-aiplatform\` | — | [docs](https://respan.ai/docs/integrations/vertex-ai.md) | -| AWS Bedrock | \`boto3\` | — | [docs](https://respan.ai/docs/integrations/aws-bedrock.md) | -| Cohere | \`cohere\` | — | [docs](https://respan.ai/docs/integrations/providers/cohere.md) | -| Together AI | \`together\` | — | [docs](https://respan.ai/docs/integrations/together-ai.md) | - -**Note:** LiteLLM in JS uses the OpenAI-compatible API, so the OpenAI auto-instrument covers it. For Python LiteLLM, see [LiteLLM guide](https://respan.ai/docs/integrations/litellm.md). For Google GenAI (\`@google/genai\`), see [Google GenAI guide](https://respan.ai/docs/integrations/google-genai.md). +| Azure OpenAI | \`openai\` (Azure config) | \`openai\` (Azure client) | [docs](https://respan.ai/docs/integrations/providers/azure.md) | +| Google Vertex AI | \`google-cloud-aiplatform\` | \`@google-cloud/vertexai\` | [docs](https://respan.ai/docs/integrations/vertex-ai.md) | +| Google GenAI | \`google-genai\` | — | [docs](https://respan.ai/docs/integrations/google-genai.md) | +| AWS Bedrock | \`boto3\` | \`@aws-sdk/client-bedrock-runtime\` | [docs](https://respan.ai/docs/integrations/aws-bedrock.md) | +| Cohere | — | \`cohere-ai\` | [docs](https://respan.ai/docs/integrations/providers/cohere.md) | +| Together AI | \`together\` | \`together-ai\` | [docs](https://respan.ai/docs/integrations/together-ai.md) | +| OpenRouter | — | \`@openrouter/sdk\` | [docs](https://respan.ai/docs/integrations/providers/openrouter.md) | +| Writer | — | \`writer-sdk\` | [docs](https://respan.ai/docs/integrations/writer.md) | +| Ollama | \`ollama\` | — | [docs](https://respan.ai/docs/integrations/ollama.md) | + +**Note:** LLM wrappers such as Python LiteLLM stay explicit-only to avoid overlapping provider spans. See the [LiteLLM guide](https://respan.ai/docs/integrations/litellm.md) for its explicit setup. **1c. Read the actual code and understand the workflow:** diff --git a/javascript-sdks/respan-cli/tests/tracing-skill.test.mjs b/javascript-sdks/respan-cli/tests/tracing-skill.test.mjs new file mode 100644 index 00000000..316c0fe4 --- /dev/null +++ b/javascript-sdks/respan-cli/tests/tracing-skill.test.mjs @@ -0,0 +1,18 @@ +import assert from 'node:assert/strict'; +import test from 'node:test'; + +import { TRACING_MD } from '../dist/lib/skill-refs.generated.js'; + +test('Auto tracing setup keeps the required Vercel AI SDK instrumentation', () => { + const autoPath = TRACING_MD.match(/### Auto path([\s\S]*?)\n---\n\n### Full path/)?.[1]; + + assert.ok(autoPath, 'expected an Auto path in the bundled tracing skill'); + assert.match(autoPath, /Vercel AI SDK/); + assert.match(autoPath, /@respan\/instrumentation-vercel/); + assert.match(autoPath, /@ai-sdk\/otel/); + assert.match(autoPath, /EveInstrumentor/); + assert.match(autoPath, /VercelAIInstrumentor/); + assert.match(autoPath, /instrumentations:\s*\[new VercelAIInstrumentor\(\)\]/); + assert.match(autoPath, /telemetry:\s*\{\s*isEnabled:\s*true/s); + assert.match(autoPath, /experimental_telemetry:\s*\{\s*isEnabled:\s*true/s); +}); diff --git a/skills/references/tracing.md b/skills/references/tracing.md index b73a871a..173500da 100644 --- a/skills/references/tracing.md +++ b/skills/references/tracing.md @@ -30,20 +30,20 @@ The API key is stored in `.env` as `RESPAN_API_KEY`. **Before analyzing anything, ask the user which setup they want.** This is the first question — the two branches diverge immediately. -1. **Auto** — the fastest path. Install the core SDK and add `Respan()` (one line). Every LLM call from a supported direct SDK — OpenAI, Anthropic, Azure OpenAI, Bedrock, Vertex, Cohere, Together, Gemini, LiteLLM — is automatically captured as a flat span. **No decorators and no framework instrumentor, even if a framework is detected.** Best for a quick start or a live demo: traces flowing in under two minutes. +1. **Auto** — the fastest working path. For a supported direct SDK, install the core SDK and add `Respan()` (one line) to capture flat LLM spans. The Vercel AI SDK is an explicit-only exception: Auto must also add `@respan/instrumentation-vercel` and `VercelAIInstrumentor`, but still adds no decorators or manual workflow wrappers. Best for a quick start or a live demo: traces flowing in under two minutes. 2. **Full** — structured setup. Adds framework-specific instrumentation and/or workflow structure on top: - **Explicit instrumentor** — for agent frameworks (LangChain, CrewAI, OpenAI Agents, Claude Agent SDK, LlamaIndex, Haystack, …) so the framework's agent / tool / chain structure is captured. - **Decorators** — wrap the user's own functions with `@workflow` / `@task` for nested spans. Follow only the branch the user picks; skip the other one. -> **Frameworks under Auto:** Auto never installs a framework instrumentor. If the app uses a framework (LangChain, CrewAI, an agents SDK) and the user picks Auto, they get only the flat LLM spans the underlying direct SDK emits — no agent / tool / chain structure. Capturing framework structure requires Full. +> **Frameworks under Auto:** Auto normally does not install a framework instrumentor. If the app uses a framework (LangChain, CrewAI, an agents SDK), Auto captures only spans from a supported underlying direct SDK. **Direct Vercel AI SDK usage is different:** the Respan facade deliberately does not auto-discover it, and AI SDK calls are not covered by core-only setup. If the project directly calls the `ai` package and no higher-level framework owns its AI SDK telemetry, follow the Vercel branch below in both Auto and Full. --- ### Auto path -**A1. Detect language and package manager** — `package.json` (JS/TS) or `pyproject.toml` / `requirements.txt` (Python). +**A1. Detect language, package manager, and integration** — inspect `package.json` and source imports for JS/TS, or `pyproject.toml` / `requirements.txt` for Python. Before choosing core-only initialization, check whether the JS/TS project directly imports and calls the `ai` package (Vercel AI SDK). Apply the priority rule first: if a higher-level integration owns AI SDK telemetry, use that integration instead of also adding Vercel instrumentation. For example, an Eve app must use `EveInstrumentor`; do not activate `VercelAIInstrumentor` or another `@ai-sdk/otel` bridge alongside it. **A2. Install the core SDK** (pin the exact current version): ```bash @@ -54,6 +54,12 @@ pip install respan-ai npm install @respan/respan ``` +If the project uses Vercel AI SDK directly, fetch and follow its [integration doc](https://respan.ai/docs/integrations/vercel-ai-sdk.md), and pin exact compatible versions of: + +- `@respan/respan` +- `@respan/instrumentation-vercel` +- `@ai-sdk/otel` when the installed `ai` major version is 7 (AI SDK 4 through 6 do not require this adapter) + **A3. Add initialization code** at the top of the entrypoint, before any LLM client is created: ```python # Python @@ -67,7 +73,33 @@ const respan = new Respan(); await respan.initialize(); ``` -Do **not** add decorators or a framework instrumentor. Go straight to **Verify**. +For a Vercel AI SDK project, use its required explicit instrumentor instead of the core-only TypeScript initialization above: + +```typescript +import { Respan } from "@respan/respan"; +import { VercelAIInstrumentor } from "@respan/instrumentation-vercel"; + +const respan = new Respan({ + instrumentations: [new VercelAIInstrumentor()], +}); +await respan.initialize(); +``` + +If the project already has a `Respan` instance, merge `VercelAIInstrumentor` into its existing `instrumentations` list instead of creating a second instance. Preserve other explicit instrumentors that cover genuinely separate direct-SDK calls; do not add a provider instrumentor solely for calls made through Vercel AI SDK. + +If the application already registers its own compatible `@ai-sdk/otel` integration, construct `VercelAIInstrumentor({ autoRegisterAISDKTelemetry: false })` so Respan translates the spans without claiming a second registration. For Next.js, follow the integration doc's root `instrumentation.ts` `register()` placement and `serverExternalPackages` configuration; keep tracing on the server runtime, not an Edge or browser path. + +Also enable Vercel telemetry on every AI SDK operation. Use the option that matches the installed AI SDK major version: + +```typescript +// AI SDK 7 +telemetry: { isEnabled: true } + +// AI SDK 4 through 6 +experimental_telemetry: { isEnabled: true } +``` + +Do **not** add decorators. For non-Vercel Auto setups, do not add a framework instrumentor. For Vercel, keep the required `VercelAIInstrumentor` and go straight to **Verify**. --- @@ -103,6 +135,8 @@ Check higher-priority categories first. If a match is found, use that instrument If a Priority 1 framework is found, use its instrumentation. Do NOT also add Priority 2 instrumentation for the same provider. +For direct Vercel AI SDK usage, the explicit `VercelAIInstrumentor` and per-operation telemetry option are required in both Auto and Full. Never replace them with core-only `new Respan()` setup, and never stack them with a higher-level integration that already owns AI SDK telemetry. + **LangChain / LangGraph setup differs by language** (one package covers both frameworks: `respan-instrumentation-langchain` / `@respan/instrumentation-langchain`). In **Python**, pass `LangChainInstrumentor()` to `Respan(instrumentations=[...])` — it patches the LangChain and LangGraph callback managers globally on init, so every chain, graph, node, LLM, tool, and retriever run is traced **automatically with no per-call setup**. `add_respan_callback(config=...)` is optional and only labels a run with a name, tags, or metadata. In **TypeScript**, create a `LangChainInstrumentor`, pass it to `new Respan({ instrumentations: [...] })`, **and** attach `instrumentor.addCallback(...)` to the **outermost** chain or graph invocation (`.invoke()` / `.stream()`, not inside a node — LangChain/LangGraph propagate it to nested runs); the TypeScript instrumentor does **not** patch globally, so without the callback no spans are emitted. (The other Priority-1 frameworks use the standard `Respan(instrumentations=[XInstrumentor()])` plugin pattern.) **Priority 2 — Direct LLM SDKs** (only if no P1 framework covers this provider):