Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .agents/skills/instrumentation/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,7 @@ Map the change before editing:
- Promise/stream behavior must be preserved. Patches need to keep subclass/helper semantics intact.
- Contain instrumentation failures. Extraction/logging bugs should be logged or ignored as appropriate, but must not break the user call path.
- Log only the useful surface. Prefer narrow, stable payloads over dumping full request/response objects; exclude redundant or overly large data when possible.
- We want to limit our instrumentation to operations that are relevant for AI generations and operations (LLMs, embeddings, media generation, ...). Things like creating entities on platforms (CRUD for Workflows of Agent entities) is irrelevant to us.
- When building instrumentation, we should always have a vendored type/interface for what we are wrapping. The type or interface should not be larger than what is relevant to the instrumentation. The type or interface should be used for typing tracing channels and also should be used to assert the type on whatever is passed into wrappers as soon as the wrapper has verified that the passed in value is plausibly what should be wrapped.

## Process
Expand Down
10 changes: 6 additions & 4 deletions .env.example
Original file line number Diff line number Diff line change
@@ -1,4 +1,6 @@

BRAINTRUST_API_KEY=your_braintrust_api_key
OPENAI_API_KEY=your_openai_api_key
ANTHROPIC_API_KEY=your_anthropic_key
BRAINTRUST_API_KEY=
OPENAI_API_KEY=
ANTHROPIC_API_KEY=
GEMINI_API_KEY=
OPENROUTER_API_KEY=
MISTRAL_API_KEY=
1 change: 1 addition & 0 deletions .github/workflows/e2e-canary.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,7 @@ jobs:
GEMINI_API_KEY: ${{ secrets.GEMINI_API_KEY }}
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
OPENROUTER_API_KEY: ${{ secrets.OPENROUTER_API_KEY }}
MISTRAL_API_KEY: ${{ secrets.MISTRAL_API_KEY }}
run: pnpm test:e2e:canary
- name: Create or update nightly canary issue
if: ${{ failure() && github.event_name == 'schedule' }}
Expand Down
3 changes: 3 additions & 0 deletions .github/workflows/integration-tests.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@ jobs:
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
GEMINI_API_KEY: ${{ secrets.GEMINI_API_KEY }}
OPENROUTER_API_KEY: ${{ secrets.OPENROUTER_API_KEY }}
MISTRAL_API_KEY: ${{ secrets.MISTRAL_API_KEY }}
steps:
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4.3.1
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
Expand Down Expand Up @@ -57,6 +58,7 @@ jobs:
GEMINI_API_KEY: ${{ secrets.GEMINI_API_KEY }}
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
OPENROUTER_API_KEY: ${{ secrets.OPENROUTER_API_KEY }}
MISTRAL_API_KEY: ${{ secrets.MISTRAL_API_KEY }}
steps:
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4.3.1
- uses: denoland/setup-deno@ff4860f9d7236f320afa0f82b7e6457384805d05 # v2.0.4
Expand Down Expand Up @@ -105,6 +107,7 @@ jobs:
GEMINI_API_KEY: ${{ secrets.GEMINI_API_KEY }}
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
OPENROUTER_API_KEY: ${{ secrets.OPENROUTER_API_KEY }}
MISTRAL_API_KEY: ${{ secrets.MISTRAL_API_KEY }}
steps:
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4.3.1
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
Expand Down
4 changes: 2 additions & 2 deletions .vscode/settings.json
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
{
"typescript.tsdk": "js/node_modules/typescript/lib",
"typescript.enablePromptUseWorkspaceTsdk": false
"js/ts.tsdk.promptToUseWorkspaceVersion": false,
"js/ts.tsdk.path": "js/node_modules/typescript/lib"
}
1 change: 1 addition & 0 deletions e2e/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -116,6 +116,7 @@ Non-hermetic scenarios require provider credentials in addition to the mock Brai
- `ANTHROPIC_API_KEY`
- `GEMINI_API_KEY` or `GOOGLE_API_KEY`
- `OPENROUTER_API_KEY`
- `MISTRAL_API_KEY`

`claude-agent-sdk-instrumentation` also uses `ANTHROPIC_API_KEY`, because it runs the real Claude Agent SDK against Anthropic in the same style as the existing live Anthropic wrapper coverage.

Expand Down
13 changes: 13 additions & 0 deletions e2e/config/pr-comment-scenarios.json
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,19 @@
{ "variantKey": "google-genai-v1460", "label": "v1.46.0" }
]
},
{
"scenarioDirName": "mistral-instrumentation",
"label": "Mistral Instrumentation",
"metadataScenario": "mistral-instrumentation",
"variants": [
{ "variantKey": "mistral-v1-3-4", "label": "v1.3.4" },
{ "variantKey": "mistral-v1-10-0", "label": "v1.10.0" },
{ "variantKey": "mistral-v1-14-1", "label": "v1.14.1" },
{ "variantKey": "mistral-v1-15-1", "label": "v1.15.1" },
{ "variantKey": "mistral-v1", "label": "latest v1" },
{ "variantKey": "mistral-v2", "label": "latest v2" }
]
},
{
"scenarioDirName": "openrouter-instrumentation",
"label": "OpenRouter Instrumentation",
Expand Down
1 change: 1 addition & 0 deletions e2e/helpers/scenario-installer.ts
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ const INSTALL_SECRET_ENV_VARS = [
"GH_TOKEN",
"OPENAI_API_KEY",
"OPENROUTER_API_KEY",
"MISTRAL_API_KEY",
] as const;

const cleanupDirs = new Set<string>();
Expand Down
Loading
Loading