Skip to content

fix: honor ls_provider for gen_ai.system in OTel translator [closes #2936]#2948

Open
langsmith-fleet[bot] wants to merge 1 commit into
mainfrom
fix/wrap-anthropic-otel-gen-ai-system
Open

fix: honor ls_provider for gen_ai.system in OTel translator [closes #2936]#2948
langsmith-fleet[bot] wants to merge 1 commit into
mainfrom
fix/wrap-anthropic-otel-gen-ai-system

Conversation

@langsmith-fleet
Copy link
Copy Markdown

Description

wrapAnthropic explicitly emits ls_provider: "anthropic" via getInvocationParams, but LangSmithToOTELTranslator.setGenAiSystem was ignoring it and only inferring gen_ai.system from model-name heuristics. For model aliases like "claude-haiku-4-5" the heuristic works (starts with "claude"), but this is fragile — any alias that doesn't happen to match a pattern causes gen_ai.system to fall back to "langchain", and Playground cannot resolve the Anthropic model dropdown.

The fix: check ls_provider in runInfo.extra.metadata first (authoritative, set explicitly by all wrappers), then fall back to model-name heuristics only when it's absent. A static LS_PROVIDER_TO_GEN_AI_SYSTEM map covers all existing providers (anthropic, openai, azure, bedrock, cohere, deepseek, gemini, groq, mistral, perplexity, vertex, xai).

This also benefits wrapOpenAI and any future wrappers for free.

Test Plan

…2936]

wrapAnthropic emits ls_provider='anthropic' via getInvocationParams but the
OTel translator's setGenAiSystem only inferred gen_ai.system from model-name
heuristics. When a model alias (e.g. 'claude-haiku-4-5') happens not to match
any heuristic pattern, gen_ai.system fell back to 'langchain', preventing
Playground from resolving the Anthropic model dropdown.

Fix: check ls_provider in metadata first (authoritative, set explicitly by
wrappers), and only fall back to model-name heuristics when it is absent.
Add a static provider→system map covering all wrappers (anthropic, openai,
azure, bedrock, etc.) plus unit tests.
@github-actions
Copy link
Copy Markdown

JS perf benchmark

Lower is better. Noisy on shared runners — treat as a signal, not a gate.

Base64-heavy payload

Single large base64 string per message — the shape the worker-offload path is optimized for.
Payload: 2511.2 KB in / 5.2 KB out, 100 runs.

metric main this PR delta
Wall time (ms) 1929.08 1916.41 -0.7%
createRun total (ms) 73.62 84.04 +14.1%
createRun p50 (ms) 0.63 0.70 +12.4%
createRun p95 (ms) 0.94 0.89 -5.5%
createRun p99 (ms) 19.87 27.19 +36.9%
createRun max (ms) 19.87 27.19 +36.9%
updateRun total (ms) 41.32 41.56 +0.6%
updateRun p95 (ms) 0.68 0.74 +8.9%
loop lag total (ms) 971.48 968.86 -0.3%
loop lag p50 (ms) 0.09 0.10 +0.3%
loop lag p95 (ms) 3.71 3.83 +3.1%
loop lag p99 (ms) 15.32 22.57 +47.4%
loop lag max (ms) 259.18 89.56 -65.4%

Structural payload

Many small strings across a wide/nested object graph. Should bypass worker offload and use sync flush.
Payload: 1239.5 KB in / 13.3 KB out, 100 runs.

metric main this PR delta
Wall time (ms) 1477.49 1472.46 -0.3%
createRun total (ms) 400.76 390.16 -2.6%
createRun p50 (ms) 3.54 3.42 -3.5%
createRun p95 (ms) 6.13 5.84 -4.7%
createRun p99 (ms) 8.82 20.23 +129.5%
createRun max (ms) 8.82 20.23 +129.5%
updateRun total (ms) 33.23 35.06 +5.5%
updateRun p95 (ms) 0.57 0.54 -5.3%
loop lag total (ms) 1026.13 1030.90 +0.5%
loop lag p50 (ms) 0.07 0.08 +12.4%
loop lag p95 (ms) 5.25 4.05 -23.0%
loop lag p99 (ms) 127.64 125.99 -1.3%
loop lag max (ms) 161.76 136.09 -15.9%

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant