fix: honor ls_provider for gen_ai.system in OTel translator [closes #2936]#2948
Open
langsmith-fleet[bot] wants to merge 1 commit into
Open
fix: honor ls_provider for gen_ai.system in OTel translator [closes #2936]#2948langsmith-fleet[bot] wants to merge 1 commit into
langsmith-fleet[bot] wants to merge 1 commit into
Conversation
…2936] wrapAnthropic emits ls_provider='anthropic' via getInvocationParams but the OTel translator's setGenAiSystem only inferred gen_ai.system from model-name heuristics. When a model alias (e.g. 'claude-haiku-4-5') happens not to match any heuristic pattern, gen_ai.system fell back to 'langchain', preventing Playground from resolving the Anthropic model dropdown. Fix: check ls_provider in metadata first (authoritative, set explicitly by wrappers), and only fall back to model-name heuristics when it is absent. Add a static provider→system map covering all wrappers (anthropic, openai, azure, bedrock, etc.) plus unit tests.
JS perf benchmarkLower is better. Noisy on shared runners — treat as a signal, not a gate. Base64-heavy payloadSingle large base64 string per message — the shape the worker-offload path is optimized for.
Structural payloadMany small strings across a wide/nested object graph. Should bypass worker offload and use sync flush.
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
wrapAnthropicexplicitly emitsls_provider: "anthropic"viagetInvocationParams, butLangSmithToOTELTranslator.setGenAiSystemwas ignoring it and only inferringgen_ai.systemfrom model-name heuristics. For model aliases like"claude-haiku-4-5"the heuristic works (starts with"claude"), but this is fragile — any alias that doesn't happen to match a pattern causesgen_ai.systemto fall back to"langchain", and Playground cannot resolve the Anthropic model dropdown.The fix: check
ls_providerinrunInfo.extra.metadatafirst (authoritative, set explicitly by all wrappers), then fall back to model-name heuristics only when it's absent. A staticLS_PROVIDER_TO_GEN_AI_SYSTEMmap covers all existing providers (anthropic, openai, azure, bedrock, cohere, deepseek, gemini, groq, mistral, perplexity, vertex, xai).This also benefits
wrapOpenAIand any future wrappers for free.Test Plan
js/src/tests/otel_translator.test.tscovering:ls_provider=anthropic,ls_provider=openai, model-name fallback, unrecognized model fallback, and alias-without-"claude" case (the key regression scenario from wrapAnthropic + LANGSMITH_TRACING_MODE=otel traces don't auto-resolve in Playground #2936)