feat!: Rename LDAIMetrics.usage and LDAIGraphMetrics.usage to .tokens#1366
Merged
Conversation
Contributor
|
@launchdarkly/js-sdk-common size report |
Contributor
|
@launchdarkly/js-client-sdk size report |
3 tasks
Contributor
|
@launchdarkly/browser size report |
Contributor
|
@launchdarkly/js-client-sdk-common size report |
jsonbailey
added a commit
to launchdarkly/python-server-sdk-ai
that referenced
this pull request
May 7, 2026
…175) ## Summary Standardizes on `tokens` as the field name for token usage across all AI metric types, aligning with the canonical spec rename. Renames: - `LDAIMetrics.usage` → `LDAIMetrics.tokens` (runner-level single-model metrics) - `AIGraphMetrics.usage` → `AIGraphMetrics.tokens` (runner-level graph metrics) - `LDAIMetricSummary.usage` → `LDAIMetricSummary.tokens` (managed-layer single-model summary) - `AIGraphMetricSummary.usage` → `AIGraphMetricSummary.tokens` (managed-layer graph summary) Updates the type definitions in `ldai/providers/types.py` and `ldai/tracker.py`, all internal usages across the SDK (`tracker.py`, `managed_agent_graph.py`) and provider packages (openai model + agent + agent-graph runners; langchain model + agent + langgraph runners; langgraph callback handler), tests, and README docstrings. The `to_dict()` serialization key on `LDAIMetrics` also changes from `usage` to `tokens`. No backward-compat aliases are kept -- these types are part of the in-progress unreleased SDK iteration after PR #173 (the prior rename batch). ## Coordinated PRs This is one of three coordinated PRs landing this rename together: - **sdk-specs** (canonical name source): launchdarkly/sdk-specs#163 - **python-server-sdk-ai** (this PR): targets `main` - **js-core** (sibling impl): launchdarkly/js-core#1366 (targets `feat/next-ai-release`) ## Test plan - [x] `make test` -- all 325 tests pass (server-ai 196, langchain 85, openai 44) - [x] `make lint` -- mypy, isort, pycodestyle all clean across all three packages - [ ] e2e validation via `hello-python-ai` chat-judge example (sibling agent) <!-- CURSOR_SUMMARY --> --- > [!NOTE] > **Medium Risk** > Broad breaking API rename from `usage` to `tokens` across SDK/provider metric types and serialization, which will require downstream code updates and could silently drop token tracking if any call sites were missed. > > **Overview** > Standardizes token accounting field names by renaming **`usage` → `tokens`** across core metric types (`LDAIMetrics`, `AIGraphMetrics`) and their managed summaries, including `LDAIMetrics.to_dict()` now emitting a `tokens` key. > > Updates all affected runners (LangChain, LangGraph, OpenAI models/agents/graphs), tracker integration (`track_tokens`, graph total token tracking), docs, and tests to use the new `tokens` field with no backward-compatible alias. > > <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit ccdc00e. Bugbot is set up for automated code reviews on this repo. Configure [here](https://www.cursor.com/dashboard/bugbot).</sup> <!-- /CURSOR_SUMMARY --> Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
joker23
approved these changes
May 7, 2026
Standardize the runner-level metrics token-usage field on `tokens` to match the summary-level types (`LDAIMetricSummary` and `LDAIGraphMetricSummary`, which already use `tokens`). This is a breaking change for anyone constructing or reading these metrics directly: the field renames from `usage` to `tokens` while the type (`LDTokenUsage`) is unchanged. External provider response shapes (OpenAI, Bedrock, Vercel `usage` properties) and the LD-internal `$ld:ai:usage:*` event keys are unaffected. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
jsonbailey
force-pushed
the
jb/align-usage-to-tokens
branch
from
a185790 to
3a6fc84
Compare
jsonbailey
changed the title
jsonbailey
added a commit
that referenced
this pull request
May 7, 2026
Merged
jsonbailey
pushed a commit
that referenced
this pull request
May 7, 2026
🤖 I have created a release *beep* *boop* --- <details><summary>server-sdk-ai: 0.20.0</summary> ## [0.20.0](server-sdk-ai-v0.19.1...server-sdk-ai-v0.20.0) (2026-05-07) ### ⚠ BREAKING CHANGES * Rename LDAIMetrics.usage and LDAIGraphMetrics.usage to .tokens ([#1366](#1366)) * Remove AIProvider deprecated methods and create*/init* aliases (AIC-2388) ([#1363](#1363)) * Build judge input as string and strip legacy judge config messages ([#1364](#1364)) * Use LDAIGraphMetricSummary for graph metric summary ([#1362](#1362)) ### Features * add Evaluator class for judge orchestration ([#1331](#1331)) ([54faa69](54faa69)) * add ManagedAgent with evaluations support ([#1334](#1334)) ([7f09c46](7f09c46)) * add ManagedGraphResult, GraphMetricSummary, and ManagedAgentGraph ([#1335](#1335)) ([09fa1db](09fa1db)) * introduce ManagedResult, RunnerResult, and LDAIMetricSummary ([#1332](#1332)) ([5040122](5040122)) * Remove AIProvider deprecated methods and create*/init* aliases (AIC-2388) ([#1363](#1363)) ([ad66314](ad66314)) * Rename LDAIMetrics.usage and LDAIGraphMetrics.usage to .tokens ([#1366](#1366)) ([ff932b7](ff932b7)) * Replace OpenAIProvider with Runner protocol implementation (AIC-2388) ([#1337](#1337)) ([e32a955](e32a955)) ### Bug Fixes * Build judge input as string and strip legacy judge config messages ([#1364](#1364)) ([c90034b](c90034b)) * Use LDAIGraphMetricSummary for graph metric summary ([#1362](#1362)) ([76a4bf2](76a4bf2)) </details> <details><summary>server-sdk-ai-langchain: 0.7.0</summary> ## [0.7.0](server-sdk-ai-langchain-v0.6.3...server-sdk-ai-langchain-v0.7.0) (2026-05-07) ### ⚠ BREAKING CHANGES * Rename LDAIMetrics.usage and LDAIGraphMetrics.usage to .tokens ([#1366](#1366)) * Remove AIProvider deprecated methods and create*/init* aliases (AIC-2388) ([#1363](#1363)) * Build judge input as string and strip legacy judge config messages ([#1364](#1364)) ### Features * Remove AIProvider deprecated methods and create*/init* aliases (AIC-2388) ([#1363](#1363)) ([ad66314](ad66314)) * Rename LDAIMetrics.usage and LDAIGraphMetrics.usage to .tokens ([#1366](#1366)) ([ff932b7](ff932b7)) * Replace LangChainProvider with Runner protocol implementation (AIC-2388) ([#1338](#1338)) ([113a0d2](113a0d2)) ### Bug Fixes * Build judge input as string and strip legacy judge config messages ([#1364](#1364)) ([c90034b](c90034b)) ### Dependencies * The following workspace dependencies were updated * devDependencies * @launchdarkly/server-sdk-ai bumped from ^0.19.1 to ^0.20.0 * peerDependencies * @launchdarkly/server-sdk-ai bumped from ^0.19.1 to ^0.20.0 </details> <details><summary>server-sdk-ai-openai: 0.6.0</summary> ## [0.6.0](server-sdk-ai-openai-v0.5.10...server-sdk-ai-openai-v0.6.0) (2026-05-07) ### ⚠ BREAKING CHANGES * Rename LDAIMetrics.usage and LDAIGraphMetrics.usage to .tokens ([#1366](#1366)) * Remove AIProvider deprecated methods and create*/init* aliases (AIC-2388) ([#1363](#1363)) * Build judge input as string and strip legacy judge config messages ([#1364](#1364)) ### Features * Remove AIProvider deprecated methods and create*/init* aliases (AIC-2388) ([#1363](#1363)) ([ad66314](ad66314)) * Rename LDAIMetrics.usage and LDAIGraphMetrics.usage to .tokens ([#1366](#1366)) ([ff932b7](ff932b7)) * Replace OpenAIProvider with Runner protocol implementation (AIC-2388) ([#1337](#1337)) ([e32a955](e32a955)) ### Bug Fixes * Build judge input as string and strip legacy judge config messages ([#1364](#1364)) ([c90034b](c90034b)) ### Dependencies * The following workspace dependencies were updated * devDependencies * @launchdarkly/server-sdk-ai bumped from ^0.19.1 to ^0.20.0 * peerDependencies * @launchdarkly/server-sdk-ai bumped from ^0.19.1 to ^0.20.0 </details> <details><summary>server-sdk-ai-vercel: 0.6.0</summary> ## [0.6.0](server-sdk-ai-vercel-v0.5.10...server-sdk-ai-vercel-v0.6.0) (2026-05-07) ### ⚠ BREAKING CHANGES * Rename LDAIMetrics.usage and LDAIGraphMetrics.usage to .tokens ([#1366](#1366)) * Remove AIProvider deprecated methods and create*/init* aliases (AIC-2388) ([#1363](#1363)) * Build judge input as string and strip legacy judge config messages ([#1364](#1364)) ### Features * Remove AIProvider deprecated methods and create*/init* aliases (AIC-2388) ([#1363](#1363)) ([ad66314](ad66314)) * Rename LDAIMetrics.usage and LDAIGraphMetrics.usage to .tokens ([#1366](#1366)) ([ff932b7](ff932b7)) * replace VercelProvider with Runner protocol implementation (AIC-2388) ([#1339](#1339)) ([d5a62de](d5a62de)) ### Bug Fixes * add zod devDependency to Vercel provider (peer dep of ai v5) ([aab6226](aab6226)) * Build judge input as string and strip legacy judge config messages ([#1364](#1364)) ([c90034b](c90034b)) ### Dependencies * The following workspace dependencies were updated * devDependencies * @launchdarkly/server-sdk-ai bumped from ^0.19.1 to ^0.20.0 * peerDependencies * @launchdarkly/server-sdk-ai bumped from ^0.19.1 to ^0.20.0 </details> --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please). <!-- CURSOR_SUMMARY --> --- > [!NOTE] > **Medium Risk** > This is primarily a versioning/release metadata update, but it ships breaking API changes in `@launchdarkly/server-sdk-ai` and provider packages that may require consumer code updates (e.g., metrics field rename and deprecated method removals). > > **Overview** > Publishes a new release for `@launchdarkly/server-sdk-ai` (`0.19.1` → `0.20.0`) and the provider packages (`server-sdk-ai-langchain` `0.6.3` → `0.7.0`, `server-sdk-ai-openai`/`server-sdk-ai-vercel` `0.5.10` → `0.6.0`), updating `.release-please-manifest.json`, package versions, and `sdkInfo.ts`. > > Updates provider `peerDependencies`/`devDependencies` to require `@launchdarkly/server-sdk-ai@^0.20.0`, and bumps all `server-ai` examples to depend on the new SDK/provider versions. Changelogs highlight *breaking changes* including renaming metrics `usage` → `tokens`, removing deprecated `AIProvider` methods/aliases, and judge-input/config cleanup (plus additional `server-sdk-ai` graph/managed result summary changes). > > <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit cb3aebc. Bugbot is set up for automated code reviews on this repo. Configure [here](https://www.cursor.com/dashboard/bugbot).</sup> <!-- /CURSOR_SUMMARY --> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Standardize the runner-level metrics token-usage field on
tokensto match the summary-level types. The summary typesLDAIMetricSummaryandLDAIGraphMetricSummaryalready usetokens; this PR aligns the runner-levelLDAIMetricsandLDAIGraphMetricswith that convention.This is one of three coordinated PRs that land together:
Breaking change
The
usageproperty onLDAIMetricsandLDAIGraphMetricsis renamed totokens. The value type (LDTokenUsage) is unchanged. Consumers that construct or read these metrics directly must rename the field.External provider response shapes (the
usageproperty on OpenAI / Bedrock / Vercel response objects) and the LD-internal$ld:ai:usage:*event keys are unaffected — those are separate concepts that happen to share the English word.What's included
LDAIMetrics.usage→LDAIMetrics.tokens(packages/sdk/server-ai/src/api/metrics/LDAIMetrics.ts)LDAIGraphMetrics.usage→LDAIGraphMetrics.tokens(packages/sdk/server-ai/src/api/graph/types.ts)packages/sdk/server-aipackages/ai-providers/server-ai-openaipackages/ai-providers/server-ai-langchainpackages/ai-providers/server-ai-vercelVerification
LDAIMetricSummary(inpackages/sdk/server-ai/src/api/model/types.ts) already usestokens— no change neededLDAIGraphMetricSummaryalready usestokenssince feat!: consolidate graph metric summary into LDAIGraphMetricSummary #1362 — no change neededTest plan
yarn workspaces foreach -pR --topological-dev --from '@launchdarkly/server-sdk-ai' run build(server-sdk-ai + ai-providers builds clean)yarn workspace @launchdarkly/server-sdk-ai test(239 passed)yarn workspace @launchdarkly/server-sdk-ai-openai test(39 passed)yarn workspace @launchdarkly/server-sdk-ai-langchain test(38 passed)yarn workspace @launchdarkly/server-sdk-ai-vercel test(20 passed)yarn workspace ... lintfor each touched package (no errors)🤖 Generated with Claude Code
Note
Medium Risk
Breaking API rename from
metrics.usagetometrics.tokensacross runners/graphs; low logical risk but will break downstream consumers that read or construct these metrics.Overview
Breaking change: Standardizes runner-level token-usage metrics by renaming
LDAIMetrics.usagetoLDAIMetrics.tokens, andLDAIGraphMetrics.usagetoLDAIGraphMetrics.tokens.Updates all provider runners/helpers (LangChain, OpenAI, Vercel) plus managed-layer tracking (
LDAIConfigTrackerImpl,ManagedAgentGraph) and test assertions to read/write the newtokensfield while leaving underlying token values and provider responseusageshapes unchanged.Reviewed by Cursor Bugbot for commit 3a6fc84. Bugbot is set up for automated code reviews on this repo. Configure here.