Skip to content

feat: support conversation history directly in AI Provider model runners#1371

Merged
jsonbailey merged 1 commit into
mainfrom
jb/message-history-in-providers
May 14, 2026
Merged

feat: support conversation history directly in AI Provider model runners#1371
jsonbailey merged 1 commit into
mainfrom
jb/message-history-in-providers

Conversation

@jsonbailey

@jsonbailey jsonbailey commented May 7, 2026

Copy link
Copy Markdown
Contributor

Summary

Port the multi-turn conversation history pattern from the Python AI SDK provider runners to the js-core AI SDK provider runners. Mirrors the behaviour shipped in python-server-sdk-ai #166.

Each provider model runner now keeps an internal conversation history that is seeded from the AI config's messages on construction and grows with each successful call. On every run() invocation:

  1. The user prompt is appended to the existing history before being sent to the model.
  2. If the call succeeds and produces non-empty content, the user prompt and the assistant's reply are persisted to the history.
  3. If the call fails or returns empty content, the history is left unchanged so the next call can retry cleanly.

The public run(input) interface is unchanged and still returns the same RunnerResult. The history is purely internal state on the runner instance.

Per-provider approach

  • OpenAI (OpenAIModelRunner.ts) — private LDMessage[] history, mirroring the Python OpenAI runner. The OpenAI Chat Completions API has no native conversation state.
  • LangChain (LangChainModelRunner.ts) — uses InMemoryChatMessageHistory from @langchain/core/chat_history, the JS equivalent of langchain_core.chat_history. This mirrors the Python LangChain runner.
  • Vercel (VercelModelRunner.ts) — private ModelMessage[] history (Vercel AI SDK native v5 type). No analogue exists in the Python SDK; chose ModelMessage[] to avoid re-converting on every call.

Behaviour change for multi-turn callers (medium risk)

Callers that invoke run() multiple times on the same runner instance will now accumulate conversation history across calls. Token usage will grow with each turn since the full history is sent on every call. Callers that want stateless behaviour should construct a new runner per call.

Test plan

  • Multi-turn accumulation across two successful run() calls (per provider)
  • No accumulation when run() throws (per provider)
  • No accumulation when run() returns empty content (per provider)
  • Config / system messages still prepended ahead of accumulated history on every call (per provider)
  • Existing tests still pass (43 OpenAI / 42 LangChain / 24 Vercel)
  • yarn workspaces foreach -pR --topological-dev --from '@launchdarkly/server-sdk-ai' run build succeeds
  • Lint passes for each touched package

Note

Medium Risk
Changes run() semantics for repeated calls by accumulating prior prompts/responses, which can increase token usage and alter model outputs; failures/empty content are explicitly excluded from history to reduce runaway state.

Overview
Adds stateful multi-turn support to the LangChainModelRunner, OpenAIModelRunner, and VercelModelRunner by keeping an internal conversation history seeded from config.messages and appending it to every run() call.

History is only persisted after successful calls with non-empty content; exceptions or empty/multimodal responses leave history unchanged to allow clean retries. New unit tests in each provider verify accumulation behavior and that config/system messages stay prepended ahead of the growing history.

Reviewed by Cursor Bugbot for commit b74c851. Bugbot is set up for automated code reviews on this repo. Configure here.

@github-actions

github-actions Bot commented May 7, 2026

Copy link
Copy Markdown
Contributor

@launchdarkly/js-sdk-common size report
This is the brotli compressed size of the ESM build.
Compressed size: 26281 bytes
Compressed size limit: 29000
Uncompressed size: 128971 bytes

@github-actions

github-actions Bot commented May 7, 2026

Copy link
Copy Markdown
Contributor

@launchdarkly/js-client-sdk-common size report
This is the brotli compressed size of the ESM build.
Compressed size: 38487 bytes
Compressed size limit: 39000
Uncompressed size: 211236 bytes

@github-actions

github-actions Bot commented May 7, 2026

Copy link
Copy Markdown
Contributor

@launchdarkly/js-client-sdk size report
This is the brotli compressed size of the ESM build.
Compressed size: 31906 bytes
Compressed size limit: 34000
Uncompressed size: 113658 bytes

@github-actions

github-actions Bot commented May 7, 2026

Copy link
Copy Markdown
Contributor

@launchdarkly/browser size report
This is the brotli compressed size of the ESM build.
Compressed size: 179498 bytes
Compressed size limit: 200000
Uncompressed size: 830837 bytes

Each provider model runner now keeps an internal conversation history that is
seeded from the AI config's messages and grows with each successful call. The
user prompt and the assistant's reply are appended to history only when the
call succeeds and produces non-empty content, so failed calls leave history
unchanged for retries. Mirrors python-server-sdk-ai #166.

- OpenAI: private LDMessage[] history, mirrored on the Python OpenAI runner.
- LangChain: uses InMemoryChatMessageHistory from @langchain/core/chat_history
  to mirror the Python LangChain runner.
- Vercel: private ModelMessage[] history (Vercel-native types).
@jsonbailey jsonbailey force-pushed the jb/message-history-in-providers branch from 017efa0 to b74c851 Compare May 11, 2026 14:20
@jsonbailey jsonbailey marked this pull request as ready for review May 12, 2026 13:38
@jsonbailey jsonbailey requested a review from a team as a code owner May 12, 2026 13:38
@jsonbailey jsonbailey merged commit b246631 into main May 14, 2026
46 of 48 checks passed
@jsonbailey jsonbailey deleted the jb/message-history-in-providers branch May 14, 2026 20:00
@github-actions github-actions Bot mentioned this pull request May 14, 2026
jsonbailey pushed a commit that referenced this pull request May 19, 2026
🤖 I have created a release *beep* *boop*
---


<details><summary>browser: 0.1.21</summary>

##
[0.1.21](browser-v0.1.20...browser-v0.1.21)
(2026-05-19)


### Dependencies

* The following workspace dependencies were updated
  * dependencies
    * @launchdarkly/js-client-sdk bumped from 4.6.5 to 4.7.0
</details>

<details><summary>jest: 1.0.16</summary>

##
[1.0.16](jest-v1.0.15...jest-v1.0.16)
(2026-05-19)


### Dependencies

* The following workspace dependencies were updated
  * dependencies
* @launchdarkly/react-native-client-sdk bumped from ~10.17.4 to ~10.17.5
</details>

<details><summary>js-client-sdk: 4.7.0</summary>

##
[4.7.0](js-client-sdk-v4.6.5...js-client-sdk-v4.7.0)
(2026-05-19)


### Features

* wire registerDebugOverrides through client common
([#1368](#1368))
([9011c2a](9011c2a))


### Dependencies

* The following workspace dependencies were updated
  * dependencies
    * @launchdarkly/js-client-sdk-common bumped from 1.26.3 to 1.27.0
</details>

<details><summary>js-client-sdk-common: 1.27.0</summary>

##
[1.27.0](js-client-sdk-common-v1.26.3...js-client-sdk-common-v1.27.0)
(2026-05-19)


### Features

* wire registerDebugOverrides through client common
([#1368](#1368))
([9011c2a](9011c2a))
</details>

<details><summary>react-native-client-sdk: 10.17.5</summary>

##
[10.17.5](react-native-client-sdk-v10.17.4...react-native-client-sdk-v10.17.5)
(2026-05-19)


### Dependencies

* The following workspace dependencies were updated
  * dependencies
    * @launchdarkly/js-client-sdk-common bumped from 1.26.3 to 1.27.0
</details>

<details><summary>react-sdk: 4.0.2</summary>

##
[4.0.2](react-sdk-v4.0.1...react-sdk-v4.0.2)
(2026-05-19)


### Dependencies

* The following workspace dependencies were updated
  * dependencies
    * @launchdarkly/js-client-sdk bumped from ^4.6.5 to ^4.7.0
</details>

<details><summary>server-sdk-ai: 1.0.0</summary>

##
[1.0.0](server-sdk-ai-v0.20.0...server-sdk-ai-v1.0.0)
(2026-05-19)


### ⚠ BREAKING CHANGES

* Remove bedrock-specific tracker method
([#1385](#1385))
* Remove `LDAIClient.agent` — use `LDAIClient.agentConfig` instead
* Remove `LDAIClient.agents` — use `LDAIClient.agentConfigs` instead
* Remove `LDAIClient.createChat` — use `LDAIClient.createModel` instead
* Remove `LDAIClient.initChat` — use `LDAIClient.createModel` instead
* Remove `ChatResponse` type and the `api/chat` module — use
`RunnerResult` from `api/model` instead
* Change `Judge.evaluateMessages` parameter type from `ChatResponse` to
`RunnerResult` (method retained per AI SDK spec Requirement 1.1.3)
* Remove `evaluationMetricKeys` (plural) field from `LDAIJudgeConfig`
and `LDAIJudgeConfigDefault` — use `evaluationMetricKey` (singular)
instead
* Remove `LDAIConfigTracker.trackOpenAIMetrics` — use
`tracker.trackMetricsOf(getAIMetricsFromResponse, fn)` from
`@launchdarkly/server-sdk-ai-openai` instead
* Remove `LDAIConfigTracker.trackVercelAISDKGenerateTextMetrics` — use
`tracker.trackMetricsOf(getAIMetricsFromResponse, fn)` from
`@launchdarkly/server-sdk-ai-vercel` instead
* Remove `createOpenAiUsage` helper — use `getAIMetricsFromResponse`
from `@launchdarkly/server-sdk-ai-openai` instead
* Remove `createVercelAISDKTokenUsage` helper — use
`getAIMetricsFromResponse` from `@launchdarkly/server-sdk-ai-vercel`
instead
* Remove `LDAIClient.config` — use `LDAIClient.completionConfig` instead

### Features

* Change `Judge.evaluateMessages` parameter type from `ChatResponse` to
`RunnerResult` (method retained per AI SDK spec Requirement 1.1.3)
([86951b0](86951b0))
* Remove `ChatResponse` type and the `api/chat` module — use
`RunnerResult` from `api/model` instead
([86951b0](86951b0))
* Remove `createOpenAiUsage` helper — use `getAIMetricsFromResponse`
from `@launchdarkly/server-sdk-ai-openai` instead
([86951b0](86951b0))
* Remove `createVercelAISDKTokenUsage` helper — use
`getAIMetricsFromResponse` from `@launchdarkly/server-sdk-ai-vercel`
instead
([86951b0](86951b0))
* Remove `evaluationMetricKeys` (plural) field from `LDAIJudgeConfig`
and `LDAIJudgeConfigDefault` — use `evaluationMetricKey` (singular)
instead
([86951b0](86951b0))
* Remove `LDAIClient.agent` — use `LDAIClient.agentConfig` instead
([86951b0](86951b0))
* Remove `LDAIClient.agents` — use `LDAIClient.agentConfigs` instead
([86951b0](86951b0))
* Remove `LDAIClient.config` — use `LDAIClient.completionConfig` instead
([86951b0](86951b0))
* Remove `LDAIClient.createChat` — use `LDAIClient.createModel` instead
([86951b0](86951b0))
* Remove `LDAIClient.initChat` — use `LDAIClient.createModel` instead
([86951b0](86951b0))
* Remove `LDAIConfigTracker.trackOpenAIMetrics` — use
`tracker.trackMetricsOf(getAIMetricsFromResponse, fn)` from
`@launchdarkly/server-sdk-ai-openai` instead
([86951b0](86951b0))
* Remove `LDAIConfigTracker.trackVercelAISDKGenerateTextMetrics` — use
`tracker.trackMetricsOf(getAIMetricsFromResponse, fn)` from
`@launchdarkly/server-sdk-ai-vercel` instead
([86951b0](86951b0))
* Remove bedrock-specific tracker method
([#1385](#1385))
([f7dbee8](f7dbee8))


### Bug Fixes

* Make judge runners non-multi-turn
([#1383](#1383))
([3d8f488](3d8f488))
* Move ManagedAgentGraph alongside other managed types
([#1384](#1384))
([22dd76d](22dd76d))
</details>

<details><summary>server-sdk-ai-langchain: 0.8.0</summary>

##
[0.8.0](server-sdk-ai-langchain-v0.7.0...server-sdk-ai-langchain-v0.8.0)
(2026-05-19)


### Features

* Support conversation history directly in AI Provider model runners
([#1371](#1371))
([b246631](b246631))


### Bug Fixes

* Make judge runners non-multi-turn
([#1383](#1383))
([3d8f488](3d8f488))


### Dependencies

* The following workspace dependencies were updated
  * devDependencies
    * @launchdarkly/server-sdk-ai bumped from ^0.20.0 to ^1.0.0
  * peerDependencies
    * @launchdarkly/server-sdk-ai bumped from ^0.20.0 to ^1.0.0
</details>

<details><summary>server-sdk-ai-openai: 0.7.0</summary>

##
[0.7.0](server-sdk-ai-openai-v0.6.0...server-sdk-ai-openai-v0.7.0)
(2026-05-19)


### Features

* Support conversation history directly in AI Provider model runners
([#1371](#1371))
([b246631](b246631))


### Bug Fixes

* Make judge runners non-multi-turn
([#1383](#1383))
([3d8f488](3d8f488))


### Dependencies

* The following workspace dependencies were updated
  * devDependencies
    * @launchdarkly/server-sdk-ai bumped from ^0.20.0 to ^1.0.0
  * peerDependencies
    * @launchdarkly/server-sdk-ai bumped from ^0.20.0 to ^1.0.0
</details>

<details><summary>server-sdk-ai-vercel: 0.7.0</summary>

##
[0.7.0](server-sdk-ai-vercel-v0.6.0...server-sdk-ai-vercel-v0.7.0)
(2026-05-19)


### Features

* Support conversation history directly in AI Provider model runners
([#1371](#1371))
([b246631](b246631))


### Bug Fixes

* Make judge runners non-multi-turn
([#1383](#1383))
([3d8f488](3d8f488))


### Dependencies

* The following workspace dependencies were updated
  * devDependencies
    * @launchdarkly/server-sdk-ai bumped from ^0.20.0 to ^1.0.0
  * peerDependencies
    * @launchdarkly/server-sdk-ai bumped from ^0.20.0 to ^1.0.0
</details>

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).

<!-- CURSOR_SUMMARY -->
---

> [!NOTE]
> **Medium Risk**
> Primarily a release/metadata PR (version bumps and changelog updates),
but it includes a `@launchdarkly/server-sdk-ai` major version bump to
`1.0.0`, which signals breaking API changes for downstream consumers.
> 
> **Overview**
> **Release-please version rollup.** Updates
`.release-please-manifest.json`, package `version` fields, and
associated `CHANGELOG.md` entries across the monorepo.
> 
> Notable bumps include `@launchdarkly/server-sdk-ai` to **`1.0.0`**
(breaking-change release per changelog) and propagation of dependency
bumps (`@launchdarkly/js-client-sdk-common` to `1.27.0`, browser SDK to
`4.7.0`, React Native to `10.17.5`, React SDK to `4.0.2`, and AI
provider packages to `0.7.x/0.8.0`), along with updating embedded
SDK/wrapper version strings (e.g., `BrowserInfo`, `PlatformInfo`,
`LDReactClient`).
> 
> <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit
bebd031. Bugbot is set up for automated
code reviews on this repo. Configure
[here](https://www.cursor.com/dashboard/bugbot).</sup>
<!-- /CURSOR_SUMMARY -->

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants