Skip to content

Gemini plugin and Vercel providers build the whole history as an array on every send #3869

Open

Description

@acoliver

Part of #3868 (follow-ups to #854).

After #3852, OpenAI chat, OpenAI Responses and Anthropic request bodies are built one journal row at a time and streamed, so no whole-history array exists during a send. Two providers are a declared exception:

  • Gemini plugin (plugins/google-gemini): it sends through @google/genai generateContent/generateContentStream, which take the full request object, so the plugin reads every row into an array first.
  • Vercel (packages/providers/src/openai-vercel): the AI SDK's streamText/generateText take a message array.

Effect

Peak memory during a send on these two providers grows with context size. Retained memory between turns is unaffected, because the array is released when the call ends.

Options

  • Gemini: the plugin is now single-auth, so it could POST the REST endpoint directly with a streamed body, the same way the Responses route does.
  • Vercel: route sends through the OpenAI-compatible chat path and keep the AI SDK only where it adds something, or drop the provider if it overlaps fully with the OpenAI provider.

Until then, the array must be released at call end and must not be duplicated.

Activity

  1. added this to the 0.12.0 milestone on Oct 13, 2026
  2. added
    Context ManagementIssues related to managing the context sent to the model.
    Provider Supportthings that support an inference provider but not a particular model
    on Oct 13, 2026
  3. added theissue type on Oct 13, 2026
  4. github-actions commented on Oct 13, 2026

    @github-actions
    Contributor

    LLxprt Issue Planner — infrastructure failure

    The autonomous planner failed with exit code 1. Please inspect the workflow logs (Run planner agent section) and re-run once resolved.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Context ManagementIssues related to managing the context sent to the model.Provider Supportthings that support an inference provider but not a particular model

    Type

    Projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions