Skip to content

fix(openai): omit prompt_cache_key for openai-compatible providers - #636

Merged
kevincodex1 merged 1 commit into
mainfrom
fix/omit-prompt-cache-key-compatible
Jul 10, 2026
Merged

fix(openai): omit prompt_cache_key for openai-compatible providers#636
kevincodex1 merged 1 commit into
mainfrom
fix/omit-prompt-cache-key-compatible

Conversation

@gnanam1990

@gnanam1990 gnanam1990 commented Jul 10, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • Interactive Zero always has a session ID, so completion requests were sending OpenAI’s prompt_cache_key.
  • Strict openai-compatible gateways (e.g. NVIDIA NIM) reject that field with Unsupported parameter(s): prompt_cache_key, while plain zero exec usually has no session and works.
  • This PR disables prompt_cache_key for ProviderKindOpenAICompatible and keeps it for official OpenAI cache routing.
  • ZERO_DISABLE_PROMPT_CACHE_KEY=1 remains a global kill switch.

Fixes #624

Changes

  • internal/providers/openai/provider.go — add DisablePromptCacheKey option and gate request mapping
  • internal/providers/factory.go — set the flag for openai-compatible profiles only
  • Tests covering OpenAI vs openai-compatible serialization

Test plan

  • go test ./internal/providers/openai/ ./internal/providers/ -count=1 -run 'TestOpenAIRequestPromptCacheKey|TestPromptCacheKeyOnlyOnOfficialOpenAI'
  • Interactive zero against an openai-compatible provider (e.g. NIM) — send a prompt, no prompt_cache_key validation error
  • Official OpenAI multi-turn still receives prompt_cache_key when a session is present
  • ZERO_DISABLE_PROMPT_CACHE_KEY=1 still forces omit on OpenAI

Summary by CodeRabbit

  • Bug Fixes

    • OpenAI-compatible gateways no longer receive the OpenAI-specific prompt_cache_key field, preventing request rejections.
    • Official OpenAI requests continue to include the prompt cache key when provided.
    • Existing environment-based controls for disabling prompt cache keys remain supported.
  • Tests

    • Added coverage verifying prompt cache key behavior for official OpenAI and compatible gateways.

Interactive TUI always has a session ID, so every completion request
forwarded OpenAI's prompt_cache_key. Strict openai-compatible gateways
(e.g. NVIDIA NIM) reject unknown fields with a 400, while plain zero
exec usually has no session and omits the field.

Disable prompt_cache_key for ProviderKindOpenAICompatible; keep it for
official OpenAI. ZERO_DISABLE_PROMPT_CACHE_KEY remains a global kill
switch.

Fixes #624
@github-actions

Copy link
Copy Markdown
Contributor

Zero automated PR review

Verdict: No blockers found

Blockers

  • None found.

Validation

  • [pass] Diff hygiene: git diff --check
  • [pass] Tests: go test ./...
  • [pass] Build: go run ./cmd/zero-release build
  • [pass] Smoke build: go run ./cmd/zero-release smoke

Scope

Head: 346776be01e7
Changed files (5): internal/providers/factory.go, internal/providers/factory_test.go, internal/providers/openai/provider.go, internal/providers/openai/provider_test.go, internal/providers/openai/types.go

This deterministic review checks validation status and basic diff hygiene. A human reviewer still owns product judgment and design quality.

@coderabbitai

coderabbitai Bot commented Jul 10, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 71b2a8de-f044-4ede-ab94-86e877850cd6

📥 Commits

Reviewing files that changed from the base of the PR and between dc06fe7 and 346776b.

📒 Files selected for processing (5)
  • internal/providers/factory.go
  • internal/providers/factory_test.go
  • internal/providers/openai/provider.go
  • internal/providers/openai/provider_test.go
  • internal/providers/openai/types.go

Walkthrough

Changes

Prompt cache key routing

Layer / File(s) Summary
Provider option wiring
internal/providers/openai/provider.go, internal/providers/openai/types.go, internal/providers/factory.go
Adds DisablePromptCacheKey and enables it when constructing OpenAI-compatible providers.
Request filtering and validation
internal/providers/openai/provider.go, internal/providers/openai/provider_test.go, internal/providers/factory_test.go
Omits prompt_cache_key for compatible providers while retaining it for official OpenAI, with provider and factory tests covering serialization.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Possibly related PRs

  • Gitlawb/zero#515: Related OpenAI provider prompt-cache key forwarding and kill-switch behavior.

Suggested reviewers: Vasanthdev2004, kevincodex1

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 66.67% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately summarizes the main change: omitting prompt_cache_key for OpenAI-compatible providers.
Linked Issues check ✅ Passed The changes address #624 by disabling prompt_cache_key for OpenAI-compatible providers and adding coverage for the behavior.
Out of Scope Changes check ✅ Passed The remaining changes are supporting code and tests for the same prompt_cache_key behavior, with no clear unrelated additions.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/omit-prompt-cache-key-compatible

Comment @coderabbitai help to get the list of available commands.

@kevincodex1
kevincodex1 merged commit 1af5882 into main Jul 10, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

interactive shell sends prompt_cache_key on chat-completions endpoints but zero exec doesn't

2 participants