Combined server enhancements: OpenAI API compliance, prompt caching, concurrency#21

Merged

eloe merged 32 commits into

mainfrom

feature/combined-bastion

Apr 9, 2026

Commits on Apr 5, 2026

feat: OpenAI Responses API compliance with tool calling support

eloe
and
claude
committed
fix: suppress tool call tokens from streaming text deltas

eloe
and
claude
committed
feat: add prompt prefix caching to server endpoints

eloe
and
claude
committed
feat: combined OpenAI Responses + prompt prefix caching

eloe
and
claude
committed
feat: add concurrency guard and finish_reason=tool_calls to combined branch

eloe
and
claude
committed
test: comprehensive tests for prompt cache, concurrency guard, finish_reason

eloe
and
claude
committed
feat: add stop sequences support for both endpoints

eloe
and
claude
committed
fix: handle TurboQuant KV cache in prompt cache trimming

eloe
and
claude
committed
Merge feature/stop-sequences into combined-bastion
eloe
committed
feat: add stop sequences, tool_choice, and TurboQuant cache fix

eloe
and
claude
committed
fix: stop sequences pass strings not token IDs to stopping criteria

eloe
and
claude
committed

Commits on Apr 6, 2026

Commits on Apr 7, 2026

Commits on Apr 8, 2026