[Bug]: /v1/messages prompt caching is a no-op — cache_control ignored, usage always 0

[jjksam]

Summary

The Anthropic-compatible /v1/messages endpoint silently ignores prompt caching. A request with cache_control returns the exact same response id and usage.input_tokens as the same request without it, and cache_creation_input_tokens / cache_read_input_tokens are always 0. Adding the anthropic-beta: prompt-caching-2024-07-31 header changes nothing.

Steps to reproduce

1. Send a /v1/messages request without cache_control; record the response id and usage.
2. Send the same request with cache_control on system; record the response.
3. Compare id, usage.input_tokens, cache_creation_input_tokens, cache_read_input_tokens.

# with cache_control
curl https://apihub.agnes-ai.com/v1/messages \
  -H "x-api-key: $AGNES_PAY_API_KEY" \
  -H "anthropic-version: 2023-06-01" \
  -H "Content-Type: application/json" \
  -d '{
    "model":"agnes-2.0-flash","max_tokens":20,
    "system":[{"type":"text","text":"You are terse.","cache_control":{"type":"ephemeral"}}],
    "messages":[{"role":"user","content":"say ok"}]
  }'

Expected behavior

Per Anthropic's prompt caching docs, marking a cache breakpoint should cause cache_creation_input_tokens > 0 on first write (and cache_read_input_tokens > 0 on a subsequent cached read). The response usage should reflect cache state.

Actual behavior

• id and usage.input_tokens are identical with and without cache_control — the field is dropped and the request is treated as a plain request.
• cache_creation_input_tokens and cache_read_input_tokens are always 0.
• The anthropic-beta: prompt-caching-2024-07-31 header has no effect.

Impact

Anthropic clients (e.g. Claude Code) attach cache_control to large system/tools blocks. With caching a no-op, every turn re-sends the full prefix → large token-cost and latency overhead, and clients that key off usage to decide cache hits break.

Environment

• Model: agnes-2.0-flash
• Endpoint: POST /v1/messages
• Client: curl (also reproducible via Claude Code pointed at the Anthropic-compatible base URL)
• API base: https://apihub.agnes-ai.com/v1

Related observation

/v1/messages returns the same id as the underlying OpenAI chat completion (a fixed hex id), and different request shapes can share the same id. If Anthropic-semantics dedup/caching is planned, an independent id namespace would avoid collisions.

[yacha6]

Thank you for the detailed report and clear reproduction steps. We’re aware of the prompt-caching behavior you described on the Anthropic-compatible /v1/messages endpoint, including the cache-related usage fields remaining at 0 and the response ID behavior.

Our team is reviewing this. We’ll share updates in this issue when more information becomes available.