Skip to content

Cache probe (RFC-0015): track the §10 known limitations #180

Description

@eric8810

RFC-0015 (cache trace/probe) is marked IMPLEMENTED, but it carries a dedicated "§10 已知限制与后续工作" section with eight open items that nothing currently tracks. Listing them verbatim as the checklist:

  • Unprototyped scenarios: Anthropic 3-field/20-block (R-1.4), Bedrock quota (R-1.5), response cache headers (R-4.1), gateway stripping (R-4.2), aggregation A1–A5, concurrent shard locks — tests to be added at implementation stage.
  • [UNVERIFIED] external behaviors: Anthropic cache_read reporting accuracy; Gemini API-side TTL; OpenAI 5.6+ "minimum retention" upper-bound semantics for ttl; whether vLLM V1 null is fixed.
  • Real-provider replay verification: Azure 128-quantization vs OpenAI 5.6+ byte-exact dual-track dispatch, and byte-proxy deviation — needs real API response replays.
  • Prototype sync: the (j+1)B block upper bound and byte-proxy W→B capping rules (F2/F3) need syncing into the prototype verdict logic.
  • FFI meta size: serializing a 200 KB request_body doubles through FFI; add a meta_cap_bytes config.
  • RFC-0014 dependency: hang probe data off the span tree (generate → http_request) and add the TTFT observation point.
  • RFC-0024 dependency: switch probe session grouping to CallOptions.session_id once it lands (replacing wrapper-instance injection).
  • Cluster/routed deployments: node-local caches are not shared, so 0% hit-rate is the norm behind routers; R-2.2's low-hit warning is currently suppressed to a note when route_affinity_known=false. Decide whether anything stronger is possible or this stays a documented limitation.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions