Repository navigation
Description
Describe the Bug:
GeminiContextCacheManager contains a method _find_count_of_contents_to_cache specifically designed to exclude the dynamic instruction_provider content from the cache fingerprint — but it is never called. As a result, using the documented static_instruction + instruction pattern for context caching produces a permanently unstable fingerprint, making the cache never hit.
Root cause (code walkthrough):
static_instruction + instruction is the ADK-recommended pattern for context caching: the static part goes to system_instruction (stable, fingerprinted), while the instruction provider result is appended to llm_request.contents as a user-role Content (dynamic, should be excluded from the fingerprint).
The intended mechanism for excluding this dynamic content exists in gemini_context_cache_manager.py:
def _find_count_of_contents_to_cache(self, contents):
"""Find the number of contents to cache based on user content strategy.
Strategy: Find the last continuous batch of user contents and cache
all contents before them.
"""
last_user_batch_start = len(contents)
for i in range(len(contents) - 1, -1, -1):
if contents[i].role == "user":
last_user_batch_start = i
else:
break
return last_user_batch_start
At turn 1, with instruction_provider appending a user-role block at the end, all contents are user-role → this function returns N=0 → fingerprint = hash(system_instruction + tools) only → stable across all turns.
However, in handle_context_caching, the actual fingerprint count is computed as:
No existing cache metadata - return fingerprint-only metadata
total_contents_count = len(llm_request.contents) # ← bug: should use _find_count_of_contents_to_cache
fingerprint = self._generate_cache_fingerprint(llm_request, total_contents_count)
return CacheMetadata(fingerprint=fingerprint, contents_count=total_contents_count)
_find_count_of_contents_to_cache is defined but never called anywhere in the codebase.
Why this breaks turn-by-turn:
Turn 1 contents (after instruction_provider appends): [user_msg_1, dynamic_ctx_t1] → N=2, fingerprint covers both
Turn 2 contents (first N=2): [user_msg_1, model_resp_1] — model response now occupies the slot where dynamic_ctx_t1 was
Fingerprint mismatch → N reset to 4 (total contents) → same problem repeats every turn
Cache is never created
Steps to Reproduce:
Create an LlmAgent with static_instruction (stable string) and instruction (dynamic provider returning session-dependent content)
Enable ContextCacheConfig on the App
Run a multi-turn conversation
Enable GOOGLE_ADK_LOG_LEVEL=DEBUG and observe logs
Expected Behavior:
The instruction_provider content (user-role, appended at end of contents) is excluded from the cache fingerprint. The fingerprint covers only system_instruction + tools, which is stable across turns. The cache is created on turn 2 and reused on subsequent turns as long as system_instruction and tools do not change.
Observed Behavior:
The fingerprint includes the instruction_provider content (via len(llm_request.contents)). Since that content changes each turn (or is displaced by the model's response in the first-N window), the fingerprint changes on every turn. Debug logs show:
Cache content fingerprint mismatch
Fingerprints don't match, returning fingerprint-only metadata
The cache is never created. cache_hit_pct = 0%.
Proposed Fix:
In handle_context_caching, replace len(llm_request.contents) with the existing (but uncalled) _find_count_of_contents_to_cache:
Before (buggy):
total_contents_count = len(llm_request.contents)
After (fix):
total_contents_count = self._find_count_of_contents_to_cache(llm_request.contents)
This aligns the implementation with the documented static_instruction + instruction pattern and with the evident design intent of _find_count_of_contents_to_cache.
Environment Details:
ADK Library Version: google-adk==1.32.0
Desktop OS: macOS (Darwin 24.6.0)
Python Version: 3.13.11
Model Information:
LiteLLM: No
Model: gemini-2.0-flash-lite (Gemini API)
🟡 Optional Information
Minimal Reproduction Code:
from google.adk.agents import LlmAgent
from google.adk.apps.app import App
from google.adk.agents.context_cache_config import ContextCacheConfig
from google.adk.agents.readonly_context import ReadonlyContext
from google.adk.models import Gemini
_STATIC_PROMPT = "You are a helpful assistant. " * 300 # large enough to exceed 4096 tokens with tools
def dynamic_instruction(context: ReadonlyContext) -> str:
# Simulates per-turn dynamic content (e.g. session state)
return f"<session_state>turn_data={context.state.get('turn', 0)}</session_state>"
agent = LlmAgent(
name="test_agent",
model=Gemini(model="gemini-2.0-flash-lite"),
static_instruction=_STATIC_PROMPT,
instruction=dynamic_instruction,
)
app = App(
name="test",
root_agent=agent,
context_cache_config=ContextCacheConfig(ttl_seconds=1800, min_tokens=4096),
)
Run multi-turn: observe "fingerprint mismatch" in DEBUG logs on every turn
How often has this issue occurred?: Always (100%)
Additional Context:
The workaround is to inject dynamic content via a before_model_callback that calls llm_request.contents.insert(0, ...) instead of using instruction_provider. Because the dynamic block is then at position 0 on every turn, the first-N fingerprint window consistently starts with it, and the fingerprint is stable as long as the dynamic content itself doesn't change. This is semantically equivalent to the intended instruction_provider behavior but should not be necessary.
Activity
rshashank17 commentedon Jul 1, 2026
Hey there! Wow, thank you so much for this incredibly detailed and thorough bug report. We really appreciate you diving deep into the
GeminiContextCacheManagerand pinpointing exactly why thestatic_instructionandinstructionpattern isn't working as expected for context caching.Your explanation of
_find_count_of_contents_to_cachebeing defined but never called, and the precise code walkthrough showinglen(llm_request.contents)being used instead of the intended logic, makes the root cause crystal clear. The minimal reproduction code and the debug log observations ('Cache content fingerprint mismatch') are extremely helpful for confirming this behavior ingoogle-adk==1.32.0.It definitely sounds like a legitimate bug in how the context cache fingerprint is computed, preventing cache hits when dynamic instruction content is present. We'll take a closer look at your proposed fix in
handle_context_cachingto useself._find_count_of_contents_to_cache(llm_request.contents).Thanks again for not only identifying this issue but also for providing such a comprehensive analysis, reproduction steps, and a clear path to resolution!
github-actions commentedon Oct 8, 2026
Are you satisfied with the resolution of your issue?
Yes
No