Live ingestion fails with 'Server disconnected' when a single extraction LLM call exceeds ~120s

[alan5543]

Summary

Long-running, non-streaming Gemini extraction calls (≳120s of generation time) consistently fail with ServerDisconnectedError: Server disconnected in some network environments. This breaks demo/seed.py --live (and any sync whose batches produce long extraction calls), while normal Slack/Teams syncs with short messages are unaffected.

This is NOT related to the ADK 2.x migration — it was systematically root-caused during the migration's production verification (see evidence below).

Evidence (all probe-verified)

Test	Result
Direct google-genai call, 1 article (~2KB), fresh client	✅ Succeeds in 16-24s
Direct google-genai call, 7 articles (~15KB), fresh client — zero ADK/beever-atlas code	❌ ServerDisconnectedError at 123s
Same client after 4 min idle	✅ Succeeds (not a stale-connection issue)
Pipeline on ADK 1.30 (main, SequentialAgent)	❌ Identical failure
Pipeline on ADK 2.1.0 (Workflow graph)	❌ Identical failure
Pipeline via LiteLLM funnel (LLM_USE_LITELLM_FOR_GEMINI=true)	❌ Identical failure (different HTTP stack, same drop)
Pipeline with single-article batches (BATCH_MAX_PROMPT_TOKENS=800)	✅ 6/7 batches succeed, 23 facts + 165 entities ingested

Mechanism: during non-streaming generation, zero bytes flow on the connection until the full response is ready. Something in the network path (NAT/router idle timeout, ISP middlebox, or server-side limit) kills connections that stay quiet for ~2 minutes. Both aiohttp (google-genai) and LiteLLM's stack report this as "Server disconnected".

Why production syncs usually don't hit this

Slack/Teams messages are short → token-aware batching produces batches whose extraction completes in seconds. The demo corpus (7 dense Wikipedia articles, ~4K tokens) fits inside the default BATCH_MAX_PROMPT_TOKENS=6000 budget → one giant extraction call → >120s → dropped.

Suggested fixes (any of)

1. Use streaming for extraction calls (generate_content_stream) — bytes flow during generation, keeping the connection alive. Most robust fix.
2. Add a defensive cap: when a batch's estimated output tokens imply >60s generation, split it further (the output-aware batching already exists via BATCH_MAX_OUTPUT_TOKENS — its default may be too generous for slow networks).
3. Retry transport errors at the batch level: process_batch_with_retry-style splitting on ServerDisconnectedError / APIConnectionError (currently the batch just fails).
4. Document BATCH_MAX_PROMPT_TOKENS tuning for restrictive networks in the demo README.

Repro

# In an affected network environment:
python demo/seed.py --live --force          # fails: all 7 articles in one batch
BATCH_MAX_PROMPT_TOKENS=800 \
  python demo/seed.py --live --force        # mostly succeeds: 1 article per batch

🤖 Generated with Claude Code