fix: BQ analytics plugin — fork detection, GCS offload, and agent response logging#1
Draft
caohy1988 wants to merge 1 commit intohaiyuan-eng-google:mainfrom
Draft
Conversation
79104f8 to
79245d9
Compare
79245d9 to
93d5d81
Compare
3bd8b6a to
609883c
Compare
…ponse logging Three fixes to the BigQuery Agent Analytics Plugin: 1. **False-positive fork detection after pickle (google#5528):** When deployed via Vertex AI Agent Engine, __getstate__ sets _init_pid = 0. On the server, _ensure_started() checked os.getpid() != 0 which always triggered _reset_runtime_state(), producing misleading "Fork detected" warnings and adding cold-start latency. Fix: skip reset when _init_pid == 0 (pickle sentinel), and record os.getpid() after successful startup so fork detection works for the rest of the instance lifetime. 2. **GCS text offload byte/character unit mismatch (google#5561):** The offload decision mixed inline_text_limit (32KB, byte-based) and max_content_length (character-based) in a single min() comparison. For multi-byte text (CJK, emoji), this produced false offloads. Fix: evaluate each limit in its own unit — byte_len vs inline_text_limit, char_len vs max_length — offload if either is exceeded. 3. **Missing final agent response in BQ (issue google#87):** The plugin logged LLM_RESPONSE (pre-callback raw output) and AGENT_COMPLETED (latency only, no content), but never captured the final response events emitted by agents after callback modifications. Fix: detect final response events in on_event_callback via a strict guard and log as AGENT_RESPONSE with source_event_author from event.author. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
609883c to
b2eca40
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Three fixes to the BigQuery Agent Analytics Plugin in one PR, with full test coverage.
Fix 1: False-positive fork detection after pickle (google#5528)
When deployed via Vertex AI Agent Engine,
__getstate__sets_init_pid = 0. On the server,_ensure_started()checksos.getpid() != self._init_pid— sinceos.getpid()is never0, this always triggers_reset_runtime_state(), producing a misleading "Fork detected (parent PID 0, child PID xx)" warning and adding cold-start latency.Fix:
_reset_runtime_statewhen_init_pid == 0(pickle sentinel)_init_pid = os.getpid()after successful startupos.register_at_forkand PID check (when_init_pid != 0)Fix 2: GCS text offload byte/character unit mismatch (google#5561)
The offload decision mixed
inline_text_limit(32KB, byte-based) andmax_content_length(character-based) in a singlemin(). For multi-byte text (CJK, emoji), this produced false offloads.Fix: Evaluate each limit in its own unit —
byte_lenvsinline_text_limit,char_lenvsmax_length. Offload if either is exceeded.Fix 3: Missing final agent response (haiyuan-eng-google/BigQuery-Agent-Analytics-SDK#87)
The plugin logged
LLM_RESPONSE(pre-callback raw output) andAGENT_COMPLETED(latency only), but never captured the final response events emitted by agents after callback modifications.Fix: Detect final response events in
on_event_callbackvia a strict guard and log asAGENT_RESPONSE:Each
AGENT_RESPONSErow includes:response_text— the actual formatted response contentsource_event_author— the agent that produced the response (fromevent.author)source_event_id/source_event_branch— for cross-referencingQuery the latest agent response for an invocation:
Test plan
Fork detection tests (2)
test_no_reset_after_unpickle— unpickled plugin skips reset, records_init_pid == os.getpid()test_reset_on_real_fork— stale non-zero PID triggers resetGCS offload tests (5)
test_multibyte_text_offloaded_by_byte_limit— 10K emoji (40KB) offloaded via byte limittest_ascii_under_both_limits_stays_inline— small ASCII stays inlinetest_text_exceeding_char_limit_offloaded— ASCII over char limit offloadedtest_multibyte_under_char_and_byte_limits_stays_inline— regression: 3K emoji (12K bytes) with max_length=10000 stays inlinetest_no_offloader_falls_back_to_truncate— truncates inline without offloaderAGENT_RESPONSE tests (5)
test_logs_final_text_response— logs response withsource_event_authorfromevent.authortest_skips_function_call_events— function calls not loggedtest_skips_function_response_events— function responses (even withskip_summarization) not loggedtest_skips_partial_events— partial streaming chunks not loggedtest_skips_long_running_tool_events— long-running tool pauses not logged🤖 Generated with Claude Code