feat(review_hog): stamp ai_session_id on every review turn generation - #75592
Closed
sortafreel wants to merge 3 commits into
Closed
feat(review_hog): stamp ai_session_id on every review turn generation#75592sortafreel wants to merge 3 commits into
sortafreel wants to merge 3 commits into
Conversation
Recover the turn's sandbox runs by the Temporal workflow-id branding every run already carries (_sandbox_workflow_id_prefix), through a new read-only tasks facade helper, and stamp sandbox_task_ids + summed token totals + run count onto reviewhog_review_completed. sandbox_task_ids joins to $ai_generation.task_id for exact per-turn dollar cost — the gateway's cost numbers stay the single source of truth instead of duplicating pricing server-side. Prefix matching is separator-suffixed so PR 12 never matches PR 123, bounded to the turn's window (Temporal allows one running turn per PR), and best-effort: a lookup failure nulls the linkage properties, never the capture. Generated-By: PostHog Code Task-Id: 77db6432-21d6-400a-ad79-7ce81def58cf
Thread an ai_session_id through the tasks facade into run state (mirroring ai_stage exactly), and set it from ReviewHog to {report_id}:r{run_index} on every LLM call a turn makes — sandbox stages via Task.create_and_run, one-shots via a direct gateway header. $ai_session_id is first-class LLM-analytics taxonomy (materialized column + bloom-filter index), so a turn's generations become directly filterable by review turn and group as one session in AI observability. The sandbox path lights up once the agent-server forwards the new state key (companion PostHog/code change); the one-shot path works immediately.
Generated-By: PostHog Code
Task-Id: 77db6432-21d6-400a-ad79-7ce81def58cf
Reserved $-keys are stripped at the ai-gateway header boundary, so callers transport the session key unreserved (ai_session_id, mirroring ai_stage) and the llm-gateway callback promotes it to the native $ai_session_id — materialized column, bloom-filter index, AIO session grouping — where the event's properties are final, on both success and failure captures. Covered by a new callback test. Generated-By: PostHog Code Task-Id: 77db6432-21d6-400a-ad79-7ce81def58cf
|
Merging to
After your PR is submitted to the merge queue, this comment will be automatically updated with its status. If the PR fails, failure details will also be posted here |
Contributor
|
Hey @sortafreel! 👋 It looks like your git author email on this PR isn't your
You can fix it for this repo with: git config user.email "you@posthog.com"Or set it globally with |
2 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
ReviewHog's LLM generations can't be attributed to the PR review they served.
$ai_generationevents carry task-level keys only (task_id,ai_stage), so per-review cost or latency questions need multi-hop joins, and ad-hoc generation queries can't filter by review at all — a review turn's ~hundreds of generations have no common key.Changes
Task.create_and_runaccepts anai_session_idand stashes it in run state, mirroringai_stageexactly; threaded throughcreate_task_and_trigger/MultiTurnSession.{report_id}:r{run_index}on every LLM call a review turn makes — sandbox stages through the task chain, one-shot chunking/selection/dedup through the direct gateway header.ai_session_id) because$-keys are stripped at the ai-gateway header boundary; the llm-gateway callback promotes it to the native$ai_session_id(materialized column + bloom-filter index, AI-observability session grouping) where the event's properties are final — on success and failure captures.How did you test this code?
ai_session_idalongsideai_stage(threaded and omitted cases) — the regression being a silently dropped key that breaks the whole stamping chain downstream.ai_session_id→$ai_session_idpromotion — losing it would keep transport working while silently breaking native session grouping, which no other test observes.ruff check/ruff formatpass and all files compile; pytest and repo-wide mypy run in CI.Automatic notifications
Docs update
ARCHITECTURE.md (the product's own doc) is updated in this PR.
🤖 Agent context
Autonomy: Human-driven (agent-assisted)
/writing-testsand/writing-code-comments(repo skills, applied as gates).$ai_session_idfrom clients — the shared header builders document that$-keys are silently dropped at the ai-gateway boundary, so a reserved key would work on one routing path and vanish on the other; promotion happens where properties are final and covers both gateway routings' futures. Session id format{report_id}:r{run_index}gives per-turn equality and per-report prefix matching.Why: requested so a review turn's generations carry one queryable key — completing the cost-attribution story started in #75565.
Created with PostHog Code