Skip to content

feat(tasks): route selected sandbox products to the ai-gateway - #72770

Merged
trunk-io[bot] merged 1 commit into
masterfrom
brandon/sandbox-ai-gateway-env
Jul 29, 2026
Merged

feat(tasks): route selected sandbox products to the ai-gateway#72770
trunk-io[bot] merged 1 commit into
masterfrom
brandon/sandbox-ai-gateway-env

Conversation

@brandonleung

@brandonleung brandonleung commented Jul 21, 2026

Copy link
Copy Markdown
Contributor

Problem

Signals' sandbox stages (scout, research, implementation, repo_selection) make model calls through the legacy Python llm-gateway. To move them to the new Go ai-gateway, the sandbox needs the gateway URL and the list of products allowed to use it. Neither is plumbed in today.

Changes

Two new settings: SANDBOX_AI_GATEWAY_URL and SANDBOX_AI_GATEWAY_PRODUCTS. When both are set, they're injected into the sandbox as AI_GATEWAY_URL / AI_GATEWAY_PRODUCTS, and the agent-server (PostHog/code#3659) routes the listed products to the new gateway. Everything else stays on the Python gateway.

The sandbox blocks unknown env vars and network egress by default, so most of the diff is allowlisting:

  • The two keys are reserved (user-supplied env can't override them) and added to the agentsh env allowlist. Without the allowlist entry, agentsh strips them and the product silently stays on the old gateway.
  • The gateway hostname is added to the network allowlist in both layers (agentsh and Modal): static entries for ai-gateway.us/eu.posthog.com, plus the hostname parsed from the URL setting. The parsed entry covers dev's ai-gateway.dev.posthog.dev.
  • Setting values are validated as DNS names via Django's DomainNameValidator, so wildcards, IPs, and malformed labels narrow the policy rather than widening it. Misconfigurations log a warning.
  • The session-init egress probe now checks the gateway hosts, so a blocked gateway shows up by name in startup diagnostics.

How did you test this code?

Unit tests pin the env and network allowlists per setting, the hostname validation, the reserved-key guard, and that injection only happens when both settings are set. ruff and repo-wide mypy clean.

Inert until the sandbox chart sets both values; PostHog/code#3659 must deploy first. Rollback: clear either setting.

Refs PostHog/ai-gateway#357.

@brandonleung
brandonleung marked this pull request as ready for review July 22, 2026 03:02
@pr-assigner-resolver-posthog
pr-assigner-resolver-posthog Bot requested a review from a team July 22, 2026 03:02
@greptile-apps

greptile-apps Bot commented Jul 22, 2026

Copy link
Copy Markdown
Contributor
Prompt To Fix All With AI
Fix the following 1 code review issue. Work through them one at a time, proposing concise fixes.

---

### Issue 1 of 1
products/tasks/backend/logic/services/agentsh.py:24-26
**Configured Gateway Host Remains Blocked**

When `SANDBOX_AI_GATEWAY_URL` uses a valid host other than these two PostHog domains, restricted sandboxes receive the URL but the network policy still denies the connection. This makes the new setting fail for self-hosted or environment-specific gateway deployments unless every caller separately adds the host to `allowed_domains`.

Reviews (1): Last reviewed commit: "feat(tasks): route selected sandbox prod..." | Re-trigger Greptile

Comment on lines 24 to +26
"gateway.eu.posthog.com",
"ai-gateway.us.posthog.com",
"ai-gateway.eu.posthog.com",

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Configured Gateway Host Remains Blocked

When SANDBOX_AI_GATEWAY_URL uses a valid host other than these two PostHog domains, restricted sandboxes receive the URL but the network policy still denies the connection. This makes the new setting fail for self-hosted or environment-specific gateway deployments unless every caller separately adds the host to allowed_domains.

Prompt To Fix With AI
This is a comment left during a code review.
Path: products/tasks/backend/logic/services/agentsh.py
Line: 24-26

Comment:
**Configured Gateway Host Remains Blocked**

When `SANDBOX_AI_GATEWAY_URL` uses a valid host other than these two PostHog domains, restricted sandboxes receive the URL but the network policy still denies the connection. This makes the new setting fail for self-hosted or environment-specific gateway deployments unless every caller separately adds the host to `allowed_domains`.

How can I resolve this? If you propose a fix, please make it concise.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that's expected

@github-actions

github-actions Bot commented Jul 22, 2026

Copy link
Copy Markdown
Contributor

🤖 CI report

Playwright — all passed

All tests passed.

View test results →

⚠️ Backend snapshots — 7 updated (7 modified, 0 added, 0 deleted)

Query snapshots: Backend query snapshots updated

Changes: 7 snapshots (7 modified, 0 added, 0 deleted)

What this means:

  • Query snapshots have been automatically updated to match current output
  • These changes reflect modifications to database queries or schema

Next steps:

  • Review the query changes to ensure they're intentional
  • If unexpected, investigate what caused the query to change

Review snapshot changes →

⚠️ Backend coverage — 74.0% of changed backend lines covered — 50 uncovered

🧪 Backend test coverage

Patch coverage — changed backend lines (products + core): ███████████████░░░░░ 74.0% (143 / 193)

File Patch Uncovered changed lines
products/tasks/backend/temporal/process_task/tests/test_provision_sandbox.py 0.0% 110, 146–147, 151, 154–155, 160, 166, 179–181, 184–185, 187, 190–193, 197, 202, 204–205
products/tasks/backend/temporal/process_task/activities/get_sandbox_for_repository.py 0.0% 244
products/tasks/backend/temporal/process_task/tests/test_utils.py 0.0% 1072, 1076, 1080–1082, 1084, 1088–1091, 1093–1094, 1097–1099, 1101–1105
products/tasks/backend/temporal/process_task/utils.py 20.0% 1197, 1229–1230, 1234
products/tasks/backend/temporal/process_task/activities/provision_sandbox.py 33.3% 168, 344
products/tasks/backend/logic/services/agentsh.py 98.3% 112

🤖 Agents: add a test covering the lines above, or note why under "How did you test this code?". Machine-readable gap list: the patch-coverage artifact on this run (gh run download 30415387339 -n patch-coverage), or the coverage-data block at the end of this comment.

Per-product line coverage (touched products)
Product Coverage Lines
platform_features ██░░░░░░░░░░░░░░░░░░ 12.1% 7 / 58
batch_exports ████████░░░░░░░░░░░░ 39.5% 8,593 / 21,741
demo ███████████░░░░░░░░░ 56.2% 1,497 / 2,663
warehouse_sources_queue ████████████░░░░░░░░ 59.2% 148 / 250
tasks ██████████████░░░░░░ 69.7% 31,432 / 45,123
data_tools ██████████████░░░░░░ 70.0% 63 / 90
ai_gateway ███████████████░░░░░ 75.0% 9 / 12
cdp ████████████████░░░░ 81.0% 3,146 / 3,883
signals ████████████████░░░░ 81.1% 23,836 / 29,387
wizard █████████████████░░░ 84.8% 935 / 1,102
data_modeling █████████████████░░░ 85.3% 7,343 / 8,611
notebooks █████████████████░░░ 85.7% 7,525 / 8,785
actions █████████████████░░░ 86.6% 717 / 828
cohorts █████████████████░░░ 86.9% 5,648 / 6,496
product_tours ██████████████████░░ 87.9% 1,303 / 1,482
data_warehouse ██████████████████░░ 88.2% 12,137 / 13,757
exports ██████████████████░░ 88.4% 6,950 / 7,862
business_knowledge ██████████████████░░ 89.0% 4,391 / 4,936
engineering_analytics ██████████████████░░ 89.4% 6,441 / 7,202
dashboards ██████████████████░░ 89.5% 5,989 / 6,693
visual_review ██████████████████░░ 89.5% 5,837 / 6,522
conversations ██████████████████░░ 89.5% 17,117 / 19,124
alerts ██████████████████░░ 90.0% 4,342 / 4,827
mcp_analytics ██████████████████░░ 90.2% 2,883 / 3,198
links ██████████████████░░ 90.6% 183 / 202
streamlit_apps ██████████████████░░ 90.7% 2,630 / 2,901
error_tracking ██████████████████░░ 91.0% 10,925 / 12,004
slack_app ██████████████████░░ 91.0% 9,533 / 10,472
marketing_analytics ██████████████████░░ 91.1% 11,905 / 13,074
stamphog ██████████████████░░ 91.1% 4,056 / 4,450
mcp_store ██████████████████░░ 91.9% 4,257 / 4,634
product_analytics ███████████████████░ 92.5% 5,849 / 6,321
managed_migrations ███████████████████░ 92.6% 1,556 / 1,681
early_access_features ███████████████████░ 92.6% 1,287 / 1,390
notifications ███████████████████░ 92.6% 1,017 / 1,098
ai_observability ███████████████████░ 92.8% 15,328 / 16,517
surveys ███████████████████░ 93.1% 5,771 / 6,197
web_analytics ███████████████████░ 93.2% 14,826 / 15,906
posthog_ai ███████████████████░ 93.2% 1,326 / 1,422
approvals ███████████████████░ 93.3% 3,437 / 3,682
reminders ███████████████████░ 93.4% 468 / 501
legal_documents ███████████████████░ 93.8% 1,628 / 1,736
workflows ███████████████████░ 93.9% 6,919 / 7,372
endpoints ███████████████████░ 94.2% 8,655 / 9,192
tracing ███████████████████░ 94.5% 2,670 / 2,826
skills ███████████████████░ 94.6% 3,158 / 3,337
review_hog ███████████████████░ 94.6% 6,912 / 7,303
messaging ███████████████████░ 94.7% 2,885 / 3,048
experiments ███████████████████░ 95.4% 25,415 / 26,637
logs ███████████████████░ 95.5% 10,435 / 10,928
growth ███████████████████░ 96.1% 3,245 / 3,376
annotations ███████████████████░ 96.2% 732 / 761
revenue_analytics ███████████████████░ 96.3% 1,887 / 1,960
replay_vision ███████████████████░ 96.4% 15,773 / 16,367
feature_flags ███████████████████░ 96.4% 17,371 / 18,023
user_interviews ███████████████████░ 96.5% 2,638 / 2,734
access_control ███████████████████░ 96.9% 870 / 898
customer_analytics ███████████████████░ 97.1% 9,798 / 10,086
warehouse_sources ███████████████████░ 97.2% 346,658 / 356,568
data_catalog ████████████████████ 97.7% 2,555 / 2,615
analytics_platform ████████████████████ 98.0% 2,153 / 2,197
metrics ████████████████████ 98.2% 2,491 / 2,536
pulse ████████████████████ 98.4% 2,017 / 2,049
live_debugger ████████████████████ 99.2% 613 / 618
field_notes ████████████████████ 99.4% 158 / 159

Report-only. Patch coverage = changed backend lines covered vs origin/master. Sorted lowest first.
Known gaps: lines covered only by Temporal tests show as uncovered; core line numbers may drift if master changed the same file.

@charlesvien
charlesvien requested a review from a team July 22, 2026 16:32
@posthog-bot-comment-resolver

posthog-bot-comment-resolver Bot commented Jul 23, 2026

Copy link
Copy Markdown

🔀 Tried to auto-resolve conflicts with master but this one needs a human.

I won't retry until the branch or master moves.

@richardsolomou richardsolomou left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could you also add one activity-level case for get_sandbox_for_repository as well? deleting the environment_variables.update(ai_gateway_env_vars()) wiring there still leaves the helper and Modal provisioning tests green.

Comment thread products/tasks/backend/logic/services/agentsh.py Outdated
@trunk-io

trunk-io Bot commented Jul 28, 2026

Copy link
Copy Markdown

😎 Merged successfully - details.

@brandonleung
brandonleung enabled auto-merge (squash) July 28, 2026 21:21
@brandonleung

Copy link
Copy Markdown
Contributor Author

/trunk merge

@brandonleung
brandonleung force-pushed the brandon/sandbox-ai-gateway-env branch from 2ca20c9 to 046d59e Compare July 29, 2026 01:52
@brandonleung
brandonleung requested a review from a team as a code owner July 29, 2026 01:52
@brandonleung
brandonleung force-pushed the brandon/sandbox-ai-gateway-env branch from 046d59e to 52726aa Compare July 29, 2026 01:54
@brandonleung

Copy link
Copy Markdown
Contributor Author

/trunk merge

@brandonleung
brandonleung disabled auto-merge July 29, 2026 02:21
@trunk-io
trunk-io Bot merged commit fb4845a into master Jul 29, 2026
238 checks passed
@trunk-io
trunk-io Bot deleted the brandon/sandbox-ai-gateway-env branch July 29, 2026 03:08
@deployment-status-posthog

deployment-status-posthog Bot commented Jul 29, 2026

Copy link
Copy Markdown

Deploy status

Environment Status Deployed At Workflow
dev ✅ Deployed 2026-07-29 03:35 UTC Run
prod-us ✅ Deployed 2026-07-29 03:49 UTC Run
prod-eu ✅ Deployed 2026-07-29 03:49 UTC Run

fuziontech added a commit that referenced this pull request Jul 29, 2026
…ritable-default

* origin/master: (105 commits)
  fix(ingestion): split oversize batches instead of retrying them forever (#74531)
  fix(data-warehouse): fix clickhouse direct-query database and label (#74417)
  fix(error-tracking): only offer creatable alert wizard combinations (#74206)
  chore(experiments): remove dead AA test bayesian feature flag constants (#74524)
  fix(mcp-analytics): stop shifting dashboard buckets by the project offset (#74244)
  fix(retention): scope dwh variant scans to each arm's own entity (#74218)
  chore(customer-analytics): render billing charts via quill-charts directly (#74136)
  feat(agentic-provisioning): authenticate partners as oauth clients (#74089)
  chore(brand): bump `@posthog/brand` to 0.9.0 (#69488)
  fix(mcp-analytics): probe setup from event definitions (#74098)
  refactor(agentic-provisioning): restructure into drf class-based views (#73935)
  fix(pendo): paginate aggregation endpoints via filter cursor, not skip (#74360)
  fix(marketing-analytics): convert conversion goal revenue to the team base currency (#73047)
  feat(conversations): add access control to support tickets (#70081)
  feat(replay-vision): instrument user-facing events for GA (#74444)
  feat(tasks): route selected sandbox products to the ai-gateway (#72770)
  fix(replay-vision): honor observation naming conventions in titles (#74361)
  fix(marketing-analytics): resolve default source aliases in utm audit matching (#74052)
  fix(marketing-analytics): guard missing revenue column in bing report (#74053)
  feat(marketing-analytics): add demo data generator command (#74054)
  ...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants