From a4868f6f68cfdbf043e5d8ac165c16383899ad0b Mon Sep 17 00:00:00 2001 From: Richard Solomou Date: Fri, 31 Jul 2026 12:27:52 +0300 Subject: [PATCH 01/20] chore(ai-gateway): document migration parity Generated-By: PostHog Code Task-Id: e2e45fbc-1c5e-4ef2-9688-bcc941c2229c --- AGENTS.md | 1 + services/llm-gateway/PARITY.md | 37 ++++++++++++++++++++++++++++++++++ 2 files changed, 38 insertions(+) create mode 100644 services/llm-gateway/PARITY.md diff --git a/AGENTS.md b/AGENTS.md index 1626806b59c5..73923894a387 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -165,6 +165,7 @@ See [.agents/security.md](.agents/security.md) for security guidelines — least - **Object storage is SeaweedFS — do not add new MinIO dependencies.** Both S3-compatible stores in the dev/CI stack are SeaweedFS: the `objectstorage` service (S3 API on `:19000`) backs general object storage (`OBJECT_STORAGE_*` settings — exports, media uploads, error-tracking source maps, query cache, tasks), and the `seaweedfs` service (S3 API on `:8333`) backs session replay v2 (`SESSION_RECORDING_V2_S3_*` settings). MinIO now survives only as migration tooling: `docker-compose.hobby.yml` keeps it as a source for `bin/migrate-storage-hobby`, and `bin/upgrade-objectstorage` starts a throwaway MinIO to salvage objects off the pre-swap volume. Outside that, don't add docker-compose services, scripts, tests, or docs that stand up a `minio/minio` container. Code that talks to object storage should go through the existing `OBJECT_STORAGE_*` / `SESSION_RECORDING_V2_S3_*` config and a standard S3 client rather than hardcoding an endpoint — that keeps backends swappable. Note the `objectstorage` service registers its credentials at runtime via a bootstrap loop and returns `InvalidAccessKeyId` until that completes, so anything depending on it must wait for its readiness sentinel rather than just for the container to start. - **Temporal activity payloads have a ~2 MiB hard limit — pass large data by reference, not by value.** Activity inputs and outputs are serialized across a gRPC boundary that Temporal caps at ~2 MiB per payload (the server rejects larger payloads via `blobSizeLimitError`). As a conservative field-level rule, if a field could exceed ~256 KB once serialized (serialized query results, exported file contents, LLM context, rendered HTML, image bytes, unbounded `list[dict[str, Any]]`), write it to Postgres / S3 / object storage from _inside_ the activity and return only the reference (row ID, S3 key). The workflow already has access to any row ID created earlier in the same run; it does not need the content to flow back through. Shuttling large data through the workflow on the way to persistence is a foreseeable failure mode that produces `PayloadSizeError` (`TMPRL1103`) the moment the underlying data crosses the limit. - **Outbound calls to a third-party API that need rate-limiting or egress telemetry belong in `posthog/egress/` — add a `/` incarnation (GitHub is the reference) and route callers through its gated, recorded transport, never hand-rolled `requests`. See `posthog/egress/README.md`.** +- **Prefer [`PostHog/ai-gateway`](https://github.com/PostHog/ai-gateway) over `services/llm-gateway` for new LLM gateway callers when it supports the required contract.** Use the shared builders in `posthog/llm/gateway_client.py` for callers that need a temporary fallback to the Python gateway. Before migrating or adding a caller, read [`services/llm-gateway/PARITY.md`](services/llm-gateway/PARITY.md) and verify its authentication, attribution, billing, API shape, model, and provider requirements. Do not add new behavior to both gateways by default; add it to the Go gateway or document the parity blocker that requires the Python gateway. ## Code Style diff --git a/services/llm-gateway/PARITY.md b/services/llm-gateway/PARITY.md new file mode 100644 index 000000000000..ada50a5574fa --- /dev/null +++ b/services/llm-gateway/PARITY.md @@ -0,0 +1,37 @@ +# AI gateway parity + +Use [`PostHog/ai-gateway`](https://github.com/PostHog/ai-gateway) for new LLM gateway callers when it supports the caller's contract. `services/llm-gateway` remains available for callers that depend on a gap below. + +This is a migration checklist, not a promise that similarly named endpoints behave identically. Check the current implementations before moving a caller: + +- Python gateway: [`services/llm-gateway`](./README.md) +- Go gateway: [`PostHog/ai-gateway`](https://github.com/PostHog/ai-gateway), especially `docs/product.md` +- Shared PostHog client builders: [`posthog/llm/gateway_client.py`](../../posthog/llm/gateway_client.py) + +## Supported migration baseline + +The Go gateway supports the main unprefixed OpenAI Chat Completions, OpenAI Responses, Anthropic Messages, Anthropic token-counting, and model-list surfaces. It supports streaming and non-streaming calls, `phs_` and `pha_` credentials, end-user attribution with `X-PostHog-Distinct-Id`, trace attribution with `X-PostHog-Trace-Id`, and event properties with the `X-PostHog-Properties` JSON header. + +Use `build_openai_client`, `build_async_openai_client`, or `build_async_anthropic_client` from `posthog/llm/gateway_client.py` when an existing Django caller can use this baseline. These builders select the Go gateway when configured and retain the Python gateway fallback during rollout. + +## Known parity gaps + +| Area | Python gateway | Go gateway | Migration check | +| --- | --- | --- | --- | +| Product policy and attribution | Product-prefixed routes select model allowlists, authentication rules, billing, quotas, and `$ai_product`. | Routes are not product-prefixed. Callers can attach `ai_product` as an event property, but product policy and budgets are not equivalent yet. | Do not migrate a caller that relies on Python `ProductConfig` enforcement or an unbilled product until the Go gateway has the matching trusted policy. | +| Billing and limits | Supports per-product and per-user cost limits, plan and quota checks, billable credit buckets, and unbilled products. | Uses prepaid wallet admission and a front-line rate limit. The credential billing mode is carried but does not yet change settlement. | Confirm the caller should debit the Go gateway wallet and does not require Python quota or free-tier behavior. | +| Authentication | Accepts `phx_` personal API keys and `pha_` OAuth tokens, with product-specific rules. | Accepts `phs_` project secret keys and `pha_` OAuth tokens from the gateway credential cache. | Provision a supported credential and confirm its scope, revocation, team, and model allowlist projections. | +| Provider coverage | Supports direct OpenAI and Anthropic plus Bedrock, OpenRouter, Fireworks, Cloudflare Workers AI, Modal, and Baseten routes. | Supports OpenAI, Anthropic, Azure OpenAI, Bedrock, and selected Modal-hosted models. | Keep callers that require OpenRouter, Fireworks, Cloudflare, or Baseten on Python until that provider and model are in the Go catalog. | +| API surfaces | Adds product-prefixed variants and OpenAI audio transcription. Its usage endpoint reports product cost and quota status. | Adds normalized router, request-usage, wallet, and idempotency surfaces, but has no audio transcription endpoint. | Check the exact path and response contract. The two usage APIs are different products, not aliases. | +| Model identifiers and translation | LiteLLM accepts a broad set of model IDs and can route Anthropic models through the OpenAI-compatible surface. | The catalog owns canonical models and aliases. The normalized router translates selected Anthropic models to OpenAI shapes, but the reverse translation is not available. | Confirm the requested model is in `GET /v1/models` and works on the chosen API shape. | +| Provider selection and fallback | Supports explicit Bedrock selection and opt-in Bedrock fallback headers, plus gateway-specific routing for several hosted models. | Chooses between configured hosts using health and priority, with strict `X-PostHog-Provider` pinning when needed. | Verify whether the caller requires a fixed provider, automatic failover, or the Python fallback semantics. | +| Request metadata | Accepts per-key property and feature-flag headers. | Accepts one `X-PostHog-Properties` JSON object and dedicated distinct-id, trace-id, and provider headers. | Convert metadata to the JSON header. Do not assume Python's per-key or feature-flag headers are read. | + +## Adding a caller + +1. Start with the Go gateway and a shared builder. +2. Check every applicable row above against the caller's actual request, credential, model, provider, attribution, and billing behavior. +3. If a gap blocks the caller, keep the Python fallback and record the specific gap in the change. Avoid adding a second implementation to both gateways unless both services must own the behavior during migration. +4. Test the request against the selected gateway. Include streaming, provider errors, attribution properties, and billing behavior when the caller depends on them. + +Update this file when a gap closes or a new incompatibility is found. Keep detailed Go gateway behavior in its own `docs/product.md`; this file should only describe migration-relevant differences. From a05e4f2c23c5ecfe4cbfb01c11f0ce51f4e79fe8 Mon Sep 17 00:00:00 2001 From: Richard Solomou Date: Fri, 31 Jul 2026 12:43:43 +0300 Subject: [PATCH 02/20] chore(ai-gateway): format parity guide Generated-By: PostHog Code Task-Id: e2e45fbc-1c5e-4ef2-9688-bcc941c2229c --- services/llm-gateway/PARITY.md | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/services/llm-gateway/PARITY.md b/services/llm-gateway/PARITY.md index ada50a5574fa..3410ee34bb0e 100644 --- a/services/llm-gateway/PARITY.md +++ b/services/llm-gateway/PARITY.md @@ -16,16 +16,16 @@ Use `build_openai_client`, `build_async_openai_client`, or `build_async_anthropi ## Known parity gaps -| Area | Python gateway | Go gateway | Migration check | -| --- | --- | --- | --- | -| Product policy and attribution | Product-prefixed routes select model allowlists, authentication rules, billing, quotas, and `$ai_product`. | Routes are not product-prefixed. Callers can attach `ai_product` as an event property, but product policy and budgets are not equivalent yet. | Do not migrate a caller that relies on Python `ProductConfig` enforcement or an unbilled product until the Go gateway has the matching trusted policy. | -| Billing and limits | Supports per-product and per-user cost limits, plan and quota checks, billable credit buckets, and unbilled products. | Uses prepaid wallet admission and a front-line rate limit. The credential billing mode is carried but does not yet change settlement. | Confirm the caller should debit the Go gateway wallet and does not require Python quota or free-tier behavior. | -| Authentication | Accepts `phx_` personal API keys and `pha_` OAuth tokens, with product-specific rules. | Accepts `phs_` project secret keys and `pha_` OAuth tokens from the gateway credential cache. | Provision a supported credential and confirm its scope, revocation, team, and model allowlist projections. | -| Provider coverage | Supports direct OpenAI and Anthropic plus Bedrock, OpenRouter, Fireworks, Cloudflare Workers AI, Modal, and Baseten routes. | Supports OpenAI, Anthropic, Azure OpenAI, Bedrock, and selected Modal-hosted models. | Keep callers that require OpenRouter, Fireworks, Cloudflare, or Baseten on Python until that provider and model are in the Go catalog. | -| API surfaces | Adds product-prefixed variants and OpenAI audio transcription. Its usage endpoint reports product cost and quota status. | Adds normalized router, request-usage, wallet, and idempotency surfaces, but has no audio transcription endpoint. | Check the exact path and response contract. The two usage APIs are different products, not aliases. | -| Model identifiers and translation | LiteLLM accepts a broad set of model IDs and can route Anthropic models through the OpenAI-compatible surface. | The catalog owns canonical models and aliases. The normalized router translates selected Anthropic models to OpenAI shapes, but the reverse translation is not available. | Confirm the requested model is in `GET /v1/models` and works on the chosen API shape. | -| Provider selection and fallback | Supports explicit Bedrock selection and opt-in Bedrock fallback headers, plus gateway-specific routing for several hosted models. | Chooses between configured hosts using health and priority, with strict `X-PostHog-Provider` pinning when needed. | Verify whether the caller requires a fixed provider, automatic failover, or the Python fallback semantics. | -| Request metadata | Accepts per-key property and feature-flag headers. | Accepts one `X-PostHog-Properties` JSON object and dedicated distinct-id, trace-id, and provider headers. | Convert metadata to the JSON header. Do not assume Python's per-key or feature-flag headers are read. | +| Area | Python gateway | Go gateway | Migration check | +| --------------------------------- | --------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------ | +| Product policy and attribution | Product-prefixed routes select model allowlists, authentication rules, billing, quotas, and `$ai_product`. | Routes are not product-prefixed. Callers can attach `ai_product` as an event property, but product policy and budgets are not equivalent yet. | Do not migrate a caller that relies on Python `ProductConfig` enforcement or an unbilled product until the Go gateway has the matching trusted policy. | +| Billing and limits | Supports per-product and per-user cost limits, plan and quota checks, billable credit buckets, and unbilled products. | Uses prepaid wallet admission and a front-line rate limit. The credential billing mode is carried but does not yet change settlement. | Confirm the caller should debit the Go gateway wallet and does not require Python quota or free-tier behavior. | +| Authentication | Accepts `phx_` personal API keys and `pha_` OAuth tokens, with product-specific rules. | Accepts `phs_` project secret keys and `pha_` OAuth tokens from the gateway credential cache. | Provision a supported credential and confirm its scope, revocation, team, and model allowlist projections. | +| Provider coverage | Supports direct OpenAI and Anthropic plus Bedrock, OpenRouter, Fireworks, Cloudflare Workers AI, Modal, and Baseten routes. | Supports OpenAI, Anthropic, Azure OpenAI, Bedrock, and selected Modal-hosted models. | Keep callers that require OpenRouter, Fireworks, Cloudflare, or Baseten on Python until that provider and model are in the Go catalog. | +| API surfaces | Adds product-prefixed variants and OpenAI audio transcription. Its usage endpoint reports product cost and quota status. | Adds normalized router, request-usage, wallet, and idempotency surfaces, but has no audio transcription endpoint. | Check the exact path and response contract. The two usage APIs are different products, not aliases. | +| Model identifiers and translation | LiteLLM accepts a broad set of model IDs and can route Anthropic models through the OpenAI-compatible surface. | The catalog owns canonical models and aliases. The normalized router translates selected Anthropic models to OpenAI shapes, but the reverse translation is not available. | Confirm the requested model is in `GET /v1/models` and works on the chosen API shape. | +| Provider selection and fallback | Supports explicit Bedrock selection and opt-in Bedrock fallback headers, plus gateway-specific routing for several hosted models. | Chooses between configured hosts using health and priority, with strict `X-PostHog-Provider` pinning when needed. | Verify whether the caller requires a fixed provider, automatic failover, or the Python fallback semantics. | +| Request metadata | Accepts per-key property and feature-flag headers. | Accepts one `X-PostHog-Properties` JSON object and dedicated distinct-id, trace-id, and provider headers. | Convert metadata to the JSON header. Do not assume Python's per-key or feature-flag headers are read. | ## Adding a caller From 1a76aa8c30dcd8a0795a1bd2c394ca222b2e88c5 Mon Sep 17 00:00:00 2001 From: Richard Solomou Date: Fri, 31 Jul 2026 12:44:39 +0300 Subject: [PATCH 03/20] chore(ai-gateway): simplify parity guide Generated-By: PostHog Code Task-Id: e2e45fbc-1c5e-4ef2-9688-bcc941c2229c --- services/llm-gateway/PARITY.md | 58 +++++++++++++++++++--------------- 1 file changed, 32 insertions(+), 26 deletions(-) diff --git a/services/llm-gateway/PARITY.md b/services/llm-gateway/PARITY.md index 3410ee34bb0e..048c27f2908c 100644 --- a/services/llm-gateway/PARITY.md +++ b/services/llm-gateway/PARITY.md @@ -1,37 +1,43 @@ -# AI gateway parity +# Choosing an AI gateway -Use [`PostHog/ai-gateway`](https://github.com/PostHog/ai-gateway) for new LLM gateway callers when it supports the caller's contract. `services/llm-gateway` remains available for callers that depend on a gap below. +✅ **Default to [`PostHog/ai-gateway`](https://github.com/PostHog/ai-gateway)** for new callers. -This is a migration checklist, not a promise that similarly named endpoints behave identically. Check the current implementations before moving a caller: +⚠️ **Keep `services/llm-gateway`** when the Go gateway does not support the caller's contract. -- Python gateway: [`services/llm-gateway`](./README.md) -- Go gateway: [`PostHog/ai-gateway`](https://github.com/PostHog/ai-gateway), especially `docs/product.md` -- Shared PostHog client builders: [`posthog/llm/gateway_client.py`](../../posthog/llm/gateway_client.py) +## ✅ Ready on the Go gateway -## Supported migration baseline +- OpenAI Chat Completions and Responses +- Anthropic Messages and token counting +- Streaming and non-streaming requests +- `phs_` and `pha_` credentials +- Distinct ID, trace, and custom property attribution +- OpenAI, Anthropic, Azure OpenAI, Bedrock, and selected Modal models -The Go gateway supports the main unprefixed OpenAI Chat Completions, OpenAI Responses, Anthropic Messages, Anthropic token-counting, and model-list surfaces. It supports streaming and non-streaming calls, `phs_` and `pha_` credentials, end-user attribution with `X-PostHog-Distinct-Id`, trace attribution with `X-PostHog-Trace-Id`, and event properties with the `X-PostHog-Properties` JSON header. +Existing Django callers should use `build_openai_client`, `build_async_openai_client`, or `build_async_anthropic_client` from [`posthog/llm/gateway_client.py`](../../posthog/llm/gateway_client.py). These builders keep the Python fallback during rollout. -Use `build_openai_client`, `build_async_openai_client`, or `build_async_anthropic_client` from `posthog/llm/gateway_client.py` when an existing Django caller can use this baseline. These builders select the Go gateway when configured and retain the Python gateway fallback during rollout. +## ⚠️ Check before migrating -## Known parity gaps +| Area | Blocker | Action | +| ---------------------- | ----------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------- | +| Product policy | The Go gateway does not enforce Python `ProductConfig` rules or trusted `$ai_product` policy. | Keep callers that depend on product model allowlists, auth rules, or unbilled products on Python. | +| Billing and limits | Go uses prepaid wallet admission. Python has product and user limits, plan checks, quotas, and unbilled products. | Confirm the call should debit the Go wallet. | +| Authentication | Python accepts `phx_` and `pha_`. Go accepts `phs_` and `pha_`. | Provision a supported credential and verify its scope, team, and model allowlist. | +| Providers | Go does not yet support OpenRouter, Fireworks, Cloudflare, or Baseten. | Keep models on Python until their provider is in the Go catalog. | +| API surfaces | Go has no audio transcription endpoint. The gateways also expose different usage APIs. | Check the exact path and response contract. | +| Models and translation | Go owns a stricter model catalog and does not translate OpenAI models to Anthropic Messages. | Confirm the model appears in `GET /v1/models` and supports the chosen API shape. | +| Fallback behavior | Python has opt-in Bedrock fallback. Go routes by host health unless `X-PostHog-Provider` pins a host. | Confirm the required provider and fallback behavior. | +| Request metadata | Go reads `X-PostHog-Properties`, not Python's per-key property or feature flag headers. | Convert metadata to the JSON properties header. | -| Area | Python gateway | Go gateway | Migration check | -| --------------------------------- | --------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------ | -| Product policy and attribution | Product-prefixed routes select model allowlists, authentication rules, billing, quotas, and `$ai_product`. | Routes are not product-prefixed. Callers can attach `ai_product` as an event property, but product policy and budgets are not equivalent yet. | Do not migrate a caller that relies on Python `ProductConfig` enforcement or an unbilled product until the Go gateway has the matching trusted policy. | -| Billing and limits | Supports per-product and per-user cost limits, plan and quota checks, billable credit buckets, and unbilled products. | Uses prepaid wallet admission and a front-line rate limit. The credential billing mode is carried but does not yet change settlement. | Confirm the caller should debit the Go gateway wallet and does not require Python quota or free-tier behavior. | -| Authentication | Accepts `phx_` personal API keys and `pha_` OAuth tokens, with product-specific rules. | Accepts `phs_` project secret keys and `pha_` OAuth tokens from the gateway credential cache. | Provision a supported credential and confirm its scope, revocation, team, and model allowlist projections. | -| Provider coverage | Supports direct OpenAI and Anthropic plus Bedrock, OpenRouter, Fireworks, Cloudflare Workers AI, Modal, and Baseten routes. | Supports OpenAI, Anthropic, Azure OpenAI, Bedrock, and selected Modal-hosted models. | Keep callers that require OpenRouter, Fireworks, Cloudflare, or Baseten on Python until that provider and model are in the Go catalog. | -| API surfaces | Adds product-prefixed variants and OpenAI audio transcription. Its usage endpoint reports product cost and quota status. | Adds normalized router, request-usage, wallet, and idempotency surfaces, but has no audio transcription endpoint. | Check the exact path and response contract. The two usage APIs are different products, not aliases. | -| Model identifiers and translation | LiteLLM accepts a broad set of model IDs and can route Anthropic models through the OpenAI-compatible surface. | The catalog owns canonical models and aliases. The normalized router translates selected Anthropic models to OpenAI shapes, but the reverse translation is not available. | Confirm the requested model is in `GET /v1/models` and works on the chosen API shape. | -| Provider selection and fallback | Supports explicit Bedrock selection and opt-in Bedrock fallback headers, plus gateway-specific routing for several hosted models. | Chooses between configured hosts using health and priority, with strict `X-PostHog-Provider` pinning when needed. | Verify whether the caller requires a fixed provider, automatic failover, or the Python fallback semantics. | -| Request metadata | Accepts per-key property and feature-flag headers. | Accepts one `X-PostHog-Properties` JSON object and dedicated distinct-id, trace-id, and provider headers. | Convert metadata to the JSON header. Do not assume Python's per-key or feature-flag headers are read. | - -## Adding a caller +## Migration checklist 1. Start with the Go gateway and a shared builder. -2. Check every applicable row above against the caller's actual request, credential, model, provider, attribution, and billing behavior. -3. If a gap blocks the caller, keep the Python fallback and record the specific gap in the change. Avoid adding a second implementation to both gateways unless both services must own the behavior during migration. -4. Test the request against the selected gateway. Include streaming, provider errors, attribution properties, and billing behavior when the caller depends on them. +2. Check the caller against every applicable blocker above. +3. Keep the Python fallback when blocked. Record the specific gap in the change. +4. Test the selected gateway, including streaming, provider errors, attribution, and billing when relevant. + +## References + +- Python gateway: [`services/llm-gateway`](./README.md) +- Go gateway: [`PostHog/ai-gateway`](https://github.com/PostHog/ai-gateway), especially `docs/product.md` -Update this file when a gap closes or a new incompatibility is found. Keep detailed Go gateway behavior in its own `docs/product.md`; this file should only describe migration-relevant differences. +Update this file when a gap closes or a new incompatibility is found. From 999f75958245294b9bb28dca3cfe39606140ad9c Mon Sep 17 00:00:00 2001 From: Richard Solomou Date: Fri, 31 Jul 2026 13:15:46 +0300 Subject: [PATCH 04/20] chore(ai-gateway): formalize migration guidance Generated-By: PostHog Code Task-Id: e2e45fbc-1c5e-4ef2-9688-bcc941c2229c --- .../migrating-llm-gateway-callers/SKILL.md | 91 +++++++++++++++ AGENTS.md | 3 +- services/llm-gateway/PARITY.md | 106 +++++++++++++----- 3 files changed, 172 insertions(+), 28 deletions(-) create mode 100644 .agents/skills/migrating-llm-gateway-callers/SKILL.md diff --git a/.agents/skills/migrating-llm-gateway-callers/SKILL.md b/.agents/skills/migrating-llm-gateway-callers/SKILL.md new file mode 100644 index 000000000000..433336cc441d --- /dev/null +++ b/.agents/skills/migrating-llm-gateway-callers/SKILL.md @@ -0,0 +1,91 @@ +--- +name: migrating-llm-gateway-callers +description: > + Guides migration from services/llm-gateway to PostHog/ai-gateway and refreshes the gateway parity record. Use when adding or migrating an LLM caller; changing either gateway's auth, attribution, billing, endpoint, provider, model, routing, or metadata contract; modifying posthog/llm/gateway_client.py or gateway settings; reviewing a services/llm-gateway change; or updating services/llm-gateway/PARITY.md. +--- + +# Migrating LLM gateway callers + +The Go gateway is the default. Treat `services/llm-gateway` as frozen unless an active caller is blocked by a verified parity gap. + +Read [`services/llm-gateway/PARITY.md`](../../../services/llm-gateway/PARITY.md) before changing a caller or either gateway contract. + +## Decide where work belongs + +1. Describe the caller's identity, billing owner, API shape, model, provider, metadata, and failure requirements. +2. Match those requirements against the use-case and parity tables in `PARITY.md`. +3. Use the Go gateway when every required contract is supported. +4. Keep or change the Python gateway only for a named blocker that affects an active caller. + +An existing Python integration is not a blocker by itself. Prefer a shared builder in `posthog/llm/gateway_client.py` when it supports the caller. + +## Gate Python gateway changes + +Before editing `services/llm-gateway`, establish all of the following: + +- The active caller and user-visible or operational need are known. +- The missing Go contract is identified from implementation evidence. +- Implementing or waiting for the Go contract is not suitable for this change. +- The Python change is limited to the blocker and does not create a broader competing feature. +- The PR explains the blocker and migration condition. + +If the Go gateway already supports the contract, migrate the caller instead. If neither gateway supports it, add it to Go unless the blocked caller needs a temporary Python fix. + +## Refresh parity from source + +Audit current default branches. Do not rely on the existing parity table or README summaries alone. + +### 1. Record source revisions + +- Record `origin/master` for `PostHog/posthog`. +- Read the current `PostHog/ai-gateway` main SHA with `gh api repos/PostHog/ai-gateway/commits/main`. +- Use an authenticated checkout of `PostHog/ai-gateway` when code inspection is needed. + +### 2. Inspect the contracts + +For the Python gateway, inspect: + +- `services/llm-gateway/src/llm_gateway/api/` for routes and wire behavior +- `services/llm-gateway/src/llm_gateway/auth/` for accepted credentials +- `services/llm-gateway/src/llm_gateway/products/config.py` for trusted product policy, models, and billing +- `services/llm-gateway/src/llm_gateway/rate_limiting/` for budgets and limits +- `services/llm-gateway/src/llm_gateway/callbacks/` for event attribution +- `posthog/llm/gateway_client.py` and real call sites for migration needs + +For the Go gateway, inspect: + +- `internal/httpapi/routes.go` and dispatch packages for API shapes +- `internal/auth/` and `internal/principal/` for credential and identity policy +- `internal/httpapi/admission.go`, `internal/ledger/`, and `internal/quota/` for billing and limits +- `internal/catalog/`, `internal/router/`, and `internal/dispatch/` for models, providers, translation, and failover +- `internal/emitter/` and request parsing for attribution and metadata +- `docs/product.md` for intended contracts and explicitly deferred work, verified against code + +### 3. Classify each difference + +- ✅ **Supported:** a caller can migrate without losing a required contract. +- ⛔ **Blocking:** an active use case would lose auth, trusted attribution, billing policy, API, provider, or wire behavior. +- 🔎 **Verify:** support exists, but the caller must confirm configuration or behavior. + +Do not call telemetry labels trusted attribution. A caller-supplied `ai_product` property does not replace product authentication, authorization, or billing policy. + +### 4. Update the parity record + +Update `services/llm-gateway/PARITY.md` when evidence changes: + +- Move use cases between sections. +- Add or remove only migration-relevant contracts. +- Update the source SHAs and verification date. +- Keep the document decision-oriented. Put detailed Go design in `PostHog/ai-gateway`. + +When a gap closes, identify PostHog callers that can migrate. A parity refresh is incomplete if the table changes but obvious affected callers remain unmentioned in the PR. + +## Validate + +Run: + +```sh +pnpm exec oxfmt services/llm-gateway/PARITY.md .agents/skills/migrating-llm-gateway-callers/SKILL.md +pnpm exec markdownlint-cli2 --config .config/.markdownlint-cli2.jsonc services/llm-gateway/PARITY.md .agents/skills/migrating-llm-gateway-callers/SKILL.md +git diff --check +``` diff --git a/AGENTS.md b/AGENTS.md index 73923894a387..5efd1f846ae4 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -165,7 +165,7 @@ See [.agents/security.md](.agents/security.md) for security guidelines — least - **Object storage is SeaweedFS — do not add new MinIO dependencies.** Both S3-compatible stores in the dev/CI stack are SeaweedFS: the `objectstorage` service (S3 API on `:19000`) backs general object storage (`OBJECT_STORAGE_*` settings — exports, media uploads, error-tracking source maps, query cache, tasks), and the `seaweedfs` service (S3 API on `:8333`) backs session replay v2 (`SESSION_RECORDING_V2_S3_*` settings). MinIO now survives only as migration tooling: `docker-compose.hobby.yml` keeps it as a source for `bin/migrate-storage-hobby`, and `bin/upgrade-objectstorage` starts a throwaway MinIO to salvage objects off the pre-swap volume. Outside that, don't add docker-compose services, scripts, tests, or docs that stand up a `minio/minio` container. Code that talks to object storage should go through the existing `OBJECT_STORAGE_*` / `SESSION_RECORDING_V2_S3_*` config and a standard S3 client rather than hardcoding an endpoint — that keeps backends swappable. Note the `objectstorage` service registers its credentials at runtime via a bootstrap loop and returns `InvalidAccessKeyId` until that completes, so anything depending on it must wait for its readiness sentinel rather than just for the container to start. - **Temporal activity payloads have a ~2 MiB hard limit — pass large data by reference, not by value.** Activity inputs and outputs are serialized across a gRPC boundary that Temporal caps at ~2 MiB per payload (the server rejects larger payloads via `blobSizeLimitError`). As a conservative field-level rule, if a field could exceed ~256 KB once serialized (serialized query results, exported file contents, LLM context, rendered HTML, image bytes, unbounded `list[dict[str, Any]]`), write it to Postgres / S3 / object storage from _inside_ the activity and return only the reference (row ID, S3 key). The workflow already has access to any row ID created earlier in the same run; it does not need the content to flow back through. Shuttling large data through the workflow on the way to persistence is a foreseeable failure mode that produces `PayloadSizeError` (`TMPRL1103`) the moment the underlying data crosses the limit. - **Outbound calls to a third-party API that need rate-limiting or egress telemetry belong in `posthog/egress/` — add a `/` incarnation (GitHub is the reference) and route callers through its gated, recorded transport, never hand-rolled `requests`. See `posthog/egress/README.md`.** -- **Prefer [`PostHog/ai-gateway`](https://github.com/PostHog/ai-gateway) over `services/llm-gateway` for new LLM gateway callers when it supports the required contract.** Use the shared builders in `posthog/llm/gateway_client.py` for callers that need a temporary fallback to the Python gateway. Before migrating or adding a caller, read [`services/llm-gateway/PARITY.md`](services/llm-gateway/PARITY.md) and verify its authentication, attribution, billing, API shape, model, and provider requirements. Do not add new behavior to both gateways by default; add it to the Go gateway or document the parity blocker that requires the Python gateway. +- **`services/llm-gateway` is under an unofficial code freeze while callers move to [`PostHog/ai-gateway`](https://github.com/PostHog/ai-gateway).** New callers and features belong on the Go gateway by default. A Python gateway change needs a documented parity blocker for an active caller and must stay limited to that blocker. Read [`services/llm-gateway/PARITY.md`](services/llm-gateway/PARITY.md) and invoke `/migrating-llm-gateway-callers` before adding or migrating a caller, or changing either gateway contract. ## Code Style @@ -239,3 +239,4 @@ ALWAYS invoke the matching skill **before** writing or reviewing code in these a - `/authoring-ci-workflows` — adding or editing any `.github/workflows` workflow, composite action, or reusable workflow - `/reviewing-personhog-protocol` — any personhog coordination-protocol change (leases, fencing, handoffs, supervisors, budgets, warming, changelog semantics), and any request for an exhaustive review of personhog code - `/gating-production-deploys` — any workflow that builds and pushes a production image or dispatches a deploy +- `/migrating-llm-gateway-callers` — adding or migrating an LLM gateway caller; changing `services/llm-gateway`, `posthog/llm/gateway_client.py`, gateway settings, auth, attribution, billing, providers, models, or API contracts; or refreshing gateway parity documentation diff --git a/services/llm-gateway/PARITY.md b/services/llm-gateway/PARITY.md index 048c27f2908c..5d04ee6b40e2 100644 --- a/services/llm-gateway/PARITY.md +++ b/services/llm-gateway/PARITY.md @@ -1,43 +1,95 @@ # Choosing an AI gateway -✅ **Default to [`PostHog/ai-gateway`](https://github.com/PostHog/ai-gateway)** for new callers. +✅ **Default to [`PostHog/ai-gateway`](https://github.com/PostHog/ai-gateway)** for new callers and features. -⚠️ **Keep `services/llm-gateway`** when the Go gateway does not support the caller's contract. +⚠️ **Use `services/llm-gateway` temporarily** only when an active caller is blocked by a gap below. -## ✅ Ready on the Go gateway +## Migration policy -- OpenAI Chat Completions and Responses -- Anthropic Messages and token counting -- Streaming and non-streaming requests -- `phs_` and `pha_` credentials -- Distinct ID, trace, and custom property attribution -- OpenAI, Anthropic, Azure OpenAI, Bedrock, and selected Modal models +`services/llm-gateway` is under an unofficial code freeze. Everyone should move to the Go gateway unless they can name a contract it does not support. -Existing Django callers should use `build_openai_client`, `build_async_openai_client`, or `build_async_anthropic_client` from [`posthog/llm/gateway_client.py`](../../posthog/llm/gateway_client.py). These builders keep the Python fallback during rollout. +A PR that changes the Python gateway must: -## ⚠️ Check before migrating +1. Name the active caller and its use case. +2. Name the exact Go gateway parity gap. +3. Explain why the change cannot wait for or land in the Go gateway. +4. Limit the Python change to the blocked caller's needs. +5. Update this document when it discovers or closes a gap. -| Area | Blocker | Action | -| ---------------------- | ----------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------- | -| Product policy | The Go gateway does not enforce Python `ProductConfig` rules or trusted `$ai_product` policy. | Keep callers that depend on product model allowlists, auth rules, or unbilled products on Python. | -| Billing and limits | Go uses prepaid wallet admission. Python has product and user limits, plan checks, quotas, and unbilled products. | Confirm the call should debit the Go wallet. | -| Authentication | Python accepts `phx_` and `pha_`. Go accepts `phs_` and `pha_`. | Provision a supported credential and verify its scope, team, and model allowlist. | -| Providers | Go does not yet support OpenRouter, Fireworks, Cloudflare, or Baseten. | Keep models on Python until their provider is in the Go catalog. | -| API surfaces | Go has no audio transcription endpoint. The gateways also expose different usage APIs. | Check the exact path and response contract. | -| Models and translation | Go owns a stricter model catalog and does not translate OpenAI models to Anthropic Messages. | Confirm the model appears in `GET /v1/models` and supports the chosen API shape. | -| Fallback behavior | Python has opt-in Bedrock fallback. Go routes by host health unless `X-PostHog-Provider` pins a host. | Confirm the required provider and fallback behavior. | -| Request metadata | Go reads `X-PostHog-Properties`, not Python's per-key property or feature flag headers. | Convert metadata to the JSON properties header. | +Bug, security, and reliability fixes for blocked callers are valid reasons. Convenience, an existing Python client, or an unfamiliar Go API are not parity gaps. + +When a gap closes, new work uses the Go gateway and affected callers should migrate. Do not add the same feature to both gateways by default. + +## Choose by use case + +### ✅ Use the Go gateway + +| Use case | Why it fits | +| ----------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------ | +| Customer LLM traffic | The caller uses a project secret or OAuth credential, the team wallet should pay, and standard OpenAI or Anthropic APIs are enough. | +| Server-to-server PostHog call | A `phs_` credential, team wallet billing, and informational event properties provide enough policy and attribution. | +| Stock SDK proxy | The caller needs OpenAI Chat Completions or Responses, Anthropic Messages or token counting, streaming, idempotency, or the model catalog. | +| Gateway-managed routing | The caller accepts Go host selection across OpenAI, Anthropic, Azure OpenAI, Bedrock, or selected Modal models. | + +Existing Django callers should use `build_openai_client`, `build_async_openai_client`, or `build_async_anthropic_client` from [`posthog/llm/gateway_client.py`](../../posthog/llm/gateway_client.py). These builders keep a temporary Python fallback during rollout. + +### ⛔ Stay on the Python gateway for now + +| Use case | Blocking contract | +| ------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| PostHog Desktop, PostHog Code, cloud agents, onboarding, Wizard, and similar first-party products | Trusted product identity, OAuth application allowlists, server-minted credential requirements, per-product model policy, and product billing or unbilled policy are not available together in Go. An `ai_product` event property is not an authorization or billing boundary. | +| Product or user budget enforcement | Python supports product and user cost limits, plan checks, quota buckets, and unbilled products. Go currently admits against the team wallet and does not branch settlement on credential billing mode. | +| OpenRouter, Fireworks, Cloudflare Workers AI, or Baseten | These provider paths are not available in Go. | +| OpenAI audio transcription | Go has no transcription endpoint. | +| OpenAI models through Anthropic Messages | Go does not provide this reverse translation. | +| Python product usage status | The Python product usage and quota API is different from Go request usage and wallet APIs. | + +### 🔎 Verify before switching + +These are compatibility checks, not automatic blockers: + +- **Model:** it appears in `GET /v1/models` and supports the requested API shape and capabilities. +- **Credential:** Go accepts the credential type and projects the correct team, scope, and revocation state. +- **Billing:** the request should reserve and debit the Go wallet. +- **Attribution:** `X-PostHog-Distinct-Id`, `X-PostHog-Trace-Id`, and `X-PostHog-Properties` provide enough event context. Caller-supplied properties are not trusted policy. +- **Provider behavior:** health-based routing or strict `X-PostHog-Provider` pinning matches the caller's fallback requirements. +- **Wire behavior:** request fields, streaming chunks, errors, timeouts, and retries match what the caller handles. +- **Metadata:** Python per-key property and feature flag headers are converted to the Go JSON properties header. + +## Parity map + +| Contract | Go gateway today | Python-only behavior that can block migration | +| ---------------------------- | --------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------- | +| Authentication | `phs_` project secrets and `pha_` OAuth credentials with `llm_gateway:read`; credential resolves team and revocation state. | `phx_` personal keys and product-specific API key or OAuth application rules. | +| Trusted first-party identity | No trusted product identity or product-scoped authorization boundary. | Product route selects allowlisted OAuth apps, server credential requirements, and model policy. | +| Attribution | Distinct ID, trace ID, custom JSON properties, team project token, provider, usage, and cost. | Product route owns `$ai_product` and `$ai_billable`; per-key properties and feature flag headers. | +| Billing and limits | Wallet reservation and settlement, idempotency, balance read, request usage read, and front-line rate limiting. | Product and user cost windows, plans, quota buckets, free tiers, unbilled products, and product usage status. | +| OpenAI APIs | Chat Completions, Responses, bare Responses alias, and normalized router chat. | Audio transcription and broader LiteLLM translation. | +| Anthropic APIs | Messages and token counting, including Bedrock-hosted models. | OpenAI models exposed through the Anthropic shape and Python-specific Bedrock opt-in behavior. | +| Providers | OpenAI, Anthropic, Azure OpenAI, Bedrock, and selected Modal models. | OpenRouter, Fireworks, Cloudflare Workers AI, and Baseten. | +| Models | Gateway-owned catalog, canonical IDs and aliases, capability checks, and router categories. | Broader LiteLLM model acceptance and Python product allowlists. | +| Routing and failure behavior | Health-based host choice, circuit breakers, hosted-provider failover, and strict provider pinning. | Caller opt-in Bedrock fallback and provider-specific Python routing. | +| Event metadata | One `X-PostHog-Properties` JSON object plus dedicated distinct ID, trace ID, and provider headers. | `X-POSTHOG-PROPERTY-*` and `X-POSTHOG-FLAG-*` headers. | ## Migration checklist -1. Start with the Go gateway and a shared builder. -2. Check the caller against every applicable blocker above. -3. Keep the Python fallback when blocked. Record the specific gap in the change. -4. Test the selected gateway, including streaming, provider errors, attribution, and billing when relevant. +1. Describe the caller and identify which identity controls its access and spend. +2. Check every relevant contract above against the real request and response. +3. Start with a shared Go-capable builder where one exists. +4. Test success, streaming if used, provider errors, attribution, and billing. +5. Keep the Python fallback only for a named blocker. Record that blocker in the PR. + +## Refreshing this document + +Run `/migrating-llm-gateway-callers` after either gateway changes auth, attribution, billing, endpoints, providers, models, routing, or event metadata. The skill audits implementation sources in both repositories and updates this file. + +Last verified on 2026-07-31 against: + +- `PostHog/posthog` master at `801dfa4bd67e534ee4c09ec4adc21cde66d3497d` +- `PostHog/ai-gateway` main at `ea8230b6dbc4ebf6c4be83d359e561477a13b215` ## References - Python gateway: [`services/llm-gateway`](./README.md) +- Shared clients: [`posthog/llm/gateway_client.py`](../../posthog/llm/gateway_client.py) - Go gateway: [`PostHog/ai-gateway`](https://github.com/PostHog/ai-gateway), especially `docs/product.md` - -Update this file when a gap closes or a new incompatibility is found. From 7501b886a077266a3278de439004455c4634953c Mon Sep 17 00:00:00 2001 From: Richard Solomou Date: Fri, 31 Jul 2026 13:20:40 +0300 Subject: [PATCH 05/20] chore(ai-gateway): split parity and migration skills Generated-By: PostHog Code Task-Id: e2e45fbc-1c5e-4ef2-9688-bcc941c2229c --- .../auditing-llm-gateway-parity/SKILL.md | 70 +++++++++++ .../migrating-llm-gateway-callers/SKILL.md | 110 ++++++++---------- AGENTS.md | 5 +- services/llm-gateway/PARITY.md | 4 +- 4 files changed, 123 insertions(+), 66 deletions(-) create mode 100644 .agents/skills/auditing-llm-gateway-parity/SKILL.md diff --git a/.agents/skills/auditing-llm-gateway-parity/SKILL.md b/.agents/skills/auditing-llm-gateway-parity/SKILL.md new file mode 100644 index 000000000000..51b06e061603 --- /dev/null +++ b/.agents/skills/auditing-llm-gateway-parity/SKILL.md @@ -0,0 +1,70 @@ +--- +name: auditing-llm-gateway-parity +description: > + Audits services/llm-gateway against PostHog/ai-gateway and updates services/llm-gateway/PARITY.md from current implementation evidence. Use when either gateway changes auth, attribution, billing, endpoints, providers, models, routing, or metadata; when reviewing a Python gateway change; or when asked to refresh, verify, or report gateway parity. This skill updates the parity record but does not migrate callers. +--- + +# Auditing LLM gateway parity + +Compare the current default branches and update [`services/llm-gateway/PARITY.md`](../../../services/llm-gateway/PARITY.md). Keep this audit separate from caller migration. + +## Record source revisions + +- Record `origin/master` for `PostHog/posthog`. +- Read the current `PostHog/ai-gateway` main SHA with `gh api repos/PostHog/ai-gateway/commits/main`. +- Use an authenticated checkout of `PostHog/ai-gateway` when code inspection is needed. + +Audit implementation code. README files and the existing parity table are starting points, not proof. + +## Inspect each contract + +For the Python gateway, inspect: + +- `services/llm-gateway/src/llm_gateway/api/` for routes and wire behavior +- `services/llm-gateway/src/llm_gateway/auth/` for accepted credentials +- `services/llm-gateway/src/llm_gateway/products/config.py` for trusted product policy, models, and billing +- `services/llm-gateway/src/llm_gateway/rate_limiting/` for budgets and limits +- `services/llm-gateway/src/llm_gateway/callbacks/` for event attribution +- `posthog/llm/gateway_client.py` and real call sites for required contracts + +For the Go gateway, inspect: + +- `internal/httpapi/routes.go` and dispatch packages for API shapes +- `internal/auth/` and `internal/principal/` for credential and identity policy +- `internal/httpapi/admission.go`, `internal/ledger/`, and `internal/quota/` for billing and limits +- `internal/catalog/`, `internal/router/`, and `internal/dispatch/` for models, providers, translation, and failover +- `internal/emitter/` and request parsing for attribution and metadata +- `docs/product.md` for intended and deferred contracts, verified against code + +Check both request and response behavior. Matching route names do not prove parity for headers, streaming, errors, timeouts, retries, billing, or emitted events. + +## Classify the evidence + +- ✅ **Supported:** the Go gateway satisfies the contract. +- ⛔ **Blocking:** an active use case would lose auth, trusted attribution, billing policy, API, provider, or wire behavior. +- 🔎 **Verify:** support exists, but a caller must confirm configuration or behavior. + +Do not treat caller-supplied telemetry as trusted policy. An `ai_product` event property does not replace product authentication, authorization, or billing. + +For each difference, identify at least one affected use-case class. Remove details that do not change a migration decision. + +## Update the parity record + +Update `services/llm-gateway/PARITY.md` with the evidence: + +- Move use cases between supported, blocked, and verify sections. +- Add or remove migration-relevant contracts in the parity map. +- Update the source SHAs and verification date. +- Keep detailed Go design in `PostHog/ai-gateway`. + +When a gap closes, name obvious caller classes that are newly eligible to migrate. Do not modify those callers unless the user also asks for migration work. Use `/migrating-llm-gateway-callers` for that separate job. + +## Validate + +Run: + +```sh +pnpm exec oxfmt services/llm-gateway/PARITY.md .agents/skills/auditing-llm-gateway-parity/SKILL.md +pnpm exec markdownlint-cli2 --config .config/.markdownlint-cli2.jsonc services/llm-gateway/PARITY.md .agents/skills/auditing-llm-gateway-parity/SKILL.md +git diff --check +``` diff --git a/.agents/skills/migrating-llm-gateway-callers/SKILL.md b/.agents/skills/migrating-llm-gateway-callers/SKILL.md index 433336cc441d..161f2ad53616 100644 --- a/.agents/skills/migrating-llm-gateway-callers/SKILL.md +++ b/.agents/skills/migrating-llm-gateway-callers/SKILL.md @@ -1,91 +1,75 @@ --- name: migrating-llm-gateway-callers description: > - Guides migration from services/llm-gateway to PostHog/ai-gateway and refreshes the gateway parity record. Use when adding or migrating an LLM caller; changing either gateway's auth, attribution, billing, endpoint, provider, model, routing, or metadata contract; modifying posthog/llm/gateway_client.py or gateway settings; reviewing a services/llm-gateway change; or updating services/llm-gateway/PARITY.md. + Migrates an LLM caller from services/llm-gateway to PostHog/ai-gateway. Use when adding a gateway caller, converting an existing Python gateway integration, adopting shared Go-capable client builders, changing gateway URLs or headers for a caller, or removing a Python fallback. Inventories the caller's contract, checks the parity record, implements the supported migration, updates tests, and stops with a documented blocker when Go parity is missing. --- # Migrating LLM gateway callers -The Go gateway is the default. Treat `services/llm-gateway` as frozen unless an active caller is blocked by a verified parity gap. +Use [`services/llm-gateway/PARITY.md`](../../../services/llm-gateway/PARITY.md) as the current decision record. If it is stale or the migration reveals a new gap, run `/auditing-llm-gateway-parity` before continuing. -Read [`services/llm-gateway/PARITY.md`](../../../services/llm-gateway/PARITY.md) before changing a caller or either gateway contract. +## Inventory the caller -## Decide where work belongs +Find the production call site, client construction, settings, deployment wiring, and tests. Record: -1. Describe the caller's identity, billing owner, API shape, model, provider, metadata, and failure requirements. -2. Match those requirements against the use-case and parity tables in `PARITY.md`. -3. Use the Go gateway when every required contract is supported. -4. Keep or change the Python gateway only for a named blocker that affects an active caller. +- caller and user-facing use case +- credential source and required authorization policy +- billing owner, budgets, and whether the call is intentionally unbilled +- API shape, model, provider, streaming, and tool or structured-output requirements +- distinct ID, trace, product, team, feature flag, and custom property attribution +- timeout, retry, fallback, and error-handling behavior +- usage or quota APIs read by the caller -An existing Python integration is not a blocker by itself. Prefer a shared builder in `posthog/llm/gateway_client.py` when it supports the caller. +Search for the product name, `LLM_GATEWAY`, `AI_GATEWAY`, gateway client helpers, and gateway headers. Follow configuration into sandbox or deployment code when the call does not run in Django. -## Gate Python gateway changes +## Check whether migration is supported -Before editing `services/llm-gateway`, establish all of the following: +Match every required contract to the parity record and current code. -- The active caller and user-visible or operational need are known. -- The missing Go contract is identified from implementation evidence. -- Implementing or waiting for the Go contract is not suitable for this change. -- The Python change is limited to the blocker and does not create a broader competing feature. -- The PR explains the blocker and migration condition. +- If all required contracts are supported, continue with the migration. +- If a 🔎 item applies, verify it against the caller's actual model, request, and configuration. +- If a ⛔ gap applies, stop the migration. Report the exact blocker and keep the Python path. -If the Go gateway already supports the contract, migrate the caller instead. If neither gateway supports it, add it to Go unless the blocked caller needs a temporary Python fix. +An existing Python client or product route is not a blocker by itself. -## Refresh parity from source +## Implement the migration -Audit current default branches. Do not rely on the existing parity table or README summaries alone. +Prefer the smallest existing pattern that matches the caller: -### 1. Record source revisions +1. Use `build_openai_client`, `build_async_openai_client`, or `build_async_anthropic_client` from `posthog/llm/gateway_client.py` for Django callers when possible. +2. Use the slugless Go base URL. Do not carry a Python `/{product}/` path into the Go URL. +3. Use a supported `phs_` or `pha_` credential with `llm_gateway:read`. Do not weaken auth or expose a shared secret to an untrusted runtime. +4. Send event labels in one `X-PostHog-Properties` JSON object. Use the dedicated distinct ID and trace ID headers where required. +5. Treat `ai_product` as telemetry only. Do not use it to replace trusted product auth or billing policy. +6. Confirm the canonical model and API shape against the Go model catalog. +7. Preserve the caller's provider and fallback requirements. Use provider pinning only when the caller requires a specific host. +8. Keep a Python fallback only when rollout needs it, and make the switch explicit in settings or the shared builder. -- Record `origin/master` for `PostHog/posthog`. -- Read the current `PostHog/ai-gateway` main SHA with `gh api repos/PostHog/ai-gateway/commits/main`. -- Use an authenticated checkout of `PostHog/ai-gateway` when code inspection is needed. +For sandbox callers, follow the existing `SANDBOX_AI_GATEWAY_URL` and product rollout patterns rather than inventing another environment contract. -### 2. Inspect the contracts +## Update tests and docs -For the Python gateway, inspect: +Invoke `/writing-tests` before changing tests. -- `services/llm-gateway/src/llm_gateway/api/` for routes and wire behavior -- `services/llm-gateway/src/llm_gateway/auth/` for accepted credentials -- `services/llm-gateway/src/llm_gateway/products/config.py` for trusted product policy, models, and billing -- `services/llm-gateway/src/llm_gateway/rate_limiting/` for budgets and limits -- `services/llm-gateway/src/llm_gateway/callbacks/` for event attribution -- `posthog/llm/gateway_client.py` and real call sites for migration needs +Cover the observable migration contract at the lowest useful level: -For the Go gateway, inspect: +- selected base URL and absence of the Python product slug +- credential and required headers without asserting secret values +- converted JSON properties and trace attribution +- model and API shape +- rollout fallback when one remains +- failure behavior the caller handles -- `internal/httpapi/routes.go` and dispatch packages for API shapes -- `internal/auth/` and `internal/principal/` for credential and identity policy -- `internal/httpapi/admission.go`, `internal/ledger/`, and `internal/quota/` for billing and limits -- `internal/catalog/`, `internal/router/`, and `internal/dispatch/` for models, providers, translation, and failover -- `internal/emitter/` and request parsing for attribution and metadata -- `docs/product.md` for intended contracts and explicitly deferred work, verified against code +Update nearby docs and settings descriptions when the operator workflow changes. Do not update the parity record unless implementation evidence changed; use `/auditing-llm-gateway-parity` when it did. -### 3. Classify each difference +## Verify -- ✅ **Supported:** a caller can migrate without losing a required contract. -- ⛔ **Blocking:** an active use case would lose auth, trusted attribution, billing policy, API, provider, or wire behavior. -- 🔎 **Verify:** support exists, but the caller must confirm configuration or behavior. +Run the narrow caller tests and the formatter or type checker for touched code. Exercise a real request when credentials and a safe development gateway are available. Confirm the event attribution and billing behavior when those contracts matter. -Do not call telemetry labels trusted attribution. A caller-supplied `ai_product` property does not replace product authentication, authorization, or billing policy. +Before handing off, summarize: -### 4. Update the parity record - -Update `services/llm-gateway/PARITY.md` when evidence changes: - -- Move use cases between sections. -- Add or remove only migration-relevant contracts. -- Update the source SHAs and verification date. -- Keep the document decision-oriented. Put detailed Go design in `PostHog/ai-gateway`. - -When a gap closes, identify PostHog callers that can migrate. A parity refresh is incomplete if the table changes but obvious affected callers remain unmentioned in the PR. - -## Validate - -Run: - -```sh -pnpm exec oxfmt services/llm-gateway/PARITY.md .agents/skills/migrating-llm-gateway-callers/SKILL.md -pnpm exec markdownlint-cli2 --config .config/.markdownlint-cli2.jsonc services/llm-gateway/PARITY.md .agents/skills/migrating-llm-gateway-callers/SKILL.md -git diff --check -``` +- migrated caller and use case +- Go contracts relied on +- fallback left in place, if any +- parity blocker, if migration stopped +- checks run and any environment limitation diff --git a/AGENTS.md b/AGENTS.md index 5efd1f846ae4..ed5696812259 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -165,7 +165,7 @@ See [.agents/security.md](.agents/security.md) for security guidelines — least - **Object storage is SeaweedFS — do not add new MinIO dependencies.** Both S3-compatible stores in the dev/CI stack are SeaweedFS: the `objectstorage` service (S3 API on `:19000`) backs general object storage (`OBJECT_STORAGE_*` settings — exports, media uploads, error-tracking source maps, query cache, tasks), and the `seaweedfs` service (S3 API on `:8333`) backs session replay v2 (`SESSION_RECORDING_V2_S3_*` settings). MinIO now survives only as migration tooling: `docker-compose.hobby.yml` keeps it as a source for `bin/migrate-storage-hobby`, and `bin/upgrade-objectstorage` starts a throwaway MinIO to salvage objects off the pre-swap volume. Outside that, don't add docker-compose services, scripts, tests, or docs that stand up a `minio/minio` container. Code that talks to object storage should go through the existing `OBJECT_STORAGE_*` / `SESSION_RECORDING_V2_S3_*` config and a standard S3 client rather than hardcoding an endpoint — that keeps backends swappable. Note the `objectstorage` service registers its credentials at runtime via a bootstrap loop and returns `InvalidAccessKeyId` until that completes, so anything depending on it must wait for its readiness sentinel rather than just for the container to start. - **Temporal activity payloads have a ~2 MiB hard limit — pass large data by reference, not by value.** Activity inputs and outputs are serialized across a gRPC boundary that Temporal caps at ~2 MiB per payload (the server rejects larger payloads via `blobSizeLimitError`). As a conservative field-level rule, if a field could exceed ~256 KB once serialized (serialized query results, exported file contents, LLM context, rendered HTML, image bytes, unbounded `list[dict[str, Any]]`), write it to Postgres / S3 / object storage from _inside_ the activity and return only the reference (row ID, S3 key). The workflow already has access to any row ID created earlier in the same run; it does not need the content to flow back through. Shuttling large data through the workflow on the way to persistence is a foreseeable failure mode that produces `PayloadSizeError` (`TMPRL1103`) the moment the underlying data crosses the limit. - **Outbound calls to a third-party API that need rate-limiting or egress telemetry belong in `posthog/egress/` — add a `/` incarnation (GitHub is the reference) and route callers through its gated, recorded transport, never hand-rolled `requests`. See `posthog/egress/README.md`.** -- **`services/llm-gateway` is under an unofficial code freeze while callers move to [`PostHog/ai-gateway`](https://github.com/PostHog/ai-gateway).** New callers and features belong on the Go gateway by default. A Python gateway change needs a documented parity blocker for an active caller and must stay limited to that blocker. Read [`services/llm-gateway/PARITY.md`](services/llm-gateway/PARITY.md) and invoke `/migrating-llm-gateway-callers` before adding or migrating a caller, or changing either gateway contract. +- **`services/llm-gateway` is under an unofficial code freeze while callers move to [`PostHog/ai-gateway`](https://github.com/PostHog/ai-gateway).** New callers and features belong on the Go gateway by default. A Python gateway change needs a documented parity blocker for an active caller and must stay limited to that blocker. Read [`services/llm-gateway/PARITY.md`](services/llm-gateway/PARITY.md). Invoke `/auditing-llm-gateway-parity` for gateway contract changes and parity refreshes, and `/migrating-llm-gateway-callers` when moving a caller. ## Code Style @@ -239,4 +239,5 @@ ALWAYS invoke the matching skill **before** writing or reviewing code in these a - `/authoring-ci-workflows` — adding or editing any `.github/workflows` workflow, composite action, or reusable workflow - `/reviewing-personhog-protocol` — any personhog coordination-protocol change (leases, fencing, handoffs, supervisors, budgets, warming, changelog semantics), and any request for an exhaustive review of personhog code - `/gating-production-deploys` — any workflow that builds and pushes a production image or dispatches a deploy -- `/migrating-llm-gateway-callers` — adding or migrating an LLM gateway caller; changing `services/llm-gateway`, `posthog/llm/gateway_client.py`, gateway settings, auth, attribution, billing, providers, models, or API contracts; or refreshing gateway parity documentation +- `/auditing-llm-gateway-parity` — changing either gateway's auth, attribution, billing, endpoints, providers, models, routing, or metadata contract; reviewing a `services/llm-gateway` change; or refreshing `services/llm-gateway/PARITY.md` +- `/migrating-llm-gateway-callers` — adding an LLM gateway caller or migrating an existing caller from `services/llm-gateway` to `PostHog/ai-gateway`, including shared client and gateway setting changes made for that migration diff --git a/services/llm-gateway/PARITY.md b/services/llm-gateway/PARITY.md index 5d04ee6b40e2..c6a6bc050962 100644 --- a/services/llm-gateway/PARITY.md +++ b/services/llm-gateway/PARITY.md @@ -73,6 +73,8 @@ These are compatibility checks, not automatic blockers: ## Migration checklist +Run `/migrating-llm-gateway-callers` to inventory and convert a caller. + 1. Describe the caller and identify which identity controls its access and spend. 2. Check every relevant contract above against the real request and response. 3. Start with a shared Go-capable builder where one exists. @@ -81,7 +83,7 @@ These are compatibility checks, not automatic blockers: ## Refreshing this document -Run `/migrating-llm-gateway-callers` after either gateway changes auth, attribution, billing, endpoints, providers, models, routing, or event metadata. The skill audits implementation sources in both repositories and updates this file. +Run `/auditing-llm-gateway-parity` after either gateway changes auth, attribution, billing, endpoints, providers, models, routing, or event metadata. The skill audits implementation sources in both repositories and updates this file without migrating callers. Last verified on 2026-07-31 against: From 7fe1a4aefd4b890db219e996e84f980829d22c46 Mon Sep 17 00:00:00 2001 From: Richard Solomou Date: Fri, 31 Jul 2026 13:23:11 +0300 Subject: [PATCH 06/20] chore(ai-gateway): refresh parity audit Generated-By: PostHog Code Task-Id: e2e45fbc-1c5e-4ef2-9688-bcc941c2229c --- services/llm-gateway/PARITY.md | 44 +++++++++++++++++----------------- 1 file changed, 22 insertions(+), 22 deletions(-) diff --git a/services/llm-gateway/PARITY.md b/services/llm-gateway/PARITY.md index c6a6bc050962..5313a2b8c1d3 100644 --- a/services/llm-gateway/PARITY.md +++ b/services/llm-gateway/PARITY.md @@ -35,14 +35,14 @@ Existing Django callers should use `build_openai_client`, `build_async_openai_cl ### ⛔ Stay on the Python gateway for now -| Use case | Blocking contract | -| ------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| PostHog Desktop, PostHog Code, cloud agents, onboarding, Wizard, and similar first-party products | Trusted product identity, OAuth application allowlists, server-minted credential requirements, per-product model policy, and product billing or unbilled policy are not available together in Go. An `ai_product` event property is not an authorization or billing boundary. | -| Product or user budget enforcement | Python supports product and user cost limits, plan checks, quota buckets, and unbilled products. Go currently admits against the team wallet and does not branch settlement on credential billing mode. | -| OpenRouter, Fireworks, Cloudflare Workers AI, or Baseten | These provider paths are not available in Go. | -| OpenAI audio transcription | Go has no transcription endpoint. | -| OpenAI models through Anthropic Messages | Go does not provide this reverse translation. | -| Python product usage status | The Python product usage and quota API is different from Go request usage and wallet APIs. | +| Use case | Blocking contract | +| ------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| PostHog Desktop, PostHog Code, cloud agents, onboarding, Wizard, and similar first-party products | Trusted product identity, OAuth application allowlists, OAuth user attribution, server-minted credential requirements, per-product model policy, and product billing or unbilled policy are not available together in Go. An `ai_product` event property is not an authorization or billing boundary. | +| Product or user budget enforcement | Python supports product and user cost limits, plan checks, quota buckets, and unbilled products. Go currently admits against the team wallet and does not branch settlement on credential billing mode. | +| OpenRouter, Fireworks, Cloudflare Workers AI, or Baseten | These provider paths are not available in Go. | +| OpenAI audio transcription | Go has no transcription endpoint. | +| OpenAI models through Anthropic Messages | Go does not provide this reverse translation. | +| Python product usage status | The Python product usage and quota API is different from Go request usage and wallet APIs. | ### 🔎 Verify before switching @@ -51,25 +51,25 @@ These are compatibility checks, not automatic blockers: - **Model:** it appears in `GET /v1/models` and supports the requested API shape and capabilities. - **Credential:** Go accepts the credential type and projects the correct team, scope, and revocation state. - **Billing:** the request should reserve and debit the Go wallet. -- **Attribution:** `X-PostHog-Distinct-Id`, `X-PostHog-Trace-Id`, and `X-PostHog-Properties` provide enough event context. Caller-supplied properties are not trusted policy. +- **Attribution:** `X-PostHog-Distinct-Id`, `X-PostHog-Trace-Id`, and `X-PostHog-Properties` provide enough event context. Go does not derive the event distinct ID from OpenAI `user`, Anthropic `metadata.user_id`, or the OAuth user. Caller-supplied properties are not trusted policy. - **Provider behavior:** health-based routing or strict `X-PostHog-Provider` pinning matches the caller's fallback requirements. - **Wire behavior:** request fields, streaming chunks, errors, timeouts, and retries match what the caller handles. - **Metadata:** Python per-key property and feature flag headers are converted to the Go JSON properties header. ## Parity map -| Contract | Go gateway today | Python-only behavior that can block migration | -| ---------------------------- | --------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------- | -| Authentication | `phs_` project secrets and `pha_` OAuth credentials with `llm_gateway:read`; credential resolves team and revocation state. | `phx_` personal keys and product-specific API key or OAuth application rules. | -| Trusted first-party identity | No trusted product identity or product-scoped authorization boundary. | Product route selects allowlisted OAuth apps, server credential requirements, and model policy. | -| Attribution | Distinct ID, trace ID, custom JSON properties, team project token, provider, usage, and cost. | Product route owns `$ai_product` and `$ai_billable`; per-key properties and feature flag headers. | -| Billing and limits | Wallet reservation and settlement, idempotency, balance read, request usage read, and front-line rate limiting. | Product and user cost windows, plans, quota buckets, free tiers, unbilled products, and product usage status. | -| OpenAI APIs | Chat Completions, Responses, bare Responses alias, and normalized router chat. | Audio transcription and broader LiteLLM translation. | -| Anthropic APIs | Messages and token counting, including Bedrock-hosted models. | OpenAI models exposed through the Anthropic shape and Python-specific Bedrock opt-in behavior. | -| Providers | OpenAI, Anthropic, Azure OpenAI, Bedrock, and selected Modal models. | OpenRouter, Fireworks, Cloudflare Workers AI, and Baseten. | -| Models | Gateway-owned catalog, canonical IDs and aliases, capability checks, and router categories. | Broader LiteLLM model acceptance and Python product allowlists. | -| Routing and failure behavior | Health-based host choice, circuit breakers, hosted-provider failover, and strict provider pinning. | Caller opt-in Bedrock fallback and provider-specific Python routing. | -| Event metadata | One `X-PostHog-Properties` JSON object plus dedicated distinct ID, trace ID, and provider headers. | `X-POSTHOG-PROPERTY-*` and `X-POSTHOG-FLAG-*` headers. | +| Contract | Go gateway today | Python-only behavior that can block migration | +| ---------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Authentication | `phs_` project secrets and `pha_` OAuth credentials with the literal `llm_gateway:read` scope; credential resolves team and revocation state. | `phx_` personal keys, wildcard OAuth scope handling, and product-specific API key or OAuth application rules. | +| Trusted first-party identity | No trusted product identity or product-scoped authorization boundary. | Product route selects allowlisted OAuth apps, server credential requirements, and model policy. | +| Attribution | Distinct ID comes from `X-PostHog-Distinct-Id` or falls back to team ID. Supports trace ID, custom JSON properties, team project token, provider, usage, and cost. | OAuth user identity, body-derived OpenAI or Anthropic end-user identity, gateway-owned `$ai_product` and `$ai_billable`, per-key properties, and feature flag headers. | +| Billing and limits | Wallet reservation and settlement, idempotency, balance read, request usage read, and front-line rate limiting. | Product and user cost windows, plans, quota buckets, free tiers, unbilled products, and product usage status. | +| OpenAI APIs | Chat Completions, Responses, bare Responses alias, and normalized router chat. | Audio transcription and broader LiteLLM translation. | +| Anthropic APIs | Messages and token counting, including Bedrock-hosted models. | OpenAI models exposed through the Anthropic shape and Python-specific Bedrock opt-in behavior. | +| Providers | OpenAI, Anthropic, Azure OpenAI, Bedrock, and selected Modal models. | OpenRouter, Fireworks, Cloudflare Workers AI, and Baseten. | +| Models | Gateway-owned catalog, canonical IDs and aliases, capability checks, and router categories. | Broader LiteLLM model acceptance and Python product allowlists. | +| Routing and failure behavior | Health-based host choice, circuit breakers, hosted-provider failover, and strict provider pinning. | Caller opt-in Bedrock fallback and provider-specific Python routing. | +| Event metadata | One `X-PostHog-Properties` JSON object plus dedicated distinct ID, trace ID, and provider headers. | `X-POSTHOG-PROPERTY-*` and `X-POSTHOG-FLAG-*` headers. | ## Migration checklist @@ -87,7 +87,7 @@ Run `/auditing-llm-gateway-parity` after either gateway changes auth, attributio Last verified on 2026-07-31 against: -- `PostHog/posthog` master at `801dfa4bd67e534ee4c09ec4adc21cde66d3497d` +- `PostHog/posthog` master at `c49d3105235288233cd5f618a6cb27d03d8e241c` - `PostHog/ai-gateway` main at `ea8230b6dbc4ebf6c4be83d359e561477a13b215` ## References From 8bb1416fef76b32b0092359af00fddbc4aea0210 Mon Sep 17 00:00:00 2001 From: Richard Solomou Date: Fri, 31 Jul 2026 13:31:08 +0300 Subject: [PATCH 07/20] chore(ai-gateway): add migration examples Generated-By: PostHog Code Task-Id: e2e45fbc-1c5e-4ef2-9688-bcc941c2229c --- .../migrating-llm-gateway-callers/SKILL.md | 2 ++ .../references/migration-examples.md | 25 +++++++++++++++++++ 2 files changed, 27 insertions(+) create mode 100644 .agents/skills/migrating-llm-gateway-callers/references/migration-examples.md diff --git a/.agents/skills/migrating-llm-gateway-callers/SKILL.md b/.agents/skills/migrating-llm-gateway-callers/SKILL.md index 161f2ad53616..034396cc6187 100644 --- a/.agents/skills/migrating-llm-gateway-callers/SKILL.md +++ b/.agents/skills/migrating-llm-gateway-callers/SKILL.md @@ -36,6 +36,8 @@ An existing Python client or product route is not a blocker by itself. Prefer the smallest existing pattern that matches the caller: +Read [migration examples](references/migration-examples.md) for verified PRs covering Django clients, staged workload rollout, sandbox wiring, and attribution continuity. Follow the contract demonstrated by the relevant example rather than copying its code mechanically. + 1. Use `build_openai_client`, `build_async_openai_client`, or `build_async_anthropic_client` from `posthog/llm/gateway_client.py` for Django callers when possible. 2. Use the slugless Go base URL. Do not carry a Python `/{product}/` path into the Go URL. 3. Use a supported `phs_` or `pha_` credential with `llm_gateway:read`. Do not weaken auth or expose a shared secret to an untrusted runtime. diff --git a/.agents/skills/migrating-llm-gateway-callers/references/migration-examples.md b/.agents/skills/migrating-llm-gateway-callers/references/migration-examples.md new file mode 100644 index 000000000000..8fb2acfafcfb --- /dev/null +++ b/.agents/skills/migrating-llm-gateway-callers/references/migration-examples.md @@ -0,0 +1,25 @@ +# Migration examples + +These merged PRs show different migration shapes. Read the relevant diff before implementing a similar change because gateway contracts have continued to evolve. + +## Django and direct-provider callers + +- [#64448: make cluster labeling routable through the AI gateway](https://github.com/PostHog/posthog/pull/64448) introduces an environment-gated OpenAI client, validates paired settings, preserves direct-provider fallback, checks model availability before rollout, and tests both routes. +- [#65044: route OpenAI summarization through the AI gateway](https://github.com/PostHog/posthog/pull/65044) extracts shared sync and async builders, uses the slugless Go URL and project secret, keeps `trust_env=False`, retains the Python fallback, and prevents duplicate `$ai_generation` capture. +- [#68329: route the PR-approval agent through the AI gateway](https://github.com/PostHog/posthog/pull/68329) covers a non-Django SDK process. It validates and translates gateway environment variables, switches away from the traced SDK wrapper in gateway mode, keeps a direct fallback, and tests credential and metadata wiring without exposing secret values. + +## Incremental product rollout + +- [#71947: add the Signals AI gateway client and per-call opt-in](https://github.com/PostHog/posthog/pull/71947) adds the Anthropic builder and an opt-in seam without moving traffic. It converts per-key metadata to `X-PostHog-Properties`, strips `/v1` for the Anthropic SDK, and preserves Python behavior for callers that have not opted in. +- [#71948: route Signals grouping through the AI gateway](https://github.com/PostHog/posthog/pull/71948) uses that seam to move one workload at a time. Reverting the small opt-in returns only grouping to Python. +- [#71949: route Signals emission stages through the AI gateway](https://github.com/PostHog/posthog/pull/71949) handles batch and per-call metadata explicitly. Its tests cover both the Go JSON properties blob and Python per-key fallback headers. + +## Sandbox and deployment wiring + +- [#72770: route selected sandbox products to the AI gateway](https://github.com/PostHog/posthog/pull/72770) treats migration as more than a client change. It pairs URL and product rollout settings, reserves them against user overrides, updates both egress enforcement layers, validates configured hostnames, extends startup diagnostics, and keeps rollback to clearing either setting. + +## Post-migration parity checks + +- [#74249: send a stable team trace ID to the Go gateway](https://github.com/PostHog/posthog/pull/74249) fixes trace fragmentation found after a migration. It matches Python's existing trace derivation, sends the dedicated Go header, and pins cross-gateway compatibility with fixed expected IDs. + +Use these examples for their decisions and verification boundaries. Re-check [`services/llm-gateway/PARITY.md`](../../../../services/llm-gateway/PARITY.md) before adopting an older pattern. From 9ba15ec2d8085131baaacfafb2bf0389c440722d Mon Sep 17 00:00:00 2001 From: Richard Solomou Date: Fri, 31 Jul 2026 13:40:40 +0300 Subject: [PATCH 08/20] chore(ai-gateway): add cross-repository migration examples Generated-By: PostHog Code Task-Id: e2e45fbc-1c5e-4ef2-9688-bcc941c2229c --- .../references/migration-examples.md | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/.agents/skills/migrating-llm-gateway-callers/references/migration-examples.md b/.agents/skills/migrating-llm-gateway-callers/references/migration-examples.md index 8fb2acfafcfb..96eb88e98b5e 100644 --- a/.agents/skills/migrating-llm-gateway-callers/references/migration-examples.md +++ b/.agents/skills/migrating-llm-gateway-callers/references/migration-examples.md @@ -18,6 +18,14 @@ These merged PRs show different migration shapes. Read the relevant diff before - [#72770: route selected sandbox products to the AI gateway](https://github.com/PostHog/posthog/pull/72770) treats migration as more than a client change. It pairs URL and product rollout settings, reserves them against user overrides, updates both egress enforcement layers, validates configured hostnames, extends startup diagnostics, and keeps rollback to clearing either setting. +## Other PostHog repositories + +- [PostHog/code #3354: route PR review through the AI gateway](https://github.com/PostHog/code/pull/3354) shows a standalone Claude Agent SDK migration. It validates paired settings, strips `/v1` before the SDK restores its messages path, sets both Anthropic auth variables, avoids duplicate capture by bypassing the traced wrapper in gateway mode, and keeps direct-provider fallback. +- [PostHog/code #3659: route selected products to the AI gateway](https://github.com/PostHog/code/pull/3659) selects a gateway per sandbox request instead of per process. It requires both the Go URL and an allowlist, keeps unlisted products on Python, converts attribution to one bounded ASCII-safe JSON header, and gives each workload a distinct product tag. +- [PostHog/SherlockHog #104: route the agent through the slugless AI gateway](https://github.com/PostHog/SherlockHog/pull/104) covers a TypeScript Claude Agent SDK service. The paired Go settings take precedence, the old settings remain available for rollback, and tests pin partial-configuration fallback and base URL translation. +- [PostHog/SherlockHog #112: tag the AI product through `X-PostHog-Properties`](https://github.com/PostHog/SherlockHog/pull/112) is a warning example. The initial cutover reused Python's per-property headers, so traffic reached Go without product attribution. The follow-up keeps each gateway's metadata format separate and verifies that the Go route no longer emits the legacy form. +- [PostHog/charts #13131: repoint worker deployments to the AI gateway](https://github.com/PostHog/charts/pull/13131) demonstrates that merged client support does not move traffic by itself. It wires the regional URL and app-specific secret into every deployment that runs the migrated caller, with removal of either setting as rollback. + ## Post-migration parity checks - [#74249: send a stable team trace ID to the Go gateway](https://github.com/PostHog/posthog/pull/74249) fixes trace fragmentation found after a migration. It matches Python's existing trace derivation, sends the dedicated Go header, and pins cross-gateway compatibility with fixed expected IDs. From 60b51a50054b5138fe511a48cacf56d85c44b16b Mon Sep 17 00:00:00 2001 From: Richard Solomou Date: Fri, 31 Jul 2026 13:45:46 +0300 Subject: [PATCH 09/20] chore(ai-gateway): document migration follow-up sequences Generated-By: PostHog Code Task-Id: e2e45fbc-1c5e-4ef2-9688-bcc941c2229c --- .../references/migration-examples.md | 35 +++++++++++++++---- 1 file changed, 28 insertions(+), 7 deletions(-) diff --git a/.agents/skills/migrating-llm-gateway-callers/references/migration-examples.md b/.agents/skills/migrating-llm-gateway-callers/references/migration-examples.md index 96eb88e98b5e..4d4ce317e030 100644 --- a/.agents/skills/migrating-llm-gateway-callers/references/migration-examples.md +++ b/.agents/skills/migrating-llm-gateway-callers/references/migration-examples.md @@ -1,6 +1,6 @@ # Migration examples -These merged PRs show different migration shapes. Read the relevant diff before implementing a similar change because gateway contracts have continued to evolve. +These PRs show different migration shapes and rollout states. Check each PR's current status and read the relevant diff before implementing a similar change because gateway contracts have continued to evolve. ## Django and direct-provider callers @@ -18,13 +18,34 @@ These merged PRs show different migration shapes. Read the relevant diff before - [#72770: route selected sandbox products to the AI gateway](https://github.com/PostHog/posthog/pull/72770) treats migration as more than a client change. It pairs URL and product rollout settings, reserves them against user overrides, updates both egress enforcement layers, validates configured hostnames, extends startup diagnostics, and keeps rollback to clearing either setting. -## Other PostHog repositories +## Cross-repository migration sequences -- [PostHog/code #3354: route PR review through the AI gateway](https://github.com/PostHog/code/pull/3354) shows a standalone Claude Agent SDK migration. It validates paired settings, strips `/v1` before the SDK restores its messages path, sets both Anthropic auth variables, avoids duplicate capture by bypassing the traced wrapper in gateway mode, and keeps direct-provider fallback. -- [PostHog/code #3659: route selected products to the AI gateway](https://github.com/PostHog/code/pull/3659) selects a gateway per sandbox request instead of per process. It requires both the Go URL and an allowlist, keeps unlisted products on Python, converts attribution to one bounded ASCII-safe JSON header, and gives each workload a distinct product tag. -- [PostHog/SherlockHog #104: route the agent through the slugless AI gateway](https://github.com/PostHog/SherlockHog/pull/104) covers a TypeScript Claude Agent SDK service. The paired Go settings take precedence, the old settings remain available for rollback, and tests pin partial-configuration fallback and base URL translation. -- [PostHog/SherlockHog #112: tag the AI product through `X-PostHog-Properties`](https://github.com/PostHog/SherlockHog/pull/112) is a warning example. The initial cutover reused Python's per-property headers, so traffic reached Go without product attribution. The follow-up keeps each gateway's metadata format separate and verifies that the Go route no longer emits the legacy form. -- [PostHog/charts #13131: repoint worker deployments to the AI gateway](https://github.com/PostHog/charts/pull/13131) demonstrates that merged client support does not move traffic by itself. It wires the regional URL and app-specific secret into every deployment that runs the migrated caller, with removal of either setting as rollback. +Treat these as linked sequences. A client PR alone does not prove that traffic moved or retained its expected attribution. + +### ✅ SherlockHog: merged end-to-end sequence + +1. [PostHog/SherlockHog #104: route the agent through the slugless AI gateway](https://github.com/PostHog/SherlockHog/pull/104) adds the paired Go settings, base URL translation, project-secret auth, Python fallback, and route-selection tests for a TypeScript Claude Agent SDK service. +2. [PostHog/SherlockHog #111: boot with either complete gateway pair](https://github.com/PostHog/SherlockHog/pull/111) fixes the transitional startup contract so an AI-gateway-only deployment can boot while preserving the Python rollback route. +3. [PostHog/SherlockHog #112: tag the AI product through `X-PostHog-Properties`](https://github.com/PostHog/SherlockHog/pull/112) corrects attribution after the initial cutover reused Python's per-property headers. It keeps each gateway's metadata format separate and verifies that the Go route no longer emits the legacy form. +4. [PostHog/charts #12919: cut development over to the slugless AI gateway](https://github.com/PostHog/charts/pull/12919) wires the development URL and secret first, with explicit deployment rendering and rollback checks. +5. [PostHog/charts #12920: cut production over to the slugless AI gateway](https://github.com/PostHog/charts/pull/12920) applies the regional production configuration after the development stage. + +All five changes are merged. Still verify the current deployment and live attribution before using this sequence as proof of present-day behavior. + +### 🚧 Sandbox Signals stages: production rollout incomplete + +1. [PostHog/code #3659: route selected products to the AI gateway](https://github.com/PostHog/code/pull/3659) selects a gateway per sandbox request. It requires both the Go URL and an allowlist, keeps unlisted products on Python, converts attribution to one bounded ASCII-safe JSON header, and gives each workload a distinct product tag. +2. [PostHog/posthog #72770: route selected sandbox products to the AI gateway](https://github.com/PostHog/posthog/pull/72770) passes the settings into sandboxes, reserves them against user overrides, updates both egress enforcement layers, validates configured hostnames, and extends startup diagnostics. +3. [PostHog/charts #13358: route Signals sandbox stages in development](https://github.com/PostHog/charts/pull/13358) activates the client and Django support for four workloads in development. +4. [PostHog/charts #13361: route Signals sandbox stages in production](https://github.com/PostHog/charts/pull/13361) is the production follow-up and remains an open draft. Do not describe this sequence as a completed production migration until it merges and the live route is verified. + +### 🚧 Stamphog: implementation without PR evidence of activation + +- [PostHog/code #3354: route PR review through the AI gateway](https://github.com/PostHog/code/pull/3354) shows a standalone Claude Agent SDK implementation. It validates paired settings, strips `/v1` before the SDK restores its messages path, sets both Anthropic auth variables, avoids duplicate capture by bypassing the traced wrapper in gateway mode, and keeps direct-provider fallback. The PR explicitly left secret population and a live gateway request unverified, and no merged follow-up PR was found that closes those steps. + +### ✅ Deployment-only example + +- [PostHog/charts #13131: repoint worker deployments to the AI gateway](https://github.com/PostHog/charts/pull/13131) demonstrates that merged client support does not move traffic by itself. It wires the regional URL and app-specific secret into every deployment that runs an already-migrated caller, with removal of either setting as rollback. Use it as deployment guidance, not as a standalone caller migration. ## Post-migration parity checks From dafabf624e903bb22b09119eb2f35c74e3c8a086 Mon Sep 17 00:00:00 2001 From: Richard Solomou Date: Fri, 31 Jul 2026 13:51:15 +0300 Subject: [PATCH 10/20] chore(ai-gateway): record caller migration status Generated-By: PostHog Code Task-Id: e2e45fbc-1c5e-4ef2-9688-bcc941c2229c --- .../references/migration-examples.md | 12 ++++++--- services/llm-gateway/PARITY.md | 26 +++++++++++++++++++ 2 files changed, 34 insertions(+), 4 deletions(-) diff --git a/.agents/skills/migrating-llm-gateway-callers/references/migration-examples.md b/.agents/skills/migrating-llm-gateway-callers/references/migration-examples.md index 4d4ce317e030..8873b4de6ca3 100644 --- a/.agents/skills/migrating-llm-gateway-callers/references/migration-examples.md +++ b/.agents/skills/migrating-llm-gateway-callers/references/migration-examples.md @@ -5,6 +5,7 @@ These PRs show different migration shapes and rollout states. Check each PR's cu ## Django and direct-provider callers - [#64448: make cluster labeling routable through the AI gateway](https://github.com/PostHog/posthog/pull/64448) introduces an environment-gated OpenAI client, validates paired settings, preserves direct-provider fallback, checks model availability before rollout, and tests both routes. +- [#65043: route the eval-report agent through the AI gateway](https://github.com/PostHog/posthog/pull/65043) moves the AI observability eval-report agent after the shared route exists and tests the selected client path. - [#65044: route OpenAI summarization through the AI gateway](https://github.com/PostHog/posthog/pull/65044) extracts shared sync and async builders, uses the slugless Go URL and project secret, keeps `trust_env=False`, retains the Python fallback, and prevents duplicate `$ai_generation` capture. - [#68329: route the PR-approval agent through the AI gateway](https://github.com/PostHog/posthog/pull/68329) covers a non-Django SDK process. It validates and translates gateway environment variables, switches away from the traced SDK wrapper in gateway mode, keeps a direct fallback, and tests credential and metadata wiring without exposing secret values. @@ -13,6 +14,9 @@ These PRs show different migration shapes and rollout states. Check each PR's cu - [#71947: add the Signals AI gateway client and per-call opt-in](https://github.com/PostHog/posthog/pull/71947) adds the Anthropic builder and an opt-in seam without moving traffic. It converts per-key metadata to `X-PostHog-Properties`, strips `/v1` for the Anthropic SDK, and preserves Python behavior for callers that have not opted in. - [#71948: route Signals grouping through the AI gateway](https://github.com/PostHog/posthog/pull/71948) uses that seam to move one workload at a time. Reverting the small opt-in returns only grouping to Python. - [#71949: route Signals emission stages through the AI gateway](https://github.com/PostHog/posthog/pull/71949) handles batch and per-call metadata explicitly. Its tests cover both the Go JSON properties blob and Python per-key fallback headers. +- [#71950: route Signals safety through the AI gateway](https://github.com/PostHog/posthog/pull/71950) opts the safety filter and report safety judge into the shared Go-capable client. +- [#71951: route Signals eval summarization through the AI gateway](https://github.com/PostHog/posthog/pull/71951) applies the same narrow per-call rollout to eval summaries. +- [#72769: tag eval-fixture generation as `signals_eval`](https://github.com/PostHog/posthog/pull/72769) preserves workload attribution for Signals eval generation. ## Sandbox and deployment wiring @@ -32,16 +36,16 @@ Treat these as linked sequences. A client PR alone does not prove that traffic m All five changes are merged. Still verify the current deployment and live attribution before using this sequence as proof of present-day behavior. -### 🚧 Sandbox Signals stages: production rollout incomplete +### 🚧 Signals sandbox Scouts: production rollout paused 1. [PostHog/code #3659: route selected products to the AI gateway](https://github.com/PostHog/code/pull/3659) selects a gateway per sandbox request. It requires both the Go URL and an allowlist, keeps unlisted products on Python, converts attribution to one bounded ASCII-safe JSON header, and gives each workload a distinct product tag. 2. [PostHog/posthog #72770: route selected sandbox products to the AI gateway](https://github.com/PostHog/posthog/pull/72770) passes the settings into sandboxes, reserves them against user overrides, updates both egress enforcement layers, validates configured hostnames, and extends startup diagnostics. 3. [PostHog/charts #13358: route Signals sandbox stages in development](https://github.com/PostHog/charts/pull/13358) activates the client and Django support for four workloads in development. -4. [PostHog/charts #13361: route Signals sandbox stages in production](https://github.com/PostHog/charts/pull/13361) is the production follow-up and remains an open draft. Do not describe this sequence as a completed production migration until it merges and the live route is verified. +4. [PostHog/charts #13361: route Signals sandbox stages in production](https://github.com/PostHog/charts/pull/13361) is the production follow-up and remains an open draft. The rollout is paused pending auth and attribution support. Do not describe this sequence as a completed production migration until the blocker closes, the change merges, and the live route is verified. -### 🚧 Stamphog: implementation without PR evidence of activation +### ✅ StampHog: migrated caller -- [PostHog/code #3354: route PR review through the AI gateway](https://github.com/PostHog/code/pull/3354) shows a standalone Claude Agent SDK implementation. It validates paired settings, strips `/v1` before the SDK restores its messages path, sets both Anthropic auth variables, avoids duplicate capture by bypassing the traced wrapper in gateway mode, and keeps direct-provider fallback. The PR explicitly left secret population and a live gateway request unverified, and no merged follow-up PR was found that closes those steps. +- [PostHog/posthog #68329: route the PR-approval agent through the AI gateway](https://github.com/PostHog/posthog/pull/68329) and [PostHog/code #3354: route PR review through the AI gateway](https://github.com/PostHog/code/pull/3354) show the migration in both repositories. The implementation validates paired settings, strips `/v1` before the SDK restores its messages path, sets both Anthropic auth variables, avoids duplicate capture by bypassing the traced wrapper in gateway mode, and keeps direct-provider fallback. ### ✅ Deployment-only example diff --git a/services/llm-gateway/PARITY.md b/services/llm-gateway/PARITY.md index 5313a2b8c1d3..00c2a3592d0f 100644 --- a/services/llm-gateway/PARITY.md +++ b/services/llm-gateway/PARITY.md @@ -20,6 +20,32 @@ Bug, security, and reliability fixes for blocked callers are valid reasons. Conv When a gap closes, new work uses the Go gateway and affected callers should migrate. Do not add the same feature to both gateways by default. +## Current rollout inventory + +This inventory records known caller rollout state as of 2026-07-31. It complements the contract audit below. A migrated caller is evidence that its specific contract works, not proof that every caller in the same product can move. + +### ✅ Migrated to the Go gateway + +- AI observability summarization, eval reports, and clustering +- SherlockHog +- StampHog +- Signals grouping, safety, eval summarization, emission, and `signals_eval` + +Signals scout code has Go-capable paths, but the sandbox production rollout remains paused as described below. + +### ⏳ Still on the Python gateway + +| Caller | Current reason or state | +| ----------------------------- | ------------------------------------------------------------------------------------------------------------------- | +| PostHog AI (`ai`) | Requires first-party product policy that is not yet available together in Go. | +| PostHog Desktop | Requires trusted auth and attribution that are not yet available together in Go. | +| Session Replay | Not yet migrated. Inventory its exact contract before changing either gateway. | +| Wizard | Requires first-party product policy that is not yet available together in Go. | +| Agent package | Migration is in progress. Re-check the current implementation and deployment state before making related changes. | +| Scouts | Migration is paused pending auth and attribution support. Do not infer readiness from the migrated Signals callers. | +| Warehouse semantic enrichment | A migration is in draft. Re-check the draft and current parity before starting overlapping work. | +| Survey summary | Partially migrated. Some Gemini calls use the Python gateway's Haiku path as a temporary fallback. | + ## Choose by use case ### ✅ Use the Go gateway From 8b5d902cc164b68c26e2edf00ae5fa943e200438 Mon Sep 17 00:00:00 2001 From: Richard Solomou Date: Fri, 31 Jul 2026 13:55:17 +0300 Subject: [PATCH 11/20] chore(ai-gateway): verify caller rollout inventory Generated-By: PostHog Code Task-Id: e2e45fbc-1c5e-4ef2-9688-bcc941c2229c --- .../references/migration-examples.md | 4 +- services/llm-gateway/PARITY.md | 38 ++++++++++--------- 2 files changed, 23 insertions(+), 19 deletions(-) diff --git a/.agents/skills/migrating-llm-gateway-callers/references/migration-examples.md b/.agents/skills/migrating-llm-gateway-callers/references/migration-examples.md index 8873b4de6ca3..76e60cfd353f 100644 --- a/.agents/skills/migrating-llm-gateway-callers/references/migration-examples.md +++ b/.agents/skills/migrating-llm-gateway-callers/references/migration-examples.md @@ -43,9 +43,9 @@ All five changes are merged. Still verify the current deployment and live attrib 3. [PostHog/charts #13358: route Signals sandbox stages in development](https://github.com/PostHog/charts/pull/13358) activates the client and Django support for four workloads in development. 4. [PostHog/charts #13361: route Signals sandbox stages in production](https://github.com/PostHog/charts/pull/13361) is the production follow-up and remains an open draft. The rollout is paused pending auth and attribution support. Do not describe this sequence as a completed production migration until the blocker closes, the change merges, and the live route is verified. -### ✅ StampHog: migrated caller +### 🚧 StampHog: split rollout -- [PostHog/posthog #68329: route the PR-approval agent through the AI gateway](https://github.com/PostHog/posthog/pull/68329) and [PostHog/code #3354: route PR review through the AI gateway](https://github.com/PostHog/code/pull/3354) show the migration in both repositories. The implementation validates paired settings, strips `/v1` before the SDK restores its messages path, sets both Anthropic auth variables, avoids duplicate capture by bypassing the traced wrapper in gateway mode, and keeps direct-provider fallback. +- [PostHog/posthog #68329: route the PR-approval agent through the AI gateway](https://github.com/PostHog/posthog/pull/68329) and [PostHog/code #3354: route PR review through the AI gateway](https://github.com/PostHog/code/pull/3354) show Go-capable standalone PR-review paths. The implementation validates paired settings, strips `/v1` before the SDK restores its messages path, sets both Anthropic auth variables, avoids duplicate capture by bypassing the traced wrapper in gateway mode, and keeps direct-provider fallback. Do not treat these PRs as a complete StampHog migration: the hosted Temporal worker is still deployed with the Python `/stamphog/v1` product route. ### ✅ Deployment-only example diff --git a/services/llm-gateway/PARITY.md b/services/llm-gateway/PARITY.md index 00c2a3592d0f..ddb0cccaea9a 100644 --- a/services/llm-gateway/PARITY.md +++ b/services/llm-gateway/PARITY.md @@ -22,29 +22,33 @@ When a gap closes, new work uses the Go gateway and affected callers should migr ## Current rollout inventory -This inventory records known caller rollout state as of 2026-07-31. It complements the contract audit below. A migrated caller is evidence that its specific contract works, not proof that every caller in the same product can move. +This inventory records caller implementation and deployment state verified on 2026-07-31. It complements the contract audit below. A migrated caller is evidence that its specific contract works, not proof that every caller in the same product can move. ### ✅ Migrated to the Go gateway - AI observability summarization, eval reports, and clustering -- SherlockHog -- StampHog -- Signals grouping, safety, eval summarization, emission, and `signals_eval` +- SherlockHog, whose client prefers Go when the paired settings are present and whose development and production overlays provide them +- Signals grouping, safety, eval summarization, emission, and `signals_eval`, whose workers have regional Go gateway settings -Signals scout code has Go-capable paths, but the sandbox production rollout remains paused as described below. +### 🚧 Partially migrated or in progress -### ⏳ Still on the Python gateway +| Caller | Verified state | +| ---------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | +| StampHog | Standalone PR-review paths are Go-capable. The hosted Temporal worker still points `AI_GATEWAY_URL` at the Python `/stamphog/v1` product route, so StampHog is not fully migrated. | +| Agent package | Supports selecting Go per product, but rollout is not complete across callers and deployments. | +| Signals sandbox Scouts | Go routing is enabled in the development task-agent overlay. Production overlays do not enable it, and the production follow-up remains paused pending auth and attribution support. | +| Warehouse error triage | The warehouse-sources-admin deployment has Go gateway settings. Do not confuse this caller with warehouse semantic enrichment. | +| Survey summary | Uses mixed paths. Some former Gemini calls use the Python gateway's Haiku path as a temporary fallback. Audit each summary call before changing shared configuration. | -| Caller | Current reason or state | -| ----------------------------- | ------------------------------------------------------------------------------------------------------------------- | -| PostHog AI (`ai`) | Requires first-party product policy that is not yet available together in Go. | -| PostHog Desktop | Requires trusted auth and attribution that are not yet available together in Go. | -| Session Replay | Not yet migrated. Inventory its exact contract before changing either gateway. | -| Wizard | Requires first-party product policy that is not yet available together in Go. | -| Agent package | Migration is in progress. Re-check the current implementation and deployment state before making related changes. | -| Scouts | Migration is paused pending auth and attribution support. Do not infer readiness from the migrated Signals callers. | -| Warehouse semantic enrichment | A migration is in draft. Re-check the draft and current parity before starting overlapping work. | -| Survey summary | Partially migrated. Some Gemini calls use the Python gateway's Haiku path as a temporary fallback. | +### ⏳ Not migrated to the Go gateway + +| Caller | Verified reason or state | +| ----------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------- | +| PostHog AI (`ai`) | Still relies on Python gateway and first-party product contracts. | +| PostHog Desktop | Requires trusted auth and attribution that are not yet available together in Go. | +| Session Replay | Its current AI call sites do not use the shared Go-capable gateway builders. Inventory the exact provider and auth contract before migration. | +| Wizard | Still has Python product policy and direct Gemini paths. It requires first-party product policy that is not yet available together in Go. | +| Warehouse semantic enrichment | Uses the Python `warehouse_semantic_enrichment` product client, and its worker deployment has only Python gateway settings. | ## Choose by use case @@ -113,7 +117,7 @@ Run `/auditing-llm-gateway-parity` after either gateway changes auth, attributio Last verified on 2026-07-31 against: -- `PostHog/posthog` master at `c49d3105235288233cd5f618a6cb27d03d8e241c` +- `PostHog/posthog` master at `0350fb13ed8321a5b4199a111b31052e2218360b` - `PostHog/ai-gateway` main at `ea8230b6dbc4ebf6c4be83d359e561477a13b215` ## References From b2e92634e24c711bf31cb105958ca7c7fa45f0af Mon Sep 17 00:00:00 2001 From: Richard Solomou Date: Fri, 31 Jul 2026 14:09:14 +0300 Subject: [PATCH 12/20] chore(ai-gateway): focus guidance on migration decisions Generated-By: PostHog Code Task-Id: e2e45fbc-1c5e-4ef2-9688-bcc941c2229c --- .../references/migration-examples.md | 21 +++++++------ services/llm-gateway/PARITY.md | 30 ------------------- 2 files changed, 10 insertions(+), 41 deletions(-) diff --git a/.agents/skills/migrating-llm-gateway-callers/references/migration-examples.md b/.agents/skills/migrating-llm-gateway-callers/references/migration-examples.md index 76e60cfd353f..dd72c7132492 100644 --- a/.agents/skills/migrating-llm-gateway-callers/references/migration-examples.md +++ b/.agents/skills/migrating-llm-gateway-callers/references/migration-examples.md @@ -1,6 +1,6 @@ # Migration examples -These PRs show different migration shapes and rollout states. Check each PR's current status and read the relevant diff before implementing a similar change because gateway contracts have continued to evolve. +These PRs show different migration shapes. Read the relevant diff before implementing a similar change because gateway contracts have continued to evolve. Use them for implementation decisions, not as an inventory of current rollout state. ## Django and direct-provider callers @@ -22,11 +22,9 @@ These PRs show different migration shapes and rollout states. Check each PR's cu - [#72770: route selected sandbox products to the AI gateway](https://github.com/PostHog/posthog/pull/72770) treats migration as more than a client change. It pairs URL and product rollout settings, reserves them against user overrides, updates both egress enforcement layers, validates configured hostnames, extends startup diagnostics, and keeps rollback to clearing either setting. -## Cross-repository migration sequences +## Cross-repository patterns -Treat these as linked sequences. A client PR alone does not prove that traffic moved or retained its expected attribution. - -### ✅ SherlockHog: merged end-to-end sequence +### Standalone TypeScript service 1. [PostHog/SherlockHog #104: route the agent through the slugless AI gateway](https://github.com/PostHog/SherlockHog/pull/104) adds the paired Go settings, base URL translation, project-secret auth, Python fallback, and route-selection tests for a TypeScript Claude Agent SDK service. 2. [PostHog/SherlockHog #111: boot with either complete gateway pair](https://github.com/PostHog/SherlockHog/pull/111) fixes the transitional startup contract so an AI-gateway-only deployment can boot while preserving the Python rollback route. @@ -34,20 +32,21 @@ Treat these as linked sequences. A client PR alone does not prove that traffic m 4. [PostHog/charts #12919: cut development over to the slugless AI gateway](https://github.com/PostHog/charts/pull/12919) wires the development URL and secret first, with explicit deployment rendering and rollback checks. 5. [PostHog/charts #12920: cut production over to the slugless AI gateway](https://github.com/PostHog/charts/pull/12920) applies the regional production configuration after the development stage. -All five changes are merged. Still verify the current deployment and live attribution before using this sequence as proof of present-day behavior. +Together these show the client, transitional boot contract, attribution correction, staged deployment, and rollback boundaries. -### 🚧 Signals sandbox Scouts: production rollout paused +### Per-product sandbox routing 1. [PostHog/code #3659: route selected products to the AI gateway](https://github.com/PostHog/code/pull/3659) selects a gateway per sandbox request. It requires both the Go URL and an allowlist, keeps unlisted products on Python, converts attribution to one bounded ASCII-safe JSON header, and gives each workload a distinct product tag. 2. [PostHog/posthog #72770: route selected sandbox products to the AI gateway](https://github.com/PostHog/posthog/pull/72770) passes the settings into sandboxes, reserves them against user overrides, updates both egress enforcement layers, validates configured hostnames, and extends startup diagnostics. 3. [PostHog/charts #13358: route Signals sandbox stages in development](https://github.com/PostHog/charts/pull/13358) activates the client and Django support for four workloads in development. -4. [PostHog/charts #13361: route Signals sandbox stages in production](https://github.com/PostHog/charts/pull/13361) is the production follow-up and remains an open draft. The rollout is paused pending auth and attribution support. Do not describe this sequence as a completed production migration until the blocker closes, the change merges, and the live route is verified. -### 🚧 StampHog: split rollout +These show how to select Go per request, pass configuration through a sandbox boundary, enforce egress, and activate a narrow set of workloads. + +### Standalone PR-review agent -- [PostHog/posthog #68329: route the PR-approval agent through the AI gateway](https://github.com/PostHog/posthog/pull/68329) and [PostHog/code #3354: route PR review through the AI gateway](https://github.com/PostHog/code/pull/3354) show Go-capable standalone PR-review paths. The implementation validates paired settings, strips `/v1` before the SDK restores its messages path, sets both Anthropic auth variables, avoids duplicate capture by bypassing the traced wrapper in gateway mode, and keeps direct-provider fallback. Do not treat these PRs as a complete StampHog migration: the hosted Temporal worker is still deployed with the Python `/stamphog/v1` product route. +- [PostHog/posthog #68329: route the PR-approval agent through the AI gateway](https://github.com/PostHog/posthog/pull/68329) and [PostHog/code #3354: route PR review through the AI gateway](https://github.com/PostHog/code/pull/3354) validate paired settings, strip `/v1` before the SDK restores its messages path, set both Anthropic auth variables, avoid duplicate capture by bypassing the traced wrapper in gateway mode, and retain an explicit fallback. -### ✅ Deployment-only example +### Deployment activation - [PostHog/charts #13131: repoint worker deployments to the AI gateway](https://github.com/PostHog/charts/pull/13131) demonstrates that merged client support does not move traffic by itself. It wires the regional URL and app-specific secret into every deployment that runs an already-migrated caller, with removal of either setting as rollback. Use it as deployment guidance, not as a standalone caller migration. diff --git a/services/llm-gateway/PARITY.md b/services/llm-gateway/PARITY.md index ddb0cccaea9a..0ba9fbad585a 100644 --- a/services/llm-gateway/PARITY.md +++ b/services/llm-gateway/PARITY.md @@ -20,36 +20,6 @@ Bug, security, and reliability fixes for blocked callers are valid reasons. Conv When a gap closes, new work uses the Go gateway and affected callers should migrate. Do not add the same feature to both gateways by default. -## Current rollout inventory - -This inventory records caller implementation and deployment state verified on 2026-07-31. It complements the contract audit below. A migrated caller is evidence that its specific contract works, not proof that every caller in the same product can move. - -### ✅ Migrated to the Go gateway - -- AI observability summarization, eval reports, and clustering -- SherlockHog, whose client prefers Go when the paired settings are present and whose development and production overlays provide them -- Signals grouping, safety, eval summarization, emission, and `signals_eval`, whose workers have regional Go gateway settings - -### 🚧 Partially migrated or in progress - -| Caller | Verified state | -| ---------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | -| StampHog | Standalone PR-review paths are Go-capable. The hosted Temporal worker still points `AI_GATEWAY_URL` at the Python `/stamphog/v1` product route, so StampHog is not fully migrated. | -| Agent package | Supports selecting Go per product, but rollout is not complete across callers and deployments. | -| Signals sandbox Scouts | Go routing is enabled in the development task-agent overlay. Production overlays do not enable it, and the production follow-up remains paused pending auth and attribution support. | -| Warehouse error triage | The warehouse-sources-admin deployment has Go gateway settings. Do not confuse this caller with warehouse semantic enrichment. | -| Survey summary | Uses mixed paths. Some former Gemini calls use the Python gateway's Haiku path as a temporary fallback. Audit each summary call before changing shared configuration. | - -### ⏳ Not migrated to the Go gateway - -| Caller | Verified reason or state | -| ----------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------- | -| PostHog AI (`ai`) | Still relies on Python gateway and first-party product contracts. | -| PostHog Desktop | Requires trusted auth and attribution that are not yet available together in Go. | -| Session Replay | Its current AI call sites do not use the shared Go-capable gateway builders. Inventory the exact provider and auth contract before migration. | -| Wizard | Still has Python product policy and direct Gemini paths. It requires first-party product policy that is not yet available together in Go. | -| Warehouse semantic enrichment | Uses the Python `warehouse_semantic_enrichment` product client, and its worker deployment has only Python gateway settings. | - ## Choose by use case ### ✅ Use the Go gateway From ea7a78591def1ecd75a9d5434ebda59752918499 Mon Sep 17 00:00:00 2001 From: Richard Solomou Date: Fri, 31 Jul 2026 14:38:19 +0300 Subject: [PATCH 13/20] chore(ai-gateway): add migration candidate discovery skill Generated-By: PostHog Code Task-Id: e2e45fbc-1c5e-4ef2-9688-bcc941c2229c --- .../SKILL.md | 73 +++++++++++++++++++ AGENTS.md | 3 +- 2 files changed, 75 insertions(+), 1 deletion(-) create mode 100644 .agents/skills/finding-llm-gateway-migration-candidates/SKILL.md diff --git a/.agents/skills/finding-llm-gateway-migration-candidates/SKILL.md b/.agents/skills/finding-llm-gateway-migration-candidates/SKILL.md new file mode 100644 index 000000000000..71c9cb1c4e3d --- /dev/null +++ b/.agents/skills/finding-llm-gateway-migration-candidates/SKILL.md @@ -0,0 +1,73 @@ +--- +name: finding-llm-gateway-migration-candidates +description: > + Finds and ranks callers that could move from services/llm-gateway to PostHog/ai-gateway. Use when asked what to migrate next, to find low-risk gateway migration candidates, to audit remaining Python gateway callers, or to identify callers blocked by Go gateway parity. Searches code and deployment wiring, inventories each caller's required contract, filters out unsupported migrations, and returns an evidence-backed shortlist without changing callers. +--- + +# Finding LLM gateway migration candidates + +Use [`services/llm-gateway/PARITY.md`](../../../services/llm-gateway/PARITY.md) as the current contract decision record. If implementation evidence contradicts it, run `/auditing-llm-gateway-parity` before ranking candidates. + +This skill produces a point-in-time shortlist. Do not turn the parity document into a fleet-wide migration tracker. + +## Discover callers + +Search the current default branches for Python gateway use, including: + +- `LLM_GATEWAY_URL`, `LLM_GATEWAY_API_KEY`, and product-slug URLs +- `get_llm_client`, `get_async_llm_client`, and `get_async_anthropic_gateway_client` +- direct OpenAI, Anthropic, or agent SDK clients configured with a gateway base URL +- Python product names from `services/llm-gateway/src/llm_gateway/products/config.py` +- deployment, sandbox, workflow, and secret wiring in other PostHog repositories + +Trace each result to the production call site. Exclude tests, development-only tools, dead code, and callers already configured exclusively for Go. Treat shared worker configuration as evidence about availability, not proof that every call in that process uses Go. + +Search open and recently merged PRs across involved repositories before proposing work. A migration may already be underway even when the default branch still uses Python. + +## Inventory each candidate + +Record only the contracts that affect the migration decision: + +- user-facing use case and production entry point +- credential source and trusted authorization policy +- billing owner, budget enforcement, and billable or unbilled behavior +- API shape, model, provider, streaming, tools, and structured output +- distinct ID, trace, product, team, and custom attribution +- retry, timeout, fallback, and error behavior +- deployment and egress changes needed to activate and roll back the route + +Do not infer requirements from a product name. Read the call and its configuration. + +## Classify readiness + +Match the inventory to the parity record and current Go implementation: + +- ✅ **Ready:** required contracts are supported and the remaining work is client, test, or deployment wiring. +- 🔎 **Verify:** support probably exists, but model availability, wire behavior, billing, attribution, or deployment configuration needs a targeted check. +- ⛔ **Blocked:** migration would lose a required auth, policy, billing, provider, API, or attribution contract. +- 🚧 **In progress:** an open or recently merged PR already owns the migration. + +Name the exact evidence for every status. An existing Python helper is migration effort, not a blocker. An `ai_product` property is telemetry, not trusted identity or billing policy. + +## Rank the shortlist + +Prioritize candidates that: + +1. Have no ⛔ contract. +2. Use a shared Go-capable builder or a stock SDK with a simple base URL change. +3. Already use a Go-supported model, provider, and API shape. +4. Need informational attribution rather than trusted product identity. +5. Have a narrow deployment boundary, explicit fallback, and cheap verification path. + +Lower the rank for broad shared-process switches, unverified billing changes, cross-repository deployment work, or missing production tests. Do not rank by code size alone. + +## Report candidates + +Return a small decision-oriented table with: + +| Candidate | Readiness | Why | Required work | Evidence | +| --------- | --------- | --- | ------------- | -------- | + +List the strongest ready candidate first. For blocked callers, state the exact missing Go contract and where it was verified. For in-progress work, link the existing PR instead of proposing duplicate work. + +Do not modify callers while running this skill. Once a candidate is selected, run `/migrating-llm-gateway-callers` for that caller. diff --git a/AGENTS.md b/AGENTS.md index 2fcab1312b9f..4820e32966fb 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -168,7 +168,7 @@ See [.agents/security.md](.agents/security.md) for security guidelines — least - **Object storage is SeaweedFS — do not add new MinIO dependencies.** Both S3-compatible stores in the dev/CI stack are SeaweedFS: the `objectstorage` service (S3 API on `:19000`) backs general object storage (`OBJECT_STORAGE_*` settings — exports, media uploads, error-tracking source maps, query cache, tasks), and the `seaweedfs` service (S3 API on `:8333`) backs session replay v2 (`SESSION_RECORDING_V2_S3_*` settings). MinIO now survives only as migration tooling: `docker-compose.hobby.yml` keeps it as a source for `bin/migrate-storage-hobby`, and `bin/upgrade-objectstorage` starts a throwaway MinIO to salvage objects off the pre-swap volume. Outside that, don't add docker-compose services, scripts, tests, or docs that stand up a `minio/minio` container. Code that talks to object storage should go through the existing `OBJECT_STORAGE_*` / `SESSION_RECORDING_V2_S3_*` config and a standard S3 client rather than hardcoding an endpoint — that keeps backends swappable. Note the `objectstorage` service registers its credentials at runtime via a bootstrap loop and returns `InvalidAccessKeyId` until that completes, so anything depending on it must wait for its readiness sentinel rather than just for the container to start. - **Temporal activity payloads have a ~2 MiB hard limit — pass large data by reference, not by value.** Activity inputs and outputs are serialized across a gRPC boundary that Temporal caps at ~2 MiB per payload (the server rejects larger payloads via `blobSizeLimitError`). As a conservative field-level rule, if a field could exceed ~256 KB once serialized (serialized query results, exported file contents, LLM context, rendered HTML, image bytes, unbounded `list[dict[str, Any]]`), write it to Postgres / S3 / object storage from _inside_ the activity and return only the reference (row ID, S3 key). The workflow already has access to any row ID created earlier in the same run; it does not need the content to flow back through. Shuttling large data through the workflow on the way to persistence is a foreseeable failure mode that produces `PayloadSizeError` (`TMPRL1103`) the moment the underlying data crosses the limit. - **Outbound calls to a third-party API that need rate-limiting or egress telemetry belong in `posthog/egress/` — add a `/` incarnation (GitHub is the reference) and route callers through its gated, recorded transport, never hand-rolled `requests`. See `posthog/egress/README.md`.** -- **`services/llm-gateway` is under an unofficial code freeze while callers move to [`PostHog/ai-gateway`](https://github.com/PostHog/ai-gateway).** New callers and features belong on the Go gateway by default. A Python gateway change needs a documented parity blocker for an active caller and must stay limited to that blocker. Read [`services/llm-gateway/PARITY.md`](services/llm-gateway/PARITY.md). Invoke `/auditing-llm-gateway-parity` for gateway contract changes and parity refreshes, and `/migrating-llm-gateway-callers` when moving a caller. +- **`services/llm-gateway` is under an unofficial code freeze while callers move to [`PostHog/ai-gateway`](https://github.com/PostHog/ai-gateway).** New callers and features belong on the Go gateway by default. A Python gateway change needs a documented parity blocker for an active caller and must stay limited to that blocker. Read [`services/llm-gateway/PARITY.md`](services/llm-gateway/PARITY.md). Invoke `/auditing-llm-gateway-parity` for gateway contract changes and parity refreshes, `/finding-llm-gateway-migration-candidates` when deciding what to migrate next, and `/migrating-llm-gateway-callers` when moving a selected caller. ## Code Style @@ -243,4 +243,5 @@ ALWAYS invoke the matching skill **before** writing or reviewing code in these a - `/reviewing-personhog-protocol` — any personhog coordination-protocol change (leases, fencing, handoffs, supervisors, budgets, warming, changelog semantics), and any request for an exhaustive review of personhog code - `/gating-production-deploys` — any workflow that builds and pushes a production image or dispatches a deploy - `/auditing-llm-gateway-parity` — changing either gateway's auth, attribution, billing, endpoints, providers, models, routing, or metadata contract; reviewing a `services/llm-gateway` change; or refreshing `services/llm-gateway/PARITY.md` +- `/finding-llm-gateway-migration-candidates` — finding, auditing, or ranking callers that could move from `services/llm-gateway` to `PostHog/ai-gateway`, including requests for the next or lowest-risk migration candidate - `/migrating-llm-gateway-callers` — adding an LLM gateway caller or migrating an existing caller from `services/llm-gateway` to `PostHog/ai-gateway`, including shared client and gateway setting changes made for that migration From 7970fb91731a1f2b4a8f5dd0c2d9f44324d74c36 Mon Sep 17 00:00:00 2001 From: Richard Solomou Date: Fri, 31 Jul 2026 14:43:24 +0300 Subject: [PATCH 14/20] chore(ai-gateway): clarify internal spend ownership Generated-By: PostHog Code Task-Id: e2e45fbc-1c5e-4ef2-9688-bcc941c2229c --- .../skills/auditing-llm-gateway-parity/SKILL.md | 2 ++ .../SKILL.md | 7 +++++-- .../migrating-llm-gateway-callers/SKILL.md | 4 +++- services/llm-gateway/PARITY.md | 16 ++++++++-------- 4 files changed, 18 insertions(+), 11 deletions(-) diff --git a/.agents/skills/auditing-llm-gateway-parity/SKILL.md b/.agents/skills/auditing-llm-gateway-parity/SKILL.md index 51b06e061603..feb85ef09403 100644 --- a/.agents/skills/auditing-llm-gateway-parity/SKILL.md +++ b/.agents/skills/auditing-llm-gateway-parity/SKILL.md @@ -46,6 +46,8 @@ Check both request and response behavior. Matching route names do not prove pari Do not treat caller-supplied telemetry as trusted policy. An `ai_product` event property does not replace product authentication, authorization, or billing. +Do not treat Python's unbilled flag as an automatic blocker. An internal workload can move to Go with a PostHog-owned team credential when debiting that wallet is the intended way to attribute PostHog spend. It is blocked only when it must preserve customer-specific billing policy or must debit no wallet. + For each difference, identify at least one affected use-case class. Remove details that do not change a migration decision. ## Update the parity record diff --git a/.agents/skills/finding-llm-gateway-migration-candidates/SKILL.md b/.agents/skills/finding-llm-gateway-migration-candidates/SKILL.md index 71c9cb1c4e3d..b3d0453997bb 100644 --- a/.agents/skills/finding-llm-gateway-migration-candidates/SKILL.md +++ b/.agents/skills/finding-llm-gateway-migration-candidates/SKILL.md @@ -30,7 +30,7 @@ Record only the contracts that affect the migration decision: - user-facing use case and production entry point - credential source and trusted authorization policy -- billing owner, budget enforcement, and billable or unbilled behavior +- spend owner, budget enforcement, current Python billing behavior, and the team wallet that should own Go spend - API shape, model, provider, streaming, tools, and structured output - distinct ID, trace, product, team, and custom attribution - retry, timeout, fallback, and error behavior @@ -49,6 +49,8 @@ Match the inventory to the parity record and current Go implementation: Name the exact evidence for every status. An existing Python helper is migration effort, not a blocker. An `ai_product` property is telemetry, not trusted identity or billing policy. +Python's unbilled flag is not a blocker when an internal workload should debit a PostHog-owned team wallet for spend attribution. Verify that the Go credential resolves to that team. Treat billing as blocked only when migration would charge a customer incorrectly, lose required customer budget policy, or violate a requirement to debit no wallet. + ## Rank the shortlist Prioritize candidates that: @@ -57,7 +59,8 @@ Prioritize candidates that: 2. Use a shared Go-capable builder or a stock SDK with a simple base URL change. 3. Already use a Go-supported model, provider, and API shape. 4. Need informational attribution rather than trusted product identity. -5. Have a narrow deployment boundary, explicit fallback, and cheap verification path. +5. Can use a customer wallet intentionally or a PostHog-owned wallet for internal spend. +6. Have a narrow deployment boundary, explicit fallback, and cheap verification path. Lower the rank for broad shared-process switches, unverified billing changes, cross-repository deployment work, or missing production tests. Do not rank by code size alone. diff --git a/.agents/skills/migrating-llm-gateway-callers/SKILL.md b/.agents/skills/migrating-llm-gateway-callers/SKILL.md index 034396cc6187..baf98569cb20 100644 --- a/.agents/skills/migrating-llm-gateway-callers/SKILL.md +++ b/.agents/skills/migrating-llm-gateway-callers/SKILL.md @@ -14,7 +14,7 @@ Find the production call site, client construction, settings, deployment wiring, - caller and user-facing use case - credential source and required authorization policy -- billing owner, budgets, and whether the call is intentionally unbilled +- spend owner, budgets, current Python billing behavior, and the team wallet that should own Go spend - API shape, model, provider, streaming, and tool or structured-output requirements - distinct ID, trace, product, team, feature flag, and custom property attribution - timeout, retry, fallback, and error-handling behavior @@ -32,6 +32,8 @@ Match every required contract to the parity record and current code. An existing Python client or product route is not a blocker by itself. +Python's unbilled flag is not a blocker when an internal workload should debit a PostHog-owned team wallet for spend attribution. Confirm that the Go credential resolves to that team. Stop only when migration would charge a customer incorrectly, lose required customer budget policy, or violate a requirement to debit no wallet. + ## Implement the migration Prefer the smallest existing pattern that matches the caller: diff --git a/services/llm-gateway/PARITY.md b/services/llm-gateway/PARITY.md index 0ba9fbad585a..afe791014f23 100644 --- a/services/llm-gateway/PARITY.md +++ b/services/llm-gateway/PARITY.md @@ -24,12 +24,12 @@ When a gap closes, new work uses the Go gateway and affected callers should migr ### ✅ Use the Go gateway -| Use case | Why it fits | -| ----------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------ | -| Customer LLM traffic | The caller uses a project secret or OAuth credential, the team wallet should pay, and standard OpenAI or Anthropic APIs are enough. | -| Server-to-server PostHog call | A `phs_` credential, team wallet billing, and informational event properties provide enough policy and attribution. | -| Stock SDK proxy | The caller needs OpenAI Chat Completions or Responses, Anthropic Messages or token counting, streaming, idempotency, or the model catalog. | -| Gateway-managed routing | The caller accepts Go host selection across OpenAI, Anthropic, Azure OpenAI, Bedrock, or selected Modal models. | +| Use case | Why it fits | +| ----------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Customer LLM traffic | The caller uses a project secret or OAuth credential, the team wallet should pay, and standard OpenAI or Anthropic APIs are enough. | +| Server-to-server PostHog call | A `phs_` credential, team wallet billing, and informational event properties provide enough policy and attribution. Internal spend can use a PostHog-owned team wallet. | +| Stock SDK proxy | The caller needs OpenAI Chat Completions or Responses, Anthropic Messages or token counting, streaming, idempotency, or the model catalog. | +| Gateway-managed routing | The caller accepts Go host selection across OpenAI, Anthropic, Azure OpenAI, Bedrock, or selected Modal models. | Existing Django callers should use `build_openai_client`, `build_async_openai_client`, or `build_async_anthropic_client` from [`posthog/llm/gateway_client.py`](../../posthog/llm/gateway_client.py). These builders keep a temporary Python fallback during rollout. @@ -38,7 +38,7 @@ Existing Django callers should use `build_openai_client`, `build_async_openai_cl | Use case | Blocking contract | | ------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | PostHog Desktop, PostHog Code, cloud agents, onboarding, Wizard, and similar first-party products | Trusted product identity, OAuth application allowlists, OAuth user attribution, server-minted credential requirements, per-product model policy, and product billing or unbilled policy are not available together in Go. An `ai_product` event property is not an authorization or billing boundary. | -| Product or user budget enforcement | Python supports product and user cost limits, plan checks, quota buckets, and unbilled products. Go currently admits against the team wallet and does not branch settlement on credential billing mode. | +| Product or user budget enforcement | Python supports product and user cost limits, plan checks, quota buckets, and requests that debit no wallet. Go currently admits against the credential owner's team wallet and does not branch settlement on credential billing mode. | | OpenRouter, Fireworks, Cloudflare Workers AI, or Baseten | These provider paths are not available in Go. | | OpenAI audio transcription | Go has no transcription endpoint. | | OpenAI models through Anthropic Messages | Go does not provide this reverse translation. | @@ -50,7 +50,7 @@ These are compatibility checks, not automatic blockers: - **Model:** it appears in `GET /v1/models` and supports the requested API shape and capabilities. - **Credential:** Go accepts the credential type and projects the correct team, scope, and revocation state. -- **Billing:** the request should reserve and debit the Go wallet. +- **Billing:** identify which team owns the Go credential and should pay. Internal workloads can debit a PostHog-owned team wallet so spend stays attributable without charging a customer. - **Attribution:** `X-PostHog-Distinct-Id`, `X-PostHog-Trace-Id`, and `X-PostHog-Properties` provide enough event context. Go does not derive the event distinct ID from OpenAI `user`, Anthropic `metadata.user_id`, or the OAuth user. Caller-supplied properties are not trusted policy. - **Provider behavior:** health-based routing or strict `X-PostHog-Provider` pinning matches the caller's fallback requirements. - **Wire behavior:** request fields, streaming chunks, errors, timeouts, and retries match what the caller handles. From dc19c39a9c13cdc8d510c815a84cfeb5f41b5ed6 Mon Sep 17 00:00:00 2001 From: Richard Solomou Date: Fri, 31 Jul 2026 14:48:26 +0300 Subject: [PATCH 15/20] chore(ai-gateway): strengthen candidate verification Generated-By: PostHog Code Task-Id: e2e45fbc-1c5e-4ef2-9688-bcc941c2229c --- .../SKILL.md | 26 ++++++++++++++++--- 1 file changed, 22 insertions(+), 4 deletions(-) diff --git a/.agents/skills/finding-llm-gateway-migration-candidates/SKILL.md b/.agents/skills/finding-llm-gateway-migration-candidates/SKILL.md index b3d0453997bb..0add2cc407f9 100644 --- a/.agents/skills/finding-llm-gateway-migration-candidates/SKILL.md +++ b/.agents/skills/finding-llm-gateway-migration-candidates/SKILL.md @@ -24,6 +24,18 @@ Trace each result to the production call site. Exclude tests, development-only t Search open and recently merged PRs across involved repositories before proposing work. A migration may already be underway even when the default branch still uses Python. +## Prove the production path + +Before classifying a caller, follow it through every runtime boundary: + +1. Find the API, task, workflow, schedule, or command that invokes it. +2. For Temporal, map the workflow and activity to the exact task queue, then map that queue to its deployed worker. +3. Inspect every environment overlay that runs the caller. Do not borrow configuration from another worker in the same product. +4. Resolve the configured URL by host and path. Variable names are not proof: an `AI_GATEWAY_URL` can still contain a Python product-slug URL. +5. Confirm the current call can succeed. A broad exception handler with a deterministic fallback can hide missing credentials or a route that never runs. + +A secret reference proves that a credential is injected. It does not prove which team owns it, whether the wallet is funded, or whether its scope is correct. Keep the candidate at 🔎 until those properties are verified from an authoritative source, unless the same credential is already proven on a comparable Go workload. + ## Inventory each candidate Record only the contracts that affect the migration decision: @@ -31,13 +43,17 @@ Record only the contracts that affect the migration decision: - user-facing use case and production entry point - credential source and trusted authorization policy - spend owner, budget enforcement, current Python billing behavior, and the team wallet that should own Go spend -- API shape, model, provider, streaming, tools, and structured output +- API shape and model as one pair, provider, streaming, tools, and structured output - distinct ID, trace, product, team, and custom attribution - retry, timeout, fallback, and error behavior - deployment and egress changes needed to activate and roll back the route Do not infer requirements from a product name. Read the call and its configuration. +Check model and API shape together. Python can expose an Anthropic model through OpenAI Chat Completions, while Go's native routes do not generally cross-translate shapes. A migration may need both a gateway change and an SDK-shape change. + +Separate billing identity from event attribution. Go debits the credential's team. A caller-supplied `team_id` or `ai_product` property can preserve reporting context but cannot select the wallet. + ## Classify readiness Match the inventory to the parity record and current Go implementation: @@ -51,6 +67,8 @@ Name the exact evidence for every status. An existing Python helper is migration Python's unbilled flag is not a blocker when an internal workload should debit a PostHog-owned team wallet for spend attribution. Verify that the Go credential resolves to that team. Treat billing as blocked only when migration would charge a customer incorrectly, lose required customer budget policy, or violate a requirement to debit no wallet. +Use ✅ only when the production entry point, worker or process, target credential, model and API pair, attribution conversion, and rollback path are all supported by evidence. Use 🔎 when any of those facts depends on an unverified secret, runtime state, or external configuration. + ## Rank the shortlist Prioritize candidates that: @@ -68,9 +86,9 @@ Lower the rank for broad shared-process switches, unverified billing changes, cr Return a small decision-oriented table with: -| Candidate | Readiness | Why | Required work | Evidence | -| --------- | --------- | --- | ------------- | -------- | +| Candidate | Readiness | Why | Required work | Evidence gaps | +| --------- | --------- | --- | ------------- | ------------- | -List the strongest ready candidate first. For blocked callers, state the exact missing Go contract and where it was verified. For in-progress work, link the existing PR instead of proposing duplicate work. +List the strongest ready candidate first. Name the exact production process and deployment for each candidate. For blocked callers, state the missing Go contract and where it was verified. For in-progress work, link the existing PR instead of proposing duplicate work. For 🔎 candidates, state the one check that would promote or reject them. Do not modify callers while running this skill. Once a candidate is selected, run `/migrating-llm-gateway-callers` for that caller. From c3ddab9c564d5db7a527d02b13bdd6c857e7df60 Mon Sep 17 00:00:00 2001 From: Richard Solomou Date: Fri, 31 Jul 2026 14:51:27 +0300 Subject: [PATCH 16/20] chore(ai-gateway): treat credential creation as migration work Generated-By: PostHog Code Task-Id: e2e45fbc-1c5e-4ef2-9688-bcc941c2229c --- .../skills/finding-llm-gateway-migration-candidates/SKILL.md | 4 ++-- .agents/skills/migrating-llm-gateway-callers/SKILL.md | 2 ++ 2 files changed, 4 insertions(+), 2 deletions(-) diff --git a/.agents/skills/finding-llm-gateway-migration-candidates/SKILL.md b/.agents/skills/finding-llm-gateway-migration-candidates/SKILL.md index 0add2cc407f9..048b1d963b49 100644 --- a/.agents/skills/finding-llm-gateway-migration-candidates/SKILL.md +++ b/.agents/skills/finding-llm-gateway-migration-candidates/SKILL.md @@ -34,7 +34,7 @@ Before classifying a caller, follow it through every runtime boundary: 4. Resolve the configured URL by host and path. Variable names are not proof: an `AI_GATEWAY_URL` can still contain a Python product-slug URL. 5. Confirm the current call can succeed. A broad exception handler with a deterministic fallback can hide missing credentials or a route that never runs. -A secret reference proves that a credential is injected. It does not prove which team owns it, whether the wallet is funded, or whether its scope is correct. Keep the candidate at 🔎 until those properties are verified from an authoritative source, unless the same credential is already proven on a comparable Go workload. +A missing `phs_` credential is deployment work, not a parity blocker. Once the intended paying team is known, create a project secret with `llm_gateway:read` in the PostHog dashboard and fund or configure that team wallet as part of the migration. A secret reference proves only that a credential is injected; verify ownership, scope, and funding during implementation or rollout. ## Inventory each candidate @@ -67,7 +67,7 @@ Name the exact evidence for every status. An existing Python helper is migration Python's unbilled flag is not a blocker when an internal workload should debit a PostHog-owned team wallet for spend attribution. Verify that the Go credential resolves to that team. Treat billing as blocked only when migration would charge a customer incorrectly, lose required customer budget policy, or violate a requirement to debit no wallet. -Use ✅ only when the production entry point, worker or process, target credential, model and API pair, attribution conversion, and rollback path are all supported by evidence. Use 🔎 when any of those facts depends on an unverified secret, runtime state, or external configuration. +Use ✅ when the production entry point, worker or process, intended paying team, model and API pair, attribution conversion, and rollback path are supported by evidence. The token itself may still need to be created and wired. Use 🔎 when the spend owner, required policy, runtime behavior, or another migration contract is unknown. ## Rank the shortlist diff --git a/.agents/skills/migrating-llm-gateway-callers/SKILL.md b/.agents/skills/migrating-llm-gateway-callers/SKILL.md index baf98569cb20..7db74c864a84 100644 --- a/.agents/skills/migrating-llm-gateway-callers/SKILL.md +++ b/.agents/skills/migrating-llm-gateway-callers/SKILL.md @@ -51,6 +51,8 @@ Read [migration examples](references/migration-examples.md) for verified PRs cov For sandbox callers, follow the existing `SANDBOX_AI_GATEWAY_URL` and product rollout patterns rather than inventing another environment contract. +If the target process does not have a Go credential, create a `phs_` project secret for the intended paying team in the PostHog dashboard, grant `llm_gateway:read`, and wire it through the existing deployment secret mechanism. Credential creation is normal migration work, not a parity exception. + ## Update tests and docs Invoke `/writing-tests` before changing tests. From db110000afffb05b0f373d4ab516827490b3e320 Mon Sep 17 00:00:00 2001 From: Richard Solomou Date: Fri, 31 Jul 2026 14:53:47 +0300 Subject: [PATCH 17/20] chore(ai-gateway): refresh migration guidance Generated-By: PostHog Code Task-Id: e2e45fbc-1c5e-4ef2-9688-bcc941c2229c --- .../references/migration-examples.md | 1 - services/llm-gateway/PARITY.md | 2 +- 2 files changed, 1 insertion(+), 2 deletions(-) diff --git a/.agents/skills/migrating-llm-gateway-callers/references/migration-examples.md b/.agents/skills/migrating-llm-gateway-callers/references/migration-examples.md index dd72c7132492..bc39bf1f5923 100644 --- a/.agents/skills/migrating-llm-gateway-callers/references/migration-examples.md +++ b/.agents/skills/migrating-llm-gateway-callers/references/migration-examples.md @@ -7,7 +7,6 @@ These PRs show different migration shapes. Read the relevant diff before impleme - [#64448: make cluster labeling routable through the AI gateway](https://github.com/PostHog/posthog/pull/64448) introduces an environment-gated OpenAI client, validates paired settings, preserves direct-provider fallback, checks model availability before rollout, and tests both routes. - [#65043: route the eval-report agent through the AI gateway](https://github.com/PostHog/posthog/pull/65043) moves the AI observability eval-report agent after the shared route exists and tests the selected client path. - [#65044: route OpenAI summarization through the AI gateway](https://github.com/PostHog/posthog/pull/65044) extracts shared sync and async builders, uses the slugless Go URL and project secret, keeps `trust_env=False`, retains the Python fallback, and prevents duplicate `$ai_generation` capture. -- [#68329: route the PR-approval agent through the AI gateway](https://github.com/PostHog/posthog/pull/68329) covers a non-Django SDK process. It validates and translates gateway environment variables, switches away from the traced SDK wrapper in gateway mode, keeps a direct fallback, and tests credential and metadata wiring without exposing secret values. ## Incremental product rollout diff --git a/services/llm-gateway/PARITY.md b/services/llm-gateway/PARITY.md index afe791014f23..1b6576ff570b 100644 --- a/services/llm-gateway/PARITY.md +++ b/services/llm-gateway/PARITY.md @@ -87,7 +87,7 @@ Run `/auditing-llm-gateway-parity` after either gateway changes auth, attributio Last verified on 2026-07-31 against: -- `PostHog/posthog` master at `0350fb13ed8321a5b4199a111b31052e2218360b` +- `PostHog/posthog` master at `75c2b2ff6845097a3eb6dc5214e3f4f660395fb8` - `PostHog/ai-gateway` main at `ea8230b6dbc4ebf6c4be83d359e561477a13b215` ## References From cd00300f32fcfe3209449cf2d610229e6e3ba6ab Mon Sep 17 00:00:00 2001 From: Richard Solomou Date: Fri, 31 Jul 2026 15:24:38 +0300 Subject: [PATCH 18/20] docs(ai-gateway): clarify staged rollout fallback Generated-By: PostHog Code Task-Id: e2e45fbc-1c5e-4ef2-9688-bcc941c2229c --- services/llm-gateway/PARITY.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/services/llm-gateway/PARITY.md b/services/llm-gateway/PARITY.md index 1b6576ff570b..38d2a10efe63 100644 --- a/services/llm-gateway/PARITY.md +++ b/services/llm-gateway/PARITY.md @@ -79,7 +79,7 @@ Run `/migrating-llm-gateway-callers` to inventory and convert a caller. 2. Check every relevant contract above against the real request and response. 3. Start with a shared Go-capable builder where one exists. 4. Test success, streaming if used, provider errors, attribution, and billing. -5. Keep the Python fallback only for a named blocker. Record that blocker in the PR. +5. Keep the Python fallback only for staged rollout or a named blocker. Record the rollout plan or blocker in the PR. ## Refreshing this document From 9b07cf8c0999e6cc723b2e2a637607a29c3d2159 Mon Sep 17 00:00:00 2001 From: Richard Solomou Date: Fri, 31 Jul 2026 16:02:25 +0300 Subject: [PATCH 19/20] chore(ai-gateway): audit in-flight parity changes Generated-By: PostHog Code Task-Id: e2e45fbc-1c5e-4ef2-9688-bcc941c2229c --- .agents/skills/auditing-llm-gateway-parity/SKILL.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/.agents/skills/auditing-llm-gateway-parity/SKILL.md b/.agents/skills/auditing-llm-gateway-parity/SKILL.md index feb85ef09403..e2276d3087c4 100644 --- a/.agents/skills/auditing-llm-gateway-parity/SKILL.md +++ b/.agents/skills/auditing-llm-gateway-parity/SKILL.md @@ -6,11 +6,12 @@ description: > # Auditing LLM gateway parity -Compare the current default branches and update [`services/llm-gateway/PARITY.md`](../../../services/llm-gateway/PARITY.md). Keep this audit separate from caller migration. +Compare the current PostHog working tree with the Go gateway's current default branch and update [`services/llm-gateway/PARITY.md`](../../../services/llm-gateway/PARITY.md). Keep this audit separate from caller migration. ## Record source revisions -- Record `origin/master` for `PostHog/posthog`. +- Fetch `origin/master` and record its SHA as the PostHog baseline. +- Audit the current working tree, including committed and uncommitted changes relative to that baseline. Do not ignore an in-flight gateway change because it is not on `master` yet. - Read the current `PostHog/ai-gateway` main SHA with `gh api repos/PostHog/ai-gateway/commits/main`. - Use an authenticated checkout of `PostHog/ai-gateway` when code inspection is needed. From f2c294800853c6d953dbeee0b3f371fb9b3bc5be Mon Sep 17 00:00:00 2001 From: Richard Solomou Date: Fri, 31 Jul 2026 16:03:47 +0300 Subject: [PATCH 20/20] chore(ai-gateway): audit pending Go changes Generated-By: PostHog Code Task-Id: e2e45fbc-1c5e-4ef2-9688-bcc941c2229c --- .agents/skills/auditing-llm-gateway-parity/SKILL.md | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/.agents/skills/auditing-llm-gateway-parity/SKILL.md b/.agents/skills/auditing-llm-gateway-parity/SKILL.md index e2276d3087c4..4b4e3a024312 100644 --- a/.agents/skills/auditing-llm-gateway-parity/SKILL.md +++ b/.agents/skills/auditing-llm-gateway-parity/SKILL.md @@ -6,14 +6,15 @@ description: > # Auditing LLM gateway parity -Compare the current PostHog working tree with the Go gateway's current default branch and update [`services/llm-gateway/PARITY.md`](../../../services/llm-gateway/PARITY.md). Keep this audit separate from caller migration. +Compare the current implementations of both gateways and update [`services/llm-gateway/PARITY.md`](../../../services/llm-gateway/PARITY.md). Keep this audit separate from caller migration. ## Record source revisions - Fetch `origin/master` and record its SHA as the PostHog baseline. - Audit the current working tree, including committed and uncommitted changes relative to that baseline. Do not ignore an in-flight gateway change because it is not on `master` yet. -- Read the current `PostHog/ai-gateway` main SHA with `gh api repos/PostHog/ai-gateway/commits/main`. -- Use an authenticated checkout of `PostHog/ai-gateway` when code inspection is needed. +- Read the current `PostHog/ai-gateway` main SHA with `gh api repos/PostHog/ai-gateway/commits/main` and record it as the Go baseline. +- If the audit is for an in-flight Go change, inspect that PR branch or working tree relative to `main`. Treat its contracts as pending until the change merges; do not record them as currently supported. +- Otherwise, inspect an authenticated checkout of `PostHog/ai-gateway` at `main`. Audit implementation code. README files and the existing parity table are starting points, not proof.