Community Update — Post-GA Catch-Up (June 15 – July 6, 2026)

[missBerg]

Our first community update since the v1.0.0 GA announcement — so this one's a catch-up covering the weeks on either side of the release: 1.0 is out and the API is stable, the team spent the post-GA stretch hardening real-world edge cases, the next MCP protocol version has become the headline roadmap item, a batch of new design proposals opened up, and the last three community calls surfaced how people are running the gateway at scale. Thanks to everyone who shipped, filed, reviewed, and showed up to the calls. 🎉

🛣️ The road ahead — post-1.0

v1.0.0 is GA — released June 23. The core control-plane API (AIGatewayRoute, AIServiceBackend, BackendSecurityPolicy, GatewayConfig, MCPRoute, all at v1beta1) is now a stable contract for the 1.x series, and upgrading from v0.7 is a drop-in — no resource changes. If you haven't read it yet, the announcement lays out the stability commitment.

With GA behind us, the near-term roadmap is anchored by one big item:

• The next MCP protocol revision (Release Candidate July 28) — tracked in #2323. It's a full conformance matrix: stateless Streamable HTTP sessions, the MRTR result-type changes, cacheable tools/list results, and auth tightening — with backward compatibility for clients and backends still on the 2025-11-25 era. From the community calls, @Hritik003 and @jappievw are driving the scoping, with roughly a month to land the controller and ext_proc changes and settle the two-version compatibility story. If you build on the MCP proxy, this is the thread to weigh in on now.

Also brewing, and worth tracking:

• Google A2A support — #2070. The data-plane pieces already exist upstream (Envoy has an A2A decoder filter that publishes protocol fields into dynamic metadata, and native Envoy RBAC can act on them). What's missing is a high-level AI Gateway API to configure it — 100% on the roadmap.
• Quota-aware routing continues to build on the rate-limit filter injection that landed in v0.7.

✨ What's shipped since GA

Post-GA, the team pointed its energy at hardening 1.0 for the edge cases production surfaces.

Bug fixes & stability

• No more extproc crash on Bedrock streaming marshal errors — #2289 by @Aias00. The OpenAI-to-Bedrock streaming translator returned an error instead of panic(), so a bad chunk no longer takes down the ext_proc container mid-request.
• Empty gzip bodies no longer abort a good response — #2291 by @ChrisJBurns. A gzip-encoded response with an empty body was being treated as a fatal decode error (seen in production as a 200 with zero bytes sent); it's now passed through cleanly.
• Nil OpenAI response body no longer panics — #2240 by @kunwarshivam. A JSON null body with a 200 status used to SIGSEGV the ext_proc; it's now treated as an empty response with zero usage. Closes #2273.
• Correct Bedrock tool-result coalescing — #2237 by @Aias00. Fixes an index misalignment in the Anthropic-to-Bedrock translator when system messages are present, which could read the wrong message or panic.
• Fix silent 500s on same-namespace v1.0.0 installs — #2304 by @kothari-karan. Dedups the candidate namespaces in getObjectsForGateway, fixing a v1.0.0 regression where running the controller in the Gateway's own namespace returned HTTP 500 for any route added after startup.

Observability & usage

• Reasoning tokens counted in Gemini streaming metrics — #2259 by @hustxiayang. Streaming token usage for Gemini reasoning models now folds reasoning tokens into the output total, matching non-streaming accounting — so operators get accurate token and cost metrics on streamed responses.

Developer experience

• Codex support for the Responses API — #2317 by @xiaolin593. Adds namespace and tool_search tool support to the Responses API so the latest Codex CLI (≥ 0.120.0) works through the gateway instead of failing on the missing fields.

💡 New design proposals & feature requests

A strong stretch for proposals — worth a read if any land in your wheelhouse:

• Scale-from-zero / hibernate-aware routing — #2327 by @mohitgurnani1. A coldStart field so the gateway holds a buffered request and fires an optional wake-up trigger instead of returning an immediate 503 when a GPU backend is scaled to zero. Comes with a data-plane PoC and a design doc up in #2328.
• Router-phase request/response mutation via an external service — draft #2314 by @siddharth1036 (proposal #2086). Raised on the June 29 call: because every request lands on the AI Gateway catch-all route, there's no clean way to run a per-rule ext_proc. The proposal calls an external service at the router-filter phase to mutate the body (model selection, system-prompt injection, redaction, semantic routing) through a fixed interface.
• Customizable MCP tool-name prefix — #2316 by @dmitrysDV. A per-backend toolPrefix/alias so multiplexed MCP tool names don't have to carry the raw backend name — billing__get-invoices instead of acme-billing-mcp__get-invoices — while preserving the routing guarantee the prefix exists for.
• Required flag for MCP forward headers — #2309 by @Hritik003. An optional required on forwarded headers so a missing per-user header is rejected up front instead of silently falling back to the shared backend credential. Implementation is up in #2312.
• version command for the controller — #2322 by @adhityaravi. A way to probe the deployed controller version, mirroring envoy-gateway version, so automated deployments can keep controller and ext_proc image versions in sync.

🗣️ From the community calls (June 15, 22, 29)

The three calls since GA had a clear throughline: people are putting 1.0 into production, and the questions are about scale and real-world topology.

• Running multi-tenant at scale. @PraveenKumarInjam (large SaaS fleet) and Charan Teja from ServiceNow (~7,000 instances, ~10 rules each) asked how the gateway holds up with per-tenant rate limits expressed as CRDs. Maintainer guidance from @johnugeorge and @nacx: avoid one route per customer — do rule- and header-based routing on models instead, and split configuration across CRDs that reference a parent rather than one oversized resource. The team asked for a written tenancy model to give concrete advice, and to weigh whether pushing rate-limit config to the rate-limit service over xDS is warranted.
• Files API sticky routing. Continuing an earlier thread, the leaning is to reuse the MCP "envelope" session-stickiness approach — synthesize an ID the client echoes back — rather than relying on Set-Cookie, since client cookie support varies across SDKs.
• The ext_proc circuit-breaker cap. The max_requests: 1024 limit on the ext_proc service constrains gateway scaling. Consensus is leaning toward removing the cap (the upstream circuit breaker already protects ext_proc), but an upgrade-test failure needs an Envoy-side look before it can land cleanly.
• Retry observability. A proposal to record intermediate failed retry attempts got steered toward first using Envoy's existing per-cluster retry and circuit-breaking metrics before adding anything to the GenAI metrics namespace — with an open question about model-labeled retries.
• A first-party MCP adopter. @jappievw (FreshBooks) is building on the gateway and targeting production in about four weeks. Two gaps flagged for issues: non-200 MCP responses (e.g. a 401/403 with WWW-Authenticate) currently collapse into a 500, blocking step-up auth; and the server name/version is exposed and not yet configurable.
• AWS private VPC endpoints. The Bedrock :authority-based host fix (#2296) was reverted because it broke an end-to-end Amazon-signature test that only runs on main with real credentials — it needs reopening with live e2e proof.
• Security. Thanks to Yan Avlasov for the ongoing security work — a new fuzzing harness surfaced a large batch of potential issues now being triaged.

👀 PRs looking for review

If you have review bandwidth, these are open and substantive:

• #2319 — Stream reasoning_content as plain string, signatures via thinking_blocks by @dragon3 — fixes object-shaped reasoning_content breaking OpenAI-compatible clients (#2318)
• #2314 — Router-phase request mutation via a generic external service by @siddharth1036 — the proposal from the June 29 call
• #2325 — Add Cohere rerank to Hugging Face TEI translation by @mohitgurnani1
• #2190 — Forward reasoning_effort as reasoning_config to AWS Bedrock by @Killusions
• #2284 — Add OpenAI to first-party Anthropic translator by @JAORMX
• #2321 — Add topologySpreadConstraints and PodDisruptionBudget to the Helm chart by @andrewrothstein

🙏 Thanks to this window's contributors

@mathetake, @bannu8615, @Aias00, @ChrisJBurns, @kunwarshivam, @gavrissh, @johnugeorge, @xiaolin593, @hustxiayang, @kothari-karan, @mohitgurnani1, @Hritik003, @adhityaravi, @dragon3, @dmitrysDV, @siddharth1036, @Killusions, @JAORMX, @andrewrothstein, @nacx, @jappievw, @aabchoo, @yanavlasov, @PatilHrushikesh, @PraveenKumarInjam — and everyone who showed up to triage, review, and discuss.

See you next week!