What I'm asking
I built an Enterprise AI Gateway reference implementation that pulls together several capabilities this repo covers as standalone labs. I'd like to move it into AI-Gateway as a new lab so it lives alongside the existing samples.
The repo is live and working today. You can review the full source, docs, and test suite there.
Why it belongs here
The individual labs are the right place to learn each capability on its own. What's missing is a sample that shows how they fit together in an enterprise setup: three teams sharing one APIM gateway with different token quotas, multi-region failover with circuit breakers, MCP tools governed through the same gateway, and a chargeback dashboard that breaks cost down by team.
This sample composes patterns from backend-pool-load-balancing, token-rate-limiting, token-metrics-emitting, finops-framework, and the MCP labs into one deployment. It doesn't replace any of them.
What the sample includes
- Full-stack Terraform: APIM, Foundry (2 regions), App Insights, Log Analytics, API Center. One
terraform apply.
- Per-team token quotas: Three products (Alpha 50K TPM, Beta 20K TPM, Gamma 500 TPM) with
llm-token-limit enforcement.
- Multi-region failover: Backend pool with priority routing and circuit breakers that read
Retry-After headers.
- Token chargeback:
llm-emit-token-metric with team/model/region dimensions, KQL queries, and an Azure Monitor workbook.
- MCP tool governance: MCP servers registered in APIM with per-session rate limits, correlation IDs, and audit logging.
- API Center discovery: APIM APIs and MCP servers synced to API Center as a searchable catalog.
- Automated test suite: 15 tests (7 LLM gateway, 4 MCP governance, 4 MCP rate limit) runnable outside the notebook.
Gaps it fills
- No existing lab deploys the full stack with Terraform (only
backend-pool-load-balancing-tf exists, and it covers pools only).
- No existing lab combines LLM gateway and MCP governance in a single deployment.
- No existing lab ships a runnable test suite alongside notebook validation.
zero-to-production is the closest match but skips chargeback dashboards and MCP governance.
Proposed structure
Following existing lab conventions: README.MD with frontmatter, Jupyter notebook walkthrough, clean-up-resources.ipynb, Terraform files, APIM policy XMLs, test scripts, and result.png.
Proposed path: labs/enterprise-ai-gateway/
One question
The sample is Terraform only. Happy to add Bicep in a follow-up if that's preferred. Wanted to check before investing the effort.
What I'm asking
I built an Enterprise AI Gateway reference implementation that pulls together several capabilities this repo covers as standalone labs. I'd like to move it into AI-Gateway as a new lab so it lives alongside the existing samples.
The repo is live and working today. You can review the full source, docs, and test suite there.
Why it belongs here
The individual labs are the right place to learn each capability on its own. What's missing is a sample that shows how they fit together in an enterprise setup: three teams sharing one APIM gateway with different token quotas, multi-region failover with circuit breakers, MCP tools governed through the same gateway, and a chargeback dashboard that breaks cost down by team.
This sample composes patterns from
backend-pool-load-balancing,token-rate-limiting,token-metrics-emitting,finops-framework, and the MCP labs into one deployment. It doesn't replace any of them.What the sample includes
terraform apply.llm-token-limitenforcement.Retry-Afterheaders.llm-emit-token-metricwith team/model/region dimensions, KQL queries, and an Azure Monitor workbook.Gaps it fills
backend-pool-load-balancing-tfexists, and it covers pools only).zero-to-productionis the closest match but skips chargeback dashboards and MCP governance.Proposed structure
Following existing lab conventions:
README.MDwith frontmatter, Jupyter notebook walkthrough,clean-up-resources.ipynb, Terraform files, APIM policy XMLs, test scripts, andresult.png.Proposed path:
labs/enterprise-ai-gateway/One question
The sample is Terraform only. Happy to add Bicep in a follow-up if that's preferred. Wanted to check before investing the effort.