Repository navigation
Support Docker (container) deployments beyond development aigw-cli #1899
Description
Activity
Hi! Big user of AIGW in standalone mode here :)
Could you detail what kind of documentation you are missing? The current related docs are:
- The aigw run docs, that detail the different config options, env vars, and how to use MCP, tracing, and others.
- The aigw README contains several examples using Docker compose that show how to use the AIGW with Ollama with several configurations.
- The examples directory contains examples that are, in general, ready to use in standalone mode (I use MCP examples in standalone mode all the time).
I'm trying to understand if it is more of a missing documentation issue, poor docs organization, and info hard to find, etc.
Howdy! Good point and handy links!
I can be more clear. When learning about Envoy's AI Gateway, my first stop is: https://aigateway.envoyproxy.io/ > Get Started
At https://aigateway.envoyproxy.io/docs/getting-started/installation, I would expect to see an option for using docker/compose (or a section on
aigwsimilar to https://aigateway.envoyproxy.io/docs/cli/aigwrun).Then under https://aigateway.envoyproxy.io/docs/getting-started/basic-usage/, I would expect to see some configuration via config file or disclaimers like (I think) you can't use a
config.yamlAND--mcp-configwithaigw run. I haven't played with multipleconfig.yamlfiles, but that would be neat to document (currently working out of a giant mega yaml config).Later for https://aigateway.envoyproxy.io/docs/getting-started/connect-providers, same thing, samples for rolling docker/compose.
In addition to not being a clear, well-lit, path in the central docs - you'll see things in the blog that downplay the role of AIGW like in: https://aigateway.envoyproxy.io/blog/mcp-implementation/
All features are supported in standalone mode, allowing them to start the Envoy AI Gateway locally on their machine with a single command and start leveraging all MCP features. These configurations can be used as-is in production environments, as there is full compatibility between local standalone mode and Kubernetes.
That gives to me: "Tinker with this tool locally, when ready to actually use Envoy's AI Gateway, Kubernetes is here"
The main ask / goal is to make 2 very clear lanes for usage: docker/compose or kubernetes; if the project intends on folks using it in such a way!
Also happy to add to the docs if there's alignment.
Thanks for the detailed feedback.
We definitely want to encourage using it in Docker environments as well. IMO, standalone mode goes a long way in facilitating adoption, and it is a first-class citizen, so it shouldn't appear to be a toy.
It would be awesome if you could contribute your ideas to the docs. As a user who has been confused by them, I think you would be the best to address those concerns the right way to help future newcomers to the project. If you are willing to contribute the changes, I'll be happy to review and merge!Awesome! I ran into a couple issues that caused
aigw runto panic (mainly if you proxy an AnthropicAPI backend and curl an OpenAI call or the reverse). I also had some strange EOF errors integrating non-OpenAI models with an OSS UI. I'll open bugs for those!In any event, I switched AI gateways / rev proxies for now.
Unfortunately that means going deep into other docs 😅 - I'll still look to find time to PR those doc changes unless someone beats me to it!
- added 2 commits that reference this issue
on Mar 14, 2026 - added a commit that references this issue
on Aug 29, 2026 Hi @nacx ! I'd like to pick this up since it looks like it's still open.
My plan for a first PR:- Add a new "Running with Docker" page under the CLI docs covering
docker runwith a mounted config and a Docker Compose setup using the
publishedenvoyproxy/ai-gateway-cliimage (env vars, custom config,
MCP config, health check, version pinning). - Link it from Getting Started so Docker/Compose is a clear path alongside
Kubernetes.
I'd keep larger restructuring of the Getting Started section for a follow-up.
Does that scope sound good?
Also cc @missBerg , I noticed the "docs: add Docker/standalone getting-started
guide" commits from March referencing this issue. Is that still in progress?
Happy to build on it or leave it to you if so. I'd also keep my changes in a
separate page so they don't conflict with #2617.- Add a new "Running with Docker" page under the CLI docs covering
Description:
Is supporting/running AIGW as a standalone container (outside of Kubernetes) on the roadmap?
I've fallen in love with using the AI Gateway in my container-based homelab to track costs, traces, and consolidate LLMs / MCPs. However it feels very - undocumented - to use https://github.com/envoyproxy/ai-gateway/tree/main/cmd/aigw + a
config.yaml.I'd imagine by opening up to more users (supporting compose and kubernetes) would increase adoption / understanding as well as translating to the more robust Kubernetes option when deploying to production.
Proposal:
Or is there a technical limitation to aigw? Feel free to close if not planned.