Support chaining proxyNetwork through an upstream proxy

[gmarmstrong]

Description

When proxyNetwork is enabled, proxy environment variables passed to an exec are rewritten to point to BuildKit's internal proxy. But since that proxy currently connects directly to the destination, the original proxy settings (HTTP_PROXY, HTTPS_PROXY) are bypassed.

As discussed in #6740 (comment), BuildKit could support proxy chaining:

ExecOp -> BuildKit proxy -> upstream proxy -> destination

The upstream configuration could come from the buildkitd configuration/environment.

[gmarmstrong]

In my own case, this would help reduce boilerplate code in Dockerfiles by allowing proxy routing to just be configured once at the builder level. :)

[TommyLike]

Consolidating from #7031 (now closed) as suggested. Thanks @tonistiigi and @thompson-shaun for the pointers. Sharing our design notes, a working implementation, and a response to the "configure through the client instead" question.

The right layer depends on the deployment model — and both have a place. For a multi-tenant daemon, forwarding the upstream-proxy config from the client is clearly the better fit, and I agree with that direction. But there's a large class of deployments where the daemon is centrally managed (a shared build farm / a fleet of buildkitd pods in K8s), and there the value is the opposite: configure it once at the builder level, with zero awareness required from the people submitting builds. That's the same motivation @gmarmstrong raised here ("configure once at the builder level"), so it might be worth not limiting this to the client only.

This is not only about routing/audit — it's also about caching. Our concrete use case: many buildkitd instances in Kubernetes share a single Squid caching gateway, so packages fetched during RUN (apk/dnf/pip/…) are cached once and reused across all builds and all daemons — transparent build acceleration via cache hits, plus resilience when the origin is slow. Caching is inherently an infrastructure, configure-once concern. Pushing it to every buildctl invocation reintroduces exactly the per-invocation boilerplate this issue set out to remove.

Framing: the built-in exec proxy is already daemon-side infrastructure — it MITMs RUN traffic and injects a generated CA at the daemon today. It is itself an outbound HTTP agent, so letting it chain to an upstream forward proxy is really just making it behave like any proxy-aware HTTP client that honors an upstream proxy — completing proxyNetwork's own proxy semantics rather than adding a foreign concept.

Proposal — support both, not either/or:

• Client-forwarded upstream-proxy config as the flexible / multi-tenant path (your suggestion).
• An optional daemon-level default (buildkitd.toml) for centrally-managed clusters, taking effect only when proxyNetwork is enabled.

Reference implementation (works today, validated end-to-end against a Squid SSL-bump caching proxy, incl. the three-hop CA trust chain): TommyLike/buildkit, branch feature/upstream-proxy-config. It adds a [proxy] section (upstreamURL + optional upstreamCACert) that only applies when proxyNetwork is on. Happy to reshape it toward a client-forwarded (session-based) design if that's the preferred direction — would appreciate your read on whether a hybrid (client-forwarded + optional daemon default) is acceptable before I invest in the rewrite.

[gmarmstrong]

@TommyLike, I'd appreciate your feedback on #6996.

[gmarmstrong]

Fixed by #7074.