-
Notifications
You must be signed in to change notification settings - Fork 894
Open
Description
Summary
When OTEL HTTP request tracing is enabled for native workers, Windmill's local TLS-intercepting proxy causes certain upstream servers (e.g. Ubiquiti UniFi controllers running Tomcat) to return HTTP 400 Bad Request — but only when the script runs on Bun. The same script forced to run on Node.js (//nodejs directive) works fine through the same OTEL proxy.
This suggests the issue is specific to how the OTEL proxy handles Bun's fetch requests, rather than the TLS interception itself being incompatible with the upstream server.
Environment
- Windmill version: v1.646.0
- Bun version: 1.3.8 (worker default)
- Infrastructure: Kubernetes, with envoy egress proxy
Steps to reproduce
- Enable OTEL HTTP request tracing in Windmill settings for native workers
- Create a Bun script that makes HTTPS requests to a UniFi controller (
tls: { rejectUnauthorized: false }) - Run the script — requests fail with HTTP 400
- Add
//nodejsdirective to the same script — requests succeed through the same OTEL proxy
Expected behavior
Bun and Node.js scripts should behave consistently when routed through the OTEL tracing proxy.
Actual behavior
- Bun → 400 Bad Request from upstream
- Node.js → 200 OK from upstream
- Bun with OTEL tracing disabled → 200 OK
Workaround
Disable OTEL HTTP request tracing for native workers.
Additional notes
- Windmill overrides the worker's
HTTPS_PROXYto its local OTEL proxy (http://127.0.0.1:<port>) and setsNO_PROXY="", routing all traffic through the MITM proxy - The OTEL proxy presents a "Windmill OTEL Tracing Proxy CA" certificate
- A secondary request: it would be useful to allow
NO_PROXYoverrides or per-script opt-out from OTEL tracing for cases where specific endpoints are incompatible
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels