grafana_alloy: redact URL credentials on the OTLP path too (traces and OTLP logs) #86

What are you trying to do?

Make sure a credential embedded in a URL never leaves the host through any part of the grafana_alloy pipeline.

#85 (fixing #84) adds loki.process "redact_urls". Every journald line now has URL userinfo, path, query and fragment replaced with <redacted> before it ships to Loki. That stage only covers the journal source. The OTLP receiver (otelcol.receiver.otlp "local", loopback 4317/4318) feeds otelcol.processor.transform "decdn_identity" → batch → otelcol.exporter.otlphttp "cloud", and nothing on that path redacts:

• traces: span attributes, span event attributes (e.g. exception.message) and span names;
• logs pushed over OTLP: the body and the attributes;
• metrics: in theory label values, though unlikely.

Today this is a gap in defense in depth, not a confirmed leak:

• decdn-node is the only OTLP sender. It already sanitizes at its tracing sites: upstream security: chain RPC transport errors leak the full rpc_url (path/query API keys) to operator output decdn#954/#958 sends chain-RPC errors through decdn_common::redact, including the OTLP exporter's own error path (crates/node/src/commands/otlp.rs).
• sponsord exports no OTLP.

But any span field one of those sites misses, or any future OTLP sender on the host, would ship a raw RPC URL to Grafana Cloud Tempo/Loki unredacted.

What would help?

Mirror the journald rule on the OTLP path with OTTL replace_pattern / replace_all_patterns, either in a new otelcol.processor.transform "redact_urls" ahead of decdn_identity or as extra statements in it. Use the same two regexes as redact_urls (keyed on ://, keep scheme and host, stop at whitespace, " and \):

• span context: attributes and name; spanevent context: attributes;
• log context: body and attributes.

Keep the rendering safe:

• the regexes go inside OTTL string literals inside an Alloy backtick string, so watch the escaping;
• error_mode = "propagate" already fails loud.

To prove it, extend ansible/tests/alloy-config/validate.sh. Run the rendered processors in the real pinned Alloy and push an OTLP/HTTP JSON span and log record whose attributes and body carry https://u:SECRET@h/v2/SECRET?x=SECRET. Capture the output with an otelcol.exporter.debug (verbosity detailed) swapped in for the cloud exporter. Assert no SECRET gets through and the redacted text matches the journald stage.

This changes the rendered config.alloy on every Grafana-enabled host, so expect one Alloy restart on the next deploy. Note it in the CHANGELOG.

Deploy path

Ansible (and therefore the decdn.node collection and cloud-init, which run the same role).