From 574e6c993bcd0c3cdedefcd0432f404ec81273cb Mon Sep 17 00:00:00 2001 From: Ant Somers Date: Mon, 5 Oct 2026 13:57:29 +0300 Subject: [PATCH 1/3] fix(grafana_alloy): redact URLs in journald lines before they ship to Loki (#84) The journald pipeline sent every line to Grafana Cloud Loki unchanged. When a daemon error quoted the RPC URL, the provider API key went with it. sponsord does this today: "error sending request for url (...)". A new loki.process "redact_urls" stage now runs first on every unit's line, ahead of the level stages. Two stage.replace stages, matched on "://", replace userinfo, path, query and fragment with and keep the scheme and host. The redacted part ends at whitespace, '"' or '\', so decdn-node's JSON lines stay valid. The stage has no knob. validate.sh now runs the stage in the real pinned Alloy with lines that carry URLs. It checks the exact redacted text, that no SECRET-* marker gets through, that both daemons' levels still parse, and that lines with no URL come out unchanged. It also fails any rendered config whose journal source skips the redactor. Closes #84 Co-Authored-By: Claude Opus 5.5 --- AGENTS.md | 4 +- ansible/galaxy/CHANGELOG.md | 9 +++ ansible/molecule/grafana-cloud/verify.yml | 5 ++ ansible/roles/grafana_alloy/README.md | 6 ++ .../grafana_alloy/templates/config.alloy.j2 | 38 +++++++++- ansible/tests/alloy-config/validate.sh | 76 +++++++++++++++---- 6 files changed, 121 insertions(+), 17 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index 7d1ccb4..87c78af 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -109,7 +109,9 @@ addresses) the repo carries, and they carry their upstream commit. (Alloy's in-process `node_exporter`, curated collector set, `systemd` collector scoped to the units that matter), **journald** → Grafana Cloud Loki, Alloy's own health, and the daemon's OTLP spans. Host metrics and logs carry `job="integrations/node_exporter"` - so Grafana Cloud's prebuilt Linux Server dashboards work unmodified. The API token is the + so Grafana Cloud's prebuilt Linux Server dashboards work unmodified. Every journald line + passes `loki.process "redact_urls"` first (URL userinfo/path/query → ``, no + knob), so an RPC URL a daemon error quotes never ships its API key. The API token is the only credential, and it has two homes: operator-provisioned on the host (`0600 /etc/grafana-alloy.env`) or carried by `grafana_alloy_api_token` from a git-ignored `host_vars//secret.yml`, in which case the role authors that file itself as a diff --git a/ansible/galaxy/CHANGELOG.md b/ansible/galaxy/CHANGELOG.md index 7112cbf..16241e8 100644 --- a/ansible/galaxy/CHANGELOG.md +++ b/ansible/galaxy/CHANGELOG.md @@ -125,6 +125,15 @@ collection adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html) ### Fixed +- `grafana_alloy` no longer ships URL credentials to Grafana Cloud Loki (#84). + The journald pipeline forwarded every line verbatim, so a daemon error that + quoted the RPC URL (sponsord's `error sending request for url (…)`) put its + provider API key in Loki. A new `loki.process "redact_urls"` stage now runs on + every unit's line before anything else. It replaces userinfo, path, query and + fragment with `` and keeps the scheme and host. Lines with no URL, + or with nothing after the host, are unchanged. The rendered `config.alloy` + changes on every host with Grafana Cloud logs on, so the next deploy restarts + Alloy once. - A `release` install no longer fails intermittently at "Remove the download staging directory" with `rmtree failed: [Errno 2] No such file or directory: 'S.gpg-agent.extra'`. Signature verification auto-started a `gpg-agent` diff --git a/ansible/molecule/grafana-cloud/verify.yml b/ansible/molecule/grafana-cloud/verify.yml index 9171e8a..2b563fa 100644 --- a/ansible/molecule/grafana-cloud/verify.yml +++ b/ansible/molecule/grafana-cloud/verify.yml @@ -134,6 +134,11 @@ 'loki.relabel "journal_rules"' in ga_config - >- '__journal__systemd_unit' in ga_config + # URL credentials are redacted before any journal line ships (#84). + - >- + 'loki.process "redact_urls"' in ga_config + - >- + 'forward_to = [loki.process.redact_urls.receiver]' in ga_config # The daemon's level from its JSON body, not journald's priority # (which is info for every stdout line). - >- diff --git a/ansible/roles/grafana_alloy/README.md b/ansible/roles/grafana_alloy/README.md index c44b996..0190f6e 100644 --- a/ansible/roles/grafana_alloy/README.md +++ b/ansible/roles/grafana_alloy/README.md @@ -231,6 +231,12 @@ deliberately lean and every lever is a variable: (`grafana_alloy_logs_drop_priority_regex` / `_drop_unit_regex`, `""` disables either), and `grafana_alloy_logs_max_age` bounds the catch-up burst after an outage — without it a restart can replay days of journal in one go. +- Every journald line, from every unit, has URL credentials redacted before it + leaves the host: userinfo, path, query and fragment become ``, so + `https://u:p@rpc.example.io/v2/KEY?x=y` ships as + `https://@rpc.example.io/`. Scheme and host are kept, so the + provider is still identifiable. An RPC URL usually carries its provider API key, + and a daemon error can quote it. The redaction has no knob. - decdn-node's `level` label comes from its JSON `level` field, not the journald priority (journald reports every stdout line as `info`), and the priority drop regex is applied to that JSON level for this unit. Values follow journald's diff --git a/ansible/roles/grafana_alloy/templates/config.alloy.j2 b/ansible/roles/grafana_alloy/templates/config.alloy.j2 index 42cabe9..4af127e 100644 --- a/ansible/roles/grafana_alloy/templates/config.alloy.j2 +++ b/ansible/roles/grafana_alloy/templates/config.alloy.j2 @@ -6,7 +6,7 @@ // // Pipeline: scrape the node's /metrics and push it to Grafana Cloud Prometheus; // scrape the MACHINE (Alloy's in-process node_exporter) and Alloy itself; ship -// journald to Grafana Cloud Loki; receive OTLP traces (plus any logs/metrics +// journald, with URL credentials redacted, to Grafana Cloud Loki; receive OTLP traces (plus any logs/metrics // pushed over localhost) on gRPC 4317 / HTTP 4318 bound to loopback only; stamp // resource identity on everything, probabilistically sample traces, batch, then // export via OTLP/HTTP. @@ -352,7 +352,41 @@ loki.source.journal "host" { max_age = {{ grafana_alloy_logs_max_age | to_json }} format_as_json = false relabel_rules = loki.relabel.journal_rules.rules - forward_to = [loki.process.daemon_level.receiver] + forward_to = [loki.process.redact_urls.receiver] +} + +{# Credentials embedded in URLs never leave the host. An RPC URL usually carries + its provider API key in the path or query, and a daemon error can quote it + (sponsord's "error sending request for url (...)"; decdn-node strips URLs at + the source, decdn_common::redact). This runs on EVERY unit's line, first, and + has no knob: it is the secret model, not a cost guardrail. + + Keyed on "://", not a scheme list, so any scheme in any case is caught (the + same rule as upstream's redact_bare_urls). The scheme and host survive, so + the provider is still identifiable; userinfo, then everything from the first + / ? or # after the host, becomes . A URL with nothing after its + host, and a line with no URL, pass through unchanged. The redacted tail runs + to whitespace, '"' or '\': it over-consumes punctuation such as a closing ')' + on purpose (')' is legal in a path, and leaking a key is worse than losing a + bracket), but stops at the JSON string delimiters so decdn-node's line stays + valid JSON for the level stage below. stage.replace rewrites every match on + the line. Only URL-shaped text is caught: a Debug-printed url::Url struct + would still leak, which is why the daemons redact at the source as well. + validate.sh runs these stages in the real Alloy against sample lines. #} +loki.process "redact_urls" { + forward_to = [loki.process.daemon_level.receiver] + + // userinfo: greedy, so it runs to the LAST @ of the authority. + stage.replace { + expression = `://([^/?#\s"\\]+)@` + replace = "" + } + + // path, query and fragment. + stage.replace { + expression = `://[^/?#\s"\\]+([/?#][^\s"\\]*)` + replace = "/" + } } {# The daemon's own severity. decdn-node logs JSON to stdout, and journald gives diff --git a/ansible/tests/alloy-config/validate.sh b/ansible/tests/alloy-config/validate.sh index 508c0d1..e82a8c5 100755 --- a/ansible/tests/alloy-config/validate.sh +++ b/ansible/tests/alloy-config/validate.sh @@ -151,8 +151,16 @@ assert_has defaults.alloy 'systemd {' "the per-unit systemd collector" # guardrail anchored with its alternation grouped. Each unit-name needle is one # of the three sites a decdn-node.service rename must touch. assert_has defaults.alloy 'loki.process "daemon_level"' "the daemon JSON-level stage" -assert_has defaults.alloy 'forward_to = [loki.process.daemon_level.receiver]' "the journal source feeding the daemon-level stage" +assert_has defaults.alloy 'forward_to = [loki.process.daemon_level.receiver]' "the URL redactor feeding the daemon-level stage" assert_has defaults.alloy 'forward_to = [loki.relabel.journal_identity.receiver]' "the daemon-level stage feeding the identity relabeller" +# Every configuration that ships the journal redacts it first (#84): the +# redactor exists and the journal source feeds it, not the stage after it. +for config in "${configs[@]}"; do + name="$(basename "$config")" + grep -qF 'loki.source.journal "host"' "$config" || continue + assert_has "$name" 'loki.process "redact_urls"' "the journald URL redactor" + assert_has "$name" 'forward_to = [loki.process.redact_urls.receiver]' "the journal source feeding the URL redactor" +done # shellcheck disable=SC2016 # literal backticks: Alloy's raw-string syntax assert_has defaults.alloy 'selector = `{unit="decdn-node.service"}`' "the daemon unit scoping of the level stage" assert_has defaults.alloy 'target_label = "__priority_guardrail"' "the daemon's exemption from the journald priority drop" @@ -254,9 +262,9 @@ assert_has colocated.alloy 'prometheus.scrape "sponsord"' "the sponsord scrape o assert_has colocated.alloy '"__address__" = "127.0.0.1:8090"' "the default sponsord listener" echo "ok: per-daemon scrape toggles" -# --- Gate 1c: the daemon's level comes from its JSON body ---------------------- -# Loading proves syntax, not behaviour. Run the RENDERED journal_rules and -# loki.process "daemon_level" blocks in the real Alloy, fed sample lines through +# --- Gate 1c: URLs are redacted, and the daemon's level comes from its line ---- +# Loading proves syntax, not behaviour. Run the RENDERED journal_rules, +# redact_urls and daemon_level blocks in the real Alloy, fed sample lines through # loki.source.api, and read what loki.echo prints. loki.source.api strips # __-prefixed labels on push, so the lifted rules read journal_* instead of # __journal_* (the only edit), and a final labeldrop removes those raw labels @@ -285,11 +293,14 @@ samples_json() { # nanosecond timestamp ["$1","2026-10-05T07:59:10.000005Z ERROR sponsord: s-error"], ["$1","Error: s-raw read PaymentPool.usdc()"], ["$1","Caused by: s-cause connection refused"], - ["$1","s-plain an unlevelled line"]]}, + ["$1","s-plain an unlevelled line"], + ["$1","Error: s-rpcurl read PaymentPool.usdc(): error sending request for url (https://arb-sepolia.example-provider.io/v2/SECRET-KEY?x=SECRET-Q)"]]}, {"stream":{"journal__systemd_unit":"other.service","journal_priority_keyword":"info"},"values":[ ["$1","{\"level\":\"ERROR\",\"fields\":{\"message\":\"m-other-info\"}}"]]}, {"stream":{"journal__systemd_unit":"other.service","journal_priority_keyword":"error"},"values":[ - ["$1","m-other-error"]]}, + ["$1","m-other-error"], + ["$1","o-twourls wss://SECRET-USER@a.example/SECRET-A and HTTP://b.example?SECRET-B=1 end"], + ["$1","o-nopath https://c.example:443 done"]]}, {"stream":{"journal__systemd_unit":"decdn-node.service","journal_priority_keyword":"info"},"values":[ ["$1","{\"level\":\"TRACE\",\"fields\":{\"message\":\"m-trace\"}}"], ["$1","{\"level\":\" DEBUG\",\"fields\":{\"message\":\"m-debug\"}}"], @@ -299,7 +310,8 @@ samples_json() { # nanosecond timestamp ["$1","{\"level\":\"ERROR\",\"fields\":{\"message\":\"m-error\"}}"], ["$1","{\"fields\":{\"message\":\"m-nolevel\"}}"], ["$1","{\"level\":5,\"fields\":{\"message\":\"m-weird\"}}"], - ["$1","m-text is not json"]]} + ["$1","m-text is not json"], + ["$1","{\"level\":\"WARN\",\"fields\":{\"message\":\"m-rpcurl via https://u:SECRET-PASS@rpc.example.io:8545/v3/SECRET-KEY#SECRET-FRAG failed\"}}"]]} ]} JSON } @@ -323,7 +335,7 @@ loki.source.api "in" { listen_port = $api_port } relabel_rules = loki.relabel.journal_rules.rules - forward_to = [loki.process.daemon_level.receiver] + forward_to = [loki.process.redact_urls.receiver] } loki.echo "out" { } @@ -332,11 +344,14 @@ ALLOY awk '/^loki\.relabel "journal_rules" \{/ { on = 1 } on && /^\}/ { print "\n rule {\n regex = \"journal_.*\"\n action = \"labeldrop\"\n }"; print; exit } on { print }' "$src" | sed 's/"__journal_/"journal_/g' + awk '/^loki\.process "redact_urls" \{/ { on = 1 } on { print } on && /^\}/ { exit }' "$src" awk '/^loki\.process "daemon_level" \{/ { on = 1 } on { print } on && /^\}/ { exit }' "$src" \ | sed 's|forward_to = \[loki\.relabel\.journal_identity\.receiver\]|forward_to = [loki.echo.out.receiver]|' } >"$config" - if ! grep -qF 'action = "labeldrop"' "$config" || ! grep -qF 'forward_to = [loki.echo.out.receiver]' "$config"; then - fail "could not lift journal_rules and daemon_level out of $1" + if ! grep -qF 'action = "labeldrop"' "$config" \ + || ! grep -qF 'loki.process "redact_urls"' "$config" \ + || ! grep -qF 'forward_to = [loki.echo.out.receiver]' "$config"; then + fail "could not lift journal_rules, redact_urls and daemon_level out of $1" fi "$alloy" run "$config" --storage.path="$data" \ @@ -382,10 +397,21 @@ kept() { # marker, level, unit dropped() { # marker ! grep -qF "$1" "$harness_log" || harness_fail "$1 should have been dropped" } +# URL redaction (#84). Every credential in the samples is spelled SECRET-*, so +# no_secrets is the blanket check; redacted pins the exact shape that survives. +redacted() { # marker, the redacted text its line must carry + grep -F "$1" "$harness_log" | grep -qF "$2" || harness_fail "$1 should carry \"$2\"" +} +unchanged() { # the whole line, which must come out byte-for-byte + grep -qF "entry=\"$1\" " "$harness_log" || harness_fail "\"$1\" should pass through unchanged" +} +no_secrets() { + ! grep -qF 'SECRET' "$harness_log" || harness_fail "a URL credential (SECRET-*) left the pipeline" +} # Defaults (guardrail "debug|trace"), sponsord not enabled: its plain-text lines # are not re-levelled and stand at journald's info. -run_level_harness defaults.alloy 17 +run_level_harness defaults.alloy 21 kept s-trace info sponsord.service kept s-error info sponsord.service kept m-warn-uc warning decdn-node.service @@ -402,11 +428,28 @@ dropped m-debug # padded + lower-cased before the drop dropped m-other-debug # the journald-priority drop, other units echo "ok: daemon log level follows the JSON body (defaults)" +# URL redaction, on every unit, ahead of the level stages. +no_secrets +# scheme and host survive; path and query go, and so does reqwest's closing ')' +redacted s-rpcurl 'error sending request for url (https://arb-sepolia.example-provider.io/"' +kept s-rpcurl info sponsord.service # sponsord not enabled: journald's level +# userinfo, path and fragment go; the JSON stays valid, so the level still parses +redacted m-rpcurl 'via https://@rpc.example.io:8545/ failed' +kept m-rpcurl warning decdn-node.service +# every URL on the line, any scheme in any case, a query with no path +redacted o-twourls 'o-twourls wss://@a.example/ and HTTP://b.example/ end' +unchanged 'o-nopath https://c.example:443 done' # nothing after the host: untouched +unchanged 'm-text is not json' # no URL: untouched +echo "ok: URL credentials are redacted before the level stages (defaults)" + # Guardrail "info|debug|trace": the daemon's every line is journald-info, so it # must be judged by its JSON level — errors and warnings survive, and lines with # no JSON level are never dropped for a priority they did not choose. -run_level_harness priorityinfo.alloy 7 +run_level_harness priorityinfo.alloy 10 +no_secrets dropped s-error # sponsord not enabled: not exempt either +dropped s-rpcurl +kept m-rpcurl warning decdn-node.service # redacted JSON is still judged by its level kept m-warn-uc warning decdn-node.service kept m-warn-lc warning decdn-node.service kept m-error error decdn-node.service @@ -422,7 +465,9 @@ dropped m-other-debug echo "ok: daemon log level follows the JSON body (info in the guardrail)" # sponsord enabled: its level is parsed from the plain-text line. -run_level_harness sponsordonly.alloy 15 +run_level_harness sponsordonly.alloy 19 +no_secrets +kept s-rpcurl error sponsord.service # the "Error: " match still sees the redacted line kept s-info info sponsord.service kept s-warn warning sponsord.service kept s-error error sponsord.service @@ -436,7 +481,10 @@ echo "ok: sponsord log level follows its text line (defaults)" # Co-located, guardrail "info|debug|trace": both daemons exempt from the # journald-priority drop and judged by the level they logged. -run_level_harness colocated.alloy 12 +run_level_harness colocated.alloy 16 +no_secrets +kept s-rpcurl error sponsord.service +kept m-rpcurl warning decdn-node.service kept s-warn warning sponsord.service kept s-error error sponsord.service kept s-raw error sponsord.service From b4ed128c0efa2834be23be1dd3a95d83ba92a572 Mon Sep 17 00:00:00 2001 From: Ant Somers Date: Mon, 5 Oct 2026 14:10:59 +0300 Subject: [PATCH 2/3] fix(grafana_alloy): redact URLs that JSON escaping hides from the redactor Both stages looked for a literal "://", and the redacted part stopped at any backslash. So JSON written by other units could still ship credentials. PHP's json_encode writes https:\/\/host\/..., which matched neither stage, so the whole URL went through. Go's encoding/json escapes '&' as &, which stopped the redacted part mid-query and let the rest of the query through. The separators may now be '\/', and the redacted part runs across every JSON escape except \" and \\. Those still end it, so the line stays valid JSON. validate.sh adds a sample for each case (escaped slashes, &, and a URL quoted inside a JSON string). Each sample fails on its own when its half of the fix is removed. Co-Authored-By: Claude Opus 5.5 --- ansible/galaxy/CHANGELOG.md | 3 +- ansible/roles/grafana_alloy/README.md | 3 +- .../grafana_alloy/templates/config.alloy.j2 | 31 +++++++++++-------- ansible/tests/alloy-config/validate.sh | 19 +++++++++--- 4 files changed, 36 insertions(+), 20 deletions(-) diff --git a/ansible/galaxy/CHANGELOG.md b/ansible/galaxy/CHANGELOG.md index 16241e8..60ecfdd 100644 --- a/ansible/galaxy/CHANGELOG.md +++ b/ansible/galaxy/CHANGELOG.md @@ -130,7 +130,8 @@ collection adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html) quoted the RPC URL (sponsord's `error sending request for url (…)`) put its provider API key in Loki. A new `loki.process "redact_urls"` stage now runs on every unit's line before anything else. It replaces userinfo, path, query and - fragment with `` and keeps the scheme and host. Lines with no URL, + fragment with `` and keeps the scheme and host, including in JSON + that escapes the URL (`https:\/\/…`, `\u0026`). Lines with no URL, or with nothing after the host, are unchanged. The rendered `config.alloy` changes on every host with Grafana Cloud logs on, so the next deploy restarts Alloy once. diff --git a/ansible/roles/grafana_alloy/README.md b/ansible/roles/grafana_alloy/README.md index 0190f6e..273b66c 100644 --- a/ansible/roles/grafana_alloy/README.md +++ b/ansible/roles/grafana_alloy/README.md @@ -235,7 +235,8 @@ deliberately lean and every lever is a variable: leaves the host: userinfo, path, query and fragment become ``, so `https://u:p@rpc.example.io/v2/KEY?x=y` ships as `https://@rpc.example.io/`. Scheme and host are kept, so the - provider is still identifiable. An RPC URL usually carries its provider API key, + provider is still identifiable. URLs inside JSON that escapes them (`\/`, + `\u0026`) are caught too. An RPC URL usually carries its provider API key, and a daemon error can quote it. The redaction has no knob. - decdn-node's `level` label comes from its JSON `level` field, not the journald priority (journald reports every stdout line as `info`), and the priority drop diff --git a/ansible/roles/grafana_alloy/templates/config.alloy.j2 b/ansible/roles/grafana_alloy/templates/config.alloy.j2 index 4af127e..48a1970 100644 --- a/ansible/roles/grafana_alloy/templates/config.alloy.j2 +++ b/ansible/roles/grafana_alloy/templates/config.alloy.j2 @@ -361,30 +361,35 @@ loki.source.journal "host" { the source, decdn_common::redact). This runs on EVERY unit's line, first, and has no knob: it is the secret model, not a cost guardrail. - Keyed on "://", not a scheme list, so any scheme in any case is caught (the - same rule as upstream's redact_bare_urls). The scheme and host survive, so - the provider is still identifiable; userinfo, then everything from the first - / ? or # after the host, becomes . A URL with nothing after its - host, and a line with no URL, pass through unchanged. The redacted tail runs - to whitespace, '"' or '\': it over-consumes punctuation such as a closing ')' - on purpose (')' is legal in a path, and leaking a key is worse than losing a - bracket), but stops at the JSON string delimiters so decdn-node's line stays - valid JSON for the level stage below. stage.replace rewrites every match on - the line. Only URL-shaped text is caught: a Debug-printed url::Url struct - would still leak, which is why the daemons redact at the source as well. + Keyed on "://" (or ":\/\/"), not a scheme list, so any scheme in any case + is caught (the same rule as upstream's redact_bare_urls). The scheme and + host survive, so the provider is still identifiable; userinfo, then + everything from the first / ? or # after the host, becomes . A + URL with nothing after its host, and a line with no URL, pass through + unchanged. The redacted tail runs to whitespace, '"', '\"' or '\\': it + over-consumes punctuation such as a closing ')' on purpose (')' is legal in + a path, and leaking a key is worse than losing a bracket), but stops at the + JSON string delimiters so decdn-node's line stays valid JSON for the level + stage below. Other units may log JSON that escapes URL characters, so both + stages read through it: a separator may be '\/' (PHP's json_encode writes + https:\/\/host\/...), and the tail runs across every other escape, such as + Go encoding/json's \u0026 for '&', which would otherwise end it mid-query. + stage.replace rewrites every match on the line. Only URL-shaped text is + caught: a Debug-printed url::Url struct would still leak, which is why the + daemons redact at the source as well. validate.sh runs these stages in the real Alloy against sample lines. #} loki.process "redact_urls" { forward_to = [loki.process.daemon_level.receiver] // userinfo: greedy, so it runs to the LAST @ of the authority. stage.replace { - expression = `://([^/?#\s"\\]+)@` + expression = `:\\?/\\?/([^/?#\s"\\]+)@` replace = "" } // path, query and fragment. stage.replace { - expression = `://[^/?#\s"\\]+([/?#][^\s"\\]*)` + expression = `:\\?/\\?/[^/?#\s"\\]+((?:\\?/|[?#])(?:\\[^\s"\\]|[^\s"\\])*)` replace = "/" } } diff --git a/ansible/tests/alloy-config/validate.sh b/ansible/tests/alloy-config/validate.sh index e82a8c5..e91e7ce 100755 --- a/ansible/tests/alloy-config/validate.sh +++ b/ansible/tests/alloy-config/validate.sh @@ -300,7 +300,10 @@ samples_json() { # nanosecond timestamp {"stream":{"journal__systemd_unit":"other.service","journal_priority_keyword":"error"},"values":[ ["$1","m-other-error"], ["$1","o-twourls wss://SECRET-USER@a.example/SECRET-A and HTTP://b.example?SECRET-B=1 end"], - ["$1","o-nopath https://c.example:443 done"]]}, + ["$1","o-nopath https://c.example:443 done"], + ["$1","o-escslash {\"url\":\"https:\\\\/\\\\/u:SECRET-ESCPASS@esc.example\\\\/v2\\\\/SECRET-ESCKEY?x=SECRET-ESCQ\"}"], + ["$1","o-escamp {\"url\":\"https://amp.example/v2/k?a=1\\\\u0026b=SECRET-AMP\"}"], + ["$1","o-escquote {\"m\":\"say \\\\\"https://q.example/v2/SECRET-QK\\\\\" ok\"}"]]}, {"stream":{"journal__systemd_unit":"decdn-node.service","journal_priority_keyword":"info"},"values":[ ["$1","{\"level\":\"TRACE\",\"fields\":{\"message\":\"m-trace\"}}"], ["$1","{\"level\":\" DEBUG\",\"fields\":{\"message\":\"m-debug\"}}"], @@ -411,7 +414,7 @@ no_secrets() { # Defaults (guardrail "debug|trace"), sponsord not enabled: its plain-text lines # are not re-levelled and stand at journald's info. -run_level_harness defaults.alloy 21 +run_level_harness defaults.alloy 24 kept s-trace info sponsord.service kept s-error info sponsord.service kept m-warn-uc warning decdn-node.service @@ -438,6 +441,12 @@ redacted m-rpcurl 'via https://@rpc.example.io:8545/ failed' kept m-rpcurl warning decdn-node.service # every URL on the line, any scheme in any case, a query with no path redacted o-twourls 'o-twourls wss://@a.example/ and HTTP://b.example/ end' +# JSON another unit may write (needles are as loki.echo prints them, logfmt- +# escaped): PHP's \/ separators, Go's \u0026 for '&' mid-query, and a URL +# quoted inside a JSON string, whose \" must survive so the JSON stays valid +redacted o-escslash 'https:\\/\\/@esc.example/\"}"' +redacted o-escamp 'https://amp.example/\"}"' +redacted o-escquote 'say \\\"https://q.example/\\\" ok\"}"' unchanged 'o-nopath https://c.example:443 done' # nothing after the host: untouched unchanged 'm-text is not json' # no URL: untouched echo "ok: URL credentials are redacted before the level stages (defaults)" @@ -445,7 +454,7 @@ echo "ok: URL credentials are redacted before the level stages (defaults)" # Guardrail "info|debug|trace": the daemon's every line is journald-info, so it # must be judged by its JSON level — errors and warnings survive, and lines with # no JSON level are never dropped for a priority they did not choose. -run_level_harness priorityinfo.alloy 10 +run_level_harness priorityinfo.alloy 13 no_secrets dropped s-error # sponsord not enabled: not exempt either dropped s-rpcurl @@ -465,7 +474,7 @@ dropped m-other-debug echo "ok: daemon log level follows the JSON body (info in the guardrail)" # sponsord enabled: its level is parsed from the plain-text line. -run_level_harness sponsordonly.alloy 19 +run_level_harness sponsordonly.alloy 22 no_secrets kept s-rpcurl error sponsord.service # the "Error: " match still sees the redacted line kept s-info info sponsord.service @@ -481,7 +490,7 @@ echo "ok: sponsord log level follows its text line (defaults)" # Co-located, guardrail "info|debug|trace": both daemons exempt from the # journald-priority drop and judged by the level they logged. -run_level_harness colocated.alloy 16 +run_level_harness colocated.alloy 19 no_secrets kept s-rpcurl error sponsord.service kept m-rpcurl warning decdn-node.service From 1ca11cbf911e2780be038c5d3a5902bf15bac8d1 Mon Sep 17 00:00:00 2001 From: Ant Somers Date: Mon, 5 Oct 2026 14:34:10 +0300 Subject: [PATCH 3/3] fix(grafana_alloy): close the redactor's userinfo and nested-JSON leaks PR review found three shapes that still shipped credentials to Loki. All three were reproduced in the real Alloy 1.19.2: - A password with an unencoded / ? or # stopped the userinfo stage, so the username and the start of the password shipped: postgres://u:pa#ss@db/app came out as postgres://u:pa/. - A JSON escape in the userinfo (Go's &) stopped both stages, and the whole URL shipped, path key included. - JSON nested in a JSON string (\\u0026) cut the query short, so the rest of it shipped. The userinfo stage now runs to the last @ of the token, and every part of the match reads JSON escapes as whole pairs; only \" ends the match. The trade-off: a URL with an @ in its path loses its host (https://@x/). Losing information beats leaking a key. validate.sh: - adds samples for each leak, plus the existing shapes nothing tested yet (@ in a password, a fragment straight after the host, IPv6 with userinfo, userinfo with no path, a line with no URL but @ # ? :); - puts the escaped and nested samples in decdn-node's stream, so its level label proves the redacted JSON still parses; - now fails any config where a loki.source.* bypasses the redactor or a later hop has a second feeder; - renames the unchanged helper to entry_is, since it also checks redacted output. Docs: - Restore the header comment to its main text, so config.alloy changes only on hosts with logs on, as the CHANGELOG says. - Move the README bullet into its own section, out of the cost guardrails. - State the limits instead of overclaiming: sponsord does not redact at the source; / escapes, percent-encoded URLs, deeper nesting, journald line splits and OTLP (#86) are not covered. Co-Authored-By: Claude Opus 5.5 --- AGENTS.md | 5 +- ansible/galaxy/CHANGELOG.md | 18 +++-- ansible/roles/grafana_alloy/README.md | 33 +++++++-- .../grafana_alloy/templates/config.alloy.j2 | 73 +++++++++++-------- ansible/tests/alloy-config/validate.sh | 64 +++++++++++++--- 5 files changed, 134 insertions(+), 59 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index 87c78af..3c19e15 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -110,8 +110,9 @@ addresses) the repo carries, and they carry their upstream commit. to the units that matter), **journald** → Grafana Cloud Loki, Alloy's own health, and the daemon's OTLP spans. Host metrics and logs carry `job="integrations/node_exporter"` so Grafana Cloud's prebuilt Linux Server dashboards work unmodified. Every journald line - passes `loki.process "redact_urls"` first (URL userinfo/path/query → ``, no - knob), so an RPC URL a daemon error quotes never ships its API key. The API token is the + passes `loki.process "redact_urls"` ahead of the level stages (URL userinfo, path, query + and fragment → ``, scheme and host kept, no knob), so an RPC URL quoted in a + daemon error ships without its API key; OTLP data is not redacted (#86). The API token is the only credential, and it has two homes: operator-provisioned on the host (`0600 /etc/grafana-alloy.env`) or carried by `grafana_alloy_api_token` from a git-ignored `host_vars//secret.yml`, in which case the role authors that file itself as a diff --git a/ansible/galaxy/CHANGELOG.md b/ansible/galaxy/CHANGELOG.md index 60ecfdd..cc80314 100644 --- a/ansible/galaxy/CHANGELOG.md +++ b/ansible/galaxy/CHANGELOG.md @@ -125,16 +125,18 @@ collection adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html) ### Fixed -- `grafana_alloy` no longer ships URL credentials to Grafana Cloud Loki (#84). - The journald pipeline forwarded every line verbatim, so a daemon error that +- `grafana_alloy` no longer ships URL credentials in journald lines to Grafana + Cloud Loki (#84). Every line was forwarded verbatim, so a daemon error that quoted the RPC URL (sponsord's `error sending request for url (…)`) put its provider API key in Loki. A new `loki.process "redact_urls"` stage now runs on - every unit's line before anything else. It replaces userinfo, path, query and - fragment with `` and keeps the scheme and host, including in JSON - that escapes the URL (`https:\/\/…`, `\u0026`). Lines with no URL, - or with nothing after the host, are unchanged. The rendered `config.alloy` - changes on every host with Grafana Cloud logs on, so the next deploy restarts - Alloy once. + every unit's line ahead of the level stages. It replaces the userinfo with + `` and everything after the host with `/`, keeping the + scheme and host. That holds for passwords with unencoded `/ ? # @`, and for + URLs inside JSON that escapes them (`https:\/\/…`, `\u0026`, one level of + nesting). A URL with an `@` in its path loses its host. A line with no URL, or + with a URL that has no userinfo and nothing after its host, is unchanged. OTLP + logs and spans are not redacted (#86). The rendered `config.alloy` changes on + every host with Grafana Cloud logs on, so the next deploy restarts Alloy once. - A `release` install no longer fails intermittently at "Remove the download staging directory" with `rmtree failed: [Errno 2] No such file or directory: 'S.gpg-agent.extra'`. Signature verification auto-started a `gpg-agent` diff --git a/ansible/roles/grafana_alloy/README.md b/ansible/roles/grafana_alloy/README.md index 273b66c..f8d02cc 100644 --- a/ansible/roles/grafana_alloy/README.md +++ b/ansible/roles/grafana_alloy/README.md @@ -206,6 +206,32 @@ Changing `grafana_alloy_host_job` / `grafana_alloy_logs_job` away from `integrations/node_exporter` silently un-wires the prebuilt dashboards; that is why they are knobs with a warning rather than hard-coded literals. +## What is redacted before logs leave the host + +An RPC URL usually carries its provider API key, and a daemon error can quote +it. So every journald line, from every unit, passes `loki.process "redact_urls"` +before anything ships. It has no knob. + +- The userinfo becomes ``, and everything from the first `/`, `?` or + `#` after the host becomes `/`. The scheme and host are kept, so the + provider is still identifiable: + `https://u:p@rpc.example.io/v2/KEY?x=y` ships as + `https://@rpc.example.io/`. +- A password with unencoded `/ ? # @` is covered. The price is that a URL with an + `@` in its path loses its host: `https://h/users/@x/y` ships as + `https://@x/`. +- URLs inside JSON that escapes them are covered: `https:\/\/…`, `\u0026` and one + level of JSON nested in a JSON string. The JSON stays valid. +- A line with no URL, or with a URL that has no userinfo and nothing after its + host, is unchanged. + +Not covered: `\u002f`-escaped separators, fully percent-encoded URLs, deeper JSON +nesting, a URL journald splits across two records (lines over `LineMax`, 48K by +default), and OTLP logs and spans ([#86](https://github.com/decdn/devops/issues/86)). +decdn-node also strips URLs from its own errors; sponsord does not +([decdn/sponsord#38](https://github.com/decdn/sponsord/issues/38)), so for its +lines this stage is the only guard. + ## Machine monitoring: what it costs and how to bound it Grafana Cloud bills on active series and log volume, so the defaults are @@ -231,13 +257,6 @@ deliberately lean and every lever is a variable: (`grafana_alloy_logs_drop_priority_regex` / `_drop_unit_regex`, `""` disables either), and `grafana_alloy_logs_max_age` bounds the catch-up burst after an outage — without it a restart can replay days of journal in one go. -- Every journald line, from every unit, has URL credentials redacted before it - leaves the host: userinfo, path, query and fragment become ``, so - `https://u:p@rpc.example.io/v2/KEY?x=y` ships as - `https://@rpc.example.io/`. Scheme and host are kept, so the - provider is still identifiable. URLs inside JSON that escapes them (`\/`, - `\u0026`) are caught too. An RPC URL usually carries its provider API key, - and a daemon error can quote it. The redaction has no knob. - decdn-node's `level` label comes from its JSON `level` field, not the journald priority (journald reports every stdout line as `info`), and the priority drop regex is applied to that JSON level for this unit. Values follow journald's diff --git a/ansible/roles/grafana_alloy/templates/config.alloy.j2 b/ansible/roles/grafana_alloy/templates/config.alloy.j2 index 48a1970..fc28c52 100644 --- a/ansible/roles/grafana_alloy/templates/config.alloy.j2 +++ b/ansible/roles/grafana_alloy/templates/config.alloy.j2 @@ -6,7 +6,7 @@ // // Pipeline: scrape the node's /metrics and push it to Grafana Cloud Prometheus; // scrape the MACHINE (Alloy's in-process node_exporter) and Alloy itself; ship -// journald, with URL credentials redacted, to Grafana Cloud Loki; receive OTLP traces (plus any logs/metrics +// journald to Grafana Cloud Loki; receive OTLP traces (plus any logs/metrics // pushed over localhost) on gRPC 4317 / HTTP 4318 bound to loopback only; stamp // resource identity on everything, probabilistically sample traces, batch, then // export via OTLP/HTTP. @@ -355,41 +355,51 @@ loki.source.journal "host" { forward_to = [loki.process.redact_urls.receiver] } -{# Credentials embedded in URLs never leave the host. An RPC URL usually carries - its provider API key in the path or query, and a daemon error can quote it - (sponsord's "error sending request for url (...)"; decdn-node strips URLs at - the source, decdn_common::redact). This runs on EVERY unit's line, first, and - has no knob: it is the secret model, not a cost guardrail. - - Keyed on "://" (or ":\/\/"), not a scheme list, so any scheme in any case - is caught (the same rule as upstream's redact_bare_urls). The scheme and - host survive, so the provider is still identifiable; userinfo, then - everything from the first / ? or # after the host, becomes . A - URL with nothing after its host, and a line with no URL, pass through - unchanged. The redacted tail runs to whitespace, '"', '\"' or '\\': it - over-consumes punctuation such as a closing ')' on purpose (')' is legal in - a path, and leaking a key is worse than losing a bracket), but stops at the - JSON string delimiters so decdn-node's line stays valid JSON for the level - stage below. Other units may log JSON that escapes URL characters, so both - stages read through it: a separator may be '\/' (PHP's json_encode writes - https:\/\/host\/...), and the tail runs across every other escape, such as - Go encoding/json's \u0026 for '&', which would otherwise end it mid-query. - stage.replace rewrites every match on the line. Only URL-shaped text is - caught: a Debug-printed url::Url struct would still leak, which is why the - daemons redact at the source as well. - validate.sh runs these stages in the real Alloy against sample lines. #} +{# Credentials embedded in URLs do not leave the host in journald lines. An RPC + URL usually carries its provider API key, and a daemon error can quote it: + sponsord's "error sending request for url (...)" does. decdn-node also strips + URLs at the source (decdn_common::redact); sponsord does not, so for sponsord + this stage is the only guard. It runs on EVERY unit's line, ahead of the level + stages, and has no knob: it is the secret model, not a cost guardrail. + + Both stages key on "://" (or JSON's ":\/\/"), not a scheme list, so any + scheme in any case is caught, as in upstream's redact_bare_urls. Unlike that + function, the scheme and host are kept, so the provider stays identifiable: + 1. userinfo -> . It runs to the LAST @ of the whitespace-free + token, so a password holding an unencoded / ? # or @ is covered too. + The cost: a URL with an @ in its path loses its host + (https://h/users/@x/y ships as https://@x/). + 2. everything from the first / ? # (or \/) after the host -> /, + whichever separator it was. + A URL with no userinfo and nothing after its host, and a line with no URL, + pass through unchanged. + + The redacted text ends at whitespace, '"' or '\"' and over-consumes anything + else, such as a closing ')' (legal in a path; losing a bracket beats leaking + a key). JSON escapes are read as whole pairs (\u0026, \/, \\), so Go's + encoding/json, PHP's json_encode and one level of JSON nested in a JSON + string are all covered, and the line stays valid JSON for the level stage + below. stage.replace rewrites every match on the line. + + Not covered: a \u002f-escaped separator, a fully percent-encoded URL, deeper + JSON nesting, a URL journald split across two records (lines over LineMax, + 48K by default) and a Debug-printed url::Url struct. OTLP data does not pass + through here (#86). A line no expression matches ships as it was, so keep + `replace` a literal: a template that fails to render also passes the line + through. validate.sh runs these stages in the real Alloy against sample + lines. #} loki.process "redact_urls" { forward_to = [loki.process.daemon_level.receiver] - // userinfo: greedy, so it runs to the LAST @ of the authority. + // 1. userinfo (as upstream's redact_userinfo, which writes ***@ instead). stage.replace { - expression = `:\\?/\\?/([^/?#\s"\\]+)@` + expression = `:\\?/\\?/((?:\\[^\s"]|[^\s"\\])+)@` replace = "" } - // path, query and fragment. + // 2. path, query and fragment; the separator is normalised to /. stage.replace { - expression = `:\\?/\\?/[^/?#\s"\\]+((?:\\?/|[?#])(?:\\[^\s"\\]|[^\s"\\])*)` + expression = `:\\?/\\?/(?:\\[^\s"\\/]|[^/?#\s"\\])+((?:\\?/|[?#])(?:\\[^\s"]|[^\s"\\])*)` replace = "/" } } @@ -399,9 +409,10 @@ loki.process "redact_urls" { them level="info" — WARN and ERROR included. The real level is the `level` field of the tracing-subscriber JSON body, e.g. {"level":"WARN",...}. - This runs after loki.source.journal, where the __journal* fields are already - gone, so it keys off the `unit` label journal_rules produced. The unit name - is the one decdn_node installs; it is also hard-coded in the journal_rules + This runs after loki.source.journal (via redact_urls, which rewrites only + the line), where the __journal* fields are already gone, so it keys off the + `unit` label journal_rules produced. The unit name is the one decdn_node + installs; it is also hard-coded in the journal_rules priority exemption above and the service_name rule below — a rename must be mirrored in all three (validate.sh pins them). diff --git a/ansible/tests/alloy-config/validate.sh b/ansible/tests/alloy-config/validate.sh index e91e7ce..af8dee9 100755 --- a/ansible/tests/alloy-config/validate.sh +++ b/ansible/tests/alloy-config/validate.sh @@ -155,12 +155,25 @@ assert_has defaults.alloy 'forward_to = [loki.process.daemon_level.receiver]' "t assert_has defaults.alloy 'forward_to = [loki.relabel.journal_identity.receiver]' "the daemon-level stage feeding the identity relabeller" # Every configuration that ships the journal redacts it first (#84): the # redactor exists and the journal source feeds it, not the stage after it. +journal_configs=0 for config in "${configs[@]}"; do name="$(basename "$config")" grep -qF 'loki.source.journal "host"' "$config" || continue assert_has "$name" 'loki.process "redact_urls"' "the journald URL redactor" assert_has "$name" 'forward_to = [loki.process.redact_urls.receiver]' "the journal source feeding the URL redactor" + # ...and nothing reaches the stages after it another way: every log source + # feeds the redactor. + sources="$(grep -c '^loki\.source\.' "$config")" + [ "$(grep -c 'loki\.process\.redact_urls\.receiver' "$config")" -eq "$sources" ] \ + || fail "$name: a loki.source.* component does not feed the URL redactor" + # Each later hop has exactly one feeder, so nothing can join the chain past it. + for receiver in loki.process.daemon_level loki.relabel.journal_identity loki.write.cloud; do + [ "$(grep -cF "$receiver.receiver" "$config")" -eq 1 ] \ + || fail "$name: $receiver has a feeder other than its predecessor, which bypasses the URL redactor" + done + journal_configs=$((journal_configs + 1)) done +[ "$journal_configs" -gt 0 ] || fail "no rendered configuration ships the journal; the redactor check examined nothing" # shellcheck disable=SC2016 # literal backticks: Alloy's raw-string syntax assert_has defaults.alloy 'selector = `{unit="decdn-node.service"}`' "the daemon unit scoping of the level stage" assert_has defaults.alloy 'target_label = "__priority_guardrail"' "the daemon's exemption from the journald priority drop" @@ -303,7 +316,15 @@ samples_json() { # nanosecond timestamp ["$1","o-nopath https://c.example:443 done"], ["$1","o-escslash {\"url\":\"https:\\\\/\\\\/u:SECRET-ESCPASS@esc.example\\\\/v2\\\\/SECRET-ESCKEY?x=SECRET-ESCQ\"}"], ["$1","o-escamp {\"url\":\"https://amp.example/v2/k?a=1\\\\u0026b=SECRET-AMP\"}"], - ["$1","o-escquote {\"m\":\"say \\\\\"https://q.example/v2/SECRET-QK\\\\\" ok\"}"]]}, + ["$1","o-escquote {\"m\":\"say \\\\\"https://q.example/v2/SECRET-QK\\\\\" ok\"}"], + ["$1","o-hashpw postgres://SECRET-HU:SECRET-pa#ss@db.example/app y"], + ["$1","o-slashpw https://SECRET-SU:SECRET-ab/cd+ef==@rpc.example/v2/SECRET-SK y"], + ["$1","o-atpw https://u:SECRET-P@SS@at.example/v2/SECRET-AK y"], + ["$1","o-frag https://app.example#access_token=SECRET-TOK y"], + ["$1","o-ipv6 http://u:SECRET-V6PW@[2001:db8::1]:8545/SECRET-V6 y"], + ["$1","o-userhost redis://:SECRET-RP@cache.example:6379 y"], + ["$1","o-atpath https://h.example/users/@alice/x y"], + ["$1","o-noturl ops@example.com at 10:30, ref #42 path a/b?c"]]}, {"stream":{"journal__systemd_unit":"decdn-node.service","journal_priority_keyword":"info"},"values":[ ["$1","{\"level\":\"TRACE\",\"fields\":{\"message\":\"m-trace\"}}"], ["$1","{\"level\":\" DEBUG\",\"fields\":{\"message\":\"m-debug\"}}"], @@ -314,7 +335,9 @@ samples_json() { # nanosecond timestamp ["$1","{\"fields\":{\"message\":\"m-nolevel\"}}"], ["$1","{\"level\":5,\"fields\":{\"message\":\"m-weird\"}}"], ["$1","m-text is not json"], - ["$1","{\"level\":\"WARN\",\"fields\":{\"message\":\"m-rpcurl via https://u:SECRET-PASS@rpc.example.io:8545/v3/SECRET-KEY#SECRET-FRAG failed\"}}"]]} + ["$1","{\"level\":\"WARN\",\"fields\":{\"message\":\"m-rpcurl via https://u:SECRET-PASS@rpc.example.io:8545/v3/SECRET-KEY#SECRET-FRAG failed\"}}"], + ["$1","{\"level\":\"WARN\",\"fields\":{\"url\":\"https://user:p\\\\u0026SECRET-EU@rpc.example/v2/SECRET-EK\",\"message\":\"m-escuser\"}}"], + ["$1","{\"level\":\"WARN\",\"fields\":{\"message\":\"m-nested body={\\\\\"url\\\\\":\\\\\"https://n.example/v2/k?a=1\\\\\\\\u0026key=SECRET-NK\\\\\"}\"}}"]]} ]} JSON } @@ -401,12 +424,13 @@ dropped() { # marker ! grep -qF "$1" "$harness_log" || harness_fail "$1 should have been dropped" } # URL redaction (#84). Every credential in the samples is spelled SECRET-*, so -# no_secrets is the blanket check; redacted pins the exact shape that survives. +# no_secrets is the blanket check; redacted pins text the marker's line must +# contain, and entry_is pins a whole line. redacted() { # marker, the redacted text its line must carry grep -F "$1" "$harness_log" | grep -qF "$2" || harness_fail "$1 should carry \"$2\"" } -unchanged() { # the whole line, which must come out byte-for-byte - grep -qF "entry=\"$1\" " "$harness_log" || harness_fail "\"$1\" should pass through unchanged" +entry_is() { # the whole line exactly as it must come out (redacted or untouched) + grep -qF "entry=\"$1\" " "$harness_log" || harness_fail "no entry came out as \"$1\"" } no_secrets() { ! grep -qF 'SECRET' "$harness_log" || harness_fail "a URL credential (SECRET-*) left the pipeline" @@ -414,7 +438,7 @@ no_secrets() { # Defaults (guardrail "debug|trace"), sponsord not enabled: its plain-text lines # are not re-levelled and stand at journald's info. -run_level_harness defaults.alloy 24 +run_level_harness defaults.alloy 34 kept s-trace info sponsord.service kept s-error info sponsord.service kept m-warn-uc warning decdn-node.service @@ -447,14 +471,32 @@ redacted o-twourls 'o-twourls wss://@a.example/ and HTTP://b redacted o-escslash 'https:\\/\\/@esc.example/\"}"' redacted o-escamp 'https://amp.example/\"}"' redacted o-escquote 'say \\\"https://q.example/\\\" ok\"}"' -unchanged 'o-nopath https://c.example:443 done' # nothing after the host: untouched -unchanged 'm-text is not json' # no URL: untouched +entry_is 'o-nopath https://c.example:443 done' # no userinfo, nothing after the host: untouched +entry_is 'm-text is not json' # no URL: untouched +# userinfo runs to the token's LAST @, so unencoded / ? # or @ in a password +# cannot cut it short; an @ in a path costs the host instead (over-redaction) +entry_is 'o-hashpw postgres://@db.example/ y' +entry_is 'o-slashpw https://@rpc.example/ y' +entry_is 'o-atpw https://@at.example/ y' +entry_is 'o-frag https://app.example/ y' +entry_is 'o-ipv6 http://@[2001:db8::1]:8545/ y' +entry_is 'o-userhost redis://@cache.example:6379 y' +entry_is 'o-atpath https://@alice/ y' +# userinfo read through a JSON escape, and one level of JSON nested in a JSON +# string; both stay valid JSON, so the node's level still parses +entry_is '{\"level\":\"WARN\",\"fields\":{\"url\":\"https://@rpc.example/\",\"message\":\"m-escuser\"}}' +entry_is '{\"level\":\"WARN\",\"fields\":{\"message\":\"m-nested body={\\\"url\\\":\\\"https://n.example/\\\"}\"}}' +kept m-escuser warning decdn-node.service +kept m-nested warning decdn-node.service +# none of the characters a broader expression might trip on changes a line +entry_is 'o-noturl ops@example.com at 10:30, ref #42 path a/b?c' +entry_is '{\"level\":\"INFO\",\"fields\":{\"message\":\"m-info\"}}' echo "ok: URL credentials are redacted before the level stages (defaults)" # Guardrail "info|debug|trace": the daemon's every line is journald-info, so it # must be judged by its JSON level — errors and warnings survive, and lines with # no JSON level are never dropped for a priority they did not choose. -run_level_harness priorityinfo.alloy 13 +run_level_harness priorityinfo.alloy 23 no_secrets dropped s-error # sponsord not enabled: not exempt either dropped s-rpcurl @@ -474,7 +516,7 @@ dropped m-other-debug echo "ok: daemon log level follows the JSON body (info in the guardrail)" # sponsord enabled: its level is parsed from the plain-text line. -run_level_harness sponsordonly.alloy 22 +run_level_harness sponsordonly.alloy 32 no_secrets kept s-rpcurl error sponsord.service # the "Error: " match still sees the redacted line kept s-info info sponsord.service @@ -490,7 +532,7 @@ echo "ok: sponsord log level follows its text line (defaults)" # Co-located, guardrail "info|debug|trace": both daemons exempt from the # journald-priority drop and judged by the level they logged. -run_level_harness colocated.alloy 19 +run_level_harness colocated.alloy 29 no_secrets kept s-rpcurl error sponsord.service kept m-rpcurl warning decdn-node.service