feat(ansible): allow the grafana_alloy API token via git-ignored inventory

[thiras]

What

GC_API_TOKEN gets the same dual-home treatment decdn_rpc_url received in #39. Setting grafana_alloy_api_token (git-ignored host_vars/<node>/secret.yml, like everything else sensitive) makes grafana_alloy author /etc/grafana-alloy.env itself — a single JSON-encoded GC_API_TOKEN="…" line at root:root 0600 (no_log). Leaving it empty keeps today's operator-provisioned posture unchanged; the value never transits the control machine.

Provenance machinery (#39 parity)

After every enabled converge the role records <source> <sha256> of the env file into /etc/grafana-alloy.env.sha256 (root 0600), recorded last — after handler flush and service start, checksum-guarded under --check. Preflight uses it for two fail-loud guards:

State	Guard
var empty + record says inventory + checksum matches	refuse to adopt the role-authored file as host-provisioned ("your secret.yml went missing")
var set + foreign/unknown provenance on an existing file	require grafana_alloy_overwrite_host_file: true before rewriting

Authoring is all-or-nothing: extra hand-added GC_* lines are discarded by a rewrite, so preflight's per-key gates force migrating them to their inventory variables first. An out-of-band edit on the host path now notifies a restart before the flush/record (same ordering as decdn_node :1408/:1611). Teardown removes the record behind the managed-by marker, so disable→enable re-seeds cleanly. Checksums pinned to sha256 explicitly so the .sha256 cross-checks against sha256sum.

Zero template changes — config.alloy.j2 keeps sys.env("GC_API_TOKEN"); only who fills the file changes.

Tests

• validation: rescue matcher updated for the renamed existence assert; new cases ga-token-var-shape (whitespace token) and ga-collision (foreign fixture + token var, no overwrite knob); expected-tags multiset extended
• New scenario molecule/grafana-cloud-token: no staged env file, token via playbook vars; verifies 0600 root:root regular file, exact single-line content, <source> <sha256> record vs live stat checksum, no token literal outside the env file, running service, idempotence, plus an end-state rotation pass
• Docs: role README §Credentials gains "The API token has two homes now"; repo-level README setup paragraph + config table rows; both .example templates cross-reference the other path

Out of scope

Helm chart (Secret-oriented by design) and lint-alloy matrix (renders config only).

Local gates: make lint, lint-ansible (production profile), lint-alloy (real binary, 8 combos), molecule grafana-cloud-token / grafana-cloud / validation — all green.

[Copilot]

🟡 Changes recommended

Unresolved provenance, rotation, path-collision, write-order, and token-leakage issues block approval.

Get a fresh assessment by requesting another Copilot review.

Pull request overview

Adds inventory-backed Grafana Alloy API-token provisioning with provenance tracking, overwrite safeguards, documentation, and Molecule coverage.

Changes:

• Adds token authoring, checksum records, rotation detection, and validation.
• Updates role and repository documentation.
• Adds validation cases and a dedicated token scenario.

File summaries

File	Summary
ansible/roles/grafana_alloy/tasks/validate-paths.yml	Validates checksum-file paths.
ansible/roles/grafana_alloy/tasks/preflight.yml	Adds token and provenance gates.
ansible/roles/grafana_alloy/tasks/main.yml	Authors, restarts, and records token files.
ansible/roles/grafana_alloy/README.md	Documents token workflows.
ansible/roles/grafana_alloy/files/grafana-alloy.env.example	Documents dual credential paths.
ansible/roles/grafana_alloy/defaults/main.yml	Defines token and provenance options.
ansible/README.md	Updates setup and scenario documentation.
ansible/molecule/validation/converge.yml	Adds token validation cases.
ansible/molecule/grafana-cloud-token/verify.yml	Verifies authoring, provenance, and rotation.
ansible/molecule/grafana-cloud-token/molecule.yml	Defines the new scenario.
ansible/molecule/grafana-cloud-token/files/alloy-stub	Supplies the test binary stub.
ansible/molecule/grafana-cloud-token/converge.yml	Configures the token scenario.
ansible/inventory/host_vars/decdn-node-1/secret.yml.example	Shows the token variable.

Review details

Suppressed comments (5)

ansible/roles/grafana_alloy/README.md:94

• This documentation promises that rotation is just an inventory edit plus deploy, but the current provenance guard rejects that edit whenever the new token changes the file checksum (before the rewrite runs). Update the documented workflow together with the guard, otherwise operators following this section will hit the overwrite assertion on every rotation.

With it set, the role authors `/etc/grafana-alloy.env` itself — as a **token-only**
file (`GC_API_TOKEN="…"` at `root:root 0600`, JSON-encoded so quotes/backslashes
survive), rewritten wholesale on every converge. Runtime behaviour is identical:
`config.alloy` still reads `sys.env("GC_API_TOKEN")`; only who fills the file
changes. Rotation is an inventory edit + deploy instead of per-host shell work,
and provenance tracking (a `<source> <sha256>` record next to the file) catches

ansible/roles/grafana_alloy/tasks/main.yml:283

• This copy runs before alloy validate, service start, and the provenance-record write. If a later task fails after a token rotation changes the file, the old record remains; the next retry sees an inventory checksum mismatch and demands grafana_alloy_overwrite_host_file, even though the mismatch was caused by this role. Defer the authoring write until failure-prone validation succeeds, or track the pending role write so retries do not classify it as foreign.

    - name: Write the token-only secret environment file (inventory-supplied)
      ansible.builtin.copy:
        dest: "{{ grafana_alloy_secret_file }}"
        owner: root
        group: root

ansible/roles/grafana_alloy/tasks/main.yml:283

• This inventory path reads grafana_alloy_api_token on the Ansible control machine and sends it to the target via copy, so it necessarily transits the control machine. That contradicts the PR description's claim that the value never does; clarify that claim (it is only true for the empty-variable, host-provisioned path) so operators understand the different threat model.

    - name: Write the token-only secret environment file (inventory-supplied)
      ansible.builtin.copy:
        dest: "{{ grafana_alloy_secret_file }}"
        owner: root
        group: root

ansible/roles/grafana_alloy/tasks/preflight.yml:460

• This condition treats a normal inventory-token rotation as a foreign edit: after the first inventory converge, changing the token necessarily makes the live file checksum differ from the recorded checksum, so this assert aborts before the copy task can rotate it. The new grafana-cloud-token verify play exercises exactly that path and will fail here; adjust the provenance/rotation design (or explicitly require and document an opt-in for rotation) while preserving the out-of-band-edit guard.

    - _ga_recorded_sum | length > 0
    - >-
      _ga_record_source != 'inventory'
      or _ga_recorded_sum != (_ga_secret_stat.stat.checksum | default(''))

ansible/roles/grafana_alloy/tasks/preflight.yml:460

• This condition deliberately skips files with no provenance record, so an existing hand-edited/unknown file is overwritten without requiring grafana_alloy_overwrite_host_file. That can discard its extra GC_* settings; it also contradicts the PR contract and README, which say unknown/untracked existing files require the explicit opt-in. Treat an empty provenance record as foreign for this guard (and remove or revise the now-unneeded warning path).

    - grafana_alloy_api_token | length > 0
    - _ga_secret_stat.stat.exists
    - _ga_recorded_sum | length > 0
    - >-
      _ga_record_source != 'inventory'
      or _ga_recorded_sum != (_ga_secret_stat.stat.checksum | default(''))

• Files reviewed: 13/13 changed files
• Comments generated: 3
• Review effort level: Lite

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.