Skip to content

Wire the Athena CUR datasource so the finance board's cost-reconciliation panels work in prod #59

Description

@stxkxs

Problem

The agent-finance dashboard's four reconciliation panels set "datasource": "athena-cur"
(dashboards/base/platform/agent-finance.yaml:50,66,143,169), but no Athena GrafanaDatasource
CR exists and the AMG workspace neither enables the Athena plugin nor grants the role Athena/Glue/S3
access. So the board's reason to exist — CUR-truth spend vs the operator's in-cluster estimate, and
estimate-drift — is dead in prod. The CloudWatch panels were fixed in #58; this is the remaining
datasource gap on that board.

The Athena backend is real: the eks-agent-platform cost-pipeline provisions the Athena workgroup,
Glue database, CUR tables, and the invocation_cost_reconciliation view.

Fix (cross-repo)

landing-zone (managed-monitoring component):

  • Add ATHENA to aws_grafana_workspace.data_sources (currently [PROMETHEUS, CLOUDWATCH]) so AMG
    enables the grafana-athena-datasource plugin.
  • Grant the AMG service role athena:StartQueryExecution/GetQueryExecution/GetQueryResults,
    glue:GetTable/GetDatabase/GetPartitions, and s3:GetObject/ListBucket on the CUR results bucket +
    the Athena query-output location.

eks-gitops (dashboards/base/datasources/):

  • Add athena-cur.yaml GrafanaDatasource (type grafana-athena-datasource, uid athena-cur,
    jsonData: catalog AwsDataCatalog, database, workgroup, defaultRegion us-west-2, output-location).
  • The database/workgroup/output-location are per-env (each env's cost-pipeline) — wire them as a
    PLACEHOLDER + per-env overlay patch (same pattern as the AMP workspace URL), sourced from the
    cost-pipeline SSM/Terraform outputs.
  • The finance board references a ${cost_database} it doesn't define yet — add the templating var.

Why not done inline

Cross-repo (landing-zone IAM + plugin + eks-gitops CR + per-env workgroup wiring), and the Athena
plugin/IAM path can't be validated offline — it needs a live AMG workspace to confirm the query path.
Until then the four finance reconciliation panels stay no-data; the board's CloudWatch + kube_customresource
spend panels work.

Acceptance

  • landing-zone AMG enables Athena + role can query the CUR view.
  • eks-gitops athena-cur datasource CR renders in all overlays with per-env workgroup substitution.
  • The four finance panels return CUR data against a live workspace.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions