[karl-power]

Summary

clickstack_query_tile and clickstack_query_tiles return data for PromQL tiles instead of "PromQL tiles cannot be queried through MCP yet", so an agent can check the PromQL tiles it builds.

• Runs on all three PromQL backends: a Prometheus endpoint, ClickHouse's Prometheus HTTP API (26.6+), and the ClickHouse table function on older servers.
• Queries follow the dashboard's rules for auto granularity (floored at the source's minAutoGranularity), bucket alignment, and instant vs range queries, so values match the tile.
• Rows use the app's chart shape, { __hdx_time_bucket, series_name, value }, with series named by the legend template. seriesLabels lists each series' labels once.
• Number, pie and bar tiles return one reduced value per series. Dashboard variables apply through variableValues.
• A rejected expression is a user error; an unreachable, timed-out or failing backend is a server error. Error messages never include the host's credentials.
• Range queries over 11,000 points are rejected on every backend, matching Prometheus's own limit.
• Each query gets a promql.query span with hyperdx.promql.backend and hyperdx.promql.query_kind.
• Backend selection and the ClickHouse Prometheus API helpers move from routers/api/prometheus.ts to controllers/promql.ts so MCP and the router share them. The router behaves as before; its tests pass unchanged apart from an import path.
• New buildPrometheusRequest builds every Prometheus request (URL, params, Basic auth, redacted target) for both the router's streaming proxy and MCP's buffered fetch. The one visible change: a host with malformed percent-encoding in its userinfo gets the proxy's "Invalid Connection host" 400.

How to test

The expressions below use count by (job) ({__name__=~".+"}), which counts series per job and returns data for any metrics in the table.

Tile results

1. Call clickstack_save_dashboard:

   {
     "name": "PromQL",
     "tiles": [
       {
         "name": "Series",
         "config": {
           "configType": "promql",
           "displayType": "line",
           "sourceId": "<id>",
           "expressions": [
             { "expression": "count by (job) ({__name__=~\".+\"})" },
             { "expression": "count({__name__=~\".+\"})", "alias": "Total" }
           ],
           "legendTemplate": "{{job}}"
         }
       }
     ]
   }

2. Call clickstack_query_tile with the dashboard and tile IDs and startTime one hour ago. It returns rows { "__hdx_time_bucket", "series_name", "value" } from both expressions (one series per job, plus Total), bucketed at 1 minute, and a seriesLabels entry per series.
3. Call clickstack_query_tile with startTime about an hour ago, at 30 seconds past a minute (e.g. ...T10:00:30Z). The first bucket is at 10:00:00. Turn alignment off with clickstack_patch_dashboard (the config is replaced, not merged, so send all of it) and query again: the first sample is at 10:00:30.
4. Call clickstack_query_tiles on the dashboard. The tile reports status: "ok" and hasData: true.

Filters

5. Add a job label filter and use it in the tile with clickstack_save_dashboard (it replaces the tiles and filters, so the tile keeps its id):

   {
     "id": "<dashboard id>",
     "name": "PromQL",
     "tiles": [
       {
         "id": "<tile id>",
         "name": "Series",
         "config": {
           "configType": "promql",
           "displayType": "line",
           "sourceId": "<id>",
           "expressions": [
             { "expression": "count by (job) ({__name__=~\".+\", job=~\"$job\"})" }
           ],
           "legendTemplate": "{{job}}"
         }
       }
     ],
     "filters": [
       {
         "type": "PROMETHEUS_LABEL",
         "name": "Job",
         "sourceId": "<id>",
         "label": "job",
         "variableName": "job"
       }
     ]
   }

   Then call clickstack_query_tile with this added to the dashboard and tile IDs. Only that job's series comes back.

   { "variableValues": [{ "name": "job", "values": ["<one job from step 2>"] }] }

References

• Linear Issue: HDX-5302
• Related PRs: feat: read and write PromQL tiles and label filters in MCP #3516 (base of this PR), feat: describe PromQL sources in MCP #3496 (PromQL sources in MCP)

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated
hyperdx-oss	Ready	Preview	Nov 9, 2026 5:14pm UTC
hyperdx-storybook	Ready	Preview	Nov 9, 2026 5:14pm UTC

[changeset-bot]

🦋 Changeset detected

Latest commit: 0a18b0c

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 3 packages

Name	Type
@hyperdx/api	Patch
@hyperdx/app	Patch
@hyperdx/otel-collector	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

[github-actions]

🔴 Tier 4 — Critical

Touches authentication, tenancy data models, the public API or shipped database config — or substantially changes the query rendering engine, background tasks, the OTel pipeline, image build, or release CI.

Why this tier:

• Large diff: 1110 production lines changed (threshold: 1000)
• Touches API routes or data models — hidden complexity risk

Review process: Deep review from a domain expert. Synchronous walkthrough may be required.
SLA: Schedule synchronous review within 2 business days.

Stats

• Production files changed: 5
• Production lines changed: 1110 (+ 809 in test files, excluded from tier calculation)
• Branch: claude/hdx-5302-mcp-promql-query-tile
• Author: karl-power

To override this classification, remove the review/tier-4 label and apply a different review/tier-* label. Manual overrides are preserved on subsequent pushes.

[github-actions]

Duplicate code analysis

3 new clones introduced by this PR.

• packages/api/src/routers/api/prometheus.ts:355-391 ↔ packages/api/src/routers/api/prometheus.ts:465-497 — 37 lines
• packages/api/src/routers/api/prometheus.ts:355-373 ↔ packages/api/src/routers/api/prometheus.ts:594-614 — 19 lines
• packages/api/src/routers/api/prometheus.ts:374-391 ↔ packages/api/src/routers/api/prometheus.ts:804-818 — 18 lines

If one of these duplicates a helper that already exists, import it instead
of redefining it. If the duplication is deliberate, ignore this comment —
the check is advisory and never fails the build.

---

jscpd compared this branch against origin/main
across app, api and common-utils. Only new clones are listed; the codebase
currently holds 110 in total (1.21%).
Run yarn dupes locally to reproduce, or yarn dupes:all for the full picture.

[github-actions]

🔵 minor — Table and heatmap PromQL tiles return line-chart rows, not what the dashboard shows

In the app, table tiles go through toTableRows: one column per label plus Timestamp/Value, capped at MAX_TABLE_ROWS, with limit sent on instant queries. Histogram heatmaps go through toHistogramRows, which groups by le and rejects series without le (both in packages/app/src/utils/promqlChartQuery.ts). Here both kinds fall into the generic series_name/value shape. No row/series limit is applied and heatmap le is never checked. Branch on displayType === Table and heatmap?.mode === 'histogram' like queryPromqlChartConfig does, ideally through the shared helper suggested above, or document the difference in the tool description.

_{Advisory. Fix if it is a small defect in code this PR changes; otherwise reply in the thread. Do not widen the PR or touch files it did not already change. How to respond}

[karl-power]

Fixed in the stacked follow-up #3529: MCP now uses the app's shared table and histogram row builders, so table tiles return a column per label (capped at MAX_TABLE_ROWS, with limit on instant queries) and histogram heatmaps return rows keyed by le.

[github-actions]

PR Review

If you are a coding agent acting for the author, read this first.
Nothing below blocks merge automatically; a maintainer decides. What they will expect:

• 🔴 critical or 🟠 major in code this PR adds or changes: fix before asking for review.
• 🔵 minor in code this PR changes: your call. Fix if small, otherwise reply.
• Anything about surrounding code, or asking you to widen the change (hoist a helper,
  dedupe with another file, fix other call sites): never fix here, whatever the
  severity. Reply with one sentence; if it is critical, say so plainly so a human sees it.

One commit per review round. After two rounds, stop and ask a maintainer to review
scope rather than addressing more automated findings. Full rule: AGENTS.md.

6 finding(s): 🔴 0 critical · 🟠 2 major · 🔵 4 minor

6 posted as inline comment(s) on the changed lines.

---

_{Severity is the reviewer's own estimate and is used for ordering, not filtering. No finding blocks merge automatically; a maintainer decides.}

[github-actions]

Deep Review

✅ No critical issues found.

The diff was reviewed by the reviewers listed below plus orchestrator-level analysis of the full diff. I independently traced the credential-redaction paths (buildPrometheusRequest strips userinfo into a Basic-auth header and exposes redactedTarget; every throw in fetchPrometheusData names only redactedTarget), the series-name alignment in toPromqlTileResult (renderPromqlSeriesNames returns names grouped per-expression in series order, and names.flat() lines up index-for-index with expressions.flatMap(r => r.result); the reducer's early return on a null value does not desync the index), the runPromqlQuery resolution guard, and the prometheus.ts router refactor (connection/databaseName/tableName and chBackend.client/connectionId remain correctly wired). No correctness or security regression surfaced in that analysis.

🟡 P2 — recommended

• packages/api/src/mcp/tools/query/promqlTile.ts:124 — toSamples handles scalar and string result types with new behavior (the string branch throws a user-category PromQLQueryError, the scalar branch emits a single metric: {} row), but no test exercises either path.
  • Fix: Add unit coverage driving a scalar response through toPromqlTileResult to assert its row shape, and a string response to assert it throws a user-category PromQLQueryError.
  • _{testing-reviewer}

🔵 P3 nitpicks (5)

• packages/api/src/controllers/promql.ts:639 — The max-resolution guard (stepSec <= 0 || (endSec - startSec) / stepSec > PROMETHEUS_MAX_RESOLUTION) is now re-implemented in both runPromqlQuery and queryRangeHandler with divergent user-facing wording, so the shared limit has two sources of truth.
  • Fix: Extract a single assertWithinMaxResolution(startSec, endSec, stepSec) helper in the controller and call it from the router.
  • _{maintainability-reviewer}
• packages/api/src/mcp/tools/query/promqlTile.ts:183 — toPromqlTileResult relies on positional alignment between two independent flattenings (names.flat() vs expressions.flatMap(({ result }) => result)); a future divergence in count or order would silently mislabel series (falling back to '') rather than failing, and no test covers a multi-expression tile with differing per-expression series counts.
  • Fix: Carry names grouped alongside each expression's result (zip per-expression) so the correspondence is structural, and add a multi-expression test.
  • _{maintainability-reviewer}
• packages/api/src/mcp/tools/query/helpers.ts:708 — The non-PromQLQueryError fallback to clickHouseErrorResult(e) for the table-function backend is untested, so the table-function ClickHouse-error path is unverified.
  • Fix: Add a case that forces the table-function backend (ClickHouse < 26.6) into a ClickHouse error and asserts the result surfaces via clickHouseErrorResult.
  • _{testing-reviewer}
• packages/api/src/controllers/promql.ts:342 — The word upstream names both the clickhousePrometheusUpstream host-URL builder and the upstream field returned by clickhousePrometheusRequest, which reads ambiguously.
  • Fix: Rename the string builder to something like clickhousePrometheusHostWithLimits to disambiguate it from the request builder.
  • _{maintainability-reviewer}
• packages/api/src/routers/api/prometheus.ts:355 — The four proxy handlers (queryRangeHandler, queryHandler, queryExemplarsHandler, handleLabelLookup) still repeat the metric-timing, missing-query 400, resolvePromqlTarget error block, and isPrometheusEndpoint proxy branch; an advisory duplicate-code check flagged these clones and they remain.
  • Fix: Extract the shared per-handler scaffolding into a common wrapper, or waive deliberately since the check is advisory and never fails the build.
  • _{previous-comments-reviewer}

---

Reviewers (4): testing, maintainability, previous-comments, learnings-researcher (plus orchestrator analysis of correctness, security, and the router refactor).

Testing gaps:

• scalar and string result types in toSamples are never exercised end-to-end.
• Table-function backend ClickHouse-error fallback in runConfigTile is untested.
• The probe-returns-false-despite-version-OK fallback in resolveClickhouseQueryBackend (handler not configured on a 26.6+ server) is not explicitly covered.
• toPromqlTileResult alignment is not tested with multiple expressions returning differing series counts.
• Prior review items for table and histogram-heatmap row shapes are acknowledged as deferred to the stacked follow-up refactor: Share PromQL chart planning and rows between app and MCP #3529, not regressions in this diff.

_{No finding blocks merge automatically. A maintainer will expect P0/P1 findings in code this PR changes to be fixed; P2/P3 are your call -- fix or reply. Never fix findings about surrounding code here; reply instead. Do not widen the PR. How to respond}

[github-actions]

E2E Test Results

✅ All tests passed • 463 passed • 1 skipped • 1170s

Status	Count
✅ Passed	463
❌ Failed	0
⚠️ Flaky	3
⏭️ Skipped	1

Tests ran across 4 shards in parallel.

View full report →

[github-actions]

🟠 major — PromQL result normalization and row shaping copied from the app instead of shared

toSamples copies normalizeInstantResult and toSampledSeries, and toPromqlTileResult copies toChartRows and reduceBucketRows, all from packages/app/src/utils/promqlChartQuery.ts. SampledSeries/PromqlExpressionSamples restate the app's SampledSeries/ExpressionResult types, and the string-result error message is copied word for word. The REQUIRED DRY rule says not to add a second definition. The PR promises 'values match the tile', so any later change to the app's naming, reducing or bucketing will quietly make MCP disagree with the dashboard. Move these helpers into common-utils/src/core/promql.ts and import them in both the app and the API.

_{Nothing blocks merge automatically, but a maintainer will expect this fixed if it is a real defect in code this PR changes. If it is about surrounding code, reply and say so instead of patching. Do not widen the PR. How to respond}

[github-actions]

🔵 minor — Table and histogram tiles get chart rows the dashboard never shows, and instant table queries have no row cap

For DisplayType.Table the app uses toTableRows: one column per label, plus Timestamp/Value, capped at MAX_TABLE_ROWS. Its instant query also sends limit: MAX_TABLE_ROWS to Prometheus. Heatmaps in histogram mode use toHistogramRows (rows keyed by le). See packages/app/src/utils/promqlChartQuery.ts:98,350-356. MCP sends both through series_name/value rows and puts no limit on the instant request, so a large vector from a Prometheus endpoint is loaded whole before trimToolResponse cuts it. Branch on the display type the same way the app does (best done with the shared helpers), and pass a series limit for table instant queries.

_{Advisory. Fix if it is a small defect in code this PR changes; otherwise reply in the thread. Do not widen the PR or touch files it did not already change. How to respond}

[teeohhem]

Looks good. Tested locally