feat(database_observability.postgres): Add index-usage collectors - #7068
Merged
Merged
Conversation
…dex detection
Adds two opt-in collectors, disabled by default and enabled via
enable_collectors: table_stats emits pg_stat_user_tables_seq_scan/idx_scan/
n_live_tup; index_stats emits pg_stat_user_indexes_idx_scan_total,
pg_index_properties{is_primary}, and pg_index_size_bytes. Both fan out
across every database the connection can reach (reusing schema_details'
discovery/connection mechanism, extracted into a shared multi_database.go),
fixing the vendored postgres_exporter's single-DSN limitation for these
system views natively rather than waiting on the unmerged upstream
prometheus-community/postgres_exporter#1378.
Metric names/labels match the existing pg_stat_user_tables_* fields, and,
for the new index metrics, the shape proposed by the unmerged upstream
prometheus-community/postgres_exporter#1071.
Also fixes a redundant-connection issue in the multi-database fan-out
(connectToDatabase): it always opened a fresh connection via factory(),
even for the one database that's already the database the initial
connection points to, since a freshly-sql.Open'd *sql.DB is never
pointer-equal to the initial one. It now detects that case up front and
reuses the initial connection directly.
Validated against a live db-o11y-postgres-16 test instance hosting real
books_store data: every emitted series across both collectors matched the
database source exactly.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Contributor
|
💻 Deploy preview deleted (feat(database_observability.postgres): Add index-usage collectors). |
gaantunes
marked this pull request as ready for review
September 9, 2026 19:35
Drop the source system view names (pg_stat_user_tables/pg_stat_user_indexes) from the collector table, matching the terse, purpose-focused style already used for the other rows in this table. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Shorten doc comments in the new postgres collectors: drop "KG insight" task-framing and restate-the-code prose, keep the genuinely non-obvious rationale (multi-database scoping, the redundant-connection gotcha, upstream-name matching) but state it more directly. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Renames pg_stat_user_tables_*, pg_stat_user_indexes_idx_scan_total, pg_index_properties, and pg_index_size_bytes to database_observability_pg_stat_user_tables_*, database_observability_pg_stat_user_indexes_idx_scan_total, database_observability_pg_index_properties, and database_observability_pg_index_size_bytes -- matching the namespace ConnectionInfo and the logs collector's error metrics already use. This also closes a latent registry collision: table_stats' prior names were identical to the real embedded postgres_exporter's own stat_user_tables collector output, so enabling both table_stats and the prometheus_exporter block for the same target would have hit the same "duplicate metrics collector registration" conflict fixed for mysql's table_stats/index_stats a few commits back. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
clayton-cornell
added
the
type/docs
Contributor
|
All looks good for doc changes :-) |
…dex_properties Per Cristian's review on the postgres PR: - Move discoverDatabases/connectToDatabase and selectAllDatabases into dsn.go, since DSN parsing and per-database connection fan-out go together; drop the now-empty multi_database.go/_test.go. - Drop the pg_index_properties metric and its is_primary column/join: schema_details already surfaces primary-key info, and index_stats has no equivalent in the mysql PR. This also removes the double-append needed only to build that metric's label list. - Trim the now-stale "field names match upstream" comment on index_stats. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Rather than a separate constant-1 index_properties metric, attach is_primary, is_unique, and is_partial directly as labels on index_size_bytes. These are 1:1 with an index the same way schemaname and indexrelname already are, so this adds no series -- a separate info-style metric would have cost a full extra series per index for re-emitting the same identifying labels. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
4 tasks
Mirrors the mysql_index_stats/mysql_table_stats subsystem naming from the mysql PR: pg_stat_user_indexes/pg -> pg_index_stats (idx_scan_total, size_bytes), pg_stat_user_tables -> pg_table_stats (seq_scan, idx_scan, n_live_tup). SQL/log references to the actual pg_stat_user_indexes and pg_stat_user_tables views are unchanged -- only the metric namespace segment moves. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
seq_scan and idx_scan are registered as CounterValue (cumulative since the last stats reset), so per Prometheus convention they need a _total suffix, matching pg_index_stats_idx_scan_total. n_live_tup stays as-is since it's a GaugeValue, not cumulative. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
row_count is the cross-engine name settled on for this gauge, so mysql's upcoming equivalent (mysql.innodb_table_stats.n_rows) can share it. Not row_total: this is a live snapshot that can go up or down, not a monotonic counter, so a _total suffix would be misleading. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
"the primary key" instead of "a primary key" -- a table has exactly one, in either engine. Found while auditing all new metrics across both PRs for cross-engine consistency. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
gaantunes
added a commit
that referenced
this pull request
Sep 11, 2026
…ls and a new row_count metric (#7095) ### Brief description of Pull Request Adds two signals to the `index_stats`/`table_stats` collectors, mirroring the same additions made to the postgres collectors in #7068: - `is_primary`/`is_unique` labels on `mysql_index_stats_size_bytes`. `is_primary` is derived from `INDEX_NAME = 'PRIMARY'` (no extra query needed); `is_unique` is joined in from `information_schema.statistics.NON_UNIQUE`. - `mysql_table_stats_row_count`, a new gauge sourced from `mysql.innodb_table_stats.n_rows` (the sibling table to `mysql.innodb_index_stats`, already used for index size). ### Pull Request Details Both are needed for the missing/unused-index Knowledge Graph insight: - Without `is_primary`/`is_unique`, the unused-index insight has no signal to avoid recommending the removal of an index that's actually backing a primary key or unique constraint just because its usage counters look low. These are folded onto the existing `size_bytes` gauge as labels rather than a separate metric, since both attributes are 1:1 with an index the same way `schema`/`table`/`index` already are — this adds no additional series. - Without `row_count`, the missing-index insight can't tell a table that's genuinely being hammered with full scans from one that's just small enough that a full scan doesn't matter — the same gap closed on the postgres side by `pg_table_stats_row_count`. ### Issue(s) fixed by this Pull Request ### Notes to the Reviewer ### PR Checklist - [ ] Documentation added - [ ] Tests updated - [ ] Config converters updated - [ ] This pull request was substantially generated with AI assistance (see the [GenAI policy](https://github.com/grafana/alloy/blob/main/docs/developer/genai.md)) 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
cristiangreco
approved these changes
Sep 14, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Brief description of Pull Request
Split out of #7003 (originally combined postgres + mysql; splitting per one-logical-change-per-PR).
Adds two opt-in collectors to
database_observability.postgres, disabled by default and enabled viaenable_collectors, emitting only the metrics needed for missing-index and unused-index KG insights:table_stats—pg_stat_user_tables_seq_scan/idx_scan/n_live_tup.index_stats—pg_stat_user_indexes_idx_scan_total,pg_index_properties{is_primary},pg_index_size_bytes.Both fan out across every database the connection can reach, reusing the discovery/connection mechanism already used by
schema_details(extracted into a sharedmulti_database.go). This fixes the vendoredpostgres_exporter's single-DSN limitation for these system views natively, rather than waiting on the unmerged upstreamprometheus-community/postgres_exporter#1378.Metric names/labels match the existing
pg_stat_user_tables_*fields, and, for the new index metrics, the shape proposed by the unmerged upstreamprometheus-community/postgres_exporter#1071.Also fixes a redundant-connection issue in the multi-database fan-out (
connectToDatabase): it always opened a fresh connection viafactory(), even for the one database that's already the database the initial connection points to, since a freshly-sql.Open'd*sql.DBis never pointer-equal to the initial one. It now detects that case up front and reuses the initial connection directly instead.Validated against a live
db-o11y-postgres-16test instance hosting realbooks_storedata: every emitted series across both collectors matched the database source exactly.Pull Request Details
Issue(s) fixed by this Pull Request
Notes to the Reviewer
PR Checklist