Skip to content

clickpipes: add new CDC resource utilization metrics #6488

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
morsapaes merged 2 commits into from
Jul 4, 2026
Merged

clickpipes: add new CDC resource utilization metrics #6488

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
6d0a94f
clickpipes: add new CDC resource utilization metrics
morsapaes Jul 3, 2026
b92c778
Split into sections
morsapaes Jul 4, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 21 additions & 6 deletions docs/integrations/data-ingestion/clickpipes/monitoring.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,6 @@ doc_type: 'reference'
keywords: ['ClickPipes', 'monitoring', 'metrics', 'Prometheus', 'observability']
---


In addition to in-console monitoring, ClickPipes exposes metrics to a [Prometheus-compatible endpoint](/integrations/prometheus) for scraping. These metrics are published with other ClickHouse Cloud service metrics, and allow you to integrate ClickPipes monitoring with your existing observability stack (e.g., [Grafana](/integrations/prometheus#integrating-with-grafana), [Datadog](/integrations/prometheus#integrating-with-datadog)).

## Metric labels {#metric-labels}
Expand Down Expand Up @@ -89,14 +88,30 @@ ClickPipes_Latency{clickhouse_org="11dfa1ec-767d-43cb-bfad-618ce2aaf959",clickho
| `ClickPipes_Latency` | Gauge (_avg_) | Time in milliseconds between when a record was produced at the source and when it was written to ClickHouse. |
| `ClickPipes_SourceReplicationLatency_MiB` | Gauge (_avg_) | **CDC ClickPipes** Replication lag at the source in MiB. For Postgres ClickPipes, this reflects the replication slot lag. |

### Resource usage {#metrics-resources}
### Resource utilization {#metrics-resources}

#### Streaming and object storage ClickPipes {#metrics-resources-streaming-os}

`ClickPipes_Replica_*` metrics report resource utilization for streaming and object storage ClickPipes. Each ClickPipe has **dedicated compute**, so these values reflect utilization for individual ClickPipes.

| Metric | Type | Description |
|----------------------------------|----------------|-------------|
| `ClickPipes_Replica_CPUUsage` | Gauge (_avg_) | Average CPU usage of the ingestion replicas, in millicores. |
| `ClickPipes_Replica_CPULimit` | Gauge (_last_) | Configured CPU limit of the ingestion replicas, in millicores. |
| `ClickPipes_Replica_MemoryUsage` | Gauge (_avg_) | Average memory usage of the ingestion replicas, in bytes. |
| `ClickPipes_Replica_MemoryLimit` | Gauge (_last_) | Configured memory limit of the ingestion replicas, in bytes. |

#### CDC ClickPipes {#metrics-resources-cdc}

`ClickPipes_CDC_*` metrics report resource utilization for CDC ClickPipes. All CDC ClickPipes in the same service **share compute**, so these values reflect the combined usage of every CDC ClickPipe in the service.

| Metric | Type | Description |
|----------------------------------|----------------|-------------|
| `ClickPipes_Replica_CPUUsage` | Gauge (_avg_) | Average CPU usage of the ingestion replicas, in millicores. |
| `ClickPipes_Replica_CPULimit` | Gauge (_last_) | Configured CPU limit of the ingestion replicas, in millicores. |
| `ClickPipes_Replica_MemoryUsage` | Gauge (_avg_) | Average memory usage of the ingestion replicas, in bytes. |
| `ClickPipes_Replica_MemoryLimit` | Gauge (_last_) | Configured memory limit of the ingestion replicas, in bytes. |
| `ClickPipes_CDC_CPUUsage` | Gauge (_avg_) | Average CPU usage, in millicores. Shared across all CDC ClickPipes in the service. |
| `ClickPipes_CDC_CPULimit` | Gauge (_last_) | Configured CPU limit, in millicores. Shared across all CDC ClickPipes in the service. |
| `ClickPipes_CDC_MemoryUsage` | Gauge (_avg_) | Average memory usage, in bytes. Shared across all CDC ClickPipes in the service. |
| `ClickPipes_CDC_MemoryLimit` | Gauge (_last_) | Configured memory limit, in bytes. Shared across all CDC ClickPipes in the service. |
| `ClickPipes_CDC_NetworkReceiveBytes` | Gauge | Inbound network bandwidth, in bytes (delta per scrape window). Shared across all CDC ClickPipes in the service. |

### State and progress {#metrics-state}

Expand Down
Loading