Skip to content

chore(deps): Update cAdvisor to v0.60.5 - #7079

Open
dehaansa wants to merge 17 commits into
mainfrom
update-cadvisor-v0.60.5
Open

dehaansa wants to merge 17 commits into
mainfrom
update-cadvisor-v0.60.5

Conversation

@dehaansa

Copy link
Copy Markdown
Contributor

Brief description of Pull Request

Updates the cAdvisor dependency from v0.54.1 to v0.60.5 (Grafana grafana-v0.60.5-noglobals fork).

Pull Request Details

cAdvisor v0.60 moved most of its library packages into a new github.com/google/cadvisor/lib submodule. The root module points /lib at a local ./lib replace, which Go ignores when the module is consumed as a dependency. To make it resolve, collector/builder-config.yaml now carries a second require/replace pair that redirects github.com/google/cadvisor/lib to the same Grafana fork commit, and the go.mod files are regenerated with make generate-otel-collector-distro.

The integration imports are rewritten to the new /lib paths. container/docker and info/v2 stayed in the root module, so those imports are unchanged. No cAdvisor API signatures changed.

Stacked on top of #7039 (the cAdvisor integration test); this PR targets that branch so its diff is only the dependency update.

Issue(s) fixed by this Pull Request

Notes to the Reviewer

PR Checklist

  • Documentation added
  • Tests updated
  • Config converters updated
  • This pull request was substantially generated with AI assistance (see the GenAI policy)

dehaansa and others added 8 commits September 4, 2026 16:26
Add a docker integration test for the cAdvisor exporter. The test scrapes
a single prometheus.exporter.cadvisor instance into Mimir and asserts that
cAdvisor metrics appear. cAdvisor uses its raw cgroup driver, so the test
needs a privileged container but no container runtime.

This POC includes a temporary discovery test. It lists every cAdvisor
metric that reaches Mimir so the assertion list can be pinned from a real
Linux run. The discovery test fails on purpose to surface its output.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Replace the temporary discovery test with the real assertion list, pinned
from a CI run. The list covers every cAdvisor collector family: build and
version, cpu, memory, filesystem, network, and blkio.

Leave out container_pressure_* (needs kernel CONFIG_PSI) and
container_health_state (needs a Docker HEALTHCHECK on the target container),
since neither is guaranteed on every host.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
… blocked pod start

A read-only host /var/run mount stops Kubernetes from mounting the service
account token, so the Alloy pod failed with RunContainerError. The raw cgroup
driver does not need /var/run. Only the docker and containerd plugins use it,
and this test does not use them.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
cAdvisor v0.60 split its library packages into a new
github.com/google/cadvisor/lib submodule. Its in-repo local replace is
ignored from a dependency, so add a second require and replace pair that
redirect /lib to the Grafana fork. Rewrite the integration imports to the
/lib paths; container/docker and info/v2 stay in the root module.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@dehaansa
dehaansa requested a review from a team as a code owner September 10, 2026 15:22
@github-actions

github-actions Bot commented Sep 10, 2026

Copy link
Copy Markdown
Contributor

🔍 Dependency Review

Below are the dependency changes detected in go.mod files, with required code updates (if any), supporting evidence, and minimal example diffs to adopt the upgrades.


github.com/google/cadvisor v0.54.1 -> v0.60.5 — ❌ Changes Needed

Summary of required changes:

  • cAdvisor v0.60 split most of its library code under github.com/google/cadvisor/lib/... and removed most process-global singletons/registries. Import paths and initialization must be updated.
  • The manager.New API changed to require filesystem plugins to be passed per-instance (and not via a global registry).
  • Optional collectors (perf events, resctrl/Intel RDT, and application metrics) are now injected via instance factories rather than enabled globally. Wire them explicitly to preserve existing behavior.
  • Some metric families were added (PSI, cgroup v2 memory events/pages, CPU CFS burst accounting, container_start_time_seconds). Tests that validate an allowlist/expect-list should be updated.

Minimal code changes (already reflected in this PR):

  1. Update imports from old top-level packages to the split lib module:
- "github.com/google/cadvisor/cache/memory"
- "github.com/google/cadvisor/container"
- "github.com/google/cadvisor/manager"
- "github.com/google/cadvisor/metrics"
- "github.com/google/cadvisor/storage"
- "github.com/google/cadvisor/utils/sysfs"
+ "github.com/google/cadvisor/lib/cache/memory"
+ "github.com/google/cadvisor/lib/container"
+ "github.com/google/cadvisor/lib/fs"
+ "github.com/google/cadvisor/lib/manager"
+ "github.com/google/cadvisor/lib/metrics"
+ "github.com/google/cadvisor/lib/stats"
+ "github.com/google/cadvisor/lib/storage"
+ "github.com/google/cadvisor/lib/utils/sysfs"
  1. Update container plugin imports to their new locations under lib (docker remains outside lib):
- "github.com/google/cadvisor/container/containerd"
- "github.com/google/cadvisor/container/crio"
- "github.com/google/cadvisor/container/raw"
- "github.com/google/cadvisor/container/systemd"
+ "github.com/google/cadvisor/lib/container/containerd"
+ "github.com/google/cadvisor/lib/container/crio"
+ "github.com/google/cadvisor/lib/container/raw"
+ "github.com/google/cadvisor/lib/container/systemd"

 // still:
  "github.com/google/cadvisor/container/docker"
  1. Pass filesystem plugins to manager.New (replaces former global registration); build the fsPlugins map:
+ // Filesystem plugins now selected per instance.
+ fsPlugins := map[string]fs.FsPlugin{
+   "btrfs":        btrfs.NewPlugin(),
+   "devicemapper": devicemapper.NewPlugin(),
+   "nfs":          nfs.NewPlugin(),
+   "overlay":      overlay.NewPlugin(),
+   "tmpfs":        tmpfs.NewPlugin(),
+   "vfs":          vfs.NewPlugin(),
+   "zfs":          zfs.NewPlugin(),
+ }
  1. Inject optional collectors (perf, resctrl, application metrics) to preserve behavior previously provided by globals:
+ manager.PerfManagerFactory = perf.NewManager
+ manager.ResctrlManagerFactory = func(interval time.Duration, vendorID string, inHostNamespace bool) (stats.ResctrlManager, error) {
+   return intel.NewManager(interval, intel.Setup, vendorID, inHostNamespace, c.DockerOnly)
+ }
+ manager.CollectorManagerFactory = func(handler container.ContainerHandler, readFile func(string) ([]byte, error), httpClient *http.Client) (manager.CollectorManager, error) {
+   return appmetrics.NewManager(handler, readFile, httpClient, manager.ApplicationMetricsCountLimit())
+ }
  1. Update the manager.New call to include fsPlugins (new parameter) and use the lib types:
- rm, err := manager.New(plugins, memoryStorage, sysFs, manager.HousekeepingConfigFlags, includedMetrics, &collectorHTTPClient, c.RawCgroupPrefixAllowlist, c.EnvMetadataAllowlist, c.PerfEventsConfig, time.Duration(c.ResctrlInterval), rawOpts)
+ rm, err := manager.New(plugins, fsPlugins, memoryStorage, sysFs, manager.HousekeepingConfigFlags, includedMetrics, &collectorHTTPClient, c.RawCgroupPrefixAllowlist, c.EnvMetadataAllowlist, c.PerfEventsConfig, time.Duration(c.ResctrlInterval), rawOpts)
  1. Update tests to include metrics introduced since v0.54 (PSI, CFS burst, cgroup v2 memory events/pages, container start time):
- // Two families are left out on purpose:
- //   - container_pressure_* (PSI) needs kernel CONFIG_PSI. Not every host has it.
- //   - container_health_state needs a container with a Docker HEALTHCHECK. It
- //     depends on the sibling workloads, not the exporter.
+ // container_health_state is left out on purpose: it needs a container with a
+ // Docker HEALTHCHECK, which kind's containerd runtime does not provide.

+ // Pressure stall (PSI) metrics...
+ "container_pressure_cpu_stalled_seconds_total",
+ "container_pressure_cpu_waiting_seconds_total",
+ "container_pressure_io_stalled_seconds_total",
+ "container_pressure_io_waiting_seconds_total",
+ "container_pressure_memory_stalled_seconds_total",
+ "container_pressure_memory_waiting_seconds_total",

+ // Metrics added across the 0.58–0.60 line (cgroup v2 memory events/pages, CPU
+ // burst accounting, and container start time).
+ "container_cpu_cfs_burst_periods_total",
+ "container_cpu_cfs_burst_seconds_total",
+ "container_memory_events_high_total",
+ "container_memory_events_max_total",
+ "container_memory_pgscan_total",
+ "container_memory_pgsteal_total",
+ "container_memory_workingset_refault_anon_total",
+ "container_memory_workingset_refault_file_total",
+ "container_start_time_seconds",

Notes about the new submodule:

  • cAdvisor v0.60 split the library into a separate module path github.com/google/cadvisor/lib; you must add it as a requirement (this PR does), and, when using a fork, add a replace for github.com/google/cadvisor/lib as well (also done in this PR).

Evidence:


github.com/containerd/containerd/api v1.9.0 -> v1.10.0 — ✅ Safe
  • This module provides generated API/protobuf types for containerd. Changes in 1.10.0 are additive and compatible for consumers using it transitively through other projects.
  • No code changes required in this repository.

Evidence:


github.com/containerd/ttrpc v1.2.7 -> v1.2.9 — ✅ Safe
  • Minor updates with bug fixes and internal improvements.
  • No direct usage here; upgrade is transitive. No code changes required.

Evidence:


github.com/opencontainers/cgroups v0.0.4 -> v0.0.6 — ✅ Safe
  • Patch/minor updates in the legacy cgroups module used transitively (e.g., by runc/cadvisor).
  • No API usage in this repository; no code changes needed.

Evidence:


github.com/opencontainers/selinux v1.13.0 -> v1.13.1 — ✅ Safe
  • Patch release with fixes. Used transitively by container runtimes.
  • No direct imports in this repo; no code changes required.

Evidence:


github.com/opencontainers/runc (replace) v1.2.8 -> v1.3.6 — ⚠️ Needs Review

What changed:

  • The repository pins runc to v1.3.6 (was v1.2.8) via replace to keep libcontainer subpackages (e.g., libcontainer/{cgroups,intelrdt,user}) that cAdvisor and other deps still import.
  • runc v1.4.0+ removed some of these subpackages; staying on v1.3.x is intentional to avoid breaking downstream dependencies.

Code impact:

  • No code changes are required here for v1.2.8 -> v1.3.6. This pin ensures transitive imports used by cAdvisor continue to resolve.
  • Note: The root go.mod shows a require on runc v1.4.3, but the replace forces v1.3.6; the replace takes precedence and is sufficient.

Evidence:


Notes

  • New module: github.com/google/cadvisor/lib (added due to cAdvisor v0.60 library split). This is expected and required to build against v0.60+; the PR already adds a replace to the Grafana fork for both github.com/google/cadvisor and github.com/google/cadvisor/lib.
  • Removed indirects (e.g., github.com/docker/docker, github.com/euank/go-kmsg-parser, github.com/karrick/godirwalk, github.com/moby/sys/atomicwriter, github.com/morikuni/aec) appear to be fallout from the cAdvisor refactor and general dep pruning. No direct imports in this repo were using them, so no action is needed.

dehaansa and others added 6 commits September 10, 2026 15:15
cAdvisor v0.60 (grafana-v0.60.5-noglobals fork) selects filesystem plugins
per instance instead of from a process-global registry. Build the default
plugin set (btrfs, devicemapper, nfs, overlay, tmpfs, vfs, zfs) and pass it
to manager.New, the same way the container plugins are passed. Without this
the manager has no filesystem plugins and the container_fs_* metrics are
empty. Bump the fork pseudo-version to the commit that adds this API.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The grafana-v0.60.5-noglobals fork now selects the resctrl manager per
instance through manager.ResctrlManagerFactory instead of a global registry.
The manager.New signature is unchanged and Alloy disables resctrl metrics by
default, so this is a pseudo-version bump only.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
cAdvisor v0.60 (grafana-v0.60.5-noglobals fork) turns the perf_event,
resctrl, and application-metrics collectors into per-instance injection
seams that the lean library leaves nil. Set PerfManagerFactory,
ResctrlManagerFactory, and CollectorManagerFactory in the integration so
these keep working as they did before the noglobals split. Bump the fork
pseudo-version to the commit that makes appmetrics importable.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The runc replace held v1.2.8, which govulncheck flags. runc cannot float to
latest: cAdvisor (resctrl/intel) and other dependencies still import runc's
libcontainer packages (intelrdt, user) that v1.4+ removed. v1.3.6 keeps those
packages and is not flagged, so pin the replace to v1.3.6.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Extend the pinned metric list with pressure stall (PSI) metrics and the
metrics cAdvisor v0.60 adds: cgroup v2 memory events and stats
(pgscan/pgsteal, workingset refault, memory events), CPU burst accounting,
and container start time. All are emitted on the CI runner's cgroup v2 kernel.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Address the factory-global race, portability issues in Kubernetes metric assertions, and stale vulnerability-ignore rationale.

Get a fresh assessment by requesting another Copilot review.

Pull request overview

Updates cAdvisor from v0.54.1 to v0.60.5 and adapts the integration to its split /lib module layout.

Changes:

  • Rewrites cAdvisor imports and integration wiring.
  • Updates dependency replacements, versions, and checksums.
  • Expands Kubernetes integration metric assertions.
File summaries
File Reviewed changes and findings
internal/static/integrations/cadvisor/cadvisor.go Critical (1 vote): Package-level factory globals can race during reloads or multiple exporters and overwrite instance-specific state.
integration-tests/k8s/tests/prometheus-exporter-cadvisor/k8s_test.go Moderate (3 votes): PSI metrics need capability gating. Moderate (1 vote): cgroup-v2 metrics need feature-aware assertions. Moderate (2 votes): Burst metrics require a CPU-limited fixture.
go.sum Refreshes dependency checksums.
go.mod Updates dependencies. Moderate (1 vote): Stale Docker vulnerability ignores should be updated or removed.
collector/go.sum Refreshes collector dependency checksums.
collector/go.mod Regenerates collector dependencies.
collector/builder-config.yaml Adds cAdvisor fork module replacements.
Review details

Suppressed comments (2)

go.mod:60

  • This bump removes the legacy github.com/docker/docker requirement, but .govulncheck.yaml still has five ignores whose reasons say cAdvisor is the remaining importer and that they should be revisited when cAdvisor drops that dependency. Please update or remove those entries with this change; otherwise the vulnerability ignore policy remains stale and can suppress findings under an obsolete rationale.
	github.com/google/cadvisor v0.60.5

integration-tests/k8s/tests/prometheus-exporter-cadvisor/k8s_test.go:96

  • These cgroup-v2-dependent memory metrics are not guaranteed by the test setup: the manifest only mounts the host cgroup filesystem and does not require a cgroup-v2 node. On a cgroup-v1 kind/local node cAdvisor legitimately omits these series, so QueryMetrics will time out; assert only feature-independent metrics or enforce/skip based on the cgroup version.
		"container_memory_events_high_total",
		"container_memory_events_max_total",
		"container_memory_pgscan_total",
		"container_memory_pgsteal_total",
		"container_memory_workingset_refault_anon_total",
  • Files reviewed: 5/7 changed files
  • Comments generated: 3
  • Review effort level: Lite

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment on lines +113 to +117
manager.PerfManagerFactory = perf.NewManager
manager.ResctrlManagerFactory = func(interval time.Duration, vendorID string, inHostNamespace bool) (stats.ResctrlManager, error) {
return intel.NewManager(interval, intel.Setup, vendorID, inHostNamespace, c.DockerOnly)
}
manager.CollectorManagerFactory = func(handler container.ContainerHandler, readFile func(string) ([]byte, error), httpClient *http.Client) (manager.CollectorManager, error) {
Comment on lines +80 to +86
// Pressure stall (PSI) metrics. Present on the CI runner kernel.
"container_pressure_cpu_stalled_seconds_total",
"container_pressure_cpu_waiting_seconds_total",
"container_pressure_io_stalled_seconds_total",
"container_pressure_io_waiting_seconds_total",
"container_pressure_memory_stalled_seconds_total",
"container_pressure_memory_waiting_seconds_total",
Comment on lines +90 to +91
"container_cpu_cfs_burst_periods_total",
"container_cpu_cfs_burst_seconds_total",
@thampiotr

Copy link
Copy Markdown
Contributor

@dehaansa I think there are some govulncheck exceptions waiting for this upgrade. Do you want to add them to this PR or later?

@dehaansa

Copy link
Copy Markdown
Contributor Author

@dehaansa I think there are some govulncheck exceptions waiting for this upgrade. Do you want to add them to this PR or later?

I briefly looked into it and I think they still end up transitive dependencies of the mongodb exporter :(. Planning to investigate further in a follow up.

thampiotr
thampiotr previously approved these changes Sep 14, 2026
dehaansa and others added 3 commits September 14, 2026 10:18
…ertions

PSI and cgroup v2 memory metrics are only produced on capable kernels, so
asserting them unconditionally breaks the test on hosts without CONFIG_PSI or
cgroup v2 (for example local dev). The kind node shares the host kernel, so
gate these assertions on the host's capabilities and assert them only where
cAdvisor emits them.

Also drop container_cpu_cfs_burst_* : cAdvisor only emits burst metrics for
containers with a non-zero CPU quota, which no workload in this test guarantees.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
cAdvisor v0.60 stopped importing the legacy github.com/docker/docker client.
The module still enters the build through prometheus.exporter.mongodb
(github.com/percona/percona-backup-mongodb imports docker/docker/api/types),
so the advisories still apply but through a different, client-only path. Update
the ignore reasons to name the real importer and revisit condition.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Bump the cAdvisor fork to the commit that adds manager.Config/Factories, and
pass perf, resctrl, and application-metrics factories through
manager.Config.Factories instead of assigning the package-level *Factory
globals in New(). This removes the reload/multi-instance race on those globals:
the manager now resolves each seam once at construction.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Base automatically changed from poc/cadvisor-integration-test to main September 15, 2026 10:49
@dehaansa
dehaansa dismissed thampiotr’s stale review September 15, 2026 10:49

The base branch was changed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants