Skip to content

Measure memory, image size and uncached build time - #7

Open
mvanduijker wants to merge 2 commits into
basecamp:mainfrom
mvanduijker:resource-measurements
Open

mvanduijker wants to merge 2 commits into
basecamp:mainfrom
mvanduijker:resource-measurements

Conversation

@mvanduijker

Copy link
Copy Markdown

The search for the most effective / efficient agentic programming environment should also include memory, docker image size and docker build time.

Results on my old 2 core homelab:

image

Each benchmark round samples the app container's cgroup memory when idle
and after the load (working set, anonymous memory and peak) and records the
benchmarked image's size; summary.json reports medians under `resources`.
Inconsistent counter samples are retried and then fail the run.

bin/build-images times production image builds in a fresh, empty BuildKit
builder with --no-cache --pull, and records the resulting image sizes. It
shares the benchmark lock so builds cannot overlap timed runs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Copilot AI balanced review requested due to automatic review settings October 9, 2026 18:53

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Source changes can go undetected, failed builds discard completed samples, and inconsistent memory samples may be accepted.

1 open finding
What changed in this PR

Adds resource-efficiency measurements to the shared benchmark harness.

Changes:

  • Measures container memory and image size.
  • Adds uncached, multi-round image build timing.
  • Documents and tests the new behavior.

[!TIP]
If you aren't ready for review, convert to a draft PR.
Click "Convert to draft" or run gh pr ready --undo.
Click "Ready for review" or run gh pr ready to reengage.

File Description
README.md Documents resource and build benchmarks.
bin/​build-images Adds uncached image build measurements.
bench/​test_benchmark.rb Tests memory sampling and app validation.
bench/​comparison_support.rb Adds image/source helpers and cgroup metrics.
bench/​compare.rb Records resource metrics in benchmark summaries.

🧠 Review effort: Balanced


💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread bench/comparison_support.rb Resolved
…uilds

A consistent cgroup sample's working set always contains all anonymous
memory, so a smaller value means the counters moved between reads; retry it
like a negative sample. bin/build-images now writes each completed build to
its own JSON file, so a later failure no longer discards finished builds.
summary.json is still written only after every build succeeds.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The source guard can combine builds with different inputs into one median without detecting the changes.

No open issues 🎉

1 resolved since last review

Review effort: Balanced

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants