Repository navigation
Measure memory, image size and uncached build time - #7
Open
mvanduijker wants to merge 2 commits into
Open
mvanduijker wants to merge 2 commits into
mvanduijker wants to merge 2 commits into
Conversation
Each benchmark round samples the app container's cgroup memory when idle and after the load (working set, anonymous memory and peak) and records the benchmarked image's size; summary.json reports medians under `resources`. Inconsistent counter samples are retried and then fail the run. bin/build-images times production image builds in a fresh, empty BuildKit builder with --no-cache --pull, and records the resulting image sizes. It shares the benchmark lock so builds cannot overlap timed runs. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
There was a problem hiding this comment.
🟡 Changes recommended
Source changes can go undetected, failed builds discard completed samples, and inconsistent memory samples may be accepted.
1 open finding
What changed in this PR
Adds resource-efficiency measurements to the shared benchmark harness.
Changes:
- Measures container memory and image size.
- Adds uncached, multi-round image build timing.
- Documents and tests the new behavior.
[!TIP]
If you aren't ready for review, convert to a draft PR.
Click "Convert to draft" or rungh pr ready --undo.
Click "Ready for review" or rungh pr readyto reengage.
| File | Description |
|---|---|
README.md |
Documents resource and build benchmarks. |
bin/build-images |
Adds uncached image build measurements. |
bench/test_benchmark.rb |
Tests memory sampling and app validation. |
bench/comparison_support.rb |
Adds image/source helpers and cgroup metrics. |
bench/compare.rb |
Records resource metrics in benchmark summaries. |
🧠 Review effort: Balanced
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
…uilds A consistent cgroup sample's working set always contains all anonymous memory, so a smaller value means the counters moved between reads; retry it like a negative sample. bin/build-images now writes each completed build to its own JSON file, so a later failure no longer discards finished builds. summary.json is still written only after every build succeeds. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

The search for the most effective / efficient agentic programming environment should also include memory, docker image size and docker build time.
Results on my old 2 core homelab: