feat(report): add batch benchmarking, plot generation, and reproducible Docker environment - #6
Open
cherninkiy wants to merge 7 commits into
Open
feat(report): add batch benchmarking, plot generation, and reproducible Docker environment#6cherninkiy wants to merge 7 commits into
cherninkiy wants to merge 7 commits into
Conversation
…cript Add bench_batch.c and update CMakeLists; enhance scripts/benchmark.sh with batch mode
Add scripts/plot_results.py to aggregate seq/batch CSV and render plots; embed figures in docs/report.md; update README; include copied PNGs in docs/figures for portability
Provide reproducible container build and wrapper to run benchmarks with mounted results/docs
Remove iaik_merkle_tree from third_party docs/comments; update git message template and third_party orchestration
There was a problem hiding this comment.
Code Review
This pull request completes the MVP for the batch PQC benchmarking project by introducing a new batch signing benchmark, a comprehensive reporting and plotting pipeline, and Docker support for reproducible runs. Key changes include the removal of the external Merkle tree submodule and the addition of a detailed final report. Feedback focuses on addressing a potential integer overflow in memory allocation, resolving a conflict between Docker build steps and .dockerignore, improving constant definitions for hash sizes, and ensuring build isolation when using Docker volumes.
…sh size from bits constant Check cfg->batch_size * cfg->msg_size for SIZE_MAX overflow before malloc; define BATCH_HASH_BITS and derive BATCH_HASH_SIZE.
…ule expectations Submodules must be initialized on the host; remove git submodule commands that fail due to .dockerignore excluding .git
…nflicts Mount named Docker volume 'batch-pqc_build' to /opt/batch-pqc/build so host build artifacts don't interfere with container builds.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR introduces several major improvements to the benchmarking workflow, reporting, and development environment:
bench_batch.cand batch mode inscripts/benchmark.shto measure performance under batch processing.scripts/plot_results.pyaggregates sequential and batch CSV results, generates plots, and embeds them intodocs/report.md. Figures are also copied todocs/figures/for portability.Dockerfile,docker_runhelper, andinstall_deps.shto provide a reproducible containerized build and benchmark environment with mounted results/docs.iaik_merkle_treesubmodule, updatedthird_partyorchestration and git message template accordingly.Motivation
Testing
Related
Let me know if you'd like to adjust the tone or add any specific details.