feat(report): add batch benchmarking, plot generation, and reproducible Docker environment

.dockerignore

@@ -0,0 +1,6 @@
+build
+results
+docs/figures
+.git
+*.pyc
+__pycache__

.gitmessage.txt

@@ -17,7 +17,6 @@
 #   - shipovnik
 #   - hypericum
 #   - kryzhovnik
-#   - iaik_merkle_tree
 # - Ensures reproducible builds with pinned submodule commits
 #
 # chore(repo): add .gitignore and basic CI workflow

Dockerfile

@@ -0,0 +1,35 @@
+FROM ubuntu:22.04
+
+ENV DEBIAN_FRONTEND=noninteractive
+
+RUN apt-get update && apt-get install -y \
+    build-essential \
+    cmake \
+    gcc \
+    git \
+    libcppunit-dev \
+    libntl-dev \
+    libssl-dev \
+    make \
+    pkg-config \
+    python3 \
+    python3-pip \
+    python3-matplotlib \
+  \
+  && rm -rf /var/lib/apt/lists/*
+
+WORKDIR /opt/batch-pqc
+
+# Copy repository into the image
+COPY . /opt/batch-pqc
+
+# Build the project inside the image.
+# Note: submodules are not initialized in the image build context. Ensure
+# submodules are checked out on the host before building the image (or add
+# them to the build context). Attempting to run `git submodule` here will
+# usually fail because `.git` is excluded from the Docker build context.
+RUN mkdir -p build \
+ && cmake -S . -B build \
+ && cmake --build build --parallel
+
+CMD ["/bin/bash"]

README.md

@@ -30,7 +30,7 @@ The goal is to measure the speedup of signing a batch of messages with a single
   - [Shipovnik](https://github.com/QAPP-tech/shipovnik_tc26)
   - [Hypericum](https://github.com/QAPP-tech/hypericum_tc26)
   - [Kryzhovnik](https://github.com/ElenaKirshanova/pqc_LWR_signature)
-- Sequential benchmark (`bench_seq`) with configurable paramsets, verification pass, and warmup iterations
+- Sequential and batch benchmarks (`bench_seq`, `bench_batch`) with configurable paramsets, verification pass, and warmup iterations
 - Test suite with unified `test_*` naming in CTest
 
 ## Original Repos and Forks
@@ -102,8 +102,8 @@ git submodule update --init --recursive
 # Run tests
 ./scripts/third_party.sh tests
 
-# Run benchmark (sequential signing)
-./build/bench/bench_seq --algo hypericum --batch-size 16 --iters 100 --verify 1
+# Run benchmark comparison (sequential + batch + plots)
+./scripts/benchmark.sh --algo hypericum --batch-sizes 1,4,16 --iters 100 --verify 1
 ```
 
 ### Paramset Selection
@@ -140,8 +140,11 @@ Note: Kryzhovnik constants are aligned with `security.sage` for `small/medium/la
 
 After running benchmarks, the `results/` directory will contain:
 
-- `raw_data.csv` – timings for each (algorithm, batch_size)
-- `plots/speedup_vs_batch.png` – graph for all three algorithms
+- `bench-<timestamp>/*_seq.csv` and `bench-<timestamp>/*_batch.csv` – raw benchmark slices
+- `bench-<timestamp>/raw_data.csv` – aggregated timings for each `(algorithm, batch_size)`
+- `bench-<timestamp>/plots/speedup_vs_batch.png` – graph for all three algorithms
+
+- Note: automated PDF generation has been removed from this repository; the final report is maintained as Markdown at `docs/report.md`. To produce a PDF locally, install `pandoc` and a LaTeX distribution and convert the Markdown manually.
 
 ## Current Status (MVP)
 
@@ -151,8 +154,8 @@ After running benchmarks, the `results/` directory will contain:
 - [x] Sequential benchmark `bench_seq` with warmup and corrected signature-size metric
 - [x] Merkle-based batch signer/verifier implementation
 - [x] Test coverage: `test_adapters_smoke`, `test_adapters_batch`, `test_kryzhovnik*`, `test_merkle`, `test_batch_signing`, `test_streebog`
-- [ ] Sequential vs real batch signing benchmark comparison
-- [ ] Final report (PDF)
+- [x] Sequential vs real batch signing benchmark comparison
+- [x] Final report (`docs/report.md`)
 
 **MVP is being developed in `dev` branch. After completion, a Pull Request to `main` will be opened for review.**
 

README_ru.md

@@ -33,7 +33,7 @@
   - [Шиповник](https://github.com/QAPP-tech/shipovnik_tc26)
   - [Гиперикум](https://github.com/QAPP-tech/hypericum_tc26)
   - [Крыжовник](https://github.com/ElenaKirshanova/pqc_LWR_signature)
-- Последовательный бенчмарк (`bench_seq`) с параметрами paramset, verify-проходом и прогревочными итерациями
+- Последовательный и пакетный бенчмарки (`bench_seq`, `bench_batch`) с параметрами paramset, verify-проходом и прогревочными итерациями
 - Набор тестов с унифицированными именами `test_*` в CTest
 
 ## Оригинальные репозитории и форки
@@ -107,8 +107,8 @@ git submodule update --init --recursive
 # Запуск тестов
 ./scripts/third_party.sh tests
 
-# Запуск бенчмарка (последовательная подпись)
-./build/bench/bench_seq --algo hypericum --batch-size 16 --iters 100 --verify 1
+# Запуск бенчмарков (последовательная и пакетная подпись + графики)
+./scripts/benchmark.sh --algo hypericum --batch-sizes 1,4,16 --iters 100 --verify 1
 ```
 
 ### Выбор paramset
@@ -145,8 +145,9 @@ cmake --build build --parallel
 
 После запуска бенчмарков в директории `results/` появятся:
 
-- `raw_data.csv` – замеры времени для каждого (алгоритм, размер пакета)
-- `plots/speedup_vs_batch.png` – график ускорения для всех трёх алгоритмов
+- `bench-<timestamp>/*_seq.csv` и `bench-<timestamp>/*_batch.csv` – исходные замеры
+- `bench-<timestamp>/raw_data.csv` – агрегированные замеры времени для каждого `(алгоритм, размер пакета)`
+- `bench-<timestamp>/plots/speedup_vs_batch.png` – график ускорения для всех трёх алгоритмов
 
 ## Текущий статус (MVP)
 
@@ -156,8 +157,8 @@ cmake --build build --parallel
 - [x] Последовательный бенчмарк `bench_seq` с warmup и исправленной метрикой размера подписей
 - [x] Реализация пакетной подписи/верификации на основе дерева Меркла
 - [x] Покрытие тестами: `test_adapters_smoke`, `test_adapters_batch`, `test_kryzhovnik*`, `test_merkle`, `test_batch_signing`, `test_streebog`
-- [ ] Сравнение бенчмарков: последовательная vs реальная пакетная подпись
-- [ ] Финальный отчёт (PDF)
+- [x] Сравнение бенчмарков: последовательная vs реальная пакетная подпись
+- [x] Финальный отчёт (`docs/report.md`)
 
 **MVP разрабатывается в ветке `dev`. После завершения будет открыт Pull Request в `main` для ревью.**
 

bench/CMakeLists.txt

@@ -3,3 +3,9 @@ add_executable(bench_seq bench_seq.c)
 target_link_libraries(bench_seq PRIVATE batch_bench_adapters)
 
 target_compile_features(bench_seq PRIVATE c_std_99)
+
+add_executable(bench_batch bench_batch.c)
+
+target_link_libraries(bench_batch PRIVATE batch_adapters batch_bench_adapters streebog)
+
+target_compile_features(bench_batch PRIVATE c_std_99)