diff --git a/.gitignore b/.gitignore index d5114dca..4f20baa2 100644 --- a/.gitignore +++ b/.gitignore @@ -119,3 +119,8 @@ fabric.properties .idea/ *~ *.zip + +# Generated performance test sources, builder output, and measurement results +performance-test/src/main/java/org/javahelpers/simple/builders/performance_test/ +performance-test/generated-performance-builder/ +performance-test/performance-reports/ diff --git a/README.md b/README.md index 034a6271..39902af9 100644 --- a/README.md +++ b/README.md @@ -31,6 +31,7 @@ A zero-reflection Java annotation processor that generates fluent, type-safe bui - [Elementary Builder Example](#elementary-builder-example) - [Full-Featured Examples](#full-featured-examples) - [Advanced Features](#advanced-features) +- [Performance Measurement](#performance-measurement) - [Contributing](#contributing) - [License](#license) - [Acknowledgements](#acknowledgements) @@ -464,6 +465,12 @@ Examples demonstrating special annotations and nested object relationships: These examples serve as both documentation and integration tests for the annotation processor. +## Performance Measurement + +The `performance-test` module measures annotation processing performance across different builder frameworks (Simple Builders, MinimalBuilder, RecordBuilder, and Lombok). It generates ~1000 DTO classes from a JSON catalog, compiles them repeatedly, and aggregates timing results. + +For the full guide — including quick-start, builder type shortcuts, Maven profiles, and wall-time-only mode for external processors — see [Performance Analysis Guide](performance-test/docs/PERFORMANCE_ANALYSIS.md). + ## Contributing Contributions are welcome! Please see [CONTRIBUTING.md](docs/CONTRIBUTING.md) for: diff --git a/docs/CONFIGURATION.md b/docs/CONFIGURATION.md index 624b14ec..04d3981a 100644 --- a/docs/CONFIGURATION.md +++ b/docs/CONFIGURATION.md @@ -20,6 +20,7 @@ Simple-builders supports fine-grained configuration through the `@SimpleBuilder. - [Component Filtering](#component-filtering) - [Integration](#integration) - [Reliability](#reliability) + - [Performance Tracking](#performance-tracking) - [Examples](#examples) - [Minimal Builder](#minimal-builder) - [Internal API Builder](#internal-api-builder) @@ -901,6 +902,71 @@ When enabled, builder and Jackson-module generation failures are promoted from c to errors that fail the build. By default, strict mode is disabled and generation failures are reported as warnings so compilation can continue. +--- + +### Performance Tracking + +#### `performanceTracking` + +**Default**: `false` | **Compiler Option**: `-Asimplebuilder.performanceTracking=true|false` + +Enables performance tracking during annotation processing. When enabled, the processor measures +execution times for each processing phase, method generator, builder enhancer, and per-class +processing. A summary report is logged to the compiler output at the end of processing. + +**When enabled**: A hierarchical performance report is printed to the compiler log, including: +- Total processing time and average time per class +- Phase breakdown (Configuration Resolution, Builder Definition Extraction, DTO Mapping, Code Generation) +- Top 20 slowest classes with field and collection counts +- Top 5 slowest MethodGenerators and BuilderEnhancers + +**When disabled** (default): No performance tracking occurs. The `NoOpPerformanceTracker` is used, +which has zero overhead as the JIT compiler eliminates all tracking calls. + +**Example**: +```bash +# Maven +mvn compile -Dsimplebuilder.performanceTracking=true + +# Or via compiler arg +-Asimplebuilder.performanceTracking=true +``` + +--- + +#### `performanceOutputFile` + +**Default**: *(empty, no file output)* | **Compiler Option**: `-Asimplebuilder.performanceOutputFile=path/to/report.json` + +Specifies a file path where the performance report is written as structured JSON. This is useful +for automated performance analysis and comparison across multiple runs. + +**When set**: In addition to the log output, a JSON file is written containing: +- `timestamp` — ISO-8601 timestamp of the report +- `totalClasses` — number of classes processed +- `totalProcessingTimeNanos` / `totalProcessingTimeSeconds` — total processing time +- `averagePerClassMs` — average processing time per class +- `phaseBreakdown` — hierarchical phase timings with elapsed nanos, seconds, and percentages +- `classMetrics` — per-class metrics (name, elapsed nanos/ms, field count, collection count), sorted by elapsed time descending +- `generatorStats` — per-generator stats (name, elapsed nanos, call count, avg ms/call) +- `enhancerStats` — per-enhancer stats (name, elapsed nanos, call count, avg ms/call) + +**When not set** (default): Only the log report is generated; no JSON file is written. + +**Example**: +```bash +# Maven +mvn compile \ + -Dsimplebuilder.performanceTracking=true \ + -Dsimplebuilder.performanceOutputFile=target/performance-report.json + +# Or via compiler arg +-Asimplebuilder.performanceTracking=true +-Asimplebuilder.performanceOutputFile=target/performance-report.json +``` + +**Note**: `performanceTracking` must be enabled for `performanceOutputFile` to have any effect. + ## Examples ### Minimal Builder @@ -1277,6 +1343,10 @@ methodAccess = AccessModifier.PRIVATE # Reliability -Asimplebuilder.strict=ENABLED|DISABLED + +# Performance Tracking +-Asimplebuilder.performanceTracking=true|false +-Asimplebuilder.performanceOutputFile=path/to/report.json ``` ### Complete Options Example diff --git a/performance-test/docs/PERFORMANCE_ANALYSIS.md b/performance-test/docs/PERFORMANCE_ANALYSIS.md new file mode 100644 index 00000000..985c2e50 --- /dev/null +++ b/performance-test/docs/PERFORMANCE_ANALYSIS.md @@ -0,0 +1,129 @@ +# Performance Analysis Guide + +This guide explains how to run performance analysis for different builder +frameworks using the scripts in the `performance-test` module. + +## Overview + +The performance test module generates a large set of Java classes (records and +plain classes) from a JSON catalog, then compiles them with a builder annotation +processor to measure processing time. Three scripts work together: + +| Script | Purpose | +|---|---| +| `generate_classes.py` | Generate Java source files with a chosen builder annotation | +| `run_performance_measurement.py` | Run N compilations and aggregate timing results | +| `compare_performance.py` | Compare results from multiple measurement runs side-by-side | +| `run_full_comparison.py` | Run all frameworks end-to-end and compare (convenience) | + +## Supported Builder Types + +| `--builder-type` | Annotation | Maven Profile | JSON Report | Notes | +|---|---|---|---|---| +| `simple-builder` | `@SimpleBuilder` | `simplebuilder` | Yes | Full feature set (default) | +| `simple-minimal-builder` | `@MinimalBuilder` | `minimalbuilder` | Yes | All optional features disabled | +| `record-builder` | `@RecordBuilder` | `recordbuilder` | No | RecordBuilder library (wall-time only) | +| `lombok` | `@Builder` | `lombok` | No | Lombok (wall-time only) | + +Builder types with JSON reports produce detailed phase/class/generator/enhancer +metrics. Builder types without JSON reports only measure overall wall time. + +## Quick Start + +Run all four frameworks with N runs each, then compare: + +```bash +python3 scripts/run_full_comparison.py --runs 10 +``` + +Use `--keep-builders` to copy generated builders to `generated-builders//` +so they survive Maven clean: + +```bash +python3 scripts/run_full_comparison.py --runs 30 --keep-builders +``` + +Labels are auto-generated as `sb-runs`, `mb-runs`, `rb-runs`, +`lombok-runs`. + +### Running Individual Frameworks + +For single-framework measurements, use the individual scripts directly. +Use `--force` when regenerating classes after switching builder types. + +```bash +python3 scripts/generate_classes.py --builder-type simple-builder --force +python3 scripts/run_performance_measurement.py --runs 30 --label sb-30runs --builder-type simple-builder +``` + +Results are written to `performance-test/performance-reports/