chore: cli benchmark#2560
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
size-limit report 📦
|
There was a problem hiding this comment.
Pull request overview
Adds a new @lingui/benchmarks workspace providing a CLI-focused benchmark suite (fixture generation + scenario runners) intended to measure extraction/compile/macro-transform performance across different configurations.
Changes:
- Adds a new
benchmarks/workspace with fixture generation, benchmark scenarios, and console/JSON reporting. - Updates the root workspace configuration and lockfile to include benchmark dependencies (SWC, tinybench, lingui-swc, etc.).
- Documents benchmark usage and fixture generation in
benchmarks/README.mdandbenchmarks/CLAUDE.md.
Reviewed changes
Copilot reviewed 23 out of 24 changed files in this pull request and generated 22 comments.
Show a summary per file
| File | Description |
|---|---|
| yarn.lock | Adds workspace + benchmark dependency resolutions (SWC, tinybench, lingui-swc, etc.). |
| package.json | Registers benchmarks/ as a workspace. |
| benchmarks/package.json | Defines @lingui/benchmarks scripts and dependencies. |
| benchmarks/tsconfig.json | Adds a TS config for the benchmarks workspace. |
| benchmarks/README.md | Documents running the benchmark suite and scenarios. |
| benchmarks/CLAUDE.md | Developer guide for architecture/fixtures/scenarios. |
| benchmarks/.gitignore | Ignores generated fixtures/results and SWC artifacts. |
| benchmarks/src/presets.ts | Defines benchmark presets (small/medium/large). |
| benchmarks/src/generate-fixtures.ts | Generates synthetic fixtures and catalogs for benchmarking. |
| benchmarks/src/run-benchmarks.ts | Runs benchmark scenarios and writes aggregated results. |
| benchmarks/src/reporters/console-reporter.ts | Renders console output (bars, throughput, summary). |
| benchmarks/src/reporters/json-reporter.ts | Writes machine-readable benchmark results to JSON. |
| benchmarks/src/scenarios/extract.bench.ts | Benchmarks lingui extract across Babel/SWC and worker counts. |
| benchmarks/src/scenarios/extract-template.bench.ts | Benchmarks lingui extract-template across Babel/SWC and worker counts. |
| benchmarks/src/scenarios/compile.bench.ts | Benchmarks lingui compile across worker counts. |
| benchmarks/src/scenarios/macro-transform.bench.ts | Benchmarks pure Babel vs SWC macro transforms (no CLI). |
| benchmarks/src/utils/run-cli.ts | Runs Lingui CLI in a subprocess with error surfacing. |
| benchmarks/src/utils/config-builder.ts | Writes temporary Lingui config files for fixtures. |
| benchmarks/src/utils/silence.ts | Utility to silence console/stdout during iterations. |
| benchmarks/src/utils/deterministic-random.ts | Seeded RNG utilities (currently unused). |
| benchmarks/src/generators/message-pool.ts | Message pool + variability rules for fixture generation. |
| benchmarks/src/generators/jsx-file-generator.ts | Generates TSX files using Lingui React macros. |
| benchmarks/src/generators/js-file-generator.ts | Generates TS files using Lingui core macros. |
| benchmarks/src/generators/po-catalog-generator.ts | Generates PO catalogs by extracting a real template then filling translations. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
| silence.ts ← Suppresses console/stdout during benchmark iterations | ||
| deterministic-random.ts ← Seeded PRNG (unused currently, available for future use) |
There was a problem hiding this comment.
I would suggest not shipping these unused things. We can always add them separately if needed
| "@lingui/message-utils": "workspace:*", | ||
| "@lingui/swc-plugin": "^6.3.0", | ||
| "@swc/core": "^1.11.0", | ||
| "lingui-swc": "^0.5.2", |
There was a problem hiding this comment.
This is not yet an official Lingui extractor, so better to hold off on using it for benchmarks until it goes live.
|
|
||
| Catalog overlap: 90% of unique messages have pre-existing translations. The remaining 10% simulate newly added code not yet in catalogs (empty translation string). | ||
|
|
||
| ## Adding a New Scenario |
There was a problem hiding this comment.
the example imports buildConfig and silenceConsole, but neither exists in the current code. Scenarios now use writeConfigs() + runLingui()
Description
This PR brings a CLI benchmark
Read readme.md from this PR to know more about it. Also CLAUDE.md has information about how the fixtures are generated.
Types of changes
Fixes # (issue)
Checklist