Skip to content

perf: speed up the worker, fix a long-line timeout, make the benchmark like for like - #86

Merged
rlorenzo merged 21 commits into
mainfrom
perf/worker-speedups
Oct 20, 2026
Merged

rlorenzo merged 21 commits into
mainfrom
perf/worker-speedups

Conversation

@rlorenzo

@rlorenzo rlorenzo commented Oct 20, 2026 •

Copy link
Copy Markdown
Owner

What

Speeds up the PHP worker, fixes a timeout on very long single-line arrays, and makes the benchmark compare like with like. Behaviour is unchanged: the per-rule finding counts on all 11 benchmark codebases are identical to v1.6.0, and just check passes on every commit.

Why

Running the bake-off on v1.6.0 failed on Google Site Kit: wordpress/array-alignment took 26-32 s on the Symfony polyfill data files (21k-character single-line arrays), past Mago's 30 s worker timeout. Profiling the worker found the rest of the overhead and a benchmark that did not match what it claimed.

Commits (one per improvement)

  • fix(array-alignment): measure a long line's columns incrementally. 26 s to 0.2 s on the polyfill files.
  • perf(presets): start the worker with opcache.enable_cli=1. 8-15% less CPU on large plugins; harmless without opcache.
  • perf(rules), perf(strings), perf(leading-comment), perf(phpcs-comments), perf(class-comment, file-comment): replace substr_count from byte 0, strrpos(substr(...)) prefix copies and a linear comment scan with LineIndex, a negative-offset strrpos and a map by end offset. Mostly matters on large files.
  • perf(calls), perf(empty-catch-comment), perf(escape-output), perf(superfluous-whitespace): unwrap one argument, skip files with nothing to check, reuse the shared token stream. superfluous-whitespace is a conservative pre-check; its reports are identical on all 11 codebases (420 lines).
  • refactor(rules): one shared PhpToken::tokenize of the file (FileTokens::raw) for the rules that tokenized it separately.
  • bench, docs: phpcs runs with opcache too and with the full WordPress standard (the extension ports 183 of its 186 sniffs, so that is the matching comparison); results and charts regenerated.

Numbers

CPU time against v1.6.0 (opcache plus the code changes): Yoast SEO 20.6 s to 18.0 s, WooCommerce 106 s to 94 s, Elementor 32 s to 28 s, about 12%. Most of that is opcache.

Bake-off, phpcs WordPress against mago + extension: 92.6 s vs 52.5 s overall (1.8x); 2.1-2.7x on the large plugins, about even on the small ones.

The old README figure (2-12x, 8.1x overall) came from a benchmark that ran only the 46 rules the extension had then. It now runs all 186, so the README and docs/benchmarks.md are updated to the measured numbers.

Not done

Profiling shows no single hotspot left: shared structure builds are about 27% of rule time and the rest is spread over ~185 rules. Merging the four phpcs-style tokenizers would save roughly 10% but risks WPCS parity, so it is left for a separate change.

Review

CodeRabbit's missing-docs finding is fixed: the timed rules now include Mago's own missing-docs, which covers the WordPress-Docs sniffs. The suggestion to alternate tool order is not taken: both tools get a warm-up and three runs, and the machine ran other work equally for both. CI on Mago 1.47.1 failed on an array-alignment static local that the older analyzer read as never; it is a typed property now.

@rlorenzo

Copy link
Copy Markdown
Owner Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Oct 20, 2026 •

Copy link
Copy Markdown
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@coderabbitai

coderabbitai Bot commented Oct 20, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

📝 Walkthrough

The PR updates the WordPress-versus-Mago benchmark to use PHPCS WordPress rules and publishes results from October 20, 2026. It enables CLI OPcache in worker commands and changes linting internals to share tokenization and source-position lookups, with additional rule scan and argument/comment lookup updates.

Changes: Benchmark refresh

Layer / File(s) Summary
Enable CLI OPcache for benchmark workers
bench/lib.sh, bench/run.sh, bin/generate-presets.php, wordpress-core.mago.toml, wordpress-extra.mago.toml, wordpress.mago.toml
Benchmark and generated worker commands enable PHP CLI OPcache.
Update benchmark scope and results
bench/bakeoff.sh, bench/run.sh, bench/results/2026-10-20-bakeoff.md, docs/benchmarks.md, README.md
The benchmark compares PHPCS WordPress rules with Mago’s default WordPress extension rules. The new report and documentation record timings and lint counts for 11 codebases, with an overall 1.7× speed-up.

Linting internals

Layer / File(s) Summary
Share tokenization and source-position lookups
src/Internal/*, src/Linter/Rules/*
A cached FileTokens helper supplies PHP tokens. Internal helpers and lint rules use shared newline or line-index lookups instead of local scans.
Change lint-rule scan paths
src/Linter/Rules/ArrayAlignmentRule.php, src/Linter/Rules/EmptyCatchCommentRule.php, src/Linter/Rules/SuperfluousWhitespaceRule.php, src/Linter/Rules/EscapeOutputRule.php, src/WordPressExtension.php
Array alignment memoizes width scans. Empty-catch and whitespace checks add early checks. EscapeOutputRule uses Spacing::tokens() and is registered earlier.
Update argument and comment lookup
src/Internal/Calls.php, src/Internal/LeadingComment.php
Argument lookup scans for the requested positional argument. Leading-comment lookup uses trivia indexed by span end.

Priority: ➖ Normal

Estimated code review effort: 3 (Moderate) | ~25 minutes

Change: Bug fix

Merge Risk: 🔵 Low · up to 4f3f1

The linting changes appear safe, but the published benchmark omits Mago's documentation rule while PHPCS runs it. The stated 1.7× comparison therefore overstates parity. Correct the rule selection and rerun the benchmark before publishing the README and docs figures.

Pre-merge checks: Passed 4
Check name Status Explanation
Title check Passed The title clearly summarizes the pull request's three main changes: worker performance improvements, the long-line timeout fix, and an equivalent benchmark comparison.
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Description check Passed The pull request description directly explains the performance improvements, timeout fix, benchmark changes, validation results, and deferred work reflected in the changeset.
✨ Finishing Touches: 💡 1
🛠️ Fix failing CI checks: 💡
  • Commit to this branch
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2


ℹ️ Review info
⚙️ Run configuration
  • Configuration used: Organization UI
  • Review profile: ASSERTIVE
  • Plan: Advanced
  • Run ID: fe2dc9bc-583d-4843-8d44-383326cfc7a7
📥 Commits

Reviewing files that changed from the base of the PR and between 0342f01 and 4f3f1d8.

⛔ Files ignored due to path filters: 2
  • docs/benchmarks-dark.svg is excluded by !**/*.svg
  • docs/benchmarks-light.svg is excluded by !**/*.svg
📒 Files selected for processing: 40
  • README.md
  • bench/bakeoff.sh
  • bench/lib.sh
  • bench/results/2026-10-20-bakeoff.md
  • bench/run.sh
  • bin/generate-presets.php
  • docs/benchmarks.md
  • src/Internal/BlankLinesAfterComment.php
  • src/Internal/Calls.php
  • src/Internal/DocBlockScan.php
  • src/Internal/FileTokens.php
  • src/Internal/ItemShift.php
  • src/Internal/JoinedStatement.php
  • src/Internal/LeadingComment.php
  • src/Internal/LineIndents.php
  • src/Internal/PhpText.php
  • src/Internal/PhpcsComments.php
  • src/Internal/ScopeColon.php
  • src/Internal/Strings.php
  • src/Linter/Rules/ArrayAlignmentRule.php
  • src/Linter/Rules/ClassCommentRule.php
  • src/Linter/Rules/ClassInstantiationRule.php
  • src/Linter/Rules/DisallowInlineTabsRule.php
  • src/Linter/Rules/DisallowLonelyIfRule.php
  • src/Linter/Rules/DisallowMultipleStatementsRule.php
  • src/Linter/Rules/EmptyCatchCommentRule.php
  • src/Linter/Rules/EndFileNewlineRule.php
  • src/Linter/Rules/EscapeOutputRule.php
  • src/Linter/Rules/FileCommentRule.php
  • src/Linter/Rules/FileHeaderRule.php
  • src/Linter/Rules/FunctionCallSignatureRule.php
  • src/Linter/Rules/FunctionCommentRule.php
  • src/Linter/Rules/InlineControlStructureRule.php
  • src/Linter/Rules/NonExecutableCodeRule.php
  • src/Linter/Rules/SeparateFunctionsFromOORule.php
  • src/Linter/Rules/SuperfluousWhitespaceRule.php
  • src/WordPressExtension.php
  • wordpress-core.mago.toml
  • wordpress-extra.mago.toml
  • wordpress.mago.toml
💤 Files with no reviewable changes: 1
  • src/Internal/PhpText.php

Included review availability: This review used your included allowance. Your plan provides up to 1 included review per hour; 0 remain after this review.

Comment thread bench/run.sh Resolved
Comment thread bench/run.sh
echo "| Tool | Mean of 3 runs |"
echo "|:---|---:|"
timed "phpcs WordPress-Extra (WPCS 3.4.1, --parallel=8)" phpcs
timed "phpcs WordPress (WPCS 3.4.1, --parallel=8)" phpcs

@coderabbitai coderabbitai Bot Oct 20, 2026 •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🔵 Trivial | 🏗️ Heavy lift

Alternate the tool order during timed samples.

This call starts every PHPCS sample before any Mago sample for the same codebase. Fixed order can make cache state, temperature, or CPU clock changes appear as a tool difference. Alternate the order across samples and record the executed order with the timings. (raw.githubusercontent.com) Based on learnings, fixed sequential benchmark order can systematically bias a comparison through cache, thermal, and clock-frequency drift.

Source: Learnings

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not changing this. Each tool gets a warm-up run and the mean of three timed runs, so cache state is the same for both, and the machine ran the same background load throughout. The order is fixed so the logged output matches the old results files.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@rlorenzo Thanks for the clarification. The fixed order preserves the layout of earlier results files. I accept your decision to keep it.

The order-bias concern is a benchmark-methodology caveat, not a demonstrated defect in these results. I won’t press for a change.


✏️ Learnings added
Learnt from: rlorenzo
Repo: rlorenzo/mago-wordpress PR: 86
File: bench/run.sh:115-115
Timestamp: 2026-10-20T12:16:42.185Z
Learning: In rlorenzo/mago-wordpress, bench/run.sh intentionally benchmarks PHPCS before Mago so the logged output matches earlier results files. Each tool receives one warm-up run followed by three timed runs. Preserve this intentional ordering unless the benchmark requirements change.

Note: Learnings are effective only in the context of similar code segments. To apply general coding guidelines, please configure review instructions. You can manage existing learnings in the app.

You are interacting with an AI system.

@rlorenzo
rlorenzo merged commit f1b4b3b into main Oct 20, 2026
19 checks passed
@rlorenzo
rlorenzo deleted the perf/worker-speedups branch October 20, 2026 12:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant