Skip to content

PHP: test-file evidence on PHPUnit 9 and 10, and a coverage run that loads pcov once - #558

Merged
pdbethke merged 3 commits into
mainfrom
fix/phpunit9-unit-listing
Nov 19, 2026
Merged

pdbethke merged 3 commits into
mainfrom
fix/phpunit9-unit-listing

Conversation

@pdbethke

Copy link
Copy Markdown
Owner

What was wrong

  • corral listed PHPUnit's test files with --list-test-files, which exists only from PHPUnit 11. On 9.6 and 10.5 the listing stopped at Unknown option "--list-test-files", so a project on those versions got no unit evidence: files were paired by name only and graded by the whole suite. ramsey/uuid, the PHP project in the ledger, is on PHPUnit 9.6.
  • The PHP coverage run named extension=pcov even when PHP already loads it. The second load prints a warning on stderr at every start, and PHPUnit fails a test it runs in a separate process when that process writes to stderr.
  • A failed unit listing printed the runner's whole output three times (units line, selection line, summary line).

What changed

  • The listing is --list-tests, run under a small setup that has PHP print the file each test class was loaded from (Runner.ListUnitsSetup, new). One listing for PHPUnit 9.6 to 12.5.
  • PHPUnit 9 and 10 ignore --group / --exclude-group when listing and say so; that sentence is read as no listing, so such a command is graded by the whole suite as before.
  • The coverage ini names pcov only when a probe under the same ini scan path finds it missing.
  • The two later lines give the one-line reason; the units line keeps the output.
  • VerdictGeneration 47.
  • scripts/listunitscmd prints a row's listing script, so the fixture capture runs the row's own setup instead of a copy.

Measured

  • Listing and one-file runs on PHPUnit 9.6.39, 10.5.67, 11.5.57 and 12.5.44 (PHP 8.5.4), fixtures captured from each; the real-runner test passes on all four with CORRALAI_RUNNER_ROOT set.
  • ramsey/uuid at 7a16331, certify --repo --dry-run --evidence in the bwrap jail: released build lists nothing; with the listing fix 67 test files are listed and 59 give evidence; with pcov loaded once all 67 do, and 11 source files are paired by evidence.
  • Full go test ./... and every gate were green on the listing commit; after the pcov commit and the last changelog edit I reran internal/lang, internal/reposcan, ^TestDocs, the doc generators' checks, licensing and the security gate, not the whole suite.

Not covered

  • Only the PHP lines of capture-runner-fixtures.sh were run, by hand; the whole script was not.
  • No audit with models was run on a PHPUnit 9 or 10 project.
  • No PHPUnit older than 9.6 and no PHP older than 8.5.
  • The GitHub runner where pcov was first found missing under the scan path was not re-run; that branch is tested with a stand-in php.

🤖 Generated with Claude Code

pdbethke and others added 3 commits November 18, 2026 17:30
…st class's file, because --list-test-files exists only from PHPUnit 11 and a project on 9 or 10 (ramsey/uuid among them) got no unit evidence at all; and print a failed listing's output once, not three times.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
… it under the injected ini scan path, because the second load's start-up warning on stderr failed every test PHPUnit runs in a separate process (eight ramsey/uuid test files gave no unit evidence).

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…easured on ramsey/uuid: 59 of 67 test files gave evidence with the listing fix alone and all 67 with pcov loaded once.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
@pdbethke
pdbethke merged commit 10110b6 into main Nov 19, 2026
10 checks passed
@github-actions github-actions Bot locked and limited conversation to collaborators Nov 19, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant