Repository navigation
Parallelize CI tests with verified complete coverage - #478
Merged
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Test setup dominates the serial suite. Run three deterministic ordinary-test shards and one corpus lane, each with its own PostgreSQL database. Preserve the required
checkgate by comparing all executed test IDs against a full same-revision collection manifest. Keep existing static, migration and extension checks and mechanical autofixes.Two CI trials on revision 3271201 matched all 1,471 cases and outcomes. Including provisioning and uploads, parallel jobs completed in 135s versus 201s serial, and 90s versus 214s serial. These are test-job spans, not whole-workflow timings; the sample is small and parallel execution uses more total runner time.
Share lane execution with a manual benchmark workflow, retain phase timing artifacts, and document measurements and the next fixture improvements in docs/testing-performance.md. Setup took 132.81s and 145.02s in the serial trials; optimize reseeding next.
Validation: benchmark coverage/outcome comparisons passed twice; ten focused orchestration regressions pass, including a new manifest regression confirmed failing against the preceding implementation. Full parallel CI passed on the final revision, including exact coverage verification for 1,472 cases. The first complete parallel CI workflow finished in 156s from creation to completion versus 259s for the preceding serial workflow, including static checks and orchestration overhead. This comparison spans the added manifest regression and is not a controlled same-revision benchmark.