Skip to content

Parallelize CI tests with verified complete coverage - #478

Merged
kensac merged 3 commits into
mainfrom
perf/test-suite
Sep 9, 2026
Merged

kensac merged 3 commits into
mainfrom
perf/test-suite

Conversation

@kensac

@kensac kensac commented Sep 9, 2026 •

Copy link
Copy Markdown
Owner

Test setup dominates the serial suite. Run three deterministic ordinary-test shards and one corpus lane, each with its own PostgreSQL database. Preserve the required check gate by comparing all executed test IDs against a full same-revision collection manifest. Keep existing static, migration and extension checks and mechanical autofixes.

Two CI trials on revision 3271201 matched all 1,471 cases and outcomes. Including provisioning and uploads, parallel jobs completed in 135s versus 201s serial, and 90s versus 214s serial. These are test-job spans, not whole-workflow timings; the sample is small and parallel execution uses more total runner time.

Share lane execution with a manual benchmark workflow, retain phase timing artifacts, and document measurements and the next fixture improvements in docs/testing-performance.md. Setup took 132.81s and 145.02s in the serial trials; optimize reseeding next.

Validation: benchmark coverage/outcome comparisons passed twice; ten focused orchestration regressions pass, including a new manifest regression confirmed failing against the preceding implementation. Full parallel CI passed on the final revision, including exact coverage verification for 1,472 cases. The first complete parallel CI workflow finished in 156s from creation to completion versus 259s for the preceding serial workflow, including static checks and orchestration overhead. This comparison spans the added manifest regression and is not a controlled same-revision benchmark.

@kensac kensac changed the title Profile test setup and compare isolated CI shards Parallelize CI tests with verified complete coverage Sep 9, 2026
@kensac
kensac merged commit 1662e29 into main Sep 9, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant