Skip to content

Run BATS files in parallel by default#8

Merged
c0da-ricon merged 7 commits into
mainfrom
junior/parallel-bats-default
Jul 17, 2026
Merged

Run BATS files in parallel by default#8
c0da-ricon merged 7 commits into
mainfrom
junior/parallel-bats-default

Conversation

@junior-ricon

@junior-ricon junior-ricon commented Jul 17, 2026

Copy link
Copy Markdown
Contributor

Summary

  • declare Rush 0.9.0 and run BATS files concurrently with a measured four-job default
  • keep tests within each file serial to avoid BATS 1.13's one-second semaphore polling
  • preserve CLI/environment job and runner overrides plus explicit serial debugging
  • add task-level precedence/failure tests and a real cross-file concurrency barrier
  • document parallel-state isolation for repositories copied from Template

Closes #4.

Design

Effective jobs follow caller --jobs, then BATS_NUMBER_OF_PARALLEL_JOBS, then the four-job default. Rush is required only when effective jobs exceed one. The task forwards caller arguments unchanged and adds --no-parallelize-within-files for parallel runs.

Within-file parallelism is intentionally excluded. A causal disposable BATS experiment changed only its semaphore poll from 1s to 10ms and reduced a 32-test single-file profile from 17.012s to 2.788s at two jobs and 7.849s to 1.740s at four jobs. The upstream/fork opportunity is tracked separately in #7.

Measurement

Five measured runs per candidate followed one warmup, with rotated/reversed order. Timing covers the BATS suite only, after tool setup. Profiles use isolated test-local state.

Host/profile 1 job 2 jobs 3 jobs 4 jobs 8 jobs
Local 8-CPU tiny 2.298s 1.497s 1.037s 0.906s
Local 8-CPU shell/wait 4.357s 2.351s 1.326s 0.873s
Ubuntu 4-CPU tiny 1.936s 1.237s 1.103s
Ubuntu 4-CPU shell/wait 3.735s 2.026s 1.270s
macOS 3-CPU tiny 3.704s 2.885s 2.316s 2.159s
macOS 3-CPU shell/wait 6.482s 3.694s 2.920s 2.264s

Four jobs was the smallest hosted candidate within roughly 10% of the best median across both profiles and platforms. It reduced the measured hosted profiles by about 42–66% versus serial execution. Raw hosted evidence is in run 29583418980; local raw JSON remains in Junior's Template#4 benchmark workbench. The temporary hosted benchmark workflow and fixtures are absent from the final head.

Validation

  • focused task contract: 10/10 BATS
  • full default-parallel suite: 13/13 BATS in 4.1s locally
  • full serial opt-out suite: 13/13 BATS in 7.0s locally
  • real cross-file concurrency barrier passed
  • all 9 configured codebase lints
  • README generation check
  • Bash syntax and git diff --check
  • temporary hosted benchmark and ordinary tests passed on Ubuntu/macOS
  • final exact-head run 29584113017 passed at 072f507: Ubuntu test step 2s / job 11s; macOS test step 6s / job 33s

Boundaries

No downstream repository adoption, Template#6 change, BATS fork, or merge authority is implied by this PR.

@junior-ricon
junior-ricon marked this pull request as ready for review July 17, 2026 13:30
@c0da-ricon

Copy link
Copy Markdown
Contributor

Fix-it #9 addresses one structural/correctness concern in this branch:

  • .mise/tasks/test currently owns the entire command workflow rather than remaining a thin public adapter.
  • Its raw second-pass option scan misclassifies valid BATS option values. For example, mise run test --filter --jobs fails because --jobs is treated as a wrapper control instead of the filter regex; suite-shaped filter values can also be rewritten as paths.

#9 moves the workflow to libexec/test, makes option-value handling context-aware, adds focused regressions, and updates Template's structural guidance. It also temporarily removes the direct-invocation bats-test-task lint; the generic lint split is tracked in codebase#78.

Exact signed fix-it head: 5f084f0.

@c0da-ricon
c0da-ricon merged commit 121a8af into main Jul 17, 2026
2 checks passed
@c0da-ricon
c0da-ricon deleted the junior/parallel-bats-default branch July 17, 2026 15:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Run BATS suites in parallel by default

2 participants