Skip to content

Add opt-in parallel setup across nodes - #107

Open
bgrewell wants to merge 2 commits into
mainfrom
feature/parallel-setup
Open

Add opt-in parallel setup across nodes#107
bgrewell wants to merge 2 commits into
mainfrom
feature/parallel-setup

Conversation

@bgrewell

@bgrewell bgrewell commented Aug 9, 2026

Copy link
Copy Markdown
Owner

Setup steps ran strictly sequentially, so two nodes with slow provisioning
waited on each other for no reason. Each node's chain now runs concurrently
with the others when --parallel-setup is passed: three nodes with a three
second chain each finish in three seconds rather than nine.

Steps for one node stay in order. A node's own steps routinely depend on each
other, so only different nodes overlap.

It is opt-in rather than the default. Sequential order across nodes is the
only ordering mechanism a suite has, so a suite whose step on one node depends
on a step having finished on another is relying on that order without
declaring it. Enabling this by default would break those suites silently,
which is the failure mode this codebase keeps removing.

Two flags force the sequential path and say why: --pause-on-error reads stdin
and cannot prompt for concurrent steps, and --until stops at a named step,
which needs an order to stop in.

Concurrent tasks render without a spinner. The animated completer owns a
terminal line and registers as the process's single active spinner, so two at
once overwrite each other; a quiet completer writes each task's line once, on
completion, under a shared mutex so lines never interleave.

A failure skips the rest of its own node's chain but not the other nodes'.
Every failure is reported and the run exits with the first in node-declaration
order, so the reported error does not depend on scheduling.

Setup steps ran strictly sequentially, so two nodes with slow provisioning
waited on each other for no reason. Each node's chain now runs concurrently
with the others when --parallel-setup is passed: three nodes with a three
second chain each finish in three seconds rather than nine.

Steps for one node stay in order. A node's own steps routinely depend on each
other, so only different nodes overlap.

It is opt-in rather than the default. Sequential order across nodes is the
only ordering mechanism a suite has, so a suite whose step on one node depends
on a step having finished on another is relying on that order without
declaring it. Enabling this by default would break those suites silently,
which is the failure mode this codebase keeps removing.

Two flags force the sequential path and say why: --pause-on-error reads stdin
and cannot prompt for concurrent steps, and --until stops at a named step,
which needs an order to stop in.

Concurrent tasks render without a spinner. The animated completer owns a
terminal line and registers as the process's single active spinner, so two at
once overwrite each other; a quiet completer writes each task's line once, on
completion, under a shared mutex so lines never interleave.

A failure skips the rest of its own node's chain but not the other nodes'.
Every failure is reported and the run exits with the first in node-declaration
order, so the reported error does not depend on scheduling.
Rendering goes through the line builder and completer factory the caller
supplies; the formatter itself was never referenced, which the test made
visible by passing nil.

Also records why the parallel path prefixes its errors with the node name
while the sequential path does not: parallel failures are reported together
after every group finishes, detached from the task lines that identify the
node.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant