feat: .parallel() — concurrent step groups (closes #20)#31
Merged
Conversation
- Extract 180-line parallel block into executeParallelGroup() with typed ParallelGroupResult return, reducing executeRun nesting - Add single-branch parallel group test - Add engine.cancel() during parallel execution test - Add sequential parallel groups prev-chaining test - Add @internal to BranchFailedError
Resume parallel groups per-branch instead of all-or-nothing: completed branches are skipped on retry and only the missing ones re-run, so side effects are not duplicated. Failed/completed rows now upsert in place rather than appending a new row each attempt. Track which branch caused the group to abort so onRunFailed / onFailure report the underlying cause, not a sibling that was aborted by signal propagation. Switched to Promise.allSettled so concurrent rejections do not surface as unhandled rejections. Bail out of the retry loop once the group is aborted to avoid pointless iterations. Adds tests covering per-branch resume, no-duplicate rows, hook semantics on resume, cause-of-abort reporting, no-unhandled-rejection, and no-retry-after-abort.
- README: new "Parallel Steps" section with semantics callouts and a workflow.parallel() API reference entry - docs/docs.html: matching section + API card with nav links - docs/llms.txt: parallel section + API entry kept in sync - examples/4-parallel-enrichment.ts: runnable fan-out/fan-in example - CHANGELOG: entry under 0.3.0 crediting @brianjenkins94 (#20)
0.3.0 has already been released — restore that section to its released contents and create a new 0.4.0 entry for the .parallel() work.
I am your repository's number 1 fan 🤣 This seems like a solid solution. I generally found |
CI's coverage check was failing because engine.ts dropped to ~87% lines after adding executeParallelGroup. Two small changes get us back over: - Remove three unreachable defensive blocks: the 'early-complete' branch in executeParallelGroup (guardedComplete throws ParallelCompleteError, not EarlyCompleteError, so executeStep never returns 'early-complete' for parallel branches), the EarlyCompleteError escape check in the outer catch (same reason), and the lastError-RunControlError rethrow in executeStep (RunControlError is rethrown inside the retry loop, never stored in lastError). - Add three targeted tests for genuinely-reachable paths: lease loss during success-path persistence (storage returns false from saveStepResult mid-loop), cancellation landing before the parallel group starts, and heartbeat loss mid-branch (the cause-branch fallback recognising a RunControlError reject). Coverage: 93.83% lines / 93.68% statements / 84.03% branches.
Owner
Author
|
Correction: it's actually all 5 substantive feature issues on the repo — I miscounted. With this one merged that's 2 of your 5 ideas already shipped (this + the start hooks in 0.3.0), and #21/#23/#24 are sitting in your draft #27. Genuinely appreciate the steady stream of well-thought-out reports — they've been doing a lot of the product-thinking for me. Thanks for the quick look. Merging now. 🚀 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
workflow.parallel({ branchName: handler | config, ... })for running independent steps concurrently. The next step'sprevis a typed merged record of all branch outputs.onRunFailed/onFailuresurface the branch that caused the abort, not a sibling aborted by signal propagation).ParallelCompleteErrorthrown if a branch callscomplete().This closes #20, originally suggested by @brianjenkins94. Thanks again Brian — three of the last four feature ideas have come from you.
Design note — and a question for @brianjenkins94
Your issue referenced GitHub Actions'
needs:model, which is an arbitrary DAG. I went with a structured.parallel()group instead. Reasoning:prev(a free-form DAG breaks that — what's the prev type when multiple disjoint paths could precede a step?).A → (B, C, D) → E, which is ~90% of the "parallel" use cases I could think of.What it cannot express that
needs:could:Ddepends only onA, runs alongside an independentB → Cchain).Does
.parallel()cover your actual workload? If you have a case in mind that genuinely needs arbitrary-DAG expressiveness, I'd rather know now than ship.parallel()and have it not fit. Happy to revisit the design before merging.What's in the diff
src/core/workflow.ts—.parallel()builder,ExecutionUnitmodel.src/core/engine.ts—executeParallelGroup, per-branch resume,Promise.allSettled, cause-tracking, abort-aware retry bail.src/core/__tests__/parallel-engine.test.ts— 25 tests covering builder, types, fail-fast, retries, timeouts, hooks, crash recovery, cancellation, sequential parallel groups.examples/4-parallel-enrichment.ts— runnable fan-out/fan-in example.README.md,docs/docs.html,docs/llms.txt— new "Parallel Steps" section + API reference.CHANGELOG.md— entry under0.4.0.Test plan
bun run test— 291/291 passbun run typecheck— cleanbun run build— cleanoxlint— cleannpx tsx examples/4-parallel-enrichment.ts— runs end-to-end in ~1.2s (vs ~3.2s sequential)