Skip to content

feat(flows): execute fork branches in parallel with join semantics - #501

Open
albertoperdomo2 wants to merge 2 commits into
mainfrom
feat/parallel-flow-execution
Open

albertoperdomo2 wants to merge 2 commits into
mainfrom
feat/parallel-flow-execution

Conversation

@albertoperdomo2

Copy link
Copy Markdown
Contributor

Summary

Implements the fork/join design from PARALLEL-FLOW-EXECUTION.md. Today the runner walks a single cursor and follows only the first outgoing edge of every node (nextEdge() takes getFlowOutgoingTargets(...)[0]), silently dropping the rest; merge is a pass-through, not a join. A fan-out flow that looks correct in the editor executes only its first branch, and downstream {{steps.<dropped-node>.output}} references resolve to null and fail the run with unknown_template_variable.

Changes

  • fork node type{ id, name, type: 'fork', joinNodeId }; joinNodeId explicitly names the paired merge, avoiding ambiguity about which join collects which branches.
  • Parallel execution — when the cursor reaches a fork, the runner spawns one execution fiber per outgoing edge and runs them via Promise.all. Each fiber is an independent traversal (own visited set, own step-list copy, stops at the fork's join via stopBeforeNodeId). When all branches arrive, the parent cursor executes the merge once and continues single-cursor from its outgoing edge.
  • Per-branch OpenCode sessions — a design-doc deviation, forced by the code: a workspace session executes one prompt at a time (createActiveRunAfterRuntimeStateCheck fails with session_busy on a busy session), so branches sharing the parent session would fail spuriously or interleave. Each branch creates its own session (Flow | <name> | <date> · <branch node>); branch steps are recorded per node as usual, which is what the template context consumes.
  • Failure semantics — each fork shares an abort flag: the first branch failure (or termination_unconfirmed) sets it, siblings exit on their next loop iteration, and the fork reports the failure without executing the join. Run cancellation still exits through the existing per-iteration check. Outcome precedence: termination_unconfirmed > failed > cancelled.
  • Template context — fiber step lists are merged per nodeId into the parent after the branches settle, so {{steps.<branchNode>.output}} works in post-join nodes; previousOutput resets to null after a fork (branch outputs are referenced explicitly).
  • Validation (validateFlowDefinition) — new rules: at most one outgoing edge for nodes other than condition/fork (this is the silent-drop bug, now an error — intentionally flags currently-broken flows); join must reference an existing merge and be unique per fork; ≥2 branches and no direct fork→join edge; no branch dead-ends; no human/slack inside a branch region; no join inputs from outside the region.
  • Editor — renaming a join node now remaps fork.joinNodeId (join fields previously fell through unmapped).

Design decisions documented in the OpenSpec change

  • The doc's literal fan-in rule ("merge incoming count == fork outgoing count") is replaced by structurally sound containment rules — a condition node inside a branch legitimately converges several rule targets into the join, which breaks the literal count for a correct graph.
  • human/slack nodes are rejected inside branches for now: pausing one branch while siblings run needs branch-state persistence and a fork-aware resume path.
  • Fork nodes record no run step (the runner intercepts them), so the Prisma FlowNodeType enum and the database need no migration; nodeTypeToPrisma is typed to exclude fork so the compiler keeps that invariant.
  • currentNodeId tracks the single cursor only; a retry after a fork failure resumes from the fork and re-runs the region.

OpenSpec

Adds parallel-flow-execution change with proposal, design, tasks, and a flow-execution spec delta (parallel branch execution, join failure semantics, topology validation, backward compatibility). openspec validate parallel-flow-execution --strict passes.

Test plan

  • validation.test.ts — well-formed fork accepted (nested forks too); one test per rejection rule; multi-edge rejection
  • runner.test.ts — branches run and both outputs render in the post-join template; per-branch session creation; branch failure fails the run without executing the join
  • editor-graph.test.ts — join rename remaps joinNodeId
  • Full flows suite (27 files, 201 tests) passes; eslint clean on changed files
  • tsc --noEmit: no new errors (only the pre-existing tests/providers-* local-env noise, untouched by this diff)
  • CI green

Notes / follow-ups

  • Editor authoring UI for fork nodes (palette entry, join picker, canvas rendering) is deliberately out of scope; fork flows are expressible via templates and the API.
  • ⚠️ fix(flows): keep gateway tokens fresh across flow execution #500 also touches executeFlowNodes (between-step token refresh), so whichever merges second will need a rebase — the changes are complementary, not conflicting in intent.

albertoperdomo2 and others added 2 commits September 2, 2026 11:01
The flow runner walked a single cursor and followed only the first
outgoing edge of every node, silently dropping the rest; `merge` was a
pass-through that neither waited for branches nor aggregated outputs.
Fan-out flows parsed and validated but lost branches at runtime, and
downstream `{{steps.<nodeId>.output}}` references to the dropped nodes
resolved to null.

- Add a `fork` node type (`joinNodeId` names its paired `merge`) and
  execute branches as concurrent fibers in the runner, each with its
  own OpenCode session (a workspace session runs one prompt at a time,
  so sharing the parent session would fail spuriously with
  session_busy), own visited set, and a shared fail-fast flag.
- Merge branch steps into the parent template context after the
  branches settle, so post-join nodes can reference branch outputs;
  the join executes once on the parent cursor and `previousOutput`
  resets to null after a fork.
- First branch failure (or unconfirmed termination) aborts sibling
  branches on their next loop iteration and fails the fork without
  executing the join; run cancellation still exits through the
  existing per-iteration check.
- Validate topology up front: single outgoing edge outside
  condition/fork (the silent-drop bug, now an error), join must be an
  existing unique merge, at least two real branches, no branch dead
  ends, no human/slack nodes inside branches, and no join inputs from
  outside the branch region. Definitions without forks are unchanged.

Fork nodes record no run step, so the Prisma FlowNodeType enum and the
database are untouched. Editor authoring UI for forks is a follow-up.

Co-Authored-By: Claude Code <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant