Skip to content

sdk: concurrent f.llm calls beyond ~5 are dispatched after their 30s lease has expired #561

Description

@khaliqgant

Problem

With flows run --local-agent on relayflows 2.0.29, a Promise.all over 9 f.llm calls fails every time. A body with 5 passes. It needs no application code:

// runtime-parallel-llm-repro.flow.ts
const xs = await Promise.all([1,2,3,4,5,6,7,8,9].map((i) =>
  f.llm(`Return {"x": ${i}+${input.n}} as JSON only.`, { output: s, model: "claude-haiku-4-5-20251001" })));
FAILED [protocol_error] relayflowd could not complete the run request: Agent lease is already expired for 01M365AS4PJ4NRG4VFG684CEFV/llm-3.

From the failing child's journal (capacity 1), attempt 1 was started with a 30s lease and never heartbeated. The kernel journaled lease_expired at +30s and retried, and attempt 2 succeeded. The worker's withWorkerLease then threw already expired at its first check. In other words, the worker got the dispatch after the lease had run out.

Suspect: WorkerSlots (authored-worker-step.ts) is meant to hold overflow calls before run.start, so the kernel never leases a step the worker can't take. Yet queued calls end up holding kernel leases they don't use. Either admission isn't bounded the way the comment in worker-slots.ts says, or the LlmWorker's dispatch handling is delayed. This regressed or survived #554, which is in 2.0.29.

Scope

Find out why a queued LLM dispatch reaches the worker after its lease deadline, and fix it so N concurrent f.llm calls run at most capacity at a time with no expired leases. The fatal-error half is #560. This issue is about never producing the stale dispatch in the first place.

Acceptance

  • A test that runs 9+ concurrent f.llm calls against the real kernel with a fake CLI completes with no lease_expired attempt in any child journal, at capacity 1 and at the default.
  • The repro above passes locally.
  • examples/prompt-lab can restore Promise.all in jobs/shared.ts runEngine and jobs/new-agency.ts (feat(examples): prompt-lab — the Prompt Lab product brief as one relayflow #559).
  • Mutation-verified per AGENTS.md.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    garden-readyScoped and ready for an agent to pick up

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions