Skip to content

perf(server): skip already-published task outcomes in maintenance - #105

Open
kvnloo wants to merge 4 commits into
CopilotKit:mainfrom
kvnloo:perf/maintain-outcome-marker-safe
Open

kvnloo wants to merge 4 commits into
CopilotKit:mainfrom
kvnloo:perf/maintain-outcome-marker-safe

Conversation

@kvnloo

@kvnloo kvnloo commented Oct 1, 2026

Copy link
Copy Markdown
Contributor

Problem

maintain() currently calls publishOutcome() for every durable task on every 60s pass. publishOutcome() immediately re-reads the task, so steady-state work grows with lifetime task count rather than active/recoverable work.

A downstream PGlite benchmark of the equivalent skip logic measured:

  • 5k tasks / 50 active: ~8104ms → ~105ms per maintenance pass
  • 20k tasks / 50 active: ~32.4s extrapolated → ~478ms

Change

Record the exact notification outcome key in state.publishedOutcome after publication, and skip rows whose current outcome key already matches that marker.

The marker write uses a new nested JSON compare-and-set primitive instead of replacing the whole state object. That matters because publication can race another same-status state update; sibling state must survive.

Crash semantics stay conservative:

  • terminal state committed but publication missed → no marker, maintenance republishes;
  • notification committed but marker missed → notification dedupe makes replay safe;
  • same-status outcome changes (for example a new waiting_input question) fail the old marker CAS and are republished on the next pass.

Tests

New outcome-level regressions cover:

  • recovery once + zero task re-reads on the next steady-state pass;
  • concurrent sibling state written between notification and marker is preserved;
  • a same-status question change is not hidden by a stale marker.

No connector, UI, or task-execution behavior changes.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant