Skip to content

Feed incremental engine streams instead of recomputing the buffer - #100

Merged
Maharsh17 merged 2 commits into
mainfrom
perf/incremental-worker-state
Sep 25, 2026
Merged

Maharsh17 merged 2 commits into
mainfrom
perf/incremental-worker-state

Conversation

@Maharsh17

Copy link
Copy Markdown
Member

Each event now feeds a RunDistanceStream, a SessionSegmenter and the goal stream together, in one place, so a recompute reads their state instead of redoing the whole buffer. The output is the same as the batch calls give over the buffer until it wraps. A test checks exactly that.

What changes

This bumps the engine to f03612b, which is main again with the scaling fixes from agent-lm-packages #8. The humanize and hat tests are updated for the engine's explicit hat list and its not connected section.

All 267 tests pass, including test_run_identity and test_audit_fixes.

Closes #99

… the buffer

Each event now feeds a RunDistanceStream, a SessionSegmenter and the goal stream in lockstep, so a recompute reads their state instead of re-parsing every run and re-segmenting the whole buffer. Run indices count from the worker's first event, so they stay joined to the goal stream after the rolling buffer wraps. The unused per-event list is dropped from the episodes payload, and stored goal runs are released except the latest. Renderer tests are updated for the explicit hat list and the not connected section.
@Maharsh17
Maharsh17 merged commit c268049 into main Sep 25, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Workers recompute the whole buffer on every event

1 participant