Skip to content

perf(events): SSE dirty-flag + tear-proof job serialization - #305

Merged
thcp merged 1 commit into
mainfrom
feat/sse-dirty-flag
Jul 17, 2026
Merged

perf(events): SSE dirty-flag + tear-proof job serialization#305
thcp merged 1 commit into
mainfrom
feat/sse-dirty-flag

Conversation

@thcp

@thcp thcp commented Jul 17, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • PR-3A of Phase 3 (Performance). Adds Job.version, bumped by _set() on every field write. Internal-only (excluded from to_state() and to_record(), same as cancel_requested).
  • job_events's SSE loop now compares job.version to the last-sent version instead of re-serializing + string-diffing to_state() on every 0.2s tick — idle connections do one int compare per tick instead of a full to_state() + json.dumps(). At the 200-connection cap that removes ~1,000 serializations/s.
  • Closes Non-atomic multi-field job updates visible to SSE #285 for real: if job.version changes while to_state() is mid-call (a concurrent _set() from the pipeline thread), the snapshot may mix pre- and post-write fields — a torn read. The loop now detects that (version read before vs. after serializing) and discards the snapshot instead of yielding it, retrying immediately rather than sleeping.
  • Already-terminal jobs (done/error/cancelled) close the stream right after the initial snapshot instead of idling on int-compares.

Closes #289

Test plan

  • New tests/test_events_stream.py drives the SSE generator directly: initial snapshot always sent, no duplicate frames while state is static, a frame is emitted on _set() and the stream closes after a terminal transition, a simulated torn read (monkeypatched to_state() bumping version mid-call) is discarded and re-serialized consistently, an already-terminal job closes after one frame.
  • Full suite: 188 passed, 12 skipped
  • ruff check / ruff format --check clean

Adds Job.version, bumped by _set() on every field write. The SSE stream
now compares versions instead of re-serializing + string-diffing on every
0.2s tick -- idle connections drop from a full to_state()+json.dumps per
tick to one int compare, eliminating ~1,000 serializations/s at the
200-connection cap.

Also closes #285 for real: if job.version changes while to_state() is
mid-call, the snapshot may mix pre- and post-write fields (a torn read).
The stream loop now detects that (version read before vs. after
serializing) and discards the snapshot instead of yielding it, retrying
immediately.

Already-terminal jobs (done/error/cancelled) now close the stream right
after the initial snapshot instead of idling.

Closes #289
@thcp
thcp merged commit c896b9c into main Jul 17, 2026
8 checks passed
@thcp
thcp deleted the feat/sse-dirty-flag branch July 17, 2026 10:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

SSE polls at 5 Hz per connection Non-atomic multi-field job updates visible to SSE

1 participant