Skip to content

feat: detect event loop blocking in development - #7672

Open
harsh21234i wants to merge 11 commits into
reflex-dev:mainfrom
harsh21234i:fix/7619-event-loop-blocking
Open

harsh21234i wants to merge 11 commits into
reflex-dev:mainfrom
harsh21234i:fix/7619-event-loop-blocking

Conversation

@harsh21234i

@harsh21234i harsh21234i commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Warn in development when an event blocks the asyncio event loop.

  • A watchdog thread puts a probe on the loop every 1–10 ms. If the probe does not run within event_loop_blocking_threshold seconds (default 0.2), the loop is blocked.
  • While the loop is still blocked, the monitor reads asyncio.current_task(loop) and gets the handler from the event task's name (reflex_event|<event>|…). This covers the whole event, including computed vars recalculated after the handler.
  • The first warning appears at once, so a handler that never returns is reported too. It names the handler and the innermost line of app code, then shows the stack of the event task. A second line gives the total duration when the loop responds.
  • Detection is on by default in development and always off in production. Set event_loop_blocking_threshold=0, or REFLEX_EVENT_LOOP_BLOCKING_THRESHOLD=0, to turn it off.
  • There is no cost on the event path. The monitor uses about 1% of one core while idle.

Example:

Event handler State.http_sync is blocking the event loop at my_app/my_app.py:115 in http_sync (threshold: 0.200s), including state updates such as computed vars. Move blocking I/O to a thread pool or use a non-blocking async API.
Blocking stack (most recent call last):
  ...
  File ".../my_app/my_app.py", line 115, in http_sync
    urllib.request.urlopen(SLOW_URL).read()
  ...
  File ".../socket.py", line 729, in readinto
    return self._sock.recv_into(b)
Event handler State.http_sync blocked the event loop for 0.501s in total.

Closes #7619

Testing

  • Unit tests for the monitor, the event processor, the config and the app wiring. The tests cover sync, async and post-handler blocking, a hung handler, a stale frame snapshot, threshold 0, and the env var.
  • Dev server with 11 blocking workloads: sync sleep, CPU loop, sqlite, pandas, sync HTTP, a blocking async handler, a background event, a helper function and a slow computed var. All 11 are detected with the correct handler and app line, on Python 3.11 and 3.14, with asyncio, uvloop and rloop. There were no warnings for awaiting handlers or blocks shorter than the threshold.
  • The durations agree with py-spy within 30 ms.
  • Event throughput with the monitor on is 97.6–99.1% of the throughput with it off.

@harsh21234i
harsh21234i requested review from a team and Alek99 as code owners October 27, 2026 14:25

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 9 files

Reply with feedback, questions, or to request a fix.

View guided diff | Re-trigger cubic

Comment thread news/+7619.feature.md
Comment thread packages/reflex-base/src/reflex_base/event/processor/blocking.py
Comment thread tests/units/test_config.py
Comment thread reflex/app.py
Comment thread packages/reflex-base/src/reflex_base/config.py Outdated
@greptile-apps

greptile-apps Bot commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerView in GreptileConfidence Score: 4/5

[Medium impact] The warnings affect event processing in development apps.

The code appears safe to merge once the new test helper meets the repository’s docstring requirement.

Findings

  1. P2 New helper lacks a docstring ▶

Summary

Adds development-time warnings for event handlers that block the asyncio loop.

  • Blocked event handlers now produce stack and duration warnings.
  • Apps can set the delay that triggers blocking warnings.

Reviews (11) · Last reviewed commit: "feat: disable blocking detection with 0 ..." · Reviewed by Greptile

Comment thread reflex/app.py
Comment thread packages/reflex-base/src/reflex_base/config.py Outdated
Comment thread packages/reflex-base/src/reflex_base/event/processor/blocking.py Outdated
Comment thread packages/reflex-base/src/reflex_base/event/processor/blocking.py Outdated
@codspeed

codspeed Bot commented Oct 27, 2026 •

Copy link
Copy Markdown

Merging this PR will degrade performance by 4.2%

1 warning

⚠️ 20 benchmarks are dominated by system calls

System calls cannot be simulated consistently and are left out of the measure. The reported value understates the real cost. Switch to the Walltime instrument to measure them.

Learn more about system calls

See affected benchmarks

❌ 1 regressed benchmark
✅ 147 untouched benchmarks

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

Benchmark BASE HEAD Efficiency
❌ test_isinstance_container[list_optional] 283.1 ms 295.5 ms -4.2%

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.


Comparing harsh21234i:fix/7619-event-loop-blocking (e192a31) with main (db9af76)

Open in CodSpeed

Comment thread packages/reflex-base/src/reflex_base/event/processor/blocking.py Outdated

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 3 files (changes from recent commits).

Reply with feedback, questions, or to request a fix.

View guided diff | Re-trigger cubic

Comment thread tests/units/reflex_base/event/processor/test_blocking.py Outdated
Comment thread tests/units/reflex_base/event/processor/test_blocking.py Outdated
@masenf masenf added on deck PRs lined up to review / merge next documentation Improvements or additions to documentation labels Oct 27, 2026
harsh21234i and others added 3 commits October 28, 2026 16:30
Name the event task running on the blocked loop instead of tracking
handler frames per event, so blocking in computed vars after the handler
is reported. Log after the loop responds with the real duration. Remove
the per-event hooks and the duplicated otel branch, and simplify the
threshold validation.
Comment thread packages/reflex-base/src/reflex_base/event/processor/blocking.py Outdated

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 7 files (changes from recent commits).

Reply with feedback, questions, or to request a fix.

View guided diff | Re-trigger cubic

Comment thread packages/reflex-base/src/reflex_base/event/processor/blocking.py
Capture the loop thread's frames from the event task's entry to the
blocking line when the threshold is crossed, so the warning points at
the code that blocked, including computed vars and C-extension calls.
Comment thread packages/reflex-base/src/reflex_base/event/processor/blocking.py Outdated
Log the handler and its stack as soon as the threshold is reached, so a
handler that never returns is reported, then log the total duration once
the loop responds. Report nothing when the frame snapshot no longer
contains the event task's entry frame.
Comment thread tests/units/reflex_base/event/processor/test_blocking.py
Use 0 instead of None to turn detection off, so
REFLEX_EVENT_LOOP_BLOCKING_THRESHOLD=0 works: the env parser cannot
produce None for optional fields. Name the innermost frame of app code
(under the app root, outside the stdlib and installed packages) in the
first warning line, so library frames do not hide the line to fix.
Comment on lines +715 to +717
async def _execute_event(self: EventProcessor, **kwargs):
await execute_event(self, **kwargs)
time.sleep(0.12) # noqa: ASYNC251

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 New helper lacks a docstring

The new _execute_event helper has no docstring. The repository guide requires Google-style docstrings on all functions, including nested test helpers. Add a short summary explaining that this helper simulates blocking after the handler returns. This requirement must be satisfied before merging.

Suggested change
async def _execute_event(self: EventProcessor, **kwargs):
await execute_event(self, **kwargs)
time.sleep(0.12) # noqa: ASYNC251
async def _execute_event(self: EventProcessor, **kwargs):
"""Simulate blocking after the event handler returns."""
await execute_event(self, **kwargs)
time.sleep(0.12) # noqa: ASYNC251

Context Used: CLAUDE.md (source)

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

test code

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is test code, but CLAUDE.md requires Google-style docstrings on all functions and doesn’t exempt test helpers. Since this nested function simulates blocking after the handler returns, the one-line docstring is still appropriate.

Comment on lines +82 to +83
if task is None or not (name := task.get_name()).startswith(EVENT_TASK_PREFIX):
return None

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we probably should still report blocking for any task, not just reflex_event| tasks. that way if there is an error with the lock waiter or the lost and found subscriber, those can still be surfaced here instead of detecting the blocking, but not reporting anything. The message can be slightly different when we detect blocking in an arbitrary task.

Comment on lines +699 to +702
assert (
"is blocking the event loop at "
"tests/units/reflex_base/event/processor/test_event_processor.py:"
) in caplog.text

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

use platform-aware path separators.

Suggested change
assert (
"is blocking the event loop at "
"tests/units/reflex_base/event/processor/test_event_processor.py:"
) in caplog.text
exp_path = Path("tests", "units", "reflex_base", "event", "processor", "test_event_processor.py")
assert (
f"is blocking the event loop at {path}:"
) in caplog.text

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation on deck PRs lined up to review / merge next

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Implement event loop blocking detection

3 participants