Skip to content

fix(cache): run sync handlers in the threadpool - #134

Merged
allen0099 merged 1 commit into
masterfrom
fix/cache-sync-handler-threadpool
Sep 25, 2026
Merged

allen0099 merged 1 commit into
masterfrom
fix/cache-sync-handler-threadpool

Conversation

@allen0099

Copy link
Copy Markdown
Owner

Closes #100.

Problem

@cache wraps every handler in an async wrapper, so FastAPI no longer sends a def handler to its threadpool. get_response() then called the sync handler inline, on the event loop, and blocking I/O in it stalled every other request on the worker.

The coroutine check also missed an object with an async def __call__, and a sync callable that returns a coroutine; both produced an un-awaited coroutine that then failed to encode.

Change

  • get_response() awaits coroutine handlers and runs everything else with starlette.concurrency.run_in_threadpool.
  • New _is_coroutine_callable(): inspect.iscoroutinefunction(func) (which already unwraps functools.partial), or the same check on type(func).__call__. A class counts as sync, because its type's __call__ is type.__call__.
  • An awaitable result is awaited as a safety net.

A note on the issue text: a functools.partial of an async function was already handled, since inspect.iscoroutinefunction unwraps partials. It stays in the test as a regression case.

Tests

  • test_sync_handler_runs_off_the_event_loop_thread records the thread ident of a sync handler and an async handler, and asserts they differ.
  • test_handler_returning_a_coroutine_is_awaited is parametrized over a partial, an async __call__ object, and a sync function returning a coroutine.

Against the old get_response(), 3 of these 4 cases fail; the partial case passes.

Checks

  • ruff check / format on fastapi_cachex, tests and scripts
  • mypy strict on the package; mypy on tests and scripts
  • pytest against live Redis and Memcached (CACHEX_REQUIRE_LIVE_SERVERS=1): 720 passed, 100% coverage
  • zensical build --strict

The CHANGELOG entry sits at the top of ### Fixed, next to #133's, so whichever merges second needs a trivial rebase.

The @cache wrapper is async, so FastAPI no longer offloads a def handler
and get_response() called it on the event loop. Run sync handlers with
run_in_threadpool, detect an async __call__ on the handler's type, and
await any awaitable a sync callable returns.

Closes #100
@allen0099
allen0099 merged commit 0c06e6a into master Sep 25, 2026
10 checks passed
@allen0099
allen0099 deleted the fix/cache-sync-handler-threadpool branch September 26, 2026 11:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

@cache runs sync handlers on the event loop

1 participant