Skip to content

feat(proxy): one locked get_or_create for lazy defaults; CacheBackend falls back - #274

Merged
allen0099 merged 1 commit into
masterfrom
feat/proxy-get-or-create-112
Sep 26, 2026
Merged

allen0099 merged 1 commit into
masterfrom
feat/proxy-get-or-create-112

Conversation

@allen0099

Copy link
Copy Markdown
Owner

Summary

Closes #112.

Three dependencies each had their own "use the registered instance, else create a default" logic:

Function Before
get_backend_or_fallback() (proxy.py) its own double-checked lock
get_app_cache() (dependencies.py) its own double-checked lock
get_state_manager() (state/dependencies.py) no lock

Without a lock, concurrent first requests could each build and register a StateManager, and the last set() won. Separately, CacheBackend (get_cache_backend) had no fallback at all. It answered 500 (BackendNotFoundError) until some @cache route had run and installed the MemoryBackend fallback.

Changes

  • ProxyBase.get_or_create(factory):
    • Returns the registered instance. Otherwise it runs factory under the class's lock and registers the result. The factory runs at most once, even with concurrent first callers.
    • If the factory raises, nothing is registered and the error propagates.
    • The lock is a threading.Lock, like the two it replaces, because these are sync FastAPI dependencies that run in worker threads.
    • Each proxy class gets its own lock through __init_subclass__. The default CacheManager is built inside CacheManagerProxy's lock and creates the backend fallback via BackendProxy.get_or_create. A single shared non-reentrant lock would deadlock there.
  • Built on get_or_create: get_backend_or_fallback(), get_app_cache() and get_state_manager(). The two module-level locks are gone.
  • get_cache_backend / CacheBackend now return get_backend_or_fallback(), the same backend @cache and AppCache use.
  • get_state_manager() still has no memory fallback. OAuth states must be readable by whichever worker handles the callback. With no backend it still raises BackendNotFoundError and registers nothing, as STATE.md already documents.
  • Docs:
    • HTTP_CACHING.md "By path or pattern" (EN and zh-TW): CacheBackend now falls back; BackendProxy.get() still raises.
    • CLAUDE.md is updated.

Not breaking: a registered instance is returned exactly as before.

Tests

Scope: uv run pytest -q without live servers (806 passed, 190 live-server tests skipped). Also pre-commit on all files and both strict docs builds.

New and changed tests:

  • tests/test_proxybackend.py, run on a private test proxy:
    • get_or_create returns the registered instance without calling the factory;
    • 8 racing threads run the factory once;
    • a raising factory registers nothing and leaves the lock usable;
    • a factory that calls get_backend_or_fallback() finishes. It runs in a thread with a 5 s join, so a shared lock fails the test instead of hanging.
  • tests/state/test_proxy.py:
    • concurrent first get_state_manager() calls share one instance, using a slow StateManager to widen the race window;
    • with no backend it raises and registers nothing.
  • tests/test_dependencies.py: the old "raises BackendNotFoundError" test became "CacheBackend falls back to a registered MemoryBackend shared with AppCache".

Mutation checks (each fails only the tests listed):

Mutation Failing tests
get_or_create without the lock the new race test, and the existing test_get_app_cache_concurrent_first_calls_share_one_backend
one lock shared by all proxy classes the nested-proxy test
get_cache_backend back to BackendProxy.get() the CacheBackend fallback test
get_state_manager back to the unlocked version the state race test, 2/2 runs

CHANGELOG

Changed

  • CacheBackend falls back to a MemoryBackend like @cache and
    AppCache.
    It used to answer 500 (BackendNotFoundError) until some
    @cache route had registered the fallback. The three lazy defaults
    (get_backend_or_fallback, get_app_cache, get_state_manager) now share
    ProxyBase.get_or_create(factory), which runs the factory at most once
    under a per-class lock; get_state_manager could previously register two
    managers under concurrent first requests. States still have no memory
    fallback. (#112)

… falls back

get_backend_or_fallback() and get_app_cache() each had their own
double-checked lock, and get_state_manager() had none, so concurrent first
requests could each build and register a StateManager. Add
ProxyBase.get_or_create(factory) with a per-class threading.Lock and build
all three on it. The lock is per class so the default CacheManager can
create the fallback backend inside its own creation.

get_cache_backend (CacheBackend) now uses get_backend_or_fallback(), so it
no longer answers 500 until some @cache route has installed the fallback.
get_state_manager still has no memory fallback.

Closes #112
@allen0099 allen0099 added this to the 0.3.8 milestone Sep 26, 2026
@allen0099 allen0099 added enhancement New feature or request fastapi-integration FastAPI dependencies, middleware and request handling labels Sep 26, 2026
@allen0099
allen0099 merged commit 7d20980 into master Sep 26, 2026
11 checks passed
@allen0099
allen0099 deleted the feat/proxy-get-or-create-112 branch September 26, 2026 23:04
allen0099 added a commit that referenced this pull request Sep 27, 2026
Copies the CHANGELOG sections of #207, #208, #209, #211, #212, #272,
#273, #274, #275, #276, #279 and #284 into Unreleased. The #273 entry
drops expire_if_equals from its list of methods that changed, since that
primitive is new in 0.3.8.
allen0099 added a commit that referenced this pull request Sep 27, 2026
Copies the CHANGELOG sections of #207, #208, #209, #211, #212, #272,
#273, #274, #275, #276, #279 and #284 into Unreleased. The #273 entry
drops expire_if_equals from its list of methods that changed, since that
primitive is new in 0.3.8.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request fastapi-integration FastAPI dependencies, middleware and request handling

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Unify lazy get-or-create for the proxies; CacheBackend and get_state_manager lack fallback and locking

1 participant