Skip to content

test: run pytest in strict mode and fail on unexpected warnings - #292

Merged
allen0099 merged 3 commits into
masterfrom
test/strict-pytest-config-189
Sep 27, 2026
Merged

allen0099 merged 3 commits into
masterfrom
test/strict-pytest-config-189

Conversation

@allen0099

@allen0099 allen0099 commented Sep 27, 2026 •

Copy link
Copy Markdown
Owner

Closes #189

Summary

Tightens [tool.pytest.ini_options]:

  • asyncio_mode = "auto": removes all 383 per-test @pytest.mark.asyncio. asyncio_default_fixture_loop_scope = "function" stays.
  • testpaths = ["tests"], --strict-markers and xfail_strict = true.
  • filterwarnings = ["error"]: a warning now fails the test that raised it.

Every warning the suite used to emit is fixed in the test that emits it. The master CI run with live servers reported 57 warnings.

  • Memcached warnings the tests exercise on purpose. These are the clear() flush warning in test_memcached_clear, clear_path(include_params=True), clear_pattern, get_all_keys() and get_cache_data(). The tests now assert them with pytest.warns(..., match=...).
  • Memcached resets in fixtures and setup. They go through a new flush_memcached() in tests/live_servers.py. It silences only the Memcached.clear() flushes ALL warning, and only around that call.
  • Redis clear_pattern("/nonexistent/*"). The test now asserts the path-shaped RuntimeWarning.
  • JWT HS384/HS512 round-trip. It uses a 64-byte key, so neither our short-key UserWarning nor PyJWT's InsecureKeyLengthWarning fires. The dedicated short-key test still covers the warning.
  • Sync tests that called asyncio.run(). There were four in test_cache_invalidate.py, one in test_cache.py and two in session/test_hardening.py, and they are now async tests. Under pytest-asyncio 0.26 (the lowest env), they left an unclosed event loop behind, and its ResourceWarning failed a later test.

docs/DEVELOPMENT.md gets a short "Strict pytest settings" section, and CLAUDE.md's Test Setup gets a note.

Rebased onto master after #291, which added tests/test_examples.py. That file needed no changes: it runs green under these settings.

Live clients are closed inside their test

With "error", the live suite failed because Redis and Memcached clients were dropped without being closed. The garbage collector found them during some later test, and their ResourceWarning failed that test through PytestUnraisableExceptionWarning:

  • Redis: unclosed asyncio transports, mostly from test_redis.py, test_cache_manager.py[RedisBackend] and state/test_manager.py.
  • Memcached: unclosed pool sockets from the concurrency tests in test_memcached.py and from test_cache_status_headers.py.

The backends have no close method, so the tests now close the clients themselves:

  • A new autouse fixture, close_network_clients in tests/conftest.py, records every AsyncRedisCacheBackend and MemcachedBackend a test builds. It records through __new__, so constructor warnings still point at the test. At teardown it closes each client on the test's own loop: await client.aclose() for Redis, client.close() for Memcached.
  • test_cached_route_with_ttl_zero_is_served drives Redis through TestClient, so it closes the client via client.portal, the loop that owns those connections.

No ignores were added for these.

Global ignores

Only one:

  • ignore:The anyio.abc.BlockingPortal alias is deprecated:DeprecationWarning. starlette 1.0.0, our declared floor that the lowest tox env installs, references anyio.abc.BlockingPortal when starlette.testclient is imported, and current anyio deprecates that alias. Later starlette releases don't use it, and neither this package nor its tests do. The warning fires at import time, so no test-level fix is possible.

Verification

  • Live runs (CACHEX_TEST_REDIS_PORT / CACHEX_TEST_MEMCACHED_PORT set, CACHEX_REQUIRE_LIVE_SERVERS=1), each with 0 warnings:
    • uv run pytest and uv run pytest -p no:randomly: 1048 passed each.
    • tox -e lowest (pytest 8.3.5, pytest-asyncio 0.26.0, starlette 1.0.0, redis 5.3.0): 1048 passed.
    • tox -e py310 and tox -e py314: 1048 passed each.
  • Without servers, each with 0 warnings:
    • uv run pytest: 857 passed, 191 skipped.
    • tox -e lowest: 857 passed, 191 skipped.
    • Before this PR it was 845 passed and 190 skipped with 6 warnings, and a live run on master emitted 57 warnings.
  • Negative checks, both reverted afterwards:
    • A temporary warnings.warn("x") in escape_key_component made 99 tests fail.
    • A temporary unregistered marker made collection fail.
  • Leaked resources: a throwaway plugin ran gc.collect() after every test and found none, on either the latest or the lowest environment, live included.

CHANGELOG

None: test-only.

Update

@allen0099
allen0099 force-pushed the test/strict-pytest-config-189 branch from b8aef41 to 65e9844 Compare September 27, 2026 11:16
- asyncio_mode = "auto" and drop the 383 per-test @pytest.mark.asyncio
- testpaths, --strict-markers and xfail_strict = true
- filterwarnings = ["error"]: expected warnings are asserted with
  pytest.warns, live Memcached resets go through flush_memcached, and the
  HS384/HS512 round-trip uses a key long enough for both
- sync tests that called asyncio.run() are async tests now; under
  pytest-asyncio 0.26 they left an unclosed event loop behind
- one global ignore: starlette 1.0.0's deprecated anyio alias, hit only by
  the lowest-dependencies env
With filterwarnings = ["error"], a client a test dropped without closing
failed whichever later test the garbage collector happened to run in:
ResourceWarnings for unclosed asyncio transports (Redis) and sockets
(Memcached pools), surfaced as PytestUnraisableExceptionWarning.

An autouse fixture records each AsyncRedisCacheBackend and MemcachedBackend
as it is built (via __new__, so constructor warnings keep pointing at the
test) and closes its client at teardown on the test's loop. The one sync
test that drives Redis through TestClient closes it on the portal loop that
owns the connections.
The cleanup fixture imported pymemcache and redis at the top of
tests/conftest.py, so the whole suite failed to collect without those
extras. Match the client's type by module name instead.
@allen0099
allen0099 force-pushed the test/strict-pytest-config-189 branch from 65e9844 to baa881d Compare September 27, 2026 11:21
@allen0099
allen0099 merged commit 9f3613f into master Sep 27, 2026
12 checks passed
@allen0099
allen0099 deleted the test/strict-pytest-config-189 branch September 27, 2026 11:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Tests: tighten the pytest configuration

1 participant