feat(redis)!: remove the clear_pattern prefix fallback and the encoding option

changelog.d/125.removed.md

@@ -0,0 +1,6 @@
+**Redis `clear_pattern()` no longer retries a pattern with the key prefix
+stripped.** A pattern always matches the logical key, as on every other
+backend, so one that starts with the backend's `key_prefix` now clears only
+logical keys that themselves start with it, and the `DeprecationWarning` is
+gone. Leave the prefix out: `clear_pattern("fastapi_cachex:GET|||*")` becomes
+`clear_pattern("GET|||*")`.

changelog.d/126.removed.md

@@ -0,0 +1,8 @@
+**The Redis `encoding` option is removed; the client reads raw bytes.**
+`AsyncRedisCacheBackend` no longer takes `encoding` or `decode_responses` and
+raises `TypeError` for either. `RedisConfig` has no `encoding` field and now
+ignores one like any other unknown field. Replies go straight to the entry
+codec. Entries were always UTF-8 JSON, so data stored with the default
+encoding reads back unchanged. Remove the argument:
+`AsyncRedisCacheBackend(host="redis", encoding="utf-8")` becomes
+`AsyncRedisCacheBackend(host="redis")`.

docs/BACKENDS.md

@@ -77,9 +77,9 @@ BackendProxy.set(backend)
   given to `clear_path()` are matched literally, so `*`, `?`, `[` or `]` in them cannot
   reach keys outside the prefix or miss the path
 - `clear_pattern()` matches the logical key, the key without the backend prefix, and
-  always adds the prefix itself. Before 0.3.8 a pattern that started with the prefix
-  was matched with the prefix stripped. That form still works when it is the only one
-  that matches anything, with a `DeprecationWarning`, until 0.4.0
+  always adds the prefix itself, so leave the prefix out of the pattern. A pattern that
+  starts with the prefix is not stripped: it matches only logical keys that start with
+  the prefix (see [Migrating to 0.4.0](MIGRATING_0_4.md#redis-clear-pattern))
 
 **Configuring from a model**: `RedisConfig` is a pydantic model with the same
 settings and validation, which is handy when they come from environment
@@ -103,11 +103,9 @@ backend = AsyncRedisCacheBackend.load_from_config(config)
 BackendProxy.set(backend)
 ```
 
-Leave `encoding` out, of both `RedisConfig` and `AsyncRedisCacheBackend`: it is deprecated
-and removed in 0.4.0, and setting it at all emits a `DeprecationWarning`. Entries are always
-written as UTF-8 JSON, and the client decodes replies with `encoding`, so any value other than
-UTF-8 also corrupts non-ASCII content on the way back (with `"latin-1"`, a stored `b"\xe9"`
-reads back as `b"\xc3\xa9"`), and the backend emits a `RuntimeWarning` for it. See
+The client reads raw bytes, and entries are written as UTF-8 JSON. There is no
+`encoding` option: 0.4.0 removed it from both `RedisConfig` and `AsyncRedisCacheBackend`,
+and passing `encoding` or `decode_responses` to the backend raises `TypeError`. See
 [Migrating to 0.4.0](MIGRATING_0_4.md#redis-encoding).
 
 Keep `protocol=2` unless you need RESP3 features *and* your `hiredis` build

docs/MIGRATING_0_4.md

@@ -303,7 +303,7 @@ add_routes(app, dependencies=[], include_content_preview=True)
 
 ### Redis encoding {#redis-encoding}
 
-The Redis client reads raw bytes in 0.4.0, and the `encoding` option is removed from `AsyncRedisCacheBackend` and `RedisConfig` ([#126](https://github.com/allen0099/FastAPI-CacheX/issues/126)). Entries were always written as UTF-8, so leaving it out changes nothing. In 0.3.9, a UTF-8 `encoding` passed to `AsyncRedisCacheBackend` emits a `DeprecationWarning`, and any other value emits only its `RuntimeWarning`, which also announces the removal. A `RedisConfig` that sets `encoding` emits the `DeprecationWarning` when it is passed to `load_from_config()`, plus the `RuntimeWarning` for a value other than UTF-8.
+The Redis client reads raw bytes in 0.4.0, and the `encoding` option is removed from `AsyncRedisCacheBackend` and `RedisConfig` ([#126](https://github.com/allen0099/FastAPI-CacheX/issues/126)). Entries were always written as UTF-8, so leaving it out changes nothing. In 0.3.9, a UTF-8 `encoding` passed to `AsyncRedisCacheBackend` emits a `DeprecationWarning`, and any other value emits only its `RuntimeWarning`, which also announces the removal. A `RedisConfig` that sets `encoding` emits the `DeprecationWarning` when it is passed to `load_from_config()`, plus the `RuntimeWarning` for a value other than UTF-8. In 0.4.0, passing `encoding` or `decode_responses` to `AsyncRedisCacheBackend` raises `TypeError`, and `RedisConfig` ignores an `encoding` value like any other unknown field. `decode_responses` gets no warning in 0.3.9: it only ever accepted `True`, its default, so passing it had no effect.
 
 ```python
 # Before

fastapi_cachex/backends/config.py

@@ -16,14 +16,6 @@ class RedisConfig(BaseModel):
         default=None, description="Redis server password"
     )
     db: int = Field(default=0, ge=0, description="Redis database number")
-    encoding: str = Field(
-        default="utf-8",
-        description=(
-            "Deprecated, removed in 0.4.0: leave it unset. Character encoding "
-            "the client decodes replies with; setting it emits a "
-            "DeprecationWarning in load_from_config()."
-        ),
-    )
     socket_timeout: float = Field(
         default=1.0, description="Timeout for socket operations in seconds"
     )

fastapi_cachex/backends/redis.py

@@ -1,14 +1,11 @@
 """Redis cache backend implementation."""
 
-import codecs
 import logging
 import time
-import warnings
 from collections.abc import Callable
 from collections.abc import Iterable
 from typing import TYPE_CHECKING
 from typing import Any
-from typing import Literal
 
 from fastapi_cachex.backends.codec import decode_entry
 from fastapi_cachex.backends.codec import encode_entry
@@ -79,49 +76,25 @@ def _escape_glob(text: str) -> str:
 """
 
 
-_ENCODING_REMOVED = (
-    "Version 0.4.0 removes it: the client will read raw bytes, and entries "
-    "are always UTF-8. Remove the argument; UTF-8 is what you get without it "
-    "(https://github.com/allen0099/FastAPI-CacheX/issues/126)."
-)
+# Constructor keywords that 0.4.0 removed. They would otherwise fall through
+# **kwargs to redis-py and change how keys and replies are encoded.
+_REMOVED_KWARGS = ("encoding", "decode_responses")
 
 
-def _is_utf8(encoding: str) -> bool:
-    """Whether ``encoding`` names UTF-8 (unknown names count as UTF-8 here).
+def _key_text(key: bytes | str) -> str | None:
+    """A key SCAN returned, as text; ``None`` if it is not UTF-8.
 
-    An unknown name is left for the client to reject itself.
+    The client reads raw bytes, so keys arrive as ``bytes`` (as ``str`` only
+    through a caller-supplied pool that decodes replies). This backend writes
+    every key as UTF-8 text, so a key that is not UTF-8 is none of its own and
+    cannot be named as a logical key.
     """
+    if isinstance(key, str):
+        return key
     try:
-        return codecs.lookup(encoding).name == "utf-8"
-    except LookupError:
-        return True
-
-
-def _warn_encoding(encoding: str) -> None:
-    r"""Warn about an explicitly passed ``encoding``.
-
-    UTF-8 (under any alias such as ``"UTF8"`` or ``"utf_8"``) only gets the
-    ``DeprecationWarning`` for the parameter's removal in 0.4.0. Anything else
-    gets a ``RuntimeWarning``: the shared codec writes UTF-8 JSON, and the
-    client decodes each reply with ``encoding``, so under e.g. latin-1 a stored
-    ``b"\xe9"`` reads back as ``b"\xc3\xa9"``.
-    """
-    if _is_utf8(encoding):
-        warnings.warn(
-            f"AsyncRedisCacheBackend(encoding={encoding!r}) is deprecated. "
-            f"{_ENCODING_REMOVED}",
-            DeprecationWarning,
-            stacklevel=3,
-        )
-    else:
-        warnings.warn(
-            f"AsyncRedisCacheBackend(encoding={encoding!r}) will corrupt non-ASCII "
-            "cached content: entries are always written as UTF-8, and replies "
-            "are decoded with this encoding. Remove the argument (UTF-8 is the "
-            "default); the encoding parameter will be removed in version 0.4.0.",
-            RuntimeWarning,
-            stacklevel=3,
-        )
+        return key.decode("utf-8")
+    except UnicodeDecodeError:
+        return None
 
 
 class AsyncRedisCacheBackend(BaseCacheBackend):
@@ -131,7 +104,7 @@ class AsyncRedisCacheBackend(BaseCacheBackend):
     applications. Keys are namespaced with 'fastapi_cachex:' by default.
     """
 
-    client: "AsyncRedis[str]"
+    client: "AsyncRedis[bytes]"
     key_prefix: str
 
     def __init__(
@@ -140,8 +113,6 @@ def __init__(
         port: int = 6379,
         password: str | None = None,
         db: int = 0,
-        encoding: str | None = None,
-        decode_responses: Literal[True] = True,
         socket_timeout: float = 1.0,
         socket_connect_timeout: float = 1.0,
         key_prefix: str = DEFAULT_REDIS_PREFIX,
@@ -155,13 +126,6 @@ def __init__(
             port: Redis port
             password: Redis password
             db: Redis database number
-            encoding: Deprecated; leave it out. Character encoding the client
-                decodes replies with, UTF-8 when omitted. Entries are always
-                written as UTF-8 JSON, so any other encoding corrupts non-ASCII
-                content on the way back, and a ``RuntimeWarning`` says so.
-                Passing it at all emits a ``DeprecationWarning``: the parameter
-                is removed in 0.4.0.
-            decode_responses: Whether to decode response automatically
             socket_timeout: Timeout for socket operations (in seconds)
             socket_connect_timeout: Timeout for socket connection (in seconds)
             key_prefix: Prefix for all cache keys (default: 'fastapi_cachex:')
@@ -172,6 +136,9 @@ def __init__(
 
         Raises:
             CacheXError: If redis-py is not installed
+            TypeError: If ``encoding`` or ``decode_responses`` is passed; both
+                were removed in 0.4.0. The client always reads raw bytes, and
+                entries are always UTF-8.
         """
         try:
             # Import top-level package first so tests that monkeypatch
@@ -185,10 +152,15 @@ def __init__(
             )
             raise CacheXError(msg) from exc
 
-        if encoding is None:
-            encoding = "utf-8"
-        else:
-            _warn_encoding(encoding)
+        for name in _REMOVED_KWARGS:
+            if name in kwargs:
+                msg = (
+                    f"AsyncRedisCacheBackend() got an unexpected keyword argument "
+                    f"{name!r}: it was removed in version 0.4.0. Remove it; the "
+                    "client reads raw bytes and entries are always UTF-8 "
+                    "(https://github.com/allen0099/FastAPI-CacheX/issues/126)."
+                )
+                raise TypeError(msg)
 
         # `protocol` is not in the types-redis stubs (added in redis-py 5.x).
         # Pass it via **kwargs so mypy doesn't complain about an unknown keyword.
@@ -198,8 +170,9 @@ def __init__(
             port=port,
             password=password,
             db=db,
-            encoding=encoding,
-            decode_responses=decode_responses,
+            # Replies stay bytes: the codec decodes entries itself, and the
+            # Lua compare-and-* scripts get back exactly the bytes read.
+            decode_responses=False,
             socket_timeout=socket_timeout,
             socket_connect_timeout=socket_connect_timeout,
             **kwargs,
@@ -224,23 +197,7 @@ def load_from_config(config: RedisConfig) -> "AsyncRedisCacheBackend":
 
         Returns:
             An instance of AsyncRedisCacheBackend
-
-        Warns:
-            DeprecationWarning: ``config`` sets ``encoding`` explicitly; the
-                field is removed in 0.4.0.
-            RuntimeWarning: That ``encoding`` is not UTF-8, which corrupts
-                non-ASCII content read back (emitted by the constructor).
         """
-        encoding: str | None = None
-        if "encoding" in config.model_fields_set:
-            warnings.warn(
-                f"RedisConfig(encoding={config.encoding!r}) is deprecated. "
-                f"{_ENCODING_REMOVED}",
-                DeprecationWarning,
-                stacklevel=2,
-            )
-            if not _is_utf8(config.encoding):
-                encoding = config.encoding  # keeps the RuntimeWarning
         return AsyncRedisCacheBackend(
             host=config.host,
             port=config.port,
@@ -252,7 +209,6 @@ def load_from_config(config: RedisConfig) -> "AsyncRedisCacheBackend":
             socket_connect_timeout=config.socket_connect_timeout,
             key_prefix=config.key_prefix,
             protocol=config.protocol,
-            encoding=encoding,
         )
 
     async def aclose(self) -> None:
@@ -283,14 +239,16 @@ async def _scan_keys(self, pattern: str) -> list[str]:
         Uses SCAN instead of KEYS so the server is never blocked. SCAN may
         return a key more than once (when the keyspace shrinks mid-iteration),
         so the result is deduplicated, keeping the order keys were first seen.
+        Keys that are not UTF-8 are left out (see ``_key_text``).
         """
         cursor = 0
         keys: dict[str, None] = {}
         while True:
             cursor, page = await self.client.scan(
                 cursor, match=pattern, count=_BATCH_SIZE
             )
-            keys.update(dict.fromkeys(page))
+            texts = (_key_text(key) for key in page)
+            keys.update(dict.fromkeys(text for text in texts if text is not None))
             if cursor == 0:
                 return list(keys)
 
@@ -300,7 +258,8 @@ async def _delete_matching(
         """Delete every key matching ``pattern``, one SCAN page at a time.
 
         With ``keep``, only the matching keys it returns ``True`` for (given
-        the full, prefixed key) are deleted.
+        the full, prefixed key as text) are deleted; a key that is not UTF-8
+        is kept. Without it, every match is deleted, UTF-8 or not.
 
         Each page is deleted as it arrives, so the keyspace is never held in
         memory. Deleting keys SCAN already returned is safe: SCAN still returns
@@ -318,7 +277,11 @@ async def _delete_matching(
                 cursor, match=pattern, count=_BATCH_SIZE
             )
             if keep is not None:
-                page = [key for key in page if keep(key)]
+                page = [
+                    key
+                    for key in page
+                    if (text := _key_text(key)) is not None and keep(text)
+                ]
             if page:
                 deleted += await self.client.delete(*page)
             if cursor == 0:
@@ -509,12 +472,9 @@ async def clear_pattern(self, pattern: str) -> int:
 
         Only ``pattern`` is a live glob; the backend's key prefix is matched
         literally and always added, so ``pattern`` matches the logical key like
-        on every other backend.
-
-        Before 0.3.8 a pattern that started with the key prefix was matched
-        with the prefix stripped instead. When a pattern like that clears
-        nothing, the old form is still tried, and a ``DeprecationWarning`` is
-        emitted if it clears anything. That retry will be removed in 0.4.0.
+        on every other backend. A pattern that itself starts with the key
+        prefix gets it twice, so ``"fastapi_cachex:GET*"`` matches only logical
+        keys that start with ``fastapi_cachex:``.
 
         Args:
             pattern: A glob pattern to match cache keys against
@@ -524,23 +484,6 @@ async def clear_pattern(self, pattern: str) -> int:
         """
         full_pattern = self._prefix_pattern + pattern
         cleared_count = await self._delete_matching(full_pattern)
-        if (
-            cleared_count == 0
-            and self.key_prefix
-            and pattern.startswith(self.key_prefix)
-        ):
-            full_pattern = self._prefix_pattern + pattern.removeprefix(self.key_prefix)
-            cleared_count = await self._delete_matching(full_pattern)
-            if cleared_count:
-                warnings.warn(
-                    f"clear_pattern({pattern!r}) matched only with the backend's "
-                    f"key prefix {self.key_prefix!r} stripped. Patterns match the "
-                    "logical key, without the backend prefix; pass "
-                    f"{pattern.removeprefix(self.key_prefix)!r} instead. The "
-                    "stripped retry will be removed in version 0.4.0.",
-                    DeprecationWarning,
-                    stacklevel=2,
-                )
         warn_if_path_shaped(pattern, cleared_count)
         logger.debug(
             "Redis CLEAR_PATTERN; pattern=%s removed=%s", full_pattern, cleared_count

i18n/zh-TW/docs/BACKENDS.md

@@ -52,7 +52,7 @@ BackendProxy.set(backend)
 - 預設以 `fastapi_cachex:` 前綴建立命名空間；多租戶情境可傳入 `key_prefix="myapp:cache:"`
 - `clear()`、`clear_pattern()` 與 `clear_path()` 只刪除這個後端 `key_prefix` 之下的鍵；同一台伺服器上其他應用程式的鍵不受影響
 - 只有傳給 `clear_pattern()` 的模式是萬用字元（glob）模式。鍵前綴與傳給 `clear_path()` 的路徑都以字面值比對，因此其中的 `*`、`?`、`[` 或 `]` 不會觸及前綴以外的鍵，也不會漏掉該路徑
-- `clear_pattern()` 比對的是邏輯鍵，也就是不含後端前綴的鍵，並一律自行加上前綴。0.3.8 以前，以前綴開頭的模式會先去掉前綴再比對。在只有這種寫法能比對到項目時，它仍可使用，但會發出 `DeprecationWarning`，直到 0.4.0 為止
+- `clear_pattern()` 比對的是邏輯鍵，也就是不含後端前綴的鍵，並一律自行加上前綴，因此模式中不要寫出前綴。以前綴開頭的模式不會被去掉前綴：它只會比對到本身以前綴開頭的邏輯鍵（見[遷移至 0.4.0](MIGRATING_0_4.md#redis-clear-pattern)）
 
 **從模型設定**：`RedisConfig` 是具有相同設定項與驗證的 pydantic 模型，當設定來自環境變數或設定檔時很方便：
 
@@ -74,7 +74,7 @@ backend = AsyncRedisCacheBackend.load_from_config(config)
 BackendProxy.set(backend)
 ```
 
-請不要設定 `encoding`，`RedisConfig` 與 `AsyncRedisCacheBackend` 皆然：它已棄用並將於 0.4.0 移除，只要設定就會發出 `DeprecationWarning`。項目一律以 UTF-8 JSON 寫入，而用戶端會以 `encoding` 解碼回應，因此 UTF-8 以外的值還會在讀回時破壞非 ASCII 內容（使用 `"latin-1"` 時，儲存的 `b"\xe9"` 會讀回成 `b"\xc3\xa9"`），後端也會因此發出 `RuntimeWarning`。請參閱[遷移至 0.4.0](MIGRATING_0_4.md#redis-encoding)。
+用戶端直接讀取原始位元組，項目則以 UTF-8 JSON 寫入。沒有 `encoding` 選項：0.4.0 已從 `RedisConfig` 與 `AsyncRedisCacheBackend` 移除它，傳入 `encoding` 或 `decode_responses` 給後端會引發 `TypeError`。請參閱[遷移至 0.4.0](MIGRATING_0_4.md#redis-encoding)。
 
 除非你需要 RESP3 的功能，*而且*你的 `hiredis` 建置支援它（RESP3 需要 hiredis >= 3.0），否則請保留 `protocol=2`。Redis 8.0 支援 RESP3，但較舊的 hiredis 會無法協商使用它。