Skip to content

Commit b45650e

Browse files
committed
fix(cache): percent-encode | and % in cache-key host and path
The default key builder joined the raw Host header and decoded path with |||, so a Host of 'example.com|||/p' on GET /x stored the response under the key of GET /p%7C%7C%7C/x. Encode | and % in both components with escape_key_component; clear_path encodes its argument the same way and the monitoring routes decode for display. Recommend TrustedHostMiddleware in the HTTP caching guide. Closes #230
1 parent 1d2bd73 commit b45650e

13 files changed

Lines changed: 216 additions & 12 deletions

File tree

‎CHANGELOG.md‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -232,6 +232,16 @@ Note that 0.3.3 was never released; 0.3.4 follows 0.3.2.
232232
then drop at once, and reports only non-numeric values as "not a counter".
233233
`@cache` rejects such a `ttl` when the decorator is applied.
234234
([#229](https://github.com/allen0099/FastAPI-CacheX/issues/229))
235+
- **A `|||` in the `Host` header or path can no longer poison another
236+
path's cache entry.** The default key builder joined the raw host and
237+
decoded path with `|||`, so `GET /x` with `Host: example.com|||/p` stored
238+
its response under the key of `GET /p%7C%7C%7C/x`. `|` and `%` in the host
239+
and path are now percent-encoded (`escape_key_component` in
240+
`fastapi_cachex.types`); `clear_path()` encodes its argument the same way
241+
and the monitoring routes decode for display. Keys whose host or path
242+
contains `|` or `%` change, so those entries are cached afresh once. The
243+
HTTP caching guide now recommends `TrustedHostMiddleware`.
244+
([#230](https://github.com/allen0099/FastAPI-CacheX/issues/230))
235245

236246
## [0.3.7] - 2026-09-25
237247

‎CLAUDE.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -53,7 +53,7 @@ The library has four independent subsystems:
5353
- Cache flow: check `no-store` → check `no-cache` → check ETag (`If-None-Match`) → check TTL-based cache hit → execute handler → store result.
5454
- Fails open by default (`fail_open=True`): a backend error on `get` is logged and treated as a miss, one on `set` is logged and the response served unstored. `fail_open=False` propagates the error.
5555
- Only GET requests are cached; other methods bypass the cache entirely.
56-
- Cache keys follow the format `method|||host|||path|||query_params` (separator defined in `types.py`).
56+
- Cache keys follow the format `method|||host|||path|||query_params` (separator defined in `types.py`). Host and path go through `escape_key_component` (`|` → `%7C`, `%` → `%25`) so client input cannot inject the separator; `clear_path` encodes its argument and `routes.py` decodes for display.
5757
- `BackendProxy` is a non-instantiable class-level singleton (via `ProxyMeta`). Call `BackendProxy.set(backend)` at app startup; `BackendProxy.get()` raises `BackendNotFoundError` if unset. Falls back to `MemoryBackend` automatically inside `@cache` if no backend is set.
5858
- Cache values are stored as `CacheEntry(fingerprint, content, media_type)` dataclass (defined in `types.py`).
5959

‎docs/CACHE_FLOW.md‎

Lines changed: 13 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -49,10 +49,16 @@ When a request arrives, the `@cache` decorator does the following:
4949

5050
```python
5151
from fastapi_cachex.types import CACHE_KEY_SEPARATOR # "|||"
52+
from fastapi_cachex.types import escape_key_component
5253

5354
# Cache key format (default_key_builder in fastapi_cachex/cache.py)
5455
cache_key = CACHE_KEY_SEPARATOR.join(
55-
[request.method, request.headers.get("host", "unknown"), request.url.path, query]
56+
[
57+
request.method,
58+
escape_key_component(request.headers.get("host", "unknown")),
59+
escape_key_component(request.url.path),
60+
query,
61+
]
5662
)
5763

5864
# For example:
@@ -64,6 +70,12 @@ The separator is `|||` rather than a colon because the host itself may contain a
6470
port (`127.0.0.1:8000`); with a colon the key could not be split reliably, and
6571
`clear_path()` needs to recover the path from the key.
6672

73+
The host and path are percent-encoded first: `|` becomes `%7C` and `%` becomes
74+
`%25` (`escape_key_component` in `fastapi_cachex/types.py`). Both come from the
75+
client, and a raw `|||` in either would shift the components so that one
76+
request's key could equal another's. The query string is URL-encoded already.
77+
The monitoring routes decode them again for display.
78+
6779
Query parameters are joined in the order the request sent them
6880
(`str(request.query_params)`) and are **not sorted**, so `?page=1&limit=10` and
6981
`?limit=10&page=1` are two separate cache entries. If you want them treated as

‎docs/HTTP_CACHING.md‎

Lines changed: 26 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -137,6 +137,28 @@ This ensures that:
137137
Query parameters are taken in the order the client sent them, without sorting, so
138138
`?a=1&b=2` and `?b=2&a=1` are two distinct cache entries for the same logical request.
139139

140+
The host and path come from the client, so `|` and `%` in them are percent-encoded
141+
(`%7C` and `%25`). A `Host` header or path containing `|||` therefore cannot shift
142+
the components and make one request's key equal another's. The query string is
143+
URL-encoded already. `clear_path()` takes the path as your application sees it
144+
(`request.url.path`) and encodes it the same way; `clear_pattern()` matches the
145+
stored key, so write `%7C` there for a `|`. Before 0.3.8 both were stored as sent,
146+
so after upgrading, entries for a host or path containing `|` or `%` are cached
147+
afresh once.
148+
149+
The host is still whatever the client sends. Unless a reverse proxy or load
150+
balancer in front of the app already rejects unknown hosts, add Starlette's
151+
`TrustedHostMiddleware`, so a forged `Host` gets a `400` instead of filling the
152+
cache with entries no one else will request:
153+
154+
```python
155+
from starlette.middleware.trustedhost import TrustedHostMiddleware
156+
157+
app.add_middleware(
158+
TrustedHostMiddleware, allowed_hosts=["example.com", "*.example.com"]
159+
)
160+
```
161+
140162
All backends automatically namespace keys with a prefix (e.g., `fastapi_cachex:`)
141163
to avoid conflicts with other applications. `CacheManager` (see
142164
[Application cache](APP_CACHE.md)) uses a separate, simpler `cache:`-prefixed key
@@ -166,6 +188,7 @@ from fastapi import Request, Response
166188

167189
from fastapi_cachex import cache
168190
from fastapi_cachex.types import CACHE_KEY_SEPARATOR
191+
from fastapi_cachex.types import escape_key_component
169192

170193

171194
# 1. Keep it out of the shared cache entirely.
@@ -183,8 +206,9 @@ def per_user_key(request: Request) -> str:
183206
user_id = getattr(request.state, "user_id", "anonymous")
184207
return (
185208
f"{request.method}{CACHE_KEY_SEPARATOR}"
186-
f"{request.headers.get('host', 'unknown')}{CACHE_KEY_SEPARATOR}"
187-
f"{request.url.path}{CACHE_KEY_SEPARATOR}"
209+
f"{escape_key_component(request.headers.get('host', 'unknown'))}"
210+
f"{CACHE_KEY_SEPARATOR}"
211+
f"{escape_key_component(request.url.path)}{CACHE_KEY_SEPARATOR}"
188212
f"{request.query_params}{CACHE_KEY_SEPARATOR}{user_id}"
189213
)
190214

‎fastapi_cachex/backends/memory.py‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,7 @@
1212
from fastapi_cachex.types import CacheItem
1313
from fastapi_cachex.types import counter_entry
1414
from fastapi_cachex.types import counter_value
15+
from fastapi_cachex.types import escape_key_component
1516

1617
from .base import BaseCacheBackend
1718
from .base import validate_delta
@@ -311,14 +312,15 @@ async def clear_path(self, path: str, include_params: bool = False) -> int:
311312
Returns:
312313
Number of cache entries cleared
313314
"""
315+
key_path = escape_key_component(path)
314316

315317
def matches(key: str) -> bool:
316318
parsed = _split_http_key(key)
317319
if parsed is None:
318320
# Direct key match (custom key format without separators)
319321
return key == path
320322
cache_path, has_params = parsed
321-
return cache_path == path and (include_params or not has_params)
323+
return cache_path == key_path and (include_params or not has_params)
322324

323325
cleared_count = await self._evict(matches)
324326
logger.debug(

‎fastapi_cachex/backends/redis.py‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -16,6 +16,7 @@
1616
from fastapi_cachex.exceptions import CacheXError
1717
from fastapi_cachex.types import CACHE_KEY_SEPARATOR
1818
from fastapi_cachex.types import CacheEntry
19+
from fastapi_cachex.types import escape_key_component
1920

2021
from .base import BaseCacheBackend
2122
from .base import validate_delta
@@ -395,10 +396,11 @@ async def clear_path(self, path: str, include_params: bool = False) -> int:
395396
# exact path is matched: default_key_builder always appends a separator
396397
# after the path, so keys with no query params end with "|||". The
397398
# path is a literal, not a glob: "/files/[draft]" means those brackets.
399+
# It is stored with "|" and "%" percent-encoded, so match it that way.
398400
suffix = "*" if include_params else ""
399401
pattern = (
400402
f"{self._prefix_pattern}*{CACHE_KEY_SEPARATOR}"
401-
f"{_escape_glob(path)}{CACHE_KEY_SEPARATOR}{suffix}"
403+
f"{_escape_glob(escape_key_component(path))}{CACHE_KEY_SEPARATOR}{suffix}"
402404
)
403405
cleared_count = await self._delete_matching(pattern)
404406

‎fastapi_cachex/cache.py‎

Lines changed: 9 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -40,6 +40,7 @@
4040
from .types import CACHE_KEY_SEPARATOR
4141
from .types import CacheEntry
4242
from .types import CacheKeyBuilder
43+
from .types import escape_key_component
4344

4445
if TYPE_CHECKING:
4546
from fastapi.routing import APIRoute
@@ -60,6 +61,11 @@ def default_key_builder(request: Request) -> str:
6061
6162
Generates cache key in format: method|||host|||path|||query_params
6263
64+
``|`` and ``%`` in the host and path are percent-encoded (see
65+
``escape_key_component``), so a ``Host`` header or path containing
66+
``|||`` cannot make one request's key equal another's. The query string
67+
is already URL-encoded and never contains ``|``.
68+
6369
Args:
6470
request: The FastAPI Request object
6571
@@ -68,8 +74,9 @@ def default_key_builder(request: Request) -> str:
6874
"""
6975
key = (
7076
f"{request.method}{CACHE_KEY_SEPARATOR}"
71-
f"{request.headers.get('host', 'unknown')}{CACHE_KEY_SEPARATOR}"
72-
f"{request.url.path}{CACHE_KEY_SEPARATOR}"
77+
f"{escape_key_component(request.headers.get('host', 'unknown'))}"
78+
f"{CACHE_KEY_SEPARATOR}"
79+
f"{escape_key_component(request.url.path)}{CACHE_KEY_SEPARATOR}"
7380
f"{request.query_params}"
7481
)
7582
logger.debug("Built cache key: %s", key)

‎fastapi_cachex/routes.py‎

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,7 @@
1010
from .proxy import BackendProxy
1111
from .types import CACHE_KEY_SEPARATOR
1212
from .types import CacheEntry
13+
from .types import unescape_key_component
1314

1415
if TYPE_CHECKING:
1516
from fastapi import FastAPI
@@ -107,7 +108,9 @@ def _parse_cache_key(cache_key: str) -> tuple[str, str, str, str]:
107108
"""
108109
key_parts = cache_key.split(CACHE_KEY_SEPARATOR, CACHE_KEY_MAX_PARTS)
109110
if len(key_parts) >= CACHE_KEY_MIN_PARTS:
110-
method, host, path = key_parts[0], key_parts[1], key_parts[2]
111+
method = key_parts[0]
112+
host = unescape_key_component(key_parts[1])
113+
path = unescape_key_component(key_parts[2])
111114
query_params = key_parts[3] if len(key_parts) > CACHE_KEY_MIN_PARTS else ""
112115
return method, host, path, query_params
113116

‎fastapi_cachex/types.py‎

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -14,6 +14,26 @@
1414
# Type for custom cache key builder function
1515
CacheKeyBuilder = Callable[[Request], str]
1616

17+
_KEY_ESCAPES = {"%": "%25", "|": "%7C"}
18+
_KEY_UNESCAPES = {escaped: char for char, escaped in _KEY_ESCAPES.items()}
19+
_KEY_UNESCAPE_RE = re.compile("%25|%7C")
20+
21+
22+
def escape_key_component(value: str) -> str:
23+
"""Percent-encode ``|`` and ``%`` so ``value`` cannot contain the separator.
24+
25+
The host header and the decoded URL path are client-controlled and may
26+
contain ``|||``; left as is, one request's components could line up into
27+
another request's key. Encoding ``%`` as well keeps the mapping reversible,
28+
so two different values never share an encoding.
29+
"""
30+
return value.replace("%", "%25").replace("|", "%7C")
31+
32+
33+
def unescape_key_component(value: str) -> str:
34+
"""Reverse ``escape_key_component``."""
35+
return _KEY_UNESCAPE_RE.sub(lambda match: _KEY_UNESCAPES[match.group()], value)
36+
1737

1838
# Status replayed for entries stored before ``CacheEntry`` carried a status code.
1939
DEFAULT_STATUS_CODE = 200

‎i18n/zh-TW/docs/CACHE_FLOW.md‎

Lines changed: 9 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -46,10 +46,16 @@ private? ── 是 → 執行 handler;比對 If-None-Match 決定回傳 304
4646

4747
```python
4848
from fastapi_cachex.types import CACHE_KEY_SEPARATOR # "|||"
49+
from fastapi_cachex.types import escape_key_component
4950

5051
# 快取鍵格式(fastapi_cachex/cache.py 中的 default_key_builder)
5152
cache_key = CACHE_KEY_SEPARATOR.join(
52-
[request.method, request.headers.get("host", "unknown"), request.url.path, query]
53+
[
54+
request.method,
55+
escape_key_component(request.headers.get("host", "unknown")),
56+
escape_key_component(request.url.path),
57+
query,
58+
]
5359
)
5460

5561
# 例如:
@@ -59,6 +65,8 @@ cache_key = CACHE_KEY_SEPARATOR.join(
5965

6066
分隔符號使用 `|||` 而不是冒號,是因為 host 本身可能包含連接埠(`127.0.0.1:8000`);若使用冒號,快取鍵就無法可靠地拆分,而 `clear_path()` 需要從快取鍵中取回路徑。
6167

68+
host 與路徑會先經過百分比編碼:`|` 變成 `%7C`,`%` 變成 `%25`(`fastapi_cachex/types.py` 中的 `escape_key_component`)。兩者都來自用戶端,其中若出現未編碼的 `|||`,各段就會錯位,使某個請求的快取鍵可能與另一個請求相同。查詢字串本來就經過 URL 編碼。監控路由顯示時會再解碼。
69+
6270
查詢參數依請求送出的順序串接(`str(request.query_params)`),**不會排序**,因此 `?page=1&limit=10` 與 `?limit=10&page=1` 是兩個不同的快取項目。若希望兩者視為同一個,請傳入自訂的 `key_builder` 將查詢字串正規化。
6371

6472
這個快取鍵格式讓每個維度各自獨立快取:

0 commit comments

Comments
 (0)