Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions changelog.d/254.fixed.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
**Responses served from the cache carry an `Age` header, so downstream caches no longer keep them for up to twice the `ttl`.**
A hit, and a 304 answered from the stored entry's ETag, used to send
`Cache-Control: max-age=<ttl>` with no `Age`, so a browser or CDN restarted
the freshness clock on every hit. They now send `Age`, the whole seconds since
the entry was stored, clamped to `0`–`ttl` against clock skew between hosts;
downstream subtracts it from `max-age` (RFC 9111 §4.2.3). `CacheEntry` has a
new optional `stored_at` field (epoch seconds, wall clock) that `@cache` sets
and the Redis/Memcached codec stores. Entries written by older releases decode
with `stored_at=None` and are served without `Age`; responses the handler
renders (misses, `no_cache`, bypassed requests) never carry one. An `Age`
header the handler sets is no longer stored and replayed.
32 changes: 24 additions & 8 deletions docs/CACHE_FLOW.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,12 +28,12 @@ Read the backend entry
↓
Request carries If-None-Match?
├─ and no-cache → run the handler first to compute the current ETag; match → 304
├─ otherwise → compare with the cached entry's ETag; match → 304
├─ otherwise → compare with the cached entry's ETag; match → 304 with Age
└─ no match / no header → continue
↓
Cached entry exists, ttl is set, and no-cache is off?
├─ yes → respond with the cached content (including the stored status code
│ and headers; the handler does **not** run)
│ and headers, plus Age; the handler does **not** run)
└─ no → run the handler
├─ non-2xx (or 206) → return as-is and **do not write**
│ (an existing good entry is not overwritten)
Expand Down Expand Up @@ -185,12 +185,15 @@ entry = CacheEntry(
media_type="application/json",
status_code=200, # replayed with the original status code
headers={"Vary": "Accept-Encoding"}, # headers sent back on replay
stored_at=1702650540.5, # epoch seconds when @cache stored it; drives Age
)
```

The TTL is not stored in `CacheEntry`: expiry is the backend's responsibility
(`MemoryBackend` keeps it in `CacheItem.expiry`, Redis uses `SET ... EX`,
Memcached uses the exptime).
Memcached uses the exptime). `stored_at` is wall-clock time (`time.time()`),
since the process that serves an entry may not be the one that stored it; it
is `None` for entries written by releases before 0.3.9.

If no backend has been configured with `BackendProxy.set()`, the decorator
creates a `MemoryBackend` on the first request, registers it and logs a
Expand All @@ -217,14 +220,15 @@ if client_etag and no_cache:
if etag_matches(client_etag, fresh.etag):
return not_modified(...) # 304
elif client_etag and entry and etag_matches(client_etag, entry.fingerprint):
return not_modified(...) # 304, handler does not run
return not_modified(..., age_headers(entry, ttl)) # 304, handler does not run

if entry and not no_cache:
return Response( # 200, handler does not run
content=entry.content,
status_code=entry.status_code,
media_type=entry.media_type,
headers={**(entry.headers or {}), "ETag": entry.fingerprint, ...},
headers={**(entry.headers or {}), "ETag": entry.fingerprint, ...,
**age_headers(entry, ttl)}, # Age: now - stored_at, clamped to 0..ttl
)

response, body, etag = await render() # miss (reused if no-cache already rendered)
Expand All @@ -235,10 +239,18 @@ if etag is None:
if marked_private_or_no_store(response) or "set-cookie" in response.headers:
return response # one caller's response: not written
if not entry or entry.fingerprint != etag:
await backend.set(cache_key, CacheEntry(...), ttl=ttl)
await backend.set(cache_key, CacheEntry(..., stored_at=time.time()), ttl=ttl)
return response
```

> [!NOTE]
> **`Age` on responses served from the backend.** A hit and a 304 answered from
> the stored ETag carry `Age: <seconds since stored_at>`, clamped to `0`–`ttl`
> against clock skew between hosts; `Cache-Control` keeps `max-age=<ttl>`, and
> a downstream cache subtracts `Age` from it (RFC 9111 §4.2.3). Responses the
> handler just rendered, including every `no_cache` response and every bypass,
> carry no `Age`, and neither do entries without `stored_at`.

> [!NOTE]
> "Non-2xx is not written" is deliberate: a transient error must not wipe out
> the last good cached response, nor be replayed later as a 200. `206 Partial
Expand Down Expand Up @@ -307,6 +319,7 @@ intermediate cache would lose those fields after revalidation (RFC 9110
media_type="application/json",
status_code=200,
headers=None,
stored_at=1702650540.5,
),
expiry=1702650600.5, # epoch seconds; None means never expires
),
Expand Down Expand Up @@ -334,15 +347,18 @@ and the standard library `json` otherwise:
"content": "<response bytes decoded as latin-1>",
"media_type": "application/json",
"status_code": 200,
"headers": {"Vary": "Accept-Encoding"}
"headers": {"Vary": "Accept-Encoding"},
"stored_at": 1702650540.5
}
```

- `content` uses a **latin-1 round-trip**, not base64: latin-1 maps one-to-one
onto bytes, so any byte sequence can be placed in JSON text and recovered
unchanged.
- Entries written by older releases, without the `status_code`/`headers`
fields, remain readable and decode to `200` with no extra headers.
fields, remain readable and decode to `200` with no extra headers. Those
without `stored_at` (before 0.3.9) decode with `stored_at=None` and are
served without an `Age` header.
- Any decode failure (broken JSON, missing fields, wrong types) is treated as a
**cache miss** and returns `None` instead of raising.
- `increment()` leaves a **bare integer** behind (written by the Redis/Memcached
Expand Down
23 changes: 23 additions & 0 deletions docs/HTTP_CACHING.md
Original file line number Diff line number Diff line change
Expand Up @@ -111,6 +111,29 @@ When a cached entry is valid (within TTL):
- **Without `ttl`** (`ttl=None`): Nothing is read from or written to the backend, as with `private=True`. The handler runs on every request, and `If-None-Match` gets a 304 only when it matches the freshly rendered response, so an old ETag never gets a 304 once the content has changed
- **With `ttl=0`**: Sends `max-age=0` and otherwise behaves like `ttl=None`. A negative `ttl`, a non-`int` one (such as `1.5` or `True`) and one above `MAX_TTL` (see [TTL values](BACKENDS.md#ttl-values)) are rejected with `CacheXError` when the decorator is applied

### The `Age` header

A response served from a stored entry carries an `Age` header: the whole
number of seconds since `@cache` stored it (RFC 9111 §5.1). That covers a
cache hit and a 304 answered from the stored entry's ETag. `Cache-Control`
still says `max-age=<ttl>`, and a browser or CDN subtracts `Age` from it
(RFC 9111 §4.2.3), so a response stored 50 seconds into a 60-second ttl is
reused downstream for at most 10 more seconds. Without `Age`, a hit just
before the entry expired restarted the downstream clock, and the content could
be reused for up to twice the ttl.

```
GET /items → 200, Cache-Control: max-age=60, no Age (the handler ran)
GET /items → 200, Cache-Control: max-age=60, Age: 42 (served 42 s after it was stored)
```

The time an entry was stored comes from the wall clock of the process that
stored it and is read by whichever process serves it, so `Age` is clamped to
`0`–`ttl` in case two hosts' clocks disagree. No `Age` is sent when the handler
runs (a miss, `no_cache=True`, a bypassed request) or for an entry stored by
a release before 0.3.9, which does not record the time. An `Age` header the
handler sets itself is not stored.

Only successful responses are stored. A response the handler *returns* with a
non-2xx status (for example `Response(..., status_code=404)`) is passed straight
through and never cached, so a transient error cannot replace or poison the last
Expand Down
14 changes: 13 additions & 1 deletion fastapi_cachex/backends/codec.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,8 @@
it is installed and the standard library ``json`` module otherwise.
"""

import math

from fastapi_cachex.types import COUNTER_FINGERPRINT
from fastapi_cachex.types import DEFAULT_STATUS_CODE
from fastapi_cachex.types import CacheEntry
Expand Down Expand Up @@ -45,12 +47,20 @@ def encode_entry(entry: CacheEntry) -> bytes:
"media_type": entry.media_type,
"status_code": entry.status_code,
"headers": entry.headers,
"stored_at": entry.stored_at,
},
)
# orjson returns bytes, stdlib json returns str
return serialized if isinstance(serialized, bytes) else serialized.encode("utf-8")


def _stored_at(value: object) -> float | None:
"""``stored_at`` from a document; anything but a finite number is unknown."""
if isinstance(value, bool) or not isinstance(value, (int, float)):
return None
return float(value) if math.isfinite(value) else None


def decode_entry(raw: str | bytes | None) -> CacheEntry | None:
"""Rebuild a ``CacheEntry`` from a stored value.

Expand All @@ -61,7 +71,8 @@ def decode_entry(raw: str | bytes | None) -> CacheEntry | None:
cache miss.

Documents written before entries carried a status code and headers simply
lack those keys and decode to a plain ``200`` with no extra headers.
lack those keys and decode to a plain ``200`` with no extra headers; those
written before entries carried ``stored_at`` decode with ``None``.
"""
if raw is None:
return None
Expand All @@ -76,6 +87,7 @@ def decode_entry(raw: str | bytes | None) -> CacheEntry | None:
media_type=data.get("media_type"),
status_code=data.get("status_code", DEFAULT_STATUS_CODE),
headers=data.get("headers"),
stored_at=_stored_at(data.get("stored_at")),
)
except _DECODE_ERRORS:
return None
49 changes: 46 additions & 3 deletions fastapi_cachex/cache.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@
import inspect
import logging
import threading
import time
import warnings
from collections.abc import Awaitable
from collections.abc import Callable
Expand Down Expand Up @@ -62,6 +63,11 @@

_NO_STORE = DirectiveType.NO_STORE.value

# Wall clock behind ``CacheEntry.stored_at`` and the ``Age`` header. Wall time,
# not monotonic, because an entry stored by one process or host is served by
# another. A module attribute so tests can move time without sleeping.
_now = time.time


def build_cache_key(request: Request, *components: str | int) -> str:
"""Build the default cache key for ``request``, plus extra components.
Expand Down Expand Up @@ -466,6 +472,7 @@ def __str__(self) -> str:
"etag",
"cache-control",
"content-type",
"age",
}
)

Expand Down Expand Up @@ -499,6 +506,28 @@ def _cacheable_headers(response: Response) -> dict[str, str] | None:
_REVALIDATION_HEADERS = frozenset({"content-location", "expires", "vary"})


def _age_headers(entry: CacheEntry, ttl: int | None) -> dict[str, str]:
"""The ``Age`` header for a response served from a stored ``entry``.

``Cache-Control`` keeps ``max-age=<ttl>`` on a hit: RFC 9111 §4.2.3 has a
downstream cache compute the remaining freshness as ``max-age`` minus
``Age``, so a copy stored here N seconds ago is fresh downstream for
``ttl - N`` more seconds, and the total never reaches twice the ttl.

``Age`` is a non-negative integer number of seconds (RFC 9111 §5.1). The
value is clamped to ``[0, ttl]``: ``stored_at`` may come from another
host's clock, and the backend never keeps an entry longer than ``ttl``, so
anything outside that range is clock skew. An entry without ``stored_at``
(written by an older release) gets no ``Age`` at all.
"""
if entry.stored_at is None:
return {}
age = max(0.0, _now() - entry.stored_at)
if ttl is not None:
age = min(age, ttl)
return {"age": str(int(age))}


def _revalidation_headers(headers: Mapping[str, str] | None) -> dict[str, str]:
"""The subset of a response's headers that a 304 must repeat."""
if not headers:
Expand Down Expand Up @@ -643,21 +672,27 @@ def _etag_matches(if_none_match: str | None, etag: str) -> bool:


def _not_modified(
etag: str, cache_control: str, headers: Mapping[str, str] | None = None
etag: str,
cache_control: str,
headers: Mapping[str, str] | None = None,
age: Mapping[str, str] | None = None,
) -> Response:
"""Build the 304 for a successful revalidation.

``headers`` is what the 200 for this resource would have carried; RFC 9110
§15.4.5 requires the fields that steer caching to be repeated on the 304,
otherwise a cache that stored the 200 would drop them on refresh. ``Date``
is added by Starlette and the other two are set here.
is added by Starlette and the other two are set here. ``age`` is the
``Age`` header (see ``_age_headers``) when the 304 is answered from a
stored entry.
"""
return Response(
status_code=HTTP_304_NOT_MODIFIED,
headers={
**_revalidation_headers(headers),
"ETag": etag,
"Cache-Control": cache_control,
**(age or {}),
},
)

Expand Down Expand Up @@ -1220,8 +1255,14 @@ async def serve(*args: Any, **kwargs: Any) -> Response:
logger.debug(
"304 Not Modified (cached ETag match); key=%s", cache_key
)
# Answered from the stored entry, so the 304 says how old
# that entry is: a cache refreshing its copy with this 304
# takes the new Age with it (RFC 9111 §4.3.4).
return _not_modified(
cached_data.fingerprint, cache_control, cached_data.headers
cached_data.fingerprint,
cache_control,
cached_data.headers,
_age_headers(cached_data, ttl),
)

# If we don't have If-None-Match header, check if we have a valid cached copy
Expand All @@ -1236,6 +1277,7 @@ async def serve(*args: Any, **kwargs: Any) -> Response:
**(cached_data.headers or {}),
"ETag": cached_data.fingerprint,
"Cache-Control": cache_control,
**_age_headers(cached_data, ttl),
},
)

Expand Down Expand Up @@ -1284,6 +1326,7 @@ async def serve(*args: Any, **kwargs: Any) -> Response:
media_type=_media_type_of(current_response),
status_code=current_response.status_code,
headers=_cacheable_headers(current_response),
stored_at=_now(),
),
ttl=ttl,
)
Expand Down
7 changes: 7 additions & 0 deletions fastapi_cachex/types.py
Original file line number Diff line number Diff line change
Expand Up @@ -59,13 +59,20 @@ class CacheEntry:
``status_code`` and ``headers`` default to a plain ``200`` with no extra
headers, so entries built by older callers (and documents written by older
releases) keep their previous behaviour.

``stored_at`` is when ``@cache`` stored the response, in epoch seconds
from the wall clock (``time.time()``), since an entry written by one
process or host may be served by another. It drives the ``Age`` header on
a hit; ``None`` (entries written by older releases, and anything not
stored by ``@cache``) sends no ``Age``.
"""

fingerprint: str
content: bytes
media_type: str | None = None
status_code: int = DEFAULT_STATUS_CODE
headers: dict[str, str] | None = None
stored_at: float | None = None


@dataclass
Expand Down
Loading
Loading