Skip to content

fix(redis): warn when the backend is configured with a non-UTF-8 encoding - #157

Merged
allen0099 merged 1 commit into
masterfrom
fix/redis-non-utf8-encoding-warning
Sep 26, 2026
Merged

allen0099 merged 1 commit into
masterfrom
fix/redis-non-utf8-encoding-warning

Conversation

@allen0099

Copy link
Copy Markdown
Owner

Closes #122.

Problem

The shared codec always writes UTF-8 JSON, but the Redis client decodes every reply with the configured encoding. With anything else, non-ASCII content comes back corrupted and nothing reports it. Reproduced on a live Redis:

utf-8    b'\xe9'
latin-1  b'\xc3\xa9'

Change

  • AsyncRedisCacheBackend.__init__ emits a RuntimeWarning when encoding is not UTF-8. The name is normalised with codecs.lookup, so "UTF8" and "utf_8" are accepted silently. load_from_config() goes through the same check. The warning points at the caller's line.
  • An unknown codec is not warned about: redis-py already raises LookupError when the client is built.
  • It warns rather than rejects, so existing configurations keep starting. 0.4.0: drop the Redis encoding option and read raw bytes #126 drops the parameter in 0.4.0.
  • The encoding docstring, docs/BACKENDS.md and the changelog explain why the encoding has to stay UTF-8.

Tests

  • UTF-8 aliases do not warn. latin-1, utf-16 and ascii warn, attributed to the test file. load_from_config warns. An unknown codec raises LookupError without the warning.
  • A live Redis test pins the corruption the warning describes.

With the warning call removed, the five warning tests fail. The full suite passes against live Redis and Memcached (CACHEX_REQUIRE_LIVE_SERVERS=1): 799 passed, 100% coverage. mypy --strict is clean.

This and #156 each add a ### Fixed section to ## [Unreleased]. Whichever merges second needs a small CHANGELOG rebase.

@allen0099 allen0099 added this to the 0.3.8 milestone Sep 26, 2026
@allen0099 allen0099 added bug Something isn't working backends Cache backends and their atomic primitives labels Sep 26, 2026
…ding

Entries are always written as UTF-8 JSON, but the client decodes replies
with the configured encoding, so under encoding="latin-1" a stored b"\xe9"
read back as b"\xc3\xa9" without any error. AsyncRedisCacheBackend, and so
load_from_config(), now emit a RuntimeWarning for any encoding other than
UTF-8 (aliases such as "UTF8" are accepted). An unknown codec is still
rejected by redis-py itself. #126 removes the parameter in 0.4.0.

Closes #122
@allen0099
allen0099 force-pushed the fix/redis-non-utf8-encoding-warning branch from acd3f21 to 0133733 Compare September 26, 2026 11:31
@allen0099
allen0099 merged commit 44420b1 into master Sep 26, 2026
11 checks passed
@allen0099
allen0099 deleted the fix/redis-non-utf8-encoding-warning branch September 26, 2026 11:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backends Cache backends and their atomic primitives bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Warn when the Redis backend is configured with a non-UTF-8 encoding

1 participant