Skip to content

Handle Memcached "object too large" consistently outside @cache #261

Description

@allen0099

Split out of #228 (fixed for @cache in #259).

Problem

Memcached rejects values larger than its item size limit (1 MB by default, memcached -I) with MemcacheServerError: object too large for cache. #259 makes @cache log this and serve the response unstored. Every other writer still gets the raw pymemcache error:

  • CacheManager.set / add / get_or_set
  • StateManager.create_state (large metadata)
  • SessionManager (large session data)
  • direct backend.set / set_if_absent callers

Redis has a much larger limit (512 MB per value), so the same code can work on Redis and fail on Memcached.

Options

  1. Keep raising, with a clearer error. Translate the error into a CacheXError subclass that names the key's size and the server limit. This is the least surprising option for writers whose value must be stored: sessions and states must never be dropped silently.
  2. Skip the store for pure caches. CacheManager.set could log and skip like @cache, since a cache miss later is harmless. get_or_set would still return the computed value. add would need a decision, because it reports whether it stored.
  3. Check the size before sending. Add a max_item_size option on MemcachedBackend so the check happens without a round trip.

A reasonable split is 1 for sessions, states and the backend API, and 2 for CacheManager. Whatever is chosen, BACKENDS.md should document it per caller. The limit itself is already listed there as of #259.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    backendsCache backends and their atomic primitivescache-managerApplication-level CacheManagerenhancementNew feature or request

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions