Repository navigation
Use native shared fragments and simplify the local hot-path cache - #350
thomasklemm wants to merge 13 commits into
Conversation
|
Updated in A single attachment or mention on a page previously bypassed the collection cache for every message on it. Mentions are common in real rooms, but the benchmark fixtures contain none, so this didn't show up. Pages now render in consecutive runs: cacheable runs keep their content-keyed hits and only messages with further dependencies render fresh. A new Local production-mode probe, unrelated commit before each read, p50: room with a mention in every eighth message 119 → 73 ms, matching search 257 → 129 ms, room without mentions unchanged. Unit (588 runs) and system tests (27 runs) pass; hosted CI is green. The description has the details. Generated by Claude Code |
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
There was a problem hiding this comment.
🔵 Needs a closer look
Cross-worker caching, authorization snapshots, and production SQLite contention require final human validation after dependency #345 lands.
0 open findings
What changed in this PR
Migrates shared Rails caching to Solid Cache while retaining a bounded per-worker cache for completed responses and authorization snapshots.
Changes:
- Adds Solid Cache configuration, schema, restore handling, and documentation.
- Replaces private fragment caching with content-keyed native Rails fragments.
- Consolidates local response and record caching under one
MemoryStorebudget.
[!TIP]
If you aren't ready for review, convert to a draft PR.
Click "Convert to draft" or rungh pr ready --undo.
Click "Ready for review" or rungh pr readyto reengage.
| File | Description |
|---|---|
Gemfile |
Adds Solid Cache. |
Gemfile.lock |
Locks Solid Cache 1.0.10. |
README.md |
Documents the revised cache architecture. |
app/controllers/application_controller.rb |
Enables shared fragment-cache safeguards. |
app/controllers/concerns/cached_responses.rb |
Simplifies local response-cache integration. |
app/controllers/concerns/fragment_caching.rb |
Namespaces and conditionally enables fragments. |
app/controllers/messages/by_bots_controller.rb |
Preloads JSON presentation dependencies. |
app/controllers/users/sidebars_controller.rb |
Preloads direct-room participants. |
app/helpers/messages_helper.rb |
Implements content-keyed, run-based rendering. |
app/helpers/rooms_helper.rb |
Uses preloaded direct-room users. |
app/models/fragment_cache.rb |
Removes the private fragment store. |
app/models/message/pagination.rb |
Updates presentation preloading behavior. |
app/models/record_cache.rb |
Shares the local response-cache store. |
app/models/response_cache.rb |
Replaces FIFO bookkeeping with MemoryStore. |
app/views/messages/_actions.html.erb |
Uses native conditional fragment caching. |
app/views/messages/_message.html.erb |
Removes duplicate nested message caching. |
app/views/messages/_message.json.jbuilder |
Renders JSON without fragment caching. |
app/views/messages/boosts/_boost.json.jbuilder |
Removes cached boost JSON. |
app/views/messages/index.html.erb |
Uses run-based message rendering. |
app/views/rooms/refreshes/show.turbo_stream.erb |
Uses run-based rendering for refreshes. |
app/views/rooms/show.html.erb |
Uses run-based room rendering. |
app/views/searches/index.html.erb |
Uses run-based search rendering. |
app/views/users/_user.json.jbuilder |
Renders current user JSON. |
app/views/users/sidebars/rooms/_direct.html.erb |
Removes direct-room fragment caching. |
app/views/users/sidebars/show.html.erb |
Renders direct memberships fresh. |
bench/README.md |
Clarifies cold-cache measurement. |
bench/message_hot_paths.rb |
Clears every applicable cache store. |
config/cache.yml |
Configures Solid Cache limits. |
config/database.yml |
Adds dedicated cache databases. |
config/environments/production.rb |
Selects Solid Cache in production. |
config/initializers/fragment_cache.rb |
Removes custom Jbuilder/store overrides. |
config/initializers/version.rb |
Defines deployment-specific presentation namespaces. |
db/cache_schema.rb |
Adds the Solid Cache schema. |
db/schema.rb |
Normalizes schema-file ending. |
docs/self-hosting.md |
Documents operation, backup, and restore. |
docs/upgrades/solid-cache.md |
Adds migration and rollback guidance. |
hooks/post-restore |
Removes disposable cache files during restore. |
test/controllers/cached_responses_test.rb |
Verifies sidebar response reuse. |
test/controllers/fragment_cache_test.rb |
Expands fragment correctness coverage. |
test/helpers/messages_helper_test.rb |
Tests native presentation reuse. |
test/models/record_cache_test.rb |
Tests shared-budget snapshots. |
test/models/response_cache_test.rb |
Tests MemoryStore admission and eviction. |
test/scripts/solid_cache_restore_test.rb |
Tests cache cleanup during restore. |
🧠 Review effort: Balanced
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: af029e3012
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| # Cache in memory for now | ||
| config.cache_store = :redis_cache_store | ||
| # Persist the Rails cache in its own SQLite database. | ||
| config.cache_store = :solid_cache_store |
There was a problem hiding this comment.
Prepare Solid Cache before starting benchmark Puma
When bench/compare_http.rb is run with an existing benchmark seed created before this migration (and thus containing only production.sqlite3), the candidate now needs the solid_cache_entries schema, but the driver copies the seed and launches bundle exec puma directly at lines 42–49 without the normal db:prepare startup step. SQLite can create the cache file on first connection but not its table, so the login or first fragment-cache access fails with a missing-table error instead of producing benchmark results; prepare each copied storage directory or load db/cache_schema.rb before starting Puma.
Useful? React with 👍 / 👎.
|
This PR under concurrent multi-worker HTTP load, the measurement the description lists as missing. All numbers come from one laptop (Docker Desktop on Windows 11, server container pinned to 4 CPUs with a 2.5 GB limit, overlay storage). They add to the single-process timings above; they don't replace them. Setup. Production images built from each head, plus the official v1.5.6 image for reference:
On Today's One process, no Puma (same laptop, same data). 100 people each view room pages six times. A quiet instance has no writes between views. An active one refreshes someone else's presence between views, which an instance with open tabs does all the time. Room page p50:
On the base, 49% of fragment reads hit when the instance is active, and 91% on this PR. The base's message fragment keys carry the viewer's user id and session digest, in a per-process store, so each person renders each message once per worker. Here they're shared. This doesn't contradict the "unrelated commit before each read" rows in the description: those read with one session, and with one viewer the base's keys hit too. The gap shows once many people read the same rooms. RSS growth of that process over 300 people × 6 views: base +257 MB, this PR +62 MB. Written by Claude Code (AI assistant) on behalf of @namespaceMarcello, who directs this work. |
|
Thanks @namespaceMarcello, this is the measurement the description was missing; I added a short summary there. I also merged current |
|
Most of what still separates this PR from v1.5.6 on a room page is deriving the fragment keys, and a change on top of Where the time goes.
The whole room page, same process, someone else's presence refreshed between views, 1,000 pages per run, three runs:
The change. The same rows and the same slices, read as stored: What was checked, on the same laptop:
Under load (the multi-worker run from the earlier comment: 3 Puma workers × 5 threads on 4 CPUs, 2.5 GB, 300 WebSocket viewers, an HTTP driver at 4 then 24 concurrent people):
Two runs each for this PR and the commit, in opposite order, one for v1.5.6; every run with the commit did better than every run without it on each row. Every accepted message was stored, in every run. The gain is smaller than in one process: these two pages are about two in five of the driver's requests. Untouched by the commit and well apart from v1.5.6: Our open #357 and #382 touch the same templates and response cache; none of these images include them. Written by Claude Code (AI assistant) on behalf of @namespaceMarcello, who directs this work. |
1d019ed to
966515e
Compare
Amp-Thread-ID: https://ampcode.com/threads/T-01a11acc-6153-7316-a06c-fcf2d713cd64 Co-authored-by: Amp <amp@ampcode.com>
Amp-Thread-ID: https://ampcode.com/threads/T-01a11acc-6153-7316-a06c-fcf2d713cd64 Co-authored-by: Amp <amp@ampcode.com>
Amp-Thread-ID: https://ampcode.com/threads/T-01a11acc-6153-7316-a06c-fcf2d713cd64 Co-authored-by: Amp <amp@ampcode.com>
Amp-Thread-ID: https://ampcode.com/threads/T-01a11acc-6153-7316-a06c-fcf2d713cd64 Co-authored-by: Amp <amp@ampcode.com>
Amp-Thread-ID: https://ampcode.com/threads/T-01a11acc-6153-7316-a06c-fcf2d713cd64 Co-authored-by: Amp <amp@ampcode.com>
Amp-Thread-ID: https://ampcode.com/threads/T-01a11acc-6153-7316-a06c-fcf2d713cd64 Co-authored-by: Amp <amp@ampcode.com>
The Solid Cache generator evaluates ENV.fetch("DATABASE") once, when it
writes cache.yml. Copied verbatim, it ran on every boot, so an unrelated
DATABASE variable in the host environment would redirect the cache.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01N8uPpsqMRJrp8jqATFJW45
Build on the Solid Cache migration without sacrificing observer-validated response and authorization reuse. Replace custom FIFO bookkeeping with Rails MemoryStore and move safe content-keyed collections to Rails.cache. Remove private Jbuilder hooks and keep unsafe nested JSON uncached. Amp-Thread-ID: https://ampcode.com/threads/T-01a11fe5-588c-71af-905f-84af46bd6951
One attachment, mention or other embedded attachment on a page bypassed the collection cache for all of its messages. Mentions are common, so many real pages rendered entirely uncached. Render consecutive runs instead: cacheable runs keep their content-keyed hits, and only the messages with further dependencies render fresh. Local production-mode probe, 60 requests with an unrelated commit before each read, p50 over two runs: an 80-message room with a mention in every eighth message 119 -> 73 ms; a search matching those messages 257 -> 129 ms; a room without mentions unchanged (50 -> 49 ms). Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01N8uPpsqMRJrp8jqATFJW45
Serialize each message body once per render: the fragment key, the cacheability check and the presentation each walked the rich text document. Collection-cached messages no longer also write the text presentation and message controls as separate Solid Cache entries; those nested fragments remain for messages rendered fresh, such as direct rooms and mentions. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01N8uPpsqMRJrp8jqATFJW45
Replace the helper's instance-variable memo with an explicit fragment_cached local: render_messages decides cacheability once per message and the partials read it, so no view state lives in helper instance variables. Ahead-of-time compilers such as Roundhouse lower partial locals but treat view instance variables as controller state. Cold renders stay as fast as with the memo. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01N8uPpsqMRJrp8jqATFJW45
Roundhouse lowers a collection render only where it appears in a template, and drops a statement-form `cache_if` block. Built through Roundhouse, the helper-side `render_messages` emitted a single nil message instead of the page, and `_actions` emitted nothing at all. Message pages now iterate `message_runs` in the view and render each run with the same collection cache and `fragment_cached` local. The action controls cache through `cache_message_actions`, an output helper with a block, under the same presentation namespace as before. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BMME5CgsJpvz4dB74xsCEs
6399714 to
49e1b2d
Compare
|
Closing in favour of #412, which shares these fragments between viewers with a one-file change to the existing keys, and #413, which takes the cost out of deriving them. Thanks @namespaceMarcello for the load runs and the stored-value keys. The 500 for a body stored as invalid text happens on main too; would you open your fix for it as its own PR? |
Requested by Thomas · project thread
Stack
Depends on #345; please merge the Solid Cache migration first. This PR is opened directly against upstream
main, with headthomasklemm:perf/native-fragment-cache. GitHub cannot target the dependency branch because it exists only in the fork, so this PR currently includes #345's commits too. Once #345 lands, the remaining diff will be this follow-up.The isolated follow-up is five commits on the rebased #345 head. Compare only the changes after #345.
Design
A fully shared/native replacement was prototyped and rejected after pronounced warm-request regressions. Keep the observer-validated per-worker response/authorization hot path; make presentation reuse Rails-native where its dependencies are safe.
MemoryStorefor completed HTML/gzip responses and independent authorization snapshots. Retain admission limits, auth/access checks, cold-render collapse and SQLitedata_versionchecks before/after reads and on admission.CAMPFIRE_RESPONSE_CACHE_MB=0disables this local cache.Rails.cache.fetchfor viewer-independent HTML fragments, backed by Use Solid Cache for the production Rails cache #345's shared Solid Cache. Content-derived keys cover actual body, room, creator and boost/booster presentation inputs, including SQL edits without timestamp changes.GIT_REVISIONto share across workers.message_runsand render each run themselves, and the action controls cache through thecache_message_actionsblock helper. This keeps the page compilable ahead of time through Roundhouse, which lowers collection renders only in templates and drops statement-formcache_ifblocks. Detached and primary-transaction renders bypass shared fragments. Remove duplicate full-message caching, the private fragment store and Jbuilder private-method overrides. JSON renders fresh; direct sidebars preload participants rather than cache incomplete membership dependencies.fragment_cachedpartial local tells the nested partials whether their message is already collection-cached. Collection-cached messages do not also write their text presentation and controls as separate Solid Cache entries; those nested fragments remain only for messages rendered fresh, such as in direct rooms or with mentions.Oracle reviewed the hybrid and found no blocking correctness/security issue. This is not approval to merge and not a universal performance improvement.
Measurements against the parent
Re-measured on the current heads: parent #345 at
3a59b4e, this PR atf5be953. Sequential ABBA·ABBA, eight independent processes (four per revision), Ruby 4.0.7 + YJIT, production environment with actual Solid Cache (Redis only for Action Cable on the parent), fresh database per process with 20 users, 80 deterministic messages (boosts on every seventh) and 16 authenticated sessions. 200-request warmup per workload on one session; 30 timed samples per run (15 for previously unwarmed sessions). Values average each run's p50.Warm whole-response hits stay around 1.8–2.2 ms in both versions, within run-to-run noise. No universal speedup claim. Shared fragments remove duplicate per-viewer rendering. With an unrelated commit before each read both versions are within noise. Fully cold renders were 35–40% slower than the parent on the previous head (
e67f1a8) and are now 6–10% faster: profiling showed rich-text serialization repeated per message plus about 80 nested Solid Cache reads and inserts per cold page. Every revision's cold room p50 fell in a narrow band (parent 158.8–166.2 ms, this PR 149.0–157.6 ms). JSON throughput has not been benchmarked. These are single-process integration timings; concurrent multi-worker load is below.Pages with mentions or attachments
The fixture data above contains no mentions, so it does not show that a single mention or attachment previously turned off the collection cache for a whole page. Local production-mode probe with Solid Cache, 60 requests with an unrelated commit before each read, p50 over two runs:
Concurrent load, measured by @namespaceMarcello
Docker, 3 Puma workers × 5 threads, 4 CPUs / 2.5 GB, 1,000 people, 300 holding WebSockets, a mixed chat workload. One run per image:
05c5a2caf029e3)With many viewers the base's per-viewer fragment keys hit 49% of the time on an active instance, these shared keys 91%.
Local verification on the current head
Rebased onto upstream
mainatca9b963through #345, without merge commits. The resulting tree matches the previous PR plus current main, except for the inherited lockfile regenerated withbundle lock. This retains the previous integration's caching, text-node auto-linking, script policy, and bumped message/text-presentation cache keys. The measurements above describe the historical revisions explicitly named there; they were not rerun for this history-only refresh.49e1b2d,bin/rails db:prepare teston Ruby 4.0.9: 659 runs, 3,964 assertions, 0 failures/errors, 2 skips.bin/rubocop: 335 files, no offenses.