Repository navigation
Replace Redis with Solid Cable - #347
thomasklemm wants to merge 3 commits into
Conversation
|
Updated in
Unit suite (582 runs) and RuboCop pass locally on Ruby 4.0.7. Production Generated by Claude Code |
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
There was a problem hiding this comment.
🟡 Changes recommended
The migration command uses an invalid Resque API, and the benchmark cannot reliably start either dependency set or prepare the new databases.
3 open findings
What changed in this PR
Replaces Redis-backed cache, jobs, and Action Cable with dedicated SQLite-backed Solid adapters.
[!TIP]
If you aren't ready for review, convert to a draft PR.
Click "Convert to draft" or rungh pr ready --undo.
Click "Ready for review" or rungh pr readyto reengage.
Changes:
- Adds Solid Cache, Queue, and Cable databases and configuration.
- Removes Redis, Resque, and Kredis runtime dependencies.
- Updates startup, restore, benchmarking, and deployment documentation.
| File | Description |
|---|---|
test/scripts/solid_cache_restore_test.rb |
Tests removal of cache and cable databases. |
test/scripts/post_restore_test.rb |
Tests primary restore and queue cleanup. |
Procfile |
Replaces Redis/Resque processes with Solid Queue. |
lib/tasks/resque.rake |
Removes Resque lifecycle tasks. |
lib/rails_ext/sqlite_wal_checkpoint.rb |
Generalizes fork documentation. |
hooks/post-restore |
Clears disposable databases during restore. |
Gemfile.lock |
Locks Solid adapter dependencies. |
Gemfile |
Replaces Redis dependencies with Solid adapters. |
docs/upgrades/solid-queue.md |
Adds Solid Queue migration guidance. |
docs/upgrades/solid-cache.md |
Adds Solid Cache migration guidance. |
docs/upgrades/solid-cable.md |
Adds Solid Cable migration guidance. |
docs/self-hosting.md |
Updates backup, restore, and upgrade instructions. |
docs/development.md |
Removes development Redis instructions. |
Dockerfile |
Removes the Redis package. |
db/schema.rb |
Normalizes schema formatting. |
db/queue_schema.rb |
Adds Solid Queue schema. |
db/cache_schema.rb |
Adds Solid Cache schema. |
db/cable_schema.rb |
Adds Solid Cable schema. |
config/resque-pool.yml |
Removes Resque pool configuration. |
config/redis.conf |
Removes Redis configuration. |
config/recurring.yml |
Schedules queue record cleanup. |
config/queue.yml |
Configures Solid Queue processes. |
config/initializers/sqlite_wal_checkpoint.rb |
Updates fork lifecycle documentation. |
config/initializers/solid_queue.rb |
Manages checkpoint threads around queue forks. |
config/initializers/resque.rb |
Removes Resque initialization. |
config/environments/production.rb |
Enables Solid Cache and Queue. |
config/environments/development.rb |
Uses in-memory development caching. |
config/database.yml |
Defines dedicated adapter databases. |
config/cache.yml |
Configures Solid Cache retention and size. |
config/cable.yml |
Enables Solid Cable outside development/test. |
bin/start-app |
Moves database preparation out of web startup. |
bin/setup |
Removes Redis setup. |
bin/jobs |
Adds the Solid Queue executable. |
bin/boot |
Prepares databases before launching services. |
bench/README.md |
Documents the frozen Redis benchmark baseline. |
bench/message_hot_paths.rb |
Generalizes adapter isolation wording. |
bench/compare_http.rb |
Adjusts baseline Redis environment handling. |
app/models/room.rb |
Generalizes cache-store documentation. |
.github/workflows/ci.yml |
Removes Redis CI services. |
🧠 Review effort: Balanced
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| environment_variables = { RAILS_ENV: "production", SECRET_KEY_BASE: "isolated-benchmark-fixture-key", DISABLE_SSL: true, | ||
| SKIP_TELEMETRY: true, RAILS_LOG_LEVEL: "fatal", WEB_CONCURRENCY: 1, JOB_CONCURRENCY: 1, RAILS_MAX_THREADS: 5 | ||
| } | ||
| environment_variables[:REDIS_URL] = "redis://#{redis}:6379/0" if side == "before" |
There was a problem hiding this comment.
Fixed in 9300e44: the driver accepts --baseline-image for the before side (defaulting to --image), and the bench README notes when to pass it.
Generated by Claude Code
| command.concat environment(environment_variables) | ||
| command.concat [ options[:image], "bundle", "exec", "puma", "-C", "config/puma.rb" ] |
There was a problem hiding this comment.
Fixed in 9300e44: the after container now runs bin/rails db:prepare before execing Puma, so a frozen seed gets the cache, cable and queue databases. The baseline side still starts Puma directly to stay frozen.
Generated by Claude Code
| require "resque" | ||
| info = Resque.info | ||
| puts "pending=#{info[:pending]} working=#{info[:working]} failed=#{info[:failed]}" | ||
| Resque.queue_sizes.each { |queue, size| puts "#{queue}=#{size}" } |
There was a problem hiding this comment.
Leaving this as is: Resque.queue_sizes is defined in Resque 2.7.0 (lib/resque.rb, "Returns a hash, mapping queue names to queue sizes"), pipelining LLEN over Resque.queues. The drain check works on the release being upgraded from.
Generated by Claude Code
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 7ae38af30e
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: f5f8a87c57
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
f4619a0 to
6277f5b
Compare
6277f5b to
775adcd
Compare
775adcd to
8b22b56
Compare
Rails.cache moves from Redis to Solid Cache, in its own SQLite database that db:prepare creates from db/cache_schema.rb. Entries aren't copied, so the cache starts cold after the upgrade. The cache database keeps SQLite's own WAL checkpoints, since the checkpoint contender only checkpoints the primary database. The restore hook deletes it: a copy taken from a running volume can be torn, and the cache refills on demand. The HTTP benchmark prepares the databases before starting Puma, so a seed made before a new database or migration still boots.
Jobs move from Resque in the container's Redis to Solid Queue in its own SQLite database, so jobs queued when the container stops are still there when it starts again. Redis doesn't persist (save "", appendonly no), so jobs still queued in Resque at the upgrade are lost, as on any restart. The queue database is backed up and restored with the primary. A room deletion or banned user's content removal still queued at the backup would otherwise never run after a restore. The two databases are copied one after the other and the app writes them in separate commits, so a job enqueued while the backup runs can be missing from it: a room deleted in that moment comes back closed and without members, its messages left. A job running at the backup comes back claimed by its process, and fails once that process is pruned as dead. bin/boot prepares the databases before starting the web server and the workers, which now start together and both need the queue's tables. Solid Queue forks its workers from a supervisor, so the supervisor stops its checkpoint contender before forking and each worker starts its own, as Puma's workers do. Workers poll every 0.1 s, as Resque did. The queue's pool is five, for the worker's three threads plus its poller and heartbeat, independent of the web server's RAILS_MAX_THREADS. The test environment gets a queue database, so the backup test can use real Solid Queue records; jobs still run on the test adapter. The benchmark drivers take --baseline-image, since a baseline from before this change needs Resque's gems, which the current image no longer has, and prepare the databases before starting Puma, so a seed made before the queue database existed still boots.
With the cache and the jobs on Solid Cache and Solid Queue, Action Cable was the last thing using Redis. It moves to Solid Cable, in its own SQLite database, and Redis goes: the server process, the redis and kredis gems, the Debian package, bin/setup's container and CI's service. Development uses Solid Cable too, rather than the async adapter, because bin/dev runs several Puma workers and a broadcast has to reach sockets held by the others. Development caching uses the memory store. Broadcasts are transient, so the restore hook deletes the cable database with the cache's and db:prepare recreates both.
8b22b56 to
4540f47
Compare


Stacked on #345 and #346, which should merge first; only the last commit (
4540f47) is new here.With the cache on Solid Cache and the jobs on Solid Queue, Action Cable is the last thing using Redis. It moves to Solid Cable, in its own SQLite database, and Redis goes: the server process, the
redisandkredisgems, the Debian package,bin/setup's container and CI's service.Development uses Solid Cable as well, because
bin/devruns several Puma workers and a broadcast has to reach sockets held by the others. Broadcasts are transient, so the restore hook deletes the cable database together with the cache's.