A pure spinel-Ruby Redis client speaking RESP2 over spinel's sp_net
sockets. The require string is redis and the command surface mirrors
redis-rb's contracts, so code written
against redis-rb (or transpiled from it) resolves here unchanged.
require "redis"
r = Redis.new("127.0.0.1", 6379)
r.set("k", "v")
r.get("k") # => "v"
r.get("missing") # => nil
r.exists?("k") # => true
r.hgetall("h") # => {"f" => "1", ...}Redis.new also takes redis-rb's url: form, with the password,
ACL username and database number in the URL or as keywords:
r = Redis.new(url: "redis://:s3cret@cache.internal:6380/2")
r = Redis.new(url: ENV["REDIS_URL"], db: 3) # db: overrides the URL'sThe URL is read the way redis-rb 5 reads it: a user with no password
is the password, the password is percent-decoded, and the database
comes from the path or a db= query parameter. After connecting the
client sends AUTH when there is a password and SELECT when the
database isn't 0. A refusal there is a Redis::CannotConnectError, as
in redis-rb. RedisURL.redact(url) hides the password for logs.
Errors use redis-rb's classes, so one rescue works against either
client. An error reply is a Redis::CommandError (or its
WrongTypeError, PermissionError or OutOfMemoryError subclass),
and a socket that fails is a Redis::CannotConnectError or
Redis::ConnectionError, both Redis::BaseConnectionError. All of
them are Redis::BaseError, a StandardError. Messages are
unchanged from 0.1.1.
read_timeout: bounds how long a command waits for its reply, in
seconds, as in redis-rb. When it runs out the call raises
Redis::TimeoutError (a Redis::BaseConnectionError) and the
connection is closed, since a late reply would otherwise be read as the
next command's. redis-rb reconnects at that point; here every later
call raises Redis::ConnectionError, and the caller opens a new client.
The default, nil, waits as long as the server takes. Pub/sub reads
have no deadline.
connect_timeout: bounds the connect the same way. A server that
doesn't answer in time is a Redis::CannotConnectError, as in redis-rb,
and the deadline covers every address the host resolves to. Both need a
Spinel whose sp_net has the timed calls (sp_net_recv_some_timeout
from matz/spinel#9584, and sp_net_connect_timeout).
r = Redis.new(url: ENV["REDIS_URL"], connect_timeout: 1.0, read_timeout: 1.0)brpop takes redis-rb's timeout: keyword and pops from the first of
the keys that has an element. It returns [key, value], or [] when
the timeout passes (redis-rb returns nil there). While it waits, the
read deadline is its timeout plus read_timeout:, as redis-rb sets it,
so a server that stops answering is still noticed. timeout: 0 waits
for a value as long as it takes, with no read deadline:
r = Redis.new(url: ENV["REDIS_URL"], read_timeout: 5.0)
r.brpop(["queue:critical", "queue:default"], timeout: 2)
# => ["queue:default", job], or [] after 2 s (the read gives up at 7 s)rpush takes one value or an Array of them, and zremrangebyscore and
zremrangebyrank trim a sorted set; their bounds go out with to_s, so
"-inf", "+inf" and "(5" work as Redis reads them.
Lua scripts run by SHA, as Sidekiq's scheduler runs its
ZPOPBYSCORE script. script(:load, lua) returns the SHA, as
redis-rb's does, and evalsha takes the keys and arguments
positionally, as redis-rb's positional form does. A script the server
doesn't hold (after a restart or SCRIPT FLUSH) raises
Redis::NoScriptError, so the caller loads it again and retries:
sha = r.script(:load, lua)
r.evalsha(sha, ["schedule"], [Time.now.to_f.to_s]) # => String or nilPipelining takes redis-rb's block form. The commands go out in one write, every reply is read, and the first error reply is raised after that, so the connection stays in step:
r.pipelined do |p|
p.sadd("queues", "default")
p.lpush("queue:default", job)
end # => [1, 1]The block's recorder has set (with ex: and nx:), del, unlink,
exists, incrby, expire, lpush, rpush, rpop, sadd, hset,
zadd, zremrangebyscore and zremrangebyrank. A pipeline is not a
transaction.
multi takes the same block and recorder, and wraps the commands in
MULTI ... EXEC, still in one write, so the server runs them as one
transaction. This is how Sidekiq writes a job to its dead set:
r.multi do |tx|
tx.zadd("dead", now, job)
tx.zremrangebyscore("dead", "-inf", now - 15552000)
tx.zremrangebyrank("dead", 0, -10000)
end # => [true, 0, 0]Errors are raised the way redis-rb raises them. If the server refuses
to queue a command (an unknown command or a wrong argument count),
nothing runs and that refusal is raised, not EXECABORT. If a command
fails while EXEC runs (WRONGTYPE), the others still ran, as Redis
does it, and the first error is raised once every reply is read.
Either way the connection stays in step. An empty block sends nothing
and returns [].
Pub/sub mirrors redis-rb's block DSL. A subscribed connection is dedicated (RESP2), so pubsub gets its own; the loop blocks until the subscription count drains to zero, and handlers drive unsubscription:
ps = Redis.pubsub("127.0.0.1", 6379)
ps.subscribe("news") do |on|
on.subscribe { |ch, count| }
on.message { |ch, msg| ps.unsubscribe_all if msg == "quit" }
on.unsubscribe { |ch, count| }
end
# psubscribe / on.pmessage for patterns; subscribe_many for several channelsThe blocking subscribe read parks scheduler-aware under SP_THREADS, so
a subscribe loop in one fiber coexists with other work.
Embedders with their own event loop skip the blocking loop entirely:
subscribe_start sends the command, fd exposes the socket, and
drain(listener) does one read plus dispatch of every complete push —
park on the fd however you like between drains. This is the surface
Tep::RedisFeed (roundhouse) rides to feed tep's in-process broadcast
registry from cross-process Redis pub/sub:
l = RedisListener.new
l.message { |ch, msg| Broadcast.publish(ch, msg) }
ps.subscribe_start("timeline:1")
loop { ps.drain(l) if fd_readable?(ps.fd) } # io_wait / poll betweenNo C, no FFI beyond the eight sp_net externs the compiler links into
every binary: protocol encode/decode is pure Ruby (compiled to native
code by spinel), the same architecture as redis-rb's own default driver.
Bulk strings are binary-safe end to end (write_bytes with explicit
lengths on the send side, :binstr reads on the receive side — embedded
NUL and CRLF bytes round-trip through a real server; the live test
proves it).
Three layers, each testable one level down:
| file | what | tested how |
|---|---|---|
redis/resp.rb |
RESP2 encoding (pure functions) + incremental reply parser | dual-runtime: same test runs compiled and under CRuby |
redis/client.rb |
typed per-command methods over an injected transport duck | dual-runtime, against a scripted transport |
redis/errors.rb |
redis-rb's error classes | dual-runtime, against a scripted transport |
redis/url.rb |
redis:// URL parsing and redaction |
dual-runtime |
redis/pipeline.rb |
the pipelined recorder |
dual-runtime, against a scripted transport |
redis/pubsub.rb |
SUBSCRIBE-mode dispatch loop + block-DSL listener | dual-runtime (scripted) + compiled-only live test |
redis/sock.rb + redis/connection.rb |
the real sp_net transport |
compiled-only live test against a real redis-server (committed snapshot) |
The parser is a try_next/reply state machine rather than a
value-or-nil API: booleans and typed ivars keep every call site
monomorphic. Commands are individual typed methods (incrby(key, n),
lrange(key, start, stop)) built on a typed Array<String> plumbing
layer — there is deliberately no public call(*args) funnel.
sp_net recv parks scheduler-aware under SP_THREADS, so blocking-style
calls are already fiber-friendly.
examples/ ports redis-rb v5.4.1's examples; each has a gated copy in
test/ running against a test-owned server, so spin publish gates on
the ported behavior:
basic.rb,incr-decr.rb,list.rb,sets.rb— verbatim flows (these forcedltrimandsinterinto the client surface). One correction to upstream:basic.rb'sr['foo'] = 'bar'sugar was removed from redis-rb in 5.0 — as written upstream it falls throughmethod_missingand sends a literal[]=command to the server. The oracle harness caught this; the port (and this client) uses set/get.pubsub.rb— reshaped from interactive (redis-cli in a second terminal) to self-driving: a second connection publishes from inside the subscriber's callbacks.trap(:INT)and the rescue/retry reconnect wrapper are not ported (ledger).prefork.rb— theunicorn/example's lesson translated tosp_net_forkdiscipline: close before fork, connect per worker after.- Not ported:
dist_redis.rb/sentinel*(v0.1 exclusions below) andconsistency.rb(a random-driven soak tool; the API it exercises is covered by the lanes above).
spin test # resp + client + pubsub lanes also run under CRuby and must match
sh oracle/run.sh # replay every snapshot flow through the REAL redis-rb gemThe live lanes start their own redis-server on private ports (no
persistence, shut down after) and carry committed .expected snapshots;
they need redis-server/redis-cli on PATH.
oracle/run.sh is the second conformance axis: each oracle/*.rb is a
twin of a snapshot-gated flow, run under CRuby against the real
redis-rb gem (no -I, so require "redis" resolves to the gem), and
its output is diffed against the same committed snapshots the compiled
client froze. Zero hand-authored expectations; a diff on either side is
a contract divergence. Snapshot-determinism rules the oracle enforced:
set replies are sorted, and unsubscribes are explicit per-channel (the
no-arg form's confirmation order is server-internal and flaps between
runs).
Deliberately not in this release, recorded rather than implied:
- RESP3 — RESP2 only; no client-side caching push. A dedicated connection per subscriber is the model, as in redis-rb's default.
- TLS —
sp_nethas no TLS yet (matz/spinel#1054); TLS-only managed Redis endpoints (ElastiCache in-transit encryption etc.) need a local stunnel/nginx hop for now.rediss://URLs raise anArgumentErrorsaying so. - Sentinel / Cluster /
dist_redis— single connection to a single server. Mastodon supports Sentinel via env, so this is a future ledger entry, not a never. - Unix sockets,
WATCH, and the SCAN family beyondsscan_each— straightforward additions when a consumer needs them.unix://URLs raise anArgumentError.multihas noWATCH/UNWATCH/DISCARD, and its block is the recorder, so a command can't be sent outside the transaction from inside it. sscan_eachtakes a block, withmatch:andcount:; redis-rb's Enumerator form (no block) andsscan,scan,hscanandzscanaren't mirrored.- Variadic command forms are narrowed to typed arities (
del(key),zadd(key, score, member)).lpush,rpushandsaddtake one value or an Array, andunlinkone key or an Array, as redis-rb's do;saddreturns the Integer added-count, redis-rb's legacy Boolean mode is not mirrored.hsettakes a field and value or a Hash of them, not redis-rb's flathset(key, f1, v1, f2, v2)splat.settakesex:andnx:(and answers true or false withnx:, as redis-rb does);px:,xx:,keepttl:andget:aren't mirrored.zaddandzremtake one member and return a Boolean, as redis-rb's single-member forms do.zadd'snx:/xx:/incr:options andzrange'swith_scores:are not mirrored;zscorereads one score. Redis.newtakeshostandportpositionally or aurl:, not redis-rb'shost:/port:keywords, and it doesn't readREDIS_URLfrom the environment by itself as redis-rb does. Noreconnect_attempts: once a connection is lost every later call raises, and the caller opens a new one.- Bulk replies are binary (
ASCII-8BIT) Strings, so they are binary-safe; redis-rb returns them tagged UTF-8. The bytes are the same, but==against a UTF-8 literal with non-ASCII characters is false here. Compare bytes, orforce_encoding("UTF-8")the reply. brpopreturns[]rather thannilwhen its timeout passes, which keeps one return type for spinel. It takes its keys as an Array, not redis-rb's splat, and isn't in the pipeline recorder (a blocking command holds every reply queued behind it).blpop,brpoplpushand thelmovefamily aren't mirrored.scripttakes only:loadand:flush, andevalshatakes(sha, keys, argv)positionally, not redis-rb'skeys:/argv:keywords. A script's reply must be a bulk string or nil; an Integer or Array reply raisesRedis::BaseErrorinstead.evaland the otherSCRIPTsubcommands aren't mirrored.pipelinedandmultitake noexception: false, and their recorder has only the commands listed above. Their return values hold the coerced replies, as redis-rb's do, with an error reply's message in its place.
Three compiler defects were found while building this, filed with minimal repros, and all three were fixed upstream the same day (spinel a7e42e90):
- matz/spinel#1773 —
&&evaluated a later operand's receiver before an earlier operand's side effect. Fixed (38923ee9). - matz/spinel#1775 —
return <expr>insidebeginpopped the rescue frame before evaluating. Fixed (5c56c94c). - matz/spinel#1774 — user
class named
Valcollided with a runtime typedef. Fixed (3e7173e9 — user classes now get a distinct C stem).
The in-tree code kept the workaround-era shapes (sequential statements, assign-then-return) — they're valid Ruby either way and keep the package compiling on pre-fix spinel builds.