Compare against theine and sturdyc, and on features

CHANGELOG.md

@@ -6,6 +6,13 @@ While the major version is 0, the public API may change in any release.
 
 ## [Unreleased]
 
+### Documentation
+
+- The comparison benchmarks include theine and sturdyc, `make bench-compare` adds a serial
+  `Get` and five runs of each, and the README tables are medians of those runs.
+- The README compares features with otter, theine, sturdyc and ttlcache, including what
+  they do that this library does not, and points to sturdyc for background refresh.
+
 ## [0.7.0] - 2026-09-30
 
 ### Added

Makefile

@@ -15,7 +15,7 @@ bench:
 
 .PHONY: bench-compare
 bench-compare:
-	cd benchmarks && go test -run '^$$' -bench 'Get$$|Set$$|Mixed$$' -benchmem ./...
+	cd benchmarks && go test -run '^$$' -bench 'Get$$|GetSerial$$|Set$$|Mixed$$' -benchmem -count=5 ./...
 	cd benchmarks && go test -run '^$$' -bench HitRate -benchtime=300000x ./...
 
 .PHONY: cover

README.md

@@ -493,65 +493,102 @@ calling it is still better than relying on when the collector gets around to it.
 
 The `benchmarks/` module measures this cache against the ones it would otherwise be
 replacing. It is a separate module so the root stays dependency-free; `make bench-compare`
-runs it. Same machine as above, 256-byte values, a budget sized to 4096 of them. The
-sanecache rows are its knobs, each one added to the row above it.
+runs it. Same machine as above, Go 1.24, 256-byte values, a budget sized to 4096 of them,
+medians of five runs. The sanecache rows are its knobs, each one added to the row above it.
 
 | ns/op | serial `Get` | `Get` ×8 | `Set` ×8 | 90/10 ×8 |
 |---|---:|---:|---:|---:|
-| sanecache, defaults | 61 | 139 | 226 | 151 |
-| ⤷ `Shards: 16` | 64 | 42 | 91 | 49 |
-| ⤷ + `ClockGranularity` | 34 | 32 | 84 | 42 |
-| ⤷ + `ClearOnFull` | 32 | 20 | 90 | 39 |
-| [otter](https://github.com/maypok86/otter) v2 | 77 | 14 | 263 | 30 |
-| [ristretto](https://github.com/dgraph-io/ristretto) v2 | 86 | 21 | 324 | 65 |
-| [golang-lru](https://github.com/hashicorp/golang-lru) `v2/expirable` | 52 | 142 | 190 | 165 |
-| [ttlcache](https://github.com/jellydator/ttlcache) v3 | 63 | 177 | 267 | 192 |
+| sanecache, defaults | 62 | 141 | 229 | 156 |
+| ⤷ `Shards: 16` | 64 | 42 | 90 | 49 |
+| ⤷ + `ClockGranularity` | 33 | 32 | 81 | 38 |
+| ⤷ + `ClearOnFull` | 31 | 19 | 86 | 38 |
+| [otter](https://github.com/maypok86/otter) v2 | 80 | 14 | 268 | 29 |
+| [theine](https://github.com/Yiling-J/theine-go) | 91 | 13 | 174 | 57 |
+| [ristretto](https://github.com/dgraph-io/ristretto) v2 | 87 | 21 | 252 | 70 |
+| [sturdyc](https://github.com/viccon/sturdyc) | 62 | 32 | 121 | 67 |
+| [golang-lru](https://github.com/hashicorp/golang-lru) `v2/expirable` | 54 | 140 | 185 | 168 |
+| [ttlcache](https://github.com/jellydator/ttlcache) v3 | 63 | 175 | 264 | 193 |
 
 The first column is what one lookup costs; the second is what eight goroutines get out of
 the same machine. Read them together, because they say opposite things. Per operation this
-cache is *cheaper* than otter and ristretto — they spend their time maintaining a frequency
-sketch and a set of ring buffers that only pay off later. What they buy with it is scaling:
-otter turns eight cores into 5.4x the throughput, this one into 1.6x, and unsharded into
-less than 1x, because reads take a lock and locks are where cores queue.
+cache is *cheaper* than otter, theine and ristretto — they spend their time maintaining a
+frequency sketch and a set of ring buffers that only pay off later. What they buy with it is
+scaling: otter and theine turn eight cores into 6x to 7x the throughput, this one into 1.6x,
+and unsharded into less than 1x, because reads take a lock and locks are where cores queue.
 
 So the honest shape of it is not "slower". It is: **the same work per operation, and less
 of the machine used to do it in parallel.** Whether that matters is a question about the
-read rate. At one lookup per request, the gap between 20 ns and 14 ns is six nanoseconds
+read rate. At one lookup per request, the gap between 19 ns and 13 ns is six nanoseconds
 against a request budget measured in milliseconds. It starts to matter when one request
 does thousands of lookups, or when the cache more or less is the service.
 
-Writes are the other way round: 91 ns against 190 to 324. An admission policy has to decide
-whether to accept each write and update its sketch, and that costs more than taking a lock
-does. A write-heavy cache is the case where this library is simply faster.
+Writes are the other way round: 81 to 90 ns against 121 to 268. An admission policy has to
+decide whether to accept each write and update its sketch, and that costs more than taking a
+lock does; sturdyc, closest here, has no admission policy either. A write-heavy cache is the
+case where this library is simply faster.
 
 The number that usually matters more than any of those is how much of a fixed budget each
 policy turns into hits. 4096 entries against a key space of 100,000:
 
 | %hit | zipf s=1.20 | zipf s=1.01 | zipf + scan |
 |---|---:|---:|---:|
 | sanecache (LRU) | 87.3 | 65.8 | 78.0 |
-| sanecache (ClearOnFull) | 83.1 | 58.0 | 73.7 |
-| otter v2 | 88.9 | **71.4** | 80.8 |
-| ristretto v2 | 87.4 | 68.7 | 78.9 |
+| sanecache (ClearOnFull) | 83.2 | 57.9 | 73.7 |
+| otter v2 | 88.9 | **71.3** | **80.8** |
+| theine | **89.0** | **71.3** | 80.7 |
+| ristretto v2 | 87.3 | 68.7 | 79.0 |
+| sturdyc | 84.9 | 61.6 | 75.3 |
 | golang-lru `expirable` | 87.3 | 65.8 | 78.0 |
 | ttlcache v3 | 87.3 | 65.8 | 78.0 |
 
-When the hot set fits, every policy looks the same and the extra machinery buys 1.6 points.
-When it does not, W-TinyLFU is worth 5.6 points of hit rate over LRU — and 5.6 points of
+When the hot set fits, every policy looks the same and the extra machinery buys 1.7 points.
+When it does not, W-TinyLFU is worth 5.5 points of hit rate over LRU — and 5.5 points of
 upstream traffic is worth more than every nanosecond in the table above it. That is the real
-reason to pick otter, and it is a better one than throughput.
+reason to pick otter or theine, and it is a better one than throughput.
 
-The same table prices the cheap read lock: `ClearOnFull` buys its 40% by giving up 7.8
+The same table prices the cheap read lock: `ClearOnFull` buys its 40% by giving up 7.9
 points of hit rate. It is a trade for caches whose access order is flat, not a free win.
+sturdyc makes a milder version of the same trade: when a shard is full it drops the
+least recently used tenth of it at once, which costs it 4 points against a plain LRU.
+
+### The features are not the difference either
+
+Loading a cold key once, loading many keys in one call and a budget in something other
+than entries are not unique to this library. As of otter v2.3.0,
+[theine](https://github.com/Yiling-J/theine-go) v0.6.2,
+[sturdyc](https://github.com/viccon/sturdyc) v1.1.6 and ttlcache v3.4.1:
+
+| | one load per key | batch load | budget by cost | "not found" remembered | value over budget |
+|---|---|---|---|---|---|
+| sanecache | `GetOrLoad` | `GetManyOrLoad` | `MaxBytes` + `Cost` | `SetNegative` | `ErrTooLarge` |
+| otter v2 | `Get` + `Loader` | `BulkGet` | `MaximumWeight` + `Weigher` | no: `ErrNotFound` deletes | accepted, then evicted |
+| theine | loading cache | — | `Cost` | — | `Set` returns `false` |
+| sturdyc | `GetOrFetch` | `GetOrFetchBatch` | — | `WithMissingRecordStorage` | — |
+| ttlcache v3 | `SuppressedLoader` | — | `WithMaxCost` | — | evicts everything, itself last |
+
+They also do things this library does not. otter and sturdyc refresh entries in the
+background before they expire, so a hot key never goes cold; otter and theine can save the
+cache to a file and load it back; sturdyc coalesces refreshes into batches and can sit in
+front of a distributed store. sturdyc can also hold several value types in one cache, with
+typed package-level functions on top, though its capacity is a count of entries rather than
+bytes.
+
+What is left is the combination rather than any single feature: a byte budget that refuses
+what does not fit, missing records charged against that budget, and value types with costs
+of their own [under one budget](#several-value-types-under-one-budget). If background
+refresh matters more to you than that, otter or sturdyc is the better fit.
 
 ## When to use something else
 
 - You want the best possible hit rate for a given memory budget, and admission policies and
   access-frequency estimation are worth the complexity → [otter](https://github.com/maypok86/otter).
   The hit-rate table above is the size of the prize, and it is the biggest number on this page.
 - Your read path is hot enough that how well a cache scales across cores is a real
-  difference → otter or [ristretto](https://github.com/dgraph-io/ristretto), and budget for
-  `Wait()` in the tests.
+  difference → otter or [ristretto](https://github.com/dgraph-io/ristretto); with ristretto,
+  budget for `Wait()` in the tests.
+- You call an upstream API and want entries refreshed before they expire, refreshes
+  coalesced into batches, or a distributed store behind the cache →
+  [sturdyc](https://github.com/viccon/sturdyc).
 - You are caching hundreds of megabytes and GC pressure from millions of live pointers is
   your actual problem → [bigcache](https://github.com/allegro/bigcache) or
   [freecache](https://github.com/coocood/freecache), which reduce GC scanning by storing entries
@@ -564,7 +601,7 @@ points of hit rate. It is a trade for caches whose access order is flat, not a f
 
 ## Status
 
-v0.6. The API above is what exists and is tested; expect it to move before v1.
+v0.7. The API above is what exists and is tested; expect it to move before v1.
 
 Contributions are welcome — see [CONTRIBUTING.md](CONTRIBUTING.md) for what this
 library optimises for before proposing a change.

benchmarks/compare_test.go

@@ -19,11 +19,13 @@ import (
 	"testing"
 	"time"
 
+	"github.com/Yiling-J/theine-go"
 	"github.com/andared/sanecache"
 	"github.com/dgraph-io/ristretto/v2"
 	"github.com/hashicorp/golang-lru/v2/expirable"
 	"github.com/jellydator/ttlcache/v3"
 	"github.com/maypok86/otter/v2"
+	"github.com/viccon/sturdyc"
 )
 
 const (
@@ -61,9 +63,11 @@ func factories() []factory {
 			return openSane(16, time.Millisecond, sanecache.ClearOnFull)
 		}},
 		{"otter", openOtter},
+		{"theine", openTheine},
 		{"ristretto", openRistretto},
 		{"golang-lru-expirable", openExpirable},
 		{"ttlcache", openTTLCache},
+		{"sturdyc", openSturdyc},
 	}
 }
 
@@ -108,6 +112,28 @@ func (o otterCache) wait() {}
 
 func (o otterCache) close() {}
 
+type theineCache struct{ c *theine.Cache[string, []byte] }
+
+func openTheine() cache {
+	c, err := theine.NewBuilder[string, []byte](capacity * valueSize).
+		Cost(func(v []byte) int64 { return int64(len(v)) }).
+		Build()
+	if err != nil {
+		panic(err)
+	}
+
+	return theineCache{c}
+}
+
+func (t theineCache) get(key string) bool { _, ok := t.c.Get(key); return ok }
+
+// Cost 0 asks theine to call the Cost function given to the builder.
+func (t theineCache) set(key string, v []byte) { t.c.SetWithTTL(key, v, 0, benchTTL) }
+
+func (t theineCache) wait() {}
+
+func (t theineCache) close() { t.c.Close() }
+
 type ristrettoCache struct {
 	c *ristretto.Cache[string, []byte]
 }
@@ -180,6 +206,26 @@ func (t ttlCache) wait() {}
 
 func (t ttlCache) close() { t.c.Stop() }
 
+type sturdycCache struct{ c *sturdyc.Client[[]byte] }
+
+func openSturdyc() cache {
+	// Entry count rather than bytes: this cache has no notion of cost. Shards
+	// and the share evicted when full are the values from its own README.
+	// Its expiry sweeper never exits and keeps the client alive, so every run
+	// would leak a full cache into the ones after it; nothing expires mid-run,
+	// so turning it off changes nothing else.
+	return sturdycCache{sturdyc.New[[]byte](capacity, 10, benchTTL, 10,
+		sturdyc.WithNoContinuousEvictions())}
+}
+
+func (s sturdycCache) get(key string) bool { _, ok := s.c.Get(key); return ok }
+
+func (s sturdycCache) set(key string, v []byte) { s.c.Set(key, v) }
+
+func (s sturdycCache) wait() {}
+
+func (s sturdycCache) close() {}
+
 // keySet precomputes keys so that the benchmarks measure the cache rather than
 // strconv.
 func keySet(n int) []string {

benchmarks/go.mod

@@ -7,19 +7,23 @@ go 1.24.0
 replace github.com/andared/sanecache => ../
 
 require (
+	github.com/Yiling-J/theine-go v0.6.2
 	github.com/andared/sanecache v0.1.0
 	github.com/dgraph-io/ristretto/v2 v2.4.2
 	github.com/hashicorp/golang-lru/v2 v2.0.7
 	github.com/jellydator/ttlcache/v3 v3.4.1
 	github.com/maypok86/otter/v2 v2.3.0
+	github.com/viccon/sturdyc v1.1.6
 )
 
 require (
 	github.com/cespare/xxhash/v2 v2.3.0 // indirect
 	github.com/davecgh/go-spew v1.1.1 // indirect
 	github.com/dustin/go-humanize v1.0.1 // indirect
+	github.com/klauspost/cpuid/v2 v2.0.9 // indirect
 	github.com/pmezard/go-difflib v1.0.0 // indirect
 	github.com/stretchr/testify v1.11.1 // indirect
+	github.com/zeebo/xxh3 v1.0.2 // indirect
 	golang.org/x/sync v0.16.0 // indirect
 	golang.org/x/sys v0.36.0 // indirect
 	gopkg.in/yaml.v3 v3.0.1 // indirect

benchmarks/go.sum

@@ -1,3 +1,5 @@
+github.com/Yiling-J/theine-go v0.6.2 h1:1GeoXeQ0O0AUkiwj2S9Jc0Mzx+hpqzmqsJ4kIC4M9AY=
+github.com/Yiling-J/theine-go v0.6.2/go.mod h1:08QpMa5JZ2pKN+UJCRrCasWYO1IKCdl54Xa836rpmDU=
 github.com/cespare/xxhash/v2 v2.3.0 h1:UL815xU9SqsFlibzuggzjXhog7bL6oX9BbNZnL2UFvs=
 github.com/cespare/xxhash/v2 v2.3.0/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs=
 github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
@@ -8,23 +10,38 @@ github.com/dgryski/go-farm v0.0.0-20240924180020-3414d57e47da h1:aIftn67I1fkbMa5
 github.com/dgryski/go-farm v0.0.0-20240924180020-3414d57e47da/go.mod h1:SqUrOPUnsFjfmXRMNPybcSiG0BgUW2AuFH8PAnS2iTw=
 github.com/dustin/go-humanize v1.0.1 h1:GzkhY7T5VNhEkwH0PVJgjz+fX1rhBrR7pRT3mDkpeCY=
 github.com/dustin/go-humanize v1.0.1/go.mod h1:Mu1zIs6XwVuF/gI1OepvI0qD18qycQx+mFykh5fBlto=
+github.com/google/go-cmp v0.6.0 h1:ofyhxvXcZhMsU5ulbFiLKl/XBFqE1GSq7atu8tAmTRI=
+github.com/google/go-cmp v0.6.0/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY=
 github.com/hashicorp/golang-lru/v2 v2.0.7 h1:a+bsQ5rvGLjzHuww6tVxozPZFVghXaHOwFs4luLUK2k=
 github.com/hashicorp/golang-lru/v2 v2.0.7/go.mod h1:QeFd9opnmA6QUJc5vARoKUSoFhyfM2/ZepoAG6RGpeM=
 github.com/jellydator/ttlcache/v3 v3.4.1 h1:bOdXmXiycyK6E6Qjyuj5vl+/vU3SCOoDs8a86NbHjAQ=
 github.com/jellydator/ttlcache/v3 v3.4.1/go.mod h1:j7LO12PNghFg5+0v9budMAT4rDK4JY969jb9vOdOBBk=
+github.com/klauspost/cpuid/v2 v2.0.9 h1:lgaqFMSdTdQYdZ04uHyN2d/eKdOMyi2YLSvlQIBFYa4=
+github.com/klauspost/cpuid/v2 v2.0.9/go.mod h1:FInQzS24/EEf25PyTYn52gqo7WaD8xa0213Md/qVLRg=
+github.com/kr/pretty v0.1.0 h1:L/CwN0zerZDmRFUapSPitk6f+Q3+0za1rQkzVuMiMFI=
+github.com/kr/pretty v0.1.0/go.mod h1:dAy3ld7l9f0ibDNOQOHHMYYIIbhfbHSm3C4ZsoJORNo=
+github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY=
+github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE=
 github.com/maypok86/otter/v2 v2.3.0 h1:8H8AVVFUSzJwIegKwv1uF5aGitTY+AIrtktg7OcLs8w=
 github.com/maypok86/otter/v2 v2.3.0/go.mod h1:XgIdlpmL6jYz882/CAx1E4C1ukfgDKSaw4mWq59+7l8=
 github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
 github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
 github.com/stretchr/testify v1.11.1 h1:7s2iGBzp5EwR7/aIZr8ao5+dra3wiQyKjjFuvgVKu7U=
 github.com/stretchr/testify v1.11.1/go.mod h1:wZwfW3scLgRK+23gO65QZefKpKQRnfz6sD981Nm4B6U=
+github.com/viccon/sturdyc v1.1.6 h1:ISrV4YrmkakI1iON09gYNF4YA5gkR3ZOC5EErbJ3S8Q=
+github.com/viccon/sturdyc v1.1.6/go.mod h1:OCBEgG/i48uugKQ498UQlfMHmf5j8MYY8a4BApfVnMo=
+github.com/zeebo/assert v1.3.0 h1:g7C04CbJuIDKNPFHmsk4hwZDO5O+kntRxzaUoNXj+IQ=
+github.com/zeebo/assert v1.3.0/go.mod h1:Pq9JiuJQpG8JLJdtkwrJESF0Foym2/D9XMU5ciN/wJ0=
+github.com/zeebo/xxh3 v1.0.2 h1:xZmwmqxHZA8AI603jOQ0tMqmBr9lPeFwGg6d+xy9DC0=
+github.com/zeebo/xxh3 v1.0.2/go.mod h1:5NWz9Sef7zIDm2JHfFlcQvNekmcEl9ekUZQQKCYaDcA=
 go.uber.org/goleak v1.3.0 h1:2K3zAYmnTNqV73imy9J1T3WC+gmCePx2hEGkimedGto=
 go.uber.org/goleak v1.3.0/go.mod h1:CoHD4mav9JJNrW/WLlf7HGZPjdw8EucARQHekz1X6bE=
 golang.org/x/sync v0.16.0 h1:ycBJEhp9p4vXvUZNszeOq0kGTPghopOL8q0fq3vstxw=
 golang.org/x/sync v0.16.0/go.mod h1:1dzgHSNfp02xaA81J2MS99Qcpr2w7fw1gpm99rleRqA=
 golang.org/x/sys v0.36.0 h1:KVRy2GtZBrk1cBYA7MKu5bEZFxQk4NIDV6RLVcC8o0k=
 golang.org/x/sys v0.36.0/go.mod h1:OgkHotnGiDImocRcuBABYBEXf8A9a87e/uXjp9XT3ks=
-gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405 h1:yhCVgyC4o1eVCa2tZl7eS0r+SDo693bJlVdllGtEeKM=
 gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
+gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127 h1:qIbj1fsPNlZgppZ+VLlY7N33q108Sa+fhmuc+sWQYwY=
+gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
 gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
 gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=