Repository navigation
once-campfire-jai, a Campfire rewrite in Jai - #11
lucasmeijer wants to merge 3 commits into
Conversation
once-campfire-jai, a Jai port of Campfire, as an optional jai app: select it with --apps jai and JAI_IMAGE (default once-campfire-jai:app; source checkout ../once-campfire-jai). It is not in the default app list. Its topology record reports its event loops, request workers and readers, and the response cache size it shares with the Rust port (CAMPFIRE_RESPONSE_CACHE_MB). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
There was a problem hiding this comment.
🔵 Needs a closer look
Jai image compatibility and runtime topology remain unconfirmed by an independently validated benchmark run.
No open issues 🎉
What changed in this PR
Adds opt-in Jai support to the shared Campfire benchmark runner without changing the default implementation list.
Changes:
- Accepts
jaiand records its runtime topology. - Documents
JAI_IMAGEand explicit Jai selection.
[!TIP]
If you aren't ready for review, convert to a draft PR.
Click "Convert to draft" or rungh pr ready --undo.
Click "Ready for review" or rungh pr readyto reengage.
| File | Description |
|---|---|
| README.md | Documents Jai support and selection. |
| bench/compare.rb | Allows Jai and reports its runtime configuration. |
Review effort: Balanced
Jai's compiler is in closed beta, so nobody else can build its image from source. jai now defaults to the published build, ghcr.io/lucasmeijer/once-campfire-jai:e56ba63, which the harness pulls on first use (any image named with a registry path is pulled when it isn't local; local tags behave as before). With that image the source checkout must be at the revision it was built from, so the results record the source of the binary they measured. JAI_IMAGE still selects a build of one's own. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The image the harness pulls by default is now ghcr.io/lucasmeijer/once-campfire-jai:2ecd487 (registry digest sha256:bbbe5c64...). Since e56ba63 the app's event loop answers repeated identical page requests itself, large pages are gzipped from kept pieces, and live-update broadcasts are held back by slow subscribers' queues instead of disconnecting those subscribers.
|
As a Jai fan, this is very neat to see. Why do you think you got so many gains compared to C and Rust? Do you think it's honestly because LLMs are better at writing good code with Jai? Or rather than, to put it bluntly, someone who has access to Jai probably has more knowledge about performance and how things should work, including benchmarking, and you maybe just steered the LLM towards a better goal? Genuinely curious |
|
|
Makes sense. Having written quite a bit of high perf backend Rust it does make me wonder what is really the reason why Rust did so poorly in comparison. Comparison vs C makes sense to me because I'd kinda assume better libraries around strings/networking could make quite a bit of difference, where especially in Rust one could get quite a lot for very little work. Curious that Jai did so well, nice to see :) Maybe this is a good example of something to study. |
Hi. I rewrote Campfire in Jai, and I'd love for it to be included in the official benchmark. It passes all of the harness's checks: every response matches its route contract and every acknowledged write is audited, with nothing invalid in about 57 million validated responses.
There is one big catch:
ghcr.io/lucasmeijer/once-campfire-jai:2ecd487, built from2ecd487.Measurements
One session on one machine: a 16-vCPU AMD EPYC (Genoa) cloud VM. Each app ran in its production image on four vCPUs. The HTTP rows come from this harness; the others come from its load generator, through the scripts in
bench/scenarios. C# is devmobasa/once-campfire-dotnet, which this harness doesn't list.A caveat first: most rewrites, this one included, benchmaxx the page workloads. The harness reads each page with 16 connections that share one signed-in session and ask for one URL over and over, so a cache of finished pages answers nearly every request. Jai goes furthest: its event loop resends the previous identical response itself. Real users rarely repeat a request, and every new message invalidates the caches, so those numbers say little about production. The workloads that do reflect production are posting a message, the first page load after a new message, live updates at a steady pace, and memory.
¹ C connected only 144–235 of the clients; its figures are for those. ² Rust disconnects clients that fall 256 messages behind, so not every client got every message. Jai used to do the same, and failed the 1,000-client row too. Its posts are now held back by slow subscribers' queues instead, as Rails and C# effectively do. A post is answered once its message is queued for every subscriber.
The revisions measured were Rails
ce7b5d4, C873a51f, Rust6dae2fd, Jai2ecd487and C#3286f07, all latest at the time. Full method, ranges and caveats are in docs/benchmarks.md.What changed since the first version of this PR
2ecd487; it wase56ba63. The only change in this PR's code is that revision.