From a721a7c90b1165936f233c31e8bd23b598e46dc1 Mon Sep 17 00:00:00 2001 From: sepehr-safari Date: Wed, 23 Sep 2026 17:47:18 +0300 Subject: [PATCH] docs: take the benchmark numbers from the released binary The numbers came from a local build made with the release flags. They now come from the 0.3.1 binary on the release page, run through the same script, which is the build people actually run. Signing is the one that moved, about 31,000 a second rather than 22,700; everything else is within a few percent. The Linux aarch64 download is 1.75 MB, which rounds to 1.8. --- BENCHMARKS.md | 38 +++++++++++++++++++------------------- README.md | 2 +- 2 files changed, 20 insertions(+), 20 deletions(-) diff --git a/BENCHMARKS.md b/BENCHMARKS.md index c5c5a05..ae6a735 100644 --- a/BENCHMARKS.md +++ b/BENCHMARKS.md @@ -9,7 +9,7 @@ python3 bench/run.py zig-out/bin/deed It needs python3 and [websocat](https://github.com/vi/websocat). Everything happens on your machine: the relay it talks to is [`bench/relay.py`](bench/relay.py) on 127.0.0.1, and nothing is sent anywhere else. Each timing is the best of several runs, because on a shared machine noise only ever adds time. -Measured on an Apple M2 Pro with macOS 26.6, with deed 0.3.1 built as it is released (`ReleaseSafe`, stripped). +Measured on an Apple M2 Pro with macOS 26.6, running the deed 0.3.1 binary from the release page (`ReleaseSafe`, stripped), installed with the one-line installer. ## Size @@ -18,7 +18,7 @@ Measured on an Apple M2 Pro with macOS 26.6, with deed 0.3.1 built as it is rele | macOS, Apple Silicon | 2.3 MB | 1.6 MB | | macOS, Intel | 2.4 MB | 1.7 MB | | Linux, x86_64 | 2.7 MB | 1.8 MB | -| Linux, aarch64 | 2.4 MB | 1.7 MB | +| Linux, aarch64 | 2.4 MB | 1.8 MB | The Linux binaries are statically linked, so that is everything they need. @@ -26,12 +26,12 @@ The Linux binaries are statically linked, so that is everything they need. | | | | --- | --- | -| starting any process at all (`/usr/bin/true`), for scale | 1.56 ms | +| starting any process at all (`/usr/bin/true`), for scale | 1.57 ms | | `deed version` | 2.34 ms | -| `deed key generate` | 2.37 ms | -| `deed decode ` | 2.41 ms | -| `deed event`, signing one | 2.41 ms | -| `deed verify`, one | 3.02 ms | +| `deed key generate` | 2.39 ms | +| `deed decode ` | 2.43 ms | +| `deed event`, signing one | 2.46 ms | +| `deed verify`, one | 3.05 ms | | peak memory, `deed version` | 1.7 MB | deed's own share of a one-shot command is under a millisecond; the rest is the operating system starting a process. Verifying one event adds about 0.6 ms to that. @@ -40,10 +40,10 @@ deed's own share of a one-shot command is under a millisecond; the rest is the o | 10,000 records through stdin | | | --- | --- | -| signing (`deed event -`) | 22,660 per second | -| verifying (`deed verify`) | 33,270 per second | -| decoding npubs (`deed decode`) | 586,718 per second | -| peak memory, verifying 10,000 | 3.8 MB | +| signing (`deed event -`) | 30,986 per second | +| verifying (`deed verify`) | 32,523 per second | +| decoding npubs (`deed decode`) | 568,831 per second | +| peak memory, verifying 10,000 | 3.9 MB | ## The local store @@ -51,13 +51,13 @@ deed's own share of a one-shot command is under a millisecond; the rest is the o | | | | --- | --- | -| storing them from a loopback relay (`req --store`), each signature checked first | 16,864 per second | -| the store file for them | 94 MB | +| storing them from a loopback relay (`req --store`), each signature checked first | 17,379 per second | +| the store file for them | 95 MB | | one event by id (`req --local -i`) | 3.25 ms | -| one profile (`-a -k 0 -l 1`) | 3.30 ms | -| one author's latest 50 notes | 3.47 ms | -| the latest 500 notes (`-k 1 -l 500`) | 4.94 ms | -| every note, 98,000 of them | 284 ms | +| one profile (`-a -k 0 -l 1`) | 3.21 ms | +| one author's latest 50 notes | 3.50 ms | +| the latest 500 notes (`-k 1 -l 500`) | 5.03 ms | +| every note, 98,000 of them | 286 ms | The query times are the whole command, starting deed included, so a lookup costs about a millisecond more than `deed version` does. Events are written to the store in batches, in one transaction each, and a batch is stored before any of it is printed. @@ -65,7 +65,7 @@ The query times are the whole command, starting deed included, so a lookup costs | | | | --- | --- | -| `req` for 500 events | 44.5 ms, of which the test relay starting a Python process is about 20 | -| `publish` 1,000 events, waiting for each one's OK | 6,884 per second | +| `req` for 500 events | 43.9 ms, of which the test relay starting a Python process is about 20 | +| `publish` 1,000 events, waiting for each one's OK | 6,933 per second | These leave the network out, so they measure deed and not the distance to a relay. Against a real relay, the round trip to it dominates. diff --git a/README.md b/README.md index 7a33551..0d64de3 100644 --- a/README.md +++ b/README.md @@ -107,7 +107,7 @@ Scripts branch on these, so they are part of the interface and not free to drift ## How fast it is -A one-shot command runs in about 2.3 ms and under 2 MB of memory. deed signs about 22,000 events a second and verifies about 33,000, stores 100,000 events from a relay at about 17,000 a second, and answers a lookup from that store in about 3 ms, start to finish. The binaries are 2.3 to 2.7 MB. [BENCHMARKS.md](BENCHMARKS.md) has the full set and how to reproduce every number. +A one-shot command runs in about 2.3 ms and under 2 MB of memory. deed signs and verifies about 31,000 events a second each, stores 100,000 events from a relay at about 17,000 a second, and answers a lookup from that store in about 3 ms, start to finish. The binaries are 2.3 to 2.7 MB. [BENCHMARKS.md](BENCHMARKS.md) has the full set and how to reproduce every number. ## Build