diff --git a/BENCHMARKS.md b/BENCHMARKS.md index c5c5a05..ae6a735 100644 --- a/BENCHMARKS.md +++ b/BENCHMARKS.md @@ -9,7 +9,7 @@ python3 bench/run.py zig-out/bin/deed It needs python3 and [websocat](https://github.com/vi/websocat). Everything happens on your machine: the relay it talks to is [`bench/relay.py`](bench/relay.py) on 127.0.0.1, and nothing is sent anywhere else. Each timing is the best of several runs, because on a shared machine noise only ever adds time. -Measured on an Apple M2 Pro with macOS 26.6, with deed 0.3.1 built as it is released (`ReleaseSafe`, stripped). +Measured on an Apple M2 Pro with macOS 26.6, running the deed 0.3.1 binary from the release page (`ReleaseSafe`, stripped), installed with the one-line installer. ## Size @@ -18,7 +18,7 @@ Measured on an Apple M2 Pro with macOS 26.6, with deed 0.3.1 built as it is rele | macOS, Apple Silicon | 2.3 MB | 1.6 MB | | macOS, Intel | 2.4 MB | 1.7 MB | | Linux, x86_64 | 2.7 MB | 1.8 MB | -| Linux, aarch64 | 2.4 MB | 1.7 MB | +| Linux, aarch64 | 2.4 MB | 1.8 MB | The Linux binaries are statically linked, so that is everything they need. @@ -26,12 +26,12 @@ The Linux binaries are statically linked, so that is everything they need. | | | | --- | --- | -| starting any process at all (`/usr/bin/true`), for scale | 1.56 ms | +| starting any process at all (`/usr/bin/true`), for scale | 1.57 ms | | `deed version` | 2.34 ms | -| `deed key generate` | 2.37 ms | -| `deed decode ` | 2.41 ms | -| `deed event`, signing one | 2.41 ms | -| `deed verify`, one | 3.02 ms | +| `deed key generate` | 2.39 ms | +| `deed decode ` | 2.43 ms | +| `deed event`, signing one | 2.46 ms | +| `deed verify`, one | 3.05 ms | | peak memory, `deed version` | 1.7 MB | deed's own share of a one-shot command is under a millisecond; the rest is the operating system starting a process. Verifying one event adds about 0.6 ms to that. @@ -40,10 +40,10 @@ deed's own share of a one-shot command is under a millisecond; the rest is the o | 10,000 records through stdin | | | --- | --- | -| signing (`deed event -`) | 22,660 per second | -| verifying (`deed verify`) | 33,270 per second | -| decoding npubs (`deed decode`) | 586,718 per second | -| peak memory, verifying 10,000 | 3.8 MB | +| signing (`deed event -`) | 30,986 per second | +| verifying (`deed verify`) | 32,523 per second | +| decoding npubs (`deed decode`) | 568,831 per second | +| peak memory, verifying 10,000 | 3.9 MB | ## The local store @@ -51,13 +51,13 @@ deed's own share of a one-shot command is under a millisecond; the rest is the o | | | | --- | --- | -| storing them from a loopback relay (`req --store`), each signature checked first | 16,864 per second | -| the store file for them | 94 MB | +| storing them from a loopback relay (`req --store`), each signature checked first | 17,379 per second | +| the store file for them | 95 MB | | one event by id (`req --local -i`) | 3.25 ms | -| one profile (`-a -k 0 -l 1`) | 3.30 ms | -| one author's latest 50 notes | 3.47 ms | -| the latest 500 notes (`-k 1 -l 500`) | 4.94 ms | -| every note, 98,000 of them | 284 ms | +| one profile (`-a -k 0 -l 1`) | 3.21 ms | +| one author's latest 50 notes | 3.50 ms | +| the latest 500 notes (`-k 1 -l 500`) | 5.03 ms | +| every note, 98,000 of them | 286 ms | The query times are the whole command, starting deed included, so a lookup costs about a millisecond more than `deed version` does. Events are written to the store in batches, in one transaction each, and a batch is stored before any of it is printed. @@ -65,7 +65,7 @@ The query times are the whole command, starting deed included, so a lookup costs | | | | --- | --- | -| `req` for 500 events | 44.5 ms, of which the test relay starting a Python process is about 20 | -| `publish` 1,000 events, waiting for each one's OK | 6,884 per second | +| `req` for 500 events | 43.9 ms, of which the test relay starting a Python process is about 20 | +| `publish` 1,000 events, waiting for each one's OK | 6,933 per second | These leave the network out, so they measure deed and not the distance to a relay. Against a real relay, the round trip to it dominates. diff --git a/README.md b/README.md index 7a33551..0d64de3 100644 --- a/README.md +++ b/README.md @@ -107,7 +107,7 @@ Scripts branch on these, so they are part of the interface and not free to drift ## How fast it is -A one-shot command runs in about 2.3 ms and under 2 MB of memory. deed signs about 22,000 events a second and verifies about 33,000, stores 100,000 events from a relay at about 17,000 a second, and answers a lookup from that store in about 3 ms, start to finish. The binaries are 2.3 to 2.7 MB. [BENCHMARKS.md](BENCHMARKS.md) has the full set and how to reproduce every number. +A one-shot command runs in about 2.3 ms and under 2 MB of memory. deed signs and verifies about 31,000 events a second each, stores 100,000 events from a relay at about 17,000 a second, and answers a lookup from that store in about 3 ms, start to finish. The binaries are 2.3 to 2.7 MB. [BENCHMARKS.md](BENCHMARKS.md) has the full set and how to reproduce every number. ## Build