Skip to content

Benchmark numbers from the released binary #25

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
sepehr-safari merged 1 commit into from
Sep 23, 2026
Merged

Benchmark numbers from the released binary #25

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
38 changes: 19 additions & 19 deletions BENCHMARKS.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ python3 bench/run.py zig-out/bin/deed

It needs python3 and [websocat](https://github.com/vi/websocat). Everything happens on your machine: the relay it talks to is [`bench/relay.py`](bench/relay.py) on 127.0.0.1, and nothing is sent anywhere else. Each timing is the best of several runs, because on a shared machine noise only ever adds time.

Measured on an Apple M2 Pro with macOS 26.6, with deed 0.3.1 built as it is released (`ReleaseSafe`, stripped).
Measured on an Apple M2 Pro with macOS 26.6, running the deed 0.3.1 binary from the release page (`ReleaseSafe`, stripped), installed with the one-line installer.

## Size

Expand All @@ -18,20 +18,20 @@ Measured on an Apple M2 Pro with macOS 26.6, with deed 0.3.1 built as it is rele
| macOS, Apple Silicon | 2.3 MB | 1.6 MB |
| macOS, Intel | 2.4 MB | 1.7 MB |
| Linux, x86_64 | 2.7 MB | 1.8 MB |
| Linux, aarch64 | 2.4 MB | 1.7 MB |
| Linux, aarch64 | 2.4 MB | 1.8 MB |

The Linux binaries are statically linked, so that is everything they need.

## Starting, and one event at a time

| | |
| --- | --- |
| starting any process at all (`/usr/bin/true`), for scale | 1.56 ms |
| starting any process at all (`/usr/bin/true`), for scale | 1.57 ms |
| `deed version` | 2.34 ms |
| `deed key generate` | 2.37 ms |
| `deed decode <npub>` | 2.41 ms |
| `deed event`, signing one | 2.41 ms |
| `deed verify`, one | 3.02 ms |
| `deed key generate` | 2.39 ms |
| `deed decode <npub>` | 2.43 ms |
| `deed event`, signing one | 2.46 ms |
| `deed verify`, one | 3.05 ms |
| peak memory, `deed version` | 1.7 MB |

deed's own share of a one-shot command is under a millisecond; the rest is the operating system starting a process. Verifying one event adds about 0.6 ms to that.
Expand All @@ -40,32 +40,32 @@ deed's own share of a one-shot command is under a millisecond; the rest is the o

| 10,000 records through stdin | |
| --- | --- |
| signing (`deed event -`) | 22,660 per second |
| verifying (`deed verify`) | 33,270 per second |
| decoding npubs (`deed decode`) | 586,718 per second |
| peak memory, verifying 10,000 | 3.8 MB |
| signing (`deed event -`) | 30,986 per second |
| verifying (`deed verify`) | 32,523 per second |
| decoding npubs (`deed decode`) | 568,831 per second |
| peak memory, verifying 10,000 | 3.9 MB |

## The local store

100,000 signed events from 1,000 authors: a profile, a follow list and 98 notes each.

| | |
| --- | --- |
| storing them from a loopback relay (`req --store`), each signature checked first | 16,864 per second |
| the store file for them | 94 MB |
| storing them from a loopback relay (`req --store`), each signature checked first | 17,379 per second |
| the store file for them | 95 MB |
| one event by id (`req --local -i`) | 3.25 ms |
| one profile (`-a <pubkey> -k 0 -l 1`) | 3.30 ms |
| one author's latest 50 notes | 3.47 ms |
| the latest 500 notes (`-k 1 -l 500`) | 4.94 ms |
| every note, 98,000 of them | 284 ms |
| one profile (`-a <pubkey> -k 0 -l 1`) | 3.21 ms |
| one author's latest 50 notes | 3.50 ms |
| the latest 500 notes (`-k 1 -l 500`) | 5.03 ms |
| every note, 98,000 of them | 286 ms |

The query times are the whole command, starting deed included, so a lookup costs about a millisecond more than `deed version` does. Events are written to the store in batches, in one transaction each, and a batch is stored before any of it is printed.

## Relays, over loopback

| | |
| --- | --- |
| `req` for 500 events | 44.5 ms, of which the test relay starting a Python process is about 20 |
| `publish` 1,000 events, waiting for each one's OK | 6,884 per second |
| `req` for 500 events | 43.9 ms, of which the test relay starting a Python process is about 20 |
| `publish` 1,000 events, waiting for each one's OK | 6,933 per second |

These leave the network out, so they measure deed and not the distance to a relay. Against a real relay, the round trip to it dominates.
2 changes: 1 addition & 1 deletion README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -107,7 +107,7 @@ Scripts branch on these, so they are part of the interface and not free to drift

## How fast it is

A one-shot command runs in about 2.3 ms and under 2 MB of memory. deed signs about 22,000 events a second and verifies about 33,000, stores 100,000 events from a relay at about 17,000 a second, and answers a lookup from that store in about 3 ms, start to finish. The binaries are 2.3 to 2.7 MB. [BENCHMARKS.md](BENCHMARKS.md) has the full set and how to reproduce every number.
A one-shot command runs in about 2.3 ms and under 2 MB of memory. deed signs and verifies about 31,000 events a second each, stores 100,000 events from a relay at about 17,000 a second, and answers a lookup from that store in about 3 ms, start to finish. The binaries are 2.3 to 2.7 MB. [BENCHMARKS.md](BENCHMARKS.md) has the full set and how to reproduce every number.

## Build

Expand Down
Loading