Skip to content

Conversation

@jackkleeman
Copy link
Contributor

  1. Use a lookup table (3844 pairs of u8s) for turning bytes corresponding to 2 base62 characters into those two characters, reducing the number of div/rem operations needed
  2. Avoid branching when handling 21/21 or 11/10 digit cases
  3. Avoid binary search when determining digit count, use instead an approximation with correction

Overall improvements on my M2 max are 30-50%, decoding benchmarks unaffected:

encode/standard_new_fixed
                        time:   [21.810 ns 21.865 ns 21.924 ns]
                        change: [−20.781% −18.469% −16.145%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 9 outliers among 100 measurements (9.00%)
  4 (4.00%) low mild
  3 (3.00%) high mild
  2 (2.00%) high severe
encode/standard_new_random
                        time:   [16.702 ns 16.863 ns 17.027 ns]
                        change: [−36.855% −35.101% −33.665%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 16 outliers among 100 measurements (16.00%)
  8 (8.00%) low mild
  5 (5.00%) high mild
  3 (3.00%) high severe
encode/standard_new_random_u64
                        time:   [12.128 ns 12.258 ns 12.373 ns]
                        change: [−31.413% −29.568% −27.826%] (p = 0.00 < 0.05)
                        Performance has improved.
encode/standard_bytes_fixed
                        time:   [8.9859 ns 9.0062 ns 9.0267 ns]
                        change: [−40.511% −38.713% −37.367%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 4 outliers among 100 measurements (4.00%)
  4 (4.00%) high mild
encode/standard_bytes_random
                        time:   [10.264 ns 10.311 ns 10.362 ns]
                        change: [−40.286% −39.525% −38.883%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 4 outliers among 100 measurements (4.00%)
  3 (3.00%) high mild
  1 (1.00%) high severe
encode/standard_bytes_random_u64
                        time:   [4.8844 ns 4.9894 ns 5.0998 ns]
                        change: [−45.120% −43.545% −41.757%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 6 outliers among 100 measurements (6.00%)
  2 (2.00%) low mild
  2 (2.00%) high mild
  2 (2.00%) high severe
encode/standard_buf_fixed
                        time:   [9.1049 ns 9.1667 ns 9.2424 ns]
                        change: [−39.683% −37.537% −35.580%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 4 outliers among 100 measurements (4.00%)
  2 (2.00%) high mild
  2 (2.00%) high severe
encode/standard_buf_random
                        time:   [9.8441 ns 9.8764 ns 9.9082 ns]
                        change: [−40.796% −39.898% −39.266%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 3 outliers among 100 measurements (3.00%)
  2 (2.00%) low mild
  1 (1.00%) high mild
encode/standard_buf_random_u64
                        time:   [4.4342 ns 4.4420 ns 4.4501 ns]
                        change: [−50.769% −49.654% −48.998%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 6 outliers among 100 measurements (6.00%)
  1 (1.00%) low severe
  4 (4.00%) low mild
  1 (1.00%) high mild
encode/alternative_new_fixed
                        time:   [21.840 ns 21.922 ns 22.024 ns]
                        change: [−21.341% −19.820% −18.214%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 4 outliers among 100 measurements (4.00%)
  2 (2.00%) high mild
  2 (2.00%) high severe
encode/alternative_new_random
                        time:   [17.275 ns 17.436 ns 17.593 ns]
                        change: [−32.089% −31.037% −30.045%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 1 outliers among 100 measurements (1.00%)
  1 (1.00%) high mild
encode/alternative_new_random_u64
                        time:   [12.161 ns 12.281 ns 12.410 ns]
                        change: [−25.044% −23.586% −22.151%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 5 outliers among 100 measurements (5.00%)
  4 (4.00%) low mild
  1 (1.00%) high mild
encode/alternative_bytes_fixed
                        time:   [9.0809 ns 9.2128 ns 9.4044 ns]
                        change: [−37.003% −36.148% −35.102%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 5 outliers among 100 measurements (5.00%)
  1 (1.00%) high mild
  4 (4.00%) high severe
encode/alternative_bytes_random
                        time:   [10.398 ns 10.504 ns 10.611 ns]
                        change: [−39.402% −38.730% −38.121%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 3 outliers among 100 measurements (3.00%)
  3 (3.00%) high mild
encode/alternative_bytes_random_u64
                        time:   [4.8215 ns 4.9078 ns 4.9969 ns]
                        change: [−44.269% −42.694% −40.968%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 20 outliers among 100 measurements (20.00%)
  14 (14.00%) low mild
  5 (5.00%) high mild
  1 (1.00%) high severe
encode/alternative_buf_fixed
                        time:   [9.1571 ns 9.1818 ns 9.2060 ns]
                        change: [−37.374% −37.108% −36.842%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 1 outliers among 100 measurements (1.00%)
  1 (1.00%) high mild
encode/alternative_buf_random
                        time:   [9.8021 ns 9.8417 ns 9.8796 ns]
                        change: [−39.731% −39.315% −38.920%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 1 outliers among 100 measurements (1.00%)
  1 (1.00%) high mild
encode/alternative_buf_random_u64
                        time:   [4.5678 ns 4.6589 ns 4.7722 ns]
                        change: [−48.095% −47.371% −46.465%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 9 outliers among 100 measurements (9.00%)
  3 (3.00%) high mild
  6 (6.00%) high severe

Co-authored-by: François Bernier <frankbernier@gmail.com>
@fbernier fbernier merged commit dc62ff9 into fbernier:master Jan 23, 2026
9 checks passed
@fbernier
Copy link
Owner

great stuff again. Thanks!

@jackkleeman jackkleeman deleted the encode-pair branch January 23, 2026 17:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants