Typo in readme file

[dadr]

In your Readme.md within the hashing algorithms section you show a command line code snippet:

$ cat /proc/crypto | grep crc32c_intel
On my system, I had to grep for crc32c-intel

[nethappen]

You're absolutely right — that information is a bit outdated:

It's possible that the crc32c module is built directly into your kernel. In such cases, /proc/crypto may show only the driver name (like crc32c-intel) instead of the original module name (e.g. crc32c_intel).

In the latest kernels (starting from 6.14), Intel hardware support has been integrated into a unified crc32c module, which might appear simply as crc32c or crc32c_x86.

Currently you can check if crc32 works in accelerated version
$ cat /proc/crypto | grep crc32

If you have: crc32c-x86 - this is old crc32c-intel driver

You can also check the performance comparing algorithms by running blocksync-fast --benchmark and looking at the performance numbers to see if they're high enough.

[dadr]

I wonder if you might have an opinion about this: I tried to install the libxxhash-dev package that you recommended in my comment from last year. (My Ubuntu 24.04 had an apt mis-match between the -dev and libxxhash0 package versions, so I downgraded the libxxhash0 package to 0.8.2 to get them to match.) I recompiled blocksync-fast and then ran it with the --benchmark-algos. The good news is that I saw benchmark results for XXH* after doing this. The bad news is that CRC32 came in at 12.08 GiB/s and XXH3LOW came in at only 5.52 GiB/s
I have a Xeon processor with all the extended instructions you mention. Does the xxhash benchmark seem too low? And do you think that I should try to fix XXH, accept it as it is, or use CRC32. I'm syncing over 10TB of data about once a week in a btrfs partition. Thanks!

[nethappen]

Performance may vary between systems, which is why it's important to be aware of this and run a benchmark using $blocksync-fast --benchmark

There are several possible reasons for performance differences. For example, the CRC32 algorithm is often supported directly by the kernel (and potentially hardware-accelerated), while xxHash requires an additional library and relies on CPU instruction set support.

To achieve high speeds with xxHash, the CPU should support both AVX2 and SSE2 — which most modern processors do. You can check whether your CPU supports these features with:
$ lscpu | grep -o 'avx[^ ]*'
or
$ cat /proc/cpuinfo | grep flags | head -n 1

Note that on systems like Debian/Ubuntu, xxHash is often delivered as a precompiled package. However, it's unclear which compiler flags were used — it may or may not include AVX2-optimized code, and it might lack runtime CPU detection. Many distributions compile for maximum compatibility instead of performance.

I recommend starting with a performance test:
$ xxhsum -b

This will show whether AVX2 is detected and give you native benchmark of the hashing speed.

With blocksync-fast, since you compile it directly on your system, the build process will detect and use your CPU flags.

That said, I’ve encountered a strange case: blocksync-fast versions up to 1.05 performed very well, but starting from 1.06 — performance for xxh3 family (64-bit, 128-bit, and 64/2 variants) dropped drastically, by a factor of several times. I’m not yet sure why. The source code changed, but I haven’t found anything that clearly explains this behavior.

Even more oddly, running the exact same binary on an identical system with a slightly newer CPU results in normal, high performance again. So the issue seems hardware- or microarchitecture-specific.

That said, if you’re currently seeing speeds like 5.52 GiB/s, and that’s already faster than your disk’s sequential read speed, I’d suggest sticking with xxHash for now — especially since you've already generated checksum files using it. The issue may be resolved in a future release, and performance could return to expected levels.