Performance regression in zlib-rs backend, more zeroing of memory than before

[mstange]

I believe #502 introduced a performance regression for users of the zlib-rs feature, introducing more zeroing of memory than before.

Edit 2026-04-15: Actually it seems the real regression here was async-compression's switch to the new decompress_uninit API, and the fact that decompress_uninit for the zlib-rs backend uses the default implementation, which results in excessive repeated zeroing of the same large output buffer when called repeatedly for small input chunks.

Profile with async-compression-0.4.30 and flate2-1.1.2: https://share.firefox.dev/4c0g66E
Profile with async-compression-0.4.41 and flate2-1.1.7: https://share.firefox.dev/4sXPEC8

In practice this is causing a download that took 2.7 seconds to now take 9.1 seconds.

cc @NobodyXu @Byron

[NobodyXu]

Yes it's missing, maybe it's added after my PR, IIRC it wasn't there

[NobodyXu]

It seems like safe API of zlib-rs is missing uninitialized buffer, so we'd need to add it there first

[Byron]

Thanks a lot for reporting!

I'm trying to use an agent to whip up a performance test that would be able to catch such regressions in future. My hope is that from there a fix can also be attempted and validated.

[fintelia]

I believe this may only be a regression for users that directly use the Decompress::decompress_uninit API. I measured via my corpus-bench utility which uses flate2::write::ZlibDecoder and actually saw an almost 2% performance increase since version 1.1.2.

[Byron]

Thanks for chiming in @fintelia!

Meantime, it doesn't look like the regression could be reproduced. @mstange could we have the code that runs into this?
Thanks everyone!

[mstange]

Meantime, it doesn't look like the regression could be reproduced. @mstange could we have the code that runs into this? Thanks everyone!

The branch in that PR is based on Byron@aedc7a6 which is from 2023 and doesn't have #502.

Also, @fintelia's comment is probably right and you need to have a benchmark which calls decompress_uninit the way compression_codecs does: https://github.com/Nullus157/async-compression/blob/41915738959e1fececdc6cf2ebbd41b2c6ca6a5c/crates/compression-codecs/src/flate/decoder.rs#L32

In the profile I linked at the top of this issue, you can see that I am using flate2 via the async_compression crate (which uses compression_codecs internally to call into flate2).

[mstange]

I've managed to make a benchmark that reproduces the problem:

        bench_case("Decompress::chunked_decompress_uninit_with_large_output_buf", plain.len(), {
            const CHUNK_OUT: usize = 2 * 1024 * 1024;
            const CHUNK_IN: usize = 2048;
            let mut chunk = vec![MaybeUninit::<u8>::uninit(); CHUNK_OUT].into_boxed_slice();
            move || {
                let mut decoder = Decompress::new(true);
                let mut result = Vec::with_capacity(plain.len());
                loop {
                    let prior_out = decoder.total_out();
                    let in_start = decoder.total_in() as usize;
                    let in_end = (in_start + CHUNK_IN).min(zlib.len());
                    let status = decoder
                        .decompress_uninit(
                            &zlib[in_start..in_end],
                            &mut chunk,
                            FlushDecompress::None,
                        )
                        .unwrap();
                    let bytes_written = (decoder.total_out() - prior_out) as usize;
                    result.extend_from_slice(unsafe {
                        std::slice::from_raw_parts(chunk.as_ptr() as *const u8, bytes_written)
                    });
                    if status == Status::StreamEnd {
                        break;
                    }
                }
                result
            }
        }),

Profile: https://share.firefox.dev/4tRSzMJ

Three pieces are needed:

1. Calling decompress_uninit rather than decompress,
2. Calling it repeatedly with small inputs, and
3. Passing a large output buffer.

With backends that implement decompress_uninit by zeroing out the entire output buffer, decompress_uninit will be slower then decompress because for decompress the caller only needs to initialize the buffer once, whereas the uninit variant will zero the entire buffer on every call.

[Byron]

Thanks so much.

I'm trying to whip up a benchmark based on the one above to set a baseline prior breakage and then set up a CI job that tests against the baseline post-breakage so that we have CI to reproduce it and a framework to check performance of more critical low level functionality, maybe to prevent such issues in future.

[mstange]

Well, if I understand the situation correctly, it's not really fair to say that there was breakage in flate2 - rather, flate2 introduced a new API which has subpar performance for certain backends, then async-compression switched to that new API, and it was this switch which resulted in the overall regression in my use.

Here's the commit which introduced the decompress_uninit call in async-compression: Nullus157/async-compression@e894423
I haven't dug into the exact details.