Invitation to provide feedback on New Parquet Encoding: Adaptive Lossless Floating Point Compression

[alamb]

Describe the enhancement requested

👋 from the Parquet community

The Parquet community is working on a proposal to implement ALP: Adaptive Lossless floating-Point Compression, (SIGMOD 2024) as a new Parquet encoding

arrow-go is listed on the Parquet Implementation Status page

Since you may feel pressure to implement newly added encodings, I think having your feedback on the spec as a stakeholder would be immensely valuable.

The proposed addition is here : Parquet ALP Encoding Specification (Google Doc)

There is also a prototype C++ implementation for review here

• GH-48701: [C++][Parquet] Add ALPpd encoding arrow#48345

See also: mailing list discussion (link) and parquet-format PR

Thank you for your consideration

Component(s)

Parquet

[zeroshade]

Thanks @alamb for the nudge here!

I'll take a look at the format PR and the prototype implementation and provide feedback. I should be able to find some time this week to do so. It'll give me something to do under all this snow ❄️ 😆

[alamb]

Note that the parquet-format PR is out of date with respect to the google doc spec -- please give me a few more days to migrate the content over to parquet format (or use the google doc spec directly)

[alamb]

(the basic algorithm is the same, but the details of where the headers go is still being worked out)