Skip to content

feat: expose ColumnByteStatisticsProvider for join costing - #36

Closed
alexanderbianchi wants to merge 1 commit into
alex.bianchi/iceberg-selected-file-stats-reviewfrom
alex.bianchi/column-byte-statistics-provider-review
Closed

alexanderbianchi wants to merge 1 commit into
alex.bianchi/iceberg-selected-file-stats-reviewfrom
alex.bianchi/column-byte-statistics-provider-review

Conversation

@alexanderbianchi

Copy link
Copy Markdown

Stacked on #35. This PR contains only the provider, its export, tests, and registration docs. Sessions can register ColumnByteStatisticsProvider to estimate derived-column bytes for join costing. It does not opt sessions in by default and leaves row estimates alone.

The join-selection regression compares build sides with and without registry costing and checks the output schema. Additional tests cover missing sizes, outer joins, and overflow.

Test plan: cargo test --locked -p datafusion-distributed --lib column_byte_statistics.

Signed-off-by: alex.bianchi <alex.bianchi@datadoghq.com>

Copy link
Copy Markdown
Author

Superseded by ddoghq/dd-source#196361: the provider and its tests now live in the Iceberg connector and registration is restricted to the main stateful Iceberg release. #35 remains the standalone selected-file statistics change.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant