Skip to content

Handle filterless exact sum overflow - #26

Open
ila wants to merge 3 commits into
mainfrom
codex/filterless-exact-overflow
Open

Handle filterless exact sum overflow#26
ila wants to merge 3 commits into
mainfrom
codex/filterless-exact-overflow

Conversation

@ila

@ila ila commented Sep 2, 2026

Copy link
Copy Markdown
Member

Thanks for reviewing.

Summary

  • preserve exact HUGEINT and DECIMAL(38) sums when lower per-PU or upper filterless aggregation exceeds 128 bits
  • retain cancellation and order independence by delaying saturation until the public SQL result boundary
  • cover direct aggregates, compiler-rewritten queries, parallel combine, HUGEINT_MIN, and clipped bound * count overflow

Root cause and design

The lower partial and each upper exact bin previously accumulated directly into hugeint_t, so a valid query could throw before positive and negative contributions canceled. Early saturation is not valid because it makes the result order-dependent.

The common path remains the existing inline hugeint_t. On actual overflow, the state is promoted into an arena-allocated 192-bit representation: a DuckDB HUGEINT residue plus a signed count of 128-bit overflows. The residue is updated through DuckDB's unsigned 128-bit representation so wrapping is defined on every compiler. Upper bins use sparse sidecars, leaving the existing 32-byte dense bin layout unchanged.

This PR is based directly on main.

Validation

  • ninja -C build/release
  • the exact SQL query that failed x64 Linux CI returns 1
  • filterless SQL suite: 173 assertions passed
  • full SQL suite passed
  • PAC test runner passed
  • real compiler-rewritten DECIMAL queries returned the exact cancellation result
  • 30M-row review timings: BIGINT 0.725s, DECIMAL 0.806s, lower pair 0.433s; grouped 50k-state run 36.0s

Base automatically changed from codex/filterless-lower-fusion to main September 2, 2026 17:41
@ila
ila force-pushed the codex/filterless-exact-overflow branch from 2191711 to fb108b5 Compare September 2, 2026 17:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant