Fuse filterless lower aggregate partials - #24
Merged
Conversation
ila
marked this pull request as ready for review
September 1, 2026 13:01
Fuse filterless AVG release
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Thanks for reviewing.
Summary
This PR fuses the two lower, per-privacy-unit partial aggregates used by
dp_filterlessinto one paired aggregate for every SUM or COUNT component. The paired state returnsSTRUCT(answer, histogram), so the upper filterless aggregate receives the same two partials as before while the lower plan updates only one aggregate state per component.This is an internal execution optimization. It does not change the public SQL surface, privacy-budget allocation, sampling rule, clipping-bound selection, noise calibration, group suppression, or released result types.
Why
Filterless needs two views of each contribution:
answer: rows selected by the user predicate;histogram: rows selected by the fixed, predicate-independent privacy-unit sample.Previously, the per-PU aggregation computed those views with two independent aggregate states for each rewritten component:
SUM, COUNT, and each internal AVG component therefore repeated aggregate dispatch, state lookup, value conversion, and accumulation. This overhead grows with query width.
The new lower plan performs both updates in one callback:
The existing upper
filterless_sumandfilterless_countaggregates still consume separate answer and histogram expressions. The compiler extracts the two struct fields after per-PU aggregation, so the privacy mechanism above this boundary is unchanged.Semantics preserved
COUNT(expr)uses an explicit non-NULL marker, whileCOUNT(*)counts every qualifying row.DECIMAL(38, scale)accumulation.SUMorCOUNT(expr)when all values are NULL.Implementation
priv_filterless_sum_pairfunctions for the supported numeric types and apriv_filterless_count_pairfunction for count markers.answerandhistogramfields for the existing upper aggregate.PAC aggregate callbacks themselves are not reused: they own 64-lane, arena-backed PAC state, whereas this lower aggregate owns exactly two scalar partials with different activation predicates. Sharing those callbacks would couple unrelated state layouts instead of removing duplication.
Plan and correctness coverage
The filterless SQL suite verifies:
dp_filterless_sample_bits=0and6;C_u > 1top-k contribution bounding;A real four-thread grouped query with
C_u=2also returned the expected BIGINT, DECIMAL, floating SUM, and COUNT values for both retained groups.Validation
Performance
Randomized fresh-process SF30 measurements compare this PR with the checkpoint where exact integer and DECIMAL lower partials were still unfused:
The benefit increases with aggregate width because fusion removes one lower state and update path per component. The BIGINT measurements had more run-to-run variance, so the table reports conservative medians rather than individual best runs.
A separate ten-pair comparison of five floating/count aggregates against the earlier specialized fused implementation differed by 0.09%. This confirms that replacing separate specialized pair implementations with the shared generic implementation is performance-neutral while substantially reducing code.