Keep MAX buckets whose values are all non-positive#116
Open
PDGGK wants to merge 1 commit into
Open
Conversation
The milliseconds aggregation path projected MAX over the numeric COALESCE expression. IoTDB's grouped max accumulator seeds FLOAT/DOUBLE state with Float/Double.MIN_VALUE -- the smallest positive value, not the most negative one -- and only marks a group initialized when value >= state, so a bucket whose numeric maximum is zero or negative comes back NULL. This DAO reads a NULL aggregate as an empty bucket and skips it, so a MAX downsampling query over, for example, a sub-zero sensor silently lost whole buckets instead of failing. Project -MIN(-x) instead: the grouped min accumulator seeds with MAX_VALUE and is not affected, and IEEE-754 negation is exact, so the result is identical over finite values on servers with and without the upstream fix. The projection is shared with the calendar path, whose non-grouped accumulators track an explicit initialized flag rather than a sentinel, so the substitution is exact there too and an empty bucket still yields NULL. The long and string channels are already correct and keep using MAX. Verified against apache/iotdb 2.0.8 and 2.0.10 containers. See apache/iotdb#18300, fixed on master but not yet in a released version. Signed-off-by: Zihan Dai <99155080+PDGGK@users.noreply.github.com>
PDGGK
force-pushed
the
fix/grouped-max-non-positive
branch
from
July 25, 2026 08:39
c9c9a72 to
b1f761e
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The milliseconds aggregation path projects the numeric aggregate over a
COALESCEexpression:IoTDB's grouped max accumulator seeds FLOAT/DOUBLE state with
Float.MIN_VALUE/Double.MIN_VALUE— the smallest positive values, not the most negative ones — and only marks a group initialized whenvalue >= state. A bucket whose numeric maximum is zero or negative therefore never sets the flag and comes back asNULL. This is apache/iotdb#18300, fixed on master but not yet in any released version (the 2.0.11 cherry-pick is under discussion on dev@), so every release the module supports is affected.aggregatedEntryreads a NULL numeric aggregate as "this bucket has no value" and returnsnull, andreadMillisecondsAggregatedQueryskips such buckets. So aMAXdownsampling query over telemetry that is always zero or negative — a sub-zero temperature sensor, a gauge that sits at 0 — silently returned fewer points, with no error raised anywhere.Verified against the released
apache/iotdb:2.0.8andapache/iotdb:2.0.10standalone images: a bucket of-5.0, -3.0and a bucket of0.0, 0.0both returnagg_num = NULL, while a positive bucket is correct.Scope of the problem, also verified on both images:
MIN,AVG,SUMandCOUNTare unaffected.GROUP BY, and the non-grouped accumulators track an explicit initialized flag instead of a sentinel.MAX(long_v)channel, whoseLongBigArrayseeds with the trueLong.MIN_VALUE.Fix
Project
-MIN(-x)instead ofMAX(x)for the numeric channel. The grouped min accumulator seeds withMAX_VALUEand is not affected, and IEEE-754 negation is exact, so this returns the same value asMAX(x)over finite values, on servers with and without the upstream fix, and costs no extra query. The long and string channels are already correct —MAX(long_v)seeds with the trueLong.MIN_VALUE, and the binary channel is flag-based — so they keep usingMAX.The projection is shared with the calendar path, so its SQL changes too. That path's non-grouped accumulators track an explicit initialized flag rather than a sentinel, so the substitution is exact there as well and an empty bucket still returns
NULLand is still skipped.Tests
Two integration tests against a real IoTDB container:
maxKeepsBucketsWhoseValuesAreAllNonPositiveAgainstRealIoTDB— buckets of all-negative and all-zero doubles keep theirMAX(andMINis unchanged).maxOverNonPositiveLongOnlyAndMixedBucketsKeepsResultTypeAgainstRealIoTDB— covers the long-only channel and the mixed promotion, since they take different paths.Both fail on the previous projection with
bucket count expected: <2> but was: <0>and<1>respectively — the buckets disappear — and pass with the fix. The existing aggregation coverage only ever used positive doubles, which is why this was not caught earlier.Full module build is green: 190 unit tests and 54 container integration tests.