[BugFix] Fix bin/chart NPE with null values (#5174)

[qianheng-aws]

Description

Fix NullPointerException when using bin then chart with null values in the binned field.

Root cause: The bin bucket functions (SpanBucketFunction, MinspanBucketFunction, RangeBucketFunction) declared a non-nullable VARCHAR(2000) return type via ReturnTypes.VARCHAR_2000, even though they return null for null inputs. This caused Calcite's optimizer to remove the IS NOT NULL filters (added by visitAggregation's nonNullGroupMask logic) as trivially true, allowing null group keys to reach the Enumerable aggregation's TreeMap, which crashed with NPE in compareTo.

Fix:

1. Declare the return type as nullable using ReturnTypes.VARCHAR_2000.andThen(SqlTypeTransforms.FORCE_NULLABLE) so that the null-filtering logic in visitAggregation works correctly
2. Add nullsLast to chart command sort operations as a defensive measure

Related Issues

Resolves opensearch-project#5174

Check List

• New functionality includes testing
• Commits signed per DCO (-s)
• spotlessCheck passed
• Unit tests passed
• Integration tests passed

[qianheng-aws]

Decision Log

Root Cause: Bin bucket functions (SpanBucketFunction, MinspanBucketFunction, RangeBucketFunction) declared ReturnTypes.VARCHAR_2000 as their return type inference, producing a non-nullable type. Despite the implementations returning null for null inputs, Calcite's type system believed the result was always non-null. This caused Calcite's optimizer to remove IS NOT NULL filters as trivially true, allowing null group keys from binning null values to reach the Enumerable aggregation's TreeMap, which crashed with Cannot invoke "java.lang.Comparable.compareTo(Object)" because "v0" is null.

Approach: Two-part fix:

1. Changed return type inference in all three bucket functions from ReturnTypes.VARCHAR_2000 to ReturnTypes.VARCHAR_2000.andThen(SqlTypeTransforms.FORCE_NULLABLE), making the return type nullable so the IS NOT NULL filter in visitAggregation is preserved by the optimizer.
2. Added nullsLast() to chart command sort operations (lines 3208 and 3278 in CalciteRelNodeVisitor) as a defensive measure against any remaining null-in-sort scenarios.

Alternatives Rejected:

• Adding null checks in the sort comparator: Would be a workaround, not a fix for the actual type mismatch.
• Filtering nulls in visitChart before sort: The existing mechanism in visitAggregation already handles this correctly once the type is correct; adding another filter would be redundant.

Pitfalls:

• The existing chart unit tests (CalcitePPLChartTest) already expected NULLS LAST in the SparkSQL output, because the RelToSqlConverter's SparkSQL dialect naturally emits NULLS LAST for ASC sorts. This masked the fact that the runtime Enumerable execution could still fail on nulls.
• The DEFAULT_USE_NULL = Literal.TRUE default in Chart means column splits can legitimately have null values that get labeled "NULL" string, so the non-null filter must only apply to fields marked in nonNullGroupMask.

Things to Watch:

• Other UDF functions that return null for null inputs but declare non-nullable return types may have similar issues. Worth auditing functions in udf/binning/ and other PPLFuncImpTable registrations.
• The NullPolicy.ANY used by these functions means Calcite generates code that passes nulls through to the implementor rather than short-circuiting. This interacts with the return type declaration - both must be consistent.