Repository navigation
fix(expression): type JSON paths in pushed-down CEL filters - #180
Merged
Merged
Conversation
ClickHouse infers a dynamic JSON path's type per row. One String or Array at a path breaks every filter transpiled for that path. `doc.a > 10` failed with NO_COMMON_TYPE. `IN` failed on any JSON path. String functions failed on the first Int64 row. `doc.p == "8080"` ran but matched the number 8080 as well. transpile_to_clickhouse takes json_columns. For paths under those columns, the other operand picks the typed sub-column. Numbers use `.:Int64`, `.:UInt64` and `.:Float64`. Strings use `.:String`. Bools use `.:Bool`. Equality returns false rather than NULL for a row of another type, so `!=` and a negated `==` keep CEL's answer. Conversions, size(), string methods and arithmetic read the sub-columns they accept. A comparison raises TranspileError if only the data can determine its type. The integration test runs 46 filters against a JSON column holding six types at one path. It checks each against evaluate_condition row by row.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
ClickHouse infers a dynamic JSON path's type per row. One String or Array at a path breaks every filter transpiled for that path.
doc.a > 10failed with NO_COMMON_TYPE.INfailed on any JSON path. String functions failed on the first Int64 row.doc.p == "8080"ran but matched the number 8080 as well.transpile_to_clickhouse takes json_columns. For paths under those columns, the other operand picks the typed sub-column. Numbers use
.:Int64,.:UInt64and.:Float64. Strings use.:String. Bools use.:Bool.Equality returns false rather than NULL for a row of another type, so
!=and a negated==keep CEL's answer. Conversions, size(), string methods and arithmetic read the sub-columns they accept. A comparison raises TranspileError if only the data can determine its type.The integration test runs 46 filters against a JSON column holding six types at one path. It checks each against evaluate_condition row by row.