Skip to content

TUR-134: MLIR: unwind a typed list of any value type into a typed column - #1452

Merged
sulaimansuhas merged 1 commit into
mainfrom
suhasnarreddy/tur-134-mlir-unwind-a-typed-list-of-datetimes
Nov 11, 2026
Merged

sulaimansuhas merged 1 commit into
mainfrom
suhasnarreddy/tur-134-mlir-unwind-a-typed-list-of-datetimes

Conversation

@sulaimansuhas

Copy link
Copy Markdown
Contributor

Unwind a list of datetimes, durations, points, maps, embeddings or paths into a column of that type instead of type-erased cells.

Stacked on #1448 (bot stack main): review and merge #1448 first.

@sulaimansuhas

Copy link
Copy Markdown
Contributor Author

Note

AI-generated summary.

Summary

Architecture of the change: unwoundElementType decides the typed chunk nl.unwind yields, the old allow-list is removed, and a new comparePathColumn lets a path column be a sort key

unwoundElementType picks the column type an UNWIND of a list yields. It used an allow-list of six element types (node, edge, string, list, f64, i1/i64). Every other typed list fell into !storage.any cells. The allow-list is gone; the rule is now the type system's own:

 unwoundElementType(list<T>)
-  if T not in {node_id, edge_id, string, list, f64, i1, i64}
-    return !storage.any
-  if T is node_id or edge_id
-    return T
-  return nullable<T>
+  if holdsItsOwnNull(T)                       # any, node_id, edge_id, path
+    return T
+  else if elementTypeToValueType(T) == Invalid  # mixed list, nulls alone
+    return !storage.any
+  return nullable<T>                          # datetime, duration, point, map, embedding, ...

Two follow-ons make that rule reach every case:

literalElementType          DictionaryAttr  -> !storage.map
                            DenseF32ArrayAttr -> !storage.embedding
                            so [{a: 1}, {a: 2}] and [(1, 5), (2, 3)] get a shared element type

sort key dispatch           NLChunkKind::Path used to throw "a path has no order here"
  comparePathColumn         now orders it, the empty (null) path last
    operator<=>(PathView)   lexicographic over IDs, shared with CellOrder's Path case

The path sort key is needed because UNWIND [p, null] AS q ... ORDER BY q used to sort !storage.any cells through CellOrder. It now sorts a path column, which had no comparator.

Evidence

Run locally on 074e4b87, binaries up to date (ninja -n reported no work):

test_query_ir_unwind_typed_list        13 tests  PASSED   (new)
test_query_ir_path_sort_key             2 tests  PASSED   (new)
test_query_ir_list_of_maps             21 tests  PASSED
test_query_ir_datetime_component_cell   7 tests  PASSED
test_query_ir_duration_component_cell   7 tests  PASSED

UnwindTypedListTest checks both the rows and the EXPLAIN (nl) iterator type of each unwind:

UNWIND [duration(1000000), duration(90061000000)] AS d RETURN d.hours, d.seconds
  before: !nl.iter<!nl.chunk<!storage.any>, ...>
  after:  !nl.iter<!nl.chunk<!storage.nullable<!storage.duration>>, ...>   rows (0, 1), (25, 90061)

MATCH p = ... UNWIND [p, null] AS q WITH q ORDER BY q RETURN length(q)
  after:  !storage.path column, rows 1, null

The same holds for datetimes, points, maps, embeddings, collected properties (collect(n.took)) and collected paths. DateTimeComponentCellTest and DurationComponentCellTest gain a case that reads a field off a map value and a mixed-list element, because an unwind no longer reaches the type-erased cell path those tests covered. I did not run the full ctest suite.

Merge Danger

Door: two-way

No stored format or wire format changes. The change only picks a different chunk type at lowering time.

Blast Radius: query-wide

Every query that unwinds a typed list of a type outside the old allow-list now reads a typed column, not !storage.any cells. Any nl op or executor kernel without a case for nullable<datetime|duration|point|map|embedding> or path can now fail on such a query, as the path sort key did. The new tests cover field access, comparison, + duration, DISTINCT, grouping and ORDER BY. Other consumers, such as the list functions over an unwound value, are not covered here.

@sulaimansuhas
sulaimansuhas force-pushed the suhasnarreddy/tur-134-mlir-unwind-a-typed-list-of-datetimes branch from 074e4b8 to 5f7bf8d Compare November 11, 2026 05:03
Base automatically changed from suhasnarreddy/tur-70-listsmap-loader-reading-1-byte-at-a to main November 11, 2026 10:26
@sulaimansuhas
sulaimansuhas force-pushed the suhasnarreddy/tur-134-mlir-unwind-a-typed-list-of-datetimes branch from 5f7bf8d to 07f8a0c Compare November 11, 2026 10:26
@sulaimansuhas
sulaimansuhas marked this pull request as ready for review November 11, 2026 10:27
@sulaimansuhas
sulaimansuhas requested a review from rjb32 as a code owner November 11, 2026 10:27
@sulaimansuhas
sulaimansuhas merged commit 0c21f26 into main Nov 11, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants