Skip to content

tp: add INTERVAL FLATTEN to pipelines - #7692

Draft
LalitMaganti wants to merge 1 commit into
dev/lalitm/interval-flatten-grammarfrom
dev/lalitm/interval-flatten
Draft

LalitMaganti wants to merge 1 commit into
dev/lalitm/interval-flatten-grammarfrom
dev/lalitm/interval-flatten

Conversation

@LalitMaganti

@LalitMaganti LalitMaganti commented Sep 29, 2026 •

Copy link
Copy Markdown
Member
FROM spans
|> INTERVAL FLATTEN PER cpu AGGREGATE COUNT(*) AS n, SUM(weight) AS w

Cuts overlapping rows at every start and end into disjoint segments, one
row per segment with its bounds, the PER keys and the aggregates. A row of
no width becomes a segment of no width counting the rows spanning it.
Bounds follow INTERVAL INTERSECTION: null ts or dur is skipped, negative
is an error. Only COUNT(*) and SUM for now.

The operator needs its input grouped by the keys and ordered by ts within
each group. Lowering tracks what order is known and adds Sort and GroupBy
only when needed, e.g. no Sort for a dataframe sorted on ts. Its output
keeps that order, so a following stage needs neither.

Against the stdlib macros it replaces, on real traces (release build,
materializing the result as a table):

Workload Macro Macro time FLATTEN time
sched by machine_id intervals_overlap_count_by_group! 329 ms 35 ms
thread_state by state intervals_overlap_count_by_group! 986 ms 94 ms
slice by track_id intervals_overlap_count_by_group! 1213 ms 62 ms
sched intervals_overlap_count! 264 ms 28 ms
Chrome slices interval_self_intersect! 1352 ms 7 ms

  FROM spans
  |> INTERVAL FLATTEN PER cpu AGGREGATE COUNT(*) AS n, SUM(weight) AS w

Cuts overlapping rows at every start and end into disjoint segments, one
row per segment with its bounds, the PER keys and the aggregates. A row of
no width becomes a segment of no width counting the rows spanning it.
Bounds follow INTERVAL INTERSECTION: null ts or dur is skipped, negative
is an error. Only COUNT(*) and SUM for now.

The operator needs its input grouped by the keys and ordered by ts within
each group. Lowering tracks what order is known and adds Sort and GroupBy
only when needed, e.g. no Sort for a dataframe sorted on ts. Its output
keeps that order, so a following stage needs neither.
@LalitMaganti
LalitMaganti added this pull request to stack #7680 September 29, 2026 23:53
@github-actions

Copy link
Copy Markdown

🎨 Perfetto UI Builds & Tests

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant