Skip to content

[CORE][UI] Fix fallback info for V2 writes and align plan with Spark SQL tab#11853

Open
Zouxxyy wants to merge 2 commits intoapache:mainfrom
Zouxxyy:dev/spark-ui
Open

[CORE][UI] Fix fallback info for V2 writes and align plan with Spark SQL tab#11853
Zouxxyy wants to merge 2 commits intoapache:mainfrom
Zouxxyy:dev/spark-ui

Conversation

@Zouxxyy
Copy link
Copy Markdown
Contributor

@Zouxxyy Zouxxyy commented Mar 30, 2026

What changes are proposed in this pull request?

Fix Gluten UI not displaying fallback info for V2 writes (e.g., Iceberg INSERT with sort order), and align the plan display with Spark SQL tab (consistent operator IDs, complete plan tree).

Root causes:

  1. Plan traversal skips CommandResultExec and V2CommandExec, missing V2 write fallback reasons.
  2. In AQE mode, per-stage events have partial plans with inconsistent operator IDs; last stage overwrites earlier ones.

Fix: Add CommandResultExec recursion in GlutenExplainUtils/GlutenImplicits/GlutenFallbackReporter. Add GlutenQueryExecutionListener that generates complete fallback data from the fully materialized QueryExecution after execution ends, posted as event for both live UI and History Server.

We also considered a listener-side merge approach (accumulating per-stage events), but it cannot produce consistent operator IDs or a complete plan tree.

How was this patch tested?

Added test in VeloxIcebergSuite. Verified on History Server.

before

image

after

image

Was this patch authored or co-authored using generative AI tooling?

Generated-by: Kiro (Claude Opus 4.6), co-authored with Zouxxyy through iterative design, code review, and testing.

@github-actions github-actions bot added CORE works for Gluten Core VELOX labels Mar 30, 2026
@github-actions
Copy link
Copy Markdown

Run Gluten Clickhouse CI on x86

@Zouxxyy Zouxxyy changed the title [GLUTEN-UI] Fix fallback info for V2 writes and align plan with Spark SQL tab [UI] Fix fallback info for V2 writes and align plan with Spark SQL tab Mar 30, 2026
@Zouxxyy Zouxxyy changed the title [UI] Fix fallback info for V2 writes and align plan with Spark SQL tab [CORE] Fix fallback info for V2 writes and align plan with Spark SQL tab Mar 30, 2026
@github-actions
Copy link
Copy Markdown

Run Gluten Clickhouse CI on x86

@Zouxxyy Zouxxyy changed the title [CORE] Fix fallback info for V2 writes and align plan with Spark SQL tab [CORE][UI] Fix fallback info for V2 writes and align plan with Spark SQL tab Mar 31, 2026
@Zouxxyy
Copy link
Copy Markdown
Contributor Author

Zouxxyy commented Mar 31, 2026

CC @zhztheplayer @JkSelf @jinchengchenghh for a look, thanks

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CORE works for Gluten Core VELOX

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant