[FEA] Give the AutoTuner a signal for GPU memory skew within a stage

[amahussein]

Is your feature request related to a problem? Please describe.

The AutoTuner has no way to notice that a stage's GPU memory demand is unevenly distributed across its tasks, and that blind spot has a shape worth stating plainly.

Its existing signals in this area are all reachable only through a side door:

• Spill is only examined on stages that shuffle, so a stage that reads from files and spills heavily is skipped entirely.
• Skew detection is defined in terms of shuffle read volume, so skew that originates on the scan side does not register.
• The GPU memory pressure path is failure driven, so it stays silent unless a task actually failed with an out of memory error.

An application can therefore spill a large amount of GPU memory, take a real slowdown for it, and receive no comment and no configuration change from the tuner, because none of the three conditions above is met. From the user's point of view the tool simply has nothing to say about the thing that went wrong.

This matters more now than it used to. Recent plugin versions size GPU task admission at runtime from observed memory usage, and the AutoTuner defers to that mechanism rather than recommending a concurrency value itself. That is a reasonable division of labour, but it means when the runtime estimate is a poor fit for a stage, nothing in the offline tooling notices or reports it.

Describe the solution you'd like

Give the AutoTuner the ability to look at how GPU memory demand is distributed across the tasks of a stage, recognise when it is strongly uneven, and report that.

At the concept level:

• Consume per-stage GPU memory signals rather than only a single summary value per stage.
• Detect stages where a small number of tasks account for a disproportionate share of the memory demand, which is the pattern that causes the runtime estimate to fit most tasks and not the ones that matter.
• Report the finding in the tuner's output, identifying the stage and characterising the imbalance, so a user has something actionable to look at.
• Keep it observational to begin with. Reporting the condition is useful on its own and is separable from changing any recommendation. Whether the tuner should also adjust a configuration in response is a further question, and deliberately not part of this.

The intent is that this is a step toward better GPU memory advice generally, not a fix for one workload shape.

Describe alternatives you've considered

• Extending the existing shuffle-oriented skew and spill checks to cover this case. Those checks are built around shuffle volume and drive a shuffle-partition remedy, which is not meaningful for a stage that reads from files, so widening them would produce advice that cannot help.
• Relying on the plugin's runtime estimator alone. It adapts within a run and has information the tools do not, but it works from partial information as a stage progresses and cannot see the outcome. Offline tooling sees the completed run and can characterise the whole distribution, which is a genuinely different vantage point.
• Waiting until a specific remediation is agreed before doing anything. Reporting the condition has value independently, and building it first means any later remediation is designed against evidence rather than assumption.

Additional context

This depends on #2131, which adds the per-stage distribution reporting that a detector of this kind would consume. That issue is deliberately diagnostics only; this one is about acting on what it exposes.

Two scoping notes:

• The specific measure of imbalance, and the point at which a stage is considered skewed, are intentionally left open here. There is currently no body of real-world data to choose them from, which is part of what [FEA] Report per-stage distribution statistics for GPU task metrics #2131 exists to produce.
• This applies to the profiling path. The signals involved only exist in event logs from applications that ran with the plugin, so there is nothing equivalent available when analysing a CPU application.

Out of scope: any change to the plugin's runtime estimator, and any decision about which configuration property a future remediation would set.