Audit false-extern accounting and true fallback coverage

[eellison]

Problem

The model-attribution pipeline currently puts every node rejected by
torch._inductor.fx_passes.fusion_regions.is_fusible_node on the extern side
of the accounting boundary. That combines several materially different cases:

1. a true ATen/eager fallback;
2. an op with a conditional Inductor lowering;
3. an Inductor-lowered op excluded by a conservative fusion/capture rule.

This makes extern coverage hard to interpret and can hide useful compiler
targets. We should audit the current extern set, fix false-extern accounting,
and migrate actionable scopes into canonical repro and oracle coverage.

The numbers below come from the B200 occurrence manifests at PyTorch commit
fef6016ce793323baa162ff8ede53bfe434c1f86. They are direct-op timings
multiplied by model occurrence counts. Extern replay also fabricates tensor
values: in particular, it does not preserve the observed index duplication or
histogram distribution. These numbers show workload presence, not actual
compiled-path time or recoverable headroom; only a faithful compiled repro
and valid full-scope oracle can establish either.

Initial audit set

op/scope	current compiler status	corpus evidence	required investigation
Large-input aten.cat.default	Not a fallback. CUDA has a real cat() lowering that selects pointwise_cat or ConcatKernel. is_fusible_node rejects cats above max_pointwise_cat_inputs=8, so the capture pipeline calls them extern.	62 nodes, all in torchbench/densenet121: 31 inference and 31 training cats, with 9-25 inputs. 1,127.5us is currently attributed as direct-op time across the two model modes.	Capture/price the lowering actually generated by Inductor without simply raising the pointwise-cat cap. Preserve producer/consumer context, then use an oracle to decide whether any compiler optimization is justified.
aten.cumsum.default	Conditional Inductor scan lowering exists. A static CUDA probe emits Triton with tl.associative_scan, but initialization also creates fallback_cumsum = fallback_handler(...), adding the overload to the global fallbacks set. is_fusible_node therefore rejects even shapes that lower successfully.	44 occurrences across 20 models, 419.4us currently attributed as extern. GPT-OSS contributes 24 occurrences / 177.4us for an int32[32] prefix sum.	Make classification reflect the actual lowering for the concrete backend/shape, add canonical scan repros, and measure whether producer/consumer fusion exposes further headroom.
aten.histc.default	True fallback today (make_fallback(aten.histc); source notes that Histogram IR is missing).	GPT-OSS has 24 occurrences / 197.4us of standalone attribution: 4,000 int32 expert IDs, 32 bins. It feeds the 32-element cumsum used as grouped-MM expert offsets. Current replay does not preserve the [0,31] expert-ID distribution.	Build a faithful histogram/offset workload repro. Evaluate general Histogram IR versus a narrower sorted-expert-ID histogram + prefix-sum lowering. After a lowering exists, recapture the resulting full partition and build its oracle; do not price isolated histc as if it were the final optimization boundary.
Open-source bf16 aten.index_add.default	True conditional fallback. The Inductor decomposition returns NotImplemented for non-fbcode bf16 and leaves native index_add in place, avoiding the very slow accumulating-index_put path tracked in pytorch/pytorch#137425. Other dtypes decompose to accumulating index_put.	60 occurrences across training: Longformer has 36 occurrences / 140,941.8us and XLNet has 24 / 13,392.4us in standalone attribution. Current replay uses fabricated random indices; XLNet uses unique/iota-like indices while Longformer has structured duplication, so the aggregate charge may be dominated by artificial contention.	Extract workload repros that preserve index range, uniqueness/duplication, layout, and surrounding use. Compare native ATen with candidate Inductor strategies. After a lowering exists, recapture neighboring partitions and build a full-scope oracle; shape-only random indices are not a performance oracle.

Migration requirements

For each item:

• Verify whether each concrete occurrence generates Inductor IR or an ATen
  fallback. Do not infer execution solely from membership in the global
  fallbacks set, and preserve legitimate conditional fallbacks.
• Define the correct optimization boundary, including adjacent
  producers/consumers when they determine fusion or eliminate an
  intermediate.
• Add an exact workload repro and correctness test preserving shape, dtype,
  stride, output contract, and performance-relevant value properties such
  as index uniqueness/contention or sorted/ranged IDs.
• For false externs, recapture the actual lowered partition. For true
  fallbacks, first land a profitable lowering or document why the observed
  case cannot yet be lowered, then recapture because neighboring partition
  boundaries may change.
• Add a numerically faithful full-scope oracle/floor and record
  current-versus-oracle B200 timings. An isolated-op oracle must not price a
  larger fused partition. Include no-CD/CD measurements where scheduler
  tuning is applicable.
• Identify and benchmark the corresponding Inductor optimization. If no
  profitable implementation exists, retain the scope as explicitly
  measured/unpriced rather than claiming headroom.
• Regenerate occurrence manifests and oracle/model-headroom reports.
  Lowered scopes must leave the generic extern bucket; true fallbacks must
  remain explicitly labeled until their lowering lands.
• Add regression tests for classification, extraction, occurrence
  accounting, and canonical replay.

Completion criteria

The audit is complete when every listed occurrence is classified from its
actual compile behavior, each actionable scope has a faithful workload repro,
lowered scopes have recaptured canonical partitions and valid full-scope
oracles, and the model rollup no longer treats lowered cat/cumsum scopes as
indistinguishable from true histc/bf16-index_add fallbacks.