Skip to content

level3 conv-family: accept seed to make ref ↔ cand verification possible - #145

Open
zhshgmail wants to merge 1 commit into
Just-it:mainfrom
zhshgmail:feat/conv-family-seed-input
Open

level3 conv-family: accept seed to make ref ↔ cand verification possible#145
zhshgmail wants to merge 1 commit into
Just-it:mainfrom
zhshgmail:feat/conv-family-seed-input

Conversation

@zhshgmail

Copy link
Copy Markdown

Problem

All five level-3 conv ops (6_ConvStandard1d, 7_ConvStandard2d, 8_ConvStandard3d, 9_ConvDepthwise2d, 10_ConvTranspose2d) have Model.forward() that constructs nn.ConvNd(...) inline on every call. The verification harness utils/verification_ascendc.py:306 does torch.manual_seed(0) once before the per-case for-loop. Inside each iteration it calls ref_model.forward(*args) then cand_model.forward(*args) back-to-back.

Because nn.ConvNd.__init__ draws random weights from the global RNG, and the RNG state advances after the ref call, cand draws different weights than ref. Verification then compares two different random outputs → ref ≠ cand by construction, regardless of whether the candidate kernel is correct.

This makes the conv family structurally unverifiable. It hits any worker porting nn.Conv*d to a candidate kernel: even an exact PyTorch re-implementation (e.g. F.conv1d(x, conv.weight, conv.bias, ...)) would diverge from the ref because the ref's conv.weight ≠ the ones the cand sees.

Discovered during an op-gen cold-start of op#6 ConvStandard1d on Ascend950PR (2026-04-28). The downstream workaround would be a workspace-local model.py with torch.manual_seed(...) inside forward — but that has to be reapplied per-op per-cold-start. Fixing it upstream unblocks every future cold-start at no cost.

Fix

Add a new optional seed kwarg (default 0) to each conv forward(). At the start of forward(), call torch.manual_seed(seed) so the inline conv construction draws weights from a fixed RNG state. Both ref and cand pass the same seed → both construct conv with identical weights → comparison is meaningful.

Also adds .to(x.device) after construction so the conv's weights track the input device (this is the existing CPU-vs-NPU device-mismatch error that the inline-construction pattern produces when called with NPU input — was already a separate bug, fixed here in passing).

get_input_groups() reads seed from the JSON case attrs (defaulting to 0) so existing JSON cases continue to work unchanged.

Backward compat

  • seed is optional with default 0. Existing JSON cases that don't include {"name": "seed", ...} continue to work via attr_inputs.get("seed", 0).
  • ModelNew / candidate implementations that don't pass seed will get seed=0 by default. They MUST also seed before any RNG-dependent construction (or extract weights from the seed-0 conv) to match ref.

Verification

Tested on A5 NPU (Ascend950PR_9589, container a5ops_dev3):

ref_preflight on op#6 ConvStandard1d: verdict=RUNNABLE
  (was FAILED with "Expected all tensors to be on the same device" before this fix)

Two consecutive `Model().forward(x, ..., seed=42)` calls — bit-exact identical output:
  6_ConvStandard1d: ✓ bit-exact
  7_ConvStandard2d: ✓ bit-exact
  9_ConvDepthwise2d: ✓ bit-exact
  10_ConvTranspose2d: ✓ bit-exact

(op#8 ConvStandard3d skipped in determinism test due to different x rank, but fix is identical pattern across all five files.)

Files changed

  • benchmarks/NPUKernelBench/level3/6_ConvStandard1d.py
  • benchmarks/NPUKernelBench/level3/7_ConvStandard2d.py
  • benchmarks/NPUKernelBench/level3/8_ConvStandard3d.py
  • benchmarks/NPUKernelBench/level3/9_ConvDepthwise2d.py
  • benchmarks/NPUKernelBench/level3/10_ConvTranspose2d.py

🤖 Generated with Claude Code

…ion possible

Affects level3/6_ConvStandard1d.py, 7_ConvStandard2d.py, 8_ConvStandard3d.py,
9_ConvDepthwise2d.py, 10_ConvTranspose2d.py.

## Problem

All five conv ops have `Model.forward()` that constructs `nn.ConvNd(...)` inline
on every call. The verification harness `utils/verification_ascendc.py:306` does
`torch.manual_seed(0)` ONCE before the per-case for-loop. Inside each iteration,
it calls `ref_model.forward(*args)` then `cand_model.forward(*args)` back-to-back.

Because `nn.ConvNd.__init__` draws random weights from the global RNG, and the
RNG state advances after the ref call, **cand draws different weights than ref**.
Verification then compares two different random outputs → ref ≠ cand by
construction, regardless of whether the candidate kernel is correct.

This makes the conv family structurally unverifiable. Hits any worker porting
`nn.Conv1d/2d/3d/Depthwise/Transpose2d` to a candidate kernel: even an exact
re-implementation in PyTorch (e.g. `F.conv1d(x, conv.weight, conv.bias, ...)`)
would diverge from the ref because the ref's `conv.weight` ≠ the ones the cand
sees.

## Fix

Add a new optional `seed` kwarg (default 0) to each conv `forward()`. At the
start of `forward()`, call `torch.manual_seed(seed)` so the inline conv
construction draws weights from a fixed RNG state. Both ref and cand pass the
same seed → both construct conv with identical weights → comparison works.

Also adds `.to(x.device)` after construction so the conv's weights track the
input device (handles the existing CPU-vs-NPU device-mismatch error that the
inline-construction pattern triggers when called with NPU input).

`get_input_groups()` extracts `seed` from the JSON case attrs (defaulting to 0)
so existing JSON cases continue to work unchanged. Adding a `seed` entry to
existing cases is optional — the default of 0 is sufficient for verification.

## Verification

Tested on A5 NPU (Ascend950PR_9589, container a5ops_dev3):
- ref_preflight on op#6 ConvStandard1d: verdict=RUNNABLE (was FAILED with
  "Expected all tensors to be on the same device" before this fix)
- Two consecutive `Model().forward(x, ..., seed=42)` calls produce bit-exact
  identical output for op#6/7/9/10. Verified by `torch.equal(out1, out2)`.
  (op#8 ConvStandard3d skipped in test due to different x rank, but
  fix is identical pattern.)

## Backward compat

- `seed` is optional with default 0. Existing JSON cases that don't include
  `{"name": "seed", ...}` continue to work — `attr_inputs.get("seed", 0)`
  returns 0.
- ModelNew / candidate implementations that don't pass seed will get seed=0
  by default. They MUST also seed before any RNG-dependent construction
  (or extract weights from the seed-0 conv) to match ref.

## Why now

Caught during op-gen cold-start of op#6 ConvStandard1d on Ascend950PR
(2026-04-28). Worker repeatedly hit the structural unverifiability before
the harness behavior was understood; downstream fix in worker model.py
(workspace-local OL-89 prose-spec extension with `torch.manual_seed`)
unblocks individual workers but the upstream fix unblocks every future
cold-start at no cost.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant