Skip to content

fix(storage): thread seed through iota fill so eval couples differ (#104)#113

Open
RealDiligent wants to merge 1 commit into
zeokin:mainfrom
RealDiligent:fix/iota-fill-respects-seed
Open

fix(storage): thread seed through iota fill so eval couples differ (#104)#113
RealDiligent wants to merge 1 commit into
zeokin:mainfrom
RealDiligent:fix/iota-fill-respects-seed

Conversation

@RealDiligent

@RealDiligent RealDiligent commented Jul 7, 2026

Copy link
Copy Markdown
Contributor

Summary

Fixes issue #104: _fill_iota() ignored the seed argument, so generate(..., fill="iota") always produced the same symmetric (i+j)%97 pattern. The eval harness offsets seeds to build distinct A_i and B_i couples (seed+2i / seed+2i+1), but with iota those offsets were no-ops — every couple had A == B and all --pairs were byte-identical, silently benchmarking A @ A instead of general A @ B.

Threads seed into _fill_iota as (i+j+seed) % 97 in both strategy/storage.py and matmul/storage.py, mirroring _fill_random / _fill_lowrank. Reopens the fix from closed PR #105 on current main.

Result

metric value
accuracy 1.0 (exact baseline unchanged; restores distinct A/B couples for --fill iota)
time complexity O(N³) unchanged
latency N/A
VRAM usage N/A

Regime measured: CPU-only regression tests (tests/test_iota_fill.py). No strategy cost claim.

Raw scorecard (paste python -m eval … output or --json)
Engine/input fix — not a strategy improvement PR.
Local CI (mirrors eval-policy.yml):
  tests/test_iota_fill.py: 5 passed
  pytest tests/ strategy/tests/ eval/tests/: 83 passed, 15 skipped (no GPU)
New tests verify: distinct seeds -> distinct matrices; eval couple offsets -> A != B; couples differ.
Closes #104

Checklist

  • I ran the scorer on unseen couples — no hardcoding of seeds/matrices. (N/A: input-generation fix)
  • Accuracy and latency come from the same run at the same dtype. (N/A)
  • This is an improvement (every cost axis down, accuracy held) or I
    state honestly which axis it trades — see the one rule in CONTRIBUTING.md.
    This PR restores eval harness semantics for --fill iota; not a strategy submission.
  • Correctness gates pass:
    python eval/tests/test_eval.py,
    python strategy/tests/test_subspace.py,
    python tests/test_correctness.py.
  • I named the device and dtype so a reviewer can reproduce the numbers.

<----ia--------e------- relabel refresh: 2026-07-07T14:23:28Z -->

@github-actions github-actions Bot added area:strategy Smart strategies / transforms (strategy/) area:matmul Exact engine and tile schedules (matmul/) area:tests Correctness gates and test suites (tests/) status:needs-review Awaiting maintainer review status:queued-gpu Feat/strategy PR passed non-GPU gates and is queued for the next batched GPU evaluation labels Jul 7, 2026
@github-actions

github-actions Bot commented Jul 7, 2026

Copy link
Copy Markdown

Gate chain passed. This PR is queued for the next batched GPU evaluation window.

Co-authored-by: Cursor <cursoragent@cursor.com>
@github-actions github-actions Bot added type:bug Something is incorrect or broken status:ready-non-gpu Fix/docs PR cleared non-GPU triage; maintainer review only and removed status:queued-gpu Feat/strategy PR passed non-GPU gates and is queued for the next batched GPU evaluation labels Jul 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:matmul Exact engine and tile schedules (matmul/) area:strategy Smart strategies / transforms (strategy/) area:tests Correctness gates and test suites (tests/) status:needs-review Awaiting maintainer review status:ready-non-gpu Fix/docs PR cleared non-GPU triage; maintainer review only type:bug Something is incorrect or broken

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant