Skip to content

Unify sweep trees into conf/scaling/ + Python metric presets - #17

Open
philipnickel wants to merge 1 commit into
masterfrom
refactor/hydra-scaling-group-and-python-metrics
Open

philipnickel wants to merge 1 commit into
masterfrom
refactor/hydra-scaling-group-and-python-metrics

Conversation

@philipnickel

Copy link
Copy Markdown
Collaborator

What

Two readability/maintainability refactors of the Hydra/hydra-zen config surface, verified to leave the composed configs unchanged.

1. conf/scaling/ axis group — single source of truth

The swept sim.* values were duplicated across two parallel trees (conf/sweep/* for timing, conf/nsight_sweep/* for NCU):

value before after
9-value particle list 4 files 1 (scaling/particle_count), shared
${ppc_grid:…} derivation 2 files 1 (scaling/weak)
density grid list + N=20M 2 files 1 (scaling/density)
sm operating point 2–3 files 1 (scaling/sm)

Both wrapper trees now pull the axis via defaults: - /scaling: <axis> and keep only their genuine differences (backend list ±jax, plots vs nsight_plot callback, output-dir roots).

2. Metric presets as a Python source of truth — nsight_metric_sets.py

conf/nsight_metrics/{roofline,full}.yaml hand-duplicated the roofline counters. FULL_METRICS is now composed as TIMING + ROOFLINE + extras, so the roofline subset is defined once. Registered as the nsight_metrics config group (package nsight.analyze), imported by profile_nsight.py before Hydra composes. conf/nsight_metrics/ is removed.

Verification

  • --cfg job/--cfg hydra for all 9 sweep selections + 3 metric presets is byte-identical before/after, except:
    1. the full preset's metric order (set proven identical: 59 metrics, no dupes — order is irrelevant to NCU and the long-format parquet);
    2. the expected new scaling: entry under hydra.runtime.choices.
  • 68 tests pass, ruff clean.
  • Runtime-validated on an A100 across all five backends: sim=benchmark, both timing sweeps (${ppc_grid:…} derivation confirmed: 100k→G=28, 200k→G=36), and NCU collection driven by the Python presets (roofline 15/15, full 59/59, zero missing), including an nsight_sweep multirun that produced all five figures.

Follow-up (not in this PR)

P2 consistency pass: rename wrappers to the canonical axis names (weak_scalingweak, particle_densitydensity), hoist the duplicated gpu_kind resolver, normalize indentation. Two stale .claude/CLAUDE.md pointers to conf/nsight_metrics/ should also be updated (the edit was auto-blocked as self-modification).

🤖 Generated with Claude Code

…c presets

Two readability/maintainability refactors against the Hydra/hydra-zen config
surface, verified to leave the composed configs unchanged.

1. conf/scaling/ axis group (single source of truth)
   The swept sim.* values were duplicated across two parallel trees
   (conf/sweep/* for timing, conf/nsight_sweep/* for NCU): the 9-value
   particle list lived in 4 files, the ${ppc_grid:N} derivation in 2, the
   density grid list in 2, the sm operating point in 2-3. They now live once
   per axis in conf/scaling/{particle_count,weak,density,sm}.yaml; both wrapper
   trees pull them via `defaults: - /scaling: <axis>` and keep only their
   genuine differences (backend list +/- jax, plots vs nsight_plot callback,
   output-dir roots).

2. Metric presets as a Python source of truth (nsight_metric_sets.py)
   conf/nsight_metrics/{roofline,full}.yaml hand-duplicated the roofline
   counters. FULL_METRICS is now composed as TIMING + ROOFLINE + extras, so the
   roofline subset is defined once. Registered as the nsight_metrics config
   group (package nsight.analyze), imported by profile_nsight.py before Hydra
   composes; conf/nsight_metrics/ is removed.

Verification: --cfg job/hydra for all 9 sweep selections + 3 metric presets is
byte-identical before/after, except (a) the full preset's metric *order* (set
proven identical, 59 metrics, no dupes; order is irrelevant to NCU and the
long-format parquet) and (b) the expected new `scaling:` entry under
hydra.runtime.choices. 68 tests pass; ruff clean. Runtime-validated on an A100
across all five backends: benchmark, both timing sweeps (ppc_grid derivation
confirmed), and NCU collection (roofline 15/15, full 59/59) incl. an
nsight_sweep multirun with all five figures.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant