feat: configurable initial t for the adaptive GAH batch norm - #13
Merged
Merged
Conversation
The learnable t of the adaptive GAH mean was hard-coded to 0.5, so the
t_init options of downstream configs (spdnet-training
batchnorm_t_gah_init) had no effect. mean_options={"t_init": t} now sets
it (validated in [0, 1]; default 0.5 unchanged). Docstrings list the
adaptive GAH mean type.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PQdVCDbXCd8gvf1Y4TufJR
There was a problem hiding this comment.
Copilot review overview
🟡 Changes recommended
Endpoint initialization is inaccurate, and unsupported scalar-variance behavior is documented as available.
Review effort: Balanced
Findings: 1
Open (1)
What changed in this PR
Adds configurable initialization for adaptive GAH batch normalization.
Changes:
- Reads and validates
mean_options["t_init"]. - Documents the option and adds initialization/gradient tests.
| File | Description |
|---|---|
src/yetanotherspdnet/nn/batchnorm.py |
Configures adaptive GAH initialization and updates docstrings. |
tests/nn/test_batchnorm.py |
Tests initialization, gradients, and validation. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Comment on lines
+359
to
+360
| if not 0.0 <= t_init <= 1.0: | ||
| raise ValueError(f"t_init must lie in [0, 1], got {t_init}") |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
BatchNormSPDMean(mean_type="adaptive_geometric_arithmetic_harmonic")always started its learnabletat 0.5.mean_optionswas ignored for this mean. As a result, spdnet-training'sbatchnorm_t_gah_initoption has never had any effect: its forwarding is also wrong, see the companion spdnet-training PR.What
mean_options={"t_init": t}sets the initialt. The value is validated in [0, 1], where 0 is harmonic and 1 is arithmetic. The sigmoid parametrization'sright_inversemaps it to the unconstrained parameter.adaptive_geometric_arithmetic_harmonicand document the option.ValueError.Not in scope, pre-existing
BatchNormSPDMeanScalarVariancehas no dispersion function for the adaptive GAH mean:_init_stdhas no branch for it, and the forward raisesAttributeError: ... no attribute 'std_fun'. This is unchanged here.Test
pytest tests/nn/test_batchnorm.py tests/test_model.py: 3209 passed.ruff format --check/ruff check(0.16.7): clean.