Skip to content

Add eigenstrapping null generation#13

Open
VictorBarnes wants to merge 63 commits intoNSBLab:devfrom
VictorBarnes:feature/eigenstrapping
Open

Add eigenstrapping null generation#13
VictorBarnes wants to merge 63 commits intoNSBLab:devfrom
VictorBarnes:feature/eigenstrapping

Conversation

@VictorBarnes
Copy link
Collaborator

@VictorBarnes VictorBarnes commented Feb 17, 2026

  • Created new module nulls.py with a main function eigenstrap to generate eigenstrapped nulls. This function uses _eigenstrap_single to generate each null.
  • Added eigenstrap as a class method to EigenSolver
  • Added get_eigengroup_inds to eigen.py
  • Removed joblib as optional dependency and made it a project dependency
    • Removed the check for joblib in io.py and removed corresponding test in test_io.py
  • Added tests in test_nulls.py
  • Added a tutorial in eigenstrapping.ipynb

Commit f6f95f1 contains code to exactly match the nulls generated in the original eigenstrapping code by Nikitas Koussis if we need to go back to compare. The key differences include:

  • Set the default eigendecomposition method to project the data into mass-orthonormal space (as discussed above). See also this lapy issue. Regress is also included as a decomposition method and is a good option to use when the input data contains NaNs. The mass matrix is an optional input accordingly.
  • Consolidated add_res and permute into one argument residual which can either be None, add or permute
  • Use all modes in computation (including constant mode). This is to be consistent with the rest of the toolbox where all main functions always take in all modes as an input. This also makes eigenstrap cleaner. To avoid division by zero in the normalisation, we set the first eigenvalue to be 1. The first mode doesn't get rotated but keeping this in will preserve the mean of the nulls when resample=None.
  • Require all modes, data, and mass matrix to already be masked when passed in. Again, this keeps this cleaner and also avoids conflicts with the EigenSolver which already does masking. Since eigenstrap is a method of that class, adding a masking option in eigenstrap would lead to conflicts with the EigenSolver mask.
  • Removed normalize as an option.
  • Seeding is handled in two stages to ensure reproducibility when parallelising. First seed is used to initialize a master random number generator (RNG) that generates an integer seed for each null map. Then each null uses its allocated integer to generate its own RNG to use for all rotations/permutations of that null. This ensures that each null is independent of the number of parallel jobs used.
  • Refactor resample to include multiple options: 'match' to match the sorted of the original data, 'zscore' to z-score and rescale to original mean std, 'mean' to preserve the mean, and 'range' to preserve the minimum and maximum. Default is None for no resampling.

- assume all modes are given and discard the first
- rename to `eigenstrap`
- always perform spheroid transform
- remove joblib test in test_io.py
- refactor eigen.get_eigengroup_inds to take in n_modes as only input argument
- update toml to use latest lapy version on github
Other minor changes include:
- improve docstrings
- fix randomize bug
- update uv.lock to include joblib and github version of lapy
magnesium2400 and others added 21 commits February 17, 2026 15:22
Modify lab neuromodes PR NSBLab#13 to simplify dealing with first eigenmode
magnesium2400 and others added 22 commits February 20, 2026 19:31
… squeeze bug in last line. some minor changes to formatting otherwise
…or reproducibility across different n_nulls in eigenstrap()
Support multi-map eigenstrapping + speed up computation + adjust docs
- Fixed bug when resample="exact"
- Improved docstrings and comments
- Used resample="exact" in tutorial
- Added L and R meylinmap files for fsLR 4k
- Updated the L and R medmasks to correctly mask out all medial wall vertices (cortical vertices reduced from 3636 to 3619)
- Updated all corresponding tests and test data
- Updated nulls.eigenstrap docstring
- Using _eigenstrap_multiple caused "test_internull_corrs" to fail. Therefore, reverted to _eigenstrap_single for now.
- Fixed deprecation bug in eigenstrapping_sphere.ipynb
@magnesium2400
Copy link
Collaborator

magnesium2400 commented Feb 27, 2026

why doesnt the nulls sphere PSD match the original any more? test_psd_preservation still passes i presume

@VictorBarnes
Copy link
Collaborator Author

why doesnt the nulls sphere PSD match the original any more? test_psd_preservation still passes i presume

Looks like I just forgot to rerun that notebook before committing. The PSD looks like that when resample="range". Whoever makes the next commit can just make sure to rerun it.

magnesium2400 and others added 6 commits February 27, 2026 22:47
…two functions that can be used to generate rotation matrices. set names of these functions to _rotate_coeffs_qr and _scipy (from _eigenstrap_multiple and _single). change input/output of _rotate_coeffs_scipy (nee _eigenstrap_single) so that it does concatenation internally -- but there should not be any changes to functionality. these can now be called using eigenstrap(..., rotation_method={qr,scipy}). update cortex and sphere notebooks to each use one method (for ongoing testing). all tests pass with both method (_qr is 5-20x faster than _scipy). TODO : compare _scipy results with original eigenstrapping
- Add `eigenstrapping_match_orig.ipynb` to test our implementation matches original
- Change `eigenstrapping.ipynb` to use `resample="exact"`
- Fix missing import in `test_nulls.py`
- Remove jupyter_bokeh from toml
- Update eigenstrap docstring
- Exactly match default version of original eigenstrapping in validation notebook
- Save original nulls as test data to use for validation notebook and tests
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants