Skip to content

Commit 576dee6

Browse files
committed
add tests and improve plotting
1 parent e202a66 commit 576dee6

3 files changed

Lines changed: 320 additions & 13 deletions

File tree

CLAUDE.md

Lines changed: 95 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,95 @@
1+
# CLAUDE.md
2+
3+
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
4+
5+
## Overview
6+
7+
`meg_utils` is a shared Python utility library for MEG (Magnetoencephalography) analysis, designed to be used as a git submodule across multiple projects. It wraps MNE-Python and scikit-learn with MEG-specific conveniences.
8+
9+
## Commands
10+
11+
### Install (editable)
12+
```bash
13+
pip install -e .
14+
pip install pytest # for running tests
15+
```
16+
17+
### Run all tests
18+
```bash
19+
pytest tests/ -v
20+
```
21+
22+
### Run a single test file
23+
```bash
24+
pytest tests/test_decoding.py -v
25+
```
26+
27+
### Run a single test class or method
28+
```bash
29+
pytest tests/test_misc.py::TestLongDfToArray::test_roundtrip_basic -v
30+
```
31+
32+
### Sync all instances of meg_utils across Nextcloud repos
33+
```bash
34+
python _synchronize_meg_utils.py
35+
```
36+
37+
## Repository Structure
38+
39+
The package lives under `meg_utils/` (the inner directory). The root also contains legacy top-level `.py` stubs (e.g., `decoding.py`, `plotting.py`) that are **not** the active source — always edit files under `meg_utils/`.
40+
41+
```
42+
meg_utils/ ← Python package (the real source)
43+
__init__.py ← imports all submodules + exposes Stop
44+
_constants.py ← MEGIN TRIUX sensor index arrays (idx_grad, idx_mag)
45+
conversion.py ← FIF↔EDF conversion
46+
data.py ← load_meg, load_epochs, load_segments, load_events
47+
decoding.py ← classifiers, CV, heatmaps, stratify, reduce_dimensions
48+
misc.py ← file utilities, hashing, to_long_df / long_df_to_array
49+
pipeline.py ← DataPipeline (sklearn-style MNE processing steps)
50+
plotting.py ← sensor plots, GIF creation, figure helpers
51+
preprocessing.py ← MEG rescaling, autoreject, robust scaling
52+
sigproc.py ← filtering, resampling, alpha peak, oscillation tools
53+
tests/
54+
test_decoding.py
55+
test_misc.py
56+
test_sigproc.py
57+
_synchronize_meg_utils.py ← pulls updates in all Nextcloud copies of this repo
58+
```
59+
60+
## Architecture
61+
62+
### Module responsibilities
63+
64+
- **`data.py`**: High-level loaders that compose preprocessing into single calls (`load_meg`, `load_epochs`, `load_segments`). Accepts ICA, autoreject, custom filter functions, and event filtering.
65+
66+
- **`pipeline.py`**: sklearn-style `DataPipeline` extending `sklearn.pipeline.Pipeline`. Each step (`LoadRawStep`, `FilterStep`, `ResampleStep`, `CropStep`, `EpochingStep`, `NormalizationStep`, `ToArrayStep`) declares `type_in`/`type_out` tuples; `DataPipeline.check_steps()` validates the chain at construction time. Use `pipeline.set_params_all(n_jobs=4)` to propagate a parameter to all steps that accept it.
67+
> **Note**: `ICAStep` and `StratifyStep` raise warnings/exceptions — they are ChatGPT drafts that need manual verification before use.
68+
69+
- **`decoding.py`**: Time-resolved decoding utilities. Key patterns:
70+
- `cross_validation_across_time(data_x, data_y, clf)` expects `data_x` shape `(n_samples, n_features, n_timepoints)` and returns a long-format DataFrame of per-fold accuracy per timepoint.
71+
- `LogisticRegressionOvaNegX`: one-vs-all LR that accepts an external `neg_x` "null" class during `fit()`.
72+
- `TimeEnsembleVoting`: trains one classifier clone per time slice of a 3D array and aggregates via VotingClassifier.
73+
- `reduce_dimensions(data, axis, n_components)`: fits PCA along any axis and returns `(reduced_data, reducer)` where `reducer` can be called on new data.
74+
- `save_clf(clf, filename)`: saves `.pkl.gz` + JSON sidecar with hyperparams and calling code line.
75+
76+
- **`misc.py`**:
77+
- `to_long_df(arr, columns, value_name, **col_labels)` / `long_df_to_array(df, columns, value_name)`: round-trip conversion between N-D numpy arrays and long-format DataFrames. Fortran-order linearisation. Dimensions named `None`, `False`, or `'_'` are skipped.
78+
- `NumpyEncoder`: JSON encoder for numpy types (used in `save_clf`).
79+
- `Stop`: raises `KeyboardInterrupt` subclass that exits a script cleanly to REPL without traceback — use `raise Stop` instead of `raise SystemExit` in interactive sessions.
80+
- `telegram_callback`: decorator that sends Telegram messages on function begin/finish/error via `telegram_send`.
81+
82+
- **`sigproc.py`**: Array-level signal processing (no MNE objects required for most functions). Includes `bandpass`, `notch`, `resample`, `estimate_peak_alpha_freq`, `get_alpha_peak`, `fit_curve`, `interpolate`, `sliding_window`, `wave_speed_cm`.
83+
84+
- **`_constants.py`**: `idx_grad` and `idx_mag` — NumPy integer arrays of gradiometer and magnetometer channel indices for the MEGIN TRIUX 306-channel system (102 mags, 204 grads).
85+
86+
### Data conventions
87+
88+
- MEG epoch arrays follow the shape `(n_samples, n_channels, n_timepoints)` throughout.
89+
- Default sampling frequency after preprocessing is **100 Hz**.
90+
- Default epoch window: `tmin=-0.1`, `tmax=0.5` seconds.
91+
- `cross_validation_across_time` requires **balanced classes** (equal samples per class) — use `stratify()` first if needed.
92+
93+
### Testing approach
94+
95+
Tests use `unittest.TestCase` (decoding, sigproc) and plain pytest classes (misc). They do not require real MEG data — all tests use synthetically generated numpy arrays. CI runs against Python 3.10, 3.12, and 3.14.

meg_utils/plotting.py

Lines changed: 30 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -453,7 +453,8 @@ def make_fig(
453453
sns.despine(fig)
454454
return fig, axs, *axs_bottom
455455

456-
def savefig(fig, file, tight=True, despine=True, metadata=None, **kwargs):
456+
def savefig(fig, file, tight=True, despine=True, metadata=None,
457+
save_vector=True, **kwargs):
457458
"""
458459
Save a Matplotlib figure to a specified file with optional adjustments and metadata.
459460
@@ -475,6 +476,10 @@ def savefig(fig, file, tight=True, despine=True, metadata=None, **kwargs):
475476
despine : bool, optional
476477
If True, removes the top and right spines from the figure using
477478
`sns.despine()`. Default is True.
479+
save_vector : bool, optional
480+
If True, saves additional SVG and EPS copies of the figure to a
481+
'vectors/' subfolder in the same directory as `file`. The `dpi`
482+
kwarg is excluded when saving vector formats. Default is True.
478483
metadata : dict | str | False | None, optional
479484
Metadata to embed in the image file:
480485
- dict: Custom metadata key-value pairs
@@ -520,21 +525,33 @@ def savefig(fig, file, tight=True, despine=True, metadata=None, **kwargs):
520525
file = file + '.png'
521526
fig.savefig(file, **kwargs)
522527

523-
# Add metadata to PNG or JPG files if provided
528+
# Resolve metadata to a dict (or None if disabled)
524529
if metadata is False:
525-
# Explicitly skip metadata
526-
pass
530+
resolved_metadata = None
527531
elif metadata is None:
528-
# Auto-generate default metadata
529-
metadata = _generate_default_metadata()
530-
_add_image_metadata(file, metadata)
532+
resolved_metadata = _generate_default_metadata()
531533
elif isinstance(metadata, str):
532-
# Convert string to dict
533-
metadata = {'metadata': metadata}
534-
_add_image_metadata(file, metadata)
535-
elif isinstance(metadata, dict):
536-
# Use provided metadata dict
537-
_add_image_metadata(file, metadata)
534+
resolved_metadata = {'metadata': metadata}
535+
else:
536+
resolved_metadata = metadata
537+
538+
if save_vector:
539+
vec_dir = os.path.join(out_dir, 'vectors') if out_dir else 'vectors'
540+
os.makedirs(vec_dir, exist_ok=True)
541+
basename = os.path.splitext(os.path.basename(file))[0]
542+
vec_kwargs = {k: v for k, v in kwargs.items() if k != 'dpi'}
543+
for ext in ('svg', 'eps'):
544+
vec_file = os.path.join(vec_dir, f'{basename}.{ext}')
545+
fig.savefig(vec_file, **vec_kwargs)
546+
if resolved_metadata:
547+
json_file = os.path.join(vec_dir, f'{basename}.json')
548+
with open(json_file, 'w') as f:
549+
json.dump(resolved_metadata, f, indent=4)
550+
551+
# Add metadata to PNG or JPG files if provided
552+
if resolved_metadata:
553+
_add_image_metadata(file, resolved_metadata)
554+
538555

539556

540557
def _generate_default_metadata():

tests/test_plotting.py

Lines changed: 195 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,195 @@
1+
#!/usr/bin/env python3
2+
# -*- coding: utf-8 -*-
3+
"""
4+
Tests for meg_utils.plotting — focusing on savefig and vector output.
5+
"""
6+
import sys; sys.path.append('../..')
7+
8+
import json
9+
import os
10+
import tempfile
11+
12+
import matplotlib
13+
matplotlib.use('Agg')
14+
import matplotlib.pyplot as plt
15+
import pytest
16+
from PIL import Image
17+
18+
from meg_utils.plotting import savefig
19+
20+
21+
# ---------------------------------------------------------------------------
22+
# Helpers
23+
# ---------------------------------------------------------------------------
24+
25+
def _simple_fig():
26+
fig, ax = plt.subplots()
27+
ax.plot([1, 2, 3], [1, 4, 9])
28+
return fig
29+
30+
31+
# ---------------------------------------------------------------------------
32+
# savefig — basic file creation
33+
# ---------------------------------------------------------------------------
34+
35+
class TestSavefig:
36+
37+
def test_saves_png(self, tmp_path):
38+
fig = _simple_fig()
39+
out = str(tmp_path / 'plot.png')
40+
savefig(fig, out, metadata=False)
41+
assert os.path.exists(out)
42+
plt.close('all')
43+
44+
def test_default_extension_added(self, tmp_path):
45+
fig = _simple_fig()
46+
out = str(tmp_path / 'plot')
47+
savefig(fig, out, metadata=False)
48+
assert os.path.exists(out + '.png')
49+
plt.close('all')
50+
51+
def test_saves_jpg(self, tmp_path):
52+
fig = _simple_fig()
53+
out = str(tmp_path / 'plot.jpg')
54+
savefig(fig, out, metadata=False)
55+
assert os.path.exists(out)
56+
plt.close('all')
57+
58+
def test_saves_svg(self, tmp_path):
59+
fig = _simple_fig()
60+
out = str(tmp_path / 'plot.svg')
61+
savefig(fig, out, metadata=False)
62+
assert os.path.exists(out)
63+
plt.close('all')
64+
65+
def test_creates_output_directory(self, tmp_path):
66+
fig = _simple_fig()
67+
out = str(tmp_path / 'subdir' / 'deep' / 'plot.png')
68+
savefig(fig, out, metadata=False)
69+
assert os.path.exists(out)
70+
plt.close('all')
71+
72+
73+
# ---------------------------------------------------------------------------
74+
# savefig — vector output (save_vector=True)
75+
# ---------------------------------------------------------------------------
76+
77+
class TestSavefigVector:
78+
79+
def test_vector_creates_vectors_subdir(self, tmp_path):
80+
fig = _simple_fig()
81+
out = str(tmp_path / 'plot.png')
82+
savefig(fig, out, save_vector=True, metadata=False)
83+
assert os.path.isdir(str(tmp_path / 'vectors'))
84+
plt.close('all')
85+
86+
def test_vector_creates_svg(self, tmp_path):
87+
fig = _simple_fig()
88+
out = str(tmp_path / 'plot.png')
89+
savefig(fig, out, save_vector=True, metadata=False)
90+
assert os.path.exists(str(tmp_path / 'vectors' / 'plot.svg'))
91+
plt.close('all')
92+
93+
def test_vector_creates_eps(self, tmp_path):
94+
fig = _simple_fig()
95+
out = str(tmp_path / 'plot.png')
96+
savefig(fig, out, save_vector=True, metadata=False)
97+
assert os.path.exists(str(tmp_path / 'vectors' / 'plot.eps'))
98+
plt.close('all')
99+
100+
def test_vector_disabled(self, tmp_path):
101+
fig = _simple_fig()
102+
out = str(tmp_path / 'plot.png')
103+
savefig(fig, out, save_vector=False, metadata=False)
104+
assert not os.path.isdir(str(tmp_path / 'vectors'))
105+
plt.close('all')
106+
107+
def test_vector_basename_matches_source(self, tmp_path):
108+
fig = _simple_fig()
109+
out = str(tmp_path / 'my_figure.png')
110+
savefig(fig, out, save_vector=True, metadata=False)
111+
assert os.path.exists(str(tmp_path / 'vectors' / 'my_figure.svg'))
112+
assert os.path.exists(str(tmp_path / 'vectors' / 'my_figure.eps'))
113+
plt.close('all')
114+
115+
def test_vector_dpi_kwarg_excluded(self, tmp_path):
116+
"""dpi should not be passed to vector formats (no error raised)."""
117+
fig = _simple_fig()
118+
out = str(tmp_path / 'plot.png')
119+
savefig(fig, out, save_vector=True, metadata=False, dpi=300)
120+
assert os.path.exists(str(tmp_path / 'vectors' / 'plot.svg'))
121+
plt.close('all')
122+
123+
def test_vector_metadata_json_written(self, tmp_path):
124+
fig = _simple_fig()
125+
out = str(tmp_path / 'plot.png')
126+
savefig(fig, out, save_vector=True, metadata={'key': 'value'})
127+
json_file = str(tmp_path / 'vectors' / 'plot.json')
128+
assert os.path.exists(json_file)
129+
with open(json_file) as f:
130+
data = json.load(f)
131+
assert data['key'] == 'value'
132+
plt.close('all')
133+
134+
def test_vector_no_json_when_metadata_false(self, tmp_path):
135+
fig = _simple_fig()
136+
out = str(tmp_path / 'plot.png')
137+
savefig(fig, out, save_vector=True, metadata=False)
138+
json_file = str(tmp_path / 'vectors' / 'plot.json')
139+
assert not os.path.exists(json_file)
140+
plt.close('all')
141+
142+
def test_vector_svg_for_svg_source(self, tmp_path):
143+
"""When saving an SVG directly, vectors/ still gets SVG and EPS copies."""
144+
fig = _simple_fig()
145+
out = str(tmp_path / 'plot.svg')
146+
savefig(fig, out, save_vector=True, metadata=False)
147+
assert os.path.exists(str(tmp_path / 'vectors' / 'plot.svg'))
148+
assert os.path.exists(str(tmp_path / 'vectors' / 'plot.eps'))
149+
plt.close('all')
150+
151+
152+
# ---------------------------------------------------------------------------
153+
# savefig — metadata embedding
154+
# ---------------------------------------------------------------------------
155+
156+
class TestSavefigMetadata:
157+
158+
def test_png_metadata_dict(self, tmp_path):
159+
fig = _simple_fig()
160+
out = str(tmp_path / 'plot.png')
161+
savefig(fig, out, metadata={'author': 'test', 'experiment': 'MEG'})
162+
img = Image.open(out)
163+
assert img.text.get('author') == 'test'
164+
assert img.text.get('experiment') == 'MEG'
165+
img.close()
166+
plt.close('all')
167+
168+
def test_png_metadata_string(self, tmp_path):
169+
fig = _simple_fig()
170+
out = str(tmp_path / 'plot.png')
171+
savefig(fig, out, metadata='my note')
172+
img = Image.open(out)
173+
assert img.text.get('metadata') == 'my note'
174+
img.close()
175+
plt.close('all')
176+
177+
def test_png_metadata_false_no_chunks(self, tmp_path):
178+
fig = _simple_fig()
179+
out = str(tmp_path / 'plot.png')
180+
savefig(fig, out, metadata=False)
181+
img = Image.open(out)
182+
# No custom text chunks should be present
183+
assert 'author' not in img.text
184+
img.close()
185+
plt.close('all')
186+
187+
def test_png_metadata_none_auto(self, tmp_path):
188+
"""metadata=None should auto-generate at least script_path."""
189+
fig = _simple_fig()
190+
out = str(tmp_path / 'plot.png')
191+
savefig(fig, out, save_vector=False, metadata=None)
192+
img = Image.open(out)
193+
assert 'script_path' in img.text
194+
img.close()
195+
plt.close('all')

0 commit comments

Comments
 (0)