Skip to content

Add RC-KT analysis operators - #590

Merged
skim0119 merged 21 commits into
mainfrom
feat/rc-kt-operators
Aug 31, 2026
Merged

Add RC-KT analysis operators#590
skim0119 merged 21 commits into
mainfrom
feat/rc-kt-operators

Conversation

@skim0119

@skim0119 skim0119 commented Aug 26, 2026

Copy link
Copy Markdown
Collaborator

Summary

Add reusable RC-KT analysis operators, numerical corrections required by the SI, grouped HDF5 imports, serializable result objects, deterministic seeds, algorithm-version cache fields, tests, and 0.5.0 migration notes.

Corrected existing implementations

  • BAKS: include the Bayesian beta term (n_spikes ** (4/5) by default), correct the Gaussian exponent, and use numerically stable bandwidth-ratio scaling.
  • Spike decay: replace the former alpha-shaped response with the causal rho * exp(-rho * tau) kernel and exclude future spikes.
  • Avalanche branching ratio: divide by the number of valid temporal transitions rather than avalanche size/channel count.
  • DirectedConnectivity: replace the Granger statistic previously reported as transfer entropy with pairwise discrete transfer entropy; use empirical shuffled-ISI bootstrap p-values; exclude self-connections; normalize by eligible directed pairs; and seed each channel pair deterministically.

These changes intentionally alter numerical results. Affected operators expose an algorithm_version in their cache identity and are documented as a MiV-OS 0.5.0 numerical migration.

New operators and APIs

  • BayesianAdaptiveKernelSmoother
  • BranchingRatio
  • TTLPulseDecoder
  • TTLPulseDecoder and FixedDurationTrializer as explicit serial nodes
  • ExponentialSpikeEncoder
  • KernelRank
  • SpectralRadius
  • RidgeReadout with stratified holdout, cross-validated ridge selection, balanced accuracy, and confusion matrix
  • KnowledgeTransferInputBuilder
  • KnowledgeTransferTrialSelector and GPFALatentProjector
  • standalone terminal KnowledgeTransfer

ImportSignal also gains HDF5 group and container selection so aligned Ephys and Stimulus groups can enter normal MiV graphs.

Paired-input knowledge transfer

KnowledgeTransferInputBuilder verifies that expert and student recordings contain the same number of trials and the same stimulus-label sequence. The downstream modules remain explicit:

expert_trials >> kt_input
student_trials >> kt_input
kt_input >> expert_trial_stream >> expert_gpfa
kt_input >> student_trial_stream >> student_gpfa
expert_gpfa >> student_gpfa
expert_gpfa >> expert_features >> expert_readout
student_gpfa >> student_features
expert_features >> transplant
student_features >> transplant
expert_readout >> transplant

Pipeline(transplant).run(...)
kt_result = transplant.output()

The graph performs the following explicit sequence:

  1. Fit expert GPFA and learn the bounded multiscale kernel parameters.
  2. Fit student GPFA while passing the expert result so those kernel parameters remain frozen.
  3. Extract paired endpoint latent features and fit the expert readout.
  4. Fit per-band ridge coordinate maps and assemble the block-diagonal transplant.
  5. Optionally perform prior-centered student refinement.

Every stage is independently runnable and cacheable. The expert-to-student edge is intentional: it is the serial dependency that freezes expert kernel parameters during the student fit. KnowledgeTransferResult includes paired latent features, labels, the expert readout, coordinate transform, transplanted weights, and refined weights. The lower-level pre-embedded latent-array API remains available.

Callback and persistence review note

  • Result caching uses the normal MiV dataclass cacher, including the terminal KT result.
  • BAKS, TTL decoding, fixed trialization, exponential encoding, GPFA projection, kernel rank, spectral-radius fitting, ridge readout, branching ratio, connectivity, and KT alignment expose standalone plot_* diagnostics.
  • Existing DirectedConnectivity matrix and histogram callbacks remain active, and the formerly disabled node-wise callback now runs when an MEA geometry is supplied.
  • Cohort/publication figures and tables are intentionally produced by the scripts in Add full RC-KT reproducibility workflow #591 because they aggregate multiple operators and recordings.
  • Routing-only nodes (KnowledgeTransferInputBuilder and trial selectors) intentionally have no plots.

Dependency

Built on #588. The direct miv-state-space dependency temporarily tracks feat/rc-kt-hgpfa; package publication will be handled separately.

Validation

Focused operator tests pass, including a synthetic fully modular pipeline test that verifies the student GPFA receives the expert reference and operator callbacks create their diagnostic artifacts. Ruff and Python compilation also pass. The full-suite pass is intentionally deferred to the follow-up test-fix review.

@skim0119
skim0119 force-pushed the feat/rc-kt-operators branch from 451d632 to 2072af5 Compare August 27, 2026 01:48
@skim0119

Copy link
Copy Markdown
Collaborator Author

Merge after checking unittests

@skim0119
skim0119 changed the base branch from pub/RC-KT to main August 29, 2026 03:27
@codecov-commenter

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 86.25%. Comparing base (b489106) to head (0df109f).
⚠️ Report is 3 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #590      +/-   ##
==========================================
+ Coverage   86.17%   86.25%   +0.08%     
==========================================
  Files          28       28              
  Lines        1157     1157              
  Branches      115      115              
==========================================
+ Hits          997      998       +1     
+ Misses        135      134       -1     
  Partials       25       25              
Flag Coverage Δ
mpi-tests 86.25% <ø> (+0.08%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@skim0119
skim0119 merged commit 3eb199b into main Aug 31, 2026
2 checks passed
@skim0119
skim0119 deleted the feat/rc-kt-operators branch August 31, 2026 02:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants