Add full RC-KT reproducibility workflow

[skim0119]

Summary

• add the authoritative RC-KT manifest schema and example
• stream Intan RHS and Open Ephys data into aligned Ephys/Stimulus MiV HDF5 groups without downsampling
• add full-cohort spontaneous, RC, hierarchical-GPFA, and knowledge-transplant programs
• add 32-node allocation-attached Parsl/Slurm launchers with no nested scheduler submission
• add summaries, cohort tables, diagnostic plots, failure isolation, documentation, and structural tests

Knowledge-transfer workflow

KT.py treats the expert and student recordings as paired responses to the same experimental input. Their independently imported, filtered, detected, decoded, and trialized streams feed one KnowledgeTransferInputBuilder, followed by explicit GPFA, projection, readout, and transplant nodes:

expert_trials >> paired_input
student_trials >> paired_input
paired_input >> expert_trial_stream >> expert_gpfa
paired_input >> student_trial_stream >> student_gpfa
expert_gpfa >> student_gpfa
expert_gpfa >> expert_features >> expert_readout
student_gpfa >> student_features
expert_features >> transplant
student_features >> transplant
expert_readout >> transplant

Pipeline(transplant).run(output, cache, skip_plot=True)
transfer = transplant.output()

The GPFA fits, latent projections, expert readout, and transplant each retain independent caches and callbacks. The exposed expert_gpfa >> student_gpfa edge is intentional and limited to passing frozen expert kernel parameters into the student observation fit. The terminal result contains the paired latent features, expert readout, block-diagonal transform, and transplanted/refined weights.

The script then evaluates immediate transplant, a student scratch baseline, and prior-centered refinement over increasing student-training fractions. It records whether the expert kernel parameters were frozen for the student fit.

Parsl and Slurm topology

• Each launcher requests one allocation: 32 nodes, partition normal, 48 hours, one task and four cores per node.
• run_tasks() wraps each independent scientific unit in a Parsl python_app.
• AdHocProvider attaches workers to the existing allocation through local/SSH channels and never invokes sbatch.
• Parallel units are one recording for conversion, diagnostics, and RC, and one expert/student pair for KT.
• Each allocated node runs one four-core worker. Outside Slurm, the same API uses local threaded or serial execution.
• The target cluster must permit passwordless SSH among allocated nodes for the AdHocProvider configuration.

Outputs

• Per-script summary.json with configuration, versions, seed, included recordings, runtimes, failures, and headline metrics.
• CSV cohort tables and per-recording diagnostic plots.
• Preflight characterization panels, RC confusion matrices and longitudinal performance, and KT latent/learning-curve plots.
• Independent failures identify the recording or pair without stopping unrelated cohort tasks.

Dependencies

Built on operator PR #590 and dependency PR #588. The branch intentionally contains their commits until those PRs are integrated into pub/RC-KT.

Protocol conventions for review

The manifest, rather than filenames, is authoritative for inclusion and expert/student pairing. The code records the inferred GPFA timescale bounds and exposes implementation conventions not stated explicitly in the SI.

Validation

Focused lint, compilation, manifest, HDF5 conversion, grouped-import, Slurm, import-smoke, operator, callback, and paired-input KT checks pass locally (17 tests). Real data and cluster execution are intentionally not acceptance requirements.

[skim0119]

Block merge. Release state space embedding modules

[codecov-commenter]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 86.25%. Comparing base (7030e3b) to head (6cb3425).

Additional details and impacted files

@@            Coverage Diff             @@
##           pub/RC-KT     #591   +/-   ##
==========================================
  Coverage      86.25%   86.25%           
==========================================
  Files             28       28           
  Lines           1157     1157           
  Branches         115      115           
==========================================
  Hits             998      998           
  Misses           134      134           
  Partials          25       25           

Flag	Coverage Δ
mpi-tests	86.25% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.