Skip to content
CLiMBlab-StanfordPublic

About

An engine for analyzing neural data

Resources

Contributing

Stars

0 stars

Watchers

0 watching

Forks

Repository files navigation

nro

nro processes anatomical and functional MRI from BIDS datasets. It produces preprocessed images, cleaned time courses, dynamic-connectivity series, small brain parcels, and individualized functional networks. Users request results; the orchestration layer finds their dependencies and distributes ready work across a site-wide Slurm worker pool.

The design separates scientific computation from scheduling. Modules declare their complete steps before execution; a common runner handles freshness, logging, and resumption. Public outputs have explicit contracts. Changed inputs, configurations, or contracts can invalidate results, while equivalent requests share existing work.

Derivatives are produced by modules with configurable settings. Sequences of configured modules are organized into workflows.

All module artifacts live under BIDS/PROJECT/derivatives/nro/MODULE/CONFIG_ID-LINEAGE_DIGEST/. Anatomy and functional processing have separate directories, so workflows can select either configuration independently.

The main processing graph contains these modules:

  • anat prepares anatomical images.
  • func prepares each functional run.
  • clean removes unwanted signal from functional data.
  • dynconn packages cleaned vertex- or voxel-level signals for interactive dynamic-connectivity viewing.
  • microparcellation divides the brain into small regions and measures their connectivity.
  • networks estimates individualized functional networks from either microparcel connectivity or dynamic-connectivity time series.
  • firstlevels estimates task effects from func outputs. Registered task models specify run, session, and subject contrasts; volume and surface results include effect, variance, t, and degrees-of-freedom maps.

The arrows show dependencies:

[anat] ──► [func] ──┬──► [clean] ──┬──► [dynconn] ────────────┐
                    │              └──► [microparcellation] ──┴──► [networks]
                    └──► [firstlevels]

[anat] ── "direct anatomical inputs" ──► {clean, microparcellation, networks, firstlevels}

Only one of the two configured inputs to `networks` is active in a workflow.

nro follows Semantic Versioning, and main contains released code. Each published version has an annotated Git tag and a corresponding GitHub Release. During the 0.x series, public interfaces may still change. The project retains older behavior when the benefit is clear and the implementation remains small, readable, and inexpensive to run. See the development guide for the release policy and the branching tutorial for development and release workflows.

Installation

Choose the guide for your site:

From a Linux checkout, run ./install. First setup creates a locked Python environment and asks where data, work files, private state, and processing resources belong. Personal and development installations are editable. Shared releases use staged, non-editable environments so a checkout update cannot alter the active command before maintenance succeeds. The system can use Slurm or run workers locally.

The user launcher is installed in ~/.local/bin; no environment activation is needed. The POSIX shell bootstrap downloads checksum-pinned uv and CPython, so the host does not need Python. It does require curl or wget, sha256sum, and tar. A FreeSurfer license is required when FreeSurfer-dependent processing runs, but not during installation. Some managed hosts need an administrator to enable the container runtime. See installation details for shared deployments, unattended setup, permissions, and dependency acquisition.

export PATH="$HOME/.local/bin:$PATH"
nro doctor

Basic usage

Run the complete workflow for one participant:

nro run -p 01 -P example

With no --module, requests reach every workflow endpoint: currently dynconn, networks, and firstlevels. Firstlevels selects models in set main for matching tasks. Shared upstream work runs once.

Run through a particular module, or select several participants:

nro run -p 01 02 -P example -m microparcellation

clean and downstream modules default to fsnative at 2 mm smoothing. Request only the space/smoothing combinations you need; multiple values produce their cross-product:

nro run -p 01 -P example -m networks \
  --space fsnative T1w --smoothing 0 2

The main workflow estimates networks from microparcellation with ICA. Select repeated clustering with -w clustering, the slower OSLOM backend with -w oslom, or dynconn-backed ICA with -w dynconn-networks.

Request task-effect maps separately:

nro models list
nro run -p 01 -P example -m firstlevels -s fsnative -S 2

New definitions stores start without task models. Add reviewed models to set main when they should run by default. See first-level models for model registration, supported BIDS Stats Models features, and inference limitations.

Edit a model or create it from a task's BIDS events when it is missing:

nro def edit model newtask
nro def edit model newtask main

The same command supports config CLASS ID, workflow ID, and source markup. See definition authoring for validation, local drafts, and shared-store safeguards.

Run nro help for task-oriented guidance from the installed CLI, or nro help --command COMMAND for a command's current syntax. nro --version reports which package release the selected installation uses.

Check progress and inspect a failed task:

nro status -P example
nro log -p 01 -P example -m func

Create one Workbench scene from matching completed derivatives:

nro scene -p 01 -P example -s fsnative -S 2 --open

The default scene links to its source derivatives. Add --publish to copy its inputs into a portable scene directory. --open runs wb_view through an X11 Slurm allocation on the site's viewing partition; it does not consume an nro worker slot. Add --serverless to run the viewer on the current host instead.

Render all named maps in a selection as static images:

nro render -p 01 -P example -m networks firstlevels

Seed-driven connectivity is skipped unless --seeds supplies subject-specific T1w coordinates. See the viewing guide for the seed file format and rendering controls.

Common controls are:

nro stop -p 01 -P example
nro set ls
nro get
nro set concurrency=100
nro purge -p 01 -P example
nro purge --excluded
nro gc -p 01 -P example

purge previews and confirms destructive cleanup. A bare purge selects all nro-controlled derivatives across all projects; --logs limits cleanup to eligible logs. purge --excluded removes derivatives that used paths now excluded by source markup, together with their downstream consumers. gc preserves controlled derivatives while removing unclaimed files from the matching public and private derivative namespaces.

The direct module form remains equivalent, for example:

.nro-env/bin/python -m nro.bin.run -p 01 -P example

Registration quality-control images can be generated separately:

nro qc registration 01 -p example

The equivalent engine entry point is .nro-env/bin/python -m nro.qc registration 01 -p example.

Workflow settings are selected with -w; the default is main. Configuration, model, optional source-markup, and protected site settings live in a separate, version-controlled definitions store. Use nro paths show to find it and nro def validate to check its contents.

Documentation

The documentation home links the full guide. Pages build as a Sphinx site compatible with Read the Docs.

Topic Guide
First use New installation quickstart, CLIMBLAB quickstart, installation
Work structure Design, concepts, work-item lifecycle
Scientific processing Module guides, denoising, software and methods sources
Configuration Workflows and parameters
Command-line interface Command reference
Cloud acquisition Flywheel-to-BIDS ingestion
Python development API guide, development and documentation builds

The module guides describe conditional processing paths and public artifact layouts. Working derivatives use BIDS-like names but are not claimed to be fully BIDS compliant. Publication and destructive cleanup are separate user commands.

Development

Run the test suite with:

.nro-env/bin/python -m pytest -q

The default command runs the development suite. The development guide gives commands for the integration tier and the complete suite.

Install with --dev to include test dependencies. The documentation build needs only docs/requirements.txt; it does not install neuroimaging containers or import the processing package. See the development guide for the build command.

AI-assisted development

This codebase was developed almost entirely with AI coding assistance. Human maintainers selected the requirements, reviewed the generated code, ran the applicable tests, and accept responsibility for the published result.

The tools and models used are recorded in AI_PROVENANCE.md. Where available, individual commits also contain Assisted-by: trailers.

About

An engine for analyzing neural data

Resources

Contributing

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages