Add quick-start and full-analysis notebooks, fix fetch_band performance#11
Open
TincaSubic wants to merge 9 commits into
Open
Add quick-start and full-analysis notebooks, fix fetch_band performance#11TincaSubic wants to merge 9 commits into
TincaSubic wants to merge 9 commits into
Conversation
- docs/quick_start.ipynb + docs/quick_start.py: end-to-end dLEM example on H1-hESC Hi-C (chr10:20.5–22.5 Mb, 10 kb). Trains with correct settings (multinomial loss, band rows 1–169, slowdown=0.025) and produces a split contact map with L/R cohesin speed tracks. - docs/data/example_chr10.cool: minimal Hi-C file (308 KB) extracted from 4DNFI9GMP2J8, chromosome stored as ref_region with coords shifted to 0; docs/data/create_example_data.py regenerates it. - .gitignore: exempt docs/data/ from the data/ exclusion rule. - dlem/api.py: TODO comment on early-stopping verbose message — it reads "no MSE improvement" even when loss_type='multinomial'. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Cleaned up corruption from accidental re-run: removed duplicate section-3 and section-5 markdown headers, restored train_dlem code cell, fixed _log_eo/combined cell type from markdown back to code, and added the missing section-4 heading. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…adata - pyproject.toml: add seaborn>=0.13.0 as runtime dependency (used by quick_start for the vlag colormap); remove pytest from runtime deps (test-only tool, already in dev dependency-group) - docs/quick_start.ipynb: fix kernelspec from stale 'dlem' to 'python3' so jupyter execute works in any env; strip invalid execution_count/ outputs from markdown cells; add required execution_count/outputs to code cells that were missing them after the corruption repair Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- docs/full_analysis.ipynb: downloads H1-hESC Hi-C (4DNFI9GMP2J8, ~13 GB)
and CTCF ChIP-seq (ENCFF269OPL) from public sources, fits dLEM on full
chr10 at 10 kb, and produces three figures:
full_analysis_chr10.png — chromosome-wide L/R/CTCF/barrier tracks
full_analysis_locus.png — split contact map at chr10:20.5–22.5 Mb
with CTCF peak marks below the map
full_analysis_scatter.png — cohesin activity vs CTCF signal scatter
Downloads are skipped on re-run if files already exist.
- .gitignore: exclude docs/data/*.mcool and docs/data/*.bed.gz
(too large to commit; generated by the notebook)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
pip install dlem-jax installs CPU-only jax by default (works everywhere). pip install "dlem-jax[cuda]" adds jax[cuda12] for NVIDIA GPU support. README now shows both install commands clearly. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- dlem/__init__.py: on import, detect NVIDIA GPU + CPU jax and print an actionable message with the exact reinstall command; silent when GPU is already in use or no GPU is present - docs/quick_start.py + quick_start.ipynb: print "Loading and processing Hi-C data..." before fetch_band so users know the pause is expected (adaptive coarse-graining is slow on CPU) - docs/full_analysis.ipynb: same progress message, with extra note that full chr10 may take a few minutes on CPU Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Switch matrix fetching from hictkpy to cooler (already a project dep): - Pre-create bal_mat/raw_mat handles outside the loop (one cooler.matrix() call instead of two hictkpy fetches per block iteration) - Replace jnp.diag(sub, k) with np.diagonal(sub, k) to eliminate JAX dispatch overhead inside the width-iteration inner loop Result: example_chr10.cool fetch drops from ~90s to ~1s (bit-identical output, max abs diff = 0). mcool and full-chr10 paths also improve. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Switch CTCF data source from narrowPeak to bigWig (ENCFF269OPL) via pyBigWig one-call binning (type='max', matches paper Table 2) - Enable %matplotlib inline for interactive figure display - Remove redundant Hi-C progress print (fetch_band is fast after fix) - Fix ctcf_signal .values AttributeError (numpy array, not pandas Series) - Fix CTCF panel: fill_between continuous signal, height_ratios=[1,5,1] - Add docs/data/*.bigWig to .gitignore - Correct notebook overview markdown: bigWig format, ~1.7 GB size Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description:
Summary
docs/quick_start.ipynb/docs/quick_start.py: self-contained example that fits dLEM to a bundled H1-hESC Hi-C excerpt (chr10:19–26 Mb) and reproduces the reference locussplit contact map (chr10:20.5–22.5 Mb). Runnable headlessly via
python docs/quick_start.pyorjupyter execute docs/quick_start.ipynb.docs/full_analysis.ipynb: end-to-end notebook that downloads 4DNFI9GMP2J8 Micro-C (~13 GB) and ENCFF269OPL CTCF ChIP-seq bigWig (~1.7 GB), fits dLEM on full chr10, andcompares inferred cohesin speed tracks (L/R) to CTCF signal. CTCF binned via pyBigWig
type='max', matching Table 2 of the paper.dlem/api.py: fixed ~100× slowdown infetch_bandon single-resolution.coolfiles — now reads diagonals directly viacooler+np.diagonalinstead of routing throughhictkpy.dlem/__init__.py: GPU availability hint on import.pyproject.toml: addedseaborndependency;[cuda]optional extra forjax[cuda12].README.md: GPU install instructions and quick-start usage..gitignore: exclude large downloaded data files (*.mcool,*.bigWig,*.bed.gz) fromdocs/data/.Test plan
pip install -e .in a clean environment — all deps resolve,import dlemsucceedspython docs/quick_start.pyruns to completion, producesdocs/quick_start_patch.pngjupyter execute docs/quick_start.ipynbproduces the same PNGfull_analysis.ipynbruns end-to-end after data downloads; overview and locus figures save correctly🤖 Generated with Claude Code