TargetGene

Pointer-style Top-DEG target prioritization for perturbation transcriptomics.

The model receives one Top-K DEG candidate set and returns logits over the same K candidates. It does not classify against a global all-gene vocabulary.

Data Policy

The initial benchmark target is Replogle et al. 2022 K562 Essential Perturb-seq. DEG views are generated offline by repeated downsampling after a target-level split. All views from the same target_gene_id stay in exactly one split.

Control detection is strict: for K562 Essential use obs["gene"] == "non-targeting" as the control pool. Do not use substring matching such as nt, because it can incorrectly classify real target genes like INTS2 or TRNT1 as controls. Targets are filtered by min_cells_per_group before target selection and split, so a requested max_targets count should correspond to generated targets.

Default processed files:

data/processed/gene_vocab.json
data/processed/geneformer_embeddings.npy
data/processed/train_views.npz
data/processed/val_views.npz
data/processed/test_views.npz

Each .npz contains candidate IDs, DEG statistics, TI, label position, target ID, and target_in_topk. Missing neg_log10_fdr is computed at load time.

Environment

Use the requested conda environment:

conda activate targetgene

Install optional dependencies for full data download/config support:

pip install -r requirements.txt

The current core tests use numpy, scipy, torch, anndata, scanpy, and pydeseq2.

Workflow

Inspect shared data:

conda run -n targetgene python scripts/inspect_anndata.py /NFS_DATA/Perturb-seq/replogle2022/K562_[E]/zenodo/ReplogleWeissman2022_K562_essential.h5ad

The default configs point to the shared NFS H5AD:

/NFS_DATA/Perturb-seq/replogle2022/K562_[E]/zenodo/ReplogleWeissman2022_K562_essential.h5ad

If a local copy is needed, the downloader still uses the K562 Essential Zenodo H5AD URL by default:

conda run -n targetgene python scripts/download_data.py \
  --output data/raw/k562_essential.h5ad \
  --url <DIRECT_H5AD_URL>

Generate DEG-like perturbation rankings with the maintained no-TI track:

conda run -n targetgene python scripts/analyze_full_normal_deg.py \
  --config configs/full_normal_deg_geneformer_scanpy_wilcoxon.yaml

This path uses Scanpy rank_genes_groups(method="wilcoxon"), which is a Wilcoxon rank-sum test for independent groups, not a paired Wilcoxon signed-rank test. The output should be interpreted as a DEG-like perturbation signature/ranking rather than replicate-aware formal DEG discovery.

The TI-style batch-level pyDESeq2 backend can coexist with the Wilcoxon path by switching only --deg-method or the config file. It sums raw counts per batch for target and non-targeting control cells, fits ~batch + condition, and writes TRADE-compatible global features:

conda run -n targetgene python scripts/analyze_full_normal_deg.py \
  --config configs/full_normal_deg_geneformer_ti_style_batch_pydeseq2.yaml \
  --deg-method ti_style_batch_pydeseq2 \
  --batch-column batch

For timestamped reruns, override the output directory explicitly:

conda run -n targetgene python scripts/analyze_full_normal_deg.py \
  --config configs/full_normal_deg_geneformer_scanpy_wilcoxon.yaml \
  --output-dir /NFS_DATA/Perturb-seq/replogle2022/K562_[E]/targetgene_scanpy_wilcoxon_<TIMESTAMP>

Build Geneformer embeddings aligned to gene_vocab.json:

conda run -n targetgene python scripts/build_geneformer_embeddings.py \
  --gene-vocab data/processed/<TRACK>/gene_vocab.json \
  --h5ad-path /NFS_DATA/Perturb-seq/replogle2022/K562_[E]/zenodo/ReplogleWeissman2022_K562_essential.h5ad \
  --output data/processed/<TRACK>/geneformer_embeddings.npy \
  --report reports/<TIMESTAMP>_geneformer_embedding_report.json \
  --model ctheodoris/Geneformer \
  --model-subfolder Geneformer-V2-104M \
  --token-dictionary-filename geneformer/token_dictionary_gc104M.pkl

Create train/val/test multiview shards from a prepared manifest:

conda run -n targetgene python scripts/make_final_multiview_dataset.py \
  --filtered-h5ad data/interim/<FILTERED>.h5ad \
  --manifest data/processed/<TRACK>/dataset_manifest.json \
  --output data/processed/<TRACK>/train_views.npz \
  --split train \
  --deg-method scanpy_wilcoxon

Run coverage and baselines:

conda run -n targetgene python scripts/compute_coverage.py data/processed/<TRACK>/test_views.npz
conda run -n targetgene python scripts/run_baselines.py data/processed/<TRACK>/test_views.npz --baseline rank1_deg

Train and evaluate:

conda run -n targetgene python scripts/train_model.py --config <MODEL_CONFIG>
conda run -n targetgene python scripts/evaluate_model.py --config <MODEL_CONFIG> --checkpoint <CHECKPOINT>

Tests

This repository uses unittest, so pytest is optional:

conda run -n targetgene python -m unittest discover -s tests

Notes

geneformer_embeddings.npy is expected to be aligned with gene_vocab.json where index 0 is reserved for unknown/padding. The --make-random-embeddings flag creates a random table for smoke tests and ablations; replace it with Geneformer-derived embeddings for the main experiment.

Legacy one-off scripts and dated configs are kept under archive/ for record keeping and are not part of the maintained two-track workflow.