I/O library for Illumina MethylationEPIC v2 (EPICv2) beta arrays.
EpicV2IO loads beta files, retains cg probes, optionally applies QC exclusions from the shipped Peters et al. probe annotation manifest, and provides helpers for probe-ID parsing and file summaries. It is designed as a lightweight dependency for analysis packages (for example betaSieve) and for ML or custom workflows that need clean EPICv2 matrices.
- Load EPICv2 beta matrices (
.csv= comma-delimited,.txt= tab-delimited) - Ship a compact Peters et al. EPICv2 manifest as Parquet
- Optional QC exclusions via manifest flags (mismatch / missing position, cross-hybridisation, sex / mitochondrial chromosomes)
- Parse EPICv2
cg…_…probe identifiers into site, design, and replicate components - Summarise probe-type and chromosome composition of a betas file
pip install epicv2iopip install git+https://github.com/MarStreicher/EpicV2IO.gitconda env create -f environment.yml
conda activate env_epicv2io- Python ≥ 3.9
- pandas ≥ 1.5
- pyarrow ≥ 10
- openpyxl
Dependencies are installed automatically with pip.
EpicV2IO ships a compact Parquet derived subset of the extended EPICv2 probe manifest from Peters et al. (2024). Only the columns required for QC filtering and annotation are retained (see table below). This is an adaptation of third-party open materials, not original EpicV2IO content.
| Column | Role |
|---|---|
IlmnID |
Probe identifier (join key with betas index) |
CHR |
Chromosome (chr1 … chr22, chrX, chrY, chrM, …) |
MismatchPos |
Illumina vs SeSAMe location discrepancy flag |
MissingPos |
Missing / unresolved genomic position flag |
CH_BLAT |
Cross-hybridisation evidence (BLAT) |
CH_WGBS_evidence |
Cross-hybridisation evidence (WGBS) |
Provenance
- Paper (CC BY 4.0): Peters et al., BMC Genomics (2024) 25:251 —
https://doi.org/10.1186/s12864-024-10027-5 - Supplementary CSV: Additional file 4 of that article
- Bioconductor:
EPICv2manifest/ AnnotationHub AH116484 (Artistic-2.0)
The rebuild script is scripts/build_peters_manifest_parquet.R. Full attribution
text is in NOTICE and src/epicv2io/resources/README.md.
Provide a small, versioned annotation table that travels with the package so loaders and downstream tools do not need a separate Bioconductor / AnnotationHub download at runtime.
from epicv2io.manifest import load_peters_manifest, MANIFEST_COLUMNS
manifest = load_peters_manifest()
print(manifest.shape)
print(list(manifest.columns))
print(manifest.head(3))Example result:
(937690, 6)
['IlmnID', 'CHR', 'MismatchPos', 'MissingPos', 'CH_BLAT', 'CH_WGBS_evidence']
IlmnID CHR MismatchPos MissingPos CH_BLAT CH_WGBS_evidence
0 cg00000029_TC11 chr16 N N N N
1 cg00000109_TC21 chr3 N N N N
2 cg00000155_BC11 chr12 N N N N
Subset columns if you only need chromosome mapping:
chr_map = load_peters_manifest(columns=["IlmnID", "CHR"])Read a beta-value matrix, keep cg probes only, drop annotation / control columns, and optionally remove probes that match enabled Peters QC flags. Returns either a probes × samples frame or a samples × probes (wide) frame for ML-style workflows.
Expected input
- Rows: probe IDs (
IlmnID), including at least some IDs starting withcg - Columns: samples (plus optional annotation columns such as
CHR, which are dropped) - Values: beta values in [0, 1]
- Delimiter: from file extension (
.csv→ comma,.txt→ tab)
from pathlib import Path
from epicv2io import BetasLoader
loader = BetasLoader(Path("betas.csv"))
# probes × samples (cg only; no QC flags)
betas = loader.load_data()
print(betas.shape)
print(betas.iloc[:3, :3])
# samples × probes
betas_wide = loader.load_data_wide(
exclude_mismatch_pos=True,
exclude_missing_pos=True,
exclude_ch_blat=True,
exclude_ch_wgbs_evidence=True,
exclude_sex_chromosomes=True,
exclude_mitochondrial=True,
)
print(betas_wide.shape)
print(betas_wide.iloc[:2, :3])Console summary printed during a filtered load (counts are dataset-dependent):
Candidate cg CpGs (betas ∩ manifest, cg rows): 850,000
exclude_mismatch_pos: 1,234 CpGs match this rule (among candidates)
exclude_missing_pos: 567 CpGs match this rule (among candidates)
exclude_ch_blat: 8,901 CpGs match this rule (among candidates)
exclude_ch_wgbs_evidence: 2,345 CpGs match this rule (among candidates)
exclude_sex_chromosomes: 19,000 CpGs match this rule (among candidates)
exclude_mitochondrial: 12 CpGs match this rule (among candidates)
Union (unique CpGs removed by any enabled rule): 28,500
Remaining after union: 821,500
(821500, 48)
Example matrix result (load_data):
Sample_A Sample_B Sample_C
IlmnID
cg00000029_TC11 0.12 0.15 0.11
cg00000109_TC21 0.88 0.91 0.87
cg00000155_BC11 0.45 0.44 0.48
Example wide result (load_data_wide):
IlmnID cg00000029_TC11 cg00000109_TC21 cg00000155_BC11
sample_id
Sample_A 0.12 0.88 0.45
Sample_B 0.15 0.91 0.44
After loading, exclusion statistics are available on the loader:
print(loader.load_candidate_cg_count)
print(loader.load_exclusion_counts_by_rule)
print(loader.load_exclusion_union_count)Manifest cg probes absent from the betas file:
missing = loader.get_manifest_cpgs_missing_in_betas()
print(len(missing), missing[:5])All default to False. Truthy manifest values are Y, TRUE, and 1 (case-insensitive).
| Argument | Manifest column / rule |
|---|---|
exclude_mismatch_pos |
MismatchPos |
exclude_missing_pos |
MissingPos |
exclude_ch_blat |
CH_BLAT |
exclude_ch_wgbs_evidence |
CH_WGBS_evidence |
exclude_sex_chromosomes |
CHR in chrX / chrY |
exclude_mitochondrial |
CHR == chrM |
When any exclusion flag is enabled, at least one probe ID must overlap the shipped manifest (IlmnID); otherwise a ValueError is raised.
Parse EPICv2 cg probe names of the form cgDDDD_XXXX into components used for duplicate grouping (same site / same design / replicate suffix).
Pattern: ^cg(\d+)_([TB][CO][12])(\d+)$
| Component | Meaning | Example (cg00000029_TC11) |
|---|---|---|
site_id |
CpG site | cg00000029 |
design_type |
Design block[TB][CO][12] |
TC1 |
design_id |
Site + design (exact-duplicate group key) | cg00000029_TC1 |
replicate_id |
Numeric replicate suffix | 1 |
from epicv2io import CgProbeId
probe = "cg00000029_TC11"
print(CgProbeId.parse(probe))
print(CgProbeId.site_id(probe))
print(CgProbeId.design_id(probe))
print(CgProbeId.design_type(probe))
print(CgProbeId.replicate_id(probe))Example result:
('cg00000029', 'cg00000029_TC1', 'TC1', '1')
cg00000029
cg00000029_TC1
TC1
1
Non-matching IDs return None:
print(CgProbeId.parse("rs12345678")) # NoneQuick inventory of a betas file: sample/column counts, probe-type mix from ID prefixes (cg, ch, rs, nv, control), and chromosome distribution via the Peters manifest. Useful before a full load.
Delimiter handling matches BetasLoader: .csv files are comma-delimited and .txt files are tab-delimited (epicv2io.io.infer_separator).
from pathlib import Path
from epicv2io import BetasAnalyzer
BetasAnalyzer(Path("betas.csv")).summarise()Example result:
Betas file: betas.txt
Header (first 10 columns): ['IlmnID', 'Sample_A', 'Sample_B', ...]...
Number of columns (samples + index): 49
Number of sample columns: 48
Total probe rows (index size): 937,055
Probe type counts (from index prefix):
----------------------------------------
cg 850,000 (90.71%)
ch 50,000 ( 5.34%)
rs 20,000 ( 2.13%)
nv 10,000 ( 1.07%)
control 5,000 ( 0.53%)
other 2,055 ( 0.22%)
----------------------------------------
total 937,055 (100.00%)
Probe chr counts (from manifest):
----------------------------------------
chr1 75,000 ( 8.00%)
...
undefined 12,000 ( 1.28%)
----------------------------------------
total 937,055 (100.00%)
Distribution of autosomal vs sex chromosomes (from manifest):
----------------------------------------
sex 25,000 ( 2.67%)
autosomal 912,055 (97.33%)
----------------------------------------
total 937,055 (100.00%)
If you use EpicV2IO in published work, please cite:
Streicher M. EpicV2IO: I/O library for reading Illumina MethylationEPIC v2 BeadChip array files. GitHub repository. 2026.
If you use the shipped probe annotation (or any analysis that relies on it), please cite the Peters et al. EPICv2 manifest paper (CC BY 4.0):
Peters TJ, Meyer B, Ryan L, Achinger-Kawecka J, Song J, Campbell EM, Qu W, Nair S, Loi-Luu P, Stricker P, Lim E, Stirzaker C, Clark SJ, Pidsley R. Characterisation and reproducibility of the HumanMethylationEPIC v2.0 BeadChip for DNA methylation profiling. BMC Genomics. 2024;25:251. https://doi.org/10.1186/s12864-024-10027-5
Optionally also acknowledge the Bioconductor distribution used to build the Parquet subset:
Peters TJ, Pidsley R. EPICv2manifest: Illumina Infinium MethylationEPIC v2.0 extended manifest. Bioconductor annotation package (AnnotationHub AH116484).
BSD 3-Clause License (see LICENSE) covers this software only.
The packaged Peters et al. manifest subset is third-party open material under the article’s CC BY 4.0 terms (and, for the AnnotationHub build path, Bioconductor Artistic-2.0). Attribution requirements for that data are described in NOTICE; they are separate from the EpicV2IO software licence.