Skip to content

BREAKING CHANGE: v0.16 -> CBQ input/output implementation - #95

Open
alejandrogzi wants to merge 2 commits into
bede:mainfrom
alejandrogzi:v0.0.16-cbq-input
Open

BREAKING CHANGE: v0.16 -> CBQ input/output implementation#95
alejandrogzi wants to merge 2 commits into
bede:mainfrom
alejandrogzi:v0.0.16-cbq-input

Conversation

@alejandrogzi

@alejandrogzi alejandrogzi commented Aug 6, 2026

Copy link
Copy Markdown

This PR includes the necessary implemenation fordeacon filter to accept CBQ (BINSEQ) input, detected automatically, and writes CBQ when the output path ends in .cbq. Benchmark output is consistent between any format combination.

Main change against #31 is only CBQ support, dropping VBQ/BQ.

Benchmark: FASTQ vs CBQ, deacon filter 0.16.0

Input: 500k real viral reads (~4 kb each, 1.9 GB) from rsviruses17900 (Zenodo), filtered against an index built from the 17,900 virus contigs (rsviruses17900.fa.gz); keep-all thresholds (-a 1 -r 0), -t 8, 16-core box; hyperfine min/median of 3 + one /usr/bin/time -v pass for peak RSS.

arm min (s) median (s) peak RSS (MB)
fq->fq 2.95 3.05 662
fq->fq.gz 4.02 4.06 681
fq->cbq 2.86 2.91 674
fq.gz->fq.gz 4.26 4.32 687
fq.gz->cbq 3.08 3.09 671
cbq->fq 2.98 3.01 813
cbq->fq.gz 4.01 4.03 830
cbq->cbq 2.98 3.06 840
fq(r1+r2)->fq 3.27 3.46 663
fq(r1+r2)->cbq 3.05 3.16 675
cbq->fq (paired) 3.31 3.36 814
cbq->cbq (paired) 2.96 2.98 841

Findings

  • CBQ output is free: fq->cbq (2.86 s) beats plain fq->fq (2.95 s) + fq->fq.gz (4.02 s); gz output is the slowest arm by ~40%.
  • CBQ beats gzip on size: 197.5 MB vs 330.1 MB (40% smaller) on this data; 10× smaller than plain fq.
  • CBQ input costs ~150 MB RSS (mmap + embedded index: 662→813 MB); negligible on speed.
  • gz input adds ~0.2 s regardless of output format. CBQ roundtrips preserve counts exactly (seqs_out identical within each input group).

*Paired reads were derived by half-splitting real reads.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant