Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,9 +7,12 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

## [Unreleased]

## [0.16.0] - 2026-08-05

### Added

- Support for filtering using smaller static fuse filter indexes (`.pidx`). These can be generated from standard Deacon indexes using `deacon index freeze`. Fuse filters reduce index size from ~64 bits/k-mer to ~36 bits/k-mer using default 32 bit fingerprints, or to ~18 bits/k-mer using 16 bit fingerprints. Default 32 bit fingerprints deliver a negligible false positive rate of ~2<sup>-32</sup>.
- `deacon filter` accepts CBQ (BINSEQ) input, detected automatically, and writes CBQ when the output path ends in `.cbq`; paired reads are stored as native paired records in a single file.
- Browser-based streaming FASTA/FASTQ filtering with `deacon-wasm`.
- `deacon index filter` keeps or discards index minimizers based on a complexity threshold (`-c`), with a choice of `--algorithm` (`-a`).
- `deacon filter --complexity-threshold` (`-c`) ignores minimizers below a kdust complexity threshold at filter time.
Expand Down
88 changes: 80 additions & 8 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

5 changes: 4 additions & 1 deletion Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,8 @@ rand = { version = "0.9", optional = true }
serde_json = { version = "1.0", optional = true }
niffler = { version = "3.0.1", default-features = false, optional = true }
paraseq = { version = "0.4.3", default-features = false, features = ["anyhow", "niffler"], optional = true }
# Binary sequence format (CBQ) support. Defaults disabled: binseq's own paraseq integration is not needed.
binseq = { version = "0.9.4", default-features = false, features = ["anyhow"], optional = true }
ensure_simd = { version = "0.1.0", optional = true }

# Compression deps (optional)
Expand All @@ -60,10 +62,11 @@ predicates = "3.0"
tempfile = "3.20"
rstest = "0.26"
nix = { version = "0.31", features = ["fs"] }
binseq = { version = "0.9.4", default-features = false, features = ["anyhow"] }

[features]
scalar = ["simd-minimizers/scalar", "packed-seq/scalar"]
cli = ["rayon", "parking_lot", "indicatif", "paraseq", "ensure_simd", "clap", "rand", "niffler", "serde_json"]
cli = ["rayon", "parking_lot", "indicatif", "paraseq", "binseq", "ensure_simd", "clap", "rand", "niffler", "serde_json"]
# Disable for faster builds.
compression = ["zstd", "liblzma", "flate2", "gzp", "paraseq/default"]
# Use to still handle .gz files when "compression" is not enabled.
Expand Down
6 changes: 5 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -89,7 +89,7 @@ Prebuilt pangenome indexes are provided. These can be downloaded using the links

### Filtering

The main command `deacon filter` accepts an index path followed by up to two FASTA/FASTQ file paths, depending on whether input sequences originate from stdin, a single file, or paired input files. Indexes are built with `deacon index build`. Paired inputs are supported as either two separate or one interleaved file/stream when using `--interleaved`, and may be written either to separate paired output files or one interleaved file. For paired sequences, distinct minimizer hits originating from either mate are counted. By default, input sequences must meet both an absolute threshold of 2 minimizer hits (`-a 2`) and a relative threshold of 1% of minimizers (`-r 0.01`) to pass the filter. Filtering can be inverted for e.g. host depletion using the `--deplete` (`-d`) flag. Gzip, Zstandard, and xz compression formats are detected automatically by file extension. Paired read headers can be validated using `--check-pairs`.
The main command `deacon filter` accepts an index path followed by up to two FASTA/FASTQ file paths, depending on whether input sequences originate from stdin, a single file, or paired input files. Indexes are built with `deacon index build`. Paired inputs are supported as either two separate or one interleaved file/stream when using `--interleaved`, and may be written either to separate paired output files or one interleaved file. For paired sequences, distinct minimizer hits originating from either mate are counted. By default, input sequences must meet both an absolute threshold of 2 minimizer hits (`-a 2`) and a relative threshold of 1% of minimizers (`-r 0.01`) to pass the filter. Filtering can be inverted for e.g. host depletion using the `--deplete` (`-d`) flag. Gzip, Zstandard, and xz compression formats are detected automatically by file extension. Paired read headers can be validated using `--check-pairs`. CBQ files (the columnar binary format of the [BINSEQ](https://www.biorxiv.org/content/10.1101/2025.04.08.647863v2) family) are detected on input by magic bytes, and written when the output path ends in `.cbq`; CBQ pairing is native, so paired reads stay in one file (do not pass `--output2`).

#### Examples

Expand Down Expand Up @@ -131,6 +131,10 @@ zcat r12.fq.gz | deacon filter -d panhuman-1.k31w15.idx - - > filt12.fq
# Save summary JSON
deacon filter -d panhuman-1.k31w15.idx reads.fq.gz -o filt.fq.gz -s summary.json

# CBQ (BINSEQ) input/output; paired reads stay in one CBQ file
deacon filter panhuman-1.k31w15.idx reads.fq.gz -o filt.cbq
deacon filter panhuman-1.k31w15.idx filt.cbq -o filt.fq.gz

# Replace read headers with incrementing integers
deacon filter -d -R panhuman-1.k31w15.idx reads.fq.gz > filt.fq

Expand Down
2 changes: 1 addition & 1 deletion deacon-py/Cargo.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[package]
name = "deacon-py"
version = "0.16.0-rc4"
version = "0.16.0"
edition = "2024"
license = "MIT"
publish = false
Expand Down
2 changes: 1 addition & 1 deletion deacon-py/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -157,7 +157,7 @@ impl Index {
let mins = Arc::clone(&self.minimizers);
let (k, w) = (self.k, self.w);
let summary = py
.detach(|| run_with_index(&mins, &IndexHeader::new(k, w), &cfg))
.detach(|| run_with_index(mins, &IndexHeader::new(k, w), &cfg))
.map_err(to_pyerr)?;
Ok(pythonize::pythonize(py, &summary)?.unbind())
}
Expand Down
Loading