Home

fqtrim
trimming&filtering of next-gen reads

Overview

fqtrim is a versatile stand-alone utility that can be used to trim adapters, poly-A tails, terminal unknown bases (Ns) and low quality 3' regions in reads from high-throughput next-generation sequencing machines. The program allows for inexact matching of adapters and poly-A sequences (thus accounting for mismatches and indels due to sequencing errors). This utility can also apply a low-complexity ("dust") filter to the reads, or count and collapse duplicate reads which can be particularly useful for micro-RNA analysis pipelines.
fqtrim can be used as a pre-processing or filtering step for next-generation sequence analysis pipelines (e.g. mapping, assembly) or as a post-processing utility for the analysis and potential recovery of unmapped reads or singletons resulting from such a pipeline.

Obtaining and installing fqtrim

The source archive can be downloaded here: fqtrim-0.9.7.tar.gz
In order to build the fqtrim program from the source package, just unpack and run the 'make release' command:

tar xvfz fqtrim-N.NN.tar.gz
cd fqtrim-N.NN
make release

A pre-built Linux x86_64 binary package: fqtrim-0.9.7.Linux_x86_64.tar.gz
Simply unpack this archive and copy the fqtrim executable in a directory of your choice.

Licensing and contact Information

fqtrim is free, open source software released under an Artistic License.
You can contact us about fqtrim at: gpertea@jhu.edu

DOI

Usage

The program can take as input read sequence data in FASTA or FASTQ format (compressed or streamed at stdin) and can process paired-end reads in a consistent manner (i.e. not breaking the pairs and producing two distinct output files with the paired reads, optionally compressed). The basic usage template is:

fqtrim [<options>] <input_file(s)>..

Input files can also be compressed FASTA or FASTQ files - but only the basic Linux compression extensions are recognized: gz and bz2. Options and input files can be provided in mixed order (options always start with the dash ('-') character followed by an alphanumeric character). When paired-reads should be provided as input (two separate files) and kept together, the two file names should be only separated by a comma or a colon character (no spaces, so the two file names appear as one argument to the program).
Unless the -o option is provided (see below), the trimmed/processed reads are printed at stdout. The special input file name '-' (single dash, without quotes) will direct fqtrim to process a stream of FASTA or FASTQ formatted records from stdin. The main options are explained below.

Option	Description
-o <outsuffix>	Write the trimmed/filtered reads to file(s) named <input>.<outsuffix> in the current working directory. The suffix should include the file extension. If the extension is .gz, .gzip, or .bz2, the output is compressed automatically. Note: If the input file is - (reads are streamed from stdin), this option specifies the complete output filename, not just the suffix.
--outdir <outdir>	When used with -o, write output file(s) to <outdir> instead of the current directory.
-l <minlen>	Minimum read length after trimming. Reads shorter than this threshold, either before or after trimming, are discarded. Default: 16.
-5 <DNAseq>	Trim the specified adapter/primer sequence from the 5' end of each read (for example: -5 CGACAGGTTCAGAGTTCTACAGTCCGACGATC). Only one -5 option may be specified.
-3 <DNAseq>	Trim the specified adapter/primer sequence from the 3' end of each read (for example: -3 TCGTATGCCGTCTTCTGCTTG). Only one -3 option may be specified.
-f <filename>	Read multiple adapter sequences from a file instead of using -5 and -3. Each line contains a 5' adapter and/or a 3' adapter separated by a tab, space, comma, colon, or semicolon (\t, space, ,, :, or ;). Empty fields are allowed. Example: CGACAGGTTCAGAGTTCTACAGTCCGACGATC,TCGTATGCCGTCTTCTGCTTG This is equivalent to: CGACAGGTTCAGAGTTCTACAGTCCGACGATC, ,TCGTATGCCGTCTTCTGCTTG If a line contains no delimiter, the sequence is searched at both the 5' and 3' ends. To specify only a 3' adapter: ,TCGTATGCCGTCTTCTGCTTG
-a <minmatch>	Minimum suffix-prefix overlap between a read and an adapter required for trimming. Default: 6. The default is intentionally permissive and may result in over-trimming due to short accidental matches. It is primarily useful for post-processing previously rejected reads (for example, unmapped or singleton reads).
-A	Disable automatic poly-A/poly-T trimming. By default, poly-A is trimmed from the 3' end and poly-T from the 5' end of each read. This behavior originates from transcriptome (EST) processing and may be undesirable for genomic or RNA-Seq data.
-y <minpolyLen>	Minimum poly-A/poly-T run length to trim. Default: 6. Increasing this value reduces false positives.
-q <minqv> [-w <winsize>] [-t <maxtrim>]	Enable 3' quality trimming using a sliding window. Trimming begins when the average quality falls below <minqv>. The quality threshold is independent of whether the input uses Phred-33 or Phred-64 encoding. -w specifies the sliding window size (default: 6). -t limits the maximum number of bases removed from the 3' end.
-m <maxpercN>	Maximum percentage of N bases allowed after trimming. Default: 5. Terminal Ns are removed automatically, and reads exceeding this percentage afterward are discarded.
-n <prefix>	Rename reads using <prefix> followed by a read counter. If -C is also used, the suffix _x<N> is appended, where <N> is the duplicate count.
-r <report.txt>	Write a trimming report listing all modified reads and discarded reads. The report contains three columns: 1. Read name 2. Comma-separated trimming operations 3. A one-letter discard ("trash") code, if applicable Operation format: - 5 or 3 = trimmed end - Q = quality trimming - N = N trimming - A = poly-A trimming - T = poly-T trimming - V (or a, b, c, ...) = adapter/vector trimming. Lowercase letters identify individual adapters when using -f together with --aidx. - Final number = number of bases removed
-s1 or -s2	For paired-end reads, disable trimming of read 1 or read 2 while still discarding the pair if the other read fails. Intended primarily for single-cell sequencing, where one read contains only barcode information.
-T	Append the number of bases trimmed from the 5' and 3' ends to each FASTA/FASTQ header in the output.
-D	Apply a low-complexity (DUST) filter and discard reads with more than 50% low-complexity sequence.
-C	Collapse duplicate reads and append _x<N> to the read name, where <N> is the number of identical reads. Because all read sequences are stored in memory, this option is recommended only for relatively small datasets (for example, microRNA experiments).
-p <numcpus>	Use <numcpus> CPU threads to accelerate processing, particularly when multiple adapters are provided. This option is currently incompatible with -C.
-Q	Convert quality values between Phred-33 and Phred-64. fqtrim automatically detects the input encoding and converts it to the alternate representation.
-M	Disable paired-read name consistency checking. Normally, fqtrim verifies that paired-end reads follow the expected naming convention, but some datasets use different formats.

Common usage example

Cleaning up noisy exome data (paired reads) with Ns in the read sequence, allowing a minimum length of 25 bases for trimmed reads and maintaining the pairing of the reads:

fqtrim -A -l25 -o trimmed.fq.gz exome_reads_1.fastq.gz,exome_reads_2.fastq.gz 

Note that for non-transcriptomic reads the -A option is advised. In this example, the output of fqtrim will be written in two compressed files with the suffix ".trimmed.fq.gz".