[DimaMolod]

Builds HMMER 3.4 in both prediction images with the jackhmmer speed-up patches that AlphaFold 3 ships since google-deepmind/alphafold3@03f4f48. Snakemake runs feature generation in these images, so the speed-up reaches workflow users without changes elsewhere.

The patches

patch	what it does	upstream
jackhmmer_seq_limit.patch	--seq_limit, already in the AF3 image	AF3
easel_fast_fasta_parsing.patch	residues are copied while the read buffer is scanned, not in a second pass; reported 2-2.5x faster jackhmmer	EddyRivasLab/easel#83
jackhmmer_parallel_fasta.patch	on a rewindable FASTA each worker opens the database and claims 512 KB chunks, so one reader thread no longer limits jackhmmer; reported UniRef90 98 s -> 38 s at 8 CPUs, 94 s -> 18 s at 24	EddyRivasLab/hmmer#353

Both upstream issues are open with no maintainer response. The author's identity test was 3 queries against PDB seqres.

Changes

• docker/hmmer/: the three patches (Apache-2.0, copied from that commit; the seq_limit patch is byte-identical to the submodule's), SHA256SUMS, and build_hmmer.sh PREFIX. The script downloads HMMER 3.4, checks all checksums, applies the patches with --fuzz=0 in AF3's order, and installs. It then checks that serial (--cpu 0) and threaded (--cpu 4) jackhmmer give identical hits, alignment and target count on a generated ~6 MB database.
• AF3 image: HMMER is built by the script; it no longer reads alphafold3/docker/.
• AF2 image: HMMER is built in an ubuntu:20.04 stage and overwrites conda's binaries, like hhblits. The build fails if jackhmmer -h lacks --seq_limit.
• docs/backend_installation.md: how to put the same build into a conda env.

Not changed: AP still runs jackhmmer with 8 CPUs (JACKHMMER_N_CPU, AF2 default), and the Snakemake feature walltime is unchanged. Both should follow from measurements.

Tested so far

• build_hmmer.sh on an EMBL compute node (gcc 14.3): patches apply with --fuzz=0, build and self-check pass (21 s).
• Same script in ubuntu:20.04 with gcc 9 (apptainer, EMBL node): build and self-check pass.
• On the self-check database, stock 3.4 and the patched build agree byte-for-byte at --cpu 0 and --cpu 4.
• Identity check on the real AF2/AF3 databases (SLURM 63210385): 5 proteins (101-1210 aa) x {AF3 uniref90, mgnify, small BFD, uniprot; AF2 uniref90}, a tRNA against Rfam with nhmmer, and AF3's hmmsearch template search on pdb_seqres. Each image's current build at --cpu 8 vs the patched one at 8 and 16, with the exact AF2/AF3 flags. Alignments (-A Stockholm), target counts and hit tables are identical in all 57 comparisons. The tblout target-name column was padded differently in 2 of them (EGFR x small BFD). That is pre-existing: p7_tophits_GetMaxNameLength() reads unsrt[0..N-1], which after --seq_limit are the first hits found, and the seq_limit-only build alone gives width 87 at --cpu 4/8 and 71 at 16. Details: /scratch/dima/hmmer_speedup_test/identity/RESULT.txt.

Wall time on our storage (BeeGFS /scratch/AlphaFold_DBs, up to 7 jobs per node, so indicative). AF3 columns are the four protein searches added up; AF3 runs them concurrently.

query	length	AF3, current build 8 CPUs	patched 8	patched 16	AF2 uniref90 speed-up, 8 / 16
SUMO1	101	2074 s	537 s	287 s	x1.4 / x3.1
HBA	142	2564 s	565 s	299 s	x5.0 / x11.1
p53	393	3360 s	888 s	519 s	x1.6 / x3.1
HSP90A	732	3125 s	1406 s	936 s	x3.6 / x6.9
EGFR	1210	3624 s	2991 s	2128 s	x1.1 / x1.9

The gain shrinks with query length: long queries were already limited by the search itself, not by reading, so for them more CPUs is what helps. hmmsearch on pdb_seqres and nhmmer on Rfam take the same time before and after.

End-to-end

create_individual_features.py with the Snakemake defaults, in the 2.11.0 images, using either each image's HMMER or the patched build compiled inside that image's OS. Databases were on /scratch_cached. The figures are wall time of the whole feature job; current and patched ran side by side on the same nodes.

query	length	AF3, 8 CPUs	AF3, 16 CPUs	AF2
HBA	142	808 -> 332 s (x2.4)	629 -> 195 s (x3.2)	2420 -> 760 s (x3.2)
p53	393	1289 -> 816 s (x1.6)	852 -> 466 s (x1.8)	1798 -> 1798 s
HSP90A	732	1834 -> 1638 s (x1.1)	984 -> 878 s (x1.1)	3233 -> 3184 s
EGFR	1210	4332 -> 4141 s (x1.05)	2397 -> 2332 s (x1.03)	~9970 -> 9959 s

• AF3 features are identical for all four proteins; the metadata differs only in date, output dir and HMMER binary paths.
• AF2: the uniref90, mgnify, uniprot and template-hit files are byte-identical. For HBA and EGFR the final MSA still differed, because HHblits' bfd_uniref_hits.a3m changes from run to run with the same binary (two runs of the current image already differ). p53 and HSP90A features are identical.
• The gain is largest for short and medium proteins. Long ones are limited by the search itself, and AF2's time for them is mostly HHblits.
• Going from 8 to 16 CPUs speeds up the current build too (EGFR 4332 -> 2397 s), so raising Snakemake's feature_threads is worth doing on its own, separately from this PR.

Raw numbers: /scratch/dima/hmmer_speedup_test/e2e/FINAL.txt.

🤖 Generated with Claude Code

[chatgpt-codex-connector]

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-10-20T01:42:29.607456Z	5b5a34e	Draft marked ready

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.