Skip to content

Port cascade bug fixes from Astra sandbox #4

Description

@jwestrob

Summary

Four critical bugs found and fixed in ~/astra/astra/search.py (the local Astra copy that Sharur uses) while running KOFAM --cascade on the DPANN dataset (3,564 genomes). KOFAM crashed at 187/3564 genomes due to compounding memory/performance issues. These fixes need to be ported to Aksha.

Bugs Fixed

1. HMM grouping inside per-genome loop (performance → OOM)

  • Function: hmmsearch() in search.py
  • Problem: Cascade groups all ~26,833 KOFAM HMMs by best available threshold — but this was recomputed for every genome. 26,833 × 3,564 = 95.6M wasted iterations + memory fragmentation.
  • Fix: Move grouping + per-group kwargs construction above the per-genome loop.

2. Missing e-value fallback for cascade mode (correctness)

  • Problem: HMMs without GA/TC/NC thresholds got pyhmmer's default E=10.0 instead of the intended 1e-15 fallback.
  • Fix: kwargs.setdefault('E', 1e-15) when cascade is active and no bitscore threshold exists.

3. Invalid bit_cutoffs='cascade' in define_kwargs (dead code)

  • Problem: define_kwargs() set kwargs['bit_cutoffs'] = 'cascade', not a valid pyhmmer value. Never reached pyhmmer (overwritten downstream) but was confusing.
  • Fix: Removed. define_kwargs() now only sets preferred_cutoff in cascade mode.

4. combine_results loads all DataFrames into memory (OOM)

  • Problem: Loaded every per-genome TSV into pandas then pd.concat() — O(N) memory.
  • Fix: Stream line-by-line with constant memory.

Vault Reference

agent-captures/2026-03-08_aksha-cascade-bug-fixes.md

Checklist

  • Port all four fixes to aksha/search.py
  • Add test case for cascade mode on multi-genome directory
  • Migrate Sharur from Astra to Aksha (pip install -e ~/aksha)

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions