Skip to content

Fix spectrum, tree, IO, and plotting bugs; add calculate_mutrate - #17

Draft
kpotoh wants to merge 1 commit into
devfrom
cursor/fix-bugs-and-improvements-dfb3
Draft

Fix spectrum, tree, IO, and plotting bugs; add calculate_mutrate#17
kpotoh wants to merge 1 commit into
devfrom
cursor/fix-bugs-and-improvements-dfb3

Conversation

@kpotoh

@kpotoh kpotoh commented Aug 12, 2026

Copy link
Copy Markdown
Contributor

Fixes several correctness bugs found while exploring the library (issue #13) and adds calculate_mutrate. Version bump: 0.0.15 → 0.0.16. All 97 tests pass.

Spectrum calculation (calculate_mutspec / MutRate)

  • Keep RawMutSpec (obs/exp) separate from scaled MutSpec so scaling no longer overwrites the unscaled rates.
  • All-zero scale no longer yields NaNs.
  • Accept 12-component Mut values (C>T) as well as 192 (A[C>T]G).
  • New calculate_mutrate() for unscaled observed/expected rates (exported from the package).
  • collapse_mutspec accepts ObsNum/ExpNum as aliases of ObsFr/ExpFr and copies input to avoid SettingWithCopy.
  • complete_sbs192_columns fills missing columns in one concat (no fragmentation warning).
  • jackknife_spectra_sampling / calc_edgewise_spectra copy inputs instead of mutating caller frames.
  • nobs_cutoff added; misspelled nobs_cuttof kept as an alias.
  • filter_outlier_branches falls back to ProbaFull if ProbaMut is missing.

Codon annotation

  • Sequences whose length is not divisible by 3: incomplete last codon is skipped; warn via warnings.warn (CodonAnnotation had no logger — that was a crash).
  • _prepare_codontable now actually raises ValueError for invalid genetic-code arguments.
  • get_syn_codons default is set() not dict().
  • Empty mutation DataFrames get the expected columns.
  • MutSpecExtractor._derive_mutspec now calls branch.process_branch() (serial extraction was broken after the parallel refactor).
  • dump_table skips empty frames.

Tree

  • TreeNode.parent / .up and iter_edges().
  • Tree loads from a Newick string or a file path.
  • node_parent catches StopIteration rather than BaseException.
  • get_tree_height handles zero distances for geometric mean.
  • calc_phylocoefs handles tree_height <= 0.
  • get_ingroup_root no longer raises UnboundLocalError when both root children are leaves.

IO

  • GenomeStates reads the states file (was using a leftover gappy-sites path); path_to_gappy_sites=None works.
  • GenesStates passes states_fmt into read_alignment; parameterized SQL; groupby.size(); valid RuntimeError (illegal file= keyword removed).
  • read_genbank_ref uses gene_qualifier and ftr.location.strand (BioPython 1.88 has no SeqFeature.strand).

Plotting and scripts

  • plot_mutspec(show=False) with caller ax= no longer plt.close()s that figure (README multi-panel example was broken).
  • scripts/calculate_mutspec.py: filter_short_exp_seqs actually drops IQR outliers; pivot(index=..., columns=..., values=...) for pandas 3.
  • scripts/collect_mutations.py: empty dump handling; separate header flags for ms12 vs ms192 files.
  • scripts/collect_mutations_parallel.py: same Branch.process_branch fix as the library.
  • basic_logger() uses a named logger and does not attach duplicate handlers.

Tests

  • tests/test_mutspec_calc.py now asserts real RawMutSpec values (old lookups never matched).
  • New tests/test_bugfixes.py regression coverage.
  • tests/test_codon_ann.py: real extract_mutations_simple test.

Suggested follow-ups (not in this PR)

These remain from issue #13 / README TODOs:

  1. HGT annotation — new way of annotating mutations from HGT (no HGT source in this repo).
  2. HGT parallelization — integrate parallel mutation extraction from the HGT project.
  3. Split scripts — separate mutation collection vs annotation into distinct CLIs.
  4. Merge tree.py into phylo_tree.py — existing TODO; node_parent / get_tree_height still live in a leftover module.
  5. Seaborn boxplot vert deprecation — 204 MatplotlibDeprecationWarnings from seaborn internals vs matplotlib 3.11 (orientation instead of vert). Needs a seaborn upgrade or a local workaround in plot_mutspec.
  6. Broader renamesnobs_cutoff is the start; remaining names like nobs_cuttof alias, tree_heigth typos in git history, and script --rewrite_db TODOs could be cleaned up in a dedicated pass.
  7. Logging configbasic_logger still optionally reads a config file; inline the small config as noted in utils/logging.py.
Open in Web Open in Cursor 

Keep RawMutSpec separate from scaled MutSpec, accept 12-component Mut
values, and add calculate_mutrate. Fix CodonAnnotation (CDS length,
ValueError, logger crash), GenomeStates/GenesStates IO, MutSpecExtractor
serial extraction, plot_mutspec closing caller axes, and several pandas 3
compat issues. Add regression tests and bump to 0.0.16.

Co-authored-by: kpotoh <kpotoh@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants