Skip to content

OOM error when calculating distance matrix #97

@PatPotsdam

Description

@PatPotsdam

Dear astral-Team,

Thanks for this great software package.

Describe the bug
I am trying to run a moderately large dataset with ~1500 gene trees and ~140 species using the astral-MP. Surprisingly, I am not able to run it on my university's HPC cluster. I run out of memory even with 60 threads and the option -Xmx1300g (the maximum possible on my cluster). I thought this option would set a memory limit for astral and it just runs longer but actually it stops with OOM error when calculating distance matrix.

To Reproduce
java -Xmx1300g -D"java.library.path=/path/to/Astral.5.15.5/lib/" -jar /path/to/Astral.5.15.5/astral.5.15.5.jar -T 60 -i genes.treefile -o genes_astral.tre -t 2 2>genes_astral.log

Log file
last 4 lines of the log before it stopped:
...
Using polytree-based weight calculation.
Number of gene trees: 15079
15079 trees have missing taxa
Calculating distance matrix (for completion of X) ....with 4 distance matrices for parallellism

slurm error file:
slurmstepd: error: Detected 1 oom_kill event in StepId=702545.batch. Some of the step tasks have been OOM Killed.

** Version
5.15.5

Thanks in advance and all the best
Patrick

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions