Stats calculations not running on the GUI

[microscopycore-ismms]

Hi,

We tried to run the stats calculations and ran into the following issue (see below).

25-06-12 09:40:51: /home/oconnr03/miniconda/envs/ClearMapUi39/lib/python3.9/site-packages/scipy/stats/_axis_nan_policy.py:563: RuntimeWarning: Precision loss occurred in moment calculation due to catastrophic cancellation. This occurs when the data are nearly identical. Results may be unreliable. res = hypotest_fun_out(*samples, axis=axis, **kwds) 25-06-12 09:45:17: /home/oconnr03/miniconda/envs/ClearMapUi39/lib/python3.9/site-packages/ClearMap2-2.1.4-py3.9-linux-x86_64.egg/ClearMap/Alignment/utils.py:19: FutureWarning: The default dtype for empty Series will be 'object' instead of 'float64' in a future version. Specify a dtype explicitly to silence this warning. structs = pd.Series() 25-06-12 09:49:19: /home/oconnr03/miniconda/envs/ClearMapUi39/lib/python3.9/site-packages/scipy/stats/_axis_nan_policy.py:563: RuntimeWarning: Precision loss occurred in moment calculation due to catastrophic cancellation. This occurs when the data are nearly identical. Results may be unreliable. res = hypotest_fun_out(*samples, axis=axis, **kwds) 25-06-12 09:49:19:
Traceback (most recent call last):
File "/home/oconnr03/miniconda/envs/ClearMapUi39/lib/python3.9/site-packages/scipy/init.py", line 137, in getattr
return globals()[name]
KeyError: 'arange'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/home/oconnr03/miniconda/envs/ClearMapUi39/lib/python3.9/site-packages/ClearMap2-2.1.4-py3.9-linux-x86_64.egg/ClearMap/gui/tabs.py", line 1627, in make_group_stats_tables
df = self.main_window.wrap_in_thread(make_summary, self.params.results_folder,
File "/home/oconnr03/miniconda/envs/ClearMapUi39/lib/python3.9/site-packages/ClearMap2-2.1.4-py3.9-linux-x86_64.egg/ClearMap/gui/run_gui.py", line 619, in wrap_in_thread
return result.get()
File "/home/oconnr03/miniconda/envs/ClearMapUi39/lib/python3.9/multiprocessing/pool.py", line 771, in get
raise self._value
File "/home/oconnr03/miniconda/envs/ClearMapUi39/lib/python3.9/multiprocessing/pool.py", line 125, in worker
result = (True, func(*args, **kwds))
File "/home/oconnr03/miniconda/envs/ClearMapUi39/lib/python3.9/site-packages/ClearMap2-2.1.4-py3.9-linux-x86_64.egg/ClearMap/Analysis/Statistics/group_statistics.py", line 480, in make_summary
total_df = generate_summary_table(aggregated_dfs)
File "/home/oconnr03/miniconda/envs/ClearMapUi39/lib/python3.9/site-packages/ClearMap2-2.1.4-py3.9-linux-x86_64.egg/ClearMap/Analysis/Statistics/group_statistics.py", line 424, in generate_summary_table
total_df['q_value'] = clearmap_FDR.estimate_q_values(p_vals)
File "/home/oconnr03/miniconda/envs/ClearMapUi39/lib/python3.9/site-packages/ClearMap2-2.1.4-py3.9-linux-x86_64.egg/ClearMap/Analysis/Statistics/MultipleComparisonCorrection.py", line 133, in estimate_q_values
lam = scipy.arange(0, 0.90, 0.01)
File "/home/oconnr03/miniconda/envs/ClearMapUi39/lib/python3.9/site-packages/scipy/init.py", line 139, in getattr
raise AttributeError(
AttributeError: Module 'scipy' has no attribute 'arange'

[crousseau]

@microscopycore-ismms I just pushed a patch to the master branch if you want to try. It's a typo that was fixed on the dev branch but remained on the main branch. This kind of issues should be sorted soon when I merge the recent changes.

[microscopy-Core-ISMMS]

Okay thanks. But now, I am not able to install ClearMap on my Linux server. And I get the following error.

Creating entry
wrote /home/narasa01-local/.local/share/applications/menulibre-clearmap2.desktop
./install_gui.sh: line 279: /home/narasa01-local/.gnome/apps/menulibre-clearmap2.desktop: No such file or directory
chmod: cannot access '/home/narasa01-local/miniconda3/envs/ClearMapUi39/lib/python3.9/site-packages/start_gui.sh': No such file or directory
(base) narasa01-local@mabc-linux-server:/ClearMap2$ conda activate ClearMapUi39
(ClearMapUi39) narasa01-local@mabc-linux-server:/ClearMap2$ clearmap-ui
/home/narasa01-local/miniconda3/envs/ClearMapUi39/lib/python3.9/site-packages/scipy/init.py:155: UserWarning: A NumPy version >=1.18.5 and <1.25.0 is required for this version of SciPy (detected version 1.26.4
warnings.warn(f"A NumPy version >={np_minversion} and <{np_maxversion}"
Traceback (most recent call last):
File "/home/narasa01-local/miniconda3/envs/ClearMapUi39/bin/clearmap-ui", line 33, in
sys.exit(load_entry_point('ClearMap2==2.1.4', 'gui_scripts', 'clearmap-ui')())
File "/home/narasa01-local/miniconda3/envs/ClearMapUi39/bin/clearmap-ui", line 25, in importlib_load_entry_point
return next(matches).load()
File "/home/narasa01-local/miniconda3/envs/ClearMapUi39/lib/python3.9/importlib/metadata.py", line 86, in load
module = import_module(match.group('module'))
File "/home/narasa01-local/miniconda3/envs/ClearMapUi39/lib/python3.9/importlib/init.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "", line 1030, in _gcd_import
File "", line 1007, in _find_and_load
File "", line 986, in _find_and_load_unlocked
File "", line 680, in _load_unlocked
File "", line 850, in exec_module
File "", line 228, in _call_with_frames_removed
File "/home/narasa01-local/miniconda3/envs/ClearMapUi39/lib/python3.9/site-packages/ClearMap/gui/run_gui.py", line 75, in
from ClearMap.gui.gui_logging import Printer
File "/home/narasa01-local/miniconda3/envs/ClearMapUi39/lib/python3.9/site-packages/ClearMap/gui/gui_logging.py", line 12, in
import lxml.html
ModuleNotFoundError: No module named 'lxml'
/home/narasa01-local/miniconda3/envs/ClearMapUi39/lib/python3.9/site-packages/torch/distributed/distributed_c10d.py:181: UserWarning: torch.distributed.reduce_op is deprecated, please use torch.distributed.ReduceOp instead
warnings.warn(

[crousseau]

Hi @microscopy-Core-ISMMS ,

Yes that's probably because the install was more geared towards desktop install. Cluster support is planned but not really supported yet.
What I can advise to do is

1. ensure you can install a conda env on the cluster
2. create it using the .yml file (ClearMapUi39.yml on master (v2.1) or ClearMap3.yml in the more recent versions on dev, 3.1 should make it to master soonish)
3. install the config files into your ~/.clearmap folders yourself. The issue is that this may not be a proper folder ona cluster.

Please let me know if you hit other snags