Skip to content

multiprocessing hangs in iterative solver #24

Description

@bonfus

For some reason, the second time (?!) the code hits this line:

https://github.com/stefanch/sGDML/blob/master/sgdml/utils/desc.py#L345

during sgdml all, multiprocessing imap blocks the execution and ctrl+c gives this backtrace:

^C[CRIT] Traceback (most recent call last):
         File "/usr/lib/python3.9/multiprocessing/pool.py", line 853, in next
           item = self._items.popleft()
       IndexError: pop from an empty deque

       During handling of the above exception, another exception occurred:

       Traceback (most recent call last):
         File "/home/pie/Venvs/sgdmlenv/lib/python3.9/site-packages/sgdml/cli.py", line 1083, in train
           model = gdml_train.train(
         File "/home/pie/Venvs/sgdmlenv/lib/python3.9/site-packages/sgdml/train.py", line 927, in train
           R_desc, R_d_desc = desc.from_R(
         File "/home/pie/Venvs/sgdmlenv/lib/python3.9/site-packages/sgdml/utils/desc.py", line 345, in from_R
           map_func(partial(_from_r, lat_and_inv=lat_and_inv), R)
         File "/usr/lib/python3.9/multiprocessing/pool.py", line 858, in next
           self._cond.wait(timeout)
         File "/usr/lib/python3.9/threading.py", line 312, in wait
           waiter.acquire()
       KeyboardInterrupt

Disabling parallelism in this function is an acceptable workaround (not much time spent here).

Using numpy compiled with MKL support and 12 cores.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions