Is CloudComPy multiple thread or process safe?

[jp123909]

Hi all,

I tried using multiprocessing.Pool to run ransac_sd in parallel, but the process hangs and never terminates.
I suspect there might be a conflict between Python's multiprocessing module and OpenMP.

Do you have any insights or suggestions?

Thank you in advance.

[prascle]

Hi,

Difficult question...
CloudCompare uses a lot of parallel algorithms and CloudComPy takes advantages of this, but I don't know for sure if it's process safe. The only experience I have with Python multiprocessing module is issue #45 and I am not sure it's conclusive.
I don't know if the problem is related to CloudComPy itself or to the Ransac_sd plugin.
Ransac_sd is a tricky piece of code and parallelism with OpenMP has been disabled in CloudCompare (and CloudComPy) after several problems (broken build on Linux, infinite loops on Windows...).
This does not explain why it does not work with Python's multiprocessing module, which works at a higher level.

What is your operating System ? (Windows, Linux, MacOS ?)

Regards,
Paul

[jp123909]

Hi Paul

Thank you for your reply. My OS is Ubuntu 22.04.
I found that not only Ransac_SD but also cloudComPy.computeNormals does not work with Python's multiprocessing module.
I suspect the issue is related to QtConcurrent, which is used internally in CloudCompare.
It seems that Python's multiprocessing cannot correctly handle the close event of Qt threads.

I would appreciate it if a pure Ransac_SD plugin without Qt (since the GUI is unnecessary for development) could be made available in the future, although I understand this might require a lot of effort.

So for now, I think the simplest solution is to use subprocess to run the functions, though I'll need to create some shared memory.
I'll give it a try.

Regards

[jp123909]

Update

The problem was resolved by creating a new thread and placing the multiprocessing process inside it.

For reference, see:
https://stackoverflow.com/questions/69794912/pyqt5-single-qprogressbar-with-multiprocessing-gets-stuck

[prascle]

Thank you, that's very interesting!
I'll try to add something about this to the cloudComPy documentation, it could be very useful!

Regards,
Paul