-
Notifications
You must be signed in to change notification settings - Fork 3
Performance profiling
The following timing tests were done on an Intel(R) Xeon(R) CPU E5-1620 v3 @ 3.50GHz CPU with 16 GB DDR3 ECC RAM, nVidia GM107GL [Quadro K2200] and SATA 6 Gb/s spinning disk.
Galario and emcee packages both support multiprocessing. The underlying C++ code in galario supports both OpenMP and CUDA parallelisation. The emcee code supports single node parallelisation using the multiprocessing python infrastructure and MPI on single or multiple nodes.
By default galario tries to use all available CPU cores for the image to complex plane transformation. Does the competition for resources hinder performance when emcee is run in parallel mode? What is the most optimal use of threads?
The tests show that when emcee is used in multiprocessing or MPI modes then galario should be instructed to either use GPU or a single CPU thread. If this is not ensured, then the runtime may increase by a factor of 10, due to resource competition. The number of CUDA core blocks (8 to 32) affect the performance marginally. On the test machine (old, entry level GPU) the MPI CPU mode of galario is almost a factor of 2 faster than GPU computation.
Note that in SiDE emcee fitting the most time consuming step is the radiative transfer modelling. Nevertheless, it is important to optimize the image to uv space transformation and model fitting as much as possible.
| Galario threads | emcee threads (multi) | Runtime [s] |
|---|---|---|
| 1 | 1 | 1044 |
| 1 | 8 | 305.08 |
| 8 | 1 | 422.93 |
| 8 | 8 | 2346.1 |
| Galario threads | emcee threads (MPI) | Runtime [s] |
|---|---|---|
| 1 | 8 | 291 |
| 1 | 2 | 1053 |
| 2 | 4 | 328 |
| 2 | 2 | 649 |
| 4 | 2 | 504.8 |
| 8 | 8 | 2688 |
| Galario (CUDA) threads | emcee threads (multi) | Runtime [s] |
|---|---|---|
| 8 | 1 | 633.5 |
| 16 | 1 | 636.1 |
| 16 | 8 | 536.4 |
| 32 | 1 | 645 |
| Galario (CUDA) threads | emcee threads (MPI) | Runtime [s] |
|---|---|---|
| 16 | 2 | 641 |
| 16 | 8 | 539 |
The RADMC-3D code has multiple parameters for speeding up the thermal Monte Carlo radiative transfer computation.
Single model computed with nphot = 100 000 and modified random walk switched on.
| ifast | Runtime [s] | delta T [K] | delta F [Jy] |
|---|---|---|---|
| 0 | 10.0 | ? | ? |
| 1 | 7.40 | ? | ? |
| 2 | 8.03 | ? | ? |
The aim of SiDE is not to produce high precision radiative transfer models at each modelling step, but to probe a large parameter space in order to find best fit parameters within the model framework (i.e. assumptions on geometry, density distribution, etc.). Precision is traded for computation speed. The user should always recompute the best fit parameter model with much higher number of photon packages and optimalization switched off (ifast = 0). The modified_random_walk flag might still be necessary to deal with optically thick regions.