ICP: GUI Matrix Wildly Different from Python Matrix

[stellarpower]

Hi,

I'll see if I can find to make an MRE later, as right now I have spent too much time on this.

Are there any known quirks when working with the ICP function in CloudComPy compared to using the CC native GUI? Or should a trivial opening of the two files and calling into the function with the defaults match almost exactly opening the two files in the GUI and using the ICP tool, also with the UI defaults?

I am simply loading in a cloud and a reference mesh, aligning as per the ICP tutorial in the docs, cloning the aligned result, applying the matrix, and then saving the lot to file, which I then inspect in ccViewer - nothing complicated thus far. Today I've modified the cloudcompare GUI build to trace all the arguments passed to the ICP implementation, and what I see logged is identical to what I am calling into from Python (most are at their defaults anyway). Whilst the GUI aligns the two pretty well, the matrix that comes out of the Python call is off by about 30 degrees and barely an improvement over the starting transformation between the two.

Are there any obvious causes of something like this? Or is it possible/expected that the GUI and Python results may differ without some pre-processing or careful pinning of circumstances?

Thanks.

[prascle]

Hello,
I'm sorry for the very late reply, I was full time on another project for 2 months... I hope you've found a solution.
There's a difference between Python's default settings and those of the GUI: the convergence method. The Python default uses a maximum of 20 iterations, whereas the GUI default uses a convergence criterion maxErrorConvergence (1.e-5).
I've built a better test than the very simple one in the documentation, to check behavior on a more realistic case.

The data can be downloaded here and the corresponding script is below.
The two parts to be aligned require a first approximate alignment for the ICP algorithm to work. Then I get the same result with the GUI and Python.

If you have a case that works with the CloudCompare GUI and not with Python, I'm interested.
I'll probably change CloudComPy's default settings to match CloudCompare's.

Best regards,
Paul

#!/usr/bin/env python3

import os
import sys
import math
import psutil

os.environ["_CCTRACE_"]="ON" # only if you want C++ debug traces

import cloudComPy as cc

objs = cc.importFile("chimeres.bin")
chimere_1_mesh      = objs[0][0] # mesh, first part of the object
chimere_2_mesh      = objs[0][1] # mesh, second part of the object
chimere_1_cloud     = objs[1][0] # cloud, first part of the object
chimere_2_cloud     = objs[1][1] # cloud, second part of the object
chimere_2_manualRot = objs[1][2] # cloud, second part of the object, very approximately aligned (hand rotation with the CloudCompare GUI)

chimere_1_mesh.setName("chimere_1_mesh")
chimere_2_mesh.setName("chimere_2_mesh")
chimere_1_cloud.setName("chimere_1_cloud")
chimere_2_cloud.setName("chimere_2_cloud")
chimere_2_manualRot.setName("chimere_2_manualRot")

# ICP with chimere_1_mesh and chimere_2_cloud does not work (neither with GUI nor with Python): ICP requires some first alignment

# ICP with the second part "pre aligned" by hand: OK, same result as CloudCompare GUI

res=cc.ICP(data=chimere_2_manualRot, model=chimere_1_mesh, finalOverlapRatio=0.5, method=cc.CONVERGENCE_TYPE.MAX_ERROR_CONVERGENCE)

tr1 = res.transMat
print(tr1.toString())
chimere_2_manualRot_transformed = chimere_2_manualRot.cloneThis() # = res.aligned.cloneThis() # note: res.aligned is data!
chimere_2_manualRot_transformed.applyRigidTransformation(tr1)
chimere_2_manualRot_transformed.setName("chimere2_manualRot_transformed_afterICP")

cc.SaveEntities([chimere_1_mesh, chimere_2_manualRot, chimere_2_manualRot_transformed],"chimeres_ICP.bin")

[stellarpower]

I'm sorry for the very late reply, I was full time on another project for 2 months...

No worries, perennially in the same situation myself. Appreciate the time responding.

So, I'm struggling to find my notes from last time I looked at this; if I can get hold of them will see if I can mock up an MRE. I noticed that the convergence criteria were different, but I modified the cloudCompare GUI from source (IIRC, I made sure to start from the same tag as the Python build used) to spit out more in essence trace logging information - so I was getting a different matrix even when I diffed the arguments that ccRegistrationTools::ICP() received against what I had specified on the Python side. I gave up before adding the same trace logging into the cloudComPy build too, but that aside, the matrices differed when the parameters were all identical - which was what was stumping me. As far as I was aware, ICP should be broadly deterministic, excepting the random subsampling, so with the exact same arguments, I was expecting a matrix back that looksed pretty similar to the one from the GUI. But from memory, the orientation portion was quite markedly different, and the Python alignment was over 180 degrees out, whilst the GUI had performed a satisfactory convergence.

But until I find those notes, to sanity-check, would you expect the code above to perform the same as this in the UI?

All in, I get the impression the code within CloudCompare itself could probably do with a bit of a refactor - I see there's duplication between the ICP function within qCC and the pyCC.cpp file here, and without having dived in properly, I saw a few bits that were a bit coupled to the UI/command-line interface whilst I was having a browse, and seen forum posts regarding GUI commands missing equivalents from the CLI, so longer-term these overall would probably be best pulled out upstream and put into the/a library if possible to separate.

[prascle]

Hi,
I ran the CloudCompare GUI and my Python script with 20 iterations and an RMS difference of 1.E-5 (all other parameters are the same as yours).
Clearly, 20 iterations are not enough in this case, and the matrices are not the same from one run to the next (but similar) :

GUI with 20 iterations: not converged (Final RMS*: 1.00383 (computed on 49999 points))
0.998947501183 0.045837983489 0.001680309884 5.549425601959
-0.045791473240 0.998720467091 -0.021460168064 -8.492719650269
-0.002661850769 0.021360635757 0.999768316746 0.645483553410
0.000000000000 0.000000000000 0.000000000000 1.000000000000

another run of GUI with 20 iterations: not converged
0.999026358128 0.044114358723 0.000527049240 5.338826179504
-0.044090352952 0.998759269714 -0.023150173947 -8.811495780945
-0.001547650318 0.023104395717 0.999731838703 0.616607248783
0.000000000000 0.000000000000 0.000000000000 1.000000000000

Python script with 20 iterations: not converged
0.999045491219 0.043590109795 0.002821651986 5.807701110840
-0.043508864939 0.998757183552 -0.024311551824 -9.067401885986
-0.003877888201 0.024165578187 0.999700427055 0.571819543839
0.000000000000 0.000000000000 0.000000000000 1.000000000000

another run of Python script with 20 iterations: not converged
0.999029994011 0.044034127146 0.000142479999 5.248332500458
-0.044018905610 0.998759925365 -0.023259034380 -8.841768264771
-0.001166494563 0.023230200633 0.999729454517 0.602434396744
0.000000000000 0.000000000000 0.000000000000 1.000000000000

When using the RMS difference of 1.e-5 for convergence, both the GUI and the Python script give a converged result and the matrices are almost identical (with an accuracy of ~1.e-5):

GUI with RMS difference 1.E-5 : converged
0.996447682381 0.059144437313 0.059949912131 21.347190856934
-0.054463900626 0.995549321175 -0.076910503209 -19.967372894287
-0.064231924713 0.073372185230 0.995234012604 -2.493510007858
0.000000000000 0.000000000000 0.000000000000 1.000000000000

Python script with RMS difference 1.E-5 : converged
0.996444761753 0.059185355902 0.059957753867 21.347774505615
-0.054505549371 0.995548665524 -0.076889686286 -19.963775634766
-0.064241603017 0.073348298669 0.995235145092 -2.493304967880
0.000000000000 0.000000000000 0.000000000000 1.000000000000

So, from my point of view, based on this particular case, the GUI and Python behaviors are consistent, and there's a non-negligible random effect visible in the early steps.
Incidentally, I've just discovered that using 100% overlap in this case instead of 50% provides better convergence (both GUI and Python), which is counter-intuitive for me...
You're right about the duplication of code. Some CloudCompare algorithms are not clearly separated from the GUI. CloudComPy was designed to use CloudCompare libraries, but for some parts of the code, the simplest solution was duplication in CloudComPy.
A partial restructuring of CloudCompare might be a good idea, but this needs to be coordinated with the CloudCompare developer team and I don't think it's a priority.
In the meantime, I'm checking the sources for differences, and I don't think there are any significant ones for ICP.
Anyway, if you have an example where the GUI and Python behaviors are different, I'm interested!
Paul

[stellarpower]

Sure thing, I'll get you more details tomorrow (most likely). I found some notes but there's not as much in there as I thought so will probably need to fish around and try to see if I can recreate the exact same scenario I had. I have a bit of a bad habit of saving things to a ramdisk when I think they'll be temporary, and then of course they end up not being so, on the odd occasion I finally reboot or my desktop crashes and I have to close down about 150 windows, stuff disappears!! But I'm sure I remember taking something really simple like a blender cube and finding differences between the two I couldn't explain.

I started with a 100% overlap (largely following the GUI defaults, especially as it worked visually, planning to tweak it after I had it scripted), and whilst again as you say, the code should be the same between the two, and I think the random sampling was quite a way above the number of points I think I had (maybe 250 or so in each cloud), so I'd expect almost identical results, maybe this is too high, maybe my clouds are just sparser than normal, so I guess it is possible that the algorithm didn't properly converge in that many steps, and the results diverged simply because it hadn't properly reached a solution yet.

On that note I find the GUI options a little bit confusing. I think it may have changed with a later build, but the way the radio buttons appear as a tree, obfuscates a bit what the algorithm is doing. Just from general knowledge, I'd expect to be able to select only one of maximum number of iterations, minimal overlap, or minimal residuals as a stopping criterion, an whilst I did log what was passed in, I didn't get as far as stepping through, and I saw from some of the CC code that some arguments sort of override others, rather than using e.g. an overload to express mutually-exclusive options, so it might be it wasn't set up quite as I thought it was.

for some parts of the code, the simplest solution was duplication

Yeah, if I had a pound for every time!

A partial restructuring of CloudCompare might be a good idea, but this needs to be coordinated with the CloudCompare developer team

Agreed, just mentioning as a sidenote, nothing that code in this project can really do about that. In an ideal world the core would be agnostic to the frontend, and then be it through something like SWIG or even just using native C++ compile-time stuff (which would have the bonus of automagically generating command-line flags) the bindings would be generated programmatically and less code would be needed.

P.S.: I noticed since I upgraded meshlab that there's a button on the dialogue for some filters that lets me copy a call for their python API to achieve the same affect. I find it has similar issues - some of these appear to be tied to the GUI, some not, the flow isn't always consistent with each filter, but, in spite of that it's a nice feature to have when it is there.