Different Results on iBims Dataset

[xlDownxl]

Hello everyone,

first of all, thanks for the great work and repository!

I tried to reproduce the results on the iBims dataset using the --ibims flag on the inference script and the original iBims evaluation code. These are the results I am getting with the pretrained 101 model:

rel = 0.18252209
sq_rel = 0.31786945
rms = 1.007896
log10 = 0.093049966
thr1 = 0.67061347
thr2 = 0.8888803
thr3 = 0.953198
dde_0 = 0.8467216
dde_p = 0.014197104
dde_m = 0.1390813
pe_fla = 3.289500754786499
pe_ori = 9.120578994338139
dbe_acc = 2.7233768
dbe_com = 25.422318

The global metrics are as reported in the paper, but for some reason I receive different results for the iBims specific metrics, most notably the DBE com which is 7.03 (or 6.59), while my scripts report 25.422.
I am using the same torch and torchvision version as specified in the environment.yaml file.
Has anyone an idea on where this difference in results comes from?

Thank You!

[EryiXie]

Ahhh.. I am so sorry that I forget to reply to you. I just tested again and I get the same result as the paper reported. To output .mat format depth with double float accuracy for iBims, there is a specified args "--ibims1" or "--ibims_pd" for it. I assumed if the result you got is very different, could be the case that you use depth output as png file, which is rescaled with a depth_shift factor = 512 to fit into (0, 256x256-1).
You can try to run:
python simple_inference.py --config=PlaneRecNet_101_config --trained_model=weights/PlaneRecNet_101_9_125000.pth --ibims1=./ibims1/ibims1_core_mat:./ibims1/ibims1_core_mat/test
And then use the evaluation script given by iBims1 dataset.