Why are intrinsics injected via raymap while poses use PRoPE?

[germemoe]

Hi, thanks for the great work!

I noticed that Pi3X seems to convert intrinsics into ray maps and inject them via ray_embed, while extrinsics are injected through PRoPE. Could you explain the motivation for separating them this way, instead of injecting both intrinsics and extrinsics together through PRoPE?

Have you tried an ablation with full camera geometry injected only through PRoPE?

[yyfz]

Hi, germemoe!

When I previously used PRoPE, I introduced some bugs, so the experimental results showed PRoPE performing worse than GTA. I later found the bugs, but I did not rerun the experiments. Intuitively, I feel that PRoPE should be better. So if you would like to retrain the model, I recommend using PRoPE for both pose and intrinsics.

[germemoe]

Thanks for the clarification! That’s very helpful.