Advice for long-duration stability in online 6DoF estimation?

[barikata1984]

Hello, thanks for sharing this great project!

I am attempting to use this repository for real-time object tracking and 6DoF pose estimation for robotic manipulation. My pipeline is slightly modified from the original as follows:

• Initial Frame Acquisition: SAM3 (https://ai.meta.com/sam3/)
• Feature Matching: EfficientLoFTR (https://github.com/zju3dv/EfficientLoFTR)
• Segmentation Tracking: Cutie (https://github.com/hkchengrex/Cutie).

And, my environment is as follows and works at about 15 FPS in most cases:

• CPU: Ryzen 9950X3D
• RAM: 128GB
• GPU: RTX 5090
• Virtual Env: Docker

However, I have encountered stability issues during long runs like:

• Freezing: The prediction process sometimes freezes/hangs without any terminal error messages. This usually happens around 10 minutes into a task.
• GUI Crash: The visualization GUI sometimes crashes (disappears) abruptly. In some cases, the estimation process continues in the background, but other times it fails completely.
• Drift: Significant pose drift accumulates over time, even when the system is otherwise stable.

Could you provide any advice on debugging or alleviating these stability issues or preventing drift over long durations?

Thanks in advance!