关于深度倒装的问题

[Smezzdl1996]

Dear author, thank you for your great work. I tried to apply your method to self-supervised visual odometry in SLAM. However, when using your provided test script run.py, I found that the inferred depth seems to have smaller values for the farthest points. I verified this by visualizing the depth values on a heatmap, as shown in the figure.

my test code is listed as
`import numpy as np
import matplotlib.pyplot as plt
import cv2
import os
def visualize_depth_maps(
npy_path,
output_dir="depth_visualizations",
colormap='magma',
show_values=False,
step=20
):
if not os.path.exists(output_dir):
os.makedirs(output_dir)
depth_maps = np.load(npy_path)
print(f"Loaded depth maps of shape {depth_maps.shape}")

for i in range(depth_maps.shape[0]):
    depth = depth_maps #[i]
    plt.figure(figsize=(8, 6))
    plt.imshow(depth, cmap=colormap)
    plt.colorbar(label='Depth (normalized)')
    plt.title(f'Depth Map {i+1}')
    plt.axis('off')
    if show_values:
        h, w = depth.shape
        for y in range(0, h, step):
            for x in range(0, w, step):
                plt.text(
                    x, y, f"{depth[y, x]:.2f}",
                    color='white',
                    fontsize=3,
                    ha='center', va='center',
                    bbox=dict(facecolor='black', alpha=0.3, lw=0)
                )
    output_path = os.path.join(output_dir, f'depth_map_{i+1}.png')
    plt.savefig(output_path, bbox_inches='tight', pad_inches=0.05, dpi=300)
    plt.close()
    print(f"Saved: {output_path}")

if name == "main":
npy_path = "aaa.npy"
visualize_depth_maps(
npy_path,
output_dir="depth_visualizations",
colormap='magma',
show_values=True,
step=100
)
`
I would like to know what caused this issue and I’m really looking forward to your reply. Alternatively, is there any way to fix this problem? I actually only need disp for fine-tuning in self-supervised visual odometry.

[heyoeyo]

The relative depth models predict 'inverse depth', which comes from the earlier MiDaS models that they're based on. There's quite a few issues about it, like #93 or #268 from the MiDaS repo. I also have an explainer on github with pictures, which might help.

[Yang-Xijie]

@heyoeyo Thank you so much to explain this in detail!
An extra note: 3DGS (https://github.com/graphdeco-inria/gaussian-splatting?tab=readme-ov-file#depth-regularization) can use mono depths for supervision. To convert the "normalized inv depth" or "normalized depth" to "true depth", yes, we can adopt the know "d_min" and "d_max" to carry out the map. We can also use the average and variance to align the "normalized inv depth" or "normalized depth" to another "given depth". Just like 3DGS does in https://github.com/graphdeco-inria/gaussian-splatting/blob/54c035f7834b564019656c3e3fcc3646292f727d/utils/make_depth_scale.py#L54-L60.

[heyoeyo]

We can also use the average and variance to align the "normalized inv depth" or "normalized depth" to another "given depth"

Ya, that's a good point! This is similar to what MiDaS (page 6 of a related paper) and depth-anything (page 3 of the DA paper) seem to do for supervised training as well.

[Smezzdl1996]

Thank you for your reply. I have successfully resolved the depth issue. I applied min-max normalization to the output inverse depth, mapping the model’s relative depth predictions to the [0, 1] range as disparity values, which are then used as the supervision signal for self-supervised pose regression. as
for f_i in self.opt.frame_ids: for scale_i in self.opt.scales: disp_temp = self.models["DepthAnythingV2"](inputs["color", f_i, scale_i]) disp_temp = (disp_temp - disp_temp.min()) / (disp_temp.max() - disp_temp.min() + 1e-8) outputs["disp", f_i]["disp", scale_i] = disp_temp.unsqueeze(1)