Instance prediction only for trunk class

[harshit-1801]

Instead of instance prediction on all the classes (2,3,4), I want to predict only for class 2 i.e. trunk. For that I changed the 'SemIDforInstance' variable in treeins panoptic dataset file. Do I also need to tune parameters of 'get_instances' function of PanopticResults to make it work or retrain the embed head, offset head or scorer Unet after freezing the backbone and semantic head, or do I need to change clustering parameters?

Edit - Increased the cluster_radius_search to 3 * grid_size instead of 1.5 and bandwidth to 0.8 for meanshift clustering from 0.6 and min_score to 1e-5 from 0.5(because scorer without retraining gives very less score to only trunk). The results from trunk instance segmentation are better now but still some instances are over segmented.

[bxiang233]

Instead of instance prediction on all the classes (2,3,4), I want to predict only for class 2 i.e. trunk. For that I changed the 'SemIDforInstance' variable in treeins panoptic dataset file. Do I also need to tune parameters of 'get_instances' function of PanopticResults to make it work or retrain the embed head, offset head or scorer Unet after freezing the backbone and semantic head, or do I need to change clustering parameters?

Edit - Increased the cluster_radius_search to 3 * grid_size instead of 1.5 and bandwidth to 0.8 for meanshift clustering from 0.6 and min_score to 1e-5 from 0.5(because scorer without retraining gives very less score to only trunk). The results from trunk instance segmentation are better now but still some instances are over segmented.

Hi,

I’m not 100% sure about your exact setting. Do you still keep the semantic head predicting 5 classes, or do you reduce it to just ground, low vegetation, trunk?

Anyway, if it were me, I would probably retrain the network once with only trunk as the instance class. The changes required should be minor, and in fact it should be easier than the case with leaves, since the hardest part is usually overlapping leaves. Other clustering parameters (like bandwidth) don’t necessarily need to be changed.

Also, I remember there are some hooks in the code for intermediate visualization, e.g. _dump_visuals/_dump_visuals_fortest:
https://github.com/prs-eth/ForAINet/blob/341e46f16884c521233e4457b72acb60e6824736/PointCloudSegmentation/torch_points3d/metrics/panoptic_tracker_pointgroup_treeins_partseg.py#L302

You can try enabling or modifying those to analyze the outputs more easily.

Since I haven’t used this project environment for a while, so I can’t test it directly now, but you can explore these ideas further. Looking forward to your feedback, and good luck with your experiments!