About class-agnostic evluation for coco20k instance segmentation

[hynnsk]

Hi, thank you for the great work! I have one question to ask for your help.

I understand that eval_instances.py with args.eval_most_confident=False performs evaluation of multi-object instance segmentation on the COCO20k dataset. The code looks like this:

        gt_object = COCO(args.gt_file)
        pred_object = gt_object.loadRes(args.input_file)
        results_cls = COCOeval(gt_object, pred_object, 'segm')
        results_cls.evaluate()
        results_cls.accumulate()
        results_cls.summarize()
        print('Eval classes ... ')
        print(gt_object.cats)
        print(colored('segm AP50 result is {}'.format(results_cls.stats[1]), 'yellow'))

For class agnostic evaluation, I think args.input_file should be mask_rcnn_agnostic_coco20k.json.

But, I noticed that all 'category_id' values in mask_rcnn_agnostic_coco20k.json are set to 0, which causes COCOeval to return AP50 of 0.

Am I following the correct procedure, or is there another way to evaluate class-agnostic evaluation?

Thank you.