During training and validation a subbtle halow is often drawn around insect.

[qgeissmann]

The inpainting halo

What the function is for

telea_inpaint_polys in src/flat_bug/augmentations.py paints unwanted animals out of a
training crop. It takes two lists of polygons:

• polys — instances to remove (usually ones cut by the crop boundary)
• exclude_polys — instances to keep (the ones that stay labelled)

It works at 1/6 resolution for speed, then upsamples the result back.

How it was supposed to work

1. Draw both lists into one mask. Both, not just polys, so that Telea's fast-marching
   doesn't reach into a kept animal and smear its colour into the region being erased.
2. Dilate that mask by one pixel, so the inpainting also covers the soft edge where an
   animal fades into the background.
3. Inpaint everything the mask marks.
4. Afterwards, subtract the kept instances back out of the mask, so the original pixels
   show through for them and only the unwanted animals are actually replaced.

Where it went wrong

Step 4 subtracted the bare polygons — but step 2 had dilated them.

# before
for p in exclude_polys:
    inpaint_bitmap = cv2.drawContours(inpaint_bitmap, [p // downscale_factor], color=0, ...)

Drawing at color=0 erases exactly the polygon and nothing more. The dilation had grown
every contour in the mask by one low-res pixel, kept ones included, and that grown ring was
never taken back. So a one-pixel-wide ring at 1/6 scale — roughly 6 px at full resolution
— stayed marked as "inpaint me", tight around every animal that was being kept.

Those ring pixels got the smeared Telea output pasted over them. The visible result is a
faint outline tracing every labelled instance: exactly the "weird contour, like they were
synthetic" seen in the AgriVolt crops.

Since it also happens during validation, the model cheats and gives inflated validation performance.
Fortunatly, we end to end eval is not afected

[qgeissmann]

fixed in 03e1bd9:
Trained default vs fix for 50 epoch. as expected, we fixes model shows lower validation performance but higher end2end performance. This is because the model learnt the halo shortcut instead of the true detection.

[asgersvenning]

Wow, that seems like quite a large improvement!

[qgeissmann]

yes... found it really by chance... some images were too big to annotate, so I tiled them and removed edge instances using the same inpainting code, but then when zooming in the image i could see a faint halo around instances I was about to annotate!