How to improve impainting.

[iamdhar]

Hi, I was cleaning some colored panels and in most of the bubbles, the impainting color is directly put on the original text that was supposed to be cleared, which makes it look weird.

I did the same panel on the cotrans demo site and it cleaned it pretty well drawing the stuff that was under it even where there are multiple colors behind the text bubble. How do I make the outputs of the panel cleaner just like the contrans demo? Is there even any way?

[iamdhar]

In summary, if i change the impainting models, like the one being used by cotrans, will the results improve? What model are we using currently in panel cleaner?

Can you suggest me some good newer models? And where do i place them inside the directory?

[VoxelCubes]

Panel Cleaner isn't cotrans. It works differently. It detects text (using a model) and then uses an algorithm to try out the best mask to cover the text. The output of this comic text detector is black and white (see the "raw output" in the image details), so it can only cover text with a single color. That is called masking.
For more complex bubbles like you have, the inpainting model must be used. It uses a model (lama inpainting) to inpaint the text that the masker could not adequately clean. The text in your image was cleaned by the masker, so make sure inpainting is enabled in your profile (off by default) and then lower the setting in Masker: Mask Max Standard Deviation. You can see which masks failed to be cleaned by the masker using the output Fitment Quality. If the text is crossed out, then the inpainter gets the chance to act, as you'll then see in the Inpainter outputs at the bottom of the list.
(Btw, to see image details, you click on an image in the list)

Hope that helps!

If yoy want to change the inpainting model, you can find where the model lives by clicking "Open Log" in the bottom right. The new window will tell you where your Cache Directory is (near the top). In there you will find a file called lama inpainting, you can replace it with any other lama compatible model file. Just rename the old file to something else, and rename the new one to the original name. Hope that works!

Good luck!

[iamdhar]

yeah i had the impainting enabled and still the results were like this, so youre telling me i have to reduce that setting from the gui to make the impainter work for those parts? will it break anything? how much should i reduce it to? it is 15 by default. Sorry for bothering you but if i want the impainter to work most of the times so that i can get better cleans, what settings do i have to tweak?

edit: i am messing with the Mask Max Stnd Dev setting giving different values and so far, 8 seems good. Will this affect the b/w panels i clean in any way?

Even though it was 8, still some of the bubbles were cleaned by the mask, if i set it to 0, will every mask get impainted? is that how it works?

[iamdhar]

Also also, sorry again, but I have a gtx 1650 gpu and by what I could read online, it is CUDA enabled. Currently the cleaning process uses my cpu only and i wanna know if i can also use my gpu for it so that the process gets quicker.

I have no idea how to use CUDA so can you please help…

Thanks again dude, you’ve been a huge help!!

[VoxelCubes]

For cuda questions, please refer to the FAQ: https://github.com/VoxelCubes/PanelCleaner/blob/master/docs/faq.md#frequently-asked-questions

CUDA can be a massive pain to deal with, so prepare yourself for that.

As for forcing inpainting instead of masking, you can achieve that by setting the Mask Max Std Dev to 0 and Mask Growth Steps to 1. Masking is generally better (and faster) than inpainting, unless you're dealing with colored pages like you are. Using a really low tolerance and only testing one mask using those settings though will result in almost all masks using the inpainter.

Just try it out and make a different profile for black and white series that don't have those problems. Tweak settings and look at the detailed outputs to see what works. Just remember that it'll never be perfect, AI has flaws. So I'd recommend outputting the masks instead of the cleaned images. That way you can layer the masks on the originals in gimp, photoshop, clip studio etc. and simply erase the mask where it messed up.
You can also use the psd output option in General: Layered Export to get a .psd file that already has everything layered properly (those files can also be imported into gimp no problem). Clip studio support maybe one day.

Good luck and happy tinkering. Each series has different optimal settings. The defaults are just good enough for most of them.