Generation on similar classes domain

[GLivshits]

Hi. You show quite good results on conditional generation on classical classification benchmarks. But how such approach would behave on the biometry domains, e.g. huge number of similar classes with at most 20 images per class?
This is not an issue, I just want to ask you for opinion regarding this question.

[ArantxaCasanova]

Hi,
thanks for the interesting question.

Given your description, it seems to be a very challenging domain. I will talk about the three main characteristics you provided:

1. few samples per class: we have shown in the paper (Table 4) that for the long-tailed class dataset ImageNet-LT, IC-GAN provides better quality/diversity than BigGAN. It is unclear whether this finding would translate to a dataset where all classes have very few images, but I would expect it does, given that the majority of classes in ImageNet-LT have very few images themselves.

2. huge number of classes: as it is done in BigGAN, we obtain an embedding per class to condition the generator and to do the projection in the discriminator. On top of my head, it seems scaling the number of classes would not result in a very obvious issue. I am not aware of any work that tries to scale BigGAN to more than a thousand classes, but I would expect a correlated success with that of BigGAN's for this type of scenario.

3. similar classes: It depends on how similar they are, but I suspect we are talking about a closer similarity than two close but different dog breeds in ImageNet. In here we might be limited by the BigGAN backbone and/or the GAN loss formulation: it all depends on the degree of finegrained details the backbone is able to generate/discriminate in order to generate class-conditional images.

Note that IC-GAN can be used with other GAN backbones if you find one that is better suited to your domain. We have only experimented with point 1 above, so this is just a hypothesis: Finding a proper backbone would specially tackle point 3 and/or 2, while the IC-GAN formulation could directly mitigate point 1.