Too many S3 requests to dataset_logo_original for dataset thumbnail

[cesarpferreira]

What steps does it take to reproduce the issue?

• When does this issue occur?

When trying to load a dataset thumbnail and the dataset author/admin didn't define a thumbnail.

• Which page(s) does it occurs on?

It occurs on pages that require a dataset thumbnail on load.

• What happens?

Dataverse tries to access a S3 file that doesn't exist, this creates a lot of unwanted requests to the S3 endpoint.

• To whom does it occur (all users, curators, superusers)?

All users.

• What did you expect to happen?

I have configure the feature flag disable-dataset-thumbnail-autoselect to On, but documentation isn't clear on the full effects of this setting. In dataset table there is a column usegenericthumbnail, but the Thumbnail+Widgets page for datsets doesn't have a way to enable this setting. Is this option configured by this feature flag?
I have also configured S3 following this section of the documentation, but in the same table the column storagedrive is set to null for all entries. Shouldn't this have the storage driver ID configured ?

Which version of Dataverse are you using?
6.6

Any related open or closed issues to this bug report?
No

[pdurbin]

See also https://dataverse.zulipchat.com/#narrow/channel/378866-troubleshooting/topic/S3.20and.20dataset_logo_original/near/554651961

[bikramj]

Any solution or workaround for this issue? In 24hours our custom S3 endpoint recorded 537K requests to dataset_logo_original with a 404 and only 5K with 200! Can we somehow disable this check at all?

[cesarpferreira]

@bikramj the workaround we implemented to prevent these 404 requests was setting the feature flag disable-dataset-thumbnail-autoselect to True and we also changed in the dataset table the value of column usegenericthumbnail to True for all the datasets that didn't have a thumbnail file and were causing the failed requests. I would suggest invert the default behavior/value of this feature flag, since the dataset thumbnail isn't something mandatory or available at the dataset creation.