Set separate 'thumbnail_url' for SMK

[stacimc]

Problem

The SMK provider script ingests records with ~2MB images. These files are overloading our thumbnail service and result in an unacceptable 6-10s load time. If these requests to the thumbnail server time out, the frontend will default to using the full image url (so a page with many results from SMK could end up downloading ~20 2MB images).

We would like to keep the high quality image for image_url, but supply a separate thumbnail_url with a smaller size.

Description

This was actually originally implemented in this way in WordPress/openverse-api#506, so much of this code can simply be reintroduced. For a given record, it will generate:

  "image_url": "https://iip.smk.dk/iiif/jp2/KKSgb5100_34.TIF.jp2/full/!2048,/0/default.jpg",         # Note filesize given as 2048
  "thumbnail_url": "https://iip.smk.dk/iiif/jp2/KKSgb5100_34.TIF.jp2/full/!400,/0/default.jpg".  # Note filesize is 400

Currently, thumbnails are hard-coded to None for images by the ImageStore. This was done in WordPress/openverse-api#519, because at the time the thumbnail_url was always redundant (since thumbnails are generated by the server). Now that we have a use case for needing separate image and thumbnail urls, we should remove this hard-coding.

Additional context

This issue handles adding the data to the records in the Catalog. There will also need to be a change to the API to actually use the thumbnail_url in thumbnail generation. API issue: #675

Implementation

• 🙋 I would be interested in implementing this feature.

[obulat]

I wonder if it makes sense to have a separate column for thumbnail that is used only by a handful (currently, only SMK) of providers, or would it be better to save thumbnail to the metadata column?
I feel like it's better not to have a separate column if only one (or several) providers use it. If we decide to save thumbnail to the metadata object, we could either save the list of such providers and check if the provider is in it when generating thumbnails (I think this would be faster) or check if there's a thumbnail field in the metadata column, and use it, or fallback on the image url otherwise.

[zackkrida]

Anecdotally, I've noticed that the 512px endpoint seems to have quite fast responses: https://iip.smk.dk/iiif/jp2/KKSgb5100_34.TIF.jp2/full/!512,/0/default.jpg

[AetherUnbound]

+1 to having this in the meta_data column! In Elasticsearch we're already accessing meta_data to pull out some other information that could be indexed:

https://github.com/WordPress/openverse-api/blob/1f32bb8b30bfd0c0bcc0454808a24b5a3e1ed53d/ingestion_server/ingestion_server/elasticsearch_models.py#L240

Perhaps we could have Elasticsearch look there first before serving a thumbnail link from the API! That way the image doesn't even have to be processed on the thumbnail server if a thumbnail URL already exists, we can just return the thumbnail URL in the API search response 😮