Issues with indexing by Google Dataset Search

[andrewsu]

Just starting a thread to track notes on whether CViSB datasets are being indexed on Google Dataset Search.

Currently, there are five datasets on data.cvisb.org (all listed in https://data.cvisb.org/assets/sitemap.xml):

• https://data.cvisb.org/dataset/hla
• https://data.cvisb.org/dataset/ebola-virus-seq
• https://data.cvisb.org/dataset/lassa-virus-seq
• https://data.cvisb.org/dataset/sarscov2-virus-seq
• https://data.cvisb.org/dataset/systems-serology

Two datasets are indexed (SARS-CoV-2, HLA) (https://datasetsearch.research.google.com/search?query=site%3Adata.cvisb.org)

Google Search Console reports 1 error, 0 "valid with warning" and 0 "valid" (https://search.google.com/search-console/datasets?resource_id=https%3A%2F%2Fdata.cvisb.org%2F). Oddly, the one error is for the HLA dataset (one of the successfully-indexed datasets). The error relates to having an object of type Organization under Citation.

Using the Rich Results Testing tool, that error shows up for 3 datasets (Ebola, Lassa, HLA) -- of those three, HLA is successfully indexed in Google Dataset Search. Two datasets (SARS-CoV-2 and systems serology) show up as "Page is eligible for rich results", but only systems serology is successfully indexed. The URL inspection tool on Google Search Console confirms that the datasets are successfully detected -- I just requested re-indexing in the hopes that those datasets will show up in Google Dataset Search (but I seem to recall doing this before).

And one last note that at different times, I have seen all five datasets successfully indexed and also three datasets successfully indexed. As far as I know, we have not changed anything on our end that would explain those changes. From now, will try to track that here...

[andrewsu]

Possibly triggered by the reindexing I requested yesterday, all five datasets are currently being returned (https://datasetsearch.research.google.com/search?query=site%3Adata.cvisb.org). Keeping this issue open for the moment just to track any changes...

[andrewsu]

The same search as above (https://datasetsearch.research.google.com/search?query=site%3Adata.cvisb.org) is now returning 3 datasets

[andrewsu]

Note that https://datasetsearch.research.google.com/search?query=site%3Adata.cvisb.org is currently indexing all 5 datasets on data.cvisb.org. (Possibly due to the change in #68 implemented last week?) Will continue monitoring to assess stability.

Also noting that the search console (https://search.google.com/search-console/datasets?resource_id=https%3A%2F%2Fdata.cvisb.org%2F) is still not showing that those datasets are indexed (unchanged from screenshot above).

[andrewsu]

Unfortunately, https://datasetsearch.research.google.com/search?query=site%3Adata.cvisb.org is back down to 3 datasets (the same three as in #67 (comment)). Searching on the top-level domain https://datasetsearch.research.google.com/search?query=site%3Acvisb.org also returns the same three datasets. So it doesn't look like it's an issue with indexing our dev site instead of our prod site. Emailing Natasha to ask for feedback...

[andrewsu]

Today, https://datasetsearch.research.google.com/search?query=site%3Adata.cvisb.org is back up to 5 datasets (same as in #67 (comment))