Improve discoverability of Risk Data Library Catalog from Google Dataset Search

[pzwsk]

Risk Data Library
is now indexed by Google Dataset Search (good news). However, only a few metadata seem to be indexed.

Analyzing and enhancing how Google Dataset Search robot is indexing our dataset pages should help improve discoverability of risk data library datasets through that specific website but also through other Search Engines.

See for instance https://datasetsearch.research.google.com/search?src=0&query=fathom%20flood&docid=L2cvMTFsdzM0Njk3bA%3D%3D

[pzwsk]

See also #14

[lydiascarf]

@pzwsk Do we have access to Google Search Console for the RDL JKAN site? If you go to this page and hit "Start Now", you can attempt to get set up using domain verification. This will require access to the site's DNS records. If you give me access to the Risk Data Library JKAN domain settings, I can do this myself and set up our account. Someone else who already has access to the domain settings can also set it up. Either way, we'll then be able to see what's wrong from Google's perspective.

That said, after doing a little bit of research, I suspect there are a few things we can do to get our datasets harvested:

1. Submit a sitemap to Google. I would like to start by trying to generate one with a Jekyll plugin. It hasn't been updated in a while, but that's generally true of a lot of the Jekyll ecosystem, and it shouldn't represent a vulnerability in this case. If this doesn't work, we can investigate alternatives.
2. There needs to be a JSON+LD block of metadata in our page source, as mentioned in this CKAN site. According to the documentation, this would be a block in the canonical pages, i.e. in the dataset template.

It turns out that I was wrong about the data.json file being significant to Google. While we should address the fact that our data.json file currently shows 0 datasets in production then continue to make it DCAT compliant, this is not a priority for Google, which is instead focused on structured data embedded in page source.

Some general sources:
https://datasetsearch.research.google.com/help
https://developers.google.com/search/docs/appearance/structured-data/dataset
🌟 https://pauldambra.dev/structured-data-with-jekyll.html 🌟

[matamadio]

Related: #12