Investigate and add last updated value to data outputs

[andrewtavis]

Terms

• I have searched open and closed feature requests
• I agree to follow Scribe-Data's Code of Conduct

Description

A long term goal for Scribe is that data updates will only need to be for the specific data that's been updated since the last time that the user requested new data. One way this could be achieved is by getting the last time that the Wikidata item was edited and adding this as a last_updated field in the JSON and SQLite tables. This would allow us to serve data that meets the condition that last_updated >= last_app_data_update.

This issue would investigate how to get the last time that a Wikidata entry was edited via both the JSON dump and the Wikidata Query Service queries and add this to the outputs of both processes 🚀

Contribution

Happy to support someone who has interest in working on this!

@axif0 might pick this up for Outreachy, but could also review if someone else had interest 📶✈

[axif0]

Wikidata entry was edited via both the JSON dump and the Wikidata Query Service queries

Are we want like this?

{
"modified":"2024-11-12T16:46:51Z",
  "L87": {
    "nominativePlural": "Regenbögen",
    "genitiveSingular": "Regenbogens",
    "nominativeSingular": "Regenbogen",
    "dativeSingular": "Regenbogen",
    "accusativeSingular": "Regenbogen"
  }
}

Add an extra parameter modified to show the last timestamp.

[andrewtavis]

I think that having the timestamp in the LID object might make more sense so that the LID is the base unit? With that sounds really good though!

[axif0]

For clarifying, last update, we need to add schema:dateModified ?lastModified . in all sparql queries. The date will save as "lastModified": "2024-12-14T18:10:24Z" in JSON.

query Link

Is it correct ?

[andrewtavis]

Looks great, @axif0! We'd also need this to be ignored in the checks?

[axif0]

We'd also need this to be ignored in the checks?

Are you referring to this commit e363282 for add modified date in missing forms queries ?

[andrewtavis]

Exactly this, @axif0 :) Thanks so much for you attention to detail!