You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This was discussed briefly in #78, but I'll use this issue as a way to better express my ideas around the bigger problem here. Our data.(js|min.js|yml) files are going to be changing quite often while this site is in development, not just in their values but also in their keys.
For example, if we were to make some large changes to the data file. Let's say, for example, the desired effect of our change is to move the wikidata link of an organization to a new key:
When we deploy the code, it will still expect a "wikidata_url" key to exist, despite the name being changed to "wikidata_link". On top of that, the caching code will use the existing organization's object instead of making another API request to GCI (because we don't expect their returned data to change) so the "wikidata_url" will stick around along with the new "wikidata_link" leading to some very cluttered data.
So how do we fix this?
Pretty straight-forward. Adding a CACHE_VERSION or DATA_VERSION constant in our code and putting that value in our data.(js|min.js|yml) files will allow us to check if our data format is up to date. We would increment this version every time a breaking change is made to the data format (be it adding or removing a key, or updating the name of an existing key). If the DATA_VERSION we expect and the DATA_VERSION of the data file we're reading from don't match up, we ignore it and re-fetch all the data.
The text was updated successfully, but these errors were encountered:
Ignoring the cache based on a schema version change is too simplistic; our rendered version would regress too often. Instead, we would eventually need 'migrations'; scripts which convert from one version to the other. But I dont see that being particularly important to build as an independent feature of this build system. It is a lot of work to build generically. Maybe we can find an existing system in Node? but I doubt it. It is something we can build using the django framework for the community project.
There is also the added complexity that if we do have downstream users of the js/yaml, we need to maintain the old keys anyway, so we would need to version the filenames if there is breaking change to the contents.
At the moment, each PR that includes key renames should also be removing old keys. We forgot that in #83 ? Send a PR to kill the old names, and we are done for the moment, but can revisit this if renames become more frequent.
This was discussed briefly in #78, but I'll use this issue as a way to better express my ideas around the bigger problem here. Our
data.(js|min.js|yml)
files are going to be changing quite often while this site is in development, not just in their values but also in their keys.For example, if we were to make some large changes to the data file. Let's say, for example, the desired effect of our change is to move the wikidata link of an organization to a new key:
When we deploy the code, it will still expect a "wikidata_url" key to exist, despite the name being changed to "wikidata_link". On top of that, the caching code will use the existing organization's object instead of making another API request to GCI (because we don't expect their returned data to change) so the "wikidata_url" will stick around along with the new "wikidata_link" leading to some very cluttered data.
So how do we fix this?
Pretty straight-forward. Adding a
CACHE_VERSION
orDATA_VERSION
constant in our code and putting that value in ourdata.(js|min.js|yml)
files will allow us to check if our data format is up to date. We would increment this version every time a breaking change is made to the data format (be it adding or removing a key, or updating the name of an existing key). If theDATA_VERSION
we expect and theDATA_VERSION
of the data file we're reading from don't match up, we ignore it and re-fetch all the data.The text was updated successfully, but these errors were encountered: