Skip to content

Commit

Permalink
docs: Update Refresh stage after changes in when things are deleted
Browse files Browse the repository at this point in the history
  • Loading branch information
jarofgreen committed Apr 24, 2024
1 parent 3584252 commit 3b6a332
Showing 1 changed file with 2 additions and 1 deletion.
3 changes: 2 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -134,7 +134,8 @@ Service Loop (when container starts)
- Checks for `stale_datasets` - `document.last_seen` is from a previous run (so no longer in registry)
- `clean_datasets()`
- Removes `stale_datasets` from Activity lake, decided it wasn't worth updating `changed_datasets` from activity lake because filenames are hash of `iati_identifier` so less likely to change.
- Removes `changed_datasets` and `stale_datasets` from source xml blob container and Solr.
- Removes `stale_datasets` from source and clean xml blob container and Solr.
- Removes `changed_datasets`from source and clean xml blob container. Not Solr as this will be removed later, and we want the older data to be available to data store users during processing.
- Removes `stale_datasets` from DB documents table
- `reload(retry_errors)`
- `retry_errors` is True after RETRY_ERRORS_AFTER_LOOP refreshes.
Expand Down

0 comments on commit 3b6a332

Please sign in to comment.