Incremental Loading - Stage 1 - Implement checks (4) - No new resources

[Ben-Hodgkiss]

Overview
The final check required is that no new resources have been downloaded. With this check in place, the gatekeeper can act to stop processing of a collection if all the checks pass. In this case, the process log should be uploaded to S3, but no further action is required.

Pull Request(PR):

Tech Approach

• Check the downloaded resources, and ensure that they are all in resources.csv (indicating they have previously been processed)
• Use this information to implement the check.

Acceptance Criteria/Tests

• New resources are correctly identified as such to ensure the collection is processed.
• The process log is updated correctly.

Resourcing & Dependencies

• Developer.