[ORCA-306] Get and update Challenge submissions

[jaymedina]

problem

Helper methods needed to be added for basic operations with Synapse, such as retrieving the submissions with a certain submission status and updating submission statuses. This is for the new Sage Bionetworks Challenges backend workflow for handling submissions, which includes running an automated Airflow process that retrieves submissions and updates their status before piping them over to the Nextflow workflow.

solution

• Add a method to get submissions of a desired submission status (default is RECEIVED)
• Add a method to update submission statuses of input submission(s)
• Add tests
• Add challengeutils as a dependency
• Add documentation where appropriate
• Fix failing CI jobs

testing & preview

Unit tests were added to test_ops.py to cover basic functionality and error-handling.

Further, here are some screenshots from actual communications with the API:

[swarmia]

✅  Linked to Story ORCA-306 · Implement py-orca functionality for getting new challenge submissions and updating their statuses

[thomasyu888]

Awesome - I know you're still actively working on it, so just some preliminary comments.

[thomasyu888]

This package is going to be deprecated in favor of cnb-tools eventually. For now it's ok, but the package isn't being regularly maintained.

Probably need to add this to the pipfile.

Maybe some of the challenge module should be added to the python client.

[jaymedina]

Hi @thomasyu888 thanks for the comments. Since this PR is a blocker for ORCA-302, I'd like to give a status update:

I see no way of mocking the get_submissions test since it requires retrieving submissions from an actual submissions view, so I'm constructing a set of integration tests where I build up and tear down a Project with an Evaluation queue for testing. This is what I'm working on right now and will aim to wrap up before the end of the day. @BWMac

[thomasyu888]

Hi @thomasyu888 thanks for the comments. Since this PR is a blocker for ORCA-302, I'd like to give a status update:

I see no way of mocking the get_submissions test since it requires retrieving submissions from an actual submissions view, so I'm constructing a set of integration tests where I build up and tear down a Project with an Evaluation queue for testing. This is what I'm working on right now and will aim to wrap up before the end of the day. @BWMac

See something like this: https://github.com/Sage-Bionetworks/Genie/blob/429dafbaed3bc78bbf71ff6bfffb7d88a6f49d4d/tests/test_clinical.py#L15-L46. There is a way to create unit tests by mocking the behavior of tableQuery which may be lower lift than an integration test. That said, those tests I wrote are a bit dated. There are most likely other ways to achieve the mocking of a table query.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 100.00%. Comparing base (d15c5b7) to head (e0c160c).

❗ Current head e0c160c differs from pull request most recent head 0b8c6d2. Consider uploading reports for the commit 0b8c6d2 to get more accurate results

Additional details and impacted files

@@            Coverage Diff            @@
##              main       #37   +/-   ##
=========================================
  Coverage   100.00%   100.00%           
=========================================
  Files           28        28           
  Lines          920       932   +12     
  Branches       140       141    +1     
=========================================
+ Hits           920       932   +12     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[BWMac]

My summary review comment got lost somehow, but I think your tests need some work. See my comments about mocking. I like to use the patch method from unittest.mock. agora-data-tools has some examples that you can reference.

[BWMac]

LGTM. Thanks! Let me know when the new version has been released.

[sonarqubecloud]

Quality Gate passed

Issues
1 New issue
0 Accepted issues

Measures
0 Security Hotspots
No data about Coverage
0.0% Duplication on New Code

See analysis details on SonarCloud