File parsing and validation #70
Conversation
- Added update_submission fuction to submission_repo.py - Added test_update_submission function to pytest using SessionLocal mock and session_generator to simulate multiple sessions and transaction doing updates and fetches
src/services/submission_processor.py
Outdated
| validator_version = imeta.version("regtech-data-validator") | ||
|
|
||
| # Set VALIDATION_IN_PROGRESS | ||
| await update_submission(SubmissionDAO(submitter=submission_id, state=SubmissionState.VALIDATION_IN_PROGRESS)) |
There was a problem hiding this comment.
submitter is going to be the username or email (see #52). The SubmissionDAO id field should be set to the submission_id. As it stands, each state update is creating a new entry in the database instead of updating the submission with that id.
| ) | ||
| await update_submission( | ||
| SubmissionDAO( | ||
| submitter=submission_id, |
| else: | ||
| await update_submission( | ||
| SubmissionDAO( | ||
| submitter=submission_id, |
| @@ -46,7 +46,7 @@ async def upload_file( | |||
| ): | |||
| content = await file.read() | |||
| await submission_processor.upload_to_storage(lei, submission_id, content) | |||
| background_tasks.add_task(submission_processor.validate_submission, lei, submission_id, content) | |||
| background_tasks.add_task(submission_processor.validate_submission, lei, submission_id, content, background_tasks) | |||
There was a problem hiding this comment.
let's go with getting submission updated before the background_task kicks in; we don't need 2 nested bg tasks. so have the submission dao updated here to say validation in progress; then just 1 bg task that does the validation and update the submission dao when validation completes.
when we do the first submission update with the validation in progress, pass in the session already attached to the request. so
repo.update_submission(the_dao, request.state.db_session)then the bg task's one, call it without passing in a session.
There was a problem hiding this comment.
actually, we'll deal with this in the refactor that will happen in #51; just address the submission_id comment.
Coverage report
Click to see where and how coverage changed
This report was generated by python-coverage-comment-action |
||||||||||||||||||||||||||||||||||||||||||||||||
There was a problem hiding this comment.
I think this is good to go after removing the unnecessary mock lines in def test_unauthed_upload_file. There is still the issue of the submitter being set to the submitter_id, but this will all be rewritten with #51 and I don't think it's worth the extra effort here to correct all that (would take faking up a submitter email/username, and submission ids). Just make sure that is corrected there.
tests/api/routers/test_filing_api.py
Outdated
| mock_upload = mocker.patch("services.submission_processor.upload_to_storage") | ||
| mock_upload.return_value = None | ||
| mock_validate_submission = mocker.patch("services.submission_processor.validate_submission") | ||
| mock_validate_submission.return_value = None |
There was a problem hiding this comment.
Technically these mock return values aren't needed because the unauthed hits and returns from the endpoint prior to any actual processing in the function.
Closes #5
validate_submissioninto separate functions for specific tasks.update_submissionto utilize new sessions when making db calls.submission_processor.