Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use gost API endpoints in FFE email notification #4000

Draft
wants to merge 6 commits into
base: main
Choose a base branch
from

Conversation

TylerHendrickson
Copy link
Member

@TylerHendrickson TylerHendrickson commented Feb 14, 2025

Ticket #3908

Description

This PR updates the arpa-exporter email notification, replacing the single presigned S3 URL for the zip file with a gost API URL. This adds an additional measure of security, ensuring that users must be authenticated with the ARPA Reporter tool before downloading an S3-hosted file. It also includes an additional link in the email notification that allows users to download the accompanying metadata CSV.

Additionally, this PR addresses # TODO comments in arpa-exporter/src/worker.py by refactoring the UploadInfo model in the following ways:

  • Renames the directory_location field to path_in_zip
  • Drops any fields not actively referenced in the arpa-exporter source code.

Finally, a new endpoint handler is added to the ARPA reporter API, at /api/exports/:tenantId/getFullFileExport/:downloadType(archive|metadata). When this endpoint receives an authenticated request, a presigned URL is generated for the requested object and a redirect is issued to the client to download the file as an attachment. This endpoint corresponds to the links provided to the user in the arpa-exporter email notification.

@TylerHendrickson TylerHendrickson self-assigned this Feb 14, 2025
@github-actions github-actions bot added enhancement New feature or request javascript Pull requests that update Javascript code python labels Feb 14, 2025
Copy link

QA Summary

View the workflow summary for individual coverage reports if this comment is truncated.

QA Check Result
🌐 Client Tests
🔗 Server Tests
🤝 E2E Tests
🐍 ARPA Exporter Tests
📏 ESLint
🧹 TFLint
🧼 Ruff
🛁 mypy

Test Coverage

Coverage report for `packages/client`
St File % Stmts % Branch % Funcs % Lines Uncovered Line #s
🔴 All files 37.3 37.21 33.92 38.9
🔴  src 0 100 100 0
🔴   App.vue 0 100 100 0 2-9
🔴  src/arpa_reporter 0 100 100 0
🔴   App.vue 0 100 100 0 2-13
🟡  ...ter/components 63 60 55.31 64.51
🟢   AlertBox.vue 100 100 100 100
🟡   ...oadButton.vue 60 55.55 37.5 60 7,60-67
🟢   ...ileButton.vue 100 100 100 100
🟢   ...ttonSmall.vue 100 100 100 100
🟢   ...mplateBtn.vue 100 100 100 100
🟡   ...avigation.vue 65.62 63.63 52.63 65.62 ...01-207,216-223
🔴   StandardForm.vue 45 50 41.66 45.45 ...25-129,136-158
🟢  ...porter/helpers 84.61 79.48 87.5 84.61
🟢   form-helpers.js 84.21 79.48 85.71 84.21 7,16,25,81-83
🟢   short-uuid.js 100 100 100 100
🔴  ...eporter/router 0 0 0 0
🔴   index.js 0 0 0 0 19-124
🔴  ...reporter/store 3.92 0 2.17 4.12
🔴   index.js 3.92 0 2.17 4.12 11-14,32-261
🔴  ...reporter/views 47.39 33.43 47.97 49.38
🟢   AgenciesView.vue 100 0 100 100 16
🔴   AgencyView.vue 40.74 36.36 50 45.83 53-62,70-96
🟢   HomeView.vue 93.47 90.24 100 93.47 114,147,180
🔴   LoginView.vue 29.62 36.36 20 30.76 1,6,21-36,74-102
🔴   ...plateView.vue 27.02 35.71 45.45 25 ...2,30-37,69-113
🔴   ...ploadView.vue 24.24 16.66 33.33 26.66 1,30-31,116-144
🔴   ...eriodView.vue 48 18.18 75 52.17 64-90
🟡   ...riodsView.vue 57.57 28.57 60 59.37 101,132,149-171
🔴   ...pientView.vue 40 10.52 30.76 41.02 73-93,110-152
🔴   ...ientsView.vue 48.48 8.33 46.66 53.33 ...46,165-190,203
🟡   UploadView.vue 61.95 43.75 62.5 61.79 ...42-443,449-450
🔴   UploadsView.vue 16.66 0 0 18.18 61,112-289
🔴   UserView.vue 46.34 33.33 68.75 48.64 84,97-137
🟡   UsersView.vue 60 7.69 62.5 66.66 91-92,105-107,123
🟡  src/components 54.7 44.55 55.47 55.42
🔴   ...vityTable.vue 18.75 0 28.57 19.35 115-185
🟡   BaseLayout.vue 69.56 53.84 60 69.56 172,219-231
🔴   CopyButton.vue 30 33.33 40 30 6,54-63
🟢   ...tActivity.vue 80.76 62.85 83.33 80.76 ...31-136,179,201
🟢   GrantNotes.vue 90 78.26 85 89.79 ...87,207,216-217
🔴   GrantsTable.vue 43.58 39.47 41.86 45.27 ...72-476,484-564
🔴   ...dUploader.vue 24.24 33.33 37.5 24.24 16,25,74-111
🔴   SearchFilter.vue 40.74 28 37.5 40 ...52,64,67,72-82
🔴   ShareGrant.vue 38.7 64.28 23.07 40 ...08-112,131-152
🟢   ...ivityItem.vue 96.15 90.9 100 96.15 64
🟡   UserAvatar.vue 70 75 100 62.5 35-37
🟢   ...eaderText.vue 100 100 100 100
🔴  ...ponents/Modals 29.21 44.34 33.1 32.94
🔴   ...anization.vue 17.39 65.62 30 22.22 1-14,149-178
🔴   AddTeam.vue 45.45 55.55 61.53 57.69 ...06,212,224-247
🔴   AddUser.vue 36.66 66.66 60 37.5 ...40,145,148-176
🔴   ...anization.vue 15.38 64.28 14.28 16.66 1-17,60-80
🔴   EditTeam.vue 18.18 26.43 33.33 20.83 ...32,211,219-304
🔴   EditUser.vue 21.05 66.66 25 22.22 1,102-129
🔴   ...ilsLegacy.vue 22.03 0 0 24.07 131,177,205-369
🟢   ...Followers.vue 83.33 88.88 57.14 83.33 10,114-115
🟡   ImportTeams.vue 50 41.17 50 53.33 28,64-69,81-82
🔴   ImportUsers.vue 42.85 60 40 46.15 29,65-80
🔴   ...archPanel.vue 27.9 15.78 23.52 28.57 ...68-178,211-255
🔴   SearchPanel.vue 21.62 63.26 32 27.58 ...77-380,386-458
🔴  src/helpers 18.12 17.92 18.6 18.88
🟢   constants.js 100 100 100 100
🟢   currency.js 100 100 100 100
🟡   dates.js 66.66 100 33.33 100
🔴   fetchApi.js 6 13.79 5.26 6.12 10-12,20-132
🔴   filters.js 4 0 0 4.54 19-51
🔴   form-helpers.js 0 0 0 0 5-82
🟡   gtag.js 77.77 90 75 77.77 12,51
🟢   testHelpers.js 100 100 100 100
🔴   ...patWarning.js 0 0 0 0 39-61
🟢  ...s/featureFlags 92.3 100 85.71 92.3
🟡   index.js 80 100 80 80 8
🟢   utils.js 100 100 100 100
🔴  src/mixin 20 0 28.57 20
🔴   ...zableTable.js 20 0 28.57 20 16-31,36-37,42
🔴  src/router 18.91 16.66 11.11 18.91
🔴   index.js 18.91 16.66 11.11 18.91 ...81-182,186-205
🟢  src/store 100 100 100 100
🟢   index.js 100 100 100 100
🔴  src/store/modules 2.82 0 4.48 2.94
🔴   agencies.js 5.26 100 8.33 5.55 13-70
🔴   alerts.js 20 100 20 20 10-24
🔴   grants.js 0.99 0 0.97 1.03 61-456
🔴   organization.js 33.33 100 33.33 33.33 21-25
🔴   roles.js 20 100 20 25 13-22
🔴   tenants.js 11.11 100 14.28 12.5 13-32
🔴   users.js 2.43 0 4.76 2.5 17-100
🔴  src/views 45.16 38.12 33.98 46.95
🔴   ...orterView.vue 25.58 51.85 18.18 26.82 ...,64,87,112-154
🟡   ...boardView.vue 50 17.64 50 52 89-98,114-125
🔴   ...tailsView.vue 32.97 14.28 17.07 33.69 ...98-437,443-464
🟢   GrantsView.vue 100 100 100 100
🟡   LoginView.vue 56 38.88 44.44 58.33 25,140-162
🟡   MyGrantsView.vue 73.33 85.71 62.5 73.33 1,80,91,99
🟡   ...ofileView.vue 78.26 80 42.85 78.26 1,32,63,136-140
🟢   NotFoundView.vue 100 100 100 100
🔴   ...tionsView.vue 47.05 57.14 41.66 53.33 ...97-100,114-118
🔴   ...ivityView.vue 46.42 23.8 43.75 46.42 ...01,114,120-134
🔴   TeamsView.vue 44.44 88.88 41.66 53.33 1,58,142,156-163
🟡   UsersView.vue 50 66.66 36.36 53.84 ...16-121,133-139
Coverage report for `packages/server`
St File % Stmts % Branch % Funcs % Lines Uncovered Line #s
🟡 All files 59.99 53.61 56.7 60.08
🟢  src 81.81 0 50 81.81
🟢   configure.js 81.81 0 50 81.81 47-54,83-85
🟢  src/arpa_reporter 100 71.42 100 100
🟢   configure.js 100 33.33 100 100 34-41
🟢   environment.js 100 100 100 100
🟢   use-request.js 100 100 100 100
🟡  src/arpa_reporter/db 50.73 45.88 50 51.9
🟡   arpa-subrecipients.js 53.19 50 38.46 54.54 23-60,101,113-122
🔴   reporting-periods.js 37.2 46.87 40 38.09 46,77-156
🟢   settings.js 100 83.33 100 100 13
🟡   uploads.js 50 28.57 52.38 51.42 18-29,84,99-124,141-150
🔴  src/arpa_reporter/lib 31.06 33.8 36.14 29.9
🟢   arpa-ec-codes.js 100 100 100 100
🔴   audit-report.js 23.52 21.56 26.56 23.26 ...53-554,579-709,757-783
🟡   ensure-async-context.js 75 100 50 100
🟢   format.js 90.62 90 90 91.3 41-42
🟡   log.js 75 50 50 75 13,25
🟡   preconditions.js 66.66 33.33 100 66.66 3
🔴   spreadsheet.js 9.09 0 0 9.09 15-32
🟢   validation-error.js 85.71 100 50 85.71 16
🔴  src/arpa_reporter/routes 37.32 14.28 13.88 37.84
🔴   agencies.js 22.58 0 0 23.33 13-21,26-53
🟡   application_settings.js 75 100 0 75 10-11
🟡   audit-report.js 68.91 58.33 100 68.91 57-58,64-78,100-116
🟡   exports.js 54.12 55.55 66.66 54.12 62-76,99-100,112-172
🔴   reporting-periods.js 20 0 0 20.43 ...25-137,143-149,154-180
🔴   subrecipients.js 23.8 0 0 23.8 12-13,17-27,31-48,52-63
🔴   uploads.js 28.28 7.89 9.09 29.16 ...33-154,164-166,173-180
🔴   users.js 19.6 0 0 20 15-35,39-44,48-81
🔴  src/arpa_reporter/services 47.13 37.32 52.44 47.48
🔴   generate-arpa-report.js 41.91 7.77 59.25 42.34 ...987,996-1009,1083-1150
🔴   get-template.js 45.94 33.33 40 45.94 18-39,51,60,68-79
🟡   persist-upload.js 69.66 90 73.07 69.76 ...64-206,227-241,279-301
🔴   records.js 20.75 0 11.11 21.15 38-204,221-276
🔴   revalidate-uploads.js 37.5 100 0 37.5 5-14
🔴   validate-upload.js 43.56 58.51 37.03 44.2 ...52,371,393,411-688,703
🟢   validation-rules.js 98.18 90 90.9 100 157,173
🟡  src/db 73.78 70.28 68 73.89
🟢   connection.js 100 50 100 100 6
🟢   constants.js 100 100 100 100
🟡   helpers.js 75 83.33 50 75 5,21-22
🟢   index.js 81.51 76.8 81.15 81.56 ...97-1463,1645-1646,1653
🟢   saved_search_migration.js 92 88.23 71.42 93.61 5,69,134
🔴   tenant_creation.js 10.58 2.7 ...[Comment body truncated]

Copy link

Terraform Summary

Step Result
🖌 Terraform Format & Style
⚙️ Terraform Initialization
🤖 Terraform Validation
📖 Terraform Plan

Hint: If "Terraform Format & Style" failed, run terraform fmt -recursive from the terraform/ directory and commit the results.

Output

Validation Output
stdout:
Success! The configuration is valid.


-------------------------------------
stderr:

Plan Summary
CHANGE RESOURCE
update module.api.aws_ecs_service.default[0]
module.api.module.grant_digest_scheduled_task.aws_iam_role_policy.default[0]
module.api.module.grant_digest_scheduled_task.aws_scheduler_schedule.default[0]
module.arpa_audit_report.aws_ecs_service.default
module.arpa_exporter[0].aws_ecs_service.default
module.arpa_treasury_report.aws_ecs_service.default
module.consume_grants.aws_ecs_service.default
module.website.aws_s3_object.deploy-config[0]
module.website.aws_s3_object.origin_dist_artifact["assets/style-C-Atp7Bz.js.map"]
recreate module.api.aws_ecs_task_definition.default[0]
module.arpa_audit_report.aws_ecs_task_definition.consumer
module.arpa_exporter[0].aws_ecs_task_definition.consumer
module.arpa_treasury_report.aws_ecs_task_definition.consumer
module.consume_grants.aws_ecs_task_definition.consume_grants

Pusher: @TylerHendrickson, Action: pull_request_target, Workflow: Continuous Integration

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request javascript Pull requests that update Javascript code python
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant