Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix app crash on big csv downloads #909

Merged
merged 7 commits into from
Aug 10, 2023
Merged

Fix app crash on big csv downloads #909

merged 7 commits into from
Aug 10, 2023

Conversation

echaidemenos
Copy link
Collaborator

Chunks dates into 6 month intervals while creating the CSV. Appends each chunk to a temporally file, which will be sent as the response

Copy link

@pullrequest pullrequest bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ This pull request was sent to the PullRequest network.


Check the status or cancel PullRequest code review here.

Copy link

@pullrequest pullrequest bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PullRequest Breakdown

Reviewable lines of change

+ 313
- 98

79% TypeScript
11% TSX
9% Jest Snapshot (tests)
1% TypeScript (tests)
<1% JSON

Generated lines of change

+ 10
- 0

Type of change

Fix - These changes are likely to be fixing a bug or issue.

Copy link

@pullrequest pullrequest bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a great example of unbounded work causing issues, as per my notes on the previous PR that implemented this stuff. Even with this mitigations in place, I'd still recommend applying limits to restrict range size. Probably there is a maximum filesize supported by your load balancer, also? Other than this, the changes here are looking good, other than the notes left inline for your consideration.

Image of Graham C Graham C


Reviewed with ❤️ by PullRequest

packages/api/src/time-series/time-series.service.ts Outdated Show resolved Hide resolved
packages/api/src/utils/time-series.utils.ts Outdated Show resolved Hide resolved
@github-actions
Copy link

github-actions bot commented Aug 4, 2023

Build succeeded and deployed at https://aqualink-app-909.surge.sh
(hash 9808d33 deployed at 2023-08-04T10:21:17)

Copy link

@pullrequest pullrequest bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PullRequest reviewed the updates made to #909 up until the latest commit (9808d33). No further issues were found.

Reviewed by:

Image of Graham C Graham C

@@ -37,20 +30,16 @@ const Header = ({ site }: HeaderProps) => {
You can find example file formats here:{' '}
{exampleFiles.map((file, i) => (
<span key={file}>
<Button
<a
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why did we need to make this change?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

these are some static files we point to, to download. It is just more straightforward to create a link with the file url, than to add custom download logic

Copy link

@pullrequest pullrequest bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PullRequest reviewed the updates made to #909 since our last review was posted. This includes comments that have been posted by non-PullRequest reviewers. No further issues were found.

Reviewed by:

Image of Graham C Graham C

@ericboucher ericboucher merged commit 7af5526 into master Aug 10, 2023
3 checks passed
@ericboucher ericboucher deleted the large-csv branch August 10, 2023 07:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants