Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: include commas in the profile size, handle failure case of profile more than 500kB in size #5241

Closed
wants to merge 25 commits into from

Conversation

yashasvibajpai
Copy link
Member

@yashasvibajpai yashasvibajpai commented Oct 28, 2024

Description

On debugging for the issue, we came across the two pointers addressed here.

  1. commas in JSON were unaccounted for in the final size of payload.
  2. we will throw an error in case a single profile's size itself crosses the 500 kb threshold set by Klaviyo themselves. Reference

Linear Ticket

Related to INT-2820, INT-2840

Security

  • The code changed/added as part of this pull request won't create any security issues with how the software is being used.

@yashasvibajpai yashasvibajpai self-assigned this Oct 28, 2024
@yashasvibajpai yashasvibajpai requested review from koladilip and ItsSudip and removed request for koladilip October 28, 2024 11:27
Copy link

codecov bot commented Oct 28, 2024

Codecov Report

Attention: Patch coverage is 48.93617% with 96 lines in your changes missing coverage. Please review.

Project coverage is 73.58%. Comparing base (94aa254) to head (25fe1a5).

Files with missing lines Patch % Lines
...destinationmanager/klaviyobulkupload/apiService.go 12.76% 82 Missing ⚠️
...tionmanager/klaviyobulkupload/klaviyobulkupload.go 80.00% 8 Missing and 2 partials ⚠️
warehouse/internal/mocks/utils/mock_uploader.go 85.71% 4 Missing ⚠️
Additional details and impacted files
@@                Coverage Diff                 @@
##           release/1.38.x    #5241      +/-   ##
==================================================
+ Coverage           73.40%   73.58%   +0.18%     
==================================================
  Files                 424      425       +1     
  Lines               60243    59965     -278     
==================================================
- Hits                44221    44126      -95     
+ Misses              13578    13373     -205     
- Partials             2444     2466      +22     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@yashasvibajpai yashasvibajpai changed the title fix: include commas in the profile size, handle failure case of profile more than 5mb in size fix: include commas in the profile size, handle failure case of profile more than 500kB in size Nov 6, 2024
@@ -333,6 +338,15 @@ func (kbu *KlaviyoBulkUploader) Upload(asyncDestStruct *common.AsyncDestinationS
return kbu.generateKlaviyoErrorOutput("Error while parsing JSON.", err, importingJobIDs, destinationID)
}
profileStructure := kbu.ExtractProfile(input)
// if profileStructure length is more than 5 mb, throw an error
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
// if profileStructure length is more than 5 mb, throw an error
// if profileStructure length is more than 500 kB, throw an error

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updated

@yashasvibajpai yashasvibajpai changed the base branch from master to release/1.38.x November 15, 2024 05:47
@github-actions github-actions bot deleted the fix.klaviyo.bulk.upload.rate.cfd branch January 19, 2025 02:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants