Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Transactions Download functionality Improvements #10211

Merged
merged 43 commits into from
Feb 7, 2025

Conversation

jessy-p
Copy link
Contributor

@jessy-p jessy-p commented Jan 22, 2025

Fixes #10255 which is part of the Epic #9969

Summary

The PR moves Transaction exports to a single implementation via Transact Platform Service API. This does away with the redundant JS based export for a single page export, and email + server export for larger data, that led to data inconsistency and additional maintenance overhead. With this PR, the merchant gets a browser download via an async service request, if it takes within 6 seconds ( 1 + 6 retries at 1 second interval ). There is always an email sent with the export file as a backup. If the file takes longer than 6 seconds to generate, the merchant will anyway get it on email.

For security reasons, we are unable to store the export file in the uploads folder on the server. We were guided by the systems team to asynchronously poll the job id of the export and use that to get the URL of the exported report.

Changes proposed in this Pull Request

  • removes the @woocommerce/csv-export based JS export on the Transactions page
  • Uses the server transactions/download endpoint for all downloads instead.
  • The export API returns an export_id
  • Polls the transactions/download/<export_id> endpoint to get the download URL.
  • If the file is available within 6 retries ( 6 seconds) , the file is downloaded immediately and additionally emailed to the merchant. Else, it will be emailed to the merchant once the file gets ready.

By removing the browser export and using only server based export, the functionality is simplified and made consistent. We also preserve the functionality for the merchant by making the file available immediately in the browser whenever possible.

Testing instructions

  • The necessary API changes to the Transact Platform Service is already merged

  • Checkout this branch on Pressable test site via Jetpack Beta, or Local + Sandbox.

  • Browse to Payments > Transactions

  • Click the Export button in the top right corner of the report list

  • You should see a snack bar notice with copy and appearance similar to this screenshot:
    image

  • Download various numbers of rows or filtered rows and check that the file is downloaded. An email is also sent to the merchant with the download URL.

  • If the cron job is not running/slow, i.e the file is not available, the file will not be downloaded and a message displayed that the file will be emailed. When the job runs, the email is sent.

Here is a video of the complete flow ( please watch in full screen since the snack bar notice is obscured by the player controls in small screen ):

Screen.Recording.2025-02-07.at.8.51.30.PM.mov

  • Run npm run changelog to add a changelog file, choose patch to leave it empty if the change is not significant. You can add multiple changelog files in one PR by running this command a few times.
  • Covered with tests (or have a good reason not to test in description ☝️)
  • Tested on mobile (or does not apply)

Post merge

@jessy-p jessy-p marked this pull request as draft January 22, 2025 01:17
@botwoo
Copy link
Collaborator

botwoo commented Jan 22, 2025

Test the build

Option 1. Jetpack Beta

  • Install and activate Jetpack Beta.
  • Use this build by searching for PR number 10211 or branch name update/9969-poll-async-job in your-test.site/wp-admin/admin.php?page=jetpack-beta&plugin=woocommerce-payments

Option 2. Jurassic Ninja - available for logged-in A12s

🚀 Launch a JN site with this branch 🚀

ℹ️ Install this Tampermonkey script to get more options.


Build info:

  • Latest commit: 0545283
  • Build time: 2025-02-07 16:07:42 UTC

Note: the build is updated when a new commit is pushed to this PR.

Copy link
Contributor

github-actions bot commented Jan 22, 2025

Size Change: +492 B (0%)

Total Size: 1.28 MB

Filename Size Change
release/woocommerce-payments/dist/index.js 232 kB +383 B (0%)
release/woocommerce-payments/dist/multi-currency-switcher-block.js 61.1 kB +21 B (0%)
release/woocommerce-payments/dist/multi-currency.js 57.9 kB +30 B (0%)
release/woocommerce-payments/dist/order.js 42.5 kB +24 B (0%)
release/woocommerce-payments/dist/payment-gateways.js 39 kB +31 B (0%)
release/woocommerce-payments/dist/settings.js 223 kB +3 B (0%)
ℹ️ View Unchanged
Filename Size
release/woocommerce-payments/assets/css/admin.css 1.37 kB
release/woocommerce-payments/assets/css/admin.rtl.css 1.37 kB
release/woocommerce-payments/assets/css/success.css 189 B
release/woocommerce-payments/assets/css/success.rtl.css 190 B
release/woocommerce-payments/dist/blocks-checkout-rtl.css 2.66 kB
release/woocommerce-payments/dist/blocks-checkout.css 2.66 kB
release/woocommerce-payments/dist/blocks-checkout.js 55.2 kB
release/woocommerce-payments/dist/cart-block.js 17.2 kB
release/woocommerce-payments/dist/cart.js 5.73 kB
release/woocommerce-payments/dist/checkout-rtl.css 1.27 kB
release/woocommerce-payments/dist/checkout.css 1.27 kB
release/woocommerce-payments/dist/checkout.js 34.5 kB
release/woocommerce-payments/dist/express-checkout-rtl.css 236 B
release/woocommerce-payments/dist/express-checkout.css 236 B
release/woocommerce-payments/dist/express-checkout.js 15.7 kB
release/woocommerce-payments/dist/frontend-tracks.js 854 B
release/woocommerce-payments/dist/index-rtl.css 35.5 kB
release/woocommerce-payments/dist/index.css 35.5 kB
release/woocommerce-payments/dist/multi-currency-analytics.js 1.08 kB
release/woocommerce-payments/dist/multi-currency-rtl.css 4.29 kB
release/woocommerce-payments/dist/multi-currency.css 4.29 kB
release/woocommerce-payments/dist/order-rtl.css 740 B
release/woocommerce-payments/dist/order.css 740 B
release/woocommerce-payments/dist/payment-gateways-rtl.css 1.34 kB
release/woocommerce-payments/dist/payment-gateways.css 1.34 kB
release/woocommerce-payments/dist/plugins-page-rtl.css 386 B
release/woocommerce-payments/dist/plugins-page.css 386 B
release/woocommerce-payments/dist/plugins-page.js 20.1 kB
release/woocommerce-payments/dist/product-details-rtl.css 433 B
release/woocommerce-payments/dist/product-details.css 436 B
release/woocommerce-payments/dist/product-details.js 12.5 kB
release/woocommerce-payments/dist/settings-rtl.css 11.5 kB
release/woocommerce-payments/dist/settings.css 11.4 kB
release/woocommerce-payments/dist/subscription-edit-page.js 703 B
release/woocommerce-payments/dist/subscription-product-onboarding-modal-rtl.css 524 B
release/woocommerce-payments/dist/subscription-product-onboarding-modal.css 524 B
release/woocommerce-payments/dist/subscription-product-onboarding-modal.js 20.2 kB
release/woocommerce-payments/dist/subscription-product-onboarding-toast.js 730 B
release/woocommerce-payments/dist/subscriptions-empty-state-rtl.css 120 B
release/woocommerce-payments/dist/subscriptions-empty-state.css 120 B
release/woocommerce-payments/dist/subscriptions-empty-state.js 19.3 kB
release/woocommerce-payments/dist/tokenized-express-checkout-rtl.css 236 B
release/woocommerce-payments/dist/tokenized-express-checkout.css 236 B
release/woocommerce-payments/dist/tokenized-express-checkout.js 16.6 kB
release/woocommerce-payments/dist/tos-rtl.css 235 B
release/woocommerce-payments/dist/tos.css 235 B
release/woocommerce-payments/dist/tos.js 21.8 kB
release/woocommerce-payments/dist/woopay-direct-checkout.js 6.13 kB
release/woocommerce-payments/dist/woopay-express-button.js 23.3 kB
release/woocommerce-payments/dist/woopay-rtl.css 4.31 kB
release/woocommerce-payments/dist/woopay.css 4.28 kB
release/woocommerce-payments/dist/woopay.js 71 kB
release/woocommerce-payments/includes/subscriptions/assets/css/plugin-page.css 625 B
release/woocommerce-payments/includes/subscriptions/assets/js/plugin-page.js 814 B
release/woocommerce-payments/vendor/automattic/jetpack-assets/build/i18n-loader.js 2.46 kB
release/woocommerce-payments/vendor/automattic/jetpack-assets/build/jetpack-script-data.js 772 B
release/woocommerce-payments/vendor/automattic/jetpack-assets/src/js/i18n-loader.js 1.02 kB
release/woocommerce-payments/vendor/automattic/jetpack-assets/src/js/script-data.js 69 B
release/woocommerce-payments/vendor/automattic/jetpack-connection/babel.config.js 163 B
release/woocommerce-payments/vendor/automattic/jetpack-connection/dist/identity-crisis.css 2.47 kB
release/woocommerce-payments/vendor/automattic/jetpack-connection/dist/identity-crisis.js 14.2 kB
release/woocommerce-payments/vendor/automattic/jetpack-connection/dist/identity-crisis.rtl.css 2.47 kB
release/woocommerce-payments/vendor/automattic/jetpack-connection/dist/jetpack-connection.css 10 kB
release/woocommerce-payments/vendor/automattic/jetpack-connection/dist/jetpack-connection.js 28.4 kB
release/woocommerce-payments/vendor/automattic/jetpack-connection/dist/jetpack-connection.rtl.css 10 kB
release/woocommerce-payments/vendor/automattic/jetpack-connection/dist/jetpack-sso-admin-create-user.css 198 B
release/woocommerce-payments/vendor/automattic/jetpack-connection/dist/jetpack-sso-admin-create-user.js 280 B
release/woocommerce-payments/vendor/automattic/jetpack-connection/dist/jetpack-sso-admin-create-user.rtl.css 198 B
release/woocommerce-payments/vendor/automattic/jetpack-connection/dist/jetpack-sso-login.css 625 B
release/woocommerce-payments/vendor/automattic/jetpack-connection/dist/jetpack-sso-login.js 333 B
release/woocommerce-payments/vendor/automattic/jetpack-connection/dist/jetpack-sso-login.rtl.css 626 B
release/woocommerce-payments/vendor/automattic/jetpack-connection/dist/jetpack-sso-users.js 424 B
release/woocommerce-payments/vendor/automattic/jetpack-connection/dist/tracks-ajax.js 521 B
release/woocommerce-payments/vendor/automattic/jetpack-connection/dist/tracks-callables.js 585 B
release/woocommerce-payments/vendor/automattic/jetpack-connection/src/sso/jetpack-sso-admin-create-user.css 215 B
release/woocommerce-payments/vendor/automattic/jetpack-connection/src/sso/jetpack-sso-admin-create-user.js 521 B
release/woocommerce-payments/vendor/automattic/jetpack-connection/src/sso/jetpack-sso-login.css 721 B
release/woocommerce-payments/vendor/automattic/jetpack-connection/src/sso/jetpack-sso-login.js 412 B
release/woocommerce-payments/vendor/automattic/jetpack-connection/src/sso/jetpack-sso-users.js 632 B
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/css/about.css 1.04 kB
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/css/admin-empty-state.css 294 B
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/css/admin-order-statuses.css 408 B
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/css/admin.css 3.59 kB
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/css/checkout.css 301 B
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/css/modal.css 746 B
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/css/view-subscription.css 574 B
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/css/wcs-upgrade.css 414 B
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/js/admin/admin-pointers.js 543 B
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/js/admin/admin.js 9.4 kB
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/js/admin/jstz.js 6.78 kB
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/js/admin/jstz.min.js 3.84 kB
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/js/admin/meta-boxes-coupon.js 545 B
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/js/admin/meta-boxes-subscription.js 2.52 kB
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/js/admin/moment.js 22.2 kB
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/js/admin/moment.min.js 11.7 kB
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/js/admin/payment-method-restrictions.js 1.29 kB
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/js/admin/wcs-meta-boxes-order.js 507 B
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/js/frontend/payment-methods.js 358 B
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/js/frontend/single-product.js 428 B
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/js/frontend/view-subscription.js 1.38 kB
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/js/frontend/wcs-cart.js 782 B
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/js/modal.js 1.09 kB
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/js/wcs-upgrade.js 1.26 kB
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/build/index.css 391 B
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/build/index.js 3.04 kB

compressed-size-action

@jessy-p jessy-p changed the title Draft PR: polling and fallback implementation for Transactions export job Remove JS based export and poll for Transactions CSV from Transactions Export job. Jan 22, 2025
@jessy-p jessy-p changed the title Remove JS based export and poll for Transactions CSV from Transactions Export job. Transactions Download functionality Improvements Jan 22, 2025
- The response is changed to an array
- If the file times out, don't show an error. The file may be emailed.
@jessy-p jessy-p marked this pull request as ready for review January 23, 2025 02:28
@Jinksi Jinksi requested review from Jinksi and a team January 23, 2025 05:25
@Jinksi Jinksi removed the request for review from a team January 23, 2025 05:26
Jessy and others added 5 commits February 5, 2025 11:49
- Removed the file URL check that blocks local testing
- Changed isDownloading to isExportInProgress
- Removed the track event as it is not user triggered.
@shendy-a8c
Copy link
Contributor

7e10a7d fixes unit test due to changes from d7e1f6d.

@Jinksi
Copy link
Member

Jinksi commented Feb 6, 2025

✅ Failing e2e test is unrelated, due to the removal of the date settings notice in #10129:

image

playwright-report.zip (from this test run)

Copy link
Member

@Jinksi Jinksi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This tests well! I've tested it on a Pressable site against with various filters enabled, all is working smoothly. It feels magic to have a transact-platform-server generated CSV appear so instantly as a client-side download 🌟

Flicking my network off immediately after clicking download, to force the polling to fail, results in the CSV still being emailed, and the UI gracefully cancelling the polling process ✅

I've added some minor commits that:

  • Remove unused sprintf import in cfead56
  • Add extra documentation for export response interfaces in b068840: I was getting confused reading ExportResponse and ExportURLResponse and thought that we can make it clearer what these response objects contain and their purpose in the export flow.
  • Add isBusy prop to DownloadButton to show loading state during export in b52e61f - this provides better user feedback while the export is being processed, following WP component patterns for loading states. If this change is incorrect, e.g. I missed a discussion about this already, please feel free to revert it 👍
Button.isBusy.mov

Pulling some thoughts from our discussion in Slack (p1738811085595819-slack-C02BW3Z8SHK):

Do you think the notice stays for too much time ?

No, I think it is a sufficient amount of time for the merchant to read the message, and the Snackbar can be dismissed on click.

Is the polling working reliably (just once, I felt like it polled once, and failed - worked all the other times). I haven't changed anything about it, so not sure if it was just a issue with my build that one time.

It has been reliable for me when testing on a Pressable site 👍

Significance: minor
Type: update

Replaced JS export with downloading Transactions CSV from service.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe this could be more descriptive, to explain the changes from a merchant's perspective.

Suggested change
Replaced JS export with downloading Transactions CSV from service.
Enhanced transaction CSV exports with async transact platform service processing, improving the CSV export experience.

(My attempt, please modify or ignore 😉)

Copy link
Contributor

@nagpai nagpai Feb 7, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated the change log with hints from what you have shared, Eric - ef7b796

@haszari
Copy link
Contributor

haszari commented Feb 6, 2025

Add isBusy prop to DownloadButton to show loading state during export in b52e61f - this provides better user feedback while the export is being processed, following WP component patterns for loading states.

🙇🏻 Thank you @Jinksi !

Copy link
Contributor

@haszari haszari left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Quick review - tested locally only. Looking good, we can maybe polish the flow a little more but we can do that as follow up.

  • In my first test, the download didn't happen, and felt strange when the button was still spinning but the snackbar had disappeared. Would be nice if we can sync them, but this is maybe an edge case.
  • The snackbar is a little sad, it should be be gently triumphant!
Screenshot 2025-02-07 at 3 57 35 PM

__(
'There was a problem generating your export.',
'Processing your export. The file will download automatically and will be emailed to %s',
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This wording feels a little sad! Not a blocker but here are some ideas:

  • Can we add a symbol / emoji to make it more happy / triumphant?
  • Add a full stop at end of sentence so it doesn't dangle
  • If possible, might be helpful to use different formatting for the email (bold or something).

✅ Now processing your export. The file will automatically download and will be emailed to [email protected].

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have added an icon and full stop. I was unable to add a bold font face for the email, since createNotice supports only string.

image

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmmm @haszari @nagpai @jessy-p I feel like this first part of the message sounds a bit weird.. could we say: "We’re processing your export. The file will download automatically and be emailed to [email]."

In terms of the celebratory symbol and layout, could we remove the side padding and keep the style consistent with the original component without diverging too much? If we want, we could add 🚀 or 🎉 inline instead.

For example:

We’re processing your export. 🎉 The file will download automatically and be emailed to [email].
or
We’re processing your export. The file will download automatically and be emailed to [email]. 🚀

I'm just thinking ✅ means finished. But if you want to keep it, then please, could it be inline?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We’re processing your export. 🎉 The file will download automatically and be emailed to [email].

Sounds good to me, thanks for following up and refining this @rogermattic !

Also works without the emoji, if that's preferable:

We’re processing your export. The file will download automatically and be emailed to [email].

@nagpai
Copy link
Contributor

nagpai commented Feb 7, 2025

Since we have approvals, and no more blocking feedback remains, I am merging this branch. It will also help speed up Disputes and Payout exports. If any major bugs are found, we can reopen.

@nagpai nagpai added this pull request to the merge queue Feb 7, 2025
Merged via the queue into develop with commit e18716b Feb 7, 2025
25 checks passed
@nagpai nagpai deleted the update/9969-poll-async-job branch February 7, 2025 16:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Transaction CSV exports should use export functionality from server
8 participants