Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Mishandling of headers timeout when sending out a big body #186

Open
vrurg opened this issue Aug 5, 2023 · 0 comments
Open

Mishandling of headers timeout when sending out a big body #186

vrurg opened this issue Aug 5, 2023 · 0 comments

Comments

@vrurg
Copy link
Contributor

vrurg commented Aug 5, 2023

The setup (encountered while implementing uploads for Google Cloud Storage API):

  1. Instantiated Cro::HTTP::Client, HTTP 1.1, not persistent.
  2. A file of at least few hundred megabytes in size. What is considered 'sufficiently big' depends on connection bandwidth.
  3. Best observed with request body implemented as a supply reading and emitting file's data in chunks.
  4. Setting the timeout to an unreasonable low value helps with observing the problem.

What happens:

  1. The headers timeout is setup before the request is sent, i.e. before $pipeline.send-request($request-object) is done.
  2. Since the outgoing transfer takes really long the time out goes off before the entire body sending is completed.
  3. I'm not sure what exactly happens internally at this point, but the exception thrown doesn't cause the body supply to be aborted. An attempt to re-use the same instance of Cro::HTTP::Client for another request either resumes the previous request sending its data, or simply await until the sending is complete. Either way, I observe the body supply to continue its read/emit cycle.
  4. The second request normally gets aborted with a consequent timeout exception.

So far, I'm trying to figure out a workaround which doesn't require re-instantiating the client per each API request. Instantiation is needed because Google Cloud API specifics make it necessary to remove any body serializers/parsers except for blob and supply fallbacks. Lack of knowing of Cro's guts and lack of time doesn't let me to debug deeper and, perhaps, find a solution for this. But overall the problem makes a lot of heavy-weight use cases very tricky to implement and might result users rejecting Cro as the tool of their choice.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant