Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Register call_requests in background at metatx #35

Merged
merged 9 commits into from
May 8, 2024

Conversation

kompotkot
Copy link
Collaborator

@kompotkot kompotkot commented May 1, 2024

During operations with waggle service we notices few bottlenecks and problems:

  • Long lists of call requests force users to split them into multiple lists (<400) and require us to increase the nginx timeout to a high value
  • There is no detailed exceptions if few call request IDs already presented in database

With new PR moonstream-to/api#1062 for metatx and this one for waggle next improvements:

  • Before pushing call requests - make a check if this call requests represented in database (if "metatx" set in request body)
  • Track jobs of adding call_request batches with Bugout journals
  • Response to user not waiting background jobs

Possible "metatx" values:

  • no_metatx - do not push call_requests to journal, sign it and return back
  • no_check_metatx - do not check existing call_requests before push to metatx

Additional response fields:

  • job_entry_id - uuid of entry of job
  • job_entry_url - full url to job in spire

image

@kompotkot
Copy link
Collaborator Author

kompotkot commented May 5, 2024

@bugout-dev check

  • Create journal metatx-jobs

Copy link

@kellan-simiotics kellan-simiotics left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lg

@@ -292,15 +299,15 @@ func (c *BugoutAPIClient) FindGroup(token, groupId string) (Group, int, error) {
var requestBodyBytes []byte
request, requestErr := http.NewRequest("GET", fmt.Sprintf("%s/group/find?group_id=%s", c.BroodBaseURL, groupId), bytes.NewBuffer(requestBodyBytes))
if requestErr != nil {
return group, 500, requestErr
return group, 0, requestErr

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why are we sending a status code 0 on error?

Copy link
Collaborator Author

@kompotkot kompotkot May 7, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I want to pass reasonable status codes back to client, so we need to proxy it. When getting 0, it means failed on waggle side, not from other API

}
request.Header.Add("Authorization", fmt.Sprintf("Bearer %s", token))
request.Header.Add("Accept", "application/json")
request.Header.Add("Content-Type", "application/json")

response, responseErr := c.HTTPClient.Do(request)
if responseErr != nil {
return group, 500, requestErr
return group, 0, requestErr

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same question as above.

@kompotkot kompotkot merged commit eed0b4a into main May 8, 2024
1 check passed
@kompotkot kompotkot deleted the backgound-register-call-requests branch May 8, 2024 08:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants