Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add retry utility for unprocessed items in batch writes #1287

Merged
merged 3 commits into from
Jan 9, 2025

Conversation

pvlugter
Copy link
Contributor

@pvlugter pvlugter commented Jan 7, 2025

Refs #1286

Make the retry mechanism generic, and add a specific utility method for batch writes. Use this method internally.

Copy link
Member

@patriknw patriknw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looking good

minBackoff: Duration,
maxBackoff: Duration,
randomFactor: Double,
onRetry: Procedure3[BatchWriteItemResponse, Integer, Duration],
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What should this callback be used for, other than logging? shall we have a convenient variant without the callback parameter and log by default in that case? Maybe that convenience should also create the exception by default, so without failOnMaxRetries parameter.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, we probably don't need the flexible callbacks for the convenience method at all. Will build in the logging and exception.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated in 70f0100. Always provide logging and the exception for the convenience batch writes. Move common parts to share across java and scala. Rename to a more general Requests.batchWriteWithRetries, if there are other request utilities that we end up providing.

Copy link
Member

@patriknw patriknw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM after internal api markers


private[akka] val log: Logger = LoggerFactory.getLogger(getClass)

private[akka] def batchUnprocessedTotal(response: BatchWriteItemResponse): Int =
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Internal api markers on the private[akka]

/**
 * INTERNAL API
 */
@InternalApi

@patriknw patriknw merged commit 3d8f3ec into main Jan 9, 2025
22 checks passed
@patriknw patriknw deleted the retry-batch-writes branch January 9, 2025 06:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants