Add a cap on total queue files #20
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This commit adds a `max_disk_files` option to `channel_with_explicit_capacity` that allows the caller to set the total amount of on-disk consumption. The caller may choose to loop on a send which results in `hopper::Error::Full` or drop the data and try again later. Hopper will _not_ block sends when there is no space available -- in memory or disk -- to store the item. This work is done in the service of supporting postmates/cernan#411 and postmates/cernan#412. Signed-off-by: Brian L. Troutwine <[email protected]>
ekimekim
reviewed
Feb 17, 2018
tsantero
suggested changes
Feb 20, 2018
README.md
Outdated
|
|
||
| `hopper::channel_with_max_bytes` takes a `max_disk_files` argument, defining the | ||
| total number of overflow files that can exist concurrently. If all memory and | ||
| disk buffers are full sends into the queue will fail but the error result is |
There was a problem hiding this comment.
the sentence beginning If all memory... is difficult to read. the verbal noun sends can be confusing. i'd also strongly recommend breaking the error result subordinate clause into it's own sentence.
There was a problem hiding this comment.
how about are full, sends into the queue fill fail. However, the error result is implemented such that the caller...
This clarifies the noun-ness of sends by explicitly breaking the if clause into predicate and effect, and avoids the word "written" which can be ambiguous when used to mean 'how the code is written' while talking about a system that writes to disk.
The language in the README was a touch confusing. If it's still confusing it's no longer confusing in the same way. Signed-off-by: Brian L. Troutwine <[email protected]>
tsantero
approved these changes
Feb 20, 2018
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This commit adds a
max_disk_filesoption tochannel_with_explicit_capacitythat allows the caller to set the total amount of on-disk consumption. The caller may choose to loop on a send which results inhopper::Error::Fullor drop the data and try again later. Hopper will not block sends when there is no space available -- in memory or disk -- to store the item.This work is done in the service of supporting postmates/cernan#411 and postmates/cernan#412.
Signed-off-by: Brian L. Troutwine [email protected]