Oplogtoredis: Add Configurable Denylist to HTTP Server #64
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
kinda surprised that we're adding this funtionality (presumably for us to ... toggle off/on during some restores or something). Were the other options not possible? |
A full declarative approach doesn't look possible. Hence there must be some kind of communication between our tooling and the service. Given that, this seemed like the least intrusive approach |
alex-goodisman
force-pushed
the
alex.denylist
branch
from
c821a05 to
3a8f826
Compare
alex-goodisman
force-pushed
the
alex.denylist
branch
from
3a8f826 to
1e6ccd5
Compare
torywheelwright
approved these changes
Apr 26, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This adds a new feature- a configurable denylist.
The denylist is a list of rules. Each rule is a list of keys and a regex.
Every incoming oplog message will be checked against every rule. The keys will be used sequentially to index into the oplog message, and if the value found is a string, it will be checked against the regex. If the regex is found for any rule in the denylist, the oplog message will be skipped and will not be reported to Redis.
The denylist can be viewed by HTTP methods (GET /denylist and GET /denylist/:ruleID), and can be configured in the same manner (PUT /denylist and DELETE /denylist/:ruleID).
Default behavior is unchanged, since the denylist is empty.
This is useful when an external context knows that some oplog messages are not necessary to be transferred. The external context can inform Oplogtoredis via the denylist, and then monitor the internal state via the HTTP API.