Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add options for filtering by database and/or collection #8

Open
benweissmann opened this issue Mar 23, 2018 · 2 comments
Open

Add options for filtering by database and/or collection #8

benweissmann opened this issue Mar 23, 2018 · 2 comments
Labels
good first issue Good for newcomers help wanted Extra attention is needed performance

Comments

@benweissmann
Copy link
Contributor

We should think about the different ways that someone might want to use this + redis-oplog and provide configuration options for limiting how much of the oplog is consumed. Some ideas:

  • The default configuration of redis-oplog assumes that there's only one database on the MongoDB server that's being used with redis-oplog. Maybe oplogtoredis should have a matching "single database" mode where we publish redis messages without the database name, so this simple use-case can be accommodated without using the globalChannelPrefix option in redis-oplog

  • The MongoDB server might have other unrelated databases on it, and we should let users whitelist or blacklist databases that we don't need to process

  • The MongoDB database might have some collections that don't need real-time updates or have very high write throughputs that we can't support. We should let users whitelist or blacklist specific collections as needed.

However we handle this, we should provide guidance on how to appropriately configure oplogtoredis and redis-oplog so they work together correctly.

@benweissmann benweissmann added this to the 0.1 milestone Mar 23, 2018
@benweissmann benweissmann removed this from the 0.1 milestone May 8, 2018
@benweissmann benweissmann added help wanted Extra attention is needed good first issue Good for newcomers labels May 8, 2018
@benweissmann
Copy link
Contributor Author

Removing from 0.1 milestone -- will come back to this later.

@radekmie
Copy link

radekmie commented Dec 3, 2021

Maybe oplogtoredis should have a matching "single database" mode where we publish redis messages without the database name, so this simple use-case can be accommodated without using the globalChannelPrefix option in redis-oplog

Does that mean that oplogtoredis listens to all of the databases in a cluster once it's set up? It's not clearly stated in the docs but it's an important note.

For example, we do have multiple environments hosted in a single cluster, and we planned to have two oplogtoredis instances for each database. If my assumptions about the above are true, we'd only need one, right?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
good first issue Good for newcomers help wanted Extra attention is needed performance
Projects
None yet
Development

No branches or pull requests

3 participants