Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

api: support skipping record when bulk indexing #92

Open
slint opened this issue Nov 26, 2018 · 0 comments
Open

api: support skipping record when bulk indexing #92

slint opened this issue Nov 26, 2018 · 0 comments

Comments

@slint
Copy link
Member

slint commented Nov 26, 2018

It should be possible to skip/abort indexing for a record during the RecordIndexer._prepare_record stage. There are two options for that, both involving behavior in the receivers of the before_record_index signal:

  • In the receiver, raise a standardized exception (e.g. raise SkipIndexing()) which will be handled of course appropriately in the _prepare_record method. This solution is backward compatible but uses exceptions (which are slow-ish)
  • In the receiver, return None, or an empty dictionary ({}). The result of the ._prepare_record should be checked in the .index and ._index_action methods if the resulting data is empty. The problem with this solution is that current implementations of receivers are probably not aware that the passed json dictionary parameter might end up being empty...

The use case behind this is that it might be e.g. that a record gets sent for bulk indexing, but is later (soft)deleted while it's still in the bulk indexing queue. When the bulk indexer consumer runs, it should skip this record.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant