Why is delete_documents slow?

[thobson]

I'm not sure if I've missed something but assuming I have a single trivial doc in my elasticsearch store:

"_source": {
    "content": "the queen of England is Elizabeth 2",
    "content_type": "text",
    "folder": "root",
    "_split_id": 0
}

This call takes around 2 seconds:

filters = {'folder': 'root'}
document_store.delete_documents(filters=filters)

Whereas the corresponding elastic search query:

http://localhost:9200/documents/_delete_by_query

{
  "query": {
    "bool": {
      "must": [
          { "term": {"folder": "root" } }
      ]
    }
  }
}

takes only 6 ms

Any ideas?

[thobson]

I think I might have worked out what's going on.

It seems it's the refresh_type='wait_for' that's causing the issue

Now I need to work out why wait_for is so slow...

[danielbichuetti]

Hi @thobson!

This parameter refers to a parameter into Elasticsearch which sets how the changes are made visible.
When you choose “wait_for” you are telling that you want a reply only when changes are fully applied. This time is completely dependent on your cluster setup.

wait_for
Wait for the changes made by the request to be made visible by a refresh before replying. This doesn’t force an immediate refresh, rather, it waits for a refresh to happen.

Further reference:
https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-refresh.html#docs-refresh

[thobson]

Thanks @danielbichuetti

[ErfolgreichCharismatisch]

Hi @danielbichuetti Can you reply to #3235 ?