Saving and Loading embeddings in elasticsearch document store

[MariaDavid30]

Hai, I am trying to update embeddings of approximately 4000 documents in the elastic search document store using the below command.

document_store.update_embeddings(retriever=retriever, update_existing_embeddings=False)

I am running it in my local system(jupyter notebook). I am using a docker container for elastic search. Updating embeddings is taking hours to run. Is there a method to save and load the embeddings the subsequent times.

Requesting help!

[anakin87]

Maybe this comment answers your questions.
Let us know if you need further help.

[MariaDavid30]

Hey @anakin87 I've seen the comment. Does the comment mean that it will not take the same amount of time the next time I run the "document_store.update_embeddings(retriever=retriever, update_existing_embeddings=False)"?

[anakin87]

I think you are right.
Furthermore, if you reload the same document store (already containing embeddings), you don't need to call update_embeddings, unless you have new documents to encode.

[ryanholtschneider2]

How do you reload a elasticsearch document store?
I was trying to get my embeddings from a previous run of a docker container by saving the elasticsearch files to a local volume, but I cannot seem to reload them -

Here is the command I was running -

docker run -d -p 9200:9200 -e "discovery.type=single-node" -v $(pwd)/esdata:/usr/share/elasticsearch/data elasticsearch:7.17.6

[MariaDavid30]

Thank you @anakin87, got it.