How do I use models locally for privacy reasons?

[ErfolgreichCharismatisch]

My code is this

from haystack.nodes import TransformersSummarizer, TransformersTranslator
from haystack import Document

translator = TransformersTranslator(model_name_or_path="path/to/opus-mt-de-en")
summarizer = TransformersSummarizer(model_name_or_path="google/pegasus-xsum",progress_bar=True)

As the models have already been downloaded, I switched off the internet connection(as data to be processed is sensitive) and tried the model with it.

I got

ERROR:posthog:error uploading: HTTPSConnectionPool(host='tm.hs.deepset.ai', port=443): Max retries exceeded with url: /batch/ (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x000001F5B34A3BE0>: Failed to establish a new connection: [Errno 11001] getaddrinfo failed'))
ERROR:posthog:error uploading: HTTPSConnectionPool(host='tm.hs.deepset.ai', port=443): Max retries exceeded with url: /batch/ (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x000001F5B34A3820>: Failed to establish a new connection: [Errno 11001] getaddrinfo failed'))

So I got the error message not only for the relative path whose model has already been downloaded, but also for the absolute path. Also I noticed the error uploading, which confused me, as I don't understand what should be uploaded.

I found solutions, like #486 (comment) but the FARM framework does not accept google/pegasus-xsum, also it is for QA and not for summarization.

How do I use the models locally for privacy reasons?

Maybe @bglearning , @vblagoje or @bogdankostic can help.

[danielbichuetti]

Hi @ErfolgreichCharismatisch !

The upload error you are looking at is caused by telemetry. You can disable it by setting environment variable HAYSTACK_TELEMETRY_ENABLED=False or by calling disable_telemetry().

Haystack telemetry is totally anonymized.

Regarding the Transformers models, you can use locally cached models or manually downloaded, the location will be set/get by env vars:

• TRANSFORMERS_CACHE
• HF_HOME + transformers/.
• XDG_CACHE_HOME + /huggingface/transformers.

But I think that if you don't remove the virtual env or clean these directories, you can use it offline. Remember to turn off telemetry.