Use existing Weaviate Schema in DocumentStore

[janwika]

Is there a way to use an existing Weaviate instance with documents stored as a Haystack DocumentStore? It appears that you can only instantiate a new schema through WeaviateDocumentStore or am i not seeing something?

[tstadel]

@janwika in order to reuse an existing instance just pass the name of your weaviate class to the index param of WeaviateDocumentStore.

[heiningair]

Hi,
I have the same issue. The WeaviateDocumentStore is not working when trying to connect to an existing Weaviate instance.

I load the store like this:

document_store = WeaviateDocumentStore(
                host='http://local.weaviate.instance',
                embedding_dim=1536,
                port=80,
                index="Text",
                content_field="text",
                recreate_index=False,
                custom_schema=custom_schema
            )

When I then try to retrieve documents with it, it fails and throws an exception in the _convert_weaviate_result_to_document() function.
It turns out that in line 321 it says:

if props.get(self.content_field) is not None:
            # Converting JSON-string to original datatype (string or nested list)
            content = json.loads(str(props.get(self.content_field))) 

Meaning, whenever a content_field is set (which is always the case) then it assumes that the actual content is a json value. This breaks the code if the content isnt a json value.

Is there any example out there, on how to configure a WeaviateVectorStore that is working with an existing instance of weaviate?

[ZanSara]

@heiningair it can only work if your documents were written to Weaviate by Haystack as well, so unless you wrote your documents to Weaviate in the format Haystack expects, there's no good way to make it work.

[heiningair]

@ZanSara actually it is just as much as replacing this line (l.no.:321)

content = json.loads(str(props.get(self.content_field)))

with this

content = props.get(self.content_field)

I simply created a CustomWeaviateDocumentStore for now. I know this is just a workaround. But this could easily be made configurable.
It just makes sense to let the CustomWeaviateDocumentStore work with existing Weaviate instances. There should also be a mention of it ("weaviate store not working for existing instances") in the documentation.

For now I will use this workaround. But if you want, I'd be happy to create a PR for it, so others can also profit from that. While my search for the workaround I came across others looking for a solution for that.