Unstructured chunking error unpacking response, missing languages field

[tomlynchRNA]

Using local unstructured, not an error for every point

[2024-09-13T00:49:34Z ERROR vector_db_proxy::data::processing_incoming_messages] Error embedding response from unstructuredIO. Er
ror: An error occurred while unpacking the successful response. Error: reqwest::Error { kind: Decode, source: Error("missing fiel
d `languages`", line: 1, column: 157) }

chunking config for the datasource (default):

test> db.datasources.find({name:'chunked'}, {status:1, recordCount:1, chunkingConfig:1})
[
  {
    _id: ObjectId('66e38a9aa07c1acf3db1db15'),
    status: 'ready',
    recordCount: { total: 0 },
    chunkingConfig: {
      partitioning: 'auto',
      strategy: 'basic',
      max_characters: 500,
      new_after_n_chars: 500,
      overlap: 0,
      similarity_threshold: 0.5,
      overlap_all: false,
      file_type: 'txt'
    }
  }
]

[ragyabraham]

Is that new? I haven't seen that field in the API before

[tomlynchRNA]

from my understanding, its unpacking the response into whatever struct we handle the response as, which is missing a field, not a field that we send in the request