GLiNER Inference Issue

[taghreed34]

I'm serving a version of GLiNER (transformer-based zero-shot NER model) on Openvino, and the model takes these as inputs:

input_ids

attention_mask
words_mask
text_lengths

span_idx

span_mask

For an input like this:

{
"text": "Monica Geller works as a dentist at NY General Hospital in USA.",
"req_entities": [
"Person",
"Location",
"Organization",
"Job Position",
"Workplace"
],
"isEnglish": [
"True"
]
}

the output of the preprocessed text is something like that (jsonified request to be sent to OVMS):

JSON Input: {"inputs": {"input_ids": [[1, 2569, 4052, 20773, 1504, 1504, 12590, 2569, 4052, 20773, 1504, 1504, 7792, 2569, 4052, 20773, 1504, 1504, 6946, 2569, 4052, 20773, 1504, 1504, 6094, 18172, 2569, 4052, 20773, 1504, 1504, 24411, 2569, 4052, 74606, 1504, 1504, 13258, 64146, 885, 283, 266, 8301, 288, 4413, 1849, 3505, 267, 2222, 323, 2]], "attention_mask": [[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]], "words_mask": [[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 0]], "span_idx": [[[0, 0], [0, 1], [0, 2], [0, 3], [0, 4], [0, 5], [0, 6], [0, 7], [0, 8], [0, 9], [0, 10], [0, 11], [1, 1], [1, 2], [1, 3], [1, 4], [1, 5], [1, 6], [1, 7], [1, 8], [1, 9], [1, 10], [1, 11], [1, 12], [2, 2], [2, 3], [2, 4], [2, 5], [2, 6], [2, 7], [2, 8], [2, 9], [2, 10], [2, 11], [2, 12], [2, 13], [3, 3], [3, 4], [3, 5], [3, 6], [3, 7], [3, 8], [3, 9], [3, 10], [3, 11], [3, 12], [3, 13], [3, 14], [4, 4], [4, 5], [4, 6], [4, 7], [4, 8], [4, 9], [4, 10], [4, 11], [4, 12], [4, 13], [4, 14], [4, 15], [5, 5], [5, 6], [5, 7], [5, 8], [5, 9], [5, 10], [5, 11], [5, 12], [5, 13], [5, 14], [5, 15], [5, 16], [6, 6], [6, 7], [6, 8], [6, 9], [6, 10], [6, 11], [6, 12], [6, 13], [6, 14], [6, 15], [6, 16], [6, 17], [7, 7], [7, 8], [7, 9], [7, 10], [7, 11], [7, 12], [7, 13], [7, 14], [7, 15], [7, 16], [7, 17], [7, 18], [8, 8], [8, 9], [8, 10], [8, 11], [8, 12], [8, 13], [8, 14], [8, 15], [8, 16], [8, 17], [8, 18], [8, 19], [9, 9], [9, 10], [9, 11], [9, 12], [9, 13], [9, 14], [9, 15], [9, 16], [9, 17], [9, 18], [9, 19], [9, 20], [10, 10], [10, 11], [10, 12], [10, 13], [10, 14], [10, 15], [10, 16], [10, 17], [10, 18], [10, 19], [10, 20], [10, 21], [11, 11], [11, 12], [11, 13], [11, 14], [11, 15], [11, 16], [11, 17], [11, 18], [11, 19], [11, 20], [11, 21], [11, 22], [12, 12], [12, 13], [12, 14], [12, 15], [12, 16], [12, 17], [12, 18], [12, 19], [12, 20], [12, 21], [12, 22], [12, 23]]], "span_mask": [[true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, false, true, true, true, true, true, true, true, true, true, true, false, false, true, true, true, true, true, true, true, true, true, false, false, false, true, true, true, true, true, true, true, true, false, false, false, false, true, true, true, true, true, true, true, false, false, false, false, false, true, true, true, true, true, true, false, false, false, false, false, false, true, true, true, true, true, false, false, false, false, false, false, false, true, true, true, true, false, false, false, false, false, false, false, false, true, true, true, false, false, false, false, false, false, false, false, false, true, true, false, false, false, false, false, false, false, false, false, false, true, false, false, false, false, false, false, false, false, false, false, false]], "text_lengths": [[13]]}}

I keep getting the following error:

{'error': 'Could not parse input content. Not valid ndarray detected'}

And I don't understand what's wrong!

Here is the model metadata:

{
"modelSpec": {
"name": "GLiNER",
"signatureName": "",
"version": "1"
},
"metadata": {
"signature_def": {
"@type": "type.googleapis.com/tensorflow.serving.SignatureDefMap",
"signatureDef": {
"serving_default": {
"inputs": {
"attention_mask": {
"dtype": "DT_INT64",
"tensorShape": {
"dim": [
{
"size": "-1",
"name": ""
},
{
"size": "-1",
"name": ""
}
],
"unknownRank": false
},
"name": "attention_mask"
},
"span_mask": {
"dtype": "DT_BOOL",
"tensorShape": {
"dim": [
{
"size": "-1",
"name": ""
},
{
"size": "-1",
"name": ""
}
],
"unknownRank": false
},
"name": "span_mask"
},
"span_idx": {
"dtype": "DT_INT64",
"tensorShape": {
"dim": [
{
"size": "-1",
"name": ""
},
{
"size": "-1",
"name": ""
},
{
"size": "-1",
"name": ""
}
],
"unknownRank": false
},
"name": "span_idx"
},
"text_lengths": {
"dtype": "DT_INT64",
"tensorShape": {
"dim": [
{
"size": "-1",
"name": ""
},
{
"size": "-1",
"name": ""
}
],
"unknownRank": false
},
"name": "text_lengths"
},
"input_ids": {
"dtype": "DT_INT64",
"tensorShape": {
"dim": [
{
"size": "-1",
"name": ""
},
{
"size": "-1",
"name": ""
}
],
"unknownRank": false
},
"name": "input_ids"
},
"words_mask": {
"dtype": "DT_INT64",
"tensorShape": {
"dim": [
{
"size": "-1",
"name": ""
},
{
"size": "-1",
"name": ""
}
],
"unknownRank": false
},
"name": "words_mask"
}
},
"outputs": {
"logits": {
"dtype": "DT_FLOAT",
"tensorShape": {
"dim": [
{
"size": "-1",
"name": ""
},
{
"size": "-1",
"name": ""
},
{
"size": "12",
"name": ""
},
{
"size": "-1",
"name": ""
}
],
"unknownRank": false
},
"name": "logits"
},
"5801": {
"dtype": "DT_INT64",
"tensorShape": {
"dim": [
{
"size": "-1",
"name": ""
},
{
"size": "-1",
"name": ""
}
],
"unknownRank": false
},
"name": "5801"
},
"onnx::Shape_6062": {
"dtype": "DT_FLOAT",
"tensorShape": {
"dim": [
{
"size": "-1",
"name": ""
},
{
"size": "-1",
"name": ""
},
{
"size": "512",
"name": ""
}
],
"unknownRank": false
},
"name": "onnx::Shape_6062"
},
"onnx::NonZero_5838": {
"dtype": "DT_BOOL",
"tensorShape": {
"dim": [
{
"size": "-1",
"name": ""
},
{
"size": "-1",
"name": ""
}
],
"unknownRank": false
},
"name": "onnx::NonZero_5838"
},
"prompts_embedding": {
"dtype": "DT_FLOAT",
"tensorShape": {
"dim": [
{
"size": "-1",
"name": ""
},
{
"size": "-1",
"name": ""
},
{
"size": "512",
"name": ""
}
],
"unknownRank": false
},
"name": "prompts_embedding"
}
},
"methodName": "",
"defaults": {}
}
}
}
}
}

[mzegla]

Looks like your JSON is correct. Is it possible to change true/false values to 1/0 in span_mask?