docker NVILA inference error

[JIA-HONG-CHU]

I follow your instruction by running:

1. docker build -t vila-server:latest .
2. docker run --gpus all --ipc=host --ulimit memlock=-1 --ulimit stack=67108864
   -v ./hub:/root/.cache/huggingface/hub
   -it --rm -p 8000:8000
   -e VILA_MODEL_PATH=Efficient-Large-Model/NVILA-15B
   -e VILA_CONV_MODE=auto
   vila-server:latest
3. from openai import OpenAI

client = OpenAI(
base_url="http://localhost:8000",
api_key="MYKEY",
)
response = client.chat.completions.create(
messages=[
{
"role": "user",
"content": [
{"type": "text", "text": "What’s in this image?"},
{
"type": "image_url",
"image_url": {
"url": "https://blog.logomyway.com/wp-content/uploads/2022/01/NVIDIA-logo.jpg",
# Or you can pass in a base64 encoded image
# "url": "data:image/png;base64,<base64_encoded_image>",
},
},
],
}
],
model="NVILA-15B",
)
print(response.choices[0].message.content)

Encountered the error:
openai.UnprocessableEntityError: Error code: 422 - {'detail': [{'loc': ['body', 'model'], 'msg': "unexpected value; permitted: 'VILA1.5-3B', 'VILA1.5-3B-AWQ', 'VILA1.5-3B-S2', 'VILA1.5-3B-S2-AWQ', 'Llama-3-VILA1.5-8B', 'Llama-3-VILA1.5-8B-AWQ', 'VILA1.5-13B', 'VILA1.5-13B-AWQ', 'VILA1.5-40B', 'VILA1.5-40B-AWQ'", 'type': 'value_error.const', 'ctx': {'given': 'NVILA-15B', 'permitted': ['VILA1.5-3B', 'VILA1.5-3B-AWQ', 'VILA1.5-3B-S2', 'VILA1.5-3B-S2-AWQ', 'Llama-3-VILA1.5-8B', 'Llama-3-VILA1.5-8B-AWQ', 'VILA1.5-13B', 'VILA1.5-13B-AWQ', 'VILA1.5-40B', 'VILA1.5-40B-AWQ']}}]}

[Lyken17]

oh, for NVILA the docker image and serving scripts is not updated to support yet. Please stay tuned and we will update an version in the coming week.

[wesen02]

Is there any news about the docker updates?