Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Can llama3.2 vision accept no image? #174

Open
TQA-task opened this issue Oct 10, 2024 · 2 comments
Open

Can llama3.2 vision accept no image? #174

TQA-task opened this issue Oct 10, 2024 · 2 comments

Comments

@TQA-task
Copy link

Hello,

thank you for thegreat work.
I have been trying to expermint with the model and see how it works.

My question is:
Can I use llama3.2 vision to cover cases where there is no image?

I tried set the value of the image variable to None or even delete it but I alawys get:
ValueError: Invalid input type. Must be a single image, a list of images, or a list of batches of images.
OR
ValueError: No image were provided, but there are image tokens in the prompt

@ashwinb
Copy link
Contributor

ashwinb commented Oct 10, 2024

Can you show your code?

@TQA-task
Copy link
Author

like in this:

question = row['Question']
options = row['AnswerChoices']

message = (f"Answer the question with the correct option letter. "
               f"Question: {question}\nOptions: {options}")

messages = [
        {"role": "user", "content": [
            {"type": "text", "text": message}
        ]}
    ]

  
input_text = processor.apply_chat_template(messages, add_generation_prompt=True)

inputs = processor(
        input_text,
        add_special_tokens=False,
        return_tensors="pt"
    ).to(model.device)

output = model.generate(**inputs, max_new_tokens=30)
response = processor.decode(output[0])

return response

I have also tried sending None for the image but got error

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants