[Usage] pre-requisites for multi-image and multi-prompt generation

[sykverse]

Describe the issue

Hi there, thank you for sharing this awesome project.

I have one question about the requried packages for multi-image and multi-prompt generation.
In the following link, it says that the version of the "transformers" library should be newer than 4.35.3.
(https://huggingface.co/llava-hf/vip-llava-7b-hf)

At first I installed all the necessary packages including transformers by following the instructions in a README file. (BTW, with the required versions of packages, I've got the same error discussed in #10 when I followed the steps in https://huggingface.co/llava-hf/vip-llava-7b-hf#using-pipeline )

In order to utilize the multi-image and prompt generation functions, I re-installed the transformers to satisfy the condition (4.36.0). However, when I tried to do an inference, there was a conflict between the transforemers and tokenizer libraries.

Could you share which version of packages you used when testing the multi-image and prompt generation options (transformers, tokenziers, accelerate, bitsandbytes and so on)?

ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
vip-llava 1.1.3 requires tokenizers<0.14,>=0.12.1, but you have tokenizers 0.15.2 which is incompatible.
vip-llava 1.1.3 requires transformers==4.31.0, but you have transformers 4.36.0 which is incompatible.

[mu-cai]

Hi,

Thanks for trying our project!
In my experiments, I do not conduct multi-image generation experiments. But this should be possible if you update the code a bit.

Besides, what if you ignore the errors appears during the environment building stage? This will not affect your actual code execution process.

Thanks.

[sykverse]

Hi,

Thanks for the reply.
If there's a conflict between libraries, an error message about the tensor size mismatch popped up. (#10)
Could you share the version of CUDA when you used to build this project?

Thanks a lot.

[mu-cai]

cuda is 12.0.

can you use my code instead of huggingface code to conduct inference?