Can you add an option to use semi precision?

[niaka]

My GPU is RTX4090, and most of the models loaded through this plugin cannot work on this GPU.
The console always prompts for low memory.

I found a model card for flan-t5-xxl on the Huggingface website.
There is a description of how to use semi precision, but I don't understand how to write Python code.

Can you add an option to use semi precision?
And thank you very much for your work!

[Tps-F]

Good idea!

[Tps-F]

I'm busy this week, I'll write back next week.