GPTFast llama loader released

[biship]

https://github.com/MDK8888/GPTFast

Per reddit: https://www.reddit.com/r/LocalLLaMA/comments/1b0ejca/gptfast_accelerate_your_hugging_face_transformers/

This library does the following:
quantizes the model to int8
adds kv caching
adds speculative decoding
adds kv caching to the speculative decoding model
compiles the speculative model and main model with some extra options to squeeze out as much performance as possible
sends the models to CUDA if available

Does this library makes Oobabooga work easier not having to maintain the HF loader?
Would GPTFast be worth implementing as an additional loader (if it offers benefits).
Perhaps GPTFast makes too many assumptions or forces a method of processing that removes user configurability.