New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Adding a new model? #43

Open

gursimar opened this issue Aug 22, 2024 · 0 comments

gursimar commented Aug 22, 2024 •

edited

Loading

Hi guys,

I want to add the LLava 1.5 (https://github.com/haotian-liu/LLaVA/blob/main/docs/MODEL_ZOO.md) that includes a ViT transfomer and Vicuna LLM.

Since the SwiftTransformer backend is complex to modify, what are the possible alternative backends that already support Llava 1.5?
Alternatively, is there a backend which is based on python where we can easily add the LLava 1.5 model?
What broad steps are required to change the backend? Can the SwiftTransfomer based ops in the "torch.ops.block_migration_ops" package be used with the new backend without any modification?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment