New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Deploying olmOCR-7B-0225-preview-GGUF using Ollama, how to call it with OpenAI #39

Open

netty2019 opened this issue Feb 27, 2025 · 1 comment

netty2019 commented Feb 27, 2025

🚀 The feature, motivation and pitch

Alternatives

No response

Additional context

No response

Collaborator

jakep-allenai commented Feb 27, 2025

The model is based on https://huggingface.co/Qwen/Qwen2-VL-7B-Instruct so it may not be supported by Ollama at this time.
Try VLLM or SGlang, which is what we tested on.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment