Skip to content

Issues: vllm-project/vllm

[Roadmap] vLLM Roadmap Q1 2025
#11862 opened Jan 8, 2025 by simon-mo
Open 2
vLLM's V1 Engine Architecture
#8779 opened Sep 24, 2024 by simon-mo
Open 11
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

1,171 Open 4,894 Closed
1,171 Open 4,894 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

[Bug]: [TPU] Prefix caching + w8a8 + long context results in degraded performance and corrupted output bug Something isn't working
#12371 opened Jan 23, 2025 by kiratp
1 task done
3
Release v0.7.0 release Related to new version release
#12365 opened Jan 23, 2025 by simon-mo
5 tasks
2
[Bug]: Inference with gguf returns garbage bug Something isn't working
#12364 opened Jan 23, 2025 by q0dr
1 task done
[RFC]: Refactor config-format and load-format as plugins RFC
#12363 opened Jan 23, 2025 by maxdebayser
1 task done
2
[New Model]: Request for supporting microsoft/phi-4 Model new model Requests to new models
#12358 opened Jan 23, 2025 by yash7verma
1 task done
2
[Usage]: When running models on multiple GPUs, workload does not get split usage How to use vllm
#12354 opened Jan 23, 2025 by ArturDev42
1 task done
[Bug]: Cannot serve Qwen2.5 in OpenVINO bug Something isn't working
#12350 opened Jan 23, 2025 by cheng358
1 task done
1
[Usage]: how to use tool calling with auto option, setting the tool works usage How to use vllm
#12349 opened Jan 23, 2025 by balachandarsv
1 task done
[Usage]: Why does it consume so much memory? usage How to use vllm
#12346 opened Jan 23, 2025 by HouLingLXH
1 task done
[Usage]: fp8 sparse gemm in vllm/csrc/sparse/cutlass/sparse_scaled_mm__xxx usage How to use vllm
#12344 opened Jan 23, 2025 by zhink
1 task done
[Bug]: Why are the vLLM and Hugging Face Transformers inference results inconsistent? bug Something isn't working
#12343 opened Jan 23, 2025 by Molasse
1 task done
1
[Bug]: Run multiple LLMs inference one by one with multiple TP always pending on the second one in Model list bug Something isn't working
#12337 opened Jan 23, 2025 by alexhegit
1 task done
[Usage]: How to log incoming requests (inputs and outputs) in vllm serve ? usage How to use vllm
#12336 opened Jan 23, 2025 by thangld201
1 task done
1
[Misc]: RoPE vs Sliding Windows misc
#12328 opened Jan 22, 2025 by ccruttjr
[Bug]: Speculative decoding does not work bug Something isn't working speculative-decoding
#12323 opened Jan 22, 2025 by JohnConnor123
1 task done
1
[Usage]: Is it possible to speed up the generation speed by adding another video card? usage How to use vllm
#12322 opened Jan 22, 2025 by JohnConnor123
1 task done
[Performance]: Unable to produce the result of throughput & latency claimed on vLLM dashboard v0 performance Performance-related issues
#12315 opened Jan 22, 2025 by Neildave999
1 task done
[Performance]: It takes too much time to Add a request. performance Performance-related issues
#12314 opened Jan 22, 2025 by HuXinjing
1 task done
[Usage]: File Access Error When Using RunAI Model Streamer with S3 in VLLM usage How to use vllm
#12311 opened Jan 22, 2025 by nskumz
1 task done
1
4
[Bug]: Possible GPU Memory Utilization issue/bug for embeddings model bug Something isn't working
#12308 opened Jan 22, 2025 by mmubeen-6
1 task done
1
[Bug]: CUDA Exception on multi-gpus with concurrent users bug Something isn't working
#12307 opened Jan 22, 2025 by hahmad2008
1 task done
1
[Usage]: How can I use LLMEngine to perform distributed inference for multimodal large models, such as Qwen-VL? usage How to use vllm
#12305 opened Jan 22, 2025 by frederichen01
1 task done
1
[Bug]: Linting pre-commit hook does not apply yapf fixes; yapf fails quietly bug Something isn't working
#12302 opened Jan 22, 2025 by afeldman-nm
1 task done
4
[Bug]: build docker error bug Something isn't working
#12300 opened Jan 22, 2025 by jordane95
1 task done
[Feature]: DeepSeek-R1 tool choice && Function Call feature request
#12297 opened Jan 22, 2025 by warlockedward
1 task done
ProTip! Exclude everything labeled bug with -label:bug.