Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

request limit限制可以设置成一个队列吗 #2496

Open
jasinliu opened this issue Oct 29, 2024 · 1 comment
Open

request limit限制可以设置成一个队列吗 #2496

jasinliu opened this issue Oct 29, 2024 · 1 comment
Labels
feature
Milestone
v0.16

Comments

@jasinliu
Copy link

jasinliu commented Oct 29, 2024 •
edited
Loading

Feature request / 功能建议

request limit限制可以设置成一个队列吗

Motivation / 动机

现在的request limit到达上限后直接抛出错误，可以设置成队列，让后续请求排队等待，这样可以充分榨干显卡

Your contribution / 您的贡献

暂时没有，如果提供一些指导的话，可以尝试做贡献
@XprobeBot XprobeBot modified the milestones: v0.15, v0.16 Oct 29, 2024
@qinxuye
Copy link
Contributor

qinxuye commented Nov 1, 2024

目前还不支持，欢迎贡献。目前已经引入队列了，但是没有和 request_limit 打通。

cc：#2473

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
feature
Projects
None yet
Milestone
v0.16
Development

No branches or pull requests
3 participants
@qinxuye @jasinliu @XprobeBot