[Bug] sglang doesn't stop the generation when the request is canceled

[tchaton]

Checklist

• 1. I have searched related issues but cannot get the expected help.
• 2. The bug has not been fixed in the latest version.
• 3. Please note that if the bug-related issue you submitted lacks corresponding environment info and a minimal reproducible demo, it will be challenging for us to reproduce and resolve the issue, reducing the likelihood of receiving feedback.
• 4. If the issue you raised is not a bug but a question, please raise a discussion at https://github.com/sgl-project/sglang/discussions/new/choose Otherwise, it will be closed.
• 5. Please use English, otherwise it will be closed.

Describe the bug

I have been experimenting with sglang. Amazing work there !

I have found that sgland continues to generate even if the request has been canceled by the client.

It would be great if it wasn't the case.

Reproduction

Start the server, send a request, cancel it.

Environment

I was using 2 * H200 to serve R1 600B

[jhinpan]

Thank you for your suggestion and kind words! We will try to introduce a cancellation token or flag within the client-server communication. When the client cancels the request, this token is triggered, signaling the generation routine to stop. This could potentially avoid wasting computational resources and ensures that only active requests consume server capacity.

If you want, you are also welcome to contribute and raise a PR about this enhancement!