You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
@Paidinful I changed my code to handle batch inputs. the pipeline includes batching and unbatching in the middle few times and in the process some input may be unprocessable which i remove from batch and continue processing. Even the output passed to decode function is correct. But if any request should be given 500, then all request will get 500 in response
🐛 Bug
To Reproduce
Setup litserve api with batch processing (max_batch_size>1) and raise HTTPexception (from fastapi) either in encode_request or decode_request.
Code sample
Expected behavior
Only the request with error should return 500 and other request should return output as expected.
The text was updated successfully, but these errors were encountered: