Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Raising HTTPexception on single request from encode_request or decode_request raise same for all request in batch #340

Open
bimarshak7 opened this issue Oct 23, 2024 · 1 comment
Labels
bug Something isn't working help wanted Extra attention is needed

Comments

@bimarshak7
Copy link

bimarshak7 commented Oct 23, 2024

🐛 Bug

To Reproduce

Setup litserve api with batch processing (max_batch_size>1) and raise HTTPexception (from fastapi) either in encode_request or decode_request.

Code sample

def decode(self, output):
        if "error" in output:
            raise HTTPException(status_code=500, detail="Error processing file")

        
        return output

Expected behavior

Only the request with error should return 500 and other request should return output as expected.

@bimarshak7 bimarshak7 added bug Something isn't working help wanted Extra attention is needed labels Oct 23, 2024
@bimarshak7
Copy link
Author

@Paidinful I changed my code to handle batch inputs. the pipeline includes batching and unbatching in the middle few times and in the process some input may be unprocessable which i remove from batch and continue processing. Even the output passed to decode function is correct. But if any request should be given 500, then all request will get 500 in response

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working help wanted Extra attention is needed
Projects
None yet
Development

No branches or pull requests

1 participant