Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bug: finish_reason isn't used in batch request processors #543

Open
RyanMarten opened this issue Feb 26, 2025 · 1 comment
Open

Bug: finish_reason isn't used in batch request processors #543

RyanMarten opened this issue Feb 26, 2025 · 1 comment
Labels
curator-batch Related batch request processor P2 Should do it soon

Comments

@RyanMarten
Copy link
Contributor

this should be added to GenericResponse
the finish reason should also be mapped to the standard set with the following

from litellm.litellm_core_utils.core_helpers import map_finish_reason
...
        # Get stop reason
        finish_reason = raw_response.get("stop_reason", "unknown")
        finish_reason = map_finish_reason(finish_reason)

        return GenericResponse(
            response_message=response_message,
            response_errors=response_errors,
            raw_response=raw_response,
            raw_request=None,
            generic_request=generic_request,
            created_at=batch.created_at,
            finished_at=batch.finished_at,
            token_usage=token_usage,
            response_cost=cost,
            finish_reason=finish_reason,
        )
@RyanMarten
Copy link
Contributor Author

fixing this for anthropic in #536
but gemini and openai also need to be updated.

this is so requests can be retried on invalid finish reasons

@RyanMarten RyanMarten added curator-batch Related batch request processor P2 Should do it soon labels Mar 2, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
curator-batch Related batch request processor P2 Should do it soon
Projects
None yet
Development

No branches or pull requests

1 participant