Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Native Azure OpenAI Backend with Batch API Support #538

Open
tolgayan opened this issue Feb 25, 2025 · 1 comment
Open

Add Native Azure OpenAI Backend with Batch API Support #538

tolgayan opened this issue Feb 25, 2025 · 1 comment
Assignees
Labels
curator-batch Related batch request processor P1 Important

Comments

@tolgayan
Copy link

Description

The library currently supports batch operations for OpenAI, but it does not natively support Azure OpenAI. Although Azure OpenAI is accessible via the litellm backend, batch mode is not supported when using litellm. It would be highly beneficial to add a native Azure OpenAI backend that also supports batch API operations.

Current Behavior

  • OpenAI Batch Support: Batch operations are supported natively for OpenAI.
  • Azure OpenAI Integration: Azure OpenAI is only available through the litellm backend.
  • Limitation: Batch mode cannot be utilized with the litellm backend.

Expected Behavior

  • Native Integration: Provide a native backend for Azure OpenAI.
  • Batch API Support: The new Azure backend should support batch operations, consistent with the OpenAI implementation.

Thank you for considering this feature request.

@kartik4949 kartik4949 self-assigned this Feb 25, 2025
@kartik4949
Copy link
Contributor

Hello @tolgayan Thanks for the feature request.
We will be soon working on this feature!
Thanks

@RyanMarten RyanMarten added P1 Important curator-batch Related batch request processor and removed enhancement labels Mar 2, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
curator-batch Related batch request processor P1 Important
Projects
None yet
Development

No branches or pull requests

3 participants