Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Extend our eval library / workflow code to support LLM-as-judge #862

Open
aittalam opened this issue Feb 13, 2025 · 0 comments
Open

Extend our eval library / workflow code to support LLM-as-judge #862

aittalam opened this issue Feb 13, 2025 · 0 comments
Assignees
Labels
api Changes which impact API/presentation layer backend

Comments

@aittalam
Copy link
Member

See rationale here.

The goal of this task is to have all the code in place to run a pipeline which provides LLM-as-judge evaluation of a given dataset (following the design we aligned on and extending eval library with some aggregate method to have e.g. mean score from models)

@aittalam aittalam self-assigned this Feb 13, 2025
@ividal ividal added backend api Changes which impact API/presentation layer labels Feb 13, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api Changes which impact API/presentation layer backend
Projects
Development

No branches or pull requests

2 participants