chore(llmobs): refactor out ragas base evaluator

[lievan]

Creates a RagasBaseEvaluator class that ragas evaluators can share. This is a split-out version of this PR that has incorporated changes based on the first round of comments from @Yun-Kim and @Kyle-Verhoog

This class contains shared logic for:

• checking ragas dependencies are present, and throwing otherwise
• submit_and_evaluate function, which generally takes a span event, calls evaluate to generate a score, and then submits an evaluation
• extracting out question, contexts, and answer from an LLM span

Misc changes

• rename tests to specify they are faithfulness-specific tests
• throw when we parse an unsupported evaluator instead of logging a warning

Checklist

• PR author has checked that all the criteria below are met
• The PR description includes an overview of the change
• The PR description articulates the motivation for the change
• The change includes tests OR the PR description describes a testing strategy
• The PR description notes risks associated with the change, if any
• Newly-added code is easy to change
• The change follows the library release note guidelines
• The change includes or references documentation updates if necessary
• Backport labels are set (if applicable)

Reviewer Checklist

• Reviewer has checked that all the criteria below are met
• Title is accurate
• All changes are related to the pull request's stated goal
• Avoids breaking API changes
• Testing strategy adequately addresses listed risks
• Newly-added code is easy to change
• Release note makes sense to a user of the library
• If necessary, author has acknowledged and discussed the performance implications of this PR as reported in the benchmarks PR comment
• Backport labels are set in a manner that is consistent with the release branch maintenance policy

[github-actions]

CODEOWNERS have been resolved as:

ddtrace/llmobs/_evaluators/ragas/base.py                                @DataDog/ml-observability
ddtrace/llmobs/_evaluators/ragas/faithfulness.py                        @DataDog/ml-observability
ddtrace/llmobs/_evaluators/runner.py                                    @DataDog/ml-observability
tests/llmobs/_utils.py                                                  @DataDog/ml-observability
tests/llmobs/test_llmobs_evaluator_runner.py                            @DataDog/ml-observability
tests/llmobs/llmobs_cassettes/tests.llmobs.test_llmobs_ragas_evaluators.emits_traces_and_evaluations_on_exit.yaml  @DataDog/ml-observability
tests/llmobs/llmobs_cassettes/tests.llmobs.test_llmobs_ragas_evaluators.test_ragas_faithfulness_emits_traces.yaml  @DataDog/ml-observability
tests/llmobs/llmobs_cassettes/tests.llmobs.test_llmobs_ragas_evaluators.test_ragas_faithfulness_submits_evaluation.yaml  @DataDog/ml-observability
tests/llmobs/llmobs_cassettes/tests.llmobs.test_llmobs_ragas_evaluators.test_ragas_faithfulness_submits_evaluation_on_span_with_custom_keys.yaml  @DataDog/ml-observability
tests/llmobs/llmobs_cassettes/tests.llmobs.test_llmobs_ragas_evaluators.test_ragas_faithfulness_submits_evaluation_on_span_with_question_in_messages.yaml  @DataDog/ml-observability
tests/llmobs/test_llmobs_ragas_evaluators.py                            @DataDog/ml-observability

[datadog-dd-trace-py-rkomorn]

Datadog Report

Branch report: evan.li/ragas-base
Commit report: 7d7b245
Test service: dd-trace-py

✅ 0 Failed, 289 Passed, 1179 Skipped, 7m 22.32s Total duration (29m 15.6s time saved)

[pr-commenter]

Benchmarks

Benchmark execution time: 2025-01-09 21:02:05

Comparing candidate commit 967bddd in PR branch evan.li/ragas-base with baseline commit 04ee68f in branch main.

Found 0 performance improvements and 0 performance regressions! Performance is the same for 394 metrics, 2 unstable metrics.

[Kyle-Verhoog]

just one suggestion for the telemetry then good to go