[MLOB-2035] create generic llmobs decorator

[ncybul]

Please read this doc for more details about the initiative and plan to make span kinds optional.

This PR updates the LLMObs SDK to make span kinds optional. In order to accomplish this, a new generic decorator @wrap is added along with a generic LLMObs.trace inline method. The end goal of this work will be to have these generic methods replace span kind specific decorators and inline methods (e.g. @llm or @workflow) for a faster onboarding experience.

Using the generic @wrap decorator

The first step in a user's onboarding journey will now involve decorating their functions with @wrap. This will capture the work done inside that function as an llmobs span with no span kind defined.

@wrap
def select_vacation():
    location_ideas = brainstorm_vacation_locations()
    vacation = select_location_from_ideas(location_ideas)
    res = format_output(vacation)
    LLMObs.annotate(
        input_data="Selecting vacation...",
        output_data=res,
    )
    return res

Once a user is comfortable with choosing span kinds that match the work done inside a function, they can enrich the decorator by passing in the kind argument. This will produce llmobs spans of that kind.

@wrap(kind="workflow")
def select_vacation():
    location_ideas = brainstorm_vacation_locations()
    vacation = select_location_from_ideas(location_ideas)
    res = format_output(vacation)
    LLMObs.annotate(
        input_data="Selecting vacation...",
        output_data=res,
    )
    return res

Using the generic LLMObs.trace inline method

Similarly to the @wrap decorator, users can start off by creating inline spans with no span kind defined.

def format_output(vacation):
    with LLMObs.trace(name="format_output") as span:
        res = "Your dream vacation is " + vacation
        LLMObs.annotate(
            span,
            input_data=vacation,
            output_data=res,
        )
    res = "Your dream vacation is " + vacation
    return res

They can then add additional arguments to flesh out their instrumentation.

def format_output(vacation):
    with LLMObs.trace(kind="task", name="format_output") as span:
        res = "Your dream vacation is " + vacation
        LLMObs.annotate(
            span,
            input_data=vacation,
            output_data=res,
        )
    res = "Your dream vacation is " + vacation
    return res

Checklist

• PR author has checked that all the criteria below are met
• The PR description includes an overview of the change
• The PR description articulates the motivation for the change
• The change includes tests OR the PR description describes a testing strategy
• The PR description notes risks associated with the change, if any
• Newly-added code is easy to change
• The change follows the library release note guidelines
• The change includes or references documentation updates if necessary
• Backport labels are set (if applicable)

Reviewer Checklist

• Reviewer has checked that all the criteria below are met
• Title is accurate
• All changes are related to the pull request's stated goal
• Avoids breaking API changes
• Testing strategy adequately addresses listed risks
• Newly-added code is easy to change
• Release note makes sense to a user of the library
• If necessary, author has acknowledged and discussed the performance implications of this PR as reported in the benchmarks PR comment
• Backport labels are set in a manner that is consistent with the release branch maintenance policy

[github-actions]

CODEOWNERS have been resolved as:

ddtrace/llmobs/_llmobs.py                                               @DataDog/ml-observability
ddtrace/llmobs/_trace_processor.py                                      @DataDog/ml-observability
ddtrace/llmobs/decorators.py                                            @DataDog/ml-observability

[pr-commenter]

Benchmarks

Benchmark execution time: 2025-01-08 19:41:07

Comparing candidate commit 354b389 in PR branch nicole-cybul/add-generic-decorator with baseline commit 6613185 in branch main.

Found 0 performance improvements and 0 performance regressions! Performance is the same for 394 metrics, 2 unstable metrics.