How would I swap in the "ollama/llama2" model into the RestrictToTopic validator

[widgetface]

I hope you can assist me. I'm trying to swap in a different model to the RestrictToTopic using llmlite. From the Documentation I think I have the right approach.

I have the folloing code :

response =  Tennis and basketball are both popular sports, but they have some significant differences.
Tennis is an individual sport played with a racket and a small, hollow ball called a tennis ball. The game is played on a rectangular court with a net in the middle. The objective is to hit the ball over the net and into the opponent's court, while ensuring that the ball bounces once on your side of the court. The player who wins a set by scoring four points (with a margin of two points) wins the match.
ts."
guard = Guard().use(
    RestrictToTopic(
        litellm.completion,
        model="ollama/llama2",
        llm_callable="ollama/llama2",
        disable_classifier=False,
        disable_llm=False,
        valid_topics=["food"],
        invalid_topics=["music", "music"],
        on_fail="exception",
    )
)

guard.validate(response)

hen I run this code, I get an error:

    RestrictToTopic(
TypeError: RestrictToTopic.__init__() got multiple values for argument 'valid_topics'

Can anyone point out where I am going wrong. Thanks

[CalebCourier]

Hi @widgetface, the answer depends on how you intend to use ollama/llama2 within the validator. By default, this validator uses both a zero-shot classification model and an LLM model to perform an "ensemble" approach when extracting topics from the provided text.

Using Only ollama/llama2

If you want to use only ollama/llama2 to perform this function, you should disable the zero-shot model by passing disable_classifier=True and set the llm_callable to a Callable that takes the text to be validated and a list of topics, and returns a string with any topics found. Below is the default method that does this with an OpenAI model. Yours will likely look similar but with litellm and your chosen model instead:

def openai_callable(text: str, topics: List[str]) -> str:
    api_key, api_base = self.get_client_args()
    client = OpenAI(api_key=api_key, base_url=api_base)
    response = client.chat.completions.create(
        model=llm_callable,
        response_format={"type": "json_object"},
        messages=[
            {
                "role": "user",
                "content": f"""
                    Given a text and a list of topics, return a valid json list of which topics are present in the text. If none, just return an empty list.

                    Output Format:
                    -------------
                    "topics_present": []

                    Text:
                    ----
                    "{text}"

                    Topics: 
                    ------
                    {topics}

                    Result:
                    ------ """,
            },
        ],
    )
    return json.loads(response.choices[0].message.content)["topics_present"]

Using ollama/llama2 Alongside Zero-Shot Classification

If you want to continue using the default ensemble approach but swap out gpt-4o for ollama/llama2, you can do as I said above but leave disable_classifier=False.

Using ollama/llama2 As The Zero-Shot Classification

This is not possible because ollama/llama2 is not a transformers pipeline compatible zero-shot-classification model. However, you can disable the zero-shot classification like I described in the first scenario, or you can choose an appropriate zero-shot-classification model from here: https://huggingface.co/models?pipeline_tag=zero-shot-classification