Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tool choice docs #349

Open
wants to merge 3 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions fern/pages/text-generation/tools/tool-use.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -306,6 +306,18 @@ if not response.tool_calls:
co.chat(message=message, tools=tools, tool_results=[])
```

## Forcing Tool Use

During the tool calling step, the model may decide to either:
- make tool call(s)
- or, respond to a user message directly.

You can force the model to make tool call(s), i.e. to not respond directly, by setting the `force_single_step` parameter to `True`.

This is equivalent to setting the `tool_choice` as `REQUIRED` in the v2 API.

While if `force_single_step=True` and `tool_results` are passed, this is equivalent to specifying `tool_choice` as `NONE`.

## Single-Step Tool Use and Chat History

Single-step tool use functions as part of a two-part conversational process. Here’s how that works:
Expand Down
36 changes: 31 additions & 5 deletions fern/pages/v2/text-generation/tools/tool-use.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -411,17 +411,43 @@ Start: 177 | End: 204 | Text: 'Laptop: $1,000, 15 in stock'
Start: 207 | End: 232 | Text: 'Tablet: $300, 25 in stock'
```

## Forcing Tool Use

During the tool calling step, the model may decide to either:
- make tool call(s)
- or, respond to a user message directly.

You can force the model to make tool call(s), i.e. to not respond directly, by setting the `tool_choice` parameter to `REQUIRED`.

Alternatively, you can force the model to respond directly, i.e. to not make tool call(s), by setting the `tool_choice` parameter to `NONE`.

By default, if you don’t specify the `tool_choice` parameter, then the model will decide whether it's more appropriate to call tools or to respond directly.

```python PYTHON {5}
response = co.chat(
model="command-r-plus-08-2024",
messages=messages,
tools=tools,
tool_choice="REQUIRED" # optional, to force tool calls
# tool_choice="NONE" # optional, to force a direct response
)
```

<Note>This parameter is only compatible with the [Command R7B](https://docs.cohere.com/v2/docs/command-r7b) and newer models.</Note>

## Structured Outputs (Tools)

Setting the `strict_tools` parameter to True will enforce each tool call to follow the specified tool schema. To learn more about this feature, visit the [Structured Outputs documentation](https://docs.cohere.com/v2/docs/structured-outputs).

Note that `strict_tools` is currently an experimental feature.

```python PYTHON {4}
response = co.chat(model="command-r-plus-08-2024",
messages=messages,
tools=tools,
strict_tools=True)
```python PYTHON {5}
response = co.chat(
model="command-r-plus-08-2024",
messages=messages,
tools=tools,
strict_tools=True
)
```

## How to Get Good Answers With Tool Use
Expand Down
Loading