diff --git a/.DS_Store b/.DS_Store
new file mode 100644
index 0000000..5008ddf
Binary files /dev/null and b/.DS_Store differ
diff --git a/lab-tools-prompting.ipynb b/lab-tools-prompting.ipynb
deleted file mode 100644
index 9704130..0000000
--- a/lab-tools-prompting.ipynb
+++ /dev/null
@@ -1,509 +0,0 @@
-{
- "cells": [
- {
- "cell_type": "markdown",
- "id": "77f66dbe-192b-471c-9cb8-e9b365e61bbb",
- "metadata": {},
- "source": [
- "# Lab | Tools prompting\n",
- "\n",
- "**Replace the existing two tools decorators, by creating 3 new ones and adjust the prompts accordingly**"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "14b94240",
- "metadata": {},
- "source": [
- "### How to add ad-hoc tool calling capability to LLMs and Chat Models\n",
- "\n",
- ":::{.callout-caution}\n",
- "\n",
- "Some models have been fine-tuned for tool calling and provide a dedicated API for tool calling. Generally, such models are better at tool calling than non-fine-tuned models, and are recommended for use cases that require tool calling. Please see the [how to use a chat model to call tools](/docs/how_to/tool_calling) guide for more information.\n",
- "\n",
- "In this guide, we'll see how to add **ad-hoc** tool calling support to a chat model. This is an alternative method to invoke tools if you're using a model that does not natively support tool calling.\n",
- "\n",
- "We'll do this by simply writing a prompt that will get the model to invoke the appropriate tools. Here's a diagram of the logic:\n",
- "\n",
- "
\n",
- "\n",
- ""
- ]
- },
- {
- "cell_type": "markdown",
- "id": "a0a22cb8-19e7-450a-9d1b-6848d2c81cd1",
- "metadata": {},
- "source": [
- "## Setup\n",
- "\n",
- "We'll need to install the following packages:"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "8c556c5e-b785-428b-8e7d-efd34a2a1adb",
- "metadata": {},
- "outputs": [],
- "source": [
- "%pip install --upgrade --quiet langchain langchain-community"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "897bc01e-cc2b-4400-8a64-db4aa56085d3",
- "metadata": {},
- "source": [
- "If you'd like to use LangSmith, uncomment the below:"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 26,
- "id": "5efb4170-b95b-4d29-8f57-09509f3ba6df",
- "metadata": {},
- "outputs": [],
- "source": [
- "import getpass\n",
- "import os\n",
- "# os.environ[\"LANGCHAIN_TRACING_V2\"] = \"true\"\n",
- "# os.environ[\"LANGCHAIN_API_KEY\"] = getpass.getpass()"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "7ec6409b-21e5-4d0a-8a46-c4ef0b055dd3",
- "metadata": {},
- "source": [
- "You can select any of the given models for this how-to guide. Keep in mind that most of these models already [support native tool calling](/docs/integrations/chat/), so using the prompting strategy shown here doesn't make sense for these models, and instead you should follow the [how to use a chat model to call tools](/docs/how_to/tool_calling) guide.\n",
- "\n",
- "```{=mdx}\n",
- "import ChatModelTabs from \"@theme/ChatModelTabs\";\n",
- "\n",
- "\n",
- "```\n",
- "\n",
- "To illustrate the idea, we'll use `phi3` via Ollama, which does **NOT** have native support for tool calling. If you'd like to use `Ollama` as well follow [these instructions](/docs/integrations/chat/ollama/)."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 24,
- "id": "424be968-2806-4d1a-a6aa-5499ae20fac5",
- "metadata": {},
- "outputs": [],
- "source": [
- "from langchain_community.llms import Ollama\n",
- "\n",
- "model = Ollama(model=\"phi3\")"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "68946881",
- "metadata": {},
- "source": [
- "## Create a tool\n",
- "\n",
- "First, let's create an `add` and `multiply` tools. For more information on creating custom tools, please see [this guide](/docs/how_to/custom_tools)."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 4,
- "id": "4548e6fa-0f9b-4d7a-8fa5-66cec0350e5f",
- "metadata": {},
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "--\n",
- "multiply\n",
- "Multiply two numbers together.\n",
- "{'x': {'title': 'X', 'type': 'number'}, 'y': {'title': 'Y', 'type': 'number'}}\n",
- "--\n",
- "add\n",
- "Add two numbers.\n",
- "{'x': {'title': 'X', 'type': 'integer'}, 'y': {'title': 'Y', 'type': 'integer'}}\n"
- ]
- }
- ],
- "source": [
- "from langchain_core.tools import tool\n",
- "\n",
- "\n",
- "@tool\n",
- "def multiply(x: float, y: float) -> float:\n",
- " \"\"\"Multiply two numbers together.\"\"\"\n",
- " return x * y\n",
- "\n",
- "\n",
- "@tool\n",
- "def add(x: int, y: int) -> int:\n",
- " \"Add two numbers.\"\n",
- " return x + y\n",
- "\n",
- "\n",
- "tools = [multiply, add]\n",
- "\n",
- "# Let's inspect the tools\n",
- "for t in tools:\n",
- " print(\"--\")\n",
- " print(t.name)\n",
- " print(t.description)\n",
- " print(t.args)"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 5,
- "id": "be77e780",
- "metadata": {},
- "outputs": [
- {
- "data": {
- "text/plain": [
- "20.0"
- ]
- },
- "execution_count": 5,
- "metadata": {},
- "output_type": "execute_result"
- }
- ],
- "source": [
- "multiply.invoke({\"x\": 4, \"y\": 5})"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "15dd690e-e54d-4209-91a4-181f69a452ac",
- "metadata": {},
- "source": [
- "## Creating our prompt\n",
- "\n",
- "We'll want to write a prompt that specifies the tools the model has access to, the arguments to those tools, and the desired output format of the model. In this case we'll instruct it to output a JSON blob of the form `{\"name\": \"...\", \"arguments\": {...}}`."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 6,
- "id": "2063b564-25ca-4729-a45f-ba4633175b04",
- "metadata": {},
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "multiply(x: float, y: float) -> float - Multiply two numbers together.\n",
- "add(x: int, y: int) -> int - Add two numbers.\n"
- ]
- }
- ],
- "source": [
- "from langchain_core.output_parsers import JsonOutputParser\n",
- "from langchain_core.prompts import ChatPromptTemplate\n",
- "from langchain_core.tools import render_text_description\n",
- "\n",
- "rendered_tools = render_text_description(tools)\n",
- "print(rendered_tools)"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 17,
- "id": "f02f1dce-76e7-4ca9-9bac-5af496131fe1",
- "metadata": {},
- "outputs": [],
- "source": [
- "system_prompt = f\"\"\"\\\n",
- "You are an assistant that has access to the following set of tools. \n",
- "Here are the names and descriptions for each tool:\n",
- "\n",
- "{rendered_tools}\n",
- "\n",
- "Given the user input, return the name and input of the tool to use. \n",
- "Return your response as a JSON blob with 'name' and 'arguments' keys.\n",
- "\n",
- "The `arguments` should be a dictionary, with keys corresponding \n",
- "to the argument names and the values corresponding to the requested values.\n",
- "\"\"\"\n",
- "\n",
- "prompt = ChatPromptTemplate.from_messages(\n",
- " [(\"system\", system_prompt), (\"user\", \"{input}\")]\n",
- ")"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 18,
- "id": "f8623e03-60eb-4439-b57b-ecbcebc61b58",
- "metadata": {},
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "{\n",
- " \"name\": \"add\",\n",
- " \"arguments\": {\n",
- " \"x\": 3,\n",
- " \"y\": 1132\n",
- " }\n",
- "}\n"
- ]
- }
- ],
- "source": [
- "chain = prompt | model\n",
- "message = chain.invoke({\"input\": \"what's 3 plus 1132\"})\n",
- "\n",
- "# Let's take a look at the output from the model\n",
- "# if the model is an LLM (not a chat model), the output will be a string.\n",
- "if isinstance(message, str):\n",
- " print(message)\n",
- "else: # Otherwise it's a chat model\n",
- " print(message.content)"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "14df2cd5-b6fa-4b10-892d-e8692c7931e5",
- "metadata": {},
- "source": [
- "## Adding an output parser\n",
- "\n",
- "We'll use the `JsonOutputParser` for parsing our models output to JSON."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 19,
- "id": "f129f5bd-127c-4c95-8f34-8f437da7ca8f",
- "metadata": {},
- "outputs": [
- {
- "data": {
- "text/plain": [
- "{'name': 'multiply', 'arguments': {'x': 13.0, 'y': 4.0}}"
- ]
- },
- "execution_count": 19,
- "metadata": {},
- "output_type": "execute_result"
- }
- ],
- "source": [
- "from langchain_core.output_parsers import JsonOutputParser\n",
- "\n",
- "chain = prompt | model | JsonOutputParser()\n",
- "chain.invoke({\"input\": \"what's thirteen times 4\"})"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "e1f08255-f146-4f4a-be43-5c21c1d3ae83",
- "metadata": {},
- "source": [
- ":::{.callout-important}\n",
- "\n",
- "๐ Amazing! ๐ We now instructed our model on how to **request** that a tool be invoked.\n",
- "\n",
- "Now, let's create some logic to actually run the tool!\n",
- ":::"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "8e29dd4c-8eb5-457f-92d1-8add076404dc",
- "metadata": {},
- "source": [
- "## Invoking the tool ๐\n",
- "\n",
- "Now that the model can request that a tool be invoked, we need to write a function that can actually invoke \n",
- "the tool.\n",
- "\n",
- "The function will select the appropriate tool by name, and pass to it the arguments chosen by the model."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 20,
- "id": "faee95e0-4095-4310-991f-9e9465c6738e",
- "metadata": {},
- "outputs": [],
- "source": [
- "from typing import Any, Dict, Optional, TypedDict\n",
- "\n",
- "from langchain_core.runnables import RunnableConfig\n",
- "\n",
- "\n",
- "class ToolCallRequest(TypedDict):\n",
- " \"\"\"A typed dict that shows the inputs into the invoke_tool function.\"\"\"\n",
- "\n",
- " name: str\n",
- " arguments: Dict[str, Any]\n",
- "\n",
- "\n",
- "def invoke_tool(\n",
- " tool_call_request: ToolCallRequest, config: Optional[RunnableConfig] = None\n",
- "):\n",
- " \"\"\"A function that we can use the perform a tool invocation.\n",
- "\n",
- " Args:\n",
- " tool_call_request: a dict that contains the keys name and arguments.\n",
- " The name must match the name of a tool that exists.\n",
- " The arguments are the arguments to that tool.\n",
- " config: This is configuration information that LangChain uses that contains\n",
- " things like callbacks, metadata, etc.See LCEL documentation about RunnableConfig.\n",
- "\n",
- " Returns:\n",
- " output from the requested tool\n",
- " \"\"\"\n",
- " tool_name_to_tool = {tool.name: tool for tool in tools}\n",
- " name = tool_call_request[\"name\"]\n",
- " requested_tool = tool_name_to_tool[name]\n",
- " return requested_tool.invoke(tool_call_request[\"arguments\"], config=config)"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "f4957532-9e0c-47f6-bb62-0fd789ac1d3e",
- "metadata": {},
- "source": [
- "Let's test this out ๐งช!"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 21,
- "id": "d0ea3b2a-8fb2-4016-83c8-a5d3e78fedbc",
- "metadata": {},
- "outputs": [
- {
- "data": {
- "text/plain": [
- "15.0"
- ]
- },
- "execution_count": 21,
- "metadata": {},
- "output_type": "execute_result"
- }
- ],
- "source": [
- "invoke_tool({\"name\": \"multiply\", \"arguments\": {\"x\": 3, \"y\": 5}})"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "715af6e1-935d-4bc0-a3d2-646ecf8a329b",
- "metadata": {},
- "source": [
- "## Let's put it together\n",
- "\n",
- "Let's put it together into a chain that creates a calculator with add and multiplication capabilities."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 22,
- "id": "0555b384-fde6-4404-86e0-7ea199003d58",
- "metadata": {},
- "outputs": [
- {
- "data": {
- "text/plain": [
- "53.83784653"
- ]
- },
- "execution_count": 22,
- "metadata": {},
- "output_type": "execute_result"
- }
- ],
- "source": [
- "chain = prompt | model | JsonOutputParser() | invoke_tool\n",
- "chain.invoke({\"input\": \"what's thirteen times 4.14137281\"})"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "b4a9c5aa-f60a-4017-af6f-1ff6e04bfb61",
- "metadata": {},
- "source": [
- "## Returning tool inputs\n",
- "\n",
- "It can be helpful to return not only tool outputs but also tool inputs. We can easily do this with LCEL by `RunnablePassthrough.assign`-ing the tool output. This will take whatever the input is to the RunnablePassrthrough components (assumed to be a dictionary) and add a key to it while still passing through everything that's currently in the input:"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 23,
- "id": "45404406-859d-4caa-8b9d-5838162c80a0",
- "metadata": {},
- "outputs": [
- {
- "data": {
- "text/plain": [
- "{'name': 'multiply',\n",
- " 'arguments': {'x': 13, 'y': 4.14137281},\n",
- " 'output': 53.83784653}"
- ]
- },
- "execution_count": 23,
- "metadata": {},
- "output_type": "execute_result"
- }
- ],
- "source": [
- "from langchain_core.runnables import RunnablePassthrough\n",
- "\n",
- "chain = (\n",
- " prompt | model | JsonOutputParser() | RunnablePassthrough.assign(output=invoke_tool)\n",
- ")\n",
- "chain.invoke({\"input\": \"what's thirteen times 4.14137281\"})"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "1797fe82-ea35-4cba-834a-1caf9740d184",
- "metadata": {},
- "source": [
- "## What's next?\n",
- "\n",
- "This how-to guide shows the \"happy path\" when the model correctly outputs all the required tool information.\n",
- "\n",
- "In reality, if you're using more complex tools, you will start encountering errors from the model, especially for models that have not been fine tuned for tool calling and for less capable models.\n",
- "\n",
- "You will need to be prepared to add strategies to improve the output from the model; e.g.,\n",
- "\n",
- "1. Provide few shot examples.\n",
- "2. Add error handling (e.g., catch the exception and feed it back to the LLM to ask it to correct its previous output)."
- ]
- }
- ],
- "metadata": {
- "kernelspec": {
- "display_name": "Python 3 (ipykernel)",
- "language": "python",
- "name": "python3"
- },
- "language_info": {
- "codemirror_mode": {
- "name": "ipython",
- "version": 3
- },
- "file_extension": ".py",
- "mimetype": "text/x-python",
- "name": "python",
- "nbconvert_exporter": "python",
- "pygments_lexer": "ipython3",
- "version": "3.11.8"
- }
- },
- "nbformat": 4,
- "nbformat_minor": 5
-}
diff --git a/lab_tools_prompting.ipynb b/lab_tools_prompting.ipynb
new file mode 100644
index 0000000..6800ca6
--- /dev/null
+++ b/lab_tools_prompting.ipynb
@@ -0,0 +1,716 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "id": "77f66dbe-192b-471c-9cb8-e9b365e61bbb",
+ "metadata": {
+ "id": "77f66dbe-192b-471c-9cb8-e9b365e61bbb"
+ },
+ "source": [
+ "# Lab | Tools prompting\n",
+ "\n",
+ "**Replace the existing two tools decorators, by creating 3 new ones and adjust the prompts accordingly**"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "14b94240",
+ "metadata": {
+ "id": "14b94240"
+ },
+ "source": [
+ "### How to add ad-hoc tool calling capability to LLMs and Chat Models\n",
+ "\n",
+ ":::{.callout-caution}\n",
+ "\n",
+ "Some models have been fine-tuned for tool calling and provide a dedicated API for tool calling. Generally, such models are better at tool calling than non-fine-tuned models, and are recommended for use cases that require tool calling. Please see the [how to use a chat model to call tools](/docs/how_to/tool_calling) guide for more information.\n",
+ "\n",
+ "In this guide, we'll see how to add **ad-hoc** tool calling support to a chat model. This is an alternative method to invoke tools if you're using a model that does not natively support tool calling.\n",
+ "\n",
+ "We'll do this by simply writing a prompt that will get the model to invoke the appropriate tools. Here's a diagram of the logic:\n",
+ "\n",
+ "
\n",
+ "\n",
+ ""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "a0a22cb8-19e7-450a-9d1b-6848d2c81cd1",
+ "metadata": {
+ "id": "a0a22cb8-19e7-450a-9d1b-6848d2c81cd1"
+ },
+ "source": [
+ "## Setup\n",
+ "\n",
+ "We'll need to install the following packages:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 1,
+ "id": "8c556c5e-b785-428b-8e7d-efd34a2a1adb",
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/"
+ },
+ "id": "8c556c5e-b785-428b-8e7d-efd34a2a1adb",
+ "outputId": "8413e797-1fa7-468c-94b3-31e6e06120a3"
+ },
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "\u001b[2K \u001b[90mโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ\u001b[0m \u001b[32m2.5/2.5 MB\u001b[0m \u001b[31m10.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
+ "\u001b[2K \u001b[90mโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ\u001b[0m \u001b[32m76.4/76.4 kB\u001b[0m \u001b[31m4.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
+ "\u001b[2K \u001b[90mโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ\u001b[0m \u001b[32m49.6/49.6 kB\u001b[0m \u001b[31m2.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
+ "\u001b[?25h"
+ ]
+ }
+ ],
+ "source": [
+ "%pip install --upgrade --quiet langchain langchain-community langchain_ollama"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 2,
+ "id": "44b18db9",
+ "metadata": {
+ "id": "44b18db9"
+ },
+ "outputs": [],
+ "source": [
+ "import os\n",
+ "from dotenv import load_dotenv, find_dotenv\n",
+ "_ = load_dotenv(find_dotenv())"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "897bc01e-cc2b-4400-8a64-db4aa56085d3",
+ "metadata": {
+ "id": "897bc01e-cc2b-4400-8a64-db4aa56085d3"
+ },
+ "source": [
+ "If you'd like to use LangSmith, uncomment the below:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "5efb4170-b95b-4d29-8f57-09509f3ba6df",
+ "metadata": {
+ "id": "5efb4170-b95b-4d29-8f57-09509f3ba6df"
+ },
+ "outputs": [],
+ "source": [
+ "os.environ[\"LANGCHAIN_TRACING_V2\"] = \"true\"\n",
+ "os.environ[\"LANGCHAIN_ENDPOINT\"]=\"https://api.smith.langchain.com\"\n",
+ "os.environ[\"LANGCHAIN_PROJECT\"]=\"NAME_OF_YOUR_PROJECT\"\n",
+ "LANGSMITH_API_KEY = os.getenv(\"LANGSMITH_API_KEY\")"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "7ec6409b-21e5-4d0a-8a46-c4ef0b055dd3",
+ "metadata": {
+ "id": "7ec6409b-21e5-4d0a-8a46-c4ef0b055dd3"
+ },
+ "source": [
+ "You can select any of the given models for this how-to guide. Keep in mind that most of these models already [support native tool calling](/docs/integrations/chat/), so using the prompting strategy shown here doesn't make sense for these models, and instead you should follow the [how to use a chat model to call tools](/docs/how_to/tool_calling) guide.\n",
+ "\n",
+ "```{=mdx}\n",
+ "import ChatModelTabs from \"@theme/ChatModelTabs\";\n",
+ "\n",
+ "\n",
+ "```\n",
+ "\n",
+ "To illustrate the idea, we'll use `phi3` via Ollama, which does **NOT** have native support for tool calling. If you'd like to use `Ollama` as well follow [these instructions](/docs/integrations/chat/ollama/).\n",
+ "\n",
+ "Bear in mind, this model is around 2gb and will be downloaded in your system. To run it, if you don't have a GPU it will run inference on CPU and might be slow. Feel free to skip and use `gpt-4o-mini` from OpenAI instead."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 3,
+ "id": "5b14026b",
+ "metadata": {
+ "id": "5b14026b"
+ },
+ "outputs": [],
+ "source": [
+ "#from langchain_ollama import OllamaLLM\n",
+ "\n",
+ "#model = OllamaLLM(model=\"phi3\")"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "a38fb7b6",
+ "metadata": {
+ "id": "a38fb7b6"
+ },
+ "source": [
+ "Should you not have ae access to `Ollama`, you can also use `gpt-4o-mini` from OpenAI. If you'd like to use `gpt-4o-mini`, uncomment the below and make sure to have the `langchain_openai` package installed."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 10,
+ "id": "7_eQwjYkgRiV",
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/"
+ },
+ "id": "7_eQwjYkgRiV",
+ "outputId": "a0f5d177-18cd-4c2d-f73d-137c7c44d9bc"
+ },
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "Collecting langchain_openai\n",
+ " Downloading langchain_openai-0.3.0-py3-none-any.whl.metadata (2.7 kB)\n",
+ "Requirement already satisfied: langchain-core<0.4.0,>=0.3.29 in /usr/local/lib/python3.11/dist-packages (from langchain_openai) (0.3.29)\n",
+ "Requirement already satisfied: openai<2.0.0,>=1.58.1 in /usr/local/lib/python3.11/dist-packages (from langchain_openai) (1.59.6)\n",
+ "Collecting tiktoken<1,>=0.7 (from langchain_openai)\n",
+ " Downloading tiktoken-0.8.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (6.6 kB)\n",
+ "Requirement already satisfied: PyYAML>=5.3 in /usr/local/lib/python3.11/dist-packages (from langchain-core<0.4.0,>=0.3.29->langchain_openai) (6.0.2)\n",
+ "Requirement already satisfied: jsonpatch<2.0,>=1.33 in /usr/local/lib/python3.11/dist-packages (from langchain-core<0.4.0,>=0.3.29->langchain_openai) (1.33)\n",
+ "Requirement already satisfied: langsmith<0.3,>=0.1.125 in /usr/local/lib/python3.11/dist-packages (from langchain-core<0.4.0,>=0.3.29->langchain_openai) (0.2.10)\n",
+ "Requirement already satisfied: packaging<25,>=23.2 in /usr/local/lib/python3.11/dist-packages (from langchain-core<0.4.0,>=0.3.29->langchain_openai) (24.2)\n",
+ "Requirement already satisfied: pydantic<3.0.0,>=2.5.2 in /usr/local/lib/python3.11/dist-packages (from langchain-core<0.4.0,>=0.3.29->langchain_openai) (2.10.5)\n",
+ "Requirement already satisfied: tenacity!=8.4.0,<10.0.0,>=8.1.0 in /usr/local/lib/python3.11/dist-packages (from langchain-core<0.4.0,>=0.3.29->langchain_openai) (9.0.0)\n",
+ "Requirement already satisfied: typing-extensions>=4.7 in /usr/local/lib/python3.11/dist-packages (from langchain-core<0.4.0,>=0.3.29->langchain_openai) (4.12.2)\n",
+ "Requirement already satisfied: anyio<5,>=3.5.0 in /usr/local/lib/python3.11/dist-packages (from openai<2.0.0,>=1.58.1->langchain_openai) (3.7.1)\n",
+ "Requirement already satisfied: distro<2,>=1.7.0 in /usr/local/lib/python3.11/dist-packages (from openai<2.0.0,>=1.58.1->langchain_openai) (1.9.0)\n",
+ "Requirement already satisfied: httpx<1,>=0.23.0 in /usr/local/lib/python3.11/dist-packages (from openai<2.0.0,>=1.58.1->langchain_openai) (0.27.2)\n",
+ "Requirement already satisfied: jiter<1,>=0.4.0 in /usr/local/lib/python3.11/dist-packages (from openai<2.0.0,>=1.58.1->langchain_openai) (0.8.2)\n",
+ "Requirement already satisfied: sniffio in /usr/local/lib/python3.11/dist-packages (from openai<2.0.0,>=1.58.1->langchain_openai) (1.3.1)\n",
+ "Requirement already satisfied: tqdm>4 in /usr/local/lib/python3.11/dist-packages (from openai<2.0.0,>=1.58.1->langchain_openai) (4.67.1)\n",
+ "Requirement already satisfied: regex>=2022.1.18 in /usr/local/lib/python3.11/dist-packages (from tiktoken<1,>=0.7->langchain_openai) (2024.11.6)\n",
+ "Requirement already satisfied: requests>=2.26.0 in /usr/local/lib/python3.11/dist-packages (from tiktoken<1,>=0.7->langchain_openai) (2.32.3)\n",
+ "Requirement already satisfied: idna>=2.8 in /usr/local/lib/python3.11/dist-packages (from anyio<5,>=3.5.0->openai<2.0.0,>=1.58.1->langchain_openai) (3.10)\n",
+ "Requirement already satisfied: certifi in /usr/local/lib/python3.11/dist-packages (from httpx<1,>=0.23.0->openai<2.0.0,>=1.58.1->langchain_openai) (2024.12.14)\n",
+ "Requirement already satisfied: httpcore==1.* in /usr/local/lib/python3.11/dist-packages (from httpx<1,>=0.23.0->openai<2.0.0,>=1.58.1->langchain_openai) (1.0.7)\n",
+ "Requirement already satisfied: h11<0.15,>=0.13 in /usr/local/lib/python3.11/dist-packages (from httpcore==1.*->httpx<1,>=0.23.0->openai<2.0.0,>=1.58.1->langchain_openai) (0.14.0)\n",
+ "Requirement already satisfied: jsonpointer>=1.9 in /usr/local/lib/python3.11/dist-packages (from jsonpatch<2.0,>=1.33->langchain-core<0.4.0,>=0.3.29->langchain_openai) (3.0.0)\n",
+ "Requirement already satisfied: orjson<4.0.0,>=3.9.14 in /usr/local/lib/python3.11/dist-packages (from langsmith<0.3,>=0.1.125->langchain-core<0.4.0,>=0.3.29->langchain_openai) (3.10.14)\n",
+ "Requirement already satisfied: requests-toolbelt<2.0.0,>=1.0.0 in /usr/local/lib/python3.11/dist-packages (from langsmith<0.3,>=0.1.125->langchain-core<0.4.0,>=0.3.29->langchain_openai) (1.0.0)\n",
+ "Requirement already satisfied: annotated-types>=0.6.0 in /usr/local/lib/python3.11/dist-packages (from pydantic<3.0.0,>=2.5.2->langchain-core<0.4.0,>=0.3.29->langchain_openai) (0.7.0)\n",
+ "Requirement already satisfied: pydantic-core==2.27.2 in /usr/local/lib/python3.11/dist-packages (from pydantic<3.0.0,>=2.5.2->langchain-core<0.4.0,>=0.3.29->langchain_openai) (2.27.2)\n",
+ "Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.11/dist-packages (from requests>=2.26.0->tiktoken<1,>=0.7->langchain_openai) (3.4.1)\n",
+ "Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.11/dist-packages (from requests>=2.26.0->tiktoken<1,>=0.7->langchain_openai) (2.3.0)\n",
+ "Downloading langchain_openai-0.3.0-py3-none-any.whl (54 kB)\n",
+ "\u001b[2K \u001b[90mโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ\u001b[0m \u001b[32m54.2/54.2 kB\u001b[0m \u001b[31m2.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
+ "\u001b[?25hDownloading tiktoken-0.8.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.2 MB)\n",
+ "\u001b[2K \u001b[90mโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ\u001b[0m \u001b[32m1.2/1.2 MB\u001b[0m \u001b[31m21.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
+ "\u001b[?25hInstalling collected packages: tiktoken, langchain_openai\n",
+ "Successfully installed langchain_openai-0.3.0 tiktoken-0.8.0\n"
+ ]
+ }
+ ],
+ "source": [
+ "!pip install langchain_openai"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 12,
+ "id": "90655f01",
+ "metadata": {
+ "id": "90655f01"
+ },
+ "outputs": [],
+ "source": [
+ "from langchain_openai import ChatOpenAI\n",
+ "from google.colab import userdata\n",
+ "\n",
+ "OPENAI_API_KEY = userdata.get('OPENAI_API_KEY')\n",
+ "model = ChatOpenAI(model= \"gpt-4o-mini\", openai_api_key=OPENAI_API_KEY)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "68946881",
+ "metadata": {
+ "id": "68946881"
+ },
+ "source": [
+ "## Create a tool\n",
+ "\n",
+ "First, let's create an `add` and `multiply` tools. For more information on creating custom tools, please see [this guide](/docs/how_to/custom_tools)."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 13,
+ "id": "4548e6fa-0f9b-4d7a-8fa5-66cec0350e5f",
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/"
+ },
+ "id": "4548e6fa-0f9b-4d7a-8fa5-66cec0350e5f",
+ "outputId": "2fcf37c7-89cb-4af7-bce8-922dece159e7"
+ },
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "--\n",
+ "multiply\n",
+ "Multiply two numbers together.\n",
+ "{'x': {'title': 'X', 'type': 'number'}, 'y': {'title': 'Y', 'type': 'number'}}\n",
+ "--\n",
+ "add\n",
+ "Add two numbers.\n",
+ "{'x': {'title': 'X', 'type': 'integer'}, 'y': {'title': 'Y', 'type': 'integer'}}\n"
+ ]
+ }
+ ],
+ "source": [
+ "from langchain_core.tools import tool\n",
+ "\n",
+ "\n",
+ "@tool\n",
+ "def multiply(x: float, y: float) -> float:\n",
+ " \"\"\"Multiply two numbers together.\"\"\"\n",
+ " return x * y\n",
+ "\n",
+ "\n",
+ "@tool\n",
+ "def add(x: int, y: int) -> int:\n",
+ " \"Add two numbers.\"\n",
+ " return x + y\n",
+ "\n",
+ "\n",
+ "tools = [multiply, add]\n",
+ "\n",
+ "# Let's inspect the tools\n",
+ "for t in tools:\n",
+ " print(\"--\")\n",
+ " print(t.name)\n",
+ " print(t.description)\n",
+ " print(t.args)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 14,
+ "id": "be77e780",
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/"
+ },
+ "id": "be77e780",
+ "outputId": "445d9efb-2f93-4823-8dd8-150a4c6877ae"
+ },
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "20.0"
+ ]
+ },
+ "execution_count": 14,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "multiply.invoke({\"x\": 4, \"y\": 5})"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "15dd690e-e54d-4209-91a4-181f69a452ac",
+ "metadata": {
+ "id": "15dd690e-e54d-4209-91a4-181f69a452ac"
+ },
+ "source": [
+ "## Creating our prompt\n",
+ "\n",
+ "We'll want to write a prompt that specifies the tools the model has access to, the arguments to those tools, and the desired output format of the model. In this case we'll instruct it to output a JSON blob of the form `{\"name\": \"...\", \"arguments\": {...}}`."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 15,
+ "id": "2063b564-25ca-4729-a45f-ba4633175b04",
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/"
+ },
+ "id": "2063b564-25ca-4729-a45f-ba4633175b04",
+ "outputId": "6f2b6066-cba1-4789-c428-6c9b8ab4f10f"
+ },
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "multiply(x: float, y: float) -> float - Multiply two numbers together.\n",
+ "add(x: int, y: int) -> int - Add two numbers.\n"
+ ]
+ }
+ ],
+ "source": [
+ "from langchain_core.output_parsers import JsonOutputParser\n",
+ "from langchain_core.prompts import ChatPromptTemplate\n",
+ "from langchain_core.tools import render_text_description\n",
+ "\n",
+ "rendered_tools = render_text_description(tools)\n",
+ "print(rendered_tools)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 16,
+ "id": "f02f1dce-76e7-4ca9-9bac-5af496131fe1",
+ "metadata": {
+ "id": "f02f1dce-76e7-4ca9-9bac-5af496131fe1"
+ },
+ "outputs": [],
+ "source": [
+ "system_prompt = f\"\"\"\\\n",
+ "You are an assistant that has access to the following set of tools.\n",
+ "Here are the names and descriptions for each tool:\n",
+ "\n",
+ "{rendered_tools}\n",
+ "\n",
+ "Given the user input, return the name and input of the tool to use.\n",
+ "Return your response as a JSON blob with 'name' and 'arguments' keys.\n",
+ "\n",
+ "The `arguments` should be a dictionary, with keys corresponding\n",
+ "to the argument names and the values corresponding to the requested values.\n",
+ "\"\"\"\n",
+ "\n",
+ "prompt = ChatPromptTemplate.from_messages(\n",
+ " [(\"system\", system_prompt), (\"user\", \"{input}\")]\n",
+ ")"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 17,
+ "id": "f8623e03-60eb-4439-b57b-ecbcebc61b58",
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/"
+ },
+ "id": "f8623e03-60eb-4439-b57b-ecbcebc61b58",
+ "outputId": "1a5e1a48-68a2-466f-92b4-67f0904ab297"
+ },
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "{\n",
+ " \"name\": \"add\",\n",
+ " \"arguments\": {\n",
+ " \"x\": 3,\n",
+ " \"y\": 1132\n",
+ " }\n",
+ "}\n"
+ ]
+ }
+ ],
+ "source": [
+ "chain = prompt | model\n",
+ "message = chain.invoke({\"input\": \"what's 3 plus 1132\"})\n",
+ "\n",
+ "# Let's take a look at the output from the model\n",
+ "# if the model is an LLM (not a chat model), the output will be a string.\n",
+ "if isinstance(message, str):\n",
+ " print(message)\n",
+ "else: # Otherwise it's a chat model\n",
+ " print(message.content)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "14df2cd5-b6fa-4b10-892d-e8692c7931e5",
+ "metadata": {
+ "id": "14df2cd5-b6fa-4b10-892d-e8692c7931e5"
+ },
+ "source": [
+ "## Adding an output parser\n",
+ "\n",
+ "We'll use the `JsonOutputParser` for parsing our models output to JSON."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 18,
+ "id": "f129f5bd-127c-4c95-8f34-8f437da7ca8f",
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/"
+ },
+ "id": "f129f5bd-127c-4c95-8f34-8f437da7ca8f",
+ "outputId": "7b20d02c-6597-42e1-9617-74adc6544be4"
+ },
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "{'name': 'multiply', 'arguments': {'x': 13.0, 'y': 4.0}}"
+ ]
+ },
+ "execution_count": 18,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "from langchain_core.output_parsers import JsonOutputParser\n",
+ "\n",
+ "chain = prompt | model | JsonOutputParser()\n",
+ "chain.invoke({\"input\": \"what's thirteen times 4\"})"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "e1f08255-f146-4f4a-be43-5c21c1d3ae83",
+ "metadata": {
+ "id": "e1f08255-f146-4f4a-be43-5c21c1d3ae83"
+ },
+ "source": [
+ ":::{.callout-important}\n",
+ "\n",
+ "๐ Amazing! ๐ We now instructed our model on how to **request** that a tool be invoked.\n",
+ "\n",
+ "Now, let's create some logic to actually run the tool!\n",
+ ":::"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "8e29dd4c-8eb5-457f-92d1-8add076404dc",
+ "metadata": {
+ "id": "8e29dd4c-8eb5-457f-92d1-8add076404dc"
+ },
+ "source": [
+ "## Invoking the tool ๐\n",
+ "\n",
+ "Now that the model can request that a tool be invoked, we need to write a function that can actually invoke\n",
+ "the tool.\n",
+ "\n",
+ "The function will select the appropriate tool by name, and pass to it the arguments chosen by the model."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 19,
+ "id": "faee95e0-4095-4310-991f-9e9465c6738e",
+ "metadata": {
+ "id": "faee95e0-4095-4310-991f-9e9465c6738e"
+ },
+ "outputs": [],
+ "source": [
+ "from typing import Any, Dict, Optional, TypedDict\n",
+ "\n",
+ "from langchain_core.runnables import RunnableConfig\n",
+ "\n",
+ "\n",
+ "class ToolCallRequest(TypedDict):\n",
+ " \"\"\"A typed dict that shows the inputs into the invoke_tool function.\"\"\"\n",
+ "\n",
+ " name: str\n",
+ " arguments: Dict[str, Any]\n",
+ "\n",
+ "\n",
+ "def invoke_tool(\n",
+ " tool_call_request: ToolCallRequest, config: Optional[RunnableConfig] = None\n",
+ "):\n",
+ " \"\"\"A function that we can use the perform a tool invocation.\n",
+ "\n",
+ " Args:\n",
+ " tool_call_request: a dict that contains the keys name and arguments.\n",
+ " The name must match the name of a tool that exists.\n",
+ " The arguments are the arguments to that tool.\n",
+ " config: This is configuration information that LangChain uses that contains\n",
+ " things like callbacks, metadata, etc.See LCEL documentation about RunnableConfig.\n",
+ "\n",
+ " Returns:\n",
+ " output from the requested tool\n",
+ " \"\"\"\n",
+ " tool_name_to_tool = {tool.name: tool for tool in tools}\n",
+ " name = tool_call_request[\"name\"]\n",
+ " requested_tool = tool_name_to_tool[name]\n",
+ " return requested_tool.invoke(tool_call_request[\"arguments\"], config=config)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "f4957532-9e0c-47f6-bb62-0fd789ac1d3e",
+ "metadata": {
+ "id": "f4957532-9e0c-47f6-bb62-0fd789ac1d3e"
+ },
+ "source": [
+ "Let's test this out ๐งช!"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 20,
+ "id": "d0ea3b2a-8fb2-4016-83c8-a5d3e78fedbc",
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/"
+ },
+ "id": "d0ea3b2a-8fb2-4016-83c8-a5d3e78fedbc",
+ "outputId": "17fad954-6abc-406a-f963-28a629a20b18"
+ },
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "15.0"
+ ]
+ },
+ "execution_count": 20,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "invoke_tool({\"name\": \"multiply\", \"arguments\": {\"x\": 3, \"y\": 5}})"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "715af6e1-935d-4bc0-a3d2-646ecf8a329b",
+ "metadata": {
+ "id": "715af6e1-935d-4bc0-a3d2-646ecf8a329b"
+ },
+ "source": [
+ "## Let's put it together\n",
+ "\n",
+ "Let's put it together into a chain that creates a calculator with add and multiplication capabilities."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 21,
+ "id": "0555b384-fde6-4404-86e0-7ea199003d58",
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/"
+ },
+ "id": "0555b384-fde6-4404-86e0-7ea199003d58",
+ "outputId": "eeca49b3-406e-4058-d38a-f895b89d55a5"
+ },
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "53.83784653"
+ ]
+ },
+ "execution_count": 21,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "chain = prompt | model | JsonOutputParser() | invoke_tool\n",
+ "chain.invoke({\"input\": \"what's thirteen times 4.14137281\"})"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "b4a9c5aa-f60a-4017-af6f-1ff6e04bfb61",
+ "metadata": {
+ "id": "b4a9c5aa-f60a-4017-af6f-1ff6e04bfb61"
+ },
+ "source": [
+ "## Returning tool inputs\n",
+ "\n",
+ "It can be helpful to return not only tool outputs but also tool inputs. We can easily do this with LCEL by `RunnablePassthrough.assign`-ing the tool output. This will take whatever the input is to the RunnablePassrthrough components (assumed to be a dictionary) and add a key to it while still passing through everything that's currently in the input:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 22,
+ "id": "45404406-859d-4caa-8b9d-5838162c80a0",
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/"
+ },
+ "id": "45404406-859d-4caa-8b9d-5838162c80a0",
+ "outputId": "a7e2072f-d533-49f0-8f01-d371b73df9d7"
+ },
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "{'name': 'multiply',\n",
+ " 'arguments': {'x': 13, 'y': 4.14137281},\n",
+ " 'output': 53.83784653}"
+ ]
+ },
+ "execution_count": 22,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "from langchain_core.runnables import RunnablePassthrough\n",
+ "\n",
+ "chain = (\n",
+ " prompt | model | JsonOutputParser() | RunnablePassthrough.assign(output=invoke_tool)\n",
+ ")\n",
+ "chain.invoke({\"input\": \"what's thirteen times 4.14137281\"})"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "1797fe82-ea35-4cba-834a-1caf9740d184",
+ "metadata": {
+ "id": "1797fe82-ea35-4cba-834a-1caf9740d184"
+ },
+ "source": [
+ "## What's next?\n",
+ "\n",
+ "This how-to guide shows the \"happy path\" when the model correctly outputs all the required tool information.\n",
+ "\n",
+ "In reality, if you're using more complex tools, you will start encountering errors from the model, especially for models that have not been fine tuned for tool calling and for less capable models.\n",
+ "\n",
+ "You will need to be prepared to add strategies to improve the output from the model; e.g.,\n",
+ "\n",
+ "1. Provide few shot examples.\n",
+ "2. Add error handling (e.g., catch the exception and feed it back to the LLM to ask it to correct its previous output)."
+ ]
+ }
+ ],
+ "metadata": {
+ "colab": {
+ "provenance": []
+ },
+ "kernelspec": {
+ "display_name": "base",
+ "language": "python",
+ "name": "python3"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.12.7"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}