Gemini_Redis.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "aiHV6ip2f5id"
      },
      "source": [
        "# LLM Reference Architecture using Redis & Google Cloud Platform\n",
        "\n",
        "<a href=\"https://colab.research.google.com/github/RedisVentures/redis-google-llms/blob/main/BigQuery_Palm_Redis.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>\n",
        "\n",
        "This notebook serves as a getting started guide for working with LLMs on Google Cloud Platform with Redis Enterprise.\n",
        "\n",
        "## Intro\n",
        "Google's Vertex AI has expanded its capabilities by introducing [Generative AI](https://cloud.google.com/vertex-ai/docs/generative-ai/learn/overview). This advanced technology comes with a specialized [in-console studio experience](https://cloud.google.com/vertex-ai/docs/generative-ai/start/quickstarts/quickstart), a [dedicated API](https://cloud.google.com/vertex-ai/docs/generative-ai/start/quickstarts/api-quickstart) and [Python SDK](https://cloud.google.com/vertex-ai/docs/python-sdk/use-vertex-ai-python-sdk) designed for deploying and managing instances of Google's powerful Gemini language models.\n",
        "\n",
        "Redis Enterprise offers robust vector database features, with an efficient API for vector index creation, management, distance metric selection, similarity search, and hybrid filtering. When coupled with its versatile data structures - including lists, hashes, JSON, and sets - Redis Enterprise shines as the optimal solution for crafting high-quality Large Language Model (LLM)-based applications. It embodies a streamlined architecture and exceptional performance, making it an instrumental tool for production environments.\n",
        "\n",
        "Below we will work through several design patterns with Vertex AI LLMs and Redis Enterprise that will ensure optimal production performance.\n",
        "\n",
        "___\n",
        "## Contents\n",
        "- Setup\n",
        "    1. Prerequisites\n",
        "    2. Obtain Dataset\n",
        "    3. Generate Embeddings\n",
        "    4. Create Index\n",
        "    5. Query\n",
        "- Building a RAG Pipeline from scratch\n",
        "- Demo\n",
        "\n",
        "___"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "rK2rWODkw-kX"
      },
      "source": [
        "# Setup"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "37rbBPKdL09o"
      },
      "source": [
        "## 1. Prerequisites\n",
        "Before we begin, we must install some required libraries, authenticate with Google, create a Redis database, and initialize other required components.\n",
        "\n",
        "### Install required libraries"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 1,
      "metadata": {
        "id": "pc-IxYu3wnQm",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 1000
        },
        "outputId": "2b333ebc-0cd3-4e38-c8c7-6dc256d7939b",
        "collapsed": true
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Collecting redisvl\n",
            "  Downloading redisvl-0.2.0-py3-none-any.whl (63 kB)\n",
            "\u001b[?25l     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m0.0/63.9 kB\u001b[0m \u001b[31m?\u001b[0m eta \u001b[36m-:--:--\u001b[0m\r\u001b[2K     \u001b[91m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[90m╺\u001b[0m\u001b[90m━\u001b[0m \u001b[32m61.4/63.9 kB\u001b[0m \u001b[31m2.4 MB/s\u001b[0m eta \u001b[36m0:00:01\u001b[0m\r\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m63.9/63.9 kB\u001b[0m \u001b[31m1.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: google-cloud-aiplatform in /usr/local/lib/python3.10/dist-packages (1.52.0)\n",
            "Collecting langchain-community\n",
            "  Downloading langchain_community-0.2.1-py3-none-any.whl (2.1 MB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m2.1/2.1 MB\u001b[0m \u001b[31m26.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting unstructured[pdf]\n",
            "  Downloading unstructured-0.14.3-py3-none-any.whl (2.0 MB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m2.0/2.0 MB\u001b[0m \u001b[31m48.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting gradio\n",
            "  Downloading gradio-4.31.5-py3-none-any.whl (12.3 MB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m12.3/12.3 MB\u001b[0m \u001b[31m34.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting coloredlogs (from redisvl)\n",
            "  Downloading coloredlogs-15.0.1-py2.py3-none-any.whl (46 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m46.0/46.0 kB\u001b[0m \u001b[31m1.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: numpy in /usr/local/lib/python3.10/dist-packages (from redisvl) (1.25.2)\n",
            "Requirement already satisfied: pydantic<3,>=2 in /usr/local/lib/python3.10/dist-packages (from redisvl) (2.7.1)\n",
            "Requirement already satisfied: pyyaml in /usr/local/lib/python3.10/dist-packages (from redisvl) (6.0.1)\n",
            "Collecting redis>=5.0.0 (from redisvl)\n",
            "  Downloading redis-5.0.4-py3-none-any.whl (251 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m252.0/252.0 kB\u001b[0m \u001b[31m4.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: tabulate<1,>=0.9.0 in /usr/local/lib/python3.10/dist-packages (from redisvl) (0.9.0)\n",
            "Requirement already satisfied: tenacity>=8.2.2 in /usr/local/lib/python3.10/dist-packages (from redisvl) (8.3.0)\n",
            "Requirement already satisfied: google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.*,!=2.4.*,!=2.5.*,!=2.6.*,!=2.7.*,<3.0.0dev,>=1.34.1 in /usr/local/lib/python3.10/dist-packages (from google-cloud-aiplatform) (2.11.1)\n",
            "Requirement already satisfied: google-auth<3.0.0dev,>=2.14.1 in /usr/local/lib/python3.10/dist-packages (from google-cloud-aiplatform) (2.27.0)\n",
            "Requirement already satisfied: proto-plus<2.0.0dev,>=1.22.0 in /usr/local/lib/python3.10/dist-packages (from google-cloud-aiplatform) (1.23.0)\n",
            "Requirement already satisfied: protobuf!=3.20.0,!=3.20.1,!=4.21.0,!=4.21.1,!=4.21.2,!=4.21.3,!=4.21.4,!=4.21.5,<5.0.0dev,>=3.19.5 in /usr/local/lib/python3.10/dist-packages (from google-cloud-aiplatform) (3.20.3)\n",
            "Requirement already satisfied: packaging>=14.3 in /usr/local/lib/python3.10/dist-packages (from google-cloud-aiplatform) (24.0)\n",
            "Requirement already satisfied: google-cloud-storage<3.0.0dev,>=1.32.0 in /usr/local/lib/python3.10/dist-packages (from google-cloud-aiplatform) (2.8.0)\n",
            "Requirement already satisfied: google-cloud-bigquery!=3.20.0,<4.0.0dev,>=1.15.0 in /usr/local/lib/python3.10/dist-packages (from google-cloud-aiplatform) (3.21.0)\n",
            "Requirement already satisfied: google-cloud-resource-manager<3.0.0dev,>=1.3.3 in /usr/local/lib/python3.10/dist-packages (from google-cloud-aiplatform) (1.12.3)\n",
            "Requirement already satisfied: shapely<3.0.0dev in /usr/local/lib/python3.10/dist-packages (from google-cloud-aiplatform) (2.0.4)\n",
            "Requirement already satisfied: docstring-parser<1 in /usr/local/lib/python3.10/dist-packages (from google-cloud-aiplatform) (0.16)\n",
            "Requirement already satisfied: SQLAlchemy<3,>=1.4 in /usr/local/lib/python3.10/dist-packages (from langchain-community) (2.0.30)\n",
            "Requirement already satisfied: aiohttp<4.0.0,>=3.8.3 in /usr/local/lib/python3.10/dist-packages (from langchain-community) (3.9.5)\n",
            "Collecting dataclasses-json<0.7,>=0.5.7 (from langchain-community)\n",
            "  Downloading dataclasses_json-0.6.6-py3-none-any.whl (28 kB)\n",
            "Collecting langchain<0.3.0,>=0.2.0 (from langchain-community)\n",
            "  Downloading langchain-0.2.1-py3-none-any.whl (973 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m973.5/973.5 kB\u001b[0m \u001b[31m26.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting langchain-core<0.3.0,>=0.2.0 (from langchain-community)\n",
            "  Downloading langchain_core-0.2.1-py3-none-any.whl (308 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m308.5/308.5 kB\u001b[0m \u001b[31m18.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting langsmith<0.2.0,>=0.1.0 (from langchain-community)\n",
            "  Downloading langsmith-0.1.63-py3-none-any.whl (122 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m122.8/122.8 kB\u001b[0m \u001b[31m6.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: requests<3,>=2 in /usr/local/lib/python3.10/dist-packages (from langchain-community) (2.31.0)\n",
            "Requirement already satisfied: chardet in /usr/local/lib/python3.10/dist-packages (from unstructured[pdf]) (5.2.0)\n",
            "Collecting filetype (from unstructured[pdf])\n",
            "  Downloading filetype-1.2.0-py2.py3-none-any.whl (19 kB)\n",
            "Collecting python-magic (from unstructured[pdf])\n",
            "  Downloading python_magic-0.4.27-py2.py3-none-any.whl (13 kB)\n",
            "Requirement already satisfied: lxml in /usr/local/lib/python3.10/dist-packages (from unstructured[pdf]) (4.9.4)\n",
            "Requirement already satisfied: nltk in /usr/local/lib/python3.10/dist-packages (from unstructured[pdf]) (3.8.1)\n",
            "Requirement already satisfied: beautifulsoup4 in /usr/local/lib/python3.10/dist-packages (from unstructured[pdf]) (4.12.3)\n",
            "Collecting emoji (from unstructured[pdf])\n",
            "  Downloading emoji-2.12.1-py3-none-any.whl (431 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m431.4/431.4 kB\u001b[0m \u001b[31m25.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting python-iso639 (from unstructured[pdf])\n",
            "  Downloading python_iso639-2024.4.27-py3-none-any.whl (274 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m274.7/274.7 kB\u001b[0m \u001b[31m8.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting langdetect (from unstructured[pdf])\n",
            "  Downloading langdetect-1.0.9.tar.gz (981 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m981.5/981.5 kB\u001b[0m \u001b[31m11.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25h  Preparing metadata (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
            "Collecting rapidfuzz (from unstructured[pdf])\n",
            "  Downloading rapidfuzz-3.9.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.4 MB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m3.4/3.4 MB\u001b[0m \u001b[31m48.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting backoff (from unstructured[pdf])\n",
            "  Downloading backoff-2.2.1-py3-none-any.whl (15 kB)\n",
            "Requirement already satisfied: typing-extensions in /usr/local/lib/python3.10/dist-packages (from unstructured[pdf]) (4.11.0)\n",
            "Collecting unstructured-client (from unstructured[pdf])\n",
            "  Downloading unstructured_client-0.22.0-py3-none-any.whl (28 kB)\n",
            "Requirement already satisfied: wrapt in /usr/local/lib/python3.10/dist-packages (from unstructured[pdf]) (1.14.1)\n",
            "Collecting onnx (from unstructured[pdf])\n",
            "  Downloading onnx-1.16.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (15.9 MB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m15.9/15.9 MB\u001b[0m \u001b[31m57.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting pdf2image (from unstructured[pdf])\n",
            "  Downloading pdf2image-1.17.0-py3-none-any.whl (11 kB)\n",
            "Collecting pdfminer.six (from unstructured[pdf])\n",
            "  Downloading pdfminer.six-20231228-py3-none-any.whl (5.6 MB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m5.6/5.6 MB\u001b[0m \u001b[31m50.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting pikepdf (from unstructured[pdf])\n",
            "  Downloading pikepdf-8.15.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.4 MB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m2.4/2.4 MB\u001b[0m \u001b[31m30.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting pillow-heif (from unstructured[pdf])\n",
            "  Downloading pillow_heif-0.16.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (7.5 MB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m7.5/7.5 MB\u001b[0m \u001b[31m46.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting pypdf (from unstructured[pdf])\n",
            "  Downloading pypdf-4.2.0-py3-none-any.whl (290 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m290.4/290.4 kB\u001b[0m \u001b[31m19.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting pytesseract (from unstructured[pdf])\n",
            "  Downloading pytesseract-0.3.10-py3-none-any.whl (14 kB)\n",
            "Collecting google-cloud-vision (from unstructured[pdf])\n",
            "  Downloading google_cloud_vision-3.7.2-py2.py3-none-any.whl (459 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m459.6/459.6 kB\u001b[0m \u001b[31m22.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting effdet (from unstructured[pdf])\n",
            "  Downloading effdet-0.4.1-py3-none-any.whl (112 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m112.5/112.5 kB\u001b[0m \u001b[31m9.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting unstructured-inference==0.7.33 (from unstructured[pdf])\n",
            "  Downloading unstructured_inference-0.7.33-py3-none-any.whl (56 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m56.2/56.2 kB\u001b[0m \u001b[31m3.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting unstructured.pytesseract>=0.3.12 (from unstructured[pdf])\n",
            "  Downloading unstructured.pytesseract-0.3.12-py3-none-any.whl (14 kB)\n",
            "Collecting layoutparser (from unstructured-inference==0.7.33->unstructured[pdf])\n",
            "  Downloading layoutparser-0.3.4-py3-none-any.whl (19.2 MB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m19.2/19.2 MB\u001b[0m \u001b[31m38.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting python-multipart (from unstructured-inference==0.7.33->unstructured[pdf])\n",
            "  Downloading python_multipart-0.0.9-py3-none-any.whl (22 kB)\n",
            "Requirement already satisfied: huggingface-hub in /usr/local/lib/python3.10/dist-packages (from unstructured-inference==0.7.33->unstructured[pdf]) (0.23.1)\n",
            "Requirement already satisfied: opencv-python!=4.7.0.68 in /usr/local/lib/python3.10/dist-packages (from unstructured-inference==0.7.33->unstructured[pdf]) (4.8.0.76)\n",
            "Collecting onnxruntime>=1.17.0 (from unstructured-inference==0.7.33->unstructured[pdf])\n",
            "  Downloading onnxruntime-1.18.0-cp310-cp310-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl (6.8 MB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m6.8/6.8 MB\u001b[0m \u001b[31m59.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: matplotlib in /usr/local/lib/python3.10/dist-packages (from unstructured-inference==0.7.33->unstructured[pdf]) (3.7.1)\n",
            "Requirement already satisfied: torch in /usr/local/lib/python3.10/dist-packages (from unstructured-inference==0.7.33->unstructured[pdf]) (2.3.0+cu121)\n",
            "Collecting timm (from unstructured-inference==0.7.33->unstructured[pdf])\n",
            "  Downloading timm-1.0.3-py3-none-any.whl (2.3 MB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m2.3/2.3 MB\u001b[0m \u001b[31m59.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: transformers>=4.25.1 in /usr/local/lib/python3.10/dist-packages (from unstructured-inference==0.7.33->unstructured[pdf]) (4.41.1)\n",
            "Collecting aiofiles<24.0,>=22.0 (from gradio)\n",
            "  Downloading aiofiles-23.2.1-py3-none-any.whl (15 kB)\n",
            "Requirement already satisfied: altair<6.0,>=4.2.0 in /usr/local/lib/python3.10/dist-packages (from gradio) (4.2.2)\n",
            "Collecting fastapi (from gradio)\n",
            "  Downloading fastapi-0.111.0-py3-none-any.whl (91 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m92.0/92.0 kB\u001b[0m \u001b[31m11.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting ffmpy (from gradio)\n",
            "  Downloading ffmpy-0.3.2.tar.gz (5.5 kB)\n",
            "  Preparing metadata (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
            "Collecting gradio-client==0.16.4 (from gradio)\n",
            "  Downloading gradio_client-0.16.4-py3-none-any.whl (315 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m315.9/315.9 kB\u001b[0m \u001b[31m33.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting httpx>=0.24.1 (from gradio)\n",
            "  Downloading httpx-0.27.0-py3-none-any.whl (75 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m75.6/75.6 kB\u001b[0m \u001b[31m8.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: importlib-resources<7.0,>=1.3 in /usr/local/lib/python3.10/dist-packages (from gradio) (6.4.0)\n",
            "Requirement already satisfied: jinja2<4.0 in /usr/local/lib/python3.10/dist-packages (from gradio) (3.1.4)\n",
            "Requirement already satisfied: markupsafe~=2.0 in /usr/local/lib/python3.10/dist-packages (from gradio) (2.1.5)\n",
            "Collecting orjson~=3.0 (from gradio)\n",
            "  Downloading orjson-3.10.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (142 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m142.5/142.5 kB\u001b[0m \u001b[31m16.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: pandas<3.0,>=1.0 in /usr/local/lib/python3.10/dist-packages (from gradio) (2.0.3)\n",
            "Requirement already satisfied: pillow<11.0,>=8.0 in /usr/local/lib/python3.10/dist-packages (from gradio) (9.4.0)\n",
            "Collecting pydub (from gradio)\n",
            "  Downloading pydub-0.25.1-py2.py3-none-any.whl (32 kB)\n",
            "Collecting ruff>=0.2.2 (from gradio)\n",
            "  Downloading ruff-0.4.6-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (8.8 MB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m8.8/8.8 MB\u001b[0m \u001b[31m84.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting semantic-version~=2.0 (from gradio)\n",
            "  Downloading semantic_version-2.10.0-py2.py3-none-any.whl (15 kB)\n",
            "Collecting tomlkit==0.12.0 (from gradio)\n",
            "  Downloading tomlkit-0.12.0-py3-none-any.whl (37 kB)\n",
            "Collecting typer<1.0,>=0.12 (from gradio)\n",
            "  Downloading typer-0.12.3-py3-none-any.whl (47 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m47.2/47.2 kB\u001b[0m \u001b[31m6.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: urllib3~=2.0 in /usr/local/lib/python3.10/dist-packages (from gradio) (2.0.7)\n",
            "Collecting uvicorn>=0.14.0 (from gradio)\n",
            "  Downloading uvicorn-0.30.0-py3-none-any.whl (62 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m62.4/62.4 kB\u001b[0m \u001b[31m8.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: fsspec in /usr/local/lib/python3.10/dist-packages (from gradio-client==0.16.4->gradio) (2023.6.0)\n",
            "Collecting websockets<12.0,>=10.0 (from gradio-client==0.16.4->gradio)\n",
            "  Downloading websockets-11.0.3-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (129 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m129.9/129.9 kB\u001b[0m \u001b[31m9.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: aiosignal>=1.1.2 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0.0,>=3.8.3->langchain-community) (1.3.1)\n",
            "Requirement already satisfied: attrs>=17.3.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0.0,>=3.8.3->langchain-community) (23.2.0)\n",
            "Requirement already satisfied: frozenlist>=1.1.1 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0.0,>=3.8.3->langchain-community) (1.4.1)\n",
            "Requirement already satisfied: multidict<7.0,>=4.5 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0.0,>=3.8.3->langchain-community) (6.0.5)\n",
            "Requirement already satisfied: yarl<2.0,>=1.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0.0,>=3.8.3->langchain-community) (1.9.4)\n",
            "Requirement already satisfied: async-timeout<5.0,>=4.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0.0,>=3.8.3->langchain-community) (4.0.3)\n",
            "Requirement already satisfied: entrypoints in /usr/local/lib/python3.10/dist-packages (from altair<6.0,>=4.2.0->gradio) (0.4)\n",
            "Requirement already satisfied: jsonschema>=3.0 in /usr/local/lib/python3.10/dist-packages (from altair<6.0,>=4.2.0->gradio) (4.19.2)\n",
            "Requirement already satisfied: toolz in /usr/local/lib/python3.10/dist-packages (from altair<6.0,>=4.2.0->gradio) (0.12.1)\n",
            "Collecting marshmallow<4.0.0,>=3.18.0 (from dataclasses-json<0.7,>=0.5.7->langchain-community)\n",
            "  Downloading marshmallow-3.21.2-py3-none-any.whl (49 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m49.3/49.3 kB\u001b[0m \u001b[31m6.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting typing-inspect<1,>=0.4.0 (from dataclasses-json<0.7,>=0.5.7->langchain-community)\n",
            "  Downloading typing_inspect-0.9.0-py3-none-any.whl (8.8 kB)\n",
            "Requirement already satisfied: googleapis-common-protos<2.0.dev0,>=1.56.2 in /usr/local/lib/python3.10/dist-packages (from google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.*,!=2.4.*,!=2.5.*,!=2.6.*,!=2.7.*,<3.0.0dev,>=1.34.1->google-cloud-aiplatform) (1.63.0)\n",
            "Requirement already satisfied: grpcio<2.0dev,>=1.33.2 in /usr/local/lib/python3.10/dist-packages (from google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.*,!=2.4.*,!=2.5.*,!=2.6.*,!=2.7.*,<3.0.0dev,>=1.34.1->google-cloud-aiplatform) (1.64.0)\n",
            "Requirement already satisfied: grpcio-status<2.0.dev0,>=1.33.2 in /usr/local/lib/python3.10/dist-packages (from google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.*,!=2.4.*,!=2.5.*,!=2.6.*,!=2.7.*,<3.0.0dev,>=1.34.1->google-cloud-aiplatform) (1.48.2)\n",
            "Requirement already satisfied: cachetools<6.0,>=2.0.0 in /usr/local/lib/python3.10/dist-packages (from google-auth<3.0.0dev,>=2.14.1->google-cloud-aiplatform) (5.3.3)\n",
            "Requirement already satisfied: pyasn1-modules>=0.2.1 in /usr/local/lib/python3.10/dist-packages (from google-auth<3.0.0dev,>=2.14.1->google-cloud-aiplatform) (0.4.0)\n",
            "Requirement already satisfied: rsa<5,>=3.1.4 in /usr/local/lib/python3.10/dist-packages (from google-auth<3.0.0dev,>=2.14.1->google-cloud-aiplatform) (4.9)\n",
            "Requirement already satisfied: google-cloud-core<3.0.0dev,>=1.6.0 in /usr/local/lib/python3.10/dist-packages (from google-cloud-bigquery!=3.20.0,<4.0.0dev,>=1.15.0->google-cloud-aiplatform) (2.3.3)\n",
            "Requirement already satisfied: google-resumable-media<3.0dev,>=0.6.0 in /usr/local/lib/python3.10/dist-packages (from google-cloud-bigquery!=3.20.0,<4.0.0dev,>=1.15.0->google-cloud-aiplatform) (2.7.0)\n",
            "Requirement already satisfied: python-dateutil<3.0dev,>=2.7.2 in /usr/local/lib/python3.10/dist-packages (from google-cloud-bigquery!=3.20.0,<4.0.0dev,>=1.15.0->google-cloud-aiplatform) (2.8.2)\n",
            "Requirement already satisfied: grpc-google-iam-v1<1.0.0dev,>=0.12.4 in /usr/local/lib/python3.10/dist-packages (from google-cloud-resource-manager<3.0.0dev,>=1.3.3->google-cloud-aiplatform) (0.13.0)\n",
            "Requirement already satisfied: anyio in /usr/local/lib/python3.10/dist-packages (from httpx>=0.24.1->gradio) (3.7.1)\n",
            "Requirement already satisfied: certifi in /usr/local/lib/python3.10/dist-packages (from httpx>=0.24.1->gradio) (2024.2.2)\n",
            "Collecting httpcore==1.* (from httpx>=0.24.1->gradio)\n",
            "  Downloading httpcore-1.0.5-py3-none-any.whl (77 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m77.9/77.9 kB\u001b[0m \u001b[31m9.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: idna in /usr/local/lib/python3.10/dist-packages (from httpx>=0.24.1->gradio) (3.7)\n",
            "Requirement already satisfied: sniffio in /usr/local/lib/python3.10/dist-packages (from httpx>=0.24.1->gradio) (1.3.1)\n",
            "Collecting h11<0.15,>=0.13 (from httpcore==1.*->httpx>=0.24.1->gradio)\n",
            "  Downloading h11-0.14.0-py3-none-any.whl (58 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m58.3/58.3 kB\u001b[0m \u001b[31m8.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: filelock in /usr/local/lib/python3.10/dist-packages (from huggingface-hub->unstructured-inference==0.7.33->unstructured[pdf]) (3.14.0)\n",
            "Requirement already satisfied: tqdm>=4.42.1 in /usr/local/lib/python3.10/dist-packages (from huggingface-hub->unstructured-inference==0.7.33->unstructured[pdf]) (4.66.4)\n",
            "Collecting langchain-text-splitters<0.3.0,>=0.2.0 (from langchain<0.3.0,>=0.2.0->langchain-community)\n",
            "  Downloading langchain_text_splitters-0.2.0-py3-none-any.whl (23 kB)\n",
            "Collecting jsonpatch<2.0,>=1.33 (from langchain-core<0.3.0,>=0.2.0->langchain-community)\n",
            "  Downloading jsonpatch-1.33-py2.py3-none-any.whl (12 kB)\n",
            "Collecting packaging>=14.3 (from google-cloud-aiplatform)\n",
            "  Downloading packaging-23.2-py3-none-any.whl (53 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m53.0/53.0 kB\u001b[0m \u001b[31m6.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: contourpy>=1.0.1 in /usr/local/lib/python3.10/dist-packages (from matplotlib->unstructured-inference==0.7.33->unstructured[pdf]) (1.2.1)\n",
            "Requirement already satisfied: cycler>=0.10 in /usr/local/lib/python3.10/dist-packages (from matplotlib->unstructured-inference==0.7.33->unstructured[pdf]) (0.12.1)\n",
            "Requirement already satisfied: fonttools>=4.22.0 in /usr/local/lib/python3.10/dist-packages (from matplotlib->unstructured-inference==0.7.33->unstructured[pdf]) (4.51.0)\n",
            "Requirement already satisfied: kiwisolver>=1.0.1 in /usr/local/lib/python3.10/dist-packages (from matplotlib->unstructured-inference==0.7.33->unstructured[pdf]) (1.4.5)\n",
            "Requirement already satisfied: pyparsing>=2.3.1 in /usr/local/lib/python3.10/dist-packages (from matplotlib->unstructured-inference==0.7.33->unstructured[pdf]) (3.1.2)\n",
            "Requirement already satisfied: pytz>=2020.1 in /usr/local/lib/python3.10/dist-packages (from pandas<3.0,>=1.0->gradio) (2023.4)\n",
            "Requirement already satisfied: tzdata>=2022.1 in /usr/local/lib/python3.10/dist-packages (from pandas<3.0,>=1.0->gradio) (2024.1)\n",
            "Requirement already satisfied: annotated-types>=0.4.0 in /usr/local/lib/python3.10/dist-packages (from pydantic<3,>=2->redisvl) (0.7.0)\n",
            "Requirement already satisfied: pydantic-core==2.18.2 in /usr/local/lib/python3.10/dist-packages (from pydantic<3,>=2->redisvl) (2.18.2)\n",
            "Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.10/dist-packages (from requests<3,>=2->langchain-community) (3.3.2)\n",
            "Requirement already satisfied: greenlet!=0.4.17 in /usr/local/lib/python3.10/dist-packages (from SQLAlchemy<3,>=1.4->langchain-community) (3.0.3)\n",
            "Requirement already satisfied: click>=8.0.0 in /usr/local/lib/python3.10/dist-packages (from typer<1.0,>=0.12->gradio) (8.1.7)\n",
            "Collecting shellingham>=1.3.0 (from typer<1.0,>=0.12->gradio)\n",
            "  Downloading shellingham-1.5.4-py2.py3-none-any.whl (9.8 kB)\n",
            "Requirement already satisfied: rich>=10.11.0 in /usr/local/lib/python3.10/dist-packages (from typer<1.0,>=0.12->gradio) (13.7.1)\n",
            "Requirement already satisfied: soupsieve>1.2 in /usr/local/lib/python3.10/dist-packages (from beautifulsoup4->unstructured[pdf]) (2.5)\n",
            "Collecting humanfriendly>=9.1 (from coloredlogs->redisvl)\n",
            "  Downloading humanfriendly-10.0-py2.py3-none-any.whl (86 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m86.8/86.8 kB\u001b[0m \u001b[31m11.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: torchvision in /usr/local/lib/python3.10/dist-packages (from effdet->unstructured[pdf]) (0.18.0+cu121)\n",
            "Requirement already satisfied: pycocotools>=2.0.2 in /usr/local/lib/python3.10/dist-packages (from effdet->unstructured[pdf]) (2.0.7)\n",
            "Collecting omegaconf>=2.0 (from effdet->unstructured[pdf])\n",
            "  Downloading omegaconf-2.3.0-py3-none-any.whl (79 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m79.5/79.5 kB\u001b[0m \u001b[31m10.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting starlette<0.38.0,>=0.37.2 (from fastapi->gradio)\n",
            "  Downloading starlette-0.37.2-py3-none-any.whl (71 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m71.9/71.9 kB\u001b[0m \u001b[31m8.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting fastapi-cli>=0.0.2 (from fastapi->gradio)\n",
            "  Downloading fastapi_cli-0.0.4-py3-none-any.whl (9.5 kB)\n",
            "Collecting ujson!=4.0.2,!=4.1.0,!=4.2.0,!=4.3.0,!=5.0.0,!=5.1.0,>=4.0.1 (from fastapi->gradio)\n",
            "  Downloading ujson-5.10.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (53 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m53.6/53.6 kB\u001b[0m \u001b[31m6.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting email_validator>=2.0.0 (from fastapi->gradio)\n",
            "  Downloading email_validator-2.1.1-py3-none-any.whl (30 kB)\n",
            "Requirement already satisfied: six in /usr/local/lib/python3.10/dist-packages (from langdetect->unstructured[pdf]) (1.16.0)\n",
            "Requirement already satisfied: joblib in /usr/local/lib/python3.10/dist-packages (from nltk->unstructured[pdf]) (1.4.2)\n",
            "Requirement already satisfied: regex>=2021.8.3 in /usr/local/lib/python3.10/dist-packages (from nltk->unstructured[pdf]) (2024.5.15)\n",
            "Requirement already satisfied: cryptography>=36.0.0 in /usr/local/lib/python3.10/dist-packages (from pdfminer.six->unstructured[pdf]) (42.0.7)\n",
            "Collecting pillow<11.0,>=8.0 (from gradio)\n",
            "  Downloading pillow-10.3.0-cp310-cp310-manylinux_2_28_x86_64.whl (4.5 MB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m4.5/4.5 MB\u001b[0m \u001b[31m21.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting Deprecated (from pikepdf->unstructured[pdf])\n",
            "  Downloading Deprecated-1.2.14-py2.py3-none-any.whl (9.6 kB)\n",
            "Collecting deepdiff>=6.0 (from unstructured-client->unstructured[pdf])\n",
            "  Downloading deepdiff-7.0.1-py3-none-any.whl (80 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m80.8/80.8 kB\u001b[0m \u001b[31m9.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting jsonpath-python>=1.0.6 (from unstructured-client->unstructured[pdf])\n",
            "  Downloading jsonpath_python-1.0.6-py3-none-any.whl (7.6 kB)\n",
            "Collecting mypy-extensions>=1.0.0 (from unstructured-client->unstructured[pdf])\n",
            "  Downloading mypy_extensions-1.0.0-py3-none-any.whl (4.7 kB)\n",
            "Requirement already satisfied: cffi>=1.12 in /usr/local/lib/python3.10/dist-packages (from cryptography>=36.0.0->pdfminer.six->unstructured[pdf]) (1.16.0)\n",
            "Collecting ordered-set<4.2.0,>=4.1.0 (from deepdiff>=6.0->unstructured-client->unstructured[pdf])\n",
            "  Downloading ordered_set-4.1.0-py3-none-any.whl (7.6 kB)\n",
            "Collecting dnspython>=2.0.0 (from email_validator>=2.0.0->fastapi->gradio)\n",
            "  Downloading dnspython-2.6.1-py3-none-any.whl (307 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m307.7/307.7 kB\u001b[0m \u001b[31m31.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: google-crc32c<2.0dev,>=1.0 in /usr/local/lib/python3.10/dist-packages (from google-resumable-media<3.0dev,>=0.6.0->google-cloud-bigquery!=3.20.0,<4.0.0dev,>=1.15.0->google-cloud-aiplatform) (1.5.0)\n",
            "Collecting jsonpointer>=1.9 (from jsonpatch<2.0,>=1.33->langchain-core<0.3.0,>=0.2.0->langchain-community)\n",
            "  Downloading jsonpointer-2.4-py2.py3-none-any.whl (7.8 kB)\n",
            "Requirement already satisfied: jsonschema-specifications>=2023.03.6 in /usr/local/lib/python3.10/dist-packages (from jsonschema>=3.0->altair<6.0,>=4.2.0->gradio) (2023.12.1)\n",
            "Requirement already satisfied: referencing>=0.28.4 in /usr/local/lib/python3.10/dist-packages (from jsonschema>=3.0->altair<6.0,>=4.2.0->gradio) (0.35.1)\n",
            "Requirement already satisfied: rpds-py>=0.7.1 in /usr/local/lib/python3.10/dist-packages (from jsonschema>=3.0->altair<6.0,>=4.2.0->gradio) (0.18.1)\n",
            "Collecting antlr4-python3-runtime==4.9.* (from omegaconf>=2.0->effdet->unstructured[pdf])\n",
            "  Downloading antlr4-python3-runtime-4.9.3.tar.gz (117 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m117.0/117.0 kB\u001b[0m \u001b[31m12.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25h  Preparing metadata (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
            "Requirement already satisfied: flatbuffers in /usr/local/lib/python3.10/dist-packages (from onnxruntime>=1.17.0->unstructured-inference==0.7.33->unstructured[pdf]) (24.3.25)\n",
            "Requirement already satisfied: sympy in /usr/local/lib/python3.10/dist-packages (from onnxruntime>=1.17.0->unstructured-inference==0.7.33->unstructured[pdf]) (1.12)\n",
            "Requirement already satisfied: pyasn1<0.7.0,>=0.4.6 in /usr/local/lib/python3.10/dist-packages (from pyasn1-modules>=0.2.1->google-auth<3.0.0dev,>=2.14.1->google-cloud-aiplatform) (0.6.0)\n",
            "Requirement already satisfied: markdown-it-py>=2.2.0 in /usr/local/lib/python3.10/dist-packages (from rich>=10.11.0->typer<1.0,>=0.12->gradio) (3.0.0)\n",
            "Requirement already satisfied: pygments<3.0.0,>=2.13.0 in /usr/local/lib/python3.10/dist-packages (from rich>=10.11.0->typer<1.0,>=0.12->gradio) (2.16.1)\n",
            "Requirement already satisfied: exceptiongroup in /usr/local/lib/python3.10/dist-packages (from anyio->httpx>=0.24.1->gradio) (1.2.1)\n",
            "Requirement already satisfied: safetensors in /usr/local/lib/python3.10/dist-packages (from timm->unstructured-inference==0.7.33->unstructured[pdf]) (0.4.3)\n",
            "Requirement already satisfied: networkx in /usr/local/lib/python3.10/dist-packages (from torch->unstructured-inference==0.7.33->unstructured[pdf]) (3.3)\n",
            "Collecting nvidia-cuda-nvrtc-cu12==12.1.105 (from torch->unstructured-inference==0.7.33->unstructured[pdf])\n",
            "  Using cached nvidia_cuda_nvrtc_cu12-12.1.105-py3-none-manylinux1_x86_64.whl (23.7 MB)\n",
            "Collecting nvidia-cuda-runtime-cu12==12.1.105 (from torch->unstructured-inference==0.7.33->unstructured[pdf])\n",
            "  Using cached nvidia_cuda_runtime_cu12-12.1.105-py3-none-manylinux1_x86_64.whl (823 kB)\n",
            "Collecting nvidia-cuda-cupti-cu12==12.1.105 (from torch->unstructured-inference==0.7.33->unstructured[pdf])\n",
            "  Using cached nvidia_cuda_cupti_cu12-12.1.105-py3-none-manylinux1_x86_64.whl (14.1 MB)\n",
            "Collecting nvidia-cudnn-cu12==8.9.2.26 (from torch->unstructured-inference==0.7.33->unstructured[pdf])\n",
            "  Using cached nvidia_cudnn_cu12-8.9.2.26-py3-none-manylinux1_x86_64.whl (731.7 MB)\n",
            "Collecting nvidia-cublas-cu12==12.1.3.1 (from torch->unstructured-inference==0.7.33->unstructured[pdf])\n",
            "  Using cached nvidia_cublas_cu12-12.1.3.1-py3-none-manylinux1_x86_64.whl (410.6 MB)\n",
            "Collecting nvidia-cufft-cu12==11.0.2.54 (from torch->unstructured-inference==0.7.33->unstructured[pdf])\n",
            "  Using cached nvidia_cufft_cu12-11.0.2.54-py3-none-manylinux1_x86_64.whl (121.6 MB)\n",
            "Collecting nvidia-curand-cu12==10.3.2.106 (from torch->unstructured-inference==0.7.33->unstructured[pdf])\n",
            "  Using cached nvidia_curand_cu12-10.3.2.106-py3-none-manylinux1_x86_64.whl (56.5 MB)\n",
            "Collecting nvidia-cusolver-cu12==11.4.5.107 (from torch->unstructured-inference==0.7.33->unstructured[pdf])\n",
            "  Using cached nvidia_cusolver_cu12-11.4.5.107-py3-none-manylinux1_x86_64.whl (124.2 MB)\n",
            "Collecting nvidia-cusparse-cu12==12.1.0.106 (from torch->unstructured-inference==0.7.33->unstructured[pdf])\n",
            "  Using cached nvidia_cusparse_cu12-12.1.0.106-py3-none-manylinux1_x86_64.whl (196.0 MB)\n",
            "Collecting nvidia-nccl-cu12==2.20.5 (from torch->unstructured-inference==0.7.33->unstructured[pdf])\n",
            "  Using cached nvidia_nccl_cu12-2.20.5-py3-none-manylinux2014_x86_64.whl (176.2 MB)\n",
            "Collecting nvidia-nvtx-cu12==12.1.105 (from torch->unstructured-inference==0.7.33->unstructured[pdf])\n",
            "  Using cached nvidia_nvtx_cu12-12.1.105-py3-none-manylinux1_x86_64.whl (99 kB)\n",
            "Requirement already satisfied: triton==2.3.0 in /usr/local/lib/python3.10/dist-packages (from torch->unstructured-inference==0.7.33->unstructured[pdf]) (2.3.0)\n",
            "Collecting nvidia-nvjitlink-cu12 (from nvidia-cusolver-cu12==11.4.5.107->torch->unstructured-inference==0.7.33->unstructured[pdf])\n",
            "  Downloading nvidia_nvjitlink_cu12-12.5.40-py3-none-manylinux2014_x86_64.whl (21.3 MB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m21.3/21.3 MB\u001b[0m \u001b[31m60.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: tokenizers<0.20,>=0.19 in /usr/local/lib/python3.10/dist-packages (from transformers>=4.25.1->unstructured-inference==0.7.33->unstructured[pdf]) (0.19.1)\n",
            "Collecting httptools>=0.5.0 (from uvicorn>=0.14.0->gradio)\n",
            "  Downloading httptools-0.6.1-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (341 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m341.4/341.4 kB\u001b[0m \u001b[31m34.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting python-dotenv>=0.13 (from uvicorn>=0.14.0->gradio)\n",
            "  Downloading python_dotenv-1.0.1-py3-none-any.whl (19 kB)\n",
            "Collecting uvloop!=0.15.0,!=0.15.1,>=0.14.0 (from uvicorn>=0.14.0->gradio)\n",
            "  Downloading uvloop-0.19.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.4 MB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m3.4/3.4 MB\u001b[0m \u001b[31m67.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting watchfiles>=0.13 (from uvicorn>=0.14.0->gradio)\n",
            "  Downloading watchfiles-0.22.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.2 MB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.2/1.2 MB\u001b[0m \u001b[31m49.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: scipy in /usr/local/lib/python3.10/dist-packages (from layoutparser->unstructured-inference==0.7.33->unstructured[pdf]) (1.11.4)\n",
            "Collecting iopath (from layoutparser->unstructured-inference==0.7.33->unstructured[pdf])\n",
            "  Downloading iopath-0.1.10.tar.gz (42 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m42.2/42.2 kB\u001b[0m \u001b[31m4.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25h  Preparing metadata (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
            "Collecting pdfplumber (from layoutparser->unstructured-inference==0.7.33->unstructured[pdf])\n",
            "  Downloading pdfplumber-0.11.0-py3-none-any.whl (56 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m56.4/56.4 kB\u001b[0m \u001b[31m7.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: pycparser in /usr/local/lib/python3.10/dist-packages (from cffi>=1.12->cryptography>=36.0.0->pdfminer.six->unstructured[pdf]) (2.22)\n",
            "Requirement already satisfied: mdurl~=0.1 in /usr/local/lib/python3.10/dist-packages (from markdown-it-py>=2.2.0->rich>=10.11.0->typer<1.0,>=0.12->gradio) (0.1.2)\n",
            "Collecting portalocker (from iopath->layoutparser->unstructured-inference==0.7.33->unstructured[pdf])\n",
            "  Downloading portalocker-2.8.2-py3-none-any.whl (17 kB)\n",
            "Collecting pypdfium2>=4.18.0 (from pdfplumber->layoutparser->unstructured-inference==0.7.33->unstructured[pdf])\n",
            "  Downloading pypdfium2-4.30.0-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.8 MB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m2.8/2.8 MB\u001b[0m \u001b[31m90.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: mpmath>=0.19 in /usr/local/lib/python3.10/dist-packages (from sympy->onnxruntime>=1.17.0->unstructured-inference==0.7.33->unstructured[pdf]) (1.3.0)\n",
            "Building wheels for collected packages: ffmpy, langdetect, antlr4-python3-runtime, iopath\n",
            "  Building wheel for ffmpy (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
            "  Created wheel for ffmpy: filename=ffmpy-0.3.2-py3-none-any.whl size=5584 sha256=4a246b2d6c476603c97ede4ea813b21cc294654d5fd1e1e3f3edae0294c86ef4\n",
            "  Stored in directory: /root/.cache/pip/wheels/bd/65/9a/671fc6dcde07d4418df0c592f8df512b26d7a0029c2a23dd81\n",
            "  Building wheel for langdetect (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
            "  Created wheel for langdetect: filename=langdetect-1.0.9-py3-none-any.whl size=993227 sha256=6506fc3e8eaea7efe256e8a35215d94a82259e3ff05368c5b4c942de00352cda\n",
            "  Stored in directory: /root/.cache/pip/wheels/95/03/7d/59ea870c70ce4e5a370638b5462a7711ab78fba2f655d05106\n",
            "  Building wheel for antlr4-python3-runtime (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
            "  Created wheel for antlr4-python3-runtime: filename=antlr4_python3_runtime-4.9.3-py3-none-any.whl size=144554 sha256=bbdaeecb21d60240939c6c994934ed32decf60e284169d4f6cda20729005279a\n",
            "  Stored in directory: /root/.cache/pip/wheels/12/93/dd/1f6a127edc45659556564c5730f6d4e300888f4bca2d4c5a88\n",
            "  Building wheel for iopath (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
            "  Created wheel for iopath: filename=iopath-0.1.10-py3-none-any.whl size=31532 sha256=6acff56c46639790ab035a659f2b4442ed10883d95bfe85c44d1d53dfce3bd8d\n",
            "  Stored in directory: /root/.cache/pip/wheels/9a/a3/b6/ac0fcd1b4ed5cfeb3db92e6a0e476cfd48ed0df92b91080c1d\n",
            "Successfully built ffmpy langdetect antlr4-python3-runtime iopath\n",
            "Installing collected packages: pydub, filetype, ffmpy, antlr4-python3-runtime, websockets, uvloop, ujson, tomlkit, shellingham, semantic-version, ruff, redis, rapidfuzz, python-multipart, python-magic, python-iso639, python-dotenv, pypdfium2, pypdf, portalocker, pillow, packaging, orjson, ordered-set, onnx, omegaconf, nvidia-nvtx-cu12, nvidia-nvjitlink-cu12, nvidia-nccl-cu12, nvidia-curand-cu12, nvidia-cufft-cu12, nvidia-cuda-runtime-cu12, nvidia-cuda-nvrtc-cu12, nvidia-cuda-cupti-cu12, nvidia-cublas-cu12, mypy-extensions, langdetect, jsonpointer, jsonpath-python, humanfriendly, httptools, h11, emoji, dnspython, Deprecated, backoff, aiofiles, watchfiles, uvicorn, unstructured.pytesseract, typing-inspect, starlette, pytesseract, pillow-heif, pikepdf, pdf2image, nvidia-cusparse-cu12, nvidia-cudnn-cu12, marshmallow, jsonpatch, iopath, httpcore, email_validator, deepdiff, coloredlogs, typer, redisvl, pdfminer.six, onnxruntime, nvidia-cusolver-cu12, langsmith, httpx, dataclasses-json, unstructured-client, pdfplumber, langchain-core, gradio-client, fastapi-cli, unstructured, layoutparser, langchain-text-splitters, google-cloud-vision, fastapi, timm, langchain, gradio, unstructured-inference, langchain-community, effdet\n",
            "  Attempting uninstall: pillow\n",
            "    Found existing installation: Pillow 9.4.0\n",
            "    Uninstalling Pillow-9.4.0:\n",
            "      Successfully uninstalled Pillow-9.4.0\n",
            "  Attempting uninstall: packaging\n",
            "    Found existing installation: packaging 24.0\n",
            "    Uninstalling packaging-24.0:\n",
            "      Successfully uninstalled packaging-24.0\n",
            "  Attempting uninstall: typer\n",
            "    Found existing installation: typer 0.9.4\n",
            "    Uninstalling typer-0.9.4:\n",
            "      Successfully uninstalled typer-0.9.4\n",
            "\u001b[31mERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.\n",
            "imageio 2.31.6 requires pillow<10.1.0,>=8.3.2, but you have pillow 10.3.0 which is incompatible.\n",
            "spacy 3.7.4 requires typer<0.10.0,>=0.3.0, but you have typer 0.12.3 which is incompatible.\n",
            "weasel 0.3.4 requires typer<0.10.0,>=0.3.0, but you have typer 0.12.3 which is incompatible.\u001b[0m\u001b[31m\n",
            "\u001b[0mSuccessfully installed Deprecated-1.2.14 aiofiles-23.2.1 antlr4-python3-runtime-4.9.3 backoff-2.2.1 coloredlogs-15.0.1 dataclasses-json-0.6.6 deepdiff-7.0.1 dnspython-2.6.1 effdet-0.4.1 email_validator-2.1.1 emoji-2.12.1 fastapi-0.111.0 fastapi-cli-0.0.4 ffmpy-0.3.2 filetype-1.2.0 google-cloud-vision-3.7.2 gradio-4.31.5 gradio-client-0.16.4 h11-0.14.0 httpcore-1.0.5 httptools-0.6.1 httpx-0.27.0 humanfriendly-10.0 iopath-0.1.10 jsonpatch-1.33 jsonpath-python-1.0.6 jsonpointer-2.4 langchain-0.2.1 langchain-community-0.2.1 langchain-core-0.2.1 langchain-text-splitters-0.2.0 langdetect-1.0.9 langsmith-0.1.63 layoutparser-0.3.4 marshmallow-3.21.2 mypy-extensions-1.0.0 nvidia-cublas-cu12-12.1.3.1 nvidia-cuda-cupti-cu12-12.1.105 nvidia-cuda-nvrtc-cu12-12.1.105 nvidia-cuda-runtime-cu12-12.1.105 nvidia-cudnn-cu12-8.9.2.26 nvidia-cufft-cu12-11.0.2.54 nvidia-curand-cu12-10.3.2.106 nvidia-cusolver-cu12-11.4.5.107 nvidia-cusparse-cu12-12.1.0.106 nvidia-nccl-cu12-2.20.5 nvidia-nvjitlink-cu12-12.5.40 nvidia-nvtx-cu12-12.1.105 omegaconf-2.3.0 onnx-1.16.1 onnxruntime-1.18.0 ordered-set-4.1.0 orjson-3.10.3 packaging-23.2 pdf2image-1.17.0 pdfminer.six-20231228 pdfplumber-0.11.0 pikepdf-8.15.1 pillow-10.3.0 pillow-heif-0.16.0 portalocker-2.8.2 pydub-0.25.1 pypdf-4.2.0 pypdfium2-4.30.0 pytesseract-0.3.10 python-dotenv-1.0.1 python-iso639-2024.4.27 python-magic-0.4.27 python-multipart-0.0.9 rapidfuzz-3.9.2 redis-5.0.4 redisvl-0.2.0 ruff-0.4.6 semantic-version-2.10.0 shellingham-1.5.4 starlette-0.37.2 timm-1.0.3 tomlkit-0.12.0 typer-0.12.3 typing-inspect-0.9.0 ujson-5.10.0 unstructured-0.14.3 unstructured-client-0.22.0 unstructured-inference-0.7.33 unstructured.pytesseract-0.3.12 uvicorn-0.30.0 uvloop-0.19.0 watchfiles-0.22.0 websockets-11.0.3\n"
          ]
        },
        {
          "output_type": "display_data",
          "data": {
            "application/vnd.colab-display-data+json": {
              "pip_warning": {
                "packages": [
                  "PIL",
                  "google",
                  "pydevd_plugins"
                ]
              },
              "id": "3b79e51e64a24b3ea311607a182d9806"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "{'status': 'ok', 'restart': True}"
            ]
          },
          "metadata": {},
          "execution_count": 1
        }
      ],
      "source": [
        "!pip install -U redisvl google-cloud-aiplatform langchain-community unstructured[pdf] gradio\n",
        "\n",
        "import IPython\n",
        "\n",
        "app = IPython.Application.instance()\n",
        "app.kernel.do_shutdown(True)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "jr_IviwqFS7K"
      },
      "source": [
        "### Using Free Redis Cloud account on GCP\n",
        "You can also use Forever Free instance of Redis Cloud. To activate it:\n",
        "- Head to https://redis.com/try-free/\n",
        "- Register (using gmail-based registration is the easiest)\n",
        "- Create New Subscription\n",
        "- Use the following options:\n",
        "    - Fixed plan, Google Cloud\n",
        "    - New 30Mb Free database\n",
        "- Create new RedisStack DB\n",
        "\n",
        "If you are registering at Redis Cloud for the first time - the last few steps would be performed for you by default. Capture the host, port and default password of the new database. You can use these instead of default `localhost` based in the following code block."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "p5kx9ePDwwp6"
      },
      "source": [
        "^^^ If prompted press the Restart button to restart the kernel. ^^^\n",
        "\n",
        "### Install Redis locally (optional)\n",
        "If you have a Redis db running elsewhere with [Redis Stack](https://redis.io/docs/about/about-stack/) installed, you don't need to run it on this machine. You can skip to the \"Connect to Redis server\" step."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 1,
      "metadata": {
        "id": "vs4KZURX4XpT",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "793ec489-129a-4596-b5bf-f093f40ef56c"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "deb [signed-by=/usr/share/keyrings/redis-archive-keyring.gpg] https://packages.redis.io/deb jammy main\n",
            "Starting redis-stack-server, database path /var/lib/redis-stack\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "gpg: cannot open '/dev/tty': No such device or address\n",
            "curl: (23) Failed writing body\n"
          ]
        }
      ],
      "source": [
        "%%sh\n",
        "curl -fsSL https://packages.redis.io/gpg | sudo gpg --dearmor -o /usr/share/keyrings/redis-archive-keyring.gpg\n",
        "echo \"deb [signed-by=/usr/share/keyrings/redis-archive-keyring.gpg] https://packages.redis.io/deb $(lsb_release -cs) main\" | sudo tee /etc/apt/sources.list.d/redis.list\n",
        "sudo apt-get update  > /dev/null 2>&1\n",
        "sudo apt-get install redis-stack-server  > /dev/null 2>&1\n",
        "redis-stack-server --daemonize yes"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "zvDp8WNz4XpU"
      },
      "source": [
        "### Connect to Redis server\n",
        "Replace the connection params below with your own if you are connecting to an external Redis instance."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 2,
      "metadata": {
        "id": "duCyNgfZ4XpU",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "12d97af1-9d7d-471f-8c10-4aee284a8fe5"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "True"
            ]
          },
          "metadata": {},
          "execution_count": 2
        }
      ],
      "source": [
        "import os\n",
        "import redis\n",
        "\n",
        "# Redis connection params\n",
        "REDIS_HOST = os.getenv(\"REDIS_HOST\", \"localhost\") #\"redis-12110.c82.us-east-1-2.ec2.cloud.redislabs.com\"\n",
        "REDIS_PORT = os.getenv(\"REDIS_PORT\", \"6379\")      #12110\n",
        "REDIS_PASSWORD = os.getenv(\"REDIS_PASSWORD\", \"\")  #\"pobhBJP7Psicp2gV0iqa2ZOc1WdXXXXX\"\n",
        "\n",
        "# Create Redis client\n",
        "redis_client = redis.Redis(\n",
        "  host=REDIS_HOST,\n",
        "  port=REDIS_PORT,\n",
        "  password=REDIS_PASSWORD\n",
        ")\n",
        "\n",
        "# Test connection\n",
        "redis_client.ping()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 3,
      "metadata": {
        "id": "0Rrz76w96dF3",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "a7fd6b35-736b-4ff6-b2be-c2ff963adc96"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "True"
            ]
          },
          "metadata": {},
          "execution_count": 3
        }
      ],
      "source": [
        "# Clear Redis database (optional)\n",
        "redis_client.flushdb()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "BpjxW-kou-FY"
      },
      "source": [
        "### Authenticate to Google Cloud"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 4,
      "metadata": {
        "id": "SeTJb51SKs_W",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "68d9a5dc-5e6f-4959-b2ff-7533c580d473"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Authenticated\n"
          ]
        }
      ],
      "source": [
        "from google.colab import auth\n",
        "auth.authenticate_user()\n",
        "print('Authenticated')"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 5,
      "metadata": {
        "id": "8Yil6twAvIuH",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "c0a1240c-b892-4bd0-cf71-6e43952bc422"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "PROJECT_ID:··········\n",
            "REGION:us-central1\n"
          ]
        }
      ],
      "source": [
        "from getpass import getpass\n",
        "\n",
        "# input your GCP project ID and region for Vertex AI\n",
        "PROJECT_ID = getpass(\"PROJECT_ID:\") #'central-beach-194106'\n",
        "REGION = input(\"REGION:\") #'us-central1'"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "9vDGqjHmVgjB"
      },
      "source": [
        "## 2. Obtain dataset\n",
        "\n",
        "Below pull the dataset from ..."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 6,
      "metadata": {
        "id": "Pxshas3XpgdV",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "d4085964-d3f5-430d-c48d-0959ccd1d8dc"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "--2024-05-29 17:44:32--  https://www.irs.gov/pub/irs-pdf/p5718.pdf\n",
            "Resolving www.irs.gov (www.irs.gov)... 23.201.171.228, 2600:1408:5400:4b2::f50, 2600:1408:5400:48d::f50\n",
            "Connecting to www.irs.gov (www.irs.gov)|23.201.171.228|:443... connected.\n",
            "HTTP request sent, awaiting response... 200 OK\n",
            "Length: 8749823 (8.3M) [application/pdf]\n",
            "Saving to: ‘resources/p5718.pdf’\n",
            "\n",
            "p5718.pdf           100%[===================>]   8.34M  --.-KB/s    in 0.1s    \n",
            "\n",
            "2024-05-29 17:44:32 (60.5 MB/s) - ‘resources/p5718.pdf’ saved [8749823/8749823]\n",
            "\n"
          ]
        }
      ],
      "source": [
        "#Procure a dataset - downloading a publication from IRS\n",
        "!mkdir resources\n",
        "!wget https://www.irs.gov/pub/irs-pdf/p5718.pdf -P resources/"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "0Kh0ObD4xZtK"
      },
      "source": [
        "### Create text embeddings with Vertex AI embedding model\n",
        "Use the [Vertex AI API for text embeddings](https://cloud.google.com/vertex-ai/docs/generative-ai/embeddings/get-text-embeddings), developed by Google.\n",
        "\n",
        "> Text embeddings are a dense vector representation of a piece of content such that, if two pieces of content are semantically similar, their respective embeddings are located near each other in the embedding vector space. This representation can be used to solve common NLP tasks, such as:\n",
        "> - **Semantic search**: Search text ranked by semantic similarity.\n",
        "> - **Recommendation**: Return items with text attributes similar to the given text.\n",
        "> - **Classification**: Return the class of items whose text attributes are similar to the given text.\n",
        "> - **Clustering**: Cluster items whose text attributes are similar to the given text.\n",
        "> - **Outlier Detection**: Return items where text attributes are least related to the given text.\n",
        "\n",
        "The Vertex AI text-embeddings API lets you create a text embedding using Generative AI on Vertex AI. The `textembedding-gecko` model accepts a maximum of 3,072 input tokens (i.e. words) and outputs 768-dimensional vector embeddings."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "x3o_9ehYuEpA"
      },
      "source": [
        "### Set up embeddings\n",
        "We define a helper function to create embeddings from a list of texts convert them to a byte string for efficient storage in Redis.\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 7,
      "metadata": {
        "id": "zrpKY4W5yb0M"
      },
      "outputs": [],
      "source": [
        "from redisvl.utils.vectorize import VertexAITextVectorizer\n",
        "\n",
        "vectorizer = VertexAITextVectorizer(\n",
        "    model = \"text-embedding-004\",\n",
        "    api_config = {\"project_id\": PROJECT_ID, \"location\": REGION}\n",
        ")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "peFHIY351eDc"
      },
      "source": [
        "## 3. Generate Embeddings\n",
        "The next step is to create chunks of the pdf and then embed each chunk as a vector."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 8,
      "metadata": {
        "id": "tzkRiknVXOtc",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "b597c6b5-45d7-4b41-f46a-cdea0659efd0"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "[nltk_data] Downloading package punkt to /root/nltk_data...\n",
            "[nltk_data]   Unzipping tokenizers/punkt.zip.\n",
            "[nltk_data] Downloading package averaged_perceptron_tagger to\n",
            "[nltk_data]     /root/nltk_data...\n",
            "[nltk_data]   Unzipping taggers/averaged_perceptron_tagger.zip.\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Done preprocessing. Created 44 chunks of the original pdf resources/p5718.pdf\n"
          ]
        }
      ],
      "source": [
        "from langchain.text_splitter import RecursiveCharacterTextSplitter\n",
        "from langchain.document_loaders import UnstructuredFileLoader\n",
        "\n",
        "doc = \"resources/p5718.pdf\"\n",
        "\n",
        "# set up the file loader/extractor and text splitter to create chunks\n",
        "text_splitter = RecursiveCharacterTextSplitter(\n",
        "    chunk_size=2500, chunk_overlap=0\n",
        ")\n",
        "loader = UnstructuredFileLoader(\n",
        "    doc, mode=\"single\", strategy=\"fast\"\n",
        ")\n",
        "\n",
        "# extract, load, and make chunks\n",
        "chunks = loader.load_and_split(text_splitter)\n",
        "\n",
        "print(\"Done preprocessing. Created\", len(chunks), \"chunks of the original pdf\", doc)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 9,
      "metadata": {
        "id": "OETsrYvfuzmX",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "9eb60a85-6e59-4a34-e028-34d7b211cfdd"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "True"
            ]
          },
          "metadata": {},
          "execution_count": 9
        }
      ],
      "source": [
        "# Embed each chunk content\n",
        "embeddings = vectorizer.embed_many([chunk.page_content for chunk in chunks], as_buffer=True)\n",
        "\n",
        "# Check to make sure we've created enough embeddings, 1 per document chunk\n",
        "len(embeddings) == len(chunks)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "WGVt7-DNr80e"
      },
      "source": [
        "## 4. Create Index\n",
        "\n",
        "Now that we have created embeddings that represent the text in our dataset, we will create an index that enables efficient search over the embeddings.\n",
        "\n",
        "**Why do we need to enable search???**\n",
        "Using Redis for vector search allows us to retrieve chunks of text data that are **similar** or **relevant** to an input question or query. This will be extremely helpful for our sample generative ai / LLM application."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 10,
      "metadata": {
        "id": "9mNa5LNn4XpX"
      },
      "outputs": [],
      "source": [
        "from redisvl.schema import IndexSchema\n",
        "from redisvl.index import SearchIndex\n",
        "\n",
        "\n",
        "index_name = \"redisvl\"\n",
        "\n",
        "schema = IndexSchema.from_dict({\n",
        "  \"index\": {\n",
        "    \"name\": index_name,\n",
        "    \"prefix\": \"chunk\"\n",
        "  },\n",
        "  \"fields\": [\n",
        "    {\n",
        "        \"name\": \"chunk_id\",\n",
        "        \"type\": \"tag\",\n",
        "        \"attrs\": {\n",
        "            \"sortable\": True\n",
        "        }\n",
        "    },\n",
        "    {\n",
        "        \"name\": \"content\",\n",
        "        \"type\": \"text\"\n",
        "    },\n",
        "    {\n",
        "        \"name\": \"text_embedding\",\n",
        "        \"type\": \"vector\",\n",
        "        \"attrs\": {\n",
        "            \"dims\": vectorizer.dims,\n",
        "            \"distance_metric\": \"cosine\",\n",
        "            \"algorithm\": \"flat\",\n",
        "            \"datatype\": \"float32\"\n",
        "        }\n",
        "    }\n",
        "  ]\n",
        "})"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 11,
      "metadata": {
        "id": "cNjHD3D94_Sc"
      },
      "outputs": [],
      "source": [
        "# Create an index from schema and the client\n",
        "index = SearchIndex(schema, redis_client)\n",
        "index.create(overwrite=True, drop=True)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 12,
      "metadata": {
        "id": "VOzL5qB-uzrE"
      },
      "outputs": [],
      "source": [
        "# Load expects an iterable of dictionaries\n",
        "data = [\n",
        "    {\n",
        "        'chunk_id': f'{i}',\n",
        "        'content': chunk.page_content,\n",
        "        'text_embedding': embeddings[i]\n",
        "    } for i, chunk in enumerate(chunks)\n",
        "]\n",
        "\n",
        "# RedisVL handles batching automatically\n",
        "keys = index.load(data, id_field=\"chunk_id\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "G6CmHY3-6wB1"
      },
      "source": [
        "## 5. Query\n",
        "Now we can use RedisVL to perform a variety of vector search operations."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 13,
      "metadata": {
        "id": "d9HKH8kO5T3E",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 35
        },
        "outputId": "17299b9c-26d5-4352-aedb-b73474ed6bb3"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "'*=>[KNN 3 @text_embedding $vector AS vector_distance] RETURN 3 chunk_id content vector_distance SORTBY vector_distance ASC DIALECT 2 LIMIT 0 3'"
            ],
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            }
          },
          "metadata": {},
          "execution_count": 13
        }
      ],
      "source": [
        "from redisvl.query import VectorQuery\n",
        "\n",
        "query = \"What is TCC?\"\n",
        "\n",
        "query_embedding = vectorizer.embed(query)\n",
        "\n",
        "vector_query = VectorQuery(\n",
        "    vector=query_embedding,\n",
        "    vector_field_name=\"text_embedding\",\n",
        "    num_results=3,\n",
        "    return_fields=[\"chunk_id\", \"content\"],\n",
        "    return_score=True\n",
        ")\n",
        "\n",
        "# show the raw redis query\n",
        "str(vector_query)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 14,
      "metadata": {
        "id": "W_-AlXmE5dUK",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "ad823e6e-fe74-47e0-e2dc-f689ca4375f2"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[{'id': 'chunk:7',\n",
              "  'vector_distance': '0.500085473061',\n",
              "  'chunk_id': '7',\n",
              "  'content': 'IRIS uses QuickAlerts, an IRS e-mail service, to disseminate information quickly regarding IRIS issues to subscribers. This service keeps tax professionals up to date on IRIS issues throughout the year, with emphasis on issues during the filing season. After subscribing, customers will receive “round the clock” communication issues such as electronic specifica- tions and system information needed for Software Developers and Transmitters to transmit to IRS. New subscribers may sign up through the “subscription page” link located on the QuickAlerts “more” e-file Benefits for Tax Professionals page.\\n\\n9\\n\\nPublication 5718\\n\\n1.3 Registration and Application Process\\n\\nExternal users must register with the current IRS credential service provider and complete the IRIS Application for Transmitter Control Code (TCC) to submit transmissions using the IRIS intake platform. Information returns filed through the IRIS A2A system cannot be filed using any other intake platform TCC. These include:\\n\\ne-File Application (MeF)\\n\\nAffordable Care Act (ACA) Application for TCC (AIR)\\n\\nPartnership Bipartisan Budget Act (PBBA) Application for TCC\\n\\n\\n\\nInformation Returns (IR) Application for TCC (FIRE)\\n\\n\\n\\nIRIS TCC for the Taxpayer Portal\\n\\n1.3.1 Registration\\n\\nBefore completing the IRIS Application for TCC, each user must create an account or sign-in using their existing credentials to validate their identities using the latest authentication process.\\n\\nFor more information, please visit How to register for IRS online self-help tools | Internal Revenue Service.\\n\\n1.3.2 Who should apply for an IRIS TCC\\n\\nIf you are transmitting information returns to the IRS or if you are developing software to file information returns electronically, you must apply for one or more TCCs using the IRIS Appli- cation for TCC available online. A single application can be used to apply for multiple roles and the necessary TCCs. The IRS encourages transmitters who file for multiple issuers to submit one application and use the assigned TCC for all issuers. The purpose of the TCC is to identify the business acting as the transmitter of the file. As a transmitter, you may transmit files for as many companies as you need to under one TCC. The IRIS Application for TCC contains three separate roles: Software Developer, Transmitter, and Issuer. Complete the IRIS Application for TCC if your firm or organization is performing one or more of the following roles:'},\n",
              " {'id': 'chunk:1',\n",
              "  'vector_distance': '0.502500534058',\n",
              "  'chunk_id': '1',\n",
              "  'content': 'Processing Year 2024 Revisions After 12-2023\\n\\nLocation\\n\\nUpdate\\n\\nSection 6.1.1\\n\\n2-Step Correction Step 2\\n\\nPublication 5718\\n\\nTable of Contents 1. Introduction ���������������������������������������������������������������������������������������������������������� 7\\n\\n1.1 Purpose ���������������������������������������������������������������������������������������������������������������������� 8\\n\\n1.2 Communications �������������������������������������������������������������������������������������������������������� 9\\n\\n1.2.1 IRIS Web Site ���������������������������������������������������������������������������������������������������� 9\\n\\n1.3 Registration and Application Process �������������������������������������������������������������������� 10\\n\\n1.3.1 Registration ��������������������������������������������������������������������������������������������������� 10\\n\\n1.3.2 Who should apply for an IRIS TCC ��������������������������������������������������������������� 10\\n\\n1.3.3 Third-Party Transmitters ������������������������������������������������������������������������������� 12\\n\\n1.3.4 Things you need to know before completing the IRIS ��������������������������������� 12\\n\\n1.3.5 Access the IRIS Application for TCC ������������������������������������������������������������� 13\\n\\n1.3.6 Application Approved/Completed ����������������������������������������������������������������� 13\\n\\n1.3.7 Revise Current TCC Information �������������������������������������������������������������������� 14\\n\\n1.3.8 Deleted TCCs ������������������������������������������������������������������������������������������������� 14\\n\\n1.4 Transmitter and Issuer TCCs ����������������������������������������������������������������������������������� 14\\n\\n1.5 Software Developer TCCs ��������������������������������������������������������������������������������������� 14\\n\\n1.6 API Client ID ������������������������������������������������������������������������������������������������������������ 15\\n\\n2. Transmissions and Submissions ���������������������������������������������������������������������� 18\\n\\n2.1 Transmission/Submission Definitions and Limitations ����������������������������������������� 18\\n\\n2.2 Uniquely Identifying the Transmission �������������������������������������������������������������������� 19\\n\\n3. Transmitting Information Returns �������������������������������������������������������������������� 20'},\n",
              " {'id': 'chunk:11',\n",
              "  'vector_distance': '0.505242109299',\n",
              "  'chunk_id': '11',\n",
              "  'content': 'When your IRIS Application for TCC is approved and completed, a five-character alphanu- meric TCC that begins with the letter ‘D’ will be assigned to your business. An approval letter will be sent via United States Postal Service (USPS) to the address listed on the application, informing you of your TCC. You can also sign into your IRIS Application for TCC to view your TCCs on the Application Summary page.\\n\\n13\\n\\nPublication 5718\\n\\nIf your application is in Completed status for more than 45 days and your TCC has not been assigned, contact the Help Desk.\\n\\n1.3.7 Revise Current TCC Information\\n\\nAs changes occur, you must update and maintain your IRIS TCC Application. Some changes will require all ROs or Authorized Delegates (ADs) on the application to re-sign the Appli- cation Submission page. Below are examples of when an application would need to be re-signed (this list is not all inclusive):\\n\\nFirm’s DBA Name change\\n\\nRole changes or additions\\n\\nAdd, delete or change RO and/or AD\\n\\nNote: Changes submitted on an IRIS TCC Application do not change the address of IRS tax records just as a change of address to IRS tax records does not automatically update infor- mation on an IRIS TCC Application.\\n\\nChanges that require a firm to acquire a new Employer Identification Number (EIN) require a new IRIS TCC Application. Firms that change their form of organization, such as from a sole proprietorship to a corporation, generally require the firm to acquire a new EIN.\\n\\n1.3.8 Deleted TCCs\\n\\nYour TCCs will remain valid if you transmit information returns or extensions of time to file. If you don’t use your TCC for three consecutive years, your TCC will be deleted. Once your TCC is deleted it cannot be reactivated. You’ll need to submit a new IRIS Application for TCC.\\n\\n1.4 Transmitter and Issuer TCCs\\n\\nDepending on the roles selected on the application, one or more TCCs will be assigned. Each TCC will have an indicator of Test “T” or Production “P” and status of Active, Inactive, or Dropped. Transmitters and Issuers are issued a TCC in Test “T” status until required Communication Testing is conducted in the ATS environment and passed. Once Commu- nication Testing is passed, the Transmitter should contact the Help Desk to request to be moved to Production “P” status. For more information about Communication Testing for Transmitters, refer to Publication 5719, Information Returns Intake System (IRIS) Test Package for Information Returns.\\n\\n1.5 Software Developer TCCs'}]"
            ]
          },
          "metadata": {},
          "execution_count": 14
        }
      ],
      "source": [
        "# execute the query with RedisVL\n",
        "index.query(vector_query)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 15,
      "metadata": {
        "id": "-vT4bTYB5h74",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "81e773a8-e9ec-4fce-afa6-6e2a1308213b"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "7 0.500085473061\n",
            "1 0.502500534058\n",
            "11 0.505242109299\n"
          ]
        }
      ],
      "source": [
        "# paginate through results\n",
        "for result in index.paginate(vector_query, page_size=1):\n",
        "    print(result[0][\"chunk_id\"], result[0][\"vector_distance\"], flush=True)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 16,
      "metadata": {
        "id": "t9MjgGiCLLqv",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 35
        },
        "outputId": "0f2e8768-6bb9-4943-c46c-ae44b0b6c023"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "'@content:(Social Security)=>[KNN 3 @text_embedding $vector AS vector_distance] RETURN 3 chunk_id content vector_distance SORTBY vector_distance ASC DIALECT 2 LIMIT 0 3'"
            ],
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            }
          },
          "metadata": {},
          "execution_count": 16
        }
      ],
      "source": [
        "from redisvl.query.filter import Text\n",
        "\n",
        "query = \"What is TCC?\"\n",
        "\n",
        "query_embedding = vectorizer.embed(query)\n",
        "\n",
        "text_filter = Text(\"content\") % \"Social Security\"\n",
        "\n",
        "vector_query = VectorQuery(\n",
        "    vector=query_embedding,\n",
        "    vector_field_name=\"text_embedding\",\n",
        "    num_results=3,\n",
        "    return_fields=[\"chunk_id\", \"content\"],\n",
        "    return_score=True,\n",
        "    filter_expression=text_filter\n",
        ")\n",
        "\n",
        "# show the raw redis query\n",
        "str(vector_query)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 17,
      "metadata": {
        "id": "ZXci2AeGUMtE",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "8cdf34ee-d4a0-46eb-f11a-096a928b9de1"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[{'id': 'chunk:9',\n",
              "  'vector_distance': '0.572170257568',\n",
              "  'chunk_id': '9',\n",
              "  'content': 'Select the role of Transmitter on your application. Note: The TCC for a Transmitter can be used to transmit your own returns and others. You may not use an Issuer TCC to transmit information returns for others.\\n\\n11\\n\\nPublication 5718\\n\\n1.3.3 Third-Party Transmitters\\n\\nIf you do not have an in-house programmer familiar with XML or do not wish to purchase A2A software that is certified to support the information returns that you plan to file, you can file through a Third-Party Transmitter or use the online Taxpayer Portal. Visit www.irs.gov/ iris for additional information.\\n\\nOnly those persons listed as an Authorized User on the IRIS Application for TCC qualify to receive information about a Receipt ID associated with a TCC listed on that application.\\n\\nIf your Third-Party Transmitter needs technical assistance regarding a Receipt ID associated with records that were submitted on behalf of your organization, they should contact the Help Desk.\\n\\nWhen filing through a Third-Party Transmitter obtain the following for each submission filed on your behalf:\\n\\nA copy of all electronic records within each submission, along with the Receipt ID for the transmission in which they were filed.\\n\\nThe transmission Acknowledgement that includes the Status that is returned when processing is complete (Accepted, Accepted With Errors, Partially Accepted, Rejected) and a detailed list of errors, if any.\\n\\nNote: The items cited above are critical to your ability to make corrections should your Third- party Transmitter go out of business or be otherwise unavailable to file corrections on your behalf.\\n\\n1.3.4 Things you need to know before completing the IRIS\\n\\nA responsible official (RO) initiates and submits the IRIS Application for TCC electronically. Each RO must sign the terms of agreement using their five-digit PIN they created when they initially accessed the system. An application will receive a tracking number after saving it. Completing the application in a single session isn’t a requirement.\\n\\nThe following information is necessary to complete each application:\\n\\nFirm’s business structure\\n\\nFirm’s (EIN) (the system doesn’t allow firms to use a Social Security Number (SSN) or Individual Taxpayer Identification Number (ITIN)\\n\\nFirm’s legal business name and business type\\n\\nFirm’s doing business as name when it’s different from the legal business name\\n\\nBusiness phone (phone country code and phone number)\\n\\nBusiness address (this must be a physical location, not a post office box)'}]"
            ]
          },
          "metadata": {},
          "execution_count": 17
        }
      ],
      "source": [
        "# execute the query with RedisVL\n",
        "index.query(vector_query)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "82-AbKHxItif"
      },
      "source": [
        "# Building a RAG Pipeline from Scratch\n",
        "We're going to build a complete RAG pipeline from scratch incorporating the following components:\n",
        "\n",
        "- Standard retrieval and chat completion\n",
        "- Dense content representation to improve accuracy\n",
        "- Query re-writing to improve accuracy\n",
        "- Semantic caching to improve performance\n",
        "- Conversational session history to improve personalization"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 18,
      "metadata": {
        "id": "a6BsbxUG7kVc"
      },
      "outputs": [],
      "source": [
        "#@title Setup RedisVL *AsyncSearchIndex*\n",
        "\n",
        "from redis.asyncio import Redis\n",
        "from redisvl.index import AsyncSearchIndex\n",
        "\n",
        "# Create Redis client\n",
        "redis_client = Redis(\n",
        "    host=REDIS_HOST,\n",
        "    port=REDIS_PORT,\n",
        "    password=REDIS_PASSWORD\n",
        ")\n",
        "\n",
        "index = AsyncSearchIndex(index.schema, redis_client)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 19,
      "metadata": {
        "id": "sSbXjA896Ami"
      },
      "outputs": [],
      "source": [
        "#@title Setup VertexAI Generative Model with Safety Settings\n",
        "from vertexai.generative_models import GenerativeModel, Part, HarmCategory, HarmBlockThreshold\n",
        "\n",
        "\n",
        "model = GenerativeModel(\"gemini-1.5-flash-001\")\n",
        "\n",
        "# Define safety settings\n",
        "safety_settings = {\n",
        "    HarmCategory.HARM_CATEGORY_HATE_SPEECH: HarmBlockThreshold.BLOCK_ONLY_HIGH,\n",
        "    HarmCategory.HARM_CATEGORY_DANGEROUS_CONTENT: HarmBlockThreshold.BLOCK_ONLY_HIGH,\n",
        "    HarmCategory.HARM_CATEGORY_SEXUALLY_EXPLICIT: HarmBlockThreshold.BLOCK_ONLY_HIGH,\n",
        "    HarmCategory.HARM_CATEGORY_HARASSMENT: HarmBlockThreshold.BLOCK_ONLY_HIGH,\n",
        "}\n",
        "\n",
        "# Define generation config\n",
        "generation_config = {\n",
        "    \"max_output_tokens\": 2048,\n",
        "    \"temperature\": 0.5,\n",
        "    \"top_p\": 1\n",
        "}"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "_Ep2DbPB_Rmu"
      },
      "source": [
        "### Baseline Retrieval Augmented Generation\n",
        "\n",
        "Below we build a simple RAG pipeline with three helper methods:\n",
        "\n",
        "\n",
        "*   `answer_question` -- full RAG operation\n",
        "    * `retrieve_context` -- search Redis for relevant sources\n",
        "    * `promptify`  -- combine system instructions, user question, and sources\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 20,
      "metadata": {
        "id": "zmma7Cjd7kZ9"
      },
      "outputs": [],
      "source": [
        "async def answer_question(index: AsyncSearchIndex, query: str):\n",
        "    \"\"\"Answer the user's question\"\"\"\n",
        "\n",
        "    SYSTEM_PROMPT = \"\"\"You are a helpful tax analyst assistant that has access\n",
        "    to publications from the IRS\n",
        "    \"\"\"\n",
        "\n",
        "    query_vector = vectorizer.embed(query)\n",
        "\n",
        "    # Fetch context from Redis using vector search\n",
        "    context = await retrieve_context(index, query_vector)\n",
        "\n",
        "    prompt = f'''\n",
        "    System: {SYSTEM_PROMPT}\n",
        "    User: {promptify(query, context)}\n",
        "    '''\n",
        "\n",
        "    responses = model.generate_content(\n",
        "        [prompt],\n",
        "        generation_config=generation_config,\n",
        "        safety_settings=safety_settings,\n",
        "        stream=False\n",
        "    )\n",
        "    # Response provided by LLM\n",
        "    if(responses.candidates[0].finish_reason.value == 1):\n",
        "        return(responses.candidates[0].content.parts[0].text)\n",
        "    else:\n",
        "        return(f\"Content has been blocked for {responses.candidates[0].finish_reason.name} reasons.\")\n",
        "\n",
        "\n",
        "async def retrieve_context(index: AsyncSearchIndex, query_vector) -> str:\n",
        "    \"\"\"Fetch the relevant context from Redis using vector search\"\"\"\n",
        "    results = await index.query(\n",
        "        VectorQuery(\n",
        "            vector=query_vector,\n",
        "            vector_field_name=\"text_embedding\",\n",
        "            return_fields=[\"content\"],\n",
        "            num_results=3\n",
        "        )\n",
        "    )\n",
        "    content = \"\\n\".join([result[\"content\"] for result in results])\n",
        "    return content\n",
        "\n",
        "\n",
        "def promptify(query: str, context: str) -> str:\n",
        "    return f'''Use the provided context below derived from public documenation to answer the user's question. If you can't answer the user's\n",
        "    question, based on the context; do not guess. Do your best finding the answer in the context, but if there is no context at all,\n",
        "    respond with \"I don't know\".\n",
        "\n",
        "    User question:\n",
        "\n",
        "    {query}\n",
        "\n",
        "    Helpful context:\n",
        "\n",
        "    {context}\n",
        "\n",
        "    Answer:\n",
        "    '''"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 21,
      "metadata": {
        "id": "wIaYNgNxA4D1"
      },
      "outputs": [],
      "source": [
        "# Generate a list of questions\n",
        "questions = [\n",
        "    \"What is TCC?\",\n",
        "    \"Who should apply for an IRIS TCC?\",\n",
        "    \"What is a JWK?\",\n",
        "    \"Should I buy a yacht??\"\n",
        "]"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 22,
      "metadata": {
        "id": "uy7rh-stIIii"
      },
      "outputs": [],
      "source": [
        "import asyncio\n",
        "\n",
        "results = await asyncio.gather(*[\n",
        "    answer_question(index, question) for question in questions\n",
        "])"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 23,
      "metadata": {
        "id": "T_HZnaBo6ylG",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "6c0d1baa-dc27-4ad5-8983-5dd3a98ee9bd"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "What is TCC?: \n",
            "TCC stands for Transmitter Control Code. \n",
            "\n",
            "\n",
            "\n",
            "Who should apply for an IRIS TCC?: \n",
            "If you are transmitting information returns to the IRS or if you are developing software to file information returns electronically, you must apply for one or more TCCs using the IRIS Application for TCC available online. \n",
            "\n",
            "\n",
            "\n",
            "What is a JWK?: \n",
            "A JSON Web Key Set (JWKs) is used for e-Services API authentication. It contains a public key that validates the API consumer application. \n",
            "\n",
            "\n",
            "\n",
            "Should I buy a yacht??: \n",
            "I don't know. \n",
            "\n",
            "\n",
            "\n"
          ]
        }
      ],
      "source": [
        "for question, result in zip(questions,results):\n",
        "  print(question+\": \\n\"+result+\"\\n\\n\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "TN3Ok2zJMhdt"
      },
      "source": [
        "# Improve performance and cut costs with LLM Semantic Caching"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 24,
      "metadata": {
        "id": "QzX8lQ35Mpee"
      },
      "outputs": [],
      "source": [
        "from redis import Redis\n",
        "from redisvl.extensions.llmcache import SemanticCache\n",
        "\n",
        "# Create Redis client\n",
        "redis_client = Redis(\n",
        "  host=REDIS_HOST,\n",
        "  port=REDIS_PORT,\n",
        "  password=REDIS_PASSWORD\n",
        ")\n",
        "\n",
        "# Create the Semantic Cache\n",
        "llmcache = SemanticCache(\n",
        "    name=\"llmcache\",\n",
        "    vectorizer=vectorizer,\n",
        "    redis_client=redis_client,\n",
        "    ttl=120,\n",
        "    distance_threshold=0.2\n",
        ")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 25,
      "metadata": {
        "id": "vaovoOKbMrSG"
      },
      "outputs": [],
      "source": [
        "from functools import wraps\n",
        "\n",
        "\n",
        "# Create an LLM caching decorator\n",
        "def cache(func):\n",
        "    @wraps(func)\n",
        "    async def wrapper(index, query_text, *args, **kwargs):\n",
        "        query_vector = llmcache._vectorizer.embed(query_text)\n",
        "\n",
        "        # Check the cache with the vector\n",
        "        if result := llmcache.check(vector=query_vector):\n",
        "            return result[0]['response']\n",
        "\n",
        "        response = await func(index, query_text, query_vector=query_vector)\n",
        "        llmcache.store(query_text, response, query_vector)\n",
        "        return response\n",
        "    return wrapper\n",
        "\n",
        "\n",
        "@cache\n",
        "async def answer_question(index: AsyncSearchIndex, query: str, **kwargs):\n",
        "    \"\"\"Answer the user's question\"\"\"\n",
        "\n",
        "    SYSTEM_PROMPT = \"\"\"You are a helpful tax analyst assistant that has access\n",
        "    to publications from the IRS\n",
        "    \"\"\"\n",
        "\n",
        "    # Fetch context from Redis using vector search\n",
        "    context = await retrieve_context(index, kwargs[\"query_vector\"])\n",
        "\n",
        "    prompt = f'''\n",
        "    System: {SYSTEM_PROMPT}\n",
        "    User: {promptify(query, context)}\n",
        "    '''\n",
        "\n",
        "    responses = model.generate_content(\n",
        "        [prompt],\n",
        "        generation_config=generation_config,\n",
        "        safety_settings=safety_settings,\n",
        "        stream=False\n",
        "    )\n",
        "    # Response provided by LLM\n",
        "    if(responses.candidates[0].finish_reason.value == 1):\n",
        "        return(responses.candidates[0].content.parts[0].text)\n",
        "    else:\n",
        "        return(f\"Content has been blocked for {responses.candidates[0].finish_reason.name} reasons.\")\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 26,
      "metadata": {
        "id": "a4d6PJG01hcz"
      },
      "outputs": [],
      "source": [
        "from datetime import datetime"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 27,
      "metadata": {
        "id": "yAMpnoVIP7G1",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "5ab491b9-13cc-4981-fa6a-0db7fdf2f0b5"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Total time: 0:00:00.700675\n"
          ]
        }
      ],
      "source": [
        "query = \"What is a JWK?\"\n",
        "\n",
        "startTime = datetime.now()\n",
        "await answer_question(index, query)\n",
        "print(f\"Total time: {datetime.now() - startTime}\")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 28,
      "metadata": {
        "id": "aepPokugQBNt",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "5cf7ed3d-fb48-40f1-f678-284b92964a67"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Total time: 0:00:00.102795\n"
          ]
        }
      ],
      "source": [
        "# Now try again with semantic caching enabled!\n",
        "query = \"What's a JWK?\"\n",
        "\n",
        "startTime = datetime.now()\n",
        "await answer_question(index, query)\n",
        "print(f\"Total time: {datetime.now() - startTime}\")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "H6jxgpzWA2Ng"
      },
      "outputs": [],
      "source": [
        "import gradio as gr\n",
        "\n",
        "async def respond(message, history):\n",
        "    print(message)\n",
        "    result = await answer_question(index, message)\n",
        "    print(result)\n",
        "    return result\n",
        "\n",
        "gr.ChatInterface(respond).launch(debug=True)"
      ]
    }
  ],
  "metadata": {
    "colab": {
      "gpuType": "T4",
      "provenance": []
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}