speech/getting-started/speech_recognition.ipynb

{
  "cells": [
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "ur8xi4C7S06n"
      },
      "outputs": [],
      "source": [
        "# Copyright 2023 Google LLC\n",
        "#\n",
        "# Licensed under the Apache License, Version 2.0 (the \"License\");\n",
        "# you may not use this file except in compliance with the License.\n",
        "# You may obtain a copy of the License at\n",
        "#\n",
        "#     https://www.apache.org/licenses/LICENSE-2.0\n",
        "#\n",
        "# Unless required by applicable law or agreed to in writing, software\n",
        "# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
        "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
        "# See the License for the specific language governing permissions and\n",
        "# limitations under the License."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "JAPoU8Sm5E6e"
      },
      "source": [
        "# Get started with Chirp on Google Cloud\n",
        "\n",
        "<table align=\"left\">\n",
        "\n",
        "  <td>\n",
        "    <a href=\"https://colab.research.google.com/github/GoogleCloudPlatform/generative-ai/blob/main/speech/getting-started/speech_recognition.ipynb\">\n",
        "      <img src=\"https://cloud.google.com/ml-engine/images/colab-logo-32px.png\" alt=\"Colab logo\"> Run in Colab\n",
        "    </a>\n",
        "  </td>\n",
        "  <td>\n",
        "    <a href=\"https://github.com/GoogleCloudPlatform/generative-ai/blob/main/speech/getting-started/speech_recognition.ipynb\">\n",
        "      <img src=\"https://cloud.google.com/ml-engine/images/github-logo-32px.png\" alt=\"GitHub logo\">\n",
        "      View on GitHub\n",
        "    </a>\n",
        "  </td>\n",
        "  <td>\n",
        "    <a href=\"https://console.cloud.google.com/vertex-ai/workbench/deploy-notebook?download_url=https://raw.githubusercontent.com/GoogleCloudPlatform/generative-ai/main/speech/getting-started/speech_recognition.ipynb\">\n",
        "      <img src=\"https://lh3.googleusercontent.com/UiNooY4LUgW_oTvpsNhPpQzsstV5W8F7rYgxgGBD85cWJoLmrOzhVs_ksK_vgx40SHs7jCqkTkCk=e14-rj-sc0xffffff-h130-w32\" alt=\"Vertex AI logo\">\n",
        "      Open in Vertex AI Workbench\n",
        "    </a>\n",
        "  </td>\n",
        "</table>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5da949dba939"
      },
      "source": [
        "| | |\n",
        "|-|-|\n",
        "|Author(s) | [Ivan Nardini](https://github.com/inardini) |"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "tvgnzT1CKxrO"
      },
      "source": [
        "## Overview\n",
        "\n",
        "This notebook demonstrates how to use Chirp for converting voice to text in several languages.\n",
        "\n",
        "Currently, there are some main challenges in Automatic Speech Recognition, ASR in short. Supervised learning approaches in ASR are not scalable. They are required large amounts of high quality labeled data which has a reasonable language coverage. Also, they are computational expensive. \n",
        "\n",
        "As foundational model, Chirp is the next generation of Google's speech-to-text models. Representing the culmination of years of research, the first version of Chirp is now available for Speech-to-Text. With Chirp, you get access to the first 2B parameter speech model which achieved 98% accuracy on English and over 300% relative improvement in tail languages.\n",
        "\n",
        "For details, check out the official documentation about [Chirp](https://cloud.google.com/speech-to-text/v2/docs/chirp-model)."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "d975e698c9a4"
      },
      "source": [
        "### Objective\n",
        "\n",
        "In this tutorial, you will use Chirp to transcribe short and long audio clips. You will also evaluate the model results using automatic speech recognition system metrics.\n",
        "\n",
        "This tutorial uses the following Google Cloud ML services and resources:\n",
        "\n",
        "- Cloud Storage\n",
        "- [Cloud Speech-to-Text API (v2)](https://cloud.google.com/speech-to-text/v2/docs)\n",
        "\n",
        "The steps performed include:\n",
        "\n",
        "- Get audio file to process\n",
        "- Transcribe short audio files\n",
        "- Transcribe long audio files\n",
        "- Evaluate transcriptions"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "aed92deeb4a0"
      },
      "source": [
        "### Costs\n",
        "\n",
        "This tutorial uses billable components of Google Cloud:\n",
        "\n",
        "* Speech-to-text\n",
        "* Cloud Storage\n",
        "\n",
        "Learn about [Speech-to-text pricing](https://cloud.google.com/speech-to-text/pricing),\n",
        "and [Cloud Storage pricing](https://cloud.google.com/storage/pricing),\n",
        "and use the [Pricing Calculator](https://cloud.google.com/products/calculator/)\n",
        "to generate a cost estimate based on your projected usage."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "i7EUnXsZhAGF"
      },
      "source": [
        "## Getting Started\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "NNSWiCNPjh_p"
      },
      "source": [
        "### Install Vertex AI SDK, other packages and their dependencies\n",
        "\n",
        "Install the following packages required to execute this notebook."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "2b4ef9b72d43"
      },
      "outputs": [],
      "source": [
        "# Install the packages\n",
        "! pip3 install --user --upgrade google-cloud-speech librosa jiwer protobuf"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "gbWwuHK8j1xm"
      },
      "source": [
        "### Colab only: Uncomment the following cell to restart the kernel."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "yStiAHorWE3Q"
      },
      "source": [
        "***Colab only***: Uncomment the following cell to restart the kernel or use the button to restart the kernel. For Vertex AI Workbench you can restart the terminal using the button on top."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "f200f10a1da3"
      },
      "outputs": [],
      "source": [
        "# Automatically restart kernel after installs so that your environment can access the new packages\n",
        "# import IPython\n",
        "\n",
        "# app = IPython.Application.instance()\n",
        "# app.kernel.do_shutdown(True)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "opUxT_k5TdgP"
      },
      "source": [
        "### Authenticating your notebook environment\n",
        "* If you are using **Colab** to run this notebook, uncomment the cell below and continue.\n",
        "* If you are using **Vertex AI Workbench**, check out the setup instructions [here](https://github.com/GoogleCloudPlatform/generative-ai/tree/main/setup-env)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "vbNgv4q1T2Mi"
      },
      "outputs": [],
      "source": [
        "# from google.colab import auth\n",
        "\n",
        "# auth.authenticate_user()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "YbMFqPZ3tnwz"
      },
      "source": [
        "### Set Google Cloud project information\n",
        "\n",
        "To get started using Speech-to-Text, Cloud Storage, and Vertex AI, you must have an existing Google Cloud project and [enable the APIs](https://console.cloud.google.com/flows/enableapi?apiid=aiplatform.googleapis.com,storage.googleapis.com,speech.googleapis.com).\n",
        "\n",
        "Learn more about [setting up a project and a development environment](https://cloud.google.com/vertex-ai/docs/start/cloud-environment).\n",
        "\n",
        "Please note the **available regions** for Chirp, see [documentation](https://cloud.google.com/speech-to-text/v2/docs/speech-to-text-supported-languages)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "GjSsu6cmUdEx"
      },
      "outputs": [],
      "source": [
        "PROJECT_ID = \"[your-project-id]\"  # @param {type:\"string\"}\n",
        "\n",
        "REGION = \"us-central1\"  # @param {type:\"string\"}"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "zgPO1eR3CYjk"
      },
      "source": [
        "### Create a Cloud Storage bucket\n",
        "\n",
        "Create a storage bucket to store intermediate artifacts such as datasets."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "MzGDU7TWdts_"
      },
      "outputs": [],
      "source": [
        "BUCKET_URI = \"gs://your-bucket-name-unique\"  # @param {type:\"string\"}"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "-EcIXiGsCePi"
      },
      "source": [
        "**Only if your bucket doesn't already exist**: Run the following cell to create your Cloud Storage bucket."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "NIq7R4HZCfIc"
      },
      "outputs": [],
      "source": [
        "! gsutil mb -l $REGION -p $PROJECT_ID $BUCKET_URI"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "960505627ddf"
      },
      "source": [
        "### Import libraries"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "PyQmSRbKA8r-"
      },
      "outputs": [],
      "source": [
        "import json\n",
        "from pathlib import Path as p\n",
        "from pprint import pprint\n",
        "\n",
        "from IPython.display import Audio as play\n",
        "from google.api_core.client_options import ClientOptions\n",
        "from google.cloud.speech_v2 import SpeechClient\n",
        "from google.cloud.speech_v2.types import cloud_speech\n",
        "import jiwer\n",
        "import librosa\n",
        "import pandas as pd"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "v5CEc4-Wrjk2"
      },
      "source": [
        "### Helper functions"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "kYx2wwhjrmD6"
      },
      "outputs": [],
      "source": [
        "def get_stt_metric(truth, transcription, metric=\"wer\"):\n",
        "    \"\"\"\n",
        "    A function to calculate common automatic speech recognition system metrics using JiWER library.\n",
        "    Default metrics is Word error rate (WER) which is a measure of the misrecognized words.\n",
        "    Possible metrics are:\n",
        "    - Word information lost (WIL) is a measure of the amount of information that is lost when the model transcribes a word.\n",
        "    - Word information preserved (WIP) is a measure of the amount of information that is preserved.\n",
        "    \"\"\"\n",
        "    chirp_evaluation = jiwer.compute_measures(truth, transcription)\n",
        "    return chirp_evaluation.get(metric)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5eaHokbL2nS9"
      },
      "source": [
        "### Prepare data\n",
        "\n",
        "You load some existing audio file from a public Google Cloud Storage bucket. In this tutorial, you will use `brooklyn.flac` and `vr.wav` audio files."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "0KqGXuRVuBDf"
      },
      "outputs": [],
      "source": [
        "data_folder = p.cwd() / \"data\"\n",
        "p(data_folder).mkdir(parents=True, exist_ok=True)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "F4BTONqwyP_t"
      },
      "source": [
        "#### Short audio\n",
        "\n",
        "As short audio, let's use `brooklyn.flac` which is less than 60 seconds."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "rIwPXEltAbo9"
      },
      "outputs": [],
      "source": [
        "short_audio_uri = (\n",
        "    \"gs://cloud-samples-tests/speech/brooklyn.flac\"  # @param {type:\"string\"}\n",
        ")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "d888b7b3ebf0"
      },
      "source": [
        "To verify if the audio length, you can use `librosa`, a python package for music and audio analysis. "
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "_bo-d_3Dtdq8"
      },
      "outputs": [],
      "source": [
        "short_audio_path = str(data_folder / \"short_audio.flac\")\n",
        "! gsutil cp {short_audio_uri} {short_audio_path}\n",
        "\n",
        "short_audio_duration = librosa.get_duration(path=short_audio_path)\n",
        "if short_audio_duration > 60:\n",
        "    raise Exception(\n",
        "        f\"The audio is longer than 60 sec. Please use a GCS url for audio longer than 60 sec. Actual length: {short_audio_duration}\"\n",
        "    )"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "8t0EX8FIyOq2"
      },
      "outputs": [],
      "source": [
        "play(short_audio_path)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "aqHNSH7VyTpj"
      },
      "source": [
        "#### Long audio\n",
        "\n",
        "For long audio, you can use `vr.wav` which is longer than 60 seconds."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "n8GhiYMCAfxH"
      },
      "outputs": [],
      "source": [
        "long_audio_origin_uri = (\n",
        "    \"gs://cloud-samples-tests/speech/vr.wav\"  # @param {type:\"string\"}\n",
        ")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "b734a8bab825"
      },
      "source": [
        "As before, let's verify the audio length."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "mmEDV95kuDIv"
      },
      "outputs": [],
      "source": [
        "long_audio_uri = f\"{BUCKET_URI}/data/vr.wav\"  # @param {type:\"string\"}\n",
        "long_audio_path = str(data_folder / \"long_audio.wav\")\n",
        "!gsutil cp {long_audio_origin_uri} {long_audio_uri}\n",
        "!gsutil cp {long_audio_uri} {long_audio_path}\n",
        "\n",
        "long_audio_duration = librosa.get_duration(path=long_audio_path)\n",
        "if long_audio_duration < 60:\n",
        "    raise Exception(\n",
        "        f\"The audio is less than 1 min. Actual length: {long_audio_duration}\"\n",
        "    )"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "yUwHPo7gyXen"
      },
      "outputs": [],
      "source": [
        "play(long_audio_path)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "mVQiapdNKDH9"
      },
      "source": [
        "## Transcribe short audio files (< 1 min)\n",
        "\n",
        "To transcribe short audio files, you need to create a Recognizer which allows you to define the model used for recognition and a list of settings used for recognition. Then you use the Recognizer to run a recognizer request which generates the transcription.\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "lZ1BLy_3LQ8R"
      },
      "outputs": [],
      "source": [
        "client = SpeechClient(\n",
        "    client_options=ClientOptions(api_endpoint=f\"{REGION}-speech.googleapis.com\")\n",
        ")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "oLMSUoQDKPOW"
      },
      "source": [
        "### Create a recognizer\n",
        "\n",
        "First, you need to initiate a Recognizer which uses the Chirp model and transcribe the audio in English.\n",
        "\n",
        "See [the documentation](https://cloud.google.com/python/docs/reference/speech/latest/google.cloud.speech_v2.types.CreateRecognizerRequest) to learn more about how to configure the `CreateRecognizerRequest` request."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "S7E1VlwDKOfp"
      },
      "outputs": [],
      "source": [
        "language_code = \"en-US\"\n",
        "recognizer_id = f\"chirp-{language_code.lower()}-test\"\n",
        "\n",
        "recognizer_request = cloud_speech.CreateRecognizerRequest(\n",
        "    parent=f\"projects/{PROJECT_ID}/locations/{REGION}\",\n",
        "    recognizer_id=recognizer_id,\n",
        "    recognizer=cloud_speech.Recognizer(\n",
        "        language_codes=[language_code],\n",
        "        model=\"chirp\",\n",
        "    ),\n",
        ")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "EHV7GbFMpAP2"
      },
      "source": [
        "Then, you create a Speech-to-Text [Recognizer](https://cloud.google.com/speech-to-text/v2/docs/recognizers) that uses Chirp running a create operation."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "AarUGGQ_MEn7"
      },
      "outputs": [],
      "source": [
        "create_operation = client.create_recognizer(request=recognizer_request)\n",
        "recognizer = create_operation.result()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "D5W-cuFWMkXA"
      },
      "outputs": [],
      "source": [
        "recognizer"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ch0QOMWUMtao"
      },
      "source": [
        "### Transcribe a short audio\n",
        "\n",
        "After you create a Speech-to-Text Recognizer that uses Chirp, you are ready to transcribe your audio.\n",
        "\n",
        "You can create a recognition configuration and the associated recognition request."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "81cuIpuOM6z8"
      },
      "outputs": [],
      "source": [
        "with open(short_audio_path, \"rb\") as f:\n",
        "    content = f.read()\n",
        "\n",
        "short_audio_config = cloud_speech.RecognitionConfig(\n",
        "    features=cloud_speech.RecognitionFeatures(\n",
        "        enable_automatic_punctuation=True, enable_word_time_offsets=True\n",
        "    ),\n",
        "    auto_decoding_config={},\n",
        ")\n",
        "\n",
        "short_audio_request = cloud_speech.RecognizeRequest(\n",
        "    recognizer=recognizer.name, config=short_audio_config, content=content\n",
        ")\n",
        "\n",
        "short_audio_response = client.recognize(request=short_audio_request)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "UTtkQFskprZz"
      },
      "source": [
        "Below you can see the transcribed audio."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "P_XpEZyypsm7"
      },
      "outputs": [],
      "source": [
        "short_audio_transcription = short_audio_response.results[0].alternatives[0].transcript\n",
        "pprint(short_audio_transcription)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "JiXyeFqkxdkN"
      },
      "source": [
        "## Transcribe long audio files (> 1 min)\n",
        "\n",
        "To transcribe long audio files, you use the same process described above but you can run a batch recognition request which uses some audio uploaded on a bucket."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "-18yzs6mENBi"
      },
      "outputs": [],
      "source": [
        "transcriptions_folder = p.cwd() / \"transcriptions\"\n",
        "p(transcriptions_folder).mkdir(parents=True, exist_ok=True)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "fCiDAJG3Jqsq"
      },
      "source": [
        "### Transcribe a long audio\n",
        "\n",
        "Unlike the short audio transcription, the recognition request requires information about the bucket location of the audio file to transcribe and the bucket destination of transcriptions."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "QVsIxZvEzCrk"
      },
      "outputs": [],
      "source": [
        "long_audio_config = cloud_speech.RecognitionConfig(\n",
        "    features=cloud_speech.RecognitionFeatures(\n",
        "        enable_automatic_punctuation=True, enable_word_time_offsets=True\n",
        "    ),\n",
        "    auto_decoding_config={},\n",
        ")\n",
        "\n",
        "long_audio_request = cloud_speech.BatchRecognizeRequest(\n",
        "    recognizer=recognizer.name,\n",
        "    recognition_output_config={\n",
        "        \"gcs_output_config\": {\"uri\": f\"{BUCKET_URI}/transcriptions\"}\n",
        "    },\n",
        "    files=[{\"config\": long_audio_config, \"uri\": long_audio_uri}],\n",
        ")\n",
        "\n",
        "\n",
        "long_audio_operation = client.batch_recognize(request=long_audio_request)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "CpLcENeirIET"
      },
      "source": [
        "Below you can see the result of the transcription job."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "YgiGZa-brIc4"
      },
      "outputs": [],
      "source": [
        "long_audio_result = long_audio_operation.result()\n",
        "print(long_audio_result)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "aX0sghRDJ-YG"
      },
      "source": [
        "### Get transcriptions\n",
        "\n",
        "To see the result of transcription job, you get the generated transcription file."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "wPOpmAyeCgU1"
      },
      "outputs": [],
      "source": [
        "transcriptions_uri = long_audio_result.results[long_audio_uri].uri\n",
        "transcriptions_file_path = str(data_folder / \"transcriptions.text\")\n",
        "\n",
        "! gsutil cp {transcriptions_uri} {transcriptions_file_path}"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "lq3L4SXSEYif"
      },
      "outputs": [],
      "source": [
        "transcriptions = json.loads(open(transcriptions_file_path).read())\n",
        "transcriptions = transcriptions[\"results\"]\n",
        "transcriptions = [\n",
        "    transcription[\"alternatives\"][0][\"transcript\"]\n",
        "    for transcription in transcriptions\n",
        "    if \"alternatives\" in transcription.keys()\n",
        "]\n",
        "long_audio_transcription = \" \".join(transcriptions)\n",
        "print(long_audio_transcription)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "svfbUjshYWTp"
      },
      "outputs": [],
      "source": [
        "transcriptions"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2WJn7OLkKCw0"
      },
      "source": [
        "## Evaluate transcriptions\n",
        "\n",
        "Finally, you may want to evaluate Chirp transcriptions. To do so, you can use `[JiWER](https://github.com/jitsi/jiwer)`, a simple and fast Python package which supports several metrics. In this tutorial, you use:\n",
        "\n",
        "- **Word error rate (WER)** which is the most common metric. It calculates the number of words that are incorrectly recognized divided by the total number of words in the reference transcript.\n",
        "- **Word information lost (WIL)** is a measure of the amount of information that is lost when the model transcribes a word. It is based on the uncorrected number of phonemes over the total number of phonemes in the word.\n",
        "- **Word information preserved (WIP)** is a measure of the amount of information that is preserved and it is calculated as the number of phonemes that are correctly recognized divided by the total number of phonemes in the word."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "gobHdyhyiVVQ"
      },
      "outputs": [],
      "source": [
        "audio_uris = [short_audio_uri, long_audio_uri]\n",
        "actual_transcriptions = [\n",
        "    \"\"\"\n",
        "    how old is the Brooklyn Bridge?\n",
        "    \"\"\",\n",
        "    \"\"\"\n",
        "    so okay, so what am I doing here? why am I here at GDC talking about VR video?\n",
        "    um, it's because I believe um, my favorite games, I love games, I believe in games,\n",
        "    my favorite games are the ones that are all about the stories, I love narrative game design,\n",
        "    I love narrative-based games and I think that when it comes to telling stories in VR,\n",
        "    bringing together capturing the world with narrative-based games and narrative-based game design,\n",
        "    is going to unlock some of the killer apps and killer stories of the medium,\n",
        "    so I'm really here looking for people who are interested in telling those sort of stories,\n",
        "    that are planning projects around telling those types of stories,\n",
        "    um and I would love to talk to you, so if this sounds like your project,\n",
        "    if you're looking at blending VR video and interactivity to tell a story,\n",
        "    I want to talk to you, um, I want to help you, so if this sounds like you,\n",
        "    please get in touch, please come find me, I'll be here all week, I have pink\n",
        "    I work for Google um and I would love to talk with you further about\n",
        "    um VR video, interactivity and storytelling.\n",
        "    \"\"\",\n",
        "]\n",
        "hypothesis_transcriptions = [short_audio_transcription, long_audio_transcription]\n",
        "\n",
        "evaluations = []\n",
        "for a, t, h in zip(audio_uris, actual_transcriptions, hypothesis_transcriptions):\n",
        "    evaluation = {}\n",
        "    evaluation[\"audio_uri\"] = a\n",
        "    evaluation[\"truth\"] = t\n",
        "    evaluation[\"hypothesis\"] = h\n",
        "    evaluation[\"wer\"] = get_stt_metric(t, h, \"wer\")\n",
        "    evaluation[\"wil\"] = get_stt_metric(t, h, \"wil\")\n",
        "    evaluation[\"wip\"] = get_stt_metric(t, h, \"wip\")\n",
        "    evaluations.append(evaluation)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "pDL9yvksbGjC"
      },
      "outputs": [],
      "source": [
        "evaluations_df = pd.DataFrame.from_dict(evaluations)\n",
        "evaluations_df.reset_index(inplace=True, drop=True)\n",
        "evaluations_df"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "o5IMNXMNt2kL"
      },
      "source": [
        "## Conclusion\n",
        "\n",
        "In this tutorial, you learned how to use Chirp for converting English audio to text.\n",
        "\n",
        "Although Chirp does not currently support many of the Speech-to-Text features, including Speech adaptation and Diarization, Chirp is a powerful new speech-to-text model that can accurately transcribe audio in over 100 languages. It is different from previous speech models because it uses a universal encoder that is trained on data in many different languages. This allows Chirp to achieve state-of-the-art accuracy, even for languages with limited training data.\n",
        "\n",
        "Chirp is ideal for developers who need to transcribe audio in multiple languages. It can be used for a variety of tasks, such as video captioning, content transcription, and speech recognition."
      ]
    }
  ],
  "metadata": {
    "colab": {
      "name": "speech_recognition.ipynb",
      "toc_visible": true
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}