notebooks/community/model_garden/model_garden_pipeline_templates_t5x.ipynb

{
  "cells": [
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "copyright"
      },
      "outputs": [],
      "source": [
        "# Copyright 2023 Google LLC\n",
        "#\n",
        "# Licensed under the Apache License, Version 2.0 (the \"License\");\n",
        "# you may not use this file except in compliance with the License.\n",
        "# You may obtain a copy of the License at\n",
        "#\n",
        "#     https://www.apache.org/licenses/LICENSE-2.0\n",
        "#\n",
        "# Unless required by applicable law or agreed to in writing, software\n",
        "# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
        "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
        "# See the License for the specific language governing permissions and\n",
        "# limitations under the License."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "title:generic,gcp"
      },
      "source": [
        "# Get started with Model Garden Pipeline Templates for T5X models\n",
        "\n",
        "\n",
        "<table align=\"left\">\n",
        "\n",
        "  <td>\n",
        "    <a href=\"https://colab.research.google.com/github/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/community/model_garden/model_garden_template_pipelines_t5x.ipynb\">\n",
        "      <img src=\"https://cloud.google.com/ml-engine/images/colab-logo-32px.png\" alt=\"Colab logo\"> Run in Colab\n",
        "    </a>\n",
        "  </td>\n",
        "  <td>\n",
        "    <a href=\"https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/community/model_garden/model_garden_template_pipelines_t5x.ipynb\">\n",
        "      <img src=\"https://cloud.google.com/ml-engine/images/github-logo-32px.png\" alt=\"GitHub logo\">\n",
        "      View on GitHub\n",
        "    </a>\n",
        "  </td>\n",
        "  <td>\n",
        "    <a href=\"https://console.cloud.google.com/vertex-ai/workbench/deploy-notebook?download_url=https://raw.githubusercontent.com/GoogleCloudPlatform/vertex-ai-samples/main/notebooks/communitymodel_garden/model_garden_template_pipelines_t5x.ipynb\">\n",
        "      <img src=\"https://lh3.googleusercontent.com/UiNooY4LUgW_oTvpsNhPpQzsstV5W8F7rYgxgGBD85cWJoLmrOzhVs_ksK_vgx40SHs7jCqkTkCk=e14-rj-sc0xffffff-h130-w32\" alt=\"Vertex AI logo\">\n",
        "      Open in Vertex AI Workbench\n",
        "    </a>\n",
        "  </td>                                                                                               \n",
        "</table>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "overview:mlops"
      },
      "source": [
        "## Overview\n",
        "\n",
        "\n",
        "This tutorial demonstrates how to modify, compile and execute a prebuilt Vertex AI Model Garden pipeline template with Vertex AI Pipelines.\n",
        "\n",
        "Learn more about [Create a pipeline template](https://cloud.google.com/vertex-ai/docs/pipelines/create-pipeline-template)."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "objective:mlops,stage4,get_started_vertex_model_evaluation"
      },
      "source": [
        "### Objective\n",
        "\n",
        "In this tutorial, you learn how to use a prebuilt pipeline template with `Vertex AI Pipelines` to fine-tune a T5X text classification model, where the model is accessed from `Vertex AI Model Garden`.\n",
        "\n",
        "This tutorial uses the following Google Cloud ML services:\n",
        "\n",
        "- `Vertex AI Pipelines`\n",
        "- `Vertex AI Training`\n",
        "- `Vertex AI Model Garden`\n",
        "- `Google Cloud Pipeline Components`\n",
        "\n",
        "\n",
        "The steps performed include:\n",
        "\n",
        "- Create a user-defined repository in the `Artifact Registry`.\n",
        "- Upload the prebuilt pipeline template to the `Artifact Registry`.\n",
        "- Create a pipeline job with the prebuilt pipeline template to fine-tune a T5X model.\n",
        "- Execute the pipeline using `Vertex AI Pipelines`.\n",
        "    - Load T5X model from Vertex AI Model Garden\n",
        "    - Fine-tune train the model\n",
        "- Obtain the Vertex AI Model resource from the pipeline artifacts.\n",
        "- Deploy the model to a Vertex AI Endpoint\n",
        "- Make a prediction"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "dataset:bank,lbn"
      },
      "source": [
        "### Model\n",
        "\n",
        "This tutorial uses a pre-trained T5 text classification model from `Vertex AI Model Garden`, which is then fine-tuned (transfer learning) on a dataset of text phrases which are classified as either FirstClass or SecondClass.\n",
        "\n",
        "Learn more about [Text-to-text transfer transformer](https://github.com/google-research/text-to-text-transfer-transformer). "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "costs"
      },
      "source": [
        "### Costs\n",
        "\n",
        "This tutorial uses billable components of Google Cloud:\n",
        "\n",
        "* Vertex AI\n",
        "* Cloud Storage\n",
        "* Dataflow\n",
        "\n",
        "Learn about [Vertex AI\n",
        "pricing](https://cloud.google.com/vertex-ai/pricing), [Cloud Storage\n",
        "pricing](https://cloud.google.com/storage/pricing), and [Dataflow pricing](https://cloud.google.com/dataflow/pricing)\n",
        "and use the [Pricing\n",
        "Calculator](https://cloud.google.com/products/calculator/)\n",
        "to generate a cost estimate based on your projected usage."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "install_mlops"
      },
      "source": [
        "## Installations\n",
        "\n",
        "Install the packages required for executing this notebook.\n",
        "\n",
        "*Note:* This tutorial requires KFP 2.x."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "install_mlops"
      },
      "outputs": [],
      "source": [
        "import os\n",
        "\n",
        "! pip3 install --upgrade google-cloud-aiplatform \\\n",
        "                         google-cloud-pipeline-components \\\n",
        "                         kfp==2.0.0b15"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "restart"
      },
      "source": [
        "### Colab only: Uncomment the following cell to restart the kernel"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "D-ZBOjErv5mM"
      },
      "outputs": [],
      "source": [
        "# Automatically restart kernel after installs so that your environment can access the new packages\n",
        "# import IPython\n",
        "\n",
        "# app = IPython.Application.instance()\n",
        "# app.kernel.do_shutdown(True)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "before_you_begin"
      },
      "source": [
        "## Before you begin\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "before_you_begin:nogpu"
      },
      "source": [
        "### Set your project ID\n",
        "\n",
        "**If you don't know your project ID**, try the following:\n",
        "* Run `gcloud config list`.\n",
        "* Run `gcloud projects list`.\n",
        "* See the support page: [Locate the project ID](https://support.google.com/googleapi/answer/7014113)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "set_project_id"
      },
      "outputs": [],
      "source": [
        "PROJECT_ID = \"[your-project-id]\"  # @param {type:\"string\"}\n",
        "\n",
        "# Set the project id\n",
        "! gcloud config set project {PROJECT_ID}"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "region"
      },
      "source": [
        "#### Region\n",
        "\n",
        "You can also change the `REGION` variable used by Vertex AI. Learn more about [Vertex AI regions](https://cloud.google.com/vertex-ai/docs/general/locations)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "region"
      },
      "outputs": [],
      "source": [
        "REGION = \"us-central1\"  # @param {type: \"string\"}"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "c4ccf556d4ea"
      },
      "source": [
        "### Enable APIs\n",
        "\n",
        "You can enable the required APIs using `gcloud`."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "619529337e6d"
      },
      "outputs": [],
      "source": [
        "! gcloud services enable compute.googleapis.com         \\\n",
        "                         containerregistry.googleapis.com  \\\n",
        "                         aiplatform.googleapis.com  \\\n",
        "                         artifactregistry.googleapis.com"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "gcp_authenticate"
      },
      "source": [
        "### Authenticate your Google Cloud account\n",
        "\n",
        "Depending on your Jupyter environment, you may have to manually authenticate. Follow the relevant instructions below."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "FvQeFm3Gv5mR"
      },
      "source": [
        "**1. Vertex AI Workbench**\n",
        "* Do nothing as you are already authenticated."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "ce6043da7b33"
      },
      "outputs": [],
      "source": [
        "# ! gcloud auth login"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "0367eac06a10"
      },
      "source": [
        "**3. Colab, uncomment and run:**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "21ad4dbb4a61"
      },
      "outputs": [],
      "source": [
        "# from google.colab import auth\n",
        "# auth.authenticate_user()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "c13224697bfb"
      },
      "source": [
        "**4. Service account or other**\n",
        "* See how to grant Cloud Storage permissions to your service account at https://cloud.google.com/storage/docs/gsutil/commands/iam#ch-examples."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "bucket:mbsdk"
      },
      "source": [
        "### Create a Cloud Storage bucket\n",
        "\n",
        "Create a storage bucket to store intermediate artifacts such as datasets."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "bucket"
      },
      "outputs": [],
      "source": [
        "BUCKET_URI = f\"gs://your-bucket-name-{PROJECT_ID}-unique\"  # @param {type:\"string\"}"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "create_bucket"
      },
      "source": [
        "**Only if your bucket doesn't already exist**: Run the following cell to create your Cloud Storage bucket."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "create_bucket"
      },
      "outputs": [],
      "source": [
        "! gsutil mb -l $REGION $BUCKET_URI"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "set_service_account"
      },
      "source": [
        "#### Service Account\n",
        "\n",
        "**If you don't know your service account**, try to get your service account using `gcloud` command by executing the second cell below."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "set_service_account"
      },
      "outputs": [],
      "source": [
        "SERVICE_ACCOUNT = \"[your-service-account]\"  # @param {type:\"string\"}"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "autoset_service_account"
      },
      "outputs": [],
      "source": [
        "import sys\n",
        "\n",
        "IS_COLAB = \"google.colab\" in sys.modules\n",
        "if (\n",
        "    SERVICE_ACCOUNT == \"\"\n",
        "    or SERVICE_ACCOUNT is None\n",
        "    or SERVICE_ACCOUNT == \"[your-service-account]\"\n",
        "):\n",
        "    # Get your service account from gcloud\n",
        "    if not IS_COLAB:\n",
        "        shell_output = !gcloud auth list 2>/dev/null\n",
        "        SERVICE_ACCOUNT = shell_output[2].replace(\"*\", \"\").strip()\n",
        "\n",
        "    if IS_COLAB:\n",
        "        shell_output = ! gcloud projects describe  $PROJECT_ID\n",
        "        project_number = shell_output[-1].split(\":\")[1].strip().replace(\"'\", \"\")\n",
        "        SERVICE_ACCOUNT = f\"{project_number}-compute@developer.gserviceaccount.com\"\n",
        "\n",
        "    print(\"Service Account:\", SERVICE_ACCOUNT)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "set_service_account:pipelines"
      },
      "source": [
        "#### Set service account access for Vertex AI Pipelines\n",
        "\n",
        "Run the following commands to grant your service account access to read and write pipeline artifacts in the bucket that you created in the previous step -- you only need to run these once per service account."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "set_service_account:pipelines"
      },
      "outputs": [],
      "source": [
        "! gsutil iam ch serviceAccount:{SERVICE_ACCOUNT}:roles/storage.objectCreator  \n",
        "! gsutil iam ch serviceAccount:{SERVICE_ACCOUNT}:roles/storage.objectViewer  "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "setup_vars"
      },
      "source": [
        "### Set up variables\n",
        "\n",
        "Next, set up some variables used throughout the tutorial.\n",
        "### Import libraries and define constants"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "import_kfp"
      },
      "outputs": [],
      "source": [
        "import json\n",
        "import os\n",
        "\n",
        "import google.cloud.aiplatform as aiplatform\n",
        "from kfp.registry import RegistryClient"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "init_aip:mbsdk,all"
      },
      "source": [
        "### Initialize Vertex AI SDK for Python\n",
        "\n",
        "Initialize the Vertex AI SDK for Python for your project and corresponding bucket."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "init_aip:mbsdk,all"
      },
      "outputs": [],
      "source": [
        "aiplatform.init(project=PROJECT_ID, location=REGION, staging_bucket=BUCKET_URI)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2d242773d707"
      },
      "source": [
        "### Enable Artifact Registry API\n",
        "You must enable the Artifact Registry API service for your project.\n",
        "\n",
        "<a href=\"https://cloud.google.com/artifact-registry/docs/enable-service\">Learn more about Enabling service</a>."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "162b5e8883c2"
      },
      "outputs": [],
      "source": [
        "! gcloud services enable artifactregistry.googleapis.com\n",
        "\n",
        "if os.getenv(\"IS_TESTING\"):\n",
        "    ! sudo apt-get update --yes && sudo apt-get --only-upgrade --yes install google-cloud-sdk-cloud-run-proxy google-cloud-sdk-harbourbridge google-cloud-sdk-cbt google-cloud-sdk-gke-gcloud-auth-plugin google-cloud-sdk-kpt google-cloud-sdk-local-extract google-cloud-sdk-minikube google-cloud-sdk-app-engine-java google-cloud-sdk-app-engine-go google-cloud-sdk-app-engine-python google-cloud-sdk-spanner-emulator google-cloud-sdk-bigtable-emulator google-cloud-sdk-nomos google-cloud-sdk-package-go-module google-cloud-sdk-firestore-emulator kubectl google-cloud-sdk-datastore-emulator google-cloud-sdk-app-engine-python-extras google-cloud-sdk-cloud-build-local google-cloud-sdk-kubectl-oidc google-cloud-sdk-anthos-auth google-cloud-sdk-app-engine-grpc google-cloud-sdk-pubsub-emulator google-cloud-sdk-datalab google-cloud-sdk-skaffold google-cloud-sdk google-cloud-sdk-terraform-tools google-cloud-sdk-config-connector\n",
        "    ! gcloud components update --quiet"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "9b773e8d2bd2"
      },
      "source": [
        "## Create repo in Artifact Registry\n",
        "\n",
        "First, you create your own (user-defined) repository in the `Artifact Registry`. You use this repository to upload and retreive your pipeline templates."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "520de849cee2"
      },
      "outputs": [],
      "source": [
        "REPO_NAME = \"my-docker-repo-unique\"\n",
        "\n",
        "! gcloud artifacts repositories create {REPO_NAME} --location={REGION} --repository-format=KFP"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "1611d3517c0f"
      },
      "source": [
        "### Upload the pipeline template\n",
        "\n",
        "Next, you instantiate a client interface to the Artifact Registry. Then with the `upload_pipeline()` method you upload your pipeline template."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "7f002e57998a"
      },
      "outputs": [],
      "source": [
        "T5X_YAML = \"gs://cloud-samples-data/vertex-ai/dataset-management/datasets/t5_finetuning/pipeline.yaml\"\n",
        "\n",
        "! gsutil cp {T5X_YAML} pipeline.yaml\n",
        "\n",
        "client = RegistryClient(\n",
        "    host=f\"https://{REGION}-kfp.pkg.dev/{PROJECT_ID}/quickstart-kfp-repo\"\n",
        ")\n",
        "\n",
        "templateName, versionName = client.upload_pipeline(\n",
        "    file_name=\"pipeline.yaml\",\n",
        "    tags=[\"v1\", \"latest\"],\n",
        "    extra_headers={\n",
        "        \"description\": \"This is a pipeline template for fine-tuning a T5 model.\"\n",
        "    },\n",
        ")\n",
        "\n",
        "! rm pipeline.yaml"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "02f82754fc0d"
      },
      "source": [
        "### View your artifacts in your registry\n",
        "\n",
        "Next, using the `gcloud artifacts files` command you view the artifacts, inclusive of the pipeline template, in your artifacts repository."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "b2f641eb2056"
      },
      "outputs": [],
      "source": [
        "! gcloud artifacts files list  --repository={REPO_NAME} --location={REGION}"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "968a46a3cb6d"
      },
      "source": [
        "## Load and execute the pipeline job\n",
        "\n",
        "Next, you create a Vertex AI Pipeline job from your T5 pipeline template by instantiating a PipelineJob(), with the following parameters:\n",
        "\n",
        "- `display_name`: The human readable name for the pipeline job.\n",
        "- `template_path`: The path to the pipeline template in the Artifact Registry.\n",
        "- `enable_caching`: On re-runs, use the results from previous successful and unchanged steps.\n",
        "- `pipeline_root`: A Cloud storage location for storing pipeline results.\n",
        "- `parameter_values`: The parameters and values that are input to the template pipeline. In this example, they are:\n",
        "TODO\n",
        "    - `project`: Your project ID.\n",
        "    - `class_labels`: A list of valid class labels, in cardinal order.\n",
        "    - `root_dir`: A Cloud Storage scratch area.\n",
        "    - `training_data_path`: A Cloud Storage location to the training data.\n",
        "    - `ground_truth_gcs_source_uris`: A Cloud Storage location to evaluation data."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "a3c502fc7e41"
      },
      "outputs": [],
      "source": [
        "PIPELINE_ROOT = f\"{BUCKET_URI}/pipeline_root/t5_finetuning\"\n",
        "\n",
        "job = aiplatform.PipelineJob(\n",
        "    display_name=\"t5x-finetuning\",\n",
        "    template_path=f\"https://{REGION}-kfp.pkg.dev/{PROJECT_ID}/quickstart-kfp-repo/{templateName}/{versionName}\",\n",
        "    pipeline_root=PIPELINE_ROOT,\n",
        "    enable_caching=False,\n",
        "    parameter_values={\n",
        "        \"project_id\": PROJECT_ID,\n",
        "        \"accelerator_count\": 32,\n",
        "        \"feature_keys\": \"question\",\n",
        "        \"label_key\": \"answer\",\n",
        "        \"training_data_path\": \"gs://cloud-llm-public/tfds/natural_questions_open/1.0.0_shortened/natural_questions_open-train.tfrecord-00000-of-00001\",\n",
        "        \"validation_data_path\": \"gs://cloud-llm-public/tfds/natural_questions_open/1.0.0_shortened/natural_questions_open-validation.tfrecord-00000-of-00001\",\n",
        "    },\n",
        ")\n",
        "\n",
        "job.run()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view_pipleline_results:bqml"
      },
      "source": [
        "### View the pipeline results"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "view_pipleline_results:bqml"
      },
      "outputs": [],
      "source": [
        "PROJECT_NUMBER = job.gca_resource.name.split(\"/\")[1]\n",
        "print(PROJECT_NUMBER)\n",
        "\n",
        "\n",
        "def print_pipeline_output(job, output_task_name):\n",
        "    JOB_ID = job.name\n",
        "    print(JOB_ID)\n",
        "    artifact = \"\"\n",
        "    for _ in range(len(job.gca_resource.job_detail.task_details)):\n",
        "        TASK_ID = job.gca_resource.job_detail.task_details[_].task_id\n",
        "        EXECUTE_OUTPUT = (\n",
        "            PIPELINE_ROOT\n",
        "            + \"/\"\n",
        "            + PROJECT_NUMBER\n",
        "            + \"/\"\n",
        "            + JOB_ID\n",
        "            + \"/\"\n",
        "            + output_task_name\n",
        "            + \"_\"\n",
        "            + str(TASK_ID)\n",
        "            + \"/executor_output.json\"\n",
        "        )\n",
        "        GCP_RESOURCES = (\n",
        "            PIPELINE_ROOT\n",
        "            + \"/\"\n",
        "            + PROJECT_NUMBER\n",
        "            + \"/\"\n",
        "            + JOB_ID\n",
        "            + \"/\"\n",
        "            + output_task_name\n",
        "            + \"_\"\n",
        "            + str(TASK_ID)\n",
        "            + \"/gcp_resources\"\n",
        "        )\n",
        "        EVALUATION_METRICS = (\n",
        "            PIPELINE_ROOT\n",
        "            + \"/\"\n",
        "            + PROJECT_NUMBER\n",
        "            + \"/\"\n",
        "            + JOB_ID\n",
        "            + \"/\"\n",
        "            + output_task_name\n",
        "            + \"_\"\n",
        "            + str(TASK_ID)\n",
        "            + \"/evaluation_metrics\"\n",
        "        )\n",
        "        # Check if file exists, 0 is success\n",
        "        !gsutil -q stat $EXECUTE_OUTPUT\n",
        "        if _exit_code == 0:\n",
        "            ! gsutil cat $EXECUTE_OUTPUT\n",
        "            artifact = EXECUTE_OUTPUT\n",
        "            break\n",
        "        !gsutil -q stat $GCP_RESOURCES\n",
        "        if _exit_code == 0:\n",
        "            ! gsutil cat $GCP_RESOURCES\n",
        "            artifact = GCP_RESOURCES\n",
        "            break\n",
        "        !gsutil -q stat $EVALUATION_METRICS\n",
        "        if _exit_code == 0:\n",
        "            ! gsutil cat $EVALUATION_METRICS\n",
        "            artifact = EVALUATION_METRICS\n",
        "            break\n",
        "\n",
        "    return artifact\n",
        "\n",
        "\n",
        "print(\"model-upload\")\n",
        "artifacts = print_pipeline_output(job, \"model-upload\")\n",
        "output = !gsutil cat $artifacts\n",
        "print(output)\n",
        "output = json.loads(output[0])\n",
        "model_id = output[\"artifacts\"][\"model\"][\"artifacts\"][0][\"metadata\"][\"resourceName\"]\n",
        "print(\"\\n\\n\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "f431a9e6f025"
      },
      "source": [
        "### Delete the pipeline job\n",
        "\n",
        "The method 'delete()' will delete the pipeline job."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "00bf554abbc6"
      },
      "outputs": [],
      "source": [
        "job.delete()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "3d183db57ae2"
      },
      "source": [
        "### Deploy the model\n",
        "\n",
        "Next, you deploy the model to an endpoint:\n",
        "\n",
        "- Use the `model_id` obtained from the pipeline artifacts to instantiate a Vertex AI Model resource instance.\n",
        "- Deploy the Vertex AI Model resource to a Vertex AI Endpoint resource.\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "591ccc049ce5"
      },
      "outputs": [],
      "source": [
        "model = aiplatform.Model(model_id)\n",
        "endpoint = model.deploy(\n",
        "    accelerator_count=1,\n",
        "    accelerator_type=aiplatform.gapic.AcceleratorType.NVIDIA_TESLA_T4.name,\n",
        "    machine_type=\"n1-standard-4\",\n",
        ")\n",
        "print(endpoint)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "edb781a92864"
      },
      "source": [
        "### Make a prediction\n",
        "\n",
        "Finally, you make a prediction with the deployed model."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "72d94012a987"
      },
      "outputs": [],
      "source": [
        "endpoint.predict([\"this is a test\"])"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "cleanup:mbsdk"
      },
      "source": [
        "# Cleaning up\n",
        "\n",
        "To clean up all Google Cloud resources used in this project, you can [delete the Google Cloud\n",
        "project](https://cloud.google.com/resource-manager/docs/creating-managing-projects#shutting_down_projects) you used for the tutorial.\n",
        "\n",
        "Otherwise, you can delete the individual resources you created in this tutorial."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "cleanup:mbsdk"
      },
      "outputs": [],
      "source": [
        "delete_bucket = False\n",
        "\n",
        "endpoint.undeploy_all()\n",
        "endpoint.delete()\n",
        "model.delete()\n",
        "\n",
        "if delete_bucket or os.getenv(\"IS_TESTING\"):\n",
        "    ! gsutil rm -r $BUCKET_URI\n",
        "\n",
        "! rm -rf custom custom.tar.gz\n",
        "\n",
        "! gcloud artifacts repositories delete $REPO_NAME --project {PROJECT_ID} --location {REGION} --quiet"
      ]
    }
  ],
  "metadata": {
    "colab": {
      "name": "model_garden_pipeline_templates_t5x.ipynb",
      "toc_visible": true
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}