AML Tutorial (#255)

advanced_tutorials/aml/3_aml_online_inference.ipynb

@@ -0,0 +1,289 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# <span style=\"font-width:bold; font-size: 3rem; color:#1EB182;\"><img src=\"../../images/icon102.png\" width=\"38px\"></img> **Hopsworks Feature Store** </span><span style=\"font-width:bold; font-size: 3rem; color:#333;\">- Part 03: Online Inference</span>\n",
+    "\n",
+    "<span style=\"font-width:bold; font-size: 1.4rem;\">In this last notebook you will use your deployment for online inference.  </span>\n",
+    "\n",
+    "## **🗒️ This notebook is divided into the following sections:** \n",
+    "1. **Deployment Retrieval**: Retrieve your deployment from the model registry.\n",
+    "2. **Prediction using deployment**.\n",
+    "3. **REST endpoint usage for model serving**.\n",
+    "\n",
+    "![tutorial-flow](../../images/03_model.png)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## <span style=\"color:#ff5f27;\"> 📡 Connecting to Hopsworks Feature Store </span>"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Connected. Call `.close()` to terminate connection gracefully.\n",
+      "\n",
+      "Logged in to project, explore it here https://staging.cloud.hopsworks.ai/p/120\n",
+      "Connected. Call `.close()` to terminate connection gracefully.\n"
+     ]
+    }
+   ],
+   "source": [
+    "import hopsworks\n",
+    "\n",
+    "project = hopsworks.login()\n",
+    "\n",
+    "# Get the feature store handle for the project's feature store\n",
+    "fs = project.get_feature_store()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## <span style=\"color:#ff5f27;\">🗄 Model Registry</span>\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Connected. Call `.close()` to terminate connection gracefully.\n"
+     ]
+    }
+   ],
+   "source": [
+    "# Get the Model Registry\n",
+    "mr = project.get_model_registry()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## <span style='color:#ff5f27'>⚙️ Fetch Deployment</span>"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Access the Model Serving\n",
+    "ms = project.get_model_serving()\n",
+    "\n",
+    "# Specify the deployment name\n",
+    "deployment_name = \"amlmodeldeployment\"\n",
+    "\n",
+    "# Get the deployment with the specified name\n",
+    "deployment = ms.get_deployment(deployment_name)\n",
+    "\n",
+    "# Start the deployment and wait for it to be in a running state for up to 300 seconds\n",
+    "deployment.start(await_running=300)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## <span style='color:#ff5f27'>🔮 Predicting using deployment</span>\n",
+    "\n",
+    "\n",
+    "Finally you can start making predictions with your model!\n",
+    "\n",
+    "Send inference requests to the deployed model as follows:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Prepare input data using the input example from the model\n",
+    "data = {\n",
+    "    \"inputs\": model.input_example,\n",
+    "}\n",
+    "\n",
+    "# Make predictions using the deployed model\n",
+    "predictions = deployment.predict(data)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Prepare input data with a specific input value\n",
+    "data = {\n",
+    "    \"inputs\": [\"f1c119ed\"],\n",
+    "}\n",
+    "\n",
+    "# Make predictions using the deployed model\n",
+    "predictions = deployment.predict(data)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "> For trouble shooting one can use `get_logs` method."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "deployment.get_logs()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## <span style='color:#ff5f27'>🚀 Use REST endpoint</span>\n",
+    "\n",
+    "You can also use a REST endpoint for your model. To do this you need to create an API key with 'serving' enabled, and retrieve the endpoint URL from the Model Serving UI.\n",
+    "\n",
+    "Go to the Model Serving UI and click on the eye icon next to a model to retrieve the endpoint URL. The shorter URL is an internal endpoint that you can only reach from within Hopsworks. If you want to call it from outside, you need one of the longer URLs. \n",
+    "\n",
+    "![serving-endpoints](images/serving_endpoints.gif)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import os\n",
+    "import requests\n",
+    "\n",
+    "mr = project.get_model_registry()\n",
+    "\n",
+    "# Use the model name from the previous notebook.\n",
+    "model = mr.get_model(\n",
+    "    name=\"fraud_tutorial_model\", \n",
+    "    version=1,\n",
+    ")\n",
+    "\n",
+    "test_inputs = model.input_example\n",
+    "\n",
+    "API_KEY = \"...\"  # Put your API key here.\n",
+    "MODEL_SERVING_URL = \"...\" # Put model serving endppoint here.\n",
+    "HOST_NAME = \"...\" # Put your hopsworks model serving hostname here \n",
+    "\n",
+    "data = {\"inputs\": test_inputs}\n",
+    "headers = {\n",
+    "    \"Content-Type\": \"application/json\", \"Accept\": \"application/json\",\n",
+    "    \"Authorization\": f\"ApiKey {API_KEY}\",\n",
+    "    \"Host\": HOST_NAME}\n",
+    "\n",
+    "response = requests.post(MODEL_SERVING_URL, verify=False, headers=headers, json=data)\n",
+    "response.json()"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Now lets test feature vectors from online store\n",
+    "ids_to_score = [\n",
+    "    \"0016359b\", \n",
+    "    \"001dcc27\", \n",
+    "    \"0054a022\", \n",
+    "    \"00d6b609\", \n",
+    "    \"00e14860\", \n",
+    "    \"00e39a1b\", \n",
+    "    \"014ed5cb\", \n",
+    "    \"01ce3306\", \n",
+    "    \"01fa19ae\", \n",
+    "    \"01fa1d01\", \n",
+    "    \"036dce03\", \n",
+    "    \"03e09be4\", \n",
+    "    \"04b23f4b\",\n",
+    "]\n",
+    "\n",
+    "for node_id in ids_to_score:\n",
+    "    data = {\"inputs\": [node_id]}\n",
+    "    print(\" anomaly score for node_id \", node_id, \" : \",   deployment.predict(data)[\"outputs\"])"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Stop Deployment\n",
+    "To stop the deployment you simply run:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "deployment.stop()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## <span style=\"color:#ff5f27;\"> 🎁 Wrapping things up </span>"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "In this module you perforemed feature engineering, created feature view and traning dataset, trained advesarial anomaly detection model and depoyed it in production. To setup this pipeline in your enterprise settings contuct us.\n",
+    "\n",
+    "<img src=\"images/contuct_us.png\" width=\"400px\"></img>"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.10.13"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 4
+}