2 AutoML Classification.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# AutoML Classification with Azure ML"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src='https://github.com/retkowsky/images/blob/master/AzureMLservicebanniere.png?raw=true'>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This is a copy of the test set of the **UCI ML hand-written digits datasets** http://archive.ics.uci.edu/ml/datasets/Optical+Recognition+of+Handwritten+Digits\n",
    "\n",
    "The data set contains images of hand-written digits: 10 classes where each class refers to a digit.\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"https://www.seewald.at/img/L/digits.gif\">\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Documentation Azure ML service\n",
    "> https://docs.microsoft.com/en-us/azure/machine-learning/service/how-to-automated-ml"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1. Informations\n",
    "\n",
    "As part of the setup you have already created an Azure ML `Workspace` object. <br> For AutoML you will need to create an `Experiment` object, which is a named object in a `Workspace` used to run experiments."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'3.6.9 |Anaconda, Inc.| (default, Jul 30 2019, 19:07:31) \\n[GCC 7.3.0]'"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import sys\n",
    "sys.version"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2020-09-16 12:44:10.812443\n"
     ]
    }
   ],
   "source": [
    "import datetime\n",
    "now = datetime.datetime.now()\n",
    "print(now)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Version Azure ML : 1.13.0\n"
     ]
    }
   ],
   "source": [
    "import azureml.core\n",
    "print(\"Version Azure ML :\", azureml.core.VERSION)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "import logging\n",
    "import os\n",
    "import random\n",
    "\n",
    "from matplotlib import pyplot as plt\n",
    "from matplotlib.pyplot import imshow\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "from sklearn import datasets\n",
    "\n",
    "import azureml.core\n",
    "from azureml.core.experiment import Experiment\n",
    "from azureml.core.workspace import Workspace\n",
    "from azureml.train.automl import AutoMLConfig\n",
    "from azureml.train.automl.run import AutoMLRun"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2. Connexion workspace Azure ML service"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>SDK version</th>\n",
       "      <td>1.13.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Workspace Name</th>\n",
       "      <td>workshopAML2020</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Resource Group</th>\n",
       "      <td>workshopAML2020-rg</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Location</th>\n",
       "      <td>westeurope</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Project Directory</th>\n",
       "      <td>./sample_projects/workshop2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Experiment Name</th>\n",
       "      <td>Exemple2-automlclassificationSR</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                  \n",
       "SDK version        1.13.0                         \n",
       "Workspace Name     workshopAML2020                \n",
       "Resource Group     workshopAML2020-rg             \n",
       "Location           westeurope                     \n",
       "Project Directory  ./sample_projects/workshop2    \n",
       "Experiment Name    Exemple2-automlclassificationSR"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ws = Workspace.from_config()\n",
    "\n",
    "experiment_name = 'Exemple2-automlclassificationSR'\n",
    "\n",
    "project_folder = './sample_projects/workshop2'\n",
    "\n",
    "experiment = Experiment(ws, experiment_name)\n",
    "\n",
    "output = {}\n",
    "output['SDK version'] = azureml.core.VERSION\n",
    "output['Workspace Name'] = ws.name\n",
    "output['Resource Group'] = ws.resource_group\n",
    "output['Location'] = ws.location\n",
    "output['Project Directory'] = project_folder\n",
    "output['Experiment Name'] = experiment.name\n",
    "pd.set_option('display.max_colwidth', -1)\n",
    "pd.DataFrame(data = output, index = ['']).T"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3. Chargement des données\n",
    "\n",
    "This uses scikit-learn's [load_digits](http://scikit-learn.org/stable/modules/generated/sklearn.datasets.load_digits.html) method."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Each datapoint is a 8x8 image of a digit."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn import datasets\n",
    "\n",
    "digits = datasets.load_digits()\n",
    "\n",
    "# Exclude the first 100 rows from training so that they can be used for test.\n",
    "X_train = digits.data[100:,:]\n",
    "y_train = digits.target[100:]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Taille du dataset :  (1797, 64)\n"
     ]
    }
   ],
   "source": [
    "print(\"Taille du dataset : \", digits.data.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> On a 1797 images de 8 sur 8 pixels (=64)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exemples d'images :"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "def visu(chiffre):\n",
    "    import matplotlib.pyplot as plt \n",
    "    plt.gray() \n",
    "    plt.matshow(digits.images[chiffre]) \n",
    "    plt.show() "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<Figure size 432x288 with 0 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPoAAAECCAYAAADXWsr9AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAL40lEQVR4nO3dW4hd9RXH8d+vY7xGSaxWJBHtSAmIUHNBKgFpNYpWsS81RFCotCQPrRha0NiX4ptPYh+KELxU8IajBoq01gQVEVrtTIz1MrFoiJhEHSWRGAsR4+rD2SkxnTp7xv3/z5mzvh845MzMmb3WzOR39t7n7L2XI0IABtu3ZrsBAOURdCABgg4kQNCBBAg6kABBBxLoi6DbvsL2W7bftr2hcK37bE/Yfr1knSPqnWX7Odvjtt+wfXPhesfbftn2q02920vWa2oO2X7F9lOlazX1dtp+zfY226OFay2w/bjt7c3f8KKCtZY0P9Ph237b6ztZeETM6k3SkKR3JA1LOlbSq5LOK1jvYknLJL1e6ec7U9Ky5v7Jkv5V+OezpPnN/XmSXpL0g8I/468lPSzpqUq/052STqtU6wFJv2juHytpQaW6Q5I+kHR2F8vrhzX6hZLejogdEfG5pEcl/aRUsYh4QdLeUsufpN77EbG1uf+ppHFJiwrWi4g40Hw4r7kVOyrK9mJJV0m6p1SN2WL7FPVWDPdKUkR8HhGfVCp/qaR3IuLdLhbWD0FfJOm9Iz7epYJBmE22z5G0VL21bMk6Q7a3SZqQtDkiSta7S9Itkr4sWONoIekZ22O21xasMyzpI0n3N7sm99g+qWC9I62R9EhXC+uHoHuSzw3ccbm250t6QtL6iNhfslZEHIqICyQtlnSh7fNL1LF9taSJiBgrsfyvsTIilkm6UtIvbV9cqM4x6u3m3R0RSyV9Jqnoa0iSZPtYSddIGulqmf0Q9F2Szjri48WS9sxSL0XYnqdeyB+KiCdr1W02M5+XdEWhEislXWN7p3q7XJfYfrBQrf+KiD3NvxOSNqm3+1fCLkm7jtgiely94Jd2paStEfFhVwvsh6D/Q9L3bH+3eSZbI+lPs9xTZ2xbvX288Yi4s0K9020vaO6fIGmVpO0lakXEbRGxOCLOUe/v9mxEXF+i1mG2T7J98uH7ki6XVOQdlIj4QNJ7tpc0n7pU0pslah3lOnW42S71Nk1mVUR8YftXkv6q3iuN90XEG6Xq2X5E0g8lnWZ7l6TfRcS9peqpt9a7QdJrzX6zJP02Iv5cqN6Zkh6wPaTeE/ljEVHlba9KzpC0qff8qWMkPRwRTxesd5Okh5qV0A5JNxasJdsnSrpM0rpOl9u8lA9ggPXDpjuAwgg6kABBBxIg6EACBB1IoK+CXvhwxlmrRT3qzXa9vgq6pJq/zKp/OOpRbzbr9VvQARRQ5IAZ2wN9FM7ChQun/T0HDx7UcccdN6N6ixZN/2S+vXv36tRTT51Rvf37p3/OzYEDBzR//vwZ1du9e/e0vyci1BwdN22HDh2a0ffNFRHxP7+YWT8Edi5atWpV1Xp33HFH1XpbtmypWm/DhuInhH3Fvn37qtbrB2y6AwkQdCABgg4kQNCBBAg6kABBBxIg6EACBB1IoFXQa45MAtC9KYPeXGTwD+pdgvY8SdfZPq90YwC602aNXnVkEoDutQl6mpFJwKBqc1JLq5FJzYnytc/ZBdBCm6C3GpkUERslbZQG/zRVYK5ps+k+0COTgAymXKPXHpkEoHutLjzRzAkrNSsMQGEcGQckQNCBBAg6kABBBxIg6EACBB1IgKADCRB0IAEmtcxA7ckpw8PDVevNZOTUN7F3796q9VavXl213sjISNV6k2GNDiRA0IEECDqQAEEHEiDoQAIEHUiAoAMJEHQgAYIOJEDQgQTajGS6z/aE7ddrNASge23W6H+UdEXhPgAUNGXQI+IFSXXPOgDQKfbRgQQ6O02V2WtA/+os6MxeA/oXm+5AAm3eXntE0t8kLbG9y/bPy7cFoEtthixeV6MRAOWw6Q4kQNCBBAg6kABBBxIg6EACBB1IgKADCRB0IIGBmL22fPnyqvVqz0I799xzq9bbsWNH1XqbN2+uWq/2/xdmrwGogqADCRB0IAGCDiRA0IEECDqQAEEHEiDoQAIEHUiAoAMJtLk45Fm2n7M9bvsN2zfXaAxAd9oc6/6FpN9ExFbbJ0sas705It4s3BuAjrSZvfZ+RGxt7n8qaVzSotKNAejOtPbRbZ8jaamkl0o0A6CM1qep2p4v6QlJ6yNi/yRfZ/Ya0KdaBd32PPVC/lBEPDnZY5i9BvSvNq+6W9K9ksYj4s7yLQHoWpt99JWSbpB0ie1tze3HhfsC0KE2s9delOQKvQAohCPjgAQIOpAAQQcSIOhAAgQdSICgAwkQdCABgg4kMBCz1xYuXFi13tjYWNV6tWeh1Vb795kRa3QgAYIOJEDQgQQIOpAAQQcSIOhAAgQdSICgAwkQdCABgg4k0OYqsMfbftn2q83stdtrNAagO22OdT8o6ZKIONBc3/1F23+JiL8X7g1AR9pcBTYkHWg+nNfcGNAAzCGt9tFtD9neJmlC0uaIYPYaMIe0CnpEHIqICyQtlnSh7fOPfozttbZHbY923SSAb2Zar7pHxCeSnpd0xSRf2xgRKyJiRUe9AehIm1fdT7e9oLl/gqRVkraXbgxAd9q86n6mpAdsD6n3xPBYRDxVti0AXWrzqvs/JS2t0AuAQjgyDkiAoAMJEHQgAYIOJEDQgQQIOpAAQQcSIOhAAsxem4EtW7ZUrTfoav/99u3bV7VeP2CNDiRA0IEECDqQAEEHEiDoQAIEHUiAoAMJEHQgAYIOJEDQgQRaB70Z4vCKbS4MCcwx01mj3yxpvFQjAMppO5JpsaSrJN1Tth0AJbRdo98l6RZJXxbsBUAhbSa1XC1pIiLGpngcs9eAPtVmjb5S0jW2d0p6VNIlth88+kHMXgP615RBj4jbImJxRJwjaY2kZyPi+uKdAegM76MDCUzrUlIR8bx6Y5MBzCGs0YEECDqQAEEHEiDoQAIEHUiAoAMJEHQgAYIOJDAQs9dqz9Javnx51Xq11Z6FVvv3OTIyUrVeP2CNDiRA0IEECDqQAEEHEiDoQAIEHUiAoAMJEHQgAYIOJEDQgQRaHQLbXOr5U0mHJH3BJZ2BuWU6x7r/KCI+LtYJgGLYdAcSaBv0kPSM7THba0s2BKB7bTfdV0bEHtvfkbTZ9vaIeOHIBzRPADwJAH2o1Ro9IvY0/05I2iTpwkkew+w1oE+1maZ6ku2TD9+XdLmk10s3BqA7bTbdz5C0yfbhxz8cEU8X7QpAp6YMekTskPT9Cr0AKIS314AECDqQAEEHEiDoQAIEHUiAoAMJEHQgAYIOJOCI6H6hdvcL/RrDw8M1y2l0dLRqvXXr1lWtd+2111atV/vvt2LFYJ+OERE++nOs0YEECDqQAEEHEiDoQAIEHUiAoAMJEHQgAYIOJEDQgQQIOpBAq6DbXmD7cdvbbY/bvqh0YwC603aAw+8lPR0RP7V9rKQTC/YEoGNTBt32KZIulvQzSYqIzyV9XrYtAF1qs+k+LOkjSffbfsX2Pc0gh6+wvdb2qO26p3YBmFKboB8jaZmkuyNiqaTPJG04+kGMZAL6V5ug75K0KyJeaj5+XL3gA5gjpgx6RHwg6T3bS5pPXSrpzaJdAehU21fdb5L0UPOK+w5JN5ZrCUDXWgU9IrZJYt8bmKM4Mg5IgKADCRB0IAGCDiRA0IEECDqQAEEHEiDoQAIDMXuttrVr11atd+utt1atNzY2VrXe6tWrq9YbdMxeA5Ii6EACBB1IgKADCRB0IAGCDiRA0IEECDqQAEEHEpgy6LaX2N52xG2/7fU1mgPQjSmvGRcRb0m6QJJsD0naLWlT4b4AdGi6m+6XSnonIt4t0QyAMqYb9DWSHinRCIByWge9uab7NZJG/s/Xmb0G9Km2Axwk6UpJWyPiw8m+GBEbJW2UBv80VWCumc6m+3Visx2Yk1oF3faJki6T9GTZdgCU0HYk078lfbtwLwAK4cg4IAGCDiRA0IEECDqQAEEHEiDoQAIEHUiAoAMJEHQggVKz1z6SNJNz1k+T9HHH7fRDLepRr1a9syPi9KM/WSToM2V7NCJWDFot6lFvtuux6Q4kQNCBBPot6BsHtBb1qDer9fpqHx1AGf22RgdQAEEHEiDoQAIEHUiAoAMJ/AchD47vPuZI8QAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 288x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "visu(0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<Figure size 432x288 with 0 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPoAAAECCAYAAADXWsr9AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAALpklEQVR4nO3d/2td9R3H8ddraYvfaiPTiVixE2ZBhCVFyqSg/aJSp7S/7IcWFCYb3Q+bGDYQ3S/Vf0DcD0MoVStYK1otHbI5CxpE2HT9Emc1dWipmFaNYtOqgxX1vR/uqWRdtpzE8zm5yfv5gEvuvbk573cSXvdzzrnnnI8jQgDmtu/MdAMAyiPoQAIEHUiAoAMJEHQgAYIOJNAVQbe91vbbtt+xfU/hWo/YHrV9sGSdcfUus/2S7WHbb9q+q3C9s2y/Zvv1qt79JetVNXtsH7D9XOlaVb0jtt+wPWR7b+FavbZ32j5U/Q+vLVhrafU7nb6dtD3QyMIjYkZvknokvSvpCkkLJL0u6aqC9a6TtEzSwZZ+v0skLavuL5T0j8K/nyWdV92fL+lVST8q/Dv+WtITkp5r6W96RNKFLdV6TNLPq/sLJPW2VLdH0oeSLm9ied0woi+X9E5EHI6IU5KelLS+VLGIeFnSp6WWP0G9DyJif3X/M0nDki4tWC8i4vPq4fzqVuyoKNuLJd0iaWupGjPF9vnqDAwPS1JEnIqIsZbKr5H0bkS818TCuiHol0p6f9zjERUMwkyyvURSvzqjbMk6PbaHJI1K2hMRJes9KOluSV8XrHGmkPSC7X22NxWsc4WkjyU9Wm2abLV9bsF6422QtKOphXVD0D3Bc3PuuFzb50l6RtJARJwsWSsivoqIPkmLJS23fXWJOrZvlTQaEftKLP//WBERyyTdLOmXtq8rVGeeOpt5D0VEv6QvJBXdhyRJthdIWifp6aaW2Q1BH5F02bjHiyUdm6FeirA9X52Qb4+IZ9uqW61mDkpaW6jECknrbB9RZ5Nrte3HC9X6RkQcq76OStqlzuZfCSOSRsatEe1UJ/il3Sxpf0R81NQCuyHof5P0A9vfr97JNkj6wwz31BjbVmcbbzgiHmih3kW2e6v7Z0u6QdKhErUi4t6IWBwRS9T5v70YEbeVqHWa7XNtLzx9X9JNkop8ghIRH0p63/bS6qk1kt4qUesMG9XgarvUWTWZURHxpe1fSfqzOnsaH4mIN0vVs71D0kpJF9oekbQ5Ih4uVU+dUe92SW9U282S9NuI+GOhepdIesx2jzpv5E9FRCsfe7XkYkm7Ou+fmifpiYh4vmC9OyVtrwahw5LuKFhLts+RdKOkXzS63GpXPoA5rBtW3QEURtCBBAg6kABBBxIg6EACXRX0woczzlgt6lFvput1VdAltfnHbPUfRz3qzWS9bgs6gAKKHDBjm6NwGnTllVdO+WdOnDihRYsWTavevHlTP2Dy+PHjuuCCC6ZV7+jRo1P+mVOnTmnBggXTqnfixIlp/dxsERH/daIYQZ8FBgcHW63X29vbar3Nmze3Wm/37t2t1mvbREFn1R1IgKADCRB0IAGCDiRA0IEECDqQAEEHEiDoQAK1gt7mlEkAmjdp0KuLDP5enUvQXiVpo+2rSjcGoDl1RvRWp0wC0Lw6QU8zZRIwV9U5TanWlEnVifJtn7MLoIY6Qa81ZVJEbJG0ReLsNaDb1Fl1n9NTJgEZTDqitz1lEoDm1bqUSDVPWKm5wgAUxpFxQAIEHUiAoAMJEHQgAYIOJEDQgQQIOpAAQQcSmPrcO2jd2NhYq/Wuv/76VuutWrWq1XpzfaaWiTCiAwkQdCABgg4kQNCBBAg6kABBBxIg6EACBB1IgKADCRB0IIE6UzI9YnvU9sE2GgLQvDoj+jZJawv3AaCgSYMeES9L+rSFXgAUwjY6kEBjp6ky9xrQvRoLOnOvAd2LVXcggTofr+2Q9BdJS22P2P5Z+bYANKnOJIsb22gEQDmsugMJEHQgAYIOJEDQgQQIOpAAQQcSIOhAAgQdSIC516ahr6+v1XorV65stV7bhoaGZrqFOY8RHUiAoAMJEHQgAYIOJEDQgQQIOpAAQQcSIOhAAgQdSICgAwnUuTjkZbZfsj1s+03bd7XRGIDm1DnW/UtJv4mI/bYXStpne09EvFW4NwANqTP32gcRsb+6/5mkYUmXlm4MQHOmtI1ue4mkfkmvlmgGQBm1T1O1fZ6kZyQNRMTJCb7P3GtAl6oVdNvz1Qn59oh4dqLXMPca0L3q7HW3pIclDUfEA+VbAtC0OtvoKyTdLmm17aHq9uPCfQFoUJ25116R5BZ6AVAIR8YBCRB0IAGCDiRA0IEECDqQAEEHEiDoQAIEHUhgTsy9NjAw0Gq9++67r9V6ixYtarVe2wYHB2e6hTmPER1IgKADCRB0IAGCDiRA0IEECDqQAEEHEiDoQAIEHUiAoAMJ1LkK7Fm2X7P9ejX32v1tNAagOXWOdf+XpNUR8Xl1ffdXbP8pIv5auDcADalzFdiQ9Hn1cH51Y4IGYBaptY1uu8f2kKRRSXsigrnXgFmkVtAj4quI6JO0WNJy21ef+Rrbm2zvtb236SYBfDtT2useEWOSBiWtneB7WyLimoi4pqHeADSkzl73i2z3VvfPlnSDpEOlGwPQnDp73S+R9JjtHnXeGJ6KiOfKtgWgSXX2uv9dUn8LvQAohCPjgAQIOpAAQQcSIOhAAgQdSICgAwkQdCABgg4k4M5ZqA0v1J7Tp7H29va2Wu/48eOt1mtbf3+7x2MNDQ21Wq9tEeEzn2NEBxIg6EACBB1IgKADCRB0IAGCDiRA0IEECDqQAEEHEiDoQAK1g15N4nDANheGBGaZqYzod0kaLtUIgHLqTsm0WNItkraWbQdACXVH9Acl3S3p64K9ACikzkwtt0oajYh9k7yOudeALlVnRF8haZ3tI5KelLTa9uNnvoi514DuNWnQI+LeiFgcEUskbZD0YkTcVrwzAI3hc3QggTqTLH4jIgbVmTYZwCzCiA4kQNCBBAg6kABBBxIg6EACBB1IgKADCRB0IIEpHTADlNDX19dqvbk+99pEGNGBBAg6kABBBxIg6EACBB1IgKADCRB0IAGCDiRA0IEECDqQQK1DYKtLPX8m6StJX3JJZ2B2mcqx7qsi4pNinQAohlV3IIG6QQ9JL9jeZ3tTyYYANK/uqvuKiDhm+3uS9tg+FBEvj39B9QbAmwDQhWqN6BFxrPo6KmmXpOUTvIa514AuVWc21XNtLzx9X9JNkg6WbgxAc+qsul8saZft069/IiKeL9oVgEZNGvSIOCzphy30AqAQPl4DEiDoQAIEHUiAoAMJEHQgAYIOJEDQgQQIOpAAQQcSIOhAAgQdSICgAwkQdCABgg4kQNCBBAg6kABBBxIg6EACBB1IoFbQbffa3mn7kO1h29eWbgxAc+pO4PA7Sc9HxE9sL5B0TsGeADRs0qDbPl/SdZJ+KkkRcUrSqbJtAWhSnVX3KyR9LOlR2wdsb60mcvgPtjfZ3mt7b+NdAvhW6gR9nqRlkh6KiH5JX0i658wXMSUT0L3qBH1E0khEvFo93qlO8AHMEpMGPSI+lPS+7aXVU2skvVW0KwCNqrvX/U5J26s97ocl3VGuJQBNqxX0iBiSxLY3MEtxZByQAEEHEiDoQAIEHUiAoAMJEHQgAYIOJEDQgQTqHhmHccbGxlqtt3v37lbrrV+/vtV6K1eubLXetm3bWq3XDRjRgQQIOpAAQQcSIOhAAgQdSICgAwkQdCABgg4kQNCBBCYNuu2ltofG3U7aHmijOQDNmPQQ2Ih4W1KfJNnukXRU0q7CfQFo0FRX3ddIejci3ivRDIAyphr0DZJ2lGgEQDm1g15d032dpKf/x/eZew3oUlM5TfVmSfsj4qOJvhkRWyRtkSTb0UBvABoylVX3jWK1HZiVagXd9jmSbpT0bNl2AJRQd0qmf0r6buFeABTCkXFAAgQdSICgAwkQdCABgg4kQNCBBAg6kABBBxIg6EACjmj+/BPbH0uazjnrF0r6pOF2uqEW9ajXVr3LI+KiM58sEvTpsr03Iq6Za7WoR72ZrseqO5AAQQcS6Lagb5mjtahHvRmt11Xb6ADK6LYRHUABBB1IgKADCRB0IAGCDiTwbwuQdvD/0C3PAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 288x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "visu(1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<Figure size 432x288 with 0 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPoAAAECCAYAAADXWsr9AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAL50lEQVR4nO3d/4tVdR7H8ddrJ6UvWgPWRmQ0Gy1CBI0isiGEqxW2hfrD/qCwwcYu7g+7oexC1P6i/QPh/rAEYmmQGWWpS+y2CRkR7NaqjZs1JiUTzVpNX1ArYe3Le3+4x3Bdtzkznc+ZO/N+PuDinTt3zutzHV73nHPnnPNxRAjA1Pa9iR4AgPIoOpAARQcSoOhAAhQdSICiAwl0RdFtL7X9pu23bN9bOOth2yO2D5bMOSPvKtt7bA/aft32msJ559t+xfaBKu/+knlVZo/tV20/Uzqryhuy/ZrtAdt7C2f12t5u+1D1O7yxYNac6jWdvp2wvbaRhUfEhN4k9Uh6W9I1kqZLOiDpuoJ5N0maJ+lgS6/vCknzqvszJR0u/PosaUZ1f5qklyX9qPBr/K2kxyQ909L/6ZCkS1vKekTSL6v70yX1tpTbI+l9SVc3sbxuWKMvkPRWRByJiFOSHpe0vFRYRLwo6ZNSyz9H3nsRsb+6/6mkQUlXFsyLiPis+nJadSt2VJTt2ZJul7SpVMZEsX2xOiuGhyQpIk5FxLGW4pdIejsi3mliYd1Q9CslvXvG18MqWISJZLtP0lx11rIlc3psD0gakbQ7IkrmbZB0j6SvC2acLSQ9Z3uf7dUFc66R9KGkzdWuySbbFxXMO9NKSduaWlg3FN3neGzKHZdre4akpyStjYgTJbMi4quI6Jc0W9IC29eXyLF9h6SRiNhXYvnfYmFEzJN0m6Rf276pUM556uzmPRgRcyV9LqnoZ0iSZHu6pGWSnmxqmd1Q9GFJV53x9WxJRydoLEXYnqZOybdGxNNt5VabmS9IWlooYqGkZbaH1NnlWmz70UJZ34iIo9W/I5J2qLP7V8KwpOEztoi2q1P80m6TtD8iPmhqgd1Q9H9I+qHtH1TvZCsl/WmCx9QY21ZnH28wIh5oIe8y273V/Qsk3SzpUImsiLgvImZHRJ86v7fnI+JnJbJOs32R7Zmn70u6VVKRv6BExPuS3rU9p3poiaQ3SmSdZZUa3GyXOpsmEyoivrT9G0l/VeeTxocj4vVSeba3SVok6VLbw5LWRcRDpfLUWevdKem1ar9Zkn4fEX8ulHeFpEds96jzRv5ERLTyZ6+WXC5pR+f9U+dJeiwini2Yd7ekrdVK6IikuwpmyfaFkm6R9KtGl1t9lA9gCuuGTXcAhVF0IAGKDiRA0YEEKDqQQFcVvfDhjBOWRR55E53XVUWX1OZ/Zqu/OPLIm8i8bis6gAKKHDBjm6NwGjRjxowx/8wXX3yhadOmjSvv2muvHfPPfPzxx5o1a9a48k6ePDnmnzl+/LguueSSceUdPnx4XD83WUTE/5woNuGHwGJ08+fPbzVv586dreYNDAyM/qQGLVq0qNW8bsCmO5AARQcSoOhAAhQdSICiAwlQdCABig4kQNGBBGoVvc0pkwA0b9SiVxcZ/KM6l6C9TtIq29eVHhiA5tRZo7c6ZRKA5tUpepopk4Cpqs5JLbWmTKpOlG/7nF0ANdQpeq0pkyJio6SNEqepAt2mzqb7lJ4yCchg1DV621MmAWherQtPVPOElZorDEBhHBkHJEDRgQQoOpAARQcSoOhAAhQdSICiAwlQdCABZmoZh/7+/lbz9uzZ02re8ePHW83r6+trNS8j1uhAAhQdSICiAwlQdCABig4kQNGBBCg6kABFBxKg6EACFB1IoM6UTA/bHrF9sI0BAWhenTX6FklLC48DQEGjFj0iXpT0SQtjAVAI++hAAo2dpsrca0D3aqzozL0GdC823YEE6vx5bZukv0maY3vY9i/KDwtAk+pMsriqjYEAKIdNdyABig4kQNGBBCg6kABFBxKg6EACFB1IgKIDCTD32jisWLGi1bwDBw60mrdz585W89atW9dqXkas0YEEKDqQAEUHEqDoQAIUHUiAogMJUHQgAYoOJEDRgQQoOpBAnYtDXmV7j+1B26/bXtPGwAA0p86x7l9K+l1E7Lc9U9I+27sj4o3CYwPQkDpzr70XEfur+59KGpR0ZemBAWjOmPbRbfdJmivp5RKDAVBG7dNUbc+Q9JSktRFx4hzfZ+41oEvVKrrtaeqUfGtEPH2u5zD3GtC96nzqbkkPSRqMiAfKDwlA0+rsoy+UdKekxbYHqttPCo8LQIPqzL32kiS3MBYAhXBkHJAARQcSoOhAAhQdSICiAwlQdCABig4kQNGBBJh7bRw2bNjQat7Q0FCreW2/vl27drWalxFrdCABig4kQNGBBCg6kABFBxKg6EACFB1IgKIDCVB0IAGKDiRQ5yqw59t+xfaBau61+9sYGIDm1DnW/d+SFkfEZ9X13V+y/ZeI+HvhsQFoSJ2rwIakz6ovp1U3JmgAJpFa++i2e2wPSBqRtDsimHsNmERqFT0ivoqIfkmzJS2wff3Zz7G92vZe23ubHiSA72ZMn7pHxDFJL0haeo7vbYyI+RExv6GxAWhInU/dL7PdW92/QNLNkg6VHhiA5tT51P0KSY/Y7lHnjeGJiHim7LAANKnOp+7/lDS3hbEAKIQj44AEKDqQAEUHEqDoQAIUHUiAogMJUHQgAYoOJODOWagNL9Ru9TTW3t7eNuO0du3aVvNWrFjRal5fX9+Uzjt27FireW2LCJ/9GGt0IAGKDiRA0YEEKDqQAEUHEqDoQAIUHUiAogMJUHQgAYoOJFC76NUkDq/a5sKQwCQzljX6GkmDpQYCoJy6UzLNlnS7pE1lhwOghLpr9A2S7pH0dcGxACikzkwtd0gaiYh9ozyPudeALlVnjb5Q0jLbQ5Iel7TY9qNnP4m514DuNWrRI+K+iJgdEX2SVkp6PiJ+VnxkABrD39GBBOpMsviNiHhBnWmTAUwirNGBBCg6kABFBxKg6EACFB1IgKIDCVB0IAGKDiQwpgNmutX69etbzVuzZk2reW1re663qT4XWjdgjQ4kQNGBBCg6kABFBxKg6EACFB1IgKIDCVB0IAGKDiRA0YEEah0CW13q+VNJX0n6kks6A5PLWI51/3FEfFRsJACKYdMdSKBu0UPSc7b32V5dckAAmld3031hRBy1/X1Ju20fiogXz3xC9QbAmwDQhWqt0SPiaPXviKQdkhac4znMvQZ0qTqzqV5ke+bp+5JulXSw9MAANKfOpvvlknbYPv38xyLi2aKjAtCoUYseEUck3dDCWAAUwp/XgAQoOpAARQcSoOhAAhQdSICiAwlQdCABig4k4IhofqF28wv9Fv39/W3GacuWLa3m3XDD1D5eadeuXa3mbd68udW8tl9fRPjsx1ijAwlQdCABig4kQNGBBCg6kABFBxKg6EACFB1IgKIDCVB0IIFaRbfda3u77UO2B23fWHpgAJpTdwKHP0h6NiJ+anu6pAsLjglAw0Ytuu2LJd0k6eeSFBGnJJ0qOywATaqz6X6NpA8lbbb9qu1N1UQO/8X2att7be9tfJQAvpM6RT9P0jxJD0bEXEmfS7r37CcxJRPQveoUfVjScES8XH29XZ3iA5gkRi16RLwv6V3bc6qHlkh6o+ioADSq7qfud0vaWn3ifkTSXeWGBKBptYoeEQOS2PcGJimOjAMSoOhAAhQdSICiAwlQdCABig4kQNGBBCg6kEDdI+O62sDAQKt5bc/11nbe+vXrW81bvnx5q3lDQ0Ot5rU999q5sEYHEqDoQAIUHUiAogMJUHQgAYoOJEDRgQQoOpAARQcSGLXotufYHjjjdsL22jYGB6AZox4CGxFvSuqXJNs9kv4laUfhcQFo0Fg33ZdIejsi3ikxGABljLXoKyVtKzEQAOXULnp1Tfdlkp78P99n7jWgS43lNNXbJO2PiA/O9c2I2ChpoyTZjgbGBqAhY9l0XyU224FJqVbRbV8o6RZJT5cdDoAS6k7JdFLSrMJjAVAIR8YBCVB0IAGKDiRA0YEEKDqQAEUHEqDoQAIUHUiAogMJOKL5809sfyhpPOesXyrpo4aH0w1Z5JHXVt7VEXHZ2Q8WKfp42d4bEfOnWhZ55E10HpvuQAIUHUig24q+cYpmkUfehOZ11T46gDK6bY0OoACKDiRA0YEEKDqQAEUHEvgPgDyLPntCRcAAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 288x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "visu(2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<Figure size 432x288 with 0 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPoAAAECCAYAAADXWsr9AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAALx0lEQVR4nO3d3Ytd9RXG8efpmOBb4ki1IkacKiUgQidBQiUg00QlVkm96EUEhUhLetGKoQXR3hT/AUkuihCiVjBGNJpQpLUGfENotUmcqdHEoHHENGoUjVELDerqxdkpaZx29sT9+82ZWd8PHHLOzJm91mR4zn45++zliBCA2e1b090AgPIIOpAAQQcSIOhAAgQdSICgAwn0RdBtr7D9uu03bN9RuNZ9tg/Z3l2yznH1LrT9jO09tl+1fVvheqfafsn2WFPvrpL1mpoDtl+2/UTpWk29cduv2B61vaNwrUHbW2zvbf6GVxSstbD5nY7djthe28nCI2Jab5IGJL0p6WJJcyWNSbq0YL0rJS2WtLvS73e+pMXN/XmS9hX+/SzpzOb+HEkvSvpB4d/xV5IekvREpf/TcUnnVKr1gKSfNffnShqsVHdA0nuSLupief2wRl8i6Y2I2B8RRyU9LOnHpYpFxPOSPiq1/AnqvRsRu5r7n0raI+mCgvUiIj5rHs5pbsXOirK9QNJ1kjaWqjFdbM9Xb8VwryRFxNGIOFyp/HJJb0bE210srB+CfoGkd457fEAFgzCdbA9JWqTeWrZknQHbo5IOSdoeESXrrZN0u6SvCtY4UUh6yvZO22sK1rlY0geS7m92TTbaPqNgveOtkrS5q4X1Q9A9wddm3Xm5ts+U9JiktRFxpGStiPgyIoYlLZC0xPZlJerYvl7SoYjYWWL5/8fSiFgs6VpJv7B9ZaE6p6i3m3dPRCyS9LmkoseQJMn2XEkrJT3a1TL7IegHJF143OMFkg5OUy9F2J6jXsg3RcTjteo2m5nPSlpRqMRSSSttj6u3y7XM9oOFav1HRBxs/j0kaat6u38lHJB04Lgtoi3qBb+0ayXtioj3u1pgPwT9b5K+Z/u7zSvZKkl/mOaeOmPb6u3j7YmIuyvUO9f2YHP/NElXSdpbolZE3BkRCyJiSL2/29MRcVOJWsfYPsP2vGP3JV0jqcg7KBHxnqR3bC9svrRc0mslap3gRnW42S71Nk2mVUR8YfuXkv6s3pHG+yLi1VL1bG+WNCLpHNsHJP02Iu4tVU+9td7Nkl5p9psl6TcR8cdC9c6X9IDtAfVeyB+JiCpve1VynqStvddPnSLpoYh4smC9WyVtalZC+yXdUrCWbJ8u6WpJP+90uc2hfACzWD9sugMojKADCRB0IAGCDiRA0IEE+irohU9nnLZa1KPedNfrq6BLqvmfWfUPRz3qTWe9fgs6gAKKnDBje1afhXPJJZdM+WeOHDmi+fPnn1S9gYGBKf/MJ598orPOOuuk6u3bt++kfg79ISK+9kExgn4Stm3bVrXe4OBg1XojIyNV66FbEwWdTXcgAYIOJEDQgQQIOpAAQQcSIOhAAgQdSICgAwm0CnrNkUkAujdp0JuLDP5OvUvQXirpRtuXlm4MQHfarNGrjkwC0L02QU8zMgmYrdpc173VyKTmg/K1P7MLoIU2QW81MikiNkjaIM3+T68BM02bTfdZPTIJyGDSNXrtkUkAutdq9lozJ6zUrDAAhXFmHJAAQQcSIOhAAgQdSICgAwkQdCABgg4kQNCBBGbFpJahoaGa5fTWW29VrTfbjY2NVa03PDxctV5tTGoBkiLoQAIEHUiAoAMJEHQgAYIOJEDQgQQIOpAAQQcSIOhAAm1GMt1n+5Dt3TUaAtC9Nmv030taUbgPAAVNGvSIeF7SRxV6AVAI++hAAq2u694Gs9eA/tVZ0Jm9BvQvNt2BBNq8vbZZ0l8kLbR9wPZPy7cFoEtthizeWKMRAOWw6Q4kQNCBBAg6kABBBxIg6EACBB1IgKADCRB0IIHOznWfToODg9PdQlHPPfdc1Xrj4+NV642MjFStlxFrdCABgg4kQNCBBAg6kABBBxIg6EACBB1IgKADCRB0IAGCDiTQ5uKQF9p+xvYe26/avq1GYwC60+Zc9y8k/ToidtmeJ2mn7e0R8Vrh3gB0pM3stXcjYldz/1NJeyRdULoxAN2Z0j667SFJiyS9WKIZAGW0/piq7TMlPSZpbUQcmeD7zF4D+lSroNueo17IN0XE4xM9h9lrQP9qc9Tdku6VtCci7i7fEoCutdlHXyrpZknLbI82tx8V7gtAh9rMXntBkiv0AqAQzowDEiDoQAIEHUiAoAMJEHQgAYIOJEDQgQQIOpCAI7o/Lb32ue61Z699/PHHVeudffbZVett27atar3h4eGq9Wb7rL6I+NoJbqzRgQQIOpAAQQcSIOhAAgQdSICgAwkQdCABgg4kQNCBBAg6kECbq8Ceavsl22PN7LW7ajQGoDttruv+L0nLIuKz5vruL9j+U0T8tXBvADrS5iqwIemz5uGc5saABmAGabWPbnvA9qikQ5K2RwSz14AZpFXQI+LLiBiWtEDSEtuXnfgc22ts77C9o+smAXwzUzrqHhGHJT0racUE39sQEZdHxOUd9QagI22Oup9re7C5f5qkqyTtLd0YgO60Oep+vqQHbA+o98LwSEQ8UbYtAF1qc9T975IWVegFQCGcGQckQNCBBAg6kABBBxIg6EACBB1IgKADCRB0IIE2Z8b1vcOHD1etNzY2VrVe7Vlv69evr1qv9uy1oaGhqvXGx8er1psIa3QgAYIOJEDQgQQIOpAAQQcSIOhAAgQdSICgAwkQdCABgg4k0DrozRCHl21zYUhghpnKGv02SXtKNQKgnLYjmRZIuk7SxrLtACih7Rp9naTbJX1VsBcAhbSZ1HK9pEMRsXOS5zF7DehTbdboSyWttD0u6WFJy2w/eOKTmL0G9K9Jgx4Rd0bEgogYkrRK0tMRcVPxzgB0hvfRgQSmdCmpiHhWvbHJAGYQ1uhAAgQdSICgAwkQdCABgg4kQNCBBAg6kABBBxJwRHS/ULv7hSZWezbZ6Oho1Xrr1q2rWq/27LUbbrihar2I8IlfY40OJEDQgQQIOpAAQQcSIOhAAgQdSICgAwkQdCABgg4kQNCBBFpdM6651POnkr6U9AWXdAZmlqlcHPKHEfFhsU4AFMOmO5BA26CHpKds77S9pmRDALrXdtN9aUQctP0dSdtt742I549/QvMCwIsA0IdardEj4mDz7yFJWyUtmeA5zF4D+lSbaapn2J537L6kayTtLt0YgO602XQ/T9JW28ee/1BEPFm0KwCdmjToEbFf0vcr9AKgEN5eAxIg6EACBB1IgKADCRB0IAGCDiRA0IEECDqQwFQ+j45pMttnoa1evbpqvdqz0PoBa3QgAYIOJEDQgQQIOpAAQQcSIOhAAgQdSICgAwkQdCABgg4k0Crotgdtb7G91/Ye21eUbgxAd9qe675e0pMR8RPbcyWdXrAnAB2bNOi250u6UtJqSYqIo5KOlm0LQJfabLpfLOkDSffbftn2xmaQw3+xvcb2Dts7Ou8SwDfSJuinSFos6Z6IWCTpc0l3nPgkRjIB/atN0A9IOhARLzaPt6gXfAAzxKRBj4j3JL1je2HzpeWSXivaFYBOtT3qfqukTc0R9/2SbinXEoCutQp6RIxKYt8bmKE4Mw5IgKADCRB0IAGCDiRA0IEECDqQAEEHEiDoQALMXjsJtWeTDQ8PV603ODhYtd7IyEjVerVn2fUD1uhAAgQdSICgAwkQdCABgg4kQNCBBAg6kABBBxIg6EACkwbd9kLbo8fdjtheW6M5AN2Y9BTYiHhd0rAk2R6Q9A9JWwv3BaBDU910Xy7pzYh4u0QzAMqYatBXSdpcohEA5bQOenNN95WSHv0f32f2GtCnpvIx1Wsl7YqI9yf6ZkRskLRBkmxHB70B6MhUNt1vFJvtwIzUKui2T5d0taTHy7YDoIS2I5n+KenbhXsBUAhnxgEJEHQgAYIOJEDQgQQIOpAAQQcSIOhAAgQdSICgAwk4ovvPn9j+QNLJfGb9HEkfdtxOP9SiHvVq1bsoIs498YtFgn6ybO+IiMtnWy3qUW+667HpDiRA0IEE+i3oG2ZpLepRb1rr9dU+OoAy+m2NDqAAgg4kQNCBBAg6kABBBxL4N14NjAzFFaBBAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 288x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "visu(9)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4. Configuration AutoML\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING - The AutoMLConfig parameters, X and y, will soon be deprecated. Please refer to our documentation for the latest interface: https://aka.ms/AutoMLConfig\n"
     ]
    }
   ],
   "source": [
    "automl_config = AutoMLConfig(task = 'classification',\n",
    "                             debug_log = 'automlclassif.log',\n",
    "                             primary_metric = 'AUC_weighted', #the metric I want to use\n",
    "                             iteration_timeout_minutes = 2, # Temps max par itération\n",
    "                             experiment_timeout_minutes = 15, # Temps max pour le process AutoML\n",
    "                             enable_early_stopping=True, # Sortie si pas d'amélioration de la métrique\n",
    "                             iterations = 10,           # Possibilité de définir le nombre d'itérations max\n",
    "                             n_cross_validations = 3,    # Nombre de validations croisées\n",
    "                             verbosity = logging.INFO,\n",
    "                             X = X_train, \n",
    "                             y = y_train)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 5. AutoML\n",
    "\n",
    "Call the `submit` method on the experiment object and pass the run configuration. Execution of local runs is synchronous. Depending on the data and the number of iterations this can run for a while.\n",
    "In this example, we specify `show_output = True` to print currently running iterations to the console.\n",
    "\n",
    "`AUC_weighted` is the **metric** used."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Mise à jour si besoin (redémarrer le kernel ensuite)\n",
    "#!pip install --upgrade azureml-sdk[automl]\n",
    "#!pip install --upgrade azureml-sdk[explain,contrib]\n",
    "#!pip install interpret-community==0.14.3"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Running on local machine\n",
      "Parent Run ID: AutoML_99c06691-0a63-42f3-b16b-1d38b6d0f8ae\n",
      "\n",
      "Current status: DatasetEvaluation. Gathering dataset statistics.\n",
      "Current status: FeaturesGeneration. Generating features for the dataset.\n",
      "Current status: DatasetFeaturization. Beginning to fit featurizers and featurize the dataset.\n",
      "Current status: DatasetFeaturizationCompleted. Completed fit featurizers and featurizing the dataset.\n",
      "Current status: DatasetCrossValidationSplit. Generating individually featurized CV splits.\n",
      "\n",
      "****************************************************************************************************\n",
      "DATA GUARDRAILS: \n",
      "\n",
      "TYPE:         Class balancing detection\n",
      "STATUS:       PASSED\n",
      "DESCRIPTION:  Your inputs were analyzed, and all classes are balanced in your training data.\n",
      "              Learn more about imbalanced data: https://aka.ms/AutomatedMLImbalancedData\n",
      "\n",
      "****************************************************************************************************\n",
      "\n",
      "TYPE:         Missing feature values imputation\n",
      "STATUS:       PASSED\n",
      "DESCRIPTION:  No feature missing values were detected in the training data.\n",
      "              Learn more about missing value imputation: https://aka.ms/AutomatedMLFeaturization\n",
      "\n",
      "****************************************************************************************************\n",
      "\n",
      "TYPE:         High cardinality feature detection\n",
      "STATUS:       PASSED\n",
      "DESCRIPTION:  Your inputs were analyzed, and no high cardinality features were detected.\n",
      "              Learn more about high cardinality feature handling: https://aka.ms/AutomatedMLFeaturization\n",
      "\n",
      "****************************************************************************************************\n",
      "Current status: ModelSelection. Beginning model selection.\n",
      "\n",
      "****************************************************************************************************\n",
      "ITERATION: The iteration being evaluated.\n",
      "PIPELINE: A summary description of the pipeline being evaluated.\n",
      "DURATION: Time taken for the current iteration.\n",
      "METRIC: The result of computing score on the fitted pipeline.\n",
      "BEST: The best observed score thus far.\n",
      "****************************************************************************************************\n",
      "\n",
      " ITERATION   PIPELINE                                       DURATION      METRIC      BEST\n",
      "         0   MaxAbsScaler LightGBM                          0:04:32       0.9992    0.9992\n",
      "         1   MaxAbsScaler XGBoostClassifier                 0:00:36       0.9992    0.9992\n",
      "         2   SparseNormalizer KNN                           0:00:24       0.9988    0.9992\n",
      "         3   MinMaxScaler RandomForest                      0:00:29       0.9865    0.9992\n",
      "         4   StandardScalerWrapper SGD                      0:00:24       0.9933    0.9992\n",
      "         5   MinMaxScaler SGD                               0:00:50       0.9958    0.9992\n",
      "         6   StandardScalerWrapper RandomForest             0:00:27       0.9942    0.9992\n",
      "         7   MaxAbsScaler SGD                               0:00:30       0.9930    0.9992\n",
      "         8   VotingEnsemble                                 0:00:34       0.9997    0.9997\n",
      "         9   StackEnsemble                                  0:00:58       0.9768    0.9997\n",
      "****************************************************************************************************\n",
      "Current status: BestRunExplainModel. Best run model explanations started\n",
      "Current status: ModelExplanationDataSetSetup. Model explanations data setup completed\n",
      "Current status: PickSurrogateModel. Choosing LightGBM as the surrogate model for explanations\n",
      "Current status: EngineeredFeatureExplanations. Computation of engineered features started\n",
      "Current status: EngineeredFeatureExplanations. Computation of engineered features completed\n",
      "Current status: BestRunExplainModel. Best run model explanations completed\n",
      "****************************************************************************************************\n"
     ]
    }
   ],
   "source": [
    "local_run = experiment.submit(automl_config, show_output = True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table style=\"width:100%\"><tr><th>Experiment</th><th>Id</th><th>Type</th><th>Status</th><th>Details Page</th><th>Docs Page</th></tr><tr><td>Exemple2-automlclassificationSR</td><td>AutoML_99c06691-0a63-42f3-b16b-1d38b6d0f8ae</td><td>automl</td><td>Completed</td><td><a href=\"https://ml.azure.com/experiments/Exemple2-automlclassificationSR/runs/AutoML_99c06691-0a63-42f3-b16b-1d38b6d0f8ae?wsid=/subscriptions/70b8f39e-8863-49f7-b6ba-34a80799550c/resourcegroups/workshopAML2020-rg/workspaces/workshopAML2020\" target=\"_blank\" rel=\"noopener\">Link to Azure Machine Learning studio</a></td><td><a href=\"https://docs.microsoft.com/en-us/python/api/overview/azure/ml/intro?view=azure-ml-py\" target=\"_blank\" rel=\"noopener\">Link to Documentation</a></td></tr></table>"
      ],
      "text/plain": [
       "Run(Experiment: Exemple2-automlclassificationSR,\n",
       "Id: AutoML_99c06691-0a63-42f3-b16b-1d38b6d0f8ae,\n",
       "Type: automl,\n",
       "Status: Completed)"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "local_run"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 6. Exploration des résultats AutoML"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 6.1 Notebook Widget\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "29f15bd6ca2545c091357f9ab59fdb63",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "_AutoMLWidget(widget_settings={'childWidgetDisplay': 'popup', 'send_telemetry': False, 'log_level': 'INFO', 's…"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/aml.mini.widget.v1": "{\"status\": \"Completed\", \"workbench_run_details_uri\": \"https://ml.azure.com/experiments/Exemple2-automlclassificationSR/runs/AutoML_99c06691-0a63-42f3-b16b-1d38b6d0f8ae?wsid=/subscriptions/70b8f39e-8863-49f7-b6ba-34a80799550c/resourcegroups/workshopAML2020-rg/workspaces/workshopAML2020\", \"run_id\": \"AutoML_99c06691-0a63-42f3-b16b-1d38b6d0f8ae\", \"run_properties\": {\"run_id\": \"AutoML_99c06691-0a63-42f3-b16b-1d38b6d0f8ae\", \"created_utc\": \"2020-09-16T12:45:34.095093Z\", \"properties\": {\"num_iterations\": \"10\", \"training_type\": \"TrainFull\", \"acquisition_function\": \"EI\", \"primary_metric\": \"AUC_weighted\", \"train_split\": \"0\", \"acquisition_parameter\": \"0\", \"num_cross_validation\": \"3\", \"target\": \"local\", \"AMLSettingsJsonString\": \"{\\\"path\\\":null,\\\"name\\\":\\\"Exemple2-automlclassificationSR\\\",\\\"subscription_id\\\":\\\"70b8f39e-8863-49f7-b6ba-34a80799550c\\\",\\\"resource_group\\\":\\\"workshopAML2020-rg\\\",\\\"workspace_name\\\":\\\"workshopAML2020\\\",\\\"region\\\":\\\"westeurope\\\",\\\"compute_target\\\":\\\"local\\\",\\\"spark_service\\\":null,\\\"azure_service\\\":\\\"Microsoft.AzureNotebookVM\\\",\\\"_local_managed_run_id\\\":null,\\\"many_models\\\":false,\\\"iterations\\\":10,\\\"primary_metric\\\":\\\"AUC_weighted\\\",\\\"task_type\\\":\\\"classification\\\",\\\"data_script\\\":null,\\\"validation_size\\\":0.0,\\\"n_cross_validations\\\":3,\\\"y_min\\\":null,\\\"y_max\\\":null,\\\"num_classes\\\":10,\\\"featurization\\\":\\\"auto\\\",\\\"_ignore_package_version_incompatibilities\\\":false,\\\"is_timeseries\\\":false,\\\"max_cores_per_iteration\\\":1,\\\"max_concurrent_iterations\\\":1,\\\"iteration_timeout_minutes\\\":2,\\\"mem_in_mb\\\":null,\\\"enforce_time_on_windows\\\":false,\\\"experiment_timeout_minutes\\\":15,\\\"experiment_exit_score\\\":null,\\\"whitelist_models\\\":null,\\\"blacklist_algos\\\":[\\\"TensorFlowLinearClassifier\\\"],\\\"supported_models\\\":[\\\"SVM\\\",\\\"GradientBoosting\\\",\\\"ExtremeRandomTrees\\\",\\\"MultinomialNaiveBayes\\\",\\\"LogisticRegression\\\",\\\"SGD\\\",\\\"RandomForest\\\",\\\"LightGBM\\\",\\\"XGBoostClassifier\\\",\\\"KNN\\\",\\\"AveragedPerceptronClassifier\\\",\\\"DecisionTree\\\",\\\"TensorFlowDNN\\\",\\\"LinearSVM\\\",\\\"TensorFlowLinearClassifier\\\",\\\"BernoulliNaiveBayes\\\"],\\\"auto_blacklist\\\":true,\\\"blacklist_samples_reached\\\":false,\\\"exclude_nan_labels\\\":true,\\\"verbosity\\\":20,\\\"_debug_log\\\":\\\"automlclassif.log\\\",\\\"show_warnings\\\":false,\\\"model_explainability\\\":true,\\\"service_url\\\":null,\\\"sdk_url\\\":null,\\\"sdk_packages\\\":null,\\\"enable_onnx_compatible_models\\\":false,\\\"enable_split_onnx_featurizer_estimator_models\\\":false,\\\"vm_type\\\":null,\\\"telemetry_verbosity\\\":20,\\\"send_telemetry\\\":true,\\\"enable_dnn\\\":false,\\\"force_text_dnn\\\":false,\\\"enable_feature_sweeping\\\":true,\\\"enable_early_stopping\\\":true,\\\"early_stopping_n_iters\\\":10,\\\"metrics\\\":null,\\\"enable_ensembling\\\":true,\\\"enable_stack_ensembling\\\":true,\\\"ensemble_iterations\\\":10,\\\"enable_tf\\\":false,\\\"enable_subsampling\\\":false,\\\"subsample_seed\\\":null,\\\"enable_nimbusml\\\":false,\\\"enable_streaming\\\":false,\\\"force_streaming\\\":false,\\\"track_child_runs\\\":true,\\\"label_column_name\\\":null,\\\"weight_column_name\\\":null,\\\"cv_split_column_names\\\":null,\\\"enable_local_managed\\\":false,\\\"cost_mode\\\":1,\\\"lag_length\\\":0,\\\"metric_operation\\\":\\\"maximize\\\",\\\"preprocess\\\":true,\\\"scenario\\\":\\\"SDK-1.13.0\\\"}\", \"DataPrepJsonString\": null, \"EnableSubsampling\": \"False\", \"runTemplate\": \"AutoML\", \"azureml.runsource\": \"automl\", \"display_task_type\": \"classification\", \"dependencies_versions\": \"{\\\"azureml-widgets\\\": \\\"1.13.0\\\", \\\"azureml-train\\\": \\\"1.13.0\\\", \\\"azureml-train-restclients-hyperdrive\\\": \\\"1.13.0\\\", \\\"azureml-train-core\\\": \\\"1.13.0\\\", \\\"azureml-train-automl\\\": \\\"1.13.0\\\", \\\"azureml-train-automl-runtime\\\": \\\"1.13.0\\\", \\\"azureml-train-automl-client\\\": \\\"1.13.0.post1\\\", \\\"azureml-tensorboard\\\": \\\"1.13.0\\\", \\\"azureml-telemetry\\\": \\\"1.13.0\\\", \\\"azureml-sdk\\\": \\\"1.13.0\\\", \\\"azureml-samples\\\": \\\"0+unknown\\\", \\\"azureml-pipeline\\\": \\\"1.13.0\\\", \\\"azureml-pipeline-steps\\\": \\\"1.13.0\\\", \\\"azureml-pipeline-core\\\": \\\"1.13.0\\\", \\\"azureml-opendatasets\\\": \\\"1.9.0\\\", \\\"azureml-monitoring\\\": \\\"0.1.0a18\\\", \\\"azureml-model-management-sdk\\\": \\\"1.0.1b6.post1\\\", \\\"azureml-mlflow\\\": \\\"1.9.0\\\", \\\"azureml-interpret\\\": \\\"1.13.0\\\", \\\"azureml-explain-model\\\": \\\"1.13.0\\\", \\\"azureml-defaults\\\": \\\"1.13.0\\\", \\\"azureml-dataset-runtime\\\": \\\"1.13.0\\\", \\\"azureml-dataprep\\\": \\\"2.0.7\\\", \\\"azureml-dataprep-native\\\": \\\"20.0.2\\\", \\\"azureml-datadrift\\\": \\\"1.9.0\\\", \\\"azureml-core\\\": \\\"1.13.0\\\", \\\"azureml-contrib-services\\\": \\\"1.13.0\\\", \\\"azureml-contrib-server\\\": \\\"1.13.0\\\", \\\"azureml-contrib-reinforcementlearning\\\": \\\"1.9.0\\\", \\\"azureml-contrib-pipeline-steps\\\": \\\"1.13.0\\\", \\\"azureml-contrib-notebook\\\": \\\"1.13.0\\\", \\\"azureml-contrib-interpret\\\": \\\"1.9.0\\\", \\\"azureml-contrib-gbdt\\\": \\\"1.9.0\\\", \\\"azureml-contrib-fairness\\\": \\\"1.9.0\\\", \\\"azureml-contrib-dataset\\\": \\\"1.9.0\\\", \\\"azureml-cli-common\\\": \\\"1.9.0\\\", \\\"azureml-automl-runtime\\\": \\\"1.13.0\\\", \\\"azureml-automl-core\\\": \\\"1.13.0\\\", \\\"azureml-accel-models\\\": \\\"1.13.0\\\"}\", \"ClientSdkVersion\": \"1.13.0\", \"ClientType\": \"SDK\", \"_aml_system_scenario_identification\": \"Local.Parent\", \"environment_cpu_name\": \"AzureML-AutoML\", \"environment_cpu_version\": \"35\", \"environment_gpu_name\": \"AzureML-AutoML-GPU\", \"environment_gpu_version\": \"23\", \"root_attribution\": \"automl\", \"attribution\": \"AutoML\", \"Orchestrator\": \"AutoML\", \"_azureml.ComputeTargetType\": \"local\", \"ProblemInfoJsonString\": \"{\\\"dataset_num_categorical\\\": 0, \\\"is_sparse\\\": false, \\\"subsampling\\\": false, \\\"dataset_classes\\\": 10, \\\"dataset_features\\\": 61, \\\"dataset_samples\\\": 1697, \\\"single_frequency_class_detected\\\": false}\", \"feature_skus\": \"automatedml_sdk_guardrails\", \"azureml.git.repository_uri\": \"https://github.com/retkowsky/WorkshopAML2020\", \"mlflow.source.git.repoURL\": \"https://github.com/retkowsky/WorkshopAML2020\", \"azureml.git.branch\": \"master\", \"mlflow.source.git.branch\": \"master\", \"azureml.git.commit\": \"eb05ad565a41b5121d26c6fda4b1c6398a9243d7\", \"mlflow.source.git.commit\": \"eb05ad565a41b5121d26c6fda4b1c6398a9243d7\", \"azureml.git.dirty\": \"True\"}, \"tags\": {\"model_explain_run\": \"best_run\", \"model_explain_best_run_child_id\": \"AutoML_99c06691-0a63-42f3-b16b-1d38b6d0f8ae_8\", \"best_score\": \"0.9997226726657312\", \"best_pipeline\": \"VotingEnsemble\", \"automl_best_child_run_id\": \"AutoML_99c06691-0a63-42f3-b16b-1d38b6d0f8ae_8\"}, \"end_time_utc\": \"2020-09-16T12:56:53.007003Z\", \"status\": \"Completed\", \"log_files\": {}, \"log_groups\": [], \"run_duration\": \"0:11:18\"}, \"child_runs\": [{\"run_id\": \"AutoML_99c06691-0a63-42f3-b16b-1d38b6d0f8ae_0\", \"run_number\": 194, \"metric\": null, \"status\": \"Completed\", \"run_type\": null, \"training_percent\": \"100\", \"start_time\": \"2020-09-16T12:45:54.241715Z\", \"end_time\": \"2020-09-16T12:50:24.485336Z\", \"created_time\": \"2020-09-16T12:45:54.041319Z\", \"created_time_dt\": \"2020-09-16T12:45:54.041319Z\", \"duration\": \"0:04:30\", \"iteration\": \"0\", \"goal\": \"AUC_weighted_max\", \"run_name\": \"MaxAbsScaler, LightGBM\", \"run_properties\": \"copy=True\", \"primary_metric\": 0.99919404, \"best_metric\": 0.99919404}, {\"run_id\": \"AutoML_99c06691-0a63-42f3-b16b-1d38b6d0f8ae_1\", \"run_number\": 195, \"metric\": null, \"status\": \"Completed\", \"run_type\": null, \"training_percent\": \"100\", \"start_time\": \"2020-09-16T12:50:25.276987Z\", \"end_time\": \"2020-09-16T12:51:00.759479Z\", \"created_time\": \"2020-09-16T12:50:25.078336Z\", \"created_time_dt\": \"2020-09-16T12:50:25.078336Z\", \"duration\": \"0:00:35\", \"iteration\": \"1\", \"goal\": \"AUC_weighted_max\", \"run_name\": \"MaxAbsScaler, XGBoostClassifier\", \"run_properties\": \"copy=True\", \"primary_metric\": 0.99915093, \"best_metric\": 0.99919404}, {\"run_id\": \"AutoML_99c06691-0a63-42f3-b16b-1d38b6d0f8ae_2\", \"run_number\": 196, \"metric\": null, \"status\": \"Completed\", \"run_type\": null, \"training_percent\": \"100\", \"start_time\": \"2020-09-16T12:51:01.937092Z\", \"end_time\": \"2020-09-16T12:51:25.0882Z\", \"created_time\": \"2020-09-16T12:51:01.73358Z\", \"created_time_dt\": \"2020-09-16T12:51:01.73358Z\", \"duration\": \"0:00:23\", \"iteration\": \"2\", \"goal\": \"AUC_weighted_max\", \"run_name\": \"SparseNormalizer, KNN\", \"run_properties\": \"<azureml.automl.runtime.shared.model_wrappers.SparseNormalizer object at 0x7f1d3436c470\", \"primary_metric\": 0.99878973, \"best_metric\": 0.99919404}, {\"run_id\": \"AutoML_99c06691-0a63-42f3-b16b-1d38b6d0f8ae_3\", \"run_number\": 197, \"metric\": null, \"status\": \"Completed\", \"run_type\": null, \"training_percent\": \"100\", \"start_time\": \"2020-09-16T12:51:26.947223Z\", \"end_time\": \"2020-09-16T12:51:54.470534Z\", \"created_time\": \"2020-09-16T12:51:26.739482Z\", \"created_time_dt\": \"2020-09-16T12:51:26.739482Z\", \"duration\": \"0:00:27\", \"iteration\": \"3\", \"goal\": \"AUC_weighted_max\", \"run_name\": \"MinMaxScaler, RandomForest\", \"run_properties\": \"copy=True, feature_range=(0, 1\", \"primary_metric\": 0.98651345, \"best_metric\": 0.99919404}, {\"run_id\": \"AutoML_99c06691-0a63-42f3-b16b-1d38b6d0f8ae_4\", \"run_number\": 198, \"metric\": null, \"status\": \"Completed\", \"run_type\": null, \"training_percent\": \"100\", \"start_time\": \"2020-09-16T12:51:56.425475Z\", \"end_time\": \"2020-09-16T12:52:18.688347Z\", \"created_time\": \"2020-09-16T12:51:56.146971Z\", \"created_time_dt\": \"2020-09-16T12:51:56.146971Z\", \"duration\": \"0:00:22\", \"iteration\": \"4\", \"goal\": \"AUC_weighted_max\", \"run_name\": \"StandardScalerWrapper, SGD\", \"run_properties\": \"<azureml.automl.runtime.shared.model_wrappers.StandardScalerWrapper object at 0x7f1d34eb9390\", \"primary_metric\": 0.99329723, \"best_metric\": 0.99919404}, {\"run_id\": \"AutoML_99c06691-0a63-42f3-b16b-1d38b6d0f8ae_5\", \"run_number\": 199, \"metric\": null, \"status\": \"Completed\", \"run_type\": null, \"training_percent\": \"100\", \"start_time\": \"2020-09-16T12:52:20.391484Z\", \"end_time\": \"2020-09-16T12:53:08.898689Z\", \"created_time\": \"2020-09-16T12:52:20.210669Z\", \"created_time_dt\": \"2020-09-16T12:52:20.210669Z\", \"duration\": \"0:00:48\", \"iteration\": \"5\", \"goal\": \"AUC_weighted_max\", \"run_name\": \"MinMaxScaler, SGD\", \"run_properties\": \"copy=True, feature_range=(0, 1\", \"primary_metric\": 0.99578997, \"best_metric\": 0.99919404}, {\"run_id\": \"AutoML_99c06691-0a63-42f3-b16b-1d38b6d0f8ae_6\", \"run_number\": 200, \"metric\": null, \"status\": \"Completed\", \"run_type\": null, \"training_percent\": \"100\", \"start_time\": \"2020-09-16T12:53:11.290693Z\", \"end_time\": \"2020-09-16T12:53:36.485185Z\", \"created_time\": \"2020-09-16T12:53:10.996967Z\", \"created_time_dt\": \"2020-09-16T12:53:10.996967Z\", \"duration\": \"0:00:25\", \"iteration\": \"6\", \"goal\": \"AUC_weighted_max\", \"run_name\": \"StandardScalerWrapper, RandomForest\", \"run_properties\": \"<azureml.automl.runtime.shared.model_wrappers.StandardScalerWrapper object at 0x7f1d342c9320\", \"primary_metric\": 0.99422134, \"best_metric\": 0.99919404}, {\"run_id\": \"AutoML_99c06691-0a63-42f3-b16b-1d38b6d0f8ae_7\", \"run_number\": 201, \"metric\": null, \"status\": \"Completed\", \"run_type\": null, \"training_percent\": \"100\", \"start_time\": \"2020-09-16T12:53:39.696496Z\", \"end_time\": \"2020-09-16T12:54:06.585891Z\", \"created_time\": \"2020-09-16T12:53:39.152199Z\", \"created_time_dt\": \"2020-09-16T12:53:39.152199Z\", \"duration\": \"0:00:27\", \"iteration\": \"7\", \"goal\": \"AUC_weighted_max\", \"run_name\": \"MaxAbsScaler, SGD\", \"run_properties\": \"copy=True\", \"primary_metric\": 0.99297009, \"best_metric\": 0.99919404}, {\"run_id\": \"AutoML_99c06691-0a63-42f3-b16b-1d38b6d0f8ae_8\", \"run_number\": 202, \"metric\": null, \"status\": \"Completed\", \"run_type\": null, \"training_percent\": \"100\", \"start_time\": \"2020-09-16T12:54:08.518313Z\", \"end_time\": \"2020-09-16T12:54:41.886614Z\", \"created_time\": \"2020-09-16T12:54:07.967725Z\", \"created_time_dt\": \"2020-09-16T12:54:07.967725Z\", \"duration\": \"0:00:33\", \"iteration\": \"8\", \"goal\": \"AUC_weighted_max\", \"run_name\": \"VotingEnsemble\", \"run_properties\": \"classification_labels=None,\\n               estimators=[('0', Pipeline(memory=None,\\n     steps=[('maxabsscaler', MaxAbsScaler(copy=True\", \"primary_metric\": 0.99972267, \"best_metric\": 0.99972267}, {\"run_id\": \"AutoML_99c06691-0a63-42f3-b16b-1d38b6d0f8ae_9\", \"run_number\": 203, \"metric\": null, \"status\": \"Completed\", \"run_type\": null, \"training_percent\": \"100\", \"start_time\": \"2020-09-16T12:54:42.811075Z\", \"end_time\": \"2020-09-16T12:55:40.734369Z\", \"created_time\": \"2020-09-16T12:54:42.605539Z\", \"created_time_dt\": \"2020-09-16T12:54:42.605539Z\", \"duration\": \"0:00:58\", \"iteration\": \"9\", \"goal\": \"AUC_weighted_max\", \"run_name\": \"StackEnsemble\", \"run_properties\": \"base_learners=[('0', Pipeline(memory=None,\\n     steps=[('maxabsscaler', MaxAbsScaler(copy=True\", \"primary_metric\": 0.97679175, \"best_metric\": 0.99972267}], \"children_metrics\": {\"categories\": [0], \"series\": {\"matthews_correlation\": [{\"categories\": [\"0\", \"1\", \"2\", \"3\", \"4\", \"5\", \"6\", \"7\", \"8\", \"9\"], \"mode\": \"markers\", \"name\": \"matthews_correlation\", \"stepped\": false, \"type\": \"scatter\", \"data\": [0.9588531516939129, 0.9562257668672043, 0.9745300387895229, 0.8473841494092905, 0.926867553643623, 0.935253779924242, 0.9090925828906905, 0.9574599955679807, 0.9777813954801045, 0.8118440983176808]}, {\"categories\": [\"0\", \"1\", \"2\", \"3\", \"4\", \"5\", \"6\", \"7\", \"8\", \"9\"], \"mode\": \"lines\", \"name\": \"matthews_correlation_max\", \"stepped\": true, \"type\": \"scatter\", \"data\": [0.9588531516939129, 0.9588531516939129, 0.9745300387895229, 0.9745300387895229, 0.9745300387895229, 0.9745300387895229, 0.9745300387895229, 0.9745300387895229, 0.9777813954801045, 0.9777813954801045]}], \"balanced_accuracy\": [{\"categories\": [\"0\", \"1\", \"2\", \"3\", \"4\", \"5\", \"6\", \"7\", \"8\", \"9\"], \"mode\": \"markers\", \"name\": \"balanced_accuracy\", \"stepped\": false, \"type\": \"scatter\", \"data\": [0.9633000881733736, 0.9609461219451969, 0.9772049533000661, 0.8617819618884656, 0.9350145867832289, 0.9416610453441616, 0.9178113136223093, 0.9624733533134826, 0.9801048297732556, 0.8186541842941667]}, {\"categories\": [\"0\", \"1\", \"2\", \"3\", \"4\", \"5\", \"6\", \"7\", \"8\", \"9\"], \"mode\": \"lines\", \"name\": \"balanced_accuracy_max\", \"stepped\": true, \"type\": \"scatter\", \"data\": [0.9633000881733736, 0.9633000881733736, 0.9772049533000661, 0.9772049533000661, 0.9772049533000661, 0.9772049533000661, 0.9772049533000661, 0.9772049533000661, 0.9801048297732556, 0.9801048297732556]}], \"accuracy\": [{\"categories\": [\"0\", \"1\", \"2\", \"3\", \"4\", \"5\", \"6\", \"7\", \"8\", \"9\"], \"mode\": \"markers\", \"name\": \"accuracy\", \"stepped\": false, \"type\": \"scatter\", \"data\": [0.9628839759425456, 0.9605251362873553, 0.9770161668595015, 0.8621147211190677, 0.9340014801380073, 0.941670054306472, 0.9180952083137893, 0.9617040349395957, 0.9799660193668762, 0.8026402743467068]}, {\"categories\": [\"0\", \"1\", \"2\", \"3\", \"4\", \"5\", \"6\", \"7\", \"8\", \"9\"], \"mode\": \"lines\", \"name\": \"accuracy_max\", \"stepped\": true, \"type\": \"scatter\", \"data\": [0.9628839759425456, 0.9628839759425456, 0.9770161668595015, 0.9770161668595015, 0.9770161668595015, 0.9770161668595015, 0.9770161668595015, 0.9770161668595015, 0.9799660193668762, 0.9799660193668762]}], \"AUC_macro\": [{\"categories\": [\"0\", \"1\", \"2\", \"3\", \"4\", \"5\", \"6\", \"7\", \"8\", \"9\"], \"mode\": \"markers\", \"name\": \"AUC_macro\", \"stepped\": false, \"type\": \"scatter\", \"data\": [0.9992289225554636, 0.9991689809164891, 0.998763778791338, 0.9866364857541425, 0.993280087548554, 0.9958835391080884, 0.9940912570496043, 0.9929989769763603, 0.9997278321619597, 0.9763349195611849]}, {\"categories\": [\"0\", \"1\", \"2\", \"3\", \"4\", \"5\", \"6\", \"7\", \"8\", \"9\"], \"mode\": \"lines\", \"name\": \"AUC_macro_max\", \"stepped\": true, \"type\": \"scatter\", \"data\": [0.9992289225554636, 0.9992289225554636, 0.9992289225554636, 0.9992289225554636, 0.9992289225554636, 0.9992289225554636, 0.9992289225554636, 0.9992289225554636, 0.9997278321619597, 0.9997278321619597]}], \"f1_score_weighted\": [{\"categories\": [\"0\", \"1\", \"2\", \"3\", \"4\", \"5\", \"6\", \"7\", \"8\", \"9\"], \"mode\": \"markers\", \"name\": \"f1_score_weighted\", \"stepped\": false, \"type\": \"scatter\", \"data\": [0.9629897652902043, 0.9607122386657801, 0.9768989879460303, 0.8607540452714417, 0.9334363069898197, 0.9416824423456075, 0.9176320521400024, 0.9616662692617162, 0.9800071252416492, 0.7686542927045544]}, {\"categories\": [\"0\", \"1\", \"2\", \"3\", \"4\", \"5\", \"6\", \"7\", \"8\", \"9\"], \"mode\": \"lines\", \"name\": \"f1_score_weighted_max\", \"stepped\": true, \"type\": \"scatter\", \"data\": [0.9629897652902043, 0.9629897652902043, 0.9768989879460303, 0.9768989879460303, 0.9768989879460303, 0.9768989879460303, 0.9768989879460303, 0.9768989879460303, 0.9800071252416492, 0.9800071252416492]}], \"precision_score_weighted\": [{\"categories\": [\"0\", \"1\", \"2\", \"3\", \"4\", \"5\", \"6\", \"7\", \"8\", \"9\"], \"mode\": \"markers\", \"name\": \"precision_score_weighted\", \"stepped\": false, \"type\": \"scatter\", \"data\": [0.9643887974423323, 0.9621190907630258, 0.9776344838295447, 0.8665459402710912, 0.9353116663897838, 0.9430718918515567, 0.9190780630710956, 0.9622390608313863, 0.9806729693762462, 0.7570782516080291]}, {\"categories\": [\"0\", \"1\", \"2\", \"3\", \"4\", \"5\", \"6\", \"7\", \"8\", \"9\"], \"mode\": \"lines\", \"name\": \"precision_score_weighted_max\", \"stepped\": true, \"type\": \"scatter\", \"data\": [0.9643887974423323, 0.9643887974423323, 0.9776344838295447, 0.9776344838295447, 0.9776344838295447, 0.9776344838295447, 0.9776344838295447, 0.9776344838295447, 0.9806729693762462, 0.9806729693762462]}], \"AUC_micro\": [{\"categories\": [\"0\", \"1\", \"2\", \"3\", \"4\", \"5\", \"6\", \"7\", \"8\", \"9\"], \"mode\": \"markers\", \"name\": \"AUC_micro\", \"stepped\": false, \"type\": \"scatter\", \"data\": [0.9991625925212827, 0.9991268520936541, 0.9988294590582659, 0.9873327858016743, 0.9943157697536874, 0.9966135012878969, 0.9948399900346722, 0.9934292773498649, 0.9997502627629037, 0.9787492220432769]}, {\"categories\": [\"0\", \"1\", \"2\", \"3\", \"4\", \"5\", \"6\", \"7\", \"8\", \"9\"], \"mode\": \"lines\", \"name\": \"AUC_micro_max\", \"stepped\": true, \"type\": \"scatter\", \"data\": [0.9991625925212827, 0.9991625925212827, 0.9991625925212827, 0.9991625925212827, 0.9991625925212827, 0.9991625925212827, 0.9991625925212827, 0.9991625925212827, 0.9997502627629037, 0.9997502627629037]}], \"weighted_accuracy\": [{\"categories\": [\"0\", \"1\", \"2\", \"3\", \"4\", \"5\", \"6\", \"7\", \"8\", \"9\"], \"mode\": \"markers\", \"name\": \"weighted_accuracy\", \"stepped\": false, \"type\": \"scatter\", \"data\": [0.9624157429451962, 0.9600002216335907, 0.9768094794340296, 0.8623416009154926, 0.9329467146348507, 0.9416092557463424, 0.918162381384399, 0.9608794608555534, 0.9797723436975737, 0.7861733453101779]}, {\"categories\": [\"0\", \"1\", \"2\", \"3\", \"4\", \"5\", \"6\", \"7\", \"8\", \"9\"], \"mode\": \"lines\", \"name\": \"weighted_accuracy_max\", \"stepped\": true, \"type\": \"scatter\", \"data\": [0.9624157429451962, 0.9624157429451962, 0.9768094794340296, 0.9768094794340296, 0.9768094794340296, 0.9768094794340296, 0.9768094794340296, 0.9768094794340296, 0.9797723436975737, 0.9797723436975737]}], \"precision_score_micro\": [{\"categories\": [\"0\", \"1\", \"2\", \"3\", \"4\", \"5\", \"6\", \"7\", \"8\", \"9\"], \"mode\": \"markers\", \"name\": \"precision_score_micro\", \"stepped\": false, \"type\": \"scatter\", \"data\": [0.9628839759425456, 0.9605251362873553, 0.9770161668595015, 0.8621147211190677, 0.9340014801380073, 0.941670054306472, 0.9180952083137893, 0.9617040349395957, 0.9799660193668762, 0.8026402743467068]}, {\"categories\": [\"0\", \"1\", \"2\", \"3\", \"4\", \"5\", \"6\", \"7\", \"8\", \"9\"], \"mode\": \"lines\", \"name\": \"precision_score_micro_max\", \"stepped\": true, \"type\": \"scatter\", \"data\": [0.9628839759425456, 0.9628839759425456, 0.9770161668595015, 0.9770161668595015, 0.9770161668595015, 0.9770161668595015, 0.9770161668595015, 0.9770161668595015, 0.9799660193668762, 0.9799660193668762]}], \"f1_score_macro\": [{\"categories\": [\"0\", \"1\", \"2\", \"3\", \"4\", \"5\", \"6\", \"7\", \"8\", \"9\"], \"mode\": \"markers\", \"name\": \"f1_score_macro\", \"stepped\": false, \"type\": \"scatter\", \"data\": [0.9630586267844569, 0.9605393679245454, 0.977133423257011, 0.8590572506587087, 0.9337190208187781, 0.9415087054506376, 0.9172894226661366, 0.9620664608135705, 0.9802344671856597, 0.7810781101083885]}, {\"categories\": [\"0\", \"1\", \"2\", \"3\", \"4\", \"5\", \"6\", \"7\", \"8\", \"9\"], \"mode\": \"lines\", \"name\": \"f1_score_macro_max\", \"stepped\": true, \"type\": \"scatter\", \"data\": [0.9630586267844569, 0.9630586267844569, 0.977133423257011, 0.977133423257011, 0.977133423257011, 0.977133423257011, 0.977133423257011, 0.977133423257011, 0.9802344671856597, 0.9802344671856597]}], \"norm_macro_recall\": [{\"categories\": [\"0\", \"1\", \"2\", \"3\", \"4\", \"5\", \"6\", \"7\", \"8\", \"9\"], \"mode\": \"markers\", \"name\": \"norm_macro_recall\", \"stepped\": false, \"type\": \"scatter\", \"data\": [0.9592223201926373, 0.95660680216133, 0.9746721703334068, 0.8464244020982951, 0.9277939853146989, 0.9351789392712907, 0.9086792373581213, 0.9583037259038695, 0.9778942553036174, 0.7985046492157409]}, {\"categories\": [\"0\", \"1\", \"2\", \"3\", \"4\", \"5\", \"6\", \"7\", \"8\", \"9\"], \"mode\": \"lines\", \"name\": \"norm_macro_recall_max\", \"stepped\": true, \"type\": \"scatter\", \"data\": [0.9592223201926373, 0.9592223201926373, 0.9746721703334068, 0.9746721703334068, 0.9746721703334068, 0.9746721703334068, 0.9746721703334068, 0.9746721703334068, 0.9778942553036174, 0.9778942553036174]}], \"log_loss\": [{\"categories\": [\"0\", \"1\", \"2\", \"3\", \"4\", \"5\", \"6\", \"7\", \"8\", \"9\"], \"mode\": \"markers\", \"name\": \"log_loss\", \"stepped\": false, \"type\": \"scatter\", \"data\": [0.11264348157133698, 0.1342876872993016, 0.13080861791488663, 0.8379254131099501, 0.691834902415937, 0.39371149731702765, 0.5643508659891863, 0.516513977588911, 0.14778326607028566, 0.7031305877169217]}, {\"categories\": [\"0\", \"1\", \"2\", \"3\", \"4\", \"5\", \"6\", \"7\", \"8\", \"9\"], \"mode\": \"lines\", \"name\": \"log_loss_min\", \"stepped\": true, \"type\": \"scatter\", \"data\": [0.11264348157133698, 0.11264348157133698, 0.11264348157133698, 0.11264348157133698, 0.11264348157133698, 0.11264348157133698, 0.11264348157133698, 0.11264348157133698, 0.11264348157133698, 0.11264348157133698]}], \"recall_score_macro\": [{\"categories\": [\"0\", \"1\", \"2\", \"3\", \"4\", \"5\", \"6\", \"7\", \"8\", \"9\"], \"mode\": \"markers\", \"name\": \"recall_score_macro\", \"stepped\": false, \"type\": \"scatter\", \"data\": [0.9633000881733736, 0.9609461219451969, 0.9772049533000661, 0.8617819618884656, 0.9350145867832289, 0.9416610453441616, 0.9178113136223093, 0.9624733533134826, 0.9801048297732556, 0.8186541842941667]}, {\"categories\": [\"0\", \"1\", \"2\", \"3\", \"4\", \"5\", \"6\", \"7\", \"8\", \"9\"], \"mode\": \"lines\", \"name\": \"recall_score_macro_max\", \"stepped\": true, \"type\": \"scatter\", \"data\": [0.9633000881733736, 0.9633000881733736, 0.9772049533000661, 0.9772049533000661, 0.9772049533000661, 0.9772049533000661, 0.9772049533000661, 0.9772049533000661, 0.9801048297732556, 0.9801048297732556]}], \"f1_score_micro\": [{\"categories\": [\"0\", \"1\", \"2\", \"3\", \"4\", \"5\", \"6\", \"7\", \"8\", \"9\"], \"mode\": \"markers\", \"name\": \"f1_score_micro\", \"stepped\": false, \"type\": \"scatter\", \"data\": [0.9628839759425456, 0.9605251362873553, 0.9770161668595015, 0.8621147211190677, 0.9340014801380073, 0.941670054306472, 0.9180952083137893, 0.9617040349395957, 0.9799660193668762, 0.8026402743467068]}, {\"categories\": [\"0\", \"1\", \"2\", \"3\", \"4\", \"5\", \"6\", \"7\", \"8\", \"9\"], \"mode\": \"lines\", \"name\": \"f1_score_micro_max\", \"stepped\": true, \"type\": \"scatter\", \"data\": [0.9628839759425456, 0.9628839759425456, 0.9770161668595015, 0.9770161668595015, 0.9770161668595015, 0.9770161668595015, 0.9770161668595015, 0.9770161668595015, 0.9799660193668762, 0.9799660193668762]}], \"average_precision_score_weighted\": [{\"categories\": [\"0\", \"1\", \"2\", \"3\", \"4\", \"5\", \"6\", \"7\", \"8\", \"9\"], \"mode\": \"markers\", \"name\": \"average_precision_score_weighted\", \"stepped\": false, \"type\": \"scatter\", \"data\": [0.9947431665432841, 0.9940092587807604, 0.9954134606584472, 0.9311858267231844, 0.9683576786367728, 0.9777519220099554, 0.9684318506232525, 0.9774178932952382, 0.9979479476259953, 0.9429638428775236]}, {\"categories\": [\"0\", \"1\", \"2\", \"3\", \"4\", \"5\", \"6\", \"7\", \"8\", \"9\"], \"mode\": \"lines\", \"name\": \"average_precision_score_weighted_max\", \"stepped\": true, \"type\": \"scatter\", \"data\": [0.9947431665432841, 0.9947431665432841, 0.9954134606584472, 0.9954134606584472, 0.9954134606584472, 0.9954134606584472, 0.9954134606584472, 0.9954134606584472, 0.9979479476259953, 0.9979479476259953]}], \"recall_score_micro\": [{\"categories\": [\"0\", \"1\", \"2\", \"3\", \"4\", \"5\", \"6\", \"7\", \"8\", \"9\"], \"mode\": \"markers\", \"name\": \"recall_score_micro\", \"stepped\": false, \"type\": \"scatter\", \"data\": [0.9628839759425456, 0.9605251362873553, 0.9770161668595015, 0.8621147211190677, 0.9340014801380073, 0.941670054306472, 0.9180952083137893, 0.9617040349395957, 0.9799660193668762, 0.8026402743467068]}, {\"categories\": [\"0\", \"1\", \"2\", \"3\", \"4\", \"5\", \"6\", \"7\", \"8\", \"9\"], \"mode\": \"lines\", \"name\": \"recall_score_micro_max\", \"stepped\": true, \"type\": \"scatter\", \"data\": [0.9628839759425456, 0.9628839759425456, 0.9770161668595015, 0.9770161668595015, 0.9770161668595015, 0.9770161668595015, 0.9770161668595015, 0.9770161668595015, 0.9799660193668762, 0.9799660193668762]}], \"average_precision_score_micro\": [{\"categories\": [\"0\", \"1\", \"2\", \"3\", \"4\", \"5\", \"6\", \"7\", \"8\", \"9\"], \"mode\": \"markers\", \"name\": \"average_precision_score_micro\", \"stepped\": false, \"type\": \"scatter\", \"data\": [0.9944002366191053, 0.9938193767064769, 0.9954054644171587, 0.9319266809692449, 0.9732035117785424, 0.9828237477488685, 0.9713673698477194, 0.9787709441096024, 0.9979518516312101, 0.897643524090148]}, {\"categories\": [\"0\", \"1\", \"2\", \"3\", \"4\", \"5\", \"6\", \"7\", \"8\", \"9\"], \"mode\": \"lines\", \"name\": \"average_precision_score_micro_max\", \"stepped\": true, \"type\": \"scatter\", \"data\": [0.9944002366191053, 0.9944002366191053, 0.9954054644171587, 0.9954054644171587, 0.9954054644171587, 0.9954054644171587, 0.9954054644171587, 0.9954054644171587, 0.9979518516312101, 0.9979518516312101]}], \"average_precision_score_macro\": [{\"categories\": [\"0\", \"1\", \"2\", \"3\", \"4\", \"5\", \"6\", \"7\", \"8\", \"9\"], \"mode\": \"markers\", \"name\": \"average_precision_score_macro\", \"stepped\": false, \"type\": \"scatter\", \"data\": [0.994792988333756, 0.9939933758616979, 0.9955715246625335, 0.9308590506096994, 0.9684342559961682, 0.9779558977640334, 0.9679669511864072, 0.9775220845107778, 0.997973084114332, 0.9414505204522984]}, {\"categories\": [\"0\", \"1\", \"2\", \"3\", \"4\", \"5\", \"6\", \"7\", \"8\", \"9\"], \"mode\": \"lines\", \"name\": \"average_precision_score_macro_max\", \"stepped\": true, \"type\": \"scatter\", \"data\": [0.994792988333756, 0.994792988333756, 0.9955715246625335, 0.9955715246625335, 0.9955715246625335, 0.9955715246625335, 0.9955715246625335, 0.9955715246625335, 0.997973084114332, 0.997973084114332]}], \"recall_score_weighted\": [{\"categories\": [\"0\", \"1\", \"2\", \"3\", \"4\", \"5\", \"6\", \"7\", \"8\", \"9\"], \"mode\": \"markers\", \"name\": \"recall_score_weighted\", \"stepped\": false, \"type\": \"scatter\", \"data\": [0.9628839759425456, 0.9605251362873553, 0.9770161668595015, 0.8621147211190677, 0.9340014801380073, 0.941670054306472, 0.9180952083137893, 0.9617040349395957, 0.9799660193668762, 0.8026402743467068]}, {\"categories\": [\"0\", \"1\", \"2\", \"3\", \"4\", \"5\", \"6\", \"7\", \"8\", \"9\"], \"mode\": \"lines\", \"name\": \"recall_score_weighted_max\", \"stepped\": true, \"type\": \"scatter\", \"data\": [0.9628839759425456, 0.9628839759425456, 0.9770161668595015, 0.9770161668595015, 0.9770161668595015, 0.9770161668595015, 0.9770161668595015, 0.9770161668595015, 0.9799660193668762, 0.9799660193668762]}], \"AUC_weighted\": [{\"categories\": [\"0\", \"1\", \"2\", \"3\", \"4\", \"5\", \"6\", \"7\", \"8\", \"9\"], \"mode\": \"markers\", \"name\": \"AUC_weighted\", \"stepped\": false, \"type\": \"scatter\", \"data\": [0.9991940367053402, 0.9991509279404523, 0.9987897319884537, 0.9865134482777519, 0.9932972312022299, 0.9957899743264166, 0.9942213439244755, 0.9929700892698329, 0.9997226726657312, 0.9767917522787696]}, {\"categories\": [\"0\", \"1\", \"2\", \"3\", \"4\", \"5\", \"6\", \"7\", \"8\", \"9\"], \"mode\": \"lines\", \"name\": \"AUC_weighted_max\", \"stepped\": true, \"type\": \"scatter\", \"data\": [0.9991940367053402, 0.9991940367053402, 0.9991940367053402, 0.9991940367053402, 0.9991940367053402, 0.9991940367053402, 0.9991940367053402, 0.9991940367053402, 0.9997226726657312, 0.9997226726657312]}], \"precision_score_macro\": [{\"categories\": [\"0\", \"1\", \"2\", \"3\", \"4\", \"5\", \"6\", \"7\", \"8\", \"9\"], \"mode\": \"markers\", \"name\": \"precision_score_macro\", \"stepped\": false, \"type\": \"scatter\", \"data\": [0.9640830453384751, 0.9613595438298086, 0.9778899642927587, 0.8635227975592263, 0.9347521515769408, 0.9427446733153223, 0.9186662773583251, 0.9622760890833764, 0.9809917709323183, 0.7685683535022306]}, {\"categories\": [\"0\", \"1\", \"2\", \"3\", \"4\", \"5\", \"6\", \"7\", \"8\", \"9\"], \"mode\": \"lines\", \"name\": \"precision_score_macro_max\", \"stepped\": true, \"type\": \"scatter\", \"data\": [0.9640830453384751, 0.9640830453384751, 0.9778899642927587, 0.9778899642927587, 0.9778899642927587, 0.9778899642927587, 0.9778899642927587, 0.9778899642927587, 0.9809917709323183, 0.9809917709323183]}]}, \"metricName\": null, \"primaryMetricName\": \"AUC_weighted\", \"showLegend\": false}, \"run_metrics\": [{\"name\": \"experiment_status\", \"run_id\": \"AutoML_99c06691-0a63-42f3-b16b-1d38b6d0f8ae\", \"categories\": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11], \"series\": [{\"data\": [\"DatasetEvaluation\", \"FeaturesGeneration\", \"DatasetFeaturization\", \"DatasetFeaturizationCompleted\", \"DatasetCrossValidationSplit\", \"ModelSelection\", \"BestRunExplainModel\", \"ModelExplanationDataSetSetup\", \"PickSurrogateModel\", \"EngineeredFeatureExplanations\", \"EngineeredFeatureExplanations\", \"BestRunExplainModel\"]}]}, {\"name\": \"experiment_status_description\", \"run_id\": \"AutoML_99c06691-0a63-42f3-b16b-1d38b6d0f8ae\", \"categories\": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11], \"series\": [{\"data\": [\"Gathering dataset statistics.\", \"Generating features for the dataset.\", \"Beginning to fit featurizers and featurize the dataset.\", \"Completed fit featurizers and featurizing the dataset.\", \"Generating individually featurized CV splits.\", \"Beginning model selection.\", \"Best run model explanations started\", \"Model explanations data setup completed\", \"Choosing LightGBM as the surrogate model for explanations\", \"Computation of engineered features started\", \"Computation of engineered features completed\", \"Best run model explanations completed\"]}]}, {\"name\": \"matthews_correlation\", \"run_id\": \"AutoML_99c06691-0a63-42f3-b16b-1d38b6d0f8ae\", \"categories\": [0], \"series\": [{\"data\": [0.9777813954801045]}]}, {\"name\": \"balanced_accuracy\", \"run_id\": \"AutoML_99c06691-0a63-42f3-b16b-1d38b6d0f8ae\", \"categories\": [0], \"series\": [{\"data\": [0.9801048297732556]}]}, {\"name\": \"accuracy\", \"run_id\": \"AutoML_99c06691-0a63-42f3-b16b-1d38b6d0f8ae\", \"categories\": [0], \"series\": [{\"data\": [0.9799660193668762]}]}, {\"name\": \"AUC_macro\", \"run_id\": \"AutoML_99c06691-0a63-42f3-b16b-1d38b6d0f8ae\", \"categories\": [0], \"series\": [{\"data\": [0.9997278321619597]}]}, {\"name\": \"f1_score_weighted\", \"run_id\": \"AutoML_99c06691-0a63-42f3-b16b-1d38b6d0f8ae\", \"categories\": [0], \"series\": [{\"data\": [0.9800071252416492]}]}, {\"name\": \"precision_score_weighted\", \"run_id\": \"AutoML_99c06691-0a63-42f3-b16b-1d38b6d0f8ae\", \"categories\": [0], \"series\": [{\"data\": [0.9806729693762462]}]}, {\"name\": \"AUC_micro\", \"run_id\": \"AutoML_99c06691-0a63-42f3-b16b-1d38b6d0f8ae\", \"categories\": [0], \"series\": [{\"data\": [0.9997502627629037]}]}, {\"name\": \"weighted_accuracy\", \"run_id\": \"AutoML_99c06691-0a63-42f3-b16b-1d38b6d0f8ae\", \"categories\": [0], \"series\": [{\"data\": [0.9797723436975737]}]}, {\"name\": \"precision_score_micro\", \"run_id\": \"AutoML_99c06691-0a63-42f3-b16b-1d38b6d0f8ae\", \"categories\": [0], \"series\": [{\"data\": [0.9799660193668762]}]}, {\"name\": \"f1_score_macro\", \"run_id\": \"AutoML_99c06691-0a63-42f3-b16b-1d38b6d0f8ae\", \"categories\": [0], \"series\": [{\"data\": [0.9802344671856597]}]}, {\"name\": \"norm_macro_recall\", \"run_id\": \"AutoML_99c06691-0a63-42f3-b16b-1d38b6d0f8ae\", \"categories\": [0], \"series\": [{\"data\": [0.9778942553036174]}]}, {\"name\": \"log_loss\", \"run_id\": \"AutoML_99c06691-0a63-42f3-b16b-1d38b6d0f8ae\", \"categories\": [0], \"series\": [{\"data\": [0.14778326607028566]}]}, {\"name\": \"recall_score_macro\", \"run_id\": \"AutoML_99c06691-0a63-42f3-b16b-1d38b6d0f8ae\", \"categories\": [0], \"series\": [{\"data\": [0.9801048297732556]}]}, {\"name\": \"f1_score_micro\", \"run_id\": \"AutoML_99c06691-0a63-42f3-b16b-1d38b6d0f8ae\", \"categories\": [0], \"series\": [{\"data\": [0.9799660193668762]}]}, {\"name\": \"average_precision_score_weighted\", \"run_id\": \"AutoML_99c06691-0a63-42f3-b16b-1d38b6d0f8ae\", \"categories\": [0], \"series\": [{\"data\": [0.9979479476259953]}]}, {\"name\": \"recall_score_micro\", \"run_id\": \"AutoML_99c06691-0a63-42f3-b16b-1d38b6d0f8ae\", \"categories\": [0], \"series\": [{\"data\": [0.9799660193668762]}]}, {\"name\": \"average_precision_score_micro\", \"run_id\": \"AutoML_99c06691-0a63-42f3-b16b-1d38b6d0f8ae\", \"categories\": [0], \"series\": [{\"data\": [0.9979518516312101]}]}, {\"name\": \"average_precision_score_macro\", \"run_id\": \"AutoML_99c06691-0a63-42f3-b16b-1d38b6d0f8ae\", \"categories\": [0], \"series\": [{\"data\": [0.997973084114332]}]}, {\"name\": \"recall_score_weighted\", \"run_id\": \"AutoML_99c06691-0a63-42f3-b16b-1d38b6d0f8ae\", \"categories\": [0], \"series\": [{\"data\": [0.9799660193668762]}]}, {\"name\": \"AUC_weighted\", \"run_id\": \"AutoML_99c06691-0a63-42f3-b16b-1d38b6d0f8ae\", \"categories\": [0], \"series\": [{\"data\": [0.9997226726657312]}]}, {\"name\": \"precision_score_macro\", \"run_id\": \"AutoML_99c06691-0a63-42f3-b16b-1d38b6d0f8ae\", \"categories\": [0], \"series\": [{\"data\": [0.9809917709323183]}]}], \"run_logs\": \"\\nRun is completed.\", \"graph\": {}, \"widget_settings\": {\"childWidgetDisplay\": \"popup\", \"send_telemetry\": false, \"log_level\": \"INFO\", \"sdk_version\": \"1.13.0\"}, \"loading\": false}"
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from azureml.widgets import RunDetails\n",
    "RunDetails(local_run).show() "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "### 6.2 Métriques par itération\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>0</th>\n",
       "      <th>1</th>\n",
       "      <th>2</th>\n",
       "      <th>3</th>\n",
       "      <th>4</th>\n",
       "      <th>5</th>\n",
       "      <th>6</th>\n",
       "      <th>7</th>\n",
       "      <th>8</th>\n",
       "      <th>9</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>matthews_correlation</th>\n",
       "      <td>0.96</td>\n",
       "      <td>0.96</td>\n",
       "      <td>0.97</td>\n",
       "      <td>0.85</td>\n",
       "      <td>0.93</td>\n",
       "      <td>0.94</td>\n",
       "      <td>0.91</td>\n",
       "      <td>0.96</td>\n",
       "      <td>0.98</td>\n",
       "      <td>0.81</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>balanced_accuracy</th>\n",
       "      <td>0.96</td>\n",
       "      <td>0.96</td>\n",
       "      <td>0.98</td>\n",
       "      <td>0.86</td>\n",
       "      <td>0.94</td>\n",
       "      <td>0.94</td>\n",
       "      <td>0.92</td>\n",
       "      <td>0.96</td>\n",
       "      <td>0.98</td>\n",
       "      <td>0.82</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>accuracy</th>\n",
       "      <td>0.96</td>\n",
       "      <td>0.96</td>\n",
       "      <td>0.98</td>\n",
       "      <td>0.86</td>\n",
       "      <td>0.93</td>\n",
       "      <td>0.94</td>\n",
       "      <td>0.92</td>\n",
       "      <td>0.96</td>\n",
       "      <td>0.98</td>\n",
       "      <td>0.80</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>AUC_macro</th>\n",
       "      <td>1.00</td>\n",
       "      <td>1.00</td>\n",
       "      <td>1.00</td>\n",
       "      <td>0.99</td>\n",
       "      <td>0.99</td>\n",
       "      <td>1.00</td>\n",
       "      <td>0.99</td>\n",
       "      <td>0.99</td>\n",
       "      <td>1.00</td>\n",
       "      <td>0.98</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>f1_score_weighted</th>\n",
       "      <td>0.96</td>\n",
       "      <td>0.96</td>\n",
       "      <td>0.98</td>\n",
       "      <td>0.86</td>\n",
       "      <td>0.93</td>\n",
       "      <td>0.94</td>\n",
       "      <td>0.92</td>\n",
       "      <td>0.96</td>\n",
       "      <td>0.98</td>\n",
       "      <td>0.77</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>precision_score_weighted</th>\n",
       "      <td>0.96</td>\n",
       "      <td>0.96</td>\n",
       "      <td>0.98</td>\n",
       "      <td>0.87</td>\n",
       "      <td>0.94</td>\n",
       "      <td>0.94</td>\n",
       "      <td>0.92</td>\n",
       "      <td>0.96</td>\n",
       "      <td>0.98</td>\n",
       "      <td>0.76</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>AUC_micro</th>\n",
       "      <td>1.00</td>\n",
       "      <td>1.00</td>\n",
       "      <td>1.00</td>\n",
       "      <td>0.99</td>\n",
       "      <td>0.99</td>\n",
       "      <td>1.00</td>\n",
       "      <td>0.99</td>\n",
       "      <td>0.99</td>\n",
       "      <td>1.00</td>\n",
       "      <td>0.98</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>weighted_accuracy</th>\n",
       "      <td>0.96</td>\n",
       "      <td>0.96</td>\n",
       "      <td>0.98</td>\n",
       "      <td>0.86</td>\n",
       "      <td>0.93</td>\n",
       "      <td>0.94</td>\n",
       "      <td>0.92</td>\n",
       "      <td>0.96</td>\n",
       "      <td>0.98</td>\n",
       "      <td>0.79</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>precision_score_micro</th>\n",
       "      <td>0.96</td>\n",
       "      <td>0.96</td>\n",
       "      <td>0.98</td>\n",
       "      <td>0.86</td>\n",
       "      <td>0.93</td>\n",
       "      <td>0.94</td>\n",
       "      <td>0.92</td>\n",
       "      <td>0.96</td>\n",
       "      <td>0.98</td>\n",
       "      <td>0.80</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>f1_score_macro</th>\n",
       "      <td>0.96</td>\n",
       "      <td>0.96</td>\n",
       "      <td>0.98</td>\n",
       "      <td>0.86</td>\n",
       "      <td>0.93</td>\n",
       "      <td>0.94</td>\n",
       "      <td>0.92</td>\n",
       "      <td>0.96</td>\n",
       "      <td>0.98</td>\n",
       "      <td>0.78</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>norm_macro_recall</th>\n",
       "      <td>0.96</td>\n",
       "      <td>0.96</td>\n",
       "      <td>0.97</td>\n",
       "      <td>0.85</td>\n",
       "      <td>0.93</td>\n",
       "      <td>0.94</td>\n",
       "      <td>0.91</td>\n",
       "      <td>0.96</td>\n",
       "      <td>0.98</td>\n",
       "      <td>0.80</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>log_loss</th>\n",
       "      <td>0.11</td>\n",
       "      <td>0.13</td>\n",
       "      <td>0.13</td>\n",
       "      <td>0.84</td>\n",
       "      <td>0.69</td>\n",
       "      <td>0.39</td>\n",
       "      <td>0.56</td>\n",
       "      <td>0.52</td>\n",
       "      <td>0.15</td>\n",
       "      <td>0.70</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>recall_score_macro</th>\n",
       "      <td>0.96</td>\n",
       "      <td>0.96</td>\n",
       "      <td>0.98</td>\n",
       "      <td>0.86</td>\n",
       "      <td>0.94</td>\n",
       "      <td>0.94</td>\n",
       "      <td>0.92</td>\n",
       "      <td>0.96</td>\n",
       "      <td>0.98</td>\n",
       "      <td>0.82</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>f1_score_micro</th>\n",
       "      <td>0.96</td>\n",
       "      <td>0.96</td>\n",
       "      <td>0.98</td>\n",
       "      <td>0.86</td>\n",
       "      <td>0.93</td>\n",
       "      <td>0.94</td>\n",
       "      <td>0.92</td>\n",
       "      <td>0.96</td>\n",
       "      <td>0.98</td>\n",
       "      <td>0.80</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>average_precision_score_weighted</th>\n",
       "      <td>0.99</td>\n",
       "      <td>0.99</td>\n",
       "      <td>1.00</td>\n",
       "      <td>0.93</td>\n",
       "      <td>0.97</td>\n",
       "      <td>0.98</td>\n",
       "      <td>0.97</td>\n",
       "      <td>0.98</td>\n",
       "      <td>1.00</td>\n",
       "      <td>0.94</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>recall_score_micro</th>\n",
       "      <td>0.96</td>\n",
       "      <td>0.96</td>\n",
       "      <td>0.98</td>\n",
       "      <td>0.86</td>\n",
       "      <td>0.93</td>\n",
       "      <td>0.94</td>\n",
       "      <td>0.92</td>\n",
       "      <td>0.96</td>\n",
       "      <td>0.98</td>\n",
       "      <td>0.80</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>average_precision_score_micro</th>\n",
       "      <td>0.99</td>\n",
       "      <td>0.99</td>\n",
       "      <td>1.00</td>\n",
       "      <td>0.93</td>\n",
       "      <td>0.97</td>\n",
       "      <td>0.98</td>\n",
       "      <td>0.97</td>\n",
       "      <td>0.98</td>\n",
       "      <td>1.00</td>\n",
       "      <td>0.90</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>average_precision_score_macro</th>\n",
       "      <td>0.99</td>\n",
       "      <td>0.99</td>\n",
       "      <td>1.00</td>\n",
       "      <td>0.93</td>\n",
       "      <td>0.97</td>\n",
       "      <td>0.98</td>\n",
       "      <td>0.97</td>\n",
       "      <td>0.98</td>\n",
       "      <td>1.00</td>\n",
       "      <td>0.94</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>recall_score_weighted</th>\n",
       "      <td>0.96</td>\n",
       "      <td>0.96</td>\n",
       "      <td>0.98</td>\n",
       "      <td>0.86</td>\n",
       "      <td>0.93</td>\n",
       "      <td>0.94</td>\n",
       "      <td>0.92</td>\n",
       "      <td>0.96</td>\n",
       "      <td>0.98</td>\n",
       "      <td>0.80</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>AUC_weighted</th>\n",
       "      <td>1.00</td>\n",
       "      <td>1.00</td>\n",
       "      <td>1.00</td>\n",
       "      <td>0.99</td>\n",
       "      <td>0.99</td>\n",
       "      <td>1.00</td>\n",
       "      <td>0.99</td>\n",
       "      <td>0.99</td>\n",
       "      <td>1.00</td>\n",
       "      <td>0.98</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>precision_score_macro</th>\n",
       "      <td>0.96</td>\n",
       "      <td>0.96</td>\n",
       "      <td>0.98</td>\n",
       "      <td>0.86</td>\n",
       "      <td>0.93</td>\n",
       "      <td>0.94</td>\n",
       "      <td>0.92</td>\n",
       "      <td>0.96</td>\n",
       "      <td>0.98</td>\n",
       "      <td>0.77</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                    0    1    2    3    4    5    6    7    8  \\\n",
       "matthews_correlation             0.96 0.96 0.97 0.85 0.93 0.94 0.91 0.96 0.98   \n",
       "balanced_accuracy                0.96 0.96 0.98 0.86 0.94 0.94 0.92 0.96 0.98   \n",
       "accuracy                         0.96 0.96 0.98 0.86 0.93 0.94 0.92 0.96 0.98   \n",
       "AUC_macro                        1.00 1.00 1.00 0.99 0.99 1.00 0.99 0.99 1.00   \n",
       "f1_score_weighted                0.96 0.96 0.98 0.86 0.93 0.94 0.92 0.96 0.98   \n",
       "precision_score_weighted         0.96 0.96 0.98 0.87 0.94 0.94 0.92 0.96 0.98   \n",
       "AUC_micro                        1.00 1.00 1.00 0.99 0.99 1.00 0.99 0.99 1.00   \n",
       "weighted_accuracy                0.96 0.96 0.98 0.86 0.93 0.94 0.92 0.96 0.98   \n",
       "precision_score_micro            0.96 0.96 0.98 0.86 0.93 0.94 0.92 0.96 0.98   \n",
       "f1_score_macro                   0.96 0.96 0.98 0.86 0.93 0.94 0.92 0.96 0.98   \n",
       "norm_macro_recall                0.96 0.96 0.97 0.85 0.93 0.94 0.91 0.96 0.98   \n",
       "log_loss                         0.11 0.13 0.13 0.84 0.69 0.39 0.56 0.52 0.15   \n",
       "recall_score_macro               0.96 0.96 0.98 0.86 0.94 0.94 0.92 0.96 0.98   \n",
       "f1_score_micro                   0.96 0.96 0.98 0.86 0.93 0.94 0.92 0.96 0.98   \n",
       "average_precision_score_weighted 0.99 0.99 1.00 0.93 0.97 0.98 0.97 0.98 1.00   \n",
       "recall_score_micro               0.96 0.96 0.98 0.86 0.93 0.94 0.92 0.96 0.98   \n",
       "average_precision_score_micro    0.99 0.99 1.00 0.93 0.97 0.98 0.97 0.98 1.00   \n",
       "average_precision_score_macro    0.99 0.99 1.00 0.93 0.97 0.98 0.97 0.98 1.00   \n",
       "recall_score_weighted            0.96 0.96 0.98 0.86 0.93 0.94 0.92 0.96 0.98   \n",
       "AUC_weighted                     1.00 1.00 1.00 0.99 0.99 1.00 0.99 0.99 1.00   \n",
       "precision_score_macro            0.96 0.96 0.98 0.86 0.93 0.94 0.92 0.96 0.98   \n",
       "\n",
       "                                    9  \n",
       "matthews_correlation             0.81  \n",
       "balanced_accuracy                0.82  \n",
       "accuracy                         0.80  \n",
       "AUC_macro                        0.98  \n",
       "f1_score_weighted                0.77  \n",
       "precision_score_weighted         0.76  \n",
       "AUC_micro                        0.98  \n",
       "weighted_accuracy                0.79  \n",
       "precision_score_micro            0.80  \n",
       "f1_score_macro                   0.78  \n",
       "norm_macro_recall                0.80  \n",
       "log_loss                         0.70  \n",
       "recall_score_macro               0.82  \n",
       "f1_score_micro                   0.80  \n",
       "average_precision_score_weighted 0.94  \n",
       "recall_score_micro               0.80  \n",
       "average_precision_score_micro    0.90  \n",
       "average_precision_score_macro    0.94  \n",
       "recall_score_weighted            0.80  \n",
       "AUC_weighted                     0.98  \n",
       "precision_score_macro            0.77  "
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "children = list(local_run.get_children())\n",
    "metricslist = {}\n",
    "for run in children:\n",
    "    properties = run.get_properties()\n",
    "    metrics = {k: v for k, v in run.get_metrics().items() if isinstance(v, float)}\n",
    "    metricslist[int(properties['iteration'])] = metrics\n",
    "\n",
    "rundata = pd.DataFrame(metricslist).sort_index(1)\n",
    "rundata"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 7. Récupération du meilleur modèle"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Best run = Run(Experiment: Exemple2-automlclassificationSR,\n",
      "Id: AutoML_99c06691-0a63-42f3-b16b-1d38b6d0f8ae_8,\n",
      "Type: None,\n",
      "Status: Completed)\n"
     ]
    }
   ],
   "source": [
    "best_run, fitted_model = local_run.get_output()\n",
    "print(\"Best run =\", best_run)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Pipeline(memory=None,\n",
      "     steps=[('datatransformer', DataTransformer(enable_dnn=None, enable_feature_sweeping=None,\n",
      "        feature_sweeping_config=None, feature_sweeping_timeout=None,\n",
      "        featurization_config=None, force_text_dnn=None,\n",
      "        is_cross_validation=None, is_onnx_compatible=None, logger=None,\n",
      "        obser... warm_start=False))]))],\n",
      "               flatten_transform=None, weights=[0.1, 0.3, 0.4, 0.1, 0.1]))])\n"
     ]
    }
   ],
   "source": [
    "print(fitted_model)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<bound method Pipeline.get_params of Pipeline(memory=None,\n",
       "     steps=[('datatransformer', DataTransformer(enable_dnn=None, enable_feature_sweeping=None,\n",
       "        feature_sweeping_config=None, feature_sweeping_timeout=None,\n",
       "        featurization_config=None, force_text_dnn=None,\n",
       "        is_cross_validation=None, is_onnx_compatible=None, logger=None,\n",
       "        obser... warm_start=False))]))],\n",
       "               flatten_transform=None, weights=[0.1, 0.3, 0.4, 0.1, 0.1]))])>"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "fitted_model.get_params"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "[('datatransformer', DataTransformer(enable_dnn=None, enable_feature_sweeping=None,\n",
       "        feature_sweeping_config=None, feature_sweeping_timeout=None,\n",
       "        featurization_config=None, force_text_dnn=None,\n",
       "        is_cross_validation=None, is_onnx_compatible=None, logger=None,\n",
       "        observer=None, task=None, working_dir=None)), ('prefittedsoftvotingclassifier', PreFittedSoftVotingClassifier(classification_labels=None,\n",
       "               estimators=[('0', Pipeline(memory=None,\n",
       "     steps=[('maxabsscaler', MaxAbsScaler(copy=True)), ('lightgbmclassifier', LightGBMClassifier(boosting_type='gbdt', class_weight=None,\n",
       "          colsample_bytree=1.0, importance_type='split', learning_rate=0.1,\n",
       "          max_depth=-1, min_child_samples=20, ...tors=10, n_jobs=1, oob_score=False, random_state=None,\n",
       "            verbose=0, warm_start=False))]))],\n",
       "               flatten_transform=None, weights=[0.1, 0.3, 0.4, 0.1, 0.1]))]"
      ],
      "text/plain": [
       "[('datatransformer',\n",
       "  DataTransformer(enable_dnn=None, enable_feature_sweeping=None,\n",
       "          feature_sweeping_config=None, feature_sweeping_timeout=None,\n",
       "          featurization_config=None, force_text_dnn=None,\n",
       "          is_cross_validation=None, is_onnx_compatible=None, logger=None,\n",
       "          observer=None, task=None, working_dir=None)),\n",
       " ('prefittedsoftvotingclassifier',\n",
       "  PreFittedSoftVotingClassifier(classification_labels=None,\n",
       "                 estimators=[('0', Pipeline(memory=None,\n",
       "       steps=[('maxabsscaler', MaxAbsScaler(copy=True)), ('lightgbmclassifier', LightGBMClassifier(boosting_type='gbdt', class_weight=None,\n",
       "            colsample_bytree=1.0, importance_type='split', learning_rate=0.1,\n",
       "            max_depth=-1, min_child_samples=20, ...tors=10, n_jobs=1, oob_score=False, random_state=None,\n",
       "              verbose=0, warm_start=False))]))],\n",
       "                 flatten_transform=None, weights=[0.1, 0.3, 0.4, 0.1, 0.1]))]"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "fitted_model.steps"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Métriques du meilleur modèle :\n",
      "\n",
      "- matthews_correlation = 0.9777813954801045\n",
      "- balanced_accuracy = 0.9801048297732556\n",
      "- accuracy = 0.9799660193668762\n",
      "- AUC_macro = 0.9997278321619597\n",
      "- f1_score_weighted = 0.9800071252416492\n",
      "- precision_score_weighted = 0.9806729693762462\n",
      "- AUC_micro = 0.9997502627629037\n",
      "- weighted_accuracy = 0.9797723436975737\n",
      "- precision_score_micro = 0.9799660193668762\n",
      "- f1_score_macro = 0.9802344671856597\n",
      "- norm_macro_recall = 0.9778942553036174\n",
      "- log_loss = 0.14778326607028566\n",
      "- recall_score_macro = 0.9801048297732556\n",
      "- f1_score_micro = 0.9799660193668762\n",
      "- average_precision_score_weighted = 0.9979479476259953\n",
      "- recall_score_micro = 0.9799660193668762\n",
      "- average_precision_score_micro = 0.9979518516312101\n",
      "- average_precision_score_macro = 0.997973084114332\n",
      "- recall_score_weighted = 0.9799660193668762\n",
      "- AUC_weighted = 0.9997226726657312\n",
      "- precision_score_macro = 0.9809917709323183\n",
      "- confusion_matrix = aml://artifactId/ExperimentRun/dcid.AutoML_99c06691-0a63-42f3-b16b-1d38b6d0f8ae_8/confusion_matrix\n",
      "- accuracy_table = aml://artifactId/ExperimentRun/dcid.AutoML_99c06691-0a63-42f3-b16b-1d38b6d0f8ae_8/accuracy_table\n"
     ]
    }
   ],
   "source": [
    "print(\"Métriques du meilleur modèle :\")\n",
    "print()\n",
    "best_run_metrics = best_run.get_metrics()\n",
    "for metric_name in best_run_metrics:\n",
    "    metric = best_run_metrics[metric_name]\n",
    "    print('-', metric_name, '=', metric)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 7.1 Changement de la métrique AutoML\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Meilleur modèle :\n",
      "\n",
      "Best Run = Run(Experiment: Exemple2-automlclassificationSR,\n",
      "Id: AutoML_99c06691-0a63-42f3-b16b-1d38b6d0f8ae_8,\n",
      "Type: None,\n",
      "Status: Completed)\n",
      "\n",
      "Pipeline(memory=None,\n",
      "     steps=[('datatransformer', DataTransformer(enable_dnn=None, enable_feature_sweeping=None,\n",
      "        feature_sweeping_config=None, feature_sweeping_timeout=None,\n",
      "        featurization_config=None, force_text_dnn=None,\n",
      "        is_cross_validation=None, is_onnx_compatible=None, logger=None,\n",
      "        obser... warm_start=False))]))],\n",
      "               flatten_transform=None, weights=[0.1, 0.3, 0.4, 0.1, 0.1]))])\n"
     ]
    }
   ],
   "source": [
    "lookup_metric = \"f1_score_weighted\"\n",
    "best_run, fitted_model = local_run.get_output(metric = lookup_metric)\n",
    "print(\"Meilleur modèle :\")\n",
    "print()\n",
    "print('Best Run =', best_run)\n",
    "print()\n",
    "print(fitted_model)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Choix d'une itération particulière :"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Modèle 3 ème itération :\n",
      "Run(Experiment: Exemple2-automlclassificationSR,\n",
      "Id: AutoML_99c06691-0a63-42f3-b16b-1d38b6d0f8ae_3,\n",
      "Type: None,\n",
      "Status: Completed)\n",
      "\n",
      "Pipeline(memory=None,\n",
      "     steps=[('datatransformer', DataTransformer(enable_dnn=None, enable_feature_sweeping=None,\n",
      "        feature_sweeping_config=None, feature_sweeping_timeout=None,\n",
      "        featurization_config=None, force_text_dnn=None,\n",
      "        is_cross_validation=None, is_onnx_compatible=None, logger=None,\n",
      "        obser...n_jobs=1,\n",
      "            oob_score=False, random_state=None, verbose=0,\n",
      "            warm_start=False))])\n"
     ]
    }
   ],
   "source": [
    "iteration = 3\n",
    "\n",
    "my_run, my_model = local_run.get_output(iteration = iteration)\n",
    "print(\"Modèle\",iteration,\"ème itération :\")\n",
    "print(my_run)\n",
    "print()\n",
    "print(my_model)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 8. Application du modèle\n",
    "\n",
    "### 8.1 Chargement des données"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [],
   "source": [
    "digits = datasets.load_digits()\n",
    "X_test = digits.data[:10, :]\n",
    "y_test = digits.target[:10]\n",
    "images = digits.images[:10]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 8.2 Test\n",
    "We will try to predict 2 digits and see how our model works."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "4\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAMgAAADcCAYAAAA1H+4TAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAOeUlEQVR4nO3dfZBddX3H8fenm1CEBFYa6OSR5amZaqcNTIhiHJqStGJFiDNWSRuapVqm04ESKqLQzhg7HUOdqQOtra0TSCKJMBrCg5RicZBBKyBJiJSY0IlhMUsM2YibBKVNE77945yduVnu/vYsuWfP2buf18ydvQ/n/O53z72f+zsP9/yuIgIza+6Xqi7ArM4cELMEB8QswQExS3BAzBIcELOEMRkQSY9L+vhoz1snkm6RtGqUnmuWpNckdYzG89VJpQGR1CNpUZU1tJKkEyTtkNQ7gnm6JR3N34CNl2mp+SLicxFRStAHvy4R8eOImBQRR8t4vuMhaZmkKOtDb0IZjY5jnwT2AZNGON+TEfHeEuppa5LeDtwMbCvrOWq5iiXp7ZIektQn6Wf59RmDJjtH0vclHZD0gKTTGuZ/t6TvSeqX9ANJC0ah5rOApcDKFrf7KUkvSzok6QVJC/P7V0hal1/vyj9Fr5a0O19mfybpQknP5cvhiw1tniPpMUk/lbRf0npJnfljdwGzgG/kPdlNDe1PyKeZJulBSa9K2inpTxvaXiHpa5K+kte8TdLcVi6TBiuBfwD2l9R+PQNCVtdq4EyyF+t14IuDpvlj4E+AacARsgWFpOnAvwF/C5wG3AjcK+n04Z5U0h/mb6ahLrMSs/8jcEtea0tImg1cC1wYEZOB9wE9iVneBZwHfBS4DfgrYBHwTuAjkn57oGmyN9c04NeBmcAKgIi4Cvgx8MF8terzTZ7nbqA3n//DwOcGgpu7HLgH6AQe5M2vXeP/+Fxief9zYr55wFzgXxLL4/hFRGUXshd7UYHp5gA/a7j9OHBrw+13AIeBDuBTwF2D5v8msKxh3o+3+P/4EPBIfn0B0DuCebvJAt7fcPlR/ti5ZKtsi4CJg+ZbAazLr3cBAUxvePynwEcbbt8LLB+ihsXAs0O9Lg3tTyAL01FgcsPjK4E1DXV9a9Br83qLl3cHsAm4qKzXdOBSyx5E0kmS/lXSS5IOAk8AnYP2ouxuuP4SMBGYQtbr/EHjJxHwXmBqSbWeDHweuO44mnkqIjobLucARMROYDnZm26fpHuG2Xh/peH6601uT8prPiNv6+V8+a4jW3ZFTANejYhDDfe9BExvuL234fovgBMHVs9a5M+B5yLiyRa22VQtAwJ8ApgNvCsiTgEuzu9XwzQzG67PAv6PbF10N1kP0viGOzkibh3uSSX9UZO9SY2XZqtY55F9wn5H0l5gIzBV0l5JXSP7t98sIr4a2Qb8mWSf4n93vG2SfeIH8Jv58l3Kscs29RXvPcBpkiY33DcLePmtFJJvowy1vIdafVoIfChfxnuB9wB/37id1Sp12Is1UdKJDbePAJPJPvH6843vzzSZb6mkr5CtDvwNsCEijuYbrs9Ieh/wLbKe5d3AzohI7n6NiPXA+hHW/zzHhvU9ZOvcFwB9I2zrGPk2yHTgP4H/IVsmrfhQmwwcIFu+08n2vjV6BTi72YwRsVvS94CVkm4Efg34GFnIRiwi3vkWZusGGt8zG4ENwB1vpYaUOvQgD5O98AOXFWQbmG8j6xGeAh5pMt9dwBqy7vxE4C8gewGBK8g2mPvIepRPUtL/GhFHImLvwAV4FXgjv130uMFFTT49LwR+GbiVbDnsBc4g+7+O12fJAnyAbIfGxkGPrwT+Ol9FvbHJ/EvIes09wH3AZyLi0RbUVUhE9A9a5oeBgxFxoNXPpXwjx8yaqEMPYlZbDohZggNiluCAmCU4IGYJpRwHmTJlSnR1dZXR9Jhx9Gjrvxn+4osvtrxNgHPPPbeUdseKnp4e9u/fr2aPlRKQrq4uNm3aVEbTY0Z/f3/L2+zu7m55mwD3339/Ke2OFXPnDv1lY69imSU4IGYJDohZggNiluCAmCUUCoikS/PzoXdK+nTZRZnVxbAByc/i+yfg/WSnTy6R9I6yCzOrgyI9yDyyk412RcRhspPxryi3LLN6KBKQ6Rx7/ncvx55/bNa2igSk2SH4N51lJekaSZskberrO64zTc1qo0hAejn2nOsZZKdaHiMivhwRcyNi7umnDzsEldmYUCQgzwDnSTpL0gnAlWSDgZm1vWG/rBgRRyRdSzb4WgdwZ0SUNhaqWZ0U+jZvRDxMNvqI2bjiI+lmCQ6IWYIDYpbggJglOCBmCQ6IWUIdRndvS2vWrGl5m3PmzGl5m5bmHsQswQExS3BAzBIcELMEB8QswQExSygyaMOdkvZJen40CjKrkyI9yBrg0pLrMKulYQMSEU+Q/XKr2bjTsm0QD9pg7ahlAfGgDdaOvBfLLMEBMUsospv3buBJYLakXkkfK78ss3ooMuzPktEoxKyOvIplluCAmCU4IGYJDohZggNiljDuB23o7+8vpd0yBm1Yvnx5y9sE6OnpKaXdMnR1dY3q87kHMUtwQMwSHBCzBAfELMEBMUtwQMwSinybd6akb0vaLmmbpOtHozCzOihyHOQI8ImI2CJpMrBZ0qMR8cOSazOrXJFBG34SEVvy64eA7cD0sgszq4MRbYNI6gLOB54uoxizuikcEEmTgHuB5RFxsMnjHtXE2k6hgEiaSBaO9RGxsdk0HtXE2lGRvVgC7gC2R8QXyi/JrD6K9CDzgauASyRtzS+/X3JdZrVQZNCG7wIahVrMasdH0s0SHBCzBAfELMEBMUtwQMwSHBCzhHE/qkkZo49AOSOFdHd3t7xNKG+0lM7Ozpa3uWLFipa3meIexCzBATFLcEDMEhwQswQHxCzBATFLKHI+yImSvi/pB/moJp8djcLM6qDIcZD/BS6JiNfyMwu/K+nfI+Kpkmszq1yR80ECeC2/OTG/RJlFmdVF0XPSOyRtBfYBj0aERzWxcaFQQCLiaETMAWYA8yT9xuBpPKqJtaMR7cWKiH7gceDSJo95VBNrO0X2Yp0uqTO//jZgEbCj7MLM6qDIXqypwFpJHWSB+lpEPFRuWWb1UGQv1nNkw42ajTs+km6W4ICYJTggZgkOiFmCA2KWMKYGbXjggQda3uYNN9zQ8jYBli1bVkq7Zbj99ttLaXf16tWltDua3IOYJTggZgkOiFmCA2KW4ICYJTggZgkj+RnoDknPSvI3eW3cGEkPcj2wvaxCzOqo6DnpM4APAKvKLcesXor2ILcBNwFvlFiLWe0UOeX2MmBfRGweZjoP2mBtp0gPMh+4XFIPcA9wiaR1gyfyoA3WjoYNSETcHBEzIqILuBJ4LCKWll6ZWQ34OIhZwoi+7h4Rj5ONi2U2LrgHMUtwQMwSHBCzBAfELMEBMUtwQMwSxtSoJqeeeuqYaBNg7dq1LW9z69atLW+zTIsXL666hOPmHsQswQExS3BAzBIcELMEB8QswQExSyi0mzc/WeoQcBQ4EhFzyyzKrC5GchzkdyJif2mVmNWQV7HMEooGJID/kLRZ0jVlFmRWJ0VXseZHxB5JZwCPStoREU80TpAH5xqAWbNmtbhMs2oU6kEiYk/+dx9wHzCvyTQe1cTaTpFxsU6WNHngOvB7wPNlF2ZWB0VWsX4VuE/SwPRfjYhHSq3KrCaGDUhE7AJ+axRqMasd7+Y1S3BAzBIcELMEB8QswQExS3BAzBLG1KgmCxYsaHmb/f39LW8TyhmBpIz/H2DZsmWltNvZ2VlKu6PJPYhZggNiluCAmCU4IGYJDohZggNillAoIJI6JW2QtEPSdkkXlV2YWR0UPQ5yO/BIRHxY0gnASSXWZFYbwwZE0inAxUA3QEQcBg6XW5ZZPRRZxTob6ANWS3pW0qr81NtjSLpG0iZJm/r6+lpeqFkVigRkAnAB8KWIOB/4OfDpwRN50AZrR0UC0gv0RsTT+e0NZIExa3vDBiQi9gK7Jc3O71oI/LDUqsxqouherOuA9fkerF3A1eWVZFYfhQISEVsBj+hu446PpJslOCBmCQ6IWYIDYpbggJgljKlBG8aSMgYsOHDgQMvbBOju7i6l3XbgHsQswQExS3BAzBIcELMEB8QswQExSyjyI56zJW1tuByUtHw0ijOrWpHfKHwBmAMgqQN4meynoM3a3khXsRYCP4qIl8ooxqxuRhqQK4G7yyjErI4KByQ/m/By4OtDPO5RTaztjKQHeT+wJSJeafagRzWxdjSSgCzBq1c2zhQdm/ck4HeBjeWWY1YvRQdt+AXwKyXXYlY7PpJuluCAmCU4IGYJDohZggNiluCAmCUoIlrfqNQHFPlC4xRgf8sLKM9Yqte1FndmRDT9+kcpASlK0qaIGDODYo+lel1ra3gVyyzBATFLqDogX674+UdqLNXrWlug0m0Qs7qrugcxq7XKAiLpUkkvSNop6U0/K10XkmZK+rak7ZK2Sbq+6pqGI6kj/037h6quZTiSOiVtkLQjX8YXVV1To0pWsfLRUf6b7ByTXuAZYElE1O7XcyVNBaZGxBZJk4HNwOI61jpA0l+S/abkKRFxWdX1pEhaC3wnIlblp3WfFBH9Vdc1oKoeZB6wMyJ2RcRh4B7giopqSYqIn0TElvz6IWA7ML3aqoYmaQbwAWBV1bUMR9IpwMXAHQARcbhO4YDqAjId2N1wu5cav+kGSOoCzgeerraSpNuAm4A3qi6kgLOBPmB1vkq4StLJVRfVqKqAqMl9td6dJmkScC+wPCIOVl1PM5IuA/ZFxOaqayloAnAB8KWIOB/4OVCr7dGqAtILzGy4PQPYU1Etw5I0kSwc6yOizuflzwcul9RDttp6iaR11ZaU1Av0RsRAj7yBLDC1UVVAngHOk3RWvmF2JfBgRbUkSRLZOvL2iPhC1fWkRMTNETEjIrrIluljEbG04rKGFBF7gd2SZud3LQRqtfOjkt8ojIgjkq4Fvgl0AHdGxLYqailgPnAV8F+Stub33RIRD1dYUzu5Dliff1DuAq6uuJ5j+Ei6WYKPpJslOCBmCQ6IWYIDYpbggJglOCBmCQ6IWYIDYpbw/5c43s99EQI6AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 216x216 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAMgAAADcCAYAAAA1H+4TAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAO20lEQVR4nO3dfbBcdX3H8fenNwiGJJPEQAfy4DUMkyl0AoSAMjiUBlJREflDKLRBobapf6BkRInSdsQ+CDgjQx2txYkgkgBDeBgQEWUKFBkFuTcGJAY6AQK5YMgNJBAelAa//eOc29lc9v723OSce87dfF4zO9nde85vvzm7n/2dx98qIjCz9v6o7gLMmswBMUtwQMwSHBCzBAfELMEBMUsYlwGRdL+kvx3reZtE0sWSVozRa82R9JqknrF4vSapNSCSNko6uc4ayqDM5ZJeym9fl6SC854r6e38A9h6Ozg1X0R8LSIqCfrw9yUinouISRHxdhWvtzskHSmpX9Ib+b9HVvE647IHaaClwOnAEcB84FTg70cx/y/yD2Dr7YUqCu0Gkt4F3A6sBKYB1wK358+XqpEBkTRN0p2SBiVty+/PGjbZIZJ+KekVSbdLmt4y/wck/VzSdkmPSjqx4pI/BXwjIgYi4nngG8C5ZTQsabmk5yXtkPSkpJPy5y+RtDK/3yspJJ0naVO+zD4j6RhJj+XL4VstbR4i6d68t9sqaZWkqfnfrgPmAD/Me7KLWtqfkE9zsKQ7JL0saYOkv2tp+xJJN0n6QV7zOkkLy1gWLU4EJgBXRsTvI+KbgIBFJb9OMwNCVtc1wHvJ3qw3gW8Nm+aTwN8ABwM7gW8CSJoJ/Aj4V2A68AXgFkkHdHpRSX+Vf5hGus0ZYdbDgUdbHj+aP7dHJM0DzgeOiYjJwIeAjYlZ3g8cCvwlcCXwD8DJeS1nSvqzoaaBS8mW3Z8As4FLACLiHOA54GN5T/b1Nq9zAzCQz/8J4GtDwc2dBtwITAXu4J3vXev/8bHE8v6PEWY7HHgsdj1P6jFKWObDTSi7wTJExEvALUOPJf0bcN+wya6LiMfzv/8TsFbSp4AlwF0RcVc+3T2S+oCPkHXFqde9Hrh+N0qeBLzS8vgVYJIkRbGT3T4gaXvL45ci4hDgbWBf4DBJgxGxsUM7/xIRvwN+Kul14IaI2AIg6WfAUcB/R8QGYEM+z6CkK4CvFKgTSbOBDwKn5q+1Nt9ZcA7wX/lkDw4t/7xHWjZSexExv8jrDjN8eZM/nrwbbSU1sgeRNFHSVZKelfQq8AAwddhelE0t958F9gFmkPU6Z7R+E5G9oQdVWPJrwJSWx1OA1wqGA+ChiJjacjsEIP8gLyP7dt8i6cYOG+8vttx/s83jSQCSDszbej5fvivJll0RBwMvR8SOlueeBWa2PN7ccv8NYL+h1bOSDF/e5I93tJl2jzQyIMCFwDzg/RExBTghf751z9DslvtzgP8FtpIF57phH7j9I+KyTi8q6a/b7E1qvY20irWObAN9yBH5c3ssIq6PiA+SBT+Ay0to9tK8rfn58l3Crss2FewXgOmSWr+t5wDP704h+TbKSMv7P0eYbR0wX9plT+F8SlrmrZoQkH0k7ddym0DWVb4JbM83vtt1/0skHSZpIvDPwM35bsiVwMckfUhST97miXrnRv47RMSqNnuTWm/PjTDrD4DPS5qZf8NfCHx/9ItiV5LmSVokaV/gd2TLpIxdrZPJvoW359tsXxz29xeBue1mjIhNwM+BS/NlOx/4NLBqdwqJiMMTy/szI8x2P9ly+JykfSWdnz9/7+7UkNKEgNxF9sYP3S4h28B8N1mP8BBwd5v5riP7EG4G9gM+B///Bn4cuBgYJOtRvki1/9ergB8CvwYeJ9tJcNUo5j+uzbfnMWTbH5eRLYfNwIFk/6899VVgAdl6+4+AW4f9/VLgH/NV1C+0mf9soJesN7kN+EpE3FNCXYVExFtku9U/CWwn21lzev58qeQLpsxG1oQexKyxHBCzBAfELMEBMUtwQMwSKjnVZMaMGdHb21tF06Xbtm1bJe0ODAyU3uaUKcMPHpdj1qyOh4h2S0/P+Lh8ZOPGjWzdurXt5QmVBKS3t5e+vr4qmi7d6tWrK2l3+fLlpbe5ePHi0tsEuOyyjicZ7JZp06ZV0m7ZFi4c+WRjr2KZJTggZgkOiFmCA2KW4ICYJRQKiKRTlF0PvUHSl6ouyqwpOgYkv4rv28CHgcOAsyUdVnVhZk1QpAc5FtgQEU/n59vfSHa9hVnXKxKQmex6/fcAu15/bNa1igSk3SH4d1xlJWmppD5JfYODg3temVkDFAnIALsOkDCL7FLLXUTEdyNiYUQsPOCAjkNQmY0LRQLyCHCopPcpG9rxLLLBwMy6XseTFSNiZz5qxE+AHuDqiCh9eBWzJip0Nm8+St5dHSc06zI+km6W4ICYJTggZgkOiFmCA2KW4ICYJTTyB3TGUhWDKwA888wzpbdZ1Qgs06dP7zzRbrjppptKb/OMM84ovc0U9yBmCQ6IWYIDYpbggJglOCBmCQ6IWUKRQRuulrRF0uNjUZBZkxTpQb4PnFJxHWaN1DEgEfEA8PIY1GLWOKVtg3jQButGpQXEgzZYN/JeLLMEB8Qsochu3huAXwDzJA1I+nT1ZZk1Q5Fhf84ei0LMmsirWGYJDohZggNiluCAmCU4IGYJ42rQhv7+/tLbrGJwBYCnnnqq9Dbnzp1bepsAixcvrqTdKt4vD9pg1iAOiFmCA2KW4ICYJTggZgkOiFlCkbN5Z0u6T9J6SeskXTAWhZk1QZHjIDuBCyNijaTJQL+keyLiNxXXZla7IoM2/DYi1uT3dwDrgZlVF2bWBKPaBpHUCxwFPFxFMWZNUzggkiYBtwDLIuLVNn/3qCbWdQoFRNI+ZOFYFRG3tpvGo5pYNyqyF0vA94D1EXFF9SWZNUeRHuR44BxgkaS1+e0jFddl1ghFBm14ENAY1GLWOD6SbpbggJglOCBmCQ6IWYIDYpbggJgljKtRTbZt21Z6mwsWLCi9TahuBJIqHH300XWX0FjuQcwSHBCzBAfELMEBMUtwQMwSHBCzhCLXg+wn6ZeSHs1HNfnqWBRm1gRFjoP8HlgUEa/lVxY+KOnHEfFQxbWZ1a7I9SABvJY/3Ce/RZVFmTVF0WvSeyStBbYA90SERzWxvUKhgETE2xFxJDALOFbSnw6fxqOaWDca1V6siNgO3A+c0uZvHtXEuk6RvVgHSJqa3383cDLwRNWFmTVBkb1YBwHXSuohC9RNEXFntWWZNUORvViPkQ03arbX8ZF0swQHxCzBATFLcEDMEhwQs4S9ftCGxYsXl97meFPFcgWYNm1aJe2OJfcgZgkOiFmCA2KW4ICYJTggZgkOiFnCaH4GukfSryT5TF7ba4ymB7kAWF9VIWZNVPSa9FnAR4EV1ZZj1ixFe5ArgYuAP1RYi1njFLnk9lRgS0T0d5jOgzZY1ynSgxwPnCZpI3AjsEjSyuETedAG60YdAxIRX46IWRHRC5wF3BsRSyqvzKwBfBzELGFUp7tHxP1k42KZ7RXcg5glOCBmCQ6IWYIDYpbggJglOCBmCeNqVJMqRsno70+eQdMoVY0+0tfXV0m7Z555ZiXtjiX3IGYJDohZggNiluCAmCU4IGYJDohZQqHdvPnFUjuAt4GdEbGwyqLMmmI0x0H+PCK2VlaJWQN5FcssoWhAAvippH5JS6ssyKxJiq5iHR8RL0g6ELhH0hMR8UDrBHlwlgLMmTOn5DLN6lGoB4mIF/J/twC3Ace2mcajmljXKTIu1v6SJg/dB/4CeLzqwsyaoMgq1h8Dt0kamv76iLi70qrMGqJjQCLiaeCIMajFrHG8m9cswQExS3BAzBIcELMEB8QswQExSxhXo5rMnTu39DarGtFj9erV46LNKi1fvrzuEvaYexCzBAfELMEBMUtwQMwSHBCzBAfELKFQQCRNlXSzpCckrZd0XNWFmTVB0eMg/w7cHRGfkPQuYGKFNZk1RseASJoCnACcCxARbwFvVVuWWTMUWcWaCwwC10j6laQV+aW3u5C0VFKfpL7BwcHSCzWrQ5GATAAWAN+JiKOA14EvDZ/IgzZYNyoSkAFgICIezh/fTBYYs67XMSARsRnYJGle/tRJwG8qrcqsIYruxfossCrfg/U0cF51JZk1R6GARMRawCO6217HR9LNEhwQswQHxCzBATFLcEDMEvb6QRsuv/zy0tuEagYsWLiwmh2J/f39lbTbDdyDmCU4IGYJDohZggNiluCAmCU4IGYJRX7Ec56ktS23VyUtG4vizOpW5DcKnwSOBJDUAzxP9lPQZl1vtKtYJwFPRcSzVRRj1jSjDchZwA1VFGLWRIUDkl9NeBrQ9kcqPKqJdaPR9CAfBtZExIvt/uhRTawbjSYgZ+PVK9vLFB2bdyKwGLi12nLMmqXooA1vAO+puBazxvGRdLMEB8QswQExS3BAzBIcELMEB8QsQRFRfqPSIFDkhMYZwNbSC6jOeKrXtRb33ohoe/pHJQEpSlJfRIybQbHHU72utRxexTJLcEDMEuoOyHdrfv3RGk/1utYS1LoNYtZ0dfcgZo1WW0AknSLpSUkbJL3jZ6WbQtJsSfdJWi9pnaQL6q6pE0k9+W/a31l3LZ1ImirpZklP5Mv4uLpralXLKlY+Osr/kF1jMgA8ApwdEY379VxJBwEHRcQaSZOBfuD0JtY6RNLnyX5TckpEnFp3PSmSrgV+FhEr8su6J0bE9rrrGlJXD3IssCEino6It4AbgY/XVEtSRPw2Itbk93cA64GZ9VY1MkmzgI8CK+qupRNJU4ATgO8BRMRbTQoH1BeQmcCmlscDNPhDN0RSL3AU8HC9lSRdCVwE/KHuQgqYCwwC1+SrhCsk7V93Ua3qCojaPNfo3WmSJgG3AMsi4tW662lH0qnAlogYL7+IMwFYAHwnIo4CXgcatT1aV0AGgNktj2cBL9RUS0eS9iELx6qIaPJ1+ccDp0naSLbaukjSynpLShoABiJiqEe+mSwwjVFXQB4BDpX0vnzD7CzgjppqSZIksnXk9RFxRd31pETElyNiVkT0ki3TeyNiSc1ljSgiNgObJM3LnzoJaNTOj1p+ozAidko6H/gJ0ANcHRHr6qilgOOBc4BfS1qbP3dxRNxVY03d5LPAqvyL8mngvJrr2YWPpJsl+Ei6WYIDYpbggJglOCBmCQ6IWYIDYpbggJglOCBmCf8HrSacUWpkmn0AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 216x216 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Randomly select digits and test.\n",
    "for index in np.random.choice(len(y_test), 2, replace = False):\n",
    "    print(index)\n",
    "    predicted = fitted_model.predict(X_test[index:index + 1])[0]\n",
    "    label = y_test[index]\n",
    "    title = \"Label = %d  Esimation = %d \" % (label, predicted)\n",
    "    fig = plt.figure(1, figsize = (3,3))\n",
    "    ax1 = fig.add_axes((0,0,.8,.8))\n",
    "    ax1.set_title(title)\n",
    "    plt.imshow(images[index], cmap = plt.cm.gray_r, interpolation = 'nearest')\n",
    "    plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 9. Référencement du modèle"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Meilleur modèle"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Run(Experiment: Exemple2-automlclassificationSR,\n",
      "Id: AutoML_99c06691-0a63-42f3-b16b-1d38b6d0f8ae_8,\n",
      "Type: None,\n",
      "Status: Completed)\n",
      "\n",
      "Pipeline(memory=None,\n",
      "     steps=[('datatransformer', DataTransformer(enable_dnn=None, enable_feature_sweeping=None,\n",
      "        feature_sweeping_config=None, feature_sweeping_timeout=None,\n",
      "        featurization_config=None, force_text_dnn=None,\n",
      "        is_cross_validation=None, is_onnx_compatible=None, logger=None,\n",
      "        obser... warm_start=False))]))],\n",
      "               flatten_transform=None, weights=[0.1, 0.3, 0.4, 0.1, 0.1]))])\n",
      "\n",
      "Métriques :\n",
      "matthews_correlation = 0.9777813954801045\n",
      "balanced_accuracy = 0.9801048297732556\n",
      "accuracy = 0.9799660193668762\n",
      "AUC_macro = 0.9997278321619597\n",
      "f1_score_weighted = 0.9800071252416492\n",
      "precision_score_weighted = 0.9806729693762462\n",
      "AUC_micro = 0.9997502627629037\n",
      "weighted_accuracy = 0.9797723436975737\n",
      "precision_score_micro = 0.9799660193668762\n",
      "f1_score_macro = 0.9802344671856597\n",
      "norm_macro_recall = 0.9778942553036174\n",
      "log_loss = 0.14778326607028566\n",
      "recall_score_macro = 0.9801048297732556\n",
      "f1_score_micro = 0.9799660193668762\n",
      "average_precision_score_weighted = 0.9979479476259953\n",
      "recall_score_micro = 0.9799660193668762\n",
      "average_precision_score_micro = 0.9979518516312101\n",
      "average_precision_score_macro = 0.997973084114332\n",
      "recall_score_weighted = 0.9799660193668762\n",
      "AUC_weighted = 0.9997226726657312\n",
      "precision_score_macro = 0.9809917709323183\n",
      "confusion_matrix = aml://artifactId/ExperimentRun/dcid.AutoML_99c06691-0a63-42f3-b16b-1d38b6d0f8ae_8/confusion_matrix\n",
      "accuracy_table = aml://artifactId/ExperimentRun/dcid.AutoML_99c06691-0a63-42f3-b16b-1d38b6d0f8ae_8/accuracy_table\n"
     ]
    }
   ],
   "source": [
    "best_run, fitted_model = local_run.get_output()\n",
    "print(best_run)\n",
    "print()\n",
    "print(fitted_model)\n",
    "print()\n",
    "print(\"Métriques :\")\n",
    "best_run_metrics = best_run.get_metrics()\n",
    "for metric_name in best_run_metrics:\n",
    "    metric = best_run_metrics[metric_name]\n",
    "    print(metric_name, '=', metric)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "datatransformer\n",
      "prefittedsoftvotingclassifier\n"
     ]
    }
   ],
   "source": [
    "for step in fitted_model.named_steps:\n",
    "    print(step)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Référencement du modèle"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Model(workspace=Workspace.create(name='workshopAML2020', subscription_id='70b8f39e-8863-49f7-b6ba-34a80799550c', resource_group='workshopAML2020-rg'), name=mybestmodel, id=mybestmodel:14, version=14, tags={'Training context': 'Azure Auto ML'}, properties={'AUC': '0.9997226726657312', 'Accuracy': '0.9799660193668762'})"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from azureml.core import Model\n",
    "\n",
    "best_run.register_model(model_path='outputs/model.pkl', model_name='mybestmodel',\n",
    "                        tags={'Training context':'Azure Auto ML'},\n",
    "                        properties={'AUC': best_run_metrics['AUC_weighted'], 'Accuracy': best_run_metrics['accuracy']})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "mybestmodel - version = 14\n",
      "\t Training context : Azure Auto ML\n",
      "\t AUC : 0.9997226726657312\n",
      "\t Accuracy : 0.9799660193668762\n",
      "\n",
      "\n",
      "nyc-taxi-automl-predictor - version = 1\n",
      "\t area : auto\n",
      "\t type : regression\n",
      "\n",
      "\n",
      "resnet50 - version = 10\n",
      "\t area : Image classification\n",
      "\t type : classification\n",
      "\n",
      "\n",
      "sklearn_regression_model.pkl - version = 18\n",
      "\t area : diabetes\n",
      "\t type : regression\n",
      "\n",
      "\n",
      "diabetesmodel - version = 3\n",
      "\t area : diabetes\n",
      "\t type : regression\n",
      "\n",
      "\n",
      "Modele-SKLEARN-Regression - version = 15\n",
      "\t area : diabetes\n",
      "\t type : regression\n",
      "\t format : Scikit-Learn pkl\n",
      "\n",
      "\n",
      "diabetesmodel - version = 2\n",
      "\t area : diabetes\n",
      "\t type : regression\n",
      "\n",
      "\n",
      "mnist_model - version = 6\n",
      "\t Source : Onnx\n",
      "\n",
      "\n",
      "sklearn_regression_model.pkl - version = 17\n",
      "\t area : diabetes\n",
      "\t type : regression\n",
      "\n",
      "\n",
      "sklearn_regression_model.pkl - version = 16\n",
      "\t area : diabetes\n",
      "\t type : regression\n",
      "\n",
      "\n",
      "diabetesmodel - version = 1\n",
      "\t area : diabetes\n",
      "\t type : regression\n",
      "\n",
      "\n",
      "my-sklearn-model - version = 2\n",
      "\t area : diabetes\n",
      "\t type : regression\n",
      "\n",
      "\n",
      "my-sklearn-model - version = 1\n",
      "\t area : diabetes\n",
      "\t type : regression\n",
      "\n",
      "\n",
      "sklearn_regression_model.pkl - version = 15\n",
      "\t area : diabetes\n",
      "\t type : regression\n",
      "\n",
      "\n",
      "Modele-SKLEARN-Regression - version = 14\n",
      "\t area : diabetes\n",
      "\t type : regression\n",
      "\t format : Scikit-Learn pkl\n",
      "\n",
      "\n",
      "mnist_model - version = 5\n",
      "\t Source : Onnx\n",
      "\n",
      "\n",
      "mybestmodel - version = 13\n",
      "\t Training context : Azure Auto ML\n",
      "\t AUC : 0.9997366937568698\n",
      "\t Accuracy : 0.9799660193668762\n",
      "\n",
      "\n",
      "Exemple3-AutoML-Regression - version = 15\n",
      "\t Training context : Azure Auto ML\n",
      "\t R2 : 0.8609905269903366\n",
      "\t RMSE : 0.07272566334894777\n",
      "\n",
      "\n",
      "Diabetes - version = 22\n",
      "\t Training context : Hyperdrive\n",
      "\t AUC : 0.8482780237822777\n",
      "\t Accuracy : 0.7743333333333333\n",
      "\n",
      "\n",
      "Modele-SKLEARN-Regression - version = 13\n",
      "\t area : diabetes\n",
      "\t type : regression\n",
      "\t format : Scikit-Learn pkl\n",
      "\n",
      "\n",
      "Exemple10-Modele-TensorFlow - version = 22\n",
      "\t Framework : TensorFlow\n",
      "\t Hyperdrive : Oui\n",
      "\t GPU : Oui\n",
      "\n",
      "\n",
      "diabetes_model - version = 28\n",
      "\t Training context : Pipeline\n",
      "\n",
      "\n",
      "IBM_attrition_explainer - version = 16\n",
      "\n",
      "\n",
      "local_deploy_model - version = 16\n",
      "\n",
      "\n",
      "Boston - version = 3\n",
      "\t algo : Regression\n",
      "\t Training context : Azure ML\n",
      "\t Framework : scikit-learn\n",
      "\n",
      "\n",
      "DiabetesRidge - version = 6\n",
      "\t area : Diabetes\n",
      "\t type : Regression Ridge\n",
      "\t k : 0.4\n",
      "\t MSE : 3295.74\n",
      "\t R2 : 0.36\n",
      "\t RMSE : 57.41\n",
      "\t Framework : Azure ML SDK\n",
      "\n",
      "\n",
      "Diabetes - version = 21\n",
      "\t Training context : Hyperdrive\n",
      "\t AUC : 0.8483337479476591\n",
      "\t Accuracy : 0.7743333333333333\n",
      "\n",
      "\n",
      "Diabetes - version = 20\n",
      "\t Training context : Hyperdrive\n",
      "\t AUC : 0.8569106291712714\n",
      "\t Accuracy : 0.7902222222222223\n",
      "\n",
      "\n",
      "modele2-bayesianridge - version = 4\n",
      "\n",
      "\n",
      "modele1-ridge - version = 4\n",
      "\n",
      "\n",
      "modele2-bayesianridge - version = 3\n",
      "\n",
      "\n",
      "modele1-ridge - version = 3\n",
      "\n",
      "\n",
      "modele2-bayesianridge - version = 2\n",
      "\n",
      "\n",
      "modele1-ridge - version = 2\n",
      "\n",
      "\n",
      "modele2-bayesianridge - version = 1\n",
      "\n",
      "\n",
      "modele1-ridge - version = 1\n",
      "\n",
      "\n",
      "Modele-SKLEARN-Regression - version = 12\n",
      "\t area : diabetes\n",
      "\t type : regression\n",
      "\t format : Scikit-Learn pkl\n",
      "\n",
      "\n",
      "Diabetes - version = 19\n",
      "\t Training context : Hyperdrive\n",
      "\t AUC : 0.8568509052814499\n",
      "\t Accuracy : 0.7891111111111111\n",
      "\n",
      "\n",
      "Modele-SKLEARN-Regression - version = 11\n",
      "\t area : diabetes\n",
      "\t type : regression\n",
      "\t format : Scikit-Learn pkl\n",
      "\n",
      "\n",
      "mnist_model - version = 4\n",
      "\t Source : Onnx\n",
      "\n",
      "\n",
      "Exemple10-Modele-TensorFlow - version = 21\n",
      "\t Framework : TensorFlow\n",
      "\t Hyperdrive : Oui\n",
      "\t GPU : Oui\n",
      "\n",
      "\n",
      "diabetes_model - version = 27\n",
      "\t Training context : Pipeline\n",
      "\n",
      "\n",
      "IBM_attrition_explainer - version = 15\n",
      "\n",
      "\n",
      "local_deploy_model - version = 15\n",
      "\n",
      "\n",
      "Exemple3-AutoML-Regression - version = 14\n",
      "\t Training context : Azure Auto ML\n",
      "\t R2 : 0.8652411523249226\n",
      "\t RMSE : 0.07199246230940634\n",
      "\n",
      "\n",
      "mybestmodel - version = 12\n",
      "\t Training context : Azure Auto ML\n",
      "\t AUC : 0.9997170443630855\n",
      "\t Accuracy : 0.9793770912161106\n",
      "\n",
      "\n",
      "DiabetesRidge - version = 5\n",
      "\t area : Diabetes\n",
      "\t type : Regression Ridge\n",
      "\t k : 0.4\n",
      "\t MSE : 3295.74\n",
      "\t R2 : 0.36\n",
      "\t RMSE : 57.41\n",
      "\t Framework : Azure ML SDK\n",
      "\n",
      "\n",
      "DiabetesRidge - version = 4\n",
      "\t area : Diabetes\n",
      "\t type : Regression Ridge\n",
      "\t k : 0.4\n",
      "\t MSE : 3295.74\n",
      "\t R2 : 0.36\n",
      "\t RMSE : 57.41\n",
      "\t Framework : Azure ML SDK\n",
      "\n",
      "\n",
      "mybestmodel - version = 11\n",
      "\t Training context : Azure Auto ML\n",
      "\t AUC : 0.9997284861812671\n",
      "\t Accuracy : 0.9793770912161106\n",
      "\n",
      "\n",
      "DiabetesRidge - version = 3\n",
      "\t area : Diabetes\n",
      "\t type : Regression Ridge\n",
      "\t k : 0.4\n",
      "\t MSE : 3295.74\n",
      "\t R2 : 0.36\n",
      "\t RMSE : 57.41\n",
      "\t Framework : Azure ML SDK\n",
      "\n",
      "\n",
      "Modele-SKLEARN-Regression - version = 10\n",
      "\t area : diabetes\n",
      "\t type : regression\n",
      "\t format : Scikit-Learn pkl\n",
      "\n",
      "\n",
      "resnet50 - version = 5\n",
      "\t area : Image classification\n",
      "\t type : classification\n",
      "\n",
      "\n",
      "Modele-SKLEARN-Regression - version = 9\n",
      "\t area : diabetes\n",
      "\t type : regression\n",
      "\t format : Scikit-Learn pkl\n",
      "\n",
      "\n",
      "Modele-SKLEARN-Regression - version = 8\n",
      "\t area : diabetes\n",
      "\t type : regression\n",
      "\t format : Scikit-Learn pkl\n",
      "\n",
      "\n",
      "DiabetesRidge - version = 2\n",
      "\t area : Diabetes\n",
      "\t type : Regression Ridge\n",
      "\t k : 0.4\n",
      "\t MSE : 3295.74\n",
      "\t R2 : 0.36\n",
      "\t RMSE : 57.41\n",
      "\t Framework : Azure ML SDK\n",
      "\n",
      "\n",
      "Exemple10-Modele-TensorFlow - version = 20\n",
      "\t Framework : TensorFlow\n",
      "\t Hyperdrive : Oui\n",
      "\t GPU : Oui\n",
      "\n",
      "\n",
      "Diabetes - version = 18\n",
      "\t Training context : Hyperdrive\n",
      "\t AUC : 0.8568509052814499\n",
      "\t Accuracy : 0.7891111111111111\n",
      "\n",
      "\n",
      "diabetes_model - version = 26\n",
      "\t Training context : Pipeline\n",
      "\n",
      "\n",
      "Modele-SKLEARN-Regression - version = 7\n",
      "\t area : diabetes\n",
      "\t type : regression\n",
      "\t format : Scikit-Learn pkl\n",
      "\n",
      "\n",
      "mybestmodel - version = 10\n",
      "\t Training context : Azure Auto ML\n",
      "\t AUC : 0.9997297175014267\n",
      "\t Accuracy : 0.9793770912161106\n",
      "\n",
      "\n",
      "IBM_attrition_explainer - version = 14\n",
      "\n",
      "\n",
      "local_deploy_model - version = 14\n",
      "\n",
      "\n",
      "mnist_model - version = 3\n",
      "\t Source : Onnx\n",
      "\n",
      "\n",
      "Exemple3-AutoML-Regression - version = 13\n",
      "\t Training context : Azure Auto ML\n",
      "\t R2 : 0.8691026010014283\n",
      "\t RMSE : 0.07046677416066262\n",
      "\n",
      "\n",
      "DiabetesRidge - version = 1\n",
      "\t area : Diabetes\n",
      "\t type : Regression Ridge\n",
      "\t k : 0.4\n",
      "\t MSE : 3295.74\n",
      "\t R2 : 0.36\n",
      "\t RMSE : 57.41\n",
      "\t Framework : Azure ML SDK\n",
      "\n",
      "\n",
      "RegressionRidge - version = 12\n",
      "\t area : Diabetes\n",
      "\t type : Regression Ridge\n",
      "\t k : 0.4\n",
      "\t MSE : 3295.74\n",
      "\t R2 : 0.36\n",
      "\t RMSE : 57.41\n",
      "\t Framework : Azure ML SDK\n",
      "\n",
      "\n",
      "RegressionRidgeML - version = 6\n",
      "\t area : Diabetes\n",
      "\t type : Regression Ridge\n",
      "\t k : 0.4\n",
      "\t MSE : 3295.74\n",
      "\t R2 : 0.36\n",
      "\t RMSE : 57.41\n",
      "\t Framework : Azure ML SDK\n",
      "\n",
      "\n",
      "mnist_model - version = 2\n",
      "\t Source : Onnx\n",
      "\n",
      "\n",
      "mnist_model - version = 1\n",
      "\t Source : Onnx\n",
      "\n",
      "\n",
      "sklearn_regression_model - version = 1\n",
      "\t area : diabetes\n",
      "\t type : regression\n",
      "\n",
      "\n",
      "best_model - version = 2\n",
      "\n",
      "\n",
      "Boston - version = 2\n",
      "\t algo : Regression\n",
      "\t Training context : Azure ML\n",
      "\t Framework : scikit-learn\n",
      "\n",
      "\n",
      "RegressionRidgeML - version = 5\n",
      "\t area : Diabetes\n",
      "\t type : Regression Ridge\n",
      "\t k : 0.4\n",
      "\t MSE : 3295.74\n",
      "\t R2 : 0.36\n",
      "\t RMSE : 57.41\n",
      "\t Framework : Azure ML SDK\n",
      "\n",
      "\n",
      "best_model - version = 1\n",
      "\n",
      "\n",
      "Boston - version = 1\n",
      "\t algo : Regression\n",
      "\t Training context : Azure ML\n",
      "\t Framework : scikit-learn\n",
      "\n",
      "\n",
      "AutoML627c037107 - version = 1\n",
      "\n",
      "\n",
      "boston_model.pkl - version = 20\n",
      "\t algo : Regression\n",
      "\t Training context : Azure ML\n",
      "\t Framework : scikit-learn\n",
      "\n",
      "\n",
      "diabetes_model - version = 25\n",
      "\t Training context : Pipeline\n",
      "\n",
      "\n",
      "boston_model.pkl - version = 19\n",
      "\t algo : Regression\n",
      "\t Training context : Azure ML\n",
      "\t Framework : scikit-learn\n",
      "\n",
      "\n",
      "Modele-SKLEARN-Regression - version = 6\n",
      "\t area : diabetes\n",
      "\t type : regression\n",
      "\t format : Scikit-Learn pkl\n",
      "\n",
      "\n",
      "Exemple3-AutoML-Regression - version = 12\n",
      "\t Training context : Azure Auto ML\n",
      "\t R2 : 0.8609905269903366\n",
      "\t RMSE : 0.07272566334894777\n",
      "\n",
      "\n",
      "boston_model.pkl - version = 18\n",
      "\t algo : Regression\n",
      "\t Training context : Azure ML\n",
      "\t Framework : scikit-learn\n",
      "\n",
      "\n",
      "Diabetes - version = 17\n",
      "\t Training context : Hyperdrive\n",
      "\t AUC : 0.8568509052814499\n",
      "\t Accuracy : 0.7891111111111111\n",
      "\n",
      "\n",
      "Exemple10-Modele-TensorFlow - version = 19\n",
      "\t Framework : TensorFlow\n",
      "\t Hyperdrive : Oui\n",
      "\t GPU : Oui\n",
      "\n",
      "\n",
      "diabetes_model - version = 24\n",
      "\t Training context : Pipeline\n",
      "\n",
      "\n",
      "Modele-SKLEARN-Regression - version = 5\n",
      "\t area : diabetes\n",
      "\t type : regression\n",
      "\t format : Scikit-Learn pkl\n",
      "\n",
      "\n",
      "mybestmodel - version = 9\n",
      "\t Training context : Azure Auto ML\n",
      "\t AUC : 0.999730946696694\n",
      "\t Accuracy : 0.9793770912161106\n",
      "\n",
      "\n",
      "boston_model.pkl - version = 17\n",
      "\t algo : Regression\n",
      "\t Training context : Azure ML\n",
      "\t Framework : scikit-learn\n",
      "\n",
      "\n",
      "IBM_attrition_explainer - version = 13\n",
      "\n",
      "\n",
      "local_deploy_model - version = 13\n",
      "\n",
      "\n",
      "RegressionRidgeML - version = 4\n",
      "\t area : Diabetes\n",
      "\t type : Regression Ridge\n",
      "\t k : 0.4\n",
      "\t MSE : 3295.74\n",
      "\t R2 : 0.36\n",
      "\t RMSE : 57.41\n",
      "\t Framework : Azure ML SDK\n",
      "\n",
      "\n",
      "diabetes_model - version = 23\n",
      "\t Training context : Pipeline\n",
      "\n",
      "\n",
      "diabetes_model - version = 22\n",
      "\t Training context : Pipeline\n",
      "\n",
      "\n",
      "diabetes_model - version = 21\n",
      "\t Training context : Pipeline\n",
      "\n",
      "\n",
      "diabetes_model - version = 20\n",
      "\t Training context : Pipeline\n",
      "\n",
      "\n",
      "diabetes_model - version = 19\n",
      "\t Training context : Pipeline\n",
      "\n",
      "\n",
      "diabetes_model - version = 18\n",
      "\t Training context : Pipeline\n",
      "\n",
      "\n",
      "RegressionRidgeML - version = 3\n",
      "\t area : Diabetes\n",
      "\t type : Regression Ridge\n",
      "\t k : 0.4\n",
      "\t MSE : 3295.74\n",
      "\t R2 : 0.36\n",
      "\t RMSE : 57.41\n",
      "\t Framework : Azure ML SDK\n",
      "\n",
      "\n",
      "Defect - version = 5\n",
      "\t Training context : Azure Auto ML SDK\n",
      "\t AUC : 0.7744069264069265\n",
      "\t Accuracy : 0.8869565217391304\n",
      "\n",
      "\n",
      "amlstudio-defect-real-time - version = 1\n",
      "\t CreatedByAMLStudio : true\n",
      "\n",
      "\n",
      "census_gs_model_32 - version = 3\n",
      "\n",
      "\n",
      "census_gs_model_31 - version = 3\n",
      "\n",
      "\n",
      "census_gs_model_30 - version = 3\n",
      "\n",
      "\n",
      "census_gs_model_35 - version = 2\n",
      "\n",
      "\n",
      "census_gs_model_34 - version = 2\n",
      "\n",
      "\n",
      "census_gs_model_33 - version = 2\n",
      "\n",
      "\n",
      "census_gs_model_32 - version = 2\n",
      "\n",
      "\n",
      "census_gs_model_31 - version = 2\n",
      "\n",
      "\n",
      "census_gs_model_30 - version = 2\n",
      "\n",
      "\n",
      "census_unmitigated - version = 2\n",
      "\n",
      "\n",
      "RegressionRidgeML - version = 2\n",
      "\t area : Diabetes\n",
      "\t type : Regression Ridge\n",
      "\t k : 0.4\n",
      "\t MSE : 3295.74\n",
      "\t R2 : 0.36\n",
      "\t RMSE : 57.41\n",
      "\t Framework : Azure ML SDK\n",
      "\n",
      "\n",
      "census_gs_model_35 - version = 1\n",
      "\n",
      "\n",
      "census_gs_model_34 - version = 1\n",
      "\n",
      "\n",
      "census_gs_model_33 - version = 1\n",
      "\n",
      "\n",
      "census_gs_model_32 - version = 1\n",
      "\n",
      "\n",
      "census_gs_model_31 - version = 1\n",
      "\n",
      "\n",
      "census_gs_model_30 - version = 1\n",
      "\n",
      "\n",
      "census_unmitigated - version = 1\n",
      "\n",
      "\n",
      "Defect - version = 4\n",
      "\t Training context : Azure Auto ML SDK\n",
      "\t AUC : 0.7744069264069265\n",
      "\t Accuracy : 0.8869565217391304\n",
      "\n",
      "\n",
      "Defect - version = 3\n",
      "\t Training context : Azure Auto ML SDK\n",
      "\t AUC : 0.7744069264069265\n",
      "\t Accuracy : 0.8869565217391304\n",
      "\n",
      "\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "amlstudio-defect-real-time-inf - version = 1\n",
      "\t CreatedByAMLStudio : true\n",
      "\n",
      "\n",
      "Defect - version = 2\n",
      "\t Training context : Azure Auto ML SDK\n",
      "\t AUC : 0.778055275301434\n",
      "\t Accuracy : 0.8956521739130435\n",
      "\n",
      "\n",
      "Defect - version = 1\n",
      "\t Training context : Azure Auto ML SDK\n",
      "\t AUC : 0.778055275301434\n",
      "\t Accuracy : 0.8956521739130435\n",
      "\n",
      "\n",
      "RegressionRidgeML - version = 1\n",
      "\t area : Diabetes\n",
      "\t type : Regression Ridge\n",
      "\t k : 0.4\n",
      "\t MSE : 3295.741064355809\n",
      "\t R2 : 0.3572956390661659\n",
      "\t RMSE : 57.40854522068826\n",
      "\t Framework : Azure ML SDK\n",
      "\n",
      "\n",
      "RegressionRidge - version = 11\n",
      "\t area : Diabetes\n",
      "\t type : Regression Ridge\n",
      "\t k : 0.4\n",
      "\t MSE : 3295.741064355809\n",
      "\t R2 : 0.3572956390661659\n",
      "\t RMSE : 57.40854522068826\n",
      "\t Framework : Azure ML SDK\n",
      "\n",
      "\n",
      "RegressionRidge - version = 10\n",
      "\t area : Diabetes\n",
      "\t type : Regression Ridge\n",
      "\t k : 0.4\n",
      "\t MSE : 3295.741064355809\n",
      "\t R2 : 0.3572956390661659\n",
      "\t RMSE : 57.40854522068826\n",
      "\t Framework : Azure ML SDK\n",
      "\n",
      "\n",
      "IBM_attrition_explainer - version = 12\n",
      "\n",
      "\n",
      "local_deploy_model - version = 12\n",
      "\n",
      "\n",
      "Diabetes - version = 16\n",
      "\t Training context : Hyperdrive\n",
      "\t AUC : 0.8569106291712714\n",
      "\t Accuracy : 0.7902222222222223\n",
      "\n",
      "\n",
      "Exemple10-Modele-TensorFlow - version = 18\n",
      "\t Framework : TensorFlow\n",
      "\t Hyperdrive : Oui\n",
      "\t GPU : Oui\n",
      "\n",
      "\n",
      "diabetes_model - version = 17\n",
      "\t Training context : Pipeline\n",
      "\n",
      "\n",
      "Exemple3-AutoML-Regression - version = 11\n",
      "\t Training context : Azure Auto ML\n",
      "\t R2 : 0.8609905269903366\n",
      "\t RMSE : 0.07272566334894777\n",
      "\n",
      "\n",
      "Modele-SKLEARN-Regression - version = 4\n",
      "\t area : diabetes\n",
      "\t type : regression\n",
      "\t format : Scikit-Learn pkl\n",
      "\n",
      "\n",
      "Exemple4-AutoML-Forecast - version = 7\n",
      "\t Training context : Azure Auto ML\n",
      "\t R2 : 0.27268956675549244\n",
      "\t RMSE : 0.025273757999049212\n",
      "\n",
      "\n",
      "boston_model.pkl - version = 16\n",
      "\t algo : Regression\n",
      "\t Training context : Azure ML\n",
      "\t Framework : scikit-learn\n",
      "\n",
      "\n",
      "mybestmodel - version = 8\n",
      "\t Training context : Azure Auto ML\n",
      "\t AUC : 0.9993227376446444\n",
      "\t Accuracy : 0.9634718617426019\n",
      "\n",
      "\n",
      "RegressionRidge - version = 9\n",
      "\t area : Diabetes\n",
      "\t type : Regression Ridge\n",
      "\t k : 0.4\n",
      "\t MSE : 3295.741064355809\n",
      "\t R2 : 0.3572956390661659\n",
      "\t Framework : Azure ML\n",
      "\n",
      "\n",
      "noaaweatherds - version = 1\n",
      "\n",
      "\n",
      "Exemple4-AutoML-Forecast - version = 6\n",
      "\t Training context : Azure Auto ML\n",
      "\t R2 : 0.27268956675549244\n",
      "\t RMSE : 0.025273757999049212\n",
      "\n",
      "\n",
      "Exemple3-AutoML-Regression - version = 10\n",
      "\t Training context : Azure Auto ML\n",
      "\t R2 : 0.8609905269903366\n",
      "\t RMSE : 0.07272566334894777\n",
      "\n",
      "\n",
      "mybestmodel - version = 7\n",
      "\t Training context : Azure Auto ML\n",
      "\t AUC : 0.9993169930532767\n",
      "\t Accuracy : 0.9658275743456644\n",
      "\n",
      "\n",
      "RegressionRidge - version = 8\n",
      "\t area : Diabetes\n",
      "\t type : Regression Ridge\n",
      "\t k : 0.4\n",
      "\t MSE : 3295.741064355809\n",
      "\t R2 : 0.3572956390661659\n",
      "\t Framework : Azure ML\n",
      "\n",
      "\n",
      "IBM_attrition_explainer - version = 11\n",
      "\n",
      "\n",
      "local_deploy_model - version = 11\n",
      "\n",
      "\n",
      "boston_model.pkl - version = 15\n",
      "\t algo : Regression\n",
      "\t Training context : Azure ML\n",
      "\t Framework : scikit-learn\n",
      "\n",
      "\n",
      "Exemple10-Modele-TensorFlow - version = 17\n",
      "\t Framework : TensorFlow\n",
      "\t Hyperdrive : Oui\n",
      "\t GPU : Oui\n",
      "\n",
      "\n",
      "diabetes_model - version = 16\n",
      "\t Training context : Pipeline\n",
      "\n",
      "\n",
      "Modele-SKLEARN-Regression - version = 3\n",
      "\t area : diabetes\n",
      "\t type : regression\n",
      "\t format : Scikit-Learn pkl\n",
      "\n",
      "\n",
      "mybestmodel - version = 6\n",
      "\t Training context : Azure Auto ML\n",
      "\t AUC : 0.9993067801973444\n",
      "\t Accuracy : 0.962881891241127\n",
      "\n",
      "\n",
      "boston_model.pkl - version = 14\n",
      "\t algo : Regression\n",
      "\t Training context : Azure ML\n",
      "\t Framework : scikit-learn\n",
      "\n",
      "\n",
      "Exemple3-AutoML-Regression - version = 9\n",
      "\t Training context : Azure Auto ML\n",
      "\t R2 : 0.8267547975259869\n",
      "\t RMSE : 0.08250675651473231\n",
      "\n",
      "\n",
      "mybestmodel - version = 5\n",
      "\t Training context : Azure Auto ML\n",
      "\t AUC : 0.9993067801973444\n",
      "\t Accuracy : 0.962881891241127\n",
      "\n",
      "\n",
      "RegressionRidge - version = 7\n",
      "\t area : Diabetes\n",
      "\t type : Regression Ridge\n",
      "\t k : 0.4\n",
      "\t MSE : 3295.741064355809\n",
      "\t R2 : 0.3572956390661659\n",
      "\t Framework : Azure ML\n",
      "\n",
      "\n",
      "model_explain_model_on_amlcomp - version = 1\n",
      "\n",
      "\n",
      "IBM_attrition_explainer - version = 10\n",
      "\n",
      "\n",
      "local_deploy_model - version = 10\n",
      "\n",
      "\n",
      "IBM_attrition_explainer - version = 9\n",
      "\n",
      "\n",
      "local_deploy_model - version = 9\n",
      "\n",
      "\n",
      "Diabetes - version = 15\n",
      "\t Training context : Hyperdrive\n",
      "\t AUC : 0.8569106291712714\n",
      "\t Accuracy : 0.7902222222222223\n",
      "\n",
      "\n",
      "Exemple10-Modele-TensorFlow - version = 16\n",
      "\t Framework : TensorFlow\n",
      "\t Hyperdrive : Oui\n",
      "\t GPU : Oui\n",
      "\n",
      "\n",
      "Diabetes - version = 14\n",
      "\t Training context : Hyperdrive\n",
      "\t AUC : 0.8569106291712714\n",
      "\t Accuracy : 0.7902222222222223\n",
      "\n",
      "\n",
      "Exemple10-Modele-TensorFlow - version = 15\n",
      "\t Framework : TensorFlow\n",
      "\t Hyperdrive : Oui\n",
      "\t GPU : Oui\n",
      "\n",
      "\n",
      "diabetes_model - version = 15\n",
      "\t Training context : Pipeline\n",
      "\n",
      "\n",
      "Modele-SKLEARN-Regression - version = 2\n",
      "\t area : diabetes\n",
      "\t type : regression\n",
      "\t format : Scikit-Learn pkl\n",
      "\n",
      "\n",
      "Exemple4-AutoML-Forecast - version = 5\n",
      "\t Training context : Azure Auto ML\n",
      "\t R2 : 0.2102297702988311\n",
      "\t RMSE : 0.025719636958220327\n",
      "\n",
      "\n",
      "boston_model.pkl - version = 13\n",
      "\t algo : Regression\n",
      "\t Training context : Azure ML\n",
      "\t Framework : scikit-learn\n",
      "\n",
      "\n",
      "RegressionRidge - version = 6\n",
      "\t area : Diabetes\n",
      "\t type : Regression Ridge\n",
      "\t k : 0.4\n",
      "\t MSE : 3295.741064355809\n",
      "\t R2 : 0.3572956390661659\n",
      "\t Framework : Azure ML\n",
      "\n",
      "\n",
      "RegressionRidge - version = 5\n",
      "\t area : Diabetes\n",
      "\t type : Regression Ridge\n",
      "\t k : 0.4\n",
      "\t MSE : 3295.741064355809\n",
      "\t R2 : 0.3572956390661659\n",
      "\t Framework : Azure ML\n",
      "\n",
      "\n",
      "Exemple3-AutoML-Regression - version = 8\n",
      "\t Training context : Azure Auto ML\n",
      "\t R2 : 0.8267547975259869\n",
      "\t RMSE : 0.08250675651473231\n",
      "\n",
      "\n",
      "mybestmodel - version = 4\n",
      "\t Training context : Azure Auto ML\n",
      "\t AUC : 0.9993067801973444\n",
      "\t Accuracy : 0.962881891241127\n",
      "\n",
      "\n",
      "Diabetes - version = 13\n",
      "\t Training context : Hyperdrive\n",
      "\t AUC : 0.8569106291712714\n",
      "\t Accuracy : 0.7902222222222223\n",
      "\n",
      "\n",
      "Exemple10-Modele-TensorFlow - version = 14\n",
      "\t Framework : TensorFlow\n",
      "\t Hyperdrive : Oui\n",
      "\t GPU : Oui\n",
      "\n",
      "\n",
      "IBM_attrition_explainer - version = 8\n",
      "\n",
      "\n",
      "local_deploy_model - version = 8\n",
      "\n",
      "\n",
      "diabetes_model - version = 14\n",
      "\t Training context : Pipeline\n",
      "\n",
      "\n",
      "Modele-SKLEARN-Regression - version = 1\n",
      "\t area : diabetes\n",
      "\t type : regression\n",
      "\t format : Scikit-Learn pkl\n",
      "\n",
      "\n",
      "boston_model.pkl - version = 12\n",
      "\t algo : Regression\n",
      "\t Training context : Azure ML\n",
      "\t Framework : scikit-learn\n",
      "\n",
      "\n",
      "RegressionRidge - version = 4\n",
      "\t area : Diabetes\n",
      "\t type : Regression Ridge\n",
      "\t k : 0.3\n",
      "\t MSE : 3077.2\n",
      "\t Framework : Azure ML\n",
      "\n",
      "\n",
      "resnet50.4.accelonnx - version = 1\n",
      "\t parentModelName : resnet50\n",
      "\t parentModelFlavor : TF\n",
      "\t modelFlavor : AccelOnnx\n",
      "\t parentModelArtifactLocation : aml://artifact/LocalUpload/200430T125953-55db4332/resnet50\n",
      "\t convertModelLogLocation : aml://artifact/LocalUpload/119439fb2e6c41d39b0e5ecaa1e1bf77/conversion_log\n",
      "\t compileOptions_toolVersion : 1.0\n",
      "\t compileOptions_toolName : fpga\n",
      "\t compileOptions_input_node_names : Placeholder:0\n",
      "\t compileOptions_output_node_names : classifier/resnet_v1_50/predictions/Softmax:0\n",
      "\t compileOptions_require_fpga_conversion : True\n",
      "\t compileOptions_sourceModelFlavor : tf\n",
      "\t compileOptions_targetModelFlavor : accelonnx\n",
      "\t compileOptions_unpack : false\n",
      "\n",
      "\n",
      "resnet50 - version = 4\n",
      "\n",
      "\n",
      "RegressionRidge - version = 3\n",
      "\t area : Diabetes\n",
      "\t type : Regression Ridge\n",
      "\t k : 0.4\n",
      "\t MSE : 3295.74\n",
      "\t Framework : Azure ML\n",
      "\n",
      "\n",
      "RegressionRidge - version = 2\n",
      "\t area : Diabetes\n",
      "\t type : Regression Ridge\n",
      "\t k : 0.4\n",
      "\t MSE : 3295.74\n",
      "\t Framework : Azure ML\n",
      "\n",
      "\n",
      "RegressionRidge - version = 1\n",
      "\t area : Diabetes\n",
      "\t type : Regression Ridge\n",
      "\t k : 0.4\n",
      "\t MSE : 3295.74\n",
      "\t Framework : Azure ML\n",
      "\n",
      "\n",
      "resnet50 - version = 2\n",
      "\n",
      "\n",
      "ssdvgg.1.accelonnx - version = 1\n",
      "\t parentModelName : ssdvgg\n",
      "\t parentModelFlavor : TF\n",
      "\t modelFlavor : AccelOnnx\n",
      "\t parentModelArtifactLocation : aml://artifact/LocalUpload/200429T124534-4846cffe/saved_model\n",
      "\t convertModelLogLocation : aml://artifact/LocalUpload/ffaee3a06dba4e87a9ae84524cd868e7/conversion_log\n",
      "\t compileOptions_toolVersion : 1.0\n",
      "\t compileOptions_toolName : fpga\n",
      "\t compileOptions_input_node_names : Placeholder:0\n",
      "\t compileOptions_output_node_names : ssd_300_vgg/block4_box/Reshape_1:0,ssd_300_vgg/block7_box/Reshape_1:0,ssd_300_vgg/block8_box/Reshape_1:0,ssd_300_vgg/block9_box/Reshape_1:0,ssd_300_vgg/block10_box/Reshape_1:0,ssd_300_vgg/block11_box/Reshape_1:0,ssd_300_vgg/block4_box/Reshape:0,ssd_300_vgg/block7_box/Reshape:0,ssd_300_vgg/block8_box/Reshape:0,ssd_300_vgg/block9_box/Reshape:0,ssd_300_vgg/block10_box/Reshape:0,ssd_300_vgg/block11_box/Reshape:0\n",
      "\t compileOptions_require_fpga_conversion : True\n",
      "\t compileOptions_sourceModelFlavor : tf\n",
      "\t compileOptions_targetModelFlavor : accelonnx\n",
      "\t compileOptions_unpack : false\n",
      "\n",
      "\n",
      "ssdvgg - version = 1\n",
      "\n",
      "\n",
      "elevation-regression-model.pkl - version = 1\n",
      "\t area : weather\n",
      "\t type : linear regression\n",
      "\n",
      "\n",
      "MLTemplateAzure-master - version = 4\n",
      "\n",
      "\n",
      "MLTemplateAzure-master - version = 3\n",
      "\n",
      "\n",
      "MLTemplateAzure-master - version = 2\n",
      "\n",
      "\n",
      "MLTemplateAzure-master - version = 1\n",
      "\n",
      "\n",
      "diabetes_model - version = 13\n",
      "\t Training context : Pipeline\n",
      "\n",
      "\n",
      "diabetes_model - version = 12\n",
      "\t Training context : Pipeline\n",
      "\n",
      "\n",
      "diabetes_model - version = 11\n",
      "\t Training context : Pipeline\n",
      "\n",
      "\n",
      "Exemple10-Modele-TensorFlow - version = 13\n",
      "\t Framework : TensorFlow\n",
      "\t Hyperdrive : Oui\n",
      "\t GPU : Oui\n",
      "\n",
      "\n",
      "Exemple10-Modele-TensorFlow - version = 12\n",
      "\t Training context : TensorFlow Hyperdrive\n",
      "\t GPU : Oui\n",
      "\n",
      "\n",
      "Diabetes - version = 12\n",
      "\t Training context : Hyperdrive\n",
      "\t AUC : 0.8569106291712714\n",
      "\t Accuracy : 0.7902222222222223\n",
      "\n",
      "\n",
      "diabetes_model - version = 10\n",
      "\t Training context : Pipeline\n",
      "\n",
      "\n",
      "sklearn_regression_model.pkl - version = 14\n",
      "\t area : diabetes\n",
      "\t type : regression\n",
      "\t format : Scikit-Learn pkl\n",
      "\n",
      "\n",
      "diabetes_model - version = 9\n",
      "\t Training context : Pipeline\n",
      "\n",
      "\n",
      "boston_model.pkl - version = 11\n",
      "\t algo : Regression\n",
      "\t Training context : Azure ML\n",
      "\t Framework : scikit-learn\n",
      "\n",
      "\n",
      "mybestmodel - version = 3\n",
      "\t Training context : Azure Auto ML\n",
      "\t AUC : 0.9993193003206503\n",
      "\t Accuracy : 0.9658275743456644\n",
      "\n",
      "\n",
      "Exemple3-AutoML-Regression - version = 7\n",
      "\t Training context : Azure Auto ML\n",
      "\t R2 : 0.8736789106695474\n",
      "\t RMSE : 0.07225365701425089\n",
      "\n",
      "\n",
      "Exemple3-AutoML-Regression - version = 6\n",
      "\t Training context : Azure Auto ML\n",
      "\t R2 : 0.8736789106695474\n",
      "\t RMSE : 0.07225365701425089\n",
      "\n",
      "\n",
      "diabetes_model - version = 8\n",
      "\t Training context : Pipeline\n",
      "\n",
      "\n",
      "Exemple10-Modele-TensorFlow - version = 11\n",
      "\t Training context : TensorFlow GPU Hyperdrive\n",
      "\n",
      "\n",
      "IBM_attrition_explainer - version = 7\n",
      "\n",
      "\n",
      "local_deploy_model - version = 7\n",
      "\n",
      "\n",
      "Diabetes - version = 11\n",
      "\t Training context : Hyperdrive\n",
      "\t AUC : 0.8569106291712714\n",
      "\t Accuracy : 0.7902222222222223\n",
      "\n",
      "\n",
      "sklearn_regression_model.pkl - version = 13\n",
      "\t area : diabetes\n",
      "\t type : regression\n",
      "\t format : Scikit-Learn pkl\n",
      "\n",
      "\n",
      "sklearn_regression_model.pkl - version = 12\n",
      "\t area : diabetes\n",
      "\t type : regression\n",
      "\t format : Scikit-Learn pkl\n",
      "\n",
      "\n",
      "sklearn_regression_model.pkl - version = 11\n",
      "\t area : diabetes\n",
      "\t type : regression\n",
      "\t format : Scikit-Learn pkl\n",
      "\n",
      "\n",
      "sklearn_regression_model.pkl - version = 10\n",
      "\t area : diabetes\n",
      "\t type : regression\n",
      "\t format : Scikit-Learn pkl\n",
      "\n",
      "\n",
      "sklearn_regression_model.pkl - version = 9\n",
      "\t area : diabetes\n",
      "\t type : regression\n",
      "\t format : Scikit-Learn pkl\n",
      "\n",
      "\n",
      "sklearn_regression_model.pkl - version = 8\n",
      "\t area : diabetes\n",
      "\t type : regression\n",
      "\t format : Scikit-Learn pkl\n",
      "\n",
      "\n",
      "sklearn_regression_model.pkl - version = 7\n",
      "\t area : diabetes\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\t type : regression\n",
      "\n",
      "\n",
      "diabetes_model - version = 7\n",
      "\t Training context : Pipeline\n",
      "\n",
      "\n",
      "diabetes_model - version = 6\n",
      "\t Training context : Pipeline\n",
      "\n",
      "\n",
      "Exemple10-Modele-TensorFlow - version = 10\n",
      "\t Training context : TensorFlow GPU Hyperdrive\n",
      "\n",
      "\n",
      "Diabetes - version = 10\n",
      "\t Training context : Hyperdrive\n",
      "\t AUC : 0.8569106291712714\n",
      "\t Accuracy : 0.7902222222222223\n",
      "\n",
      "\n",
      "diabetes_model - version = 5\n",
      "\t Training context : Pipeline\n",
      "\n",
      "\n",
      "sklearn_regression_model.pkl - version = 6\n",
      "\t area : diabetes\n",
      "\t type : regression\n",
      "\n",
      "\n",
      "mybestmodel - version = 2\n",
      "\t Training context : Azure Auto ML\n",
      "\t AUC : 0.9994706712034915\n",
      "\t Accuracy : 0.9705396494881139\n",
      "\n",
      "\n",
      "mybestmodel - version = 1\n",
      "\t Training context : Azure Auto ML\n",
      "\t AUC : 0.9994706712034915\n",
      "\t Accuracy : 0.9705396494881139\n",
      "\n",
      "\n",
      "Exemple10-Modele-TensorFlow - version = 9\n",
      "\t Training context : TensorFlow GPU Hyperdrive\n",
      "\n",
      "\n",
      "Exemple10-Modele-TensorFlow - version = 8\n",
      "\t Training context : TensorFlow GPU Hyperdrive\n",
      "\n",
      "\n",
      "Exemple10-Modele-TensorFlow - version = 7\n",
      "\t Training context : TensorFlow GPU Hyperdrive\n",
      "\n",
      "\n",
      "Exemple10-Modele-TensorFlow - version = 6\n",
      "\t Training context : TensorFlow GPU\n",
      "\n",
      "\n",
      "Diabetes - version = 9\n",
      "\t Training context : Hyperdrive\n",
      "\t AUC : 0.8569106291712714\n",
      "\t Accuracy : 0.7902222222222223\n",
      "\n",
      "\n",
      "sklearn_regression_model.pkl - version = 5\n",
      "\t area : diabetes\n",
      "\t type : regression\n",
      "\n",
      "\n",
      "boston_model.pkl - version = 10\n",
      "\t algo : Regression\n",
      "\t Training context : Azure ML\n",
      "\t Framework : scikit-learn\n",
      "\n",
      "\n",
      "sklearn_regression_model.pkl - version = 4\n",
      "\t area : diabetes\n",
      "\t type : regression\n",
      "\n",
      "\n",
      "boston_model.pkl - version = 9\n",
      "\t algo : Regression\n",
      "\t Training context : Azure ML\n",
      "\t Framework : scikit-learn\n",
      "\n",
      "\n",
      "diabetes_model - version = 4\n",
      "\t Training context : Pipeline\n",
      "\n",
      "\n",
      "Diabetes - version = 8\n",
      "\t Training context : Hyperdrive\n",
      "\t AUC : 0.8569106291712714\n",
      "\t Accuracy : 0.7902222222222223\n",
      "\n",
      "\n",
      "diabetes_model - version = 3\n",
      "\t Training context : Pipeline\n",
      "\n",
      "\n",
      "Exemple10-Modele-TensorFlow - version = 5\n",
      "\t Training context : TensorFlow GPU\n",
      "\n",
      "\n",
      "sklearn_regression_model.pkl - version = 3\n",
      "\t area : diabetes\n",
      "\t type : regression\n",
      "\n",
      "\n",
      "IBM_attrition_explainer - version = 6\n",
      "\n",
      "\n",
      "local_deploy_model - version = 6\n",
      "\n",
      "\n",
      "boston_model.pkl - version = 8\n",
      "\t algo : Regression\n",
      "\t Training context : Azure ML\n",
      "\t Framework : scikit-learn\n",
      "\n",
      "\n",
      "Exemple4-AutoML-Forecast - version = 4\n",
      "\t Training context : Azure Auto ML\n",
      "\t R2 : 0.2102297702988311\n",
      "\t RMSE : 0.025719636958220327\n",
      "\n",
      "\n",
      "Exemple3-AutoML-Regression - version = 5\n",
      "\t Training context : Azure Auto ML\n",
      "\t R2 : 0.8846871539563971\n",
      "\t RMSE : 0.0687685685144201\n",
      "\n",
      "\n",
      "Exemple2-AutoML-Classif - version = 4\n",
      "\t Training context : Azure Auto ML\n",
      "\t AUC : 0.9995378939750003\n",
      "\t Accuracy : 0.9728977962866562\n",
      "\n",
      "\n",
      "Exemple10-Modele-TensorFlow - version = 4\n",
      "\t Training context : TensorFlow GPU\n",
      "\n",
      "\n",
      "Diabetes - version = 7\n",
      "\t Training context : Hyperdrive\n",
      "\t AUC : 0.8569106291712714\n",
      "\t Accuracy : 0.7902222222222223\n",
      "\n",
      "\n",
      "IBM_attrition_explainer - version = 5\n",
      "\n",
      "\n",
      "local_deploy_model - version = 5\n",
      "\n",
      "\n",
      "Exemple4-AutoML-Forecast - version = 3\n",
      "\t Training context : Azure Auto ML\n",
      "\t R2 : 0.36960142200884405\n",
      "\t RMSE : 0.02304059602768627\n",
      "\n",
      "\n",
      "Exemple3-AutoML-Regression - version = 4\n",
      "\t Training context : Azure Auto ML\n",
      "\t R2 : 0.8823064055329451\n",
      "\t RMSE : 0.06954349242467574\n",
      "\n",
      "\n",
      "Exemple2-AutoML-Classif - version = 3\n",
      "\t Training context : Azure Auto ML\n",
      "\t AUC : 0.9995067640229488\n",
      "\t Accuracy : 0.972896061079299\n",
      "\n",
      "\n",
      "Diabetes - version = 6\n",
      "\t Training context : Hyperdrive\n",
      "\t AUC : 0.8569106291712714\n",
      "\t Accuracy : 0.7902222222222223\n",
      "\n",
      "\n",
      "Diabetes - version = 5\n",
      "\t Training context : Hyperdrive\n",
      "\t AUC : 0.8569106291712714\n",
      "\t Accuracy : 0.7902222222222223\n",
      "\n",
      "\n",
      "boston_model.pkl - version = 7\n",
      "\t algo : Regression\n",
      "\t Training context : Azure ML\n",
      "\t Framework : scikit-learn\n",
      "\n",
      "\n",
      "boston_model.pkl - version = 6\n",
      "\t algo : Regression\n",
      "\t Training context : Azure ML\n",
      "\t Framework : scikit-learn\n",
      "\n",
      "\n",
      "boston_model.pkl - version = 5\n",
      "\t algo : Regression\n",
      "\t type : sklearn\n",
      "\n",
      "\n",
      "boston_model.pkl - version = 4\n",
      "\t algo : Regression\n",
      "\t type : sklearn\n",
      "\n",
      "\n",
      "Diabetes - version = 4\n",
      "\t Training context : Hyperdrive\n",
      "\t AUC : 0.8569106291712714\n",
      "\t Accuracy : 0.7902222222222223\n",
      "\n",
      "\n",
      "Diabetes - version = 3\n",
      "\t Training context : Hyperdrive\n",
      "\t AUC : 0.8569106291712714\n",
      "\t Accuracy : 0.7902222222222223\n",
      "\n",
      "\n",
      "Diabetes - version = 2\n",
      "\t Training context : Hyperdrive\n",
      "\t AUC : 0.8569106291712714\n",
      "\t Accuracy : 0.7902222222222223\n",
      "\n",
      "\n",
      "Diabetes - version = 1\n",
      "\t Training context : Hyperdrive\n",
      "\t AUC : 0.8569106291712714\n",
      "\t Accuracy : 0.7902222222222223\n",
      "\n",
      "\n",
      "Exemple10-Modele-TensorFlow - version = 3\n",
      "\t Training context : TensorFlow GPU\n",
      "\n",
      "\n",
      "Exemple10-Modele-TensorFlow - version = 2\n",
      "\t Training context : TensorFlow GPU\n",
      "\n",
      "\n",
      "sklearn_regression_model.pkl - version = 2\n",
      "\t area : diabetes\n",
      "\t type : regression\n",
      "\n",
      "\n",
      "IBM_attrition_explainer - version = 4\n",
      "\n",
      "\n",
      "local_deploy_model - version = 4\n",
      "\n",
      "\n",
      "Exemple10-Modele-TensorFlow - version = 1\n",
      "\t Training context : TensorFlow GPU\n",
      "\n",
      "\n",
      "sklearn_regression_model.pkl - version = 1\n",
      "\t area : diabetes\n",
      "\t type : regression\n",
      "\n",
      "\n",
      "diabetes_model - version = 2\n",
      "\t Training context : Pipeline\n",
      "\n",
      "\n",
      "Exemple4-AutoML-Forecast - version = 2\n",
      "\t Training context : Azure Auto ML\n",
      "\t R2 : 0.2102297702988311\n",
      "\t RMSE : 0.025719636958220327\n",
      "\n",
      "\n",
      "boston_model.pkl - version = 3\n",
      "\t algo : Regression\n",
      "\t type : sklearn\n",
      "\n",
      "\n",
      "Exemple3-AutoML-Regression - version = 3\n",
      "\t Training context : Azure Auto ML\n",
      "\t R2 : 0.8823064055329451\n",
      "\t RMSE : 0.06954349242467574\n",
      "\n",
      "\n",
      "Exemple3-AutoML-Regression - version = 2\n",
      "\t Training context : Azure Auto ML\n",
      "\t R2 : 0.8823064055329451\n",
      "\t RMSE : 0.06954349242467574\n",
      "\n",
      "\n",
      "IBM_attrition_explainer - version = 3\n",
      "\n",
      "\n",
      "local_deploy_model - version = 3\n",
      "\n",
      "\n",
      "Exemple2-AutoML-Classif - version = 2\n",
      "\t Training context : Azure Auto ML\n",
      "\t AUC : 0.9994894633959408\n",
      "\t Accuracy : 0.972896061079299\n",
      "\n",
      "\n",
      "IBM_attrition_explainer - version = 2\n",
      "\n",
      "\n",
      "local_deploy_model - version = 2\n",
      "\n",
      "\n",
      "diabetes_model - version = 1\n",
      "\t Training context : Pipeline\n",
      "\n",
      "\n",
      "IBM_attrition_explainer - version = 1\n",
      "\n",
      "\n",
      "local_deploy_model - version = 1\n",
      "\n",
      "\n",
      "boston_model.pkl - version = 2\n",
      "\t algo : Regression\n",
      "\t type : sklearn\n",
      "\n",
      "\n",
      "boston_model.pkl - version = 1\n",
      "\t algo : Regression\n",
      "\t type : sklearn\n",
      "\n",
      "\n",
      "Exemple4-AutoML-Forecast - version = 1\n",
      "\t Training context : Azure Auto ML\n",
      "\t R2 : 0.2102297702988311\n",
      "\t RMSE : 0.025719636958220327\n",
      "\n",
      "\n",
      "Exemple3-AutoML-Regression - version = 1\n",
      "\t Training context : Azure Auto ML\n",
      "\t R2 : 0.8823064055329451\n",
      "\t RMSE : 0.06954349242467574\n",
      "\n",
      "\n",
      "Exemple2-AutoML-Classif - version = 1\n",
      "\t Training context : Azure Auto ML\n",
      "\t AUC : 0.9992547179382384\n",
      "\t Accuracy : 0.9646507603948424\n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# Liste des modèles référencés\n",
    "for model in Model.list(ws):\n",
    "    print(model.name, '- version =', model.version)\n",
    "    for tag_name in model.tags:\n",
    "        tag = model.tags[tag_name]\n",
    "        print ('\\t',tag_name, ':', tag)\n",
    "    for prop_name in model.properties:\n",
    "        prop = model.properties[prop_name]\n",
    "        print ('\\t',prop_name, ':', prop)\n",
    "    print('\\n')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"https://github.com/retkowsky/images/blob/master/Powered-by-MS-Azure-logo-v2.png?raw=true\" height=\"300\" width=\"300\">"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "authors": [
   {
    "name": "savitam"
   }
  ],
  "kernelspec": {
   "display_name": "Python 3.6 - AzureML",
   "language": "python",
   "name": "python3-azureml"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}