notebooks/1_training.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "source": [
    "## Train Model\n",
    "\n",
    "*[Florian Roscheck](https://www.linkedin.com/in/florianroscheck/), 2024-04-02*\n",
    "\n",
    "In this notebook, we train the model on the TACO dataset using [Azure ML Automated Machine Learning](https://learn.microsoft.com/en-us/azure/machine-learning/concept-automated-ml?view=azureml-api-2). First, we will connect to the data assets we created earlier on Azure ML. Then, we are going to create a compute cluster for training the model. Finally, we submit the training job to the Azure ML backend. After running this notebook, you will be able to follow the training progress in the Azure ML web interface."
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "source": [
    "# Make a connection to the Azure ML workspace\n",
    "\n",
    "from azure.ai.ml import MLClient\n",
    "from azure.identity import DefaultAzureCredential\n",
    "from azureml.core import Workspace\n",
    "\n",
    "ws = Workspace.from_config()\n",
    "\n",
    "ml_client = MLClient(\n",
    "    DefaultAzureCredential(), ws.subscription_id, ws.resource_group, ws.name\n",
    ")"
   ],
   "outputs": [],
   "execution_count": 1,
   "metadata": {
    "gather": {
     "logged": 1711978752229
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "To train, we need a compute cluster. Ideally, this should be a compute cluster powerful enough to run multiple training jobs. Especially during a hackathon where there is not much time, it would be good advice to parallelize as many training jobs as possible to avoid long training run times. You can learn more about compute clusters in the [Azure ML documentation](https://learn.microsoft.com/en-us/azure/machine-learning/how-to-create-attach-compute-cluster?view=azureml-api-2&tabs=python).\n",
    "\n",
    "> Tip: If you are running on a budget, consider using low-priority VMs for training. They offer a significant price advantage, but also have some downsides. You can learn more about them in the [Azure ML documentation](https://learn.microsoft.com/en-us/azure/machine-learning/how-to-use-low-priority-batch?view=azureml-api-2&tabs=sdk)."
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "source": [
    "# Define a name for the compute cluster\n",
    "\n",
    "compute_name = \"gpu-cluster-florian-lowprio\""
   ],
   "outputs": [],
   "execution_count": 1,
   "metadata": {
    "nteract": {
     "transient": {
      "deleting": false
     }
    },
    "gather": {
     "logged": 1711978753343
    }
   }
  },
  {
   "cell_type": "code",
   "source": [
    "# Define cluster and create it\n",
    "\n",
    "from azure.ai.ml.entities import AmlCompute\n",
    "\n",
    "cluster_basic = AmlCompute(\n",
    "    name=compute_name,\n",
    "    type=\"amlcompute\",\n",
    "    size=\"STANDARD_NC8AS_T4_V3\",\n",
    "    min_instances=0,\n",
    "    max_instances=4,\n",
    "    idle_time_before_scale_down=120,\n",
    "    tier='LowPriority' # Use low-priority VM (see tip above)\n",
    ")\n",
    "ml_client.begin_create_or_update(cluster_basic)"
   ],
   "outputs": [],
   "execution_count": 1,
   "metadata": {
    "collapsed": false,
    "gather": {
     "logged": 1711978754362
    },
    "nteract": {
     "transient": {
      "deleting": false
     }
    },
    "ExecuteTime": {
     "end_time": "2024-04-02T18:54:10.486918Z",
     "start_time": "2024-04-02T18:54:10.378217Z"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "Now we are ready to start the automated machine learning training on our data. Let's get a reference to it and then feed that reference into the training job. Also note that we set the validation data size to 20% of the training data to get a reasonable estimate of the model's performance. "
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "outputs": [],
   "source": [
    "# Get reference to the data asset with the modified annotations we created earlier\n",
    "\n",
    "data = ml_client.data.get(\"TACO-annotations\", version=\"1\")"
   ],
   "metadata": {
    "nteract": {
     "transient": {
      "deleting": false
     }
    },
    "gather": {
     "logged": 1711978753222
    }
   },
   "execution_count": 1
  },
  {
   "cell_type": "code",
   "source": [
    "# Define training job\n",
    "\n",
    "from azure.ai.ml import automl\n",
    "from azure.ai.ml.constants import AssetTypes\n",
    "from azure.ai.ml import Input\n",
    "\n",
    "image_object_detection_job = automl.image_instance_segmentation(\n",
    "    compute=compute_name,\n",
    "    experiment_name='taco-flrs',\n",
    "    training_data=Input(type=AssetTypes.MLTABLE, path=data.id),\n",
    "    validation_data_size=0.2,\n",
    "    target_column_name=\"label\",\n",
    ")"
   ],
   "outputs": [],
   "execution_count": 1,
   "metadata": {
    "collapsed": false,
    "gather": {
     "logged": 1711978769238
    },
    "nteract": {
     "transient": {
      "deleting": false
     }
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "In the next cell, we define the limits for the training job. It makes sense to set the maximum number of trials to a multiple of the max_instances of the compute cluster. The parameter `max_concurrent_trials` should be set to the same number of `max_instances` of the compute cluster. This way, we can parallelize the training jobs as much as possible."
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "source": [
    "# Set job limits\n",
    "\n",
    "image_object_detection_job.set_limits(max_trials=8, max_concurrent_trials=4)"
   ],
   "outputs": [],
   "execution_count": 1,
   "metadata": {
    "collapsed": false,
    "gather": {
     "logged": 1711978771321
    },
    "nteract": {
     "transient": {
      "deleting": false
     }
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "Ready for action? Then let's submit the job to Azure ML. We can follow the progress in the Azure ML web interface."
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "source": [
    "# Submit the AutoML job\n",
    "returned_job = ml_client.jobs.create_or_update(\n",
    "    image_object_detection_job\n",
    ")  # submit the job to the backend\n",
    "\n",
    "print(f\"Created job: {returned_job}\")"
   ],
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": "Created job: compute: azureml:gpu-cluster-florian-lowprio\ncreation_context:\n  created_at: '2024-04-01T13:39:35.483134+00:00'\n  created_by: Florian Roscheck\n  created_by_type: User\ndisplay_name: olden_head_4857xq834s\nexperiment_name: taco-flrs\nid: azureml:/subscriptions/***/resourceGroups/***/providers/Microsoft.MachineLearningServices/workspaces/***/jobs/olden_head_4857xq834s\nlimits:\n  max_concurrent_trials: 2\n  max_trials: 2\n  timeout_minutes: 10080\nlog_verbosity: info\nname: olden_head_4857xq834s\noutputs: {}\nprimary_metric: mean_average_precision\nproperties: {}\nresources:\n  instance_count: 1\n  shm_size: 2g\nservices:\n  Studio:\n    endpoint: https://ml.azure.com/runs/olden_head_4857xq834s?wsid=/subscriptions/***/resourcegroups/***/workspaces/***&tid=***\n  Tracking:\n    endpoint: azureml://westeurope.api.azureml.ms/mlflow/v1.0/subscriptions/***/resourceGroups/***/providers/Microsoft.MachineLearningServices/workspaces/***?\nstatus: NotStarted\ntags: {}\ntarget_column_name: label\ntask: image_instance_segmentation\ntraining_data:\n  path: azureml:/subscriptions/***/resourceGroups/***/providers/Microsoft.MachineLearningServices/workspaces/***/data/TACO-annotations/versions/1\n  type: mltable\ntype: automl\n\n"
    }
   ],
   "execution_count": 1,
   "metadata": {
    "collapsed": false,
    "gather": {
     "logged": 1711978777235
    },
    "nteract": {
     "transient": {
      "deleting": false
     }
    }
   }
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": "While Azure Automated Machine Learning uses the [Mask R-CNN](https://arxiv.org/abs/1703.06870) model as default model for instance segmentation tasks, at the time of writing this notebook, Microsoft offers, as a preview feature, the opportunity to use different instance segmentation models. You can learn more about this [in this Jupyter Notebook by Microsoft](https://github.com/Azure/azureml-examples/blob/main/sdk/python/jobs/automl-standalone-jobs/automl-image-instance-segmentation-task-fridge-items/automl-image-instance-segmentation-task-fridge-items.ipynb), at point 4.2.1 (\"Individual runs with models from MMDetection (Preview)\")."
  },
  {
   "metadata": {},
   "cell_type": "code",
   "outputs": [],
   "execution_count": null,
   "source": ""
  }
 ],
 "metadata": {
  "kernel_info": {
   "name": "python310-sdkv2"
  },
  "kernelspec": {
   "name": "python3",
   "language": "python",
   "display_name": "Python 3 (ipykernel)"
  },
  "language_info": {
   "name": "python",
   "version": "3.10.11",
   "mimetype": "text/x-python",
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "pygments_lexer": "ipython3",
   "nbconvert_exporter": "python",
   "file_extension": ".py"
  },
  "microsoft": {
   "host": {
    "AzureML": {
     "notebookHasBeenCompleted": true
    }
   },
   "ms_spell_check": {
    "ms_spell_check_language": "en"
   }
  },
  "nteract": {
   "version": "nteract-front-end@1.0.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}