diff --git a/sdk/python/jobs/automl-standalone-jobs/automl-image-classification-multiclass-task-fridge-items/automl-image-classification-multiclass-task-fridge-items.ipynb b/sdk/python/jobs/automl-standalone-jobs/automl-image-classification-multiclass-task-fridge-items/automl-image-classification-multiclass-task-fridge-items.ipynb index 5846616896..b200de3596 100644 --- a/sdk/python/jobs/automl-standalone-jobs/automl-image-classification-multiclass-task-fridge-items/automl-image-classification-multiclass-task-fridge-items.ipynb +++ b/sdk/python/jobs/automl-standalone-jobs/automl-image-classification-multiclass-task-fridge-items/automl-image-classification-multiclass-task-fridge-items.ipynb @@ -252,7 +252,6 @@ ] }, { - "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ @@ -279,7 +278,6 @@ ] }, { - "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ @@ -638,6 +636,89 @@ "ml_client.jobs.stream(returned_job.name)" ] }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### 4.2.1 Individual runs with models from Hugging Face (Preview)\n", + "\n", + "In addition to the models supported natively by AutoML, you can launch individual runs to explore any model from HuggingFace transformers library that supports image classification. Please refer to this [documentation](https://huggingface.co/models?pipeline_tag=image-classification&library=transformers) for the list of models.\n", + "\n", + "While you can use any model from Hugging face to support this task, we have curated a set of models in our registry. We provide a set of sensible default hyperparameters for these models. You can fetch the list of curated models using code snippet below." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "registry_ml_client = MLClient(credential, registry_name=\"azureml\")\n", + "\n", + "models = registry_ml_client.models.list()\n", + "classification_models = []\n", + "for model in models:\n", + " try:\n", + " model = registry_ml_client.models.get(model.name, label=\"latest\")\n", + " if model.tags.get(\"task\", \"\") == \"image-classification\":\n", + " classification_models.append(model.name)\n", + " except Exception as ex:\n", + " print(f\"Error while accessing registry model list: {ex}\")\n", + "\n", + "classification_models" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "If you wish to try a model (say `microsoft/beit-base-patch16-224-pt22k-ft22k`), you can specify the job for your AutoML Image runs as follows:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Create the AutoML job with the related factory-function.\n", + "\n", + "image_classification_job = automl.image_classification(\n", + " compute=compute_name,\n", + " experiment_name=exp_name,\n", + " training_data=my_training_data_input,\n", + " validation_data=my_validation_data_input,\n", + " target_column_name=\"label\",\n", + ")\n", + "\n", + "image_classification_job.set_limits(timeout_minutes=60)\n", + "\n", + "image_classification_job.set_training_parameters(\n", + " model_name=\"microsoft/beit-base-patch16-224-pt22k-ft22k\"\n", + ")" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Submit the AutoML job\n", + "returned_job = ml_client.jobs.create_or_update(image_classification_job)\n", + "\n", + "print(f\"Created job: {returned_job}\")" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "ml_client.jobs.stream(returned_job.name)" + ] + }, { "cell_type": "markdown", "metadata": {}, @@ -774,6 +855,91 @@ "hd_job" ] }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### 4.3.1 Manual hyperparameter sweeping for models from Hugging Face (Preview)\n", + "\n", + "Similar to how you can use any model from Hugging face transformers library for individual runs, you can also include these models to perform a hyperparameter sweep. You can also choose a combination of models supported supported natively by [AutoML](https://docs.microsoft.com/en-us/azure/machine-learning/how-to-auto-train-image-models?tabs=CLI-v2#configure-model-algorithms-and-hyperparameters) and models from [Hugging Face](https://huggingface.co/models?pipeline_tag=image-classification&library=transformers).\n", + "\n", + "In this example, we sweep over `microsoft/beit-base-patch16-224-pt22k-ft22k`, `facebook/deit-base-patch16-224`, `seresnext`, and `resnet50` models choosing from a range of values for learning_rate, number_of_epochs, etc., to generate a model with the optimal 'accuracy'." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Create the AutoML job with the related factory-function.\n", + "\n", + "image_classification_job = automl.image_classification(\n", + " compute=compute_name,\n", + " experiment_name=exp_name,\n", + " training_data=my_training_data_input,\n", + " validation_data=my_validation_data_input,\n", + " target_column_name=\"label\",\n", + " primary_metric=ClassificationPrimaryMetrics.ACCURACY,\n", + " tags={\"my_custom_tag\": \"My custom value\"},\n", + ")\n", + "\n", + "image_classification_job.set_limits(\n", + " timeout_minutes=240,\n", + " max_trials=10,\n", + " max_concurrent_trials=2,\n", + ")\n", + "\n", + "image_classification_job.extend_search_space(\n", + " [\n", + " SearchSpace(\n", + " model_name=Choice(\n", + " [\n", + " \"microsoft/beit-base-patch16-224-pt22k-ft22k\",\n", + " \"facebook/deit-base-patch16-224\",\n", + " ]\n", + " ),\n", + " learning_rate=Uniform(0.00001, 0.0001),\n", + " number_of_epochs=Choice([10, 15]),\n", + " ),\n", + " SearchSpace(\n", + " model_name=Choice([\"seresnext\", \"resnet50\"]),\n", + " learning_rate=Uniform(0.001, 0.01),\n", + " ),\n", + " ]\n", + ")\n", + "\n", + "image_classification_job.set_sweep(\n", + " sampling_algorithm=\"Random\",\n", + " early_termination=BanditPolicy(\n", + " evaluation_interval=2, slack_factor=0.2, delay_evaluation=6\n", + " ),\n", + ")" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Submit the AutoML job\n", + "returned_job_hf = ml_client.jobs.create_or_update(\n", + " image_classification_job\n", + ") # submit the job to the backend\n", + "\n", + "print(f\"Created job: {returned_job_hf}\")" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "ml_client.jobs.stream(returned_job_hf.name)" + ] + }, { "cell_type": "markdown", "metadata": {}, @@ -1093,6 +1259,7 @@ "source": [ "from azure.ai.ml.entities import OnlineRequestSettings\n", "\n", + "# Setting the request timeout to 90 seconds.\n", "req_timeout = OnlineRequestSettings(request_timeout_ms=90000)" ] }, @@ -1285,7 +1452,9 @@ " - [Guided GradCAM](https://arxiv.org/abs/1610.02391v4) (guided_gradcam)\n", " - [Guided BackPropagation](https://arxiv.org/abs/1412.6806) (guided_backprop)\n", "\n", - "For more details on explainability with AutoML for images, refer to the [generating explanations](https://learn.microsoft.com/en-us/azure/machine-learning/how-to-auto-train-image-models?tabs=python#generate-explanations-for-predictions) and [schema](https://learn.microsoft.com/en-us/azure/machine-learning/reference-automl-images-schema#data-format-for-online-scoring-and-explainability-xai) articles." + "For more details on explainability with AutoML for images, refer to the [generating explanations](https://learn.microsoft.com/en-us/azure/machine-learning/how-to-auto-train-image-models?tabs=python#generate-explanations-for-predictions) and [schema](https://learn.microsoft.com/en-us/azure/machine-learning/reference-automl-images-schema#data-format-for-online-scoring-and-explainability-xai) articles.\n", + "\n", + "**Note**: Please note that explainability is not supported for Hugging Face models." ] }, { diff --git a/sdk/python/jobs/automl-standalone-jobs/automl-image-classification-multilabel-task-fridge-items/automl-image-classification-multilabel-task-fridge-items.ipynb b/sdk/python/jobs/automl-standalone-jobs/automl-image-classification-multilabel-task-fridge-items/automl-image-classification-multilabel-task-fridge-items.ipynb index 45f5c43565..86f5b520e1 100644 --- a/sdk/python/jobs/automl-standalone-jobs/automl-image-classification-multilabel-task-fridge-items/automl-image-classification-multilabel-task-fridge-items.ipynb +++ b/sdk/python/jobs/automl-standalone-jobs/automl-image-classification-multilabel-task-fridge-items/automl-image-classification-multilabel-task-fridge-items.ipynb @@ -235,7 +235,6 @@ ] }, { - "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ @@ -262,7 +261,6 @@ ] }, { - "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ @@ -617,6 +615,89 @@ "ml_client.jobs.stream(returned_job.name)" ] }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### 4.2.1 Individual runs with models from Hugging Face (Preview)\n", + "\n", + "In addition to the models supported natively by AutoML, you can launch individual runs to explore any model from HuggingFace transformers library that supports image classification. Please refer to this [documentation](https://huggingface.co/models?pipeline_tag=image-classification&library=transformers) for the list of models.\n", + "\n", + "While you can use any model from Hugging face to support this task, we have curated a set of models in our registry. We provide a set of sensible default hyperparameters for these models. You can fetch the list of curated models using code snippet below." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "registry_ml_client = MLClient(credential, registry_name=\"azureml\")\n", + "\n", + "models = registry_ml_client.models.list()\n", + "classification_models = []\n", + "for model in models:\n", + " try:\n", + " model = registry_ml_client.models.get(model.name, label=\"latest\")\n", + " if model.tags.get(\"task\", \"\") == \"image-classification\":\n", + " classification_models.append(model.name)\n", + " except Exception as ex:\n", + " print(f\"Error while accessing registry model list: {ex}\")\n", + "\n", + "classification_models" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "If you wish to try a model (say `microsoft/beit-base-patch16-224-pt22k-ft22k`), you can specify the job for your AutoML Image runs as follows:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Create the AutoML job with the related factory-function.\n", + "\n", + "image_classification_multilabel_job = automl.image_classification_multilabel(\n", + " compute=compute_name,\n", + " experiment_name=exp_name,\n", + " training_data=my_training_data_input,\n", + " validation_data=my_validation_data_input,\n", + " target_column_name=\"label\",\n", + ")\n", + "\n", + "image_classification_multilabel_job.set_limits(timeout_minutes=60)\n", + "\n", + "image_classification_multilabel_job.set_training_parameters(\n", + " model_name=\"microsoft/beit-base-patch16-224-pt22k-ft22k\"\n", + ")" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Submit the AutoML job\n", + "returned_job = ml_client.jobs.create_or_update(image_classification_multilabel_job)\n", + "\n", + "print(f\"Created job: {returned_job}\")" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "ml_client.jobs.stream(returned_job.name)" + ] + }, { "cell_type": "markdown", "metadata": {}, @@ -757,6 +838,95 @@ "hd_job" ] }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### 4.3.1 Manual hyperparameter sweeping for models from Hugging Face (Preview)\n", + "\n", + "Similar to how you can use any model from Hugging face transformers library for individual runs, you can also include these models to perform a hyperparameter sweep. You can also choose a combination of models supported natively by [AutoML](https://docs.microsoft.com/en-us/azure/machine-learning/how-to-auto-train-image-models?tabs=CLI-v2#configure-model-algorithms-and-hyperparameters) and models from [Hugging Face](https://huggingface.co/models?pipeline_tag=image-classification&library=transformers).\n", + "\n", + "In this example, we sweep over `microsoft/beit-base-patch16-224-pt22k-ft22k`, `facebook/deit-base-patch16-224`, and `seresnext`, models choosing from a range of values for learning_rate, number_of_epochs, validation_resize_size, etc., to generate a model with the optimal 'iou'." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Create the AutoML job with the related factory-function.\n", + "\n", + "image_classification_multilabel_job = automl.image_classification_multilabel(\n", + " compute=compute_name,\n", + " experiment_name=exp_name,\n", + " training_data=my_training_data_input,\n", + " validation_data=my_validation_data_input,\n", + " target_column_name=\"label\",\n", + " primary_metric=ClassificationMultilabelPrimaryMetrics.IOU,\n", + " tags={\"my_custom_tag\": \"My custom value\"},\n", + ")\n", + "\n", + "image_classification_multilabel_job.set_limits(\n", + " timeout_minutes=240,\n", + " max_trials=10,\n", + " max_concurrent_trials=2,\n", + ")\n", + "\n", + "image_classification_multilabel_job.extend_search_space(\n", + " [\n", + " SearchSpace(\n", + " model_name=Choice(\n", + " [\n", + " \"microsoft/beit-base-patch16-224-pt22k-ft22k\",\n", + " \"facebook/deit-base-patch16-224\",\n", + " ]\n", + " ),\n", + " learning_rate=Uniform(0.00001, 0.0001),\n", + " number_of_epochs=Choice([10, 15]),\n", + " ),\n", + " SearchSpace(\n", + " model_name=Choice([\"seresnext\"]),\n", + " learning_rate=Uniform(0.005, 0.05),\n", + " # model-specific, valid_resize_size should be larger or equal than valid_crop_size\n", + " validation_resize_size=Choice([288, 320, 352]),\n", + " validation_crop_size=Choice([224, 256]), # model-specific\n", + " training_crop_size=Choice([224, 256]), # model-specific\n", + " ),\n", + " ]\n", + ")\n", + "\n", + "image_classification_multilabel_job.set_sweep(\n", + " sampling_algorithm=\"Random\",\n", + " early_termination=BanditPolicy(\n", + " evaluation_interval=2, slack_factor=0.2, delay_evaluation=6\n", + " ),\n", + ")" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Submit the AutoML job\n", + "returned_job_hf = ml_client.jobs.create_or_update(\n", + " image_classification_multilabel_job\n", + ") # submit the job to the backend\n", + "\n", + "print(f\"Created job: {returned_job_hf}\")" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "ml_client.jobs.stream(returned_job_hf.name)" + ] + }, { "cell_type": "markdown", "metadata": {}, @@ -1074,6 +1244,7 @@ "metadata": {}, "outputs": [], "source": [ + "# Setting the request timeout to 90 seconds.\n", "req_timeout = OnlineRequestSettings(request_timeout_ms=90000)" ] }, @@ -1268,7 +1439,9 @@ " - [Guided GradCAM](https://arxiv.org/abs/1610.02391v4) (guided_gradcam)\n", " - [Guided BackPropagation](https://arxiv.org/abs/1412.6806) (guided_backprop)\n", "\n", - "For more details on explainability with AutoML for images, refer to the [generating explanations](https://learn.microsoft.com/en-us/azure/machine-learning/how-to-auto-train-image-models?tabs=python#generate-explanations-for-predictions) and [schema](https://learn.microsoft.com/en-us/azure/machine-learning/reference-automl-images-schema#data-format-for-online-scoring-and-explainability-xai) articles." + "For more details on explainability with AutoML for images, refer to the [generating explanations](https://learn.microsoft.com/en-us/azure/machine-learning/how-to-auto-train-image-models?tabs=python#generate-explanations-for-predictions) and [schema](https://learn.microsoft.com/en-us/azure/machine-learning/reference-automl-images-schema#data-format-for-online-scoring-and-explainability-xai) articles.\n", + "\n", + "**Note**: Please note that explainability is not supported for Hugging Face models." ] }, { diff --git a/sdk/python/jobs/automl-standalone-jobs/automl-image-instance-segmentation-task-fridge-items/automl-image-instance-segmentation-task-fridge-items.ipynb b/sdk/python/jobs/automl-standalone-jobs/automl-image-instance-segmentation-task-fridge-items/automl-image-instance-segmentation-task-fridge-items.ipynb index 0f3b9b19de..a203f6fd1a 100644 --- a/sdk/python/jobs/automl-standalone-jobs/automl-image-instance-segmentation-task-fridge-items/automl-image-instance-segmentation-task-fridge-items.ipynb +++ b/sdk/python/jobs/automl-standalone-jobs/automl-image-instance-segmentation-task-fridge-items/automl-image-instance-segmentation-task-fridge-items.ipynb @@ -1,7 +1,6 @@ { "cells": [ { - "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ @@ -27,7 +26,6 @@ ] }, { - "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ @@ -63,7 +61,6 @@ ] }, { - "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ @@ -105,7 +102,6 @@ ] }, { - "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ @@ -119,7 +115,6 @@ ] }, { - "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ @@ -168,7 +163,6 @@ ] }, { - "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ @@ -188,7 +182,6 @@ ] }, { - "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ @@ -227,7 +220,6 @@ ] }, { - "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ @@ -240,7 +232,6 @@ ] }, { - "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ @@ -248,7 +239,6 @@ ] }, { - "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ @@ -289,7 +279,6 @@ ] }, { - "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ @@ -337,7 +326,6 @@ ] }, { - "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ @@ -365,7 +353,6 @@ ] }, { - "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ @@ -394,7 +381,6 @@ ] }, { - "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ @@ -480,7 +466,6 @@ ] }, { - "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ @@ -509,7 +494,6 @@ ] }, { - "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ @@ -546,7 +530,6 @@ ] }, { - "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ @@ -556,7 +539,6 @@ ] }, { - "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ @@ -630,7 +612,6 @@ ] }, { - "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ @@ -662,7 +643,6 @@ ] }, { - "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ @@ -730,7 +710,89 @@ ] }, { - "attachments": {}, + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### 4.2.1 Individual runs with models from MMDetection (Preview)\n", + "\n", + "In addition to the models supported natively by AutoML, you can launch individual runs to explore any model from MMDetection version 2.28.2 that supports instance segmentation. Please refer to this [documentation](https://github.com/open-mmlab/mmdetection/blob/v2.28.2/docs/en/model_zoo.md) for the list of models.\n", + "\n", + "While you can use any model from MMDetection to support this task, we have curated a set of models in our registry. We provide a set of sensible default hyperparameters for these models. You can fetch the list of curated models using code snippet below." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "registry_ml_client = MLClient(credential, registry_name=\"azureml\")\n", + "\n", + "models = registry_ml_client.models.list()\n", + "instance_segmentation_models = []\n", + "for model in models:\n", + " try:\n", + " model = registry_ml_client.models.get(model.name, label=\"latest\")\n", + " if model.tags.get(\"task\", \"\") == \"image-segmentation\":\n", + " instance_segmentation_models.append(model.name)\n", + " except Exception as ex:\n", + " print(f\"Error while accessing registry model list: {ex}\")\n", + "\n", + "instance_segmentation_models" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "If you wish to try a model (say `mask_rcnn_swin-t-p4-w7_fpn_1x_coco`), you can specify the job for your AutoML Image runs as follows:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Create the AutoML job with the related factory-function.\n", + "\n", + "image_instance_segmentation_job = automl.image_instance_segmentation(\n", + " compute=compute_name,\n", + " experiment_name=exp_name,\n", + " training_data=my_training_data_input,\n", + " validation_data=my_validation_data_input,\n", + " target_column_name=\"label\",\n", + ")\n", + "\n", + "image_instance_segmentation_job.set_limits(timeout_minutes=60)\n", + "\n", + "image_instance_segmentation_job.set_training_parameters(\n", + " model_name=\"mask_rcnn_swin-t-p4-w7_fpn_1x_coco\"\n", + ")" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Submit the AutoML job\n", + "returned_job = ml_client.jobs.create_or_update(image_instance_segmentation_job)\n", + "\n", + "print(f\"Created job: {returned_job}\")" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "ml_client.jobs.stream(returned_job.name)" + ] + }, + { "cell_type": "markdown", "metadata": {}, "source": [ @@ -847,7 +909,6 @@ ] }, { - "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ @@ -865,7 +926,88 @@ ] }, { - "attachments": {}, + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### 4.3.1 Manual hyperparameter sweeping for models from MMDetection (Preview)\n", + "\n", + "Similar to how you can use any model from MMDetection version 2.28.2 for individual runs, you can also include these models to perform a hyperparameter sweep. You can also choose a combination of models supported natively by [AutoML](https://docs.microsoft.com/en-us/azure/machine-learning/how-to-auto-train-image-models?tabs=CLI-v2#configure-model-algorithms-and-hyperparameters) and models from [MMDetection](https://github.com/open-mmlab/mmdetection/blob/v2.28.2/docs/en/model_zoo.md).\n", + "\n", + "In this example, we sweep over `mask_rcnn_swin-t-p4-w7_fpn_1x_coco`, and `maskrcnn_resnet50_fpn`, models choosing from a range of values for learning_rate, min_size, etc., to generate a model with the optimal 'MeanAveragePrecision'.." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Create the AutoML job with the related factory-function.\n", + "\n", + "image_instance_segmentation_job = automl.image_instance_segmentation(\n", + " compute=compute_name,\n", + " experiment_name=exp_name,\n", + " training_data=my_training_data_input,\n", + " validation_data=my_validation_data_input,\n", + " target_column_name=\"label\",\n", + " primary_metric=InstanceSegmentationPrimaryMetrics.MEAN_AVERAGE_PRECISION,\n", + " tags={\"my_custom_tag\": \"My custom value\"},\n", + ")\n", + "\n", + "image_instance_segmentation_job.set_limits(\n", + " timeout_minutes=240,\n", + " max_trials=10,\n", + " max_concurrent_trials=2,\n", + ")\n", + "\n", + "image_instance_segmentation_job.extend_search_space(\n", + " [\n", + " SearchSpace(\n", + " model_name=Choice([\"maskrcnn_resnet50_fpn\"]),\n", + " learning_rate=Uniform(0.0001, 0.001),\n", + " optimizer=Choice([\"sgd\", \"adam\", \"adamw\"]),\n", + " min_size=Choice([600, 800]),\n", + " ),\n", + " SearchSpace(\n", + " model_name=Choice([\"mask_rcnn_swin-t-p4-w7_fpn_1x_coco\"]),\n", + " learning_rate=Uniform(0.00001, 0.0001),\n", + " number_of_epochs=Choice([10, 15]),\n", + " ),\n", + " ]\n", + ")\n", + "\n", + "image_instance_segmentation_job.set_sweep(\n", + " sampling_algorithm=\"Random\",\n", + " early_termination=BanditPolicy(\n", + " evaluation_interval=2, slack_factor=0.2, delay_evaluation=6\n", + " ),\n", + ")" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Submit the AutoML job\n", + "returned_job = ml_client.jobs.create_or_update(\n", + " image_instance_segmentation_job\n", + ") # submit the job to the backend\n", + "\n", + "print(f\"Created job: {returned_job}\")" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "ml_client.jobs.stream(returned_job.name)" + ] + }, + { "cell_type": "markdown", "metadata": {}, "source": [ @@ -874,7 +1016,6 @@ ] }, { - "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ @@ -891,7 +1032,6 @@ ] }, { - "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ @@ -940,7 +1080,6 @@ ] }, { - "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ @@ -976,7 +1115,6 @@ ] }, { - "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ @@ -1001,7 +1139,6 @@ ] }, { - "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ @@ -1021,7 +1158,6 @@ ] }, { - "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ @@ -1073,7 +1209,6 @@ ] }, { - "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ @@ -1081,7 +1216,6 @@ ] }, { - "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ @@ -1140,7 +1274,6 @@ ] }, { - "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ @@ -1183,13 +1316,25 @@ ] }, { - "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "### Deploy" ] }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "from azure.ai.ml.entities import OnlineRequestSettings\n", + "\n", + "# Setting the request timeout to 90 seconds. Please note that if you use a GPU compute, inference would be faster\n", + "# and this setting may not be required.\n", + "req_timeout = OnlineRequestSettings(request_timeout_ms=90000)" + ] + }, { "cell_type": "code", "execution_count": null, @@ -1202,6 +1347,7 @@ " model=registered_model.id,\n", " instance_type=\"Standard_DS4_V2\",\n", " instance_count=1,\n", + " request_settings=req_timeout,\n", " liveness_probe=ProbeSettings(\n", " failure_threshold=30,\n", " success_threshold=1,\n", @@ -1240,7 +1386,6 @@ ] }, { - "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ @@ -1264,7 +1409,6 @@ ] }, { - "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ @@ -1324,7 +1468,6 @@ ] }, { - "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ @@ -1411,7 +1554,6 @@ ] }, { - "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ @@ -1428,7 +1570,6 @@ ] }, { - "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ @@ -1446,7 +1587,6 @@ ] }, { - "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ @@ -1455,7 +1595,6 @@ ] }, { - "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [] diff --git a/sdk/python/jobs/automl-standalone-jobs/automl-image-object-detection-task-fridge-items/automl-image-object-detection-task-fridge-items.ipynb b/sdk/python/jobs/automl-standalone-jobs/automl-image-object-detection-task-fridge-items/automl-image-object-detection-task-fridge-items.ipynb index f68ca9ac1c..c153dff6a8 100644 --- a/sdk/python/jobs/automl-standalone-jobs/automl-image-object-detection-task-fridge-items/automl-image-object-detection-task-fridge-items.ipynb +++ b/sdk/python/jobs/automl-standalone-jobs/automl-image-object-detection-task-fridge-items/automl-image-object-detection-task-fridge-items.ipynb @@ -671,6 +671,91 @@ "ml_client.jobs.stream(returned_job.name)" ] }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### 4.2.1 Individual runs with models from MMDetection (Preview)\n", + "\n", + "In addition to the models supported natively by AutoML, you can launch individual runs to explore any model from MMDetection version 2.28.2 that supports object detection. Please refer to this [documentation](https://github.com/open-mmlab/mmdetection/blob/v2.28.2/docs/en/model_zoo.md) for the list of models.\n", + "\n", + "While you can use any model from MMDetection to support this task, we have curated a set of models in our registry. We provide a set of sensible default hyperparameters for these models. You can fetch the list of curated models using code snippet below." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "registry_ml_client = MLClient(credential, registry_name=\"azureml\")\n", + "\n", + "models = registry_ml_client.models.list()\n", + "object_detection_models = []\n", + "for model in models:\n", + " try:\n", + " model = registry_ml_client.models.get(model.name, label=\"latest\")\n", + " if model.tags.get(\"task\", \"\") == \"object-detection\":\n", + " object_detection_models.append(model.name)\n", + " except Exception as ex:\n", + " print(f\"Error while accessing registry model list: {ex}\")\n", + "\n", + "object_detection_models" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "If you wish to try a model (say `vfnet_r50_fpn_mdconv_c3-c5_mstrain_2x_coco`), you can specify the job for your AutoML Image runs as follows:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Create the AutoML job with the related factory-function.\n", + "\n", + "image_object_detection_job = automl.image_object_detection(\n", + " compute=compute_name,\n", + " experiment_name=exp_name,\n", + " training_data=my_training_data_input,\n", + " validation_data=my_validation_data_input,\n", + " target_column_name=\"label\",\n", + ")\n", + "\n", + "# Set limits\n", + "image_object_detection_job.set_limits(timeout_minutes=60)\n", + "\n", + "# Pass the fixed settings or parameters\n", + "image_object_detection_job.set_training_parameters(\n", + " model_name=\"vfnet_r50_fpn_mdconv_c3-c5_mstrain_2x_coco\"\n", + ")" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Submit the AutoML job\n", + "returned_job = ml_client.jobs.create_or_update(image_object_detection_job)\n", + "\n", + "print(f\"Created job: {returned_job}\")" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "ml_client.jobs.stream(returned_job.name)" + ] + }, { "cell_type": "markdown", "metadata": {}, @@ -853,6 +938,95 @@ "hd_job" ] }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### 4.3.1 Manual hyperparameter sweeping for models from MMDetection (Preview)\n", + "\n", + "Similar to how you can use any model from MMDetection version 2.28.2 for individual runs, you can also include these models to perform a hyperparameter sweep. You can also choose a combination of models supported natively by [AutoML](https://docs.microsoft.com/en-us/azure/machine-learning/how-to-auto-train-image-models?tabs=CLI-v2#configure-model-algorithms-and-hyperparameters) and models from [MMDetection](https://github.com/open-mmlab/mmdetection/blob/v2.28.2/docs/en/model_zoo.md).\n", + "\n", + "In this example, we sweep over `deformable_detr_twostage_refine_r50_16x2_50e_coco`, `sparse_rcnn_r50_fpn_300_proposals_crop_mstrain_480-800_3x_coco`, and `yolov5`, models choosing from a range of values for learning_rate, model_size, etc., to generate a model with the optimal 'MeanAveragePrecision'.." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Create the AutoML job with the related factory-function.\n", + "\n", + "image_object_detection_job = automl.image_object_detection(\n", + " compute=compute_name,\n", + " experiment_name=exp_name,\n", + " training_data=my_training_data_input,\n", + " validation_data=my_validation_data_input,\n", + " target_column_name=\"label\",\n", + " primary_metric=ObjectDetectionPrimaryMetrics.MEAN_AVERAGE_PRECISION,\n", + " tags={\"my_custom_tag\": \"My custom value\"},\n", + ")\n", + "\n", + "# Set limits\n", + "image_object_detection_job.set_limits(\n", + " timeout_minutes=240,\n", + " max_trials=10,\n", + " max_concurrent_trials=2,\n", + ")\n", + "\n", + "# Configure sweep settings\n", + "image_object_detection_job.set_sweep(\n", + " sampling_algorithm=\"random\",\n", + " early_termination=BanditPolicy(\n", + " evaluation_interval=2, slack_factor=0.2, delay_evaluation=6\n", + " ),\n", + ")\n", + "\n", + "# Define search space\n", + "image_object_detection_job.extend_search_space(\n", + " [\n", + " SearchSpace(\n", + " model_name=Choice([\"yolov5\"]),\n", + " learning_rate=Uniform(0.0001, 0.01),\n", + " model_size=Choice([\"small\", \"medium\"]), # model-specific\n", + " ),\n", + " SearchSpace(\n", + " model_name=Choice(\n", + " [\n", + " \"deformable_detr_twostage_refine_r50_16x2_50e_coco\",\n", + " \"sparse_rcnn_r50_fpn_300_proposals_crop_mstrain_480-800_3x_coco\",\n", + " ]\n", + " ),\n", + " learning_rate=Uniform(0.00001, 0.0001),\n", + " number_of_epochs=Choice([15, 20]),\n", + " ),\n", + " ]\n", + ")" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Submit the AutoML job\n", + "returned_job = ml_client.jobs.create_or_update(\n", + " image_object_detection_job\n", + ") # submit the job to the backend\n", + "\n", + "print(f\"Created job: {returned_job}\")" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "ml_client.jobs.stream(returned_job.name)" + ] + }, { "cell_type": "markdown", "metadata": {}, @@ -1190,6 +1364,19 @@ "### Deploy" ] }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "from azure.ai.ml.entities import OnlineRequestSettings\n", + "\n", + "# Setting the request timeout to 90 seconds. Please note that if you use a GPU compute, inference would be faster\n", + "# and this setting may not be required.\n", + "req_timeout = OnlineRequestSettings(request_timeout_ms=90000)" + ] + }, { "cell_type": "code", "execution_count": null, @@ -1204,6 +1391,7 @@ " model=registered_model.id,\n", " instance_type=\"Standard_DS4_V2\",\n", " instance_count=1,\n", + " request_settings=req_timeout,\n", " liveness_probe=ProbeSettings(\n", " failure_threshold=30,\n", " success_threshold=1,\n",