examples/Colab_DEMO_mouse_openfield.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "Colab_DEMO_mouse_openfield.ipynb",
      "version": "0.3.2",
      "provenance": [],
      "collapsed_sections": [],
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/AlexEMG/DeepLabCut/blob/master/examples/Colab_DEMO_mouse_openfield.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "RK255E7YoEIt",
        "colab_type": "text"
      },
      "source": [
        "# DeepLabCut Toolbox - Colab Demo on topview mouse data\n",
        "https://github.com/AlexEMG/DeepLabCut\n",
        "\n",
        "**new: no need to download from GitHub and link to Google Drive! Now you can directly use our demo data!**\n",
        "\n",
        "Nath\\*, Mathis\\* et al. *Using DeepLabCut for markerless3D  pose estimation during behavior across species. Nature Protocols, 2019\n",
        "\n",
        "This notebook demonstrates the necessary steps to use DeepLabCut on our demo data. We provide labeled mouse data from Mathis et al, 2018 Nature Neuroscience.\n",
        "\n",
        "This demo notebook mostly shows the most simple code to train and evaluate your model, but many of the functions have additional features, so please check out the overview & the protocol paper!\n",
        "\n",
        "This notebook illustrates how to use the cloud to:\n",
        "\n",
        "- load demo data\n",
        "- create a training set\n",
        "- train a network\n",
        "- evaluate a network\n",
        "- analyze a novel video\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "txoddlM8hLKm",
        "colab_type": "text"
      },
      "source": [
        "## [optional] First,  let's look at info about the Colab Environment (and decide, GPU or TPU):\n",
        "\n",
        "First, go to \"Runtime\" ->\"change runtime type\"->select \"Python3\", and then decide, \"GPU or TPU!\"\" (GPUs are faster!)\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "4C5WRoS9g5Od",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "!nvcc --version"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "_kwKDIgdySNQ",
        "colab_type": "text"
      },
      "source": [
        "If you want to use a GPU:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "HxVNyimFp-PJ",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "import tensorflow as tf\n",
        "tf.__version__"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Pm_PC1Q8lRrH",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "#let's make sure we see a GPU:\n",
        "tf.test.gpu_device_name()\n",
        "\n",
        "#from tensorflow.python.client import device_lib\n",
        "#device_lib.list_local_devices()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "StY_kCtQyUno",
        "colab_type": "text"
      },
      "source": [
        "If you selected to use a TPU: "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "oKTs9zVGyedR",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "#let's make sure we see a TPU:\n",
        "import os\n",
        "import pprint\n",
        "import tensorflow as tf\n",
        "\n",
        "if 'COLAB_TPU_ADDR' not in os.environ:\n",
        "  print('ERROR: Not connected to a TPU runtime; please see the first cell in this notebook for instructions!')\n",
        "else:\n",
        "  tpu_address = 'grpc://' + os.environ['COLAB_TPU_ADDR']\n",
        "  print ('TPU address is', tpu_address)\n",
        "\n",
        "  with tf.Session(tpu_address) as session:\n",
        "    devices = session.list_devices()\n",
        "    \n",
        "  print('TPU devices:')\n",
        "  pprint.pprint(devices)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "eNPkLottstN-",
        "colab_type": "text"
      },
      "source": [
        "## [Mandatory starting point!] Using DeepLabCut: "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Ew6r4hotoQjt",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# Clone the entire deeplabcut repo so we can use the demo data:\n",
        "!git clone -l -s git://github.com/AlexEMG/DeepLabCut.git cloned-DLC-repo\n",
        "%cd cloned-DLC-repo\n",
        "!ls"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "yDaY78dFoxyD",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "%cd /content/cloned-DLC-repo/examples/openfield-Pranav-2018-10-30\n",
        "!ls"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "q23BzhA6CXxu",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "#(this will take a few minutes to install all the dependences!)\n",
        "\n",
        "!pip install deeplabcut"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "XymV_Hnlp1OJ",
        "colab_type": "text"
      },
      "source": [
        "## PLEASE, click \"restart runtime\" from the output above before proceeding! "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "sXufoX6INe6w",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "#GUIs don't work on the cloud, so we will supress wxPython: \n",
        "import os\n",
        "os.environ[\"DLClight\"]=\"True\"\n",
        "os.environ[\"Colab\"]=\"True\"\n",
        "\n",
        "import deeplabcut"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Z7ZlDr3wV4D1",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "#create a path variable that links to the config file:\n",
        "path_config_file = '/content/cloned-DLC-repo/examples/openfield-Pranav-2018-10-30/config.yaml'\n",
        "\n",
        "# Loading example data set:\n",
        "deeplabcut.load_demo_data(path_config_file)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "c4FczXGDoEJU",
        "colab_type": "text"
      },
      "source": [
        "## Start training:\n",
        "This function trains the network for a specific shuffle of the training dataset. "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "_pOvDq_2oEJW",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "#let's also change the display and save_iters just in case Colab takes away the GPU... \n",
        "#if that happens, you can reload from a saved point. Typically, you want to train to 200,000 + iterations.\n",
        "#more info and there are more things you can set: https://github.com/AlexEMG/DeepLabCut/blob/master/docs/functionDetails.md#g-train-the-network\n",
        "\n",
        "deeplabcut.train_network(path_config_file, shuffle=1, displayiters=10,saveiters=100)\n",
        "\n",
        "#this will run until you stop it (CTRL+C), or hit \"STOP\" icon, or when it hits the end (default, 1.03M iterations). \n",
        "#Whichever you chose, you will see what looks like an error message, but it's not an error - don't worry...."
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "RiDwIVf5-3H_",
        "colab_type": "text"
      },
      "source": [
        "We recommend you run this for ~1,000 iterations, just as a demo. This should take around 20 min. Note, that **when you hit \"STOP\" you will get a KeyInterrupt \"error\"! No worries! :)**"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "xZygsb2DoEJc",
        "colab_type": "text"
      },
      "source": [
        "## Start evaluating:\n",
        "This funtion evaluates a trained model for a specific shuffle/shuffles at a particular state or all the states on the data set (images)\n",
        "and stores the results as .csv file in a subdirectory under **evaluation-results**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "nv4zlbrnoEJg",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "%matplotlib notebook\n",
        "deeplabcut.evaluate_network(path_config_file,plotting=True)\n",
        "\n",
        "# Here you want to see a low pixel error! Of course, it can only be as good as the labeler, so be sure your labels are good!"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "oxy5JG-kYKF4",
        "colab_type": "text"
      },
      "source": [
        "**Check the images**:\n",
        "You can go look in the newly created \"evalutaion-results\" folder at the images. At around 3500 iterations, the error is ~3 pixels (but this can vary on how your demo data was split for training)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "OVFLSKKfoEJk",
        "colab_type": "text"
      },
      "source": [
        "## Start Analyzing videos: \n",
        "This function analyzes the new video. The user can choose the best model from the evaluation results and specify the correct snapshot index for the variable **snapshotindex** in the **config.yaml** file. Otherwise, by default the most recent snapshot is used to analyse the video.\n",
        "\n",
        "The results are stored in hd5 file in the same directory where the video resides. \n",
        "\n",
        "**On the demo data, this should take around ~ 3 min! (The demo frames are 640x480, which should run around 35 FPS on the google-provided GPU)**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Y_LZiS_0oEJl",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "videofile_path = ['/content/cloned-DLC-repo/examples/openfield-Pranav-2018-10-30/videos/m3v1mp4.mp4'] #Enter the list of videos to analyze.\n",
        "deeplabcut.analyze_videos(path_config_file,videofile_path, videotype='.mp4')"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "pCrUvQIvoEKD",
        "colab_type": "text"
      },
      "source": [
        "## Create labeled video:\n",
        "This funtion is for visualiztion purpose and can be used to create a video in .mp4 format with labels predicted by the network. This video is saved in the same directory where the original video resides. This should run around 215 FPS on the demo video!"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "6aDF7Q7KoEKE",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "deeplabcut.create_labeled_video(path_config_file,videofile_path)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8GTiuJESoEKH",
        "colab_type": "text"
      },
      "source": [
        "## Plot the trajectories of the analyzed videos:\n",
        "This function plots the trajectories of all the body parts across the entire video. Each body part is identified by a unique color."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "gX21zZbXoEKJ",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "deeplabcut.plot_trajectories(path_config_file,videofile_path)"
      ],
      "execution_count": 0,
      "outputs": []
    }
  ]
}