CV_05_Cat_dog_Transfer_Learning_PyTorch.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "Copy of Cat_dog_Transfer_Learning_PyTorch.ipynb",
      "provenance": [],
      "private_outputs": true,
      "include_colab_link": true
    },
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.7.4"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/Gedeon-m-gedus/Image_Processing/blob/master/CV_05_Cat_dog_Transfer_Learning_PyTorch.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "FZeZiihax6Fx"
      },
      "source": [
        "# A practical example to learn Transfer learning with PyTorch\n",
        "\n",
        "#### Tranfer learning is a way to solve a computer vision problem with potentially a small dataset and without too much computer power and specialized hardware. \n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "c0vC--WfrYxP"
      },
      "source": [
        "With transfer learning, the learning process can be **faster**, **more accurate** and need **less training data**, in fact, \n",
        "\n",
        "1.   ***the size of the dataset***\n",
        "2.   ***the similarity with the original dataset (the one in which the network was initially trained) are the two keys to consider before applying transfer learning.***\n",
        "\n",
        "There are four scenarios:\n",
        "\n",
        "1. Small dataset and similar to the original: train only the (last) fully connected layer\n",
        "2. Small dataset and different to the original: train only the fully connected layers\n",
        "3. Large dataset and similar to the original: freeze the earlier layers (simple features) and train the rest of the layers\n",
        "4. Large dataset and different to the original: train the model from scratch and reuse the network architecture (using the trained weights as a started point).\n",
        "\n",
        "Note : In a network, the earlier layers capture the simplest features of the images (edges, lines…) whereas the deep layers capture more complex features in a combination of the earlier layers (for example eyes or mouth in a face recognition problem). In order to fine-tune a model, we need to retrain the final layers because the earlier layers have knowledge useful for us."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "P1cgYNfTqg-C"
      },
      "source": [
        "# **Gaol**\n",
        "\n",
        "1.   We are going to learn how transfer learning can help us to solve a problem without spending too much time training a model\n",
        "2.   Taking advantage of pretrained architectures.\n",
        "\n",
        "We'll make use resnet34 as pretrained model. \n",
        "\n",
        "## ImageNet\n",
        "<img src=\"https://github.com/Volviane/Cats_Dogs_classification/blob/master/cat_dog_tranfer_learning/.ipynb_checkpoints/data/imagenetpic.png?raw=1\">\n",
        " \n",
        "\n",
        "We will start with importing the necessary packages."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "xD09_lfLtK8J"
      },
      "source": [
        "Task and pretrained model\n",
        "\n",
        "\n",
        "---\n",
        "\n",
        "\n",
        " Dog and cat classifiction using resnet34 pretrained model.\n",
        "\n",
        "\n",
        " Needed packages\n",
        "\n",
        "\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "50V5w2VmvGzy",
        "colab": {}
      },
      "source": [
        "import torch\n",
        "import torch.nn as nn\n",
        "import torch.optim as optim\n",
        "import numpy as np\n",
        "import torchvision\n",
        "from torchvision import datasets, models, transforms\n",
        "import matplotlib.pylab as plt"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "a3mAhbhuEHF6"
      },
      "source": [
        "# Loading Cat and Dog Dataset"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "0NddywVRD10c",
        "colab": {}
      },
      "source": [
        "#https://drive.google.com/file/d/1qXKSt6jAgh20f-2PkRT-Z61PW-3E4S_z/\n",
        "\n",
        "# Load helper file\n",
        "link = \"https://drive.google.com/file/d/1qXKSt6jAgh20f-2PkRT-Z61PW-3E4S_z/\"\n",
        "\n",
        "_, id_t = link.split('d/')\n",
        "\n",
        "id = id_t.split('/')[0]\n",
        "\n",
        "print (\"Loading file ...\")\n",
        "\n",
        "print (id) # Verify that you have everything after '='\n",
        "\n",
        "# Install the PyDrive wrapper & import libraries.\n",
        "# This only needs to be done once per notebook.\n",
        "!pip install -U -q PyDrive\n",
        "from pydrive.auth import GoogleAuth\n",
        "from pydrive.drive import GoogleDrive\n",
        "from google.colab import auth\n",
        "from oauth2client.client import GoogleCredentials\n",
        "\n",
        "# Authenticate and create the PyDrive client.\n",
        "# This only needs to be done once per notebook.\n",
        "auth.authenticate_user()\n",
        "gauth = GoogleAuth()\n",
        "gauth.credentials = GoogleCredentials.get_application_default()\n",
        "drive = GoogleDrive(gauth)\n",
        "\n",
        "\n",
        "file_id = id\n",
        "downloaded = drive.CreateFile({'id':file_id})\n",
        "downloaded.FetchMetadata(fetch_all=True)\n",
        "downloaded.GetContentFile(downloaded.metadata['title'])\n",
        "print (\"Completed\")"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "GAYSRS9su3Bh",
        "colab": {}
      },
      "source": [
        "!ls data"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "eFoHTYBlEdGl",
        "colab": {}
      },
      "source": [
        "#!unzip data.zip"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "fsHaafcjEuHy",
        "colab": {}
      },
      "source": [
        "transforms = transforms.Compose([\n",
        "        transforms.Resize(256),\n",
        "        transforms.CenterCrop(224),\n",
        "        transforms.ToTensor(),\n",
        "        transforms.Normalize([0.485, 0.456, 0.406], \n",
        "                             [0.229, 0.224, 0.225])\n",
        "    ])\n",
        "\n",
        "train_set = datasets.ImageFolder(\"data/train\",transforms)\n",
        "val_set   = datasets.ImageFolder(\"data/val\",transforms)\n",
        "  \n",
        "train_loader = torch.utils.data.DataLoader(train_set, batch_size=4,\n",
        "                                       shuffle=True, num_workers=4)\n",
        "val_loader = torch.utils.data.DataLoader(val_set, batch_size=4,  \n",
        "                                       shuffle=True, num_workers=4)\n",
        "classes = train_set.classes\n",
        "device = torch.device(\"cuda:0\" if torch.cuda.is_available()\n",
        "                               else \"cpu\")\n"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "fcSevXZcFHDn",
        "colab": {}
      },
      "source": [
        "!ls data"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "nlWXyZY9FWxt"
      },
      "source": [
        "# Model Building\n",
        "\n",
        "Now, we define the  network we’ll be training. The resnet34 model was originally trained for a dataset that had 1000 class labels, but our dataset only has two class labels! We'll replace the final layer with a new, untrained layer that has only two outputs ( dog and cat)."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "a9JpHf1MFJEJ",
        "colab": {}
      },
      "source": [
        "model = models.resnet34(pretrained=True)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "Lu3H7v6UnHEF",
        "colab": {}
      },
      "source": [
        "model"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "TZAzEQyT7sxS",
        "colab": {}
      },
      "source": [
        "classes = ['cat', 'dog']\n",
        "mean , std = torch.tensor([0.485, 0.456, 0.406]),torch.tensor([0.229, 0.224, 0.225])\n",
        "\n",
        "def denormalize(image):\n",
        "  image = transforms.Normalize(-mean/std,1/std)(image) #denormalize\n",
        "  image = image.permute(1,2,0) #Changing from 3x224x224 to 224x224x3\n",
        "  image = torch.clamp(image,0,1)\n",
        "  return image\n",
        "# helper function to un-normalize and display an image\n",
        "def imshow(img):\n",
        "    img = denormalize(img) \n",
        "    plt.imshow(img)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "rq4Wr3oL05c0"
      },
      "source": [
        "## Feature Extraction & Image embedding"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "gg4ccEBAFjNT",
        "colab": {}
      },
      "source": [
        "for param in model.parameters():\n",
        "    param.requires_grad = False"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "lz-10l5en5eg",
        "colab": {}
      },
      "source": [
        "num_ftrs = model.fc.in_features #the number of input features of the last layer of thefully connected \n",
        "model.fc = nn.Linear(num_ftrs, 512) # return the last layer of the fc"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "8nVcPgHt1fm2",
        "colab": {}
      },
      "source": [
        "num_ftrs, model.fc"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "ChazBldUmdnF",
        "colab": {}
      },
      "source": [
        " for i, data in enumerate(train_loader, 0):\n",
        "        inputs, labels = data\n",
        "        break\n",
        "# plot the images in the batch, along with the corresponding labels\n",
        "fig = plt.figure(figsize=(20, 8))\n",
        "# display 8 images\n",
        "for idx in np.arange(4):\n",
        "    ax = fig.add_subplot(2, 4/2, idx+1, xticks=[], yticks=[])\n",
        "    imshow(inputs[idx])\n",
        "    ax.set_title(\"{} \".format(classes[labels[idx]]))\n",
        "\n",
        "print(inputs.shape)\n",
        "embeds = model(inputs)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "TBW-EGwh-PAa",
        "colab": {}
      },
      "source": [
        "print('shape of embedding for one batch size', embeds.shape)\n",
        "print(\"-------------------------------------------------\")\n",
        "print(embeds)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "HpMhI4CY-3Gl"
      },
      "source": [
        "### we are doing binary classification we will alter the final layer to have two neurons."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "lzUsaT5aFpzk",
        "colab": {}
      },
      "source": [
        "num_ftrs = model.fc.in_features\n",
        "model.fc = nn.Linear(num_ftrs, 2)\n",
        "model = model.to(device)\n",
        "criterion = nn.CrossEntropyLoss()\n",
        "optimizer = optim.SGD(model.parameters(), lr=0.001, momentum=0.9)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "_HwFOBzJGNrh"
      },
      "source": [
        "# Model Training"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "C3fJ22IEGHlj",
        "colab": {}
      },
      "source": [
        "for epoch in range(25):\n",
        "    running_loss = 0.0\n",
        "    for i, data in enumerate(train_loader, 0):\n",
        "        inputs, labels = data\n",
        "        inputs = inputs.to(device)\n",
        "        labels = labels.to(device)\n",
        "        \n",
        "        optimizer.zero_grad()\n",
        "        \n",
        "        outputs = model(inputs)\n",
        "        loss = criterion(outputs, labels)\n",
        "        loss.backward()\n",
        "        optimizer.step()\n",
        "        \n",
        "        running_loss += loss.item()\n",
        "    print('Train Epoch: {} [{}/{} ({:.0f}%)]\\tLoss: {:.6f}'.format(\n",
        "                epoch, i * len(data), len(train_loader.dataset),\n",
        "                100. * i / len(train_loader), running_loss))\n",
        "print('Finished Training')"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "NwlUQWpCG-H8"
      },
      "source": [
        "# Model Validation"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "OO3QC6M9GXYA",
        "colab": {}
      },
      "source": [
        "class_correct = list(0. for i in range(2))\n",
        "class_total = list(0. for i in range(2))\n",
        "with torch.no_grad():\n",
        "    for i, data in enumerate(val_loader, 0):\n",
        "            inputs, labels = data\n",
        "            inputs = inputs.to(device)\n",
        "            labels = labels.to(device)\n",
        "            outputs = model(inputs)\n",
        "            _, predicted = torch.max(outputs, 1)\n",
        "            c = (predicted == labels).squeeze()\n",
        "            for i in range(4):\n",
        "                label = labels[i]\n",
        "                class_correct[label] += c[i].item()\n",
        "                class_total[label] += 1\n",
        "for i in range(2):\n",
        "    print('Accuracy of %5s : %2d %%' % (\n",
        "        classes[i], 100 * class_correct[i] / class_total[i]))"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "FD0zpXOTH_lB"
      },
      "source": [
        "# Test with our own image"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "8W12hMmiHofU",
        "colab": {}
      },
      "source": [
        "from PIL import Image\n",
        "from torchvision.transforms import ToTensor\n",
        "model.eval()\n",
        "img_name1 = \"dog.jpeg\" # change this to the name of your image file.\n",
        "img_name2 = \"mycat.jpg\"\n",
        "def predict_image(image_path, model):\n",
        "    image = Image.open(image_path)\n",
        "    print(type(image))\n",
        "    image_tensor = ToTensor()(image)\n",
        "    #image_tensor = transforms(image)\n",
        "    image_tensor = image_tensor.unsqueeze(0)\n",
        "    image_tensor = image_tensor.to(device)\n",
        "    output = model(image_tensor)\n",
        "    index = output.argmax().item()\n",
        "    if index == 0:\n",
        "        return \"It's a Cat\"\n",
        "    elif index == 1:\n",
        "        return \"It's a Dog\"\n",
        "    else:\n",
        "        return \"None of them\"\n",
        "predict_image(img_name2,model)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "1gRvSwFOJxEN",
        "colab": {}
      },
      "source": [
        "! ls /content"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "wsPbPMH7DCkP",
        "colab": {}
      },
      "source": [
        ""
      ],
      "execution_count": 0,
      "outputs": []
    }
  ]
}