LearningAI_Chapter_3_Neural_Networks.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "LearningAI: Chapter 3 - Neural Networks.ipynb",
      "provenance": []
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "source": [
        " Logistic Regression with PyTorch. We will mostly see how to use Pytorch rather than numpy that we used in the previous 2 chapters. Pytorch has similar functionality and commands. The type of variable in pytorch is tensor compared to arrays in numpy."
      ],
      "metadata": {
        "id": "cqJGRIv-Y_Bf"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "import torch\n",
        "import matplotlib.pyplot as plt\n",
        "from IPython import display\n",
        "import time\n",
        "# let’s make the same dataset which we constructed at the start of chapter 2, but this time, in PyTorch.\n",
        "X = torch.cat((torch.arange(15, dtype=float), torch.arange(20, 35, dtype=float)), axis=0)\n",
        "y = torch.cat((torch.zeros(15, dtype=float), torch.ones(15, dtype=float)), axis=0)"
      ],
      "metadata": {
        "id": "CmsFr_CKZ-QI"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "Let’s continue rewriting our code from chapter 2, but using PyTorch instead."
      ],
      "metadata": {
        "id": "c4h3R4-waTbj"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "def BCE(y, y_pred): ##log loss error (binary cross entropy)\n",
        "  return -torch.sum((y*torch.log(y_pred) + (1-y)*torch.log(1-y_pred)))/y.shape[0]"
      ],
      "metadata": {
        "id": "qFV9dMD9aT1W"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "epoch_loss = []\n",
        "\n",
        "weights = torch.tensor([0., 0.])\n",
        "learning_rate = 1e-4\n",
        "n = X.shape[0]\n",
        "\n",
        "for epoch in range(600000+1): \n",
        "    linear = weights[0]*X + weights[1]\n",
        "    y_pred = 1/(1+torch.exp(-linear))\n",
        "    loss = BCE(y, y_pred)\n",
        "    epoch_loss.append(loss)\n",
        "\n",
        "\n",
        "    if(epoch%50000 == 0):\n",
        "        ######demo purpose only#####\n",
        "        display.display(plt.gcf())\n",
        "        display.clear_output(wait=True)\n",
        "        ##########plotting##########\n",
        "        fig, (ax0, ax1) = plt.subplots(ncols=2, constrained_layout=True)\n",
        "        fig.suptitle('epoch = {0}'.format(epoch))\n",
        "        ax0.scatter(X, y)\n",
        "        ax0.plot(X, y_pred, 'r')\n",
        "        ax0.set_title('weight0 = {0:.1f}, weight1 = {1:.1f}'.format(weights[0], weights[1]))\n",
        "        ax1.set_title('bce = {0:.2f}'.format(loss))\n",
        "        ax1.plot(epoch_loss)\n",
        "        plt.show()\n",
        "        time.sleep(1)\n",
        "        ############################\n",
        "    \n",
        "    ###simplified slope and bias derivatives with respect to loss###  \n",
        "    D_mse_wrt_slope = torch.sum(X * (y - y_pred))\n",
        "    D_mse_wrt_bias = torch.sum(y - y_pred)\n",
        "    ###########\n",
        "\n",
        "    weights[0] += learning_rate * D_mse_wrt_slope\n",
        "    weights[1] += learning_rate * D_mse_wrt_bias"
      ],
      "metadata": {
        "id": "pagZqtK1aVgg",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 330
        },
        "outputId": "90ef8f19-1644-45fe-d7a9-b1f2887ae325"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 432x288 with 0 Axes>"
            ]
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 432x288 with 2 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAbgAAAEoCAYAAAAqrOTwAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deZhcZZ33//enqpfsGwkkkD0kRFBAiAoqiqMIKqIzbjAu6KMyLjg6OjjwOMMgjo7K6KP+BkdwQ0cREB2MiMYFcMQFCbvpTiAsIQlbFrJ30tv398c5HSpNV3clXdVVderzuq666ix3nXNX1Tn1rXs551ZEYGZmljW5amfAzMysEhzgzMwskxzgzMwskxzgzMwskxzgzMwskxzgzMwskxzgzAYhaa6kkNRU7byY2f5xgDOrEZJeIekOSTslrZP05oJ1x0q6XdKu9PnYgnWS9DlJm9LH5ySp0q81q3UOcGY1QNKRwJXAJ4CJwDHA7em6FuAnwPeAycB3gJ+kywHOAV6fvuZo4LXA343Aa81qmgOc1RVJh0r6kaQNkh6S9PcF6y6SdK2kqyVtT0tDxxSsf5akmyVtkbRC0hkF60ZL+oKkNZK2SrpF0uiCXb9V0iOSNkr6RAXe2j8Dl0XEzyOiOyI2RcQD6bqTgSbgSxGxJyK+Agj4q3T92cAXImJdRKwHvgC8cwRea1bTHOCsbkjKAT8F7gYOA14OfETSqQXJXgf8EJhCUiK6TlKzpOb0tb8EDgY+BHxf0hHp6/4DOB54YfrajwO9Bdt9MXBEus8LJT2rSB7PTwPogI9B3t4J6evvlfSYpO9JmpKuOwq4J/a9r9496fK+9XcXrLu737pKvdaspjnAWT15HjAtIi6OiM6IeBD4OnBmQZrbI+LaiOgCvgiMIgkeJwDjgM+mr70RuB44Kw2c/wf4cESsj4ieiPhDROwp2O4nI6IjIu4mCQLHMICI+GxETCr2GOS9zQTeDrwBWAiMBv6/dN04YGu/9FuB8UXWbwXGpW1plXytWU1zzzCrJ3OAQ/uVhPLA7wrm1/ZNRESvpHXAoX3rIqKwVLaGpCQ4lSQQPkBxjxdM7yL58S+nDuDbEXEfgKTPAL9O1+0AJvRLPwHYXmT9BGBHRISkSr7WrKa5BGf1ZC3wUL9S0fiIeHVBmll9E2nJbCbwaPqYlS7rMxtYD2wEdgMLhptBSf9X0o5ij0Feeg9QWBVYOL0COLqwdyNJh5AVBesLS5TH9FtXqdea1TQHOKsnfwa2S/qntFNIXtKzJT2vIM3xkv4mvW7tI8Ae4E/ArSQlr4+nbXInk/QYvCot1X0L+GLaiSUv6URJrfubwYj4TESMK/YY5KXfBt4lab6kMcD5JFWoADcDPcDfS2qVdG66/Mb0+bvARyUdJulQ4GPAFSPwWrOa5gBndSMieoDTgWOBh0hKXt8g6Vbf5yfAW4CnSNq0/iYiuiKikySgvSp93VeBd0TEyvR1/wjcC9wGbAY+xwieHxHxLZJgcytJ1eke4O/TdZ0kXfnfAWwhaS98fboc4DKSDjT3An8BfpYuq/RrzWqaPOCpZYWki4DDI+Jt1c6LmVWfS3BmZpZJDnBmZpZJrqI0M7NMcgnOzMwyyQHOzMwyyQHOzMwyyQHOzMwyyQHOzMwyyQHOzMwyyQHOzMwyyQHOzMwyyQHOzMwyyQHOzMwyyQHOzMwyqS4CnKSvSfqXEtNeIenfKp2nLJM0Ox2BOl9C2rmSIh1g1GzESHpY0iuqnQ+rXXUR4CLifRHxqXJsK/0xPrzfspdLWilpl6SbJM0px7767WNuuu1d6b6KnpiSpki6WtImSRslfV/ShHLnqZiIeCQdgbpnuNuSdJGk7/Vb9mZJf0g/i5uHu4/9yMunJN0rqTsdO65Yum8NdJz0S3OspNvT93C7pGMrkmmrS0p8Lj2HN6XTGiT930paI2mnpOskTSlYd66k5ZL2SLpiRN5ARtRFgKskSVOBHwP/AkwBlgNXV2BXPwDuBA4CPgFcK2lakbT/BkwG5gELgEOAiyqQp2rZDHwJ+OwI73c18HGSUasHJOnFJJ95UZJaSEYO/x7J9/Qd4CfpcjOAc0hGQz8GOJpkNPm/GyihpKNIRlF/O8m5votkxPk+j5L8JnyrgvnNpoio2AN4F/DTgvn7gR8WzK8Fjk2nFwO/IvnxWwW8uSDdFcC/Fcx/HHiM5It/DxAkIzn3pb2U5EdsO3ArsCBd979p2p3ADuAtJAfiHwq2PRboABaX8XNYBOwBxhcs+x3wviLpfw58oGD+g8CyEve1Bjg+nX5r+n6PSuffDVyXTueA84EHgE3ANcCUdN3c9HVN6fy89LPbDvw6/Xy/1y/t2cAjwEbgE+m604BOoCv9vO/ul9f3ADdX8hgs8hl9D7hogOVNJH9Cji48pgZI90pgPelwU+myR4DTRvq9NPIDeBi4AGgDngK+DYwqWP864C5gW3qcn5Yunwh8M/0NWU8SPPJlztsfgHMK5t8N/KlI2s8AVxbML0jPm/H90v0bcEW1P/d6elS6BPdb4CRJOUmHAi3AiQCS5gPjgHskjSUJblcCBwNnAl+VdGT/DUo6Dfgo8ArgcODkAfZ7JvBJkn/Xq4FPA0TES9L1x0RSBXc1cBRwd98LI2Inyclw1EBvSNL1krYUeVxf5HM4CngwIrYXLLu72D5IAsjpkiZLmgy8gSToleK3PP2ZvBR4EHhJwfxv0+kPkfzDfClwKMkPxKVFtnkl8GeS0udFJP80+3sxcATwcuBCSc+KiF+QnLxXp5/3MSW+h73S9sBin/cWSX+7v9scxD8A/xsR9wyR7ijgnkh/dVL3UPz7tMp5K3AqSVBYBPwzgKTnA98FzgMmkZwDD6evuQLoJvn9eC7JH5b3DLTxtOpwsONvdpF87fO7wuDne//foAdIAtyiQd63laCiAS4iHiT5138syQG2DHhU0mKSH9bfRUQvcDrwcER8OyK6I+JO4EfAmwbY7JuBb0fEiojYxcBVd/8TEX+OiG7g++n+ixkHbO23bCswvsh7Oj0iJhV5nF6OfQB3kPwZ2JQ+eti3ymIwvyX5bAFOAv69YL4wwL2PpKS1LiL2kHyOb+zfWSQ9gZ8HXBgRnRFxC7B0gP1+MiI6IuJukpN1v4PZQCJpDyz2eU+KiCvLsR9Js0iqkC4sIfn+fp9WOf8ZEWsjYjPJH9mz0uXvBr4VEb+KiN6IWB8RKyUdArwa+EhE7IyIJ4H/R/Kn+Bki4sohjr9HiuSr/zGyFRhXpB3Ox1OFjEQbXF+J4iXp9M0kP7SFP7ZzgBcU/jMi+Wc2fYDtHUpStdln7QBpHi+Y3kVyABWzA+jfgWMCSWAul/3dxzXAfSQH+ASSEuX3iqTtr6/UPAPIp9t6kaS5JFUzd6Xp5gD/U/B5t5ME0kP6be9QYHP6Z6LPcD/zipO0Iu0JukPSSSW85EvAxRHR/4dmICNxzFhpCo/FNSTHK8AskvOmvzlAM/BYwbF/GUnNUTn1P0YmADv6lfqLpe1L7+NpmEYywJ2UTveVMAoD3Frgt/3+GY2LiPcPsL3HgJkF87OGmb8VFJQ20urSBenyZ5D084Ifzv6PYtWIK4D5kgr/kR1TbB8kJc7L0n+YO4CvkfzrHFJErCYJMB8iqW7bRhJ8zgFuSUvMkHzmr+r3mY+KiPX9NvkYMEXSmIJl+/OZD3RCl0xPX7JQ7PHWAXcacVR6DI2LiN+VsKuXA5dIelxSX7D+Y5Eq0BXA0f3+jR9N8e/TKqfwWJxN0i4PyfE9UGehtSTt4VMLjvsJEVGsSeKtQxx/xaoo9/ldYfDzvf9v0HygleRPrg3DSAW4lwGjI2IdSeeK00jac+5M01wPLJL0dknN6eN5kp41wPauAd4l6Vnpj25J18cVeAKYXzD/P8CzJb1B0iiSKqp7ImLlQC+OiFcV/HD2f7yqyGvuIyk5/aukUZL+muQH8UdF8ngb8B5JoyWNJglOe9uFJN2sQbq5k3zm5/L0H4ib+81DEjQ/rfSSCEnTJL1ugLyvIelZepGkFkknkvQIK9UTwFxJe481Sfn0s24Cculn0jzQi+PpSxaKPb5fakbS42oUyXHflO6371q/RSQ/MsfydJX2a0mOj/5uJint/r2kVknnpstvLDUvVjYflDRTSbf6T/B0D+hvkvxOvDztA3CYpMUR8RjwS+ALkiak6xZIeulAG4+I7w9x/BWrovwu8NF0v4cCHyNp+xvI94HXSjop/YN9MfDjvjZ7SU3pcZsH8ulx6+tOS1Gs90k5HySlgG8XzC8Hft4vzREkPR83kLQ73cjTPSyvYN9elBeQlEoeBd5PUkqYVSTtycC6gvn3pfnZQtpTk6TDykqS3pM3A3Mr8BnMTbfdQdJL9BUF694KrCiYnwf8NP0cNgO/ABYWrH8AOGWQff1d+pnMSedPT+dfUJAmR9JZZxVJVcgDwGcK8lrYi3IByR+T7cBvgMuBbw6UNl12M/CedPog4BaSTix3pMvemb6m8HHFCByHVwyw33cWSbtPL0qSTj7/t2D+ucDt6fd5B/DckTiX/NjnO3qYp3tRbiG5XGNMwfq/JvljuJ2ks9mp6fKJwH8B60jauu4Ezixz3gR8Pj1/N6fThb1udwAnFcz/LUlP3J0kl6BMKVh30QDH7UXV/vzr4aH0A6xbaSnvL0BrJJ1KMk3STOCaiHhhFfNwNbAyIv61WnkwMxtKXQa4tIrvBmAMyb+23oh4fXVzlV2SnkfyL/Qhki7V1wEnRtLb1cysJtXrnUz+DniSpFqth6Sa0ipnOkm14w7gK8D7HdzMrNbVZQnOzMxsKPVagjMzMxuUA5yZmWVS1a6lmDp1asydO7dauzc7YLfffvvGiCg2EkRN8Xlm9ahc51jVAtzcuXNZvnx5tXZvdsAkral2Hkrl88zqUbnOMVdRmplZJjnAmZlZJjnAmdUASadJWiVptaTzB1j//yTdlT7uS++Cb2aD8A07zaosveHzpcApJPdHvE3S0oho60sTEf9QkP5DJPfCNLNBDFmCk/QtSU9K+kuR9ZL0lfSf5z2Sjit/Ns0y7fnA6oh4MCI6gauAZ4zsUOAs4AcjkjOzOlZKCe4K4D9Jhn8YyKuAhenjBSR36X5BOTJnI+O6O9dzybJVPLqlg0Mnjea8U48AGHLZyxZP46aVGzL7uvNOPYLXP/ewkfgKDmPfgTvXUeQcSoc3moeH5jEbUkm36kpHg74+Ip49wLrLgJsj4gfp/Crg5EjGXSpqyZIl4e7L1Xfdneu54Mf30tHVs3dZc04g6OqJQZf1l7XXjW7O8+9/85xnBDlJt0fEkqI72E+S3gicFhHvSeffTjK00bkDpP0nYGZEfGiQ7Z1DMoYgs2fPPn7Nmrq5qsEMKN85Vo42uIH+fR5GMuaa1bhLlq3aJ7gBdPU+MzgMtOyZiTqZsW0jM7c+wdRdWxnXuYtxe5LH+M5dtHZ30dzTTXNv33MPud4echHkoxdFkItechGIQBHsHTI7XQagCt8+9dtLzuCGxS+mo6uHS5atGolS3Hr2HZl6ZrpsIGcCHxxsYxFxOcmYfSxZsmTAT+vedVu56KcruPh1R3HUoRP3P8dmdWBEO5n0+2c5kru2Ih7d0nFAr8v39nDso6t42YPLed7aFcza+gTTt28ixzN/T3uUY2fLaPY0NdOZa6Yr30RXvonuXJ6eXJ4e5Qhp73OvRCBQEtJCfdMi0ohXEPrKrufpwccP+PPZT7cBCyXNIwlsZ5IMgLkPSYuBycAfh7vD7bu7uH3NU+zYnfkhFK2BlSPAlfzvs5R/ljayDp00mvUl/ogrennNyls49b4/ctLDdzJp9w66lePuGYv4w5xjWDfxYNZNPIR1Ew9mw5jJbG8dw47WMexqHgWqXECqpEMnja74PiKiW9K5wDIgD3wrIlZIuhhYHhFL06RnAldFGYcA8UloWVaOALcUOFfSVSQN41uHan+z2nHeqUeU1Aa38KlH+czPv8zz1q5gw9hJ/HLhCdw0fwm/n3ss20aNK/q65pxoruM2uL7OJ5UWETeQDOJbuOzCfvMXlW2H9fl/w2y/DBngJP0AOBmYKmkd8K9AM0BEfI3kpHw1sBrYBbyrUpm18utrXyrWo/CJzTv4h7/8jPfd+B16Wkbx6TecxzcXnMSMyWN52eJpjF+5ge013huyDnpRVo2Hg7Qsq9qAp+5FWQfa2uDss2H5cnj96+GrX4UZM6qdq6ordy/KSip2nv3xgU2c9fU/8YP3nsCJCw6qQs7MiqulXpSWRZs3wymnQGcnXH01vOlNdduOZsWFW+Eswxzg7Jki4P3vhyefhFtvheN8c5qs8X8VawQOcPZMP/gBXHMNfPrTDm5Z5wKcZZhHE7B9rV0LH/gAvPCF8PGPVzs3ViF9BTjHN8syBzh7Wm8vvPOd0N0N3/0uNLmAn1VyHaU1AP+C2dO+8hW48Ub4+tdhwYJq58ZGgC8TsCxzCc4Sq1bB+efDa18L7353tXNjFeYCnDUCBzhL/OhHsGcPXHaZf/0aiC8TsCxzgLNEWxvMmuULuRuE/8JYI3CAs0R7Oxx5ZLVzYSPMbXCWZQ5wlvSebG+HZz2r2jmxEeJaaGsEDnAGjzwCHR0uwTUgF+AsyxzgLGl/A5fgGoqLcJZ9DnCWVE+CA1wDqtZoImYjwQHOkhLcwQfDQR42pVG4Dc4agQOcuQdlA3P5zbLMAa7RRSQlOFdPNhQX4KwROMA1uscfh61bXYJrVC7CWYY5wDU696BsSB5NwBqBA1yj6+tB6RJcVUk6TdIqSaslnV8kzZsltUlaIenKcuzX96K0LPNwOY2urQ0mToTp06udk4YlKQ9cCpwCrANuk7Q0ItoK0iwELgBeFBFPSTp4WPsczovN6oRLcI2urwelq6yq6fnA6oh4MCI6gauA1/VL817g0oh4CiAinizHjn0ZnGWZA1yjcw/KWnAYsLZgfl26rNAiYJGk30v6k6TThrND/5+xRuAqyka2aRM8+aTb3+pDE7AQOBmYCfyvpOdExJb+CSWdA5wDMHv27EE36hKcZZlLcI3MHUxqxXpgVsH8zHRZoXXA0ojoioiHgPtIAt4zRMTlEbEkIpZMmzZtwB3KrXDWABzgGpkvEagVtwELJc2T1AKcCSztl+Y6ktIbkqaSVFk+ONwduwBnWeYA18ja22HMGBiiGssqKyK6gXOBZUA7cE1ErJB0saQz0mTLgE2S2oCbgPMiYtOB7tNtcNYI3AbXyNraYPFiyPl/TrVFxA3ADf2WXVgwHcBH00c591vOzZnVFP+yNTLfZNnMMswBrlFt3w5r17r9rcG5/GZZ5gDXqFauTJ5dgmtIboOzRuAA16jcg9LwdXCWbSUFuKFuBCtptqSbJN0p6R5Jry5/Vq2s2tuhuRkWLKh2TqwKfB2cNYIhA1zBjWBfBRwJnCWpf73WP5N0bX4uyTU8Xy13Rq3M2tpg0SJockfaxuYinGVXKSW4Um4EG8CEdHoi8Gj5smgV4R6UDc1tcNYISglwpdwI9iLgbZLWkVzL86Gy5M4qY/duePBBt7+Z2+As08rVyeQs4IqImAm8GvhvSc/YtqRzJC2XtHzDhg1l2rXtt/vug95el+AamEtw1ghKCXCl3Aj23cA1ABHxR2AUMLX/hkq5CayNAPegtJQLcJZlpQS4Um4E+wjwcgBJzyIJcC6i1ar29uT2XIsWVTsnViXuRWmNYMgAV+KNYD8GvFfS3cAPgHeGb3JXu9raYP58GDWq2jmxKvNZallWUh/xEm4E2wa8qLxZs4ppa3P7W4NzG5w1At/JpNF0dcH99zvAGQDhVjjLMAe4RvPAA0mQcweThtZXgHMVpWWZA1yjaW9Pnl2Ca2iuorRG4ADXaPouEVi8uLr5sJrgApxlmQNco2lvh9mzYdy4aufEqspFOMs+B7hG09bm9jfby1fzWJY5wDWS3t5koFO3vzU8t8FZI3CAayRr1kBHh0twZtYQHOAaiXtQ1qwSBhV+p6QNku5KH+8Z1v6G82KzOuHRLhuJb7JckwoGFT6FZDiq2yQtTe8QVOjqiDi3nPt2E5xlmUtwjaS9HQ45BKZMqXZObF+lDCpcVrm0Ea7XEc4yzAGukbgHZa0qZVBhgDdIukfStZJmDbC+ZPlcX4AbzlbMapsDXKOISEpwbn+rVz8F5kbE0cCvgO8US1jKwMJ9vSh7HeEswxzgGsVjj8HWrS7B1aYhBxWOiE0RsSed/QZwfLGNlTKwcF8JrsdVlJZhDnCNwj0oa9mQgwpLmlEwewbJ2IwHLO82OGsA7kXZKNyDsmZFRLekvkGF88C3+gYVBpZHxFLg79MBhruBzcA7h7NP9QU4V1FahjnANYq2Npg0CaZPr3ZObAAlDCp8AXBBufa3t4rSAc4yzFWUjaKvg4nv0WQUVlFWOSNmFeQA1yh8iYAVUHrmuw3OsswBrhFs3AgbNriDie3VV4JzFaVlmQNcI+jrQekSnKV8obc1Age4RuBLBKyfvRd6u4rSMswBrhG0tcGYMTBrWHd3sgxxFaU1Age4RtDenlRP5vx1W+LpKkoHOMsu/+I1AvegtH58obc1Age4rNu2Ddatc/ubPUM+J3cysUxzgMu6lSuTZ5fgrJ+cfLNlyzYHuKxzD0orIie5itIyzQEu69raoKUF5s+vdk6sxiRVlA5wll0OcFnX3g6LFkGT76tt+8pJ9PRWOxdmleMAl3XuQWlF5OTLBCzbHOCyrKMDHnrI7W82IFdRWtaVFOAknSZplaTVks4vkubNktokrZB0ZXmzaQfkvvugt9clOBtQUkXpAGfZNWTDjKQ8cClwCrAOuE3S0ohoK0izkGQwxhdFxFOSDq5Uhm0/9I3i7RKcDSDnEpxlXCkluOcDqyPiwYjoBK4CXtcvzXuBSyPiKYCIeLK82bQD0t6e3J5r0aJq58RqUF6i151MLMNKCXCHAWsL5telywotAhZJ+r2kP0k6baANSTpH0nJJyzds2HBgObbStbXBggXQ2lrtnFgN8oXelnXl6mTSBCwETgbOAr4uaVL/RBFxeUQsiYgl06ZNK9Ouraj2dldPWlG5nC/0tmwrJcCtBwrHWZmZLiu0DlgaEV0R8RBwH0nAs2rp6ko6mbiDiRXhXpSWdaUEuNuAhZLmSWoBzgSW9ktzHUnpDUlTSaosHyxjPm1/PfAAdHe7BFcnSumpnKZ7g6SQtGS4+8xJ9Di+WYYNGeAiohs4F1gGtAPXRMQKSRdLOiNNtgzYJKkNuAk4LyI2VSrTVoK+HpQuwdW8gp7KrwKOBM6S9Ix/JpLGAx8Gbi3HfnPycDmWbSXdvykibgBu6LfswoLpAD6aPqwW9N1kefHi6ubDSrG3pzKApL6eym390n0K+BxwXjl2mpOrKC3bfCeTrGprg9mzYdy4aufEhjZkT2VJxwGzIuJnQ22s1N7K+Zwv9LZsc4DLKvegzAxJOeCLwMdKSV9qb+WkBFemTJrVIAe4LOrtTQY6dftbvRiqp/J44NnAzZIeBk4Alg63o0ku55stW7Y5wGXRmjXJjZZdgqsXg/ZUjoitETE1IuZGxFzgT8AZEbF8ODvN+16UlnEOcFnkHpR1pcSeymXne1Fa1nkUzCzq60HpAFc3huqp3G/5yeXYp3tRWta5BJdFbW1wyCEwZUq1c2I1zFWUlnUOcFnU1ub2NxtS0smk2rkwqxwHuKyJSKooXT1pQ8jJN1u2bHOAy5pHH4Vt21yCsyHlc/JwOZZpDnBZ09fBxAHOhuASnGWdA1zW+BIBK1FTTnQ7wFmGOcBlTXs7TJ6c9KI0G0RT3r0oLdsc4LKmrS0pvUnVzonVuKZcjq6e3mpnw6xiHOCyxjdZthI15V1FadnmAJclGzfChg1uf7OSNOVydHtIb8swB7gscQ9K2w/NedHd6ypKyy4HuCxxD0rbD/mcXIKzTHOAy5L2dhg7FmbNGjqtNbzmvDuZWLY5wGVJWxssXpzcZNBsCL4OzrLOv4RZ4h6Uth/y7kVpGecAlxXbtsG6dW5/s5I153J0u4rSMswBLitWrkyeXYKzEjXlRW/g+1FaZjnAZYV7UNp+as4np3+XLxWwjHKAy4q2Nmhpgfnzq50TqxP5XHI7N9+P0rLKAS4r2tvhiCOgqanaObE60ZQGuC5fC2cZ5QCXFX03Wba6JOk0SaskrZZ0/gDr3yfpXkl3SbpF0rAbW/uqKN3RxLLKAS4LOjrgoYfcwaROScoDlwKvAo4EzhoggF0ZEc+JiGOBzwNfHO5+m/JJCc6XClhWOcBlwapVEOESXP16PrA6Ih6MiE7gKuB1hQkiYlvB7Fhg2FGpr4rSAc6yyg02WeCbLNe7w4C1BfPrgBf0TyTpg8BHgRbgr4a706acqygt21yCy4K2tuT2XAsXVjsnVkERcWlELAD+CfjnYukknSNpuaTlGzZsKLq9vipKdzKxrHKAy4L2djj8cGhtrXZO7MCsBwrvkD0zXVbMVcDri62MiMsjYklELJk2bVrRjfR1MvFlApZVJQW4oXp4FaR7g6SQtKR8WbQhuQdlvbsNWChpnqQW4ExgaWECSYXF89cA9w93p/m9lwm4itKyacgAV2IPLySNBz4M3FruTNogurrg/vvd/lbHIqIbOBdYBrQD10TECkkXSzojTXaupBWS7iJphzt7uPttdi9Ky7hSOpns7eEFIKmvh1dbv3SfAj4HnFfWHNrgVq+G7m6X4OpcRNwA3NBv2YUF0x8u9z7dycSyrpQqyoF6eB1WmEDSccCsiPjZYBsqtfHb9oN7UNoBcicTy7phdzKRlCO56PRjQ6UttfHb9kPfTZYXL65uPqzutDblAeh0Cc4yqpQAN1QPr/HAs4GbJT0MnAAsdUeTEdLeDnPmwNix1c6J1ZnWpuT039PVU+WcmFVGKQFu0B5eEbE1IqZGxNyImAv8CTgjIpZXJMe2L/egtAM0qjkNcN0uwVk2DRngSuzhZdXQ05MMdOr2NzsAfVWUDnCWVSXdqmuoHl79lp88/GxZSdasgd27XYKzA7K3irLbVZSWTb6TST3r62DiEpwdgL0luC6X4CybHODqWd8lAi7B2QFodRucZZwDXD1ra4Pp02Hy5GrnxOpQS95VlJZtDnD1rM4EdTEAABSSSURBVL3d1ZN2wHI50ZLPuQRnmeUAV68ifImADVtrU85tcJZZDnD16tFHYft2l+BsWFqbc66itMxygKtXfT0oXYKzYWhtyruK0jLLAa5e+SbLVgatTW6Ds+xygKtXbW1J78mDD652TqyOtTTlfC9KyywHuHrV14NSqnZOrI61NruK0rLLAa5euQellUFrU47dLsFZRjnA1aMNG2DjRre/2bCNacnT4QBnGeUAV498iy4rk7GtTezc013tbJhVhANcPXIPSiuTcS1N7NzjEpxlkwNcPWprS0bwnjVr6LRmgxjTmncJzjLLAa4e9XUwcQ/KzJB0mqRVklZLOn+A9R+V1CbpHkm/kTSnHPsd19rEzs5uIqIcmzOrKQ5w9ai93e1vGSIpD1wKvAo4EjhLUv/65zuBJRFxNHAt8Ply7HtsaxO9gTuaWCY5wNWbrVth/Xq3v2XL84HVEfFgRHQCVwGvK0wQETdFxK509k/AzHLseGxrE4Db4SyTHODqzcqVybMDXJYcBqwtmF+XLivm3cDPy7HjsS3JqN5uh7Msaqp2Bmw/+SbLDU3S24AlwEsHSXMOcA7A7NmzB91eXwluhwOcZZBLcPWmvR1aW2HevGrnxMpnPVDYJXZmumwfkl4BfAI4IyL2FNtYRFweEUsiYsm0adMG3fG4vVWUDnCWPQ5w9aatDRYtgiYXvjPkNmChpHmSWoAzgaWFCSQ9F7iMJLg9Wa4d722D63SAs+xxgKs3fTdZtsyIiG7gXGAZ0A5cExErJF0s6Yw02SXAOOCHku6StLTI5vZLXwlu+24HOMseFwPqSUcHPPQQvOMd1c6JlVlE3ADc0G/ZhQXTr6jEfqeMbQHgqZ2dldi8WVW5BFdPVq2CCJfgrGwmjm5Ggqd2dVU7K2Zl5wBXT9yD0sosnxMTRjXz1C6X4Cx7HODqSXs75POwcGG1c2IZMmVsi0twlkkOcPWkrQ0WLEguEzArk0ljmt0GZ5nkAFdP3IPSKmDKmBZXUVomOcDVi64uuP9+t79Z2U0a08IWV1FaBjnA1YvVq6G72yU4K7spY5vZtHOPh8yxzCkpwFVrrCor4B6UViGHTBjF7q5etnX4Ym/LliEDXDXHqrICfQFu8eLq5sMyZ8bE0QA8tq2jyjkxK69SSnBVG6vKCrS3w5w5MHZstXNiGTN94igAHtu6u8o5MSuvUgJc1caqsgJtbW5/s4roC3CPO8BZxpS1k0nBWFWXFFl/jqTlkpZv2LChnLvOtp6e5DZdDnBWAQePb0VyCc6yp5QAV7axqvZnnCor8PDDsHu3O5hYRTTnc0wb18pjW9wGZ9lSSoCr2lhVlmpvT55dgrMKmT1lDGs27xo6oVkdGTLAVXOsKkv5EgGrsPnTxvLghp3VzoZZWZU0Hly1xqqyVHs7zJgBkyZVOyeWUfOmjmPjjnVs293FhFHN1c6OWVn4Tib1oK3NpTerqPnTkstPXIqzLHGAq3URvsmyVdyCvQFuR5VzYlY+DnC1bv162L7dJTirqLkHjaW1KceKR7dVOytmZeMAV+vcg9JGQFM+x1GHTuCedVuqnRWzsnGAq3XuQWkj5OiZk/jL+m109/RWOytmZeEAV+va22HKFDj44GrnxCqohBE7XiLpDkndkt5YiTwcM2siHV093P+k2+EsGxzgal1fD0qp2jmxCilxxI5HgHcCV1YqH0vmTAHgTw9uqtQuzEaUA1ytcw/KRlDKiB0PR8Q9QMXqD2dNGcO8qWP53f0bK7ULsxHlAFfLNmyAjRvd/pZ9+ztiR8W8+PCp/PGBTezp7qnG7s3KygGulvV1MHEJzvbDcEbtOPmIaXR09fCH1a6mtPrnAFfL+i4RcAku60oasaNUwxm146SF05g4upmf3HXAuzerGQ5wtaytDcaNg1mzhk5r9WzIETtGSktTjlc/Zzq/bHuCnXu6q5EFs7JxgKtl7e3uQdkAShmxQ9LzJK0D3gRcJmlFpfLzxuNnsauzhx/dsa5SuzAbESWNJmBV0tYGr/BADY2ghBE7biOpuqy442ZP4thZk/jmLQ/x1hfMIZ/zHyyrTy7B1aqtW+HRR93BxEacJM55yXzWbNrltjiraw5wtcodTKyKTjtqOkfPnMjnf7GKXZ1ui7P65ABXq+64I3l2gLMqyOXEv5x+JI9v280Xf3lftbNjdkAc4GpRBHz1q3D00XD44dXOjTWo582dwttOmM03bnmI36/23U2s/jjA1aJf/AJWrIB//Ef3oLSq+sSrj2T+tLF86Ad38vBGj/Zt9cUBrhb9x3/AYYfBmWdWOyfW4Ea35PnGO5bQG8G7rriNJ7btrnaWzErmAFdr7rgDbrwRPvIRaG6udm7MmD9tHN88ewlPbtvNG7/2Bx7ZtKvaWTIriQNcrfnCF2D8eHjve6udE7O9jp8zhe+/9wS27+7mjEtv4aZVT1Y7S2ZDcoCrJWvWwNVXwznnwMSJ1c6N2T6OnTWJ//nAi5g+YRTv+vZtfPKnK9jh23lZDXOAqyVf/nLSqeTDH652TswGNG/qWK774It4+wlz+PbvH+aUL/6W6+5cT09vVDtrZs/gAFcrtmyBr3896VjimytbDRvVnOdTr382P3r/C5k8poWPXH0Xp37pf/nJXevp7K7YeKxm+80BrlZcfjns2AEf+1i1c2JWkuPnTOb6D72Yr771OAR8+Kq7eOFnf8Mly1b6kgKrCb7Zci3YvDmpnnzFK+DYY6udG7OS5XLi1c+ZwWlHTed3qzfy339cw3/d/ACX3vQAR86YwGuOnsErjzyEww8eh3xNp40wB7hq27oVTj0VNm6Eiy+udm7MDkguJ166aBovXTSNx7Z28LN7HuNn9z7GJctWccmyVUyfMIoXL5zKiw+fyvFzJjNz8mgHPKs4B7hq2r4dTjsN7r4bfvxjOPHEaufIbNhmTBzNe06az3tOms+jWzr47X0buOX+jfy6/QmuvT0ZY27quBaOnTWZY2dN5FkzJrDokPEOelZ2DnDVsnMnvOY1cNtt8MMfwumnVztHZmV36KTRnPX82Zz1/Nn09Abtj23jrrVbuPORLdy59il+3f7E3rTjWptYdMg4jpg+njkHjWXOlDHMPmgMcw4ay7hW/1TZ/vNRUw0dHXDGGfD738OVV8Jf/3W1c2RWcfmcePZhE3n2YRN52wlzANi2u4v7n9jOyse3s+rx5PkXf3mcp3Z17fPag8a2MPugMcycPIbpE1qZPnE00yeMYvrEVg6ZMIqDx4+ipcl95mxfDnAjqacHrrsOPv1puOsuuOIKeMtbqp0rs6qZMKqZ4+dM4fg5U/ZZvrWji7Wbd7Fm0y7WbN65d/qedVv45dbd7BngcoSDxrYwJX0cNK5vupWD+ubHtDBhdDMTRzczYXQz41ubyHm08kwrKcBJOg34MpAHvhERn+23vhX4LnA8sAl4S0Q8XN6s1rGODvjOd5LbcK1eDQsWJHcsedObqp0zs5o0cXQzE9PSXn8RwdaOLh7ftpvHt6aPbbt5cvseNu/oZPPOTlY9vp3NOzvZ0tFFFLkGXYLxrU1MHNPMhFHJIwl+TYxtbWJsS/rcmk+n84xtbWJMS+GyZLoln3P7YQ0aMsBJygOXAqcA64DbJC2NiLaCZO8GnoqIwyWdCXwOOKCiyXV3rueSZat4dEsHh04azXmnHgGwz7KXLZ7GTSs3DJqmmq/b8uRmXrpzHad3PUr+zjs4/r7lTN21laeOOob7L7mMj3XOY93tnRz6wI0l7+/1zz3sQD5Os8yRxKQxLUwa08Li6RMGTdvd08tTu7rYvDMJfNt2d7G1o4ttfY/d3WzrSJft7uLBjTvY1tHNzj3d7OzsptQbtDTlxOjmPK3NeUY15xjdnGdUOj1q73SeUU25vcufTr/va1qbcjTnc7T0Pe+dFs353N71zU3Juua8HFyLUBT7e9OXQDoRuCgiTk3nLwCIiH8vSLMsTfNHSU3A48C0GGTjS5YsieXLl++z7Lo713PBj+9l/OYnae1J6uCbBQi6BrlBQmEapbtszgmUHOAFbzZ5TqebckIE3b0BAbnopUWQo5fenl7yvT009fakz70093QxqruTcT2dtHbvoaVzDxN37+CgXVs4eNcWpu7cwtQdm5nz1OPkSPb12LiDuOOwxXz3uNO5Y85zICe6ep7+WPryWbisv9HNef79b57jIFcjJN0eEUuqnY9SDHSeWWkigt1dvezsTAPenh52dXazY083uzp7kuc93exMp3d39bC7qzd97nl6vrv48iF+fkvWnFcS7IoExWLBsjmfI58TzXmRz4mmXI6mnMjnRXMuly4TTflc+pzM53O5vdN9657eTo7mdD5J07eP5Lkv3czJY4q+n3KdY6VUUR4GrC2YXwe8oFiaiOiWtBU4CNivYYAvWbaKjq4evnn9F3jhI/fsz0ur6qlR49k0ZiKbxk5ixcELuO7Il3HPjIX85ZDD2TBu8tMJA+gXyLpK+IvY0dXDJctWOcBlmJsBao8kRrfkGd2SZ+q41rJvPyLo7Olld2dfEEwC357uHrp6eunsTtZ3dfcm8z29dHb3FiyLvcu6Cp97Yu90V8FrOrt72dXZzdaOp9d39wbdfc/pdE9v0NUb9KSPSpg8ppk7L3xlRbZdaEQ7mUg6BzgHYPbs2c9Y/+iWDgD+64Q3cu1zXn7A+wn2La5HQfE99lmmfZb15PL0SoREILryTfQoR3cuT08uT1euid3NrexuaqGjuZWO5lZ2toymK1/5cdv6PhvLnpFuBrDaIInWpjytTXkmUptjP/b2Bj2RBLquvuDXk8x39/bS3ZMGxnT6mcuDnt7eva/p6umlN4Km3Mj0eC0lwK0HCu/+OzNdNlCadWkV5USSf5n7iIjLgcshqTrpv/7QSaNZv6WD3807rrTcN5BDJ42udhascp4PrI6IBwEkXQW8DigMcK8DLkqnrwX+U5IGawYwG65cTuQQzfnkJtv1ppQwehuwUNI8SS3AmcDSfmmWAmen028EbjyQE++8U49gdL8PsTmtrx3MQGmy9LrRzfm9nVEskwZqBuhfH71PMwDQ1wzwDJLOkbRc0vINGzZUILtm9WHIElzapnYusIykfeBbEbFC0sXA8ohYCnwT+G9Jq4HNJEFwv/W1MdVKb8haep3b36xUQ9WUmDWKIXtRVop7d1m9Kncvykr1VAafZ1afynWO+d42ZtU3Ys0AZo3Et+oyq7KRbAYwayQOcGY1ICJuAG7ot+zCgundgO/tZrYfXEVpZmaZ5ABnZmaZVLVelJI2AGsGSTKV/bzVVw2p17zXa75hZPM+JyKmjdC+hmWI86yev+9i/J7qw1DvqSznWNUC3FAkLa+XG9r2V695r9d8Q33nvVqy+Jn5PdWHkXpPrqI0M7NMcoAzM7NMquUAd3m1MzAM9Zr3es031HfeqyWLn5nfU30YkfdUs21wZmZmw1HLJTgzM7MDVpMBTtJpklZJWi3p/GrnpxhJ35L0pKS/FCybIulXku5PnycPto1qkTRL0k2S2iStkPThdHlN51/SKEl/lnR3mu9PpsvnSbo1PWauTu/paEXU2jm2v8ejEl9J83+PpOMKtnV2mv5+SWcXLD9e0r3pa74iJSMhV/qYl5SXdKek69P5AY9VSa3p/Op0/dyCbVyQLl8l6dSC5QN+j5U8HyRNknStpJWS2iWdWLPfU0TU1IPkXnwPAPOBFuBu4Mhq56tIXl8CHAf8pWDZ54Hz0+nzgc9VO59F8j4DOC6dHg/cBxxZ6/knGYZ9XDrdDNwKnABcA5yZLv8a8P5q57VWH7V4ju3v8Qi8Gvh5ejycANyaLp8CPJg+T06nJ6fr/pymVfraV6XLK3rMAx8FrgSuT+cHPFaBDwBfS6fPBK5Op49Mv6NWYF763eUH+x4reT4A3wHek063AJNq9Xuq+sk2wId3IrCsYP4C4IJq52uQ/M5l3wC3CpiRTs8AVlU7jyW+j58Ap9RT/oExwB3AC0guGm0a6Bjy4xmfW82fY0Mdj8BlwFkF6Vel688CLitYflm6bAawsmD53nSVPOaBmcBvgL8Crk9/tAc8Vklutn1iOt2UplP/76cvXbHvcbB9lOH9TAQeIu2/0f/zr7XvqRarKEsZ3biWHRIRj6XTjwOHVDMzpUirQp5LUhqq+fynVT53AU8CvyL5F7slkpGuof6OmZFW0+dYicdjsfcw2PJ1AyxnkH2Uw5eAjwO96fxBFD9Wi43avr/vdbB9DNc8YAPw7bTa9RuSxlKj31MtBrjMiOSvRk13U5U0DvgR8JGI2Fa4rlbzHxE9EXEsyb/j5wOLq5wlK5NqH4/l3Iek04EnI+L2cmyvRjSRNMv8V0Q8F9hJUl24Vy19T7UY4NYDswrmZ6bL6sUTkmYApM9PVjk/RUlqJvkx+X5E/DhdXDf5j4gtwE0kVTCTlIx0DfV3zIy0mjzH9vN4LPYeBls+c4Dlg+1juF4EnCHpYeAqkmrKL1P8WN2b93T9RGDTEO9poOWbBtnHcK0D1kXEren8tSQBrya/p1oMcKWMblzLCkdePpukLaHmpD2Tvgm0R8QXC1bVdP4lTZM0KZ0eTdJO004S6N6YJqu5fNeYmjvHDuB4XAq8I+2ldwKwNa2+Wga8UtLktJfdK0nanx4Dtkk6Id3XO/ptq+zHfERcEBEzI2IuyWd8Y0S8leLHarFR25cCZ6a9LOcBC0k6Ygz4Paavqcj5EBGPA2slHZEuejnQRq1+T+VqTC3ng6TnzX0kbSufqHZ+BsnnD4DHgC6SfzbvJqn//g1wP/BrYEq181kk7y8mKeLfA9yVPl5d6/kHjgbuTPP9F+DCdPl8kpN+NfBDoLXaea3lR62dY/t7PJJ0pLg0zf+9wJKCbf2f9DhYDbyrYPmS9Jh5APhPnr7RRcWPeeBknu5FOeCxCoxK51en6+cXvP4Tab5XkfYqHOx7rOT5ABwLLE+/q+tIekHW5PfkO5mYmVkm1WIVpZmZ2bA5wJmZWSY5wJmZWSY5wJmZWSY5wJmZWSY5wJmZWSY5wJmZWSY5wJmZWSb9//gdz0mg0MqLAAAAAElFTkSuQmCC\n"
          },
          "metadata": {
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "You might feel like the above code is different. But we can show that there is a fundamental difference between pytorch and numpy. In pytorch, we don't have to deal with derivatives. It takes care of it by using gradient descents which is a numerical algorithm to minimization rather than analytical solution to the derivatives of a function. The relevance of which is important for high dimensional complicated data sets.\n",
        "\n",
        "Let's see how it works"
      ],
      "metadata": {
        "id": "QFWHU3lObOBK"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "epoch_loss = []\n",
        "\n",
        "weights = torch.tensor([0., 0.], requires_grad=True)\n",
        "learning_rate = 1e-2\n",
        "n = X.shape[0]\n",
        "\n",
        "for epoch in range(60000+1):\n",
        "    linear = weights[0]*X + weights[1]\n",
        "    y_pred = 1/(1+torch.exp(-linear))\n",
        "    loss = BCE(y, y_pred)\n",
        "    epoch_loss.append(loss.item())\n",
        "\n",
        "\n",
        "    if(epoch%5000 == 0):\n",
        "        ######demo purpose only#####\n",
        "        display.display(plt.gcf())\n",
        "        display.clear_output(wait=True)\n",
        "        ##########plotting##########\n",
        "        fig, (ax0, ax1) = plt.subplots(ncols=2, constrained_layout=True)\n",
        "        fig.suptitle('epoch = {0}'.format(epoch))\n",
        "        ax0.scatter(X, y)\n",
        "        ax0.plot(X, y_pred.detach().numpy(), 'r')\n",
        "        ax0.set_title('weight0 = {0:.1f}, weight1 = {1:.1f}'.format(weights[0], weights[1]))\n",
        "        ax1.set_title('bce = {0:.2f}'.format(loss))\n",
        "        ax1.plot(epoch_loss)\n",
        "        plt.show()\n",
        "        time.sleep(1)\n",
        "        ############################\n",
        "\n",
        "    ###autograd in action###\n",
        "    loss.backward()\n",
        "\n",
        "    with torch.no_grad():\n",
        "        weights -= learning_rate * weights.grad\n",
        "        \n",
        "    weights.grad.zero_()"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 330
        },
        "id": "kdnv08RNugi9",
        "outputId": "b1ef0261-0c8d-4768-ece0-87fc193f9999"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 432x288 with 0 Axes>"
            ]
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 432x288 with 2 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAbgAAAEoCAYAAAAqrOTwAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deZxcdZX//9fp6uo9nT0hKwkxhIQIAcL2QxEHkGUUVBRBUXEBdcRldFQYHH6KiCCOMzqDA0ERdURQRAibGZVFRVkCFQKEBEKArGRfuju99/n+cW+HSqequzpd3bfq1vv5eNSj7vKpe09V3apzP5/7ufeauyMiIhI3ZVEHICIiMhiU4EREJJaU4EREJJaU4EREJJaU4EREJJaU4EREJJaU4ERyZGbTzMzNrDzqWESkb0pwIgXIzE4xs6fNrMnM1prZuWnz5pnZU2a2O3yelzbPzOxaM9saPq41M8vHa0WKjRKcSIExsznArcDlwHDgcOCpcF4FcDfwv8BI4GfA3eF0gIuBd4evOQx4F/Cpgb5WpBgpwUnRMrOJZvZbM9tsZq+Y2efT5n3DzO4ws9vNrCGsDR2eNn+2mT1sZjvM7HkzOyttXrWZ/buZvWZmO83sr2ZWnbbqD5nZajPbYmaXD8Jb+zpwo7s/4O4d7r7V3V8O550ElAP/6e6t7v5DwIB/COd/FPh3d1/r7uuAfwcuzMNrRYqOEpwUJTMrA+4BngEmAScDXzSz09KKnQ38BhhFUCO6y8ySZpYMX/t/wDjgc8AvzWxW+LrvAUcB/1/42q8CXWnLfQswK1znFWY2O0uMl4YJNOOjl7d3XPj6Z81sg5n9r5mNCucdCiz1va+xtzSc3j3/mbR5z/SYt7+vFSk6SnBSrI4Gxrr7le7e5u6rgJuA89LKPOXud7h7O/B9oIogeRwH1AHXhK99ELgXOD9MnB8HvuDu69y9093/5u6tacv9prs3u/szBEngcDJw92vcfUS2Ry/vbTLwYeAcYCZQDfxXOK8O2Nmj/E5gWJb5O4G68FjaQF4rUnTUG0yK1YHAxB41oQTwl7TxNd0D7t5lZmuBid3z3D29VvYaQU1wDEEifJnsXk8b3k2QGPKpGfipu78IYGZXA38M5zUC9T3K1wMNWebXA43u7ma2368dwHsRiYxqcFKs1gCv9KgVDXP3M9PKTOkeCGtmk4H14WNKOK3bVGAdsAVoAWYMNEAz+1cza8z26OWlS4H0pJI+/DxwWI9a1WHh9O756TXKw3vM29/XihQdJTgpVk8ADWb2tbBTSMLM5prZ0WlljjKz94bnrX0RaAUeAx4nqHl9NTwmdxJBj8HbwlrdzcD3w04sCTM73swq+xugu1/t7nXZHr289KfAx8zsIDOrAS4laEIFeBjoBD5vZpVmdkk4/cHw+efAl8xskplNBL4M3JKH14oUHSU4KUru3gm8E5gHvEJQ8/oxQbf6bncDHwC2ExzTeq+7t7t7G0FCOyN83Y+Aj7j78vB1/wI8CzwJbAOuZQh/K+5+M0GyeZyg6bQV+Hw4r42gK/9HgB0ExwvfHU4HuJGgA82zwHPAfeG0Ab1WpBiZmtcljszsG8Cb3P2CqGMRkWioBiciIrGkBCciIrGkJkoREYkl1eBERCSWlOBERCSWlOBERCSWlOBERCSWlOBERCSWlOBERCSWlOBERCSWlOBERCSWlOBERCSWlOBERCSWlOBERCSWiibBmdkNZvZvOZa9xcyuGuyY4szM3mpmK3Ise5KZrR3smEQyMbNXzeyUqOOQwlM0Cc7dP+3u38rHsszMzexNPaadbGbLzWy3mT1kZgfmY1091jEtXPbucF29/ijN7BQze9rMmsxsrZmdm++YsnH3v7j7rHwsK9MOh5ldYmaLzazVzG7Jx3pyjGWBma0wsy4zuzDD/H82s9fNbJeZ3ZztTt5mNieMf3v4+KOZzRn0NyBFxQLXmtnW8HGtmVkv5T9oZq+Fv/m7zGxUOL3SzH4SzmswsyVmdsbQvZPiVDQJbjCZ2RjgTuDfgFHAYuD2QVjVr4AUMBq4HLjDzMZmiWkOcGtYbjhwOPDUIMQUlfXAVcDNQ7zeZ4B/Ap7uOcPMTgMuBU4GDgQOAr6ZZTnrgfcRbC9jgIXAbYMQrxS3iwnuon44cBjBneQ/lamgmR1KcAf1DwPjgd0Ed5sHKAfWAG8j+D/4OvBrM5s2eKHHgLsP6gP4GHBP2vhLwG/SxtcA88LhQ4A/ANuAFcC5aeVuAa5KG/8qsIHgj+aTgBPcwbm77PXAfUAD8DgwI5z357BsE9AIfIBgI/xb2rJrgWbgkDx+DgcDrcCwtGl/AT6dpfytwLf2Yz1VYexjwvHLgQ6gPhz/FvCf4XAl8D1gNbARuAGoDuedBKxNW+6RBMm5AfgNwQ7AVellgS8Dm8Lv5WPhvIuBdqAt/Lzv6RHvVcAtg70dZvic/gpcmOEzvzpt/GTg9RyWVQ58Ftg91O9DDwd4FbgMWAZsB34KVKXNPxtYAuwCXgZOD6cPB34Sbq/rwm0xkefY/gZcnDb+CeCxLGWvBm5NG58R/m6GZSm/FDgn6s+/kB9DUYN7BHirmZWZ2USgAjgewMwOAuqApWZWS5DcbgXGAecBP8rU7GNmpwNfAk4B3kTwB9vTeQR73yOBlcC3Adz9xHD+4e5e5+63A4cS7NkTlmki+CEcmukNmdm9ZrYjy+PeLJ/DocAqd29Im/ZMtnUAx4XretbMNpjZ/3Y3V/TG3VuAJwn29AifXwNOSBt/JBy+hiDxziP4HCcBV2R4vxXA7wh2HEYR1ETf06PYAQR/GJMIfsTXm9lId18A/BL4bvh5v6uv95Bh/W/p5fPeYWZv6e8ys9hrOwiHx5vZ6F5i2wG0AP9F8Acl0fgQcBpBUjiYoIaDmR0D/Bz4CjACOJEgIUKwPXcQbPtHAO8g2FneR9h02Ns2ODVLXJm2qWy/+Z7/Qy8TJLiDM8QzPpz+fJZlCUPQROnuqwj2+ucRbFyLgPVmdgjBn+1f3L0LeCfwqrv/1N073D0F/BZ4f4bFngv81N2fd/fdwDcylPmduz/h7h0Ef7DzegmzDtjZY9pOYFiW9/ROdx+R5fHOfKwDmEzQVHEOMBOoJvgTzcUjwNvMrJygWeSH4XgVcDTw5/A4wMXAP7v7tjDxXk2wY9DTcQS1lB+6e7u73wk80aNMO3BlOP9+gtpaXo7huftfe/m8R7j7X/OxHvb9jrqHs31HuPsIgsR+CUENV6Lx3+6+xt23EezMnh9O/wRws7v/wd273H2duy8PE8SZwBfdvcndNwH/QebtH3e/tY9tcHWWuDJtU3VZjsPl9B9hZkmC/7SfufvybB+IBH9aQ+ERglrWm8LhHQTJ7XjeqE0cCBwb7hGnx/eLDMubSHCcrNuaDGVeTxveTbDxZNMI1PeYVk+QmPOlv+toJkjiLwKY2dXAH3Nc1yPA9wmaFZ8lqBn/hCBRrXT3rWY2DqgBnkr7rRmQyLC8icA6d0+//XvPz3xruDPRra/PfNCZWWPa6Jxe/oS69fyOuod73Q7cvcnMbgA2m9ns8M9Shlb69vgawTYLMAW4P0P5A4EksCFt+y8j83/JQGTaphp7/Jayle0uv2f7M7Mygv/ENoKdKunFUHUy6U5wbw2HHyFIcOnNZWuAR3rsFdW5+2cyLG8DQQ2n25QBxvc8wUFgAMLm0hlkqf6b2QNm1pjl8UAv6zjIzNL3xg7Ptg6C9vX0H0GmH0Q2fyOoPb2H4DNdBkwl2GPt/ry3ECTRQ9M+7+HunikpbQAm9djr7M9n3p/Y92HBKQvZPu9GM3trxpUG20/3o6/kBj22g3B4o7tvzeG1ZQQ7DJNyKCv5l749TiU4Ng/B/8qMDOXXEBwTH5O2/de7e7bDEh/qYxvM1kSZaZvK9pvv+T90EMFx8u6dXCPYUR1PcOytPctypNtQHOgjaCtuIKg9QLBXso3goG8inDaMYM/rwwR7VkmC5rTZ4fxbeKNTwxkEf7qzCf5Ufsa+nUzSO6ScxN4dJl4H3pE2PpagKeAcgk4a15LlQPAAP4fHCDp1VBEknx3A2CxlPw68QtCTrwb4NfCLtPmv0qOTRI/X/y38fN8ajv8mHH9/WpkfhMsdF45PAk7r+ZkRHDddDXyOoFZ9NsEe5FU9y/aI75Rw+BrSDp6H08rDz+E7BHukVUD5EGyLFeG6HgUuCofLwnmnh9vGHILjNQ8C12RZzqkEx20S4fb8Q4I/1arBfg967PNdvErQUjGZ4BjxXwk7CwHHhL+zkwl2QiYRdh4D7g5/A/XhvBnA2/Ic26eBF8L1TiRIYtk6lh3a/Zsl6Oj2v8BtafNvCP9D6qL+zIvlMZQb4QaCJrfu8cXAAz3KzCLo+bgZ2Br+wXT3sLyFvZPWZeGf0XrgMwQJbkqWsnv9AYcb3YZwwz83nHYKsJygVvMwMG0QPoNp4bKbCXqJnpI270PA8z3KfzP8LDaHSWBkOL2CYIchay9PgsTRDFSG45eEn9H4tDJVBMfdVoU/rBeAz2f5zOYT9ERrJEiWdwL/lqlsOO1V3khwM8PX7gDuCqd9I4wn/fGNIdgOH86w3pPS5n+JoEfpLoLeeJVp854HPhQOvz/cXhrD7+c+4LCh+j3psc+21t2LcgfBDm9N2vz3ELSINBB0OOveiRsO/A9BD+CdBMdQz8tzbAZ8l2CHfls4bGnzGwl3QsPxDxLsTDYRJOBR4fQDw221JXxN9+NDUX/+hfyw8MMramY2G3iO4M+oo6/yxS7sNfhZdz+/z8KDF8PjwA3u/tOoYhAR6U3RJjgzew/BwePuJsoud393tFHFl5m9jaDWuYWgtnkDcJC7b4g0MBGRLIr5SiafIjip+GWgk6CZUgbPLIJzdHYQnND9PiU3ESlkRVuDExER6U0x1+BERESyUoITEZFYGqormexjzJgxPm3atKhWLzJgTz311BZ3z3g3iEKi35oUs4H8ziJLcNOmTWPx4sV9FxQpUGb2WtQx5EK/NSlmA/mdqYlSRERiSQlORERiSQlOpICY2elmtsLMVprZpRnm/4eZLQkfL/a4+4aIpInsGJyI7M3MEgR3oj+V4PqIT5rZQg/uBgGAu/9zWvnPEVzwWUQy6LMGZ2Y3m9kmM3suy3wzsx+Ge5xLzezI/IcpUhKOIbjjxip3bwNuI7hzQzbnE9xdXUQyyKUGdwvw3wS3fc/kDIKrxc8EjiW4Ovex+QhOhtZdqXVct2gF63c0M3FENW8/ZCwPLd+8Z/wrpwU36E4vk2na/r4un8vKZwxfOW0W7z5iSG7zNom9b7i5liy/JTM7EJhOcMcNEckgp0t1mdk04F53n5th3o3Aw+7+q3B8BcHtR3q9TuH8+fNdXZcLx12pdVx257M0t3dmLZMsMzBo7/Rep+3v6/K5rHzGUJ1M8J33vnmfJGdmT7n7/Kwr7Sczex9wurt/Mhz/MHCsu+9z52Yz+xow2d0/l2VZFwMXA0ydOvWo114rijMaRPYxkN9ZPo7BZdrrnERwvzUpEtctWtFrcgNo79o3gWSaltPrOrsY0dLA+MZtjGhuoK5tN3WtuxnW1kxd624qO9qo7GyjoqOdys52KjraKfdOEl2dlHd1kejqJOFdmHdR5k5Z+GzdDzwYJxiH4MZcAPTYqbMsNxz/j7dcwN8PPIzm9k6uW7RiKGpx69j7ztSTw2mZnAd8NtuC3H0BsACCncls5T5402OcPvcAPnL8tH4HK1LohrSTSY+9yqFctfRh/Y7mQVluXetu5mxaxZyNqzh04yqmbV/PAY1bGde4lcrO3m/d11ZWTmt5krZEkvZEOR1l5XSUJegsS9BRVkZnWYIuK6PLjC4zIHh2jC4rww0cA7M37m5q3eO217rc9l1/sMzAYH0+PTwJzDSz6QSJ7TyCG2DuxcwOAUYCfx/oClOrd3DoxPqBLkakIOUjweW815nrXqUMvYkjqlmXjz9xdw7f8CJnvfBnTl75BNN2vFGR31wzgpVjprB40mw2DhvNxrpRbKwbzfbqYTRW1NBYWRM8V1TTkqzArXDOYpk4onrQ1+HuHWZ2CbAISAA3u/vzZnYlsNjdF4ZFzwNu8zzcCsQyJHaRuMhHglsIXGJmtxEcEN+p+4QVn6+cNmtAx+CmbFrD2cse5uxljzBtxwZaE+X8ZdoR/ObNp7DigBksmzCD9dUjc1pWIR6D6+58Mtjc/X6CG/mmT7uix/g38rvOfC5NpHD0meDM7FfAScAYM1sL/P9AEsDdbyD4MZ4JrAR2Ax8brGBl8HQfX+pvT8TL3jKJ2ddfy/Tbb8Exnp4xj6cv+Aw3jjqcF1sTe173TtSLshCpAidxFtkNT9WLMgbuuw8+8xlYuxY++1n413+FCROijmrI5LsX5WDp7bd26BW/57xjpvJv75wzxFGJ5CbqXpRSajZuhC9+EW67DQ49FB59FI4/PuqoZD+YmZooJbaU4KR/1q8Pktnrr8OVV8LXvgYVFVFHJftJTZQSZ0pwkruGBnjnO2HrVvjrX+Hoo6OOSPLAs5wHKFLslOAkNx0dcO65sHQp3HOPkltcmHpRSnwpwUnf3IPOJL//PSxYAGecEXVEkidqopQ4K5wzaaVwXX01/PjHcPnlcNFFUUcjIpITJTjp3W9/C1//OlxwAXzrW1FHI3lmupSJxJgSnGTX3g7/8i8wbx785Ce6rlNMRXUurMhg0zE4ye6WW+DVV4MTunUqQCyZoT6UEluqwUlmra1w1VVw7LHqVBJjqpNLnKkGJ5ndfDOsXg033aSmyZhTC6XElWpwsq+WFvj2t+GEE+DUU6OORgaROplInKkGJ/u66SZYtw5+/nPV3kqArmQicaUanOytuTk47+1tb4O3vz3qaGSQGWqilPhSDU72dsMNwYWUb79dtbcSoK9Y4kw1OHlDUxNccw2ccgqceGLU0cgQUQVO4koJTt7wxz/Cpk1w6aVRRyJDRveDk/hSgpM3PP00lJXp5qUlRE2UEmdKcPKGVApmzYKamqgjkSGlKpzEkxKcvCGVgiOOiDoKGUKqwEmcKcFJYMsWWLtWCS5iZna6ma0ws5VmlvFgqJmda2bLzOx5M7t1oOvUMTiJK50mIIFUKnhWgouMmSWA64FTgbXAk2a20N2XpZWZCVwGnODu281s3MDWqQQn8aUanASU4ArBMcBKd1/l7m3AbcDZPcpcBFzv7tsB3H3TQFZoaqSUGFOCk0AqBVOnwqhRUUdSyiYBa9LG14bT0h0MHGxmj5rZY2Z2eqYFmdnFZrbYzBZv3ry515XqUl0SV0pwElAHk2JRDswETgLOB24ysxE9C7n7Anef7+7zx44dm3VhaqKUOFOCE2hshBdfVIKL3jpgStr45HBaurXAQndvd/dXgBcJEp6I9KAEJ7B0abAbrwQXtSeBmWY23cwqgPOAhT3K3EVQe8PMxhA0Wa7a3xUaOgtO4ksJTtTBpEC4ewdwCbAIeAH4tbs/b2ZXmtlZYbFFwFYzWwY8BHzF3bfu7zp1PziJM50mIEGCGz0aJk+OOpKS5+73A/f3mHZF2rADXwofeVpnvpYkUlhUg5M3Ophob74kqRelxJUSXKlrb4fnnlPzZInSPo3EmRJcqVu2DNralOBKmSpwElM5Jbi+ro9nZlPN7CEzS5nZUjM7M/+hyqBQB5OSphqcxFmfCS7t+nhnAHOA881sTo9iXyfo8XUEQdfmH+U7UBkkqVRwe5yZOpWqVKkCJ3GVSw0ul+vjOVAfDg8H1ucvRBlUqRQcfjgkElFHIhEwDFc3SompXBJcLtfH+wZwgZmtJeji/Lm8RCeDq6sLlixR82QJUxOlxFm+OpmcD9zi7pOBM4FfmNk+y+7PBWBlCKxaBQ0NSnAlTvU3iatcElwu18f7BPBrAHf/O1AFjOm5oFwvACtDRB1MSp6hE70lvnJJcLlcH281cDKAmc0mSHCqohW6VArKy2Hu3KgjkYjoUl0SZ30muByvj/dl4CIzewb4FXCh68h14UulYM4cqKyMOhKJkH6oElc5XYsyh+vjLQNOyG9oMuhSKTg94/0ypUSo/iZxpiuZlKoNG2DjRh1/K3Fm0NWlOpzEkxJcqVIHEwGSiTLaOruiDkNkUCjBlaruBDdvXrRxSKQqy8toV4KTmFKCK1WpFMyYAfX1fZeV2EomlOAkvpTgSlX3PeCkpCUTZbR36BicxJMSXCnauTO4iokSXMlLlusYnMSXElwpWrIkeFaCK3kVCVMTpcSWElwpUg9KCekYnMSZElwpSqXggAOCh5S0ZKKMtg4lOIknJbhSpA4mBcvMTjezFWa20swuzTD/QjPbbGZLwscnB7K+oAanTiYSTzldqktipKUFli2Dd70r6kikBzNLANcDpxLcd/FJM1sYXgov3e3ufkk+1lmhTiYSY6rBlZrnnoPOTtXgCtMxwEp3X+XubcBtwNmDuUJ1MpE4U4IrNepgUsgmAWvSxteG03o6x8yWmtkdZjYlw/ycBefBKcFJPCnBlZpUKrh6yfTpUUci++ceYJq7Hwb8AfhZpkJmdrGZLTazxZs3Z781Y7Jcx+AkvpTgSk0qFVx/skxffQFaB6TXyCaH0/Zw963u3hqO/hg4KtOC3H2Bu8939/ljx47NusLuiy3r9o0SR/qXKyWdnbB0qZonC9eTwEwzm25mFcB5wML0AmY2IW30LIKbEO+3ikRwR7gO3TJHYki9KEvJiy/C7t1KcAXK3TvM7BJgEZAAbnb3583sSmCxuy8EPm9mZwEdwDbgwoGsM5kI9nHbOrr2DIvEhRJcKVEHk4Ln7vcD9/eYdkXa8GXAZflaX3dSU09KiSPtspWSVAoqK2H27KgjkQKRLA9rcEpwEkNKcKUklYK5cyGZjDoSKRCVe2pwOgYn8aMEVyrcdYku2UeyPOhkonPhJI6U4ErFmjWwbZsSnOxFx+AkzpTgSoU6mEgGe3pRKsFJDCnBlYpUCszgsMOijkQKSIWOwUmMKcGVilQKZs2C2tqoI5ECkn4enEjcKMGVCnUwkQyS4ZVMdAxO4kgJrhRs3Rp0MlGCkx50HpzEmRJcKVAHE8lizzE4NVFKDCnBlQIlOMmiolydTCS+lOBKQSoFU6bA6NFRRyIF5o3TBDojjkQk/5TgSoE6mEgWlWENrrVdTZQSP0pwcdfUBCtWKMFJRnsSnI7BSQzllODM7HQzW2FmK83s0ixlzjWzZWb2vJndmt8wZb8tXRpch/LII6OORApQVTIBQEu7miglfvq8H5yZJYDrgVOBtcCTZrbQ3ZellZlJcI+qE9x9u5mNG6yApZ/UwUR60V2Da1ETpcRQLjW4Y4CV7r7K3duA24Cze5S5CLje3bcDuPum/IYp+y2VCjqXTJ4cdSRSgMoTZZSXGa0dqsFJ/OSS4CYBa9LG14bT0h0MHGxmj5rZY2Z2eqYFmdnFZrbYzBZv3rx5/yKW/unuYGIWdSRSoKqSCdXgJJby1cmkHJgJnAScD9xkZiN6FnL3Be4+393njx07Nk+rlqza2+HZZ9U8Kb2qSpbRohqcxFAuCW4dMCVtfHI4Ld1aYKG7t7v7K8CLBAlPorRsGbS1KcFJryrLE+pkIrGUS4J7EphpZtPNrAI4D1jYo8xdBLU3zGwMQZPlqjzGKftDHUwkB5XJMp0mILHUZ4Jz9w7gEmAR8ALwa3d/3syuNLOzwmKLgK1mtgx4CPiKu28drKAlR6kU1NTATFWmi0Uup+SE5c4xMzez+QNdZ1V5glbV4CSG+jxNAMDd7wfu7zHtirRhB74UPqRQpFJw+OGQSEQdieQgl1NywnLDgC8Aj+djvZXJMnUykVjSlUziqqsLlixR82RxyeWUHIBvAdcCLflYaVV5QqcJSCwpwcXVqlXQ0KAEV1z6PCXHzI4Eprj7fflaaZVqcBJTSnBxpQ4msWNmZcD3gS/nUDbnc06D8+BUg5P4UYKLq1QKysth7tyoI5Hc9XVKzjBgLvCwmb0KHAcszNTRpD/nnFaWqxelxJMSXFylUjBnDlRWRh2J5K7XU3Lcfae7j3H3ae4+DXgMOMvdFw9kparBSVwpwcWV7gFXdHI8JSfvlOAkrnI6TUCKzIYNsHGjElwR6uuUnB7TT8rHOtVEKXGlGlwcqYOJ9ENlMkFrRxfB6awi8aEEF0fdCW7evGjjkKJQldRdvSWelODiKJWCGTOgvj7qSKQIVJbrrt4ST0pwcaQOJtIP1ckgwTUrwUnMKMHFzc6dwVVMlOAkR7WVQYLb3aYEJ/GiBBc3S5YEz0pwkqPaiqAz9e5WJTiJFyW4uFEPSumnmrAG19TWEXEkIvmlBBc3qRQccEDwEMnBnhqcEpzEjBJc3KiDifRT9zG4JjVRSswowcVJSwssW6YEJ/1SoxqcxJQSXJw89xx0dirBSb90N1GqBidxowQXJ+pgIvuhuqL7NAHV4CRelODiJJUKrl4yfXrUkUgRqSgvoyJRRpPOg5OYUYKLk1QquP5kmb5W6Z+aygS7W1WDk3jRP2FcdHbC0qVqnpT9UltRrhqcxI4SXFy8+CLs3q0EJ/ulpiKhY3ASO0pwcaEOJjIANZXl6kUpsaMEFxepFFRWwuzZUUciRai2IkGTjsFJzCjBxUUqBXPnQjIZdSRShGp0DE5iSAkuDtx1iS4ZkGFV5TS2tkcdhkheKcHFwZo1sG2bEpzst/qqcnY1q4lS4kUJLg7UwUQGqL46SUNLO11dHnUoInmjBBcHqRSYwWGHRR2JDJCZnW5mK8xspZldmmH+p83sWTNbYmZ/NbM5+Vjv8OokXQ6NOlVAYkQJLg5SKZg1C2pro45EBsDMEsD1wBnAHOD8DAnsVnd/s7vPA74LfD8f666vCjon7WrWcTiJDyW4OFAHk7g4Bljp7qvcvQ24DTg7vYC770obrQXy0qZYXx0kuJ1KcBIjOSW4vppN0sqdY2ZuZvPzF6L0auvWoJOJElwcTALWpI2vDaftxcw+a2YvE9TgPp9pQWZ2sZktNrPFmzdv7nPF9dXBLXPU0UTipM8El2OzCWY2DPgC8Hi+g5ReqINJyXH36919BvA14OtZyixw9/nuPn/s2LF9LnO4anASQ7nU4PpsNgl9C7gWaMljfNIXJbg4WQdMSRufHE7L5mHXy8QAABeESURBVDbg3flY8Z5jcC1KcBIfuSS4PptNzOxIYIq739fbgvrbbCI5SKVgyhQYPTrqSGTgngRmmtl0M6sAzgMWphcws5lpo/8IvJSPFQ+vUScTiZ/ygS7AzMoIenJd2FdZd18ALACYP3++TrjJB3UwiQ137zCzS4BFQAK42d2fN7MrgcXuvhC4xMxOAdqB7cBH87HuuopyzJTgJF5ySXB9NZsMA+YCD5sZwAHAQjM7y90X5ytQyaCpCVasgA98IOpIJE/c/X7g/h7Trkgb/sJgrLeszBhWWa5jcBIruTRR9tps4u473X2Mu09z92nAY4CS21BYujS4DqVqcJIHw2uSSnASK30mOHfvALqbTV4Aft3dbGJmZw12gNILdTCRPBpVW8nWpraowxDJm5yOwfXVbNJj+kkDD0tykkrBqFFBJxORARpTW8GGneoELfGhK5kUs+4OJsGxT5EBGV1Xwdam1qjDEMkbJbhi1d4Ozz6r5knJm9F1lWxrasNdHZwlHpTgitULL0BbmxKc5M3o2graO51dLbpcl8SDElyxUgcTybMxdZUAbG1UM6XEgxJcsUqloKYGDj446kgkJkbXVQCoJ6XEhhJcsUqlghucJhJRRyIxMbpWNTiJFyW4YtTVBUuWqHlS8mpMWIPb0qganMSDElwxeuUV2LVLCU7yamRtBWawuUE1OIkHJbhipA4mMgiSiTLG1FXyuk72lphQgitGqVRw7G3u3KgjkZiZOLyK9Tubow5DJC+U4IpRKgVz5kBVVdSRSMxMGF6ty3VJbCjBFSPdA04GyYQRVWzY0ayrmUgsKMEVm9dfDx5KcDIIJg6vpqmtU1czkVhQgis26mAig+iA4UGz9wYdh5MYUIIrNt0Jbt68aOOQWJo4IkxwO3QcToqfElyxSaXgoINg+PCoI5EYmjiiGoC1O1SDk+KnBFds1MFEBtH4YVVUlpfx2pamqEMRGTAluGKycye8/LISnAyasjJj2uhaXt2qBCfFTwmumDzzTPCsBBdbZna6ma0ws5VmdmmG+V8ys2VmttTM/mRmB+Y7huljanlFNTiJASW4YqIelLFmZgngeuAMYA5wvpnN6VEsBcx398OAO4Dv5juOaWNqWb1tN51dOhdOipsSXDFJpWD8eJgwIepIZHAcA6x091Xu3gbcBpydXsDdH3L33eHoY8DkfAdx0Jha2judddvV0USKmxJcMVEHk7ibBKxJG18bTsvmE8AD+Q5i2phaAF7RcTgpckpwxaK1FZYtU4ITAMzsAmA+cF2W+Reb2WIzW7x58+Z+LXvG2CDBvbSxYaBhikRKCa5YPPccdHQowcXbOmBK2vjkcNpezOwU4HLgLHfPePM2d1/g7vPdff7YsWP7FcToukrGDatk2YZd/XqdSKFRgisW6mBSCp4EZprZdDOrAM4DFqYXMLMjgBsJktumwQpkzsR6XtigGpwUNyW4YpFKwbBhwVVMJJbcvQO4BFgEvAD82t2fN7MrzeyssNh1QB3wGzNbYmYLsyxuQGZPqGflpgbaOroGY/EiQ6I86gAkR6lUcP3JMu2TxJm73w/c32PaFWnDpwxFHLMn1NPe6azc1MicifVDsUqRvNO/ZTHo7AxO8lbzpAyROROCpPbc+p0RRyKy/5TgisFLL8Hu3UpwMmQOGlNLfVU5qdXbow5FZL8pwRUDdTCRIVZWZhx14EiefFUJToqXElwxSKWgogLm9Lxqk8jgmT9tFCs3NbJjd1vUoYjsl5wSXCFcALakpVIwdy4kk1FHIiXkqANHAvDUa6rFSXHqM8EVygVgS5a7LtElkZg3ZQQViTL+/vLWqEMR2S+51OAK4gKwJWvtWti6VQlOhlxVMsGxB43i4Rf7d6kvkUKRS4IriAvAlix1MJEIve3gsazc1Miabbv7LixSYPLayWQwLwBbslIpMIPDDos6EilBbz9kHIBqcVKUcklwBXEB2JKVSsHBB0NdXdSRSAk6aEwtB46u4Q/LNkYdiki/5ZLgCuYCsCVJHUwkQmbGP755Ao+u3MKWxoz7rSIFq88EV0gXgC05W7fC6tVKcBKps+dNorPLuf/ZDVGHItIvOV1suVAuAFtyliwJnpXgJEKzDhjGIQcM43epdXzk+GlRhyOSM13JpJCpB6UUiHOOnExq9Q6WrddNUKV4KMEVsgcfhBkzYMyYqCOREnfu/ClUJxP87G+vRh2KSM6U4ArV9u3wxz/COedEHYkIw2uSvPuISdy1ZB3bmnRtSikOSnCF6p57oL0d3ve+qCMRAeDjJ0yjrbOLm/6yKupQRHKiBFeo7rgDpk6F+fOjjkQEgJnjh/GuwyZyy6Ov6pQBKQpKcIVo1y5YtCiovZlFHY3IHl84ZSatHZ3894Mrow5FpE9KcIXo3nuhrU3H36TgzBhbx/nHTOUXj72mHpVS8JTgCtEdd8DEiXDccVFHIrKPr552CCOqk1x+17N0dXnU4YhkpQRXaBob4YEHgtpbmb6eUpPDzYVPNLOnzazDzCLpgTS8Jsnl/zib1OodLFCHEylg+gctNPffDy0t6j1ZgnK8ufBq4ELg1qGNbm/vOWIS//jmCXxv0QpSq3XHbylMSnCF5o47YPx4OOGEqCORoZfLzYVfdfelQFcUAXYzM65+75sZX1/FZ3/5NJt2tUQZjkhGSnCFZPduuO8+eO97IZGIOhoZev29uXCkhlcnufHDR7GjuZ1P/nwxu9s6og5JZC9KcIXk978PkpyaJ2WAhurmwnMnDee/zj+C59bt5FO/eIqW9s5BW5dIfynBFZLf/ja47uSJJ0YdiUQjp5sL52Ioby588uzxXHvOYfx15RY+8bMnaW5TkpPCoARXKFpagstzvec9UJ7TXYwkfvq8uXChev/8KXzvfYfzt5e3ct6Cv+uYnBQEJbhC8dvfQkODTu4uYbncXNjMjjaztcD7gRvN7PnoIt7bOUdN5sYLjuKlTY2cff2jLF27I+qQpMSZezQnas6fP98XL14cyboLTlsbzJ4Nw4bB00/r/LciYWZPuXvBXyx0qH9ry9bv4qKfL2ZTQwtffscsLnrrQSTKdMk52T8D+Z3pn7QQLFgAq1bBNdcouUnRmzOxnvs+/xZOmT2eax5Yzvk3PcZLGxuiDktKkP5No9bQAFdeCW9/O5x2WtTRiOTFiJoKfvShI/nuOYexfMMuzvjBX/jOAy/Q2KpTCWToKMFF7fvfh82b4dprdecAiRUz49yjp/DQv5zEe46YxI2PrOLE7z7Egj+/rJ6WMiSU4KK0cSN873vBeW9HHx11NCKDYnRdJde9/3Du+uwJHDqxnqvvX86J1z3Ejx5eyXbdHVwGkRJclK66Cpqb4dvfjjoSkUE3b8oIfvGJY/n1p47n4PF1fPf3Kzj+mj9x2Z1LeW7dTqLq8CbxpROuovLyy3DjjXDRRXDwwVFHIzJkjpk+il9+8jiWv76LWx59lTufXsevnljDIQcM471HTuLseZMYX18VdZgSAzpNICof/CDcfTesXAkTJkQdjewHnSaQHzt2t3HP0g3c+fRaUqt3YAbzDxzJqXPGc+qcA5g+pjbqECVCA/mdqQYXheuvh1/9Cr7+dSU3KXkjair48HEH8uHjDmTV5kbuXrKePyzbyNX3L+fq+5fzpnF1nDhzLCe8aTTHTB/FsKpk1CFLkVANbqj98pdwwQVw1lnBrXGS+rEWK9XgBtfa7bv547KN/Gn5Jp54ZRutHV0kyow3TxrO8TNGc9TUkcybOoIxdZVRhyqDSDW4YnHPPfDRjwbnvN1+u5KbSC8mj6zhwhOmc+EJ02lp7yS1egd/f3kLf3t5Kzf9eRX/0+VhuWrmTRnBEVNHMndiPYccUM/wGv22RAlu6Dz8MLz//XDkkcGxtyodRBfJVVUywfEzRnP8jNF8CWhu6+S59TtZsnoHqTXbefq17dy7dMOe8hOHV3HIhHoOOWAYh0yoZ9b4YRw4uoaqpO6zWEqU4IbCE08ETZIzZsADDwTXnBSR/VZdkeDoaaM4etqoPdM27mph2YZdLN/QwPLXd7Hi9Qb+/OJmOsKanhlMHF7NQWNrmT5m78fEEdUkEzprKm6U4AZTQwN885vwgx/A5Mnwf/8Ho0dHHZVILI2vr2J8fRVvnzVuz7S2ji5e3tzIixsbeHXLbl7Z0sgrW5r43dPraEi7bFiZBa+fNKKaSSOrmTSimonh8ORwuLZSf5fFRt/YYHCH226DL38ZNmyAj388uJDyIN94UkT2VlFexuwJ9cyeUL/XdHdna1Mbr2xp4pXNTazd0cy67c2s27Gbp1dv576lG/bU/LrVViQYV1/F2GGVjBtWyfj6KsYNq2RcfSXjhlUxvr6SsXVVDKsqp0x3TygISnD51NYGDz4YJLNHHgmOt915Jxx3XNSRiUgaM2NMXSVj6ir3aubs1tnlbGpoCZNeMxt2trBpVysbG1rYvKuV59bt5MHlm9id4ZqaiTJjZE0Fo2qTjKypYHRdRTj+xqN7fERNkvrqJHUVSoqDIacEZ2anAz8AEsCP3f2aHvMrgZ8DRwFbgQ+4+6v5DbVAtbbCH/4QdPm/+27YsQNGjYIbboBPfhISOqgtUmwSZcaE4dVMGF5Nb/3TG1s72LgrSH6bGlrY0tjG9qY2tjYFz9ua2nhxYyPbm9rYvruNrixnZZUZDKtKUl9dzvDqJPVVwWN4dTCtvirJ8JpwejheW1lOXWU5NRUJaivLqSwvw3TB9r30meDMLAFcD5wKrAWeNLOF7r4srdgngO3u/iYzOw+4FvjA/gR0V2od1y1awfodzUwcUc1XTpsF0Oe0tx8yloeWbx6813V1cdPvnqDulZUc07iO95Zvo37lcqqXP09NWwsNVbWsPO4f+NXUY7h77KGM2T6cryx9Pa8xvPuISfvzkYrIIKmrLKdubB0zxtb1Wbazy9nV3B4kv91tbG1sY1dzO7ta2tnZ3B4Od7CrORhftaWRXc0d7Gxup7m977svlJfZnqRXW5nYJwEG08PncFpNRTnVFWVUJRNUJxNUV4TPyQRV4XAxd77p80RvMzse+Ia7nxaOXwbg7t9JK7MoLPN3MysHXgfGei8Lz3Ty6V2pdVx257PUbd9CTXszAEkDDDo7vTtgkuHn3dHpGI55+BzOL3MnaU4Zjnd0UtbVRcI7qaSLcu+C9g4qOtvfeHS0U9nZTm1bM/XtzdS2NVPTupvhzQ2Ma9rOuKbtjG7aQbLrjY1sR1UdK8ZNZ9nYaTwy/SgenXY47Ym9z71JlhkYtHd6r9N6ylSmOpngO+99s5JcAdGJ3jJU2jq6aOhOhGESbGrtoKmtk6bWDhpbO4Lx1g4aWzvDed3TOoP54Xhv/z2ZJBO2TwLMOF5RRnUyQWV5gsryIGlWJsveGC4v2zOvMtldJpg2aUR11ibawT7RexKwJm18LXBstjLu3mFmO4HRwJb+BHPdohU0t3dy7YM3cdYLf+7PS/OmvSxBY0UNTRXV7KqqZVPtKJaPncbm2pFsrh3JqyMn8sK4aWysG93n/dvaM7RHZJqWS5nm9k6uW7RCCU6kBFWUlzG6rpLRebhqS2tHJ01hEmxu76S5rTN4bu+kJW24ua3HvL3KdtHS1smmhhaa2zppae/a85rWjs6sTbHZrLjqdCrL8n84Z0g7mZjZxcDFAFOnTt1n/vodQa3tF0ecyYMz3rg/Wvdn5XsSiu01zbHwORjvsrLwYeGjjI6yBJ1lCTrKEnRZGW2JctoSSdrKk7QlkrSWV9BUUU1rIlmwNx7t/nxERPZXUItKMKq2YlCW7+50dDmtHV20tnfSEj63dnTR2tFFS/dw+NzS3knFIDWD5pLg1gFT0sYnh9MylVkbNlEOJ+hsshd3XwAsgKDZpOf8iSOqWbejmSenzOXJ3OIvKRNHVEcdgohIr8yMZMJIJsqoi/jcwVzS5pPATDObbmYVwHnAwh5lFgIfDYffBzzY2/G3bL5y2iyqe1xKJ1kWfFh9TetpsF831DFUJxN7Op+IiEjf+kyv4TG1S4BFBKcJ3Ozuz5vZlcBid18I/AT4hZmtBLYRJMF+6z6+VJC9KAsgBh1/iz+dkiOSP7pdjsh+yncvyvCUnBdJOyUHOD/9lBwz+yfgMHf/dHhKznvcvddTcvRbk2I2kN9Z8Z7gIBI/xwAr3X2Vu7cBtwFn9yhzNvCzcPgO4GTT2b0iGSnBiRSOTKfk9GyX3uuUHKD7lJy9mNnFZrbYzBZv3rx5kMIVKWxKcCIx5O4L3H2+u88fq4t8S4lSghMpHP05JYfeTskRESU4kUIyZKfkiJSCyHpRmtlm4LVeioyhn5f6KiDFHDsUd/xDGfuB7p7X9j8zOxP4T944Jefb6afkmFkV8AvgCMJTctx9VR/LjPNvLZs4vieI5/vq6z3t9+8ssgTXFzNbXAwXss2kmGOH4o6/mGOPShw/szi+J4jn+xrM96QmShERiSUlOBERiaVCTnALog5gAIo5diju+Is59qjE8TOL43uCeL6vQXtPBXsMTkREZCAKuQYnIiKy3woywZnZ6Wa2wsxWmtmlUcfTGzO72cw2mdlzadNGmdkfzOyl8HlklDFmY2ZTzOwhM1tmZs+b2RfC6QUfv5lVmdkTZvZMGPs3w+nTzezxcNu5PTyfTDIo9N9Zf7dPC/wwfD9LzezItGV9NCz/kpl9NG36UWb2bPiaHw7ldT3NLGFmKTO7NxzPuO2aWWU4vjKcPy1tGZeF01eY2Wlp04f8uzWzEWZ2h5ktN7MXzOz4yL8rdy+oB8H5Py8DBwEVwDPAnKjj6iXeE4EjgefSpn0XuDQcvhS4Nuo4s8Q+ATgyHB5GcCX7OcUQP2BAXTicBB4HjgN+TXBuGMANwGeijrUQH8XwO+vv9gmcCTwQbhvHAY+H00cBq8LnkeHwyHDeE2FZC197xhC+vy8BtwL3huMZt13gn4AbwuHzgNvD4Tnh91YJTA+/z0RU3y3BRcA/GQ5XACOi/q4i34gzfEjHA4vSxi8DLos6rj5insbeCW4FMCEcngCsiDrGHN/H3QS3aimq+IEa4GngWIITRsszbUt67PWZFePvrNftE7iR4PZC3eVXhPPPB25Mm35jOG0CsDxt+l7lBvm9TAb+BPwDcG/4p51x2yW4F+fx4XB5WM56fmfd5aL4bgkuGfcKYb+Ont9BVN9VITZR5nJF9UI33t03hMOvA+OjDCYXYbPHEQQ1oaKIP2ziWQJsAv5AsNe6w4Or7ENxbjtDpah+Zzlun9neU2/T12aYPhT+E/gq0BWOjyb7tpvtDhL9fb+DaTqwGfhp2Oz6YzOrJeLvqhATXKx4sLtR0F1VzawO+C3wRXfflT6vkON39053n0ewN3wMcEjEIckgKNbtMxszeyewyd2fijqWPConOFTzP+5+BNBE0CS5RxTfVSEmuFyuqF7oNprZBIDweVPE8WRlZkmCP49fuvud4eSiiR/A3XcADxE0zYyw4Cr7UJzbzlApit9ZP7fPbO+pt+mTM0wfbCcAZ5nZqwQ3tf0H4Adk33az3UGiv+93MK0F1rr74+H4HQQJL9LvqhATXC5XVC906Vd8/yjBsYOCE/ZC+gnwgrt/P21WwcdvZmPNbEQ4XE1wbOYFgkT3vrBYQcZeIAr+d7Yf2+dC4CNhD73jgJ1h89gi4B1mNjLsxfcOgmNUG4BdZnZcuK6PMATbi7tf5u6T3X0awef+oLt/iOzbbrY7SCwEzgt7WU4HZhJ0xBjy79bdXwfWmNmscNLJwDKi/q6G4oDqfhywPJOgx9TLwOVRx9NHrL8CNgDtBHsxnyBoH/8T8BLwR2BU1HFmif0tBE0GS4El4ePMYogfOAxIhbE/B1wRTj+I4Ee+EvgNUBl1rIX6KPTfWX+3T4KOF9eH7+dZYH7asj4ebhMrgY+lTZ8fbj8vA/9Nj04SQ/AeT+KNXpQZt12gKhxfGc4/KO31l4exryCtV2EU3y0wD1gcfl93EfSCjPS70pVMREQklgqxiVJERGTAlOBERCSWlOBERCSWlOBERCSWlOBERCSWlOBERCSWlOBERCSWlOBERCSW/h8GbOueTydzjQAAAABJRU5ErkJggg==\n"
          },
          "metadata": {
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Three fundamental differences between the twoe methods.\n",
        "\n",
        "1.  The *_grad parameter of the tensor lets PyTorch know that the values in that tensor are those which need to be changed, so that our logistic regression can give us the optimal BCE.\n",
        "2.  When you use PyTorch’s built-in functionality (e.g., autograd), the code is optimized and actually runs faster than writing things from scratch. Can be seen from high learning rate and low epochs.\n",
        "1. Replaced our formulae derived via calculus with the line loss.backward(). That 1 line represents backpropagation. The one line code tells PyTorch that we’re looking for the derivatives of the weights (i.e., slope and bias) with respect to our loss (which in this case is BCE)\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "\n"
      ],
      "metadata": {
        "id": "mywMk800uIiu"
      }
    }
  ]
}