deep-learning-for-fonts-strata.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Deep Learning for Fonts | Fontastic\n",
    "\n",
    "- Nischal HP | @nischalhp | VP, Engineering, omni:us\n",
    "\n",
    "- Raghotham S | @raghothams | Senior Data Scientist, Ericsson Research\n",
    "\n",
    "#### Strata Data Conference 2019, London"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Fontastic\n",
    "\n",
    "## Motivation\n",
    "\n",
    "<img src=\"imgs/london-bus.png\" alt=\"drawing\" width=\"600\"/>\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "# Fontastic\n",
    "\n",
    "### Existing Tools | What The Font\n",
    "\n",
    "<img src=\"imgs/whatthefont.png\" alt=\"drawing\" width=\"600\"/>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "# Fontastic\n",
    "\n",
    "### Existing Tools | What Font is?\n",
    "\n",
    "<img src=\"imgs/whatfontis-1.png\" alt=\"drawing\" width=\"800\" style=\"alig:center\"/>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "# Fontastic\n",
    "\n",
    "### Existing Tools | What Font is?\n",
    "\n",
    "<img src=\"imgs/whatfontis-2.png\" alt=\"drawing\" width=\"800\" style=\"alig:center\"/>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "# Fontastic\n",
    "\n",
    "### Existing Tools | Matcherator\n",
    "\n",
    "<img src=\"imgs/matcherator.png\" alt=\"drawing\" width=\"800\" style=\"alig:center\"/>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "\n",
    "# Fontastic\n",
    "\n",
    "## What do we aim to do?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "# Deep Learning for Humans\n",
    "\n",
    "* Fontastic"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "\n",
    "\n",
    "* Upcoming Projects"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Fontastic\n",
    "\n",
    "## Agenda\n",
    "\n",
    "* Data acquisition\n",
    "* Model building\n",
    "* Feature visualization"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Fontastic\n",
    "\n",
    "## Data Acquisition"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## Data Acquisition\n",
    "\n",
    "### Pass 1: Scrape Font Squirel - https://www.fontsquirrel.com/\n",
    "\n",
    "<img src=\"imgs/fontsquirrel.png\" alt=\"drawing\" width=\"800\" style=\"alig:center\"/>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "\n",
    "**Problems**\n",
    "\n",
    "1. We have images of different dimensions\n",
    "2. Even with normalizing the size, we will end up with 5-10 images per style\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## Data Acquisition\n",
    "\n",
    "### Pass 2: Scrape DaFont - https://www.dafont.com/\n",
    "\n",
    "<img src=\"imgs/dafont.png\" alt=\"drawing\" width=\"800\" style=\"alig:center\"/>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "\n",
    "**Problems**\n",
    "\n",
    "1. Old school fonts only, not updated frequently\n",
    "2. Supports only wide dimension, might not work well with inteded end use\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## Data Acquisition\n",
    "\n",
    "### Pass 3: Generate Image using PIL\n",
    "\n",
    "#### Steps\n",
    "\n",
    "1. Create 4 set of random texts\n",
    "2. Generate 4k resolution image using the TTF for every random text\n",
    "3. Take 10 random crop of size 256x256 px from the 4k image\n",
    "\n",
    "With this we have the ability to generate large number of training images \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## Data Acquisition\n",
    "\n",
    "### Pass 3: Generate Image using PIL\n",
    "\n",
    "#### Advantages\n",
    "\n",
    "1. We control the input text\n",
    "2. We control the font style and size\n",
    "3. We control the output image dinemsion\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## Data Acquisition\n",
    "\n",
    "### Pass 3: Generate Image using PIL\n",
    "\n",
    "\n",
    "<img src=\"imgs/pil-generated.jpg\" alt=\"drawing\" width=\"800\" style=\"alig:center\"/>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## Data Acquisition\n",
    "\n",
    "### Pass 3: Generate Image using PIL + Random Crop\n",
    "\n",
    "\n",
    "<img src=\"imgs/pil-rand-crop.jpg\" alt=\"drawing\" width=\"800\" style=\"alig:center\"/>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## Data Acquisition\n",
    "\n",
    "### Pass 3: Generate Image using PIL + Random Crop\n",
    "\n",
    "\n",
    "<img src=\"imgs/pil-rand-crop-all.png\" alt=\"drawing\" width=\"800\" style=\"alig:center\"/>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Fontastic\n",
    "\n",
    "## Model Building"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## Model Building\n",
    "### Phase I | Feasibility Check - FastAI"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "160cc4dc28c9479dba6d59e14065d736",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(IntProgress(value=0, description='Epoch', max=6), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[0.      1.05857 0.88758 0.6628 ]                           \n",
      "[1.      0.69731 0.59468 0.77709]                            \n",
      "[2.      0.51771 0.45974 0.84326]                            \n",
      "[3.      0.4064  0.35457 0.86119]                            \n",
      "[4.      0.34457 0.32807 0.87547]                            \n",
      "[5.      0.26355 0.24554 0.91429]                            \n",
      "\n"
     ]
    }
   ],
   "source": [
    "PATH = \"data/\"\n",
    "sz=225\n",
    "arch=resnet50\n",
    "bs=28\n",
    "\n",
    "tfms = tfms_from_model(arch, sz, aug_tfms=transforms_side_on, max_zoom=1.1)\n",
    "data = ImageClassifierData.from_paths(PATH, tfms=tfms, bs=bs, num_workers=4)\n",
    "learn = ConvLearner.pretrained(arch, data, precompute=True, ps=0.5)\n",
    "\n",
    "learn.unfreeze()\n",
    "lr=np.array([1e-4,1e-3,1e-2])\n",
    "\n",
    "learn.fit(lr, 6, cycle_len=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## Model Building - Feasibility Check - FastAI"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[200   0   3   3   0   3   7]\n",
      " [  0  94   0   0   0   0   2]\n",
      " [  3   0 423   1   0   5   0]\n",
      " [  4   0   0 105   0   0  11]\n",
      " [  0   0   0   0 179   0   1]\n",
      " [  3   0  18   0   0 195   0]\n",
      " [  3   0   0   3   2   2 206]]\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAVoAAAEmCAYAAAAjsVjMAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzsnXd8lMX2h58viaCICEgREpAqCEgvihQv2AGxgGAFQb1WrD/btZdrvZdruXpt2BXFhiAISFFE6SACFlBQCEgHAUFCOL8/3gksMdndJLubbJiHz3x2Z95558xMlrOz5505R2aGx+PxeOJHqaLugMfj8ZR0vKL1eDyeOOMVrcfj8cQZr2g9Ho8nznhF6/F4PHHGK1qPx+OJM17ReooNkg6SNFLSZknDC9HO+ZLGxbJvRYWkTpJ+KOp+eAqH/D5aT36RdB5wA9AI2ALMAx40sy8L2e6FwDVABzPbVeiOFnMkGdDAzJYUdV888cWvaD35QtINwH+AfwLVgFrAM0CvGDR/BPDj/qBko0FSalH3wRMjzMwnn6JKwKHAVqBPmDplCBTxSpf+A5Rx144HVgA3AmuAVcDF7tq9wE4g08kYBNwDvBHSdm3AgFSXHwD8TLCqXgqcH1L+Zch9HYCZwGb32iHk2mTgfmCqa2ccUDmPsWX3/+aQ/p8BnAb8CGwAbg+p3w74Gtjk6j4NlHbXvnBj2ebG2zek/VuA34DXs8vcPfWcjFYuXwNYBxxf1J8Nn8Inv6L15IdjgQOBD8PU+QdwDNACaE6gbO4IuX44gcJOI1Cm/5VU0czuJlglv2Nm5czspXAdkXQw8CRwqpkdQqBM5+VSrxLwiat7GPBv4BNJh4VUOw+4GKgKlAZuCiP6cII5SAPuAl4ALgBaA52AuyTVdXWzgOuBygRz1w24EsDMOrs6zd143wlpvxLB6v6yUMFm9hOBEn5TUlngZeAVM5scpr+eYoBXtJ78cBiwzsL/tD8fuM/M1pjZWoKV6oUh1zPd9UwzG02wmmtYwP7sBppKOsjMVpnZwlzqdAcWm9nrZrbLzN4Gvgd6htR52cx+NLPtwLsEXxJ5kUlgj84EhhEo0SfMbIuTvxBoBmBms81smpO7DHgO6BLFmO42sz9df/bBzF4AFgPTgeoEX2yeYo5XtJ78sB6oHMF2WAP4JST/iyvb00YORf0HUC6/HTGzbQQ/ty8HVkn6RFKjKPqT3ae0kPxv+ejPejPLcu+zFeHqkOvbs++XdKSkUZJ+k/Q7wYq9cpi2Adaa2Y4IdV4AmgJPmdmfEep6igFe0Xryw9fADgK7ZF6sJPjZm00tV1YQtgFlQ/KHh140s7FmdiLByu57AgUUqT/ZfcooYJ/yw7ME/WpgZuWB2wFFuCfsNiBJ5Qjs3i8B9zjTiKeY4xWtJ2rMbDOBXfK/ks6QVFbSAZJOlfSoq/Y2cIekKpIqu/pvFFDkPKCzpFqSDgVuy74gqZqk052t9k8CE0RWLm2MBo6UdJ6kVEl9gcbAqAL2KT8cAvwObHWr7StyXF8N1P3LXeF5AphtZpcQ2J7/V+heeuKOV7SefGFm/ybYQ3sHsBZYDlwNfOSqPADMAuYD3wJzXFlBZI0H3nFtzWZf5ViKYPfCSoIn8V1wD5pytLEe6OHqrifYMdDDzNYVpE/55CaCB21bCFbb7+S4fg/wqqRNks6J1JikXsApBOYSCP4OrSSdH7Mee+KCP7Dg8Xg8ccavaD0ejyfOeEXr8Xg8ccYrWo/H44kzXtF6PB5PnPFOK4qYgw6taIdWTYtcMQaklT8wIXIgwmbQJCbSJthYUhLn8NdflrFu3bqYTWNK+SPMdv3lAN0+2Pa1Y83slFjJLAhe0RYxh1ZN46Ih7ydE1n2nFPSka/4pqbtZpMSp2kTOYdbuxMjq3KFdTNuzXdsp0zD8zrgd8/4b6TRe3PGK1uPxJC8SlEop6l5ExCtaj8eT3Kj4P2ryitbj8SQ3CTTnFBSvaD0eTxLjTQcej8cTX0RSmA6Kfw/3Q35fu4pht1/ES1ecxtArezD749cA2L5lE+/eOZAXLjuZd+8cyI6tm4Hg6fSE5x7ghctO4uVrTmf1ktz8X+efcWM/pVmThjRpVJ/HHn04Jm3mxo4dO+jUoT3tW7egdfOm3H/v3SVCFpTMOfzxxx/o0K7VnlSjSgX++9QTcZMXHgWmg3Ap2pakFElzJY1y+TqSpktaLOkdSaVdeRmXX+Ku147Utle0xZBSKSn8beAtDHp2NBc8Poy5n7zJul+XMP29Fzii2TFc+vxYjmh2DNPfC9yvLp39BRtX/sIlz43l5KvuY/yz9xa6D1lZWVw3+CpGjBzD3PmLGD7sbb5btKjQ7eZGmTJlGDNuAtNnz2ParLmMHzeWGdOnJb2skjqHRx7ZkK9mzOGrGXOY8vVMDipblp6nh3NRHGdKpYRP0XMt8F1I/hFgiJk1ADYShF7CvW40s/rAEFcvfBfz0wtPYihXqSrV6jcBoHTZchxWsx5b169myfQJNOkWfKCbdDuDxdM+A2DxtAk06doLSdRo1IId235n64Y1herDzBkzqFevPnXq1qV06dL06duPUSNHFG5geSCJcuWCoAaZmZlkZmbG7QFHImWV1DkMZfLECdSpU49aR+T0rZ4oFJgOwqVoWpHSCcIevejyAroC77kqr7LX4X0vl8dd76YIG6y9oi3mbF69gtU/fUf1hs35Y9N6ylWqCgTK+I9NGwDYun41h1SuvueeQw47nK3rV+faXrSsXJlBenrNPfm0tHQyMuIXlCArK4v2bVpyRFo1unU7gXbt2ie9rJI8h9m8N/wd+vTtF3c5eSJitaL9D4Gv4t0ufxiwKSTs0gr2hj9KI/DDjLu+2dXPE69owyBpa4TrFST9xdl0rNi5fRsjHhpM10tvo0zZfIbVKuRqJrdTSfE8FZWSksL0WXNZvHQ5s2bNZOGCBUkvqyTPIcDOnTsZ/clIzjyrd1zlhCeqFW1lSbNC0j7RhSX1ANaY2ex9G/4LFsW1XPGKtnBUIBev/rEga1cmIx4azFHH9+TIDicBULbCYXtMAls3rKFshSBcVLnDqrFl3ao9925Z/9uelW9BSUtLZ8WK5XvyGRkrqFGjRpg7YkOFChXo1LkL48d9mvSySvocjhs7hhYtWlK1WrW4yolIKYVPQeTmNiHp+RwtHAecLmkZQWTjrgQr3AohgUjT2Rv7bgVQE8BdP5QgykfeXYzBMEs8kspJmiBpjqRvXUgRgIeBepLmSXpMAY9JWuDq9S2IPDPj0yfv4LCa9Wh7xsV7yuu368rCCUHEmIUTPqJ++25BefuuLJw4AjNj5ffzKFP2kEIr2jZt27JkyWKWLV3Kzp07Gf7OMLr3OL1QbebF2rVr2bRpEwDbt29n0sQJHNkwt4C2ySWrpM5hNu+9O4ze5xSh2QBiYjows9vMLN3MagP9gIlmdj4wCchervcHsg3sH7s87vpEi+CYwu+jjY4dwJlm9rsLODhN0sfArUBTM2sBIOlsoAXQnCCs9ExJX5jZqtDG3E+XywDKV/nrCidj0RwWTRpB5dpH8srgwP7e+aLrad/7Uj5+5Hrmj3+f8lWqc/qt/wGgbpsu/DzrC1647CQOKHMgp177z0IPODU1lSFPPE3P7ieTlZVF/wEDadykSaHbzY3fVq3i0kED2J2Vxe7duzmrdx9O694j6WWV1DkE+OOPP5g44TOeeLqoY0MqnvtobwGGSXoAmEsQeRj3+rqkJQQr2YjfNj5mWBgkbTWzcpIOINjG0ZnAWN4QqAMcCIwys6au/hDgWzMb6vKvA8PN7OO8ZBzeoKl5713Jg/feVTg6d2jHnNmzYjaJpcqnW5n214Sts+OzW2ebWZtYySwIfkUbHecDVYDWZpbpbDm5OXct/oeuPZ6SRJJ47/I22ug4lOCpZKakvwHZmwa3AIeE1PsC6OtOmFQhWAHPSGxXPZ79jBjso403fkUbHW8CIyXNAuYB3wOY2XpJUyUtAMYQ7MM7FviGYLvHzWb2WxH12ePZP/Deu5IbMyvnXtcRKNDc6pyXo+j/XPJ4PHEnOUwHXtF6PJ7kJUm8d3lF6/F4kpi4bu+KGV7Rejye5MabDjwejyfO+IdhHo/HE0eSZB+tV7QejyepSeRpvYLiFa3H40lahFe0nihIK39gwnwQLFi+OSFyAJrWPDRhskoqiVQgKcX/wX3uSKiUV7Qej8cTV/yK1uPxeOJMMijaZP3B4PF4PO68gsKmiE1IB0qaIekbSQsl3evKX5G01Dn2nycp2++0JD3pwo3Pl9Qqkgy/ovV4PEmLUCxWtH8CXc1sq/M9/aWkMe7a/5nZeznqnwo0cKk98Kx7zRO/ovV4PEmNpLApEhaQHYj1AJfCeULvBbzm7ptGEFusepj6XtF6PJ7kplSpUmETEaLgAjgf0vOANcB4M5vuLj3ozANDJJVxZXvCjTtCQ5Hn3sdCj9KTUMaN/ZRmTRrSpFF9Hnv04Zi3//bLz9L3lGM55+RjeGvoM/tce/2Fp2hbtwKbNqyPudx4j6uky0qUnBXLl3PKiV1peXRjWjdvyn+feiJusqJCUaTIUXAxsywX+y8daCepKXAb0AhoC1QiiCHGnlZzNBGum17RJhFZWVlcN/gqRowcw9z5ixg+7G2+W7QoZu0v+WERH73zGq9+OIG3PvmSLyeO5delPwHw28oVzPhyEofXSI+ZvGziPa6SLiuRY0pJTeWhRx9n7reLmPzl1zz37DNxkxUNQtGsaKPGzDYBk4FTzGyVMw/8CbwMtHPV9oQbd4SGIs8Vr2iTiJkzZlCvXn3q1K1L6dKl6dO3H6NGjoh8Y5Qs++lHjm7RhgMPKktqaiqt2h/H5HGjABjywO1cc+u9cdlKE+9xlXRZiRxT9erVadkyeMh+yCGH0LDRUaxcmREXWdFSWButpCqSKrj3BwEnAN9n210VNHIGsMDd8jFwkdt9cAywOWek65x4RZtErFyZQXr63i/StLR0MjJi9yGvd+RRzJ3xFZs2bmDH9j/4avJ4Vq9aweefjabK4dU58qijYyYrlHiPq6TLSuSYQvll2TK++WYubduFfeAefyKbDiJRHZgkaT4wk8BGOwp4U9K3wLdAZeABV3808DOwBHgBuDKSgKTd3iXpKzPrEKe2rwOeN7M/XH5rdliboiS38NOxXGHWqd+Qi/5+LVdfdAZlyx5Mg0ZNSUlJ5eX//ounX/0gZnJyEu9xlXRZiRxTNlu3buXcvr159PEhlC9fPq6ywiLybR7IiZnNB1rmUt41j/oGXJUfGcV+RSspNUc+BSBeStZxHVA2Fg3l7H9hSEtLZ8WKvQ87MzJWUKNGjVg1D0CvvhfxxsgveP6dMZSvUJHq6bVYueIXzuvekdM7Hc2a31ZyQc8urFu7OmYyEzGukiwrkWMCyMzM5Ly+vel37nmcceZZcZMTLYU1HSSCuClaSbUlfS/pRUkLJL0p6QQXNXaxpHaSDpY0VNJMSXMl9XL3DpA0XNJIYJyk4yVNkvQWwTIeSVvd6zOSTnfvP5Q01L0fJOkB9/4Cd/JjnqTnspW1pGfddo/Q0yCDgRoEPyUmhYznQXdyZJqkaq6siqT3Xf9nSjrOld8j6XlJ44DXYjWnbdq2ZcmSxSxbupSdO3cy/J1hdO9xeqyaB2DDurUA/JaxnEljR9L9rHMZN3MJH0/5lo+nfEvVw2vwxsjPqVylWsxkJmJcJVlWIsdkZlxx2SU0bNSIwdfdEBcZ+SH7wEJxV7TxNh3UB/oAlxHYPs4DOgKnA7cDi4CJZjbQGaNnSPrM3Xss0MzMNkg6nuCJX1MzW5pDxhdAJwIDdRqBvQUnZ5iko4C+wHFmlinpGeB8AgX4D9d+CjBBUjMze1LSDcDfXPRbgIOBaWb2D0mPApcS2GueAIaY2ZeSagFjgaPcPa2Bjma2PeekuH18lwHUrFUr6slMTU1lyBNP07P7yWRlZdF/wEAaN2kS9f3RcMuVF7F50wZSU1O5+d7HKX9ohZi2nxuJGFdJlpXIMX391VTeevN1mjY9mvZtgl/b997/IKecelpc5EXEHcEt7ig3+05MGpZqExiVG7j8a8BYM3tTUl3gA2AXcKB7hWCv2skEx9m6mNnF7t7jgbvN7G8h7W81s3KS0oD3gYHAzUBF4HJgEsH+t/4ESn2Nu/Ug4G0zu0fS5QQKL5VAQV9jZsMkLQPaZCtaSX8CB5qZSeoLnGhml0haw77bOqoQ7Lu7kcCUc2+keWrduo1NnT4r4nzGAu8m0ZMX8dIDOTnumLbMmT0rZpqxdNX6VrX342HrZDx75mwzaxMrmQUh3ivaP0Pe7w7J73ays4CzzeyH0JsktQe25WgrZx4AM8uQVBE4hWB1Wwk4B9hqZlvc1oxXzey2HDLqADcBbc1so6RXCJR+bmTa3k9iFnvnrRRwbM5Vq/u5kmt/PR5PbCku5oFwFPXDsLHANU4ZIukvT/6i5GuCB1hfAFMIFOgUd20C0FtSVSejkqQjgPIEynCzs7meGtLeFuCQKOSOA67Ozsh59/F4PImjsN67EkFRK9r7CRw4zJe0wOULwhQg1cyWAHMIVrVTAMxsEXAHwUO1+cB4oLqZfQPMBRYCQ4GpIe09D4wJfRiWB4OBNgrOQi8iMFl4PJ4EEelBWHFZ7cbNRuuJDm+j9RQHktVGW6ZaA6veb0jYOr882bPE22g9Ho8nrhQX80A4vKL1eDxJTXExD4TDK1qPx5O0SFDKr2g9Ho8nnhSfB17h8IrW4/EkNUmgZ4t8e5fH4/EUHGc6CJciNpF3FNw6kqY73yzvSCrtysu4/BJ3vXYkGV7RejyepEUUXtGyNwpuc6AFcIoCh96PEPgyaQBsBAa5+oOAjWZWHxji6oXFmw6KGCNxexgTube1YturI1eKERtnPp0wWYkka3fi9rinJOiBUjykFNZ04I7X5xYFtyuBIyyAV4F7CEKL93LvAd4DnpYkC/Mf2a9oPR5P8hKd6SDfUXCBn4BNZpbt8Co00u2eKLju+mbgsHDd9Ctaj8eTtIio9tGui3QyzMyygBbOXeuH7HV3uk+1ELF5XcsVv6L1eDxJTGx9HYREwT0GqKC9EVJCI93uiYLrrh8KbAjXrle0Ho8nqYnBroPcouB+R+DTurer1h/IDi38scvjrk8MZ58FbzrweDzJjGKyj7Y68KqLtFIKeNfMRjmPfMMUhMSaC7zk6r8EvC5pCcFKtl8kAX5Fm0Ts2LGDTh3a0751C1o3b8r9994dV3njxn5KsyYNadKoPo89+nBM2ixVSnz99i28/0TgUfLlB/vzzYd3Mmv47fzv7vNJTQ0+kj2OP5oZ79zGtGG38uWbN9OhRd2YyP/7JQOpVaMqrVs0jUl7kYjHHOZF4yPr0K5VM45t25JOx7aNm5xEjikS2TbawpgOzGy+mbU0s2Zm1tTM7nPlP5tZOzOrb2Z9zOxPV77D5eu76z9HkuEVbRJRpkwZxoybwPTZ85g2ay7jx41lxvRpcZGVlZXFdYOvYsTIMcydv4jhw97mu0WLCt3u1ef9jR+W7o2gO2zMTJqfeT9t+vyTgw48gIvPDIIbT5r+A+36PsQx/R7m8nve4Jm7zsuryXxxYf8BjBj1aUzaikS85jAco8dN5OuZc5ny9cy4tF8UY4pEDPbRxh2vaJMISZQrVw4IQj5nZmbG7fzhzBkzqFevPnXq1qV06dL06duPUSNHRL4xDGlVK3BKxya8/OFXe8rGfrn3P+msBb+QVrUiANu279xTfvBBZYjVVuOOnTpTqVKl2DQWgXjMYVFTHMckhU/FAa9ok4ysrCzat2nJEWnV6NbtBNq1ax8XOStXZpCeXnNPPi0tnYyMjEK1+dj/nc0/nviI3blsxE9NLcW53dsx/qu9ivf0vzVj3gd38MGTl3P5vW8WSnZREI85DIcQvbqfTMdj2jD0xefjIiPRY4qEYnAENxGUaEUrqYKkK4tA7vGSOsSj7ZSUFKbPmsvipcuZNWsmCxcsiIeYXE+rFcZL0qmdmrJmwxbmfrc81+tP3NaXqXOWMHXuT3vKPp40nxZnPcA5NzzPXVd2L7DsoiLWcxiJzyZ/ydTps/ng49E8/79n+HLKFzGXkegxRSY5QtmUaEULVABiomjdE8nQfLgdG8cDcVG02VSoUIFOnbswflx87I1paemsWLFXKWZkrKBGjRoFbu/YFnXp0eVovv/kXl57+GKOb3skQx+4CIDbLzuVKhXLcfO/Psj13qlzfqJuemUOq3BwgeUXBbGew0hUd21XrVqVnr3OYPbMGTGXkegxRYM3HeQDSbUlfS/pRUkLJL0p6QRJU533nHYugu1HLhjiNEnN3L33SBoqabKknyUNds0+DNSTNE/SY5KqS/rC5RdI6uTuP0nS15LmSBouqZwrXybpLklfAn1c+/+U9DlwraSeznvPXEmfSarmPPlcDlzv5HSK1RytXbuWTZs2AbB9+3YmTZzAkQ0bxar5fWjTti1Llixm2dKl7Ny5k+HvDKN7j9ML3N5dT31M/VPupFH3u7no1peZPPNHBt7xGgPOPJYTOxzFRbe9ss9qqW7Nynvet2iUTukDUlm/KbkiuMd6DsOxbds2tmzZsuf9xM/G07hJ7HdWJHJMUZEkpoPito+2PtAHuAyYSeDQoSNwOnA7wfniuWZ2hqSuwGsE3nYAGgF/IwgT/oOkZ4FbgaZm1gJA0o3AWDN70K1Qy0qqTBAl9wQz2ybpFuAG4D7X7g4z6+juvxyoYGZdXL4icIyZmaRLgJvN7EZJ/wO2mtnjsZyc31at4tJBA9idlcXu3bs5q3cfTuveI5Yi9pCamsqQJ56mZ/eTycrKov+AgTRu0iTmcp66vR+/rtrA5FdvBGDExHk89PynnNmtBef1aE/mrix2/JnJhbcMjYm8iy44lymfT2bdunXUq53OnXfdy4CBgyLfWAASNYcAa1av5txzzgJg165dnNPvXE48+ZSYy0nkmKIhyiO4RU6xiYLrVoLjnUsyJL1GoBTflFQX+IDgPPHZ2fvWJC0HmgLXA5lm9qAr/w44keCLZJSZNXXlnQlCi78BfGRm8yT1AF4hOFYHUBr42swGSVoGdDGzX9z9k4G7zexzlz8a+BfBhufSwFIzO0XSPYRRtM6pxWUANWvVav3DkmWFmrtoSeQH0nvvKjwl0XvXce3bMDuGUXAPqdnIWl7/Utg6U27sWORRcIuN6cDxZ8j73SH53QRKM5wzh9B7s8hltW5mXwCdgQyCkx0XuTbHm1kLlxqbWegSJ+fv1dD8U8DTZnY08HfgwHCDC+nH82bWxszaVK5cJZpbPB5PHiSD6aC4KdpIfAGcD8GTfQKvPL+Hqb+FwJSAu+cIYI2ZvUBwjK4VMA04TlJ9V6espCOj7M+hBEob9p59/otcj8cTJyI8CCsuVoU8bbSSyoe7MYKCixf3AC9Lmg/8wb7K7S+Y2Xr3MG0BMAZYAPyfpEwCR78XmdlaSQOAtyWVcbfeAfwYZX+GS8ogUNh1XPlI4D1JvYBrzGxKPsbo8XiiRCUgOONCgp/loaPIzhtQK5YdMbNlBPbW7PyAPK71yuXee3LkQ9vJeXbz1Vzunwj85XC4mdXOkT8+R34Eez36hJb/CDTLWe7xeGJPouzLhSFPRWtmNfO65vF4PMWFJFjQRmejldRP0u3ufbqk1vHtlsfj8UQmsMMW7mSYpJqSJkn6TkEU3Gtd+T2SMtx++HmSTgu55zYFUXB/kHRyJBkR99FKepogWFln4J8EttH/kctPbY/H40k0MTAd7AJuNLM5kg4BZksa764NyblNU1JjAh+0TYAawGeSjnThcHIlmhVtBzP7O7ADwMw2EOwZ9Xg8niKnsLsOzGyVmc1x77cQRFdIC3NLL2CYmf1pZkuBJUC7cDKiUbSZkkrh9qtKOoxgX6vH4/EUKQJSpLCJKKLg7mkvODjVEpjuiq52R/6HupOgEBIF1xEaITdXolG0/wXeB6pIuhf4Engkivs8Ho8nvkSwzzob7brsA0Iu5epD0vk4eR+4zm1ffRaoR3DMfxXBKVAoQBTciDZaM3tN0myCgGUAfcwsPr75PB6PJ5/EYteBpAMIlOybZvYBgJmtDrn+AjDKZfdEwXWERsjNlWhPhqUAmcDOfNzj8Xg8cUUED8PCpYhtBMvel4DvzOzfIeXVQ6qdSXDgCYIouP0klZFUB2gAhPVJGc2ug38QeNH60I3rLUlvmtlDEUfgiYpE+fVJ5H7DRDp6mbtsU8JktaxdIWGyErkRP3NXYh67xOOjHoOTYccBFwLfSprnym4HzpXUgqDbywj8mWBmCyW9Cywi2LFwVbgdBxCdm8QLgNZm9geApAeB2YBXtB6Pp0iJhT8DM/uS3O2uo8Pc8yDwYLQyolG0v+SolwpEDK/r8Xg8iSAlCY6GhXMqM4RgyfwHsFDSWJc/iWDngcfj8RQ5ye5UJtvwuxD4JKR8Wvy64/F4PNEjIAl8yoR1KhPebbnH4/EUNSo+zr3DEXGrlqR6koa50xE/ZqdEdM6TO1lZWRzbrhVnn9EzrnLGjf2UZk0a0qRRfR579OGkkvXgrVdzWvsGnH/asXvKft+0kWv7n8k5J7Tm2v5n8vvmYLfCnOlfcmLLWvTv2Yn+PTsx9KlHCy0/m0TNYbzlXPn3QdStdTjtW+/1/vnh+8Np1+poDi2bypzZs2IuM1oK61QmEUSzJ/YV4GWCVfqpwLvAsDj2yROB/z71BA0bHRVXGVlZWVw3+CpGjBzD3PmLGD7sbb5btChpZJ121rkMGfrePmWvPzeE1h068+5ns2ndoTOvPzdkz7XmbY7l1ZFTeHXkFAZec3OhZGeTqDlMhJzzL+zPByP2fQjfuElT3hz2Hsd17BxTWfkhFvtoE0E0irasmY0FMLOfzOwOgmizniIgY8UKPh0zmgEXxydyazYzZ8ygXr361Klbl9KlS9Onbz9GjfyLj/NiK6tlu+Mof2jFfcqmTBjDaWeeC8BpZ57LlM/y3L0TExI1h4mQc1zHzlSsVGmfsoaNjqLBkQ1jKqcgKEIqDkSjaP90Jyd+knS5pJ5A1Tj3y5MHN990PQ8+9AilSsX3gN7KlRmkp+89ZZiWlk5GRkaYO4q/rA2K0k+LAAAgAElEQVTr1lC56uEAVK56OBvXr91zbcG8mVzUsyM3DOrNz4u/i4m8RI0rkX+r4oYEpaSwqTgQzf/W64FywGCCExSXAgPj2SlP7oz5ZBRVqlShZav4+13PLQx9vOxdiZSVGw0bN+ODyfN5beSX9L7wMm694oKYtJuocRX1/BU1JSIKrplNN7MtZvarmV1oZqeb2dREdM6zL19/PZVPPhnJUUfWof+F5/L55IkMHHBhXGSlpaWzYsVeT3AZGSuoUaNGUsuqVLkq69b8BsC6Nb9R8bAg1PvBh5Sn7MHlAOhw/Ens2pXJpg3rCy0vUeNK5N+qOJIMUXDzVLSSPpT0QV4pEZ2TdIOkBS5dJ6m2pO8lvep2Qbwnqayr21rS55JmSxqb7RBC0mRJj0ia4XZMdMpD1mBJi1y7w1xZO0lfSZrrXhu68gFuHj6VtFjSo648RdIrrr/fSro+lvNx3wMPsfjn5Xz341Jeff1tuhzflaGvvB5LEXto07YtS5YsZtnSpezcuZPh7wyje4/Tk1pWx66nMPrDtwEY/eHbdOp2KgDr167esypc9M1sbPduDq1YKc92oiVR40rk36q4IcKbDYqL6SDcgYXEeQXJBQVxyS4G2hPYtKcDnwMNgUFmNlXSUOBKSU8ATwG9XPjwvgTnkLNNHKlm1s7F/LmbvS4fQ7kVqGNmf0rK9hzyPdDZzHZJOoEglM/Z7loLAgfBfwI/SHqKwHadlh2FN6SdpCM1NZUhTzxNz+4nk5WVRf8BA2ncpEnSyLrrukHMnTGVTRvX06tjEy659lYu/Pv13HHtxYwa/gbVaqTz4JOvADDp0xF8+NbLpKSmUKbMQdz3n5di8tM7UXOYCDkXX3QeX075nPXr1tGoXi1uv/NuKlasxP/dcC3r1q2lz1k9ObpZcz4a+WlM5UZEFBvzQDiUm32nOKAgQNphZnaXy98PrAVuMrNarqwrge34DuAr9vpgSAFWmdlJkiYD/3CKuRow1czq5yLvU2Ar8BHwkZltlVQTeJLADZoBB5hZI0kDgOPM7FJ37xgCxb4QmEXgjOITYJyZ/cUtkvPwfhlAzVq1Wn+/eFmh5ipakuEDWRBKqveuRJIo711djmvHnNmzYvZBrFq/qfV9bHjYOk+f1Xi2mbWJlcyCUJx9y+b1x8j5zWCu7kIza+HS0WZ2UkidP91rFm4VL+llBZEts/f4dCeIJtGaIDhbKnA/MMmtUHsCB+bS5p52zWwj0ByYDFwFvJjrAMyez/b2XrlylTyG6fF4IiFKzoGFouIL4AxJZSUdTOB4dwpQS1L2cZ9zCRzc/EAQaudYCLylSwr728nMLnZK+TQFMdFqmtkk4GagAsFOi0OB7H0yAyJ1WFJloJSZvQ/cCbTK14g9Hk++SS0VPkVCeYcbryRpvHsOM14uZpgCnlQQbny+pIj/z6NWtJLKRFs3FriolK8QeC6fTrA63EgQobK/pPlAJeBZM9sJ9AYekfQNMA/okA9xKcAbkr4F5hKEGN4EPAo8JGmqqxOJNGCyAufBrwC35aMPHo8nnwQ7Cwq9os0ON34UcAxwlYKQ4rcCE8ysATDB5SE4IdvApcsIYouFJZoIC+0IwjwcSrCabA5cYmbXRDOCwuDCSoSGlqgN7Dazy3OpOw/4y1lAMzs+5P06oHYudTKBjrmUfw0cGVJ0pyt/hUCRZtfrEVLHr2I9ngRS2EcPZraKIPgiZrZFUna48V7A8a7aqwQmwVtc+WsWPOCaJqmCpOqundz7GEU/ngR6AOtdR77BH8H1eDzFgCh9HRQ03Hi1bOXpXrNPxOY73Hg0ERZKmdkvOZbgYePjxAszWwY0LQrZHo+neBLFanFdNLsOlCPceBizQ+zDjQPLnfnAJKUA1wDeTaLH4ylypNh46FIu4caB1dkmAXcAao0rj0u48SuAG4BawGoCY/EV0Q/B4/F44kdhj+BKuYcbJwgr3t+97w+MCCm/yO0+OAbYHM4+C1GsaM1sDdAvcnc9Ho8n8cRgQZtXuPGHgXclDQJ+Bfq4a6OB04AlBDEVL44kIJpdBy+Qi/3BzPI0KHs8Hk8iyH4YVhjChBsH6JZLfSM4kBQ10dhoPwt5fyDBwYHledT1eDyexKEkD86YjZm9E5qX9DowPm498ng8nnygYhNHIW+iWdHmpA5wRKw7sr8iSq6zl0SRSEcvT335U8JkXdOxXsJkHRDNWdUYEOtPuojumG1RE42NdiN7bbSlgA3sPYrm8Xg8RUpxcRwTjrCK1m17aM5exyq7rbj6VfR4PPsdogTYaM3MJH1oZvEPUuXxeDz5RYXfdZAIorFuzIjGDZjH4/EkmuwVbbhUHMhzRSsp1cx2EXi1ulTST8A2grGZmXnl6/F4ipwkMNGGXdHOcK9nEMTpOo3gZERv9p6Q8CSYcWM/pVmThjRpVJ/HHn3YyyoGst595FbuPbMd/7r41D1lb9w7mCGX9GTIJT15qF8XhlzSE4BdmTt595Fb+PfA0xgyqAc/zZtWaPmQ2Pn7+yUDqVWjKq1bFL1/JyFSFD4VB8IpWgGY2U+5pQT1zxNCVlYW1w2+ihEjxzB3/iKGD3ub7xYt8rKKWFabU85i0CND9ym74O4nuf7FkVz/4kiadj6Zpp2CyEozRgXb0m8YOppLH3+VUc88xO7dhYvXlcj5A7iw/wBGjEpwEMa8iGA2KC6mg3CKtoqCcN+5poT10LOHmTNmUK9eferUrUvp0qXp07cfo0aOiHyjlxVXWXWbt6Ns+dz38poZ8yePpkW3YEW7+pcl1G8VRGIqV/EwDipXnhU/fFso+YmcP4COnTpTqVLhw7HHgij90RY54RRtCkHcrEPySJ4Es3JlBunpe72zpaWlk5GREeYOL6soZQEsnT+TchUrUyW9NgDV6x3FwqmfkZW1iw2rlrPixwVsXhPW8VNEEj2m4kYpKWwqDoTb3rXKzO5LWE+SFEn3AFvN7PEc5bWBUS6CbkzIbQtzvDZre1mxYd7EUbTotjfSUdvTerPm1yU8+fczqVitBkc0bUWplGjC0eVNosdU3EiGoUa00XqKD2lp6axYsdefT0bGCmrUqOFlFVNZWVm7WDBlLM3/1n1PWUpKKqdfdQfXvziSAQ8+x46tv1PZrXYLSiLHVNyQKPTDMElDJa2RtCCk7B5JGZLmuXRayLXbXATcHySdHE0/wynav7gHS0acTXmBS9dJOljSJ5K+cWV9JbWT9IGr30vSdkmlJR0o6WdXfqmkme6+9yWVzUVWa3f9a/LpRi0a2rRty5Ili1m2dCk7d+5k+DvD6N7j9FiL8bJixJLZU6lSsy4VqlTfU7Zzx3Z2bv8DgB9nfUmplFSq1W5QKDmJHFNxRBFSFLwCnJJL+RAza+HSaAAXHbcf0MTd84yLPBOWPE0HZrYhuj4WXyS1JnDK255gzqcT2J5Xmll3V+dQgv3BLd1tnYAFQFuC+Znuyj8wsxfcPQ8Ag4Cncoh8GbjGzD6X9Fisx5OamsqQJ56mZ/eTycrKov+AgTRu0iTWYrysfPLm/dfx87zpbNu8kQf7HMeJA66lXfdzmDfxkz0PwbLZumk9L958MaVUivKVq9HvtsfzaDV6Ejl/ABddcC5TPp/MunXrqFc7nTvvupcBAwfFTV44ggMLhfZH+4Uz9UVDL2CYmf0JLJW0BGgHfB3uJpVk1wWSrgUOM7O7XP5+YC1wPfAugQ11irs2HhgMPEcQp702gVLeYGbPSOoCPABUIHhIONbMLs+20QIvAN+aWS3XXjPgrdxstC4K52UANWvVav3jT7/EZwI8Maekeu9KFMe1b8Ps2bNiZpas27iZPfDG6LB1zm9d8xdgXUjR82b2fGidnM9U3P/rAcDvwCzgRjPbKOlpYJqZveHqvQSMMbP3wvUhCRyMFYq8/qCtgW+BhyTd5cqmAKcCmQTOzju69IW7/gpwtZkdDdxL4AQ9p6yovrXM7Hkza2NmbapUrhLlUDwez18RUviEi4Ibkp6P1CrBYqse0AJYBfxrj8C/EvH/fUlXtF8AZ0gqK+lggugQs4E/3DfS40CrkLrXAV+b2VrgMKARsNBdPwRY5aJlnp9TkJltAjZL6uiK/lLH4/HEFhEosXCpIJjZajPLMrPdBL9W27lL+Y6ACwVz/J00mNkcSa+w9zjxiwQ/+2dI2k2wes2O6DsdqMbeFex8YE2IW8g7XZ1fCFbDue0lvhgYKukPYGxsR+PxeHIjHntls8OMu+yZBM9tIIiA+5akfwM1gAbs1S95UqIVLYALH/zvHMV/UYJmth0oE5K/LMf1Zwl+TuS8756Q97MJ/Pdmc0/O+h6PJ4ao8HuGJb0NHA9UlrQCuBs4XlILArPAMuDvAGa2UNK7wCJgF3CVmWVFklHiFa3H4ym5CArtOMbMzs2l+KUw9R8EHsyPDK9oPR5PUpMMJ6u8ovV4PElNMhzB9YrW4/EkLbEwHSQCr2g9Hk8SI5QExgOvaD0eT1KTBAtar2g9Hk/yku29q7jjFa3H40lqkkDPekVb1Bi5O26OB4l0Br1jZ8Q93DHjwNKFc5ydHxLp6OX+8T8mTNadJx6ZMFmxxttoPR6PJ474XQcej8eTAJJAz3pF6/F4khtvOvB4PJ44IqKLC1bUeEXr8XiSFyWH6aCkO/4uUezYsYNOHdrTvnULWjdvyv333h1XeePGfkqzJg1p0qg+jz36cEzbvvryS2hwRHWObbPXq+S338zjxOM70OmY1vytY3tmz4ro5rNAxHNc8ZY1asht/OfcY3n+ir0hzFf//D2v3tCXF67oybv3XM6ff2wFYNPqFTx6RjNevLoXL17dizFP3ZVXs/kikfMXieyHYXGIgltJ0nhJi91rRVcuSU+6KLjzJbXKu+W9eEWbRJQpU4Yx4yYwffY8ps2ay/hxY5kxfVpcZGVlZXHd4KsYMXIMc+cvYviwt/lu0aKYtX/uBRfx3kef7FN29x23cvNtdzJl2mxuu+Nu7r7j1pjJyybe44q3rGYnnEW/+1/cp2z0E//g+Itv5NJnR9KwwwlMe2/v9QrVa3HJ0yO45OkRnHrNfYWSDYmdv2iJUxTcW4EJZtYAmODyEIS7auDSZeTiozo3vKJNIiRRrlw5ADIzM8nMzIzb76aZM2ZQr1596tStS+nSpenTtx+jRo6IWfvHdexMxUqV9imTxJYtWwD4/fffOfzwGjGTl028xxVvWbWObsuBhxy6T9n6FUup1bQtAHVaHsf3U8cVSkY4Ejl/UVNITWtmXwA5o373Al51718Fzggpf80CpgEVJFUnAl7RJhlZWVm0b9OSI9Kq0a3bCbRr1z4uclauzCA9fW9opLS0dDIyMuIiK5t/Pvpv7vrHLTQ5sjZ33X4zd92XL9/KUZHIcSVKVpXaR7J42gQAvpvyKVvWrdpzbfNvK3jp6jN4/eYL+HXBrELLKorPRSRKSWETQeSEWSHpskhtAtWyQ9m416quPA1YHlJvhSsL38f8DSn5kTRZUpui7kdBSUlJYfqsuSxeupxZs2aycMGCyDcVgNxOq8X7ZNnQF5/jn4/8i4U/LuPBR/7F4CsujbmMRI4rUbK6X/cgs0e9xdDBZ7Fz+zZSUksDUK5SVa56dRKDnv6IEy69lRGP3rjHfltQiuJzEYkoFrQFiYIbTlxO9s8ouM5gXSLHlk2FChXo1LkL48d9Gpf209LSWbFi7xd3RsYKatSI/U/5UN5+8zV69joTgDPO6s2c2TNjLiOR40qUrMo163Hug0MZ+OQHNO7SnQrVgxVn6gGlKVu+IgDVGzSlYvVabFixtFCyiuJzEZEYGGlzYXW2ScC9rnHlBYqCW2KUkaTakr6T9AwwB3jJ/UxYKOnePO45SdLXkuZIGi6pnKR2kj5w13tJ2i6ptKQDJf3syi+VNFPSN5Led+HMD5G01IUjR1J5Scuy87Fg7dq1bNq0CYDt27czaeIEjmzYKFbN70Obtm1ZsmQxy5YuZefOnQx/Zxjde5weF1nZVK9eg6lTPgfgi8kTqVuvQcxlJHJciZK1bdN6AGz3bqYOe5ZWp/ULyjdvYHdW4HNi46rlbFi5bI8SLihF8bkIhxSV6aAgfAz0d+/7AyNCyi9yi7ljgM0h0XLzpKTto20IXGxmV0qqZGYbJKUAEyQ1M7P52RUlVQbuAE4ws22SbgFuAP4JtHTVOhGEGW5LMFfTXfkHZvaCa+cBYJCZPSVpMtAd+AjoB7xvZpk5O+lsRJcB1KxVK+rB/bZqFZcOGsDurCx2797NWb37cFr3HhHvKwipqakMeeJpenY/maysLPoPGEjjJk1i1v6g/uczdcrnrF+/jiYNjuDWO+7mP0//j9v+7wZ27drFgQeW4T9PR/VAN1/Ee1zxlvXRIzfwy/wZbP99I09d2JlOF1zDzu1/MGfUWwA0PO5Emp14NgDLv53JF288SamUFFQqhVOvvpeDDqlQ7MZUWApruMgjCu7DwLuSBgG/An1c9dHAacAS4A/g4qhkJMpzVLyRVBuYZGZ1XP5yAmWWClQHrjGzYU4Z3gQcTrCtY4VrojTwtZkNkjQeGAw8R7B9ozaQAmwws2ckdQEeACoA5YCxZna5pOOAm82sl6SvgUvNLKwRtVXrNjZ1Wux/IueG996VXJRE713HtW/D7NmzYvZBbNyspb0x8vOwdVrXPnS2mRXpc5mStqLdBiCpDoEybWtmGyW9AhyYo66A8XmEGp5CsF8uE/iMQCGnuDZx+TPM7BtJAwi+DTGzqc6E0QVIiaRkPR5PYSmUeSBhlBgbbQ7KEyjdzZKqESjNnEwDjpNUH8DZWbO/1r8AriNY4a4FDgMaAQvd9UOAVc7+en6Odl8D3gZejuF4PB5PLkR6DlZcVHCJVLRm9g0wl0AxDgWm5lJnLTAAeFvSfALFm/1kaTpQjUDhAswH5tteO8udrs544PscTb8JVCRQth6PJ94kgaYtMaYDM1sGNA3JD8ij3vEh7ycSPOjKWWc7UCYkf1mO68+S99G7jsB7ZrYp6s57PJ4CkwymgxKjaIsDkp4iMFOcVtR98Xj2F4q/mvWKNqaY2TVF3QePZ79CRX8yLRq8ovV4PEmLSA5/tF7RejyepCYJ9KxXtB6PJ7nxpgOPx+OJM0mgZ72i9Xg8yU0S6FmvaD0eT/ISPAwr/qrWK1pPXEiko5eS4hgpJ3ecEHs3kXlx2+icBxzjw4rNO2LbYJJEwfWK1uPxJDWx0LOSlgFbgCxgl5m1kVQJeIfAe98y4Bwz21iQ9kukrwOPx7O/IKTwKR/8zcxahLhUzCsSbr7xitbj8SQ1UvhUCPKKhJtvvKL1eDxJSwzdJBowTtLskCi5eUXCzTfeRuvxeJKaKMwDlSWFxlp/PpdIuMeZ2UpJVYHxkmL6dNCvaJOIHTt20KlDe9q3bkHr5k25/9674ypv3NhPadakIU0a1eexRx8uEbISOYcrli/nlBO70vLoxrRu3pT/PvVE0sjasnYV7/2jP69e1Z3Xru7B3JGvAbBjyyY+uGsgr1x+Mh/cNZAdWzfvuWf5tzN447ozee3qHgy//cJCyc8PUZgOIoYbN7OV7nUN8CHQjrwj4eYbv6JNIsqUKcOYcRMoV64cmZmZdDu+Eyefcirt2h8Tc1lZWVlcN/gqPhkznrT0dDoe05YePU7nqMaNk1pWIucwJTWVhx59nJYtW7FlyxaOa9+Grt1OjMu4Yi2rVEoKnQfeTNV6Tdj5xzbeuvFsajXvwKKJH1Kz2bG07X0pM997gZnvv0Cn/jexY+vvTPrffZxxz/OUr1KDP1xk3rgjKFXIbQeSDgZKmdkW9/4k4D72RsJ9mH0j4eYbv6JNIiRRrlw5ADIzM8nMzIzbJsKZM2ZQr1596tStS+nSpenTtx+jRhb4c1ZsZCVyDqtXr07Llq0AOOSQQ2jY6ChWrsxIClkHV6pK1XpBdNvSZQ+mUno9tm5Yzc/TJ9K4ay8AGnftxc/TJgDwwxejqH/sCZSvUgOAshUOK8xw8kmhrbTVgC8lfQPMAD4xs08JFOyJkhYDJ7p8gfCKNsnIysqifZuWHJFWjW7dTqBdu/ZxkbNyZQbp6TX35NPS0snIiI+SSKQsSNwchvLLsmV8881c2iahrM2rM1j783ccfmRztm1ez8GVgmdCB1eqyh+bNwCwceUydmz9neH/uIi3bjibRRM/ionsSGS7SSzMrgMz+9nMmrvUxMwedOXrzaybmTVwrxsK2k+vaHMgaWs+6w+QVCNe/clJSkoK02fNZfHS5cyaNZOFC+ITaDe301bxOuqYSFmQuDnMZuvWrZzbtzePPj6E8uXLJ5Wsndu38ckjg+lyya2UKVsuz3qWlcWanxZyxp3/48x7XmTGu8+yMWNpoeVHQymFT8WB/VLRKiBWYx8AJEzRZlOhQgU6de7C+HGfxqX9tLR0VqxYviefkbGCGjXiM8xEygol3nMIgXnivL696XfueZxx5llxkxMPWVm7Mhn18LU06tKT+seeBMDBhx7Gtg3BM6FtG9ZQ9tBKAJQ77HCOaNWJAw4sy0HlK5LWpA1rl/1Q6D5EgyL8Kw7sN4pWUm1J30l6BpgDXCjpW0kLJD2So+6/JM2RNEFSFVfWQtI0SfMlfSipoqTeQBvgTUnzJB0kqZukua7toZLK/LU3BWPt2rVs2hTEfNy+fTuTJk7gyIaNItxVMNq0bcuSJYtZtnQpO3fuZPg7w+je4/Skl5XIOTQzrrjsEho2asTg626Ii4x4yTIzPnvqDirVrEurXgP2lNdt15VFEwP7+aKJI6jbvisA9dp3ZeWi2ezO2kXmn9v57cf5VEqvW+h+REUSRMHdbxStoyHwGtAduB/oCrQA2krKPvVxMDDHzFoBnwPZ+39eA24xs2bAt8DdZvYeMAs438xaEGx6fgXoa2ZHE+zquCJnJyRdJmmWpFnr1q2NuvO/rVrFKSd2pV2r5nQ6th1du53Aad175HMKoiM1NZUhTzxNz+4n0+Loozi7zzk0btIk6WUlcg6//moqb735Op9PmkT7Ni1p36Yln44ZnRSyVn43h+8mf8yK+dN547ozeeO6M1k663PanH0Jv37zFa9cfjK/fvMVbc++FIBKNetxRMuOvDH4DIbddA5NTuxN5SOOjNXw8kQRzAbFxXSgkur5KCeSagOTzKyOpF7A2WZ2kbs2CGhiZjdIygLKmNkuSXWBD4AuwLdmVsvVrwcMN7NWkiYDN5nZLEnNgafMrLOr1w24yszy/B3XqnUbmzptZryGvQ/J4E6uIOwvn+F4cvuYxPzMf+uG3qxesiBmH8QWrVrb+M+nh61TtfwBs0P8FxQJ+9s+2m3uNT9/6Pz8Ly6ZmszjKc4kwf+6/c10kM10oIukypJSgHMJzAQQzElv9/484Esz2wxslNTJlV8YUn8LcIh7/z1QW1L9XOp5PJ44kAymg/1tRQsEDiIk3QZMIvg+HG1m2TvktwFNJM0GNgN9XXl/4H+SygI/Axe78ldc+XbgWFc+XFIqMBP4XwKG5PHspxSfnQXh2G8UrZktA5qG5N8C3sqlXvZmwTtzlM8D/nJO08zeB94PKZoAtCx8jz0eTySyDywUd/YbRevxeEomXtF6PB5PnPGmA4/H44kjKkYPvMLhFa3H40luvKL1eDye+JIMpoP9dR+tx+MpIcRiH62kUyT9IGmJpAJHu82zj7Fu0OPxeBJKIZ3KuENL/wVOBRoD50qKaRgMr2g9Hk9SEwM3ie2AJc4B+E5gGEGo8ZjhbbRFzNw5s9eVLV3ql3zeVhlYF4/+eFleVpxlHRHLDsydM3ts2dKqHKHagRGi4KYBy0PyK4CYhsLwiraIMbMq+b1H0qxEeSPysrys4iArL8zslBg0k9uyN6Yu4bzpwOPx7O+sAGqG5NOBlbEU4BWtx+PZ35kJNJBUR1JpoB9BqPGY4U0Hycnzkat4WV5WiZIVN5yT/6uBsUAKMNTMFsZSxn4TYcHj8XiKCm868Hg8njjjFa3H4/HEGa9oPZ4iRpL/f1jC8X9gT0JQSAheSdWKsi/FBUkdJB1rZrtD5ydBshMiL9HjKq54RZukJNsH2NxTVxfa/RFJZWLVtqRUSVXd+zqxbDsXWQ2yZcWAo4E3JbUzM0vwyjYuX3Y5P5fmn7YDXtEmJZIUorgaSWpR1H2KBknHAN2Ba83sz1h8Wbg2OgNnS/oX8DBx8lAqqRfwHMGRzcK0UwrAzJ4DhgLPSWrpVrZx/T+pgErANEkdYt1+yOfyfEn/lnSl+7vv13hFm4SEfJhvAl4FXpD0sqTjnSeiYkEu5oILCbwjNYfYrHZcG78C5xNsNH/NzHYUtt2cSGoF3AtcbmZzJVWQVMtdy69iz/77XQ4cThCy/uV4mxGyv6DNbAPwNFDXlcf0MyPpCuAGYDFQA7hD0kmxlJFseEWbpEhqBpwJdDSztsASlz+kSDvmyLHqPtDMVgP3EGwK7yrp6FjJMrMlwLvAKOBoSW1D+hGrQznVCOb4cEm3EERQHi2pTbRfGJIOc/01Se2A64EHCb4k/ksQtr6Vux4PZdso5P0PwEBJB5hZVmEazWUVXp/gC+lZ4HFgONBTUrlkM3nFCq9okwRJFULeVwNWAzsJzmVD8IFuBgxIeOdyIUTJ3gC8JGkSwWr2OeAg4AxJhQ7LLqmLpHoEiu8uAo9Sp0s6QtKJQKdCtp8975MJVs5PA6uAG4E3CVak0bRTF7hd0qmu6HfgGzNbZWbLgdeBhcAHzowQM9umpBRJBxD8HV6RdBnwGTAXGFzItisDDdz7syQdTPA3uA7AzDYB3wDVgaz91WbrFW0S4FZlp0v6h6T+BD9hDwLmA8dISjezP4GRwJ9F2NV9cH09BbgY2A1caWaLCPx9VgdOdmfLC9r+YOBRApPEc0A54N9Aaff6Kvu6v8tv+92BtyW9DdQwsxuAdmb2mpN1EbA+yuZ2ANuALpK6Aj8B9SXdB+DMHTMIzthvLmifc/Q/e3q8V6AAABXDSURBVPV4sJllAscD7xCYDCYBVSi8O8DqwN2SXiCwj+8m+BKSpEddnXpABYLP7P6JmfmUBAkoS/DTdQ1QwZWdC/yPYDX3iLveqKj7GtLnSwlW2TcBYwgUYCngYIL/fFUL0XZX4AvX5pPAdGAiweqqFNASqF2I9tsQKKNObo7fBI4hMM20Ar4HekTRjkLepwH/AJ4AWhP8GplB8EDsNjeGw2M099nH608BRgO3A2eGXD+VYNW5EehfSFn3E3yJXOLyqQS/Xj4n+OKYCxxd1J/Hokze10ExJoedszzBf9D6wHQzu8mVNwVqAw2Bj81scVH0EyhlOWx9kv4P6E/wE/l8V3YtgfPn/8tZPx/yjiX44lkI9ADOA/oATxHYIS8xszkFHA6S0oHHgG1mdokru8/1+wVgKVDezL4L/Rvl0k7o36+0me10T/wvB6oCbxMo7CsJnJl8ZGYLCtrvXOR3I/jMDCKwBx8OjDazR0PqnAS0N7P789HuPmOWdBTBF+ogAkczH1ngqEUEq9gyZrYxFmNKVrz3rmJKjv+kRwCbzP6/vTOPt3O6+vj3l5DIRGhpa2iiiCIlFSFFhYZIQxokQWpoEhq05tfUmsrbNlpVU00d0WoREdRQU6uN6aVSBCHmt4b2I7xFhJL4vX+sfeRxmuGe55zr3Ju7v5/P/tzzDHcP5zlnnbXXXmttT5DUC5gq6Xzb3wBWAZ6xfX0Tu9vD9tzU170I96rHgTMJd655SXiNILTcsXUI2dHA7sABtv+V7LMn2H5V0lNExv+WTueXxIPAbpK+Yvs62yelqfA3iYWeF2HJnhOF5zcJ2FLS34BbCQ38UMJLYprtyQ3oL6mtohD8HLAHoUn3I36IdpP0ru2z0j0bAUMlneYwLyyVwrjGE5rrv4BfAP8ATgXmKlwO1wO+bnteQwbXjskabRujYlcrfJj/ixBQ3YHrbX9P0lqENvQ+YSscbfvZJvW1DzCDmE73ITSa24kp9k3AVcTUeB6xcv8th522THs7E1PhabZvT25JPwW6EtPTscT0+OWS9X+esHHPs/2cpEMITe0a2zeke/rZnl1Dnd8ghN3JhBCaQ9iPZxDTeQHft/1WmT4vps1tge62b5S0MnApcEga062EDfhY209L2h+4y/asGts4AJhI2MbXAkYT7/+nCZPW2sCRtv/WqHG1a5ptu8jlwwVYrvB6P+DP6fV5hB3se+m4KzAJWKeJfa38UH+HsB3/HPhUOrcz8QX/auH+HnW0NQg4A3gEOIyYjgKsQLhI/RjYuI76RwCPAccAf03tiZjWXwZ8pUSdKxILRL2AQwib7xHA1cAXiGn1xxr5LNLr4cBdhLbavTCezwLXAus3oL3TgO0KxxMINy4Rs6yezfpctsXS9A7kUngY4RbzHLBKOh5KaAiHpi/n+sArwNltoK+9qo73JDTsHdPxSknYXkVMtT8kDGpsayihzUOYHv5ARIN1LdxTqu70vxsCDxBa2ATC7PEssRDWOQn2mhZz0vuxShI8/YFb0/n1Ulu/BLo1+Jl8KbXViTBx7JfOHw7MJJlDannPUv87L+L8RcDFheO+hIvaCs3+bLbF0vQO5FL1QGAk4UxeEba9ganAoHR8HrE6vUoT+7hc0tAmEAs7N6bz3yLsoxsW+j6cpOWWbGvvJKxHFc4dRgQnDKUwA6ijjVXSj9gXCRNEL2Kq/yrwxRbW0anq+CxCE+5OmB/uTedHEQtqqzb4mXQmvAueJQIgTk+flY+n631ZONto8Y8ShR9UwgSyF7Ad4TnyJ+CsdG0cMJ0GaejLWsmLYW0M27+XNB+4X9Ig269Jeg4YI2kIsDJhk32tiX2cL+lyQvN7m5iWYnty6vttkobbfljSzU7fxJL8nfD1fJKY9mL7bEndgYOAu4H5ZSqWtD2wke2zgdfSKv2ltt+U9CRhY27Rd8T2+6nOUbavJfx7JwJb2r5N0v9J+gthpx5t+5UyfV7MONa2/ayk7wMnEmacLkTwyuqSxtp+rtDXpT6PZH//NPCgpM0IV7TjidnEV4mZ127ANEmXEWaJfW03YiFy2aPZkj6XRRfCz/FpQrsaQNhB7yAEQ7P6VLHJdk5/JxPa96FV9x0DPEN82cuaC7YjbIwrpb/PAPtU3bNyHWMZRQR87Fg4N5HQng+h4Pu5pDHwYdvoaoRf6lRCGJ0DfLtwfSMaqMkS0/qexILgtYSWeSkhEHsQU/kHgPXqaOM4QnD/lrQekD6T0wj7tdIzKv0sOkJpegdyWcLDiQWamSwMUGioTa/GvhQFSn9SsEESLo8Ax6fj0YTWvWIdbR1OOLufSoSKrkxENT0BTGrAWHomgdoXWB7Ymgjf7Z6Exw+pcfEL+Ez6exThEbIDcAsRantsazyLitAmFtV+QARWTEzt9knj+UyZ+que94GE/X1M4dxI4IxmfR7bW8mmgzaMwz2nC/AnSQNpYnitK9/AyBg2Avi3pOmE1jaCSLDSj1iQ2d72E2XaUSSb+bLtIZLOIVyR5tq+I7kUnSnpCtv1hKm+QwjbMwnTxxxgCBFJNrHQl5YGI3yaSAhzIyFkRwELCKF7MbCNpAtsv1FHnz/UbsqZcEx6Bu/bPlbS7kTE3VAiFeWRxEyg5vrT677Ay7YvlPQ+cImkpx0uW32IBD4rAP9e3PuUCbIfbTtAUk+ngIAm92MPYiV7mKRLCd/ZqYQJoSewJfCwC/bAEm1sABxMJHDZhrBnviNppMN+3c3223XUXxFUvQlvjlts35t+JE4losqW+F5XCaNtgNeAucS0/UpCy+9j+2up3jds/6NsnxfR/kBCgI8l3qt+wEhHvgsk7UkEuPyhjjYOJ4JNFhCzi3OJBa/zCZPEPOBCNzCSbZmm2Sp1Lm23sPCHuPJ3RyIhyaHECvdWhJ3zPGDtOtuq1N2ZWHB5otDu/oR9um5PCxbalzsVzu1KZJjapca6jiA8QNZNx70J08MFxFR7pwY9h7UIV7mK7/AQwuNjayJXwtrp/CbA8tXPr5ZnnV4PJMxBH0vtHgOcWRjzCzQoJ0NHKU3vQC5ts1R98Spf5OXS318XhMtFRKBCaSFIaGUPs9DOOSwJqynA0YT/Z/866u9cNZ41gHuSoFqB0A5HVo97KXUOJQIBKsJvU2Cz9LqSPazuwIBU30hicW5Xwn66IaHxzwZWSvfsQLiN1bwoVfXeTCKyw11WODeQWGzrn47zwleNJdtoM4vElW9ghKF+WdIs4J+SziIWeC6QNI1wwJ/gku5mkrYgvtzbOHIXbJounU4IlteBPW0/XrL+kYQb0muSznWYNS4gBMmd6Z79nZKgVMa9iHo6eaELVyfCvWkGcFKyow8G3pF0ke2riB0GGoLDZNKFmEksR6TDPBvYHNhMkol8xCe6RPKWwrPeiYhYuwD4nqQ9bF9h+wFJcwkB/wiR2yBTA9lGm1ksKbfA0YSr2TXAc7YnSfoUIRw3Ib7cj9ZQZxcieuiNlFvgDeAAwh3pbWJa/AzwW4c/aj3934DQVi8ksp7tTGjL71V+GCR1dg0JbhTbzwxO/X2U0GSPJ5KBHwg8afuKevq9hLZHE9r/WcAsIpXjgYR2e0USyIv9sVhK3RUt/w7b+ypyCW9PjGsm4eY1wvbzjRlNxyJrtJklsQIRKronsc/VIel8b9unKLaoqXV/rsHAkSmT1YaEW9XMdP4y4IRU1lxsDS0gacbHAlfa/lU69x4RhLAHoeH+R2rHpdQ5mght3SeVucAU2zMVWctGE+9V3RQW7QYRi1132p6axnAEcI7t30q6Ej4IIiklZNP/v5gWwC5UZCy7RNJMQovemJhVZCFblmbbLnJpe4XQKncgAiVmA3cXrh1KrDx3qaP+6wlNdut03KVwbRxwP3UmMCeimm4ivABWLZz/b2KhrRs1BlMQ2baOqvSZWKS7iphu/5EGJ7cmNPBZRAKX+4C90/md0nu0e61jaGGbDxOpLCvnOjWyjY5YskabqXZXEjEdXp6UxBnoImlXIgLoa0RG/nfraPJ6IurtNEkTnJKVK9L77UK4kJW1yQ4mhOCrwBjCr/UgRf7eObZPlHSJy7mIPQZMkHSjI9Xjz5PvamciWUvDbJeStiK05mGE5r8XsFMydVySntOcynNrFLavl7QA+Kmk921PdbJNZ8qTbbSZD1DsZTUnlcsJ++xThAa1K5E57ByX9J1UbJbYxQtzu55CuHV9idh6ZjPgkrICK9mUTyXMHQel8hyxoeJjwI9szylTd6q/NxGEIMLdrDth5hjuBsT4F8wFmxMLUgcTJpvzCIE7noiaO9n2L+ttbyl92QF42nZNAQ+ZxdBslTqX5hRCWBR9SfsQK8qPESaD8YSP6FrpemfqyJRFLHjNIsJD7wPWTOdPJRLGPAT0q6P+3sRW5qsRmybeTWyoCOHOdRN1xPwX2lmdsFXfQqSu3KTBz2VzQuOvmFX2JqUjJOzYNwAbNPvzk0ttJe+C24HxQnel1RwLHScD77Ew8//ywFjFflcLbJfNkrUlkS5xC9vDSBsSKnbvPYk0LXYNOxcU6q7s9Gpi9X0MkbN2vO2XJI0iQpdHuQH7qdl+yfa5hIljH9sP1VtnFb0JL4+t0vE9wBaSfkWYcs5wjbshZJpPFrQdDAUbE4tESNoQ+JGkY4jAgG8TCzvPEGGW4wlttmxbqxDCbz0iIxe2DyZSLE6TtIbt+2y/UHJIn0h1vk5kmToDmGh7drJznkLkYa3Hpvwf2J7nBm4/U6j3FsLvd2LyY32aeP+eIrKk/bHRbWZan2yj7aBI+gSxyDKdyCU6nJim3gq8avu0dN+aZYVgxf1LUg9CgBu42fb0dP2HwE9s/2/J+ocTXgSzgeeJhbudCY+JKcSPxAm2rytTfzNJwQOnEqGvv2l2fzL1kQVtB6IyzbZtSV2JRaKtCTvju5LGEYEIQ4i0h5PrcIA/lAhTXZFwB5tOLO50Am6rVzNLEWUHEyHA3Qjb5sbp3EgiouwV23+px7+0mSSzx2QicOCfLrlzcKb5ZEHbAVGkGxxhe5SkiwkhtWnSPjcgsmbdbvupkvXvRgQL7EMkijmPiJ66gdDSXiM8AEpl4VJsuX4b8I4jnWInYsHrJOBy27eXqbctImlVN3A3hkxzyIK2A1DlJzuC0FoPrpgE0kLLQGDzJGw/iOuvtf50vAcwwPa30vHmxAp9ZbPD18sKD0nrEBn+uxOmguOcXJ0kXQA8lharMpk2Qw5YWMapErKfJxaPhhGx/y8A2J4gaQrhGzqYsKW2mEL9uxALaJsAPSR1I7TO+yT9HujuGvIiLGIsuxBb+rxJrMb/DDhR0vqEIB9M2GYzmTZFFrTLOAUhuC8xlZ9A2P2OkvSG7RnpvrGKZDG4xDRHkWz6TEL4DSUW2N4B7pC0FmH3/X7ZcUj6GJFAZS/bj0qaSGxxcyKx9cxAYI/kbVBTophMprXJ7l0dgOTmtBcRK/8CkT/2NuCEtKhUodQuAJL6EFrw1skv9rtERNYQFmbNGmP776UHETvd9gJWTce/BjYgks+MJBLgbAOQhWymrZEF7TJIwYmfNH3flHDlGgNg+2UixPZ+4LDkgVBWk/0mcAUR7LBdcun6fTpePbWxqyM3QGmSn+xUYv+t/rbfI8wFaxBJsU8B9pG0cnH8mUxbIC+GLWNU2WQ/Cbxt+3VJk4BBwE22r07XVwPmu3zS7lGENvkDIhqrK7EN9Z2OtH17Ave5QfHyipyplXyw9xCmkG/avjFdr2s/sUymtciCdhlFsVvtEODjhMZ5LeGPORD4s+3f1Vl/JVH0Lbb3V+yGejwRQnod8KeyIbtLabcXkZZwXeBB23e3Vz/ZTMchmw6WQdLq/A62RxKhm9vafpbQNh8DvpAEVmlsv0hkkhohaZwjAfgpRK6EHYlUhQ3H9pu2b7F9vu2707ksZDNtmqzRLgNU+72mFHe9iZX/rYmNB99Ni1YvAT3coNypKVR0MjDZ9u8kLUds3ped7DOZRHbvWgYoZOGq+LFuRfiyisiKNT+FxO4IjG6UkE1t3yDpfSJR9HzbU4i8tZlMJpE12nZM1cJX0Y91RyIw4SoiL2tfIsHKuHoCBpbSl5woOpNZDFmjbadUCdmiH+vTkh4gcgpsRthoNySc+Vstj6ntW1ur7kymvZMFbTukSshWdmVdEfixpBdtX5t8Sc8FHrB9YRO7m8l0eLKgbYcUhOwoYq+tfQg/1s8BgyXdafua5HJV915WmUymPrKNtp3SLD/WTCZTO9mPtp3SLD/WTCZTO9l00I6xfbWkfwOTJZH8WI8h/FjnNbt/mUwmyIK2nZP9WDOZtk+20S4jZD/WTKbtkgVtJpPJtDJ5MSyTyWRamSxoM5lMppXJgjaTyWRamSxoM5lMppXJgjaTyWRamSxoMx8JkhZIelDSI5KmSOpeR13bSro+vf6KpOOWcG9vSd8o0cZ30nZALTpfdc/FksbU0FZfSY/U2sdM+yEL2sxHxdu2B9juD7xLbLL4AQpq/jzavs72aUu4pTdQs6DNZBpJFrSZZjAdWDdpcrMknQ/MANaSNEzSPZJmJM23J4Ck4ZIel3QnsFulIknjJf0kvf6EpGmSHkplS+A0YJ2kTZ+e7jta0v2SHpZ0SqGu4yU9Iek2YP2lDULS11M9D0maWqWlby9puqTZknZO93eWdHqh7QPqfSMz7YMsaDMfKWlPsS8DM9Op9YFLbX8eeAs4Adje9qbAX4EjU2aynxFbm38R+ORiqj+H2OF3E2BT4FHgOCJiboDtoyUNA9YDNgcGAAMlbSNpILAnkXZyN2Jr9qVxte1Bqb1ZwH6Fa32JXYh3Ai5MY9gPeN32oFT/1yWt3YJ2Mu2cnOsg81HRTdKD6fV04BfA6sDztu9N5wcTu0HcFXnL6UKkgvws8KztJwEk/QaYtIg2vgTsC2B7AfC6pJWr7hmWyt/ScU9C8PYCplWS8Ui6rgVj6i/pu4R5oidwc+HalWkvtyclPZPGMAzYuGC/XSm1PbsFbWXaMVnQZj4q3rY9oHgiCdO3iqeAW22Pq7pvALFVTyMQsWPvRVVtHF6ijYuBXWw/JGk8sG3hWnVdTm0fYrsokJHUt8Z2M+2MbDrItCXuBbaStC6ApO6S+gGPA2tLWifdN24x/387cFD6386SVgTeJLTVCjcDEwu23zUkrQb8BdhVUjdJvQgzxdLoBbwsaXlgr6prYyV1Sn3+DPBEavugdD+S+knq0YJ2Mu2crNFm2gy2X0ma4e8kdU2nT7A9W9Ik4AZJc4A7gf6LqOIwIl3kfsAC4CDb90i6K7lP3ZTstBsA9ySNei6wt+0Zkq4AHgSeJ8wbS+NE4H/S/TP5sEB/AvgzsRvxgbbfkfRzwnY7I+3p9gqwS8venUx7JmfvymQymVYmmw4ymUymlcmCNpPJZFqZLGgzmUymlcmCNpPJZFqZLGgzmUymlcmCNpPJZFqZLGgzmUymlfl/wj4ssP/mGhAAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "cm = confusion_matrix(y, preds)\n",
    "\n",
    "plot_confusion_matrix(cm, data.classes)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## Model Building - 70 Fonts - PyTorch\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "### Why PyTorch?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "* Easy to customize\n",
    "* Flexible to integrate with other visualization projects"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## Model Building - 70 Fonts - PyTorch\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "### Pretrained Model ResNet50"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "What is pretrained model? What is transfer learning?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "<img src=\"imgs/transfer-learning.png\" alt=\"drawing\" width=\"800\" style=\"alig:center\"/>\n",
    "<sub>Image Courtsey - https://medium.com/kansas-city-machine-learning-artificial-intelligen/an-introduction-to-transfer-learning-in-machine-learning-7efd104b6026</sub>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## Model Building - 70 Fonts - PyTorch\n",
    "\n",
    "\n",
    "\n",
    "### Hyper Parameter Tuning\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "\n",
    "* Learning Rate"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEOCAYAAABmVAtTAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJzt3XmcXFWd///Xp6r3PenurCQkZCGEBIJpNtmiIKKCAWQQRkBEJiLOCMLM4FdndEbHr9sMXwb5YoiCyA8U2WRRhEEBI0KADoQlCWQDsnQn3Z30vi+f3x9VafIN3Ul30rduVfX7+XjUI7dunar7Oanu++lzzz3nmLsjIiICEAk7ABERSR5KCiIi0k9JQURE+ikpiIhIPyUFERHpp6QgIiL9lBRERKSfkoKIiPRTUhARkX5KCiIi0i8j7ACGq6yszKdNmxZ2GCIiKWXlypV17l6+v3IplxSmTZtGZWVl2GGIiKQUM3tvKOV0+UhERPopKYiISD8lBRER6aekICIi/ZQURESkn5KCiIj0U1IQEUkBT63ZwYaalsCPo6QgIpLk3J2r71nJg69sDfxYgSUFM5tiZs+Y2VozW21m1wxSbpGZrYqX+XNQ8YiIpKr6tm66e51xhdmBHyvIEc09wPXu/oqZFQIrzewpd1+zu4CZlQC3Ame5+2YzGxdgPCIiKammuQOAcYU5gR8rsJaCu1e7+yvx7WZgLTB5r2J/Czzk7pvj5WqCikdEJFXtaOoEYFxR8C2FhPQpmNk04Bjgxb1emg2MMbNnzWylmV2WiHhERFJJTdPulkJqXz4CwMwKgAeBa929aYDjLwROB3KBF8xshbuv2+szlgBLAKZOnRp0yCIiSaWmOd5SSOXLRwBmlkksIdzj7g8NUGQr8IS7t7p7HbAcOHrvQu6+zN0r3L2ivHy/M7+KiKSV2uZOCnMyyM2KBn6sIO8+MuB2YK273zhIsUeAU8wsw8zygOOJ9T2IiEjcjqaOhFw6gmAvH50EXAq8YWar4vu+AUwFcPel7r7WzJ4AXgf6gJ+7+5sBxiQiknJqmjsTcukIAkwK7v4cYEMo92Pgx0HFISKS6mqaO1g4dUxCjqURzSIiSczdqWnqZFxRYloKSgoiIkmsqaOHzp6+hPUpKCmIiCSx3WMUypUURERk9xiF8bp8JCIi7897pJaCiMioV9M/75FaCiIio96Opk7ysqIUZAc+KxGgpCAiktRqmjsS1p8ASgoiIkmtprkzYXcegZKCiEhSq23uTFgnMygpiIgktZqmjoTNewRKCiIiSauls4fWrt6ErLi2m5KCiEiS2j2aebySgoiIJHLFtd2UFEREktT7SUEtBRGRUW/35SO1FEREhJrmTrIzIhTlJmY0MygpiIgkrZqmDsYVZRNb8j4xlBRERJJUItdm3i2wpGBmU8zsGTNba2arzeyafZQ91sx6zeyCoOIREUk1NQkezQzBthR6gOvd/QjgBOArZjZ370JmFgV+CDwZYCwiIilnR1NiJ8ODAJOCu1e7+yvx7WZgLTB5gKL/ADwI1AQVi4hIquno7qW5oyehk+FBgvoUzGwacAzw4l77JwPnAUsTEYeISKroX1wn3ZKCmRUQawlc6+5Ne718E3CDu/fu5zOWmFmlmVXW1tYGFaqISNLoX4YzwZePAr351cwyiSWEe9z9oQGKVAD3xm+3KgM+aWY97v7wnoXcfRmwDKCiosKDjFlEJBnsHs2cyHmPIMCkYLEz/e3AWne/caAy7j59j/J3Ar/bOyGIiIxGO0IYzQzBthROAi4F3jCzVfF93wCmAri7+hFERAZR09xJZtQYk5eZ0OMGlhTc/TlgyMPw3P3yoGIREUk1NU2dlBckdjQzaESziEhSqmnuoDzBncygpCAikpRqmjoZn+DbUUFJQUQkKdU0dyR0Gc7dlBRERJJMV08f9W3dCb/zCJQURESSTm1LOKOZQUlBRCTp7F5xLdGT4YGSgohI0tkRn/co0ZPhgZKCiEjSqe2f90hJQURk1Ktp7iRiUJqvpCAiMurVNHVSXphNNJLY0cygpCAiknR2NHeEcjsqKCmIiCQVd2dDTQsTi5UURERGvVe3NLC1vp2PzR0fyvGVFEREksjDr24jOyPCWfMmhHJ8JQURkSTR1dPHY69VceaREyjMSew6CrspKYiIJInl62qpb+vmvGMmhRaDkoKISJL47aptlOZnccqs8tBiUFIQEUkCTR3dPLVmB+ccPYnMaHinZiUFEZEk8MQb2+nq6ePcYyaHGkdgScHMppjZM2a21sxWm9k1A5T5nJm9Hn88b2ZHBxWPiEgy++2r25hels/RhxSHGkeQLYUe4Hp3PwI4AfiKmc3dq8w7wGnufhTwXWBZgPGIiCSlqoZ2Vryzk/OOmYxZ4qe22FNGUB/s7tVAdXy72czWApOBNXuUeX6Pt6wADgkqHhGRZPXIqirc4dwF4V46ggT1KZjZNOAY4MV9FPsi8IdExCMikizcnd++upWFh45hamle2OEEnxTMrAB4ELjW3ZsGKfMRYknhhkFeX2JmlWZWWVtbG1ywIiIJtqa6iXU7Wjgv5A7m3QJNCmaWSSwh3OPuDw1S5ijg58Bid985UBl3X+buFe5eUV4e3v27IiIj7ZFVVWRGjU/Nnxh2KECwdx8ZcDuw1t1vHKTMVOAh4FJ3XxdULCIiyWr5ulqOmz6WMflZYYcCBNjRDJwEXAq8YWar4vu+AUwFcPelwLeAUuDWeI97j7tXBBiTiEjS2NXaxVvbm/nHM2eHHUq/IO8+eg7Y571V7n4lcGVQMYiIJLOX3oldMT9xRmnIkbxPI5pFRELywsad5GZGmT+5JOxQ+ikpiIiEZMWmXVRMG0NWRvKcipMnEhGRUWRnSydv72jmhMOS59IRKCmIiIRixaZdQHL1J4CSgohIKFZs2kleVpT5k8OdAG9vSgoiIiF4YdNOjp02NtS1EwaSXNGIiIwCtc2dbKhpSbpLR6CkICKScCs2xcYnJFsnMygpiIgk3IpNOynIzmDepKKwQ/kAJQURkQSL9SeMISPJ+hNASUFEJKF2NHWwqbY1KfsTQElBRCShkrk/AZQUREQSasWmnRRmZ3DkpOQan7CbkoKISAKt2LSL46aPJRrZ5yTSoVFSEBFJkO2NHbxTl7z9CaCkICKSMMnenwBKCiIiCfPchjqKczM5YmLyjU/YTUlBRCQB3J3n1tdx0szSpO1PACUFEZGE2FjbwvamDk6eWR52KPsUWFIwsylm9oyZrTWz1WZ2zQBlzMxuNrMNZva6mX0oqHhERML0l/V1AJwyqyzkSPYtI8DP7gGud/dXzKwQWGlmT7n7mj3KfAKYFX8cD/w0/q+ISFp5bn0dh5bmMWVsXtih7FNgLQV3r3b3V+LbzcBaYPJexRYDd3nMCqDEzCYGFZOISBi6e/tYsWknJ89M7lYCJKhPwcymAccAL+710mRgyx7Pt/LBxCEiktJe3dxAa1dv0l86ggQkBTMrAB4ErnX3pr1fHuAtPsBnLDGzSjOrrK2tDSJMEZHAPLe+lojBiTPSJCmY2Qwzy45vLzKzr5pZyRDel0ksIdzj7g8NUGQrMGWP54cAVXsXcvdl7l7h7hXl5cndcy8isre/bKjjqENKKM7NDDuU/RpqS+FBoNfMZgK3A9OBX+3rDWZm8bJr3f3GQYo9ClwWvwvpBKDR3auHGJOISNJrbO/mtS0NnJoCl45g6Hcf9bl7j5mdB9zk7j8xs1f3856TgEuBN8xsVXzfN4CpAO6+FHgc+CSwAWgDvjDcCoiIJLMXNu6kz+HkWalxlWOoSaHbzC4GPg+cE9+3z3aQuz/HwH0Ge5Zx4CtDjEFEJOU8t6GW/Kwox0zd7xX3pDDUy0dfAE4Evufu75jZdODu4MISEUkPf1lfxwmHlZKZhEtvDmRILYX4gLOvApjZGKDQ3X8QZGAiIqluy6423tvZxuUfnhZ2KEM21LuPnjWzIjMbC7wG/MLMBus8FhERUmdqiz0NtT1THB9jcD7wC3dfCJwRXFgiIqnvuQ21TCjKYUZ5QdihDNlQk0JGfPqJC4HfBRiPiEha6O1z/rphJyfPKiN2h35qGGpS+A7wJLDR3V82s8OA9cGFJSKS2t7Y1khje3dKXTqCoXc03w/cv8fzTcBnggpKRCTVPfzqNrKiEU5NkfEJuw21o/kQM/utmdWY2Q4ze9DMDgk6OBGRVNTZ08vDq7bxsSPHMyY/K+xwhmWol49+QWxKiknEZjF9LL5PRET28tSaHTS0dfPZiin7L5xkhpoUyt39F+7eE3/cCaRWm0hEJEF+8/IWJpfkpsT6CXsbalKoM7NLzCwaf1wC7AwyMBGRVLS1vo3nNtRxwcJDiERS566j3YaaFK4gdjvqdqAauABNXici8gEPrNwKwAULU7PbdUhJwd03u/un3b3c3ce5+7nEBrKJiEhcX59zf+VWTppRlvRrMQ/mYGZoum7EohARCVBjezf/8vAbPPpaFR3dvYEd5/mNO9nW0M6Fx6ZeB/NuQ506eyCpd7FMREalv6yv5e4Vm7l7xWYKczI45+hJXLDwEI6ZUjKio41/U7mF4txMzpw7fsQ+M9EOJil8YC1lEZFkVN3QAcBtly7kiTe389ArW/nVi5uZVprHSTPLOHFGKSccVkpZQfYBH6OhrYsnV2/n4mOnkJMZHanQE26fScHMmhn45G9AbiARiYiMsKrGdvKzopw5dzwfP3IC31l8JI+/Uc0f3tzOw69u454XNwMwa1wBZ8wdz5UnT6d0mAnikVVVdPX0pfSlI9hPUnD3wkQFIiISlOqGDiaW5PZfKirMyeSzx07ls8dOpae3jze2NfLCpp28sHEnt/15I798/l2uOGk6f3fKYRTnvb/I5K7WLp54cztPrt5Onzvji3KYWJzD+KIc7l7xHvMmF3HkpOKwqjkiDubykYhISqhubGdicc6Ar2VEIxwzdQzHTB3D1YtmsqGmhZv+uI5bntnAL194l7875TAmFOXw2OtVPL9xJ719zrTSPIpzM1m3o5na5k764tdTvnfevMRVKiCBJQUzuwM4G6hx9w/8T5lZMbElPafG4/hPd9fUGSIy4qoaO5gzoWhIZWeOK+CWv/0QX/lIEzc+tY4bn1oHwJSxuSw59TDOPmoicycW9bc6enr7qGvpor6ti8PHp/7FlSBbCncCtwB3DfL6V4A17n6OmZUDb5vZPe7eFWBMIjLKdPX0UdfSycSSgVsKgzliYhE/u6yCt7Y30d3jzJtcNOCdShnRCBOKc5gwSEsk1QSWFNx9uZlN21cRoNBi/8sFwC6gJ6h4RGR02tHUgTtMKj6we2OG2sJIF2H2KdxCbObVKqAQ+Ky794UYj4ikoaqGdoBhtxRGq4MZ0XywPg6sIjYd9wLgFjMbMCWb2RIzqzSzytra2kTGKCIprroxNkZh4gG2FEabMJPCF4CHPGYD8A4wZ6CC7r7M3SvcvaK8XDN2i8jQVTXGWgqT1FIYkjCTwmbgdAAzGw8cDmwKMR4RSUPVDR0U52aSl6U78IciyFtSfw0sAsrMbCvwbSATwN2XAt8F7jSzN4iNkL7B3euCikdERqd9jVGQDwry7qOL9/N6FXBmUMcXEYFYn8KkEvUnDFWYl49ERAJX3dihlsIwKCmISNrq6O5lV2uXWgrDoKQgImnr/dtR1VIYKiUFEUlb1bsHrmmMwpApKYhI2qqKtxQ0RmHolBREJG3tbimky2R1iaCkICJpq6qxg7KCLLIzUnd5zERTUhCRtFXd2K5WwjApKYhI2qpu6FAn8zApKYhI2qpqbGeSWgrDoqQgImmppbOH5o4eJmrg2rAoKYhIWnp/jIJaCsOhpCAiaen9MQpqKQyHJhhPAHens6ePju5e2rt7ae3sYVdrN7taO/v/be/uxTDMYvOIY0ZBdpSx+dmUFmRRmp9FaUE2Y/Iyyc2MDriAuIi8Ty2FAzOqk0J3bx/1bV309jm9fU5fH/S60+eOe+xk7oA7NHd0U9PcSW1zJzXNHdQ1dwGQkxkhOzNKTkaEaCRCfVsXNc0d7GiKldvZ0kV7dy/u+44lGomd5N2dvv2UzcqIMCYvkzF5WRTlZpKfFSUnM0puZpTszCh5WVFKcjMpyc+iJDdWriQvk7H5WYzNzyInU/dsS/qrauzADMYXKSkMx6hJCqurGnlw5TaqG9upauyguqGd2pbO/Z6sBxKNGGPzs4gYdHT30dnTS0d3HwCF2RmUF2UzvjCHD00dQ1lBNvlZsZN1bmaU3KzYSXv3CXpsfhZj8j54onZ3Wrt62dnSSV1LF7tau9jZ0kl9WzcNbV3Ut3VR39ZNY1s3dfHE0xF/tHbGWiSDyc2MHX98UTYzxxW8/ygvZFJJDhlRXVWU1Ffd0M64wmwy9fM8LKMmKVQ1dHDvy5uZWJzDpJJcDj+8nInFuZQVZJERjRA1IxIxohGIxC/NRGz35RyjICeD8oJsxhVlMyYvq/8v+93cnZ4+H7EfQDOjIDuDguwMDi3NH/b7O3t6aWzvpqEt9tjV2kVDWxe72rqob+1iZ2sX2+rbefqtGu6r3Nr/vohBWUE2E4pzGFeYw4TibMbmZ8daHPmZlORmMSY/i2mleZTkZY1IXUWCEFtHQf0JwzVqksLpc8ax+t8/Hti1eDMjM5o81/mzM6KMK4wyrnD/TeeGti421LSwoaaFbQ3t7GjqYHtTJ1vr26h8bxeN7d0DtqjG5mcxozyfGeUFzBpfyEkzSzl8fKH6OyQpVDW2M2dCYdhhpJxRkxQiEZ2oBlOSl0XFtLFUTBs74Ou9fU5TezcN7d3Ut3Wxs6WLd+ta2VjbwqbaVp5as4N7X94CwPiibE6ZVc6ps8s5dVaZWhMSCnenuqGDjxw+LuxQUs6oSQpy4KIRY0x+7LLRdAa+lFXd2M5f1tXx53W1PLVmBw+s3EpGxDh1djmLF0ziY3PHk5elHzdJjMb2btq7e3Xn0QEI7LfUzO4AzgZq3H3eIGUWATcBmUCdu58WVDwSrInFuVx47BQuPHYKvX3Oqi0N/M/q7Tz6WhVPv1VDXlaUM+eO528qpvDhGaW6xCSBqmrQGIUDFeSfbncCtwB3DfSimZUAtwJnuftmM1M7L01EI8bCQ8ew8NAx3HDWHF56dxePrKri8TeqeXhVFXMmFHLlKYfx6aMnkZWhO0Nk5FU3aozCgQrsN9LdlwO79lHkb4GH3H1zvHxNULFIeCIR44TDSvn++fN56Zun86MLjqLPnX+8/zVO/uHT/N9nNtDQ1hV2mJJmNJr5wIX5Z9psYIyZPWtmK83sssEKmtkSM6s0s8ra2toEhigjKTsjyoUVU3jy2lP55RXHMXt8IT9+8m2O/99/4p/uf403tjaGHaKkieqGdjIiRllBdtihpJwwe/4ygIXA6UAu8IKZrXD3dXsXdPdlwDKAioqKAxhuJsnEzDhtdjmnzS5nbXUTd73wHg+/uo37V25lwZQSLjvxUD511EStliUHrLqxg/FFOR8YTyT7F2ZLYSvwhLu3unsdsBw4OsR4JARHTCzi++fPZ8U3TudbZ8+lqb2b6+57jZN+8DQ3/s/b7GjqCDtESUFVDe1MKlF/woEIMyk8ApxiZhlmlgccD6wNMR4JUXFuJlecPJ0/Xncad11xHEcfUsJPntnAST94mmvufZVXN9eHHaKkEI1mPnBB3pL6a2ARUGZmW4FvE7v1FHdf6u5rzewJ4HWgD/i5u78ZVDySGiLxsQ2nzi7n3bpWfvnCu9xfuZVHVlWx6PByvv6JOcyZUBR2mJLE+vqc7Y0dTJyvlsKBMD+QGeFCVFFR4ZWVlWGHIQnU0tnD3Sve49ZnNtDc2cMFHzqE686crb8EZUAbapo548bl/Punj+TzH54WdjhJw8xWunvF/srpJnFJegXZGVx12gyW//NHuPLk6bFWw4+f5UdPvEVbV0/Y4UkS6e1zbnjwDQpzMvj4kRPCDiclKSlIyijJy+Kbn5rLn64/jU/Mm8Ctz27k4zct57n1dWGHJkli2fJNrHyvnu8sPpIJGrh2QJQUJOVMGZvHTRcdw2+WnEBmJMIlt7/IP93/mgbBjXJrq5u48am3+eT8CZy7YHLY4aQsJQVJWccfVsrj15zC1Ytm8NCr2zjjxuU8/kZ12GFJCDp7evnab1ZRnJvFf5w7X3NrHQQlBUlpOZlR/vmsOTz29yczsTiHq+95hevuW0VLp/oaRpP/89R63trezI8umM/YfE3XfjA0l7GkhbmTivjt1R/m5qc3cMvT66l8t56bLlrAh6aOCTs0GUFbdrWxprrp/eVs87JYX9PCbcs3cvFxU/jonPFhh5jylBQkbWREI1z3sdmcMquMa+9dxd8sfYFrTp/FVz4yU9MdpIH3drZyzk+eo6njg63AKWNz+ean5oYQVfpRUpC0c+y0sTx+zSn868NvcuNT63j53V387LIKcjI1l1Kqau3sYcldK4lEjHuuPJ7ePmdXaxe7WrtobO9m8YJJFGTrdDYS9L8oaak4N5ObLz6GE2eU8r8eeoO//9Ur/PSShWRG1Y2Watydf3rgNdbXNPPLK47jpJllYYeU1vQbImnt4uOm8t1z5/HHtTV87Ter6O1LrRH8Aj/980Yef2M7N5w1h1NmlYcdTtpTS0HS3qUnHEpbZw/f/8Nb5Gdl8P3z5xNRH0NKePbtGn785Nucc/Qklpx6WNjhjApKCjIqfOm0GbR29nDz0xvIy47yrbPn6l72JPduXStf/fWrHD6+kB9+RmMPEkVJQUaNr31sNi2dvdzx13fIikb4+ifm6ESTpJ54cztff+h1IhHjZ5dVkJelU1Wi6H9aRg0z41/PPoLu3j5uW76J+rYu/vd588lQ53PSaOvq4TuPreHel7cwf3Ix/33RAqaMzQs7rFFFSUFGFTPjO4uPZGx+Fv/9p/XUt3Xzk4uP0e2qCdLU0c3K9+p5e3szZQXZTBmTy5SxeYwvymF1VSPX3ruKd3a28uVFM/jaGbPJylDCTjQlBRl1zIyvfWw2Y/Oz+LfHVnPZHS/x889XUJSTGXZoaaGzp5f61u7+cQQ1zR28tqWBl96t563tTQy0hEtm1OhzGFeYza+uPIETZ5QmPnABlBRkFPv8h6cxJj+L6+9bxWdvW8FdVxxHeWF22GGlrMb2bi69/UVe39r4gdfysqJ8aOoYrjl9FsdNG8uRk4upb+1iS30bW3a1s6W+DYAvnXoYJXmauyhMSgoyqn366EmMyctkyV0rueyOl/jNl05Qi+EA9PU519+3ijVVTfzDR2cyoTiHsXmx+YlKC7I4tDT/AwMHi3MzmVaWH1LEMhglBRn1TplVzm2XLuSKO19myV2V3PmF49THMEy3PruBP66t4dvnzOULJ00POxw5CIH14pjZHWZWY2Zv7qfcsWbWa2YXBBWLyP6cOruc/7rwaFZs2sW192rk83D8eV0t//XUOhYvmMTlWhM55QXZtX8ncNa+CphZFPgh8GSAcYgMyeIFk/nXs+fyxOrt/Osjb+ID9YjK/2PLrjauuTc2wOz752uAWToI7PKRuy83s2n7KfYPwIPAsUHFITIcXzx5OrXNnSz980bGFWZz7Rmzww4paXV09/Lle1bS2+csvWShBpilidC+RTObDJwHfJT9JAUzWwIsAZg6dWrwwcmodsNZh1Pb3MlNf1xPn8PXzpilv4D30t7Vy/X3r+LNbU3c/vkKdRinkTBT+03ADe7eu79fOHdfBiwDqKioUJteAmVm/OAz8zGDm/+0nq31bfzg/KM0kCrunbpWvnz3St7e0cw3P3kEpx+h1c7SSZhJoQK4N54QyoBPmlmPuz8cYkwiAGRGI/z4gqOYOjaPG59aR3VDB0svXUhx7ui+XfWJN6v5p/tfJxo17vzCcZw2W1NZp5vQ/vRx9+nuPs3dpwEPAFcrIUgyMTO+evosbrzwaCrf28UFP32erfFBVqNNd28f3/v9Gq66+xUOG1fA7796ihJCmgqspWBmvwYWAWVmthX4NpAJ4O5LgzquyEg7/0OHMKE4hy/9fys579bnuXfJCcwoLwg7rMA1dXSzfF0tT79Vw5/frmVnaxeXnXgo3/zUEWRnaBxHurJUu+2uoqLCKysrww5DRqH1O5q5+GcryIxGuO9LJ6bt7J1vbmvke79fy8vv7qKnzynJy2TR7HIWL5jMR+aMCzs8OUBmttLdK/ZbTklBZOjWVjdx0bIVFOdmcv9VJzK+KCfskEbUxtoW/mbpC2REjM8sPITT54xjwZQSTS+eBoaaFPRNiwzDEROL+OUVx7GzpZPP/fxFdrZ0hh3SiKlubOey21/CgN986URuOGsOFdPGKiGMMvq2RYZpwZQS7rj8WLbWt3Hp7S/R2N4ddkgHraGti8vidfnlFccxXeMORi0lBZEDcPxhpdx2aQXra5r525+tYE1VU9ghHbC2rh6+cOfLvLezjWWXLWTe5OKwQ5IQKSmIHKDTZpez9JKFVDd2cPZP/sK/Pbo65VoN3b19fPnuV3htSwM3X7yAD88oCzskCZmSgshBOP2I8Txz/SIuOeFQ7nrhXT76n89yX+UW+lJkltXbn3uHP6+r5bvnzuOseRPDDkeSgJKCyEEqzsvkO4vn8ejfn8yhpXn88wOvc/U9ryT9LKvVje3c/Kf1nHHEeD53/KFhhyNJQklBZITMm1zMA1d9mH88czZPrN7Or17aHHZI+/Qfv19Lb5/z7XPmhh2KJBElBZERFIkYVy+aySmzyviP363l3brWsEMa0HPr6/j969VcvWhm2g7CkwOjpCAywiIR40cXHEVG1Lj+/teSbhW3rp4+vv3om0wdm8eXTjss7HAkySgpiARgYnEu3108j5Xv1XPb8o1hh/P/+MVf32FjbSv/9um5WotaPkBJQSQgixdM4lPzJ/J/nlqXNOMYqhvb+e8/reeMI8bx0TlaB0E+SElBJCBmxnfPnUdJXhbX3beKzp7eUONx9z06l48MNRZJXlpUVSRAY/Oz+OFn5nPFnZWc9IOnKc3Ppig3g+LcTApzMunu7aO9q5fWrh7au3rJiEb4t3OOZP4hIzuquKO7l395+E1+/3o1XztjtjqXZVCaJVUkAe6v3MJL7+yisb2bpo5umtp7aO4R6Sz+AAAJEUlEQVTsJjMSITcrSl5WlNysDN6qbqKnz7nvSycyc9zIrNmweWcbV929kjXVTXz19Flcc/osohGtOT3aaOpskRT0Tl0rf7P0eTKjER748oeZXJJ7UJ/3zFs1XPubVbg7N120QP0Io5iSgkiKWl3VyEXLVlBekM19V51IWUH2ft/j7jS2d1PV0EF1YztVDe2sqW7m3pc3c8SEIpZespCppbpkNJoNNSmoT0EkyRw5qZhfXH4sl9z+Ip+/4yV+veQEcjKivLK5nr9uqOMv6+vYWNNCT5/T605vnw84FiIzaly4cAr/vvhI3XoqQ6aWgkiSevbtGv7urkrGFeZQ39ZFW1cv0Yhx9CHFzJ9cTFZGhGgkQkbEiESMopwMJpXkMrE4h8kluZQVZBNR34HEhd5SMLM7gLOBGnefN8DrnwNuiD9tAb7s7q8FFY9Iqll0+Dj++6Jj+L/PbOCjc8Zx8qwyTpxRSlFOZtihSRoL8vLRncAtwF2DvP4OcJq715vZJ4BlwPEBxiOScj45fyKfnK8prSVxAksK7r7czKbt4/Xn93i6AjgkqFhERGRokmVE8xeBP4QdhIjIaBf63Udm9hFiSeHkfZRZAiwBmDp1aoIiExEZfUJtKZjZUcDPgcXuvnOwcu6+zN0r3L2ivLw8cQGKiIwyoSUFM5sKPARc6u7rwopDRETeF+Qtqb8GFgFlZrYV+DaQCeDuS4FvAaXArWYG0DOUe2hFRCQ4Qd59dPF+Xr8SuDKo44uIyPAly91HIiKSBFJumgszawTW77GrGGgc5Pnu7T33lQF1BxHC3scbbrmB9u9v31C2E1GvfZUZSr2G8l3tuR32dzXYa6ler6Hu18/gwddpsLiGU2ak6nWou+//Th13T6kHsGyoz3dv77WvciSPP9xyA+3f374hbgder32VGUq9hvJd7fW9hfpdpWu9hrpfP4MHX6dkq9dQHql4+eixYTx/bJAyI3n84ZYbaP/+9g1l+2AN5bP2VWYo9RrKdzXUWIbiYL+rwV5L9XoNdb9+BkdGMtVrv1Lu8tHBMrNKT8O7nNKxXulYJ1C9Ukk61ml/UrGlcLCWhR1AQNKxXulYJ1C9Ukk61mmfRl1LQUREBjcaWwoiIjIIJQUREemnpCAiIv2UFPZgZhEz+56Z/cTMPh92PCPBzBaZ2V/MbKmZLQo7npFkZvlmttLMzg47lpFiZkfEv6sHzOzLYcczEszsXDP7mZk9YmZnhh3PSDGzw8zsdjN7IOxYRlLaJAUzu8PMaszszb32n2Vmb5vZBjP7+n4+ZjEwGegGtgYV61CNUJ2c2BrYOSRBnWDE6gWxNb7vCybK4RuJern7Wne/CrgQCP1WyBGq08Pu/nfA5cBnAwx3yEaoXpvc/YvBRpp4aXP3kZmdSuzkd5e7z4vviwLrgI8ROyG+DFwMRIHv7/URV8Qf9e5+m5k94O4XJCr+gYxQnercvc/MxgM3uvvnEhX/YEaoXkcRm4Igh1gdf5eY6Ac3EvVy9xoz+zTwdeAWd/9VouIfyEjVKf6+/wLucfdXEhT+oEa4XqGfK0ZS6CuvjRQfeE3o44AN7r4JwMzuJbagz/eBD1xyiE/x3RV/2htctEMzEnXaQz2QHUScwzVC39VHgHxgLtBuZo+7e1+gge/HSH1f7v4o8KiZ/R4INSmM0HdlwA+APyRDQoAR/91KK2mTFAYxGdiyx/OtwPH7KP8Q8BMzOwVYHmRgB2FYdTKz84GPAyXALcGGdlCGVS93/yaAmV1OvDUUaHQHbrjf1yLgfGIJ/PFAIztww/29+gfgDKDYzGZ6bD2VZDTc76oU+B5wjJn9r3jySHnpnhRsgH2DXi9z9zZi60Uns+HW6SFiyS7ZDate/QXc7xz5UEbUcL+vZ4FngwpmhAy3TjcDNwcXzogZbr12AlcFF0440qajeRBbgSl7PD8EqAoplpGSjnUC1SuVpGOdIH3rNSzpnhReBmaZ2XQzywIuAh4NOaaDlY51AtUrlaRjnSB96zUsaZMULLYm9AvA4Wa21cy+6O49wN8DTwJrgfvcfXWYcQ5HOtYJVK9Uqlc61gnSt14jIW1uSRURkYOXNi0FERE5eEoKIiLST0lBRET6KSmIiEg/JQUREemnpCAiIv2UFCRtmFlLgo/3czObm+BjXmtmeYk8powuGqcgacPMWty9YAQ/LyM+oClh4jOK2mAT/JnZu0CFu9clMi4ZPdRSkLRmZuVm9qCZvRx/nBTff5yZPW9mr8b/PTy+/3Izu9/MHgP+x2Ir1z1rsZXQ3jKze+InbuL7K+LbLRZbte81M1sRX78CM5sRf/6ymX1noNaMmU0zs7VmdivwCjDFzH5qZpVmttrM/j1e7qvAJOAZM3smvu9MM3vBzF6Jxz1iSVFGKXfXQ4+0eAAtA+z7FXByfHsqsDa+XQRkxLfPAB6Mb19ObGK0sfHni4BGYpOjRYhNjbD7854l9lc7xGbTPCe+/SPgX+LbvwMujm9fNUiM04A+4IQ99u0+fjR+nKPiz98FyuLbZcSmeM+PP78B+FbY34Meqf1I96mzRc4A5sb/uAcoMrNCoBj4pZnNInZCz9zjPU+5+649nr/k7lsBzGwVsZP4c3sdp4tYAgBYSWz1LoATgXPj278C/nOQON9z9xV7PL/QzJYQm95+IrHFhF7f6z0nxPf/NV6/LGJJS+SAKSlIuosAJ7p7+547zewnwDPufl58Ba5n93i5da/P6Nxju5eBf2+63d33U2Zf+o9pZtOBfwSOdfd6M7uT2LKjezNiCeziYR5LZFDqU5B09z/EZr4EwMwWxDeLgW3x7csDPP4K4DPx7YuG+J4iYkmiMd438Yk9XmsGCvf47JPMbCaAmeWZ2eyDD1lGMyUFSSd58WmQdz+uA74KVJjZ62a2hvdXyvoR8H0z+yux6/ZBuRa4zsxeInYZqHF/b3D314BXgdXAHcBf93h5GfAHM3vG3WuJJbRfm9nrxJLEnJENX0Yb3ZIqEqD4mIJ2d3czu4hYp/PisOMSGYz6FESCtRC4JX4bawNwRcjxiOyTWgoiItJPfQoiItJPSUFERPopKYiISD8lBRER6aekICIi/ZQURESk3/8PXv9I9iSEMVgAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "lrf.plot()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## Model Building - 70 Fonts - PyTorch\n",
    "\n",
    "\n",
    "\n",
    "### Hyper Parameter Tuning"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "\n",
    "* Learning Rate Scheduler"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "# Get Pretrained Model\n",
    "model_ft = models.resnet50(pretrained=True)\n",
    "\n",
    "# Customize FC Layer\n",
    "num_frts = model_ft.fc.in_features\n",
    "model_ft.fc = nn.Linear(num_frts, len(class_names))\n",
    "\n",
    "model_ft = model_ft.to(device)\n",
    "criterion = nn.CrossEntropyLoss()\n",
    "\n",
    "# Define optimizer & LR Scheduler\n",
    "optimizer_ft = optim.SGD(model_ft.parameters(), lr=0.01, momentum=0.9)\n",
    "exp_lr_scheduler = lr_scheduler.StepLR(optimizer_ft, step_size=7, gamma=0.1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## Model Building - 70 Fonts - PyTorch\n",
    "\n",
    "\n",
    "\n",
    "### Result - without LR finder & scheduler"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "# 0.74 f1-score after 40 epochs"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## Model Building - 70 Fonts - PyTorch\n",
    "\n",
    "\n",
    "\n",
    "### Result - with LR finder & scheduler"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "# 0.96 f1-score after 40 epochs"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Fontastic\n",
    "\n",
    "## Feature Visualization \n",
    "\n",
    " Mechanism to *See through the eyes of network*"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## Feature Visualization \n",
    "\n",
    "### Gradcam Analysis"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "import pickle\n",
    "from ipywidgets import interact, interactive, fixed, interact_manual\n",
    "import ipywidgets as widgets\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "\n",
    "with open('./fd727d3f-73f4-4ec6-8e89-3e15fd3801b0resnet50_grad_cam', 'rb') as f:\n",
    "    data = pickle.load(f)\n",
    "\n",
    "def show_cam(epoch_slider, image_slider, layer_slider):\n",
    "    plt.imshow(data[epoch_slider][image_slider][layer_slider])\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## Feature Visualization \n",
    "\n",
    "### Gradcam Analysis"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "153edd16f4464d4ea73ec3f2ce85c945",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "interactive(children=(IntSlider(value=0, description='epoch_slider', max=19), IntSlider(value=0, description='…"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": [
       "<function __main__.show_cam(epoch_slider, image_slider, layer_slider)>"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "interact(show_cam, epoch_slider=widgets.IntSlider(min=0, max=len(data)-1, step=1, value=0),\n",
    "                   image_slider=widgets.IntSlider(min=0, max=len(data[0])-1, step=1, value=0),\n",
    "                   layer_slider=widgets.IntSlider(min=0, max=len(data[0][0])-1, step=1, value=0))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## Feature Visualization \n",
    "\n",
    "### Activation Atlas\n",
    "\n",
    "Activation Atlases not only reveal visual abstractions within a model, but they can reveal high-level misunderstandings in a model that can be exploited. For example, by looking at an activation atlas we will be able to see why a picture of a baseball can switch the classification of an image from “grey whale” to “great white shark”.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## Feature Visualization \n",
    "\n",
    "### Activation Atlas - https://distill.pub/2019/activation-atlas/\n",
    "\n",
    "<img src=\"imgs/activation-atlas.png\" alt=\"drawing\" width=\"800\" style=\"alig:center\"/>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Extra\n",
    "## Remote Work"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "\n",
    "## Open Source\n",
    "\n",
    "### https://github.com/deep-learning-for-humans/fontastic"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Fin.\n",
    "\n",
    "## GH / twitter / everywhere\n",
    "* **@nischalhp**\n",
    "\n",
    "* **@raghothams**\n",
    "\n",
    "https://github.com/deep-learning-for-humans/fontastic\n"
   ]
  }
 ],
 "metadata": {
  "celltoolbar": "Slideshow",
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.8"
  },
  "livereveal": {
   "font-family": "Lato",
   "footer": "Strata Data Conference 2019 | @nischalhp | @raghothams ",
   "scroll": true,
   "theme": "sky",
   "transition": "none"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}