sentiment-movie-review.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Sentiment Analysis\n",
    "\n",
    "Hi 🙂, if you are seeing this notebook, you've succesfully started your first project on FloydHub, hooray! 🚀\n",
    "\n",
    "[Sentiment analysis](https://en.wikipedia.org/wiki/Sentiment_analysis) is one of the most common [NLP](https://en.wikipedia.org/wiki/Natural-language_processing) problems. The goal is to analyze a text and predict whether the underlying sentiment is positive, negative or neutral. \n",
    "*What can you use it for?* Here are a few ideas - measure sentiment of customer support tickets, survey responses, social media, and movie reviews! \n",
    "\n",
    "### Predicting sentiment of movie reviews\n",
    "\n",
    "In this notebook we will build a [Convolutional Neural Network](http://www.wildml.com/2015/11/understanding-convolutional-neural-networks-for-nlp/) (CNN) classifier to predict the sentiment (positive or negative) of movie reviews. \n",
    "\n",
    "<img src=\"https://github.com/floydhub/sentiment-analysis-template/raw/master/images/sentiment.jpg\" width=\"800\" height=\"800\" align=\"center\"/>\n",
    "\n",
    "We will use the [Stanford Large Movie Reviews](http://ai.stanford.edu/~amaas/data/sentiment/) dataset for training our model. The dataset is compiled from a collection of 50,000 reviews from IMDB. It contains an equal number of positive and negative reviews. The authors considered only highly polarized reviews. Negative reviews have scores ≤ 4 (out of 10), while positive reviews have score ≥ 7. Neutral reviews are not included. The dataset is divided evenly into training and test sets.\n",
    "\n",
    "We will:\n",
    "- Preprocess text data for NLP\n",
    "- Build and train a 1-D CNN using Keras and Tensorflow\n",
    "- Evaluate our model on the test set\n",
    "- Run the model on your own movie reviews!\n",
    "\n",
    "### Instructions\n",
    "\n",
    "- To execute a code cell, click on the cell and press `Shift + Enter` (shortcut for Run).\n",
    "- To learn more about Workspaces, check out the [Getting Started Notebook](get_started_workspace.ipynb).\n",
    "- **Tip**: *Feel free to try this Notebook with your own data and on your own super awesome sentiment classification task.*\n",
    "\n",
    "Now, let's get started! 🚀"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Initial Setup\n",
    "\n",
    "Let's start by importing some packages"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Using TensorFlow backend.\n"
     ]
    }
   ],
   "source": [
    "from keras.preprocessing import sequence\n",
    "from keras.models import Sequential\n",
    "from keras.layers import Dense, Embedding, GlobalMaxPooling1D, Flatten, Conv1D, Dropout, Activation\n",
    "from keras.preprocessing.text import Tokenizer\n",
    "\n",
    "import tensorflow as tf\n",
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "\n",
    "import os\n",
    "import re\n",
    "import string\n",
    "\n",
    "# For reproducibility\n",
    "from tensorflow import set_random_seed\n",
    "from numpy.random import seed\n",
    "seed(1)\n",
    "set_random_seed(2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Training Parameters\n",
    "\n",
    "We'll set the hyperparameters for training our model. If you understand what they mean, feel free to play around - otherwise, we recommend keeping the defaults for your first run 🙂"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Hyperparams if GPU is available\n",
    "if tf.test.is_gpu_available():\n",
    "    # GPU\n",
    "    BATCH_SIZE = 128 # Number of examples used in each iteration\n",
    "    EPOCHS = 2 # Number of passes through entire dataset\n",
    "    VOCAB_SIZE = 30000 # Size of vocabulary dictionary\n",
    "    MAX_LEN = 500 # Max length of review (in words)\n",
    "    EMBEDDING_DIM = 40 # Dimension of word embedding vector\n",
    "\n",
    "# Hyperparams for CPU training\n",
    "else:\n",
    "    # CPU\n",
    "    BATCH_SIZE = 32\n",
    "    EPOCHS = 2\n",
    "    VOCAB_SIZE = 20000\n",
    "    MAX_LEN = 90\n",
    "    EMBEDDING_DIM = 40"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Data\n",
    "\n",
    "The movie reviews dataset is already attached to your workspace (if you want to attach your own data, [check out our docs](https://docs.floydhub.com/guides/workspace/#attaching-floydhub-datasets)).\n",
    "\n",
    "Let's take a look at data. The labels are encoded in the dataset: **0** is for *negative* and **1** for a *positive* review."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train shape (rows, columns):  (25000, 2) , Validation shape (rows, columns):  (25000, 2)\n",
      "\n",
      "--- First Sample ---\n",
      "Label: 0\n",
      "Text: Watch the Original with the same title from 1944! This made for TV movie, is just god-awful! Although it does use (as far as I can tell) almost the same dialog, it just doesn't work! Is it the acting, the poor directing? OK so it's made for TV, but why watch a bad copy, when you can get your hands on the superb original? Especially as you'll be spoiled to the plot and won't enjoy the original as much, as if you've watched it first! <br /><br />There are a few things that are different from the original (it's shorter for once), but all are for the worse! The actors playing the parts here, just don't fit the bill! You just don't believe them and who could top Edward G. Robinsons performance from the original? If you want, only watch it after you've seen the original and even then you'll be very brave, if you watch it through! It's almost sacrilege!\n"
     ]
    }
   ],
   "source": [
    "DS_PATH = '/floyd/input/imdb/' # ADD path/to/dataset\n",
    "LABELS = ['negative', 'positive']\n",
    "\n",
    "# Load data\n",
    "train = pd.read_csv(os.path.join(DS_PATH, \"train.tsv\"), sep='\\t')  # EDIT WITH YOUR TRAIN FILE NAME\n",
    "val = pd.read_csv(os.path.join(DS_PATH, \"val.tsv\"), sep='\\t')  # EDIT WITH YOUR VALIDATION FILE NAME\n",
    "\n",
    "print(\"Train shape (rows, columns): \", train.shape, \", Validation shape (rows, columns): \",  val.shape)\n",
    "\n",
    "# How a row/sample looks like\n",
    "print(\"\\n--- First Sample ---\")\n",
    "print('Label:', train['label'][0])\n",
    "print('Text:', train['text'][0])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZUAAAEWCAYAAACufwpNAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAH3dJREFUeJzt3Xu8VmWd9/HPV0QxT4DuiIAJKKrHmSeVSG3ycUwTUaewMrWxJMd56KDZYWqknIlGc17kU5qW6UNJYlMeslIKT+ShswoqIorEVvEFPCDbOHhKC/09f6zfluVu35t7b9a9977h+3691muvda1rrfW79n2zf6xrrXUtRQRmZmZV2KGvAzAzs22Hk4qZmVXGScXMzCrjpGJmZpVxUjEzs8o4qZiZWWWcVKzpSJokqbWv4zCzv+akYn1C0jOl6SVJfyotn9TX8W2LJN0p6UN9HYdt23bs6wBs+xQRu7XPS1oO/EtE/KLvIuobknaMiE19HYdZVXymYv2SpF0kXSxptaSVkv6PpIE16n5e0iJJr8nl9+byBkm/lrRPqe4aSZ+RtFjSRkk/kLRTrnuNpJtyuz9Kuq3G8QZJCkmnS1ouqU3SuZJUqvNRSUslrZM0V9KIDtt+XNIjwOJO9r+rpKty2w2S7pI0JNcNlXRFtmOFpOmSdsh1H5N0q6SLcrtHJL0r130deBvw3Twb/HqW/52k2yStl7RE0rGlOK6S9A1JN0t6WtJvJb2utH7f0rZrJP1rlg+Q9B+SHpX0ZP6OB9f1wVvziwhPnvp0ApYD7+pQdh7wa2BvYBgwHzgr100CWnP+v4C7gKG5fBCwGngrMACYCvwB2DHXrwF+m/tsAVqBj+S6C4ALKc7gdwIOqRHvICCAm4HBwBjgUeBDuf4EYAnwRmAg8BXg9g7bzs1td+lk/58CrgV2yVjeBuya624Evgm8ChgO3AdMyXUfA/4CnJxt/wywvLTfO9tjzOU98nd1UtZ/G7AOeEOuvwpYC4zPdlwLXJ7rhgBtwOnAzrmvt+W6M/Oze22293Lge339PfPUO1OfB+DJU42ksgo4rLQ8GXg45ycBjwAXA7cDu5fqfa89+ZTKHgcOzPk1wHGldRcB38j584AfAWO3EG97Yji0VPZZYG7O3w6cVFo3MP/YDytt+/dd7P8TwC+Bv+tQ/jrgWWBgqewU4Mac/xiwuLRuaB5rcC53TCpTgHkdjjEbODPnrwK+VVr3PmBh6bi/rxH/Y8A7SstjgOcA9fV3zVPjJ19TsX4nu5FeQ5EM2j0OjCgtv5riD9u7I+LpUvnrgOMlfb5UtlOHbdeU5p+jOBsCOBc4G7hd0l+Ab0fE+V2EuqJDfK8txXCppItL6zcBI4GNnWzb0WUU7b9W0m7AFcB/5H4HAW2lnrYdKM62arUNYDdgQyfHeR1wiKTyuh2B9V3sr/1a2CiKxP4K+dmNAm6QVB6tdgdgL+DJTuKwbYiTivU7ERGS1lD80Wv/w/U3FGcv7Z6g+B/9DyX9Y0TMz/IVFGcMX+/BcTdSdD19StK+FMnlroj4bY1Nyn9Y/wb4f6UYPh8RP+64gaRB7YfrIo4XgC8BX5I0lqKb7UHgd8AzwJCI6Mnw4h23WQHcEhHv7sG+VgBH/dUBis9uFfC+iLinB/u1JucL9dZfXQlMl7SXpFcDZwH/Xa4QEbcA/wz8TNL+WTwT+KSkCSrsJuk9kl61pQNmvbH5v+2NwIvAS11scqakPSWNpri2cHWWXwr8u6Q35X6HSHp/ne1G0rsk7ZMX4J+iOMt5KSIeo+jCOk/S7pJ2kDRO0sF17voJYGxp+Tpgf0knSBooaSdJB0l6Yx37ug54Q95wsJOkPSS9LdddCsyQNCrb82pJPUlc1oScVKy/+hLwEMX/0BdSXFw/r2OliJgLfBy4UdJb8qziDOD/UnT5/AH4J7o4Myj5HxTXQ54GfgV8LSJ+30X9ucD9wAKKazH/nTFdCXwL+ImkpzL+I+o4frsRwPUZx2LgBjYnrA9SXOB/mOKi+tUU12rqcQFwct6tdV5ErAeOpOhGXE1xpvUVimtAXcptjwBOpLiYvxRoT27nAb8AbpP0NMUZ1vg6Y7Qmp56dRZttv7IL60/AqIhY2dfxmPUnPlMxM7PKOKmYmVll3P1lZmaV8ZmKmZlVZpt8TmXvvfeO0aNH93UYZmZN5Z577nkyIlq2Zh/bZFIZPXo0CxYs6OswzMyaiqTHt1yra+7+MjOzyjipmJlZZZxUzMysMk4qZmZWGScVMzOrjJOKmZlVxknFzMwq46RiZmaVcVIxM7PKbJNP1Pe20dPmdlq+fMYxvRyJmVnf8pmKmZlVxknFzMwq46RiZmaVcVIxM7PKOKmYmVllnFTMzKwyTipmZlYZJxUzM6uMk4qZmVWmoUlF0mBJ10p6WNISSW+XNFTSPEnL8ueQrCtJF0lqlbRI0vjSfqZk/WWSpjQyZjMz67lGn6lcCNwUEW8G9gWWANOAWyNiHHBrLgMcBYzLaSpwCYCkocB04EDgAGB6eyIyM7P+pWFJRdKewCHAZQAR8eeI2ABMBmZntdnAsTk/GbgiCncCgyUNB44E5kXEuohYD8wDJjUqbjMz67lGnqmMAdqA70m6T9J3Je0KDIuI1VlnDTAs50cAK0rbr8yyWuWvIGmqpAWSFrS1tVXcFDMzq0cjk8qOwHjgkojYH3iWzV1dAEREAFHFwSJiZkRMiIgJLS0tVezSzMy6qZFJZSWwMiLuyuVrKZLME9mtRf5cm+tXAaNK24/MslrlZmbWzzQsqUTEGmCFpDdl0eHAQ8AcoP0OrinA9Tk/Bzg57wI7CNiY3WQ3AxMlDckL9BOzzMzM+plGv6Trk8APJO0EPAqcQpHIrpF0KvA4cHzWvQE4GmgFnsu6RMQ6SecA87Pe2RGxrsFxm5lZDzQ0qUTEQmBCJ6sO76RuAKfV2M8sYFa10ZmZWdX8RL2ZmVXGScXMzCrjpGJmZpVxUjEzs8o4qZiZWWWcVMzMrDJOKmZmVhknFTMzq4yTipmZVcZJxczMKuOkYmZmlXFSMTOzyjipmJlZZZxUzMysMk4qZmZWGScVMzOrjJOKmZlVxknFzMwq46RiZmaVcVIxM7PKOKmYmVllnFTMzKwyTipmZlaZhiYVScslPSBpoaQFWTZU0jxJy/LnkCyXpIsktUpaJGl8aT9Tsv4ySVMaGbOZmfVcb5ypvDMi9ouICbk8Dbg1IsYBt+YywFHAuJymApdAkYSA6cCBwAHA9PZEZGZm/UtfdH9NBmbn/Gzg2FL5FVG4ExgsaThwJDAvItZFxHpgHjCpt4M2M7Mta3RSCeAWSfdImpplwyJidc6vAYbl/AhgRWnblVlWq/wVJE2VtEDSgra2tirbYGZmddqxwfs/OCJWSXo1ME/Sw+WVERGSoooDRcRMYCbAhAkTKtmnmZl1T0PPVCJiVf5cC/yU4prIE9mtRf5cm9VXAaNKm4/MslrlZmbWzzQsqUjaVdLu7fPARGAxMAdov4NrCnB9zs8BTs67wA4CNmY32c3ARElD8gL9xCwzM7N+ppHdX8OAn0pqP84PI+ImSfOBaySdCjwOHJ/1bwCOBlqB54BTACJinaRzgPlZ7+yIWNfAuM3MrIcallQi4lFg307K/wgc3kl5AKfV2NcsYFbVMZqZWbX8RL2ZmVXGScXMzCrjpGJmZpVxUjEzs8o4qZiZWWWcVMzMrDJOKmZmVhknFTMzq0yjB5Tcro2eNrfT8uUzjunlSMzMeofPVMzMrDJOKmZmVhknFTMzq4yTipmZVcZJxczMKuOkYmZmlXFSMTOzyjipmJlZZZxUzMysMk4qZmZWGScVMzOrjJOKmZlVxknFzMwqs8WkIun1knbO+UMlnSFpcONDMzOzZlPPmcqPgRclvQGYCYwCftjQqMzMrCnVk1ReiohNwHuBb0bE54Hh9R5A0gBJ90n6eS6PkXSXpFZJV0vaKct3zuXWXD+6tI8vZPlSSUd2p4FmZtZ76kkqf5H0QWAK8PMsG9iNY3wKWFJa/ipwQUS8AVgPnJrlpwLrs/yCrIekfYATgb8FJgHfljSgG8c3M7NeUk9SOQV4O3BuRDwmaQzw/Xp2LmkkcAzw3VwWcBhwbVaZDRyb85NzmVx/eNafDFwVES9ExGNAK3BAPcc3M7PetcWkEhEPAWcC9+byYxHx1Tr3/w3g34CXcnkvYEN2pwGsBEbk/AhgRR5jE7Ax679c3sk2L5M0VdICSQva2trqDM/MzKpUz91f7wYWAjfl8n6S5tSx3T8CayPinq2Osg4RMTMiJkTEhJaWlt44pJmZdbBjHXW+TNHddAdARCyUNLaO7d4BvEfS0cAgYA/gQmCwpB3zbGQksCrrr6K4s2ylpB2BPYE/lsrblbcxM7N+pK4L9RGxsUPZS53WLImIL0TEyIgYTXGh/baIOAm4HTguq00Brs/5OblMrr8tIiLLT8y7w8YA44C764jbzMx6WT1nKg9K+idggKRxwBnA77bimGcCV0n6CnAfcFmWXwZ8X1IrsI4iERERD0q6BngI2AScFhEvbsXxzcysQepJKp8EzgJeAK4EbgbO6c5BIuIONnefPUond29FxPPAB2psfy5wbneOaWZmvW+LSSUinqNIKmc1PhwzM2tmNZOKpJ8BUWt9RLynIRGZmVnT6upM5Wu9FoWZmW0TaiaViPhl+3yOz/VmijOXpRHx516IzczMmswWr6lIOga4FHgEEDBG0kcj4sZGB2dmZs2lnru/vg68MyJaoXi/CjAXcFIxM7NXqOfhx6fbE0p6FHi6QfGYmVkTq+dMZYGkG4BrKK6pfACYL+l9ABHxkwbGZ2ZmTaSepDIIeAL4h1xuA3YB3k2RZJxUzMwMqO/hx1N6IxAzM2t+9dz9NYZiqJbR5fp++NHMzDqqp/vrOorBHn9GHaMTm5nZ9quepPJ8RFzU8EjMzKzp1ZNULpQ0HbiFYqRiACLi3oZFZWZmTamepPI/gQ8Dh7G5+yty2czM7GX1JJUPAGM93peZmW1JPU/ULwYGNzoQMzNrfvWcqQwGHpY0n1deU/EtxWZm9gr1JJXpDY/CzMy2CfU8Uf/LLdUxMzODOq6pSDpI0nxJz0j6s6QXJT3VG8GZmVlzqedC/beADwLLKAaS/Bfg4kYGZWZmzamepEK+T2VARLwYEd8DJjU2LDMza0b1JJXn8h31CyWdJ+kz9WwnaZCkuyXdL+lBSf+Z5WMk3SWpVdLVuW8k7ZzLrbl+dGlfX8jypZKO7FFLzcys4epJKh/OeqcDzwKjgPfXsd0LwGERsS+wHzBJ0kHAV4ELIuINwHrg1Kx/KrA+yy/IekjaBzgR+FuKM6RvSxpQX/PMzKw3bTGpRMTjEfF8RDwFXARc3uH1wrW2i4h4JhcH5tQ+vMu1WT4bODbnJ+cyuf5wScryqyLihYh4DGgFDqirdWZm1qvq6ca6Q9IekoYC9wLfkXR+PTuXNEDSQmAtMA94BNgQEZuyykpgRM6PAFYA5PqNwF7l8k62KR9rqqQFkha0tbXVE56ZmVWsnu6vPfMs5X3AFRFxIPCuenaeF/b3A0ZSnF28uceRbvlYMyNiQkRMaGlpadRhzMysC/UklR0lDQeOB37ek4NExAbgduDtwGBJ7Q9djgRW5fwqius15Po9gT+WyzvZxszM+pF6ksrZwM1Aa0TMlzSW4pmVLklqkTQ453cBjgCWUCSX47LaFOD6nJ+Ty+T62yIisvzEvDtsDDAOuLuexpmZWe+qZ5iWHwE/Ki0/Sn13fw0HZuedWjsA10TEzyU9BFwl6SvAfRSvKiZ/fl9SK7CO4o4vIuJBSdcADwGbgNMi4sV6G2hmZr2nngEleyQiFgH7d1L+KJ3cvRURz1O8u6WzfZ0LnFt1jGZmVq26nqg3MzOrR8POVKy20dPmdlq+fMYxvRyJmVm16nlO5d9L8zs3NhwzM2tmNZOKpDMlvZ3Nd2oB/L7xIZmZWbPqqvvrYYoL52Ml/TqX95L0pohY2ivRmZlZU+mq+2sD8EWKsbYOBS7M8mmSftfguMzMrAl1daZyJPAl4PXA+cAi4NmIOKU3AjMzs+ZT80wlIr4YEYcDy4HvAwOAFkm/kfSzXorPzMyaSD23FN8cEQuABZI+HhEHS9q70YGZmVnzqed9Kv9WWvxIlj3ZqIDMzKx5deuJ+oi4v1GBmJlZ8/MwLWZmVhknFTMzq4yTipmZVcZJxczMKuOkYmZmlXFSMTOzyjipmJlZZZxUzMysMk4qZmZWGScVMzOrjJOKmZlVxknFzMwq07CkImmUpNslPSTpQUmfyvKhkuZJWpY/h2S5JF0kqVXSIknjS/uakvWXSZrSqJjNzGzr1PM+lZ7aBPxrRNwraXfgHknzKIbPvzUiZkiaBkwDzgSOAsbldCBwCXCgpKHAdGACELmfORGxvoGxd2r0tLm9fUgzs6bSsDOViFgdEffm/NPAEmAEMBmYndVmA8fm/GTgiijcCQyWNJzitcbzImJdJpJ5wKRGxW1mZj3XK9dUJI0G9gfuAoZFxOpctQYYlvMjgBWlzVZmWa3yjseYKmmBpAVtbW2Vxm9mZvVpeFKRtBvwY+DTEfFUeV1EBEWX1laLiJkRMSEiJrS0tFSxSzMz66ZGXlNB0kCKhPKDiPhJFj8haXhErM7urbVZvgoYVdp8ZJatAg7tUH5HI+PuK7Wu2SyfcUwvR2Jm1jONvPtLwGXAkog4v7RqDtB+B9cU4PpS+cl5F9hBwMbsJrsZmChpSN4pNjHLzMysn2nkmco7gA8DD0hamGVfBGYA10g6FXgcOD7X3QAcDbQCzwGnAETEOknnAPOz3tkRsa6BcZuZWQ81LKlExG8A1Vh9eCf1Azitxr5mAbOqi87MzBrBT9SbmVllnFTMzKwyTipmZlYZJxUzM6uMk4qZmVXGScXMzCrjpGJmZpVxUjEzs8o4qZiZWWWcVMzMrDJOKmZmVhknFTMzq4yTipmZVaahL+myavjlXWbWLHymYmZmlXFSMTOzyjipmJlZZZxUzMysMk4qZmZWGScVMzOrjJOKmZlVxknFzMwq46RiZmaVaVhSkTRL0lpJi0tlQyXNk7Qsfw7Jckm6SFKrpEWSxpe2mZL1l0ma0qh4zcxs6zVymJbLgW8BV5TKpgG3RsQMSdNy+UzgKGBcTgcClwAHShoKTAcmAAHcI2lORKxvYNxNw8O3mFl/07AzlYj4FbCuQ/FkYHbOzwaOLZVfEYU7gcGShgNHAvMiYl0mknnApEbFbGZmW6e3r6kMi4jVOb8GGJbzI4AVpXors6xWuZmZ9UN9dqE+IoKiS6sSkqZKWiBpQVtbW1W7NTOzbujtpPJEdmuRP9dm+SpgVKneyCyrVf5XImJmREyIiAktLS2VB25mZlvW20llDtB+B9cU4PpS+cl5F9hBwMbsJrsZmChpSN4pNjHLzMysH2rY3V+SrgQOBfaWtJLiLq4ZwDWSTgUeB47P6jcARwOtwHPAKQARsU7SOcD8rHd2RHS8+G9mZv1Ew5JKRHywxqrDO6kbwGk19jMLmFVhaGZm1iB+ot7MzCrjd9R3otZDhc3CD0WaWV/xmYqZmVXGScXMzCrjpGJmZpVxUjEzs8r4Qv12xBfwzazRfKZiZmaVcVIxM7PKOKmYmVllnFTMzKwyvlBvXY4g4Iv4ZtYdPlMxM7PKOKmYmVll3P1lXfKzLWbWHT5TMTOzyjipmJlZZdz9ZT3ibjEz64zPVMzMrDI+U7FK+QzGbPvmpGK9wsnGbPvgpGJ9ysnGbNviaypmZlYZn6lYv9TVeGSd8ZmNWf/QNElF0iTgQmAA8N2ImNHHIVk/0t0kVIuTk9nWaYqkImkAcDFwBLASmC9pTkQ81LeR2bbG13jMtk5TJBXgAKA1Ih4FkHQVMBlwUrFe0ZMzISci2x41S1IZAawoLa8EDixXkDQVmJqLz0ha2oPj7A082aMI+y+3qY/oq92q3hRt6ia3qTmU2/S6rd1ZsySVLYqImcDMrdmHpAURMaGikPoFt6k5uE3NwW3asma5pXgVMKq0PDLLzMysH2mWpDIfGCdpjKSdgBOBOX0ck5mZddAU3V8RsUnS6cDNFLcUz4qIBxtwqK3qPuun3Kbm4DY1B7dpCxQRVe7PzMy2Y83S/WVmZk3AScXMzCrjpJIkTZK0VFKrpGl9HU93SFou6QFJCyUtyLKhkuZJWpY/h2S5JF2U7VwkaXzfRl+QNEvSWkmLS2XdboOkKVl/maQpfdGWUiydtenLklblZ7VQ0tGldV/INi2VdGSpvN98NyWNknS7pIckPSjpU1nelJ9VF+1p9s9pkKS7Jd2f7frPLB8j6a6M8eq88QlJO+dya64fXdpXp+2tKSK2+4ni4v8jwFhgJ+B+YJ++jqsb8S8H9u5Qdh4wLeenAV/N+aOBGwEBBwF39XX8GdchwHhgcU/bAAwFHs2fQ3J+SD9r05eBz3VSd5/83u0MjMnv44D+9t0EhgPjc3534A8Ze1N+Vl20p9k/JwG75fxA4K78/V8DnJjllwIfz/lPAJfm/InA1V21t6tj+0yl8PIwMBHxZ6B9GJhmNhmYnfOzgWNL5VdE4U5gsKThfRFgWUT8CljXobi7bTgSmBcR6yJiPTAPmNT46DtXo021TAauiogXIuIxoJXie9mvvpsRsToi7s35p4ElFCNeNOVn1UV7ammWzyki4plcHJhTAIcB12Z5x8+p/fO7Fjhckqjd3pqcVAqdDQPT1RervwngFkn3qBiuBmBYRKzO+TXAsJxvprZ2tw3N0rbTsytoVns3EU3Ypuwi2Z/if8FN/1l1aA80+eckaYCkhcBaiqT9CLAhIjZllXKML8ef6zcCe9GDdjmpbBsOjojxwFHAaZIOKa+M4jy2qe8d3xbakC4BXg/sB6wGvt634fSMpN2AHwOfjoinyuua8bPqpD1N/zlFxIsRsR/FCCQHAG/ujeM6qRSaehiYiFiVP9cCP6X4Aj3R3q2VP9dm9WZqa3fb0O/bFhFP5D/2l4DvsLkroWnaJGkgxR/gH0TET7K4aT+rztqzLXxO7SJiA3A78HaK7sf2h97LMb4cf67fE/gjPWiXk0qhaYeBkbSrpN3b54GJwGKK+NvvqJkCXJ/zc4CT866cg4CNpW6L/qa7bbgZmChpSHZXTMyyfqPD9av3UnxWULTpxLwLZwwwDribfvbdzH72y4AlEXF+aVVTfla12rMNfE4tkgbn/C4U76JaQpFcjstqHT+n9s/vOOC2POOs1d7a+uruhP42Udyl8geKfsez+jqebsQ9luLujPuBB9tjp+gPvRVYBvwCGBqb7wq5ONv5ADChr9uQcV1J0c3wF4p+21N70gbgnykuJrYCp/TDNn0/Y16U/2CHl+qflW1aChzVH7+bwMEUXVuLgIU5Hd2sn1UX7Wn2z+ktwH0Z/2LgS1k+liIptAI/AnbO8kG53Jrrx26pvbUmD9NiZmaVcfeXmZlVxknFzMwq46RiZmaVcVIxM7PKOKmYmVllnFSsX5L0zJZrdXufknSbpD2q3neH49whaUIjj5HHOUPSEkk/6FC+X3lU3S62/7Kkz1UQR4ukm7Z2P7ZtcFKx7cnRwP3RYViR/qT0tHM9PgEcEREndSjfj6KtvSIi2oDVkt7RW8e0/stJxZpG/o/4x5Lm5/SOLP9yDvp3h6RHJZ1RYxcnkU8QSxqd/8v/Tr5v4pZ88vgVZxqS9pa0POc/Iuk6Fe8LWS7pdEmflXSfpDslDS0d68Mq3sOxWNIBuf2uGefduc3k0n7nSLqN4gHCju3+bO5nsaRPZ9mlFA+y3SjpM6W6OwFnAyfk8U9Q8a6T61QMjninpLd0coz/LelGSbtIer2km1QMUPprSW/OOpereDfK7/L3fFxpF9fl79e2d3351KcnT7Um4JlOyn5IMXgmwN9QDK0Bxbsvfkfxzoe9KcYsGtjJ9o8Du+f8aGATsF8uXwN8KOfvIJ/8zv0tz/mPUDxxvDvQQjGS68dy3QUUgxG2b/+dnD+EfJ8K8F+lYwymePp619zvSvIp9A4xv5Xiye5dgd0oRk3YP9ctp8N7dEpxfqu0/E1ges4fBiws/d4+B5xOkWzbn66+FRiX8wdSDNkBcDnFU9c7ULxno7V0jBHAA339vfHU91N3TrXN+tq7gH2K4ZoA2EPF6LIAcyPiBeAFSWsphl5f2WH7oVG8M6PdYxGxMOfvoUg0W3J77uNpSRuBn2X5AxRDY7S7Eop3qkjaI8dhmgi8p3QdYxBFcoR8t0gnxzsY+GlEPAsg6SfA/6IYgqNeBwPvz3huk7RX6brSyRRDmx8bEX/J3+ffAz8q/Z53Lu3ruigGWXxI0rBS+Vrgtd2IybZRTirWTHYADoqI58uF+cfvhVLRi3T+3d4kaYf8o9jZNru012Nz1/CgDvsob/NSafmlDsfsOP5RUIyD9f6IWNoh/gOBZzuJtzc8QHENZiTwGEW7N0QxZHpnyu1XaX4Q8KeGRGhNxddUrJncAnyyfUFSrT98tSyluA6xJcspup1g84iu3XUCgKSDKUbm3UgxCu8nc2RcJO1fx35+DRwr6VUqRqF+b5Z15WmKLrryPk7KYx4KPBmbb1a4D/goMEfSa7P8MUkfyPqStG8dcb6RzSP52nbMScX6q1dJWlmaPgucAUzIC84PAR/r5j7nAofWUe9rwMcl3UdxTaUnns/tL6UYnRjgHIrXui6S9GAudymKV91eTjFy7F3AdyNiS11ft1N0Ey6UdALFtZO3SloEzGDzEOftx/gNxbWVuZL2pkhAp0pqH/m6ntfivpPi92vbOY9SbNsNFe/IuCIijujrWLY1kn4FTI7iffO2HfOZim03ong51Hca/fDj9kZSC3C+E4qBz1TMzKxCPlMxM7PKOKmYmVllnFTMzKwyTipmZlYZJxUzM6vM/weydCzETL4imAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Custom Tokenizer\n",
    "re_tok = re.compile(f'([{string.punctuation}“”¨«»®´·º½¾¿¡§£₤‘’])')\n",
    "def tokenize(s): return re_tok.sub(r' \\1 ', s).split()\n",
    "\n",
    "# Plot sentence by lenght\n",
    "plt.hist([len(tokenize(s)) for s in train['text'].values], bins=50)\n",
    "plt.title('Tokens per sentence')\n",
    "plt.xlabel('Len (number of token)')\n",
    "plt.ylabel('# samples')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Data Preprocessing\n",
    "\n",
    "Before feeding the data into the model, we have to preprocess the text. \n",
    "\n",
    "- We will use the Keras `Tokenizer` to convert each word to a corresponding integer ID. Representing words as integers saves a lot of memory!\n",
    "- In order to feed the text into our CNN, all texts should be the same length. We ensure this using the `sequence.pad_sequences()` method and `MAX_LEN` variable. All texts longer than `MAX_LEN` are truncated and shorter texts are padded to get them to the same length.\n",
    "\n",
    "The *Tokens per sentence* plot (see above) is useful for setting the `MAX_LEN` training hyperparameter. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "First sample before preprocessing: \n",
      " Watch the Original with the same title from 1944! This made for TV movie, is just god-awful! Although it does use (as far as I can tell) almost the same dialog, it just doesn't work! Is it the acting, the poor directing? OK so it's made for TV, but why watch a bad copy, when you can get your hands on the superb original? Especially as you'll be spoiled to the plot and won't enjoy the original as much, as if you've watched it first! <br /><br />There are a few things that are different from the original (it's shorter for once), but all are for the worse! The actors playing the parts here, just don't fit the bill! You just don't believe them and who could top Edward G. Robinsons performance from the original? If you want, only watch it after you've seen the original and even then you'll be very brave, if you watch it through! It's almost sacrilege! \n",
      "\n",
      "First sample after preprocessing: \n",
      " [   5    1  111    2  525  354    1  201   14   73   14   44  871  293\n",
      "    9   83    7    7   47   23    3  168  180   12   23  272   36    1\n",
      "  201   42 5844   15  277   18   29   23   15    1  430    1  153  392\n",
      "    1  528  130   40   89 1180    1  985   22   40   89  261   95    2\n",
      "   34   97  347 2485 1328  236   36    1  201   44   22  178   61  103\n",
      "    9  100  871  107    1  201    2   57   92  487   27   52 2502   44\n",
      "   22  103    9  140   42  217]\n"
     ]
    }
   ],
   "source": [
    "imdb_tokenizer = Tokenizer(num_words=VOCAB_SIZE)\n",
    "imdb_tokenizer.fit_on_texts(train['text'].values)\n",
    "\n",
    "x_train_seq = imdb_tokenizer.texts_to_sequences(train['text'].values)\n",
    "x_val_seq = imdb_tokenizer.texts_to_sequences(val['text'].values)\n",
    "\n",
    "x_train = sequence.pad_sequences(x_train_seq, maxlen=MAX_LEN, padding=\"post\", value=0)\n",
    "x_val = sequence.pad_sequences(x_val_seq, maxlen=MAX_LEN, padding=\"post\", value=0)\n",
    "\n",
    "y_train, y_val = train['label'].values, val['label'].values\n",
    "\n",
    "print('First sample before preprocessing: \\n', train['text'].values[0], '\\n')\n",
    "print('First sample after preprocessing: \\n', x_train[0])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Model\n",
    "\n",
    "We will implement a model similar to Kim Yoon’s [Convolutional Neural Networks for Sentence Classification](https://arxiv.org/abs/1408.5882).\n",
    "\n",
    "![cnn for text](https://github.com/floydhub/sentiment-analysis-template/raw/master/images/cnn.png)\n",
    "*Image from [the paper](https://arxiv.org/abs/1408.5882)*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Model Parameters - You can play with these\n",
    "\n",
    "NUM_FILTERS = 250\n",
    "KERNEL_SIZE = 3\n",
    "HIDDEN_DIMS = 250"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Build model...\n",
      "WARNING:tensorflow:From /usr/local/lib/python3.6/site-packages/tensorflow/python/util/deprecation.py:497: calling conv1d (from tensorflow.python.ops.nn_ops) with data_format=NHWC is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "`NHWC` for data_format is deprecated, use `NWC` instead\n",
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "embedding_1 (Embedding)      (None, 90, 40)            800000    \n",
      "_________________________________________________________________\n",
      "dropout_1 (Dropout)          (None, 90, 40)            0         \n",
      "_________________________________________________________________\n",
      "conv1d_1 (Conv1D)            (None, 88, 250)           30250     \n",
      "_________________________________________________________________\n",
      "global_max_pooling1d_1 (Glob (None, 250)               0         \n",
      "_________________________________________________________________\n",
      "dense_1 (Dense)              (None, 250)               62750     \n",
      "_________________________________________________________________\n",
      "dropout_2 (Dropout)          (None, 250)               0         \n",
      "_________________________________________________________________\n",
      "activation_1 (Activation)    (None, 250)               0         \n",
      "_________________________________________________________________\n",
      "dense_2 (Dense)              (None, 1)                 251       \n",
      "_________________________________________________________________\n",
      "activation_2 (Activation)    (None, 1)                 0         \n",
      "=================================================================\n",
      "Total params: 893,251\n",
      "Trainable params: 893,251\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n"
     ]
    }
   ],
   "source": [
    "# CNN Model\n",
    "print('Build model...')\n",
    "model = Sequential()\n",
    "\n",
    "# we start off with an efficient embedding layer which maps\n",
    "# our vocab indices into EMBEDDING_DIM dimensions\n",
    "model.add(Embedding(VOCAB_SIZE, EMBEDDING_DIM, input_length=MAX_LEN))\n",
    "model.add(Dropout(0.2))\n",
    "\n",
    "# we add a Convolution1D, which will learn NUM_FILTERS filters\n",
    "model.add(Conv1D(NUM_FILTERS,\n",
    "                 KERNEL_SIZE,\n",
    "                 padding='valid',\n",
    "                 activation='relu',\n",
    "                 strides=1))\n",
    "\n",
    "# we use max pooling:\n",
    "model.add(GlobalMaxPooling1D())\n",
    "\n",
    "# We add a vanilla hidden layer:\n",
    "model.add(Dense(HIDDEN_DIMS))\n",
    "model.add(Dropout(0.2))\n",
    "model.add(Activation('relu'))\n",
    "\n",
    "# We project onto a single unit output layer, and squash it with a sigmoid:\n",
    "model.add(Dense(1))\n",
    "model.add(Activation('sigmoid'))\n",
    "model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])\n",
    "model.summary()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Train & Evaluate\n",
    "\n",
    "If you left the default hyperpameters in the Notebook untouched, your training should take approximately: \n",
    "\n",
    "- On CPU machine: 2 minutes for 2 epochs.\n",
    "- On GPU machine: 1 minute for 2 epochs.\n",
    "\n",
    "You should get an accuracy of > 84%. *Note*: The model will start overfitting after 2 to 3 epochs. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train on 22500 samples, validate on 2500 samples\n",
      "Epoch 1/2\n",
      "22500/22500 [==============================] - 56s 2ms/step - loss: 0.4780 - acc: 0.7473 - val_loss: 0.3608 - val_acc: 0.8412\n",
      "Epoch 2/2\n",
      "22500/22500 [==============================] - 53s 2ms/step - loss: 0.2796 - acc: 0.8832 - val_loss: 0.3510 - val_acc: 0.8484\n",
      "25000/25000 [==============================] - 5s 201us/step\n",
      "\n",
      "Accuracy:  85.052\n"
     ]
    }
   ],
   "source": [
    "# fit a model\n",
    "model.fit(x_train, y_train,\n",
    "          batch_size=BATCH_SIZE,\n",
    "          epochs=EPOCHS,\n",
    "          validation_split=0.1,\n",
    "          verbose=2)\n",
    "\n",
    "# Evaluate the model\n",
    "score, acc = model.evaluate(x_val, y_val, batch_size=BATCH_SIZE)\n",
    "print('\\nAccuracy: ', acc*100)\n",
    "\n",
    "pred = model.predict_classes(x_val)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "             precision    recall  f1-score   support\n",
      "\n",
      "   negative       0.81      0.88      0.84     11573\n",
      "   positive       0.89      0.83      0.86     13427\n",
      "\n",
      "avg / total       0.85      0.85      0.85     25000\n",
      "\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAnQAAAHlCAYAAAB8u2G0AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzs3XeYXVX18PHvSkLvNZTQqwjSi6iI9F6lCEpENAqoSFNEBQFBBRTxBVSQjgJBQEBAQDr8qNJ7CUQSQg09oYRZ7x9nD9wMk2RmmHZyvx+e++TeffY5d9+By6ystfc+kZlIkiSpvgb09QAkSZL06RjQSZIk1ZwBnSRJUs0Z0EmSJNWcAZ0kSVLNGdBJkiTVnAGdpF4XETNExGUR8UZEXPAprrNrRFzdnWPrCxFxZUQM7etxSKovAzpJkxQRu0TE3RHxdkSMKYHHF7vh0l8FBgNzZeYOXb1IZv4tMzfqhvFMJCLWjYiMiIvbtK9Y2m/o4HV+GRHnTKlfZm6amWd2cbiSZEAnqX0RsR/wB+AoquBrYeAkYOtuuPwiwBOZOaEbrtVTXgY+HxFzNbQNBZ7orjeIiv8flvSp+T8SSZ8QEbMBhwN7Z+ZFmflOZn6QmZdl5oGlz3QR8YeIeL48/hAR05Vj60bEqIjYPyJeKtm93cuxw4BDgJ1K5m+PtpmsiFi0ZMIGldffjIgREfFWRDwTEbs2tN/ScN7aEXFXKeXeFRFrNxy7ISKOiIhby3Wujoi5J/NjeB/4J7BzOX8gsBPwtzY/q+Mj4rmIeDMi/hsRXyrtmwAHN3zO+xvGcWRE3AqMAxYvbd8ux/8UERc2XP+3EXFtRESH/wVKajoGdJLa83lgeuDiyfT5GbAWsBKwIrAG8POG4/MBswELAnsAJ0bEHJl5KFXW7/zMnDkzT53cQCJiJuCPwKaZOQuwNnBfO/3mBC4vfecCfg9c3ibDtguwOzAvMC1wwOTeGzgL2K083xh4CHi+TZ+7qH4GcwJ/By6IiOkz899tPueKDed8AxgGzAKMbHO9/YEVSrD6Jaqf3dD0Po2SJsOATlJ75gJemUJJdFfg8Mx8KTNfBg6jClRafVCOf5CZVwBvA8t0cTwtwPIRMUNmjsnMh9vpsznwZGaenZkTMvNc4DFgy4Y+p2fmE5k5HhhOFYhNUmb+HzBnRCxDFdid1U6fczLz1fKevwOmY8qf84zMfLic80Gb642j+jn+HjgH+EFmjprC9SQ1OQM6Se15FZi7teQ5CQswcXZpZGn76BptAsJxwMydHUhmvkNV6vweMCYiLo+IZTswntYxLdjw+oUujOds4PvAV2gnYxkRB0TEo6XM+zpVVnJypVyA5yZ3MDPvAEYAQRV4StJkGdBJas9twHvANpPp8zzV4oZWC/PJcmRHvQPM2PB6vsaDmXlVZm4IzE+VdTulA+NpHdPoLo6p1dnAXsAVJXv2kVIS/TGwIzBHZs4OvEEViAFMqkw62fJpROxNlel7vlxfkibLgE7SJ2TmG1QLF06MiG0iYsaImCYiNo2Io0u3c4GfR8Q8ZXHBIVQlwq64D1gnIhYuCzJ+2nogIgZHxNZlLt17VKXblnaucQWwdNlqZVBE7AQsB/yri2MCIDOfAb5MNWewrVmACVQrYgdFxCHArA3HXwQW7cxK1ohYGvgV8HWq0uuPI2KypWFJMqCT1K4yH2w/qoUOL1OVCb9PtfITqqDjbuAB4EHgntLWlfe6Bji/XOu/TByEDSjjeB4YSxVc7dnONV4FtqBaVPAqVWZri8x8pStjanPtWzKzvezjVcC/qbYyGQm8y8Tl1NZNk1+NiHum9D6lxH0O8NvMvD8zn6RaKXt26wpiSWpPuHBKkiSp3szQSZIk1ZwBnSRJUs0Z0EmSJNWcAZ0kSVLNGdBJkiTV3OR2gVc3evf2811OLPWymdfZr6+HIDWtCe+Pjin36j4fvDKi237PTjP34r069u5ghk6SJKnmzNBJkqT6a/mwr0fQpwzoJElS/WV7dwRsHpZcJUmSas4MnSRJqr+W5s7QGdBJkqTaS0uukiRJqjMzdJIkqf4suUqSJNWcJVdJkiTVmRk6SZJUf24sLEmSVHOWXCVJklRnZugkSVL9ucpVkiSp3txYWJIkSbVmhk6SJNWfJVdJkqSas+QqSZKkOjNDJ0mS6s+NhSVJkmrOkqskSZLqzAydJEmqP1e5SpIk1ZwlV0mSJNWZGTpJklR/llwlSZLqLbO5ty2x5CpJklRzZugkSVL9NfmiCAM6SZJUf00+h86SqyRJUs2ZoZMkSfVnyVWSJKnmWlzlKkmSpBozQydJkurPkqskSVLNucpVkiRJdWaGTpIk1V+Tl1zN0EmSpPpraem+xxRExGkR8VJEPNTQNmdEXBMRT5Y/5yjtERF/jIinIuKBiFil4Zyhpf+TETG0oX3ViHiwnPPHiIgpjcmATpIkqXPOADZp03YQcG1mLgVcW14DbAosVR7DgD9BFQAChwJrAmsAh7YGgaXPdxrOa/ten2BAJ0mS6q8XM3SZeRMwtk3z1sCZ5fmZwDYN7Wdl5XZg9oiYH9gYuCYzx2bma8A1wCbl2KyZeXtmJnBWw7UmyTl0kiSp9jL7fGPhwZk5pjx/ARhcni8IPNfQb1Rpm1z7qHbaJ8sMnSRJUoOIGBYRdzc8hnXm/JJZyx4aXrvM0EmSpPrrxn3oMvNk4OROnvZiRMyfmWNK2fSl0j4aWKih35DSNhpYt037DaV9SDv9J8sMnSRJqr9s6b5H11wKtK5UHQpc0tC+W1ntuhbwRinNXgVsFBFzlMUQGwFXlWNvRsRaZXXrbg3XmiQzdJIkSZ0QEedSZdfmjohRVKtVfwMMj4g9gJHAjqX7FcBmwFPAOGB3gMwcGxFHAHeVfodnZutCi72oVtLOAFxZHpNlQCdJkuqvF2/9lZlfm8Sh9dvpm8Dek7jOacBp7bTfDSzfmTEZ0EmSpPrzThGSJEmqMzN0kiSp/nqx5NofGdBJkqT6s+QqSZKkOjNDJ0mS6s+SqyRJUs0Z0EmSJNWcc+gkSZJUZ2boJElS/VlylSRJqjlLrpIkSaozM3SSJKn+LLlKkiTVnCVXSZIk1ZkZOkmSVH+WXCVJkmquyQM6S66SJEk1Z4ZOkiTVX2Zfj6BPGdBJkqT6s+QqSZKkOjNDJ0mS6q/JM3QGdJIkqf7cWFiSJEl1ZoZOkiTVnyVXSZKkmmvybUssuUqSJNWcGTpJklR/llwlSZJqrskDOkuukiRJNWeGTpIk1V+T70NnQCdJkmovW1zlKkmSpBozQydJkuqvyRdFGNBJkqT6a/I5dJZcJUmSas4MnSRJqr8mXxRhQCdJkuqvyefQWXKVJEmqOTN0kiSp/po8Q2dAJ0mS6i+bew6dJVdJkqSaa8oMXUQsCjwDrJ6Zd/ftaNQbDvnrxdx03xPMOetMXHTU9wF44+1x/Pik4Tz/yussMPfsHLP3Tsw60ww88/zLHPLXi3l05Bh+sP36DN3six9d5813xnPYaZfw1OiXCOCwb2/DiksuzGMjx/CrMy/j/Q8mMHDAAA7ebQtWWGJIH31aqf865eTfsflmG/DSy6+w0srrT3Rs3x99l2OOPoTB8y/Pq6++BsBxvz+cTTdZj3Hjx7PHHvty730PAfDe+P/x4EOPAfDcc6PZdrvde/eDqP9p8pLrVJ+hi4gbIuKENs3PAfMD9/XBkNQHtv7iyvzpgG9M1Hba5TezxnKLc9nRP2KN5Rbn1H/dDMCsM8/AT76+OUM3/cInrnP0367kCyssxSW/+SEX/GovFpt/HgCOO/9qvrf1ugw/Yi/22m49/jD86p7/UFINnXXWcDbfYtdPtA8ZsgAbbrAOI0eO+qht003WY6klF2PZ5b7Innv+hBNP+PVHx8aPf5fVVt+I1VbfyGBOlZbsvkcNTfUBXXsy88PMfCEzJ/T1WNQ7Vl12UWadaYaJ2q6/5zG2+uLKAGz1xZW5/p5HAZhr1plZfvEFGTRw4q/HW+Pe5b+PP8u2X14FgGkGDfromhHw9rvvAfD2uHeZZ/ZZevTzSHV18y13MPa11z/R/rtjf8lBBx9JNsyD2nLLjTn7b/8A4I4772G22Wdjvvnm7bWxSnXSYwFdyYydFBFHRcQrEfFSRBwbEQPK8Wkj4rcRMSoixkXEXRGxcZtrbB4Rj0fEuxFxU0TsHBFZSqZExFwRcW65xviIeDgidm84/wzgy8De5byMiEXLIyNitYgYEBHPRcQP2rz30qXPKuX1bBFxcvkcb0XEjRGxWk/9/NTzxr75zkeB19yzzczYN9+ZbP/RL7/GHLPMxCF/vZgdf3ESvzz1n4x7730AfrzrZhx33tVstO+x/O68q/jhDhv2+PilqcWWW27E6NFjeOCBRyZqX3CB+Rj13PMfvR49agwLLjAfANNPPx2333YFt958GVttNdGvDjWrbOm+Rw31dIZuV2ACsDbwfeBHwE7l2OlUwdYuwPLAmcBlEbEiQEQsDFwEXA6sCPwROLrN9acH7gG2AD4LHA/8JSJaJ2bsA9xW3mv+8niu8QKZ2QKcW8baduyPZuY9ERFlHAuW91oZuAm4LiLm7+wPRf1P9a948j5saeGxkWPYYb3VGX7EXsww3bScVsq0w6+7kwN32YSrjzuAA3fZlF+e+s+eHrI0VZhhhun56U9+wC8PO7ZT5y2+5Jqs9fnN+Ppue/P7Yw9j8cUX6aERqjYsufaoRzLzkMx8IjOHA9cD60fEEsDXgB0z86bMHJGZJwBXAN8t5+4JjMjM/TLz8cz8B/Dnxotn5ujMPCYz7yvXOJkqCPxaOf4G8D4wrpRYX8jMD9sZ5znAmmVcrXYp7QBfAVYCvpqZd2bmU5n5C2AEMPHErAYRMSwi7o6Iu0/95386/lNTr5hz1pl4+fW3AHj59beYc9aZJtt/8ByzMnjOWfncEgsBsOHqy/HYyCp7cNkt97H+assBsNEan+WhEaN7cOTS1GOJJRZl0UUX5p67r+GpJ25nyJD5ueuOqxg8eB5GP/8CQxZa4KO+Cw6Zn9HPvwDA8+XPZ575HzfedBsrrbR8n4xf6i96OqB7oM3r54F5gVWAAB6JiLdbH8DmQGtQtSxwV5vz72h8EREDI+JnEfFARLxarrEdsHBnBpmZDwAPUrJ0EbFmGcffSpdVgRmBl9uMd/mG8bZ33ZMzc7XMXG2PbTbozJDUC9ZdeVkuveVeAC695V6+ssqyk+0/9+yzMHjOWXl2zCsA3PHICBZfoJrPM8/ss3D3Y88CcOcjI1h48Jw9N3BpKvLQQ4+xwJAVWXLptVhy6bUYNWoMq6+5MS+++DL/+tfVfGPXrwKw5hqr8OYbb/LCCy8x++yzMe200wIw11xzsPbnV+fRR5/oy4+hfiBbWrrtUUc9vW3JB21eJ1UQOaA8X72dPuM7cf0DgP2pSqsPAm8DR1EFjZ11DrAHcDhVYHdLZo4sxwYALwJfaue8N7vwXuplPznpAu5+7Blef3scG/7oWPbc9it8a4svceCJ5/PPm+5h/rlm55i9dwTgldff4mu//AvvjH+PAQOCc66+nYt//X1mnmF6Dvr65vz0z//ggwkfMmTeOTj829sCcMi3tuboc67gw5YWpp1mEIfsvnVfflyp3zrn7BP58jqfZ+655+TZEXdz2OHHcvoZ57Xb94orr2WTTdbj8UdvZdz48Xz72/sB8Jlll+Kkk35DS0syYEBw9DEn8OijT/bmx1B/VNNSaXeJ7KGdlSPiBuChzPx+Q9sZwNzAfsDjwHqZef0kzv81sHVmLtfQ9lOqgG2xzHw2Ii4Dxmbm0HI8gHuB1zNz3dJ2NfB0Zu7ZcJ1FabMPXUQMAUYCXwAuAX5RSrhExIbAVcCSmTmiKz+Pd28/v7n/S5P6wMzr7NfXQ5Ca1oT3R095cnI3eufI3brt9+xMPzurV8feHfpk25LMfIKqnHlGRHw1IhYvK04PiIjtSrc/A0uUlbHLlPbW+XWt/9KeoJqT98WIWBY4AViszds9C6xRVrbO3brKtp0xjQJuLO87G3BBw+H/ALcCl0TEphGxWER8PiIOi4j2snaSJKk3ucq1z+xOtfr0aOAx4F/AOlRZMkq5c3tgK+B+YF/gsHLuu+XPXwF3AldSrTp9h4/nvbU6lmphxCPAy0x+ft05VCtqr8jM11obs0pjbgZcB5xClV0cDixDNS9QkiT1pSZf5dpjJdeeEBH7UM1xmz3rNHAsuUp9wZKr1Hd6veR6+K7dV3I95G+1K7n263u5RsTeVCtdXwbWAn4BnFG3YE6SJPWwmq5O7S79OqADlgQOBuYCRlHNbzu8T0ckSZL6n5qWSrtLvw7oMnNfqrlzkiRJmoR+HdBJkiR1SE1Xp3YXAzpJklR/TV5y7cttSyRJktQNzNBJkqTaq+s9WLuLAZ0kSaq/Ji+5GtBJkqT6a/KAzjl0kiRJNWeGTpIk1Z/blkiSJNWcJVdJkiTVmRk6SZJUe9nkGToDOkmSVH9NHtBZcpUkSao5AzpJklR/LS3d9+iAiNg3Ih6OiIci4tyImD4iFouIOyLiqYg4PyKmLX2nK6+fKscXbbjOT0v74xGxcVc/vgGdJEmqv5bsvscURMSCwA+B1TJzeWAgsDPwW+C4zFwSeA3Yo5yyB/BaaT+u9CMilivnfRbYBDgpIgZ25eMb0EmSJHXeIGCGiBgEzAiMAdYD/lGOnwlsU55vXV5Tjq8fEVHaz8vM9zLzGeApYI2uDMaATpIk1V8vZugyczRwLPA/qkDuDeC/wOuZOaF0GwUsWJ4vCDxXzp1Q+s/V2N7OOZ1iQCdJkmovM7vtERHDIuLuhsewxveKiDmosmuLAQsAM1GVTPuM25ZIkiQ1yMyTgZMn02UD4JnMfBkgIi4CvgDMHhGDShZuCDC69B8NLASMKiXa2YBXG9pbNZ7TKWboJElS/fViyZWq1LpWRMxY5sKtDzwCXA98tfQZClxSnl9aXlOOX5eZWdp3LqtgFwOWAu7sysc3QydJkuqvFzcWzsw7IuIfwD3ABOBeqoze5cB5EfGr0nZqOeVU4OyIeAoYS7Wylcx8OCKGUwWDE4C9M/PDrozJgE6SJKmTMvNQ4NA2zSNoZ5VqZr4L7DCJ6xwJHPlpx2NAJ0mSas97uUqSJNVdkwd0LoqQJEmqOTN0kiSp/jp2C9aplgGdJEmqvWafQ2fJVZIkqebM0EmSpPpr8gydAZ0kSaq/Jp9DZ8lVkiSp5szQSZKk2mv2RREGdJIkqf4suUqSJKnOzNBJkqTas+QqSZJUd5ZcJUmSVGdm6CRJUu1lk2foDOgkSVL9NXlAZ8lVkiSp5szQSZKk2rPkKkmSVHdNHtBZcpUkSao5M3SSJKn2LLlKkiTVXLMHdJZcJUmSas4MnSRJqr1mz9AZ0EmSpPrL6OsR9ClLrpIkSTVnhk6SJNWeJVdJkqSayxZLrpIkSaoxM3SSJKn2LLlKkiTVXLrKVZIkSXVmhk6SJNWeJVdJkqSaa/ZVrgZ0kiSp9jL7egR9yzl0kiRJNWeGTpIk1Z4lV0mSpJozoJuEiJh1cidm5pvdPxxJkiR11uQydA8DCTSGvK2vE1i4B8clSZLUYc2+KGKSAV1mLtSbA5EkSeqqZi+5dmiVa0TsHBEHl+dDImLVnh2WJEmSOmqKAV1EnAB8BfhGaRoH/LknByVJktQZmdFtjzrqyCrXtTNzlYi4FyAzx0bEtD08LkmSpA5r9lt/daTk+kFEDKBaCEFEzAU0+Y9NkiSp/+hIhu5E4EJgnog4DNgROKxHRyVJktQJLTUtlXaXKQZ0mXlWRPwX2KA07ZCZD/XssCRJkjqurnPfuktH7xQxEPiAquzq/V8lSZL6kY6scv0ZcC6wADAE+HtE/LSnByZJktRR2RLd9qijjmTodgNWzsxxABFxJHAv8OueHJgkSVJHNfudIjpSPh3DxIHfoNImSZKkfmCSGbqIOI5qztxY4OGIuKq83gi4q3eGJ0mSNGV1LZV2l8mVXFtXsj4MXN7QfnvPDUeSJKnz3LZkEjLz1N4ciCRJkrpmiosiImIJ4EhgOWD61vbMXLoHxyVJktRhzb4PXUcWRZwBnA4EsCkwHDi/B8ckSZLUKZnd96ijjgR0M2bmVQCZ+XRm/pwqsJMkSVI/0JF96N6LiAHA0xHxPWA0MEvPDkuSJKnjXBQxZfsCMwE/pJpLNxvwrZ4clCRJUmc0+xy6KQZ0mXlHefoW8I2eHY4kSZI6a3IbC19MtZFwuzJzux4ZkSRJUifVdTFDd5lchu6EXhuFJEnSp+AcuknIzGt7cyBTuxW2PLqvhyA1nfHP39zXQ5CkXtGRRRGSJEn9mosiJEmSaq7ZS64d2VgYgIiYricHIkmSpK6ZYkAXEWtExIPAk+X1ihHx/3p8ZJIkSR2U3fioo46UXP8IbAH8EyAz74+Ir/ToqCRJkjrBkmsH+mTmyDZtH/bEYCRJkuogImaPiH9ExGMR8WhEfD4i5oyIayLiyfLnHKVvRMQfI+KpiHggIlZpuM7Q0v/JiBja1fF0JKB7LiLWADIiBkbEj4AnuvqGkiRJ3S0zuu3RQccD/87MZYEVgUeBg4BrM3Mp4NryGmBTYKnyGAb8CSAi5gQOBdYE1gAObQ0CO6sjAd2ewH7AwsCLwFqlTZIkqV9o6cbHlETEbMA6wKkAmfl+Zr4ObA2cWbqdCWxTnm8NnJWV24HZI2J+YGPgmswcm5mvAdcAm3Tl83fkXq4vATt35eKSJElTocWAl4HTI2JF4L/APsDgzBxT+rwADC7PFwSeazh/VGmbVHunTTGgi4hTaGfRR2YO68obSpIkdbek+xZFRMQwqtJoq5Mz8+SG14OAVYAfZOYdEXE8H5dXq/FkZkT02qLZjqxy/U/D8+mBbZk4mpQkSepTLd0YOpXg7eTJdBkFjMrMO8rrf1AFdC9GxPyZOaaUVF8qx0cDCzWcP6S0jQbWbdN+Q1fGPMU5dJl5fsPjTGA7YNWuvJkkSVLdZeYLVItGlylN6wOPAJcCrStVhwKXlOeXAruV1a5rAW+U0uxVwEYRMUdZDLFRaeu0rtz6azE+rglLkiT1uZZuLLl20A+Av0XEtMAIYHeqRNnwiNgDGAnsWPpeAWwGPAWMK33JzLERcQRwV+l3eGaO7cpgOjKH7jU+nkM3ABhLmzqxJElSX+rOOXQder/M+4DV2jm0fjt9E9h7Etc5DTjt045nsgFdRATV3iqjS1NLGZQkSZL6ickGdGWFxhWZuXxvDUiSJKmzOrJ/3NSsIxsL3xcRK/f4SCRJkrooiW571NEkM3QRMSgzJwArA3dFxNPAO0BQJe9WmdS5kiRJ6j2TK7neSbVp3la9NBZJkqQuafaS6+QCugDIzKd7aSySJEldYkA3afNExH6TOpiZv++B8UiSJKmTJhfQDQRmhprODpQkSU2jrosZusvkAroxmXl4r41EkiSpi1qaO56b7LYlTf6jkSRJqofJZeg+cesKSZKk/qgP7uXar0wyoOvqzWElSZJ6W7Pfl3Syt/6SJEmqg2bftqQjt/6SJElSP2aGTpIk1V5LOIdOkiSp1pp9Dp0lV0mSpJozQydJkmqv2RdFGNBJkqTa804RkiRJqjUzdJIkqfa8U4QkSVLNucpVkiRJtWaGTpIk1V6zL4owoJMkSbXX7NuWWHKVJEmqOTN0kiSp9pp9UYQBnSRJqr1mn0NnyVWSJKnmzNBJkqTaa/ZFEQZ0kiSp9po9oLPkKkmSVHNm6CRJUu1lky+KMKCTJEm1Z8lVkiRJtWaGTpIk1V6zZ+gM6CRJUu01+50iLLlKkiTVnBk6SZJUe81+6y8DOkmSVHvNPofOkqskSVLNmaGTJEm11+wZOgM6SZJUe65ylSRJUq2ZoZMkSbXnKldJkqSaa/Y5dJZcJUmSas4MnSRJqr1mXxRhQCdJkmqvpclDOkuukiRJNWeGTpIk1V6zL4owoJMkSbXX3AVXS66SJEm1Z4ZOkiTVniVXSZKkmmv2O0VYcpUkSao5M3SSJKn2mn0fOgM6SZJUe80dzllylSRJqj0zdJIkqfZc5SpJklRzzT6HzpKrJElSzZmhkyRJtdfc+TkDOkmSNBVo9jl0llwlSZJqzgydJEmqvWZfFGFAJ0mSaq+5wzkDOkmSNBVwDp0kSZJqzQydJEmqvWzyoqsZOkmSVHst3fjoiIgYGBH3RsS/yuvFIuKOiHgqIs6PiGlL+3Tl9VPl+KIN1/hpaX88Ijb+NJ/fgE6SJKnz9gEebXj9W+C4zFwSeA3Yo7TvAbxW2o8r/YiI5YCdgc8CmwAnRcTArg7GgE6SJNVeC9ltjymJiCHA5sBfy+sA1gP+UbqcCWxTnm9dXlOOr1/6bw2cl5nvZeYzwFPAGl39/AZ0kiSp9rIbHx3wB+DHfFyhnQt4PTMnlNejgAXL8wWB5wDK8TdK/4/a2zmn0wzoJEmSGkTEsIi4u+ExrOHYFsBLmfnfPhziJ0yVq1wj4lnghMw8djJ9vln6zNxb45IkST2jO+8UkZknAydP4vAXgK0iYjNgemBW4Hhg9ogYVLJwQ4DRpf9oYCFgVEQMAmYDXm1ob9V4TqdNrRm61YGTWl9EREbEV9v0OR9YvFdHpX5hvgUGc/bFf+HKWy7gipuHM3TY1wD40UF7ctkN53Hp9X/n9OEnMu/guQFYf5Mvf9R+0TVns+qaKwHwmeWXZvgVp3PFzcO57Ibz2GybDfvsM0n92c+P+j3rbL4z23z9ex+1XXXdzWy963dZ4Yub8dCjT0zU/5SzzmfTHb/FFjt/m1vv+DgJcvbwf7LN17/H1rt+l7PPv/ij9hNPPYf1tv462w/dm+2H7s1N/3dnz38o9Tu9tco1M3+amUMyc1GqRQ3XZeauwPVAa6wxFLikPL+0vKYcvy4zs7TvXFbBLgYsBXT5P96pMkOXmS93oM94YHwvDEf9zIcffsivDz2ORx54jJlmmpGLrz2HW2+4nb+ecBZ/+M2fANjtOzvz/QO+wyEH/prbbr6Ta/99IwDLLLckx//1t2yy9vaMH/cuB37/EEaOeI55B8/Nxdf+jZuvu4233ny7Lz+e1O9ss9mG7LL9Vhx8xMdFkyUXX4Q/HPULDjvmjxP1ffqZkVzrbkVlAAAbdUlEQVR57Y1ccs6feemVsXx7n59y+Xl/ZcTI57jw0n9z7l//wDSDpuF7+/+cL39hTRYesgAA39hpG3bfpe3f26Ve9RPgvIj4FXAvcGppPxU4OyKeAsZSBYFk5sMRMRx4BJgA7J2ZH3b1zfskQxcRN0TEnyPi+Ih4rTyOiYgB5fgcEXFmaR8fEf+JiM82nD9bRJwdES9FxLsRMSIiftRw/NmIOKD1eWm+oGTqni3t34yIt8vzpcuxFdqMc1hEvBIR05TXy0XE5RHxVnnvcyNivh78UakHvPziKzzywGMAvPPOOJ5+4hkGzz8vb7/9zkd9ZphxBrJk78e9M36i9tYDz474HyNHVPNZX3rxFV59eSxzzj1HL30KqT5WW2kFZpt1lonallh0YRZbZMgn+l538+1suv6XmXbaaRmywHwsPGQBHnz0CUY8+xwrfHYZZph+egYNGshqK63Af268tbc+gmogu/GfDr9n5g2ZuUV5PiIz18jMJTNzh8x8r7S/W14vWY6PaDj/yMxcIjOXycwrP83n78uS667l/T8PfBcYBrQGZWcAa1It6V0DGAf8OyJmKMd/BawAbAEsA3yLSdedVy9/fgeYv+H1RzLzCeCuMqa2YxyemR9ExPzATcBDZUwbADMDl7QGoqqfBRean+VWWJb7//sQAPsevBc33Xc5W22/Ccf/9k8f9dtws6/w7/+7kFP+fjwH7XPYJ67zuZU/y7TTTsP/nhnVa2OXpkYvvfwq8w2e56PXg+edm5defoUlF1+Ee+5/mNffeJPx777LzbfdxQsvflyMOffCy9h2tz35+VG/54033+qLoauP9fbGwv1NXwYiY4AfZuZjmTkcOAbYLyKWArYChmXmTZn5IPANqkmHrQHXIsA9mXlnZo4sEfIF7b1JQ/n19cx8YTLl2HOAr5W9YYiIhYEvlXaAPYH7M/MnmfloZj4A7EYV3K3W3gUbV8m88e4rHf25qJfMONMMnHD6MRz582M/ys4dd9RJrLPS5lx64b/5+h47fdT3miuuZ5O1t2evofvzo4P2nOg68wyem2NOOpyDfvhLMpv71jNST1li0YX51q47MGzfn/G9/X7BMkstzoAB1a+wnbbdnCuHn8aFZ5zIPHPNyTEnnNLHo5V6X18GdLfnxL/9bqPaf+UzVAHyba0HMvMN4EFgudL0J2CniLg/Io6NiC93w3jOAxagCuIAvgY8k5n/V16vCqwTEW+3Pvh4/5gl2rtgZp6cmatl5mqzTT93NwxR3WXQoEGccPoxXPqPK7n68us/cfzSf1zJxlus94n2u267l4UXWZA55pwdgJlnnolT/n48xx11EveVLJ+krpt3nrkmyry9+NIrzDtP9f/P7bfcmOGn/T/OPOkYZp1lFhZduCrZzj3nHAwcOJABAwbw1a025aFHnmj32pq69UXJtT+pW6kwAUqdeRHgWGBu4PKIOP1TXTjzJeAaPs4C7gr8raHLAOByYKU2j6WAf32a91bvO+oPv+DpJ57h9D9//K94kcU/Xj2+waZfZsRTzwKw8GIfz/NZ7nPLMs100/La2NeZZppBnHjmsfxz+L/492XX9trYpanZV764FldeeyPvv/8+o55/gf+Nep4VPrM0AK++9joAY154iWtvvJXNNlwXgJdfGfvR+dfe+H8sufgivT5u9b1mL7n25SrXNSMiGrJ0awHPU90XrXVu3U0AETEr1Zy5j4K2zHwFOJtq5ciVwLkR8b3WSYhtfAB05P5o5wAnRMTJ5f0al0zdA+wIjMzMDzr+MdXfrLrmSmy70xY89vCTXHr93wH43ZEnssOuW7PYEovQ0pI8P2oMhxxwFACbbLE+2+y4ORMmTODd8e/xo+/8FIBNt96Q1T+/CnPMORvb7bwlAD/5wS959CGzA1KjAw/9DXfd+wCvv/4m62/zdfba4xvMNuvM/Pq4PzH29TfY68BDWXapxTn5uCNZcvFF2Hi9L7HVrt9l0MCB/Gy/vRg4sPrf974H/4rX33yTQYMG8bP992LWWaptRH930qk8/uQICFhwvsEc+uMf9uXHlfpE9MWcn4i4gaqEeRrVfnErUN0P7VeZeWxE/JNqscMw4HXgSGBlYOnMHB8Rh1MFWA9TBaW/BFYtN779xMbCEfEEcANwCPBeZr7W3sbCETEj8CLwNPB+Zq7RcGwB4D7gFqob675MtY/djsD+mTnZWbhLzbNqPXO4Uo098mi7U2sl9YJp5l48evP9vrHIdt32e/bskRf16ti7Q1+WXP9GlTW7AziFap+W48qx3ak217u0/DkjsEnZOw7gPaog737gVmAWYMvJvNf+wFeo5rzdO6lOmTkOuBhYkY8XQ7Qee55qd+gW4N9UweSJZSztZQUlSVIv6eV7ufY7fZmheygzv9/rb95HzNBJvc8MndR3ejtD9/VuzNCdU8MM3VR5pwhJktRcuvNernVkQCdJkmqvrtuNdJc+Cegyc92+eF9JkqSpkRk6SZJUe3XdP667GNBJkqTaa/Y5dHW7U4QkSZLaMEMnSZJqz0URkiRJNdfsc+gsuUqSJNWcGTpJklR7fXHnq/7EgE6SJNWeq1wlSZJUa2boJElS7TX7oggDOkmSVHvNvm2JJVdJkqSaM0MnSZJqr9kXRRjQSZKk2mv2bUssuUqSJNWcGTpJklR7rnKVJEmqOVe5SpIkqdbM0EmSpNpzlaskSVLNucpVkiRJtWaGTpIk1Z4lV0mSpJpzlaskSZJqzQydJEmqvZYmXxRhQCdJkmqvucM5S66SJEm1Z4ZOkiTVnqtcJUmSaq7ZAzpLrpIkSTVnhk6SJNVes9/6y4BOkiTVXrOXXA3oJElS7XmnCEmSJNWaGTpJklR7zqGTJEmquWafQ2fJVZIkqebM0EmSpNqz5CpJklRzllwlSZJUa2boJElS7TX7PnQGdJIkqfZamnwOnSVXSZKkmjNDJ0mSas+SqyRJUs1ZcpUkSVKtmaGTJEm1Z8lVkiSp5iy5SpIkqdbM0EmSpNqz5CpJklRzllwlSZJUa2boJElS7VlylSRJqrnMlr4eQp+y5CpJklRzZugkSVLttVhylSRJqrd0laskSZLqzIBOkiTVXgvZbY8piYiFIuL6iHgkIh6OiH1K+5wRcU1EPFn+nKO0R0T8MSKeiogHImKVhmsNLf2fjIihXf38BnSSJKn2MrPbHh0wAdg/M5cD1gL2jojlgIOAazNzKeDa8hpgU2Cp8hgG/AmqABA4FFgTWAM4tDUI7CwDOkmSpE7IzDGZeU95/hbwKLAgsDVwZul2JrBNeb41cFZWbgdmj4j5gY2BazJzbGa+BlwDbNKVMbkoQpIk1V5f3forIhYFVgbuAAZn5phy6AVgcHm+IPBcw2mjStuk2jvNDJ0kSaq97MZ/ImJYRNzd8BjW3ntGxMzAhcCPMvPNicZT1W57Lco0QydJktQgM08GTp5cn4iYhiqY+1tmXlSaX4yI+TNzTCmpvlTaRwMLNZw+pLSNBtZt035DV8Zshk6SJNVeby6KiIgATgUezczfNxy6FGhdqToUuKShfbey2nUt4I1Smr0K2Cgi5iiLITYqbZ1mhk6SJNVeL98p4gvAN4AHI+K+0nYw8BtgeETsAYwEdizHrgA2A54CxgG7A2Tm2Ig4Arir9Ds8M8d2ZUAGdJIkSZ2QmbcAMYnD67fTP4G9J3Gt04DTPu2YDOgkSVLtNfutvwzoJElS7fXVtiX9hYsiJEmSas4MnSRJqj1LrpIkSTXXy6tc+x1LrpIkSTVnhk6SJNWeJVdJkqSac5WrJEmSas0MnSRJqr1s8kURBnSSJKn2LLlKkiSp1szQSZKk2nOVqyRJUs01+xw6S66SJEk1Z4ZOkiTVniVXSZKkmmv2gM6SqyRJUs2ZoZMkSbXX3Pk5iGZPUUpTEhHDMvPkvh6H1Gz87kkdZ8lVmrJhfT0AqUn53ZM6yIBOkiSp5gzoJEmSas6ATpoy5/BIfcPvntRBLoqQJEmqOTN0kiRJNWdAJ0mSVHMGdJIkSTVnQCdJklRzBnSSJEk1Z0CnphIRk/xvPiKiN8ciNZv2vn+T+05K6rhBfT0AqbdExIDMbCnPvwYsAMwNDAceycz3+nJ80tSszfdvSSCANzLzpb4dmTR18G9GahoNv0yOBn4HfBZYFzgX2MdMgdQzIiIavn+/BC4G/gU8HBE/iojBfTk+aWrgLzA1lYjYEfgasFlmfgs4ClgaeLL1F46k7pVlB/uI+BmwF7Av8BngOuAQqky5pE/BgE7NZiHglsy8r5RdzwH2zsyLI2LGiFimj8cnTZUiYkbgS8B+mfkfYCtgI+BnmflwRAws/ZzLKnWBAZ2aQkRMW54uBYyPiFWp7hN5UGb+qRz7GrBj+cUj6VNoZwrDnMDKwB0R8WXgbMr3LyJmAH4eEYun96OUusSATlO9UuY5sLz8O7ADcBfw3dZgrvxC2R6YLzPH9clApalImwVIZOYo4Gbg/wFXAD/MzL+U7rMB61Nl8CR1gQGdmsEg4DsRsShwN3Ai8CIwf0TMGxFrABdRrXrdByz7SN0hIhYHToiIvUvT/wErAtdl5umlz6zAaUAL1RQISV0QZrc1tSsB28nAbzLzvIj4DFWW7gDgfeB5YAywRWZ+EBEDM/PDvhuxNHWIiFmAM4D3MnOXMvXhWKrV5R8CT1HNa50eWN3vn9R1BnSaKkXEoMyc0PD6LGANYLnMbCkTsAdTrXB9CXistE90nqQpK/Plor1ALCK+CFwP7JSZF0XEIGAzYD1gIDAC+H+ZOcHvn9R1BnSqvYj4NfDrzHyzvP4+MBNwcWY+UdoWo9r36qTMPLG9LEDjxqeSOiYi5m3cHDgiVgceyszx5fU0wCnAdMBemfnaJK5jZk76FJxDp1or5dO1gcaFDGsAWwK3R8R+EbFGZj4D3Es18Zr2fnEYzEmdExEnUJVUiYgBEbEWcAdwbkQcBJCZH1D9ZWojqnmqlCzdRAzmpE/HDJ2mGhGxE3BpZo6PiPmotiH5FtUthi4F/gtcAHw1My/qu5FKU4eI+CzweCmXzpKZb0XEOsDWwHbAW8CfqFaXnwjMAmxn8CZ1PwM61Vabe0POD4wGrgR2zczXS/vSwArA0cBYYFXgD5m5X9+MWqq/ciuvbHj9TeCPwOcy89mImAmYBvgtsBzVXNUngMWpFh/d2/ujlqZuBnSqpYiYOzNfKc/Xyszby2rWS4HbgO+0Hi99pgN2AZal2pneiddSF7Wd7xYRK1Jl4OYD1svM/zUcW5QqY/dj4FngS05vkLqfAZ1qJyI2BL4L/AT4IbA7sHhmvlImZF8J3EQV1L3a3mIHV9NJXVO+fy+X2+cdD7yfmQeW797RwKLAOpn5XGMmr+xJ92zrKnPLrlL3MqBT7UTEDsDBVKvm5gXWzswnWn9JNAR1N1IFdWP7cLjSVKPcUeVpqo25HwY2B76cmQ+U461B3SKl/bmImDYz32+4hqvJpR7gKlfVTmZeANxDVT69i2quDiWYG5CZdwGbAF8ALi470UvqoojYrUxzGJ+ZC1AFbDsA32sN5gDKd6+1tHpdRCzaGMyVPgZzUg8woFOtNGx38F+q23TNQXVT7zVKewJk5t1U92Z9B3i7t8cpTS0iYhiwKzA2IgZFxBDgDeBVYN+IWLWxfwnqfgJ8ABzT2+OVmpUlV/V7bVazTlSuiYidgf2pbiH0uxLIEREbZebV7V1DUuc0TGf4InB72aZkGqqVqy8DewL3tvluLgyMdq6c1DsM6NSvtQnmhgIrUd0D8qbMvLS07wTsR7VtyXBgN6ptEpZK/wOXuqwhkAuq+69eCfwKOCsz/xcRc1BNf3gB2Jdq8+4Lgbsy87DGa/TJB5CaiAGdaiEifgt8k2pbksWA2ag2ET6iHN8eGEY1t2cMsFG50XcY1EmdN4nV4b8BdqK6lddZmTmqBHW3U92X9QOqv3CtXO4QIamXfOL2K1J/ExHfpZqAvUVm3hURuwCnA7NGxAyZeXBmXhgRd1L9Nz2ybI3g1iRSF7TJjG8GjMvMGzLzoIj4APheOdYa1K1OtX3Qh8CfS0nW75/Uiwzo1K+VeTrzAaeUYG4bqg1MDwGWAL4TEe9m5uGZ+VzDeQP8ZSJ1XslqtwZzvwW2AY6NiEcz88XM/EVEDKCaN/dRUAcc33CNgX7/pN5lyVX9Snsl0ogYTPWXj2mBy4G/ZubvI2IV4BqqrMBhmXlirw9YmkpFxM+BH1CtFr+t7Ty4iDiK6n7J5wHHZuarvT9KSa3M0KnfaLOr/I+BBzLz38BLmZklO9cCnFtOmZlq8+ArgNP6YszS1Cgi5gO2AvbNzFsiYsFyX+QdgbGZ+bPMPDgiZqHaD9LNu6U+ZkCnfqHNnJ2Fqe79+JOI2DYzbyrdWoDpgY0j4grgQGAkcGoJ+FxNJ3WPccB7wGfKHLrdgIVL+1oRMUdm7pWZP2j9i5gLkKS+5cbC6hcagrlfA+dQbQY8ELgsIjYo3e4FHqDaNuEeYEGqDELrLxODOamTyny4tsZRfcc2oFpZPhI4ODM3AM6i4XeHwZzUPziHTv1GRHwL+COwPtVGwQsAPwO2ALbLzKvLLvWLAXMBl5U9slxNJ3VBm8z4elTbAb1Ptd9cUm0DNHNmPtRwzg3AnZn5494fsaRJMaBTvxERRwLLZea2DW3zAKcCXwK2zcwb2pxjmVX6lCLiGOAbwGtUm3JfD5yUmReV47MAi1Ldymt+YFX/EiX1L5Zc1Z+0AKu3loBK9uBl4O9UmYOLImLNcmwggMGc9OmUzPhuwJbAWsAKVN/FPcv8OYDtgF+X56uVfeYG9vpgJU2SAZ163STm7AD8kypDcEREzNawS/1o4M/Av4DhETG3gZzUbVYEbs3Mu4C3MvMRqu1K5gR2KX3+DvwO2KzcgWWQ30Gpf3GVq3pVmzk721OVcQYA12fm3RFxEbARMFspA00D/AR4nup2QxsDnwOu64PhS7XW9nZe5R6ts1BtAdTaNk1mPl6mQJwTEYtl5jNUZVg37Zb6KQM69aqGYO5o4OtUvyQWA74dEb8DjqAq92wOPAM8DYzPzC0iYiHgLartFCR1Qpu/TC1LtZL1Jao9HG+KiB0y8wKq7x9UG3Y/DrzZeJ2293eV1D8Y0KnXRcROwM7A1uV2XkOBk6nKPRMi4gjgt8B6wKvAXeXUfYB3qFbASuqgdm7ntT0wO/Ac1TZBB1Bl42YBrgY+AL5LFfC5abBUAwZ06gtLAreUYG4Hqq1K9snMc8svlEXKNglXAETEl0oQuCvwlcx8sc9GLtVMm8zcjlSrWb8DzAR8hmqxw5+p5s2dBLxC9Rent4DPl33mBpiZk/o3Azr1qEn8IpgXGBURa1OVew7MzD+X+TxbAwtGxLOZ+XbpPw0wHfCFMmFbUge12WduQ+D3mXl5aZueatPgv1D9hWl5YDlgAnCV+zxK9eE+dOoxbTIDn6EqqY4qd364unTbqczbISJmAi4CHsvMfdpca/rMfLcXhy9NNcq9WW8F5gGOycwjGo7NBPwVeD8zh7Y5z30epZpw2xL1mDa387oEeCgiTgeCauXqBGD6iFg8IlYELqTK3u1fzouStcNgTuq6zHwB2JZqTtxXI2KVhmPvUM1VXbCd8wzmpJowQ6du1yYztx1wPNUE688Am5Ru/6Saw3MY1Sq656kmX29S9rkyMyB1s4j4HHA28CBV6fWeMm/1cuDpzNy9TwcoqcsM6NRjIuIrwDbAQ5l5Smlbn2ry9axUwdyTwFJUE7Dvy8wW5+xIPSciVqLaKHge4E5gPNXWQZ/PzPfLilh/MUg1Y8lVPaLM2fkr8E2qjUsByMxrqVa1vk0V0K2YmTdm5j0lmHPTUqkHZeZ9wI5Uf4maDfhPZq5agrlpDeakejKgU49omLPzIrBlKfW0HrsO+D0wLdUGwjTMlXNrBKmHlW2BtqNaQb5GRCxd2t/v04FJ6jJLrupRZbHDGcC9wHGZ+WDDsZWB+w3ipL5RvoN/odq65JDMfLSPhySpi8zQqUdl5v3At4CVgH0jYvmGY/e2lln7bIBSE8vMe4G9qebTvd7Hw5H0KZihU68oE7FPAUYB+2fmiD4ekqTCfR6l+jMzol5RJmLvTZUFeLZvRyOpkcGcVH9m6NSrWrdE8N6QkiR1HwM69Tr3uZIkqXsZ0EmSJNWcc+gkSZJqzoBOkiSp5gzoJEmSas6ATlK3iYgPI+K+iHgoIi6IiBk/xbXWjYh/ledbRcRBk+k7e0Ts1YX3+GVEHNDR9jZ9zoiIr3bivRaNiIc6O0ZJ6ggDOkndaXxmrpSZywPvA99rPBiVTv9/JzMvzczfTKbL7ECnAzpJmloY0EnqKTcDS5bM1OMRcRbwELBQRGwUEbdFxD0lkzczQERsEhGPRcQ9VDePp7R/MyJOKM8HR8TFEXF/eawN/AZYomQHjyn9DoyIuyLigYg4rOFaP4uIJyLiFmCZKX2IiPhOuc79EXFhm6zjBhFxd7neFqX/wIg4puG9v/tpf5CSNCUGdJK6XUQMAjYFHixNSwEnZeZngXeAnwMbZOYqwN3AfhExPdXt4bYEVgXmm8Tl/wjcmJkrAqsADwMHAU+X7OCBEbFRec81qO4jvGpErBMRq/L/27ufF5vCOI7j74+fTTPXZGPBZvxoQpLNlLKRxSzY3I0ykWQymYX4A9gp/wKiLJRYUEqaZEPTDGqYFWaK2Ch3NWHYTF+L81XHbeR0aXT0edXtnvv8ON/nuatvz/N0DhzKsv3AQIXp3I6IgYz3Ehgu1fVljAPAxZzDMDAXEQN5/xOSNlaIY2bWsRX/egBm9l/pkvQirx8DV4H1wLuImMzy3cB2YFwSwCpgAtgKvI2IWQBJ14GRRWLsA44CRMQCMCdpbVubwfw8z989FAleA7gTEfMZ426FOe2QdJ5iW7cHGCvV3co3nsxKepNzGAR2ls7X9WbsmQqxzMw64oTOzP6mrxGxq1yQSduXchHwICKG2tr91O8PCbgQEZfaYpzp4F7XgGZETEs6Buwt1bU/mT0y9qmIKCd+SOrrILaZWSXecjWzpTYJ7JG0BUBSt6R+4BXQJ2lzthv6Rf+HwGj2XS6pF/hEsfr2wxhwvHQ2b4OkdcAjoCmpS1KDYnv3dxrAB0krgcNtdQclLcsxbwJeZ+zRbI+kfkndFeKYmXXMK3RmtqQiopUrXTckrc7isxExI2kEuCdpnmLLtrHILU4DlyUNAwvAaERMSBrPx4Lcz3N024CJXCH8DByJiClJN4Fp4CPwrMKQzwFPgFZ+l8f0HngKrAFORsQ3SVcoztZNqQjeAprV/h0zs874Xa5mZmZmNectVzMzM7Oac0JnZmZmVnNO6MzMzMxqzgmdmZmZWc05oTMzMzOrOSd0ZmZmZjXnhM7MzMys5pzQmZmZmdXcd6835pwLPJBjAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 720x504 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Plot confusion matrix\n",
    "from sklearn.metrics import confusion_matrix\n",
    "from support import print_confusion_matrix\n",
    "cnf_matrix = confusion_matrix(pred, y_val)\n",
    "_ = print_confusion_matrix(cnf_matrix, LABELS)\n",
    "\n",
    "# Print Precision Recall F1-Score Report\n",
    "from sklearn.metrics import classification_report\n",
    "\n",
    "report = classification_report(pred, y_val, target_names=LABELS)\n",
    "print(report)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## It's your turn"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Test out the model you just trained. Edit the `my_review` variable and Run the Code cell below. Have fun!🎉\n",
    "\n",
    "Here are some inspirations:\n",
    "- Rian Johnson\\'s Star Wars: The Last Jedi is a satisfying, at times transporting entertainment with visual wit and a distinctly human touch. \n",
    "- All evidence points to this animated film being contrived as a money-making scheme. The result is worse than crass, it\\'s abominably bad.\n",
    "- It was inevitable that there would be the odd turkey in there. What I didn\\'t realise however, was that there could be one THIS bad.\n",
    "\n",
    "And some wrong predictions:\n",
    "- Pulp Fiction: Quentin Tarantino proves that he is the master of witty dialogue and a fast plot that doesn\\'t allow the viewer a moment of boredom or rest.\n",
    "\n",
    "Can you do better? Play around with the model hyperparameters!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "1488dc6f57fa43e2a60c177a65a5e5de",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "interactive(children=(Textarea(value='', description='review', placeholder='Type your Review here'), Button(de…"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from ipywidgets import interact_manual\n",
    "from ipywidgets import widgets\n",
    "\n",
    "def get_prediction(review):\n",
    "    # Preprocessing\n",
    "    review_np_array = imdb_tokenizer.texts_to_sequences([review])\n",
    "    review_np_array = sequence.pad_sequences(review_np_array, maxlen=MAX_LEN, padding=\"post\", value=0)\n",
    "    # Prediction\n",
    "    score = model.predict(review_np_array)[0][0]\n",
    "    prediction = LABELS[model.predict_classes(review_np_array)[0][0]]\n",
    "    print('REVIEW:', review, '\\nPREDICTION:', prediction, '\\nSCORE: ', score)\n",
    "\n",
    "interact_manual(get_prediction, review=widgets.Textarea(placeholder='Type your Review here'));"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Save the result"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pickle\n",
    "\n",
    "# Saving Tokenizer\n",
    "with open('models/tokenizer.pickle', 'wb') as handle:\n",
    "    pickle.dump(imdb_tokenizer, handle, protocol=pickle.HIGHEST_PROTOCOL)\n",
    " \n",
    "# Saving Model Weight\n",
    "model.save_weights('models/cnn_sentiment_weights.h5')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### That's all folks - don't forget to shutdown your workspace once you're done 🙂"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}