Sandip_Panesar_p2.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "6t55qCf3Fsxa"
   },
   "source": [
    "# Project 2: Topic Classification"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "j9-9G8z4Fsxd"
   },
   "source": [
    "In this project, you'll work with text data from newsgroup posts on a variety of topics. You'll train classifiers to distinguish posts by topics inferred from the text. Whereas with digit classification, where each input is relatively dense (represented as a 28x28 matrix of pixels, many of which are non-zero), here each document is relatively sparse (represented as a bag-of-words). Only a few words of the total vocabulary are active in any given document. The assumption is that a label depends only on the count of words, not their order.\n",
    "\n",
    "The `sklearn` documentation on feature extraction may be useful:\n",
    "http://scikit-learn.org/stable/modules/feature_extraction.html\n",
    "\n",
    "Each problem can be addressed succinctly with the included packages -- please don't add any more. Grading will be based on writing clean, commented code, along with a few short answers.\n",
    "\n",
    "As always, you're welcome to work on the project in groups and discuss ideas on Slack, but <b> please prepare your own write-up with your own code. </b>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "AUYW83LqFsxd"
   },
   "outputs": [],
   "source": [
    "# This tells matplotlib not to try opening a new window for each plot.\n",
    "%matplotlib inline\n",
    "\n",
    "# General libraries.\n",
    "import re\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "# SK-learn libraries for learning.\n",
    "from sklearn.pipeline import Pipeline\n",
    "from sklearn.neighbors import KNeighborsClassifier\n",
    "from sklearn.linear_model import LogisticRegression\n",
    "from sklearn.naive_bayes import BernoulliNB\n",
    "from sklearn.naive_bayes import MultinomialNB\n",
    "\n",
    "# SK-learn libraries for evaluation.\n",
    "from sklearn.metrics import confusion_matrix\n",
    "from sklearn import metrics\n",
    "from sklearn.metrics import classification_report\n",
    "\n",
    "# SK-learn library for importing the newsgroup data.\n",
    "from sklearn.datasets import fetch_20newsgroups\n",
    "\n",
    "# SK-learn libraries for feature extraction from text.\n",
    "from sklearn.feature_extraction.text import *\n",
    "\n",
    "import nltk"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "_ALNjGgCFsxg"
   },
   "source": [
    "Load the data, stripping out metadata so that only textual features will be used, and restricting documents to 4 specific topics. By default, newsgroups data is split into training and test sets, but here the test set gets further split into development and test sets.  (If you remove the categories argument from the fetch function calls, you'd get documents from all 20 topics.)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "ecYpcoxaFsxh"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "training label shape: (2034,)\n",
      "dev label shape: (676,)\n",
      "test label shape: (677,)\n",
      "labels names: ['alt.atheism', 'comp.graphics', 'sci.space', 'talk.religion.misc']\n"
     ]
    }
   ],
   "source": [
    "categories = ['alt.atheism', 'talk.religion.misc', 'comp.graphics', 'sci.space']\n",
    "newsgroups_train = fetch_20newsgroups(subset='train',\n",
    "                                      remove=('headers', 'footers', 'quotes'),\n",
    "                                      categories=categories)\n",
    "newsgroups_test  = fetch_20newsgroups(subset='test',\n",
    "                                      remove=('headers', 'footers', 'quotes'),\n",
    "                                      categories=categories)\n",
    "\n",
    "num_test = int(len(newsgroups_test.target) / 2)\n",
    "test_data, test_labels   = newsgroups_test.data[num_test:], newsgroups_test.target[num_test:]\n",
    "dev_data, dev_labels     = newsgroups_test.data[:num_test], newsgroups_test.target[:num_test]\n",
    "train_data, train_labels = newsgroups_train.data, newsgroups_train.target\n",
    "\n",
    "print('training label shape:', train_labels.shape)\n",
    "print('dev label shape:',      dev_labels.shape)\n",
    "print('test label shape:',     test_labels.shape)\n",
    "print('labels names:',         newsgroups_train.target_names)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "OHTnOke6Fsxk"
   },
   "source": [
    "### Part 1:\n",
    "\n",
    "For each of the first 5 training examples, print the text of the message along with the label."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "X8zhA06xFsxl"
   },
   "outputs": [],
   "source": [
    "# Modded the function parameters to incorporate various datasets and labels.\n",
    "def P1(num_examples, dataset, labels):\n",
    "    for a in range(num_examples):\n",
    "        print(\"Topic Label:\", labels[a])\n",
    "        print (\"\")\n",
    "        print(\"Training Example:\")\n",
    "        print(\"\")\n",
    "        print(dataset[a])\n",
    "        print(\"----------------------------------------------------------------------\")\n",
    "    return"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Topic Label: 1\n",
      "\n",
      "Training Example:\n",
      "\n",
      "Hi,\n",
      "\n",
      "I've noticed that if you only save a model (with all your mapping planes\n",
      "positioned carefully) to a .3DS file that when you reload it after restarting\n",
      "3DS, they are given a default position and orientation.  But if you save\n",
      "to a .PRJ file their positions/orientation are preserved.  Does anyone\n",
      "know why this information is not stored in the .3DS file?  Nothing is\n",
      "explicitly said in the manual about saving texture rules in the .PRJ file. \n",
      "I'd like to be able to read the texture rule information, does anyone have \n",
      "the format for the .PRJ file?\n",
      "\n",
      "Is the .CEL file format available from somewhere?\n",
      "\n",
      "Rych\n",
      "----------------------------------------------------------------------\n",
      "Topic Label: 3\n",
      "\n",
      "Training Example:\n",
      "\n",
      "\n",
      "\n",
      "Seems to be, barring evidence to the contrary, that Koresh was simply\n",
      "another deranged fanatic who thought it neccessary to take a whole bunch of\n",
      "folks with him, children and all, to satisfy his delusional mania. Jim\n",
      "Jones, circa 1993.\n",
      "\n",
      "\n",
      "Nope - fruitcakes like Koresh have been demonstrating such evil corruption\n",
      "for centuries.\n",
      "----------------------------------------------------------------------\n",
      "Topic Label: 2\n",
      "\n",
      "Training Example:\n",
      "\n",
      "\n",
      " >In article <1993Apr19.020359.26996@sq.sq.com>, msb@sq.sq.com (Mark Brader) \n",
      "\n",
      "MB>                                                             So the\n",
      "MB> 1970 figure seems unlikely to actually be anything but a perijove.\n",
      "\n",
      "JG>Sorry, _perijoves_...I'm not used to talking this language.\n",
      "\n",
      "Couldn't we just say periapsis or apoapsis?\n",
      "\n",
      " \n",
      "----------------------------------------------------------------------\n",
      "Topic Label: 0\n",
      "\n",
      "Training Example:\n",
      "\n",
      "I have a request for those who would like to see Charley Wingate\n",
      "respond to the \"Charley Challenges\" (and judging from my e-mail, there\n",
      "appear to be quite a few of you.)  \n",
      "\n",
      "It is clear that Mr. Wingate intends to continue to post tangential or\n",
      "unrelated articles while ingoring the Challenges themselves.  Between\n",
      "the last two re-postings of the Challenges, I noted perhaps a dozen or\n",
      "more posts by Mr. Wingate, none of which answered a single Challenge.  \n",
      "\n",
      "It seems unmistakable to me that Mr. Wingate hopes that the questions\n",
      "will just go away, and he is doing his level best to change the\n",
      "subject.  Given that this seems a rather common net.theist tactic, I\n",
      "would like to suggest that we impress upon him our desire for answers,\n",
      "in the following manner:\n",
      "\n",
      "1. Ignore any future articles by Mr. Wingate that do not address the\n",
      "Challenges, until he answers them or explictly announces that he\n",
      "refuses to do so.\n",
      "\n",
      "--or--\n",
      "\n",
      "2. If you must respond to one of his articles, include within it\n",
      "something similar to the following:\n",
      "\n",
      "    \"Please answer the questions posed to you in the Charley Challenges.\"\n",
      "\n",
      "Really, I'm not looking to humiliate anyone here, I just want some\n",
      "honest answers.  You wouldn't think that honesty would be too much to\n",
      "ask from a devout Christian, would you?  \n",
      "\n",
      "Nevermind, that was a rhetorical question.\n",
      "----------------------------------------------------------------------\n",
      "Topic Label: 2\n",
      "\n",
      "Training Example:\n",
      "\n",
      "AW&ST  had a brief blurb on a Manned Lunar Exploration confernce\n",
      "May 7th  at Crystal City Virginia, under the auspices of AIAA.\n",
      "\n",
      "Does anyone know more about this?  How much, to attend????\n",
      "\n",
      "Anyone want to go?\n",
      "----------------------------------------------------------------------\n"
     ]
    }
   ],
   "source": [
    "P1(5, train_data, train_labels)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "onfno6uHFsxm"
   },
   "source": [
    "### Part 2:\n",
    "\n",
    "Transform the training data into a matrix of **word** unigram feature vectors.  What is the size of the vocabulary? What is the average number of non-zero features per example?  What is the fraction of the non-zero entries in the matrix?  What are the 0th and last feature strings (in alphabetical order)?<br/>\n",
    "_Use `CountVectorization` and its `.fit_transform` method.  Use `.nnz` and `.shape` attributes, and `.get_feature_names` method._\n",
    "\n",
    "Now transform the training data into a matrix of **word** unigram feature vectors using your own vocabulary with these 4 words: [\"atheism\", \"graphics\", \"space\", \"religion\"].  Confirm the size of the vocabulary. What is the average number of non-zero features per example?<br/>\n",
    "_Use `CountVectorization(vocabulary=...)` and its `.transform` method._\n",
    "\n",
    "Now transform the training data into a matrix of **character** bigram and trigram feature vectors.  What is the size of the vocabulary?<br/>\n",
    "_Use `CountVectorization(analyzer=..., ngram_range=...)` and its `.fit_transform` method._\n",
    "\n",
    "Now transform the training data into a matrix of **word** unigram feature vectors and prune words that appear in fewer than 10 documents.  What is the size of the vocabulary?<br/>\n",
    "_Use `CountVectorization(min_df=...)` and its `.fit_transform` method._\n",
    "\n",
    "Now again transform the training data into a matrix of **word** unigram feature vectors. What is the fraction of words in the development vocabulary that is missing from the training vocabulary?<br/>\n",
    "_Hint: Build vocabularies for both train and dev and look at the size of the difference._\n",
    "\n",
    "Notes:\n",
    "* `.fit_transform` makes 2 passes through the data: first it computes the vocabulary (\"fit\"), second it converts the raw text into feature vectors using the vocabulary (\"transform\").\n",
    "* `.fit_transform` and `.transform` return sparse matrix objects.  See about them at http://docs.scipy.org/doc/scipy-0.14.0/reference/generated/scipy.sparse.csr_matrix.html.  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "# I decided rather than just return the first and last features from the bag of words to actually return the list\n",
    "# of words (features) that were associated with each training sample. \n",
    "\n",
    "# Return alphabetized feature string for any vector and any index:\n",
    "def return_string(position, index_list, bow):\n",
    "    output_list = []\n",
    "    rows, cols = index_list[position].nonzero()\n",
    "    for index in cols:\n",
    "        output_list.append(bow[index])\n",
    "    print(sorted(output_list))\n",
    "    return"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 75,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "LyVwk5RvFsxn"
   },
   "outputs": [],
   "source": [
    "# First part:\n",
    "def P2_1(dataset):\n",
    "    vec = CountVectorizer(input = 'content', binary = False, ngram_range = (1,1))\n",
    "    vec_fit = vec.fit_transform(dataset)\n",
    "    examples, words = vec_fit.shape\n",
    "    print(\"The size of the vocabulary is:\", words)\n",
    "    print(\"The average number of non-zero features per example is:\", round(vec_fit.nnz/examples, 2))\n",
    "    print(\"The fraction of non-zero entries in the matrix is:\", round(vec_fit.nnz/(examples*words), 5))\n",
    "    print(\"The first feature string is: {}\".format(vec.get_feature_names()[0]))\n",
    "    print(\"The last feature string is: {}\".format(vec.get_feature_names()[-1]))\n",
    "    return \n",
    "\n",
    "# Second part:\n",
    "def P2_2(dataset, vocab):\n",
    "    vec = CountVectorizer(input = 'content', vocabulary = vocab, binary = False, ngram_range = (1,1))\n",
    "    vec_fit = vec.fit_transform(dataset)\n",
    "    examples, words = vec_fit.shape\n",
    "    print(\"The size of the vocabulary is:\", words)\n",
    "    print(\"The average number of non-zero features per example is:\", round(vec_fit.nnz/examples, 2))\n",
    "    return\n",
    "    \n",
    "# Third part:\n",
    "def P2_3(dataset):\n",
    "    vec = CountVectorizer(input = 'content', analyzer='char', binary = False, ngram_range = (2,3))\n",
    "    vec_fit = vec.fit_transform(dataset)\n",
    "    examples, words = vec_fit.shape\n",
    "    print(\"The size of the vocabulary is:\", words)\n",
    "    print(\"The average number of non-zero features per example is:\", round(vec_fit.nnz/examples, 2))\n",
    "    return\n",
    "\n",
    "# Fourth part:\n",
    "def P2_4(dataset):\n",
    "    vec = CountVectorizer(input = 'content', min_df=10, binary = False, ngram_range = (1,1))\n",
    "    vec_fit = vec.fit_transform(dataset)\n",
    "    examples, words = vec_fit.shape\n",
    "    print(\"The size of the vocabulary is:\", words)\n",
    "    print(\"The average number of non-zero features per example is:\", round(vec_fit.nnz/examples, 2))\n",
    "    return\n",
    "\n",
    "# Fifth part:\n",
    "def P2_5(train_dataset, test_dataset):\n",
    "    vec1 = CountVectorizer(input = 'content', binary = False, ngram_range = (1,1))\n",
    "    vec_train_fit = vec1.fit_transform(train_dataset)\n",
    "    vec2 = CountVectorizer(input = 'content', binary = False, ngram_range = (1,1))\n",
    "    vec_test_fit = vec2.fit_transform(test_dataset)\n",
    "    examples1, words1 = vec_train_fit.shape\n",
    "    examples2, words2 = vec_test_fit.shape\n",
    "    # Find the number of words that are missing from the dev dataset\n",
    "    a = set(vec1.get_feature_names())\n",
    "    b = set(vec2.get_feature_names())\n",
    "\n",
    "    # Print results\n",
    "    print(\"The size of the training vocabulary is: {}\".format(words1))\n",
    "    print(\"The size of the development vocabulary is: {}\".format(words2))\n",
    "    print(\"The fraction of missing words: {}\".format(round((len(b-a)/len(b)), 2)))\n",
    "    \n",
    "    return "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The size of the vocabulary is: 26879\n",
      "The average number of non-zero features per example is: 96.71\n",
      "The fraction of non-zero entries in the matrix is: 0.0036\n",
      "The first feature string is: 00\n",
      "The last feature string is: zyxel\n"
     ]
    }
   ],
   "source": [
    "P2_1(train_data)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The size of the vocabulary is: 4\n",
      "The average number of non-zero features per example is: 0.27\n"
     ]
    }
   ],
   "source": [
    "P2_2(train_data, [\"atheism\", \"graphics\", \"space\", \"religion\"])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The size of the vocabulary is: 35478\n",
      "The average number of non-zero features per example is: 684.19\n"
     ]
    }
   ],
   "source": [
    "P2_3(train_data)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The size of the vocabulary is: 3064\n",
      "The average number of non-zero features per example is: 72.68\n"
     ]
    }
   ],
   "source": [
    "P2_4(train_data)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 76,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The size of the training vocabulary is: 26879\n",
      "The size of the development vocabulary is: 16246\n",
      "The fraction of missing words: 0.25\n"
     ]
    }
   ],
   "source": [
    "P2_5(train_data, dev_data)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "ydjkRh6LFsxp"
   },
   "source": [
    "### Part 3:\n",
    "\n",
    "Transform the training and development data to matrices of word unigram feature vectors.\n",
    "\n",
    "1. Produce several k-Nearest Neigbors models by varying k, including one with k set to optimize f1 score.  For each model, show the k value and f1 score.\n",
    "1. Produce several Naive Bayes models by varying smoothing (alpha), including one with alpha set approximately to optimize f1 score.  For each model, show the alpha value and f1 score.\n",
    "1. Produce several Logistic Regression models by varying L2 regularization strength (C), including one with C set approximately to optimize f1 score.  For each model, show the C value, f1 score, and sum of squared weights for each topic.\n",
    "\n",
    "* Why doesn't k-Nearest Neighbors work well for this problem?\n",
    "* Why doesn't Logistic Regression work as well as Naive Bayes does?\n",
    "* What is the relationship between logistic regression's sum of squared weights vs. C value?\n",
    "\n",
    "Notes:\n",
    "* Train on the transformed training data.\n",
    "* Evaluate on the transformed development data.\n",
    "* You can use `CountVectorizer` and its `.fit_transform` and `.transform` methods to transform data.\n",
    "* You can use `KNeighborsClassifier(...)` to produce a k-Nearest Neighbors model.\n",
    "* You can use `MultinomialNB(...)` to produce a Naive Bayes model.\n",
    "* You can use `LogisticRegression(C=..., solver=\"liblinear\", multi_class=\"auto\")` to produce a Logistic Regression model.\n",
    "* You can use `LogisticRegression`'s `.coef_` method to get weights for each topic.\n",
    "* You can use `metrics.f1_score(..., average=\"weighted\")` to compute f1 score."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 77,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "VvhpODdWFsxp"
   },
   "outputs": [],
   "source": [
    "def P3_1(k_value, training_data, training_labels, testing_data, testing_labels):\n",
    "    \n",
    "    # Transform raw datasets into matrices\n",
    "    vec = CountVectorizer(input = 'content', binary = False, ngram_range = (1,1))\n",
    "    vec_fit_train = vec.fit_transform(training_data)\n",
    "    vec_fit_test = vec.transform(testing_data)\n",
    "    \n",
    "    # Fit KNN model \n",
    "    master_dic = {}\n",
    "    for value in k_value:\n",
    "        mod_knn = KNeighborsClassifier(n_neighbors = value)\n",
    "        mod_knn_fit = mod_knn.fit(vec_fit_train, training_labels)\n",
    "        mod_knn_pred = mod_knn.predict(vec_fit_test)\n",
    "        mod_knn_report = classification_report(testing_labels, mod_knn_pred, output_dict = True)\n",
    "        master_dic[value] = mod_knn_report['macro avg']['f1-score']\n",
    "        print(\"For {}-nearest neighbors, the F-1 score is: {}\".format(value, round(mod_knn_report['macro avg']['f1-score'], 3)))\n",
    "        \n",
    "    # Plot a graph to look at F1 scores for various K's\n",
    "    fig, ax1 = plt.subplots(figsize=(7,4))   \n",
    "    color = 'tab:red'\n",
    "    ax1.set_title(\"Average F-1 Score For Various K-Values\")\n",
    "    ax1.set_xlabel('K')\n",
    "    ax1.set_ylabel('Average F-1 Score')\n",
    "    ax1.plot(list(master_dic.keys()), list(master_dic.values()), marker = 'o', color = color)\n",
    "    \n",
    "    return \n",
    "\n",
    "\n",
    "def P3_2(alph, training_data, training_labels, testing_data, testing_labels):\n",
    "    \n",
    "    # Transform raw datasets into matrices\n",
    "    vec = CountVectorizer(input = 'content', binary = False, ngram_range = (1,1))\n",
    "    vec_fit_train = vec.fit_transform(training_data)\n",
    "    vec_fit_test = vec.transform(testing_data)\n",
    "    \n",
    "    # Fit NB model\n",
    "    master_dic = {}\n",
    "    for value in alph:\n",
    "        BNB = BernoulliNB(alpha = value)\n",
    "        BNB_fit = BNB.fit(vec_fit_train, training_labels)\n",
    "        BNB_predict = BNB_fit.predict(vec_fit_test)\n",
    "        BNB_report = classification_report(testing_labels, BNB_predict, output_dict=True)\n",
    "        master_dic[value] = BNB_report['macro avg']['f1-score']\n",
    "        print(\"For an alpha of {}, the F-1 score is: {}\".format(value, round(BNB_report['macro avg']['f1-score'], 3)))\n",
    "     \n",
    "    # Plot a graph to look at F1 scores for various alphas\n",
    "    fig, ax1 = plt.subplots(figsize=(7,4))   \n",
    "    color = 'tab:orange'\n",
    "    ax1.set_title(\"Average F-1 Score For Various Alphas\")\n",
    "    ax1.set_xlabel('Alpha')\n",
    "    ax1.set_ylabel('Average F-1 Score')\n",
    "    ax1.plot(list(master_dic.keys()), list(master_dic.values()), marker = 'v', color = color)\n",
    "    \n",
    "    return \n",
    "\n",
    "def P3_3(c, training_data, training_labels, testing_data, testing_labels):\n",
    "    \n",
    "    # Transform raw datasets into matrices\n",
    "    vec = CountVectorizer(input = 'content', binary = False, ngram_range = (1,1))\n",
    "    vec_fit_train = vec.fit_transform(training_data)\n",
    "    vec_fit_test = vec.transform(testing_data)\n",
    "    \n",
    "    # Fit LR model\n",
    "    master_dic = {}\n",
    "    for value in c:\n",
    "        LR = LogisticRegression(C = value, solver=\"liblinear\", multi_class=\"auto\")\n",
    "        LR_fit = LR.fit(vec_fit_train, training_labels)\n",
    "        LR_predict = LR_fit.predict(vec_fit_test)\n",
    "        coef = np.sum(LR.coef_**2)\n",
    "        LR_f1 = metrics.f1_score(testing_labels, LR_predict, average=\"weighted\")\n",
    "        master_dic[value] = LR_f1\n",
    "        print(\"For a C of {}, the F-1 score is: {}\".format(value, round(LR_f1, 3)))\n",
    "        print(\"The sum of the topic squared weights is: {}\".format(round(coef, 3)))\n",
    "        print(\"------------------------------------------------------------------\")\n",
    "        \n",
    "    # Plot a graph to look at F1 scores for various C's\n",
    "    fig, ax1 = plt.subplots(figsize=(7,4))   \n",
    "    color = 'tab:green'\n",
    "    ax1.set_title(\"Average F-1 Score For Various C's\")\n",
    "    ax1.set_xlabel('C')\n",
    "    ax1.set_ylabel('Average F-1 Score')\n",
    "    ax1.plot(list(master_dic.keys()), list(master_dic.values()), marker = 'x', color = color)\n",
    "    \n",
    "    return "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 78,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "For 1-nearest neighbors, the F-1 score is: 0.369\n",
      "For 5-nearest neighbors, the F-1 score is: 0.415\n",
      "For 12-nearest neighbors, the F-1 score is: 0.408\n",
      "For 24-nearest neighbors, the F-1 score is: 0.39\n",
      "For 48-nearest neighbors, the F-1 score is: 0.417\n",
      "For 96-nearest neighbors, the F-1 score is: 0.434\n",
      "For 120-nearest neighbors, the F-1 score is: 0.441\n",
      "For 192-nearest neighbors, the F-1 score is: 0.383\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAcYAAAEWCAYAAAD8XDcGAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nOzdeXhU5fXA8e/JJCEQQhZIIJBAAMWKWrVS17qidWvEFVlUcK0gIihFQKEI4oIKKptbUawLYNUKVqt00eqv2opbrQtlESQLhDULkHXO7497g0PIMoFM7iRzPs8zD3Pfucu5k2HOvPe+i6gqxhhjjHFEeR2AMcYYE04sMRpjjDEBLDEaY4wxASwxGmOMMQEsMRpjjDEBLDEaY4wxASwxGmOalYiUiEgvr+MIFRF5QUSmeh2HOXCWGA0i8p6I7BCRNl7H0hTc8yl1v4CrHyfVse6ZIvJ3ESkUkfVB7Pt6EflORIpFZLOI/ElEEpr8JBpBRJ4TkfIa53tlE+z3SRF5vpbyn4pImYikHMh+VbW9qq472PiCJSJTReSFgOVu7t/wcRGRGuue6v5t29Wyn69E5ObmiNl4yxJjhBORLOBUQIGLQnSM6FDstwGj3C/g6sdHday3C1gI/KahHYrI6cB9wGBVTQAOB5Y2WcQc1Hs1s8b5LjmAY/tqFD0HXCoi8TXKrwHeVNXtjdy/F5+DmjH0AP4BLFPV0VpjhBNV/QDYDFxaY7tjgEOBRr+vpuWxxGiuAT7G+RIcVl0oIieKyKbAL0sRuURE/uM+jxKRCSKyVkS2icjS6hqEiGSJiLq1qx+Av7nlr7j7LBSRf4jIEQH77igiy0WkSEQ+EZF7ReTDgNd/IiIrRGS7iKwSkYFNcfKq+m9V/T0QTA3m58BHqvq5u+12VV2kqsVujG1F5BER2eCe44ci0tZ97SIR+VpEdro12sMDzm29iNzpvre7RCRaRLqKyKsiskVEvheR0QdyfiJyuHu8ne7xLwp47TkRWSAib4nILuDMGu/NR0AucFnANj5gCLDIXT5eRD5y958vInNFJDZgfRWRW0RkNbA6oOwQ93miiDzvnucGEblbRKLc12rW9Ko/V9Hu8nARWefW8L4XkaENvBe9cZLiS6o6vp5Vn8f5fxHoGpxkusP97P/B/Szv9/esccwbROS9gOVo9xyy3OU4EZklIhvFuQIxX0Ti3NfS3L/NTvdz/4/6zs80HUuM5hrgRfdxroh0BlDVj3FqU2cFrDsEeMl9Phq4GDgd6ArsAObV2PfpOLWqc93lt3F+dacBn7nHrDbPPV4XnAQdmKTjgRXusdOAwcD8wMTaTP6F8x7dIyKnyP6Xnh8GjgNOBlKA8YBfRPoALwNjgFTgLWB5YALBOacLgSTADywHvgS6Af2BMSJyLo0gIjHuft7Fed9uBV4UkcMCVhsCzAASgA/328n+SeJsIAbnbwlQBYwFOgEnubGOrLGPi4ETgL617H8OkAj0wvm8XANcG8S5xQOPA+e7tfeTgS/q2aQXTlJ8UlUnN7D754EzRaSbeywfzt8n8LLymzif5S7Af4HfNxRzHR4GegI/dfeXBdzlvvYbnB9sqe5xGorbNBVVtUeEPoBfABVAJ3f5O2BswOv3Agvd5wk4iauHu/wt0D9g3XR3X9E4/7kV6FXPsZPcdRIBn7vtYTWO/aH7/ErggxrbPwn8to59vwfsBna6j8+CeC/OBtYHsd75OMlmJ1ACzHLjjwL2AEfXss1kYGnAchROTewMd3k9cF3A6ycAP9TYx0Tg2Tpieg4oDTjfrW75qcAmICpg3ZeBqQHbPd/A+XZ3/zYZ7vKLwGP1rD8GeD1gWYGzaqyjwCHu+1YG9A147dfAe+7zqcALAa9Vf66igXj3XC8D2jZwDlOBInf93kH+33gPGB/wN98MRNexbic3rnh3+YWA9/iG6vNxl6PddbPcz0Ep7v+pgL/Zavf5fcBrwcZsj6Z7WI0xsg0D3lXVre7ySwTU1NzlS92a0aU4CWaD+1oP4HX3Ms9OnERZBXQO2H5j9RMR8YnIA+6l1yKcZADOl0oqzhfGxtq2dY91QvWx3OMNxfkVXZfRqprkPn7mxjBJfmyc8kS970wdVPVtVc3GqREOAIbjfPl1AuKAtbVs1hXYELAPv3t+3QLWqXm+XWuc7yT2fW9rejjgfDsFHHeje7xqG+o57n5U9QecmtZVItIep/a3qPp1EekjIm+6lxWLcL7MO9XYTV3H6ATEEvDe1BJfXXHtwvnBdDOQL04jqJ/Us8kynHvJfxPnPmN1/GcEfCa+DFh/ET/WlK8GXlTVSncbn4jMdC/jFgFrAs6nMboAbYAvA/7Ob+LU7gEewHk//ur+v2nwPrhpGpYYI5Q4974GAqe7X2qbcC6JHS0iRwOo6jc4/zHPZ9/LqOB82Z0f8GWcpKpxqpobsE5gw4YhOInkbJxaYlZ1KMAWoBLICFg/s8ax3q9xrPaqOqIx56yq9+mPjVMOqnWhqvpV9a8490+PBLbi/PrvXcvqeTjJDgAREZzzq+u92gh8X+N8E1T1gkaGmQdkVt+zc3Wv57h1qU4Sl7lxfRbw2gKcKw2HqmoHnAQuNbav6xhbcWqjPQLKAuPbBQS2Dt3nh5CqvqOq5+BcrfgOeLq+k1DV23ESz9+qL5Oq6nsBn4mjA1Z/BegpToOrAex7GfUa4AKc2wyJOLVf2P+8GzqHzUA5zpWS6r9zoqomurEVqepYVc3C+UFypxuPCTFLjJHrYpwaXl/gGPdxOPAB+95TegnnfuJpOF8W1Z4AZlT/+haRVBEZUM/xEnAum23D+aK4r/oFVa3CuWQ0VUTaub/8A2N4E+gjIleLSIz7+HldDR4aw21IEYdz30zcxhCxdaw7QEQGiUiyOI7HuS/2sVsrWwjMEqfhjE9ETnJr20uBC0Wkv3vf7w73vfhnHWH9GygSp0FOW3dfR4rIzxt5ev/C+WIe775nZwDZwOJG7udVnER+DwG1RVcCzmXKEvfvFvSPFffvvhTnc5TgfpZux7kUCc49w9NEpLuIJOJcTgZARDqL06ApHue9LMH5PDdkFM6Pmb9W30+vI7YSnM/kImCNqgbev6z5WZ5Rz/G+BH4qIke5P0Z/W+P8nwEedf//iIhkiMgv3XPMFpHe7g+pQvf8gjlHc5AsMUauYTj3rH5Q1U3VD2AuMFR+bFr/MnAG8LeAS64Aj+FcnnpXRIpxWraeUM/xnsepfeYC37jrBxqF8+t7E05DhpdxvnxQp9XnL4FBOLWgTcCDOJehDtZpOPcG38KprezBaaxSmx3AjTitK4twvsAfUtXqRkTjgK+AT4DtboxRqroKuAqnoclWnOSUrarltR3E/cLMxvmx8r27zTM470/Q3P1fhFPj3wrMB65R1e8auZ9d/JgcX6zx8jicqwHFODW2xnZnuBUnea/DafzzEs4PDFR1hbu//wCf4vxAqhaF8wMjD+e9Pp39G/3Udi6Kcx/z38BfRKS+y5+LcGqzNftyPuseNw/4mrp/4FRfdbkP557lKpzL0oHuwPl/8W+c5PcuTiMcgMNwkngJ8H8493ZrayBlmpg4nxNjwouIPAh0UdVhDa5sjDFNyGqMJiyI00/xpwGXKK8HXvc6LmNM5PF8JApjXAk4l0+7AgXAI8AbnkZkjIlIdinVGGOMCWCXUo0xxpgAreZSaqdOnTQrK8vrMIwxxoSRTz/9dKuqpjZmm1aTGLOysli5cqXXYRhjjAkjIrKh4bX2ZZdSjTHGmACWGI0xxpgAlhiNMcaYAJYYjTHGmACWGI0xxpgAlhiNMZ4oXL6c1Wf159vD+7L6rP4ULl/udUjGAK2ou4YxpuUoXL6c/MlT0NJSACrz8sifPAWAxOxsL0MzxmqMxpjmVzD70b1JsZqWllIw+1GPIjLmR5YYjTHNSlWpzMur9bXK/PxmjsaY/YU0MYrIeSKySkTWiMiEeta7XERURPrVKO8uIiUiMi6UcRpjQk+rqij68zusv/yKOteJTk9vxoiMqV3IEqOI+IB5OLOH9wUGi0jfWtZLAEYD/6plN7OBt0MVozEm9Pzl5ex45RXWXXAhuWPG4C8pIfHyy5G4uP3WTbr0Eg8iNGZfoWx8czywRlXXAYjIYmAA8E2N9aYDM4F9aoUicjGwDtgVwhiNMSFSVVLCziVL2f7cc1Ru2ULcEUfQ7bHHSDi7P+LzEX/C8RTMfpTK/HyiO6fhL69gx+IlJF16KTFdu3odvolgoUyM3YCNAcs5wAmBK4jIsUCmqr4ZeLlUROKBO4FzqJEwa2x/E3ATQPfu3ZsucmPMAavcto3tz/+eHS+/jL+oiHYnnUjXBx+g3UknISJ710vMzt6nBWrZmjWsHzSYjSNvIevFF4iKj/cifGNCmhillrK9syKLSBTOpdLhtax3DzBbVUsC/yPttzPVp4CnAPr162czLhvjofKcHLYvXMjOV19Dy8tJ+OUv6XjDDbQ96sigtm9zyCF0mz2Ljb++mdzxd5Ix53EkytoHmuYXysSYA2QGLGcAgU3REoAjgffc5NcFWCYiF+HULC8XkZlAEuAXkVJVnRvCeI0xB6B01Sq2Pf0MRW+/DVFRJF08gJTrrqNNz56N3lf7U0+l84QJbL7vPrbMnk3aHXeEIGJj6hfKxPgJcKiI9ARygUHAkOoXVbUQ6FS9LCLvAeNUdSVwakD5VKDEkqIx4UNV2fPpp2x9+ml2vf8Potq1I2XYMFKGXUNM584Hte/kq6+ibN1atj39DLG9epN0ycVNFLUxwQlZYlTVShEZBbwD+ICFqvq1iEwDVqrqslAd2xgTGur3U/Le+2x7+mn2fP45vuRkUsfcRvLgwfgSE5vkGCJCl7vuonz9BvKnTCG2eybtjjuuSfZtTDBEtXXcmuvXr5+uXLnS6zCMaZW0ooKit95i2zPPULZ6DTFdu5Jy/XUkXXopUW3bhuSYVTt3sv7KQVQVFZH1ylJiMzJCchzTuonIp6rar+E1f2R3to0xdfLv2cP237/AmnPPJe/OCYDQ9aGZ9H7nz6QMHRqypAjgS0oi44kFaFUVOSNGUFVSErJjGRPIEqMxZj9VO3eyZf581pzVn80zZhCT3pWMJxbQc9kbJGZnIzExzRJHm549yXjsUcrWfU/uHXegVVXNclwT2Wx2DWPMXhWbNrH9uUXsWLoU3b2b9mecQccbb/D0Hl/8SSfRZfLdbJp6DwUzH6LzxDpHlzSmSVhiNMZQtm4d2373OwqXLQe/nw4XXkDH628g7rA+XocGQPKgQZStXcf2RYuI7d2L5IEDvQ7JtGKWGI2JYHv+8x+2Pf0MxX/5CxIbS/LAgaRcey2xGd28Dm0/ne8cT/n337Np2nRiu/cg/sQTGt7ImANgrVKNiTCqyq5//pNtTz/D7o8/JqpDB5KHDiHlqquI7tjR6/DqVVVczPpBg6ncupWeSxYTm5XldUgmzB1Iq1RLjMZECK2qonjFCrY99TSl33xDdFoaKcOHkzRwIL72LWdc0vIffmD9wCvxJSeTtWQxvg4dvA7JhDHrrmGM2Y+/vJwdS5ey9oILyB0zFv/u3aTfO53ef1lBx+uubVFJESC2e3cy5jxOeU4OuWPGopWVXodkWhm7x2hMK1G4fPmP0zilp9NpxM34CwvZvuj5vdM+pQVM+9SStfv5z0mf+lvy77qbzffdT5cpk70OybQilhiNaQUKly8nf/IUtLQUgMq8PDZNngJA/Mkn0XXmg7Q78UTqm62mpUm67DKnperChcT27kXK0KFeh2RaCUuMxrQCBbMf3ZsUA/k6daL7woUeRNQ80u64nfJ169h83/3EZmXR/pRTvA7JtAJ2j9GYFq4iL4/KvLxaX6vatq2Zo2le4vPR9eGHadO7N7ljxlK2bp3XIZlWwBKjMS1URV4e+VOnsubc8+pcJzo9vRkj8oavfTyZC+YjsbFsvHkElTt2eB2SaeEsMRrTwgQmxJ2vvkbSZZeSNmkiEhe3z3oSF0fa2DEeRdm8Yrp1I2POHCrz88kdfRtaXu51SKYFs3uMxrQQFbm5bH3qaXa+9hoASZddSqebbiKma1cAopOT92mVmjZ2DInZ2V6G3Kza/exY0mfcS974O9k0/V66TLunVTU2Ms3HEqMxYW6/hHj5ZXS68ca9CbFaYnZ2RCXC2iRedBFla9ex7cknie3di47Dh3sdkmmBLDEaE6aCTYhmX6m3jaZ83ToKZj5EbFYWCWec4XVIpoWxIeGMCTOWEA+ef/du1l91FRUbfqDHyy8R1yc8ZgkxzS/shoQTkfNEZJWIrBGROidRE5HLRURFpJ+7fLyIfOE+vhSRS0IZpzHhoCI3l/wpv2XNeedT+NprJF9xOYe8+w7pv/2tJcVGimrXjsz585F2bckZMZLKVt5txTStkNUYRcQH/A84B8gBPgEGq+o3NdZLAP4ExAKjVHWliLQDylW1UkTSgS+Brqpa56CIVmM0LVVFbi5bn3yKna+/jgBJV1xOxxtvJCYCulqE2p6vvmLDVVcTd8QRdH/uWaJiY70OyTSzcKsxHg+sUdV1qloOLAYG1LLedGAmsHfYDlXdHZAE44DWcb3XmAD71BBff53kKy6n97vv0GXKFEuKTaTtUUfR9YH72fPZZ2yaPIXWcuvIhFYoG990AzYGLOcA+8wsKiLHApmq+qaIjKvx2gnAQqAHcHVttUURuQm4CaB79+5NG70xIbK3hvjaa4gIyVZDDKkO559P2dp1bJ07l9hDetPpxhu9DsmEuVAmxto6EO39uSYiUcBsYHhtG6vqv4AjRORwYJGIvK2qpTXWeQp4CpxLqU0UtzEhUZ6Ty7Ynn3QumYqQPHAgHW+6kZguXbwOrdXrdMtIytetY8us2bTp2ZOEs8/2OiQTxkKZGHOAzIDlDCBwQMcE4EjgPbcTbhdgmYhcpKp7bxaq6rcisstd124imhbHEqL3RIT0+2ZQvnEjuePvJOvFF4g7/HCvwzJhKpSNb6JxGt/0B3JxGt8MUdWv61j/PWCc2/imJ7DRbXzTA/gI+Kmqbq3reNb4xoSbmgkx6YorLCF6rKKggPUDrwQRei5dQnRqqtchmRA7kMY3IasxukltFPAO4AMWqurXIjINWKmqy+rZ/BfABBGpAPzAyPqSojHhpDwnx02If7QaYpiJSUsjc/481g+9io2jRtFj0SKiaowxa4x18DemidRMiEkDB9LxxhssIYahohUryL11NB0uvJCuDz9kY6q2YmFVYzQmUuxXQ7zySkuIYa7DOedQPnYsW2bPJrZ3L1JHjvQ6JBNGLDEac4BqTYg33UhM585eh2aC0PGmGylft5atj8+hTa9edDiv7nktTWSxxGhMI5Xn5LD1iSco/OMbSFQUyYMGOTVES4gtiojQZfp0yn/YSN6EicR0y6DtUUd6HZYJA3aP0Zgg1UyIe+8hWkJs0Sq3bWP9FQPRykqyXllqf89Wxu4xGhMC5Rs3svXJJ62G2EpFd+xIxoIFbBg8mJwRI+nx4gtEtW3rdVjGQ5YYjamDJcTIEXdYH7o+8jA5I28hb8JEus2ehUSFdPIhE8YsMRpTQ/nGjT9eMvX5SB48mI433EBM5zSvQzMhlHDmmaT95jcUzJzJljlzSLvtNq9DMh6xxGiMa7+EOGSIJcQIk3LtcMrWrWXbgido06s3idm/8jok4wFLjCbiWUI01USE9ClTqFi/gfy77iI2M4O2xxzjdVimmVmrVBOxaibEpEFX0vF6S4gGKnfsYP3AK/Hv2UPPpUuI6drV65DMAbJWqcYEofyHH9j6xJMUvuHWEIcOsYRo9hGdnEzmEwtYf+UgNo4YSdZLLxIVH+91WKaZWGI0EWOfhBgdbQnR1KtN7950mz2bjb/+Nbm/GU/GnMcRn8/rsEwzsMRoWr1aE+INNxCTZgnR1K/9qb+g86RJbL73XrbMnk3auHFeh2SagSVG02rVTIgpVw0l5frrLSGaRkkeOoSytWvY9szviO3Vm6RLL/E6JBNilhhNq1O+YYOTEJcts4RoDpqI0GXSJMrXryf/t78ltnsm7fo1qi2HaWGsVappNWomxORBV1pCNE2mqrCQ9VcOomrnTrJeWUpsZqbXIZkgWKtUE5H2ryFeRcr111lCNE3Kl5hI5hML+P7KQWwcMYKsxYvxtW/vdVgmBCwxmhartoTY8YbriU5N9To000rFZmWR8dij/HDDjeTefjuZ8+cj0fY12toEPUquiDS6E4+InCciq0RkjYhMqGe9y0VERaSfu3yOiHwqIl+5/57V2GOb1qt8wwbyJkxk7QUXUvTWW6RcdRWH/GUFnSdOsKRoQi7+xBPpcvfd7PrHBxQ89JDX4ZgQaPCnjoicDDwDtAe6i8jRwK9VdWQD2/mAecA5QA7wiYgsU9VvaqyXAIwG/hVQvBXIVtU8ETkSeAfoFvxpmdaofMMGti54gsLly62GaDyVPOhKytatZfui54nt1ZvkKwd6HZJpQsFcA5gNnAssA1DVL0XktCC2Ox5Yo6rrAERkMTAA+KbGetOBmcDeDkKq+nnA618DcSLSRlXLgjiuaWX2SYgxMaRcfTUdr7/OEqLxVOfx4yn/fj2bpk8ntkcP4k88weuQTBMJ6lKqqm6sUVQVxGbdgMDtcqhR6xORY4FMVX2znv1cBnxeW1IUkZtEZKWIrNyyZUsQIZmWZJ9Lpn/+MylXX80hK96l84Q7LSkaz0l0NN1mPUJsVg9ybruN8vXrvQ7JNJFgEuNG93KqikisiIwDvg1iO6mlbG/fEBGJwqmN3lHnDkSOAB4Efl3b66r6lKr2U9V+qfZF2WqUr19P3p0TWHv+BZYQTVjzJSSQuWABIsLGm0dQVVjodUimCQSTGG8GbsGp7eUAx7jLDckBAjv6ZAB5AcsJwJHAeyKyHjgRWBbQACcDeB24RlXXBnE808LtTYgXXEjRO++QMmyY06jGEqIJY7GZmWTMnUN5bi65Y8eiFRVeh2QOUr33GN0GNFer6tAD2PcnwKEi0hPIBQYBQ6pfVNVCoFPAsd4DxqnqShFJAv4ETFTV/zuAY5sWpHz9+h/vIcbGkjJsmHMPsVOnhjc2Jgy069eP9HvuIX/SJDbffz9dpkzxOiRzEOpNjKpaJSIDcC55NoqqVorIKJwWpT5goap+LSLTgJWquqyezUcBhwCTRWSyW/ZLVS1obBwmfFlCNK1J0qWXULZ2Ddt/t5DY3r1JGXog9QkTDhocEk5EZgCJwBJgV3W5qn4W2tAax4aEC1+Fy5dTMPtRKvPziU5PJ3noUMr/t4rC5W8isbEkDx5sCdG0ClpVRc6toyl5/30yn3yS9r84xeuQIt6BDAkXTGL8ey3Fqqph1eneEmN4Kly+nPzJU9DS0n1fiI4m5Zpr6HjdtZYQTatSVbKLDUOHUpGXR9bil2nTu7fXIUW0kCTGlsISY3hafVZ/KvPy9iuP7tyZQ99/r/kDMqYZVOTm8v3AK4mKjydryWKik5O9DiliHUhibLBVqogkisis6v6CIvKIiCQeeJgmklTm59deXmC3i03rFdOtGxlz51C5aRO5o29Dy8u9Dsk0QjDdNRYCxcBA91EEPBvKoEzrEZ2e3qhyY1qLdsceS/qMe9n9ySfkT5tGa7k6FwmCSYy9VfW3qrrOfdwD9Ap1YKZ1SLl2+H5lEhdH2tgxzR+MMc0sMTubjjf/msI/vMr2RYu8DscEKZjEuEdEflG9ICKnAHtCF5JpTfZ8+hn4fESnpYEI0V27kj59GonZ2V6HZkyzSB09moRf/pKCB2dS/N57XodjghDMIOIjgEUB9xV3AMNDFlGYqtnlIG3sGPtyb8Cujz6i+M9/ptPoW0kdWe9kLMa0WhIVRdcH7mdDTg55t99Bj8UvE9enj9dhmXoE3SpVRDoAqGpRSCM6QKFslVpblwOJi7OaTz20ooJ1F1+ClpfT683lRLVp43VIxniqYvNm1l9+BRIbS9bSJUR37Oh1SBEhVK1S7xORJFUtUtUiEUkWkXsPPMyWp2D2o/v1w9PSUgpmP+pRROFv++9foHztWjpPmmhJ0RggpnNnMubPp3LrVnJuHY3fWqqGrWDuMZ6vqjurF1R1B3BB6EIKP3V2OaijPNJVFBSwde5c2p9+Oglnnul1OMaEjbZHHUnXB+5nz2efsWnyFGupGqaCSYw+Edn7k19E2gIRVQWwLgeNU/Dww2hFBZ0nTfQ6FGPCTofzz6fTraMofOMNtj39jNfhmFoEkxhfAP4qIteLyHXACiCi2h2njR0DMTH7lElsrHU5qMXulSspWraclBuuJ7ZHD6/DMSYsdRo5kg4XXMCWWbMoWrHC63BMDQ0mRlWdCdwLHA70Baa7ZREjMTubhHPPdRZEwOeDuDjif/GL+jeMMFpZyabp9xLdNZ1ON93kdTjGhC0RIf2+GcT99Kfkjb+T0m++8TokEyCYGiOq+mfgfuD/gK0hjShMtemeCSL85Ov/krV0CbpnD5um2D2CQDsWL6Fs1So63zmBqLZtvQ7HmLAWFRdHxtw5+BIT2TjyFipsmMSwUWdiFJE3ReRI93k68F/gOuD3IhJx1xCrikuIio9HoqJoe8QRpI0dS/GKv7Bz6StehxYWKrdtY8tjjxF/8skk/PIcr8MxpkWISUsjc8F8qgoLyRl1K/6as9AYT9RXY+ypqv91n18LrFDVbOAEnAQZUfxFRUQlJOxdThk+jPiTT2bz/fdTtnath5GFh4JZs/Dv2UPnu+9CRLwOx5gWI+7ww+n20ExKv/qK/El32VWoMFBfYqwIeN4feAtAVYsBfyiDCkdVJSX4AhKjREWR/sD9RLVtS+4d4/CXlXkYnbf2fPklha++Rsqwa2jTy4bRNaaxEs4+m9Tbx1L01ltsnT/f63AiXn2JcaOI3CoilwA/A/4Me7trxNSzXavkLy7ep8YIzmWQ9PtmUPbdd2yZNcujyLylVVVsmjad6LQ0Oo2wYd+MOVAdb7iBxAED2DpnLkVvv+11OBGtvsR4PXAEzrioVwZ08j+RCJx2qqqkGF/79vuVJ5x5JslXXcX2Rc9T8o9/eBCZt3b+4VVKv/6atPHj8bWP9zocY1osEaHL9Gm0/dnPyJswkT1ffeV1SBGrzsSoqgWqegIVeIIAACAASURBVLOqDlDVdwPK/66qDwezcxE5T0RWicgaEZlQz3qXi4iKSD93uaOI/F1ESkRkbmNOKFT8RfvXGKul/WYcbQ49lLyJk6jcGjmNdit37GDLrFm0+/nP6XBhRA2GZExIRMXGkjF3DtGdOpEz8hYqNm3yOqSIFFR3jQMhIj5gHnA+Tv/HwSLSt5b1EoDRwL8CikuBycC4UMXXWP7iYnwdak+MUW3a0PWRh/GXlJA3cRLqj4xbsFsee4yqkhI63323NbgxpolEp6SQsWA+/l27yBl5C/7du70OKeKELDECxwNr3MmNy4HFwIBa1psOzMRJhgCo6i5V/TCwzEuqSlVJCVHta0+MAHF9+pB253h2ffABO37/+2aMzht7vv6anUuWkjx0CHGH2RQ6xjSluD596DrrEUq/+468CRMj5sd2uAhlYuwGbAxYznHL9hKRY4FMVX3zQA4gIjeJyEoRWblly5YDj7QBumcPVFURlbD/PcZAyYMH0/6ssyh4+BFKv/02ZPF4Tf1+Nk+/F19KCqmjRnkdjjGtUsIZZ5A2/jcUv/suWx5/3OtwIsoBJUYRmRLMarWU7e2gIyJRwGzgjgOJAUBVn1LVfqraLzU19UB306Cq4mKAfbpr1EZESJ9xL76kJKcLx549IYvJS4V/fIM9X3xB2rhx+Dp08DocY1qtlGHDSLricrY98SSFy5d7HU7EONAa4w1BrJMDZAYsZwB5AcsJwJHAeyKyHqe167LqBjjhxO8mxroa3wSKTk6m68wHKf/+e34YMZLVZ/Xn28P7svqs/q3ig11VVETBI4/Q9phjSBxwkdfhGNOqiQhdJk+m3c9/Tv5dd7P788+9Diki1DckXFEdj2KgaxD7/gQ4VER6ikgsMAhYVv2iqhaqaidVzVLVLOBj4CJVXXlwp9T0gq0xVos/6STiTz+NPR9/TGVeHqhSmZdH/uQpLT45bpk7l6rt2+k8+W4kKpRX4o0x4Mzk0+3xx4ju0oWcUbdSkZvrdUitXn3fbDuBQ1W1Q41HAtDgDL2qWgmMAt4BvgWWqurXIjJNRBqsari1yFnAcBHJqa1Fa3Pxl5QA1Nv4pqayVf/br0xLSymY/WiTxdXcSlf9jx0vvkTSoCtpe8QRXodjTMSITk4m84kFaHk5G0eMpKpkl9chtWr1Jcbngbom1HspmJ2r6luq2kdVe6vqDLdsiqouq2XdMwJri25NMkVV26tqhqp6Ni9LVVERAL4GGt8Eqqyj/1FlfoO/KcKSqrJ5+nR8CQmk3Xab1+EYE3Ha9OpFt9mzKVu7lrzx49GqKq9DarXq6+B/t6r+u47X7gxdSOHHX+zWGBvR0CQ6Pb1R5eGu6E9vsXvlSlLHjsWXlOR1OMZEpPa/OIXOkyZS8re/URChw1A2h0bdJBKRqSGKI6z5S9x7jLUMCVeXtLFjkLi4fcokLo60sS1vxq6qkl0UzJxJ3JFHknT5ZV6HY0xESxk6lOQhg9n+u4XsfPU1r8NplRrbeiIimyFWFZeAz4e0axf0NonZ2aRPn0Z0V7edks9Hl6lTSczODlGUobN1wXwqCwroMvluxOfzOhxjIl7nSZOIP/kk8qdOZffKsGuv2OI1NjFG5Lhf/uJiotq3b/SwZ4nZ2Rz6t7+SMXeOM0BAbMublKRs3Tq2L3qexMsupe3RR3sdjjEGkOhous2eTWxGBjmjbqV848aGNzJBa2xiPC4kUYS5quLioLtq1Kb9mWcS06M72559rkVNQqqqbL73XqLatSPt9tu9DscYE8CXmEjmgvmoKhtHjNjbrcwcvPr6Mb4b8HwigKpG5IB9tc3F2Bji89Fx+HBKv/qKPZ9+2oSRhVbxuyvY9c+PSB09muiOHb0OxxhTQ2xWFhmPPUb5+g3k3n4HWlnpdUitQn01xsAx1q4IdSDhrK65GBsj8eKL8SUlsW1hy5jK0r9nD5sfeIA2hx1G8qArvQ7HGFOH+BNPoMuUyez64AM2z5zpdTitQn2JseVc8wsxf3HJQdUYAaLatiVp8CBK/v53yr7/vokiC52tTz5JZX6+0+AmOtrrcIwx9UgeOJCUYdew4/nfs2PxEq/DafHqS4y9RGSZiCwPeL730VwBhoOq4qJGde6vS8qQIUh0NNsXLWqCqEKnfMMGtv9uIR0uyqZdv7AbutYYU4u08eOJP/00Nt17L7s+/tjrcFq0+qoCgXMnPhzqQMKZU2M8+FkkolNT6XBRNoWv/5HU224jOjm5CaJrepvvux+JiSFtXNjME22MaYD4fHR75BHWDxpEzm1jyFr8Mm169vQ6rBapvpFv3q/v0ZxBekn9fvwlJQ3OxRisjsOHo2Vl7Hj55SbZX1Mr/tvfKXn/fTqNGkVMWprX4RhjGsHXvj2ZTzyBREWRM2IkVYWFXofUItn0CA3w794NqvgaMYB4fdoceijxp57Kjhdfwl9W1iT7bCr+sjI233cfsb17k3L1VV6HY4w5ALEZGWTMnUN5bi45Y8agFRVeh9TiWGJsgN8dQLypaowAHa8dTtW2bRSF2RRU2373Oypycuhy911ITMsbjMAY42h33HGk33MPuz/6mE0zZrSo/tPhIOjEKCLxoQwkXFW5A4g35Uz17U46iTaHHca258Knw395Ti7bnnyKhPPOI/6kk7wOxxhzkJIuvYSON1zPzsVL2PFiUBMiGVeDiVFEThaRb3DmVEREjhaR+SGPLExUDyDemLkYGyIipFw7nPI1a9n1wQdNtt+DUfDgAxAVRec7x3sdijGmiaTefjvt+/dn8333UfLBh16H02IEU2OcDZwLbANQ1S+B00IZVDipHmapKbprBEq84AKi09LY9qz3Hf5LPviQ4hV/odPNNxPTQqfFMsbsT6Ki6DbzQdr06UPu2LGUrV3rdUgtQlCXUlW15gi1ETNDpt9NjAfbwb8miY0l+eqr2P3Rx5R++22T7rsx/OXlbJ4xg9gePUi5drhncRhjQiMqPp7M+fOQuDg23jyCyh07vA4p7AWTGDeKyMmAikisiIzDvawaCX6sMTZtYgRntApp147tzz3X5PsO1vZFiyhfv57Od99FVGysZ3EYY0InpmtXMufOoXLzZnJvHY2Wl3sdUlgLJjHeDNwCdANygGPc5QaJyHkiskpE1ojIhHrWu1xEVET6BZRNdLdbJSLnBnO8UPC7jW+ausYIzuj4SZddRuGf3qJi06Ym339DKjZtYuuCJ2jfvz/tTz212Y9vjGk+bY85hvQZM9i9ciX599wTNg3/wlGDiVFVt6rqUFXtrKppqnqVqm5raDsR8QHzgPOBvsBgEelby3oJwGjgXwFlfYFBwBHAecB8d3/Nzl9SDDExSJs2Idl/yrBrwO9nxwsvhGT/9SmYOROqqug8sc7fLMaYViQx+1d0GjmCwldfY/tz4T00pZcaHB1aRB6vpbgQWKmqb9Sz6fHAGlVd5+5nMc4wc9/UWG86MBMIHH9sALBYVcuA70Vkjbu/jxqKt6lVFTkzazR2kuJgxWZkkHDOOexYspSON4/A1755esXs+vhfFL31Np1GjSI2I6NZjmmM8V6nUaMoW7uOgpkzic3qQcKZZ3odUtgJ5lJqHM7l09Xu46dACnC9iDxaz3bdgMBGOzlu2V4iciyQqapvNnZbd/ubRGSliKzcsmVLEKfSeP7iYqI6NP1l1EAdrx2Ov7iYwtdeDelxqmlFBZtn3EtMt250vOH6ZjmmMSY8SFQUXR+4n7i+fcm7Yxylq/7ndUhhJ5jEeAhwlqrOUdU5wNnA4cAlwC/r2a62Ktbei9oiEoXTFeSOxm67t0D1KVXtp6r9UlNTa9nk4DlzMYY2MbY95hjaHnss2xc93ywTjW5/8UXKVq+h86SJRMXFhfx4xpjwEtW2LRnz5xHVvj05I0ZQua3Bu2MRJZjE2A0IvL4XD3RV1SqgvsE+c4DMgOUMIC9gOQE4EnhPRNYDJwLL3AY4DW3bbJpiLsZgpFw7nIrcXIr/8peQHqdyyxa2zplL/Gmn0v6ss0J6LGNM+Irp3JmMefOo3L6dnFG3ht3YzV4KJjHOBL4QkWdF5Dngc+Bhd4i4+r7FPwEOFZGeIhKL05hm7zyOqlqoqp1UNUtVs4CPgYtUdaW73iARaSMiPYFDgX8fwPkdNH9JcZN37q9NQv/+xHTvzrZnnw1pa7GChx9By8vpMmlSyO6bGmNahrZHHUnXBx5gz+efs2nKFGup6gqmVervgJOBP7qPX6jqM6q6S1V/U892lcAo4B2cfo9LVfVrEZkmIhc1cMyvgaU4DXX+DNzi1lCbXVVRcZPMxdgQ8flIueYaSr/8D3s+/zwkx9j92WcUvvEGKddeS2xWVkiOYYxpWTqcdy6pt42m8I1lbHv6Ga/DCQvBDiJeCuQD24FDRCSoIeFU9S1V7aOqvVV1hls2RVWX1bLuGW5tsXp5hrvdYar6dpBxNjl/cfPUGMEZ9DcqMZHtIRgmTquq2DRtOtFdutDp5l83+f6NMS1Xx5tvpsOFF7Jl1iyKVqzwOhzPBTOI+A3AP3Bqfve4/04NbVjhQauq8O/e3aQDiNcnql07kgcNovgvf6V8w4Ym3feOJUso++47Ok+4k6h27Zp038aYlk1ESJ9xL3FH/5S88XdS+k3NXnWRJZga423Az4ENqnomcCwQmr4RYcZfUj3qTfPUGAGShw5BoqPZvuj5Jttn5fbtbHn0MdqdeCIJ53o2iJAxJoxFxcWROXcuvqQkNo68hYqCAq9D8kwwibFUVUsBRKSNqn4HHBbasMJDKMdJrUtMWhodfvUrdr7+OlU7dzbJPrfMno1/925nAmJrcGOMqUN0aiqZC+ZTVVTktFQtLfU6JE8EkxhzRCQJp+HNChF5A4+6TjS3UM2s0ZCU4cPRPXvYsXjJQe9rz3/+w84/vErK1VfT5pBDmiA6Y0xrFveTn9DtoZmUfvUV+ZMmRWRL1WBapV6iqjtVdSowGfgdcHGoAwsHXtQYAeIO60P8Kaew/cUX8B/EKPjq97Np+r34OnWk0y0jmzBCY0xrltC/P2l33E7RW2+zdX7EzEu/V72JUUSiROS/1cuq+r6qLlPViJizZO89xmZqfBMo5dprqdqylaI3/3TA+9j56quUfvUVncePx9e++e6TGmNavpTrryfx4ovZOmcuRW971jHAE/UmRlX1A1+KSPdmiiesVBUVATRbd41A8aecTJs+fdh+gB3+q3buZMsjs2h73HF0+NWvQhChMaY1ExG6TLuHtscdR96Eiez56iuvQ2o2wdxjTAe+FpG/isiy6keoAwsHe+di7BD6Dv41iQgpw4dTtno1uz78v0Zvv+XxOVQVFdFl8t3W4MYYc0CiYmPJmPM40Z06kTPyFk/mjfVCMInxHuBXwDTgkYBHq+cvce8xenQZssOvLsSX2qnRHf5Lv/2WHYsXkzxkCHE/+UmIojPGRILolBQyn1iAf/duNo4ciX/3bq9DCrlgGt+8D6wHYtznnwCfhTiusFBVXIK0aYPExnpy/KjYWFKuuppd//wnpatWBbWNqrJp2nR8SUmkjr41xBEaYyJBm0MPpdvsWZR9t4q8Oyegfr/XIYVUMCPf3Aj8AXjSLeqG03Wj1fMXFzV7V42akq8ciLRty/Znnwtq/aJly9jz+eek3XE7Pg8uARtjWqf2p51G5zvHU7xiBVseq23++tYjmEuptwCnAEUAqroaSAtlUOGiqrik2btq1ORLSiLp0ksp/NOfqNhc/0gUVcXFbH7oYeKO/imJl1zSTBEaYyJF8jXXkHTFFWx78kkKl7XepibBJMaywO4ZIhJNLZMGt0b+4mLPa4wAKcOugcpKdrz4Yr3rbZ07j6pt2+hy92QkKtjx4Y0xJjgiQpcpk2l3/PHk33U3u0M0E5DXgvn2fF9EJgFtReQc4BVgeWjDCg9VJcVh0f8vtnt3Es4+mx1LltR547ts9Wq2v/ACSVdcQdujjmzmCI0xkUJiYuj22KNEd00nZ9StVOTmeh1SkwsmMU7AGTT8K+DXwFvA3aEMKlz4i8KjxghOh39/YSE7X3t9v9dU1Rnhpn17UseO8SA6Y0wkiU5OJnPBArS8nI0jRlJVssvrkJpUMIlxAPC8ql6hqper6tMaIYPnVZUU4+sQHomx3c+Ope3RR7N90SK0at85m4vffpvd//43qWPHEJ2c7FGExphI0qZXL7o9OpuytWvJGzduv++lliyYxHgR8D8R+b2IXOjeY4wI/uIST4aDq0vKtddSsXEjxX/9694y/65dbH5wJnF9+5J0xRUeRmeMiTTtTzmFzndNouS99yh4ZJbX4TSZYPoxXgscgnNvcQiwVkSeCXVgXtOKCrS0tFnnYmxIwjlnE5ORsU/Xja1PPEHl5s10nnw34vN5F5wxJiKlDBlC8pAhbF+4kJ2vvup1OE0iqKaLqloBvA0sBj7FubzaIBE5T0RWicgaEZlQy+s3i8hXIvKFiHwoIn3d8lgRedZ97UsROSPoM2oiVe4A4r4wqjGKz0fKNdew5/PP2f3555St+55tzy0i8ZJLaHfssV6HZ4yJUJ0nTST+lFPIn3oPuz/5xOtwDlowHfzPE5HngDXA5cAzOOOnNrSdD5gHnA/0BQZXJ74AL6nqUap6DDATqK6L3wigqkcB5wCPiEiz9j/wuwOIR4XJPcZqSZddCnFx/HDtday74AKorCTOWqEaYzwk0dF0mz2L2MxMcm4dTfkPP3gd0kEJJtkMxxnppo+qDlPVt1S1MojtjgfWqOo6tx/kYmrUNFW1KGAxnh/7R/YF/uquUwDsBPoFccwmU+UOIO51B/+aiv/2N6isRKtn1lalYOZDFC6PiB40xpgw5evQgcwF80HVaanqzmfbEgVzj3GQqv5RVcsAROQUEZkXxL67ARsDlnPcsn2IyC0ishanxjjaLf4SGCAi0SLSEzgOyKxl25tEZKWIrNyyZUsQIQWvegDxcGp8A1Aw+1Go3Pd3iZaWOuXGGOOh2B496Pb445Rv2EDu2NvRymDqUOEnqMuTInKMiMwUkfXAvcB3wWxWS9l+3TxUdZ6q9gbu5Mf+kQtxEulK4FHgn8B+77CqPqWq/VS1X2pqajCnErTqXztezMVYn8r8/EaVG2NMc4o/4Xi6/HYKuz78kM0PzvQ6nANSZ9cLEekDDAIGA9uAJYCo6plB7juHfWt5GUBePesvBhYAuJdqxwbE8k9gdZDHbRL+IrfGGGYDcUenp1OZt//bGJ3e4G1fY4xpFslXXEH52nVsf+452vTuRfKgQV6H1Cj11Ri/A/oD2ar6C1WdAzSmB+cnwKEi0lNEYnGS7D6jzorIoQGLF+ImPxFpJyLx7vNzgEpV/aYRxz5oXs/FWJe0sWOQuLh9yiQujjQb8cYYE0bSfjOO+NNPY9P0e9n18cdeh9Mo9SXGy4BNwN9F5GkR6U/tl0dr5db6RgHvAN8CS1X1axGZJiIXuauNEpGvReQL4HZgmFueBnwmIt/iXGK9ulFn1QSqG99EhVliTMzOJn36NKK7dgURort2JX36NBKzs70OzRhj9hKfj26PPEKbXr3IGX0bZd9/73VIQZOGRndza24X41xSPQtYBLyuqu+GPrzg9evXT1euXNlk+9t8/wPseOUVfvLZp022T2OMiTTlObmsHzgQX0ICWUuX4EtMbNbji8inqtqoXg3BtErdpaovquqvcO4TfoEzsHirVlUcHjNrGGNMSxab0Y2MOY9TkZdHzpgxaEWF1yE1qFGd5lV1u6o+qapnhSqgcOEvLg67zv3GGNMStTvuOLpMn8bujz5m04wZhPs8FBEzIHhjOXMxWmI0xpimkHTxxZSvXce2p5+mTe9DSLn6Kq9DqpNN814Hf3FJ2MzFaIwxrUHq2DG0P7s/m++/n5IPPvA6nDpZYqxDVXFR2HXuN8aYlkyiouj24IO06dOH3LG3U7Zmjdch1coSYx2cGmN4de43xpiWLio+nswF85G4ODaOGEnljh1eh7QfS4x18BcXW43RGGNCICY9ncx5c6ncvJncW0ej5eVeh7QPS4y18JeVoRUVYTeAuDHGtBZtjz6a9PvuY/fKleRPvSesWqpaq9Ra7J2L0WqMxhgTMom/upDydevYOn8+bQ45hI7XXet1SIAlxlrtnYsxzAYQN8aY1qbTqFsoW7eOgoceIjYri4Szgp2nInTsUmotfpyL0WqMxhgTShIVRdf77yPuiCPIGzeO0lWrvA7JEmNtfpyL0e4xGmNMqEW1bUvGvHlEtW/P+mHDWH36GXx7eF9Wn9WfwuXLmz+eZj9iC+CvnlnDEqMxxjSLmM5pJA0ahO4spHLzZlClMi+P/MlTmj05WmKsRVWx0/jGaozGGNN8dv7hD/uVaWkpBbMfbdY4LDHWwmqMxhjT/Crz8xtVHiqWGGvhLykGEaLi470OxRhjIkZ0enqjykPFEmMtqopLiIqPR6Ls7THGmOaSNnYMEhe3T5nExZE2dkyzxmH9GGvhLyqyuRiNMaaZJWZnA1Aw+1Eq8/OJTk8nbeyYveXNxRJjLapKSmwuRmOM8UBidnazJ8KaQnqtUETOE5FVIrJGRCbU8vrNIvKViHwhIh+KSF+3PEZEFrmvfSsiE0MZZ03+4mJreGOMMREqZIlRRHzAPOB8oC8wuDrxBXhJVY9S1WOAmcAst/wKoI2qHgUcB/xaRLJCFWtNVSXF+GzUG2OMiUihrDEeD6xR1XWqWg4sBgYErqCqRQGL8UD18OoKxItINNAWKAcC1w0pf5HVGI0xJlKFMjF2AzYGLOe4ZfsQkVtEZC1OjXG0W/wHYBeQD/wAPKyq22vZ9iYRWSkiK7ds2dJkgfuLi/FZ4xtjjIlIoUyMUkvZfhNuqeo8Ve0N3Anc7RYfD1QBXYGewB0i0quWbZ9S1X6q2i81NbVJglZVqkpKbC5GY4yJUKFMjDlAZsByBpBXz/qLgYvd50OAP6tqhaoWAP8H9AtJlDXonj1QVWVzMRpjTIQKZWL8BDhURHqKSCwwCFgWuIKIHBqweCGw2n3+A3CWOOKBE4HvQhjrXjazhjHGRLaQ9WNU1UoRGQW8A/iAhar6tYhMA1aq6jJglIicDVQAO4Bh7ubzgGeB/+Jckn1WVf8TqlgD+d3EaI1vjDEmMoW0g7+qvgW8VaNsSsDz2+rYrgSny0azsxqjMcZENhsMtAZ/iTuzhjW+McaYiGSJsYaqouq5GK3xjTHGRCJLjDXsnYuxQwePIzHGGOMFS4w1+Evce4w2JJwxxkQkS4w1VBWXgM+HtGvndSjGGGM8YImxBn9xEVHt2yNS28A9xhhjWjtLjDVUFZdYVw1jjIlglhhrsLkYjTEmsllirMHmYjTGmMhmibEGf3GJ1RiNMSaCWWKsoaq4yO4xGmNMBLPEWIPVGI0xJrJZYgygfj/+khKbi9EYYyKYJcYA/t27QRWfDSBujDERyxJjAL87gHhUB0uMxhgTqSwxBqhyBxC3xjfGGBO5LDEGqB5A3OZiNMaYyGWJMUBVsTuzhjW+McaYiBXSxCgi54nIKhFZIyITann9ZhH5SkS+EJEPRaSvWz7ULat++EXkmFDGCs5wcIB11zDGmAgWssQoIj5gHnA+0BcYXJ34Arykqkep6jHATGAWgKq+qKrHuOVXA+tV9YtQxVrtxxqjJUZjjIlUoawxHg+sUdV1qloOLAYGBK6gqkUBi/GA1rKfwcDLIYsygN9tfGM1RmOMiVyhTIzdgI0Byzlu2T5E5BYRWYtTYxxdy36upI7EKCI3ichKEVm5ZcuWgwq2cPlytj39NABrL7iAwuXLD2p/xhhjWqZQJsbaZvrdr0aoqvNUtTdwJ3D3PjsQOQHYrar/re0AqvqUqvZT1X6pqakHHGjh8uXkT56Cv8SpMVbm5ZM/eYolR2OMiUChTIw5QGbAcgaQV8/6i4GLa5QNohkuoxbMfhQtLd2nTEtLKZj9aKgPbYwxJsyEMjF+AhwqIj1FJBYnyS0LXEFEDg1YvBBYHfBaFHAFTsIMqcr8/EaVG2OMab2iQ7VjVa0UkVHAO4APWKiqX4vINGClqi4DRonI2UAFsAMYFrCL04AcVV0XqhirRaenU5m3f2U2Oj091Ic2xhgTZkKWGAFU9S3grRplUwKe31bPtu8BJ4YsuABpY8eQP3nKPpdTJS6OtLFjmuPwxhhjwkhIE2NLkZidDTj3Givz84lOTydt7Ji95cYYYyKHJUZXYna2JUJjjDE2VqoxxhgTyBKjMcYYE8ASozHGGBPAEqMxxhgTwBKjMcYYE0BUa5vQouURkS3AhoPYRSdgaxOF09xaauwtNW5oubG31Lih5cbeUuOGlht7YNw9VLVRg2m3msR4sERkpar28zqOA9FSY2+pcUPLjb2lxg0tN/aWGje03NgPNm67lGqMMcYEsMRojDHGBLDE+KOnvA7gILTU2Ftq3NByY2+pcUPLjb2lxg0tN/aDitvuMRpjjDEBrMZojDHGBLDEaIwxxgSwxAiIyHkiskpE1ojIBK/jqYuIZIrI30XkWxH5WkRuc8unikiuiHzhPi7wOtbaiMh6EfnKjXGlW5YiIitEZLX7b7LXcQYSkcMC3tcvRKRIRMaE63suIgtFpEBE/htQVut7LI7H3c/9f0TkZ2EW90Mi8p0b2+sikuSWZ4nInoD3/gmv4nbjqS32Oj8fIjLRfc9Xici53kRdZ9xLAmJeLyJfuOVh857X8z3YdJ9zVY3oB+AD1gK9gFjgS6Cv13HVEWs68DP3eQLwP6AvMBUY53V8QcS/HuhUo2wmMMF9PgF40Os4G/isbAJ6hOt7DpwG/Az4b0PvMXAB8DYgOJOC/yvM4v4lEO0+fzAg7qzA9bx+1BF7rZ8P9//rl0AboKf73eMLl7hrvP4IMCXc3vN6vgeb7HNuNUY4HlijqutUtRxYDAzwOKZaqWq+qn7mPi8GvgW6eRvVQRsALHKfLwIu9jCWhvQH1qrqwYywB+cgAQAABApJREFUFFKq+g9ge43iut7jAcDz6vgYSBKR9OaJdF+1xa2q76pqpbv4MZDR7IEFoY73vC4DgMWqWqaq3wNrcL6Dml19cYuIAAOBl5s1qCDU8z3YZJ9zS4zOG7oxYDmHFpBsRCQLOBb4l1s0yr1MsDDcLkcGUOBdEflURG5yyzqraj44H3ggzbPoGjaIfb8oWsJ7DnW/xy3ps38dzq/+aj1F5HMReV9ETvUqqAbU9vloKe/5qcBmVV0dUBZ273mN78Em+5xbYnSq1zWFdR8WEWkPvAqMUdUiYAHQGzgGyMe5BBKOTlHVnwHnA7eIyGleBxQsEYkFLgJecYv+v737ebWqigI4/l1kBEUGRoOgBAWbBwkNHARF5MOCcqIIvkGTxk2E3v/QRAJBhCAMmiS9eZMCB0aSPyJ/D0SUJxmE4eSVy8HeF7Z2730Qj/Y5l+8HLve8zXmwWCzOOm/v/c4ZS87nGUXtR8QK8Ddwsg7dAbZn5uvAp8DXEbG1V3wzzKqPUeQcOMjjN4GDy/mU6+DMU6eMzc25jbHcPbza/PwKcLtTLBuKiKcpxXAyM78FyMy1zPwnMx8Cx+k0NbORzLxdv+8Cpyhxrk2mNer33X4RzrUXOJuZazCenFezcjz42o+IZWAfcCjrglGdhrxXj3+mrNO91i/Kf5tTH2PI+RbgI+CbydjQcj7tOsgm1rmNEX4CdkXEjvpXwQFgtXNMU9V5/xPAb5n5eTPezpd/CFx88nd7i4jnIuL5yTFlY8VFSq6X62nLwHd9ItzQY3fQY8h5Y1aOV4HDddfem8Cfk6moIYiI94AjwAeZ+aAZfykinqrHO4FdwI0+UU43pz5WgQMR8UxE7KDEfub/jm8D7wCXMvPWZGBIOZ91HWQz67z3DqMhfCi7lq5Q7oJWesczJ849lCmA88Av9bMEfAVcqOOrwMu9Y50S+07KbrxzwK+TPAMvAt8DV+v3tt6xTon9WeAe8EIzNsicU5r3HWCdcqf88awcU6aYvqh1fwF4Y2BxX6OsDU1q/Vg9d3+toXPAWeD9AeZ8Zn0AKzXnl4G9Q4q7jn8JfPLEuYPJ+Zzr4KbVuY+EkySp4VSqJEkNG6MkSQ0boyRJDRujJEkNG6MkSQ0bo7QgIuKv5nipvmVge8+YpDHa0jsASZsrIt4GjgLvZubN3vFIY2NjlBZIfbjzcWApM6/3jkcaI//BX1oQEbEO3AfeyszzveORxso1RmlxrAOnKY8kk/Qf2RilxfGQ8nLZ3RHxWe9gpLFyjVFaIJn5ICL2AT9GxFpmnugdkzQ2NkZpwWTmH/WVTT9ExO+ZOdRXeUmD5OYbSZIarjFKktSwMUqS1LAxSpLUsDFKktSwMUqS1LAxSpLUsDFKktR4BFe171/b2yPjAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 504x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "P3_1([1, 5, 12, 24, 48, 96, 120, 192], train_data, train_labels, dev_data, dev_labels)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 79,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "For an alpha of 1e-10, the F-1 score is: 0.678\n",
      "For an alpha of 0.0001, the F-1 score is: 0.664\n",
      "For an alpha of 0.001, the F-1 score is: 0.66\n",
      "For an alpha of 0.01, the F-1 score is: 0.641\n",
      "For an alpha of 0.1, the F-1 score is: 0.61\n",
      "For an alpha of 0.5, the F-1 score is: 0.551\n",
      "For an alpha of 1.0, the F-1 score is: 0.521\n",
      "For an alpha of 2.0, the F-1 score is: 0.48\n",
      "For an alpha of 10.0, the F-1 score is: 0.249\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAbkAAAEWCAYAAAD7HukTAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nO3deZgU5bn+8e89AwgiogiKgooaXHABDaKJRhSNoiYuUXEhcYlLjBpjlmPUJCeaxSTnxGwnJsb4c4uicY3ExH03UQMYUBEXoiK4REARRUVgnt8fb400wyw9TPdUT/f9ua66pqu6uurpmp5+5n3qrXoVEZiZmVWjurwDMDMzKxcnOTMzq1pOcmZmVrWc5MzMrGo5yZmZWdVykjMzs6rlJGdmnUrSu5I2zTuORpJ2lzSn1OtaZXCSMyTdL+ktSavlHUspZO/ng+zLtHH6RAvr7iHpPklvS3qpiG0fL+kZSe9I+o+kv0rqU/I30Q6SLpf0YZP3e3gJtvt7SVc2s3w7SYsl9VuV7UbEGhHxQkfjay9Jx0oKSeM6e9+WHye5GidpCPApIIADyrSPbuXYbhtOy75MG6dHWlhvEXAp8F9tbVDSaOB84MiI6ANsBVxXsojp0LH6nybv90+rsO/6JosuBz4nqXeT5UcDt0bEm+3cfh6fg0LHAG9mP61GOMnZ0cCjpC+0j/74Je0s6fXCLz5JB0t6IntcJ+ksSf+WNF/SdY3/2Usakv3HfLykl4F7s+XXZ9t8W9KDkrYu2PY6kv4iaaGkSZJ+KOnhgue3lHSXpDclPVuq/8Yj4p8R8UegmJbFjsAjEfGv7LVvRsQVEfFOFmMvSRdImpW9x4cl9cqeO0DSdEkLspbmVgXv7SVJ38qO7SJJ3SRtIOlGSXMlvSjp9FV5f5K2yva3INv/AQXPXS7pd5L+JmkRsEeTY/MI8ApwSMFr6oGjgCuy+VGSHsm2/5qk30jqUbB+SDpV0vPA8wXLPpY97ivpyux9zpL0HUl12XPnSrqqYFuNn6tu2fyxkl7IWtUvShrfynHYGBgNnATsI2m9VtZ9SdLZkp5WqnBcJqlnk3W+IemN7D0fV7B8f0n/yj7HsyWdW/BcT0lXZX8vC7LPeYtxWGk4ydnRwNXZ9NEff0Q8SmrljClY9yhgQvb4dOAg0hfHBsBbwIVNtj2a1NrZJ5u/DRgKrAs8nu2z0YXZ/gaSkm1hwu0N3JXte13gSOC3hUmykzxGOkbnSdpFK5d3fwZ8HPgk0A84E2iQtDlwDXAGMAD4G/CXwmRAek/7A2sBDcBfgGnAIGBP4AxJ+9AOkrpn27mTdNy+AlwtaYuC1Y4CfgT0AR5eaSNwJekz0mgvoDvpdwmwDPga0B/4RBbrKU22cRCwEzCsme3/H9AX2JT0eTkaOK6Z9Zq+t97Ar4F9s1b1J4GprbzkaGByRNwIzABaTIiZ8aTP7WbA5sB3Cp4bmMU8CDgeuFDS2tlzi7J9rUX6fX5Z0kHZc8dkr9sQWAc4GXi/rfdqHRQRnmp0AnYFlgD9s/lngK8VPP9D4NLscR/SH/DG2fwMYM+CddfPttUNGEIqf27ayr7XytbpC9Rnr92iyb4fzh4fDjzU5PW/B77XwrbvB94DFmTT40Uci72Al4pYb19S4lgAvAv8PIu/jvSFNbyZ13wXuK5gvo7UQto9m38J+GLB8zsBLzfZxtnAZS3EdDnwQcH7nZct/xTwOlBXsO41wLkFr7uyjfe7Ufa7GZzNXw38qpX1zwBuLpgPYEyTdQL4WHbcFgPDCp77EnB/9vhc4KqC5xo/V92A3tl7PQToVcTv7XngjIJjOa3gud2BOQXzLwEnF8zvB/y7YN33gW4Fz78B7NzCfn8J/CJ7/EXgH8B2Hf3b9VT85JZcbTsGuDMi5mXzE1jxfMUE0jmZ1YDPkZLFrOy5jYGbs7LLAlLSWwYUll9mNz6QVC/pJ1l5cyHpiwRSC2AA6YtrdnOvzfa1U+O+sv2NJ/1H3ZLTI2KtbNohi+EcLe+YcVGrR6YFEXFbRHyW1FI7EDgWOCF7Hz2Bfzfzsg2AWQXbaMje36CCdZq+3w2avN9zWPHYNvWzgvfbv2C/s7P9NZrVyn5XEhEvAw8Cn5e0BqlVdkXj85I2l3SrUhl6IemcZf8mm2lpH/2BHhQcm2biaymuRaR/fk4GXlPqALRlc+tK2gXYBLg2WzQB2FbSiFZ2URjzLNKxbDQ/IpYWzL8HrJHtayeljkxzJb2dxdd4PP4I3AFcK+lVSf+TtbatjJzkalR2rmgcMDr7gnqdVHYaLmk4QEQ8TfoD35cVS5WQvgT2LfhiXSsiekbEKwXrFA5xcRQpKexFar0NaQwFmAssBQYXrL9hk3090GRfa0TEl9vzniPi/FjeMePk9ry2mW01RMQ9pPON2wDzSK2pzZpZ/VVS4gJAkkjvr6VjNRt4scn77RMR+7UzzFeBDRvPcWU2amW/LbmCVII7JIvr8YLnfkeqAAyNiDVJyVhNXt/SPuaRWokbFywrjG8RsHrBcyv8UxMRd0TEp0lVhGeAP7Swn2OymKZmn/PHsuVHt7A+rPj524h0LIsxAZgIbBgRfYGLsn0TEUsi4ryIGEYqr36mjRisBJzkatdBpJbXMGBENm0FPMSKf3gTSOffdgOuL1h+EfCj7IQ+kgZIOrCV/fUhlabmk764zm98IiKWATcB50paPfuPvDCGW4HNJX1BUvds2lEFnTdWlVIHmp6k80zKOgf0aGHdAyUdIWltJaNI55EezVpLlwI/V+o0Ui/pE1kr+Dpgf0l7Zv+5fyM7Fv9oIax/AguVOqP0yra1jaQd2/n2HiMlijOzY7Y78FmWt2iKdSPpS/88ClpxmT7AQuDd7PdW9D8e2e/9OtLnqE/2Wfo60NjZZCqwm6SNJPUllRkBkLSeUmee3qRj+S7p87yC7Hc7jtThZETB9BVgvFru8XmqpMFKnanOAYrtrdoHeDMiPsg+H0cVxLKHpG2VOu8sJCX4lWK2Esu7Xuopnwm4HbigmeXjSOdxumXzG5E6Qvy1yXp1pC+kZ4F3SGW687PnhpCdOylYfw3glmzdWaQkFsDHsucHAH8l/fFPAn4K3FPw+i2y5+eSEuW9wIgW3tv9wAlFHofdszgKp/tbWHc34B5SC+Qd4DngzILne5HOwbwCvE0q8/XKnjsYeDpb/gCwdcHrXgL2arKvDUjnz14ndep5tOk6BeteDvywhee2zvb3drb/g4t5XQv7WAZs0MwxeYaUZB4Cvk92LjV7/qPfcXPLgLVJSW0uqQX736x4DvFC0rm3mcCJLD8nt37B+1qQ/c6HNRP3EcBrQPcmy3tmv8fP0Pw5ubOz47WAlNhXL/i8zGmyrY9+f8ChpM/3O6R/zn5Ddl6R1LnoWdI/Hv8hdZzp1tIx91SaSdnBN6sokn4KDIwIX9NknUrppgAnRMTdecdiHedypVUEpevgtisoAx4P3Jx3XGbWteV9BwKzRn1I5bkNSF2yLyCVN83MVpnLlWZmVrVcrjQzs6rV5cqV/fv3jyFDhuQdhpmZVZApU6bMi4gBTZd3uSQ3ZMgQJk+enHcYZmZWQSTNam65y5VmZla1nOTMzKxqOcmZmVnVcpIzM7Oq5SRnZmZVq8v1ruyQi3aF159cefnAbeHk5gZFNjOzrqy2WnKDR0F9k1FU6nuk5WZmVnVqK8mNPhPU5C2rDkZ/K594zMysrGoryfUZCCPGL5+v75Hm+6yXX0xmZlY2tZXkILXmGrkVZ2ZW1WovyfUZCPWrpcduxZmZVbXa6l3ZqHsv6LW2W3FmZlWutpJc00sILtg8/fQlBGZmVam2ypVvv9K+5WZm1qXVVpLbfJ8Wlo/t3DjMzKxT1FaS2+tcUP2Ky1SflpuZWdWprSTXZyBsN27FZdsd7h6WZmZVqraSHKzYanMrzsysqtVekuszEHqulR67FWdmVtVqL8kB9NsEVuvrVpyZWZWrzSS32pqw7pZuxZmZVbnaTHLdV4cl7+UdhZmZlVlt3/Hk3L7pp+94YmZWlWqrJedBU83MakptJTkPmmpmVlNqK8k1DpraeNeT+u4ebsfMrIrVVpKD1Jqra7y1l9yKMzOrYrWX5PoMTBeBA6y3tVtxZmZVrPaSHMCY76TLCLr3zjsSMzMro9pMco03an79SWhoyDsaMzMrk9pMcgCDd4TFb8P85/OOxMzMyqSGk1x2bdycSfnGYWZmZVO7SW6dj0HPvjD7n3lHYmZmZVK7Sa6uLpUs50zOOxIzMyuT2k1ykJLcG0/D4nfyjsTMzMrASY6AV6bkHYmZmZVBbSe5QR9PP935xMysKtXWUDuFCofdufeHaQIPu2NmVkVqtyXnYXfMzKpe7SY5D7tjZlb1ajfJNQ6781FrTh52x8ysypQ1yUkaK+lZSTMlndXCOuMkPS1puqQJ5YxnJSu05gI23b1Td29mZuVVtiQnqR64ENgXGAYcKWlYk3WGAmcDu0TE1sAZ5YqnWR8NolqXRiV45DcQ0akhmJlZ+ZSzJTcKmBkRL0TEh8C1wIFN1jkRuDAi3gKIiDfKGE/zRp8JG+2czsXNfgxm3t3pIZiZWXmUM8kNAmYXzM/JlhXaHNhc0t8lPSppbHMbknSSpMmSJs+dO7e0UfYZCMfdBjufAmttDPf+wK05M7MqUc4kp2aWNc0e3YChwO7AkcAlktZa6UURF0fEyIgYOWDAgJIHmiLpAbufDa9NgxkTy7MPMzPrVOVMcnOADQvmBwOvNrPOLRGxJCJeBJ4lJb18bDcO+m8B9/4IGpblFoaZmZVGOZPcJGCopE0k9QCOAJo2kf4M7AEgqT+pfPlCGWNqXV097HEOzHsWnrwhtzDMzKw0ypbkImIpcBpwBzADuC4ipkv6vqQDstXuAOZLehq4D/iviJhfrpiKstUB6dZe958Py5bkGoqZmXWMoot1shg5cmRMnlzmMeCeuwMmjIPP/BJGHlfefZmZWYdJmhIRI5sur907nrRm6N7pHpYP/i8s+SDvaMzMbBU5yTVHgjHfgYWvwJTL8o7GzMxWkZNcSzYdDZvsBg9dAB8uyjsaMzNbBU5yrRnz37BoLjx2Ud6RmJnZKnCSa82GO8LmY+Hvv4L3F+QdjZmZtZOTXFv2+DZ88DY8cmHekZiZWTs5ybVl/e1g2EHw6G9h0by8ozEzs3ZwkivGHufAkvfg4V/kHYmZmbWDk1wxBmwB2x0Oky6Bha/lHY2ZmRXJSa5Yo78FDUvhoZ/lHYmZmRXJSa5Y/TaBHY6GKVfAW7PyjsbMzIrgJNceu/0XqA4e+GnekZiZWRGc5NpjzQ1gxxNg2jUw7/m8ozEzszYUneQk9S5nIF3Grl+Dbr3gvvPzjsTMzNrQra0VJH0SuARYA9hI0nDgSxFxSrmDq0hrDIAeq8P0m9JUaOC2cPLD+cRlZmYrKaYl9wtgH2A+QERMA3YrZ1AVb+g+Ky+r75GG5zEzs4pRVLkyImY3WbSsDLF0HXt+F+qaNIJVly4zMDOzilFMkpudlSxDUg9J3wRmlDmuytZnIAw/ClCaV12a77NermGZmdmKiklyJwOnAoOAOcCIbL62jfk2dFstPY4GeH2a74ZiZlZhWk1ykuqBL0TE+IhYLyLWjYjPR8T8ToqvcvUZCCPGp1bcpnvAGzPg97vBiw/lHZmZmWVaTXIRsQw4sJNi6XpGnwkb7QwH/x5OvBd69oUrD4CHfwkReUdnZlbzFG18GUv6EdAX+BOwqHF5RDxe3tCaN3LkyJg8eXIeu27b4nfgltPg6T/Dlp+Bg36bEp+ZmZWVpCkRMXKl5UUkufuaWRwRMaZUwbVHRSc5SC24R38Hd30X1toIxv0RBm6Td1RmZlWtpSTX5sXgEbFHeUKqUhJ84hQYtANcfyxcshd85ucw4qi8IzMzqzlt9q6U1FfSzyVNzqYLJLkG15aNdoYvPQiDR8Kfvwx/+Sos+SDvqMzMakoxlxBcCrwDjMumhcBl5QyqaqyxLnzhz+l+l1Muh0v38TA9ZmadqJgkt1lEfC8iXsim84BNyx1Y1ajvBnudC0dMgDdfTJcZPH9X3lGZmdWEYpLc+5J2bZyRtAvwfvlCqlJb7g9fuh/6bghXHwb3/ggaavvuaGZm5VZM78oRwBWkywgA3gKOzW7U3OkqvndlW5a8D3/9Bky9uvnnPZKBmVm7daR35VRguKQ1s/mFZYivdnTvBQdeCG+9BLP+vuJzHsnAzKykiuldeb6ktSJiYUQslLS2pB92RnBVS4JDL01Jrelyj2RgZlYyxZyT2zciFjTORMRbwH7lC6lG9BkI238B6rsvX7Z0Cfzj17DItwY1MyuFYpJcvaTVGmck9QJWa2V9K9boM0H16XG31WDYgfDob+FXw+H+n6TbhJmZ2SorJsldBdwj6XhJXwTuInVEsY4qHMlgxOdh3OXw5Udgs93h/h+nZPeP3/gicjOzVdRm70oASWOBvbLZuyLijrJG1You37uyqXdehxuOg0MvX3HQ1VemwD0/gBfugzUHpXN1I8an6+7MzGwFq3yD5oINrAPsBrwcEVNKHF/Rqi7JteXFB+Hu8+CVydBvszRY67CDoa6YRriZWW1oKcm1+E0p6VZJ22SP1weeAr4I/FHSGWWL1Fa0yW5wwt1wxDXpvN0NX4SLd4Pn7vSYdWZmbWitObBJRDyVPT6OVKb8LLATKdm1SdJYSc9KminprGaeP1bSXElTs+mEdr+DWiDBlvuli8QPvhg+WAgTDoPL9oVZj+QdnZlZxWotyS0peLwn8DeAiHgHaGhrw5LqgQuBfYFhwJGShjWz6p8iYkQ2XVJ05LWorh6GHw6nTYb9L0j3wrxsLFx1KLz2RN7RmZlVnNaS3GxJX5F0MLADcDt8dAlB91Ze12gUMDO7qfOHwLXAgR0N2IBuPWDHE+D0f8Fe58GcSfD7T8H1x8G8mXlHZ2ZWMVpLcscDWwPHAocXXBC+M8UNtTMImF0wPydb1tQhkp6QdIOkDZvbkKSTGsezmzt3bhG7rhE9Voddz4CvToNPfROeuwMuHAUTvwJvv5J3dGZmuSu6d2W7NywdBuwTESdk818ARkXEVwrWWQd4NyIWSzoZGBcRY1rbbs31rmyPd9+Ahy6AyZcCglEnprHsevfPOzIzs7Jqd+/KEpgDFLbMBgOvFq4QEfMjYnE2+wfg42WMp/qtsS7s+1P4yhTY9tDld0+578eps4qZWY0pZ5KbBAyVtImkHsARwMTCFbJLExodAMwoYzy1Y62N4KDfwimPwmZj4IGfFNw9xUMBmlntKFuSi4ilwGnAHaTkdV1ETJf0fUkHZKudLmm6pGnA6aTzf1YqA7aAw/8IJ94HG4yAO78Nv94BplwOy5bmHZ2ZWdmt0jk5Sf8dEd8vQzxt8jm5DnjxIbgn643ZbzPY4xzY+nO+e4qZdXkdvq1Xk429HBEblSSydnKS66AIeO72dF/MN6ankcjH/Dfc+314/cmV1/dI5WbWBbR7ZHBJLfVUENCrVIFZJ5Ngi31h6N7w1I1w34/S3VPWWA/qukFDQRnTI5WbWRfXWp1qATA0ItZsMvUBXuuk+Kxc6uphu3HZ3VN+Dg0NKyY4SEMAeaRyM+vCWktyVwIbt/DchDLEYnmo7w47Hg9nPAGDd1zxuXW3hmUf5hOXmVkJlO1i8HLxObkyeud1+NV2sHRxasVFA6A0EsKI8bDVZ9NdVszMKkxJLgaXdG7JIrLK02dgGqFcdfDx4+CrT8DuZ8NbL8HNJ8HPNodbTksjH3Sxf47MrDa1qyUn6fGI2KGM8bTJLbkya26k8oYGePkRmDoBpt8MSxZBv01h+FEw/AhYq9lbjpqZdZqSXEIg6V8RsX1JI2snJ7mcLX4XZvwFpl4NLz2Ey5lmVglKleTqIqLNseTKyUmugrw1C6ZdmxLeglnQow9sfRBs/3nYcKd0uYKZWSdod5KTdGdE7J09PjsiflzmGIviJFeBWipnjjgKtnM508zKb1WS3EelyUo4F9fISa7CLX4XZkxMCa+xnLnp6FTO3PIzLmeaWVm0+44ngLvPWfuttkZqwY04KvXKnHZtSng3nZjKmdscnBKey5lm1glaa8ktAB4k3cbrU9njj0TEAc29rtzckuuCGhrg5X9k5cw/u5xpZiW3KuXK0a1tMCIeKFFs7eIk18W5nGlmZVDSUQjy5CRXRT4qZ14NC16G1daErRvLmaNczjSzojnJWeVqtpy5WSpnDj8C+g7OO0Izq3BOctY1LH4Xnr4lJbxZD5PKmbtn5cz9Xc40s2Z1OMlJ6h0Ri0oeWTs5ydWQN19M5cxpE1zONLNWrXKSk/RJ4BJgjYjYSNJw4EsRcUp5Qm2dk1wNamiAWX9Prbun/wxL3nM508xW0JEk9xhwKDCx4OLwpyJim7JE2gYnuRq3+B14eqLLmWa2glW5GPwjETFbK5aGlpUqMLN2Wa0PbD8+TY3lzKkT4KYTXM40s5UUk+RmZyXLkNQDOB2YUd6wzIrQbxPY42wY/a3l5cwnr4fHr4B1Prb8YvO+g/KO1MxyUky5sj/wK2Av0t1P7gS+GhHzyx/eylyutFY1V87cbI/l5czuvfKO0MzKwJcQWO1584WsnHkNvJ31ztzmcynhDd7R5UyzKtKRjie/bmbx28DkiLilRPEVzUnO2q2hIbXqpk5I1+Atec/lTLMq05EkdzGwJXB9tugQYDqwIfBCRJxR4lhb5SRnHbL4nYKLzf+Oy5lm1aEjSe5eYO+IWJrNdyOdl/s08GREDCtDvC1ykrOSWamc2TcbCujzMHiky5lmXUhHLiEYBPQmlSjJHm8QEcskLS5hjGadq9+msMc5MPqs5eXMJ66DKZfDOkOXX2y+5gZ5R2pmq6iYJPc/wFRJ95N6V+4GnC+pN3B3GWMz6xx1dbDJbmna73/TTaKnToB7zoN7fwCb7pESnsuZZl1OUb0rJa0PjCIluX9GxKvlDqwlLldap5n/7+zemdfA27OzcmZj70yXM80qSYcuIZC0NjAU6Nm4LCIebPkV5eMkZ52uoSEN8NrYO3Pp+y5nmlWYjnQ8OQH4KjAYmArsDDwSEWPKEWhbnOQsVx8sXN478+V/gOpczjSrAB1Jck8COwKPRsQISVsC50XE4eUJtXVOclYxXM40qxgd6V35QUR8IAlJq0XEM5K2KEOMZl3LOpvBmG/D7mcvL2dOuxamXOZyplmFKKYldzNwHHAGMAZ4C+geEfuVP7yVuSVnFe2DhWnMu6kT4OVHUjlzszEp4W2xP3Tv2fY2zKzdSnLvSkmjgb7A7RHxYQnjK5qTnHUZ8/+dSplTr4GFc6BnX9jmkFTOHPRxlzPNSmiVkpykOuCJvAZIbY6TnHU5DQ3w0oNZ78yJqXdm/82ze2ce7nKmWQm0lOTqWntRRDQA0yRttIo7HSvpWUkzJZ3VynqHSgpJKwVo1uXV1aXRyz93MXzzOTjg/2D1deDuc+EXW8NVh8BTN8KSD3IO1Kz6FNPxZH1guqR/AosaF0bEAa29SFI9cCHpHpdzgEmSJkbE003W60MaiPWxdsZu1vX0XBN2ODpNheXMG77ocqZZGRST5M5bxW2PAmZGxAsAkq4FDgSebrLeD0i3DvvmKu7HrGtaZzMY8x3Y/Zzl5cyp18DkS6H/FgXlzPXzjtSsy2q1XAkQEQ8AL5F6VD4ATAIeL2Lbg4DZBfNzsmUfkbQ9sGFE3NrahiSdJGmypMlz584tYtdmXUiz5cx+cPf34BfD4KpD4ambXM40WwVttuQknQicBPQDNiMlqouAPdt6aTPLPurlknVq+QVwbFsxRMTFwMWQOp60tb5Zl9W0nDl1Qipp3nBcVs48NCtn7uByplkR2mzJAacCuwALASLieWDdIl43hzSwaqPBQOGNnfsA2wD3S3qJdLuwie58YpZZZzPY87twxpPwhT/D0H1g6tVwyRi4cCd4+Jew8LW8ozSraMUkucWF18Rlg6YW05qaBAyVtImkHsARwMTGJyPi7YjoHxFDImII8ChwQET4+gCzQnX1afTyQ/6Qypmf/TX0WtvlTLMiFNPx5AFJ5wC9JH0aOAX4S1svioilkk4D7gDqgUsjYrqk7wOTI2Ji61sws5X07AsfPyZN82amUqbLmWYtKua2XnXA8cDepPNsdwCXRHtulVJCvhjcrImGZfBi1jtzxkRY+oF7Z1rN6cgoBAcDf4uIxeUKrj2c5Mxa8cHbMP3mlPBmP5bunfmxvVLC23xf3zvTqlZHktxlpBszPwhcC9wREUvLEmURnOTMijRvJkzLRkZY+Ar0XAu2PTQlvA1czrTq0tGRwbsD+wKHA7sCd0XECSWPsghOcmbt1LAMXnwgK2f+JZUzB2y5vJzZZ2DeEZp1WIdHIcgS3VjSsDufiogBpQ2xOE5yZh3gcqZVqY6UK8eSuv/vAdwP/Am4M6+SpZOcWYm4nGlVpCNJ7lrSubjbKqHziZOcWYm5nGlVoCSDpmYb2gU4KiJOLVVw7eEkZ1ZG7y9YXs6c809Q/fJy5hb7QrfV8o7QrFktJbliLgZH0gjgKGAc8CJwU2nDM7OK0GstGHlcmuY9n90781q4/pisnHlYVs7c3uVM6xJabMlJ2px0Lu5IYD7pXNw3I2LjzgtvZW7JmXWyhmXwwv0p4T1za1bO3KqgnLle3hGatb9cKakBeAg4PiJmZsteiIhNyxppG5zkzHLkcqZVqFUpVx5CasndJ+l2UucT1yfMapnLmdbFFNO7sjdwEKlsOQa4Arg5Iu4sf3grc0vOrMK4nGkVoCS9KyX1Aw4DDo+IMSWMr2hOcmYVzOVMy0nJLiHIm5OcWRcx97nlF5u/81oaA6+xnLn+CJczraSc5MwsHw3L4IX7sovNb4Vli2HdYSnZbTvO5UwrCSc5M8vf+wtg+k1ZOXNSKmcO/XR278yxLmfaKuvQxeBmZiXRay0Y+cU0FZYzn7vd5UwrC7fkzCxfLmdaCbhcaWaV7/234KmsnPnKZJczrWguV5pZ5eu1Nux4fJrmPrv8YvMVypnjYf3hLmdaUdySM7PKtmxpdrH51fDMX7Ny5tbZxebjYI11847QKoDLlWbW9TVbzty7oJzZI+8ILScuV5pZ19diOfM26KxFbo8AAAoNSURBVNWvoHemy5mWuCVnZl2by5mGy5VmVgvefwueujErZ05xObOGuFxpZtWv19qw4wlpeuOZ7GLzPy0vZ243LiW8gdu5nFkj3JIzs+q2bGl2sXljOfNDWG+b5RebrzEg7witBFyuNDN7783l9858ZQrUdVtezhy6j8uZXZjLlWZmq/drppx5LTz7N5czq5RbcmZW21zOrAouV5qZtcXlzC7L5Uozs7asUM6ckZLdE39auZy5/vC8I7UiuSVnZtaaZUvh3/emcuazf8vKmdtm5czDXM6sEC5Xmpl11HtvLr/Y/NXHs3LmPlk5c2+XM3PkcqWZWUet3g9GnZimFcqZf4XV10kdVUYcBetvl3eklnFLzsysI1zOrAguV5qZldtH5cyr4dV/uZzZiVpKcnVl3ulYSc9KminprGaeP1nSk5KmSnpY0rByxmNmVlaN5cyT7ocvPwI7fxnmTII/jYefbwm3nQWvP5l3lDWlbC05SfXAc8CngTnAJODIiHi6YJ01I2Jh9vgA4JSIGNvadt2SM7MuZdlS+Pc9WTnztlTOHLgtjBifypm9++cdYVXIoyU3CpgZES9ExIfAtcCBhSs0JrhMb6Br1U7NzNpS3w023wfGXQnfeBb2+1kqY95+FlywBVw7PrvTypK8I61K5exdOQiYXTA/B9ip6UqSTgW+DvQAxpQxHjOzfBX2zvzP08uHAnrmVli9f8G9M7fNO9KqUc6WXHN3N12ppRYRF0bEZsC3gO80uyHpJEmTJU2eO3duicM0M8vBesNg7x/C12fAUdfBkF3gn3+Ai3ZN06O/g0Xz8o6yyyvnOblPAOdGxD7Z/NkAEfHjFtavA96KiL6tbdfn5MysajXXO3Pzsct7Z9Z3zzvCipXHxeCTgKGSNgFeAY4AjmoS1NCIeD6b3R94HjOzWtW0nDn1anjiOpczO6Cs18lJ2g/4JVAPXBoRP5L0fWByREyU9CtgL2AJ8BZwWkRMb22bbsmZWU1ZtgRmFvTObFhS0DtzHPReJ+8IK4IvBjcz6+reexOevCElvNemQl331HNzxHgY+umaLmc6yZmZVZP/TF9+78xFc7Ny5uFZOXObvKPrdE5yZmbVqNly5nYFF5vXRjnTSc7MrNrVcDnTSc7MrJa8/hRMu6ZmyplOcmZmtWjZEph5d1bOvD2VM9cfnlp32xxaNeVMJzkzs1q3aD481VjOnJbKmVuMTQnvY3t16XKmk5yZmS3XtJzZe8DycuZ6W+cdXbs5yZmZ2cqqpJzpJGdmZq1bNB+evD4lvNef6FLlTCc5MzMr3utPwtSsnPnevIovZzrJmZlZ+y1bAs/flVp3z90ODUsrspzpJGdmZh2zaN7yi80/Kmfum5Uz98y1nOkkZ2ZmpbNSOXPdbCig8WlA2E7mJGdmZqXXbDlzRHbvzEPTGHmdwEnOzMzKq9Vy5l5QX75xup3kzMys87z+5PKhgN6bv2I58+aT0vNNDdwWTn54lXbXUpKrW6WtmZmZtWbgtjD2x/D1Z+CICbDhKHjsIvjdJ2Dhq6D6Fdev7wGDR5U8jPK1Hc3MzLr1gC33T9Oieeli8ymXp9ZdIdXB6G+VfPduyZmZWefo3R92/jKc+hgMOyglNkituBHjoc96Jd+lk5yZmXW+fX+akhuUrRUHTnJmZpaHPgNT6011ZWvFgc/JmZlZXkafCXNnlK0VB05yZmaWlz4D4bjbyroLlyvNzKxqOcmZmVnVcpIzM7Oq5SRnZmZVy0nOzMyqVpe7QbOkucCsEmyqPzCvBNupRj42LfOxaZmPTct8bFpWqmOzcUQMaLqwyyW5UpE0ubk7VpuPTWt8bFrmY9MyH5uWlfvYuFxpZmZVy0nOzMyqVi0nuYvzDqCC+di0zMemZT42LfOxaVlZj03NnpMzM7PqV8stOTMzq3JOcmZmVrVqLslJGivpWUkzJZ2VdzyVQtKGku6TNEPSdElfzTumSiOpXtK/JN2adyyVRNJakm6Q9Ez2+flE3jFVCklfy/6enpJ0jaSeeceUJ0mXSnpD0lMFy/pJukvS89nPtUu5z5pKcpLqgQuBfYFhwJGShuUbVcVYCnwjIrYCdgZO9bFZyVeBGXkHUYF+BdweEVsCw/ExAkDSIOB0YGREbAPUA0fkG1XuLgfGNll2FnBPRAwF7snmS6amkhwwCpgZES9ExIfAtcCBOcdUESLitYh4PHv8DumLalC+UVUOSYOB/YFL8o6lkkhaE9gN+H8AEfFhRCzIN6qK0g3oJakbsDrwas7x5CoiHgTebLL4QOCK7PEVwEGl3GetJblBwOyC+Tn4i3wlkoYA2wOP5RtJRfklcCbQkHcgFWZTYC5wWVbKvURS77yDqgQR8QrwM+Bl4DXg7Yi4M9+oKtJ6EfEapH+2gXVLufFaS3JqZpmvoSggaQ3gRuCMiFiYdzyVQNJngDciYkresVSgbsAOwO8iYntgESUuN3VV2bmlA4FNgA2A3pI+n29UtafWktwcYMOC+cHUePmgkKTupAR3dUTclHc8FWQX4ABJL5FK3GMkXZVvSBVjDjAnIhpb/TeQkp7BXsCLETE3IpYANwGfzDmmSvQfSesDZD/fKOXGay3JTQKGStpEUg/SSeCJOcdUESSJdF5lRkT8PO94KklEnB0RgyNiCOkzc29E+D9yICJeB2ZL2iJbtCfwdI4hVZKXgZ0lrZ79fe2JO+U0ZyJwTPb4GOCWUm68Wyk3VukiYqmk04A7SD2dLo2I6TmHVSl2Ab4APClparbsnIj4W44xWdfwFeDq7B/HF4Djco6nIkTEY5JuAB4n9V7+FzV+ey9J1wC7A/0lzQG+B/wEuE7S8aR/DA4r6T59Wy8zM6tWtVauNDOzGuIkZ2ZmVctJzszMqpaTnJmZVS0nOTMzq1pOcmYVQtLBkkLSltn8kMK7tbfwmjbXMatlTnJmleNI4GF8p3qzknGSM6sA2T1DdwGOp5kkJ+lYSbdIuj0bD/F7BU/XS/pDNm7ZnZJ6Za85UdIkSdMk3Shp9c55N2aVw0nOrDIcRBqT7TngTUnN3f9xFDAeGAEcJmlktnwocGFEbA0sAA7Jlt8UETtGROMYb8eX9R2YVSAnObPKcCTp5s9kP49sZp27ImJ+RLxPutnvrtnyFyOi8VZsU4Ah2eNtJD0k6UlScty6LJGbVbCaunelWSWStA4whpSUgnRf1QB+22TVpvfga5xfXLBsGdAre3w5cFBETJN0LOmegWY1xS05s/wdClwZERtHxJCI2BB4kTQUVKFPS+qXnXM7CPh7G9vtA7yWDaE0vuRRm3UBTnJm+TsSuLnJshuBc5osexj4IzAVuDEiJrex3e+SRne/C3imBHGadTkehcCsC8jKjSMj4rS8YzHrStySMzOzquWWnJmZVS235MzMrGo5yZmZWdVykjMzs6rlJGdmZlXLSc7MzKrW/wcniJXR+iGgSAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 504x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "P3_2([1.0e-10, 0.0001, 0.001, 0.01, 0.1, 0.5, 1.0, 2.0, 10.0], train_data, train_labels, dev_data, dev_labels)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 80,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "For a C of 1e-10, the F-1 score is: 0.206\n",
      "The sum of the topic squared weights is: 0.0\n",
      "------------------------------------------------------------------\n",
      "For a C of 0.0001, the F-1 score is: 0.542\n",
      "The sum of the topic squared weights is: 0.038\n",
      "------------------------------------------------------------------\n",
      "For a C of 0.001, the F-1 score is: 0.619\n",
      "The sum of the topic squared weights is: 0.734\n",
      "------------------------------------------------------------------\n",
      "For a C of 0.01, the F-1 score is: 0.665\n",
      "The sum of the topic squared weights is: 10.594\n",
      "------------------------------------------------------------------\n",
      "For a C of 0.1, the F-1 score is: 0.697\n",
      "The sum of the topic squared weights is: 102.271\n",
      "------------------------------------------------------------------\n",
      "For a C of 0.5, the F-1 score is: 0.71\n",
      "The sum of the topic squared weights is: 373.82\n",
      "------------------------------------------------------------------\n",
      "For a C of 1.0, the F-1 score is: 0.694\n",
      "The sum of the topic squared weights is: 601.584\n",
      "------------------------------------------------------------------\n",
      "For a C of 2.0, the F-1 score is: 0.694\n",
      "The sum of the topic squared weights is: 922.101\n",
      "------------------------------------------------------------------\n",
      "For a C of 10.0, the F-1 score is: 0.685\n",
      "The sum of the topic squared weights is: 2104.343\n",
      "------------------------------------------------------------------\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAbkAAAEWCAYAAAD7HukTAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nO3deZxcZZ3v8c+3OwmEfUmEQAgJOyHsTYughFUSHRJnVATUQQGBuQRUzIzAeBEZr9fRltF7ZRyRiYCiTEQZIiRsAhEX6ARlSQIhC0sCDQlZSAhL0unf/HFOhUqlurs6qdNVXfV9Q726znOees7vnO7U7zzP2RQRmJmZ1aKGSgdgZmaWFSc5MzOrWU5yZmZWs5zkzMysZjnJmZlZzXKSMzOzmuUkZ2ZVS9KbkvapdBzWdznJWY9IeljSCklbVTqWckjX5530yzT3+kAndU+S9JCkNyS9UELb50t6VtJqSa9JulvS9mVfiR6QdJOktQXr+6kytPtjSbcUKT9M0ruSdtmcdiNiu4hYuKXx9YSkAyT9StLr6e/6KUmXS2qUNLyU371VDyc5K5mk4cCHgADGZbSMflm0240J6Zdp7vXnTuqtASYB/9hdg5JGA98Czo6I7YGDgclli5gt2lbfKVjf/9qMZTcWFN0E/J2kbQvK/x64KyKW97D9SvwdIGlf4DFgEXBoROwIfBJoAiq6g2Kbx0nOeuLvgUdJvtDOzRVKOlbSq/lffJL+VtJT6fsGSVdIWiBpmaTJuT37dM840l7PS8CDafmv0jbfkPR7SYfktb2rpN9KWiVphqRvSvpD3vyDJN0vabmkuZLOLMfKR0RrRPwMKKVncQzw54j4a/rZ5RFxc0SsTmMcKOl7kl5M1/EPkgam88ZJmi1pZdrTPDhv3V6Q9NV0266R1E/SHpJ+LWmppOclXbY56yfp4HR5K9Plj8ubd5OkH0maKmkNcFLBtvkz8DLw8bzPNALnADen082S/py23ybph5IG5NUPSZdImgfMyyvbL32/o6Rb0vV8UdLXJDWk866R9PO8tnJ/V/3S6c9JWpj2qp+X9OlONsM3gD9FxOUR0Zau29yIOCciVhbZZl+V9HLa7lxJp5S+xa1XRIRffpX0AuYD/ws4GlgH7JY3bwFwWt70r4Ar0vdfIkmOQ4GtgB8Dv0znDSfpGd4CbAsMTMvPI9lz3gr4PvBEXtu3pa9tgJEke91/SOdtm05/HugHHAW8DhzSyTo9DFzQw+1wKvBCN3U+BLxN8qV5PLBVwfzr02XvCTQCx6XregBJj/E0oD/wT+l2H5B+7gXgCWAvYCDJjurjwNXAAGAfkiR8eidx3QR8s0h5/3Q5V6XtnAysBg7M+9wb6bo0AFsXaeOfgQfypk8HlgL90+mjgWPT38tw4BngS3n1A7gf2CXv7yCA/dL3twB3pn8Xw4HngPPTedcAP89rK/d31S/9m1iVty5Duvh7eBX4fIl/Bwemf2t75C1z30r/O/Wr4PdU6QD86hsv4IMkiW1QOv0s8OW8+d8EJqXvt0+/qPdOp58BTsmrOyRtK/dlF8A+XSx7p7TOjmlCWJf7wspbdi7JfQp4pODzPwa+3knbDwNvASvT119K2BbdJrm03ljgt2m7bwLXpfE3kCTAw4t85n8Dk/OmG0h6SCem0y8A5+XNfz/wUkEbVwI/7SSmm4B38tb39bT8Q+kXfENe3V8C1+R97pZu1ndY+rsZmk7fCvygi/pfAu7Imw7g5II6AeyXbrd3gZF58y4CHk7fX0PXSW4lSS9zYDfrsA4YU+K/if2AJenfQ/+s/w36tXkvD1daqc4F7ouI19PpX5A3ZJlO/52SE1L+jiRZvJjO2xu4Ix2mWkmS9NYDu+V9flHuTXqA/9vp8OYqki92gEHAYJIvrkXFPpsu6/25ZaXL+zSwexfrdllE7JS+jkpjuErvnZjxH11umU5ExLSIOIOkZzIe+BxwQboeW5P0fgvtAbyY10ZHun575tUpXN89Ctb3KjbetoVa8tZ3UN5yF6XLy3mxi+VuIiJeAn4PfEbSdsDHSIcqYcMJHXelw9CrSI5ZDipoprNlDCLpYb6YV1YYX2dxrSHZ+bkYaFNyAtBBnVRfRrIT1q2ImE+SqK8Blki6TdIepXzWeo+TnHUrPVZ0JjA6/YJ6FfgycLikwwEiYg7Jl85YkuMwv8hrYhEwNu+LdaeI2DoiXs6rk/84jHNIksKpJL234blQSIa/2kmGPnP2KljW9IJlbRcR/9CTdY6Ib8V7J2Zc3JPPFmmrIyJ+R3K8cRTJ8Ok7wL5Fqr9CkrgAkCSS9etsWy0Cni9Y3+0j4iM9DPMVYK/cMa7UsC6W25mbSY7dfjyN6y95835EMgKwf0TsQJKMVfD5zpbxOkkva++8svz41pAMX+dstFMTEfdGxGkkCexZ4CedLOcB8o4rdicifhERH0zjCuBfS/2s9Q4nOSvFx0h6XiOBI9LXwcAjJF9oOb8ALgNOIDkml/MfwP+RtDeApMGSxnexvO1JhqaWkXxxfSs3IyLWA78BrpG0TbpHnh/DXcABkj4rqX/6Oib/5I3NpeQEmq1Jjl9J0tb5J04U1B0v6SxJOyvRDIwGHk17S5OA69KTRholfSDtBU8GPirpFEn9ga+k2+JPnYTVCqxKT4AYmLY1StIxPVy9x0gSxT+l2+xE4AySY5898WuSpPwN8npxqe1Jjo29mf7eSt7xSH/vk0n+jrZP/5YuB3InmzwBnCBpmKQdSYZsAZC0m5KTebYl2ZZvkvw9F/N14DhJ35W0e/r5/ST9XNJO+RUlHSjp5PT39g7JEHRn7VqFOMlZKc4lOcbzUkS8mnsBPwQ+rfdO9/4lcCLwYN6wJsAPgCnAfZJWk5yE8v4ulncLSa/wZWBOWj/fBJIe3qvAz9LlvgsQydmLHwbOIumdvEqyd12O6/pOIPkim0rSi3gbuK+TuiuAL5CcJbiK5Mv4uxFxazp/IvA0MANYnsbYEBFzgc8A/5+k93IGcEZErC22kPTL/wySHY/n08/cSLJ9Spa2P46kJ/468O/A30fEsz1sZw3vJbpbC2ZPJOmlrybpSfX00oVLSRLxQuAPJDtVk9Ll3p+29xTJiTh35X2ugWRn4RWSbT2a5ASqYvEvAD5AMnowW9Ib6frMTOPOtxXwbZLt9SrwPpLeqVURRfihqda3SfpXYPeIOLfbymZWV9yTsz5HyXVwh+UNA54P3FHpuMys+lTkrgJmW2h7kiHKPUhO4f4eyfVTZmYb8XClmZnVLA9XmplZzepzw5WDBg2K4cOHVzoMMzOrIo8//vjrETG4sLzPJbnhw4czc+bMSodhZmZVRNKLxco9XGlmZjXLSc7MzGqWk5yZmdUsJzkzM6tZTnJmZlaznOQ2w6RZk2hta92orLWtlUmzJlUoIjMzK8ZJbjOM2nUUE6dP3JDoWttamTh9IqN2HVXhyMzMLF+fu06uGjQPaebyoy9nwoMT+PDeH+b3i39Py+gWmoc0Vzo0MzPL4yTXA2+8+wbTnp/GlAVTePr1pwG4c8GdbNt/Wx55+RG2G7AdB+9yMMnDnM3MrNIyvUGzpDEkD8xsBG6MiG8XzP834KR0chvgfRGxE11oamqK3rzjybqOdfzx5T8yZcEUHl70MOs61nHAzgdwxOAjuOeFe2jarYlHXn6E9bGejuhg+A7DGTNiDGOHj2WfnfbptTjNzOqZpMcjommT8qySnKRG4DngNGAxyROQz46IOZ3UvxQ4MiLO66rd3kpyc5fP5c4Fd3L3wrtZ/s5ydtl6Fz4y4iOM3288q95dxcTpEzcMUba2tfKV6V9h/L7jeXb5s7S+2koQHLjzgYwdMZYxI8aw53Z7Zh6zmVm9qkSS+wBwTUScnk5fCRAR/7eT+n8Cvp4+xr5T5Upyk2ZNYtSuozY6jvbAiw9wx7w7eO2t15i7Yi79Gvpx4tATGbfvOD449IP0b+jf6Wdb21qZtWwW5406j6VvLeW+F+9j6vNTeWrpUwAcPvhwxo4Yy+nDT2fQwEHdtmFmZqWrRJL7BDAmIi5Ipz8LvD8iJhSpuzfwKDA0ItYXmX8hcCHAsGHDjn7xxaL34eyR3BmR3x39Xd5c+yY/nfVTnnz9SQAO2fUQxu83nrHDx7LT1l2OnnZr8erF3PPCPUx7fhrPrXiOBjVwzG7HcNAuB3Hngjv53ujvbegN5vcOe5uTrpn1ZZVIcp8ETi9Ics0RcWmRul8lSXCbzCtUzuHK1rZWLnvwMta0r0GI04efzsWHX8y+O+1blvYLLVi5gGnPT2Pa89N4afVLNKiBRjVy7JBjefy1xzn7oLPZb+f9UPpfgxpAbJiWNrwrXp6e8NKghk3r5X02Vy+//Nnlz/LDv/6QS4+6lFG7juKZ5c/w/b98n4lNEzli8BEbYm1oSH42qnGTsgY10E/9aFBDEoNPwDGzXlLVw5WS/gpcEhF/6q7dch+T+9idH2PBygV84dAvcNlRl5Wt3a5EBHOWz2Hawmnc/tztrGlf0yvL7W25ZLdRAsxLiLl5jWqksaFxo7obEmnDxmUNaqCxoXHTstwyGvptNL2hrU4+U2y5XbXT0PBeIi+W6DsrK7XuRuvuHQWzknWW5LK8hGAGsL+kEcDLwFnAOUUCOxDYGfhzhrEU9VjbYyxcuZB9d9yX25+7nWOHHNsrQ4WSOGTXQ1izdg1TFkxh3H7juHvh3VzRfAWHDjqUyP0X6StvGnhvXvpf8v97ZR3RsWm99CewSZu5NjroYMr8KUx7YRqn7306Y0eMZX2s3/DqiA7Wd6Q/OynriA7ao32j8vz6hXXz29ikrKB+e0f7e8voaN+obn79Yu0UW0ZfkNmOwmYk+K52LrraiSjHjkL+KEGnOwXeUegzevPwSGZJLiLaJU0A7iW5hGBSRMyWdC0wMyKmpFXPBm6LLK9lKKK1rZXLH76cIPjMyM8wbPthvXpMrPAY3KnDTq3oMblcTI+2PcpFh13E5LmTOfPAM2v6AvdiiXSjZF2QSNfHejo6ChJ8QaLPJfhiOwTl2FHoNK4y7ij0ZBl9Qak7CsUSfClJvEcjAZu5jFrbUcjdNSr/DPXcdLllep1cFsp5duWSNUu49dlbuftv72bYDsN69USLajvRozDpVvpEGOsbqnlHoXD0oFw7CsXm1duOQqfJtwdJfPXa1cxbOY+j3ncU81bM2+Lvml4/JpeVch6Tu/zhy3lq6VPc/4n7635Io9qSrlm96DaRdqzfdKegzDsKXcXQ4x2FHhyWaHuzjSVvL+Giwy5iwpGbnHjfI5U4JlfVOqKDma/O5ENDP1T3CQ4omsiahzS7F2eWsVxPKHcdbr3IjRblDo80757N903dPoVg3op5rHh3BcfsfkylQzEzqyv5h0MmHDmBltEtGz3ZpZzqNsnNeHUGAM27u6diZtabZi2btdExuOYhzbSMbmHWslllX1bdDle2vtrK0O2Gssd2e1Q6FDOzutKbh0fqqieXe6L3+o71zHxt5oazCP1EbzOz2lRXSS53bcav5/2a1WtXs8vWu/iJ3mZmNazuLiFobWtlwoMTeLv9bXYcsCPXnXidzyA0M+vjOruEoK56cpCM++6zY/Iw07MOOssJzsyshtVdkmtta2Xeynls3bg1k+dOzuSUVTMzqw51leRy12Y079bMDlvtkOm1GWZmVnl1leRy12bsvPXO9G/on+m1GWZmVnl1dZ1c7tqM2+fdTqMaAd+6ysysltVVTy6nvaOdfg11ld/NzOqSk5yZmdWsukxy62P9huFKMzOrXXWZ5No72uvusRZmZvWoLpPc+o71NDa4J2dmVuvqMsmt61jnY3JmZnWgLpNce7TTT05yZma1ri6TnIcrzczqQ10mOV9CYGZWH+oyya2P9T670sysDtRVkss9Gby9o33DdXJ+MriZWe2qqySXezL4m2vfpF9Dvw1PJfCTwc3MalNdJbncUweWvbOM51Y8x8TpE2kZ3eIbNJuZ1ai6SnKQJLqB/Qby3IrnOPPAM53gzMxqWN0luda2Vt5uf5uDdjnITwY3M6txdZXkcsfgdtxqR0buOtJPBjczq3F1leRyTwYf0DAAwE8GNzOrcZleES1pDPADoBG4MSK+XaTOmcA1QABPRsQ5WcWTezI4gBDgJ4ObmdWyzJKcpEbgeuA0YDEwQ9KUiJiTV2d/4Erg+IhYIel9WcVjZmb1J8vhymZgfkQsjIi1wG3A+II6XwCuj4gVABGxJMN4zMyszmSZ5PYEFuVNL07L8h0AHCDpj5IeTYc3NyHpQkkzJc1cunTpFgcWxBa3YWZm1S/LJKciZYXZpR+wP3AicDZwo6SdNvlQxA0R0RQRTYMHDy57oGZmVpuyTHKLgb3ypocCrxSpc2dErIuI54G5JEnPzMxsi2WZ5GYA+0saIWkAcBYwpaDOfwMnAUgaRDJ8uTDDmMzMrI5kluQioh2YANwLPANMjojZkq6VNC6tdi+wTNIc4CHgHyNiWVYxbYjNx+TMzOpCptfJRcRUYGpB2dV57wO4PH31KqnYIUMzM6sldXXHEzMzqy9OcmZmVrPqMsklo6RmZlbr6jLJwXv3rjQzs9pVt0nOzMxqX10mOV9CYGZWH0pOcpK2zTKQ3ubhSjOz2tdtkpN0XHqx9jPp9OGS/j3zyMzMzLZQKT25fwNOB5YBRMSTwAlZBmVmZlYOJQ1XRsSigqL1GcRiZmZWVqXc1muRpOOASG+0fBnp0GVf5tt6mZnVvlJ6chcDl5A88HQxcEQ6bWZmVtW67MlJagQ+GxGf7qV4zMzMyqbLnlxErAfG91Isvca39TIzqw+lDFf+UdIPJX1I0lG5V+aRZWDSrEm0trVuVNba1sqkWZMqFJGZmWWplBNPjkt/XptXFsDJ5Q8nW6N2HcXE6RNZ17EOSBLcxOkTaRndUuHIzMwsC+prQ3dNTU0xc+bMzf58a1srF9x3AQfvejBtb7bRMrqF5iHNZYzQzMx6m6THI6KpsLyUO57sKOk6STPT1/ck7ZhNmNlrHtLMgMYBzFk2hzMPPNMJzsyshpVyTG4SsBo4M32tAn6aZVBZam1rZe36tRyy6yFMnjt5k2N0ZmZWO0pJcvtGxNcjYmH6+gawT9aBZSF3DG6b/ttw6KBDaRndwsTpE53ozMxqVClJ7m1JH8xNSDoeeDu7kLIza9ksWka30L+hP5AMXbaMbmHWslkVjszMzLJQytmV/wDcnHccbgXwucwiytB5o87bpKx5SLOPy5mZ1ahuk1xEPAEcLmmHdHpV5lGZmZmVQSlnV35L0k4RsSoiVknaWdI3eyM4MzOzLVHKMbmxEbEyNxERK4CPZBeSmZlZeZSS5BolbZWbkDQQ2KqL+mZmZlWhlBNPfg78TtJPSW7ndR5wc6ZRmZmZlUEpJ558R9JTwKlp0b9ExL3ZhmVmZrblSunJERH3SJoBnAC8nm1IZmZm5dHpMTlJd0kalb4fAswiGar8maQv9VJ8ZmZmm62rE09GRETuViCfB+6PiDOA95MkOzMzs6rWVZJbl/f+FGAqQESsBjpKaVzSGElzJc2XdEWR+Z+TtFTSE+nrgp4Eb2Zm1pWujsktknQpsBg4CrgHNlxC0L+7hiU1AtcDp6VtzJA0JSLmFFT9r4iYsDnBm5mZdaWrntz5wCEk96n8VN4F4cdS2qN2moH56ZML1gK3AeO3IFYzM7Me6bQnFxFLgIuLlD8EPFRC23sCi/KmF5Mczyv0cUknAM8BX46IRYUVJF0IXAgwbNiwEhZtZmZW2h1PNpeKlEXB9G+B4RFxGPAAnVxkHhE3RERTRDQNHjy4zGGamVmtyjLJLQb2ypseCrySXyEilkXEu+nkT4CjM4zHzMzqTJZJbgawv6QRkgYAZwFT8iuk19/ljAOeyTAeMzOrM5uV5CRd3V2diGgHJgD3kiSvyRExW9K1ksal1S6TNFvSk8Bl9NGHsZqZWXUq6bZeRVwAXNtdpYiYSnp9XV7Z1XnvrwSu3MwYzMzMutRpkpPU2RPABQzMJhwzM7Py6aontxI4JiJeK5whaZPT/M3MzKpNV8fkbgH27mTeLzKIxczMrKy6uhj8a13M+2o24ZiZmZVPj86ulHRNRnGYmZmVXU8vIRjXfRUzM7Pq0NMkV+xWXWZmZlWpp0nOt90yM7M+o9MkJ+m+vPdXAkRESQ9LNTMzqwZd9eTyb/f/yawDMTMzK7euklzhY3HMzMz6lK7ueLKPpCkkJ5vk3m8QET7T0szMqlpXSW583vuWrAMxMzMrt67ueDK9NwMxMzMrtywfmmpmZlZRTnJmZlazSk5ykrbNMhAzM7Ny6zbJSTpO0hzgmXT6cEn/nnlkZmZmW6iUnty/AacDywAi4knghCyDMjMzK4eShisjovBJ4OsziMXMzKysurpOLmeRpOOAkDQAuIx06NLMzKyaldKTuxi4BNgTWAwckU6bmZlVtW57chHxOvDpXojFzMysrLpNcpL+X5HiN4CZEXFn+UMyMzMrj1KGK7cmGaKcl74OA3YBzpf0/QxjMzMz2yKlnHiyH3ByRLQDSPoRcB9wGvB0hrGZmZltkVJ6cnsC+Xc72RbYIyLWA+9mEpWZmVkZlNKT+w7whKSHSZ4tdwLwrfQ2Xw9kGJuZmdkWKeXsyv+UNBVoJklyV0XEK+nsf8wyODMzsy1R6g2a3wHagOXAfpJ8Wy8zM6t6pdyg+QLg98C9wDfSn9eU0rikMZLmSpov6You6n1CUkhqKi1sMzOz7pXSk/sicAzwYkScBBwJLO3uQ5IageuBscBI4GxJI4vU257kVmGP9SBuMzOzbpWS5N6JiHcAJG0VEc8CB5bwuWZgfkQsjIi1wG3A+CL1/oXk5JZ3SozZzMysJKUkucWSdgL+G7hf0p3AK918BpJLD/KfXrA4LdtA0pHAXhFxV1cNSbpQ0kxJM5cu7bYTaWZmBpR2duXfpm+vkfQQsCNwTwltq1hzG2ZKDSTPqvtcCTHcANwA0NTUFN1UNzMzA7pJcmkieioiRgFExPQetL0Y2Ctveigb9wC3B0YBD0sC2B2YImlcRMzswXLMzMyK6nK4MiI6gCclDduMtmcA+0sakT6H7ixgSl7bb0TEoIgYHhHDgUcBJzgzMyubUu54MgSYLakVWJMrjIhxXX0oItolTSC55KARmBQRsyVdS/IEgyldfd7MzGxLlZLkvrG5jUfEVGBqQdnVndQ9cXOXY2ZmVkwpJ55Ml7Q3sH9EPCBpG5KemZmZWVUr5Y4nXwBuB36cFu1JcjmBmZlZVSvlOrlLgOOBVQARMQ94X5ZBmZmZlUMpSe7d9I4lAEjqR971bmZmZtWqlCQ3XdJVwEBJpwG/An6bbVhmZmZbrpQkdwXJDZmfBi4iOVvya1kGZWZmVg6lXEIwHrglIn6SdTBmZmblVEpPbhzwnKSfSfpoekzOzMys6nWb5CLi88B+JMfizgEWSLox68DMzMy2VEm9sohYJ2kayVmVA0mGMC/IMjAzM7MtVcrF4GMk3QTMBz4B3EhyP0szM7OqVkpP7nMkT/W+KCLezTYcMzOz8inl3pVn5U9LOh44JyIuySwqMzOzMijpmJykI0hOOjkTeB74TZZBmZmZlUOnSU7SASQPOj0bWAb8F6CIOKmXYjMzM9siXfXkngUeAc6IiPkAkr7cK1GZmZmVQVdnV34ceBV4SNJPJJ0CqHfCMjMz23KdJrmIuCMiPgUcBDwMfBnYTdKPJH24l+IzMzPbbKXc8WRNRNwaEX8DDAWeILlps5mZWVUr5d6VG0TE8oj4cUScnFVAZmZm5dKjJGdmZtaXOMmZmVnNcpIzM7Oa5SRnZmY1y0nOzMxqlpOcmZnVLCc5MzOrWU5yZmZWs5zkzMysZjnJmZlZzco0yUkaI2mupPmSNrnfpaSLJT0t6QlJf5A0Mst4zMysvmSW5CQ1AtcDY4GRwNlFktgvIuLQiDgC+A5wXVbxmJlZ/cmyJ9cMzI+IhRGxFrgNGJ9fISJW5U1uC0SG8ZiZWZ3p6sngW2pPYFHe9GLg/YWVJF0CXA4MAIo+3UDShcCFAMOGDSt7oGZmVpuy7MkVe4r4Jj21iLg+IvYFvgp8rVhDEXFDRDRFRNPgwYPLHKaZmdWqLJPcYmCvvOmhwCtd1L8N+FiG8ZiZWZ3JMsnNAPaXNELSAOAsYEp+BUn7501+FJiXYTxmZlZnMjsmFxHtkiYA9wKNwKSImC3pWmBmREwBJkg6FVgHrADOzSoeMzOrP1meeEJETAWmFpRdnff+i1ku38zM6pvveGJmZjXLSc7MzGqWk5yZmdUsJzkzM6tZTnJmZlaznOTMzKxmOcmZmVnNcpIzM7Oa5SRnZmY1y0nOzMxqlpOcmZnVLCc5MzOrWU5yZmZWs5zkzMysZjnJmZlZzXKSMzOzmuUkZ2ZmNctJzszMapaTnJmZ1SwnOTMzq1lOcmZmVrOc5MzMrGY5yZmZWc1ykjMzs5rlJGdmZjXLSc7MzGqWk5yZmdUsJzkzM6tZdZXkJs2aRGtb60ZlrW2tTJo1qUIRmZlZluoqyY3adRQTp09kXcc6IElwE6dPZNSuoyocmZmZZSHTJCdpjKS5kuZLuqLI/MslzZH0lKTfSdo7y3iahzTTMrqFt9a9xdOvP83E6RNpGd1C85DmLBdrZmYVklmSk9QIXA+MBUYCZ0saWVDtr0BTRBwG3A58J6t4cpqHNHPAzgcwe9lszjzwTCc4M7MalmVPrhmYHxELI2ItcBswPr9CRDwUEW+lk48CQzOMB0iGKJe8tYSLDruIyXMnb3KMzszMakeWSW5PYFHe9OK0rDPnA9OKzZB0oaSZkmYuXbp0swPKHYNrGd3ChCMn0DK6hYnTJzrRmZnVqCyTnIqURdGK0meAJuC7xeZHxA0R0RQRTYMHD97sgGYtm7XRMbjcMbpZy2ZtdptmZla9+mXY9mJgr7zpocArhZUknQr8MzA6It7NMB7OG3XeJmXNQ5p9XM7MrEZl2ZObAewvaYSkAcBZwJT8CpKOBH4MjIuIJRnGYmZmdSizJBcR7cAE4F7gGWByRMyWdK2kcWm17wLbAb+S9ISkKZ00Z2Zm1mNZDlcSEVOBqQVlV+e9PzXL5ZuZWX2rqzuemJlZfctrTcEAAAQrSURBVHGSMzOzmqWIomf1Vy1JS4EXy9DUIOD1MrRTi7xtOudt0zlvm85523SuXNtm74jY5BqzPpfkykXSzIhoqnQc1cjbpnPeNp3ztumct03nst42Hq40M7Oa5SRnZmY1q56T3A2VDqCKedt0ztumc942nfO26Vym26Zuj8mZmVntq+eenJmZ1TgnOTMzq1l1l+QkjZE0V9J8SVdUOp5qIWkvSQ9JekbSbElfrHRM1UZSo6S/Srqr0rFUE0k7Sbpd0rPp388HKh1TtZD05fTf0yxJv5S0daVjqiRJkyQtkTQrr2wXSfdLmpf+3Lmcy6yrJCepEbgeGAuMBM6WNLKyUVWNduArEXEwcCxwibfNJr5IcrNx29gPgHsi4iDgcLyNAJC0J3AZ0BQRo4BGkqex1LObgDEFZVcAv4uI/YHfpdNlU1dJDmgG5kfEwohYC9wGjK9wTFUhItoi4i/p+9UkX1RdPcm9rkgaCnwUuLHSsVQTSTsAJwD/CRARayNiZWWjqir9gIGS+gHbUOSZmvUkIn4PLC8oHg/cnL6/GfhYOZdZb0luT2BR3vRi/EW+CUnDgSOBxyobSVX5PvBPQEelA6ky+wBLgZ+mQ7k3Stq20kFVg4h4GWgBXgLagDci4r7KRlWVdouINkh2toH3lbPxektyKlLmayjySNoO+DXwpYhYVel4qoGkvwGWRMTjlY6lCvUDjgJ+FBFHAmso83BTX5UeWxoPjAD2ALaV9JnKRlV/6i3JLQb2ypseSp0PH+ST1J8kwd0aEb+pdDxV5HhgnKQXSIa4T5b088qGVDUWA4sjItfrv50k6RmcCjwfEUsjYh3wG+C4CsdUjV6TNAQg/bmknI3XW5KbAewvaYSkASQHgf00ckCSSI6rPBMR11U6nmoSEVdGxNCIGE7yN/NgRHiPHIiIV4FFkg5Mi04B5lQwpGryEnCspG3Sf1+n4JNyipkCnJu+Pxe4s5yNZ/pk8GoTEe2SJgD3kpzpNCkiZlc4rGpxPPBZ4GlJT6RlV6VPdzfryqXAremO40Lg8xWOpypExGOSbgf+QnL28l+p89t7SfolcCIwSNJi4OvAt4HJks4n2TH4ZFmX6dt6mZlZraq34UozM6sjTnJmZlaznOTMzKxmOcmZmVnNcpIzM7Oa5SRn1sdI2l3SbZIWSJojaaqkAyodl1k1cpIz60PSi4rvAB6OiH0jYiRwFbBbZSMzq051dTG4WQ04CVgXEf+RK4iIJ7qob1bX3JMz61tGAb5RtFmJnOTMzKxmOcmZ9S2zgaMrHYRZX+EkZ9a3PAhsJekLuQJJx0gaXcGYzKqWb9Bs1sdI2oPkSeVHA+8AL5A85HZeJeMyq0ZOcmZmVrM8XGlmZjXLSc7MzGqWk5yZmdUsJzkzM6tZTnJmZlaznOTMzKxmOcmZmVnN+h/VEl90nj9r3QAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 504x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "P3_3([1.0e-10, 0.0001, 0.001, 0.01, 0.1, 0.5, 1.0, 2.0, 10.0], train_data, train_labels, dev_data, dev_labels)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "jWtXwAlOFsxr"
   },
   "source": [
    "ANSWER:\n",
    "\n",
    "1. K-nearest neighbors does not work well for this problem, because the vocabulary is very large, meaning that there are many different features and feature combinations that can potentially be clustered. Moreover, outside of the core constructs of grammar, it would be very difficult to cluster in terms of nearest-neighbors due to the sheer number of unique words and features, i.e. datapoints that would be considered outliers. A solution to this is to remove all common words. Nevertheless, \n",
    "\n",
    "\n",
    "2. Naive Bayes is a generative model that learns the joint probabilities of the features i.e. p(x,y). Logistic regression is a discriminative model that learns the conditional probability distribution i.e. p(x|y). Generative models have higher bias, but lower variance than discriminative models. High bias is indicative of model underfitting, however (meaning it fits the training and testing sets poorly). High variance is indicative of model overfitting (meaning it fits the training data well, but the testing data poorly). In Naive Bayes models, bias is caused by violation of the assumption of conditional independence of the features given a class and that provided all the assumptions are met for both models, Naive Bayes should not outperform logistic regression. In text classication tasks, like we have just performed, the fact that logistic regression is outperformed by Naive Bayes means that the high bias of this model caused by violation of the independence assumption is outweighed by the high variance of the logistic regression model when it comes to classifying text. The high variance for logistic regression is likely provided by the large number of features contained in the text corpus. Nevertheless, with more training examples (say >10,000), logistic regression might begin to outperform Bayes.\n",
    "\n",
    "\n",
    "3. The C value is the inverse of regularization strength, and smaller values represent a stronger regularization. It applies a progressive penalty to parameter values that are more extreme (i.e. outliers) in order to reduce overfitting (high variance). Most often to rectify this, we utilize a function that penalizes large values that might cause the model to overfit: $\\lambda \\sum \\theta_{j}^2$ where lambda is a constant multiplied by the sum of the squared parameter values. C is therefore $\\frac{1}{\\lambda}$. With higher higher lambdas, the more the extreme values will be penalized."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "dGEjsm_uFsxr"
   },
   "source": [
    "### Part 4:\n",
    "\n",
    "Transform the data to a matrix of word **bigram** feature vectors.  Produce a Logistic Regression model.  For each topic, find the 5 features with the largest weights (that's 20 features in total).  Show a 20 row (features) x 4 column (topics) table of the weights.\n",
    "\n",
    "Do you see any surprising features in this table?\n",
    "\n",
    "Notes:\n",
    "* Train on the transformed training data.\n",
    "* You can use `CountVectorizer` and its `.fit_transform` method to transform data.\n",
    "* You can use `LogisticRegression(C=0.5, solver=\"liblinear\", multi_class=\"auto\")` to produce a Logistic Regression model.\n",
    "* You can use `LogisticRegression`'s `.coef_` method to get weights for each topic.\n",
    "* You can use `np.argsort` to get indices sorted by element value. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 81,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "WN51Nv4fFsxs"
   },
   "outputs": [],
   "source": [
    "def P4(training_data, training_labels, testing_data, testing_labels):\n",
    "    \n",
    "    # Transform raw datasets into matrices\n",
    "    vec = CountVectorizer(input = 'content', binary = False, ngram_range = (2,2))\n",
    "    vec_fit_train = vec.fit_transform(training_data)\n",
    "    vec_fit_test = vec.transform(testing_data)\n",
    "    bow = vec.get_feature_names()\n",
    "    \n",
    "    # Fit LR model\n",
    "    LR = LogisticRegression(C = 0.5, solver=\"liblinear\", multi_class=\"auto\")\n",
    "    LR_fit = LR.fit(vec_fit_train, training_labels)\n",
    "    LR_predict = LR_fit.predict(vec_fit_test)\n",
    "    coef = LR.coef_\n",
    "    \n",
    "    # Get the top 5 features\n",
    "    top_five_array = np.argsort(coef)[:,-5:]\n",
    "    \n",
    "    # Create a master list of all the bigrams across all categories\n",
    "    master_list = []\n",
    "    bigrams = []\n",
    "    for row in top_five_array:\n",
    "        for index in row:\n",
    "            bigrams.append(bow[index])\n",
    "            for category in coef:\n",
    "                master_list.append(round(category[index], 2))\n",
    "    \n",
    "    # Turn this into a numpy matrix\n",
    "    master_array = np.stack(np.split(np.array(master_list), 20), axis = 1)\n",
    "    \n",
    "    # Turn this into a table \n",
    "    fig = plt.figure(figsize=(10,5))\n",
    "    ax = fig.add_subplot()\n",
    "    table = plt.table(cellText = master_array.T, rowLabels = bigrams, colLabels = newsgroups_train.target_names, loc='top')\n",
    "    table.set_fontsize(15)\n",
    "    table.scale(1, 3)\n",
    "    plt.axis('off')\n",
    "    \n",
    "    return "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 82,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAsEAAAQSCAYAAABOyyfPAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nOzdeVwV9f4/8NcBROAsHGQRUHZyTVHMXBJ3U+uaa2mpV8utn7abkkaliRXXq4m3rtdccE1NzVyy3BcsLXPB1FIQEMwNVJYDIhzO+/eHOV+PgHJk1fN6Ph7zeDCfmfnMe855n5k3c2bmqEQERERERETWxKaqAyAiIiIiqmwsgomIiIjI6rAIJiIiIiKrwyKYiIiIiKwOi2AiIiIisjosgomIiIjI6thVdQD08HF0dLyUl5dXu6rjoOrPwcHBlJeXx3+2qVSYL1RazBWyhIODw+UbN2543t2u4nOCyVIqlUqYN1QaKpUKzBUqLeYLlRZzhSzxd76o7m7nf1FEREREZHVYBBMRERGR1WERTERERERWh0UwEREREVkdFsFEREREZHVYBBMRERGR1WERTERERERWh0UwEREREVkdFsFEREREZHVYBBMRERGR1WERTI8kf39/LF++vMLX07FjR0RGRj7w8rGxsdDr9eUYEVm7++X+J598gl69elViRNS4cWOsXr26qsOgBzR8+HCMHDlSGVepVNi/f3+FrvPuY4tGo8GBAwdKvbyl81e0lJQUaDQaXLhwoapDMWNX1QEQVbTk5GQEBAQgNTUVdevWfeB+/P39ERkZiSFDhpRbbGFhYcjIyCi3/ojuZ/LkyVUdgtU5efJkVYdAd+jYsSO6du2KiIiIqg6l1AwGQ4XOX9F8fX2rXUwAzwQTEVULBQUFVR0CEVUwfs6rFxbB9NCKjo5GgwYNoNVq4evri0mTJqGwsLDIfCEhIQCA+vXrQ6PRYNq0aRb316tXL6SkpGDkyJHQaDR4+umnleWuX7+O/v37Q6vVIigoCBs2bDDr97vvvkOLFi2g1+vRsGFDrFixQpm2Z88e2Nn93xcyO3bsQPPmzaHT6eDm5oauXbsq026fie7UqRM0Gg2aNGmC48ePY+XKlQgODoazszNGjhwJo9H4AK/mw8lgMODdd99FYGAgtFotGjdujP379yM3NxdvvvkmfHx84Obmhj59+iAlJUVZrmPHjnjnnXfQt29f5X3buXMnduzYgccffxw6nQ59+/ZFdna2soxKpcLs2bPRrFkzaLVadOrUCQkJCSXGVlBQgLfffhseHh7w9PTEv/71LwQHB2Px4sUAgMWLFyM4OBgzZsxA3bp10axZMwC3ztQGBgZCo9EgKCgIs2fPVvpMTk6GSqXCggULUK9ePej1evTu3RtXrlwxW3dKSgq6dOkCjUaDxx9/HD///LMybcqUKWZ5VdJrCACrVq1Cw4YNodVqUbt2bQwfPtzyN+kRM2fOHAQEBECr1aJOnTrKmfXk5GQ8//zz8PLygl6vx1NPPYWrV68CuPclKvn5+Rg9ejQ8PDyg0+lQr149rF27FsD/5UhUVBS8vLzg4eGB8ePHmxVSL7/8Mnx8fKDVatGoUSN8/fXXZv0fP34cPXr0gLu7O2rVqoVu3bop01JSUjBgwAB4eXnBy8sLo0ePNsv5R9Frr72G2NhYTJs2DRqNBvXr18fOnTvRqlUruLi4wN3dHYMGDSrymSpJWloa2rZtW+K+t6TP+dWrVzFixAj4+PjA3d0dL7zwAi5fvlzieu6+BGPhwoUICgqCTqfD0KFDMWTIELPP593zr1u3DiEhIXB2dkZISAjWr19fJMY5c+agbt26cHFxwZgxY4o9nt7Z/xdffIEnnngCarUabdu2xfnz5/H555/Dx8cHrq6ueP/995X5b++7zp8/DwA4evQo2rVrB2dnZ9SqVQtt27bF9evXAdzad37yySeoX7++sn9et25dibGUiYhw4GDRcCttqt7atWslMTFRTCaTHDlyRDw8POR///ufiIj4+fnJsmXLREQkKSlJAEhqauoD93d3n7d16NBBXF1dZf/+/VJYWCizZs0SZ2dnycnJERGRbdu2Sa1atWTfvn1SWFgov/zyi+j1etm7d6+IiOzevVtsbW2V/ry8vGTRokViMpkkLy9Pdu3aZbb+4OBgOXXqlOTn58vgwYMlMDBQRo0aJQaDQc6dOyfu7u6yYsWKMryq5auic+WFF16Qdu3aKe/bmTNnJD4+XkaPHi1PPvmknD9/XgwGg4wYMUKaNm0qRqNRRG69b25ubnLgwAExGo0yadIk8fLykueff16uXr0qV69elYYNG8r06dPNtqVhw4YSHx8vubm5Mm7cOGnYsKHS592mTp0q9evXl7Nnz0pubq68/vrrYmdnJzExMSIiEhMTI7a2tvLWW29Jbm6ukjPLli2Tv/76S0wmk+zcuVMcHBzkxx9/FJH/y+WwsDC5ePGiZGZmSp8+faRbt27Kev38/CQoKEhOnDghRqNR3nrrLQkODlamf/TRR9KlS5f7voY5OTliZ2cnO3fuFBERg8Eg+/btK4d3rWTVZd9SktOnT4ujo6OcOHFCRESuX78uBw4ckJycHAkICJCxY8dKRkaGFBQUyM8//yxZWVkiUvy+47Z58+ZJs2bNJD09XUREUlJS5OTJkyJyK0fs7Oxk7NixkpubKwkJCfLYY4/JJ598oiy/YMECSU9PF6PRKCtXrpQaNWooy1+4cEH0er188sknYjAY5ObNm7J9+3YREblx44YEBQXJBx98ILm5uXLt2jXp2bOnvPzyyxXz4pWzsuRKhw4dZNq0acp4bGys/Prrr1JQUCAXL16UsLAwGTRokDJ92LBhMmLECLN1x8bGyunTpyU4ONisr7sV9zk3mUzSrl07GTFihGRkZEhOTo688sor0rlz5xJjvL1OEZF9+/aJg4OD7Ny5UwoKCuTrr7+WGjVqyLBhw4qd/+eff5aaNWvKli1bpKCgQDZv3iw1a9aUgwcPKjHa2dnJ5MmTJS8vT+Lj48XFxUWWL19e4nYBkFatWklqaqrk5ORIp06d5LHHHpMPPvhAbt68KceOHRN7e3v56aefRKTocbhNmzYydepUMRqNkp+fLwcOHBCDwSAiIhMnTpSGDRtKXFycmEwmSU1Nlbi4uBJjKY2/86VoPVNcIwcO9xqq64Fq/Pjx8vzzz4vIgxXB9+rv7j5v69Chg4wdO1YZNxgMAkCOHTsmIiLPPvusTJ061WyZ1157Tdmh3l0E+/n5yfvvvy8XL14sEo+fn5/861//Usa///57ASBXrlxR2p5//nl56623LNrOilSRuXL58mUBoBQktxUWFoqDg4Ns27ZNacvOzpYaNWrIzz//LCJF37eTJ08KAPn111+VtgkTJkifPn2UcQCyYMECZTwnJ8dsJ3+3oKAgWbhwoTKem5sr9vb2ZkWwg4OD5OXl3XM7+/fvLxMmTBCR/8vlHTt2KNPj4+MFgPz1118iUjRPTpw4IQAkIyNDRMyL4JJew9vb5+joKF9++aVcvXr1njGWl+q6b7nt7Nmz4uDgIKtXr5bs7GylffXq1eLp6SkFBQXFLnevIjgmJkaCg4Nl3759RZaPiYkRe3t75R8kEZH58+fLY489VmKMLVq0kC+//FJERKKiouSJJ54odr41a9ZIYGCgWdtvv/0m9vb2Jf5jV52UZxF8t02bNom7u7syXlwR/Mknn4inp+c9C0WR4j/nhw4dEkdHR7O29PR0s+PUvYrgESNGyNChQ83W065duxKL4FGjRslLL71kNv+gQYNk9OjRSoxardbsfR8wYMA9jyUA5JtvvlHGv/zyS9FqtVJYWKi0tWzZUmbPni0iRY/DHTp0kBEjRkhSUpJZvyaTSdRqtWzevLnEdT+IkopgXg5BD62VK1eiZcuWcHV1hbOzM7788kukpaVVen9eXl7K32q1GgCUrxSTkpIQFRUFvV6vDIsXLy7xDtkNGzYgPj4eTZo0QaNGjcy+Cr97XU5OTrC1tYW7u7tZ26P+deZtycnJAIB69eqZtaelpSEvLw+BgYFKm0ajgYeHB1JTU5W2u1/L4trufi39/f3Npru7uytf793tr7/+gp+fnzLu6Oho9l7dXl/NmjXN2ubMmYMmTZrAxcUFer0emzZtKpKHd8Zx++8747hXTt6ppNfw9vZt2bIFP/74I4KCgtCiRYsiX7Vbm8DAQKxYsQLz58+Ht7c32rVrh23btiE5ORmBgYFmlzaV1pAhQzBy5Ei8/fbbcHV1Rb9+/cwus/Hw8FDyE7j1ft9+r00mEz788EPUr18fzs7O0Ov1iIuLU/IlOTm52PcWuLVvSklJMds3denSBSqVCpcuXbJ4Ox5mhw8fRvfu3eHp6QmdTocXX3zxvvv+6OhoNG7cGAMHDrxv/3d/zpOSknDz5k3Url1bee2DgoLg4OBgdtlWSe7etwAoMn6n1NRUs/0hAAQFBZntDz08PGBra6uMq9Xq+x5L7t5fenh4wMbGxqytpD5iYmJgMpnQrl07BAQE4IMPPoDRaERaWhpycnJKzNvyxiKYHkqpqakYMmQIIiIicPHiRWRmZmLcuHG3z1SbufNDWZb+StPP3fz8/DBlyhRkZGQoQ3Z2NrZs2VLs/CEhIVi9ejWuXLmCefPmYdKkSdi1a5fF67UGt4u/+Ph4s3Z3d3fUrFkTSUlJSpvBYMCVK1fg4+NTpnXeLhoBIDc3F2lpaSU+caROnTo4d+6cMn7jxo0iB9a7c+qnn35CeHg45s2bh/T0dGRkZKBXr15F8vrOOG7//SBPPinpNbytY8eO2LhxI9LT0xEREYEhQ4bg7NmzFq/nUdKvXz9s374d6enpeOGFF9C7d294enoiKSnpntdQlsTOzg7h4eH47bffcO7cOTg5OeGVV15Rpl+5cgW5ubnKeHJysvJer1y5EgsWLMC6detw/fp1ZGRkICQkRMkXf3//Et9bPz8/1KtXz2zflJGRgby8PNSpU8fi7XiY3P25GzRoEEJDQ3HmzBlkZWVh5cqV9+1j8eLFMJlM6N+/P27evGnR+vz8/KBWq3Ht2jWz1/7GjRto27btfdd9974FwD2LZx8fH7P9IQAkJiaWeX9YFgEBAVi0aBHOnz+PjRs3YsGCBVi6dCnc3d2hVqtLzNvyxiKYHkoGgwEmkwnu7u6oUaMGDh48iGXLlhU7r7u7O2xsbO75oSpNf56enhZ/MN966y3Mnj0bsbGxKCwsRH5+Pg4fPozffvutyLz5+flYsmQJ0tPToVKp4OLiAhsbmwc6u2QNPDw8MGDAAIwdOxbJyckQESQkJCAxMRH//Oc/8cEHH+DChQvIzc3F+PHj0aBBAzz55JNlWufnn3+Os2fPIi8vD++99x4CAwPRqlWrYucdOnQoZsyYgaSkJOTl5WHSpEkwmUz37D8rK0s5u69SqfD999/jhx9+KDLftGnTcPnyZWRlZSE8PBxdunSBt7e3xdtT0muYkJCAy5cvY926dcjMzIStra3yPOs7zxZZm9OnT+PHH39Ebm4uatSoAWdnZ6hUKvTv3x/29vZ4++23kZmZicLCQhw8eLBU38rs2rULhw8fRkFBARwdHaFWq80+8yaTCe+99x5u3LiBxMRE/Pvf/8awYcMA3MoXOzs7uLu7w2QyYdGiRYiLi1OWHTJkCE6fPo2oqCjk5uaioKAAO3fuBAD84x//UG5Ays7Ohojgr7/+Mrth6lHl6elpdrY9KysLzs7O0Gq1SElJwWeffXbfPjQaDbZs2QKj0Yhnn30WOTk5pV7/E088gWbNmuHNN99Ubp5MS0vDqlWrSrX8P//5T6xduxa7d+9GYWEhvvnmGxw8eLDE+YcPH45169Zh69atKCwsxA8//IBvv/0WL7/8cqljLm9LlixRvhHV6/Wws7ODnZ0dVCoV/t//+3+YOHEiTpw4oeTl77//XiFxsAimh1LDhg0xdepU9O7dG3q9Hp999hlefPHFYud1dHTEtGnT8OKLL0Kv12P69OkAgJ49e+LVV18tdX8RERFYvnw5XFxc0LNnz1LF+fTTT+Orr77ChAkT4ObmBi8vL7z99tslPi9x9erVaNCgATQaDZ577jlMnToV7du3L+3LYnUWLVqEZs2aoUOHDtBqtejduzcuXbqEzz//HE888QRatmwJX19fXLx4ERs3bixzATdy5Ej069cP7u7uiIuLw4YNG5Q+P/nkEzRu3FiZd9KkSejWrRuefPJJ+Pv7w8vLC97e3kUuf7hT9+7dMXToUDz55JNwc3PD2rVr0bdv3yLzDRkyBGFhYfDx8UF+fn6ZfhimpNfQZDLhyy+/hL+/P7RaLcaNG4clS5aYXYphbfLz8zF16lTlCRBz5szBunXroFarsWvXLqSmpuKxxx6Dq6srJkyYUOLjsDQajfKUmMuXL2Po0KFwcXGBl5cXzp07h3nz5inz+vn5oU6dOggICECrVq3Qo0cPTJw4EQAwbNgwtGrVCsHBwahTpw5OnTqFsLAwZVlvb2/s2bMH27dvR926dVG7dm1ERUUBuPVV9c6dO3Hq1Ck0aNAAzs7O6NKlC44dO1ZRL1+18fbbb+O3336DXq9H48aN8dVXX2HBggXQarXo168fnn/++VL14+DggO+++w6urq7o1q0bMjIyEBsbC41Gc88zszY2Nvjuu+9gMpnQokULaLVatGrVCnv27CnVetu3b4/o6Gi88sorcHFxwaZNm9CnT58S9y1t27bFkiVL8O6778LFxQUTJ07E8uXL0bp161KtD7j1gy+ffPJJqee/n127dqFFixbQaDRo06YNXnrpJQwePBgAMH36dLzwwgvo06cPtFotOnToUGFnhlXFfX1MdC8qlUqYN1QaKpWq2EtUHkYqlQqxsbFo167dAy1vMBjg4uKCvXv3luorz+KU1w+/VFePUr6Uh8WLFyMyMvKej+KzVswVc23atEGvXr34Yzgl+DtfVHe380wwEVEFuH79On788UcUFBQgMzMTb7zxBvz8/NCyZcuqDo2IHnLr1q2DwWBAfn4+vvrqK/z2228YMGBAVYf10GERTERUAQoLCxEREYFatWohICAA58+fx6ZNm1CjRo2qDo2IHnJr165F3bp14erqirlz52L9+vWV9kSFRwkvhyCL8XIIKi1+ZUmWYL5QaTFXyBK8HIKIiIiI6G8sgomIiIjI6rAIJiIiIiKrwyKYiIiIiKwOi2AiIiIisjosgomIiIjI6rAIJiIiIiKrwyKYiIiIiKwOi2AiIiIisjosgomIiIjI6thV1oocHR0v5eXl1a6s9VHFcXBwgEpV5NcHiYpgrpAlmC9UWswVsoSDg4OpuHZVZf32tkqlEv7O96OBv9lOpcVcIUswX6i0mCtkib/zpch/TbwcgoiIiIisDotgIiIiIrI6LIKJiIiIyOqwCCYiIiIiq8MimIiIiIisDotgIiIiIrI6LIKJiIiIyOqwCCYiIiIiq8MimIiIiIisDotgIiIiIrI6LIKJiIiIyOqwCCZ6SBUWFmLChAlwd3eHVqtF//79kZ6eft/l5s6dC5VKhcjISLN2lUoFJycnaDQaZcjMzKyo8KkSJSQkoGvXrlCr1ahbty5mzpx5z/nDw8PRuHFj6HQ6eHt7Y9SoUbh27ZoyffHixbCxsTHLlRdffLGiN4OqAUtzadWqVQgLC4NOp4OdnV0lRUmVxdJ8eP/99xEQEACdTgcPDw8MGDAAKSkpyvS4uDj07NkTXl5eUKlU2L9/f4XG/0gUwSICo9FY1WEQVarPPvsMGzZswC+//ILz588DAIYOHXrPZc6dO4eZM2eiSZMmxU7ftm0bDAaDMjg7O5d73FS5CgsL0atXLzRs2BBpaWnYuHEjoqKisHr16hKXsbW1xfLly3H16lXExcXh/PnzePnll83mCQwMNMuVlStXVvSmUBV7kFxycXHB2LFjMXv27EqMlCrDg+TD0KFDcezYMWRlZSE5ORm+vr4YNGiQMt3e3h79+vXDxo0bK2MTbhWQlTHcWlXpzZ49W+rXry8ajUZ8fHzkvffeE6PRqEwHILNnz5YWLVqIg4ODHDhwQEREvvrqK2ncuLHodDpp1qyZbN26tdj+jUajeHt7y7fffmvWPnToUHnllVdERKSgoECmTp0qAQEB4uLiIp07d5bff/9dmXfYsGEyYsQIs+X9/Pxk2bJlFm3rw8bS95Iqhq+vryxYsEAZT0hIEACSlJRU4jJdunSRVatWSYcOHWTatGlm0wBIbGxsucbIXKl6u3btEkdHR8nOzlbaIiIipGPHjqXuY/PmzaLT6ZTxmJgYCQoKKtc4RZgv1V1Zcmn37t1ia2tbbrEwV6peWfctBoNBxo8fL7Vq1Sp2enkek/7OlyK1abU9E1y3bl388MMPyMrKwoYNG7Bo0SIsWLDAbJ6FCxdi9erVMBgMaN68Ob766itERUVhxYoVuH79OqZPn45+/fohISGhSP+2trYYMWKEWZ+ZmZlYu3YtRo0aBQCYMWMGli5dii1btuDixYsICwtDt27dkJWVVbEbT3QfmZmZSElJQYsWLZS2oKAg6HQ6HD9+vNhl5s2bBycnJwwcOLDEfp9//nm4ubmhVatW+Pbbb8s9bqp8cXFxqFevHjQajdIWGhqKuLi4Uvexc+dONG3a1KwtNTUVnp6e8PHxwaBBg5CUlFRuMVP1VB65RI+OB82Hr7/+Gs7OztBoNIiOjsaUKVMqONKSVdsiuH///ggICIBKpULz5s0xdOhQ7Ny502yed999F0FBQbC1tUXNmjUxZ84cfPjhhwgJCYGNjQ2eeeYZdOrUCatWrSp2HSNHjsT27dvx119/Abj1xgQFBaF169YAgJiYGISHh6NBgwaoWbMmPvzwQ9ja2uL777+v2I0nuo/b/4jdfbmCXq8v9p+0lJQUREZGYu7cuSX2uWPHDiQlJeH8+fN45513MHjwYPz444/lGzhVuuzs7FLnSXHWrVuH+fPnIzo6Wmlr3749fv/9d1y4cAGHDh2Cg4MDunXrhpycnHKNnaqXsuYSPVoeNB9eeuklZGZm4uLFi5gyZUqJl+dVhmpbBK9cuRItW7aEq6srnJ2d8eWXXyItLc1sHn9/f7PxpKQkjBs3Dnq9Xhl2796tFLl38/X1Rbdu3RATEwMAWLBggXIWGLh1piMwMFAZt7Gxgb+/P1JTU8tpK4kejFarBYAiN65lZGRAp9MVmX/kyJGIiIhAnTp1SuyzS5cucHBwgIODAwYOHIghQ4ZgxYoV5Rs4VbgVK1aY3bCm1WpLnSd3W7NmDUaNGoWNGzciNDRUaQ8MDES9evVgY2MDT09PzJ8/HxcuXMDBgwfLfXuo6pRnLtHDr7zzwdPTE6NGjcI//vEPsxtvK1O1LIJTU1MxZMgQRERE4OLFi8jMzMS4ceNuX1ussLExD9/Pzw+LFi1CRkaGMhgMhnue/RozZgwWLVqEo0eP4tSpU2Y3Fvn4+Jh9xWcymZCcnAwfHx8AgEajMTvzYTQaceXKlTJtO1Fp6PV6+Pr64siRI0pbYmIisrKyinxtDQDbt2/H5MmT4ebmBjc3N/z000/49NNPERYWVuI6bGxsinzmqPobPHiw2Q1rISEhOHPmjNm+6ujRowgJCblnPzExMRgzZgw2bdqETp063XNelUoFlUrFfHnElFcu0aOhIvLBaDQiJycHFy5cqIiQ76+4C4UrYoAFF7GfOnVKAMhPP/0kJpNJDhw4IB4eHtKhQwezi5zvvmD6q6++kkaNGsnRo0fFZDJJbm6uxMbGyh9//FHiuoxGo9SpU0eaN28ugwcPNps2ffp0eeyxx+T06dNy8+ZN+fjjj6V27dqSmZkpIrduDnF3d5fExETJy8uTiRMnip2dHW+Mo0oRGRkp9erVk8TERMnMzJQBAwZI9+7di503NTXVbGjdurVMmDBBLl26JCIiv//+u/zyyy9y8+ZNyc/Pl/Xr14ujo6Ns2LChTDEyV6qe0WiUBg0ayBtvvCG5ubly9OhR8fDwkJUrV5a4THR0tNSqVUt+/fXXYqdv3rxZUlNTxWQyydWrV2X06NHi6+trdoPMg2C+VG8PkktGo1Fu3LghW7duFVtbW7lx44bcuHFDTCZTmWJhrlQ9S/OhsLBQ/vOf/8jly5dF5NZxqU+fPuLv7y8FBQUiImIymZQcASA7d+6UGzdumD0Y4UGghBvjqmURLCIydepUcXNzE51OJ71795Y333zzvkWwiMjixYulWbNm4uzsLG5ubvL000/L8ePH77muDz/8UADInj17zNrz8/Plww8/FD8/P9Hr9dKxY0eJi4szmz5ixAjR6/VSp04d+eKLL/h0CKo0RqNRxo8fL66urqLRaKRv376SlpYmIiLLly8XtVpd4rJ3Px1i165d0qhRI3FychK9Xi8tWrS454GttJgr1UN8fLx07txZHB0dxcvLS2bMmGE2vUePHjJmzBhlHIDY2dmJWq02G2579913xcvLS5ycnMTT01P69+8vp0+fLnOczJfqz9JciomJEQBFhns9xaY0mCvVgyX5UFhYKD179hR3d3dxcnISb29veemllyQhIUGZPykpqdh8iYmJKVOcJRXBKqmkr69UKpVU1rostXjxYnz66ac4ffp0VYfyUODXnlRazBWyBPOFSou5Qpb4O19Ud7dXy2uCK1N2djaio6PxxhtvVHUoRERERFRJrLoInj17NmrXrg0/Pz+MHj26qsMhIiIiokrCyyHIYvwaikqLuUKWYL5QaTFXyBK8HIKIiIiI6G8sgomIiIjI6rAIJiIiIiKrwyKYiIiIiKwOi2AiIiIisjosgomIiIjI6rAIJiIiIiKrwyKYiIiIiKwOi2AiIiIisjosgomIiIjI6thV1oocHBxMKpWKRfcjwMHBASpVkV8fJCqCuUKWYL5QaTFXyBIODg6m4tpVlfXb2yqVSvg7348G/mY7lRZzhSzBfKHSYq6QJf7OlyL/NfHMLBERERFZHRbBRERERGR1WAQTERERkdVhEUxEREREVodFMBERERFZHRbBRERERGR1WAQTERERkdVhEUxEREREVodFMBERERFZHRbBRERERGR1WAQTERERkdVhEUz0kCosLMSECRPg7u4OrWP370cAACAASURBVFaL/v37Iz09/b7LzZ07FyqVCpGRkUpbbGwsNBqN2WBnZ4emTZtW5CZQJUlISEDXrl2hVqtRt25dzJw5857zh4eHo3HjxtDpdPD29saoUaNw7do1ZXphYSHCw8Ph4+MDrVaLJk2aYO3atRW9GVQJLM2V20wmE9q2bQuVSoXz588r7cyVR5+lx6Iff/wRjRs3hqOjIx5//HFs27ZNmXbt2jW0b98eHh4e0Ol0CAoKQmRkJESkYoIXkUoZbq3q0ZGfn1/VIVSZR+29fFhFRkbKY489JmfPnpWMjAzp16+f9OjR457LJCcnS1BQkDRp0kSmTZtW4nyFhYXi6+srUVFRZYqRuVL1jEajNGjQQF577TXJycmRw4cPi7u7u6xatarEZSZNmiRHjhyR/Px8uXLlivTo0UOee+45ZXp0dLR4eXnJn3/+KSaTSdavXy81atSQP/74o0yxMl+q1oPkym3//ve/pUuXLgJAUlNTlXbmyqPPkmPR2bNnxdHRUZYtWyY3b96U5cuXi5OTkyQlJYmISF5enpw4cUKpsRITE6VBgwYyb968MsX4d74UrU2La6yIwZKE/e9//ytNmzY1a0tISBBbW1tJTk4WEZG4uDjp1KmT6PV6CQgIkGnTponRaBQRkaSkpCIfxJiYGAkKCipxnStXrpSmTZuKVqsVT09PGT16tBgMBmW6n5+fTJ06VTp27ChOTk6ycuVKERFZv369hIaGirOzszRo0ECWL19e6u18WHHnUz34+vrKggULlPGEhAQBoOxMitOlSxdZtWqVdOjQ4Z5F8KZNm8Te3l6uXLlSphiZK1Vv165d4ujoKNnZ2UpbRESEdOzYsdR9bN68WXQ6nTL++uuvy4svvmg2j6enp6xZs6ZMsTJfqtaD5srp06clMDBQjh49WuTYy1x59FlyLPrwww+lXbt2Zm3t2rWTKVOmFNt3YmKiNGrUSN55550yxVhSEVwtL4cYPHgwzp49i0OHDiltCxcuRNeuXeHn54fMzEx069YNnTp1wqVLl/D9999j0aJFmDVr1gOv09nZGV9//TUyMjIQGxuL2NhYs6+LAWD+/PmYNWsWDAYDevfuje3bt2PEiBGYPXs2rl27hiVLluC1117Dvn37HjgOotLIzMxESkoKWrRoobQFBQVBp9Ph+PHjxS4zb948ODk5YeDAgfft/3//+x/69+8Pd3f3couZqkZcXBzq1asHjUajtIWGhiIuLq7UfezcudPs0phRo0bhxIkTOHXqFAoLC7F27VoYjUa0b9++XGOnyvUguWIymfDKK69gxowZ0Ov1RaYzVx5tlh6L4uLizOYFis+xf/zjH3B0dERgYCCys7MxZsyYConfrkJ6LSOdTodBgwZh4cKFaNmyJQoLC7FkyRLMmTMHAPD999/D3t4eERERUKlUaNiwIcLDwzFr1ixMmDDhgdbZs2dP5e/g4GCMHTsWS5cuNZtn1KhRaN68OQDA0dER0dHRePPNNxEWFgYAePLJJzFkyBAsXbqUH3CqUFlZWQBu/fN2J71er0y7U0pKCiIjI3Hw4MH79p2amooffvgBu3btKp9gqUplZ2eXOk+Ks27dOsyfPx979+5V2gIDAxEWFobHH38cNjY2qFmzJpYtWwYPD49yjZ0q14PkSnR0NDw9PdGvXz8kJycXmc5cebRZeiwqKcdOnjxp1rZ582YUFhbi0KFD2LRpE9zc3Mo58luq5ZlgABgzZgxWrlyJ3NxcbNmyBUajEc899xyAWwdpf39/qFQqZf6goCCkpqY+8Pq2b9+OsLAwuLu7Q6fTITw8HGlpaWbz+Pv7m40nJSUhKioKer1eGRYvXowLFy48cBxEpaHVagHc+i/8ThkZGdDpdEXmHzlyJCIiIlCnTp379j1//nzUr18fHTp0KJ9gqVKtWLHC7AZHrVZb6jy525o1azBq1Chs3LgRoaGhSvvYsWNx9OhRJCUlIT8/H9u3b8err75qdoMLVX9lzZWEhATMnDkTX3zxRYnrYK482iw9FlmSY7a2tmjdujWcnZ0xbty4coz6/1TbIrhly5YICgrCmjVrsHDhQgwfPhw1atQAAPj4+ODcuXNmdwsmJibCx8cHAJSvcnJycpTp9ypM8/Pz0adPHwwaNAgpKSnIyspCVFRUkbsRbWzMXy4/Pz9MmTIFGRkZypCdnY0tW7aUbeOJ7kOv18PX1xdHjhxR2hITE5GVlVXsEx22b9+OyZMnw83NDW5ubvjpp5/w6aefKt9i3GY0GrFw4cIK++qJKt7gwYNhMBiUISQkBGfOnDHbHx49ehQhISH37CcmJgZjxozBpk2b0KlTJ7Nphw8fxtChQ+Hn5wcbGxu0bdsWYWFh+OGHHypkm6hilDVX9u/fj7S0NDz++ONwc3NT/lFq2rQp/vvf/wJgrjzqLD0WhYSEmM0L3H9/ZDQaER8fX35B36m4C4UrYsADXMQ+d+5cadKkidSoUUPOnDmjtGdkZIi7u7tERkbKzZs35c8//5TAwECzO9n9/Pxk4sSJYjQa5fjx4+Lv71/ijXHZ2dliY2MjK1asEBGRkydPSv369cXPz8+sv2XLlpktt3XrVvH29pZ9+/aJ0WiUmzdvym+//SaHDh2yeFsfJg/yXlL5i4yMlHr16kliYqJkZmbKgAEDpHv37sXOm5qaaja0bt1aJkyYIJcuXTKb79tvvxVHR0e5du1aucTIXKl6t+/4f+ONNyQ3N1eOHj0qHh4eys29xYmOjpZatWrJr7/+Wuz00aNHy1NPPSXnz58XEZGDBw9KrVq1ZOnSpWWKlflStSzNlZycHLP9yoEDBwSAHDp0SLm5jrny6LPkWJSQkCCOjo7y9ddfS35+vnz99ddmT4c4cOCA7NixQ3Jzc8VoNMrevXvFw8NDIiIiyhQjHqanQ9yWlZUlGo2m2DtTjx49Kh07dhS9Xi9+fn4yZcoUKSgoUKbv2bNHGjVqJGq1Wrp27SrTpk2759Mh5s2bJ3Xq1BG1Wi0dO3aUqVOn3rcIFrl113SrVq1Er9dLrVq1JCwsTHbv3m3xtj5MuPOpHoxGo4wfP15cXV1Fo9FI3759JS0tTUREli9fLmq1usRlS3o6RPfu3WX48OHlFiNzpXqIj4+Xzp07i6Ojo3h5ecmMGTPMpvfo0UPGjBmjjAMQOzs7UavVZsNtmZmZMmbMGPH29haNRiNBQUEyffr0MsfJfKl6lubKnYp7MhNz5dFn6bHohx9+kEaNGomDg4M0atRItm7dqkzbu3evhIaGikajEa1WKw0aNJDIyEjl6V8PqqQiWCV3feVfUVQqlVi6LhFBYGAgpk+fjpdeeqmCIiNLqVSqIpeKEBWHuUKWYL5QaTFXyBJ/54vq7vZqe00wcOui/fz8fAwYMKCqQyEiIiKiR0i1fEQaALi7u8POzg4LFy6Evb19VYdDRERERI+Qan05BFVP/BqKSou5QpZgvlBpMVfIEg/l5RBERERERBWBRTARERERWR0WwURERERkdVgEExEREZHVYRFMRERERFaHRTARERERWR0WwURERERkdVgEExEREZHVYRFMRERERFaHRTARERERWR27ylqRg4ODSaVSseh+BDg4OEClKvLrg0RFMFfIEswXKi3mClnCwcHBVFy7qrJ+e1ulUgl/5/vRwN9sp9JirpAlmC9UWswVssTf+VLkvyaemSUiIiIiq8MimIiIiIisDotgIiIiIrI6LIKJiIiIyOqwCCYiIiIiq8MimIiIiIisDotgIiIiIrI6LIKJiIiIyOqwCCYiIiIiq8MimIiIiIisDovgO6xYsQIhISFVHQYRERERVTAWwXcYPHgw4uLiyqWv4cOHY+TIkeXSF1FxCgsLMWHCBLi7u0Or1aJ///5IT0+/73Jz586FSqVCZGSkWfuWLVvQokULODs7w9vbG6+//jry8vIqKnyqRAkJCejatSvUajXq1q2LmTNn3nP+VatWISwsDDqdDnZ2dkWmx8bGIjQ0FLVq1YKzszNCQ0Px7bffVlT4VImYK2QJS/MlPDwcjRs3hk6ng7e3N0aNGoVr164p09esWYPHH38cLi4ucHFxQbt27bB3794Ki59FMNFD6rPPPsOGDRvwyy+/4Pz58wCAoUOH3nOZc+fOYebMmWjSpIlZ+5UrV9CvXz+MGDEC169fx6+//oo9e/Zg2rRpFRY/VY7CwkL06tULDRs2RFpaGjZu3IioqCisXr26xGVcXFwwduxYzJ49u9jp9evXx/r163H16lVkZGRg9uzZGDJkCP7444+K2gyqBMwVssSD5IutrS2WL1+Oq1evIi4uDufPn8fLL7+sTG/dujW2b9+O69ev4+rVq3jjjTfwzDPPICMjo2I2QkQqZbi1qtJZu3at1KtXTxmPiIgQAHL27FkRETl48KDodDopKCiQnJwc6du3r9SuXVu0Wq00b95ctm3bpiyblJQkTz/9tDg7O4ter5fQ0FD5888/i11vTEyMBAUFKeMdOnSQadOmmc0DQGJjY0VE5MiRI/LUU0+JTqcTFxcXadOmjVy7dk2ioqLEzs5O7OzsRK1Wi1qtFqPRWOrtr+4seS+p4vj6+sqCBQuU8YSEBAEgSUlJJS7TpUsXWbVqVZHcPnz4sACQvLw8pe29996TZ599tkwxMleq3q5du8TR0VGys7OVtoiICOnYseN9l929e7fY2trec57CwkKJjY2VmjVrysaNG8sUK/OlajFXyBJlyZfbNm/eLDqdrthpRqNR1q5dKwDk+PHjZYr173wpUptWyzPBnTt3RkJCAlJSUgAAO3bsQHBwMHbs2KGMd+zYEXZ2djCZTOjXrx/i4+Nx9epVvPjii+jfvz/S0tIAAJMnT4avry8uX76M9PR0xMTEQK/Xl0uc48aNw9NPP41r167h8uXLmDVrFuzt7TFx4kQMHjwYw4YNg8FggMFggK2tbbmskwgAMjMzkZKSghYtWihtQUFB0Ol0OH78eLHLzJs3D05OThg4cGCRac2aNUPPnj0xb948GI1GnDt3Dhs3bkSfPn0qbBuocsTFxaFevXrQaDRKW2hoaLlc+qXX61GzZk2EhYWhVatWePrpp8vcJ1Ud5gpZojzyZefOnWjatKlZW0pKCvR6Pezt7TFgwAAMHDiwyLeX5aVaFsEuLi4IDQ3Fjh07kJWVhZMnT+L999/H9u3bAdwqgrt27QoA0Gg0GDJkCLRaLWrUqIEJEybA3t4ehw4dAgDY29vj0qVLSExMhK2tLZo2bYratWuXS5z29vZISUlBamoqatSogdatW0OtVpdL30T3kpWVBQBwdnY2a9fr9cq0O6WkpCAyMhJz584ttj8bGxsMHz4c06dPh4ODA/z9/dG8eXOzr6no4ZSdnV3qPLFURkYGDAYD1q9fj2eeeabYa0Lp4cFcIUuUNV/WrVuH+fPnIzo62qzd19cXGRkZyMrKQkxMDDp16lRuMd+tWhbBANC1a1fs2LEDu3fvRps2bfDMM89g9+7dMBgMOHDggFIE37hxA6+//joCAwOh0+mg1+tx/fp15UzwjBkzEBAQgF69esHLywuvv/46DAZDucQYExMDk8mEdu3aISAgAB988AGMRmO59E10L1qtFsCtM8J3ysjIgE6nKzL/yJEjERERgTp16hTb3+7duzFs2DDExMTg5s2buHTpErKyslgEP4RWrFgBjUajDFqtttR58iBq1qyJPn36YO/evViwYEG59EmVg7lClijPfFmzZg1GjRqFjRs3IjQ0tNh51Go1hg8fjujoaGzdurVctuFu1boI3rVrF7Zv345u3brBw8MDderUwezZs+Hq6oqGDRsCAGbNmoW9e/di586dyMzMREZGBlxcXG5fhwx3d3fMmTMHCQkJ+Omnn7Bnzx7861//KlUMGo0GOTk5yviFCxfMpgcEBGDRokU4f/48Nm7ciAULFmDp0qUAbp1ZI6ooer0evr6+OHLkiNKWmJiIrKysIl8tAcD27dsxefJkuLm5wc3NDT/99BM+/fRThIWFAQAOHz6Mpk2b4plnnoGtrS1q166NUaNGYdOmTZW2TVQ+Bg8erFyGZTAYEBISgjNnzpjty44ePVruj4M0Go2Ij48v1z6pYjFXyBLllS8xMTEYM2YMNm3aVKqzvBWZL9W2UmvXrh2ysrKwbNkydOvWDQDQpUsXzJgxQzkLDNz6WrhmzZpwdXVFfn4+Pv74Y7O7CFevXo2kpCSICJydnWFvb1/qr2GeeOIJbNiwAWlpacjOzsb7779vNn3JkiVKYazX62FnZ6f07enpicTERJhMpjK9DkQlGT16NKKiopCUlISsrCyEh4eje/fu8Pf3LzJvamoq4uLicOzYMRw7dgxPPPEExo0bh7Vr1wIA2rRpg99//x3btm2DiCA9PR3z588v8T90eni0b98efn5+mDx5Mm7cuIFjx45h3rx5GDNmTInLFBYWIi8vD/n5+QCAvLw85OXlKScX1q1bh99//x1GoxF5eXmYP38+du3ahe7du1fKNlHFYK6QJR4kX+bMmYN3330XW7duxVNPPVVk+tKlS5GQkACTyYTs7Gx8/PHHSElJQefOnStmI4q7W64iBjzAnZzdunUTDw8PMZlMIiLy/fffCwBZunSpMs+lS5eka9euolarpU6dOjJjxgwJCgqSmJgYEREJDw8XHx8fcXJyktq1a8vIkSMlJyen2PXd/XSIjIwM6du3r2i1WgkMDJQ1a9aYPR3in//8p3h6eirrnjhxovIUiLNnz8qTTz4per1enJ2d+XQIKndGo1HGjx8vrq6uotFopG/fvpKWliYiIsuXLxe1Wl3issU9+WTx4sXy+OOPi1arFXd3dxkwYICkpKSUKUbmSvUQHx8vnTt3FkdHR/Hy8pIZM2aYTe/Ro4eMGTNGGY+JiREARYbbTx75z3/+I8HBwaJWq8XFxUVat24t33zzTZnjZL5UPeYKWcLSfAFg9uSs28Nt77//vvj6+oqTk5O4ublJ586dZceOHWWOEyU8HUIlf/+3VtFUKpVU1roe1MKFCzFz5kycOnWqqkOp1lQqFar7e0nVA3OFLMF8odJirpAl/s4X1d3t1fZyiKpw5MgRBAcHV3UYRERERFTB+IySv7Vs2RI3btzAsmXLqjoUIiIiIqpgvByCLMavoai0mCtkCeYLlRZzhSzByyGIiIiIiP7GIpiIiIiIrA6LYCIiIiKyOiyCiYiIiMjqsAgmIiIiIqvDIpiIiIiIrA6LYCIiIiKyOiyCiYiIiMjqsAgmIiIiIqvDIpiIiIiIrI5dZa3IwcHBpFKpWHQ/AhwcHKBSFfn1QaIimCtkCeYLlRZzhSzh4OBgKq5dVVm/va1SqYS/8/1o4G+2U2kxV8gSzBcqLeYKWeLvfCnyXxPPzBIRERGR1WERTERERERWh0UwEREREVkdFsFEREREZHVYBBMRERGR1WERTERERERWh0UwEREREVkdFsFEREREZHVYBBMRERGR1WERTERERERW56Esgjt27IjIyMiqDuOeVCoV9u/fX9VhEBEREVExHsoimFhkE1BYWIgJEybA3d0dWq0W/fv3R3p6erHz7tmzByqVChqNRhnatm1rNo/RaMRHH30EPz8/qNVqBAUF4YcffqiMTaEqlpCQgK5du0KtVqNu3bqYOXPmPedv3LixWS45OjpCpVLhyJEjlRQxVRRLcwEAduzYgdatW0Oj0cDNzQ1jx441m3727Fn07dsXzs7OcHZ2RuvWrVFQUFBRm0CVrDyPRdeuXUP79u3h4eEBnU6HoKAgREZGQkQqJHYWwaXADytVR5999hk2bNiAX375BefPnwcADB06tMT5bW1tYTAYlOHnn382m/7qq69i27Zt2Lp1KwwGA2JjY9GwYcMK3QaqeoWFhejVqxcaNmyItLQ0bNy4EVFRUVi9enWJy5w8edIsl9555x00atQIoaGhlRg5lbcHyYU9e/ZgwIABePfdd3H16lWcP38eI0eOVKanpaUhLCwMISEhSElJwbVr1/DFF1/A1ta2MjaJKkF5HovUajXmzp2Lv/76C1lZWdixYwdWrFiB+fPnV0zwIlIpw61VlV52draMHz9eAgICRKPRSKNGjSQ2NlZERDp06CDvvPOO9OvXTzQajQQGBsp3331ntvz69eslNDRUnJ2dpUGDBrJ8+XKz6fv27ZOnnnpKXFxcJDAwUP7973+LyWQSEZHdu3eLra2tLF26VFm/iEh0dLT4+/uLRqMRb29vmTRpUonxA1DivXLlirRp00ZGjBghBQUF940vJiZGgoKCJDo6WurUqSN6vV5Gjx4tRqNRRESaNm0qAMTR0VHUarWMGDHCote2rCx9L6li+Pr6yoIFC5TxhIQEASBJSUlF5r2d0yX5888/BYD88ccf5Rojc6X627Vrlzg6Okp2drbSFhERIR07dizV8gUFBeLp6SnR0dFljoX5UrUeJBdat24t4eHhJU5/7733pFWrVuUapwhzpTopz2PR3RITE6VRo0byzjvvlCnGv/OlaG1aXGNFDJYm7AsvvCDt2rWTxMREMZlMcubMGYmPjxeRW0Wwq6ur7N+/XwoLC2XWrFni7OwsOTk5IiKybds2qVWrluzbt08KCwvll19+Eb1eL3v37hURkRMnTohGo5HvvvtOjEaj/PHHH+Lv7y9LliwRkVtvEgB58cUXJSMjQ3JycuT06dPi6OgoJ06cEBGR69evy4EDB+75gsfGxsrp06clODhYpk2bpky7X3wxMTFiZ2cnkydPlry8PImPjxcXFxezQvnOIruycedT9TIyMgSAHD161Kxdp9PJhg0bisx/O6fr1q0rtWvXlmeeeUaOHTumTJ87d67odDqZOXOmeHt7i4+Pj7z66quSlZVVpjiZK9Xf559/LiEhIWZt3377rbi4uJRq+TVr1oijo6Ncv369zLEwX6qWpblgMBjExsZGPvroI2nevLm4urpKhw4d5NChQ8o8rVq1kiFDhshzzz0nLi4u0qRJkyInpR4Ec6V6KO9j0W3PPvusODg4CADx8fGR06dPlynOkorgank5xJUrV/DNN9/gf//7HwICAqBSqfDYY48hODhYmWfgwIF46qmnYGNjg9GjRyMzMxPx8fEAgOjoaLz55psICwuDjY0NnnzySQwZMgRLly4FAMydOxfPP/88evfuDVtbWzRo0ACvvfaaMv22zz77DM7OznBycoKdnR1ERPkaUK/Xo3Xr1vfcjtjYWHTo0AFTpkxBRESE0n6/+ADA0dERH3/8MWrWrIng4GB06dIFv/32W5lfW3o0ZGVlAQCcnZ3N2vV6vTLtTg0aNMCxY8eQlJSEP//8E02bNkXnzp1x4cIFAEB6ejqysrJw6NAh/PHHH/jll19w7NgxvPPOOxW/MVSlsrOzS51HxZk3bx4GDhwIvV5fEeFRJbI0F65fvw6TyYT58+dj8eLFuHDhAp5++mk888wzyMjIAHBr3/L1119j6NChuHLlCmbOnIkRI0bwnpZHRHkfi27bvHkzDAYDDhw4gKFDh8LNza1C4q+WRXBycjIAoF69eiXO4+XlpfytVqsB3PoAA0BSUhKioqKg1+uV4fYH9Pb0lStXmk2fOnUqLl68qPRpY2MDHx8fZTwwMFC5LsXb2xvt2rXDtm3b7rkd0dHRaNy4MQYOHGjWfr/4AMDDw8Psmim1Wq1sH5FWqwUAZGZmmrVnZGRAp9MVmd/T0xMhISGws7ODXq/Hp59+ilq1aik3vt3ub9q0adDpdPDy8kJ4eDg2bNhQwVtClW3FihVmN6VotdpS59Hdzp49i507d+LVV1+tqHCpApU1F27vN15++WU0bdoU9vb2mDRpEgoKCpTrPLVaLdq0aYMBAwbAzs4O3bp1Q48ePbBx48aK3TiqFOV9LLqTra0tWrduDWdnZ4wbN65C4q+WRbC/vz8AKGd2LeXn54cpU6YgIyNDGbKzs7FlyxZl+iuvvGI2PSsrCydPnlT6UKlUUKlUZv3269cP27dvR3p6Ol544QX07t0bubm5JcaxePFimEwm9O/fHzdv3ix1fKVxd2xkXfR6PXx9fc3uxk9MTERWVhaaNm1aqj5sbGyUO26bNWsGoGheMc8ePYMHDza7KSUkJARnzpxBTk6OMs/Ro0cREhJy377mzZuHkJAQtGrVqiJDpgpS1lxwdnaGv79/sfuJ223NmjW753R6uJX3sag4RqPxgevB+667QnotIw8PDwwYMABjx45FcnIyRAQJCQlISEgo1fJvvfUWZs+ejdjYWBQWFiI/Px+HDx9WLicYO3YsVq1ahU2bNqGgoABGoxGnTp3C3r17S+zz9OnT+PHHH5Gbm4saNWrA2dkZKpUKNjYlv4QajQZbtmyB0WjEs88+q+xY7hdfaXh6elZYUtDDYfTo0YiKikJSUhKysrIQHh6O7t27K/9E3mnXrl1ISEiAyWSCwWDAlClTcPnyZXTv3h0AEBYWhiZNmuCjjz5CTk4Orly5ghkzZqBfv36VvFVU2dq3bw8/Pz9MnjwZN27cwLFjxzBv3jyMGTPmnsvl5+dj8eLFPAv8CHmQXBg7dixiYmJw6tQpGI1GzJgxAw4ODspjr8aMGYODBw/iu+++g8lkwu7du7Ft2zb06dOnsjaLKlh5HosOHjyInTt34saNGygsLMS+ffsQHR2Nnj17VkzwxV0oXBEDLLyIPSsrS958803x9fUVtVpd5OkQd95odvui5ztvFNu8ebO0atVK9Hq91KpVS8LCwmT37t3K9J9//lk6d+4srq6u4uLiIi1btpQ1a9aISPF3Lx4/flxat24tOp1OdDqdhIaGypYtW0qM/8548vPz5YUXXpA2bdooN4/cK77bT4e407Bhw8yeArFo0SKpW7eu8uSIymTpe0kVw2g0yvjx48XV1VU0Go307dtX0tLSRERk+fLlolarlXlnzZolvr6+4uTkJO7u7tK9e3f59ddfzfpLD9v+HgAAIABJREFUTk6Wnj17ilqtFm9vb3n99dfFYDCUKUbmysMhPj5eOnfuLI6OjuLl5SUzZswwm96jRw8ZM2aMWdvKlStFo9GYPUmgrJgvVc/SXDCZTPLBBx9I7dq1xdnZWTp27FjkJqlvvvlG6tWrJ05OTtK4cWP55ptvyhwnc6X6KM9j0d69eyU0NFQ0Go1otVpp0KCBREZGKk/HelAo4cY4lVTQA4jvplKppLLWRRVLpVJV2IOr6dHCXCFLMF+otJgrZIm/86XINTjV8nIIIiIiIqKKxCKYiIiIiKwOi2AiIiIisjosgomIiIjI6rAIJiIiIiKrwyKYiIiIiKwOi2AiIiIisjosgomIiIjI6rAIJiIiIiKrwyKYiIiIiKwOi2AiIiIisjosgomIiIjI6rAIJiIiIiKrY1dZK3JwcDCpVCoW3Y8ABwcHqFSqqg6DHgLMFbIE84VKi7lClnBwcDAV164SkUoJQKVSSWWtiyqWSqUC30sqDeYKWYL5QqXFXCFL/J0vRf5r4plZIiIiIrI6LIKJiIiIyOqwCCYiIiIiq8MimIiIiIisDotgIiIiIrI6LIKJiIiIyOqwCCYiIiIiq8MimIiIiIisDotgIiIiIrI6LIKJiIiIyOo8lEXw8OHDMXLkyFLPr9FocODAgQqMCNizZw/s7OwqdB1EREREVD4eyiLYUgaDAW3atCm3/hYvXozg4OBy6+82lUqF/fv3l3u/9GgqLCzEhAkT4O7uDq1Wi/79+yM9Pb3E+a9cuYJhw4bB1dUVOp0OzZo1w4ULF5TpI0eOROPGjWFnZ2fRP5n08EtISEDXrl2hVqtRt25dzJw5857zh4eHo3HjxtDpdPD29saoUaNw7dq1SoqWKlJ558Krr74KjUZjNqhUKsyaNauiN4UqgaX5smrVKoSFhUGn0933xOHcuXOhUqkQGRlZniGbsYoimOhR9Nlnn2HDhg345Zf/z969R0VVr/8Df28GFZgLoICgchGSo2LgsZtZKt61ohQ9SxM53lI8trJvGaHmSUstlYOmZUdS00ztoq4SNTXvecuyECs7IoICXhJUGEAJGJ7fH+b+OQHGCDOA836ttddy789n7/18Zh73fmbP3sNRZGdnAwCio6Mr7VtcXIxevXqhcePGOHXqFPLy8rB27VrodDq1T2hoKBYsWICnn37aJvFT/WAymRAREYF27dohJycHSUlJmDdvHj777LMq19FoNFizZg2uXLmClJQUZGdnY/To0TaMmqzBGrmwdOlSFBYWqtMXX3wBR0dHDBs2zBZDIiu6m3xxd3fHxIkT8c4779xx2+fOnUNCQgLuv//+2g7bnIjYZLq5q+orKCiQyZMnS+vWrUWn00n79u3lwIEDIiIycuRIGTt2rNp36tSp0rp1a9FqtRIYGCgLFy402xYAdd2VK1dKUFCQLFiwQFq2bCk6nU4mT54subm5EhkZKXq9Xv72t7+p/f/s8OHD0qRJE1EURbRarWi1Wtm7d6/s3btXNBqNfPrppxIYGCgGg0H+8Y9/iNForFacoaGhAkCcnZ1Fq9Waja++sfS9JOvw8/OT5cuXq/NpaWkCQDIyMir0Xbp0qbRq1UpKSkr+crt//v9VE8yV+m/Pnj3i7OwsBQUF6rLp06dLeHh4tbexZcsWMRgMNY6F+VK3bJELgwcPlkGDBtUoThHmSn1Qk3y5VTNVpVevXvLpp59K9+7dZdasWTWO9Y98qVCb1tsrwWPHjsXRo0exe/duGI1GfPnll/D29q60b/v27XHw4EEUFBRg2bJlmDp1Knbs2FHlts+dO4e8vDykp6fj4MGDePfddzFgwADExsbi2rVriIyMrPKqxqOPPoqlS5ciMDBQ/WQbHh4O4Oanoq+//hopKSlITU1FcnIyFi9eXK04U1JSAABff/01CgsLsXz58rt52chO5OfnIzMzEw888IC6LCgoCAaDASdOnKjQf+/evWjfvj1iYmLQrFkztG3bll9HEoCbx57g4GCzbwU6deqkHpOqY/fu3QgNDbVGeGRD1s6FS5cuISkpCRMmTKhxrFT3aiNfKpOYmAgXFxcMHTq0piH+pXr5JNfly5fx+eef4+eff0br1q0BAG3atKmy/4gRI9R/9+zZE08++SR2796Nfv36Vdrf2dkZM2bMgIODA8LCwhAWFoaHHnoInTt3Vrf39ttvIz8/H66urhbFPnfuXPW+p4EDB+LYsWN3HSdRVYxGIwBUyE83Nze17Xa5ubnYvXs33nnnHSxduhQnTpxA//790bx5c0RFRdkkZqqfCgoKqp1Hldm4cSOWLVuG/fv3WyM8siFr58KKFSvg6+uLPn361DhWqns1zZfKZGZmYvbs2fj2229rGl611MsrwWfPngUABAcHV6v/4sWLcf/998Pd3R1ubm7YvHkzcnJyquzv5eUFB4f/P3QXFxf4+PiYzQM332BLaDQaeHp6qvNardZsG5bGSVQVvV4P4OYV4dvl5eXBYDBU2r9ly5Z48cUX0bhxYzz44IMYMWIENm3aZJN4qf64dS/4rUmv11c7j/5s/fr1GDduHJKSktCpUydrhUxWYstcKC8vx7JlyzB+/HgoilJrYyDbqc18qcpzzz2H6dOno2XLljUNt1rqZREcEBAAADh9+vRf9j106BDi4uKQmJiI3Nxc5OXlISIi4tZ9yFZxewFdXdWJkwcGqi43Nzf4+fnhxx9/VJelp6fDaDRW+lVkx44dK80v5pz9iYqKMntQKSwsDKmpqSgqKlL7JCcnIyws7I7bWblyJWJiYrB582b06NHD2mGTFdgyF7Zv346LFy9izJgxtToGsp3aypc72blzJ6ZNmwYPDw94eHjg0KFDePvtt9G1a9faGEIF9bII9vLywpAhQzBx4kScPXsWIoK0tDSkpaVV6Gs0GtUrsIqiYOvWrdi2bZtV4/P29sbly5ctuuRfnTi9vb2rVfgTAcD48eMxb948ZGRkwGg0Ii4uDv369VM/RN5u1KhRuHLlCpYsWQKTyYSUlBSsXbsWkZGRap+SkhIUFxfDZDLBZDKhuLgYJSUlNhwR1YVu3brB398f06ZNw40bN3D8+HEkJiYiJiamynUWL16MV155BTt27MBjjz1mw2jJmqyZC4mJiYiMjDT7tpQatrvJlz+fW4qLi1FcXKxeEMzKykJKSgqOHz+O48eP48EHH8Tzzz+PDRs2WGcQlT0tZ40JFj7JaTQa5cUXXxQ/Pz/RarVV/jqEyWSSCRMmiJubm7i7u8uoUaMkKipKRo4cafZU4J9/HeJ2f376MCMjQwBIVlZWpbGVlpZKZGSkNG3aVFxdXWXfvn2VPuk4Y8YM6dWrV7Xj/PDDD6VVq1bi5uYm48ePt+j1siVL30uyjrKyMpk8ebI0a9ZMdDqdDBo0SHJyckREZM2aNaLVas367927Vzp27CguLi5y3333yXvvvWfW3r17dwFgNnXv3r1GMTJXGobTp09Lz549xdnZWXx8fCQ+Pt6svX///hITE6POAxBHR0f1F3JuTTXFfKl71siF7Oxs0Wg0sm/fvlqLk7lSP1iaLytXrqxwnkEVv2okUrE+u1uo4tchFLHibQO3UxRFbLUvsi5FUax6uwndO5grZAnmC1UXc4Us8Ue+VLj/r17eDkFEREREZE0sgomIiIjI7rAIJiIiIiK7wyKYiIiIiOwOi2AiIiIisjssgomIiIjI7rAIJiIiIiK7wyKYiIiIiOwOi2AiIiIisjssgomIiIjI7rAIJiIiIiK7wyKYiIiIiOwOi2AiIiIisjuOttqRk5NTuaIoLLrvAU5OTlAUpa7DoAaAuUKWYL5QdTFXyBJOTk7llS1XRMQmASiKIrbaF1mXoijge0nVwVwhSzBfqLqYK2SJP/KlwqcmXpklIiIiIrvDIpiIiIiI7A6LYCIiIiKyOyyCiYiIiMjusAgmIiIiIrvDIpiIiIiI7A6LYCIiIiKyOyyCiYiIiMjusAgmIiIiIrvDIpiIiIiI7E69LYIHDBiA+fPn19r2FEXBwYMHa217RERERNRw1dsieNu2bXj11VctXu/s2bNQFAXZ2dlWiIqofkhLS0Pv3r2h1WrRqlUrJCQk3LH/a6+9htatW8NgMMDLywtDhgxBZmZmpX3j4uKgKArWrFljjdCpjphMJsTGxsLT0xN6vR6DBw9Gbm5upX3feust6HQ6s0lRFEyaNEntExAQACcnJ7M+P/30k62GQ1Zi6bHllvLycnTp0qXC+XfChAmV5tKCBQusNQSyIUvzJS4uDiEhITAYDGjRogXGjRuHq1evmvU5c+YMBg0aBFdXV7i6uqJz584oLS21Svz1tgiu76z1hhD9FZPJhIiICLRr1w45OTlISkrCvHnz8Nlnn1W5TnR0NI4fPw6j0YizZ8/Cz88Pw4YNq9Dvu+++w7Zt2+Dj42PNIVAdmDt3LjZt2oSjR4+qRUp0dHSlfadNm4bCwkJ1Sk5OhqIoGDFihFm/5cuXm/W7//77rT4Osp67ObbcsnDhQri4uFRYvnTpUrMc+eKLL+Do6Fjp8YcalrvJF41GgzVr1uDKlStISUlBdnY2Ro8erbbn5OSga9euCAsLQ2ZmJq5evYr33nsPGo3GOoMQEZtMN3dVfd27d5dZs2aJiMjvv/8u48aNE09PT9Hr9dKmTRtZv359pesZDAYBIC4uLqLVauXNN98UuRmALFmyRB588EHR6XTyyCOPyK+//qquV1paKnPmzJE2bdqIq6urdOnSRY4dO6a2jxw5UoYPHy6jRo0Sd3d3mTBhgoiI/PTTT9K3b19p1qyZ+Pr6ypQpU6SkpMSisTY0lr6XVLv27Nkjzs7OUlBQoC6bPn26hIeHV2v9wsJCmTx5sjRt2tRseXFxsXTo0EEOHz4s/v7+8vHHH9c4VuZK/eHn5yfLly9X59PS0gSAZGRk/OW6kydPlr///e9my2orR27HfKlbd3tsOXXqlAQGBkpycrIAkKysrCr7Dh48WAYNGlTjWJkrda+m5yIRkS1btojBYFDnp0yZIo888kitximi5kuF2rRBXAletWoVvv/+e/z6668wGo3YvXs32rdvX2nflJQUAMCpU6dQWFiIf//732bb2bhxI3Jzc+Hr64sXXnhBbXv99dexadMmbN++HVeuXMGYMWPQr18/XLt2Te2zfv169O/fHzk5OUhISMDly5fRvXt3REZG4sKFCzhy5Ah27tyJt99+20qvBNHNHA8ODoZOp1OXderUSc39qqxbtw6urq7Q6XRYtGgRZs6cadY+c+ZM9OzZE48++qg1wqY6lJ+fj8zMTDzwwAPqsqCgIBgMBpw4ceKO6/7+++9YtWoVJkyYUKHt5ZdfRtOmTdGxY0ckJibWetxkW3dzbCkvL8eYMWMQHx8PNze3O27/0qVLSEpKqjSXqOG523PR7Xbv3o3Q0FB1fu/evWjTpg2eeeYZNG3aFKGhoVi7dm2txn27BlEEN27cGIWFhTh58iTKysrg6+tbZRF8J7GxsfDz80OTJk0watQoHDt2DMDNq+Hvvvsu4uPjERgYCI1Gg7Fjx8LHxwdbt25V13/88ccxdOhQaDQauLi4YPXq1QgLC0NMTAwaN26Mli1bYurUqVi9enWtjZ3ozwoKCuDq6mq2zM3NDUaj8Y7rDR8+HPn5+bh48SJmzpxp9tX1sWPHsH79esyZM8cqMVPdupUbd5M3GzZsQElJCYYPH262/KOPPkJ6ejp+++03xMfHY9q0aSyEG7i7ObYsWrQI3t7eiIyM/Mvtr1ixAr6+vujTp0+NY6W6d7fnols2btyIZcuWYdGiReqy3NxcrFu3DtHR0bh8+TISEhIwduxYq/2wQYMogkeMGIHnnnsOL730Epo1a4bIyEikpaVZvJ3b73PUarUoKCgAcPNFLywsREREBNzc3NQpPT3d7Ab/gIAAs+1lZGTg0KFDZuuMGTMGly5duruBElVi7dq1Zg+V6PV65Ofnm/XJy8uDwWCo1va8vb0xbtw4PPXUU7h69SpKSkowevRoLFmyxOwTPd079Ho9ANxV3iQmJiIqKqpCbnTv3h06nQ6NGjVCnz598PLLL/NhygampseWtLQ0JCQk4L333vvLfZWXl2PZsmUYP348FEWplfjJtmrzXLR+/XqMGzcOSUlJ6NSpk7pcr9fj0UcfxZAhQ+Do6Ig+ffqgf//+SEpKqvXxAA2kCHZ0dERcXByOHTuGc+fOwcXFBWPGjKm0r4OD5UPy8PCAVqvFrl27kJeXp05FRUWYMmVKldv29/dH7969zdbJz89HYWGhxTEQVSUqKsrswZKwsDCkpqaiqKhI7ZOcnIywsLBqb7OsrAxFRUW4cOECLly4gF9++QVRUVHw8PCAh4cHsrKy8K9//QtRUVHWGBLZmJubG/z8/PDjjz+qy9LT02E0Gs2+ivyzkydP4sCBA9X6+trBweHW8x/UQNT02HLw4EHk5OSgQ4cO8PDwUIuZ0NBQvP/++2Z9t2/fjosXL1Z57qb6r7bORStXrkRMTAw2b96MHj16mLV17Nix0g9JVvvgVNmNwtaYUIMH43bv3i3Hjh2TkpISKS4ulvHjx0uPHj0qXe/69evi4OAge/bsqXBT9IEDB9T5vXv3ikajUeenTZsm3bp1k9TUVBERKSgokO3bt8v58+dF5OaDcWPHjjXb5sWLF8XLy0tWrFghN27cEJPJJGfOnJFt27ZZNNaGxtL3kmpXWVmZtG3bViZNmiTXr1+X5ORk8fLykk8++aTS/iaTSd5991357bffREQkKytLBg4cKAEBAVJaWiplZWWSlZVlNrVq1UoWL14subm5NYqVuVJ/zJ49W4KDgyU9PV3y8/NlyJAh0q9fvzuuM2nSJOncuXOF5WfPnpU9e/bIjRs3pKysTPbt2yeenp6yePHiGsXIfKlblh5bioqKzI4bR44cEQDy/fffmz0sJSLy9NNPy7Bhw2otVuZK3bM0X0REFi1aJE2bNpXvvvuu0vYjR46Io6OjfPHFF2IymdSH7w4fPlyjWFHFg3ENoghet26dtGvXTrRarbi7u0u/fv3UYrUyc+bMkebNm4urq6vMnj1bfQHuVASXlpZKQkKCtGvXTvR6vXh7e8vAgQPVp1wrK4JFRH755ReJiIiQ5s2bi8FgkNDQUFmyZIlFY21oePCpe6dPn5aePXuKs7Oz+Pj4SHx8vFl7//79JSYmRkRuFsEDBgwQT09PcXFxkRYtWsjw4cMlLS2tyu3z1yHuPWVlZTJ58mRp1qyZ6HQ6GTRokOTk5IiIyJo1a0Sr1Zr1v379uri7u8uqVasqbOuXX36Rjh07ik6nE71eLyEhIfLuu+/WOEbmS92z5NjyZxkZGZX+OkR2drZoNBrZt29frcXJXKkfLM0XAOLo6ChardZsut3nn38uwcHB4uLiIiEhIfL555/XOM6qimBFbPT1laIoYqt9kXUpisKvPalamCtkCeYLVRdzhSzxR75UuKeiQdwTTERERERUm1gEExEREZHdYRFMRERERHaHRTARERER2R0WwURERERkd1gEExEREZHdYRFMRERERHaHRTARERER2R0WwURERERkd1gEExEREZHdYRFMRERERHaHRTARERER2R0WwURERERkdxxttSMnJ6dyRVFYdN8DnJycoChKXYdBDQBzhSzBfKHqYq6QJZycnMorW66IiE0CUBRFbLUvsi5FUcD3kqqDuUKWYL5QdTFXyBJ/5EuFT028MktEREREdodFMBERERHZHRbBRERERGR3WAQTERERkd1hEUxEREREdodFMBERERHZHRbBRERERGR3WAQTERERkd1hEUxEREREdodFMBERERHZHRbBlVAUBQcPHqzrMIiIiIjISu7ZInjmzJno3bv3HfucPXsWiqIgOzvbRlER1Y60tDT07t0bWq0WrVq1QkJCwh37x8XFISQkBAaDAS1atMC4ceNw9epVtX3ChAnQ6XRmk6IoWLBggbWHQjZiMpkQGxsLT09P6PV6DB48GLm5uZX2feuttyrNh0mTJpn1e/fddxEcHAytVgtfX198+OGHthgKWZGlx5ZbysvL0aVLlwrnVJPJhLi4OPj6+kKv1+P+++/Hhg0brBU+2Vhtn4tsnS/3bBFcn5SWltZ1CHQPMZlMiIiIQLt27ZCTk4OkpCTMmzcPn332WZXraDQarFmzBleuXEFKSgqys7MxevRotX3p0qUoLCxUpy+++AKOjo4YNmyYLYZENjB37lxs2rQJR48eVYuU6OjoSvtOmzbNLB+Sk5OhKApGjBih9pk9ezbee+89rFu3DgUFBTh+/Dgee+wxm4yFrONuji23LFy4EC4uLhWWL1myBB9//DF27doFo9GIWbNmYfjw4fjf//5njSGQDVnjXGTzfBERm0w3d1V9ubm5Eh0dLd7e3tK8eXP55z//KVeuXFHb/f395eOPP1bnMzIyBIBkZWXJp59+Ko0aNRKNRiNarVa0Wq2cOXOmwj4MBoMAEBcXF9FqtfLmm2+K3AxWlixZIg8++KDodDp55JFH5Ndff1XXKy0tlTlz5kibNm3E1dVVunTpIseOHVPbR44cKcOHD5dRo0aJu7u7TJgwQUREfvrpJ+nbt680a9ZMfH19ZcqUKVJSUmLR61IfWPpeUu3as2ePODs7S0FBgbps+vTpEh4eXu1tbNmyRQwGQ5XtgwcPlkGDBtUoThHmSn3i5+cny5cvV+fT0tIEgGRkZPzlupMnT5a///3v6vy1a9fE2dlZtm3bVqsxMl/q1t0eW06dOiWBgYGSnJysnodveeGFF+TZZ5816+/t7S3r16+vUazMlbpnjXORlfOlQm1ab68ER0VF4dq1azh58iR+/fVX5ObmVnnV4s+GDh2KadOmITw8XL2SERgYWKFfSkoKAODUqVMoLCzEv//9b7Vt1apV2LhxI3Jzc+Hr64sXXnhBbXv99dexadMmbN++HVeuXMGYMWPQr18/XLt2Te2zfv169O/fHzk5OUhISMDly5fRvXt3REZG4sKFCzhy5Ah27tyJt99++25fIrJTKSkpCA4Ohk6nU5d16tRJzefq2L17N0JDQyttu3TpEpKSkjBhwoQax0r1Q35+PjIzM/HAAw+oy4KCgmAwGHDixIk7rvv7779j1apVZvnw7bff4saNGzhz5gxat24NHx8fPPvss/jtt9+sNgayvrs5tpSXl2PMmDGIj4+Hm5tbhfZx48bh559/xsmTJ2EymbBhwwaUlZWhW7duVhkD2Y41zkW2zpd6WQRfuHABO3bswIIFC+Du7g53d3csWLAAX331FS5evGiTGGJjY+Hn54cmTZpg1KhROHbsGICbV87fffddxMfHIzAwEBqNBmPHjoWPjw+2bt2qrv/4449j6NCh0Gg0cHFxwerVqxEWFoaYmBg0btwYLVu2xNSpU7F69WqbjIfuHQUFBXB1dTVb5ubmBqPRWK31N27ciGXLlmHRokWVtq9YsQK+vr7o06dPjWOl+uFWbtxN3mzYsAElJSUYPny4uuzWvcQbN27E0aNH8euvv+LGjRtmt0tQw3M3x5ZFixbB29sbkZGRlbYHBgaia9eu6NChA5o0aYKRI0ciMTERXl5etRo72Z41zkW2zpd6WQRnZWUBAFq3bq0uCwoKMmuzNh8fH/XfWq0WBQUFAG4e/AsLCxEREQE3Nzd1Sk9PN3sYICAgwGx7GRkZOHTokNk6Y8aMwaVLl2wyHmq41q5da/aAkl6vR35+vlmfvLw8GAyGv9zW+vXrMW7cOCQlJaFTp04V2svLy7Fs2TKMHz8eiqLU2hiobun1egC4q7xJTExEVFSU2dWeW9ubNm0avLy84ObmhpkzZ2L37t0oKiqq5ejJWmp6bElLS0NCQgLee++9KvcxceJEJCcnIyMjAyUlJdi5cycmTJiAr7/+ulbHQtZni3ORrfOlXhbBvr6+AG7+esMt6enpZm06nc7sYHvhwgWzbTg4/PXQqtPnzzw8PKDVarFr1y7k5eWpU1FREaZMmVLltv39/dG7d2+zdfLz81FYWGhxDGRfoqKizB5SCgsLQ2pqqln+JycnIyws7I7bWblyJWJiYrB582b06NGj0j7bt2/HxYsXMWbMmFodA9UtNzc3+Pn54ccff1SXpaenw2g0VnlbDACcPHkSBw4cqHBrTMeOHQGAH5QauJoeWw4ePIicnBx06NABHh4eajETGhqK999/HwDwww8/IDo6Gv7+/nBwcECXLl3QtWtXbNu2zfoDpFpli3ORzfOlshuFrTHBwpvY+/btK08//bRcu3ZNrl69Kk899ZQMGDBAbR85cqR0795dCgoK5PLly9K/f3+zG/ITExOlTZs28vvvv1e5j+vXr4uDg4Ps2bOnwg3UBw4cUOf37t0rGo1GnZ82bZp069ZNUlNTRUSkoKBAtm/fLufPn1djGzt2rNk2L168KF5eXrJixQq5ceOGmEwmOXPmTK0/WGILlr6XVLvKysqkbdu2MmnSJLl+/bokJyeLl5eXfPLJJ1Wus2jRImnatKl89913d9z2008/LcOGDau1WJkr9cfs2bMlODhY0tPTJT8/X4YMGSL9+vW74zqTJk2Szp07V9r2xBNPSK9eveTKlStiNBolMjJS+vfvX6MYmS91y9JjS1FRkWRlZanTkSNHBIB8//336sNS48ePl8cee0yys7NFROTbb7+Vpk2byurVq2sUK3Ol7lnjXGTlfKlYm1a20BqTpQl7+fJliYqKkubNm4uXl5eMGDFCcnJy1PasrCzp2bOn6HQ6ad++vaxatcqsCL569ar07NlT3N3dxdXVVdLT0yvdz5w5c6R58+bi6uoqs2fPVl+sOxXBpaWlkpCQIO3atRO9Xi/e3t4ycOBAdd+VFcEiIr/88otERERI8+bNxWAwSGhoqCxZssSi16U+4MGn7p0+fVp69uwpzs7O4uPjI/Hx8Wbt/fv3l5iYGHUegDjnPNFaAAAgAElEQVQ6Oqq/lnJrul12drZoNBrZt29frcXJXKk/ysrKZPLkydKsWTPR6XQyaNAg9Zi6Zs2aCvlw/fp1cXd3l1WrVlW6vStXrsizzz4rer1ePUZfvny5RjEyX+qepceW293+K0235OfnS0xMjLRo0UJ0Op0EBQXJnDlzahwnc6V+qO1zkZXzpUJtqtxssz5FUcRW+yLrUhQFfC+pOpgrZAnmC1UXc4Us8Ue+VLh/q17eE0xEREREZE0sgomIiIjI7rAIJiIiIiK7wyKYiIiIiOwOi2AiIiIisjssgomIiIjI7rAIJiIiIiK7wyKYiIiIiOwOi2AiIiIisjssgomIiIjI7rAIJiIiIiK7wyKYiIiIiOwOi2AiIiIisjuOttqRk5NTuaIoLLrvAU5OTlAUpa7DoAaAuUKWYL5QdTFXyBJOTk7llS1XRMQmASiKIrbaF1mXoijge0nVwVwhSzBfqLqYK2SJP/KlwqcmXpklIiIiIrvDIpiIiIiI7A6LYCIiIiKyOyyCiYiIiMjusAgmIiIiIrvDIpiIiIiI7A6LYCIiIiKyOyyCiYiIiMjusAgmIiIiIrvDIpiIiIiI7A6L4AYmICAAa9asqbL9rbfeQkREhA0jIiIiImp4WATfY6ZNm4bNmzfXdRhkZWlpaejduze0Wi1atWqFhISEO/aPi4tDSEgIDAYDWrRogXHjxuHq1atqu8lkwqxZs9C6dWvodDp07doVJ06csPYwqB4xmUyIjY2Fp6cn9Ho9Bg8ejNzc3Cr7/+c//0FQUBD0ej3atGmD999/34bRkrVYemx57bXX0Lp1axgMBnh5eWHIkCHIzMxU2ydMmACdTmc2KYqCBQsWWHsoZAMN/VzEIthGSktL6zoEukeYTCZERESgXbt2yMnJQVJSEubNm4fPPvusynU0Gg3WrFmDK1euICUlBdnZ2Rg9erTavmDBAqxZswa7d+/G1atX0bVrV/Tr1w8FBQW2GBLVA3PnzsWmTZtw9OhRZGdnAwCio6Mr7ZuUlIQZM2Zg7dq1KCgowOrVqxEbG4udO3faMmSqZXdzbImOjsbx48dhNBpx9uxZ+Pn5YdiwYWr70qVLUVhYqE5ffPEFHB0dzfpQw3RPnItExCbTzV1VX0FBgUyePFlat24tOp1O2rdvLwcOHBARkaKiIpk0aZK0atVKmjVrJs8884ycO3dOXbd79+7y0ksvycCBA0Wn00lgYKDs2rVLdu7cKSEhIaLX62XgwIFiNBrVdQDIwoULJSwsTHQ6nYSHh8vp06erjK+kpET+7//+Tzw9PaV58+Yyb948CQoKkpUrV4qIyMqVKyUoKEjmz58vLVu2lPbt24uIyNSpU6V169ai1WolMDBQFi5cqG4zIyNDAMiyZcukTZs24urqKk8//bT89ttvah9/f3+ZM2eO9OzZU7RarYSEhMihQ4fU9hkzZkivXr2q9Tp+8skn0rZtW9HpdOLl5SUjR46s1ntj6XtJtWvPnj3i7OwsBQUF6rLp06dLeHh4tbexZcsWMRgM6vxDDz0k77zzjjpfUlIijRo1ko8++qhGsTJXGg4/Pz9Zvny5Op+WliYAJCMjo0LfhIQEefTRR82Wde7cWeLj42sUA/OlbtX02FJYWCiTJ0+Wpk2bVtln8ODBMmjQoBrHylypew3wXFShNq23V4LHjh2Lo0ePYvfu3TAajfjyyy/h7e0NAHjppZfw7bff4ttvv8W5c+fg4eGBiIgImEwmdf2PP/4YcXFxyMvLw9ChQxEdHY0PPvgA33zzDc6ePYtTp07h3XffNdvnBx98gA0bNuDy5csICQnB008/bbbN27399tvYtm0bvv32W2RkZCA7Oxvnzp0z63P27FlcuHABp0+fxvfffw8AaN++PQ4ePIiCggIsW7YMU6dOxY4dO8zWW716Nb755htkZmbCwcEBI0aMMGv/8MMPsXjxYuTn56NPnz4YOXKkxa/j9evXER0djSVLlqCgoADp6ekYO3bsX7wrVB+kpKQgODgYOp1OXdapUyekpKRUexu7d+9GaGioOl9eXn7rw6pKRHD8+PGaB0z1Xn5+PjIzM/HAAw+oy4KCgmAwGCr9KnLYsGEwGo04dOgQysvLceDAAaSmpqJ///62DJtq2d0eW9atWwdXV1fodDosWrQIM2fOrLTfpUuXkJSUhAkTJtRm2FRH7olzUWWVsTUmWPCp7bfffhMA8vPPP1doM5lM4uTkJF9//bW6rKCgQBo1aiSHDx8WkZtXgidOnKi2//LLLwJAvvvuO3VZbGysDBw40OxTwu1XQYqKiqRx48ZmV1lvFxQUJCtWrFDnr1+/Lo0bNza7Euzk5CTFxcV3HOvgwYMlNjZWRP7/leBdu3ap7adPnxYAcv78eRG5eSV4/vz5avvPP/8sACQvL09EzK8E3+l1LCoqEmdnZ1myZIlcuXLljjH+mSXvJdW+N998U7p162a2bM+ePaLRaKq1/oYNG0Sn08kPP/ygLps5c6bcd999kpqaKjdu3JBXX31VFEWRsWPH1ihW5krDkJmZKQAkPT3dbLmfn598/PHHFfqXlpbKjBkzpFGjRqLRaESj0ci7775b4ziYL3WrpseWixcvyuzZs2Xv3r2Vts+ePVsCAwOlvLy8pqEyV+qBBnguahhXgs+ePQsACA4OrtCWk5OD4uJiBAYGqst0Oh28vLyQlZWlLvPx8VH/7eLiUumyP99jEhAQYNbu6emp3hv3Z+fPn4e/v7867+zsDE9PT7M+Pj4+aNKkidmyxYsX4/7774e7uzvc3NywefNm5OTkVBnHrX/fHsft49BqtQBQ6f0yd3odXVxc8NVXX2H79u0ICgrCAw88gHXr1lU6Vqpba9euNXuoRK/XIz8/36xPXl4eDAbDX25r/fr1GDduHJKSktCpUyd1+ZQpUzBo0CD07dsXfn5+AIB27drBw8OjdgdD9ZJerweAaufVrFmzsG7dOhw/fhylpaVISUnBwoULsWLFCpvES7WjNo8tAODt7Y1x48bhqaeeMnvYCbh5hW/ZsmUYP348FEWptTGQ7dyL56J6WQTfKvxOnz5doc3T0xNNmjRBRkaGuqywsBCXL1+Gr69vjfZ7q2gEgOvXryMnJwetWrWqtG/Lli3Nbn+4ceNGhWLWwcH85T106BDi4uKQmJiI3Nxc5OXlISIiosKl/9vjuPXvquK4kzu9jgAQHh6OpKQk5ObmYvr06RgxYgTOnDlj8X7IuqKiosweLAkLC0NqaiqKiorUPsnJyQgLC7vjdlauXImYmBhs3rwZPXr0MGtr0qQJ5s+fj4yMDFy+fBmxsbFIT09HeHi4NYZE9Yybmxv8/Pzw448/qsvS09NhNBrNvqq85YcffsCgQYPQvn17KIqCkJAQDBw4EFu2bLFl2FRDtXVsuV1ZWRmKiopw4cIFs+Xbt2/HxYsXMWbMmFqLn2zrXjwX1csi+NbPrEycOBFnz56FiCAtLQ1paWlwcHDAP//5T/z73//GhQsXcP36dUyePBlt27bFww8/XKP9Lly4EGfOnEFxcTGmTJmCwMBAPPLII5X2jY6ORnx8PDIyMlBcXIypU6eivLz8jts3Go3QaDTw9PSEoijYunUrtm3bVqHfrFmz8Ntvv8FoNCIuLg69evVCixYtLB7PnV7H3377DRs3bkR+fj40Gg3c3NwA3Hxyk+q3bt26wd/fH9OmTcONGzdw/PhxJCYmIiYmpsp1Fi9ejFdeeQU7duzAY489VqH90qVL6geurKwsjBo1Co8++ij69etnrWFQPTN+/HjMmzcPGRkZ6rGnX79+Zt9M3fLYY4/hyy+/VD9g//rrr/jyyy/NruhQw2PpsaW8vBzvvfceLl++DODmN5bPP/88AgIC0LZtW7O+iYmJiIyMrPCNKTVc98K5qF4WwcDNh786duyI7t27Q6/X45lnnsGlS5cA3CxWH3zwQTz00EPw8/PDxYsXkZSUVOMC7rnnnlP/k6akpGDTpk1VbnPq1Kno06cPHn74YQQEBMDHxwctWrSocPvD7fr164fo6Gg8/PDD8PDwwIYNGzBo0KAK/UaMGIGuXbvC19cXJSUld/zjGH+lqtexvLwcS5YsQUBAAPR6PZ5//nl89NFHlZ7wqH7RaDTYvHkzfv75ZzRr1gxPPPEEYmNjzX5yaMCAAWYPn7z44oswGo3o0aOH2ddZt2RnZ6NPnz5wcXHBgw8+iICAACQlJfFrSzsyZcoURERE4KGHHkLLli1hMpnUY8+tr0FviY2NxaBBg9CnTx/odDr069cPAwcOxJQpU+oqfKoFd3Ns+eqrr9ChQwdotVo88sgjcHFxwa5du+Do6Kj2OX/+PLZu3coH4u4x98K5SPnzV/HWoiiK2Gpfd0NRFBw4cACPP/74Xa1fWFgId3d37N+/H126dLmrbZw9exatW7dGVlbWXd3+YCuKolS4hYOoMswVsgTzhaqLuUKW+CNfKlTS9fZKcH137do1bN++HaWlpcjPz8ekSZPg7++Phx56qK5DIyIiIqK/wCL4LplMJkyfPh1NmzZF69atkZ2djc2bN6NRo0Z1HRoRERER/QXeDkEW49dQVF3MFbIE84Wqi7lCluDtEEREREREf2ARTERERER2h0UwEREREdkdFsFEREREZHdYBBMRERGR3WERTERERER2h0UwEREREdkdFsFEREREZHdYBBMRERGR3WERTERERER2x9FWO3JycipXFIVF9z3AyckJilLhrw8SVcBcIUswX6i6mCtkCScnp/LKliu2+tvbiqII/873vYF/s52qi7lClmC+UHUxV8gSf+RLhU9NvDJLRERERHaHRTARERER2R0WwURERERkd1gEExEREZHdYRFMRERERHaHRTARERER2R0WwURERERkd1gEExEREZHdYRFMRERERHaHRTARERER2Z16WwQPGDAA8+fPt9n+srOzoSgKzp49a7N9EhEREVHdqLdF8LZt2/Dqq6/WdRhE9VJaWhp69+4NrVaLVq1aISEhoVrrlZeXo0uXLlAUBdnZ2WZtZ86cwaBBg+Dq6gpXV1d07twZpaWl1gif6oDJZEJsbCw8PT2h1+sxePBg5ObmVtn/P//5D4KCgqDX69GmTRu8//77atvVq1fRrVs3eHl5wWAwICgoCLNnz4aI2GIoZEXWOLbcEhcXB0VRsGbNmtoMmepQbefL6tWr0aVLF7i7u8PDwwMDBgzATz/9ZK3w628RTESVM5lMiIiIQLt27ZCTk4OkpCTMmzcPn3322V+uu3DhQri4uFRYnpOTg65duyIsLAyZmZm4evUq3nvvPWg0GmsMgerA3LlzsWnTJhw9elQ96URHR1faNykpCTNmzMDatWtRUFCA1atXIzY2Fjt37gQAaLVa/Pe//8X58+dhNBqxa9curF27FsuWLbPZeKj2WePYcst3332Hbdu2wcfHpzZDpjpkjXwpKCjAG2+8gezsbJw/fx6dOnVC3759cePGDWsMARARm0w3d1V93bt3l1mzZomISEZGhgCQ1atXS7t27USn00mfPn3kwoULVa7/ySefSGhoqOj1evH29pbx48dLYWGh2n7x4kWJiIgQg8Egbdq0kWXLlgkAycjIkJ9++kkaNWokly9fVvuXl5dLQECAfPTRRyIi8s4778jf/vY30el04uvrK1OmTJGysjK1PwBZsmSJPPjgg6LT6eSRRx6RX3/9VW0vKSmROXPmSHBwsOh0OgkMDJQNGzao7R988IGEhISIwWCQjh07yo4dOyx6/azJ0veSateePXvE2dlZCgoK1GXTp0+X8PDwO6536tQpCQwMlOTkZAEgWVlZatuUKVPkkUceqfVYmSv1h5+fnyxfvlydT0tLU495f5aQkCCPPvqo2bLOnTtLfHx8pdtOT0+X9u3by8svv1yjGJkvdcsaxxYRkeLiYunQoYMcPnxY/P395eOPP65xrMyVumetfLldYWGhAJAff/yxRrH+kS8VatMGdSX4s88+wzfffIPz58+jqKgIr7/+epV9XV1dsW7dOuTl5eHAgQM4cOAAZs+erbZHRUVBo9EgMzMT33zzDVatWqW2dejQAR07dsTatWvVZfv27cOVK1cwZMgQAECrVq2wbds2GI1GbNq0CR9++CGWL19uFsOqVauwceNG5ObmwtfXFy+88ILaNn36dKxZswbr16+H0WjE/v370aZNGwDABx98gHnz5mHt2rW4du0a5syZg8jISKSlpdXo9aN7Q0pKCoKDg6HT6dRlnTp1QkpKSpXrlJeXY8yYMYiPj4ebm1uF9r1796JNmzZ45pln0LRpU4SGhprlPzVs+fn5yMzMxAMPPKAuCwoKgsFgwIkTJyr0HzZsGIxGIw4dOoTy8nIcOHAAqamp6N+/v1m/p556Cs7OzggMDERBQQFiYmKsPhayHmscWwBg5syZ6NmzJx599NFaj5nqjrXy5Xa7d++Gi4sL7rvvvlqJ+c8aVBE8Y8YMeHh4wGAwYPjw4Th27FiVfQcMGICQkBA4ODjgvvvuw8SJE7F7924AwPnz57Fnzx785z//gaurK7y9vTFjxgyz9UePHo2VK1eq8ytXrsTQoUPVy/eDBw9G69atoSgK/v73vyM6Olrd/i2xsbHw8/NDkyZNMGrUKDVeEcGSJUsQHx+P0NBQKIqCVq1aITQ0FACwePFivP766wgLC4ODgwOeeOIJ9OjRA59++mnNX0Rq8AoKCuDq6mq2zM3NDUajscp1Fi1aBG9vb0RGRlbanpubi3Xr1iE6OhqXL19GQkICxo4di4MHD9Zq7FQ3buVGdfPGy8sLQ4YMQY8ePdC4cWP06NEDb7zxBjp06GDWb8uWLSgsLMSRI0cQHR0NDw8P6w2CrM4ax5Zjx45h/fr1mDNnTq3GSnXPGvlyu9TUVDz33HNISEiAXq+vcbyVaVBF8O33Emm1WhQUFFTZd+fOnejatSs8PT1hMBgQFxeHnJwcAFDvh/P391f7t27d2mz9Z599Fqmpqfjxxx9RUFCAjRs3YsyYMWr7J598goceegjNmjWDq6srlixZom7/r+LNyclBUVERgoODK409IyMDzz//PNzc3NRp7969OH/+/B1fH7o3rV27FjqdTp30ej3y8/PN+uTl5cFgMFS6flpaGhISEvDee+9VuQ+9Xo9HH30UQ4YMgaOjI/r06YP+/fsjKSmpVsdCdePWCaS6eTNr1iysW7cOx48fR2lpKVJSUrBw4UKsWLGiQl+NRoPOnTvD1dUVzz//vHUGQFZh7WNLSUkJRo8ejSVLlphdLaSGyRbnoltOnjyJHj164JVXXsGECRNqJf7KNKgiuLpKSkowcOBADBs2DJmZmTAajZg3b5765HLLli0BAOfOnVPXycjIMNuGm5sbBg4ciFWrVuHzzz+Hn5+f+lVOVlYWRowYgenTp+PixYvIz8/H888/X+0noz09PaHVanH69OlK2/39/fHhhx8iLy9PnQoLC/Hf//7X4teCGr6oqCgUFhaqU1hYGFJTU1FUVKT2SU5ORlhYWKXrHzx4EDk5OejQoQM8PDzQqVMnAEBoaKj6xH/Hjh2hKEqFdStbRg2Pm5sb/Pz88OOPP6rL0tPTYTQa1W+gbvfDDz9g0KBBaN++PRRFQUhICAYOHIgtW7ZUuY+ysrIqj2lUP1n72HLhwgX88ssviIqKgoeHBzw8PJCVlYV//etfiIqKsskYqfbY4lwEAD/++CPCw8MxZcoU6/9KWGU3CltjQi08GHf7zdMrV66UoKCgStctKCgQBwcHWbt2rYiI/PLLL/K3v/1N/P391T7h4eESGRkp+fn5cunSJenatWuFh0R27NghzZo1k4cffljmzZunLj958qQAkEOHDkl5ebkcOXJEvLy8pHv37mofAHLgwAF1fu/evaLRaNT5V155RUJCQuSnn36S8vJyyc7OlhMnTojIzYfi2rdvL8nJyVJeXi7Xr1+XAwcOmD1YV5csfS+pdpWVlUnbtm1l0qRJcv36dUlOThYvLy/55JNPKu1fVFQkWVlZ6nTkyBEBIN9//736QMORI0fE0dFRvvjiCzGZTOoDD4cPH65RrMyV+mP27NkSHBws6enpkp+fL0OGDJF+/fpV2vett96S4OBgSU1NFZGbx7zAwEB58803ReRmvuzatUuuX78uZWVlsn//fvHy8pLp06fXKEbmS92q7WNLWVmZWXtWVpa0atVKFi9eLLm5uTWKlblS96xxLjp48KC4ublJYmJircaKKh6MuyeLYBGRxMREadmypWi1WgkPD5c33njDrAi+cOGCPPnkk5X+OsQtJpNJfH19RaPRyMWLF822/8Ybb4iHh4cYDAZ55pln5MUXX7SoCP7999/ljTfekKCgINFqtRIUFCQbN25U21etWiUdO3YUV1dX8fDwkL59+6pFcl3jwafunT59Wnr27CnOzs7i4+NT4an9/v37S0xMTKXrVvb/SUTk888/l+DgYHFxcZGQkBD5/PPPaxwnc6X+KCsrk8mTJ0uzZs1Ep9PJoEGDJCcnR0RE1qxZI1qtVu1bWloqcXFx4u/vL1qtVnx9feXll1+WkpISERHZv3+/dOrUSXQ6nej1emnbtq3Mnj3b7Bdy7gbzpe5Z49hyO/46xL2ltvMlPDxcFEURrVZrNn3zzTc1irOqIlgRG/24uaIoYqt9kXUpisIfxadqYa6QJZgvVF3MFbLEH/lS4f6+e/KeYCIiIiKiO2ERTERERER2h0UwEREREdkdFsFEREREZHdYBBMRERGR3WERTERERER2h0UwEREREdkdFsFEREREZHdYBBMRERGR3WERTERERER2h0UwEREREdkdFsFEREREZHdYBBMRERGR3XG01Y6cnJzKFUVh0X0PcHJygqIodR0GNQDMFbIE84Wqi7lClnByciqvbLkiIjYJQFEUsdW+yLoURQHfS6oO5gpZgvlC1cVcIUv8kS8VPjXxyiwRERER2R0WwURERERkd1gEExEREZHdYRFMRERERHaHRTARERER2R0WwURERERkd1gEExEREZHdYRFMRERERHaHRTARERER2R0WwURERERkd+ptERweHo7Zs2fXyrbOnj0LRVGQnZ1daftbb72FiIiIWtlXVbZv34777rsPer0eCxYssOq+iIiIiOjO6m0RbEvTpk3D5s2brbqPSZMm4eWXX0ZBQQFefvllq+6L7n1paWno3bs3tFotWrVqhYSEhL9cZ9euXejcuTN0Oh08PDwwceJEtW39+vXo0KED3N3d4e7ujscffxz79++35hDIhj799FN07doVBoMBjo6Od+x748YN/OMf/0CbNm3g4OBQ6cWIxYsX45FHHoGLiwvuu+8+a4VN9ZSlx59Ro0ahUaNG0Ol06vT+++/bKFqyptrOhQkTJpi16XQ6KIpitYuHLIJtJD09HaGhoXe9fmlpaS1GQw2ZyWRCREQE2rVrh5ycHCQlJWHevHn47LPPqlxn3759GDJkCF555RVcuXIF2dnZeO6559T2zp07Y+fOnbh27RquXLmCSZMm4YknnkBeXp4thkRW5u7ujokTJ+Kdd975y76KoqBLly744IMP8PDDD1fap0WLFnj11Vfx2muv1XaoVM/dzfEHAEaOHInCwkJ1uv1DODVM1siFpUuXmrV98cUXcHR0xLBhw6wyhgZTBJ84cQI9e/aEu7s7AgMDMXv2bJhMpmq33y49PR1t27bFjBkzAAAzZ85E79691faAgAC89dZb6NWrF3Q6HTp06IDDhw+r7aWlpXjppZfg5eUFb29vzJ8/H/fddx9WrVpVYV8XLlyATqeDyWRC3759odPpkJqairKyMrz55psIDAxE06ZN0atXL/z888/qeqNGjUJUVBRGjx6Npk2bYtKkSTV9Ceke8c033+DcuXN4++234eLigk6dOiEmJgZLly6tcp2pU6diwoQJGDJkCJo0aQInJyd06tRJbff19YWPjw8AQESg0Whw/fp1ZGVlWX08ZH39+vXDs88+i8DAwL/s6+TkhJdeegk9evSAk5NTpX2GDBmCwYMHo2XLlrUdKtVzd3P8oXuTLXIhMTERERERaNGiRa1t83YNogjOz89Hnz590KNHD1y6dAlbt27Fhx9+qF4e/6v223377bfo2rUrpkyZgjfeeKPKfX744YdYvHixuu2RI0eqbW+//Ta2bduGb7/9FhkZGcjOzsa5c+cq3U6LFi1QWFgIAPj6669RWFiI4OBgxMfHY/Xq1fjqq69w8eJFdO3aFX369IHRaFTXXb9+Pfr374+cnJxqfd1N9iElJQXBwcHQ6XTqsk6dOiElJaXS/kVFRfjuu+/UwtfDwwPh4eE4duyYWb/MzEy4ubmhcePGGDJkCIYOHYr777/fqmMhoobF0uPPLRs3bkTTpk0RHByM2NhY9bxIDZe1c+HSpUtISkrChAkTajXu2zWIInjr1q1o3Lgxpk+fjiZNmqBdu3aIi4vD8uXLq9V+y4YNGzBw4EB89NFHGDVq1B33GRMTg5CQEGg0Gjz33HNIS0tDfn4+AGD16tV49dVXERgYCGdnZ8ybNw8ODpa9lCtXrkRcXBzatm2LJk2a4PXXX4dGo8HWrVvVPo8//jiGDh0KjUYDFxcXi7ZP966CggK4urqaLXNzczP7AHW7a9euoby8HMuWLcOqVatw4cIF9O3bt8LtDn5+fsjLy4PRaMTKlSvRo0cPq46DiBoeS48/APDCCy/gf//7H3Jzc/HFF19g//79GDdunLVDJSuzdi6sWLECvr6+6NOnT63GfbsGUQRnZWUhICAAiqKoy4KCgtSvav+q/Za5c+eif//+Zrc+VOXWV8MAoNVqAdx8wwHg/Pnz8Pf3V9udnZ3h6elp8Zhu/2rSwcEBAQEBZjEHBARYtE26N61du9bsIQG9Xq9+ILslLy8PBoOh0vX1ej0AYPTo0QgNDUXjxo0xdepUlJaWmt3mc4tWqy1oWs8AACAASURBVMWoUaOwaNEi7Nixo/YHREQNRk2PPwDwwAMPoHnz5nBwcEBISAgWLlyIDRs24Pfff7d2+FSLbJkLty7cjB8/3qy2q20Nogj29fXFuXPnICLqsvT0dPj6+lar/ZYtW7bghx9+wL/+9S+zvpZq2bKl2e0PN27cQE5OjkXb8PX1RUZGhjpfXl6Os2fPmsVs6dVlujdFRUWZPSgQFhaG1NRUFBUVqX2Sk5MRFhZW6fqurq4VPiTecqeDS1lZGU6fPl3zARBRg1XT409lbp3banIeJtuzZS5s374dFy9exJgxY2on+Kr2b9Wt15Inn3wSxcXFeOutt1BSUoJTp05h3rx5GDt2bLXab/H29sb+/fvxww8/IDo6GmVlZXcVT3R0NOLj45GRkYHi4mJMnToV5eXlFm1j1KhRmD9/PlJTU1FSUoI5c+agrKwMTz755F3FRPajW7du8Pf3x7Rp03Djxg0cP34ciYmJiImJqXKdiRMnYuXKlTh58iTKysoQHx8PJycndOnSBcDNW3zS0tJQXl6OgoICvPnmm8jMzETPnj1tNSyyIpPJhOLiYpSUlAAAiouLUVxcXGUR8vvvv6O4uBjl5eUoKytDcXGx2S/U3L5MRNTt0b3vbo4/n376qXrr1enTpzF58mQ8/fTTVT54SQ2DNXMhMTERkZGRFn/LbjERscl0c1fV1717d5k1a5Y6n5ycLOHh4eLm5ib+/v4yc+ZMKS0trVZ7RkaGAJCsrCwRETEajdKtWzd55plnpLi4WGbMmCG9evVSt+Xv7y8ff/yxOv/n9X///Xd54YUXxMPDQ5o3by5z584VPz8/WbduXZXjASAHDhxQ50tKSuT1118Xf39/cXNzk/DwcElJSVHbR44cKWPHjrXoNbMVS99Lqn2nT5+Wnj17irOzs/j4+Eh8fLxZe//+/SUmJkadLy8vl3//+9/SvHlzcXV1lfDwcElOTlbbX3vtNfHz8xMXFxfx8PCQnj17yq5du2ocJ3Olfli5cqUAqDBlZGTIN998I1qtVs6dO6f29/f3r9B35MiRavuMGTMq3V5NMV8aBkuPP927dxd3d3dxcXGRgIAAeemllyQ/P79GMTBX6gdr5EJ2drZoNBrZt29frcX5R75UqE0VsdHXEYqiiK32ZWuFhYVwd3fH/v371Str9zJFUfg1FlULc4UswXyh6mKukCX+yJcK9/81iNsh6ptr165h+/btKC0tRX5+PiZNmgR/f3889NBDdR0aEREREVUDi+C7YDKZMH36dDRt2hStW7dGdnY2Nm/ejEaNGtV1aERERERUDbwdgizGr6GoupgrZAnmC1UXc4UswdshiIiIiIj+wCKYiIiIiOwOi2AiIiIisjssgomIiIjI7rAIJiIiIiK7wyKYiIiIiOwOi2AiIiIisjssgomIiIjI7rAIJiIiIiK7wyKYiIiIiOyOo6125OTkVK4oCovue4CTkxMUpcJfHySqgLlClmC+UHUxV8gSTk5O5ZUtV2z1t7cVRRH+ne97A/9mO1UXc4UswXyh6mKukCX+yJcKn5p4ZZaIiIiI7A6LYCIiIiKyOyyCiYiIiMjusAgmIiIiIrvDIpiIiIiI7A6LYCIiIiKyOyyCiYiIiMjusAgmIiIiIrvDIpiIiIiI7A6LYCIiIiKyO/W2CB4wYADmz59f12EQERER0T2o3hbB27Ztw6uvvqrOK4qCgwcP1mFERA1LWloaevfuDa1Wi1atWiEhIeGO/ePi4hASEgKDwYAWLVpg3LhxuHr1qo2iJWuyNBdee+01tG7dGgaDAV5eXhgyZAgyMzPV9pSUFAwYMAA+Pj48Nt+DTCYTYmNj4enpCb1ej8GDByM3N7fSvvv27YOiKNDpdOrUpUsXtf3q1avo1q0bvLy8YDAYEBQUhNmzZ0NEbDUcsqKGfmypt0UwEd09k8mEiIgItGvXDjk5OUhKSsK8efPw2WefVbmORqPBmjVrcOXKFaSkpCA7OxujR4+2YdRkDXeTC9HR0Th+/DiMRiPOnj0LPz8/DBs2TG1v3LgxIiMjkZSUZIshkI3NnTsXmzZtwtGjR5GdnQ3gZk5URaPRoLCwUJ0OHz6stmm1Wvz3v//F+fPnYTQasWvXLqxduxbLli2z+jjIuu6JY4uI2GS6uavq6969u8yaNUtEREJDQ/8fe/ceFlW59g/8OzIhh2Fg5CAiIIqiiElhva/mCVBDUiuVdm0PiSnSm2W/QsLM7SFtb91qhttKXy3cCqkbtQR3vlZ4LitNwbaaQoCIRxBhACWY4f79Uc7VBCgjzIDO93Nd67pc63nWWvczc7vWPWvWGgSA2Nvbi6Ojo0yZMqVO/6NHj4pKpZLq6moREVm3bp0AkD179oiIyOXLl6VNmzZy5coVERGJjo4Wb29vUalUEhgYKCkpKYZtlZSUSFRUlLRr107UarUEBQXJgQMHTIr/fmbqe0mWt2fPHrG3t5fy8nLDsjlz5khoaGijt7Fz505Rq9VNioO50vKamgsVFRUSFxcn7dq1q7cdgBw8eLBZYmW+tA6+vr6ybt06w3xOTo4AkLy8vDp99+7dKzY2No3edm5urvTs2VNef/31JsXIXGl59+CxpU5tek9cCc7KygIAfPHFF6ioqMC6devq9Hn44YfRtm1bHD58GADw1VdfoWvXrvjyyy8N87169YKHhwcAYMCAAcjMzERpaSnmzp2L6OhonDp1CgCwdOlS3LhxA+fOnUNpaSm2b98Ob29vSwyVqFlkZWUhICAAKpXKsCwkJMTwf6kxMjIy0Lt3b3OERxZ0t7nwySefwNnZGSqVComJiZg/f76ZI6XWoKysDAUFBejTp49hmb+/P9RqNU6cOFHvOnq9Hj4+PvD09MSIESPqza2RI0fC3t4eXbp0QXl5OWJjY802BrKM++HYck8UwY3Rpk0bhIWF4auvvoKIYM+ePVi0aJFRETx06FBD/ylTpsDV1RU2NjZ47rnn0Lt3b+zbtw/Ar5fjr127hjNnzkBEEBAQgM6dO7fEsIjuSnl5OZydnY2Wubi4QKvVNmr9bdu2Ye3atUhMTDRHeGRBd5sL48aNQ1lZGS5duoT58+fjwQcfNGeY1ErcyovG5kyPHj2QmZmJvLw8/PTTT+jduzfCw8Nx8eJFo347d+5ERUUFDh8+jIkTJ8LNzc18gyCLuB+OLfdNEQwAQ4cOxVdffYUTJ07AxcUFUVFR+Pnnn1FSUoKMjAxDEVxbW4u5c+eie/fucHZ2houLC7KyslBUVAQAiI+Px5AhQzBp0iS4u7tj0qRJuHLlSksOjei2UlJSjB5McXJyQllZmVGf0tJSqNXqO24rNTUVMTExSEtLQ0hIiLlCJjNpzlwAAE9PT8TExGDkyJF8UNIKODk5AUCjc8bT0xPBwcFQKpVwcXHB3/72N7Rr1w67du2q09fGxgZ9+/aFs7Mzpk+fbp4BkNncj8eWe6YIVigUd+wzbNgwHDlyBNu3b8ewYcNgY2ODgQMH4oMPPsDly5cxaNAgAMCmTZuwbt06bNu2DdevX0dpaSmCg4MNT6s6OjrinXfewX/+8x+cPHkSFy5cQHx8vFnHR9QU48ePN3owJTg4GGfPnkVlZaWhz/HjxxEcHHzb7SQlJSE2Nhbp6ekICwszd9hkBs2VC7+n0+lQWVlZ5+oe3X9cXFzg6+uLY8eOGZbl5uZCq9U2+vaoNm3a3PbXH3Q6HbKzs5scK1nW/XhsuWeKYE9Pzzv+p+nSpQt8fHzw3nvvYdiwYQCAIUOGYOnSpejXrx8cHR0B/Pp1j1KphLu7O2pra/Hxxx8b3cOSnp6O06dPQ6/XQ6VSwc7ODkql0nyDI2pmgwYNQqdOnTB79mzcvHkTmZmZWLNmzW3vw1u5ciVmzpyJ3bt3o3///haMlszJ1Fyora3FqlWrcPXqVQBAYWEhpk+fDj8/P/To0QPArw9UV1VVoaqqCgBQXV2Nqqoq6PV6ywyKzGratGlYsmQJ8vLyoNVqkZCQgIiICPj5+dXpu2fPHuTk5KC2thYVFRWYP38+rly5goiICADAt99+i4yMDNy8eRN6vR4HDhxAYmIiIiMjLTwqam73xbGlvqflzDGhCb8OISLy8ccfi7e3t7i4uMi0adMaXC8mJkaUSqWUlZWJiMjJkycFgLz99tuGPpWVlRIVFSUqlUo8PDwkLi5OwsLCZN68eSIismLFCvH39xcHBwdxdXWVqKgow69KEJ/KvVdkZ2dLeHi42NvbS4cOHWTp0qVG7cOHD5fY2FjDPABRKpXi6OhoNDUFc6V1MCUX9Hq9REZGiru7uzg4OIiXl5eMGzdOcnJyDP3z8vIEQJ0pKSmpSXEyX1oHnU4ncXFx4urqKiqVSkaPHi1FRUUiIpKcnGx0XHj33XfF19dXHBwcxN3dXSIiIuT77783tO/fv19CQkJEpVKJk5OT9OjRQxYtWiQ6na5JMTJXWod77NhSpzZViIV+sFqhUIil9kXmpVAo+EPn1CjMFTIF84Uai7lCpvgtX+rcV3vP3A5BRERERNRcWAQTERERkdVhEUxEREREVodFMBERERFZHRbBRERERGR1WAQTERERkdVhEUxEREREVodFMBERERFZHRbBRERERGR1WAQTERERkdVhEUxEREREVodFMBERERFZHRbBRERERGR1lJbakZ2dXa1CoWDRfR+ws7ODQqFo6TDoHsBcIVMwX6ixmCtkCjs7u9r6litExCIBKBQKsdS+yLwUCgX4XlJjMFfIFMwXaizmCpnit3yp86mJV2aJiIiIyOqwCCYiIiIiq8MimIiIiIisDotgIiIiIrI6LIKJiIiIyOqwCCYiIiIiq8MimIiIiIisDotgIiIiIrI6LIKJiIiIyOqwCCYiIiIiq2NVRbBSqcS+ffssus99+/ZBqVRadJ9EREREdHtWVQQTWZOcnBwMHToUjo6O8Pb2xvLly2/bPyEhAUFBQVCr1fDy8kJMTAxKSkosFC2Zk6m58NZbb6Fz585Qq9Xw8PBAVFQUCgoK6u2bkJAAhUKB5ORkc4ROLUCv1yM+Ph7u7u5wcnLC2LFjUVxcXG/fffv2QaFQQKVSGabHHnvMqM/+/fvRt29fqNVq+Pn5YdWqVZYYBllAc59nNmzYgMceewwajQZubm6IjIzEjz/+aLb4WQQT3Yf0ej1GjRqFwMBAFBUVIS0tDUuWLMGWLVsaXMfGxgbJycm4du0asrKyUFhYiMmTJ1swajKHu8mFiRMnIjMzE1qtFvn5+fD19cVzzz1Xp9/333+PXbt2oUOHDuYcAlnY4sWLsWPHDnz33XcoLCwE8GtONMTGxgYVFRWG6ZtvvjG05efnY8SIEZgxYwauX7+OzZs3480338TWrVvNPg4yL3OcZ8rLy7FgwQIUFhbiwoULCAkJweOPP46bN2+aZxAiYpHp11013nvvvSfdu3cXlUolPj4+MmvWLNHpdIZ2APL+++/LI488IiqVSv77v/9bTp8+bWjXarXy/PPPi0ajEV9fX1m/fr3Y2NjI3r17691fXl6ePP744+Ls7CwuLi4SEhIiP/30k4iITJo0SaZMmWLUv1OnTrJx40bD/L59+2TAgAGi0WjE1dVVoqOjRURk7969YmNjI5s3b5YuXbqIWq2WZ555RrRarUmvR2ti6ntJlrdnzx6xt7eX8vJyw7I5c+ZIaGhoo7exc+dOUavVTYqDudLympoLFRUVEhcXJ+3atTNaXlVVJb169ZJvvvmmzvHwbjFfWgdfX19Zt26dYT4nJ0cASF5eXp2+t85xDXn//ffloYceMloWHR0tYWFhTYqRudLyLHGeqaioEABy7NixJsX6W77UqU1b7ZVgb29v7Nq1C1qtFjt27MDHH3+MdevWGfVZv349tm3bhuLiYvj4+OCVV14xtP2///f/kJ2djVOnTuHEiRPYsWMH9Hp9g/ubPXs2fH19ceXKFRQXFyMpKQkuLi6NivXEiROIiIjAlClTcOnSJZw/fx7PP/+8oV2v1+OLL75AVlYWzp49i+PHj2PlypUmviJEjZeVlYWAgACoVCrDspCQEGRlZTV6GxkZGejdu7c5wiMLuttc+OSTT+Ds7AyVSoXExETMnz/fqH3+/PkIDw9Hv379zBE2tZCysjIUFBSgT58+hmX+/v5Qq9U4ceJEvevo9Xr4+PjA09MTI0aMMMqt2traWxfCjJZlZmaaZwBkMZY4z2RkZMDBwQFdu3ZtUqwNabVF8NixY9G5c2coFAo8/PDDmDhxIjIyMoz6xMfHw9fXF23btkV0dDSOHj0K4Nf/YCkpKVi4cCE8PT3h7OyMJUuW3HZ/tra2uHz5MnJzc2FjY4PevXujffv2jYp19erVGDVqFKKjo9G2bVvY29sjLCzMqM/ixYuhUqnQvn17PP3004ZYicyhvLwczs7ORstcXFyg1Wobtf62bduwdu1aJCYmmiM8sqC7zYVx48ahrKwMly5dwvz58/Hggw8a2o4ePYrU1FS88847ZomZWs6tvGhszvTo0QOZmZnIy8vDTz/9hN69eyM8PBwXL14EADz++OM4ffo0Nm7ciJqaGhw6dAiffvppo49F1HqZ+zxz9uxZTJ06FcuXL4eTk1OT461Pqy2CN23ahEcffRSurq5wdnbG+++/j6KiIqM+v78PzdHREeXl5QCAoqIi/PLLL/Dz8zO0d+7c+bb7W7p0KTp37oxRo0ahQ4cOeOWVV1BRUdGoWPPz8xEQENBgu42NDdzd3euNlag5pKSkGD2Y4uTkhLKyMqM+paWlUKvVd9xWamoqYmJikJaWhpCQEHOFTGbSnLkAAJ6enoiJicHIkSNRUlKC6upqTJ48Ge+//77RFSC6P9wqNhqbM56enggODoZSqYSLiwv+9re/oV27dti1axcAICAgANu3b0diYiLat2+Pt956C5MnT4abm5v5B0PNypLnmVOnTiEsLAwzZ87Eiy++2Gxj+KNWWQSfP38eEyZMwJw5c3Dp0iWUlZVh+vTpdb5SaYi7uztsbW2Rn59vWJaXl3fHdVauXImcnBx8/fXX2LdvH/7+978DAFQqFSorKw19dTodrl69apj38/NDdna2CSMkal7jx483ejAlODgYZ8+eNcrb48ePIzg4+LbbSUpKQmxsLNLT0+t8m0H3hubKhd/T6XSorKzExYsXcfHiRZw8eRLjx4+Hm5sb3NzccP78efzP//wPxo8fb44hkQW5uLjA19cXx44dMyzLzc2FVqtt9O1Rbdq0MTpfjxgxAkePHkVJSQn279+PixcvIjQ0tLlDJzOz1Hnm2LFjCA0NxaxZs/DGG280+ziM1HejsDkmmHAT+6lTpwSAfP3111JbWyuHDx8WDw8PGTx4sNFNzgcPHjTM//Hm/OjoaOnfv79cvnxZysrKZOzYsQKgwQfjNm/eLLm5uVJbWyvFxcUSEhIiCxYsEBGRpKQkcXd3l9zcXKmqqpI33nhDlEql4UGQzMxMadu2rWzYsEF++eUXuXHjhmE/9T00MG/ePBkyZEijX4/WxpT3klqGTqeTHj16yIwZM+TGjRty/Phx8fDwkE2bNjW4TmJiorRr106+//77ZouDudLyTM0FvV4v//jHP+TKlSsiInL+/Hl5+umnxc/PT2pqakSn08n58+eNJm9vb1m5cqUUFxc3KVbmS+uwaNEiCQgIkNzcXCkrK5OoqCiJiIiot29GRoZkZ2eLXq+X8vJymTdvnjg7O0tBQYGhz/fffy/V1dVSWVkpH3zwgahUKjl58mSTYmSutDxznGcOHTokLi4usmbNmmaNFQ08GNcqi2ARkQULFoibm5uo1Wp56qmn5NVXXzWpCC4rK5MJEyaIi4tLo34dIiEhQXx8fMTBwUHat28vU6dOlcrKShERqa6ulilTpoiLi4t07NhRVq1aVedp6IyMDOnXr584OzuLm5ubvPDCC/XGJcIimCwjOztbwsPDxd7eXjp06CBLly41ah8+fLjExsYa5gGIUqkUR0dHo6kpmCutgym5oNfrJTIyUtzd3cXBwUG8vLxk3LhxkpOT0+D2+esQ9xedTidxcXHi6uoqKpVKRo8eLUVFRSIikpycbHRcePfdd8XX11ccHBzE3d1dIiIi6hQ4kZGRolarxdHRUcLDw+XIkSNNjpG50jo093kmNDRUFApFnfYDBw40Kc6GimCFNPIWg6ZSKBRiqX2ReSkUikbfmkLWjblCpmC+UGMxV8gUv+WL4o/LW+U9wURERERE5sQimIiIiIisDotgIiIiIrI6LIKJiIiIyOqwCCYiIiIiq8MimIiIiIisDotgIiIiIrI6LIKJiIiIyOqwCCYiIiIiq8MimIiIiIisDotgIiIiIrI6LIKJiIiIyOqwCCYiIiIiq6O01I7s7OxqFQoFi+77gJ2dHRQKRUuHQfcA5gqZgvlCjcVcIVPY2dnV1rdcISIWCUChUIil9kXmpVAowPeSGoO5QqZgvlBjMVfIFL/lS51PTbwyS0RERERWh0UwEREREVkdFsFEREREZHVYBBMRERGR1WERTERERERWh0UwEREREVkdFsFEREREZHVYBBMRERGR1WERTERERERWh0UwEREREVmd+64IDgoKwpYtW1o6DCIiIiJqxe67IvjkyZN49tlnWzoMIrPKycnB0KFD4ejoCG9vbyxfvvy2/RMSEhAUFAS1Wg0vLy/ExMSgpKTEqM/PP/+M0aNHw9nZGc7Ozujbty9qamrMOQyyEFPz5a233kLnzp2hVqvh4eGBqKgoFBQUGNpTU1PRq1cvaDQaaDQaDBgwAPv37zf3MMhC9Ho94uPj4e7uDicnJ4wdOxbFxcUN9r969SomTZoEV1dXqNVqPPTQQ7h48SIA4OzZs4iKikLHjh3h5OSEoKAgrFu3zlJDITNr7nPRhg0b8Nhjj0Gj0cDNzQ2RkZH48ccfzRb/fVcEE93v9Ho9Ro0ahcDAQBQVFSEtLQ1Lliy57TcgNjY2SE5OxrVr15CVlYXCwkJMnjzZ0F5UVISBAwciODgYBQUFKCkpwapVq2BjY2OJIZEZ3U2+TJw4EZmZmdBqtcjPz4evry+ee+45Q3vfvn3x5Zdf4vr167h27RpmzJiBJ554AqWlpZYYEpnZ4sWLsWPHDnz33XcoLCwE8GtO1KeqqgpDhgyBra0tzpw5g9LSUqSkpEClUgEArl+/jrCwMBw5cgRarRZr1qzBzJkzsX37douNh8zDHOei8vJyLFiwAIWFhbhw4QJCQkLw+OOP4+bNm+YZhIhYZPp1V42XmJgofn5+olKpxMvLS958801DW15enkRFRYmnp6c4OzvLY489JsXFxSIi0qlTJ9m4cWO92/zll18kJiZG3N3dxcnJSbp16yapqakiIpKUlCT+/v6yePFi8fT0FHd3d3n99delurrasH50dLR4e3uLSqWSwMBASUlJMdp+VlaWREREiJubm2g0Ghk6dKih7dy5czJ27Fjx9PQUT09PiYmJEa1Wa9Jr0lqY+l5S89qzZ4/Y29tLeXm5YdmcOXMkNDS00dvYuXOnqNVqw/ysWbPkv//7v5s1ThHmSmvQ1HypqKiQuLg4adeuXb3tOp1Otm7dKgDkxIkTTYqV+dI6+Pr6yrp16wzzOTk5AkDy8vLq9F29erV4e3sbnSvvJCoqSmbMmNGkGJkrLc8c56I/qqioEABy7NixJsX6W77UqU1b5ZXgs2fPYtasWdi5cyfKy8tx8uRJPPnkkwCAGzduIDw8HB4eHvjpp59QXFyMZcuWwdbW9o7bXb9+PY4cOYLTp09Dq9UiIyMDPXv2NLSfO3cOBQUFyM3NxeHDh5Geno5ly5YZ2gcMGIDMzEyUlpZi7ty5iI6OxqlTpwAAly5dwuDBgzF48GDk5+fj8uXLSEhIAPDrJ+Xw8HD07NkTubm5OHXqFAoLC/Hqq68258tGViIrKwsBAQGGKy0AEBISgqysrEZvIyMjA7179zbM7927F926dcNTTz2Fdu3aoXfv3khJSWnWuKll3G2+fPLJJ3B2doZKpUJiYiLmz59v1F5QUAAXFxfY2toiKioKzz77LB588EFzDIEsqKysDAUFBejTp49hmb+/P9RqNU6cOFGn/969e9GzZ0/ExsbC1dUVPXr0wLvvvtvg9m/cuIHDhw8bHX/o3mSOc1F97Q4ODujatWuTYm1QfZWxOSaY8Knt559/Fjs7O9myZYvRJwwRkS1btoinp6fU1NTUu+7trgQnJSVJ165d5cCBA3XWT0pKEltbW6msrDQsW7t2rXTr1q3BOPv06SPvv/++iIgsWbJEHnnkkXr7paamSpcuXYyWHT16VGxtbUWn0zW4/dbKlPeSmt/bb78tgwYNMlq2Z88esbGxadT6W7duFZVKJT/88INhmb+/v7Rp00ZSU1OlpqZGvvjiC2nbtq0cPHiwSbEyV1peU/Pl0qVLsmjRItm7d2+97RUVFZKUlCSrV69uaqjMl1agoKBAAEhubq7Rcl9f33rPrUOGDBEA8t5778kvv/wiR44cEVdXV0lOTq7TV6fTyZgxY2TAgAEmXTmuD3Ol5ZnjXPR7Z86cEXd3d/nwww+bHCvupSvBXbp0QUpKCtauXQsvLy8MGDAAX3zxBQAgPz8fXbp0gVKpNHm7EyZMwNSpU/Haa6/B1dUVY8aMQU5OjqHdw8MDDg4Ohnk/Pz/D/VC1tbWYO3cuunfvDmdnZ7i4uCArKwtFRUWGuAICAurdb15enuGqya1pyJAhUCgUuHz5ssnjIOty6/66W5OTkxPKysqM+pSWlkKtVt9xW6mpqYiJiUFaWhpCQkIMy52cnNCvXz9ERUVBqVRi2LBhGD58ONLS0pp9PGRezZkvAODp6YmYmBiMHDmyzsOUAODo6Ijo6GgkJiZi9+7dzTIGajlOTk4A0OiccXJyQseOHfHqq6/C1tYWjzzyCCZMmIAdO3YY9aupqcGf//xnXLp0CTt37sQDDzxgiyDrUAAAIABJREFUvkGQWVjiXHTLqVOnEBYWhpkzZ+LFF19stjH8UassggFgzJgx+PLLL1FcXIw//elPeOqpp3Djxg34+fkhLy8Per3e5G0qlUokJCTg6NGjOHfuHBwcHPDCCy8Y2q9evYobN24Y5vPz8+Ht7Q0A2LRpE9atW4dt27bh+vXrKC0tRXBw8K2r3PDz80N2dna9++3UqRMCAgJQWlpqNFVVVaFjx44mj4Osy/jx41FRUWGYgoODcfbsWVRWVhr6HD9+HMHBwbfdTlJSEmJjY5Geno6wsDCjtoceeggKhaLOOvUto9atufLl93Q6HSorKw1P/DfUp6FjIN07XFxc4Ovri2PHjhmW5ebmQqvV1vu1dWOOHVVVVRg9ejSuXr2KL774As7OzuYJnszKEuciADh27BhCQ0Mxa9YsvPHGG80+DiP1XR42xwQTvrr46aefZNeuXVJZWSm1tbWyfv16sbe3l5s3b0pFRYV06tRJXnnlFSktLRWdTieHDx82PGR2u9shMjIy5OjRo1JdXS1VVVUybdo0CQsLE5Ffb4dQKpXyyiuvyI0bN+Tnn3+W7t27y6JFi0RE5IMPPhAfHx+5fPmy1NTUyEcffSRKpVLmzZsnIiIXLlwQtVotixcvlsrKSqmurpavvvpKREQqKyulW7du8s4774hWq5Xa2lopLCyU7du3N/o1aU1MeS+p+el0OunRo4fMmDFDbty4IcePHxcPDw/ZtGlTg+skJiZKu3bt5Pvvv6+3/fDhw6JUKuXTTz8VvV5veODhm2++aVKszJWWZ2q+6PV6+cc//iFXrlwREZHz58/L008/LX5+fobbyP75z39Kdna26PV60Wq1smDBAmnbtq2cPHmySbEyX1qHRYsWSUBAgOTm5kpZWZlERUVJREREvX3z8/PF3t5eVq1aJTqdTjIzM8XNzU02b94sIiLl5eUSGhoqw4YNkxs3bjRbjMyVlmeOc9GhQ4fExcVF1qxZ06yxooHbIVplEXzixAnp27evqNVqUavVEhISIp9//rmh/eeff5ann35a3N3dxdnZWQYMGCDXrl0TkdsXwZ988okEBgaKo6OjaDQaiYiIkLNnz4qI8a9DtG/fXtzc3OTVV1813LdUWVkpUVFRolKpxMPDQ+Li4iQsLMxQBIuIHDt2TIYMGSIajUY0Go0MGzbM0FZQUCDjx48XLy8vcXJyku7du8vcuXMb/Zq0Jjz4tLzs7GwJDw8Xe3t76dChgyxdutSoffjw4RIbG2uYByBKpVIcHR2Npt/717/+JQEBAeLg4CBBQUHyr3/9q8lxMldaB1PyRa/XS2RkpLi7u4uDg4N4eXnJuHHjJCcnx9D/rbfeEl9fX3FwcBA3NzcJDw83fOhvCuZL66DT6SQuLk5cXV1FpVLJ6NGjpaioSEREkpOT6xw79u7dKw899JA4ODhI165dZdWqVYa29evXCwCxt7c3Ovb8/vh0N5grrUNzn4tCQ0NFoVDUaT9w4ECT4myoCFbIb1/nm5tCoRBL7eturF+/HosWLTK6R5jqp1Ao0JrfS2o9mCtkCuYLNRZzhUzxW77UuW+n1d4TTERERERkLiyCiYiIiMjq8HYIMhm/hqLGYq6QKZgv1FjMFTIFb4cgIiIiIvoNi2AiIiIisjosgomIiIjI6rAIJiIiIiKrwyKYiIiIiKwOi2AiIiIisjosgomIiIjI6rAIJiIiIiKrwyKYiIiIiKwOi2AiIiIisjpKS+3Izs6uVqFQsOi+D9jZ2UGhqPPXB4nqYK6QKZgv1FjMFTKFnZ1dbX3LFZb629sKhUL4d77vD/yb7dRYzBUyBfOFGou5Qqb4LV/qfGrilVkiIiIisjosgomIiIjI6rAIJiIiIiKrwyKYiIiIiKwOi2AiIiIisjosgomIiIjI6rAIJiIiIiKrwyKYiIiIiKwOi2AiIiIisjosgomIiIjI6txTRXB0dDSmTp3a0mEQERER0T2u1RbBoaGhWLRoUUuHQdQq5eTkYOjQoXB0dIS3tzeWL19+2/5vvfUWOnfuDLVaDQ8PD0RFRaGgoMDQvn79erRp0wYqlcow/fnPfzb3MMhCTM2XW2pra/HYY49BoVCgsLDQsFyv12PhwoXo3LkzVCoVBg4ciBMnTpgrfLIwvV6P+Ph4uLu7w8nJCWPHjkVxcXGD/ZctWwZ/f384OTmhW7du+OCDDwxtJSUlGDRoEDw8PKBWq+Hv749FixZBRCwxFDIzU48tCQkJCAoKglqthpeXF2JiYlBSUmJoz8rKQmRkJDp06ACFQoFDhw6ZNf5WWwQTUf30ej1GjRqFwMBAFBUVIS0tDUuWLMGWLVsaXGfixInIzMyEVqtFfn4+fH198dxzzxn16dKlCyoqKgzTpk2bzD0UsoC7yZdbVqxYAQcHhzrL3333XSQnJyMjIwMlJSUYOHAgIiIiUF5ebo4hkIUtXrwYO3bswHfffWf48DNx4sR6+6alpWHevHlISUlBeXk5NmzYgPj4eHz55ZcAAEdHR3z44Ye4cOECtFotvvrqK6SkpGDt2rUWGw+Zx90cW2xsbJCcnIxr164hKysLhYWFmDx5sqHd1tYWY8aMQVpamiWGAIiIRaZfd9U406dPlzZt2oitra04OjpKQECAiIhMmjRJJkyYIFOnThVnZ2fx8vKS1atXG6174MAB6d+/v2g0GunSpYssW7ZMamtr691PXl6eAJD169dLYGCgODg4SGRkpJSUlEhCQoK4u7tL+/btZdWqVUbrbd26VXr37i1qtVp69+4t27dvb3R7UlKS+Pv7S2JionTs2FFcXFxk2rRpotPpGv36tDRT3ktqfnv27BF7e3spLy83LJszZ46EhoY2av2KigqJi4uTdu3aGZbdysvmxlxpeXebL2fOnJEuXbrI8ePHBYCcP3/e0Pboo4/Ke++9Z5ivrq6WBx54QP75z382KVbmS+vg6+sr69atM8zn5OQIAMnLy6vTd/ny5dKvXz+jZX379pWlS5fWu+3c3Fzp2bOnvP76602KkbnS8pp6LhIR2blzp6jV6nrbAMjBgwebHOetbUl9tWl9C80xmZqwgwcPloULFxotmzRpktjZ2cmOHTtEr9fLtm3bRKlUSn5+voiI/Oc//xGVSiWfffaZ6HQ6OX36tPj5+TV4YL5VBI8aNUquXbsmxcXFEhgYKN26dZP//d//lZqaGvn8889FqVTKuXPnRETkm2++kbZt28rnn38uNTU1snPnTmnbtq18++23jWpPSkoSpVIps2fPlqqqKsnOzhaNRiPJyckmvT4tiQeflrVixQoJDg42WrZ9+3bRaDS3XS8lJUXUarUAEKVSKStXrjS0JSUlia2trbRv3168vb3l2Wefldzc3CbHylxpeXeTL3q9Xvr37y/btm0zHCd/XwT36dNHVqxYYZivrq4WpVIpr732WpNiZb60vNLSUgEgx48fN1quVqtlx44ddfpfuHBBgoKC5NChQ6LX6+XAgQPSrl07+fHHH436jRgxQuzs7ASA+Pj4yJkzZ5oUJ3Ol5d3tuej3XnvtNRkwYEC9bZYogu+52yHCw8Px5JNPok2bNhgzZgxcXFyQmZkJAPjwww/xzDPP4KmnnoKNjQ169OiBl19+GRs2bLjtNv/yl7+gXbt2cHV1xciRI/HAAw8gJiYGSqUSkZGR0Gg0OH78OAAgKSkJY8eORWRkJJRKJUaMGIHRo0fj448/blQ7ANjb2+Ptt99G27Zt0bVrVwwZMgRHjx410ytG95vy8nI4OzsbLXNxcYFWq73teuPGjUNZWRkuXbqE+fPn48EHHzS0DRo0CD/++CMuXryII0eOwM7ODsOGDUNlZaVZxkCWczf5kpiYCE9PT4wZM6be9lGjRuH9999HdnY2qqqqMGfOHOj1+jvmILV+t97DxubMrWcMwsLCYGtri7CwMCxYsAC9evUy6rdz505UVFTg8OHDmDhxItzc3Mw3CLKIuz0X3bJt2zasXbsWiYmJ5givUe65IrhDhw5G846Ojob70PLy8rBp0ya4uLgYpgULFuDSpUuN3qaDg0OdfTg4OBj2cf78eXTp0sWo3d/fH+fPn29UO/DrQcPGxqbeMRD9UUpKitEDa05OTigrKzPqU1paCrVa3ajteXp6IiYmBiNHjjQ8kNClSxcEBASgTZs28PT0xNq1a3Hx4kV8++23zT4eMq+m5ktOTg6WL1+OVatWNbiPWbNmYfTo0Xj88cfh6+sLAAgMDGRhcx9wcnICgEbnzMKFC/HJJ58gMzMTNTU1yMrKwooVK/DRRx/V6WtjY4O+ffvC2dkZ06dPN88AyGya81yUmpqKmJgYpKWlISQkxFwh31GrLYLbtDE9tE6dOuGFF15AaWmpYdJqtTh58mSzxeXj44O8vDyjZbm5ufDx8WlUO5Gpxo8fb/TAWnBwMM6ePWt0lfb48eMIDg5u9DZ1Oh0qKytx8eLFetsVCgUUCgWf4L4HNTVfDh06hKKiIvTq1Qtubm6GE1Tv3r0NT/23bdsWf//735GXl4erV68iPj4eubm5CA0NNfv4yLxcXFzg6+uLY8eOGZbl5uZCq9Wid+/edfr/8MMPGD16NHr27AmFQoGgoCA8/fTT2LlzZ4P70Ol0yM7ONkv8ZD7NdS5KSkpCbGws0tPTERYWZu6wb6vVFsGenp7IyckxaZ2XXnoJmzdvRnp6OmpqaqDT6XDq1Cns37+/2eKKjo7Gtm3bsHv3buj1euzatQvbt283PN14p3aipho0aBA6deqE2bNn4+bNm8jMzMSaNWsQGxtbb//a2lqsWrUKV69eBQAUFhZi+vTp8PPzQ48ePQAA//73v1FYWAgRQUlJCaZPnw43Nzf07dvXYuMi8zA1X/70pz/h559/RmZmJjIzM/H5558DAL744gs8//zzAIDLly8jPz8fwK/ffkVHR6Nfv36IiIiwyJjIvKZNm4YlS5YgLy8PWq0WCQkJiIiIgJ+fX52+/fv3x2effWYoak+fPo3PPvvM8OHp22+/RUZGBm7evAm9Xo8DBw4gMTERkZGRlhwSmYGpxxYAWLlyJWbOnIndu3ejf//+ddpFBFVVVaiqqgIAVFdXo6qqCnq93jyDqO9GYXNMMPEm9u+//16CgoLE2dlZevbsKSK/Phg3ZcoUo36dOnWSjRs3Gua/+eYbCQ8PF1dXV9FoNPLoo49Kampqvfuo74GPefPmyZAhQ267jy1btkivXr3EyclJevXqVWf7t2uv7yn8+sbVmpn6XlLzy87OlvDwcLG3t5cOHTrUeRJ7+PDhEhsbKyK/PuQUGRkp7u7u4uDgIF5eXjJu3DjJyckx9J85c6Z06NBBHBwcxNPTU8aOHdvkB1dEmCuthSn58kf1HSePHDkiXbt2FXt7e/Hw8JDp06cbPSF+t5gvrYNOp5O4uDhxdXUVlUolo0ePlqKiIhERSU5OFkdHR0PfmpoaSUhIkE6dOomjo6P4+PjI66+/LtXV1SIisn//fgkJCRGVSiVOTk7So0cPWbRoUZN/EYm50jqYemzBbw9mOzo6Gk233Dre/HFKSkpqUpxo4ME4hVjo606FQiGW2heZF78mp8ZirpApmC/UWMwVMsVv+aL44/JWezsEEREREZG5sAgmIiIiIqvDIpiIiIiIrA6LYCIiIiKyOiyCiYiIiMjqsAgmIiIiIqvDIpiIiIiIrA6LYCIiIiKyOiyCiYiIiMjqsAgmIiIiIqvDIpiIiIiIrA6LYCIiIiKyOiyCiYiIiMjqKC21Izs7u1qFQsGi+z5gZ2cHhULR0mHQPYC5QqZgvlBjMVfIFHZ2drX1LVeIiEUCUCgUYql9kXkpFArwvaTGYK6QKZgv1FjMFTLFb/lS51MTr8wSERERkdVhEUxEREREVodFMBERERFZHRbBRERERGR1WAQTERERkdVhEUxEREREVodFMBERERFZHRbBRERERGR1WAQTERERkdVhEUxEREREVueeKoKjo6MxderUlg6DiIiIiO5xrbYIDg0NxaJFi1o6DKJWKScnB0OHDoWjoyO8vb2xfPny2/ZPSEhAUFAQ1Go1vLy8EBMTg5KSEkP7iy++CJVKZTQpFAq8++675h4KWYCp+RIdHY0HHnjAKB8++OADQ3tqaip69eoFjUYDjUaDAQMGYP/+/eYeBlmIXq9HfHw83N3d4eTkhLFjx6K4uLjB/suWLYO/vz+cnJzQrVs3o1w5ePBgnWOLUqlE7969LTEUMrN7/VzUaotgIqqfXq/HqFGjEBgYiKKiIqSlpWHJkiXYsmVLg+vY2NggOTkZ165dQ1ZWFgoLCzF58mRD++rVq1FRUWGYPv30UyiVSjz33HOWGBKZ0d3kCwBMmjTJKCdeeuklQ1vfvn3x5Zdf4vr167h27RpmzJiBJ554AqWlpeYeDlnA4sWLsWPHDnz33XcoLCwEAEycOLHevmlpaZg3bx5SUlJQXl6ODRs2ID4+Hl9++SUAYODAgUZ5pNVq0bFjR0yYMMFi4yHzuC/ORSJikenXXTXO9OnTpU2bNmJrayuOjo4SEBAgIiKTJk2SCRMmyNSpU8XZ2Vm8vLxk9erVRuseOHBA+vfvLxqNRrp06SLLli2T2traevfzyy+/SExMjLi7u4uTk5N069ZNUlNTRUQkKSlJ/P39ZfHixeLp6Snu7u7y+uuvS3V1tWH96Oho8fb2FpVKJYGBgZKSkmK0/aysLImIiBA3NzfRaDQydOhQQ9u5c+dk7Nix4unpKZ6enhITEyNarbbRr1FLMuW9pOa3Z88esbe3l/LycsOyOXPmSGhoaKO3sXPnTlGr1Q22jx07VkaPHt2kOEWYK63B3eTLpEmTZMqUKY3avk6nk61btwoAOXHiRJNiZb60Dr6+vrJu3TrDfE5OjgCQvLy8On2XL18u/fr1M1rWt29fWbp0ab3bTk9PF1tbW7l69WqTYmSutLx78FxUpzZtlVeCV61ahYEDB+Ivf/kLKioqcObMGUPb1q1bMWrUKJSUlOAf//gHXn75ZZw7dw4AcPLkSTzxxBOIj49HUVER/v3vf2PVqlXYuHFjvftZv349jhw5gtOnT0Or1SIjIwM9e/Y0tJ87dw4FBQXIzc3F4cOHkZ6ejmXLlhnaBwwYgMzMTJSWlmLu3LmIjo7GqVOnAACXLl3C4MGDMXjwYOTn5+Py5ctISEgAAFRVVSE8PBw9e/ZEbm4uTp06hcLCQrz66qvN/lrS/ScrKwsBAQFQqVSGZSEhIcjKymr0NjIyMhr8OvLy5ctIS0vDiy++2ORYqeXdbb5s27YN7dq1Q0BAAOLj41FRUWHUXlBQABcXF9ja2iIqKgrPPvssHnzwQbOMgSynrKwMBQUF6NOnj2GZv78/1Go1Tpw4Uaf/c889B61Wi6+//hq1tbU4ePAgzp49i+HDh9e7/dWrV2Ps2LFwd3c32xjIMu6Hc1GrLIJvJzw8HE8++STatGmDMWPGwMXFBZmZmQCADz/8EM888wyeeuop2NjYoEePHnj55ZexYcOGerdla2uLiooKnDp1CjqdDj4+PkZFcJs2bbB06VLY29vD398fb7zxBpKSkgztU6ZMgaurK2xsbPDcc8+hd+/e2LdvHwBg48aN6Nq1K9588004OjrC1tYWQ4cOBQDs3LkTIoK3334b9vb20Gg0WLhwIVJSUqDX6830ytH9ory8HM7OzkbLXFxcoNVqG7X+tm3bsHbtWiQmJtbb/tFHH8HHxwfDhg1rcqzU8u4mX1555RX89NNPKC4uxqeffor9+/cjJibGqI+vry9KS0uh1WqRlJSEsLAws8RPlnUrLxqbMx4eHoiKikJYWBhsbW0RFhaGBQsWoFevXnX6nj9/Hrt27UJsbKx5gieLuh/ORfdcEdyhQwejeUdHR5SXlwMA8vLysGnTJri4uBimBQsW4NKlS/Vua8KECZg6dSpee+01uLq6YsyYMcjJyTG0e3h4wMHBwTDv5+dnuD+qtrYWc+fORffu3eHs7AwXFxdkZWWhqKgIAJCfn4+AgIB695uXl2e4inJrGjJkCBQKBS5fvnz3Lw7dl1JSUoweEnByckJZWZlRn9LSUqjV6jtuKzU1FTExMUhLS0NISEid9traWqxduxbTpk2DQqFotjGQ5TRHvvTp0wft27dHmzZtEBQUhBUrVmDr1q345Zdf6vR1dHREdHQ0EhMTsXv37mYfD1mWk5MTADQ6ZxYuXIhPPvkEmZmZqKmpQVZWFlasWIGPPvqoTt+1a9eie/fuGDx4sHmCJ7O6H89FrbYIbtPG9NA6deqEF154AaWlpYZJq9Xi5MmT9fZXKpVISEjA0aNHce7cOTg4OOCFF14wtF+9ehU3btwwzOfn58Pb2xsAsGnTJqxbtw7btm3D9evXUVpaiuDg4Fv3P8PPzw/Z2dkNxhkQEGAUZ2lpKaqqqtCxY0eTx033t/Hjxxs9KBAcHIyzZ8+isrLS0Of48eMIDg6+7XaSkpIQGxuL9PT0Bq/a/d///R8uXbpk9P+A7i3NlS+/d+t4fOv4Vh+dTtfgMY/uHS4uLvD19cWxY8cMy3Jzc6HVauv92vqHH37A6NGj0bNnTygUCgQFBeHpp5/Gzp07jfrpdDp89NFHvAp8D7sfz0Wttgj29PQ0uirbGC+99BI2b96M9PR01NTUQKfT4dSpUw3+dM+ePXvwww8/oKamBvb29nB0dIRSqTS019bWYtasWbh58yZyc3OxbNkyTJo0CcCvXxkplUq4u7ujtrYWH3/8sdF9MBMmTMCZM2ewZMkS3LhxAzU1NcjIyAAAjBw5EjU1NfjrX/+K8vJyiAguXLiATz/91NSXiazQoEGD0KlTJ8yePRs3b95EZmYm1qxZc9uTy8qVKzFz5kzs3r0b/fv3b7DfmjVrMGbMGN6vdx+5m3zZvHmz4ZcesrOzERcXhyeffBJ2dnYAgA0bNiAnJwe1tbUoLy/H22+/jYKCAoSHh1tkTGRe06ZNw5IlS5CXlwetVouEhARERETAz8+vTt/+/fvjs88+M3wAOn36ND777LM6V/fS09Nx/fp1PP/885YYAlnAfXEuqu9pOXNMMPFJzu+//16CgoLE2dlZevbsKSL1P7HcqVMn2bhxo2H+m2++kfDwcHF1dRWNRiOPPvqo4Rcf/uiTTz6RwMBAcXR0FI1GIxEREXL27FkRMf51iPbt24ubm5u8+uqrhl+HqKyslKioKFGpVOLh4SFxcXESFhYm8+bNM2z/2LFjMmTIENFoNKLRaGTYsGGGtoKCAhk/frx4eXmJk5OTdO/eXebOnWvSa9RSTH0vqfllZ2dLeHi42NvbS4cOHeo8iT18+HCJjY01zAMQpVIpjo6ORtPvFRYWio2Njezbt6/Z4mSutA6m5svgwYNFo9GIg4OD+Pn5yWuvvSZlZWWG9rfeekt8fX3FwcFB3NzcJDw8XL766qsmx8l8aR10Op3ExcWJq6urqFQqGT16tBQVFYmISHJystGxo6amRhISEqRTp07i6OgoPj4+dX5JSUQkIiJCoqOjmy1G5krrcI+di+rUpgq5zddbzUmhUIil9tUc1q9fj0WLFpl8NdoaKBSK234tSnQLc4VMwXyhxmKukCl+y5c6Nxe32tshiIiIiIjMhUUwEREREVkd3g5BJuPXUNRYzBUyBfOFGou5Qqbg7RBERERERL9hEUxEREREVodFMBERERFZHRbBRERERGR1WAQTERERkdVhEUxEREREVodFMBERERFZHRbBRERERGR1WAQTERERkdVhEUxEREREVkdpqR3Z2dnVKhQKFt33ATs7OygUdf76IFEdzBUyBfOFGou5Qqaws7OrrW+5wlJ/e1uhUAj/zvf9gX+znRqLuUKmYL5QYzFXyBS/5UudT028MktEREREVodFMBERERFZHRbBRERERGR1WAQTERERkdVhEUxEREREVodFMBERERFZHRbBRERERGR1WAQTERERkdVhEUxEREREVodFMBERERFZnXu2CC4pKUFERAScnZ3Rp0+fevuoVCocPnzYwpERERERUWt3zxbBq1evRkVFBa5du4Yffvih3j4VFRXo16+fhSMjMr+cnBwMHToUjo6O8Pb2xvLly2/bX6/XIz4+Hu7u7nBycsLYsWNRXFxs1L5w4UJ07twZKpUKAwcOxIkTJ8w9DLIQU/Nl8+bNGDhwINRqNZRKZZ32v/71r1CpVEaTQqHAjBkzzDUEshBTcyUhIQFBQUFQq9Xw8vJCTEwMSkpKDO1ZWVmIjIxEhw4doFAocOjQIXMPgSzsTueXhnz44YdQKBRYtGiRYdnZs2cRFRWFjh07wsnJCUFBQVi3bp3ZYr9ni+Dc3FwEBgbWe4Amup/p9XqMGjUKgYGBKCoqQlpaGpYsWYItW7Y0uM7ixYuxY8cOfPfddygsLAQATJw40dD+7rvvIjk5GRkZGSgpKcHAgQMRERGB8vJys4+HzOtu8kWj0eCll17Ce++9V2/77NmzUVFRYZiOHz8OhUKBCRMmmGsYZAF3kys2NjZITk7GtWvXkJWVhcLCQkyePNnQbmtrizFjxiAtLc0SQ6AWcKfzS33OnTuH5cuX48EHHzRafv36dYSFheHIkSPQarVYs2YNZs6cie3bt5sneBGxyPTrrhqvuLhYJk6cKJ6entK+fXt5/vnn5dq1ayIiMnLkSFEqlaJUKsXR0VHmzp1b7zYAyMGDBw3zBw4ckP79+4tGo5EuXbrIsmXLpLa2VkRESkpKJCoqStq1aydqtVqCgoLkwIEDIiIyb948GTJkiNG2Bw8eLAsXLrzjuvcjU99Lal579uwRe3t7KS8vNyybM2eOhIaGNriOr6+vrFu3zjCfk5MjACQvL09ERB599FF57733DO3V1dXywAMPyD//+c8mxcpcaXl3ky+37N27V2xsbO7YLy4uTh5++OEmxSnCfGlpTcmVW3bu3Clqtbretj+ek5uCudJ63On8Up9ngn+AAAAgAElEQVQhQ4bI5s2bjWqphkRFRcmMGTOaFONv+VKnNm21V4LHjx+P69ev49SpUzh9+jSKi4sNnyzS09Mxfvx4TJo0CRUVFViwYMEdt3fy5Ek88cQTiI+PR1FREf79739j1apV2LhxIwBg6dKluHHjBs6dO4fS0lJs374d3t7ejYq1KesSmSorKwsBAQFQqVSGZSEhIcjKyqq3f1lZGQoKCozunff394darTbc8lBbW3vrw6qBiCAzM9MMIyBLMjVfTPXLL79g/fr1ePHFF5tle9RymiNXMjIy0Lt3b3OER61QY84vf7RmzRo4ODjg2WefveP2b9y4gcOHD5stp1rlvQQXL17E7t27cfbsWWg0GgC/fl3bo0cPXLp0CR06dDB5mx9++CGeeeYZPPXUUwCAHj164OWXX8aGDRvw/PPPw9bWFteuXcOZM2fw8MMPIyAgoNHbbsq6RKYqLy+Hs7Oz0TIXFxdotdp6+99afrt1Ro0ahffffx8jRoyAj48P5s2bB71e3+A26d5har6YauvWraiursa4ceOaZXvUcpqaK9u2bcPatWuxf/9+c4RHrVBjzi+/V1BQgEWLFuHbb7+947b1ej0mTpyIzp074/nnn2+egP+gVV4JPn/+PACgc+fOhmX+/v5GbabKy8vDpk2b4OLiYpgWLFiAS5cuAQDi4+MxZMgQTJo0Ce7u7pg0aRKuXLnSqG03ZV2iO0lJSTF6AMnJyQllZWVGfUpLS6FWq+td38nJCQBuu86sWbMwevRoPP744/D19QUABAYGws3NrbmHQ2bW1Hwx1Zo1azB+/Hijq4d0b2jOXElNTUVMTAzS0tIQEhJirpCplWnM+eX3pk6dijlz5qBjx4633W5NTQ3+/Oc/49KlS9i5cyceeOCB5gv6d1plEezj4wMAyM/PNyzLzc01ajNVp06d8MILL6C0tNQwabVanDx5EgDg6OiId955B//5z39w8uRJXLhwAfHx8QB+/am1yspKo+1dvHjR8O/brUvUVOPHjzd6CCk4OBhnz541ysnjx48jODi43vVdXFzg6+uLY8eOGZbl5uZCq9UavmJq27Yt/v73vyMvLw9Xr15FfHw8cnNzERoaataxUfNrar6Y4tSpUzh48CBvhbhHNVeuJCUlITY2Funp6QgLCzN32NSKNOb88ntffvklZs+eDTc3N7i5ueHrr7/G3/72NwwcONDQp6qqCqNHj8bVq1fxxRdf1LnK3JxaZRHs5eWFxx9/HHFxcSgtLcX169cRFxdn+JmVu/HSSy9h8+bNSE9PR01NDXQ6HU6dOmX42iY9PR2nT5+GXq+HSqWCnZ2d4ZcnHnnkERw7dgw//PADdDodVq1ahby8PMO2b7cuUXMbNGgQOnXqhNmzZ+PmzZvIzMzEmjVrEBsb2+A606ZNw5IlS5CXlwetVouEhARERETAz88PAHD58mXDh87z588jOjoa/fr1Q0REhAVGROZ0N/mi1+tRVVWF6upqAL+elKqqqurcN75mzRr07du3WQpqanl3kysrV67EzJkzsXv3bvTv379Ou4gY8gcAqqurUVVVBb1eb7ZxkGXd6fzye+fPn0dWVhYyMzORmZmJRx55BNOnT8fWrVsB/PrTtpGRkaiursauXbvM/w1TfU/LmWOCiU9yXr16VcaPHy/t27cXDw8PmTBhghQVFRnaJ02aJFOmTLnj04C/fxL1m2++kfDwcHF1dRWNRiOPPvqopKamiojIihUrxN/fXxwcHMTV1VWioqLkypUrhnXffPNNcXNzEzc3N5kzZ47RE413Wvd+Y+p7Sc0vOztbwsPDxd7eXjp06CBLly41ah8+fLjExsYa5nU6ncTFxYmrq6uoVCoZPXq00f+nI0eOSNeuXcXe3l48PDxk+vTpRk+I3y3mSutgar4kJSUJgDrT75/2vnHjhmg0Glm/fn2zxcl8aXmm5goAwy81/X66JS8vr95cSkpKalKczJXW43bnl+TkZKN8+KM//jrE+vXrBYDY29sb5dPvc+5uoIFfh1DIHz7Zm4tCoRBL7Qv49UqGUqnEd999h//6r/+y2H6tgUKhqHNFiKg+zBUyBfOFGou5Qqb4LV8Uf1zeKm+HaA7Hjh2DjY2N0cN1RERERETAfVoEr1y5Ek888QTeeecduLu7t3Q4RERERNTK3Le3Q5D58GsoaizmCpmC+UKNxVwhU1jd7RBERERERA1hEUxEREREVodFMBERERFZHRbBRERERGR1WAQTERERkdVhEUxEREREVodFMBERERFZHRbBRERERGR1WAQTERERkdVhEUxEREREVkdpqR3Z2dnVKhQKFt33ATs7OygUdf76IFEdzBUyBfOFGou5Qqaws7OrrW+5wlJ/e1uhUAj/zvf9gX+znRqLuUKmYL5QYzFXyBS/5UudT028MktEREREVodFMBERERFZHRbBRERERGR1WAQTERERkdVhEUxEREREVodFMBERERFZHRbBRERERGR1WAQTERERkdVhEUxEREREVodFMBERERFZnVZbBPv5+SE5Odns+1EoFDh06JDZ90NERERErUerLYKbat++fVAqlS0dBpFZ5OTkYOjQoXB0dIS3tzeWL19+2/6bN2/GwIEDoVarG/x/sXr1agQEBEClUuHhhx/Gvn37zBA5tQRT8yUhIQFBQUFQq9Xw8vJCTEwMSkpKDO0bNmzAY489Bo1GAzc3N0RGRuLHH3809zDIAsxxbLnlww8/hEKhwKJFi5ozZGpher0e8fHxcHd3h5OTE8aOHYvi4uI7rtdQPuzfvx99+/aFWq2Gn58fVq1aZa7Q798imOh+pdfrMWrUKAQGBqKoqAhpaWlYsmQJtmzZ0uA6Go0GL730Et57771621NTU/GXv/wF//rXv1BWVobY2FiMGDECBQUF5hoGWcjd5IuNjQ2Sk5Nx7do1ZGVlobCwEJMnTza0l5eX4/+3d+fRUdXZ2sefSiIkqUpSIYwBEgTlgiyJBr2giGJQEWhaRRQVuKAgsHB8RZqWxhYUB0S4RvEiV67YAg2KEzijggxtixPBGYiESVHGTAhmqP3+oallTAIJSSWB3/ez1lmLOuM+VTunnqo65zBlyhTt3LlT33//vVJTU3XJJZfo0KFDtbFLCJFQHFtKbNu2TTNmzNDpp59e02Wjjj300ENaunSp1q1bp507d0qShg4desRlKuqHrVu3ql+/frr11lt14MABLV68WHfddZdeeOGF0BRvZrUy/LqpyktOTrYpU6ZY9+7dzev1WpcuXeyjjz4KTh82bJiNGDGizDLz58+377//3iIjI02Seb1e83q99swzz5S7HUn2xBNP2FlnnWU+n8+6du1q33zzTXB6YWGh3X///XbqqadaXFycnXvuufbJJ59UaV9ONFV9LVGzVqxYYVFRUZaXlxccN2nSJOvZs+dRl125cqWFh4eXGX/VVVfZ7bffXmpcmzZtbMqUKdWqlV6pe9XplxKvvfaaxcbGVjg9Pz/fJNlnn31WrVrpl7oVimNLiV69etnixYvtggsusPvuu6/atdIr9UdSUpLNnTs3+DgzM9MkWVZWVoXLVNQPTzzxhJ1xxhml5h0+fLhdeOGF1arxt34pk03r9TfBTz75pNLT07V//34NHDhQffv2VW5u7lGXS0xM1Jtvvqnw8HDl5+crPz9fw4YNq3D+Z555Ri+++KL27t2r1q1b65ZbbglO+/vf/66lS5fqrbfe0r59+3TDDTeod+/eOnDgQI3sI1BVGzZsCJ62UCI1NVUbNmw45nUGAoGSD6tBZqaMjIxjXifqh5rol/fee0+dO3c+4vTo6Gidcsop1aoVdSsUxxZJmjNnjqKjozVo0KDqloh6JicnR9u3b1eXLl2C49q1a6fY2Fh9/vnn5S5zpH4o770oEAiE7L2oXofgESNGqEuXLmrQoIEmTJigqKgovfbaazW+nfHjxyspKUkNGzbU8OHD9cknn0j6NQQ8/vjjmj59utq2bavw8HCNGDFCLVq00Ouvv17jdQCVkZeXp7i4uFLj/H5/pT4gVqR///5asGCBPvnkExUWFmrWrFnavn17tdaJ+qG6/fLiiy/qqaeeUnp6ernTN23apJEjR2rGjBmKiYmpdr2oO6E4tmzfvl1Tp07V7Nmzq1se6qGS3qhs3xytHy655BJ98803mj9/vgoLC7V27Vq9/PLLIXsvqtchuE2bNsF/ezweJSUlBc83qUktWrQI/tvr9SovL0+StHfvXuXn56t///7y+/3BYcuWLSGpAyjPwoUL5fP5gkNMTIxycnJKzZOdna3Y2Nhj3sZ//dd/afz48Ro8eLCaN2+uzz77TL169VLjxo2rWz5qWU32y5IlS3TjjTdq2bJlSk1NLTP966+/1oUXXqg777xTY8aMqbF9QO2ojWPLyJEjNWnSJLVs2bK65aIeKvngW9m+OVo/tG/fXi+99JLS09PVrFkz/e1vf9P1118fuvei8s6RCMWgYzgneNKkScHHgUDAWrdubQsXLjQzs5tuusmuueaa4PTCwkKLioqy+fPnm5nZqlWrjnh+UglJtmbNmuDj35/XFAgEzOv1ljoXGZyLVddKztvLz88Pjrv77rtr5Ly9Er/88ou1bNnSZs+eXa1a6ZW6d6z98vTTT1t8fLytXbu23OmffvqpNWnSxB577LEaq5V+qVuhOLZIskaNGllCQoIlJCRYRESERUdH23nnnVetWumV+iMpKcn+7//+L/j4u+++q/Cc4GPph4EDB9qgQYOqVaMqOCe4XofgxMRE+/TTT62goMCmTZtmCQkJlp2dbWZm8+bNsyZNmtiWLVvs8OHD9pe//MUiIiKCIXjjxo0mybZs2XLUJ6aiEGxmNnHiRDv//PNt06ZNZmaWl5dnb731ln3//fdV2p8TCQefulVUVGQdOnSwW2+91X7++Wdbv369NW3a1BYtWnTEZQ4dOmRvv/22hYeH26FDh+zQoUMWCATMzCw7O9u+/vprCwQCtnv3brvhhhusY8eO9vPPP1erVnql7h1Lv6Snp1ujRo0q/AJg7dq15vf7bc6cOTVaK/1St0JxbNmxY0epoVu3bjZ+/Hj78ccfq1UrvVJ/TJ061dq3b29btmyxnJwcGzhwoPXu3bvceSvTDx999JEVFBTYwYMH7X/+53/M5/PZV199Va0aj8sQPHny5ODdIVJTU+3DDz8MTi8oKLARI0aY3++3li1b2qxZs4J3hygxduxYa9y4scXFxdmzzz5b4RNzpBBcWFhoM2bMsI4dO1pMTIw1b97cLr/8ctuxY0eV9udEwsGn7m3evNnS0tIsKirKWrRoYdOnTy81/dJLL7XRo0cHH8+bN88klRlKPqlv377dOnXqZF6v1+Lj423IkCHVfpMyo1fqi6r2iySLiIgI3l2nZCjRs2dP83g8ZaavXr26WnXSL3Wvpo8tf8TdIU48RUVFNm7cOEtISDCfz2dXXHGF7dmzx8zMFixYUOrY8Ufl9UOfPn0sNjbWvF6vpaWl2ccff1ztGisKwR77w1V4oeLxeKy2toXQ8ng8Za7eBMpDr6Aq6BdUFr2CqvitXzx/HF+vL4wDAAAAQoEQDAAAAOcQggEAAOAcQjAAAACcQwgGAACAcwjBAAAAcA4hGAAAAM4hBAMAAMA5hGAAAAA4hxAMAAAA5xCCAQAA4BxCMAAAAJxDCAYAAIBzImprQ5GRkQGPx0PoPgFERkbK4/HUdRk4DtArqAr6BZVFr6AqIiMjA+WN95hZrRTg8XistraF0PJ4POK1RGXQK6gK+gWVRa+gKn7rlzKfmvhmFgAAAM4hBAMAAMA5hGAAAAA4hxAMAAAA5xCCAQAA4BxCMAAAAJxDCAYAAIBzCMEAAABwDiEYAAAAziEEAwAAwDn1NgT36dNHDz/8cF2XUa+MGTNGN998c12XAQAAcNyrtyH4zTff1F/+8pe6LqNeefLJJzVr1qy6LgP1QGZmpi666CJ5vV61atVKM2bMOOL8ixcvVo8ePRQbG6uIiIhy59m9e7eGDRumhIQExcbG6owzztAPP/wQivJRy2q6Xx544AH5fL5Sg8fj0a233hqqXUAtCcWxpcTs2bPl8Xg0derUmiwZ9VxxcbHGjx+vJk2aKCYmRldeeaX27t1b7rzvv/++PB5PqWPLueeeG7La6m0IrmuFhYV1XQJQruLiYvXv318dO3bUnj17tGzZMk2bNk3PPfdchcvEx8dr7NixevTRR8udfvjwYfXq1UsNGjTQxo0blZ2drYULF8rn84VqN1BLQtEvEydOVH5+fnBYv369PB6PhgwZEqrdQC0IRa+U2LZtm2bMmKHTTz+9pstGPffQQw9p6dKlWrdunXbu3ClJGjp0aIXzh4eHlzq+fPDBB6ErzsxqZfh1U5V3wQUX2H333Rd8/OGHH1pqaqr5fD7r3r27TZkyxZKTk4PTk5OT7f7777e0tDTzer3WqVMn+9e//hWcXlhYaFOmTLGTTz7Z4uPjLS0tzb744ovg9GHDhtl1111nw4cPt/j4eBszZowdPHjQrrjiCmvWrJnFxMTYmWeeacuXLw8uM2/ePGvXrp2lp6dby5Ytze/326hRo6yoqCg4T1ZWlg0cONCaN29ucXFxdu6559revXvNzGzv3r12ww03WKtWraxx48Z21VVX2Y8//ljhczJs2DAbMWKEmZkFAgGbOHGitWjRwnw+nyUnJ9tjjz1Wpef4WFX1tUTNWrFihUVFRVleXl5w3KRJk6xnz55HXXblypUWHh5eZvyTTz5prVq1soKCghqtlV6pe6Holz8aN26cnXnmmdWq04x+qWuh7JVevXrZ4sWLy7y3Hyt65fiRlJRkc+fODT7OzMw0SZaVlVVm3soec6rqt34pk02Pi2+Cc3Jy1LdvX11zzTXav3+/Hn/8cc2ZM6fMfE8//bQee+wx5eTk6OKLL9awYcOC06ZPn65nn31Wb7zxhnbt2qUePXro4osvVm5ubnCeJUuW6NJLL9WePXs0Y8YMBQIBDRgwQJs3b9a+fft07bXX6sorr9SePXuCy2zbtk0//fSTvvvuO3388cdasmSJFi9eLEn6+eeflZaWpqZNm+rbb7/V3r179cgjj6hBgwYyM11++eXyeDz68ssvtW3bNsXExOi6666r1HPyzjvv6B//+IfWrVunvLw8rVu3Tt27dz/WpxjHkQ0bNqh9+/alvqVNTU3Vhg0bjnmdK1eu1GmnnabRo0crISFBHTp00MyZM2uiXNSxUPTL7/3yyy965plnNGbMmBpZH+pOqHplzpw5io6O1qBBg6pbIo4zOTk52r59u7p06RIc165dO8XGxurzzz8vd5ni4mK1bt1azZs3V79+/WrsWFWe4yIEv/rqq/L5fLrzzjt10kkn6cwzz9QNN9xQZr7Ro0erU6dOCg8P18iRI5WZmamcnBxJ0rx58zRhwgR16NBBDRs21N///neFh4fr9ddfDy5/3nnnadCgQQoPD1d0dLR8Pp+GDBmimJgYnXTSSRo/frwaNGigjz/+OLhMVFSU7r33XjVs2FCnnHKKevXqpU8++USS9Nprr+nQoUNKT09XXFycIiIidM455ygmJkaffvqpPv30Uz3xxBOKi4tTdHS0Hn74Ya1YsSL4c8GRNGjQQIcPH9ZXX32lw4cPq1mzZkpNTa3uU43jQF5enuLi4kqN8/v9pT7QVdXevXu1fPlypaSkaNeuXVqwYIEeeOABLVy4sLrloo6Fol9+74UXXlBBQUGlP8Cj/gpFr2zfvl1Tp07V7Nmzq1sejkMlvVPZvurQoYMyMjKUlZWlb7/9Vp07d1ZaWlrIrk85LkLw999/r6SkJHk8nuC45OTkMvO1aNEi+G+v1yvp1z9qSdqxY4fatm0bnB4WFqY2bdpox44dwXFt2rQptb5Dhw7plltuUdu2bRUbGyu/368DBw6U+ia4adOmCg8PL7Xdkm1u3bpVbdu2LfdigaysLP3yyy9q1qyZ/H6//H6/2rVrp8jISG3fvv2oz0nPnj31wAMPaOrUqWratKl69+4dDN84sZScm1syxMTEBD/clcjOzlZsbOwxbyMmJkYtW7bUbbfdpgYNGuiss87SkCFDtHTp0uqWj1pWG/3ye3PmzNHgwYM5f/w4VBu9MnLkSE2aNEktW7asbrk4DsXExEhSpfuqefPmSklJUUREhPx+vx588EE1atRIb775ZkjqOy5CcMuWLbV9+/aSc4slqVJB8fdat26trKys4ONAIKCtW7eqdevWwXFhYaWfjpkzZ2rVqlV67733lJOTo+zsbMXHx5eq40jatGmjrKwsFRcXl5mWnJwsr9er/fv3Kzs7OzgcOnSo0ldCjho1SmvXrtWPP/6olJQUDRgwoFLL4fgyePDgUhcJpKSkaNOmTTp48GBwnvXr1yslJeWYt3HGGWeU+pBZorxxqN9qo19KfP3111qzZg2nQhynaqNX3nnnHU2cOFGNGzdW48aN9a9//UsPPvigevToURO7gHrO7/crKSlJn332WXDcli1blJubq86dO1dqHWFhYZXOXVV1XITgP/3pT8rLy9PMmTNVWFioDRs2aN68eVVax/Dhw/Xwww9r06ZNKigo0P3336+ioiL169evwmVyc3PVsGFDJSQkqKCgQPfee6+ys7Mrvc1+/fqpQYMG+n//7/8pJydHxcXF+vDDD5WXl6ezzjpLZ5xxhm677Tbt27dPkrRnz57g+cRH8/HHH2vt2rX65Zdf1LBhQ8XExBz19jQ4MZx//vlKTk7WxIkTdejQIWVkZGjOnDkaPXp0hcsUFxfr8OHDKigokPTr3SAOHz4cPLAMHz5c+/bt0xNPPKHi4mJt2LBBCxcu5IPVCSAU/VJizpw56tatW40EatS9UPTKjh07tGHDBmVkZCgjI0NnnXWWbrrpJr3wwgu1sk+oe6NGjdK0adOUlZWl3NxcTZgwQb179y7z67skrVixQpmZmQoEAsrPz9fkyZP1008/qXfv3iGp7bgIwX6/X6+//roWLlyo+Ph43XTTTRo+fLgaNmxY6XWMHz9e1157rS655BI1a9ZMK1as0PLly4/4M88dd9whv9+vxMREtWvXTtHR0eW+aBXxer1asWKFduzYoVNPPVUJCQkaP368CgsLFRYWpldeeUWBQEBdunRRTEyMunbtqvfff79S687Ly9Ott96qxo0bKyEhQcuXL690gMbxLTw8XK+++qq+/PJLJSQkqG/fvho/fryuueaa4Dx9+vQp9e3c/PnzFRUVpd69e6u4uFhRUVGKiorStm3bJP36y8Qbb7yhuXPnKjY2VgMHDtTkyZO5kOUEEIp+kX49XWz+/Pl8C3wCCUWvtGrVqtTQsGFDxcbGqlmzZrW+f6gbf/3rX9W/f3+dffbZatmypYqLi7VgwQJJKnMrzg0bNqhXr16KiYlR27Zt9eGHH+qdd94p9at9TfKE6ivmMhvyeKwmt3XXXXfp008/1fLly2tsnagcj8cTsp8mcGKhV1AV9Asqi15BVfzWL2XO7zsuvgmWfj2vaNeuXQoEAlqzZo3+93//V9dee21dlwUAAIDj0HFzEukXX3yhoUOHKjc3V4mJiRo/fnyp+wADAAAAlXXcng6BusPPUKgsegVVQb+gsugVVMVxfzoEAAAAUFMIwQAAAHAOIRgAAADOIQQDAADAOYRgAAAAOIcQDAAAAOcQggEAAOAcQjAAAACcQwgGAACAcwjBAAAAcE5EbW0oMjIy4PF4CN0ngMjISHk8Zf73QaAMegVVQb+gsugVVEVkZGSgvPGe2vq/tz0ej/H/fJ8Y+D/bUVn0CqqCfkFl0Suoit/6pcynJr6ZBQAAgHMIwQAAAHAOIRgAAADOIQQDAADAOYRgAAAAOIcQDAAAAOcQggEAAOAcQjAAAACcQwgGAACAcwjBAAAAcM5xGYJ79uypqVOn1nUZR+TxeLR27dq6LgMAAADlOC5DMAjZkIqLizV+/Hg1adJEMTExuvLKK7V3795y533//ffl8Xjk8/mCw7nnnltqnqKiIt1zzz1KTk6W1+tVu3bt9Oabb9bGrqCOZWZm6qKLLpLX61WrVq00Y8aMI87fqVOnUr0UFRUlj8ejzz77rJYqRqhUtRck6d1331W3bt3k8/nUuHFjjR07ttT07777TldccYXi4uIUFxenbt26qbCwMFS7gFpWk+9F+/fv1/nnn6+mTZsqNjZW7dq109SpU2VmIamdEFwJ/LGiPnrooYe0dOlSrVu3Tjt37pQkDR06tML5w8PDlZ+fHxw++OCDUtPHjBmj5cuX6+2331Z+fr7WrFmjjh07hnQfUPeKi4vVv39/dezYUXv27NGyZcs0bdo0PffccxUu89VXX5XqpTvuuEOnnXaaUlNTa7Fy1LRj6YX3339fAwcO1J133ql9+/Zp586dGjlyZHD6nj171KNHD6WkpGj79u3av3+/Zs2apfDw8NrYJdSCmnwv8nq9mj17tr7//nvl5ubq3Xff1cKFC/XUU0+Fpngzq5Xh101VXl5eno0bN85OPvlk8/l8dtppp9maNWvMzOyCCy6wO+64wwYMGGA+n8/atm1rr7zySqnlX375ZUtNTbW4uDjr0KGDLViwoNT01atXW/fu3S0+Pt7atm1rjzzyiAUCATMzW7lypYWHh9uzzz4b3L6ZWXp6urVp08Z8Pp8lJibaXXfdVWH9koL17t6928455xwbMWKEFRYWHrW+efPmWbt27Sw9Pd1atmxpfr/fRo0aZUVFRWZm1rlzZ5NkUVFR5vV6bcSIEVV6bqurqq8lQiMpKcnmzp0bfJyZmWmSLCsrq8y8JT1dkW+//dYk2TfffFOjNdIr9d+KFSssKirK8vLyguMmTZpkPXv2rNTyhYWF1rx5c0tPT692LfRL3TqWXujWrZtNmDChwul//etfrWvXrjVapxm9Up/U5HvRH23ZssVOO+00u+OOO6pV4w5IwQQAABm1SURBVG/9UjabljcyFENVG/bqq6+28847z7Zs2WKBQMA2bdpkmzdvNrNfQ3BCQoKtXbvWiouLbebMmRYXF2cHDx40M7Ply5dbo0aNbPXq1VZcXGzr1q0zv99vq1atMjOzL7/80nw+n73yyitWVFRk33zzjbVp08b+8Y9/mNmvL5Iku/baay07O9sOHjxoGzdutKioKPvyyy/NzOzAgQP273//+4hP+Jo1a2zjxo12yimn2H333RecdrT65s2bZxERETZx4kQ7fPiwbd682eLj40sF5d+H7NrGwafuZWdnmyRbv359qfGxsbG2dOnSMvOX9HSrVq2sWbNm1rdvX8vIyAhOnz17tsXGxtqMGTMsMTHRWrdubWPGjLHc3Nxq1Umv1H///d//bSkpKaXGvfTSSxYfH1+p5ZcsWWJRUVF24MCBatdCv9StqvZCfn6+hYWF2T333GNnnnmmJSQk2AUXXGAff/xxcJ6uXbvakCFD7M9//rPFx8fb6aefXuZLqWNBr9QPNf1eVKJfv34WGRlpkqx169a2cePGatVZUQiul6dD7N69W88//7yefPJJnXzyyfJ4PDr11FN1yimnBOcZNGiQunfvrrCwMI0aNUo5OTnavHmzJCk9PV233XabevToobCwMP3nf/6nhgwZomeffVaSNHv2bF111VW67LLLFB4erg4dOujmm28OTi/x0EMPKS4uTtHR0YqIiJCZBX8G9Pv96tat2xH3Y82aNbrgggs0efJkTZo0KTj+aPVJUlRUlO699141bNhQp5xyinr16qVPPvmk2s8tTgy5ubmSpLi4uFLj/X5/cNrvdejQQRkZGcrKytK3336rzp07Ky0tTT/88IMkae/evcrNzdXHH3+sb775RuvWrVNGRobuuOOO0O8M6lReXl6l+6g8c+bM0aBBg+T3+0NRHmpRVXvhwIEDCgQCeuqpp/TMM8/ohx9+0CWXXKK+ffsqOztb0q/Hln/+858aOnSodu/erRkzZmjEiBFc03KCqOn3ohKvvfaa8vPz9e9//1tDhw5V48aNQ1J/vQzBW7dulSS1b9++wnlatGgR/LfX65X06x+wJGVlZWnatGny+/3BoeQPtGT6okWLSk2fMmWKdu3aFVxnWFiYWrduHXzctm3b4HkpiYmJOu+887R8+fIj7kd6ero6deqkQYMGlRp/tPokqWnTpqXOmfJ6vcH9A2JiYiRJOTk5pcZnZ2crNja2zPzNmzdXSkqKIiIi5Pf79eCDD6pRo0bBC99K1nffffcpNjZWLVq00IQJE7R06dIQ7wlq28KFC0tdlBITE1PpPvqj7777Tu+9957GjBkTqnIRQtXthZLjxvXXX6/OnTurQYMGuuuuu1RYWBg8zzMmJkbnnHOOBg4cqIiICF188cW69NJLtWzZstDuHGpFTb8X/V54eLi6deumuLg43XTTTSGpv16G4DZt2khS8JvdqkpOTtbkyZOVnZ0dHPLy8vTGG28Ep99www2lpufm5uqrr74KrsPj8cjj8ZRa74ABA/TOO+9o7969uvrqq3XZZZfp559/rrCOZ555RoFAQFdeeaV++eWXStdXGX+sDW7x+/1KSkoqdTX+li1blJubq86dO1dqHWFhYcErbs844wxJZfuKPjvxDB48uNRFKSkpKdq0aZMOHjwYnGf9+vVKSUk56rrmzJmjlJQUde3aNZQlI0Sq2wtxcXFq06ZNuceJknFnnHHGEafj+FbT70XlKSoqOuY8eNRth2St1dS0aVMNHDhQY8eO1datW2VmyszMVGZmZqWWv/322/Xoo49qzZo1Ki4uVkFBgT799NPg6QRjx47V4sWL9eqrr6qwsFBFRUX6+uuvtWrVqgrXuXHjRr311lv6+eefddJJJykuLk4ej0dhYRU/hT6fT2+88YaKiorUr1+/4IHlaPVVRvPmzUPWFDg+jBo1StOmTVNWVpZyc3M1YcIE9e7dO/gh8vdWrFihzMxMBQIB5efna/Lkyfrpp5/Uu3dvSVKPHj10+umn65577tHBgwe1e/duTZ8+XQMGDKjlvUJtO//885WcnKyJEyfq0KFDysjI0Jw5czR69OgjLldQUKBnnnmGb4FPIMfSC2PHjtW8efP09ddfq6ioSNOnT1dkZGTwtlejR4/Whx9+qFdeeUWBQEArV67U8uXLdfnll9fWbiHEavK96MMPP9R7772nQ4cOqbi4WKtXr1Z6err69OkTmuLLO1E4FIOqeBJ7bm6u3XbbbZaUlGRer7fM3SF+f6FZyUnPv79Q7LXXXrOuXbua3++3Ro0aWY8ePWzlypXB6R988IGlpaVZQkKCxcfH29lnn21Lliwxs/KvXvz888+tW7duFhsba7GxsZaammpvvPFGhfX/vp6CggK7+uqr7ZxzzglePHKk+kruDvF7w4YNK3UXiKefftpatWoVvHNEbarqa4nQKCoqsnHjxllCQoL5fD674oorbM+ePWZmtmDBAvN6vcF5Z86caUlJSRYdHW1NmjSx3r1720cffVRqfVu3brU+ffqY1+u1xMREu+WWWyw/P79aNdIrx4fNmzdbWlqaRUVFWYsWLWz69Omlpl966aU2evToUuMWLVpkPp+v1J0Eqot+qXtV7YVAIGB33323NWvWzOLi4qxnz55lLpJ6/vnnrX379hYdHW2dOnWy559/vtp10iv1R02+F61atcpSU1PN5/NZTEyMdejQwaZOnRq8O9axUgUXxnksRDcg/iOPx2O1tS2ElsfjCdmNq3FioVdQFfQLKoteQVX81i9lzsGpl6dDAAAAAKFECAYAAIBzCMEAAABwDiEYAAAAziEEAwAAwDmEYAAAADiHEAwAAADnEIIBAADgHEIwAAAAnEMIBgAAgHMIwQAAAHAOIRgAAADOIQQDAADAORG1taHIyMiAx+MhdJ8AIiMj5fF46roMHAfoFVQF/YLKoldQFZGRkYHyxnvMrFYK8Hg8VlvbQmh5PB7xWqIy6BVUBf2CyqJXUBW/9UuZT018MwsAAADnEIIBAADgHEIwAAAAnEMIBgAAgHMIwQAAAHAOIRgAAADOIQQDAADAOYRgAAAAOIcQDAAAAOcQggEAAOCc4yoEDx8+XCNHjqzRdX7yySfq3LmzYmJidPvtt2vy5Mm66KKLKpx/+/bt8vl8+uGHH2q0DgAAANSeehuCe/bsqalTp4Z8OxMnTtSll16qvLw8Pfroo0edPykpSfn5+UpMTAx5bUBFMjMzddFFF8nr9apVq1aaMWPGEedfvHixevToodjYWEVERJSZvmbNGqWmpqpRo0aKi4tTamqqXnrppVCVj1pW1X6ZMGGCOnXqpNjYWCUmJurGG2/U/v37g9M3bNigPn36qEWLFvJ4PFq7dm2odwH1SFX76W9/+5tOPvlkxcbGqmnTpho4cKC2b99eS9Ui1IqLizV+/Hg1adJEMTExuvLKK7V3795y533//ffl8Xjk8/mCw7nnnltqHo/Ho+jo6FLz5OTkhKT2ehuCa8uWLVvUuXPnui4DqLTi4mL1799fHTt21J49e7Rs2TJNmzZNzz33XIXLxMfHa+zYsRV+0PuP//gPvfzyy9q3b5+ys7P16KOPasiQIfrmm29CtRuoJcfSL+Hh4VqwYIH27dunDRs2aOfOnbr++uuD0xs0aKABAwZo2bJltbELqEeOpZ+GDh2qjIwM5ebmauvWrUpKStI111xTi1UjlB566CEtXbpU69at086dOyX9+ppXJDw8XPn5+cHhgw8+KDPP8uXLS80TFxcXmuLNrFaGXzdVOTfddJOFhYVZgwYNzOv1Wvv27c3MbNiwYTZkyBAbOXKkxcXFWWJioj355JOlll29erV1797d4uPjrW3btvbII49YIBAodztxcXHm8XisYcOG5vV67Z133rF77rnH0tLS7Pbbb7dGjRpZy5Yt7cEHHwwuk5WVZZJsx44dld6fE01VXkvUvBUrVlhUVJTl5eUFx02aNMl69ux51GVXrlxp4eHhR5ynuLjY1qxZYw0bNrRly5ZVq1Z6pe5Vp19KvPbaaxYbG1vuNEm2Zs2aatdZsi7Ub9Xtp/z8fBs3bpw1atSoWnXQK/VHUlKSzZ07N/g4MzPTJFlWVlaZeSvzHlSTx5Tfr9PKyab18pvgWbNmqUePHrr77ruVn5+vjRs3Bqe98MIL6t+/v/bv36/HH39cN998s7Zt2yZJ+uqrr9S3b1+NHz9ee/bs0euvv65Zs2Zp/vz55W4nOztbSUlJmjt3rvLz84PnAq9evVrNmjXTrl27tHTpUs2cOVOLFi0K/Y4DlbBhwwa1b99ePp8vOC41NVUbNmyo9rr9fr8aNmyoHj16qGvXrrrkkkuqvU7UrZrol/fee49fzCDp2Pvpn//8p+Li4uTz+ZSenq7JkyeHuFLUhpycHG3fvl1dunQJjmvXrp1iY2P1+eefl7tMcXGxWrdurebNm6tfv37l9s5VV12lxo0bq2vXriE9Na9ehuAjSUtL05///GeFhYVpwIAB8vv9ysjIkCTNnj1bV111lS677DKFh4erQ4cOuvnmm/Xss89WaRstWrTQhAkT1KBBA3Xp0kWjRo3SvHnzQrE7QJXl5eWV+WnI7/crNze32uvOzs5Wfn6+Xn75ZfXt27fc84dxfKluv7z44ot66qmnlJ6eHorycJw51n667rrrlJOTo127dmny5Mk6/fTTQ1kmaknJ617ZnujQoYMyMjKUlZWlb7/9Vp07d1ZaWlqpmw28++67ysrK0s6dO3XHHXdo8ODBeuutt0JS/3EXglu0aFHqsdfrVV5eniQpKytLixYtkt/vDw5TpkzRrl27qrSN5ORkeTye4OM2bdoEz3MBatvChQtLXSAQExNT5iKB7OxsxcbG1sj2GjZsqMsvv1yrVq3S3Llza2SdqD012S9LlizRjTfeqGXLlik1NTVUJaMeq+njT/PmzXXjjTfqT3/6U6mLLXF8iomJkaRK90Tz5s2VkpKiiIgI+f1+Pfjgg2rUqJHefPPN4Dy9evVSZGSkIiMjNWjQIA0ZMkQLFy4MSf31NgSHhVW9tOTkZN1www3Kzs4ODrm5ufrqq6+qtJ5t27aVnMcsSdq6datatWpV5XqAmjB48OBSFwikpKRo06ZNOnjwYHCe9evXKyUlpUa3W1RUpM2bN9foOhF6NdUv8+bN0+jRo/Xqq6/qwgsvDHXZqKdCcfwpKirSwYMHudXoCcDv9yspKUmfffZZcNyWLVuUm5tb6VOowsLCSmWuqk6vjnobgps3b67MzMwqLTN27FgtXrxYr776qgoLC1VUVKSvv/5aq1atqtJ6du3apenTp6uwsFDr16/XU089pWHDhlVpHUConH/++UpOTtbEiRN16NAhZWRkaM6cORo9enSFyxQXF+vw4cMqKCiQJB0+fFiHDx8OHlhefPFFffHFFyoqKtLhw4f11FNPacWKFerdu3et7BNC51j65bHHHtOdd96pt99+W927dy8z3cyCPSRJBQUFOnz4sIqLi0O2H6gfqtpPgUBAs2bN0u7duyVJO3fu1E033aQ2bdqoQ4cOtVk6QmTUqFGaNm2asrKylJubqwkTJqh3795q06ZNmXlXrFihzMxMBQIB5efna/Lkyfrpp5+C7zVffvmlPvroIxUUFKiwsFCvvPKK5s+fr6uvvjo0xZd3tVwoBlXxSs6PPvrIOnXqZHFxcXbaaaeZ2a93hxgxYkSp+ZKTk23+/PnBxx988IGlpaVZQkKCxcfH29lnn21LliypcDt/XP6ee+6xCy+8MHh3iMTERLv//vuDd5jg7hBclVsfbN682dLS0iwqKspatGhh06dPLzX90ksvtdGjRwcfz5s3zySVGUqu3n388cftlFNOMa/Xa/Hx8datWzd7/vnnq10nvVI/VLVfJFlERIR5vd5SQ4mS4+Afh3nz5lWrTvrl+FCVfiouLrY+ffpYkyZNLDo62hITE+26666zzMzMatVAr9QfRUVFNm7cOEtISDCfz2dXXHGF7dmzx8zMFixYUOrYMXPmTEtKSrLo6Ghr0qSJ9e7d2z766KPg9BUrVthpp51m0dHR5vf7rUuXLrZo0aJq16gK7g7hsRB9xfxHHo/HamtbCC2PxxOynyZwYqFXUBX0CyqLXkFV/NYvnj+Or7enQwAAAAChQggGAACAcwjBAAAAcA4hGAAAAM4hBAMAAMA5hGAAAAA4hxAMAAAA5xCCAQAA4BxCMAAAAJxDCAYAAIBzCMEAAABwDiEYAAAAziEEAwAAwDkRtbWhyMjInzweT7Pa2h5CJzIyMuDxePgAhaOiV1AV9Asqi15BVURGRv5U3niPmdV2LQAAAECd4lMUAAAAnEMIBgAAgHMIwQAAAHAOIRgAAADOIQQDAADAOYRgAAAAOIcQDAAAAOcQggEAAOAcQjAAAACcQwgGAACAcwjBAAAAcA4hGAAAAM4hBAMAAMA5hGAAAAA4hxAMAAAA5xCCAQAA4BxCMAAAAJxDCAYAAIBzCMEAAABwDiEYAAAAziEEAwAAwDmEYAAAADiHEAwAAADnEIIBAADgHEIwAAAAnEMIBgAAgHMIwQAAAHAOIRgAAADOIQQDAADAOYRgAAAAOIcQDAAAAOcQggEAAOAcQjAAAACcQwgGAACAcwjBAAAAcA4hGAAAAM4hBAMAAMA5hGAAAAA4hxAMAAAA5xCCAQAA4BxCMAAAAJxDCAYAAIBzCMEAAABwDiEYAAAAziEEAwAAwDmEYAAAADiHEAwAAADnEIIBAADgHEIwAAAAnEMIBgAAgHMIwQAAAHAOIRgAAADOIQQDAADAOYRgAAAAOIcQDAAAAOcQggEAAOAcQjAAAACcQwgGAACAcwjBAAAAcA4hGAAAAM4hBAMAAMA5hGAAAAA4hxAMAAAA5xCCAQAA4BxCMAAAAJxDCAYAAIBzCMEAAABwDiEYAAAAziEEAwAAwDmEYAAAADiHEAwAAADnEIIBAADgHEIwAAAAnEMIBgAAgHMIwQAAAHAOIRgAAADOIQQDAADAOYRgAAAAOIcQDAAAAOcQggEAAOAcQjAAAACcQwgGAACAcwjBAAAAcA4hGAAAAM4hBAMAAMA5hGAAAAA4hxAMAAAA5xCCAQAA4BxCMAAAAJxDCAYAAIBzCMEAAABwDiEYAAAAziEEAwAAwDmEYAAAADiHEAwAAADnEIIBAADgHEIwAAAAnEMIBgAAgHMIwQAAAHAOIRgAAADOIQQDAADAOYRgAAAAOIcQDAAAAOcQggEAAOAcQjAAAACcQwgGAACAcwjBAAAAcA4hGAAAAM4hBAMAAMA5hGAAAAA4hxAMAAAA5xCCAQAA4BxCMAAAAJxDCAYAAIBzCMEAAABwDiEYAAAAziEEAwAAwDmEYAAAADiHEAwAAADnEIIBAADgHEIwAAAAnEMIBgAAgHMIwQAAAHAOIRgAAADOIQQDAADAOYRgAAAAOIcQDAAAAOcQggEAAOAcQjAAAACcQwgGAACAcwjBAAAAcA4hGAAAAM4hBAMAAMA5hGAAAAA4hxAMAAAA5xCCAQAA4BxCMAAAAJxDCAYAAIBzCMEAAABwDiEYAAAAziEEAwAAwDmEYAAAADiHEAwAAADnEIIBAADgHEIwAAAAnEMIBgAAgHMIwQAAAHAOIRgAAADOIQQDAADAOYRgAAAAOIcQDAAAAOcQggEAAOAcQjAAAACcQwgGAACAcwjBAAAAcA4hGAAAAM4hBAMAAMA5hGAAAAA4hxAMAAAA5xCCAQAA4BxCMAAAAJxDCAYAAIBzCMEAAABwDiEYAAAAziEEAwAAwDmEYAAAADiHEAwAAADnEIIBAADgHEIwAAAAnEMIBgAAgHMIwQAAAHAOIRgAAADOIQQDAADAOYRgAAAAOIcQDAAAAOcQggEAAOAcQjAAAACcQwgGAACAcwjBAAAAcA4hGAAAAM4hBAMAAMA5hGAAAAA4hxAMAAAA5xCCAQAA4BxCMAAAAJxDCAYAAIBzCMEAAABwDiEYAAAAziEEAwAAwDmEYAAAADiHEAwAAADnEIIBAADgHEIwAAAAnEMIBgAAgHMIwQAAAHAOIRgAAADOIQQDAADAOYRgAAAAOIcQDAAAAOcQggEAAOAcQjAAAACcQwgGAACAcwjBAAAAcA4hGAAAAM4hBAMAAMA5hGAAAAA4hxAMAAAA5xCCAQAA4BxCMAAAAJxDCAYAAIBzCMEAAABwDiEYAAAAziEEAwAAwDmEYAAAADiHEAwAAADnEIIBAADgHEIwAAAAnEMIBgAAgHMIwQAAAHAOIRgAAADOIQQDAADAOYRgAAAAOIcQDAAAAOcQggEAAOAcQjAAAACcQwgGAACAcwjBAAAAcA4hGAAAAM4hBAMAAMA5hGAAAAA4hxAMAAAA5xCCAQAA4BxCMAAAAJxDCAYAAIBzCMEAAABwDiEYAAAAziEEAwAAwDmEYAAAADiHEAwAAADnEIIBAADgHEIwAAAAnEMIBgAAgHMIwQAAAHAOIRgAAADOIQQDAADAOYRgAAAAOIcQDAAAAOcQggEAAOAcQjAAAACcQwgGAACAcwjBAAAAcA4hGAAAAM4hBAMAAMA5hGAAAAA4hxAMAAAA5xCCAQAA4BxCMAAAAJxDCAYAAIBzCMEAAABwDiEYAAAAziEEAwAAwDmEYAAAADiHEAwAAADnEIIBAADgHEIwAAAAnEMIBgAAgHMIwQAAAHAOIRgAAADOIQQDAADAOYRgAAAAOIcQDAAAAOcQggEAAOAcQjAAAACcQwgGAACAcwjBAAAAcA4hGAAAAM4hBAMAAMA5/x97pxU66A97ZQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 720x360 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "P4(train_data, train_labels, dev_data, dev_labels)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "cY67F-tXFsxt"
   },
   "source": [
    "ANSWER: \n",
    "\n",
    "Surprisingly, the most common bigrams are not necessarily related to the topics of interest, and several are found across multiple cateogories. \n",
    "\n",
    "Some of the values in this table are also misleading if the \"abs\" output is used, as 1.11 is technically larger than what the largest real probability is for the bigrams in the category in question. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "rVWhSJHHFsxu"
   },
   "source": [
    "### Part 5:\n",
    "\n",
    "To improve generalization, it is common to try preprocessing text in various ways before splitting into words. For example, you could try transforming strings to lower case, replacing sequences of numbers with single tokens, removing various non-letter characters, and shortening long words.\n",
    "\n",
    "Produce a Logistic Regression model (with no preprocessing of text).  Evaluate and show its f1 score and size of the dictionary.\n",
    "\n",
    "Produce an improved Logistic Regression model by preprocessing the text.  Evaluate and show its f1 score and size of the vocabulary.  Try for an improvement in f1 score of at least 0.02.\n",
    "\n",
    "How much did the improved model reduce the vocabulary size?\n",
    "\n",
    "Notes:\n",
    "* Train on the transformed training data.\n",
    "* Evaluate on the transformed development data.\n",
    "* You can use `CountVectorizer(preprocessor=...)` to preprocess strings with your own custom-defined function.\n",
    "* `CountVectorizer` default is to preprocess strings to lower case.\n",
    "* You can use `LogisticRegression(C=0.5, solver=\"liblinear\", multi_class=\"auto\")` to produce a logistic regression model.\n",
    "* You can use `metrics.f1_score(..., average=\"weighted\")` to compute f1 score.\n",
    "* If you're not already familiar with regular expressions for manipulating strings, see https://docs.python.org/2/library/re.html, and re.sub() in particular."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 83,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "l7gS3cGpFsxv"
   },
   "outputs": [],
   "source": [
    "def better_preprocessor(s):\n",
    "    \n",
    "    # Convert to lower case\n",
    "    s = s.lower()\n",
    "\n",
    "    # Remove stop words\n",
    "    tk = nltk.tokenize\n",
    "    text_tokens = tk.word_tokenize(s)\n",
    "    tokens_without_sw = [word for word in text_tokens if not word in ENGLISH_STOP_WORDS]\n",
    "    s = (\" \").join(tokens_without_sw)\n",
    "    \n",
    "    # Remove newlines\n",
    "    s = re.sub(r'\\n', ' ', s)\n",
    "    \n",
    "    # Replace digits \n",
    "    s = re.sub(r'\\d', '@', s)\n",
    "\n",
    "    # Remove punctuation \n",
    "    s = re.sub(r'[^A-Za-z0-9]+', ' ', s)\n",
    "    \n",
    "    # Replace digits \n",
    "    s = re.sub(r'\\d', '@', s)\n",
    "    \n",
    "    # Remove words with certain endings\n",
    "    s = re.sub('ing ', ' ', s)\n",
    "    s = re.sub('est ', ' ', s)\n",
    "    s = re.sub('ly ', ' ', s)\n",
    "    s = re.sub('ion ', ' ', s)\n",
    "    \n",
    "    return s\n",
    "\n",
    "def P5(training_data, training_labels, testing_data, testing_labels):\n",
    "    \n",
    "    # Train models with no pre-processing\n",
    "    vec1 = CountVectorizer(preprocessor = None)\n",
    "    vec_fit_train1 = vec1.fit_transform(training_data)\n",
    "    vec_fit_dev1 = vec1.transform(testing_data)\n",
    "    \n",
    "    LR1 = LogisticRegression(C = 0.5, solver = \"liblinear\", multi_class = \"auto\")\n",
    "    LR_fit1 = LR1.fit(vec_fit_train1, training_labels)\n",
    "    LR_predict1 = LR_fit1.predict(vec_fit_dev1)\n",
    "    LR1_f1 = metrics.f1_score(testing_labels, LR_predict1, average=\"weighted\")\n",
    "    \n",
    "    # Train models with pre-processing\n",
    "    vec = CountVectorizer(preprocessor = better_preprocessor)\n",
    "    vec_fit_train = vec.fit_transform(training_data)\n",
    "    vec_fit_dev = vec.transform(testing_data)\n",
    "    \n",
    "    LR = LogisticRegression(C = 0.5, solver = \"liblinear\", multi_class = \"auto\")\n",
    "    LR_fit = LR.fit(vec_fit_train, training_labels)\n",
    "    LR_predict = LR_fit.predict(vec_fit_dev)\n",
    "    LR_f1 = metrics.f1_score(testing_labels, LR_predict, average=\"weighted\")\n",
    "    \n",
    "    print(\"The non pre-processed F1 score is: {}\".format(round(LR1_f1, 3)))\n",
    "    print(\"--------------------------------------------------------------\")\n",
    "    print(\"The pre-processed F1 score is: {}\".format(round(LR_f1, 3)))\n",
    "    print(\"--------------------------------------------------------------\")\n",
    "    print(\"With pre-processing the score has improved by {}, or {}%\".format(round(LR_f1-LR1_f1, 3), round(((LR_f1-LR1_f1)/LR1_f1)*100, 2)))\n",
    "    \n",
    "    return "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 84,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The non pre-processed F1 score is: 0.71\n",
      "--------------------------------------------------------------\n",
      "The pre-processed F1 score is: 0.732\n",
      "--------------------------------------------------------------\n",
      "With pre-processing the score has improved by 0.022, or 3.13%\n"
     ]
    }
   ],
   "source": [
    "P5(train_data, train_labels, dev_data, dev_labels)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "I experimented with various pre-processing methods including tokenizing, stemming and other methods, however I found that the the included in this order of application improved the logistic regression models by the largest amount."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "Uy-WITbNFsxw"
   },
   "source": [
    "### Part 6:\n",
    "\n",
    "The idea of regularization is to avoid learning very large weights (which are likely to fit the training data, but not generalize well) by adding a penalty to the total size of the learned weights. Logistic regression seeks the set of weights that minimizes errors in the training data AND has a small total size. The default L2 regularization computes this size as the sum of the squared weights (as in Part 3 above). L1 regularization computes this size as the sum of the absolute values of the weights. Whereas L2 regularization makes all the weights relatively small, L1 regularization drives many of the weights to 0, effectively removing unimportant features.\n",
    "\n",
    "For several L1 regularization strengths ...<br/>\n",
    "* Produce a Logistic Regression model using the **L1** regularization strength.  Reduce the vocabulary to only those features that have at least one non-zero weight among the four categories.  Produce a new Logistic Regression model using the reduced vocabulary and **L2** regularization strength of 0.5.  Evaluate and show the L1 regularization strength, vocabulary size, and f1 score associated with the new model.\n",
    "\n",
    "Show a plot of f1 score vs. log vocabulary size.  Each point corresponds to a specific L1 regularization strength used to reduce the vocabulary.\n",
    "\n",
    "How does performance of the models based on reduced vocabularies compare to that of a model based on the full vocabulary?\n",
    "\n",
    "Notes:\n",
    "* Train on the transformed training data.\n",
    "* Evaluate on the transformed development data.\n",
    "* You can use `LogisticRegression(..., penalty=\"l1\")` to produce a logistic regression model using L1 regularization.\n",
    "* You can use `LogisticRegression(..., penalty=\"l2\")` to produce a logistic regression model using L2 regularization.\n",
    "* You can use `LogisticRegression(..., tol=0.015)` to produce a logistic regression model using relaxed gradient descent convergence criteria.  The gradient descent code that trains the logistic regression model sometimes has trouble converging with extreme settings of the C parameter. Relax the convergence criteria by setting tol=.015 (the default is .0001)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 97,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "l6ho31SrFsxx"
   },
   "outputs": [],
   "source": [
    "def P6(training_data, training_labels, testing_data, testing_labels, c):\n",
    "    \n",
    "    # Keep this random seed here to make comparison easier.\n",
    "    seed = np.random.seed(0)\n",
    "    \n",
    "    # Transform raw datasets into matrices\n",
    "    vec = CountVectorizer()\n",
    "    vec_fit_train = vec.fit_transform(training_data)\n",
    "    vec_fit_test = vec.transform(testing_data)\n",
    "    \n",
    "    # Create some lists\n",
    "    vocab_size = []\n",
    "    f1_scores = []\n",
    "    l2_f1_scores = []\n",
    "    \n",
    "    # Fit models with different C's\n",
    "    for value in c:\n",
    "        l1_model = LogisticRegression(solver = 'liblinear', penalty = \"l1\", C = value, tol = 0.015, random_state = seed)\n",
    "        fitted_l1_model = l1_model.fit(vec_fit_train, training_labels)\n",
    "        l1_model_predict = fitted_l1_model.predict(vec_fit_test)\n",
    "        l1_model_f1 = metrics.f1_score(testing_labels, l1_model_predict, average = \"weighted\")\n",
    "        f1_scores.append(l1_model_f1)\n",
    "        vocab_size.append(np.count_nonzero(np.count_nonzero(l1_model.coef_, axis = 0)))\n",
    "        \n",
    "        # Get index positions of all non-zero features\n",
    "        indices = np.nonzero(np.count_nonzero(l1_model.coef_, axis = 0))\n",
    "        \n",
    "        # Compile new vocabulary based upon previous output\n",
    "        new_vocab = []\n",
    "        for lis in indices:\n",
    "            for index in lis:\n",
    "                new_vocab.append(vec.get_feature_names()[index])\n",
    "                \n",
    "        # Run new LR model with l2 regularization and C = 0.5\n",
    "        new_vec = CountVectorizer(vocabulary = new_vocab)\n",
    "        new_vec_fit_train = new_vec.fit_transform(training_data)\n",
    "        new_vec_fit_test = new_vec.transform(testing_data)\n",
    "        \n",
    "        l2_model = LogisticRegression(solver = 'liblinear', penalty = \"l2\", C = 0.5, tol = 0.015, random_state = seed)\n",
    "        fitted_l2_model = l2_model.fit(new_vec_fit_train, training_labels)\n",
    "        l2_model_predict = fitted_l2_model.predict(new_vec_fit_test)\n",
    "        l2_model_f1 = metrics.f1_score(testing_labels, l2_model_predict, average = \"weighted\")\n",
    "        l2_f1_scores.append(l2_model_f1)\n",
    "    \n",
    "    # Print out model performance info\n",
    "    print(\"Model Performance Metrics:\")\n",
    "    print(\"------------------------------------------------------------------------------------\")\n",
    "    for size, f1_1, f1_2, c in zip(vocab_size, f1_scores, l2_f1_scores, c):\n",
    "        print(\"The f1 score using with L1 regularization and a C of {} is {}\".format(round(c, 1), round(f1_1, 3)))\n",
    "        print(\"The f1 score using with L2 regularization, a C of 0.5 and vocabulary size of {} is {}\".format(size, round(f1_2, 3)))\n",
    "        print(\"------------------------------------------------------------------------------------\")\n",
    "    \n",
    "    # Plot performance graphs \n",
    "    fig, ax1 = plt.subplots(figsize = (8,5))   \n",
    "    ax1.set_title(\"Model f1 Score for Various Vocabulary Sizes\")\n",
    "    ax1.set_xlabel('log(Vocabulary Size)')\n",
    "    ax1.set_ylabel('F-1 Score')\n",
    "    ax1.plot(np.log(vocab_size), f1_scores, marker = 'x', color = \"purple\", label = \"L1\")\n",
    "    ax1.plot(np.log(vocab_size), l2_f1_scores, marker = 'v', color = \"darkgoldenrod\", label = \"L2\")\n",
    "    ax1.legend()\n",
    "        \n",
    "    return"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 98,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model Performance Metrics:\n",
      "------------------------------------------------------------------------------------\n",
      "The f1 score using with L1 regularization and a C of 0.0 is 0.428\n",
      "The f1 score using with L2 regularization, a C of 0.5 and vocabulary size of 17 is 0.468\n",
      "------------------------------------------------------------------------------------\n",
      "The f1 score using with L1 regularization and a C of 0.1 is 0.652\n",
      "The f1 score using with L2 regularization, a C of 0.5 and vocabulary size of 228 is 0.689\n",
      "------------------------------------------------------------------------------------\n",
      "The f1 score using with L1 regularization and a C of 0.2 is 0.693\n",
      "The f1 score using with L2 regularization, a C of 0.5 and vocabulary size of 401 is 0.672\n",
      "------------------------------------------------------------------------------------\n",
      "The f1 score using with L1 regularization and a C of 0.3 is 0.696\n",
      "The f1 score using with L2 regularization, a C of 0.5 and vocabulary size of 523 is 0.678\n",
      "------------------------------------------------------------------------------------\n",
      "The f1 score using with L1 regularization and a C of 0.4 is 0.688\n",
      "The f1 score using with L2 regularization, a C of 0.5 and vocabulary size of 651 is 0.668\n",
      "------------------------------------------------------------------------------------\n",
      "The f1 score using with L1 regularization and a C of 0.5 is 0.691\n",
      "The f1 score using with L2 regularization, a C of 0.5 and vocabulary size of 762 is 0.673\n",
      "------------------------------------------------------------------------------------\n",
      "The f1 score using with L1 regularization and a C of 0.6 is 0.691\n",
      "The f1 score using with L2 regularization, a C of 0.5 and vocabulary size of 873 is 0.686\n",
      "------------------------------------------------------------------------------------\n",
      "The f1 score using with L1 regularization and a C of 0.7 is 0.685\n",
      "The f1 score using with L2 regularization, a C of 0.5 and vocabulary size of 953 is 0.68\n",
      "------------------------------------------------------------------------------------\n",
      "The f1 score using with L1 regularization and a C of 0.8 is 0.7\n",
      "The f1 score using with L2 regularization, a C of 0.5 and vocabulary size of 1009 is 0.677\n",
      "------------------------------------------------------------------------------------\n",
      "The f1 score using with L1 regularization and a C of 0.9 is 0.69\n",
      "The f1 score using with L2 regularization, a C of 0.5 and vocabulary size of 1070 is 0.673\n",
      "------------------------------------------------------------------------------------\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAfgAAAFNCAYAAADsL325AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nOzdd3hUZfbA8e9JBxJIqAklJMRQpAUMRUARFRBFulQp4tpdV/1t0W2WXXddV3fXvmtBRRQS6R3pKhaIJgFCJ4EkhFBCKAnp8/7+mAFDmIQAmdxJOJ/nmYeZW8+9Gebct9z3ijEGpZRSStUuHlYHoJRSSqmqpwleKaWUqoU0wSullFK1kCZ4pZRSqhbSBK+UUkrVQprglVJKqVpIE7xyGyISJiJGRLwqsew0EfmmgvkjRSRNRHJEpFvVRmoNEWknIvEickZEnrA6HmdEZIWITLU6DlcTkQ0i8osrXPd5EZlV1TFdQRyhjv8fnlbHolxDE7y6IiJyQEQKRaRxmekJjiQdZk1k570KPG6M8TfGxIvI4yISJyIFIvJxRSuKiI+IvCYi6Y4fwBQR+Xf1hF2h3wIbjDEBxpg3rmZDIjLB8TeUMtO9ROSoiAy9ku0aY4YYYz65mtgqS0RaiEixiEQ4mbdARF6tjjjcmYi0FJF5InJcRE6JyDYRmQZgjEl1/P8osThM5SKa4NXVSAEmnPsgIp2BOtaFc4HWQFKpzxnAX4EZlVj3WSAa6AkEAAOA+KoMrjK1FE6UPaar2d8CIBDoX2b6HYABVl7m9kVEqvX3xBhzCFgLTC4TS0PgTqBaLjSq0xWUtj8F0rB/dxoBU4AjVR2Xck+a4NXV+BT7D8Y5U4GZpRcQkQYiMlNEjonIQRH547lEICKeIvKqo3SRDNzlZN0PReSwiBwSkb9e6gdORHxFJAfwBBJFZD+AMWa+MWYhkFWJ4+oBLDDGZBi7A8aY88clIq1EZL7jmLJE5C3HdA/H8R10lIJnikgDx7xzzQ/3i0gqsM4xvbeIfCsiJ0UkUURuKee41mG/0HjLUavQ9hLndpqIbBKRf4vICeD50tszxuQDsVz498Px+TNjTLGIBInIUsf2sx3vW5aKaYOIvCQim4CzQJvSVdeXOB+3iEh6mWM8ICK3O973dNS4nBaRIyLyr3L+Vp9QJsED44EkY8w2x7b6iMgWRwl2i4j0KbXPhiLykYhkOI5xoWN6hcfuECEimx3bXeS4sLjksZUlIl+ISKZjO1+JSMdS8z4WkXdFZLmI5AJPO86HV6llRotIQjnnpwfwsTEm1xhTbIyJN8ascKx3vklMRG50fK/OvfJF5IBjOQ8ReUZE9ju+77GljtVPRGY5pp90nN9m5cSiqpkmeHU1vgfqi0gHR+IdB5RtW3wTaAC0wV5anALc55j3ADAU6Ia9xDymzLqfAMXAdY5lBgEVtnsaYwqMMf6Oj12NMRdV31byuJ4WkUdFpLPIz9XYjuNcChwEwoAWwBzH7GmO1wDsx+sPvFVm2/2BDsBgEWkBLMNes9AQ+DUwT0SaODmuW4Gv+bnZYQ8Vn1uAXkAy0BR4yclxfgKMEZE6jmNrANzNzxdpHsBH2Et/oUCek+OZDDyIvabjYJl5lTkf5XkdeN0YUx+IwH4x4swCoLGI9CsT00zHMTXEfo7fwF6C/RewTEQaOZb9FKgLdMR+ns41xVTm2KcA04Hm2L+nV9pssgKIdOz/J+CzMvMnYv/7BWD/m2cBA0vNv9dxHM58D7wtIuNFJLS8AIwx3zm+V/5AkGO92Y7ZTwAjsH/HmgPZwNuOeVOxfwdbYT+/D2M/V8odGGP0pa/LfgEHgNuBPwJ/x161uxrwwl7FG4a9FF0AXF9qvYewtyODvRT7cKl5gxzregHNHOvWKTV/ArDe8X4a8E0F8RngOifT/4q9RFPRsXkCjwGbHDFkAFMd824EjgFeTtZbCzxa6nM7oMhxPGGOmNqUmv874NMy21h1bl9Otr8B+EWpGCs6t9OA1Er8HfcCEx3vHwASK1g2CsguE8+LFcRY0fm4BUh39p1yvP8KeAFoXIlj+AB4z/E+EigEmjo+TwY2l1n+O8f5CQFsQFAl9uHs2F8u9fl6x349K3FszwOzytlPoON70sDx+WNgZpllfoe9lgXsF4ZngZBythcEvIy9aacESAB6OOad+056lVnnXewXRR6OzzuB20rNDyn1d5wOfAt0udQ51Ff1v7QEr67Wp9hLGNMoUz0PNAZ8uLBkdxB7qRfspYG0MvPOaQ14A4cdVX8ngf9hL+W4lDGmxBjztjGmL/Yf3JeAGSLSAXtJ5aAxptjJqs25+FjPXaycU/p4WwP3nDs+xzH2w/4DeimXOrdl91WemfxcTT+ZUu3WIlJXRP7nqGI/jT3pBsqFzSQV7aMy56M89wNtgV2Oat+KOv19AowVET/HMaw0xhwtJ4ZzcbTA/rc8YYzJLrvBKzj2g9i/rxd0Or0UsTdTveyo/j6N/UKAMtspe45nAXeLiD8wFvjaGHPY2faNMdnGmGeMMR2xn/cEYGHpWqky8TyE/QJlojHG5pjcGlhQ6ju6E/vFQjPs//9XAXMczRyviIj35ZwD5Tqa4NVVMcYcxN7Z7k5gfpnZx7Ff6bcuNS0UOOR4fxj7j2zpeeekYS+hNjbGBDpe9R0/VNXGGJNnjHkbe7Xk9Y64QsV5J7kMLj7WYi7s1FT68Y1p2EvwgaVe9YwxL1citEud27L7Ks9M4DYRuRHoDXxeat7/YS919zL2qvKbHdNLJ4eK9lHR+cjFXjVu36A9cZ5vmjDG7DXGTMB+QfcPYK6I1HO2E2PM19irrYdjr64ufaFZNoZzcRzCfv4bikigk81W5tjLfneLsP9dKjy2MiY64r4de1V3mJP9XHCOjb1z4XfASOwXNOVVz1/AGHMc+90lzbGX/C8gIjcBfwGGG2NOlZqVBgwp8z31M8YcMsYUGWNeMMZcD/TB3uRWtl+HsogmeFUV7gduNcbklp5o7LffxAIviUiAiLQGnubndvpY4Amx38oTBDxTat3DwJfAayJS39HRJ0JEyvb6rhRHRyI/7FWono7OQU57sovIk46OUnUc603F3v4ZD2zGfmHysojUc2ynr2PV2cBTIhLuKF39DYgpp7QPP5fEBjtKcn6O/ZbtzHWRSpzbSnFcoH3jiH21MSaz1OwA7O2pJx1t2c9dzrap+HzsAfxE5C5Hie+PgO+5FUXkXhFp4ihFnnRMruh2rpnYLwQCgSWlpi8H2orIRMffchz2C7Wlju/YCuAdsXeq8xaRc4m8Msd+r4hcLyJ1gReBuY6/S4XHVkYA9gvZLOwXBX+r4BjLHu9vgc7Y+yE4JSL/EJFOjmMPAB4B9hljssos1wqIAaYYe/+O0v6L/XvW2rFsExEZ7ng/wNFPxRM4jf0iR2+7cxOa4NVVM8bsN8bElTP7l9hLNMnYE8nn/Hyr2vvYq/cSsXcuKlsDMAV7NfQO7CXouVSu+tqZP2L/wX4GeykvzzHNmTzgNSATe4nsMWC0MSbZ8QN+N/aOf6lAOvbOhTiO61Ps1bkpQD7243fKGJOGvfT2e+zt+mnAb6j8/8uKzu3l+AR7KbdsE8t/sN/2eBx7p6vLunWOCs6Ho4T4KPb280OO4yjd8/wOIEnsd0S8Dow39p7/5ZmJvRQdY4wpODfRkciGYi+RZ2FPikMdpVmwl4CLgF3AUeDJyzj2T7G3kWcCftg7o1Xm2MrGfdCx3A7HvipjAY6q87IX1mXUdSx7Evv3pDUwzMlytwHB2GtKzvWkP3dL5uvAYuBLETnjiLGXY14w9v+Xp7FX3W/kMi8yleuIMZWpxVNKKeVOxH4L6EPGmDVWx6Lck5bglVKqhhGR0djb5tdZHYtyX1cympZSSimLiMgG7P0IJpfq6a7URbSKXimllKqFtIpeKaWUqoU0wSullFK1UK1pg2/cuLEJCwuzOgyllFKq2vz444/HjTFOB1KqNQk+LCyMuLjybsVWSimlah8RKTsU83laRa+UUkrVQprglVJKqVpIE7xSSilVC9WaNnhnioqKSE9PJz+/oiGs3ZOfnx8tW7bE21ufvKiUUury1eoEn56eTkBAAGFhYZTz+GO3ZIwhKyuL9PR0wsPDrQ5HKaVUDVSrq+jz8/Np1KhRjUruACJCo0aNamTNg1JKKffg0gQvIneIyG4R2ScizziZ/28RSXC89ojIyVLzporIXsdr6lXEcKWrWqqmxq2UUso9uCzBi4gn8DYwBPuDESaIyPWllzHGPGWMiTLGRAFv4ngeuIg0BJ7D/szhnsBzIhLkqlhdyd/f/6JpX331Fd27d8fLy4u5c+daEJVSSqmqtumVTaSsT7lgWsr6FDa9ssmSeFxZgu8J7DPGJBtjCoE5wPAKlp8AzHa8HwysNsacMMZkA6uBO1wYa7UKDQ3l448/ZuLEiVaHopRSqoo079GcuWPnnk/yKetTmDt2Ls17NLckHld2smsBpJX6nI69RH4REWkNhPPzs42drdvCBTGet+mVTTTv0ZzwAT93aktZn0LGlgz6/rZvle7r3JC6Hh61uguEUkpdU8IHhDMmdgyxo2Lpdn83Ej9JZEzsmAvySnVyZYZx1ohc3rNpxwNzjTEll7OuiDwoInEiEnfs2LErDNPO3a68lFJK1TxNOjQh/3Q+3732Hd0f7G5ZcgfXluDTgValPrcEMspZdjzwWJl1bymz7oayKxlj3gPeA4iOjq7wwfYrn1xJZkJmhQEHNA9g1uBZBIQEcObwGZp0aMLGFzay8YWNTpcPjgrmjv/UmpYDpZS6YtVZC+rO1jy7Bmz29z/85wfa3N6mVpbgtwCRIhIuIj7Yk/jisguJSDsgCPiu1ORVwCARCXJ0rhvkmOZSfkF+BIQEcCr1FAEhAfgF+bl6l0opVSucqwVNXpNMYU7hNVkLmrwmma0ztxJyQwhd7u1CUX4RsaNiL+p4V11cVoI3xhSLyOPYE7MnMMMYkyQiLwJxxphzyX4CMMcYY0qte0JE/oL9IgHgRWPMiauJpzIl7XNfyJv/dDNx78bR/7n+llavKKXU1aqqknXR2SJyMnM4c/gMOYdzfn6fmWP/fDgHW4mNTwd+CoCHtwc9Hu9Bsy7NqvyY3FXCJwkYm+HmP91MWP8wUtanIJ5C2rdpluQSl45kZ4xZDiwvM+3PZT4/X866M4AZLguujHPJ/VyHiLABYRd8vlJnz56lZcuW5z8//fTT3HTTTYwcOZLs7GyWLFnCc889R1JSUhUchVJKXehcyfrcb1np3zpjDHlZeRcmakfizs3MvSCZF5wuuGjb4in4N/PHP8SfgBYBhESHkLU7i9SvU/Gt78sP//6BLW9uIWJwBJ0mdKL98Pb4+PtYcBaqx5lDZ2gQ2oC2Q9vi4enB8BnDmTV4FnlZeZbEU6uHqr0cGVsyLkjm53pDZmzJuKoEb7PZnE5PT0+/4m0qpVRlne/ZPTqWoIggjm49SmB4IAunLCTnSA62oot/o7zreRMQEoB/iD/NujYj4o4I/IP9z087975u47qIx899osvWgg59bygn9p1g++zt7F22F686XrQb1o7OEztz3R3X4enj6dJjv9LaiytZ79iOYxxYf4Db/n4bHp721u+IQRFEPxrN9//5nnbD2xHWP6xqDqySNME7OPujhQ8I1yp6pVSNZiuxkbElg4LTBRyOO0y9pvUIbB1oT9SlknXp91dSyq6oFvT2v99O2rdpbPt8Gzu+2EFSTBJ+QX50GN2BzhM70/rm1ueTYlWqqPaiqtfb8s4WPH086XZ/twumD3xlIEkxScwdN5df7v0lvgG+QPV0QJRSTd81WnR0tImLi7tg2s6dO+nQoYNFEV29mh6/UjXFtx/24czRrRdND2jahT73f2tBRFXj+K7jLLpvEenfp+Pp40mPx3qw9dOtLrk3u7Kl3pKiEpLXJLN99nZ2LdhFYU4h/iH+dBzXkc4TO9M8unmVDdVdUlTC1k+3svLJlYR0DyFjSwadJnSiUdtGiKfg4emBh5fH+fel/83ancXmNzcT/Ug08R/GV3jOCk4X8K8W/6L9yPaMnDnyovmb39zMiidWcN2d1zFp2aSLLoauhoj8aIyJdjZPS/BKqWteYIue5GTtwpQUnp8mnj4EtnQ6Npfbs5XY+P4/37PuD+vw9PHEJ8CHcQvH0ebWNrS9u22VJZfSKlsL6untSeSQSCKHRFJ0tog9y/aw/fPtxL0Txw//+YGG1zWk04ROdJrQiSYdmlT6wqEwp5AjW49wOP4wmQmZZMZncnT7UUoK7MOrHNx4EID4D+Mv67g2/WMTHcd2rPBcJX6aSGFOIT0e6+F0fs9f9iT121SS5iQxd8JcUtakVMsAOFqCd2M1PX6laoqCnEy+eqcjtpKfO5KJhxfdxsTSuM3tiNScUSez9mSx6L5FpH2bRrvh7WjauSnht4ZfdS96V8s/mc/O+TvZPns7KetSMDZDcFQwzXs2Z+fcndwz957z1eVfjPmCPr/tg7EZjiTYk/qJfSfOD4dWp2EdgrsFE9wtGE8fT+LejSP6kWh+eu8nRs4aSWi/UEyJwVZiO/+vrdh2wbTUTams/NVKPH08OXvsLK36tGL07NE0CG1wQdzGGN7p+A7edb15YMsD5dY+FBcU89HNH5GxOYOb/3QzA14cUCXnraISvCZ4N1bT41eqpjiyezEJC6eArdgxRTiXLXzqNqZxm4E0bjOQRuG34VO3kWVxVsRWYuOHN35g3e/X4VXHiyFvDqHzxM418smUOZk5JMUmse3zbRz64RAAHl4eNGjdgOzk7AvGNQ0MDyQ4yp7Mg6Psr/ot6yMiF1WFV7ZqfMPrN1BwdvdF03MPNyW49Sx6PdELT297B8GU9SnMvHUmw2YMo9t93S5a55xz+45+JJq4d+OqrASvVfRKKeWEsZWw7+u/kPztqwQ07UJu1m5sJQV4ePnSe9pXnDmyjeP7V3Fs/yoyts8G8SCweQ8aRwyicZtB1A/u6hal+xP7TrDovkWkfpNK27vbMvR/QwkICbA6rCvmH+xPryd60euJXpzYf4Ltc7az+c3NZO/PplmXZkRNj7In867B+AWWPyDZld4dVXI2EiQZTFGpqd4UnWrD6l+v5rtXv+OmP95Ez8d6suXtLdRpWAf/EH82vbLJaa2Iq27DvhQtwbuYv78/OTk5F0z717/+xQcffICXlxdNmjRhxowZtG7d+qJ13SF+pWqrwrwTbFt8P8eTV9Oi61Q6DHqN3WueIS1hBq263c/1g/99flljK+HU4R85vv9Lju3/ktOZPwHgU6/p+dJ94/Db8K5TvU+1NjbD5rc2s+aZNXj5enHH63fQZXKXGllqr4irSr/lKcjJZOM711/QJ8PDy4+bHt7OgbUnWfLAEs4eP0vEoAiS1ybTYXQHDqw7UG5crhzGV6voLeQswa9fv55evXpRt25d3n33XTZs2EBMTMxF67pD/ErVRqePbCNh/kTyT6fTYdCrtIyajohQkJNJ4sKpdB0xE1//8kdgK8g9wvHktRzf/yXHU9ZSnJ9tL9236EUTR+k+oJlrE+2J/SdYPH0xB786SOSdkQx9byj1W9R32f6scqXV7Fcjc9cCEhdOg1LPPwvuMIquIz4B7B36Ft63kJ1zdwLg19CPsXPHWnJbtSb4SnDVbTLOEnxp8fHxPP7442zatOmieZrglap6h5Ni2b78Mbz9AokaOeuqe8rbbMWcyoizJ/vkLzmdmQCAr3+wvWQfMYhGYbfi7dfgEluqHGMzbHlnC2t+twYPLw/ueP0Ouk7tWutK7edU50NsjDGkfPcaezc+T/3gbuQc2/Fzx0vx4rqbniW899N4eHoDsPShpfz43o9V2mnucmkbfCVYdZvMhx9+yJAhQ1y6D6WUPRHvWf8nDm5+k8CWNxI1claFpfTK8vDwIqhlb4Ja9iay/58pyDnC8eTVHE/+kiO7F3No66eIeBLYsjeNIwbRJGIw/k06XlFCzk7JZvH0xRzYcIDr7riOu9+/m/ota1+pvbTqGoTMVlxA0sonyNj2GSHXj6XjXe+cb7Jp3nkitqI89n31F47sXkSnO98la2d9ds7feX7UvrABYW43MNo1U4Lfufq3Tkvo59hKCjl1KI7zz/kDEA8aNI/Gw9P5qE4BTbvQYeArFcZVUQl+1qxZvPXWW2zcuBFfX9+L5msJXqmqUXj2GIkLp3Li4FeE3vAQ7W77e7n/r6uSzVbMqUObOeYo3Z85Yv8N8g1oTuM2g2gSMZBGYQPw8q04SRubIe5/caz+zWrEQxj878F0m96t1pbaq1vh2eMkzJ9Idtq3RNz0ByL6PuO0yebI7sXsWPUkhblHnW7Ht247bvnVj9Uau5bgK8HD0wcf/6YU5hzBfg+G4FOvqct+BNasWcNLL71UbnJXSlWNUxk/krBgEoVnj9Np6Hu06Dyx2vbt4eFFUKs+BLXqQ9tbnif/zOGfS/e75nMo8WPEw4vAljfa2+4jBuHf+PoLEvfJAydZfP9iUtal0GZgG4Z9MOyie7HdQU0dDTAnazc/xd5DwZlDdBn+ESHX33N+nq9/MD3v/flJ5c3aDSMotB8b/tMHQ5nniYi3vfe9G7lmEvylStrgGOzi3U7YivPx8PKlz32bqqQKr6z4+HgeeughVq5cSdOmTat8+0opu/TEmexc9RQ+9ZrSa/Ia6gdHWRqPX0AILbtOoWXXKdhKijh56Ad7z/zkL9mz/k/sWf8n/Oq3PN8zP3V9IGt+Y++fM/R/Q+n+QHe3LbXXxNEAs1LWk7DgXjw8fegxaQWBLXpech2fOg3p/8sNbHz7eoyt1LEK+ATtYdXf/S9ax6qLnGsmwVeGr38wLTrfS1rCDFp0mVwlyd3Z42KXL19OTk4O99xjv1IMDQ1l8eLFV70vpZSdraSQXat/Q1r8hzQMu4Wuwz/Gp25jq8O6gIenNw1D+9EwtB9tB7xI/ulDHE9ezbH9qzi8/QvSEz7CVuxBx4cjibztHlpF17U65ApF9HuGQ9tmlR6DBhEPIvo+Y1lMFUmL/5Cdq56mXuN2dL9nLnUahFZ6XV//YFp2nUL61pmOCxrB2Io4e2IvpQdJAmsvcjTBlxHR7xlyju+ssi+ls8fFPv3001WybaXUxfLPHCZxwWROHvqesF5PEnnL83h4uP9PnV/9FrToOpWjcV2Ie6kjdZun0fUXJXiFbCU1/q+kxv8Vv/qtzlflN2zdHy+fi0uL1aW4MJczR7dxOjOBM0cSOZ2ZgK34wmfG1w1sQ1F+tktqQq+UsZWwe90fOLjlLRq3GUTXER9fsg+EM6UvaDy8fOl+zzwO75jLocSPLljOyosc9//WV7OybS5KqZojO/07EubfS0lhDl1HfEJwh9FWh1Rpp9NPs/gXi9m/aj9hA8IY9uFjBIXbB87JO5Vmb7vf/yUZ2+eQFv8h4ulDw1Z9aRwxmMYRA6nXsK3Lqu+L8k/Zk7gjkZ/OTCA3aw+lh/OtH9yNoNC+pP30IcZWBOJJzom9bHo/moatb6ZV9wdoGjn0/C1m1aG8fgEAodGP2DtbXuHFX9ka30Zh/WkU1h8RIT3xk/P30NdtGEncnOHkHNt+0TZcXXWvCV4pVeMZY0j76X12rfktdRq0JnrCYgKadLQ6rEoxxpDwUQKrnlqFrdjGnW/fSfTD0YjHz8m6ToNWtOo2nVbdpmMrLiA7/VuO77dX5+9e+wy71z5DncAwGrcZSJOIwTRsfTOe3vYq/cvt/FZ49jinj2x1JPJ4Tmcmkncy+fx834Dm1A/uRnCH0dQPjqJ+cBS+/iHnLy5MSbFjNMDpXHfT70lPnEla/IckLphsr9qOuo+WUdPxCwip6lN5EWf9AgACW/alw8B/XvX2ndX4XnfT78nY/jm24hL7Rc7RbXj6+CMeXpjzzzqonqr7a+Y2uZqopsevVHUoKcpjx6qnyNg2i8YRg+ky7EO8/QKtDqtSTh86zZIHlrBvxT5a92/N8BnDCWpzecPd5p08yLHkLzm+fzUnDm6gpOgsHp6+BIXeRJOIgZzMiOPI7kUXdX5r2XUqEX1/d75EfvpIAqczE8k/nXZ+uTqBYdRv1pX6wVEENIuifnAXfOtVXN3ubDRAYyvh2P4vSfvpfY4nr0bEg6bt7qZV9wdoGHqzy2oenD4l0NOH/o/udGmzwY6VT54f8ji4w2i2L3uEvJMpFyzj4eXHzY8kXXUc1/RIdu3bt3fbXqcVMcawa9cuTfBKVSDvVCoJ8ydxOjOeiL7PEHHT793i4S+XYowhcWYiK3+1kpLCEm7/x+30fKznBaX2K1FSnE922rf2UfX2ryL3xF7HnAs7fiEeePs1pCjv+Pn59RpeR4CjRF6/WVfqN+vqkrH1z2Ynkxb/IYcSP6Uo/wT1GrWjVfcHaN5pQpWN9ld4Nouje5eSuXM+WSlrz08XD29aRk274DkDrlD2IqekOJ8fZt7OmSP2UQ7PXWBVRRzXbIJPSUkhICCARo0a1agkb4whKyuLM2fOEB7uXiMjKeUusg5sIHHhVIytiM53v0/TyLusDqlSzmScYcmDS9i7bC+h/UIZ/tFwGl7X0CX7OpudwvHkL0n5/j8XlMy9/IJoGnnnz6Xzpp3x8q3ep8+VFOWRuXM+aT+9z6nDcXh61yOk0zhCuz9IQNNOl7290kn9xIENGFNCncA2NI4YSHrCR5iSwiorNV+J0rUJVRnHNTvQTcuWLUlPT+fYsWNWh3LZ/Pz8Lri9TillZ4zh4OY32b3+j9RrGEm30bOp16it1WFdkjGGrbO2svKJlRQXFDP434Pp9USvqy61V6RuUDihNzxEs3bDS43x4Ue/B+Is79nu6V2HFl0m0aLLJE4d/om0n94nY9vnpMfPILDljYR2f4Bm7Ybz/ScDyu1DED1hidOkHtb7SYLbjySgmWOMfltJld7+fCV8/YNp0WVytcZRq0vwSqnapbgwl6Tlj5G5cy7N2g2n013/rfaS55XIycxh6UNL2b14N636tmL4R8NpFNmoWmMo3S7s6irqK1WYd4KMbZ+R9tMHnM3ej0/dJvgGNCfn2A57z/xzxBNf/2AKczLPJ/XgDiMvTOqlVPYpga7mijiu2Sp6pVTtcTY7mfh5E7DaluMAACAASURBVMg5toPIW54nvPfTbt/0Zoxh++ztrPjlCorOFnHrS7fS61e98PCs/n4C7pLkKsMYG1kp60j96X2O7V3OBf0HHPzqtyKk49hyk/q1QhO8UqpGO7b/S7Yung4IXYd/TOM2t1kd0iXlHMlh2SPL2LVgFy17t2T4x8Np3M69RtOrCfJOpRI/bwJnjiTaJ4gnzdoOo+vImddsUi/tmm2DV0rVbMbYSN70Cvu+fomApp2IGj2buoFhVodVIWMMSbFJLH9sOYU5hQz850B6P9XbklJ7bVCnQSg3jJ33cx8CT286DHpVk3slaIJXSrml4oLTbFvyIEf3LiWk4zg6Dnnz/OAt7ir3aC7LHl3Gznk7adGzBcM/Hk6TDk2sDqvGc8VzQq4FmuCVUm4n5/guEuZN4Gx2Mu1v/weh0Y+6fYkt6Ysklj+6nILTBdz28m30+b8+eHhpqb2qVPVzQq4FmuCVUm7lyO7FbFv6IJ5efkRPXErD0JusDqlCucdyWf7YcnZ8sYPm0c0Z/vFwmnbUx0BXNX1OyOXTBK+UcgvGVsK+r/9C8rev0iAkmqhRn+FXv4XVYVVox7wdLHtkGfkn87n1pVvp+9u+WmpXbkMTvFLKcoV5J9i6aDpZKWto0XUaHQa9iqeXn9Vhlets1llWPL6C7XO2E9I9hClrp9Css7YLK/eiCV4pZanTR7aRMH8C+acPcf0db9Cq23SrQ6rQroW7WPrwUvJO5DHgLwPo+7u+eHp7Wh2WUhfRBK+UsszhpFi2L38Mb79Aet67isAWPa0OqVx5J/JY8cQKtn22jeCoYCZ/OZlmXbTUrtyXJnilVLWz2YrZs+6PHNzyFkGt+tB1xKdufevT7sW7WfrQUs4eP8stL9xCv2f7aalduT1N8EqpalWQe5TEhVPJTv2a0Bsept1tf8fD09vqsJzKy85j5a9WsvXTrTTr0oxJKyYRHBVsdVhKVYomeKVUtTmV8SMJ8ydSmJdFp6Hv0aLzRKtDAmDTK5to3qM54QN+fjzz13//mm/+/g1FZ4u4+c83c/MfbsbTR0vtqubQBK+UqhbpiTPZueopfPyb0WvyGuoHR1kd0nnNezRn7ti5jIkdQ0i3EOaOm8v+L/cTGB7I2LljCekeYnWISl02lyZ4EbkDeB3wBD4wxrzsZJmxwPPYHxeUaIyZ6JheAmxzLJZqjBnmyliVUq5hKylk1+rfkBb/IY3CBtBl+Ef41HWvh66EDwhnTOwYYkbGYGyGwjOFdJ7UmWEfDsPLV8tBqmZy2TdXRDyBt4GBQDqwRUQWG2N2lFomEngW6GuMyRaR0sM/5Rlj3OcSXyl12fLPHCZxwb2cPPQDYb2fIrL/c3h4uF/CNMaQsSWDglMFAHS7vxvDPtAyharZXPk/rSewzxiTDCAic4DhwI5SyzwAvG2MyQYwxhx1YTxKqWqUnfYtCQsmU1KYQ9cRnxDcYbTVITlVmFvI4vsXkxSThKevJ72f7E38h/GkTEq5oE1eqZrGlWMqtgDSSn1Od0wrrS3QVkQ2icj3jir9c/xEJM4xfYSzHYjIg45l4o4dO1a10SulrogxhtQf/8eWz+/Ey8efXlPXuW1yz07JZkafGSTFJOFdz5uJyydy+8u3MyZ2DHPHziVlfYrVISp1xVxZgnf26CfjZP+RwC1AS+BrEelkjDkJhBpjMkSkDbBORLYZY/ZfsDFj3gPeA4iOji67baVUNSspymPHqifJ2PYZTSLuoPOwD/D2C7Q6LKeS1yYzd+xcjM0QNT2KLvd2OV9iP9cmn7ElQ0vxqsZyZYJPB1qV+twSyHCyzPfGmCIgRUR2Y0/4W4wxGQDGmGQR2QB0A/ajlHJLeadSSZg/idOZ8UT0e5aIfs8i4n4PXjHG8P2/v2f1b1bTuENjxi8cT8PrGl60XPiAcE3uqkZzZYLfAkSKSDhwCBgPlL3pdSEwAfhYRBpjr7JPFpEg4KwxpsAxvS/wigtjVUpdhawDG0hcOBVjK6LbmFiaRt5pdUhOFZ0tYsmDS9j22TY6jOrA8I+H4xvga3VYSrmEyxK8MaZYRB4HVmG/TW6GMSZJRF4E4owxix3zBonIDqAE+I0xJktE+gD/ExEb9n4CL5fufa+Ucg/GGA5sfoM96/9EvYaRdBs9h3qNIq0Oy6mTB08SMzKGzIRMBvx1ADf9/iZEnLUkKlU7iDG1o+k6OjraxMXFWR2GUteM4sJckpY/SubOeTRrN5xOd/0XL98Aq8NyKmV9CnPHzqWkqIRRn42i7V1trQ5JqSohIj8aY6KdzXO/G1KVUm4v98R+EuZPIOf4LiJveYHw3k+7ZWnYGMMPb/zAl//3JY3aNmL8wvE0atvI6rCUqhaa4JVSl+XYvlVsXXI/ggc3jF1A4za3WR2SU0V5RSx7eBmJMxNpN7wdI2eOxLe+trera4cmeKVUpRhjI3nTK+z7+iUCmnUmatTn1A0Mszosp06lnSJ2VCwZcRn0f74//f/UH/FwvxoGpVxJE7xS6pKK8k+xfelDHN27lJCO4+g45E08vetaHZZTB786yBf3fEFRXhHjF42n3bB2VoeklCU0wSulKpRzfBfx8yaQl51M+9v/QWj0o27b3r7lnS2senIVQW2CmLZxGo3bu9dDbZSqTprglVLlOrJ7EduWPoSnVx2iJy6lYehNVofkVHF+McseW0bCjATaDm3LyFkj8WvgZ3VYSllKE7xS6iLGVsLer/5Cynev0iAkmqhRn+FXv+yjJNzD6UOniR0Vy6HNh7j5Tzdzy/O3aHu7UmiCV0qVUZh3gq2LppOVsoYWXadx/aDX8PByz97nqZtSiR0dS1FuEWPnj6XDyA5Wh6SU29AEr5Q67/SRbSTMn0D+mQyuH/ImraLuszqkcsX9L44Vv1xBYOtApqydQtOOTa0OSSm3ogleKQVARlIsScsfw9svkJ6TVhLYoqfVITlVXFDMiidW8NN7P3HdkOsY9dko6gTVsTospdyOJnilrnG2kiL2rP8jB7e8TVCrPnQd8Sm+/s2sDsupM4fPEDs6lvTv0un3bD8G/GUAHp7u98Q6pdyBJnilrmEFuUdJXDiF7NRvCL3hYdrd9nc8PL2tDsup9O/TiRkVQ8GpAsbEjqHjPR2tDkkpt6YJXqlr1KmMH0mYP5HCvCw6D32f5p0nWB1SuX764CeWP7ac+i3rc++qe2nW2T1rGJRyJ5rglboGpSfOZOeqp/Dxb0avyWuoHxxldUhOlRSWsPLJlcS9G0fEoAhGzx5NnYba3q5UZWiCV+oaYisuYOea35AeP4NGYQPoMvwjfOq652hvOUdy+GLMF6R+k0qf3/bhtr/dpu3tSl0GTfBKXSPyzxwmYf4kTmVsJqz3U0T2fw4PD/f8CTi05RAxI2PIO5HH6Nmj6TS+k9UhKVXjuOf/bqVUlcpO+5aEBfdSUphL1xEzCe4wyuqQypXwcQJLH15KQEgA9397P8FRwVaHpFSNpAleqVrMGEPaT++xa83vqNOgNdETlhDQxD17n5cUlfDl/33J5jc3E35rOGNixlC3sXs+sU6pmkATvFK1VElRHjtWPUnGts9oct0QOt/9Pt5+gVaH5VTu0Vy+GPsFBzcepPfTvRn4j4F4eGl7u1JXQxO8UrVQ3qlUEuZP5HRmAhH9niWi37OIuGfCzPgxg5iRMZw9dpaRs0bSZVIXq0NSqlbQBK9ULZN1YD2JC6dibMV0GxNL08g7rQ6pXFtnbWXJA0uo26Qu0zdNJ6R7iNUhKVVraIJXqpYwxnDgh9fZs+HP1GvUlm6jZlOvUaTVYTllK7ax+rer+f7f39O6f2vuib2Hek3rWR2WUrWKJnilaoHiwlySlj9C5s75NGs/gk53vouXb4DVYTl19vhZ5o6bS8q6FHo+0ZNBrw7C09vT6rCUqnU0wStVw+We2E/C/AnkHN9F5C0vEt77KUTE6rCcykzIZM6IOeRk5jD84+FETXXPEfSUqg00wStVgx3bt4qtS+5H8OCGcQtpHH6r1SGVa9vsbSy+fzF1G9Xlvq/vo0WPFlaHpFStpgleqRrIGBvJm15h39cvEdCsM1GjPqduYJjVYTllK7ax5tk1fPfqd4TeFMo9X9yDfzN/q8NSqtbTBK9UDVOUf4ptSx/k2N5lhHQcT8chb+Dp7Z4DwuSdyGPu+Lkkr06mx2M9GPyvwXj6aHu7UtVBE7xSNUjO8V3Ez5tAXnYy7W9/hdDoR9y2vf3I1iPMGTGHM4fOcPcHd9P9/u5Wh6TUNUUTvFI1xJHdi9i29CE8veoQPXEZDUP7WR1SuZK+SGLRtEX4BfoxbeM0WvZuaXVISl1zNMEr5eaMrYS9X/2FlO9epUFINFGjPsOvvnt2ULOV2Fj3x3VsenkTrfq04p659xAQ4p636ylV22mCV8qNFeadYOui6WSlrKFF12lcP+g1PLx8rQ7LqbzsPOZPnM++lfu44aEbGPLGEG1vV8pCmuCVclOnj2wjYf4E8s9kcP2QN2kVdZ/VIZXraNJR5gyfw6nUUwz931BuePAGq0NS6pqnCV4pN5SRFEvS8sfwrhNEz0krCWzR0+qQyrVz/k4WTFmAb4Av0zZMo1WfVlaHpJRCE7xSbsVWUsSe9X/k4Ja3CWrVh64jP8W3XjOrw3LK2Azrn1vP13/9mha9WjBu/jgCmmt7u1LuQhO8Um6iIPcoiQunkJ36DaHRj9Du1r/h4eltdVhO5Z/KZ8G9C9izdA9R06O465278PLVnxOl3In+j1TKDZzMiCNx/iQK87LofPcHNO803uqQynVs5zFiRsSQnZzNnW/fSfQj0W57L75S1zIPV25cRO4Qkd0isk9EnilnmbEiskNEkkTk81LTp4rIXsdrqivjVMpK6YmfsHnWIPDwpNfkNW6d3Hcv3s0HvT4g/2Q+U9ZNocejPTS5K+WmXFaCFxFP4G1gIJAObBGRxcaYHaWWiQSeBfoaY7JFpKljekPgOSAaMMCPjnWzXRWvUtXNVlzAzjW/IT1+Bo3CBtBl+Mf41G1kdVhOGZth44sb2fjCRppHN2fs/LE0aNXA6rCUUhVwZRV9T2CfMSYZQETmAMOBHaWWeQB4+1ziNsYcdUwfDKw2xpxwrLsauAOY7cJ4lao2+WcOkzB/EqcyNhPe+2ki+z+HeLjnPeMFpwtYMHkBuxfvpuvUrgz971C8/LR1Tyl358r/pS2AtFKf04FeZZZpCyAimwBP4HljzMpy1nXPobuUukzZad+SsOBeSgpz6TryU4Lbj7Q6pHJl7clizvA5ZO3N4o437qDn4z21Sl6pGsKVCd7Zr4Bxsv9I4BagJfC1iHSq5LqIyIPAgwChoaFXE6tSLmeMIfXH/7F77TPUaRBGjwlL8W9yvdVhlWvPsj3MnzgfTx9PpqyZQtgtYVaHpJS6DK7sZJcOlB7xoiWQ4WSZRcaYImNMCrAbe8KvzLoYY94zxkQbY6KbNGlSpcErVZVKivLYvvQhdq3+NY3bDKT3tA1um9yNzfDVX79i9t2zCYoI4oG4BzS5K1UDubIEvwWIFJFw4BAwHphYZpmFwATgYxFpjL3KPhnYD/xNRIIcyw3C3hlPqRon71QqCfMncjozgYh+zxLR71lEXHoDyxUrOFPAommL2Dl/J13u7cLQ94biXcc978VXSlXMZQneGFMsIo8Dq7C3r88wxiSJyItAnDFmsWPeIBHZAZQAvzHGZAGIyF+wXyQAvHiuw51SNUnWgfUkLpyKsRXTbcwXNI0cYnVI5Tqx7wRzhs/h+O7jDPrXIHo/2Vvb25WqwcSYi5q2a6To6GgTFxdndRhKAfb29gM/vM6eDX+mXqO2dBs9h3oNr7M6rHLtW7mPeRPmIZ7CmJgxtLmtjdUhKaUqQUR+NMZEO5un97ooVcWKC3NIWv4omTvn06z9CDrd9V+8fPytDsspYwyb/rGJtb9fS7MuzRi/cDyBYYFWh6WUqgKa4JWqQrkn9pMwfwI5x3cRecuLhPd+ym2ruQtzC1k8fTFJsUl0Gt+JYR8Ow7uutrcrVVtogleqihzbt5Kti+9HxJMbxi2kcfitVodUruzkbOaMmMOxpGPc/srt9Pl1H7e9EFFKXRlN8EpdJWNs7N/0D/Z//TcCmnUmatTn1A0Mszqscu1fvZ+54+YCMGnFJCIGRVgckVLKFTTBK3UVivJPsW3pgxzbu4yQjuPpOOQNPL3rWh2WU8YYvnvtO9b8bg1NOjZh3IJxNIxoaHVYSikX0QSv1BXKOb6T+HkTyctOpv3AfxJ6w8NuW81ddLaIxb9YzPbZ27l+zPUM/2g4Pv4+VoellHKhSiV4EWkNRBpj1ohIHcDLGHPGtaEp5b4ydy1k+7KH8fSqQ/TEZTQM7Wd1SOU6eeAkMSNjyEzM5La/30bf3/V12wsRpVTVuWSCF5EHsI/33hCIwD5s7H+B21wbmlLux9hK2PvVi6R89xoNmvcgauQs/Oq773OQUtal8MXYL7AV25i4bCKRQyKtDkkpVU0qU4J/DPujX38AMMbsPffcdqWuJYV5J9i66D6yUtbSMmoaHQa+hoeXr9VhOWWM4YfXf+DLX39J43aNGbdwHI0i3fNZ80op16hMgi8wxhSeq9ITES+cPNlNqdrs9JFtJMyfQP6ZDK4f8iatou6zOqRyFeUVsfTBpWydtZX2I9sz4pMR+Aa454WIUsp1KpPgN4rI74E6IjIQeBRY4tqwlHIfGdtjSFrxON51gug5aRWBLXpYHVK5TqWeImZkDIfjDzPgLwO46fc3IR7a3q7UtagyCf4Z4H5gG/AQsBz4wJVBKeUObCVF7F73B1Lj3iGoVR+6jvwU33rNrA6rXAc2HuCLe76gpKCECYsn0HZoW6tDUkpZqMIELyKewCfGmHuB96snJKWsV5B7hMSFU8lO/YbQ6Edod+vf8PB0z2FcjTFsfmszq55aRaPIRoxbOI7G7RpbHZZSymIVJnhjTImINBERH2NMYXUFpZSVTmbEkTB/IkV5J+h89wc07zTe6pDKVZxfzLJHlpHwcQLthrVj5Kcj8a2v7e1KqcpV0R8ANonIYiD33ERjzL9cFZRSVklP/IQdq57Czz+EXpPXUj+4q9Uhlet0+mliRsWQsSWD/s/1p/+f+2t7u1LqvMok+AzHywMIcG04SlnDVlzAzjW/IT1+Bo3CbqXL8I/wqeu+t5WlfpNK7OhYis4WMW7BONqPaG91SEopN3PJBG+MeQFARALsH02Oy6NSqhrln8kgYf69nMrYTHjvp4ns/xzi4Wl1WE4ZY4j7bxwrn1hJYHggU9dPpcn1TawOSynlhiozkl0n4FPsI9khIseBKcaYJBfHppTLZadtImHBZEoKc+k68lOC24+0OqRyFRcUs/yx5cR/GE/kXZGMmjUKv0A/q8NSSrmpylTRvwc8bYxZDyAit2DvUd/HhXEp5VLGGFJ//C+71z5LnQZh9JiwFP8m11sdVrnOZJwhdnQs6d+nc9Mfb2LACwO0vV0pVaHKJPh655I7gDFmg4jUc2FMSrlUSVEeO1b+ioztn9PkuiF0vvt9vP0CrQ6rXGnfpRE7KpaCMwWMnTeWDqM6WB2SUqoGqEyCTxaRP2Gvpge4F0hxXUhKuU7eqVQS5k/kdGYCEf1+T0S/ZxDxsDqscv34/o8sf2w5DUIbMHn1ZJp20sdAKKUqpzIJfjrwAjDf8fkrwH0H4laqHFkp60lcNBVjK6bbmC9oGjnE6pDKVVJYwoonVvDj/34kYnAEo2ePpk5QHavDUkrVIJXpRZ8NPFENsSjlEsYYDvzwOns2/Jl6jdrSbfQc6jW8zuqwypWTmUPsmFjSNqXR95m+3PrXW/HwdN9aBqWUe6pML/rVwD3GmJOOz0HAHGPMYFcHp9TVKi7MYfuyRziyawHN2o+g013/xcvH3+qwypX+Qzqxo2LJP5nPmJgxdBzb0eqQlFI1VGWq6BufS+5gL9Hr8+BVTZB7Yj8J88aTk7WbtgP+QlivJzn32GN3FD8jnmWPLCOgRQD3f3c/zbq474NtlFLurzIJ3iYiocaYVAARaY0+D165uWP7VrJ18f2IeHLDuIU0Dr/V6pDKVVJUwqqnVrHl7S20GdiG0bNHU7dRXavDUkrVcJVJ8H8AvhGRjY7PNwMPui4kpa6cMTb2b/oH+7/+GwHNOtNt1GzqBLa2Oqxy5R7NJXZMLKlfp3Ljr2/k9r/fjoeXtrcrpa5eZTrZrRSR7kBvx6SnjDHHXRuWUpevKP8U25Y+yLG9ywjpOJ6OQ97A09t9S8IZcRnEjIzhbNZZRn0+is4TOlsdklKqFik3wTuq4k8aY04ZY46LSC4wAmgrIm/p42OVO8k5vpP4eRPJO5lC+4H/JPSGh926vT1xZiJLHlyCf7A/0zdNJ6RbiNUhKaVqmYrqAmOBegAiEgV8AaQCXYF3XB+aUpWTuWsh338ygOL8k0RPWErr6EfcNrmXFJWw8smVLJy6kFZ9WvFg3IOa3JVSLlFRFX0dY0yG4/29wAxjzGtiH/YrwfWhKVUxYyth71cvkvLdazRo3oOoUZ/hF9Dc6rDKlXssl7lj53JgwwF6PdmLQf8cpO3tSimXqSjBly4C3Qo8C2CMsblr6UhdOwrPZrF18XSyUtbSMuo+Ogx8FQ8vX6vDKtfhnw4TMzKG3KO5jJg5gq6Tu1odklKqlqsowa8TkVjgMBAErAMQkRBA29+VZU4f2UrCvAnk5xym45C3aBk1zeqQKrT1s60s+cUS6japy33f3EfzG9y3lkEpVXtUlOCfBMYBIUA/Y0yRY3ow9lvnlKp2GdtjSFrxON51gug5aRWBLXpYHVK5bMU21jyzhu9e+47WN7fmni/uoV5TfRCjUqp6lJvgjTEGmONkerxLI1LKCVtJEbvX/YHUuHcIatWXriNn4lvPfUd6O5t1lrnj5pKyNoUej/dg8L8G4+ntaXVYSqlrSGUGulHKUgW5R0hcOJXs1G8IjX6Udre+hIent9VhlSszMZOYETGcyTjDsBnD6HZfN6tDUkpdgzTBK7d2MiOOhPkTKcrLpvPdH9C803irQ6rQ9pjtLLpvEXUa1uG+r++jRc8WVoeklLpGufQeHRG5Q0R2i8g+EXnGyfxpInJMRBIcr1+UmldSavpiV8ap3FN6wsdsnjUIDw9vek1Z69bJ3VZiY/XvVjNv/Dya39CcB+Me1OSulLLUFZXgRWSFMWbIJZbxBN4GBgLpwBYRWWyM2VFm0RhjzONONpFnjIm6kvhUzWYrLmDn6l+TnvARjcJupcvwj/Cp28jqsMqVdyKPeRPmsf/L/UQ/Es0d/7kDTx9tb1dKWauioWq7lzcLqEzi7QnsM8YkO7Y3BxgOlE3wSp2XfyaDhPn3cipjM+E3/h+RN/8Z8XDfZHlk2xFiRsRwKu0Ud79/N91/Ud5/G6WUql4VleC3ABu5cMCbcwIrse0WQFqpz+lALyfLjRaRm4E92B9kc24dPxGJA4qBl40xCyuxT1WDZadtImHBZEoKc+k6chbB7UdYHVKFdszdwcJpC/Gt78u0jdNodWMrq0NSSqnzKkrwO4GHjDF7y84QkTQny1+0mJNpZZ8jvwSYbYwpEJGHgU+wj5oHEGqMyRCRNtgH3dlmjNlfJo4HcTy6NjQ0tBIhKXdkjCH1x/+ye+2z1GkQRo+Jy/Bv3MHqsMplK7Gx/s/r+eZv39DyxpaMnTuWgOYBVoellFIXqCjBP0/5nfB+WYltpwOlizQtgYzSCxhjskp9fB/4R6l5GY5/k0VkA9AN2F9m/feA9wCio6PLXjyoGqCkKI8dK58gY/tsmlx3J53vfh9vvwZWh1Wu/JP5zJs4j30r9tH9ge4MeXMIXr56M4pSyv1UNNDN3ArmVaa6fAsQKSLhwCFgPDCx9AIiEmKMOez4OAx7rQEiEgScdZTsGwN9gVcqsU9Vg+SdPEj8/ImcOZJIRL/fE9HvGezPMnJPx3YcY86IOZw8cJK7/nsX0Q9FWx2SUkqV67KKHiKy1BgztDLLGmOKReRxYBXgif1pdEki8iIQZ4xZDDwhIsOwt7OfAKY5Vu8A/E9EbNhrEV520vte1WBZKetJXDQVYyuh25gvaBpZ4U0Zltu1cBcLJi/Au543U9dPJbSvNgkppdyb2EekreTCIvHGGLcclis6OtrExcVZHYa6BGMMB374D3s2PId/o3ZEjZ5NvYbXWR1WuYzNsOGFDXz14le06NmCsfPHUr9FfavDUkopAETkR2OM0+rEy2081HHo1RUrLsxh+7JHOLJrAc3aj6TTXe/i5eNvdVjlyj+Vz4LJC9izZA9R90Vx1zt34eWn7e1KqZqhovvgQ40xqaWnGWOmuz4kVRvlnthHwrwJ5GTtpu2AvxLW61eIOLvRwj0c33WcOSPmkL0/myFvDaHHoz3cOl6llCqrouLIQqA7gIjMM8aMrp6QVG1zdO8Kti35BSKe3DBuIY3Db730ShbavWQ38yfNx8vPiylrp9D65tZWh6SUUpetogRfurjSxtWBqNrHGBv7v3mZ/d/8jYBmXek26nPqBLpvsjQ2w1d//YoNz20g5IYQxi0YR4NW7nvLnlJKVaSiBG/Kea/UJRXln2Lbkgc4tm85zTtN4Po73sDTu47VYZWr4HQBC6cuZNfCXXSd0pW7/nsX3nXc95G0Sil1KRUl+K4ichp7Sb6O4z2Oz8YYo12JlVM5x3cSP28ieSdTaD/wn4Te8LBbt19n7clizog5ZO3JYvB/BtPriV5uHa9SSlVGRQPduO8TPpTbyty1gO1LH8bTpx49Ji4jqFVfq0Oq0N7le5k3cR4eXh5MXj2Z8AHhVoeklFJVQu/5UVXC2ErYu/EFUr7/Fw2a9yBq1Gf4BTS3OqxyGWP45u/fsO6P6wjuGsy4BeMIDKvMM5SUUqpm0ASvrlrh2Sy2LrqPrAPraBl1Hx0GvoqHl6/VYZWrMKeQhdMWsnPeTjpP7Mzd79+Nd11t0GiYrQAAHDtJREFUb1dK1S6a4NVVOX1kKwnzJpCfc5iOQ96iZdQ0q0Oq0In9J4gZEcOxHccY+OpAbnz6Rm1vV0rVSprg1RXL2B5D0orH8a4TRM9Jqwhs0cPqkCq0b9U+5o2fh3gI9666lza3692fSqnaSxO8umy2kiJ2r/sDqXHvENSqL11HzsS3XjOrwyqXMYZv//kta59dS9NOTRm3cBxB4UFWh6WUUi6lCV5dloLcIyQumEJ22iZCox+l3a0v4eHpvu3XhbmFLL5/MUkxSXQc15FhHw7Dp56P1WEppZTLaYJXlXby0BYSFkyiKC+bznd/QPNO460OqULZKdnEjIjhyLYj3P6P2+nzmz7a3q6UumZogleVkpbwETu//D/8/EPoNWUt9Zt1sTqkCiWvSWbuuLkYm2HSiklcN9h9H0mrlFKuoAleVchWXMDO1b8mPeEjGoXfRpdhM/D5//buO0yq+u77+Pu7S+9VEBDpRZESFxApUiSKGOoCKwIi1jxGTR6NUZPbRJN4pz3R3HeU+yGxJSjsUlaaNAWJoAKLSxVRkb50pC+w5Xv/MYPXui4rRIYzO/N5XddeTjnnzGcY3A/n/M78ToWaQcc6J3fngz9/wNuPvU2t1rVIeTOFGs1qBB1LROSSU8HLOZ06lsXq6aM4krWCxl0eoXmPp7CE6J3gMOdkDrPumcW6N9bRemhrBr06iDKVNN4uIvFJBS9F+nLHMlanjybvzAnaDZ5I3VaDgo5UrMPbDpM6OJU9q/fQ+7e96fZEN423i0hcU8HL17g72zPGs2nRk5Sv2oiOI+dQqVbroGMVa8viLUwdPpW8nDxGzh5J81uaBx1JRCRwKnj5Sl7OST6e9zBZ6ydRu9ktXPODv1G6XPReD93dWf5fy1nwyAJqtqhJyowUajaP3vMDREQuJRW8AJB9eBuZ00dybO9amnb/OU27/gyzhKBjnVNOdg5z7p/Dmn+sodWgVgx6bRBlq0Tv/PciIpeaCl44uGUxa2bcgefn8b1hU6jd7OagIxXryI4jpA5OZfeq3fR8uic9ftEDS9B4u4hIQSr4OObubF3+PJ+++0sq1WxJ+6GTqFgjur8vvu1f20hLTiP3VC4pM1JoOaBl0JFERKKSCj5O5Z45zvo5P2TvJ+nUaTWYNv3HU6pMpaBjnZO7s/KFlcz/yXyqN61Oypsp1GpVK+hYIiJRSwUfh04c+pzMaSmcOPgpLXr9hkadH47qr5TlnsplzgNzWP3yalrc2oLBEwdTrmq5oGOJiEQ1FXyc2ffZXNbNuhtLSCRpxAxqNu4VdKRiHd11lLQhaexasYseT/Wg5y97arxdROQ8qODjhHs+m5f+js1Ln6VynXZ0GPIG5atdGXSsYm1ftp20oWnknMhhRPoIWg1qFXQkEZESQwUfB3JOHWHdrHvY//lb1Gszkqtu/guJpcsHHatYGf8/g7kPzqXaldW4Y9Ed1L6qdtCRRERKFBV8jDu+/2Myp40k+8hWWvX9Ew2vvS+6x9tP5zL3wbl89LePaNavGUPfGEq5ahpvFxG5UCr4GLbnk3TWz76fxDIV6ThyDtWv6Bp0pGIdyzpGWnIaOz/YSbcnu9HrmV4kJEbvZDsiItFMBR+DPD+Pz5Y8zZYP/0zVep1oP2Qi5SrXCzpWsXZ+uJPUIamcPnqaYVOGcVXyVUFHEhEp0VTwMebMyYOsnXEnB7cuokGHcbS+8Y8klIruKVw/+vtHvPXAW1RpUIXRC0ZzWZvLgo4kIlLiqeBjyNE9a1g9fSSnju/m6lteoEG7O4KOVKy8M3nM+/E8MsZn0PT7TRk6aSjla0T3yX8iIiWFCj5GZK2fzIa5P6J0+Rp0GrWAavWSgo5UrON7jjNl2BS2L93O9Y9dT59n+2i8XUTkIlLBl3D5eTlsWvQk2zPGU/2KrrQb/A/KVqwTdKxi7Vqxi9QhqWQfymbopKG0SWkTdCQRkZijgi/BTp/Yy5r0MXy5YxlXdnyAFr1+Q0Ji6aBjFSvzlUzm/HAOlS+vzF0f3EXddnWDjiQiEpMiekzUzG42s01m9rmZPV7E82PNbL+ZrQ7/3F3guTvM7LPwT3QPJgfg8K6VfPBKd47s/ohrfvASrW78fVSXe15OHnMfmsvMcTNp2K0h92Tco3IXEYmgiO3Bm1ki8ALQF9gJrDSzme7+caFFU939R4XWrQH8EkgCHFgVXvfLSOUtSXasfoWNCx6hXOV6dB7zDlXqtA06UrFO7DvBlOFT2LZkG10e6cKNv7uRhFIabxcRiaRIHqLvBHzu7l8AmNlkYCBQuOCLchOw0N0PhdddCNwMTIpQ1hIhP/c0Gxc+ws7Vr1KzcR/aDnyFMuVrBB2rWFmrskgdnMrJ/ScZPHEwbW+P7n+MiIjEikgWfH1gR4H7O4HORSw31Mx6AJ8CP3H3HedYt36kgpYEp47uYnX6KI5kraRxl0do3uMpLCEx6FjFWvPPNcy+dzYVL6vIuGXjuPx7lwcdSUQkbkSy4Iua8NwL3Z8FTHL302Z2P/Aa0Ps818XM7gXuBWjYsOF3SxvFDm1fypr00eTlZtNu8ETqthoUdKRi5efms+CnC1j+/HIa9WxEcloyFWtXDDqWiEhcieRA6E7gigL3GwBZBRdw94Pufjp892/Atee7bnj9Ce6e5O5JtWvH3tXG3J1tK18kY9KtlCpXjevuWBz15X7ywEkm3jSR5c8vp/PDnRm1YJTKXUQkAJHcg18JNDezxsAuIAUYWXABM7vc3XeH7w4ANoZvzweeNbPq4fvfB56IYNaok5dzko/nPUzW+knUbt6fa26dQOlyVYOOVaw9q/cwedBkju85zqDXBtFuTLugI4mIxK2IFby755rZjwiVdSLwsrtvMLNngAx3nwk8ZGYDgFzgEDA2vO4hM/s1oX8kADxz9oS7eJB9eBuZ02/j2N51NOv+C5p0fQyz6D7rfN2kdcy8ayYValZg3NJx1EuK7ovbiIjEOnP/xtB2iZSUlOQZGRlBx/jODmxZxNo3x+KeR9sBL1G72c1BRypWfm4+bz/xNh/86QMadm/IsCnDqFSnUtCxRETigpmtcvci5ybXTHZRwt3Z8uFzfLbkV1Sq2ZL2QydRsUazoGMVK/tQNlNTpvLFwi/o+EBHbnruJhJLR/eZ/SIi8UIFHwVyzxxn/Zz72fvJm9RtPYSrb3mRUmWiey9479q9TB40mWO7jjHgpQF0GNch6EgiIlKACj5gJw59Tua0FE4c/JQWvX9Lo04PYVbUtwSjx4a0Dcy4cwblqpVj7L/G0qBzg6AjiYhIISr4AO37bC7rZt2FJZQiacQMajbuFXSkYuXn5bPo54tY9vtlXHH9FQyfNpxKdaP7SIOISLxSwQfAPZ/NS/+TzUv/k8p12tFhyBuUr3Zl0LGKlf1lNtNum8bm+Zu59v5r6feXfiSW0Xi7iEi0UsFfYjmnDrNu1j3s/3wu9dqM5Kqb/0Ji6fJBxyrWvg37mDxwMke2H+HWCbdy7T3XfvtKIiISKBX8JXR8/8dkThtJ9pGttOr7Jxpee1/Uj7dvnL6R9DHplK1clrHvjuWK66/49pVERCRwKvhLZM8n6ayffT+JZSrSceQcql/RNehIxfJ8Z/FTi3nvt+9Rv3N9RkwfQeV6lYOOJSIi50kFH2Gen8dnS55my4d/pmq9TrQfMpFylaN7lrdTh08xfdR0PpvzGR3u6sAtL9xCqbL6qyIiUpLot3YEnTl5kLUzxnJw62IadBhH6xv/SEKpskHHKtb+jfuZPHAyh7cc5pYXbyHp/qSoH0YQEZFvUsFHyNE9a8icfhunj+/h6lteoEG7O4KO9K0+mfEJ6aPTKV2+NGMWjeHK7tF9Zr+IiJybCj4CstZNYsO8BylTviadRi2gWr0ipwmOGp7vLHlmCUueXkK9jvUYMX0EVRpUCTqWiIh8Byr4iyg/L4dNi55ke8Z4qjfsRrtB/6BsxcuCjlWs00dPkz46nU0zN9F+bHv6j+9PqXL6ayEiUtLpN/lFcvrEXtakj+HLHcu4suMDtOj1GxISSwcdq1gHNh0gdVAqBz87SL//7kfHBzpqvF1EJEao4C+Cw7tWsHr67eScOsw1A16m3tXDg470rT6d/SnTb59OYplExrw9hkY9GwUdSURELiIV/He0I/NlNi58lHKV69F5zDtUqdM26EjF8nznvWffY/FTi7m8w+WMSB9B1YZVg44lIiIXmQr+35Sfe5qPFzzCrjWvUrNxH9oOfIUy5WsEHatYp4+dZsbYGWycvpG2o9py64RbKV0+uocRRETk36OC/zecOrqL1emjOJK1ksZdHqV5j//AEqL7wiuHPj/E5IGTObDpADc9dxOdH+6s8XYRkRimgr9Ah7YvZU36aPJys2k/5HXqtBwYdKRv9fm8z5l22zQs0Ri9YDSNezcOOpKIiESYCv48uTvbM8azadGTlK/WmI5D36JSrdZBxyqWu7Ps98t458l3qNO2DilvplCtUbWgY4mIyCWggj8PeTkn2TD3IXZvmEzt5v255tYJlC4X3SemnTl+hhnjZvDxlI9pk9KGAS8NoHQFjbeLiMQLFXwh7790Pcf2rS3yuWbdf0GTro9hlnCJU12YL7/4ksmDJrN/w376/rEvXR7povF2EZE4o4IvpFr9Thw/+Amed+Zrj9dqehNNuz0eUKrzt3nBZqamTAXg9nm307Rv04ATiYhIEKJ7VzQATbs9/o099ITEsrS55cWAEp0fd+f9P73P6/1ep0qDKtybca/KXUQkjqngCylbqS51Wg356r4llqF+uzGUrVQnwFTFyzmZw/Tbp7PwpwtpPbQ1d31wF9WbVA86loiIBEgFX4SWvZ4hITF03XazBJp2jd5D84e3Hublri+zfvJ6+vyuD8mpyZSpWCboWCIiEjAVfBHKVqpL/bajwRKo33Z01O69f/HOF0xImsDhrYe5/a3b6fazbjqZTkREAJ1kd05Nuz3O8QMbo3Lv3d358PkPWfjoQmq1rkXKmynUaBbd0+SKiMilpYI/h7KV6tJp1PygY3xDTnYOs+6ZxbrX19F6SGsGvjqQspXLBh1LRESijAq+BDmy/Qipg1PZnbmbXr/pRfcnumMJOiQvIiLfpIIvIbYu2cqUYVPIO53HbbNuo0X/FkFHEhGRKKaCj3Luzoq/rmD+T+ZTs3lNRrw5glotawUdS0REopwKPorlnspl9v2zWfPaGloOaMngfw6mbBWNt4uIyLdTwUepozuPkjoklayVWdzwqxu44T9u0Hi7iIicNxV8FNr23jamJE8hJzuHEW+OoNXAVkFHEhGREkYFH0XcnYz/yWDeQ/Oo3qQ6d7x7B7Vb1w46loiIlEAq+CiRezqXtx54i8yXMmnevzlDXh9Cuarlgo4lIiIlVESnqjWzm81sk5l9bmbnnBLOzJLNzM0sKXy/kZllm9nq8M//RDJn0I5lHeO1nq+R+VIm3X/Rndtm3qZyFxGR7yRie/Bmlgi8APQFdgIrzWymu39caLnKwEPA8kKb2Ozu7SOVL1rseH8HaUPTOH3sNMOnDaf1kNZBRxIRkRgQyT34TsDn7v6Fu58BJgMDi1ju18AfgFMRzBKVVk1Yxas9X6V0xdLc/eHdKncREbloIlnw9YEdBe7vDD/2FTPrAFzh7rOLWL+xmWWa2RIz6x7BnJdc3pk8Zt8/m9n3zaZJnybcs/IeLmtzWdCxREQkhkTyJLuivrTtXz1plgA8B4wtYrndQEN3P2hm1wJvmtnV7n70ay9gdi9wL0DDhg0vVu6IOrb7GFOSp7Dj/R10fbwrvX/Tm4REXbVXREQurkgW/E7gigL3GwBZBe5XBtoA74avYV4XmGlmA9w9AzgN4O6rzGwz0ALIKPgC7j4BmACQlJTkRLmdy3eSNiSNU4dPkZyWzNXDrg46koiIxKhIFvxKoLmZNQZ2ASnAyLNPuvsR4KtJ1c3sXeBRd88ws9rAIXfPM7MmQHPgiwhmjbjMlzOZ88M5VK5fmbs+uIs6besEHUlERGJYxAre3XPN7EfAfCAReNndN5jZM0CGu88sZvUewDNmlgvkAfe7+6FIZY2kvJw85v9kPitfWEmTvk1InpxM+Rrlg44lIiIxztyj/sj2eUlKSvKMjIxvX/ASOr73OFOGTWH7e9u5/qfX0+fZPiSU0ni7iIhcHGa2yt2TinpOM9lFyK6Vu0gbksbJgycZ8sYQrrntmqAjiYhIHFHBR8Dq11Yz+77ZVKpbibvev4u67esGHUlEROKMCv4iysvJY8GjC1jxXyto3LsxyanJVKhVIehYIiISh1TwF8mJ/SeYOnwqW9/dynU/uY6+f+ir8XYREQmMCv4i2P3RblIHp3Ji3wkG/3MwbUe1DTqSiIjEORX8d7T29bXMunsWFWpX4M6ld1Lv2npBRxIREVHB/7vyc/NZ+LOFfPjnD7nyhisZljaMipdVDDqWiIgIoIL/t5w8cJKpKVPZ8s4WOj3Yie//v++TWDox6FgiIiJfUcFfoD1r9pA6KJVju48x8JWBtB8b85esFxGREkgFfwHWp65nxp0zKF+jPHe+dyf1O9b/9pVEREQCoII/D/l5+bzz5Du8/4f3aditIcOmDqNSnUpBxxIRETknFXwhy/6wjHod69G4V2MAsg9lM/GmiWRlZJH0f5K4+bmbSSyj8XYREYlumomlkHod6zF1+FS2LN7C3nV7ebHNi2RlZNHlkS70f6G/yl1EREoE7cEX0rhXY5LTkpk8cDK5p3LJz8un31/70emBTkFHExEROW8q+CLUalmLnJM5eJ7T+cedVe4iIlLi6BB9EQ5sOkCZSmXo9kQ31k1cx5bFW4KOJCIickFU8IVsWbyFqcOnMiJ9BH2e7UNyWvJXY/IiIiIlhQq+kKyVWSSnJX91Fv3ZMfmslVkBJxMRETl/5u5BZ7gokpKSPCMjI+gYIiIil4yZrXL3pKKe0x68iIhIDFLBi4iIxCAVvIiISAxSwYuIiMQgFbyIiEgMUsGLiIjEIBW8iIhIDFLBi4iIxKCYmejGzPYD24LOUUgt4EDQIUSfQ5TQ5xAd9DlEh4v1OVzp7rWLeiJmCj4amVnGuWYYkktHn0N00OcQHfQ5RIdL8TnoEL2IiEgMUsGLiIjEIBV8ZE0IOoAA+hyihT6H6KDPITpE/HPQGLyIiEgM0h68iIhIDFLBR4CZlTOzFWa2xsw2mNnTQWeKV2aWaGaZZjY76CzxzMy2mtk6M1ttZhlB54lXZlbNzKaa2SdmttHMugSdKd6YWcvw/wdnf46a2Y8j8VqlIrFR4TTQ292Pm1lpYKmZzXX3D4MOFoceBjYCVYIOIvRyd33/Olh/Aea5e7KZlQEqBB0o3rj7JqA9hHZAgF1AeiReS3vwEeAhx8N3S4d/dLLDJWZmDYD+wN+DziISNDOrAvQAXgJw9zPufjjYVHGvD7DZ3SMySZsKPkLCh4ZXA/uAhe6+POhMceh54DEgP+ggggMLzGyVmd0bdJg41QTYD7wSHrb6u5lVDDpUnEsBJkVq4yr4CHH3PHdvDzQAOplZm6AzxRMzuxXY5+6rgs4iAHR19+8B/YAHzKxH0IHiUCnge8B4d+8AnAAeDzZS/AoPkQwApkTqNVTwERY+BPYucHPAUeJNV2CAmW0FJgO9zWxisJHil7tnhf+7j9B4Y6dgE8WlncDOAkcTpxIqfAlGP+Ajd98bqRdQwUeAmdU2s2rh2+WBG4FPgk0VX9z9CXdv4O6NCB0GW+TuowKOFZfMrKKZVT57G/g+sD7YVPHH3fcAO8ysZfihPsDHAUaKd7cRwcPzoLPoI+Vy4LXwGZIJQJq762taEq/qAOlmBqHfOW+4+7xgI8WtB4HXw4eHvwDuDDhPXDKzCkBf4L6Ivo5mshMREYk9OkQvIiISg1TwIiIiMUgFLyIiEoNU8CIiIjFIBS8iIhKDVPAiATGz49++VLHrTzWzJmb2qpndV+i5QWb21ndL+LXt/crMHr3Add41s6SLlaHAdq8zs+XhK3FtNLNfhR8fYGYXPDObmV1jZq9e7JwiQVPBi5RAZnY1kOjuXxCaLCOl0CIRneM6EsLzRpyP14B7w1NBtwHSANx9prv/7kJf193XAQ3MrOGFrisSzVTwIgGzkD+a2frwNdNHhB9PMLMXzWyDmc02s7fMLDm82u3AjPDtt4FWZnZ5eL0KhGZPfDN8//+Gt72+4HWnzWyMma01szVm9s/wYz8I7x1nmtnbZlanQNR2ZrbIzD4zs3vCy/c0s9kFtvlXMxtbxHscb2YZ4ffydIHHt5rZU2a2FHjczD4q8FxzMyvqWgKXAbvhq2s+fBxefqyZ/TV8u+D1trPN7IbwjHovm9nK8PsbWGCbs/jmP5JESjTNZCcSvCGErg/dDqgFrDSzfxGaT78RcA2hUtsIvBxepyvhPXR3zzOz6cBwQtf7HgAsdvdjZnYtodnKOgMGLDezJcAZ4OeELgJzwMxqhLe7FLjO3d3M7iZ0Nb5Hws+1Ba4DKgKZZjbnAt7jz939UHgv/R0za+vua8PPnXL3bgBmdqOZtXf31eHcrxaxreeATWb2LjAPeM3dTxVcILx3j5n9IPwe3geeJjRl8bjwVNIrzOxtdz8BZBC68MofLuA9iUQ17cGLBK8bMCm8N7oXWAJ0DD8+xd3zw/OILy6wzuWELv15VsHD9AUPz3cD0t39hLsfB6YD3YHewFR3PwDg7ofCyzcA5pvZOuCnwNUFXmOGu2eH11nMhV0wZnh47zwzvM2rCjyXWuD234E7w/8QGAG8UXhD7v4MkAQsAEYSKvlvMLPmwB+BEe6eQ2gO/MfDl3F+FygHnD0svw+odwHvRyTqqeBFgmcX+DhANqGCOmsZcLmZtQOuB86eYFfctouap/q/gb+6+zWE5sku+BqFl3cgl6//HilXaBnMrDHwKNDH3dsCcwotd6LA7WmErrJ1K7DK3Q8WFd7dN7v7eEIXTGlnZjULvWZFQmPz95y9kh2h9zzU3duHfxq6+8YCubOLei2RkkoFLxK8fwEjzCzRzGoDPYAVhA6XDw2PxdcBehZYZyPQ7OwdD11UIo3QCWhvFThk/S9gkJlVCJfeYOA94B1Ce9U1AQocoq8K7ArfvqNQzoFmVi68Tk9gJbANuMrMyppZVUKFW1gVQiV+JPw++p3rDyKcez4wHnilqGXMrL+Fr1wDNAfygMOFFnsFeMXd3yvw2HzgwbPrmlmHAs+1QFe4kxijMXiR4KUDXYA1hPaKH3P3PWY2jVBhrgc+BZYDR8LrzCFUsm8X2M4kQofVv/qqmLt/FP4K2IrwQ39390wAM/stsMTM8ggdOh8L/AqYYma7gA+BxgW2vyL8ug2BX5/dMzazNGAt8Fl4O1/j7mvMLBPYQOgKZsu+5c/jdULnJSw4x/OjgefM7CShIwi3h89DIJznSiAZaGFm48Lr3A38GngeWBsu+a2EjhQA9Aq/N5GYoavJiUQxM6vk7sfDe80rCJ0Ut8fMyhMaB+/q7nnBpry4LPR9+6ru/h+X6PXKEjrvoZu7516K1xS5FLQHLxLdZofP+C5DaK95D4C7Z5vZL4H6wPYgA15MZpYONCV0EuCl0hB4XOUusUZ78CIiIjFIJ9mJiIjEIBW8iIhIDFLBi4iIxCAVvIiISAxSwYuIiMQgFbyIiEgM+l+KoGH8vZL3DQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 576x360 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "c = list(np.arange(0.01,1,0.1))\n",
    "P6(train_data, train_labels, dev_data, dev_labels, c)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "bZmIPwExFsx1"
   },
   "source": [
    "ANSWER:\n",
    "\n",
    "Model performance is positively correlated with training vocabulary size for both types of models that we created. Nevertheless, as the models using the reduced vocabulary use l2 regularization with a fixed C-value, while the models using the full vocabulary use a variable C value and l1 regularization with a changing C-value it might be difficult to make accurate comparisons between the two. We can see from the graph above, however, that the l2 model with reduced vocabulary generally outperforms the l1 model with the full vocabulary, up to a certain point, where the l1 model then overtakes the l2 model in terms of performance. Nevertheless, performance for both plateaus after about log(5) vocabulary size. Lasso (l2) shrinks the less important (i.e. outlier) coefficients independently, while ridge (l1) shrinks outliers unevenly to 0. It would appear that after a certain vocabulary size, the colinearity of variables begins to impact the l2 model, thus allowing it to be outperformed by the l1 model. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "QmrHf8AMFsxz"
   },
   "source": [
    "### Part 7:\n",
    "\n",
    "How is `TfidfVectorizer` different than `CountVectorizer`?\n",
    "\n",
    "Produce a Logistic Regression model based on data represented in tf-idf form, with L2 regularization strength of 100.  Evaluate and show the f1 score.  How is `TfidfVectorizer` different than `CountVectorizer`?\n",
    "\n",
    "Show the 3 documents with highest R ratio, where ...<br/>\n",
    "$R\\,ratio = maximum\\,predicted\\,probability \\div predicted\\,probability\\,of\\,correct\\,label$\n",
    "\n",
    "Explain what the R ratio describes.  What kinds of mistakes is the model making? Suggest a way to address one particular issue that you see.\n",
    "\n",
    "Note:\n",
    "* Train on the transformed training data.\n",
    "* Evaluate on the transformed development data.\n",
    "* You can use `TfidfVectorizer` and its `.fit_transform` method to transform data to tf-idf form.\n",
    "* You can use `LogisticRegression(C=100, solver=\"liblinear\", multi_class=\"auto\")` to produce a logistic regression model.\n",
    "* You can use `LogisticRegression`'s `.predict_proba` method to access predicted probabilities."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "8vwU_9t2Fsx0"
   },
   "outputs": [],
   "source": [
    "def P7(train_data, train_labels, test_data, test_labels):\n",
    "    \n",
    "    # Fit tfidf vectorizer\n",
    "    tfidf = TfidfVectorizer()\n",
    "    tfidf_fit_train = tfidf.fit_transform(train_data)\n",
    "    tfidf_fit_test = tfidf.transform(test_data)\n",
    "    \n",
    "    # Fit count vectorizer\n",
    "    cv = CountVectorizer()\n",
    "    cv_fit_train = cv.fit_transform(train_data)\n",
    "    cv_fit_test = cv.transform(test_data)\n",
    "    \n",
    "    # Train LR using tfidf\n",
    "    LR1 = LogisticRegression(C = 100, solver = \"liblinear\", multi_class = \"auto\")\n",
    "    LR1_fit = LR1.fit(tfidf_fit_train, train_labels)\n",
    "    LR1_predict = LR1.predict(tfidf_fit_test)\n",
    "    LR1_f1 = metrics.f1_score(test_labels, LR1_predict, average = \"weighted\")\n",
    "    \n",
    "    # Train LR using standard countvectorizer\n",
    "    LR2 = LogisticRegression(C = 100, solver = \"liblinear\", multi_class = \"auto\")\n",
    "    LR2_fit = LR2.fit(cv_fit_train, train_labels)\n",
    "    LR2_predict = LR2.predict(cv_fit_test)\n",
    "    LR2_f1 = metrics.f1_score(test_labels, LR2_predict, average = \"weighted\")\n",
    "    \n",
    "    # Calculate top 3 probabilities\n",
    "    LR1_pred = LR1.predict_proba(tfidf_fit_test)\n",
    "    \n",
    "    position_list = []\n",
    "    for row in LR1_pred:\n",
    "        position_list.append(np.argmax(row))\n",
    "        \n",
    "    ratio_list = []\n",
    "    for index, label in enumerate(position_list):\n",
    "        if label == test_labels[index]:\n",
    "            ratio_list.append(np.max(LR1_pred[index])/LR1_pred[index][label])\n",
    "        else:\n",
    "            ratio_list.append(np.max(LR1_pred[index]/LR1_pred[index][test_labels[index]]))\n",
    "            \n",
    "    indices = list((-np.array(ratio_list)).argsort()[:3])\n",
    "    \n",
    "    # Print \n",
    "    print(\"Comparison of Model Performance\")\n",
    "    print(\"---------------------------------------------------------------------------------------------------\")\n",
    "    print(\"The f1 score of the logistic regression model using the Tfidf vectorized training dataset is: {}\".format(round(LR1_f1, 3)))\n",
    "    print(\"The f1 score of the logistic regression model using the count vectorized training dataset is: {}\".format(round(LR2_f1, 3)))\n",
    "    print(\"---------------------------------------------------------------------------------------------------\")\n",
    "    print(\"\")\n",
    "    print(\"\")\n",
    "    print(\"Documents with the 3 Largest R-Ratios\")\n",
    "    print(\"\")\n",
    "    for index, value in enumerate(indices):\n",
    "        print(\"Example {}\".format(index + 1))\n",
    "        print(\"---------------------------------------------------------------------------------------------------\")\n",
    "        print(\"The predicted label: {}\".format(newsgroups_train.target_names[position_list[value]]))\n",
    "        print(\"The real label: {}\".format(newsgroups_train.target_names[test_labels[value]]))\n",
    "        print(\"The R-ratio: {}\".format(round(ratio_list[value], 3)))\n",
    "        print(\"---------------------------------------------------------------------------------------------------\")\n",
    "        print(\"The message:\")\n",
    "        print(test_data[value])\n",
    "        print(\"\")\n",
    "        \n",
    "    return "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Comparison of Model Performance\n",
      "---------------------------------------------------------------------------------------------------\n",
      "The f1 score of the logistic regression model using the Tfidf vectorized training dataset is: 0.76\n",
      "The f1 score of the logistic regression model using the count vectorized training dataset is: 0.68\n",
      "---------------------------------------------------------------------------------------------------\n",
      "\n",
      "\n",
      "Documents with the 3 Largest R-Ratios\n",
      "\n",
      "Example 1\n",
      "---------------------------------------------------------------------------------------------------\n",
      "The predicted label: comp.graphics\n",
      "The real label: talk.religion.misc\n",
      "The R-ratio: 929.358\n",
      "---------------------------------------------------------------------------------------------------\n",
      "The message:\n",
      "I am pleased to announce that a *revised version* of _The Easy-to-Read Book\n",
      "of Mormon_ (former title: _Mormon's Book_) by Lynn Matthews Anderson is now\n",
      "available through anonymous ftp (see information below). In addition to the\n",
      "change in title, the revised ETR BOM has been shortened by several pages\n",
      "(eliminating many extraneous \"that's\" and \"of's\"), and many (minor) errors\n",
      "have been corrected. This release includes a simplified Joseph Smith Story,\n",
      "testimonies of the three and eight witnesses, and a \"Words-to-Know\"\n",
      "glossary.\n",
      "\n",
      "As with the previous announcement, readers are reminded that this is a\n",
      "not-for-profit endeavor. This is a copyrighted work, but people are welcome\n",
      "to make *verbatim* copies for personal use. People can recuperate the\n",
      "actual costs of printing (paper, copy center charges), but may not charge\n",
      "anything for their time in making copies, or in any way realize a profit\n",
      "from the use of this book. See the permissions notice in the book itself\n",
      "for the precise terms.\n",
      "\n",
      "Negotiations are currently underway with a Mormon publisher vis-a-vis the\n",
      "printing and distribution of bound books. (Sorry, I'm out of the wire-bound\n",
      "\"first editions.\") I will make another announcement about the availability\n",
      "of printed copies once everything has been worked out.\n",
      "\n",
      "FTP information: connect via anonymous ftp to carnot.itc.cmu.edu, then \"cd\n",
      "pub\" (you won't see anything at all until you do).\n",
      "\n",
      "\"The Easy-to-Read Book of Mormon\" is currently available in postscript and\n",
      "RTF (rich text format). (ASCII, LaTeX, and other versions can be made\n",
      "available; contact dba@andrew.cmu.edu for details.) You should be able to\n",
      "print the postscript file on any postscript printer (such as an Apple\n",
      "Laserwriter); let dba know if you have any difficulties. (The postscript in\n",
      "the last release had problems on some printers; this time it should work\n",
      "better.) RTF is a standard document interchange format that can be read in\n",
      "by a number of word processors, including Microsoft Word for both the\n",
      "Macintosh and Windows. If you don't have a postscript printer, you may be\n",
      "able to use the RTF file to print out a copy of the book.\n",
      "\n",
      "-r--r--r--  1 dba                   1984742 Apr 27 13:12 etrbom.ps\n",
      "-r--r--r--  1 dba                   1209071 Apr 27 13:13 etrbom.rtf\n",
      "\n",
      "For more information about how this project came about, please refer to my\n",
      "article in the current issue of _Sunstone_, entitled \"Delighting in\n",
      "Plainness: Issues Surrounding a Simple Modern English Book of Mormon.\"\n",
      "\n",
      "Send all inquiries and comments to:\n",
      "\n",
      "    Lynn Matthews Anderson\n",
      "    5806 Hampton Street\n",
      "    Pittsburgh, PA 15206\n",
      "\n",
      "Example 2\n",
      "---------------------------------------------------------------------------------------------------\n",
      "The predicted label: comp.graphics\n",
      "The real label: talk.religion.misc\n",
      "The R-ratio: 325.004\n",
      "---------------------------------------------------------------------------------------------------\n",
      "The message:\n",
      "Can anyone provide me a ftp site where I can obtain a online version\n",
      "of the Book of Mormon. Please email the internet address if possible.\n",
      "\n",
      "Example 3\n",
      "---------------------------------------------------------------------------------------------------\n",
      "The predicted label: talk.religion.misc\n",
      "The real label: alt.atheism\n",
      "The R-ratio: 287.179\n",
      "---------------------------------------------------------------------------------------------------\n",
      "The message:\n",
      "\n",
      "The 24 children were, of course, killed by a lone gunman in a second story\n",
      "window, who fired eight bullets in the space of two seconds...\n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "P7(train_data, train_labels, dev_data, dev_labels)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "bZmIPwExFsx1"
   },
   "source": [
    "ANSWER:\n",
    "\n",
    "The R-ratio may be thought of as a magnitude score of how wrong the model was in predicting the class of the message. The numerator of the R-ratio is the maximum predicted probability, corresponding to the index position of the predicted class. The denominator corresponds to the true label probability. The lower the true label's probability, the \"worse\" the model performed for that particular data point and consequently the R-ratio will be larger due to the differences between the numerators and denominators. \n",
    "\n",
    "Interestingly the example which the model performed worst on - i.e. the one with the highest R-ratio is a message belonging to the religion label, but which actually has many computer and computer-graphics related words in it. This is likely why the model predicted the wrong label. Similarly for the second worst performing example, it was another religion example, but with words like \"ftp\", \"online\" and \"internet\", demonstrating how the model might mistake it for belonging to computer graphics. Finally, the 3rd worst performing example is a little more different without any particular key words belonging to either category. \n",
    "\n",
    "A potential way to address this would be to try and identify words that seem to be common in the language and thus across categories (but not strictly categorized as \"stop words\" per se) - e.g. \"internet\", \"e-mail\" etc and remove these from the vocabulary, we can potentially modify the preprocessor for this."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "VmsDxtzoFsx1"
   },
   "source": [
    "### Part 8 EXTRA CREDIT:\n",
    "\n",
    "Produce a Logistic Regression model to implement your suggestion from Part 7."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 158,
   "metadata": {},
   "outputs": [],
   "source": [
    "def better_better_preprocessor(s):\n",
    "\n",
    "    # Convert to lower case\n",
    "    s = s.lower()\n",
    "\n",
    "    # Remove stop words\n",
    "    tk = nltk.tokenize\n",
    "    text_tokens = tk.word_tokenize(s)\n",
    "    tokens_without_sw = [word for word in text_tokens if not word in ENGLISH_STOP_WORDS]\n",
    "    s = (\" \").join(tokens_without_sw)\n",
    "    \n",
    "    # Remove some generic words like 'email', 'internet' etc.\n",
    "    remove_list = ['e-mail', 'email', 'internet', 'ftp', 'ascii', 'latex', '.edu', '.com']\n",
    "    remove = '|'.join(remove_list)\n",
    "    \n",
    "    s = re.sub(remove, ' ', s)\n",
    "    \n",
    "    # Remove newlines\n",
    "    s = re.sub(r'\\n', ' ', s)\n",
    "    \n",
    "    # Replace digits \n",
    "    s = re.sub(r'\\d', '@', s)\n",
    "\n",
    "    # Remove punctuation \n",
    "    s = re.sub(r'[^A-Za-z0-9]+', ' ', s)\n",
    "    \n",
    "    # Replace digits \n",
    "    s = re.sub(r'\\d', '@', s)\n",
    "    \n",
    "    # Remove words with certain endings\n",
    "    s = re.sub('ing ', ' ', s)\n",
    "    s = re.sub('est ', ' ', s)\n",
    "    s = re.sub('ly ', ' ', s)\n",
    "    s = re.sub('ion ', ' ', s)\n",
    "    \n",
    "    return s"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 159,
   "metadata": {},
   "outputs": [],
   "source": [
    "def P8(train_data, train_labels, test_data, test_labels):\n",
    "    \n",
    "    # Fit tfidf vectorizer\n",
    "    tfidf = TfidfVectorizer(preprocessor=better_better_preprocessor)\n",
    "    tfidf_fit_train = tfidf.fit_transform(train_data)\n",
    "    tfidf_fit_test = tfidf.transform(test_data)\n",
    "    \n",
    "    # Fit count vectorizer\n",
    "    cv = CountVectorizer(preprocessor=better_better_preprocessor)\n",
    "    cv_fit_train = cv.fit_transform(train_data)\n",
    "    cv_fit_test = cv.transform(test_data)\n",
    "    \n",
    "    # Train LR using tfidf\n",
    "    LR1 = LogisticRegression(C = 100, solver = \"liblinear\", multi_class = \"auto\")\n",
    "    LR1_fit = LR1.fit(tfidf_fit_train, train_labels)\n",
    "    LR1_predict = LR1.predict(tfidf_fit_test)\n",
    "    LR1_f1 = metrics.f1_score(test_labels, LR1_predict, average = \"weighted\")\n",
    "    \n",
    "    # Train LR using standard countvectorizer\n",
    "    LR2 = LogisticRegression(C = 100, solver = \"liblinear\", multi_class = \"auto\")\n",
    "    LR2_fit = LR2.fit(cv_fit_train, train_labels)\n",
    "    LR2_predict = LR2.predict(cv_fit_test)\n",
    "    LR2_f1 = metrics.f1_score(test_labels, LR2_predict, average = \"weighted\")\n",
    "    \n",
    "    # Calculate top 3 probabilities\n",
    "    LR1_pred = LR1.predict_proba(tfidf_fit_test)\n",
    "    \n",
    "    position_list = []\n",
    "    for row in LR1_pred:\n",
    "        position_list.append(np.argmax(row))\n",
    "        \n",
    "    ratio_list = []\n",
    "    for index, label in enumerate(position_list):\n",
    "        if label == test_labels[index]:\n",
    "            ratio_list.append(np.max(LR1_pred[index])/LR1_pred[index][label])\n",
    "        else:\n",
    "            ratio_list.append(np.max(LR1_pred[index]/LR1_pred[index][test_labels[index]]))\n",
    "            \n",
    "    indices = list((-np.array(ratio_list)).argsort()[:3])\n",
    "    \n",
    "    # Print \n",
    "    print(\"Comparison of Model Performance\")\n",
    "    print(\"---------------------------------------------------------------------------------------------------\")\n",
    "    print(\"The f1 score of the logistic regression model using the Tfidf vectorized training dataset is: {}\".format(round(LR1_f1, 3)))\n",
    "    print(\"The f1 score of the logistic regression model using the count vectorized training dataset is: {}\".format(round(LR2_f1, 3)))\n",
    "    print(\"---------------------------------------------------------------------------------------------------\")\n",
    "    print(\"\")\n",
    "    print(\"\")\n",
    "    print(\"Documents with the 3 Largest R-Ratios\")\n",
    "    print(\"\")\n",
    "    for index, value in enumerate(indices):\n",
    "        print(\"Example {}\".format(index + 1))\n",
    "        print(\"---------------------------------------------------------------------------------------------------\")\n",
    "        print(\"The predicted label: {}\".format(newsgroups_train.target_names[position_list[value]]))\n",
    "        print(\"The real label: {}\".format(newsgroups_train.target_names[test_labels[value]]))\n",
    "        print(\"The R-ratio: {}\".format(round(ratio_list[value], 3)))\n",
    "        print(\"---------------------------------------------------------------------------------------------------\")\n",
    "        print(\"The message:\")\n",
    "        print(test_data[value])\n",
    "        print(\"\")\n",
    "        \n",
    "    return "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 160,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Comparison of Model Performance\n",
      "---------------------------------------------------------------------------------------------------\n",
      "The f1 score of the logistic regression model using the Tfidf vectorized training dataset is: 0.766\n",
      "The f1 score of the logistic regression model using the count vectorized training dataset is: 0.685\n",
      "---------------------------------------------------------------------------------------------------\n",
      "\n",
      "\n",
      "Documents with the 3 Largest R-Ratios\n",
      "\n",
      "Example 1\n",
      "---------------------------------------------------------------------------------------------------\n",
      "The predicted label: comp.graphics\n",
      "The real label: talk.religion.misc\n",
      "The R-ratio: 1143.613\n",
      "---------------------------------------------------------------------------------------------------\n",
      "The message:\n",
      "I am pleased to announce that a *revised version* of _The Easy-to-Read Book\n",
      "of Mormon_ (former title: _Mormon's Book_) by Lynn Matthews Anderson is now\n",
      "available through anonymous ftp (see information below). In addition to the\n",
      "change in title, the revised ETR BOM has been shortened by several pages\n",
      "(eliminating many extraneous \"that's\" and \"of's\"), and many (minor) errors\n",
      "have been corrected. This release includes a simplified Joseph Smith Story,\n",
      "testimonies of the three and eight witnesses, and a \"Words-to-Know\"\n",
      "glossary.\n",
      "\n",
      "As with the previous announcement, readers are reminded that this is a\n",
      "not-for-profit endeavor. This is a copyrighted work, but people are welcome\n",
      "to make *verbatim* copies for personal use. People can recuperate the\n",
      "actual costs of printing (paper, copy center charges), but may not charge\n",
      "anything for their time in making copies, or in any way realize a profit\n",
      "from the use of this book. See the permissions notice in the book itself\n",
      "for the precise terms.\n",
      "\n",
      "Negotiations are currently underway with a Mormon publisher vis-a-vis the\n",
      "printing and distribution of bound books. (Sorry, I'm out of the wire-bound\n",
      "\"first editions.\") I will make another announcement about the availability\n",
      "of printed copies once everything has been worked out.\n",
      "\n",
      "FTP information: connect via anonymous ftp to carnot.itc.cmu.edu, then \"cd\n",
      "pub\" (you won't see anything at all until you do).\n",
      "\n",
      "\"The Easy-to-Read Book of Mormon\" is currently available in postscript and\n",
      "RTF (rich text format). (ASCII, LaTeX, and other versions can be made\n",
      "available; contact dba@andrew.cmu.edu for details.) You should be able to\n",
      "print the postscript file on any postscript printer (such as an Apple\n",
      "Laserwriter); let dba know if you have any difficulties. (The postscript in\n",
      "the last release had problems on some printers; this time it should work\n",
      "better.) RTF is a standard document interchange format that can be read in\n",
      "by a number of word processors, including Microsoft Word for both the\n",
      "Macintosh and Windows. If you don't have a postscript printer, you may be\n",
      "able to use the RTF file to print out a copy of the book.\n",
      "\n",
      "-r--r--r--  1 dba                   1984742 Apr 27 13:12 etrbom.ps\n",
      "-r--r--r--  1 dba                   1209071 Apr 27 13:13 etrbom.rtf\n",
      "\n",
      "For more information about how this project came about, please refer to my\n",
      "article in the current issue of _Sunstone_, entitled \"Delighting in\n",
      "Plainness: Issues Surrounding a Simple Modern English Book of Mormon.\"\n",
      "\n",
      "Send all inquiries and comments to:\n",
      "\n",
      "    Lynn Matthews Anderson\n",
      "    5806 Hampton Street\n",
      "    Pittsburgh, PA 15206\n",
      "\n",
      "Example 2\n",
      "---------------------------------------------------------------------------------------------------\n",
      "The predicted label: sci.space\n",
      "The real label: alt.atheism\n",
      "The R-ratio: 236.956\n",
      "---------------------------------------------------------------------------------------------------\n",
      "The message:\n",
      "\n",
      "The 24 children were, of course, killed by a lone gunman in a second story\n",
      "window, who fired eight bullets in the space of two seconds...\n",
      "\n",
      "\n",
      "Example 3\n",
      "---------------------------------------------------------------------------------------------------\n",
      "The predicted label: alt.atheism\n",
      "The real label: talk.religion.misc\n",
      "The R-ratio: 104.939\n",
      "---------------------------------------------------------------------------------------------------\n",
      "The message:\n",
      "Why is the NT tossed out as info on Jesus.  I realize it is normally tossed\n",
      "out because it contains miracles, but what are the other reasons?\n",
      "\n",
      "MAC\n",
      "--\n",
      "****************************************************************\n",
      "                                                    Michael A. Cobb\n",
      " \"...and I won't raise taxes on the middle     University of Illinois\n",
      "    class to pay for my programs.\"                 Champaign-Urbana\n",
      "          -Bill Clinton 3rd Debate             cobb@alexia.lis.uiuc.edu\n",
      "\n"
     ]
    }
   ],
   "source": [
    "P8(train_data, train_labels, dev_data, dev_labels)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "ANSWER:\n",
    "\n",
    "As we can see here the performance of the two models has improved marginally by utilizing a preprocessor for both the vectorizers. Nevertheless, the first two examples are still wrong - with the model performing even worse on the first one, but better on the second. Example 3 is an entirely new one. As there second two examples are rleated to atheism and religion it seems that there are common keywords here that would require identification and removal (this makes sense as the two topics are very related)."
   ]
  }
 ],
 "metadata": {
  "colab": {
   "collapsed_sections": [],
   "name": "firstname_lastname_p2.ipynb",
   "private_outputs": true,
   "provenance": [
    {
     "file_id": "https://github.com/MIDS-W207/Master/blob/master/Projects/firstname_lastname_p2.ipynb",
     "timestamp": 1559779272103
    }
   ],
   "version": "0.3.2"
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.7"
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": false,
   "sideBar": true,
   "skip_h1_title": true,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {},
   "toc_section_display": true,
   "toc_window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}