Cartpole Q-Table.ipynb

{
    "cells": [
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "# Cartpole: Q-Table"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "## Introduction"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 15,
            "metadata": {},
            "outputs": [],
            "source": [
                "import numpy as np\n",
                "import matplotlib.pyplot as plt\n",
                "plt.style.use('dark_background')\n",
                "\n",
                "import gym\n",
                "from sklearn.preprocessing import KBinsDiscretizer\n",
                "from IPython.display import clear_output"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 16,
            "metadata": {},
            "outputs": [],
            "source": [
                "name = 'CartPole-v1'\n",
                "env = gym.make(name)"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "The game: A pole is attached by an un-actuated joint to a cart, which moves along a frictionless track. The system is controlled by applying a force of +1 or -1 to the cart. The pendulum starts upright, and the goal is to prevent it from falling over. A reward of +1 is provided for every timestep that the pole remains upright. The episode ends when the pole is more than 15 degrees from vertical, or the cart moves more than 2.4 units from the center."
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "This game has a discrete action space but a continous state space. To form the Q-Table we must turn the continous state space into a discrete state space using a discretizer which uniformly divides the continous state space into a 2D grid of bins."
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 17,
            "metadata": {},
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": [
                        "Action Space: Discrete(2)\n",
                        "State Space: Box(4,)\n"
                    ]
                }
            ],
            "source": [
                "print('Action Space:', env.action_space) # Discrete action space\n",
                "print('State Space:', env.observation_space) # Continous state space"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "## Generating the Q-Table"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "### Setting the parameters"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 18,
            "metadata": {},
            "outputs": [],
            "source": [
                "n_episodes = 2000\n",
                "\n",
                "discount_rate = 0.99\n",
                "\n",
                "min_exp_rate = 0.01\n",
                "exp_decay_rate = 0.001\n",
                "\n",
                "min_learn_rate = 0.01\n",
                "learn_decay_rate = 0.001"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "### Creating the discretizer to uniformly divide the continous state into bins"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 19,
            "metadata": {},
            "outputs": [],
            "source": [
                "n_bins = (6,12)\n",
                "\n",
                "# Pole angle between the low and high from the environment (-24,24) degrees\n",
                "# Pole angular velocity between (-50,50) degrees\n",
                "lower_bounds = [env.observation_space.low[2], -np.radians(50)]\n",
                "upper_bounds = [env.observation_space.high[2], np.radians(50)]\n",
                "\n",
                "# Converts the continous state space to a discrete state space\n",
                "def discretizer(_, __, angle, pole_velocity):\n",
                "    est = KBinsDiscretizer(n_bins=n_bins, encode='ordinal', strategy='uniform')\n",
                "    est.fit([lower_bounds, upper_bounds])\n",
                "    return tuple(map(int,est.transform([[angle, pole_velocity]])[0]))"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "### Initializing the Q-Table"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 20,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/plain": [
                            "(6, 12, 2)"
                        ]
                    },
                    "execution_count": 20,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": [
                "q_table = np.zeros(n_bins + (env.action_space.n,))\n",
                "q_table.shape"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "The Q-Table has 6 bins for the angle, 12 bins for the angular velocity and 2 bins for the actions"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "### Creating helper functions"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 21,
            "metadata": {},
            "outputs": [],
            "source": [
                "# Best action given the state\n",
                "def best_action(state:tuple):\n",
                "    return np.argmax(q_table[state])"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 22,
            "metadata": {},
            "outputs": [],
            "source": [
                "# Learning rate decay\n",
                "def learning_rate(e:int) -> float  :\n",
                "    return min_learn_rate + (1 - min_learn_rate) * np.exp(-learn_decay_rate*e)"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 23,
            "metadata": {},
            "outputs": [],
            "source": [
                "# Exploration rate decay\n",
                "def exploration_rate(e:int) -> float :\n",
                "    return min_exp_rate + (1 - min_exp_rate) * np.exp(-exp_decay_rate*e)"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "### Building the Q-Table"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 24,
            "metadata": {},
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": [
                        "Episode: 0\n",
                        "Average Score: 0.27\n",
                        "\n",
                        "Episode: 100\n",
                        "Average Score: 22.48\n",
                        "\n",
                        "Episode: 200\n",
                        "Average Score: 22.86\n",
                        "\n",
                        "Episode: 300\n",
                        "Average Score: 26.18\n",
                        "\n",
                        "Episode: 400\n",
                        "Average Score: 32.36\n",
                        "\n",
                        "Episode: 500\n",
                        "Average Score: 35.18\n",
                        "\n",
                        "Episode: 600\n",
                        "Average Score: 34.43\n",
                        "\n",
                        "Episode: 700\n",
                        "Average Score: 49.01\n",
                        "\n",
                        "Episode: 800\n",
                        "Average Score: 54.37\n",
                        "\n",
                        "Episode: 900\n",
                        "Average Score: 58.57\n",
                        "\n",
                        "Episode: 1000\n",
                        "Average Score: 51.79\n",
                        "\n",
                        "Episode: 1100\n",
                        "Average Score: 95.13\n",
                        "\n",
                        "Episode: 1200\n",
                        "Average Score: 92.07\n",
                        "\n",
                        "Episode: 1300\n",
                        "Average Score: 114.44\n",
                        "\n",
                        "Episode: 1400\n",
                        "Average Score: 82.23\n",
                        "\n",
                        "Episode: 1500\n",
                        "Average Score: 140.3\n",
                        "\n",
                        "Episode: 1600\n",
                        "Average Score: 87.97\n",
                        "\n",
                        "Episode: 1700\n",
                        "Average Score: 162.42\n",
                        "\n",
                        "Episode: 1800\n",
                        "Average Score: 227.41\n",
                        "\n"
                    ]
                }
            ],
            "source": [
                "scores = []\n",
                "\n",
                "for e in range(n_episodes):\n",
                "    \n",
                "    # Discretize state into buckets\n",
                "    current_state = discretizer(*env.reset())\n",
                "    \n",
                "    done = False\n",
                "    score = 0\n",
                "    while done!=True:\n",
                "        \n",
                "        # Choose between exploration and exploitation\n",
                "        er = exploration_rate(e)\n",
                "        if np.random.random() <= er: \n",
                "            action = env.action_space.sample() # Explore\n",
                "        else:\n",
                "            action = best_action(current_state) # Exploit\n",
                "        \n",
                "        # Increment enviroment\n",
                "        observation, reward, done, _ = env.step(action)\n",
                "        new_state = discretizer(*observation)\n",
                "        \n",
                "        # Update Q-Table according to the Bellman equation\n",
                "        lr = learning_rate(e)\n",
                "        old_value = q_table[current_state][action]\n",
                "        learned_value = reward + discount_rate*np.max(q_table[new_state])\n",
                "        q_table[current_state][action] = (1-lr)*old_value + lr*learned_value\n",
                "        \n",
                "        # Update variables\n",
                "        current_state = new_state\n",
                "        score+=1\n",
                "        # env.render()\n",
                "    \n",
                "    scores.append(score)\n",
                "    \n",
                "    if len(scores)>50:\n",
                "        if sum(scores[-50:])/100 > 200:\n",
                "            break\n",
                "    \n",
                "    if e%100==0:\n",
                "        print('Episode:', e)\n",
                "        print('Average Score:', sum(scores[-100:])/100)\n",
                "        print()"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "### Plotting the scores during learning"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 27,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAEICAYAAACktLTqAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8vihELAAAACXBIWXMAAAsTAAALEwEAmpwYAABOlUlEQVR4nO2deVwU9f/HX7vLqYKgqCggkokh3ob3UWqImWEeRWVRmvotrSwt0cr6fiuyw1+HZQeaYmlEmUmWhWFqmQrlhSIp5gEpmHnnwbGf3x/LDju7M7szszN7wPv5eOyDZeZzvOczO+/Pe96f9+fz0QFgIAiCIOoVencLQBAEQagPKXeCIIh6CCl3giCIeggpd4IgiHoIKXeCIIh6CCl3giCIeggpd4IgiHoIKXei3jJgwABs3boV586dwz///INffvkFN954o7vFIgiX4ONuAQhCC4KCgrBu3To8/PDDyM7Ohp+fHwYNGoRr166pVoder4fRaFStPIJQG0Yf+tS3T69evdjZs2dFzz/00EOsqKiIXbhwge3fv5/16NGDAWA33HAD++mnn9jZs2fZvn372OjRo7k8y5YtY4sXL2bffvstu3TpEhs2bBhr3bo1+/LLL9mpU6fYn3/+yR599FEufUJCAisoKGDnz59n5eXlbOHChW5vF/o0qI/bBaAPfVT/BAUFsdOnT7Ply5ezpKQkFhISwp0bP348KysrYzfeeCMDwNq3b8/atm3LfHx82KFDh9jcuXOZr68vu/nmm9mFCxdYbGwsA0zK/dy5c6x///5Mp9OxwMBA9ttvv7HnnnuO+fr6spiYGHb48GGWmJjIALBff/2VTZw4kQFgjRs3Zn369HF7u9CnQX3cLgB96KPJ54YbbmDLli1jpaWlrKqqiq1du5a1bNmSff/99+yxxx6zST9w4EB28uRJptPpuGOrVq1izz//PANMyj0zM5M717t3b3bs2DFeGWlpaezjjz9mANjmzZvZCy+8wJo3b+72tqBPw/vQgCpRbykuLsaDDz6IqKgodO7cGW3atMFbb72FqKgoHD582CZ9mzZtUFpaCsYYd+zYsWOIiIjg/i8tLeW+R0dHo02bNjh79iz3mTdvHlq1agUAmDx5MmJjY1FcXIz8/HyMGjVKw6slCD40oEo0CP744w8sX74c06ZNQ2lpKdq3b2+T5sSJE4iKioJOp+MUfNu2bXHw4EEujaXiLy0txZEjRxAbGytYZ0lJCe655x7odDqMHTsWX375JZo3b47Lly+rfHUEYQtZ7kS9pGPHjnjyySc5qzsyMhJ33303tm/fjiVLlmD27Nno2bMnAKB9+/Zo27YtduzYgX///RdPP/00fHx8MGTIEIwePRpZWVmCdeTn5+PChQt4+umnERAQAL1ej/j4eC7c8t5770VYWBgYYzh37hwAoKamRvuLJ4ha3O4bog991P60adOGff7556ysrIxdunSJlZWVsQ8++IAFBQUxAGzatGmsuLiYXbx4kRUWFrLu3bszAKxTp05s06ZN7Ny5c2z//v1szJgxXJnLli1jL774Iq+e1q1bs1WrVrGTJ0+yM2fOsG3btrFhw4YxAOyTTz5hFRUV7OLFi2zfvn0sOTnZ7e1Cn4bz0dV+IQiCIOoR5JYhCIKoh5ByJwiCqIeQcicIgqiHkHInCIKoh3hEnPupU6dw7Ngxxfk7dOiAQ4cOqSiR+niDjADJqTYkp3p4g4yAa+WMjo5Gy5YtRc+7PWSnoKDArfm94RpJTpKzocvpDTK6Wk57dZFbhiAIoh5Cyp0gCKIeQsqdIAiiHkLKnSAIoh5Cyp0gCKIeIkm5HzlyBHv37sWuXbtQUFAAAAgNDUVubi4OHjyI3NxchISEcOnT0tJw6NAhFBcXIzExURPBCYIgCHEkW+4333wzevTogYSEBAAmBZ6Xl4fY2Fjk5eUhLS0NABAXF4eUlBTEx8cjKSkJixcvhl5PLwgEQRCuRLHWTU5ORmZmJgAgMzMTY8aM4Y5nZWWhsrISR48eRUlJCXr37q2KsARBaEeHvgmI7dcb/o0aYdDEuxDQpDHadeuC1rG2G5uI0bF/H4S2CZecPqBJY/QYeQtGTJ+Cl37dgE5DBnLn2if0RIt2bWVdgxTado3H6NmPYtDEu0TTNI+KxNhnZiO6W2fe8WEPpWLK+29CbzAAAELbhKNj/z7c+dDW4TijN/LyBAYH4ckvMjHpndcwYvoUzP32C4x79ikkzZiKhYXb0Ou2JBWvrg5JS/7++eefOHv2LBhj+PDDD5GRkYGzZ88iNDSUS3PmzBk0a9YMixYtwvbt27Fy5UoAwJIlS7B+/XqsXr2aV+aUKVMwdepUAEB8fDz279+v+CLi4uJw4MABxfldgTfICJCcauNNcn54pBAAcH2VHiW+RsRU6XHE16SoBl+VNpl9S0A1dAwYdE1a+iLfGpw28FWQua4tAdW8/9VqS3O5ANDvqgG+0NlNY67/io6hwN+02cp1VXpE1ujxs381mK4uzS/+1TDq+O11zGDEMV++wrdGavsKYfamWCOpxAEDBuDkyZNo0aIFNmzYgOLiYtG0Op1tQ1luTWYmIyMDGRkZAICCggJRAaXgbH5X4A0yAiSn2niTnAgwfX/7vXcxauYjWLpiOYZOvh+AuAKxZmHhNjCd9PSPrVyC6K7xvGPmvAsLt/H+V6stzeUCQOKIEbh05qzdNOY6W8ZEY06OaVeujMzl+O7t921ktP4fAMakPYFB994JAPjivwsw4XmTC/t0aRnCoiJt0svBPAYqhCS3zMmTJwEAf//9N9asWYPevXujoqIC4eGm16/w8HCcOnUKAFBWVoaoqCgub2RkJE6cOKFIcIIg6jkChp9LsbVFJSFnHNHSuBUydLXCoYSNGjVCkyZNuO+JiYnYt28fcnJykJqaCgBITU3F2rVrAQA5OTlISUmBn58f2rVrhw4dOiA/P1/DSyAIglBGTPeuivLpDJ4fJOLQLdOqVSusWbPGlNjHB6tWrcIPP/yAgoICZGdnY/LkyTh+/DgmTJgAACgqKkJ2djaKiopQXV2N6dOnw2i0728iCKJhwhwP+WnKrY8/jMK8zbLz6fUG5yvX+NIdKvcjR46ge/fuNsfPnDmD4cOHC+ZJT09Henq608IRBEFoidAYoRT0Si13F7poPP/dgiCIekVwyxbwb9TI3WKYkKHcLZWxTuHcHZ4+J+VOEER94vm8HMzM+tj0j7vHU/VKLXcZbhmRayTLnSCIekfLmGgAro0eEUKnUxb1IitaxlK7e1K0DEEQDQ2F8YENCKXRMq4cQCblThBEg0XxgKrCaBmdCztOUu4EQVjhZkc4lA9Yyq9HhrK1dMvIsdxFXDHkcycIwsW40C0jouBctZKsUstdcbQMKBSSIIgGgJiC08mJRnECOQOqqkPKnSAI1+IBbhlXvTzU47FjUu4EQVhRjzWeFXLcMpaGtlJ3Dr88stwJgiC0wQWvCO6K5SflThD1jMDgYDz66UeydkTi4zplJBb3rYZlLAVZ9aigpCkUkiAIxXRPGoZ23bpg6KT7FJbgymgZ11UlhDv3dya3DEEQhEa4JJ7ecvUBFy5FQMqdIAgrXOiW8aKdmHiKWeFqklKOqwUpd4IgrPCEaBlX+dzrrwqsv1dGEIRTuNuodgVKl/xVBY3bl5Q7QRCCuCRgxYuW/FUbcssQBEFohKxoGZVDIbVe/peUO0EQgrjCqBZdW8ZV6w/ICnNX2CA0iYkgCE/CNW4ZF9ShAUo7HwqFJAiCqOeQz50gCFmo5dJoCNEyrqTq6jWX1kfKnSAIQVy27K5g5W6sWwy1OzsKhSQIQg5un/UpA1duGC2ESwZua+uwvi/kliEIwi24pI/wpo5Ioaw6C+XOWxWSlDtBEHJwWRghIQ3udri2IyPlThCEx+HtHZSl/JaWO2+DbJrERBCEHLzK5+5Fsip2o5gVvXV2GlAlCEIO3m71uhSt2sqi3JqqKgDAkV17tKlLBFLuBFFP8QqrWExGF3VQ1/69rHkd5vj2FbOe5R2/dOaMpvVKVu56vR47d+7EN998AwAIDQ1Fbm4uDh48iNzcXISEhHBp09LScOjQIRQXFyMxMVF1oQmCcAxZ8I7Z+e0PmpQr5HOvvHKFl+brBW8BAGqqqjWRQbJyf/zxx3HgwAHu/7S0NOTl5SE2NhZ5eXlIS0sDAMTFxSElJQXx8fFISkrC4sWL3bpPIUE0NLzCYq/F3aIajUaX1mcZCnm19q1Bq4FVSVo3IiICo0aNwpIlS7hjycnJyMzMBABkZmZizJgx3PGsrCxUVlbi6NGjKCkpQe/evdWXnCAIQchil45Opamw1m3eol1b7ntQWHNV6pCLDhLGbL/44gu88sorCAoKwuzZszF69GicPXsWoaGhXJozZ86gWbNmWLRoEbZv346VK1cCAJYsWYL169dj9erVvDKnTJmCqVOnAgDi4+Oxf/9+xRcRFxfHe6vwRLxBRoDkVBt3yHnCYESJrxGtq3XoUG2QlCcuLg4fHikEALSr0uOorxFR1TqU+pjUw+CrPpLK2RJQ7TC9ZZpC3xqcNfBV0OCrPqgCw7aAGl5ZarWluX4AiKjWob1AG1mmMdd/FQz5tTKF1ejQqcrApRt01QAddIL5LK+33GDEQV/T20K/qwZsC6iBjgGDrklrXyESEhIEjzsscdSoUTh16hR27tyJIUOGOKxIyGoQek3MyMhARkYGAKCgoEBUQCk4m98VeIOMAMmpNu6Qs/9dYzHu2aewevVqrH7pdUl5CgoKgADT9/feew+jZj6M5cszMeyh+wGIKxBrFhZuc5jeMs1D7/8f4gb2451PSEhAYHAQXtqaCwC4ZfRtOFdeoVpbmusHgM8+y8La196ym8ZcZ0h4Kzy34WsAQF5eHlJnPcOl69OnL5jRKJjP8noTxoxCyoumgdWhQ4fi5V83oKq6CgkJ/DaQSkFBgeg5h8p9wIABuP3223HrrbciICAAwcHB+OSTT1BRUYHw8HCUl5cjPDwcp06dAgCUlZUhKiqKyx8ZGYkTJ04oEpwgCNfjaW6dFtFROFdeoU3hCi9V1TbSaODBoc993rx5iIqKQkxMDFJSUrBx40bcd999yMnJQWpqKgAgNTUVa9euBQDk5OQgJSUFfn5+aNeuHTp06ID8/HxNhCcIQhyvGFiVIKNHXoeVclek7M3XpVFnqtjRs2DBAmRnZ2Py5Mk4fvw4JkyYAAAoKipCdnY2ioqKUF1djenTp7t8RJogCCd2C/IIZepZbw9aoHU7y1LumzdvxubNmwGYBlCHDx8umC49PR3p6enOS0cQhGycVRrudsu4sn45dQnFrsuuj7cqpKIiJEMB6AThxeh9DJj73ReIv2kgd8zdylkOUjoiz3iTcIAzTe4unztBEJ5LcPPmCIuKxNhnZtucU6oUhfJF3BCrqCzHlWlTrNbI6T//KasLKOGtCqmxz52UO0HUBwQUhGLXgUC+J7/IVFSWQgH4l6Oh5a7WW469yVCnjx3HsT37BM7Qkr8EQYjg7m3qXIFHXqHMTsHufSK3DEEQYghZjmq6ZQjw/Oo27W1P2Yucoz1UCYJwKa4ckBVScC4dEHZVXQJ6nLt08rkTRP0iMDgIcYP6a1a+V0TNiFivPNk98U1CxiQmXgglbZBNEPWfB95agIcWL0Tj0BDFZWihH0LCW6pfaD1Dbsdp1wVDPneCqF+0aGtag8ng6+t8YQK6RqlPt/9dY50UxkmsrkVL37TSSUwCJyWdEwyF1AhS7gThzdQqCC1cMDSwagcZza2Dzi2uJVLuBEHgis7NA5u2ldd998BORlW3jEaQcicIAgX+Ne4WgUOt3ZFciZCu5zoAN10OKXeC8GK0tAjrk1vmul7d0aRZqOOEUpFiuYv42usOmo5du3zF9pwKKN/biSAIz0EDF4qnhFKq0cdMX/4+/in7C+kjxysvRGK4o+UxBvFQyJrqaqx97W0c+PlX5TLZgZQ7QdQzgpo3c0u9anYGWnQszSMjVKtHkuvInESnE11DYcsnWYrqlwK5ZQiinpH48GRVynGbW8Za4bpYjpDwVg7T2PYJApa7xTFzW7pyLSBS7gThZpwyUj0kBlxLXL042t3p89UpqLb93NWOpNwJoh7gKYpYDs2jIsVPuupyBNrNWCMcOcRzxcjdQ5VCIQmi4eHMc+9RS/7K7GACmjQSKYZfjqvdQ9WVlQ7TSOlMKRSSIBo4nmp016dQSM0Ruoc6W5+7KyHlThAaUq43YvSsR90thiK80dUjF8XXKMlyr/1rofldOUGLlDtBaMhBPyNueuAeu2mcMeo8aTanXEVpigMXuHidzqVL4yrBut3F4ty5v264BFLuBOFmPNVAbrBuGbHLln2jbGeoUigkQRDS0LBn8BS3jKZ9jNJLlJOPQiEJomGihvJyhSLW6XTo0DfBTgLZJQq6lTykT5GHHbcMAAqFJIiGiBrKzBXruQ+5/278J+MdxA0eoHpdNni4hpfU3uYBVbLcCYKQi5aKw7rssGjTzlFNW7VQrQ5JPmgtZ+HKeN2wbI7A4CBMeuc1i3N2LHede8YvaOEwgnAzDXXcUqxj8hRfvz0ibohFxA2xvGMBQU34icgtQxD1ny7DhmD4tAcFzzmlyzRUhHKtTa3CMp21el3RWVzXsxte/nWDYL3uClcly50gXMADby0AAPz44TLumKphcV66nruY4uPX7WQ7yViHXe9jgG+Av+wq2nbtbPe8O97OSLkThJtQw6LzAg+GOB4o/CNL30NMz24qlcaNqKpUnjwcumX8/f2xY8cO7N69G/v27cMLL7wAAAgNDUVubi4OHjyI3NxchISEcHnS0tJw6NAhFBcXIzExUSvZCYLQENnuEAVKTPjtRd7CYX6BgXbPy3kDsavYZc/ArfvujgXeHCr3a9euYejQoejevTu6d++OpKQk9OnTB2lpacjLy0NsbCzy8vKQlpYGAIiLi0NKSgri4+ORlJSExYsXQ68n1z5BWKPGA+9Jyw+4g4TkW/FK/ka0jIlWVoBKVrXD5QfcgCSt+++//wIAfH194evrC8YYkpOTkZmZCQDIzMzEmDFjAADJycnIyspCZWUljh49ipKSEvTu3Vsb6QnCi1FTMXtDhIk1dkWWeD2dhgwEALRqH2OnLBlCKURQXMslfz01Wkav12PXrl04deoUNmzYgPz8fLRq1Qrl5eUAgPLycrRs2RIAEBERgdLSUi5vWVkZIiJs9y4kCEJFPEC3q9XB2JQjQS+6u3MLbmkb+28pkzvi3HWQMRTdtGlTrFmzBo8++ih++eUXhIaGcufOnDmDZs2a4d1338W2bduwcuVKAMCSJUvw3Xff4auvvuKVNWXKFEydOhUAEB8fj/379yu+iLi4OBw4cEBxflfgDTICJKfabAmo5v0/+GpdDMN2/2pU6oA+Vw3wV6idr4IhP6AGPgzof82HV2frah3aV+uhg/hbAqt1Dv0SYLv7UFS1DqU+deohvFqHch+G66v0aFNjaxcawbhyLK/TGrN83a8ZUOpjxD8Gvgrqf9WAGgA7asvqec2AJkwneM+LfGtw2sAQV6lHC6OwrWotl+U9aVWtQ8dqg41slpiv5bKO4Td/4V2ahOh71QA/6LDTrxp+TIfOVQaUG4w46GvklessCQnCS0LIKv38+fPYtGkTkpKSUFFRgfDwcJSXlyM8PBynTp0CYLLUo6KiuDyRkZE4ceKETVkZGRnIyMgAABQUFIgKKAVn87sCb5ARIDnVZmHhNt7/ljLP/zEHTVu1wKjbRuF8xd+Kyg8Jb4XnNnyNCxcvIGHACF6dX375JQakjMOxPfvwzsQpgvlnf/UpWndoL3hu2bLlGD4llft/zZo16DdhDF555RVs/+Jrm/Q+fn549ffNNtdpjVm+Byc9iGEP3Y/ONw/mnb9pyE3wb9IYz+flAAAmTpyIv4oPCt7z1P9LR9dbbkZaWhr2bvhJsD6Djw9e2/UzJ5flPfnmm28wcf7LNrJZYq6zRbu2SPvmc9HrsmZE0ghc+ucsZmZ9jIunz+DBGbORMGYUUl58lleuMxQUFIiec+iWCQsLQ9OmTQEAAQEBGD58OIqLi5GTk4PUVNONT01Nxdq1awEAOTk5SElJgZ+fH9q1a4cOHTogPz/f6YsgCMIWKe6I6G7iMdhiil24MulJnUJn5cN2tl43uWx0bg6FdGi5t27dGpmZmTAYDNDr9cjOzsa3336Lbdu2ITs7G5MnT8bx48cxYcIEAEBRURGys7NRVFSE6upqTJ8+HUajUfMLIQhvQ83wOI+ImnGj31uxT1tLkS1mqHrk2jKFhYXo2bOnzfEzZ85g+PDhgnnS09ORnp7uvHQEQdjHA3S6UsTeOronDcdfB/5wmO7hpe/i+t697KZxdE5L3D3ISzNUCYKQzY2334pr//6LwrzNqpd95wtzceH0P3bT9LotiVPsrkK2suZts+eBljtBEOqj0+vVdaW4wkq00E93v/wcAGBWl36aiBAc1tzu+a6JN0svzE1WvbtnqJJyJwgXE9wiDM9v/EaVslz66q9yVdI7N4HZnwrXYXcLbqqf1gUgCBfTPEr9SX3u9u8StvA6IE+doUoQhIfSQJW6HDeH/Y5Pw/bzhrVlCILwPpxVKpYTmDSpT2J6NZWjKxVt3OD+5krdsp47KXeC8GLs+p7VVmS1CmrC/Dnqlqs5Fu3gQuU+/rmnue/uiHMn5U4QrkaLB11IZ6ldjxZ6UYqMUhSyQqXtkt2myC1DEISquFypKN/MQn5NcqJldILfXYpVJ7Ynd6PmVZJyJwhvxo6y6n/nHS4UxHkObf9NeWYXuD3U7BiuXb6sWllikHInCA+meWQE9D4Gh+m8NhTSQu5D+cLKXejSZE0KsrcImUs28nDP2jKk3AnCQwlq3gzz1n+J5KdniqbxJJ3udLSMGyJK6jOk3AnCzYj5jwOCmgAAOvaTsk2lB2l5ObhgQJXnc3dHO9GAKkEQltRUm3YFEnPLBIU1x9xvv3ClSJriEteFlaJ1lTuL3DIE0cAZeM8ELCzcBv9GjWCsNm3ppjcIK/deo0a4UjRZ6PSOVYutYpUx61SOBW5Hgfe6LQkGH4lLbCnoCHQ6HYVCEgQBDJp4JwCgSbNQzmUhptxdTb/xY+wnsNBhfgEBsssXs26dVY6OQiF73zHaqfIdVG76S5Y7QajDwsJtuPOFue4WQxoiysts/UqxLD0tWkYv1Rq2xB1z9AEYfLXrPHV6stwJQnX6jLvd3SIoR6eDTm9SCqKhkB6m0C2RJJpVIjm6Xd766O5ZfsBUnUnFks+dIBoA9p7zsKhI7rtZMej1nuGWcYRWM0Gt/euNmgaj882DlZXlzKxYhT53AOSWIYj6TmibcDz6yYe8Y2JKw2y5M5EN5nU8g9TDrHgl8khUgOOefUpzUdTCnW4Z2omJIFzIzQ9OlJzW7HOX6oIICmuOf8+eUyKW6khRaDaxMmLK3aosv8BAhVLZl+uuF58RPTfuuacVLedgrs8dowmk3AnCQ6l7pRdNwH31DfDHCz+tw6/Za7QXTAzeq4R21cj1X0u1nHuPuU3wuMHHR/E6PTqdniYxEYS3ERYdhX4qLM7lG+CPxIcn20TFcFafDGXW+eZBTsvjLlyzibR8Rdvyumjltend53Mny50gFDJz1VIEBgdhm5PW8tDJ9yEheRQun7/AO865ZUR97h7mZ7dA0iQjiWvLOH2ZFgXcMfdJ2dnbdol3qm4KhSQIKxo1DZY+e9ANBAYHyc4jZIX7+vsDAPwC+RN/uAFVEatv0L13yq7fZShQaK4IF+x9h5Drxb6szsyXMLllKBSSaOBEd+uMRk2Duf9f/OUHpP5fuub1No+MQMsY5a/ezlL34FuGEtqPkQ5uEYbgFmE2xxuHhmghoiSkhEJqZcXaK9edbzh1kZCk3IkGzGOfZuDhpe/yjsW7wIc8b/2XmJOTpTi/WsrDuhy7/lqROj3mTUdKtIyNW0Yjv4zD7NopXp1eT24ZomFjfgDadOzgZkk0RkCBma06TpnXwlnuRoE8NTUaCKceysLc3bWgu3bKt66D1qwKUUi5Ex6BpyyOpQhnLTOzUrOx3MXj3I0ig6zuRcG66bxrlrZwmFxL2C1ruPPqJsudaMCY108xr2HeEBFzywhGy7jNypWImAK28su7U/HWob1bhnzuRIPF7CsWUmQzVnwoa2DV1T5OZ+szW+HWpegtBlQHpIxDUPNmlpU6VafWKGkTZ/Tf/W+8ZE8YB7k1bEuublLuRAPF7Japqbb1Jcf06Iqut9wsuaykGVNVk0ttBC04s1fGeoOLWsXQrE1rjH1mNlLffMXilOcpd55IUsWzzOTkgGrjkKYSK3Udep3ObTsgOlTukZGR2LhxI4qKirBv3z489thjAIDQ0FDk5ubi4MGDyM3NRUhICJcnLS0Nhw4dQnFxMRITEzUTnqg/mN0yRhUGCuV0BKqg1sPLXwnMZoCVp7w8T7fzkNT5WE3wkeq6EEt36+MPC9brWBZt3TKAh4ZCVldXY9asWejUqRP69u2L6dOnIy4uDmlpacjLy0NsbCzy8vKQlpYGAIiLi0NKSgri4+ORlJSExYsXQy9hyy2iYaM3iLtl6juMma7Z+jmxt1WdZ/iq7SA1zp1nuIsMqEqssu/4ZMQNHiAxtZIalBXtsaGQ5eXl2LVrFwDg0qVLOHDgACIiIpCcnIzMzEwAQGZmJsaMGQMASE5ORlZWFiorK3H06FGUlJSgd28pu7cTDRmDFw+oylG0QgqsLhTSSrnbUwoe6JaxxF2dj2+Av+1BlcZElMDdUzeMf+vkVBsdHY0tW7agc+fOOH78OEJDQ7lzZ86cQbNmzbBo0SJs374dK1euBAAsWbIE69evx+rVq3llTZkyBVOnmnyj8fHx2L9/v+KLiIuLw4EDBxTndwXeICPgPjmv6BgK/Gvgx4C+10xW/JYAvqLvfs2AJgzQQ2dXzgK/alypfaYGX3U8qcdcj5S0QvkGXjVAL6LMrK+hTbUOJ3z4j1yrah0qfBgiq3Uoqz134zUDrukYCv3qFEugEUioNMl4DQw7AqS7sMSuzVo+uViWWwWGbbUyJVwzIJDZtgkDw8+1abpU6lFmYDhrMF3z9VV6lPjaKtKulXqEGPXcPd/nW4MzBmG1dUOlHi2N/E7SUVu1r9IjokYv3hYMio37hGsG7PGrQWiNDh2rDSg3GHHQ14hW1ab/1SAhIUHwuORfc+PGjbF69WrMnDkTFy9eFE0nZG0IWSsZGRnIyMgAABQUFIgKKAVn87sCb5ARcJ+cLWOiMScnC6dOnULC8GQApn1QLdntX4Mtn3yOta+9ZVfOOTlZ3HICo+4cj5Yx7bBv42bRus31yL1uc77+/QegpqrKbhozWVlZGDzxLt6xnJwc9Bk7Gp+uXImbUu8BAIwbNw7N2oRj2kfvwGg0Qq/X4/jx43hktClv01YtMP/HHMmyil2btXxysSy3cUhT/O/n7wEAY8eOxenjZTbp9QYDXt/9CwBgxowZGHz/3Ygb2A8A8Morr2DC/Dk2ef7zn4dxuGAnd88nLXod8TcNFJTn2Wefxa71G3jHHLXVG2+8gV9WfSHaFkZjjeJ5GGPHjsXDH7+Hn7fuQPbz6UgYMwopLz6Lb9Z9g4nPvayoTEsKCgpEz0lyhvv4+GD16tVYuXIl1qwxrYBXUVGB8PBwAEB4eDhOnToFACgrK0NUVBSXNzIyEidOnFAsPOH9hLWNRJPmoY4TAg7j4drc4HgGq6UxMScnCw++vUBa3RIYfH8Kugwbolp5ALhrtnXL6Hnneee81OduV2yJg46eGCkkRp1bxgMHVAFg6dKlOHDgAN58803uWE5ODlJTUwEAqampWLt2LXc8JSUFfn5+aNeuHTp06ID8/HwNRCe8hbnffoH/bvrO3WKoQvJTj+OBt/idhSxlY+cZt154q24Sk8AMVk9UcHYWDrvlP5OQti7bNouFto+/SYV1hISiZZzsCA/t+E1xXndO1HLolhkwYADuv/9+7N27lxtYnTdvHhYsWIDs7GxMnjwZx48fx4QJEwAARUVFyM7ORlFREaqrqzF9+nQPnSpNeBJSFeT1CT01lsT1GM3RMlav/nWrQto+P55uvVrLlzR9ilAq3n/NoyIklWW3Xr1AWifbqiT/d3Ts30dRXiUbrqiFQ+W+detW0cYdPny44PH09HSkp2u/VCtRD5HwIBp8fV20a49ErGSe9tHb+PtYKb56+Q1p+WsvRSwU0my5855DmfrKv1Ej9BiViO1ffC0vo8rMWctffdMTNvm2F3LqLL4BAZ47iYkgXIKMB/u1nVtQLBBV4S6sRY/t1xsDUsbJL8fG5276y735OqEIx8x9AhPmz0F7F735iMkX1jZStbIE0wpoUkf5texUnvh8GZq2bOEWY4SUO+ER1G0GLWClCvC3SCicqQgPsurlYD2/Ry/ulpHramjSzDSg7RcQ4CClcnQyxwR0ph1J6v5Xw4JWoKgd72ilgvKnJX+JBo/54fRwn7IlTlt+5ku2UiKcv1bALePp0TJS20TK7k3W2Ou8BctwUGyziNaS6vU2SLkTnoEXKXNnsPd6buOWUdFydzkK5BMcDJVZlhbNoobbxiMHVAnCFdgsOaLggQqLjkK/8WPcMDCnTn2267lb78RkaeWqUqW6yI3UtHLLqNKOihYOs09wi+ZO5XcXpNyJesPkRa+jZUw0qkVmiwrRbcQwDSWShq7OL1N3zGLFRMHF1OTuRiSQPrxDe1llyEOiWwaO3TJyOn4t3FUD75ngfCGeOomJILRHfLVAySXIHJBrHBpif5MHqfWqtkG2mFtGYIaqCnU+9dWnTpchhmT5LDs0MbeMFZ2GiK/8mPjIZETcEGstjDRZ6hmk3AmPQFVXikQryeDrKylddLfOePGXHxAYHOyMVCZkKGrrbfaUDD7WVau95SjFCuelt0py+fwFh3n8Gzeyez60dTge/2ypw3JOHTnmMI2aeOR67gThDjwpGmT41AfQqGkw2nXvIpxAZ9psJHnOTOlr6PDy62r/WEfLWPncnWgSl49DSDXcLeTKelbsLaoujZRQTvOWjXW5bYXZlLlKmoBeDPncCY9AVeWjtiKTYHTFDeqPwRPvQkirlnbTdehzo/hJB75lZyx3VyN1JybLe3XpzFmHWfwCAx2mqamSsIyxlSWt5SxVd1H/roioHzihvCRb/QKvyk1btUBAUBPcePutWFi4zaGyBviDn9ZWozVi66cAQssP1LplhHoXD3TL8JHglrFKI2UXLt9Ax5a70Wi1drvgMuT8/6W66JRCoZBEw8UDDNGO/ftg6odv4eI/Z3Dy0GEAQMvrouUV4lDp2gnVE3HLyPHTi4vlggZW8GZhmU5sgUHLNAYJ66pb78MrRZSAJvZ9+d4IWe6ER2DPBSG3DKXreEz98C0AQFDzZpxVaalXRWWSodT8G4m7FSxzDrxnQp3lLrQkg4e7ZSTJp5NvuUsxAow1VuUIyWLVYTax2FVOE2hAlWhodE8aDh8/P+7/uphv7ZXXdb26O07EmLxXamfEtrjmPuNudxAKKa9oV7sFpOl2Ha+9RGWU+UZga7k7zuNnp9NVA4qWIRoUHfrciPtefxGjnngESrTi/LwczBaI1Zbqc7/v9RfFT8oQhzfQqWRgrjZ7QFAT3uHrenYzfeEmqCqfzdksoo18uWQi9GYxIGUc2naNl5TfWimL1CK7HMGtP63e7vS6+qcKyedOuA2zMgsJb2XjUpFi+TVt2QJNW7bQRDalmywoCeE012XeS9RMr9uSTOdr3TNy48gtCW8fI1suZzDLN/aZ2XbS8K+pbpkFx+Xaw0a5C3W41tEyBm2VO1nuRMNCwJesSny7Gh4dS+XObDsc09tGXYWcX1ziLEspFG3eCkA4RNDjQyGl3ASLKCNAZIE08Nv9xuRbHRZr7buXNKGKQiEJQj3MxoxOBxtT3WOUF2N1r/AWMg2ddB/3XaeD6CbX0hC+1vOn/ubXq9wr4xqcHPCVYrkPvHu8wzTWA6pCHa61IW0dhqo6tJ474Qz+jRpxmzJ4B3VKk9urw0O2z4uKv0FRPlVn1nI63U74pAchV7dbb9YhFi0z5f03EdwiTLIc1nHu1mv2CMpCljvhyaR9m43/bv7O3WJIhnNlWPqSue8KlJeK+s6/kSnuWUpXwx9QVU8Ic1uYy2/asoVFXZ6h3J/4fDnadukEwKoTkqDde40eiZgeXbn/7fmlw6+XPmZgrLb2uQtY7lYdidaWO22zRzhFcJiXrTvNDyIHYJoh6lFYyCh5Sr1MxJe6tXXH9LvzDumyuIDITh2R/PRMAHzrV4p8XYffxPtfUpy7BKwnQ0mJllGzUxaEBlSJBonVw2fw8XFKeamp+CwHVO1UqHrdOtS5LCzL5NwTCuvR1ILkjQko8LnbbWfp5dmGQkqIltE7nvnqbZByJ9xG3YAqP2rC4Ou9EbpqdixiyxKIHJJWpobuHJ7lrlktjmFSBlStBm/1GlvuFApJ1Gs6DujL+5/7weuslY7OvdrBEgkPZfOoiLqOyolJTLxqweoGVIU0uYe4ZQDL5REsrt2qw3YrgguHObbcK/48qpVELoGUewOlcWgIb9q/K3jwrQX8A0JrpkAF/6eqbhnHaR5fucSpOHchJejIwlaqOLV0y/CjZXToO36MZnXZF4T/r+DsUxvlbtueR3cXqidTbXVHdu0FAOz+Pk+9skXw3vdfwin+t2U9Sgp24v1J090mA0/RqOi3Fop80Ol0yl6NeXkkTIZR8ZVDJ+Bzd7Yel7lldDq3DY5LMRaMVr8FvYTVJp3B/Ns7fawUs7r0c5BaHUi5N2CuT+jpbhFMWMS5AyYlIVcRh7RqibCoSLt1mBX1I8sWI6BxY0nlMjB5gQ4qRstY+GXq5OHN/PIstF21Us7ibTr7/wO2lruENN4GuWUI12H9/AjEuQPKLPdJi163X7VFme1v7IGIuFg7qetgvFBIx+lVjXO3Z7krrEYLt4yQnKZZx6pXJQnbTU9s1RxjDFuzVmP39z+KpnGGf8pOWFeoavlSIOXuYpo0C8XND96rWnmdhw5GjHn1QDdj8PHBwsJtGHjPBIdp+024A2FtLSxtC8WgNxhkK/iAIGmWuBLkvEWou12gnTIV1qOlW8aXt7+p+94sfPz5Y0liVvlXL7+B797+AIB6k5i+XvAmAKCsqFiV8pyB3DIu5u6X5+OGgX1Rkr8TpfsPOF3eg2+/CgCa+/Ei4mJx4e9/cPH0P6JpAoODAAC3THsQv6z6wm554+c/zX03zUJXP1ach9IiBSZa2UsnZaq7DQLlGnztx/p7TCSKBTNWfMB9d6d8vv7+vP8FJzHV3i/zUgVqW+7WsfbumKFKyt3FmC1MvY93TZp4MjsTlVeuYm7vm0XTmB8Qse3SuHQOfKI6gwGQssmxDJRarIyBU9zDpqQ6rkfFTsqeW8YTJzHxlKrKyr3L8JtxQSdNdr8A/j6rYm4ZoC7eXchydyY23Xq2bUz3riIptcNhd7V06VJUVFSgsLAuLCg0NBS5ubk4ePAgcnNzERISwp1LS0vDoUOHUFxcjMTERE2EJtyDX+3mxD7+/mgvMBjL7RxkMYmkUdNg0XTc/1YLSOkVKQYHeRQ7qese8KhOjhcT4/ncJdYpmsreJCZJJdvS/sYeCnPKQ23Dvf+dd2C3v5TNPADfAMeWu/m+mpcZVnv5AeuVKd3hOnWo3JcvX46kpCTesbS0NOTl5SE2NhZ5eXlIS0sDAMTFxSElJQXx8fFISkrC4sWLtV9Kk3A5Y+fNwiMfv4eWMfzNo/XctnB1P+xHli3mvgutlVJ7gv+vXu+JwSB24TYZUfB7b9KsmeBxcxOoablbLlWsKW68gT5+frw3Y6H2M9Za7GYlrFM5FNJ6ZUp34PCX+PPPP+PMmTO8Y8nJycjMzAQAZGZmYsyYMdzxrKwsVFZW4ujRoygpKUHv3r3Vl7qeExgcjIfe/z/0vG0EdyzlpWel7fnpAsy7+ph97Gb0tbvZWFotrTu0t8kvuISt5XcFu+I4coEonvTjxKu51Do79L1RrADTHwGr0qN87kJ9j5unGN/z8nzuu6BVbm25q9yeUtam1xodJASQRkdHY926dejSpQsA4OzZswi12C38zJkzaNasGRYtWoTt27dj5cqVAIAlS5Zg/fr1WL16tU2ZU6ZMwdSpUwEA8fHx2L9/v+KLiIuLw4EDzg9OaolZxl1+1bioB7pfMyCYCf+gfvOrxuVa/Tb4qg+MYPgloAY6Bgy6xh8m2RJQzaWz/O4Iy7QMDCcMDC1qdOgW10mwLS3T7/arxgU90O2aAU0truGKjqHAvwYBRqB3pQ8vHwBOfvP1mAmp0aFtjQ57/UwPWsI1A3wYsC1AmvUz+KoPdvhX45qd53PAVQMMtQrHUiZH9LhmQKmPEacN9h+TzpV67PPjv4oPvGrgXadcWtTo8LdVvW2rdWhXbcBZvRGFfspWUTT/PuS0gz2CjECPSh9eefGVelzQM5T6SFdylr9hNTBf5z96I/ZbtVXnSj2aGfWoAsO2gBq0r9LjsC8/TXi1DuUy5AfAlSOUV8pzqYSEhATB46rWZm9U2pqMjAxkZGQAAAoKCkQFlIKz+V2BWcZHP/0I7bp1waTJk3Bszz5eml6jRyKoeTOMnjWDO5aQkACDry9e27kFVVWVSEjgR8UsLNzGpbP87gjLtOHXX4en1qzE97//Bj/oBPNbpp+x4kPE9OiKhx56CEd37+XShEVHYe66bJSWlmL6bXfy8gHg5Pfx88Orv2+ua5vfCpD+4TI88vF7AIBx48fj8rnz+N/P3zu8DrNMz/zwFZq1aS2aZtCgQai6eg2NQ0Pwvy3rJZULAKmpqRgx/SHEDepvN93jM2diyuL/4x3r368/Xtv1s+S6rNmwYQO6Jw3nHVuSsQQ/LF6CDn0T8J+MdxSVa76/lvfGGQoLCzF14lReeU8+OQttu3bCLVMflCWXWjKZywOAuMED8NB7b/DOPfbY4/hj63YEBgfjpa0/4I033sAdc5/kpVnz9Rr0k7mEwsKFCzEm7Ql8tXo1+t81VlAeNSkoKBA9p0i5V1RUIDw8HOXl5QgPD8epU6cAAGVlZYiKiuLSRUZG4sSJE2LFEFbckz7fcSIVsO6EzaswCg1+2i+n7vuwKam49bH/AJCwLrfAlno2USYauR1mZn0sL4NOJ21qupAR4+w12A2FdK5otTFvbmKJu10zZgRXfOSiZUxvVtb3+IMpj6HbiKGK63QUMeYKFI125uTkIDXVFBaWmpqKtWvXcsdTUlLg5+eHdu3aoUOHDsjPz1dP2nqE1B9+aJtwpC58mXesx8hbMHSy8oExZ2N6OeVtoWHMih2wjfGtq1hn+Yd33FK5K5nEJNXnbs+6F84HdOzfR1YeqTIpys8d8wzFaSZ9B38hLE/ofNp26YTx8+dICoW0aWsnZ5SqtfGIMzi03FetWoWbbroJYWFhKC0txfPPP48FCxYgOzsbkydPxvHjxzFhgmlGYlFREbKzs1FUVITq6mpMnz7dI3owj0Lmb2bcs0/ZuAQmvvY/AMDGpZ8ok8Hid2zw9cVNqfeYRJP4g3YUGSJ2z318fU35BCx38w5DpnK10AzKyvSXuAaNcJXaaTiPGlAVwgPke3jpe/ALDMCfO3fbnLOexKQXGsR3/5ioUzhU7vfcc4/g8eHDhwseT09PR3p6unNSERyOfL1KsJxFedMD96DnqBF2UgtQtwy78GmHHbptzm6Jda/Aer1Bti6WYrkHNNFyiQKhOp0r0+41eYDytIcndD5mxW3wEVBz1pa7yiHb7ticwxqaoepNqPTAWD54ShSekFvGkogbYvHqzi346iXbxbweW7kEp4+XWglk9a8WlrsO6H/XOPnZnBDFaQXnJT73dt26CBzVwTcwQOC46zBPphMaM6lzy5gnMekFz3szNMOoHhIWHWX3vKXytPT9S52abv7hBzRpjLvT56NxSFObND6+vhg9+zGb49Fd49HrtiSb4zz5dHr5g3GaaTtnytXA51530qmytV5rXacDhtyXomkdjjD/Tu0pd6OIcrdM4600GOXe49ZEhIS3ckvdT61ZiSM+toOMNz94L+b/mCO9IIk/trnrsu2eF1MafgEB0hR8bZK+45Nx4+iRGD5NJNxNoryRcR35cjQKVH3GoOma5T+szryuOz+gKl6ms5Eo4555yqn8DvGAVwvzwL55rMcSa8t95Iypovm9lQbhltHp9Zj46n9x9mQ5Xkq8w3EGJ/Hx80P49ddxy36GX38dSgUc1bc9OUMgtx1Uc8sIK6xW17VDaZVjBWjuAAy1CthRdIwjrMPoHvn4PfyxdbukvFxVEtaWUWKJSV4+Q6DsNh2vF01+rrzCsbEhqN3tnJOBXuNNyOV2bLkfyAxRlYDZKjcIKHdHhgczGmGs9m7l3iAsd/MPrWmrli6pb+y8WXji82UI0bi+7iOGKcto57mrMEiIbuJed00KQgsLx3ozbTmY96m0RGexE5McBKMoJBJh9UZiSXVVlaIyzZ2Ys28FPj4CCk9NZMh3cFs+fngvQ3URzFa50ICqo46ewbn1YTxhQLlBKHdXhwRH1q4e2Dg0xPakM248qx/kfW+8JEsm82JK9lwNV/RwuIYN58v0MVvuIh2CK32WFve4plp4CrsScSS7ZQR3SxL+4RmNRklrjwi+jYjNFZCJwcMsdy3glLvAtTpqf2Y0er1bpkEod84N4aSy0fsYJM1WNCsX1R8ghQ9My5hoPPH5Ms4N5OjBG/XEI3bPmx8Ms0Xk5+aoCGuEo22UWe5SlbucW8OMRmmTXOx1GE4qT8HwQBWovHIVgGdMsXLGLQPGxI0WL6HeKfe2XToh8eHJvGPcA+rkA/HKjo2Yt/5Lh+k45a7RAySXRk1N0SxtO3cC4Fi5C4e21WH2uZs7usET73JWRFURXHVSpyz6wRnLXez3JlWO0DbhNsf8GzfCnS/MRWBQkEAO6QgqPBXglnv2IMtdcEDVwSs0MzKvt9w9Q/uoyOOrlgIAct9fyh1T63fm4+eH0Na2D5w1ZuUuaU0S1K6tIsG36+Pri6haBS0Ho8X6GU2ahaJ7kkJfPWCKjLETYmaJO7YWA4Q7L6VuAmf2IxCrkxmNkmZuCxkHA+8eDwDoM+52xXLZk81ZuPXRZZTf2GKzHzUxd8xCu545enNiYGCk3D0fRftaOkFN7WCZVLdMykvP4cbbR0pKO+Q+eVay3mBA8tOPm777GPDgO6/yLPNeo+3HnFszcsZUFG3eCkB7v60ceMpETKlqGAoZcUOsQGbhtMYaaW4ZTdcnkaHba6qqJd9rJTsbRcQJtJ0KmNcREg6FtJ+XGY2o8XLlXu/cMkKI/dAiO3VE+o48NGkeqmp9dW4Z/o/KevlWM1IVuxLa9ejKKXO9Xo/QcP6bR3CLMIdlWFth9iaHuAtLK1fofge3DEPfccmyy5VquVsunMbJIaJBLXeqskf49ddJSmfNiYMlivKJIadTrJvO70G/DUWhkMypztUTJkA1DOUuYskNSb0H/o0aoUMfeessB7cIs9shmH111tbOfa+/yH2P7tZZVp2KsfiRyVXGMT27Ycj9d9tarx6m3F/fs5UXmSQ0ZjB79adodV072WXf+rit0pZKl2FDBI87qzgcIkGxyHGbyJHV/Nt3JoRUbYRDIR24ZRjz+jh3z3mvVhmdXi+4boTOYjKL+Qcu90F7fuM3AIBZXfoJnq+pqvW52xlQvf0p26n5Uuhxq7xNxy2vzeDjI8sKm5H5AQDg51X8Ga/WA6piNAqWtz68UrTcp9eZWc1imyIzJi0U0lOQI6v52dJqwFYJPv5+NsccRcIwRm4Zj4W3foqFlfLG3l/x2MoltWlMlx/brzf6jhd/ZW8ZE42RAq/dQWHNBS14o51oGa1e18QH7+rq0+n1ksMBg5rXbdps414wP8ACA1WEY5iRwSjRNaMZcix3GbKaF+uaMH+ObJG0Qmiyn6MBbdPblXcr9/pruev0AGpqv/N/yNFd43nH+4wdjT5jR2PfT1sQ3bUz9v/E3xpt2kdvC1pwL/y0DoCtBV9da7n7+PrY1B3To6vCK7KPWLSNpRKR40Z5YdO33Pe4wfxlh839g7vW6vF2Goc0xZm/vGeHMnmWu3fEhjuKhGGMocbL3TL11nL3CwzkvotFPFgr3qkfvIVJ77xmMynHsiwpmC33mB7d8MbeX2XlVYp5KQBrLB9MpX7QMWlPWJXpHQ+wJ+NOt8zCwm3yfO4Cb3vfvf2BYFpvmfjj0HJn4uMix/bu10Ik1am3yn3sM7O47+Yfso1v1uoHbg5nsw6dlBsTbPbVORuLLAfrawvv0B4AXxHrDQZFbiHr2H5PiATwRs5X/M191zbM0fHvtXXt78MeOa/XbsAtcL/F3jy8xXJ32AnZGVBd/45wx+ZpeL1yN/j6YktANRL/M4l3nBcxIfJjF9w4F0BY20gkPz1T8aBQ5ZUrivI5g/VEjae++hR+gQF44vNl3LGmLVtImoRFaMP5vy2UuwYTvP4p+wsAcO3Sv6qUZ5ZRqDOvulYpmEfuW64r+evAQe67I386Y8zrtwj1euXuG+APABhstTEAb0BV1C0jfPzJLzIx+L670Kt2+zlHN7nP2NF4as1KrjMw70nqSjoNGWBz7IE3X9GkLsst8Qjp1FTWrQSphVum6uo1Uz0iC6cpRUi514isamk5EK+EbV987VR+e1iOS0lbOEzddnQ1Xq/czQQGB/GUuKXVrRew3GP7JSD+5kF2yxSatizEHfNmIfz665D4n0l47se1EiVWl3tfecHmmDPL5hLy2PntDw7T8JSuBq4t1S1Ns4hWsh74+VfUVAsr98O/73KqytPHSh0nUoil61KKW8y8JpO34vXK3WARAWIZVxwc1hxjn5lt+kdAuU/76B3Hhet0mPrhW4LbyFlinrgxfOoDmq/hTngmUlbGtFyIyl4opOW6SHJoHtlGUT4xzJ2RdSTWkkdmofLyVcE8B2qXplCKliGiltFiRgedq9FoFFy2APCeMSevV+72JgoNSDFtiKx0w+WQ8Jbo2L+Pw3TWOwkRDQ+/wEDOLSIGb1KMHf1wVaHPXO3fodn1IhRCe3RPoXAeJyf+SHVXHfhlm+yydTIsd2Zkkt/cpfD5/HTVypKKVyv3pq1acLHmgPBaSE2ahypeAe+WqSJ7gxKasO7/3nW3CIqJ7dcbVy5dsp/IwuITUy4HtxeoqlTk8O7907Bx6Qruf24BPJEwWyEqLzsXTPBX8UHHiQB8PEP+HrCWRp7DqB7GFO2f+/u67xFsBDZnfsbbEexPJ91VSvBq5W69VOgjyxbbpPnvpu8wYvoUF0lEmDdrkMunT8/Hr5+vUVka13L2RLnd85aDkEIW6vzBI7F0+my3rdlzZNdeXL5wkfu/mltGQ7o8l86cdUqGP3+TpgSNNTX4Kn2h4nqYhOUHDAruw+XzF9C90gfnyiuQ8XDd/JDTx8tkl+UsXq3cpc7yu3G0dqsu1jd8nHQnKn1LOri9wOvX8mjaqoXouZ3f/oArF+vcLUKW479nz6G6slKRUgH4kSbfvLFIcr4rFgrd8v5xlrvKm86sff1tVcrZ+lndxjkX/znjMD3P5+7ALWOsMYq6fKX63K/9e1lSOq3wauXu7sarjzirXpVanTXV1W4NPfvfcOcnnNlbPnn3D3ncgOq+n7bY3SDb3jiSPf49d577vilzFfd9T+5Gu/leTByDfldt99etrpS/ibfYIKQlW1ZkYcOHdfMv5Eb5/PKZ7W5o6SPHc2MVf4tE3Jg7qU3LV/E6tH0bN/PSfbPwXZwrr7BZhsQRGz5a5jiRC/Fq5e4to9beRGNnLXeFg9c1VVW8V+W9G34STHfiYIndwbTd3//Ifd/30xbJ9VvOHlXKsseEF8s6uqeQpyj+2LrDrvvq12xl7il7fnx7XPv3MnxrR6wqDh/hjpfk/45je/dj/aIPJcsgdRKTuXM7uC0fLyfewTt3bM8+AED+mnU2+QBgX95mm2OVV67g5aSx+OChR7HgtjsF85kNj5+Wfco7vuzxNN7/m5avBACU7j8guvKrEN8v+khyWlfg1crdnVxzcuDIE3lj3H0INdoq57fvfQgr574gqYyflq0UPL7ovml289VUV/M668wn5wmme3/SdKx48hnRcv7YuoP7frjA5L89uls4ssOazSs+AyBPuV65aBpEPXXkGIoFOh0/Brxbe+19xo4GAHS6aSCqroor9wun/sau73JFzy+aOJX7/u7907jIFbE9P+UsdbD/p59x4mAJLl+4gMorV/DOvQ/hxB91m3/88esOmzyXz18AAJwrr8Dv3/6AHatzRMs3K+yiTb8AANa/+xHOVZzipVk8aTr+785UfPG/Bdyxl5PGIqTG9NsUMyAun7+AQzt+AwBc+Pu0zXnz/gpKFgS7+m+dS02n0+Hz516WlO/a5Ss8g8OVkHJXiH8j102zPnnoMDZ8tAxbPvncqXI+eeo5bPz4E/F6RHbwOb53P47t3id47uxJ/iDid2+/j1ld+tm86h7dvVfwNbdo81Yc3F7AreOxJ3cjlj8xFwCQ9eyLNumvXLyEyitX0LxGh9/Xfc8df3F4MjKfnIf8r9fh8+de5j1Qf2zdLii7NTmvv4NZXfrZzGv4NXsNFoy+C7+v+95GaXz/rsmqvfjPGUHlWo26N8zzp0xvB75+fpxCFFuE6m87A3DXLJa3uHLxEg7+mg8AOFfOV5ILRt+F/5uQisIfN9nszmRWVrt/yOMdZ4xh4bj78NyAEXXHLDqHj6bNtJHn+3c/wvIn5uLFW8agpqoK2S/wZ0Yf31eEzCfnYXbX/vh8vkkpnvjjEGZ16YfjAtdfXVmJvw4chLG6hhugPfPXSe68lG0z/zt0tM2xT2Y/h5L833H14kWbc5ZuGmsW3TcNryXfzf2vNxiQ//U6PFvbRqX7D+DFW8YI5p3XZyg+eeo5h/Jqgdcr99UvvS543PLHYOb4viLuu9nicoS1kjJjfijM63lYYxlSZs2XL76Gt1ImiZ4389PHptfHbxa+i+8XfYS1r72Fyxcu4MDPv+L1O+4FAGzNWs3Ls3xmmk05Zop/2YZv31yMT+c8D8A0gGf2x5rX3fBnJqvIsq0A03UKuUpeSxZeauGchZvDrBBXzH4W/x12O7Z/aZrF+9OylVg6YzY+nFK3ccmKWc+g8MdNAICCtd/xytz/08+coomvMmDV3P9y56oqKzn58r9eh0+eeg7bvliDjR9/wvM/S2HX+g28/3eu+x5/Hz2OVXP/ixdvGYO8JSvw7IBEzOrSD4W1LoItn2Rx6fOWrMC8vqY1xCNr6qzMFbOeBWCy8r9/NwMbl67Aew88LDi79aePP8HGpSvw36GjsWre/wQ7OgAoL/kTuR98jJdG3FEXYFA7KPr30eP4q/ggLp+/gIXj7sOvn3/F5asoOYL/Db+d14ZimH3ixb+IdJI6HXfPzCx/Yi4+fvQprH/3I2TOnIu9G35S5EZ9ZdQEzB9k2ufX3JJSXX/vPfgI7/+D2/Lx/uQZgnLsWPONaDlHd+/F+Yq/ueicvw78AQC4cuECXrxlDN6ZOAXnyiskyeRqmLs/BQUFTuX/7vcdTG8wMABsTNoTbOA9E1hQWHPWcUBf1mPkLWzko9NYTM9uDAALCGrCHv9sKWsZE806Dx3CXtj0Les8dAh78ZcfWNq6bNb+xh4MAGseFcnluX/hy6zz0CEMAGvTsQNr1T6GAWDxNw1kfoGB7O70+aznqES2sHAbW1i4jcX2S2A6nY77//anHuO+P/Zpho38vgH+rPPQIaxr4lCm9zGwXrclsa6JQyVff+J/JrGFhdvYtIx3mH+jRuzJ7EzWpmMH1jwqks3+6lPWol1b1nFAX8G8Pn5+bObny1i77l0ZAJZfkM+6jRjGk98yfevY9uz+hS/zzpm/R9wQW1euvz/rMmwIi+2XwBqHNOWVERYdxR5ftZQFBgc7vLaB94xnD7y1gD2+ailr07GDzW8mIKgJixvU32E5Ccm3sgffXsDadevC7k6fzxYWbmPxNw9iPW8bwR58e4Fw2/j7c/ddrd9256GDmV9ggOxyLO9HcMsWgvemWURrtrBwG0v8zyS7ZcX2680ah4ZIfgb9GzVis7/6lEXExfKOP/rJR2xh4TbWoc+NTrWR1M9Pv+WzWV+uYP6NGvF+S5GdbhDNc33vXmxh4Tb26s4tdsvucWti7bPb22k5ndVnKtalTaUjRoxgxcXF7NChQ2zOnDmaNoYrG9PeJ7hFGO/B9QsMYMEtwhgAtr0gn42e/SgLaNJYk7qbR0ao/mMJDA5mjZoKK+DA4GBOaQc1b8Z74Fzxceae6/R61iyitcfLaf0ZcPd41mPkLbXtHyR4b5pFtGYGHx+Xydk8KtLj73nj0BAWENTEZdfiKcpdk52Y9Ho93nvvPdxyyy0oKytDQUEBcnJycODAAS2q8xis/bGVV65yUREG6GTFHstFzD3kDFcuXJB0TkqMsSfBjEZBt52nYxnXLeYjdvV1/VPq+sk5cvn37DlJ6bzhWuSgic+9d+/eKCkpwZEjR1BVVYWsrCwkJydrURVBEAQhgCaWe0REBEpL6yYSlJWVoU8f/gJcU6ZMwdSpppCu+Ph4FBTYj8W1R1xcnFP5XYE3yAiQnGpDcqqHN8gIeJacqvuBxo8fzzIy6gYOJ06cyN555x3NfFSe4nP3dhlJTpLTkz/eIKOr5bRXlyZumbKyMkRFRXH/R0ZG4sQJ79ntnSAIwtvRRLkXFBSgQ4cOaNeuHXx9fZGSkoKcHPFZawRBEIS6aOJzr6mpwYwZM/DDDz/AYDDg448/RlFRkeOMBEEQhCpootwBYP369Vi/fr1WxRMEQRB28PrlBwiCIAhbdDCNrLqVU6dO4dixY4rzh4WF4fRp21XgPAlvkBEgOdWG5FQPb5ARcK2c0dHRaNmypeh5t4cOOfvxhhApb5CR5CQ5PfnjDTJ6kpzkliEIgqiHkHInCIKoh9QL5f7RR561vZUQ3iAjQHKqDcmpHt4gI+A5cnrEgCpBEAShLvXCcicIgiD4kHInCIKoh3i1ch8xYgSKi4tx6NAhzJkzx62yREZGYuPGjSgqKsK+ffvw2GOmfUGff/55lJWVYdeuXdi1axdGjhzJ5UlLS8OhQ4dQXFyMxMREl8h55MgR7N27F7t27eKWJQ0NDUVubi4OHjyI3NxchISEuFXG2NhYrr127dqF8+fP4/HHH/eItly6dCkqKipQWFjIHVPSfj179sTevXtx6NAhvP322y6R87XXXsOBAwewZ88efPXVV2jatCkAU6z05cuXuXZ9//333SqnkvvsDjmzsrI4GY8cOYJdu3YBcG97WuP2eEwlH71ez0pKSlhMTAzz9fVlu3fvZnFxcW6TJzw8nPXoYdp/tUmTJuyPP/5gcXFx7Pnnn2ezZs2ySR8XF8d2797N/Pz8WLt27VhJSQnT6/Way3nkyBHWvHlz3rFXX32V2wpxzpw5bMGCBW6V0fo+nzx5krVt29Yj2nLQoEGsR48erLCw0Kn227FjB+vb17Sv7XfffceSkpI0l/OWW25hhtq9hhcsWMDJGR0dzUtn+XGHnEruszvktPy88cYb7LnnnnN7e1p+vNZy97TdnsrLy7me+9KlSzhw4AAiIiJE0ycnJyMrKwuVlZU4evQoSkpK0Lt3b1eJayNLZmYmACAzMxNjxozxGBmHDRuGw4cP4/jx46JpXCnnzz//jDNn+NsKym2/8PBwBAcHY/v27QCAFStWcHm0lHPDhg2oqakBAGzfvh2RkZF2y3CXnGJ4Wntacuedd+Kzzz6zW4Yr5LTEa5W70G5P9pSpK4mOjkaPHj2wY8cOAMCMGTOwZ88eLF26lHtld5f8jDHk5ubit99+w5QpUwAArVq1Qnl5OQBTJ2WezuwJbZySksJ7aDypLc3Ibb+IiAiUlZXZHHclkyZN4i3sFxMTg507d2LTpk0YOHAgALhVTjn32d3tOWjQIFRUVKCkpIQ75gnt6bXKXafT2RxjjLlBEj6NGzfG6tWrMXPmTFy8eBHvv/8+2rdvj+7du+PkyZNYuHAhAPfJP2DAAPTq1QsjR47E9OnTMWjQING07m5jX19f3H777fjiiy8AwOPa0hFicrlb3nnz5qG6uhorV64EAJw8eRJt27ZFz5498eSTT2LVqlUICgpym5xy77O72/Puu+/mGSCe0p5eq9w9cbcnHx8frF69GitXrsSaNWsAmBZFMxqNYIwhIyODcxe4S/6TJ08CAP7++2+sWbMGvXv3RkVFBcLDwwGYXh1PnTrlVhnNjBw5Ejt37uTk8bS2NCO3/crKynguEVfKe//99+O2227Dvffeyx2rrKzkXA47d+7E4cOHERsb6zY55d5nd7anwWDA2LFj8fnnn3PHPKk9NXPoa/kxGAzs8OHDrF27dtyAaqdOndwqU2ZmJnvzzTd5x8LDw7nvM2fOZJ999hkDwDp16sQbHDp8+LDmg5WNGjViTZo04b5v3bqVjRgxgr322mu8AcFXX33VbTJafj777DP2wAMPeFxbWg+YKWm//Px81qdPHwaYBtZGjhypuZwjRoxg+/fvZ2FhYbx0YWFhnFwxMTGsrKyMhYaGuk1OJffZHXKa23TTpk0e1Z4WH80K1vwzcuRI9scff7CSkhI2b948t8oyYMAAxhhje/bsYbt27WK7du1iI0eOZCtWrGB79+5le/bsYWvXruX9cOfNm8dKSkpYcXGxpqPm5k9MTAzbvXs32717N9u3bx/XZs2aNWM//vgjO3jwIPvxxx+5H6I7ZDR/AgMD2enTp1lwcDB3zBPactWqVezEiROssrKSlZaWskmTJilqv169erHCwkJWUlLCFi1a5BI5Dx06xI4fP879Pt9//30GgI0dO5bt27eP7d69m/3+++/stttuc6ucSu6zO+QEwJYtW8amTZvGS+vO9rT80PIDBEEQ9RCv9bkTBEEQ4pByJwiCqIeQcicIgqiHkHInCIKoh5ByJwiCqIeQcicIgqiHkHInCIKoh/w/KUGnEpNJ9hIAAAAASUVORK5CYII=",
                        "text/plain": [
                            "<Figure size 432x288 with 1 Axes>"
                        ]
                    },
                    "metadata": {},
                    "output_type": "display_data"
                }
            ],
            "source": [
                "es = [e for e in range(len(scores))]\n",
                "plt.plot(es, scores)\n",
                "plt.title('Scores')\n",
                "plt.grid()\n",
                "plt.show()"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 28,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXkAAAEICAYAAAC6fYRZAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8vihELAAAACXBIWXMAAAsTAAALEwEAmpwYAAA5N0lEQVR4nO3deVxVZf7A8Q+LwIXLZVWRVTMXXKFCLS0tV2yxbHMpLR1zyrbJmhir8VfTOFrT2DKjzZiV+9KYozVp5lbquJAhSoACgoCyCrLvnN8fl3sUAVkv93L5vl+v++rec59zzpcbfu/Dc57zfawABSGEEBbJ2tQBCCGEMB5J8kIIYcEkyQshhAWTJC+EEBZMkrwQQlgwSfJCCGHBJMmLDm/UqFHExsaaOgwhzJIkeWF0iYmJlJWV4eHhUWt7REQEiqIQEBDQquMfPnyY/v37t+oYjfniiy+oqKigR48eRj2PEG1NkrxoF4mJiUyfPl19PWjQIDQajQkjajpHR0cefvhh8vLymDlzplHOYWVlZZTjCiFJXrSLdevWMWvWLPX17NmzWbt2ba02Op2ONWvWkJmZSVJSEm+88QZWVlbY2dmRm5vLwIED1baenp4UFxfTtWtXRo8eTUpKivpeYmIiCxcuJDIykitXrrB582bs7e3V91977TUuXbrExYsXmTt3Loqi0Lt37wZjf/jhh7ly5QrvvPMOs2fPVrdHR0dz7733qq9tbGzIysoiODgYgOHDh3PkyBFyc3M5deoUo0ePVtseOHCAd999l8OHD1NcXMxNN93EU089RXR0NPn5+SQkJPDMM8/UiuNGcdvZ2fH+++9z4cIF0tPTWblyJQ4ODjf+nyI6DUUe8jDmIzExURk7dqwSGxur9O/fX7G2tlaSk5MVf39/RVEUJSAgQAGUNWvWKP/5z38UrVarBAQEKGfPnlXmzJmjAMrq1auVd999Vz3mc889p+zatUsBlNGjRyspKSm1znf8+HGlR48eipubmxIdHa3Mnz9fAZSJEycqaWlpyoABAxSNRqOsXbtWURRF6d27d4Px7927V1m2bJnSrVs3paKiQgkODlYA5a233lLWr1+vtps8ebISExOjAIq3t7eSnZ2thIaGKlZWVsq4ceOU7OxsxdPTUwGUAwcOKBcuXFAGDBig2NjYKLa2tsrkyZOVm266SQGUu+66SykqKlLP1Vjcy5cvV3bs2KG4ubkpWq1W2blzp7JkyRKT/7+Xh1k8TB6APCz8YUjyb7zxhrJkyRJl4sSJyp49exQbGxs1yVtbWyulpaVKYGCgut8zzzyjHDhwQAGUsWPHKgkJCep7hw8fVp588kkF6k/yM2fOVF8vW7ZMWblypQL6L4trk1/v3r1vmOT9/PyUqqoqZejQoQqg7N69W/nwww/VffPz8xWNRqMAyvr165W33npLAZTf//73ytq1a2sda/fu3cqsWbMU0Cf5t99++4af2/bt25UXX3yxSXEXFhaqXxCAMmLECOX8+fMm/38vD9M/ZLhGtJt169YxY8YMnnrqqTpDNZ6entjb23PhwgV124ULF/Dx8QFg//79aDQahg0bhr+/P0FBQWzfvr3Bc6Wnp6vPi4uL0Wq1AHh7e9ca2rn2eX2efPJJYmJiiIyMBGDDhg3MmDEDW1tbEhISiImJ4f7770ej0fDAAw+wceNGAAICAnj00UfJzc1VH6NGjap14fb6c0+aNImjR49y+fJlcnNzmTx5Mp6eno3G3bVrV5ycnDh58qR6rt27d9O1a9cb/myic7A1dQCi80hOTiYxMZHJkyczd+7cWu9lZ2dTXl5OQEAAMTExAPj7+3Px4kUAFEVh69atTJ8+nYyMDL799lsKCwubHUNaWhq+vr7qaz8/vxu2nzVrFv7+/qSlpQFga2uLp6cnoaGhfPPNN2zatInp06djbW1NdHQ0CQkJgD4Jr1u3rs64+rUURVGf29nZsW3bNmbNmsWOHTuorKxk+/bt6gXZG8WdnZ1NcXExAwcO5NKlS834NERnID150a7mzp3LPffcQ3Fxca3t1dXVbN26lT//+c9otVr8/f155ZVXWL9+vdpm48aNPP7448ycOVPtMTfX1q1befrpp+nfvz8ajYY//vGPDbYdMWIEvXv3ZtiwYQQFBREUFMSgQYPYsGGDegF28+bNTJgwgWeffbZWTOvXr+f+++9nwoQJWFtbY29vz+jRo9W/TK5nZ2eHvb09WVlZVFZWMmnSJCZMmNCkuBVFYdWqVSxfvlztvXt7e9faX3RekuRFuzp//jwnT56s970XXniBoqIizp8/z+HDh9m4cSOff/65+v6JEycoKirC29ubXbt2tej8u3fv5uOPP+bAgQPEx8dz9OhRAMrKyuq0nT17Njt27CAqKoqMjAz18dFHH3Hffffh5uZGeno6R48e5Y477mDLli3qvqmpqUyZMoVFixaRlZVFSkoKr732GtbW9f+TKyws5MUXX2Tr1q3k5uYyY8YMdu7c2eS4X3/9deLj4zl27Bh5eXns3buXfv36tegzEpbFCv3gvBCdUv/+/YmKisLe3p6qqipTh9NkHTVu0f6kJy86nQcffJAuXbrg6urKsmXL+OabbzpEouyocQvTkiQvOp358+eTlZVFQkICVVVVPPvss6YOqUk6atzCtGS4RgghLJj05IUQwoKZxTz5zMzMWjfBCCGEaFxAQADdunW7YRuzSPIXLlwgJCTE1GEIIUSHEh4e3mgbGa4RQggLJkleCCEsmCR5IYSwYJLkhRDCgkmSF0IICyZJXgghLJgkeSGEsGCS5IUQwkRGzXiUIRPuMeo5JMkLIYSJ3PnEYwy+5y6jnkOSvBBCmIi9oyOlRcWNN2wFSfJCCGEiDk5OlBUWGfUckuSFEMIErG1t6OJgT2mx9OSFEMLiaJydASgtKDDqeSTJCyGECWictQCU5Bca9TyS5IUQwgQ0Oh0AxfnSkxdCCItj6MnLcI0QQlggjU4/Ji89eSGEsECGJF9SIGPyQghhcWS4RgghLJijzpmqikrKS0qNeh5J8kIIYQIOWi0lRu7FgyR5IYQwCQdnLaVGLmkAkuSFEMIkHLROlBQa96IrSJIXQgiT0Gi1lBp5Zg1IkhdCCJOQ4RohhLBgDlonSmW4RgghLJN+do0keSGEsDhWVlbYOznKcI0QQlgiO0cN1tbWcuFVCCEskUZbU9KgSHryQghhcRwMdWvMYbjG19eX/fv3Ex0dTVRUFC+++CIAixcvJjU1lYiICCIiIggNDVX3CQsLIy4ujtjYWCZMmGC86IUQogNyMPTk22G4xraxBpWVlSxcuJCIiAi0Wi0nT57khx9+AGD58uV88MEHtdoHBgYybdo0Bg4ciLe3N3v37qVv375UV1cb5ycQQogOxsHZCcA87nhNT08nIiICgMLCQmJiYvDx8Wmw/ZQpU9i8eTPl5eUkJSURHx/PsGHD2i5iIYTo4DTt2JNv1ph8QEAAwcHBHD9+HIDnn3+eyMhIVq9ejaurKwA+Pj6kpKSo+6Smptb7pTBv3jzCw8MJDw/H09OzFT+CEEJ0LPZafU/eLMbkDZycnNi2bRsvv/wyBQUFrFy5kt69exMUFERaWpo6bGNlZVVnX0VR6mxbtWoVISEhhISEkJ2d3YofQQghOhaNmuTNpCdva2vLtm3b2LBhA9u3bwcgMzOT6upqFEVh1apV6pBMamoqfn5+6r6+vr5cunTJCKELIUTH5KDVUlVp/AVDoIlJfvXq1cTExLB8+XJ1m5eXl/r8oYceIioqCoCdO3cybdo07Ozs6NmzJ3369OHEiRNtHLYQQnRcGp1zuwzVQBNm14wcOZJZs2Zx+vRp9QLsokWLmD59OkFBQSiKQlJSEvPnzwcgOjqarVu3Eh0dTWVlJQsWLJCZNUIIcY32Kk4GTUjyR44cqXecfdeuXQ3us2TJEpYsWdK6yIQQwkI5aLWUFrRPT17ueBVCiHbm4Nw+q0KBJHkhhGh3Gq223YZrJMkLIUQ7k+EaIYSwYA7O7XfhVZK8EEK0MwcnGZMXQgiLZO/oiLWNDaX5kuSFEMLitGcFSpAkL4QQ7UqtJd9Od7xKkhdCiHZiZWWF1s0VaJ8yw9CEO16FEEK0jSfe/xNBE8cCMlwjhBAWx5DgAYqv5LXLOSXJCyGECeRlZrXLeSTJCyFEO7mSkQnA3n99SUVpWbucU5K8EEK0E3tHR35at4Vdn/yz3c4pSV4IIdqJvZMjZcXF7XpOSfJCCNEO7DQarK2t221+vIEkeSGEaAcONYt3lxVJT14IISyOvZMjAKVF0pMXQgiL4+Ck78nLcI0QQlggQ09eLrwKIYQFUsfkpScvhBCWx94wXCMXXoUQwvI4GIZr5MKrEEJYHunJCyGEBbN3cqSqopLKsvapWWMgSV4IIdqBg9ap3efIgyR5IYRoF/aOju1+tytIkhdCiHYhPXkhhLBg9k5m2pP39fVl//79REdHExUVxYsvvgiAm5sbe/bs4dy5c+zZswdXV1d1n7CwMOLi4oiNjWXChAlGC14IIToKBycz7clXVlaycOFCBgwYwIgRI1iwYAGBgYGEhYWxb98++vbty759+wgLCwMgMDCQadOmMXDgQCZNmsSKFSuwtpY/GIQQnZvZ9uTT09OJiIgAoLCwkJiYGHx8fJgyZQpr1qwBYM2aNTz44IMATJkyhc2bN1NeXk5SUhLx8fEMGzbMeD+BEEJ0AA5OTu1e0gCaOSYfEBBAcHAwx48fp3v37qSnpwP6L4Ju3boB4OPjQ0pKirpPamoqPj4+dY41b948wsPDCQ8Px9PTszU/gxBCmD17raN5DtcYODk5sW3bNl5++WUKCgoabGdlZVVnm6IodbatWrWKkJAQQkJCyM7ObmoYQgjR4VhZWel78uY4XANga2vLtm3b2LBhA9u3bwcgIyMDLy8vALy8vMjM1K9Cnpqaip+fn7qvr68vly5dauu4hRCiw7Bz1ADtX0sempjkV69eTUxMDMuXL1e37dy5k9mzZwMwe/ZsduzYoW6fNm0adnZ29OzZkz59+nDixAkjhC6EEB2DoW5Ne9eSB7BtrMHIkSOZNWsWp0+fVi/ALlq0iKVLl7J161bmzp1LcnIyjz76KADR0dFs3bqV6OhoKisrWbBgAdXV1cb9KYQQwowZKlCaoiffaJI/cuRIvePsAOPGjat3+5IlS1iyZEnrIhNCCAuh9uTNdUxeCCFEyxlWhTLr2TVCCCFaxt7RsGCI9OSFEMLiSE9eCCEsmL1h6T9znUIphBAChk+9n1vvm9Ts/RxMtPQfNGF2jRBCCL3H3l4EwC/f7UFpxtRwjbOWitIyqioqjBVag6QnL4QQTaDr1lV93q2nf7P2dXJzpejKlTaOqGkkyQshRBO4dr+a5HVdm1dU0cnVhaLcvLYOqUkkyQshRBNonJ2vPnfRNWtf6ckLIYSZc3DWqs8dW5TkpScvhBBmyzDXHcBR17wk7+zhTsHlnLYOqUkkyQshRBNotFd78k5uLk3ez9beHgetE4WXc40RVqMkyQshRBM4OGuprq7mcuoldJ4eDbbrMyKE+as+xsZWP0Pd2cMNgILsy+0S5/UkyQshRBM4aPVrtOZlZNaaTnm9J997h74jQpjxl8UAaN1qknyO9OSFEMJsOWi1lBQWkp+VjcsNplBWlutveBo8box+v5oLtqWFhUaPsT6S5IUQogk0zk6UFhSSl5mFrlv9Sd7N2wuXmvn0qb/GAqZdMAQkyQshRJOoPfnMbOwdHWvNtjFw9eoOQM6lNFx7dK/Zz7BgiCR5IYQwWw5aLaUFReRlZgH13/XqVpPYsxIv4OTmClytQCk9eSGEMEMjpz1M0MSxaHRaSgsLycvKBsCle7c6bWcufRuAlOiz2Hbpgp1GY9Kl/0CqUAohRIOsrKyY+sarABRdyaO0sIj8BnryhqGa8pJSclIvAuCoc8ZB60RFWRlVlZXtGPlVkuSFEKIBvgMD1ecOWidKCvSzawBcrrv46unvC8DaV9/E1q4LABoXZxycnEw2VAMyXCOEEA0yJG4AG1tbSgsLKS8ppSS/AJfr5sobLrDmZ2ZRkl8A6MsfOGidTDZUA9KTF0KIBrl796j1OvtCCgCFuVdwdK1d2sAw9n5tr12jc8be0dGkPXlJ8kII0QB3n9pJPjctA9AvyG1Y0s/A2cMdgKK8PBRFv2qUxlmLvdbJJAt4G8hwjRBCNMDd15sLp39VXxuSdVlhEfZax1ptewUPJi8zi9KCQkoK9O3unPk4Dk6mHa6RJC+EEPXwHzyAviNCyE1L59eDhwHUsfbSwsI6PXm/QQM4dzS8pl2+vl1xUc2YvAzXCCGE2bC2teGljasBiNyzn+gfj+A/KJCi3CsAlBYW43BN6WFbe3tcunUl60Kyui0p8gyVZeXYO8mYvBBCmA2bLl14Yf2/ANizcjWn9+wH4PzJU2qb0qKiWmUN3L29AMi9lKZuKysqxsHJCUcXHcV5+e0Qef0aHa5ZvXo1GRkZnDlzRt22ePFiUlNTiYiIICIigtDQUPW9sLAw4uLiiI2NZcKECcaJWgghjMS7Xx/8BvTn14OH2fPp5/W2KS2sfeHVcIE291K6uq2sqBgXr27Y2NpSlGeapf+gCUn+yy+/ZNKkSXW2L1++nODgYIKDg9m1axcAgYGBTJs2jYEDBzJp0iRWrFiBtbUM+wshOg4nV/3SfntXfYlSXV1vm9LCQmy62GJrbw+AT2A/ADLOJ11tU1SEa03pg+IrZtyTP3ToEDk5TVubcMqUKWzevJny8nKSkpKIj49n2LBhrQ5SCCHai2GR7hsNsRjG2B1qZtj4Dx5AekJirX2unVFjqkW8oRWza55//nkiIyNZvXo1rq6uAPj4+JCSkqK2SU1NxcfHp979582bR3h4OOHh4Xh6NlyAXwghrqfR6dDWLKvX1gxJvuQGSd4wW8YwZOPu401O6qXr2lxN8oUmWhUKWpjkV65cSe/evQkKCiItLY0PPvgA0BfzuZ6iKPUeY9WqVYSEhBASEkJ2dnZLwhBCdEJWVla8tGEVC/+9zijHd9TVJPmChldyKi3UJ3AHrRYra2u8evfi0tm4Wm1Ofb9PfZ6RkGiESJumRbNrMjMz1eerVq3i22+/BfQ9dz8/P/U9X19fLl26VGd/IYRoqRl/WUzXnv4A2GkcKC8pbdPja3Q6SgoKqa6qarCN4aYoB60TTq4uWNvYqIXLDNLOxbNw8O1Y29jc8FjG1qKevJeXl/r8oYceIioqCoCdO3cybdo07Ozs6NmzJ3369OHEiRNtE6kQolO4/bGHCA4d3+D7t9w7UX2urSkl0JaaMuWxtKaX7+iiU0sOF9bMob+eKRM8NKEnv3HjRsaMGYOnpycpKSksXryYMWPGEBQUhKIoJCUlMX/+fACio6PZunUr0dHRVFZWsmDBAqobuDothBDXu+m2YB556/cAnNn3I5Xl5bXed7quKNgTS9/m4yfmtWkMji46ivNvnOQNNz3N/tsS/vvhCoA6wzXmotEkP2PGjDrbPv+8/rmjAEuWLGHJkiWti0oI0Sn1GX6b+twnsC8XIqNqve/VpzcA3y7/B/f9bgEBQwdhZWXV4LW/lnB00d3woitARWmZ+tzwl4WheJm5kTtehRBmw7V7N6qrq7G2tibwzjvIS88kLzMLR1cXHnnr91RXVVFWXMKxf+/Au+/N3HLvRByctWpNmbbg6KLjSnrTE3aPPr0pupJHZVlZ441NQO5UEkKYDZdunqRExXD2f8cZP/9p3tq7g+GPTOHup59gyPi7CZo0juwLKZTkFxB7+BgAji4ujRy1eTQ6Z4qb8KWx7IFp6nNTli1ojCR5IYTZcOnejbyMTE58/Y267dE/vl6rrrtngH4Gn6FUgJNb2yZ5R13Tas1kJl5g+1/+BsDBNRvbNIa2JEleCNEol+5dGTn9kXY4TzfyMrM49f0+lt7/OLv+ri8UNnTCPWqb/Z/r58cX19xF6tSGPXl7R0dsutg2OiZvcHjjV7w5cgLHvvpPm8XQ1iTJCyFuyN7Jkd/v2MTURQt5+M3XjHYea1sbNM5adSpiVlIy+1atqdMu/tjPABTl6pP89cvwtUZTShpcry2vBxiDJHkhxA2NnPawevv+HY9PZfC4MW12bHefHtx6n74AouEcpQVXk6ZSXc2J7fqbLSNrSv5mJCYBxhmuUZN8I1MoOxJJ8kKIG3L38abgcg4r5z4PwKTnn2mzYz8Y9goz/rKYfncMR+PsDKAunWew7d33eXPkBDaE/R+Lx0y+ujpTQSFVlZVtOlzTkp68uZMkL4S4IZ/+fclNSyf+xEkOb/wKnadHmx1b6+YKQMiUyWh0+pWWru3JA1SWl1OSX0BVRQWFl2sX+irOy2/T4RqNTv9F05TZNR2FJHkhRIOcPdzxHzxAXR0pLzMLRxcdmpoiXq3VrVcAAFp396s9+WYslVd0Ja/OXbCtIT15IUSnElRTQyY+PAKA8z+fAqDP8FtbfWwHZ63ac7Z3clSX02vOhcziViR5O41GPb+BJHkhRKcyeOxo0uPPkxIVDUByzX9n/20J1jY2rTq2vaPm6nMnR7UnX3qDEr/XK87La9Fwjf+QgfzlxH5e+3pDre2OOh0VpWVme/dqS0iSF0LUy8rKCv/BA4g9ckzdVl1VpRYN6zMipFXHt3fUr6pUdCUPnaeH2qsuKWx6ki/KzWv2hVdXr+68tOEzQD//39r26peVs6cHRVeuNOt45k6SvBCiXlp3N7rY29dZ8Wj5tDmAvmZLa9jXTJlMiYpBo3Nm6IR7qKqorLWiUmOK8vKaPYVy9t9qF1Ds6n91DYzuN/ciLf58s45n7iTJCyHq5eatXzci51J6re3pcQmUFRfTrWbhjpYyLJid8PMvAAQMHcSZ/T82uHh2fYqv5GFrZ4edRtN44xr+gwcA8NlzCwHoGTxEfU/n4UF+pmWtVCdJXghRL9ce+iR/JT29znvJZ6JrJceW0LrrF/yIP3FS3bblrXebdQzDXa/NufhaWVHBqd17iT18lNLCIrz73nxNTG4mXY/VGCTJCyHq1St4CNVVVeRcTKvz3sWYc7h796hnr6ZzrOnJZyZeAPR3tDZ3Kb/m3vVqbWuDbZcupMUloCgKaXEJ9KhJ8hqdMzZdbC0uyUs9eSFELbpuXVm8bycAx7ftrHeMvOjKFbo42DP2N7OJ2P1DnXH7ptA4O1NdXU1ZUTGv3zaG6srKZh/DUKSsqeWGNdqaG65qLu6mnYsnKHQcTq4uvHNoN0C9X2odmfTkhRC1zPl4mfr81Pf76m1TcDkHgMkv/ZY3dm1r0Xkcdc6UFhSiKAqVZWUtWgu16ErzhmscapJ8Sb4+yWcmJeOo03Hr/aFqm5ifjjQ7DnMmSV4IUYvfwED1eXZKar1tTu3eW+t1z6Dmj8/rF+do3U1HhiTf1LnyaumEmp58RkIiAFN+/xKgr3xZ1YK/KMyZJHkhhKrfyBHq88+eW9jgMExFaRkxh/6nvtZ1bX49G43OudVleg37N7snX1M64dzRE/z8zS5AfwH4rw8/2ap4zJGMyQshAH0P/plPlwP6yo/XJvH6/PfDFVRVVjLo7ruY/NKznNnXvOmPjjodJc24u7U+1VVVFOfnNznJa5wNwzVXv1w2LXqHTYveaVUc5kx68kIIAKa8/jIAFWVlHG3CSkdp5xL48qUw0uIS6Brgx4DRI5t1vrboyYN+GmVTh2scnGsP13QGkuSFEIC+fgzA9ys+a3KPXFEUPnlCX1/eUFGyqdoqyTenSJk6XNPKvyA6EknyQghAX3mxJL+Ag19saLzxNcqKi6ksL8fxuoqON2JlbY2TqwsFOTnNDbOOgpwcdF09m9TWMFzTnNIJHZ0keSEEADpPD84ePYGiKM3et6y4BLuagmNNoXVzxcbWloKsy80+1/WyEpPpGuDXeENq/nooKGzWtYOOTpK8EJ2cu4/+zlVnTw8KsluWdMuKitWqkk2hrsDUBnXbi/L09Wts7e0bbRsweGCD00ItlSR5ITqxQffcxRu7vyZo0jg0zlryW9izLisuVsf0m8JwAbQ5ZYUbUlqzJqxhKOZGvPr05kJkVKvP2ZFIkheiE/OsKbN7z1z9/PCC7JZVYCwrLq61CEhj1PICBU1f6q8hV9IzAPDq3euG7aysrbF31FBkYbVpGiNJXgiBT/++AFxuYd2WsqLm9eTta5b6a4upjPHh+iqWv/3sE5w93Bs+Z82XUGlx57noCk1I8qtXryYjI4MzZ86o29zc3NizZw/nzp1jz549uLq6qu+FhYURFxdHbGwsEyZMMErQQojWc9A61Zr2mBaXwPmfI1p0rLLiEhxqFgFpCo2a5Fvfk68oLSPnkv7LaczTMxtsZ1i/tTS/80yfhCYk+S+//JJJkybV2hYWFsa+ffvo27cv+/btIywsDIDAwECmTZvGwIEDmTRpEitWrMDaWv5YEMIcPfPphwyfer/6ujXVF3PT0nH38cbKyqpJ7R20bXtT0ntTplNZXo6rV/cG27j7eANw+WLzK2Z2ZI1m4EOHDpFz3VzWKVOmsGbNGgDWrFnDgw8+qG7fvHkz5eXlJCUlER8fz7Bhw9o+aiEsiJOrC11bucpSSwQMHaQ+r66qYu+/vmjxsbISk7HTODB43JgmtXdw1qplhttCRWkZcSdO0n/UCGzt7Opt4+GrT/I5kuQb1717d9JrVotJT0+nW7duAPj4+JCSkqK2S01NxcfHp95jzJs3j/DwcMLDw/H0bNqNDEJYoncO7Sbsmy21yt0aW/9RI2q9fi1oFMlnolt8vF8PHgLqrp/aEAetE2VFxS2ak9+Q09/vx8HJiTFPzaj3fZduXQHIy8xqs3N2BG06llLfn2oN/U9ctWoVISEhhISEkN3CK/pCWJI+w29tt3ON/+0c9flP67a0+nj5WVf/DT/29qJG22uctZQVtX48/lrhO7+jqqKS0Bfmq7OFrqX1cKco9wrVlc2vW9+RtSjJZ2Rk4OWlX//Ry8uLzMxMQN9z9/O7eueZr68vly51rj+NhGgOh2vmdvsE9qP7TT2Nfk5rGxv8BgZy9N//4dWhI9nx3odtctyVc58HYPjU+7GxvXGBW42zluI2qFtzLaW6mlXPvQLAvS8/V+d9rbsbBZ1s+iS0MMnv3LmT2bNnAzB79mx27Nihbp82bRp2dnb07NmTPn36cOLEibaLVggLYmVtjVvNYtkA3n1v5vc7NnHv7+omqLZk7+SIja0tmecvtOnt/ed/OaU+13W78RCsRqejpKBtkzxA3LFw0hMSSYo8U+c9rYflLdLdFI0m+Y0bN3L06FH69etHSkoKc+bMYenSpYwfP55z584xfvx4li5dCkB0dDRbt24lOjqa3bt3s2DBAqo7UY0IIZpq8f5vePXr9fX23O+Z0/yFK3qH3ELQpHFNautgmL7YxpUYqyur1J60s+eNFxHROGvbpAJlfTLPJ9VbYsHZ3b1TJvlGFw2ZMaP+ixjjxtX/C7VkyRKWLGnaxRchOqOeQUPQdfVE19WT/qNuB+DNkRN5/J1FDLz7TlKjY5t9zOc+/wdQd1m++hiSfFuUFLieofaNrrEkr3OmJPZcm58f9GWE6ytxoHWXnrwQoh30vi1YfR4yZTI5l9Ioyc/ny5fDOPbvHfj064udpuklAm65b6L6vCl11Q1z1Nv6widAwWV9EtXe4M5TMKzvapyefGlhYa1rHQA2trY4uugkyQshjG/svFkAJJ06Q1VFJUc2/lt979cDP2HTxZZRMx4hOHR8o8dydNEx8y//p77u3rsX1rY2N9xnyPi7AShpg7ox1yvMyaG6uvqGPXlrGxscnJyMNlxTUlCIg5MT1jZXP4eB99ylj+9y50vyssarEO3MMF78yZPPYGVtXeviZ1JNhUTD7JCkU2fITUtv8Fh/Ovw9APnZl9F5erDgy5VUlpfzztgHKLqSB+iHZ+575Xn+++FKSgsKuOuJxwF9Qm5r1ZVVFF7OqXVB+Xr1rbPalgzHddA6UZyXz6gZj/DQHxYCkPJrjFHOac6kJy9EO+oVPASA7z7+FKDO7JbSgkL2r15LwWV9AvYJ7NvgsXoGDVGfL3tgmvrc1s6Odw7tRqPTYWVlRciUe7n90QeZ8vuXeDDsdwBEHfiJ3EsNf3m0RvKZXxk4ZhRWDZQ0MazHasyePIDGWV+zfuS0RwDY+f7HLbre0dFJkheinTh7evD82n8CkJGQ1GC7/364kr9MfhSAHn1vrreNu683I6dNBeDrP/+V0oJCFg6+nY+fmKe2effI9/x29d/VxB4yZTKjZuiPG/HfPa3+eRoSf+IXnNxc1bH/6/W7Q1/qJD3+vFHOX1ozNdPRRUf/USPwDPDj+xWf8ePaTUY5n7mTJC9EO3ns7T+oz5NOnb5h27LiYi6diydgyEBAf+Hw4bd+j/+Qgdja2fHGrm3ccq/+guup7/ep+12IjOLdCQ9x7lg4yVHR3BxyS73HL2qDFZkaYuhJO7nVfxE4ePIErmRkculsnFHOn5+t/ytI6+HOpBfmk5WUzIEv1hvlXB2BjMkL0Q40Omf63T6cA5+v578frmhSzZbLKRfVefQBQYO547GHuOOxh+q0u75cb25aOv+c9yIA3XoFMGD0KG69byLe/fqobbKTUzCWotwrAPzh26189txC7nh8Khnnk/j2b38HwMPXh6j9P7Vp3Zpr5Wfpa9O4dPPE3bsHkXv2U1FaZpRzdQSS5IVoB4PuuQubLrZE/nCgycktKeI0g8eOpvdtwfQZEdJgu6qKigbfy0y8QGbiBQ5+uQFbe3v8BwVy023BRhuPB7hw+uryer9Z8QEAA0aPVJO8RudMSb7x/pIouKyf4dMzaDBOri5G/Vk7AknyQhiRk5srg+65C08/HyorKrgYc7bJ+8Yd/xmA575YUee9da+9xZPv/4mM80lNPl5lWRnnT57i/MlTTd6nJRpanLv/qBGcP3kK2y5djDZHHmpm+OTk4j9YP9SVGt35ZtRcS5K8EEb03Bcr8Ordi5yLaeReTKO6qukVELMuJNfZtu61t3D16s6p3Xs5s+/HZh2vPS0aPpZnP/87fgMD1W1P/vVd3psyHTDezBqD/MxsfAf0A1CnknZWkuSFMCLD4tLuPj2IO/Zzs/YtLymts+3asgU3GqYxtbLiYj6cNgc7jYY/Hd6NrZ0dDk5O6rRPY/bkAeJPnFSTfF5G56offz2ZXSOEkVy/vkLKr81flOPc0RMkRpwmM/EC/9u6va1CazflJSW8futo3h77AAC3PaBfGKXYyL3r3f/4FwBXMjI7ZSmDa0lPXggjCRiiX17v6L//w+5P/tWiZPPPZ15Snzd1/VRzlJ+ZRUVZGQPuGgnok68xVZSW8X9332e2w1ntSZK8EEbSs+bu1kPrtrRJb9JYUw7bSxd7e/V5bisWDW8qQ0XMzk6Ga4Qwgn53DOf+hfqVkpozA8aS/X32b0n4OYLPX3iNqspKU4fTaUhPXggjGDpxLABpcQkmjsR8JP4SyYqnjbvqlahLevJCGIFGpy+Ote7VN00ciejsJMkLYQSefj78euCQDNUIk5MkL0Qbs7G1xdPfj+yUVFOHIoSMyQvRVkY8MoU+w2+joqwMO40DCeG/mDokISTJC9FWHl0cVuv1uWPhJopEiKtkuEaINuA3aECt1+E7/tupy9sK8yFJXnRYA0aP4qVNq+leUx+mrem6deXW+0Nx9nBvtO2o6fol5r7+81/Z8d5HfPV/S40SkxDNJcM1wqxY29gwetY0Rs18jHWvvtXgCkq9bwtm7t/fB/S12jMSEpt8jm69Auh7+zCObN5WZ41VAN8B/bDtYscL6/X1T+KO/cyn815o8Hjde/fitgdCyU5J5cjmbU2OQ4j2IElemAU7jYZZf/szgaNuV7c98OoLtdYsNVi4bR3eNWufVldV4T8osE6b+gy65y6mLnoVl+5dAagoLeX419/UajNk/N3M/tuSWtv6jLiNrj39yUqqW/oXYObS/wNg/Wt/bFIcQrQnGa4RZuHel5+tleCjfzpCwNBBfHDmKB+cOUqvW4YCMPDuO9UED3Bm348Mumc0C/+9Fg9fnwaPr3V34+mPlqkJHsAnsF+tNr4D+tdJ8D+u2wzAK1vXoNHp6hy3a09/fPr35dCGraT82rkXpxDmSZK8MLn7freAUTMeBeDtsQ+wcPDt/PfDlbXaPL/mU5xcXdSx75yLafxvy9dkJl0AwLtfHwaPG1Pv8QPvGsnbP36nvn516EgyEy8wZPzdWFnr/wm4eXvx288+AfTDM8semMarQ0ey872P2Pzmn7DTOPDuke+Z/bcleN18E7pu+i8Lv4H9ASlfIMyXJHlhEo4uOt764T+MePRB7p7zBAAHvthAfqZ+gYf0uAT+eOck3hw5gf2frwOg1y1DCRg6mMg9+/nzpKlse/d99n+2jn889Sz52ZfpGTS4znmGPXgfv/nHXwE4/vU3vDp0JEp1NQe/3ICzhzvLfv4R7359+N2WL9E4a8lKSubTeS+QmXhBHa+PP3F1vvuQ8Xfz2vYNLN63k163DMXRxQWAqH0/Gu/DEqIVWjUmn5iYSEFBAVVVVVRWVhISEoKbmxtbtmyhZ8+eJCUl8dhjj3HlypU2CldYih59euPq1Z1H//g6AF+8FEbU/tqJ0rBs2/7V6xg9azpPf7QMgOTTv6ptyktKOH/yFLGHjjJ43Bg8/HzJuXhJTdCP/+kNQF+X/dzRE+p+hvVTbbrYsvDfawG4nHqx3msAuWnpfPm7P2BtY8Njb/8BBycnQP/XRdKpM1RVVja4rqkQptbqnvzdd99NcHAwISH61eTDwsLYt28fffv2Zd++fYSFhTVyBGHprG1s6mzzHzKw1uuEnyMa3L8kv6DWrJXj27+t0yYp8gwaZy2LvvuK8c88BcAdj08FYP/qtbUSPOiHexYNH8vZI8fUbR88PKvBZH1m70Eiv9/Hp795kerqaja/+ScAegYNpuByToev9S4sV5vPrpkyZQpjxowBYM2aNRw8eFASfSfm078vz67+O5F79vPV20vp0fdmHlr0Cr1vDVbbVFdVUZJ/457wNx98QtKpM6THn6+3beqvserziQvmMfqpGWqPO+bwsTrtQb8O6bGvv6HfyBGsmLOAsuLiRn+elKhoXhuqX91o8svPofP04PzJU43uJ4SpWAEt7oKcP3+e3NxcFEXhn//8J6tWrSI3Nxc3Nze1TU5ODu7uN76ZJDw8XP1LQFiWt3/8Dq27/vfhx7WbGD1rOgClRUVs+eMSCrIvU1VZWWsIpqX8Bw8g9IX59L19mLrtkyfnNzjX3sDa1obqyuYvE3fH41N5+M3XWP74U6RGn232/kK0VlNyZ6t68iNHjiQtLY2uXbvyww8/EBsb2/hONebNm8czzzwDgKenZ2vCEGbKysoKrbsbFWVl2HTpoib46qoq3rxjQr03IrVG8plotr37PvcvfB6X7t349DcvUFpY1Oh+LUnwAP/b8jVHt26XoRph9pS2eCxevFhZuHChEhsbq3h5eSmA4uXlpcTGxja6b3h4eJvEIA/zejh7eigfnDmq3PH4VMXRRacE3jVSufOJxxV3X2+TxyYPeVjCoym5s8UXXh0dHdFqterzCRMmEBUVxc6dO5k9ezYAs2fPZseOHS09hejgxs9/GoDLqZcozssn5qcjHFq/hZzUSyaOTIjOo8XDNd27d2f79u36g9jasnHjRr7//nvCw8PZunUrc+fOJTk5mUcffbTNghUdg/+QgTz53p9w9+lBVlIycVJyVwiTaXGST0xMJCgoqM72nJwcxo0b15qYRAfm5OrCSxs+A/TTIv/5zEtUV7VszFsI0XpSoEy0CZfuXXlx/Sps7ewAiNyzn41/eJuqigoTRyZE5yZJXrTYyGkPo+vqSVDoODz9fNXtib9EsuH1xVRVVpowOiEESJIXLTTontFMfePVWtuObN7Gj2s2cTn1oomiEkJcT5J8J+Xu04O8zOwWDadoPdx4+iP9ykdn/3echPAI9n22pq1DFEK0AUnyFszZ04Oqiopa9Vg8/X25c+ZjamnfuGM/kxwVjU0XW/63+esm9cJDHpgMwPf/WMWeTz83TvBCiDYhSd5MaT3cqKqopCS/oEX79xl+m1ofHaCirIwu9vZ12424jT4jbgNgzOwZJEdFk5+ZReyR41xJy2DEIw8w6J7RpCck4uiiIyM+UW2/b/W6FsUmhGg/kuTNhJObKyFT7sXZw51LZ+OY8ZfFAJz+4QDnf4nk0PotTTqOs4c7/kMGMufj99RthTm5ZF1Iwevmm0g7F89/li2nrKiY7ORUbGxtGfHIFPqPup0Bo0fiP2gAoB9zv5a9o4bqqip8BvQFkJkzQnQQkuTbkbtPDyrLKwi86w7cvXvgM6Afnr4+ePj7Ym1d/83HQ8bfzZDxd6Pr6sF/l68A9DVhALVmir2To/5C6KKFOGj1lRerq6v560MzyTifdMOYqiorObJ5G0c2b8Nv0ADKi4ux1zpx70vPcvOwWwH4c+jDcpeqEB2UJHkj8/D14Y7Hp3LXk4/XW1fdoCS/gH/Of5mC7MsMHDOK3LQMFEUhJSqa3239knvmPImDVkvXAD/6DNcPl0T/dIRuPQPw9Petc7wVTz3baIK/XkpUtPp85dznm7WvEMI8tarUcFux1FLD9/7uOe6Z86T6uqSgkJKCAo59tYP0hPOkxydyOSW10eNoPdx4++B39b53JSMT1+7d2PX3f/HT2k1UlJWDokhlRCE6AaOXGhb16zP8NoZOHMvtjz4IwLfL/8GBz9e3+HiFl3NZ/vhT+A0cQGrMWcqKiricchFrWxsqSsvaKGohhCWSJN9CXXv64+7jTRd7O86fPEVxXj6u3btx6wOhTH7xtwDkZ2Xz8RPzyL2U3urzpUafrbMwhdxRKoRojCT5Jho3/2nueOwhCnNy8enft8772SmpuPt4qxdQP5r5mzZZ7UgIIVqjUyd5K2trfPr3xdbOjuQzv4IVOLro8O7bh4rSUu6oqc3SNcAPl25dgdqLUn/x0uv0DBpCz6DBoCgUZF3m+5WryUq8wJWMTFP9WEIIoerQSd7eyZHH33mD3LR00s4lcDH2HP1uH8aomY9y8MsNxBw6Rn5WFlUVlQyf+gD9Rg7ncspFKsrLsHd0ZNiD96lTDisrKrDt0qXW8SsrKriSlsHllIskRpzmq7eXUlpQiE2XLlhZWVFZXk7U/p9M8aMLIUSTdOgk3713L/oMvw1HF12d9x76w0Ie+sON94/+6QinfziAu3cPegYPoTAnl/LiEjwD/MhMvMCh9VvITLxQZz+5CUgI0VF06CSffPpX/njnJDwD/Bg4ehRlJSVcjD3HxZhz9Bw6iK69AnDUOaP1cCf11xiqq6pJPvMr/oMHUpyXx7mjsmKREMKydegkD/q7PrOSkjmYtLHW9oSfI0j4OaLefXIuprVHaEIIYXItXshbCCGE+ZMkL4QQFkySvBBCWDBJ8kIIYcEkyQshhAWTJC+EEBZMkrwQQlgwSfJCCGHBzGLRkMzMTC5cqFs+oKk8PT3Jzs5uw4iMS+I1LonX+DpazJYab0BAAN26dWu0ndLRH+Hh4SaPQeI1n4fEKzFLvFcfMlwjhBAWTJK8EEJYMItI8v/6179MHUKzSLzGJfEaX0eLuTPHaxYXXoUQQhiHRfTkhRBC1E+SvBBCWLAOneQnTpxIbGwscXFxvP7666YOBwBfX1/2799PdHQ0UVFRvPjiiwAsXryY1NRUIiIiiIiIIDQ0VN0nLCyMuLg4YmNjmTBhQrvHnJiYyOnTp4mIiCA8XL9alpubG3v27OHcuXPs2bMHV1dXs4i3b9++6mcYERFBXl4eL730ktl9vqtXryYjI4MzZ86o21rymd5yyy2cPn2auLg4Pvroo3aN97333iMmJobIyEi+/vprXFxcAP3c7OLiYvWzXrlypVnE25LfAVPGu3nzZjXWxMREIiIiAON8viafE9qSh7W1tRIfH6/06tVL6dKli3Lq1CklMDDQ5HF5eXkpwcHBCqBotVrl7NmzSmBgoLJ48WJl4cKFddoHBgYqp06dUuzs7JSePXsq8fHxirW1dbvGnJiYqHh4eNTatmzZMuX1119XAOX1119Xli5dajbxXvs7kJaWpvj7+5vd53vnnXcqwcHBypkzZ1r1mR4/flwZMWKEAijfffedMmnSpHaLd/z48YqNjY0CKEuXLlXjDQgIqNXu2ocp423J74Ap47328de//lV56623jPL5dtie/LBhw4iPjycxMZGKigo2b97MlClTTB0W6enp6jdyYWEhMTEx+Pj4NNh+ypQpbN68mfLycpKSkoiPj2fYsGHtFe4N41qzZg0Aa9as4cEHH1S3m0u8Y8eOJSEhgeTk5AbbmCreQ4cOkZOTUyeW5nymXl5e6HQ6jh07BsDatWvVfdoj3h9++IGqqioAjh07hq+v7w2PYep4G2Kun++1HnvsMTZt2nTDY7Q03g6b5H18fEhJSVFfp6am3jCZmkJAQADBwcEcP34cgOeff57IyEhWr16t/qluDj+Hoijs2bOHn3/+mXnz5gHQvXt30tPTAf0Xl+HWaXOI12DatGm1/mGY6+dr0NzP1MfHh9TU1DrbTWHOnDns2rVLfd2rVy9++eUXDh48yKhRowDMIt7m/A6YQ7wAd955JxkZGcTHx6vb2vLz7bBJ3srKqs42RVFMEEn9nJyc2LZtGy+//DIFBQWsXLmS3r17ExQURFpaGh988AFgHj/HyJEjufXWWwkNDWXBggXceeedDbY1h3gBunTpwgMPPMBXX30FYNafb2MaitFcYl+0aBGVlZVs2LABgLS0NPz9/bnlllt45ZVX2LhxI87OziaPt7m/A6aO12D69Om1Oitt/fl22CSfmpqKn5+f+trX15dLly6ZMKKrbG1t2bZtGxs2bGD79u2AvghbdXU1iqKwatUqdcjAHH6OtLQ0ALKysti+fTvDhg0jIyMDLy8vQP9nYmZmptnECxAaGsovv/yixmXOn69Bcz/T1NTUWkMkpoh91qxZ3HfffcycOVPdVl5erg49/PLLLyQkJNC3b1+Tx9vc3wFTxwtgY2PD1KlT2bJli7rNGJ+vUS40GPthY2OjJCQkKD179lQvvA4YMMDkcQHKmjVrlOXLl9fa5uXlpT5/+eWXlU2bNimAMmDAgFoXhRISEtr1Qqajo6Oi1WrV50eOHFEmTpyovPfee7UuEi5btsws4jU8Nm3apDz11FNm/flefwGtJZ/piRMnlOHDhyugv9AWGhrabvFOnDhR+fXXXxVPT89a7Tw9PdX4evXqpaSmpipubm4mj7clvwOmjNfwGR88eNDYn6/xf9mN9QgNDVXOnj2rxMfHK4sWLTJ5PIAycuRIRVEUJTIyUomIiFAiIiKU0NBQZe3atcrp06eVyMhIZceOHbV+IRctWqTEx8crsbGxRru639CjV69eyqlTp5RTp04pUVFR6ufo7u6u7N27Vzl37pyyd+9e9ZfM1PECikajUbKzsxWdTqduM7fPd+PGjcqlS5eU8vJyJSUlRZkzZ06LPtNbb71VOXPmjBIfH6988skn7RpvXFyckpycrP4er1y5UgGUqVOnKlFRUcqpU6eUkydPKvfdd59ZxNuS3wFTxgsoX3zxhTJ//vxabdv685WyBkIIYcE67Ji8EEKIxkmSF0IICyZJXgghLJgkeSGEsGCS5IUQwoJJkhdCCAsmSV4IISzY/wNkBx1aRk7kcQAAAABJRU5ErkJggg==",
                        "text/plain": [
                            "<Figure size 432x288 with 1 Axes>"
                        ]
                    },
                    "metadata": {},
                    "output_type": "display_data"
                }
            ],
            "source": [
                "def moving_average(arr, n=10):\n",
                "    ret = np.cumsum(arr)\n",
                "    ret[n:] = ret[n:] - ret[:-n]\n",
                "    return ret[n-1:]/n\n",
                "\n",
                "plt.title('Moving Average')\n",
                "plt.plot(moving_average(scores, n=100));"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "The scores are increasing over time which means it's learning"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "## Playing the game"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 29,
            "metadata": {},
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": [
                        "Episode: 9\n",
                        "Score: 500\n"
                    ]
                }
            ],
            "source": [
                "# Play the game 10 times with the calculated Q-Table\n",
                "scores = []\n",
                "for e in range(10):\n",
                "    observation = env.reset()\n",
                "    state = discretizer(*observation)\n",
                "    done = False\n",
                "    score = 0\n",
                "    while done!=True:\n",
                "        env.render()\n",
                "        action = best_action(state)\n",
                "        observation, reward, done, info = env.step(action)\n",
                "        state = discretizer(*observation)\n",
                "        score+=1\n",
                "    scores.append(score)\n",
                "    print('Episode:', e)\n",
                "    print('Score:', score)\n",
                "    clear_output(wait=True)\n",
                "env.close()"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 30,
            "metadata": {},
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": [
                        "The average score is: 500.0\n"
                    ]
                }
            ],
            "source": [
                "print('The average score is:', np.mean(scores))"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "Since 500 is the maximum score possible, the algorithm works well"
            ]
        }
    ],
    "metadata": {
        "interpreter": {
            "hash": "63fd5069d213b44bf678585dea6b12cceca9941eaf7f819626cde1f2670de90d"
        },
        "kernelspec": {
            "display_name": "Python 3.9.5 64-bit",
            "name": "python3"
        },
        "language_info": {
            "codemirror_mode": {
                "name": "ipython",
                "version": 3
            },
            "file_extension": ".py",
            "mimetype": "text/x-python",
            "name": "python",
            "nbconvert_exporter": "python",
            "pygments_lexer": "ipython3",
            "version": "3.8.5"
        },
        "orig_nbformat": 4
    },
    "nbformat": 4,
    "nbformat_minor": 2
}