tutorials/tutorial_linear_regression.solution.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Linear Regression Tutorial\n",
    "by Marc Deisenroth"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The purpose of this notebook is to practice implementing some linear algebra (equations provided) and to explore some properties of linear regression."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import scipy.linalg\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We consider a linear regression problem of the form\n",
    "$$\n",
    "y = \\boldsymbol x^T\\boldsymbol\\theta + \\epsilon\\,,\\quad \\epsilon \\sim \\mathcal N(0, \\sigma^2)\n",
    "$$\n",
    "where $\\boldsymbol x\\in\\mathbb{R}^D$ are inputs and $y\\in\\mathbb{R}$ are noisy observations. The parameter vector $\\boldsymbol\\theta\\in\\mathbb{R}^D$ parametrizes the function.\n",
    "\n",
    "We assume we have a training set $(\\boldsymbol x_n, y_n)$, $n=1,\\ldots, N$. We summarize the sets of training inputs in $\\mathcal X = \\{\\boldsymbol x_1, \\ldots, \\boldsymbol x_N\\}$ and corresponding training targets $\\mathcal Y = \\{y_1, \\ldots, y_N\\}$, respectively.\n",
    "\n",
    "In this tutorial, we are interested in finding good parameters $\\boldsymbol\\theta$."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYwAAAEGCAYAAAB2EqL0AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAP6klEQVR4nO3df6zdd13H8efLbvwQVNBeZGyTO+OdYcovPVkkGrPYVQcjKygkW4wOlSwYB3XRSOMSpkOSERPr+BGwymQYZBB+SLXF0Q1wEDPc6TJgW1lbl5GVLuyyCThBSOHtH/cM7+7ObT+9P873nHuej+Sk3x+ffvv6pu195fvjfL+pKiRJOpEf6DqAJGkyWBiSpCYWhiSpiYUhSWpiYUiSmpzSdYD1snnz5pqdne06hiRNlP3793+1qmaGrduwhTE7O0u/3+86hiRNlCRfWm6dp6QkSU0sDElSEwtDktTEwpCkDWbnvoPrsl0LQ5I2mGtvPrQu27UwJElNLAxJUhMLQ5LUZMN+cU+SpsHOfQeHXrOY3bHnMfPbt8xxxdazV/VnZaO+QKnX65Xf9JY0jWZ37OG+ay5c0e9Nsr+qesPWeUpKktTEwpAkNbEwJElNLAxJ2mC2b5lbl+1aGJK0waz2bqjlWBiSpCYWhiSpyVgURpLrkjyY5M5l1p+X5OtJ7hh83jDqjJI07cblm97vBt4GvOc4Yz5dVS8dTRxJ0lJjcYRRVbcAD3edQ5K0vLEojEYvSvK5JB9L8jPDBiS5LEk/SX9+fn7U+SRpQ5uUwrgdeHZVPR94K/BPwwZV1a6q6lVVb2ZmZqQBJWmjm4jCqKpvVNUjg+m9wKlJNnccS5KmykQURpJnJslg+lwWcj/UbSpJmi5jcZdUkvcB5wGbkxwBrgJOBaiqdwKvAH4/yTHgW8DFtVGfyy5JY2osCqOqLjnB+rexcNutJKkjE3FKSpLUPQtDktTEwpAkNbEwJElNLAxJUhMLQ5LUxMKQJDWxMCRJTSwMSVITC0OS1MTCkCQ1sTAkSU0sDElSEwtDktTEwpAkNbEwJElNLAxJUhMLQ5LUxMKQJDWxMCRJTSwMSVITC0OS1MTCkCQ1sTAkSU0sDElSk7EojCTXJXkwyZ3LrE+StyQ5nOTzSX5u1BkladqNRWEA7wYuOM76FwNzg89lwDtGkEmStMhYFEZV3QI8fJwh24D31IJbgaclOW006SRJMCaF0eB04P5F80cGyx4jyWVJ+kn68/PzIwsnSdNgUgojQ5bV4xZU7aqqXlX1ZmZmRhBLkqbHpBTGEeDMRfNnAEc7yiJJU2lSCmM38NuDu6V+Afh6VT3QdShJmiandB0AIMn7gPOAzUmOAFcBpwJU1TuBvcBLgMPAN4Hf6SapJE2vsSiMqrrkBOsL+IMRxZEkDTEpp6QkSR2zMCRJTSwMSVITC0OS1MTCkCQ1sTAkSU0sDElSEwtDktTEwpAkNbEwJElNLAxJUhMLQ5LUxMKQtCo79x3sOoJGxMKQtCrX3nyo6wgaEQtDktTEwpAkNbEwJElNxuKNe5Imw859B4des5jdsecx89u3zHHF1rNHFUsjkoW3n248vV6v+v1+1zGkDW92xx7uu+bCrmNojSTZX1W9Yes8JSVJamJhSJKaWBiSpCYWhqRV2b5lrusIGhELQ9KqeDfU9LAwJElNLAxJUpOxKIwkFyS5J8nhJDuGrH9Vkvkkdww+r+4ipyRNs86/6Z1kE/B2YCtwBLgtye6qunvJ0PdX1eUjDyhJAsbjCONc4HBV3VtV3wFuALZ1nEmStMQ4FMbpwP2L5o8Mli31G0k+n+SDSc4ctqEklyXpJ+nPz8+vR1ZJmlrjUBgZsmzpA67+GZitqucBNwHXD9tQVe2qql5V9WZmZtY4piRNt3EojCPA4iOGM4CjiwdU1UNV9e3B7N8CPz+ibJKkgXEojNuAuSRnJXkCcDGwe/GAJKctmr0IODDCfJIkxuAuqao6luRy4EZgE3BdVd2V5GqgX1W7gdcluQg4BjwMvKqzwJI0pXwfhiTp+3wfhiRp1SwMSVITC0OS1MTCkCQ1sTAkSU0sDElSEwtDktTEwpAkNbEwJElNLAxJUhMLQ5LU5ISFkeSmJM8fRRhJ0vhqOcL4E2Bnkr9f8phxSdIUOWFhVNXtVfUrwL8A/5rkqiRPXv9okqRx0nQNI0mAe4B3AK8FDiX5rfUMJm1kO/cd7DqCdNJarmF8BvgysBM4nYWXF50HnJtk13qGkzaqa28+1HUE6aS1vHHvNcBd9fg3Lb02ia9KlaQpccLCqKo7j7P6wjXMIkkaY6v6HkZV3btWQSRJ463llJSkVdi57+DQaxazO/Y8Zn77ljmu2Hr2qGJJJy2PvzSxMfR6ver3+13HkIaa3bGH+67xjK7GT5L9VdUbts5Hg0iSmlgYkqQmFoYkqYmFIXVg+5a5riNIJ20sCiPJBUnuSXI4yY4h65+Y5P2D9Z9NMjv6lNLa8W4oTaLOCyPJJuDtwIuBc4BLkpyzZNjvAf9VVT/FwiNK3jzalJKkzgsDOBc4XFX3VtV3gBuAbUvGbAOuH0x/ENgyeCCiJGlExqEwTgfuXzR/ZLBs6JiqOgZ8HfixpRtKclmSfpL+/Pz8OsWVpOk0DoUx7Ehh6bcJW8ZQVbuqqldVvZmZmTUJJ0laMA6FcQQ4c9H8GcDR5cYkOQX4EeDhkaSTJAHjURi3AXNJzkryBOBiYPeSMbuBSwfTrwA+MeRx65KkddT5wwer6liSy4EbgU3AdVV1V5KrgX5V7QbeBfxDksMsHFlc3F1iSZpOnRcGQFXtBfYuWfaGRdP/C7xy1LkkSf9vHE5JSZImgIUhSWpiYUiSmlgYkqQmFoYkqYmFIUlqYmFIkppYGJKkJhaGJKmJhSFJamJhSJKaWBiSpCYWhiSpiYUhSWpiYUiSmlgYkqQmFoYkqYmFIUlqYmFIkppYGJKkJhaGJKmJhSFJamJhSJKaWBiSpCYWhiSpiYUhSWrSaWEk+dEk+5IcGvz69GXGfTfJHYPP7lHnlCR1f4SxA7i5quaAmwfzw3yrql4w+Fw0uniSpEd1XRjbgOsH09cDL+swiyTpOLoujB+vqgcABr8+Y5lxT0rST3JrkmVLJcllg3H9+fn59cgrSVPrlPX+A5LcBDxzyKorT2IzP1FVR5P8JPCJJF+oqv9cOqiqdgG7AHq9Xq0osCRpqHUvjKo6f7l1Sb6S5LSqeiDJacCDy2zj6ODXe5N8Cngh8LjCkCStn65PSe0GLh1MXwp8dOmAJE9P8sTB9GbgF4G7R5ZQkgR0XxjXAFuTHAK2DuZJ0kvyd4MxzwH6ST4HfBK4pqosDEkasXU/JXU8VfUQsGXI8j7w6sH0vwPPHXE0SdISXR9hSJImhIUhSWpiYWhi7Nx3sOsI0lSzMDQxrr35UNcRpKlmYUiSmlgYkqQmFoYkqUmn38OQlrNz38Gh1yxmd+x5zPz2LXNcsfXsUcWSplqqNuYz+nq9XvX7/a5jaA3N7tjDfddc2HUMaUNLsr+qesPWeUpKktTEwpAkNbEwJElNLAxNjO1b5rqOIE01C0MTw7uhpG5ZGJKkJhaGJKmJhSFJamJhSJKaWBiSpCYWhiSpiYUhSWpiYUiSmlgYkqQmFoYkqYmFIUlq0mlhJHllkruSfC/J0Bd2DMZdkOSeJIeT7BhlRknSgq6PMO4Efh24ZbkBSTYBbwdeDJwDXJLknNHEkyQ9qtN3elfVAYAkxxt2LnC4qu4djL0B2Abcve4BJUnf1/URRovTgfsXzR8ZLHucJJcl6Sfpz8/PjyScJE2LdT/CSHIT8Mwhq66sqo+2bGLIsho2sKp2AbsAer3e0DGSpJVZ98KoqvNXuYkjwJmL5s8Ajq5ym5KkkzQJp6RuA+aSnJXkCcDFwO6OM0nS1On6ttqXJzkCvAjYk+TGwfJnJdkLUFXHgMuBG4EDwAeq6q6uMkvStOr6LqmPAB8Zsvwo8JJF83uBvSOMJklaYhJOSUmSxoCFIUlqYmFIkppYGMvYue9g1xEkaaxYGMu49uZDXUeQpLFiYUiSmlgYkqQmFoYkqUmnX9wbFzv3HRx6zWJ2x57HzG/fMscVW88eVSxJGiup2pgPde31etXv91f8+2d37OG+ay5cw0SSNP6S7K+qoW9A9ZSUJKmJhSFJamJhSJKaWBjL2L5lrusIkjRWLIxleDeUJD2WhSFJamJhSJKaWBiSpCYb9ot7SeaBL61iE5uBr65RnC5tlP0A92VcbZR92Sj7Aavbl2dX1cywFRu2MFYrSX+5bztOko2yH+C+jKuNsi8bZT9g/fbFU1KSpCYWhiSpiYWxvF1dB1gjG2U/wH0ZVxtlXzbKfsA67YvXMCRJTTzCkCQ1sTAkSU0sjGUkeWOSzye5I8nHkzyr60wrleQvk3xxsD8fSfK0rjOtVJJXJrkryfeSTNwtkEkuSHJPksNJdnSdZzWSXJfkwSR3dp1lNZKcmeSTSQ4M/m1t7zrTSiV5UpL/SPK5wb78+Zpu32sYwyX54ar6xmD6dcA5VfWajmOtSJJfBT5RVceSvBmgql7fcawVSfIc4HvA3wB/XFUrf63iiCXZBBwEtgJHgNuAS6rq7k6DrVCSXwYeAd5TVT/bdZ6VSnIacFpV3Z7kh4D9wMsm8e8lSYCnVNUjSU4FPgNsr6pb12L7HmEs49GyGHgKMLHNWlUfr6pjg9lbgTO6zLMaVXWgqu7pOscKnQscrqp7q+o7wA3Ato4zrVhV3QI83HWO1aqqB6rq9sH0fwMHgNO7TbUyteCRweypg8+a/eyyMI4jyZuS3A/8JvCGrvOskd8FPtZ1iCl1OnD/ovkjTOgPpo0qySzwQuCz3SZZuSSbktwBPAjsq6o125epLowkNyW5c8hnG0BVXVlVZwLvBS7vNu3xnWhfBmOuBI6xsD9jq2VfJlSGLJvYI9eNJslTgQ8Bf7jkDMNEqarvVtULWDiTcG6SNTtdeMpabWgSVdX5jUP/EdgDXLWOcVblRPuS5FLgpcCWGvMLVyfx9zJpjgBnLpo/AzjaURYtMjjf/yHgvVX14a7zrIWq+lqSTwEXAGtyY8JUH2EcT5LF72i9CPhiV1lWK8kFwOuBi6rqm13nmWK3AXNJzkryBOBiYHfHmabe4ELxu4ADVfVXXedZjSQzj94FmeTJwPms4c8u75JaRpIPAT/Nwh05XwJeU1Vf7jbVyiQ5DDwReGiw6NYJvuPr5cBbgRnga8AdVfVr3aZql+QlwF8Dm4DrqupNHUdasSTvA85j4VHaXwGuqqp3dRpqBZL8EvBp4Ass/H8H+NOq2ttdqpVJ8jzgehb+ff0A8IGqunrNtm9hSJJaeEpKktTEwpAkNbEwJElNLAxJUhMLQ5LUxMKQJDWxMCRJTSwMaYQG713YOpj+iyRv6TqT1GqqnyUldeAq4Ookz2DhqagXdZxHauY3vaURS/JvwFOB8wbvX5AmgqekpBFK8lzgNODbloUmjYUhjcjgVaDvZeEte/+TZGIemiiBhSGNRJIfBD4M/FFVHQDeCPxZp6Gkk+Q1DElSE48wJElNLAxJUhMLQ5LUxMKQJDWxMCRJTSwMSVITC0OS1OT/AE4cDzAccAchAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Define training set\n",
    "X = np.array([-3, -1, 0, 1, 3]).reshape(-1,1) # 5x1 vector, N=5, D=1\n",
    "y = np.array([-1.2, -0.7, 0.14, 0.67, 1.67]).reshape(-1,1) # 5x1 vector\n",
    "\n",
    "# Plot the training set\n",
    "plt.figure()\n",
    "plt.plot(X, y, '+', markersize=10)\n",
    "plt.xlabel(\"$x$\")\n",
    "plt.ylabel(\"$y$\");"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1. Maximum Likelihood\n",
    "We will start with maximum likelihood estimation of the parameters $\\boldsymbol\\theta$. In maximum likelihood estimation, we find the parameters $\\boldsymbol\\theta^{\\mathrm{ML}}$ that maximize the likelihood\n",
    "$$\n",
    "p(\\mathcal Y | \\mathcal X, \\boldsymbol\\theta) = \\prod_{n=1}^N p(y_n | \\boldsymbol x_n, \\boldsymbol\\theta)\\,.\n",
    "$$\n",
    "From the lecture we know that the maximum likelihood estimator is given by\n",
    "$$\n",
    "\\boldsymbol\\theta^{\\text{ML}} = (\\boldsymbol X^T\\boldsymbol X)^{-1}\\boldsymbol X^T\\boldsymbol y\\in\\mathbb{R}^D\\,,\n",
    "$$\n",
    "where \n",
    "$$\n",
    "\\boldsymbol X = [\\boldsymbol x_1, \\ldots, \\boldsymbol x_N]^T\\in\\mathbb{R}^{N\\times D}\\,,\\quad \\boldsymbol y = [y_1, \\ldots, y_N]^T \\in\\mathbb{R}^N\\,.\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let us compute the maximum likelihood estimate for a given training set"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "## EDIT THIS FUNCTION\n",
    "def max_lik_estimate(X, y):\n",
    "    \n",
    "    # X: N x D matrix of training inputs\n",
    "    # y: N x 1 vector of training targets/observations\n",
    "    # returns: maximum likelihood parameters (D x 1)\n",
    "    \n",
    "    N, D = X.shape\n",
    "    theta_ml = np.linalg.solve(X.T @ X, X.T @ y) ## <-- SOLUTION\n",
    "    return theta_ml"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "# get maximum likelihood estimate\n",
    "theta_ml = max_lik_estimate(X,y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, make a prediction using the maximum likelihood estimate that we just found"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "## EDIT THIS FUNCTION\n",
    "def predict_with_estimate(Xtest, theta):\n",
    "    \n",
    "    # Xtest: K x D matrix of test inputs\n",
    "    # theta: D x 1 vector of parameters\n",
    "    # returns: prediction of f(Xtest); K x 1 vector\n",
    "    \n",
    "    prediction = Xtest @ theta ## <-- SOLUTION\n",
    "    \n",
    "    return prediction "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, let's see whether we got something useful:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEGCAYAAABo25JHAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAbyUlEQVR4nO3dd5hV1b3G8e+PoUlVioDAMBZQESw4AgoiUgSkxd77DdeIBEFFRI3dqESaogZLjFdjSdRLLwNIEQUZUJSOIAhI7yBtZtb9Y5NcxUFm4Jyzzj77/TwPz8M5U857MnFe9l57/5Y55xARkegq4juAiIj4pSIQEYk4FYGISMSpCEREIk5FICIScUV9BzgSlSpVchkZGb5jiIiEyqxZszY65yof/HwoiyAjI4Ps7GzfMUREQsXMVuT3vE4NiYhEnIpARCTiVAQiIhGnIhARiTgVgYhIxKkIREQiTkUgIhJxKgIRkTD4aTOM7g17tsX8W6sIRESSmXMw7xMY3BBmvgYrPo/5S4TyzmIRkUjYvgZG3QcLR0C1s+GmT6Bq/Zi/jIpARCTZOAdf/Q+MfRhy90LrJ6BxV0iLz69sFYGISDLZ/D0M7w7fT4ZaTaDTi1Dx5Li+pIpARCQZ5OXCjL/CxCfB0qBDf2hwKxSJ/1KuikBExLf1C2Do3bA6G2q3CUqgfPWEvbyKQETEl5x9MG0ATH4eSpSFy1+H+leCWUJjqAhERHxYPQuGdoP18+CMy+HSvlC6kpcoKgIRkUTa9xNMega+GAxlqsC178Fpl3qNpCIQEUmU5Z/BsG6weRk0uAUueRJKlvedSkUgIhJ3e7ZB1qMw629wXAbcPAxOush3qv9QEYiIxNOiMTCiB+xcC+ffDRc/BMVL+U71CyoCEZF42LURRj8Ac/8Fx9eFa96BGuf6TpUvFYGISCw5B3M/gtG9YM92aN4HmvaAosV9JzskFYGISKxsWw0je8LiMVD9XOj0ElSp6zvVYakIRESOVl4ezP47ZP0JcvfDJU9D4z9AkTTfyQpERSAicjQ2LQ2GxC2fChkXQqdBUOEk36kKxXsRmFlN4G2gKpAHDHHODfSbSkTkMPJyYfrLMPFpSCsGHQdBg5sTPh4iFrwXAZAD3Oucm21mZYFZZpblnJvvO5iISL7WzYehXeHH2VCnHXToB+VO8J3qiHkvAufcGmDNgb/vMLMFQHVARSAiySVnH0x9IfhTsjxc+WYwJyiERwE/570Ifs7MMoBzgBn5fKwL0AUgPT09oblERFiVHYyK3rAA6l8FbZ+D0hV9p4qJpCkCMysDfATc45zbfvDHnXNDgCEAmZmZLsHxRCSq9u0K1gGmvxyc/rn+Q6jTxneqmEqKIjCzYgQl8K5z7mPfeUREAFg2GYb/EbYsh8zbodXjULKc71Qx570IzMyAN4AFzrl+vvOIiLB7K2Q9ArPfDi4FvXUkZDT1nSpuvBcB0AS4CfjWzL4+8Fwf59woj5lEJKoWjgruDt65Di74I1zcB4od4ztVXHkvAufcZ0C4l9xFJPx2bgjmA837GKrUg2v/AdUb+E6VEN6LQETEK+fgmw9hzAPBwvDFD0PTe4KbxCJCRSAi0bVtVbBXwJJxUOO8YEjc8af5TpVwRXwHEBE5Uv2zFh/ZF+blwczXYXDjYPvINn+G28dGsgRARSAiITZwwpLCf9GmpfD3jjDy3mCjmLu+gPPvCs2k0HjQqSERiYbcHJg+GD59BtJKBKeBzrkx9OMhYkFFICKpb+3cYEjcmq/htA5w6V+gXDXfqZKGikBEUlfOXpjSFz7rD8ccB1e9BXV/p6OAg6gIRCQU+mctzndNIKP3yF887t6yNj1a14GVXwZD4jYugrOugzbPQKkKiYobKuZc+Oa3ZWZmuuzsbN8xRMSzjN4jWf5s+18+uXcnTHwKZrwK5apDx4FQu5WfgEnGzGY55zIPfl5HBCKSOpZ+GgyJ2/oDnPd7aPUolCjrO1XSUxGISPjt3gLjHoav3oGKp8Bto6HWBb5ThYaKQETCbcHw4J6AXRuhaU+46AEoVtJ3qlBREYhIOO1Yx7jqb8AHE6Bq/WDDmBPO9p0qlFQEIhIuzsGc92FMb+rs3w0t/xSMi47QkLhYUxGISHhs/QGG3wNLJ0DNRsHdwZXr+E4VeioCEUl+eXmQ/QaMfyw4ImjXF877LyiicWmxoCIQkeS2cQkM6wY/fAEnt4AOA+C4Wr5TpRQVgYgkp9z98PmLMOnZYKvI370S3CGs8RAxpyIQkeSzZk4wHmLtN3B6R7j0BShbxXeqlKUiEJHksX8PTHkePhsApSrC1W9D3c6+U6U8FYGIJIcfpgdHAZuWwFnXQ5unNSQuQVQEIuLX3p0w4Qn4cgiUrwk3fgyntPSdKlJUBCLiz3cTgvsCtq2ERv8NLR6BEmV8p4ocFYGIJN5Pm4MhcV+/C5XqBBvHpzfynSqyVAQikljz/hdG3RdMDL3wPmh2v4bEeaYiEJHE2LE2KIAFw6HaWcFaQLUzfacSVAQiEm/OBaeAxvYJ9hBu9Ric3w3S9OsnWegnISLxs2UFDO8Oyz6F9Aug04tQ6RTfqeQgKgIRib28XPjyteCyUDNo/wKce7uGxCUpFYGIxNaGRcGQuJUz4JTW0KE/HFvTdyr5DUlRBGb2JtABWO+cq+c7j4gcgdz9MG0ATH4eipeGy4bAmVdrSFwIJEURAG8BLwFve84hIkfix69gaDdY9y2ccVmwX0CZyr5TSQElRRE456aYWYbvHCJSSPt3B2OiP38RSleGa96F0zv4TiWFlBRFUBBm1gXoApCenu45jYiw4vNgSNzmpdDgZmj9JBxzrO9UcgRCs4TvnBvinMt0zmVWrqxDThFv9myHET3hb+0gLwduHhpcFqoSCK3QHBGISBJYkhUMidu+Ghp3hRYPBQvDEmoqAhE5vF2bYOyD8M0HUPk0uCMLap7nO5XESFIUgZm9BzQHKpnZKuBR59wbflOJCM7BvE9g1P2wZys06wXN7oOiJXwnkxhKiiJwzl3nO4OIHGT7Ghh5LywaCSecA52GQlXd5pOKkqIIRCSJOAez34Zxj0DuXrjkKWj0Bw2JS2H6yYrI/9v8PQz/I3w/BWo1hU6DoOLJvlNJnKkIRCQYEjfjrzDxSbC0YD5Qg1s1JC4i9FMWSRH9sxYf2ReuXwBvXBJcFZRxIXSdAZmaFBol+kmLpIiBE5YU7gty9sGk5+DVC2HzMrj8dbj+AyhfPT4BJWnp1JBIFK2eFQyJWz8P6l0B7Z6H0pV8pxJPVAQiUbLvJ5j0DHwxGMpUheveh1Pb+U4lnqkIRKLi+6nBFUGbl0GDW+CSJ6Fked+pJAmoCERCqH/W4nzXBDJ6j/zF4+4ta9PjwiqQ9SjM+hscdyLcMhxObJaoqBIC5pzznaHQMjMzXXZ2tu8YIkklo/dIlj/b/pdPLhoDI3rAzrXQ+C64+CEoXspPQPHOzGY55zIPfl5HBCKpaNdGGP0AzP0XHF8XrnkHapzrO5UkKRWBSCpxDuZ+BKN7BfsGNH8QmvaEosV9J5MkpiIQSRFV2QTvXQuLx0D1TOj8Ehx/uu9YEgIqApGwy8uD2W8xufTDsCwP2jwDje6EImm+k0lIqAhEwmzTUhjeHZZPpcSJzaDjIKhwou9UEjIqApEwyssNbgr79GlIKw4dBwb3Bpj5TiYhpCIQCZt182FoV/hxNtRpBx36QbkTfKeSEFMRiIRFzl6Y2g+mvgAly8GVb8IZl+soQI6aikAkDFZlw9C7YcMCOPMaaPNnKF3RdypJESoCkWS2bxdMfBqmvxyc/rn+Q6jTxncqSTEqApFktWxyMCRuy3LIvANaPRacEhKJMRWBSLLZvRWyHgk2kK9wEtw6EjKa+k4lKUxFIJJMFo6CkT1h5zpo0j0YEVHsGN+pJMWpCESSwc4NwXygeR9DlXpw7T+gegPfqSQitGexSAwVegN552DOBzD4PFg4Alo8DF0mqQQkoVQEIjFUqA3kt62Cf1wNn3SBiqfAf0+FZvdDWrH4BRTJh04NiSRaXh7MejPYNczlQdvnoOHvNSROvFERiCTSpqUwrBusmAYnNQ9mBB2X4TmURJ2KQCQRcnNg+mD49BkoWgI6D4azb9B4CEkKKgKRI1TQDeSfbOy4aV1fWPM1nNYB2r8AZasmKqbIYWnzepEY+sUG8jl7YUpf+Kw/HHMcXPoXqNtZRwHiTVJvXm9mbYGBQBrwunPuWc+RRI7Oyi+DIXEbF8GZ10LbP0OpCr5TieTLexGYWRowGGgNrAJmmtkw59x8v8lECq8Ue2B0b5jxKpSvATd8BLVb+Y4l8psOex+BmY03s7PimKEh8J1zbplzbh/wPtA5jq8nEh9LJzK2+AMw4xU47w646wuVgIRCQY4IegH9zWwF0Mc5tybGGaoDK3/2eBXQ6OBPMrMuQBeA9PT0GEcQOQq7t8C4h+GrdyhdKh2uGw21LvCdSqTADntE4Jyb7ZxrAYwAxpjZo2YWyylY+a2c/WoF2zk3xDmX6ZzLrFy5cgxfXuQozB8GgxvB1+9B0x5UuHemSkBCp0AjJszMgEXAK0A3YImZ3RSjDKuAmj97XAP4MUbfWyQ+dqyDD26CD2+CMsfD7ycG+wUUK+k7mUihHfbUkJl9BpwEzAOmA7cCC4HuZnahc67LUWaYCdQ2sxOB1cC1wPVH+T1F4sM5mPMejHkQ9u+GFo8E46I1H0hCrCBrBHcC89yvbzjoZmYLjjaAcy7HzO4GxhJcPvqmc27e0X5fkZjb+gMMvweWToCajaHTi1C5ju9UIkftsEXgnJv7Gx9uH4sQzrlRwKhYfC+RmMvLg5mvwfjHg8ft+sJ5/wVFNLxXUsNR3UfgnFsWqyAiSWnD4mBI3MrpcHKLYEjcsbpqTVKL9xvKRJJS7n74fBBMei5YAO78Mpx9vcZDSEpSEYgcbM2cYDzE2m+C2UDt+kLZKr5TicSNikDk3/bvgcnPwbSBULoSXP0/ULeT71QicaciEAFY8UWwFrBpCZx9I7R5KpgYKhIBKgKJtr07gquBZr4G5dPhpk+CRWGRCFERSHQtGQ/Du8P21dDozuDmsBJlfKcSSTgVgUTPT5thbJ/gDuFKdeD2sZD+qzmHIpGhIpDocA7mD4VR9wUTQ5vdH/wpWsJ3MhGvVAQSDTvWwsh7YeEIqHZ2sBZQtb7vVCJJQUUgqc05+Prd4FRQzl5o9Ticfzek6f/6Iv+m/xokdW1ZHiwGL5sEtZpAx0FQ6RTfqUSSjopAUk9eLnw5BCY8AZYG7fvBubdpSJzIIagIJLVsWBSMh1j1JZzSGjoOCDaRF5FDUhFIasjdD9MGwOTnoXgZuPw1qH+VhsSJFICKQMLvx6+Co4B1c+GMy4IhcWW0r7VIQakIJLz274ZJz8LnL0LpynDNu3B6B9+pREJHq2cSTsunwStNgtNB59wAXWf8ogT6Zy32GE4kXFQEEi57tsOInvDWpZCXAzcPDfYOPubYX3zawAlLPAUUCR+dGpLwWDwORvQIhsQ17gotHoLipX2nEgk9FYEkv12bYOyD8M0HUPk0uCMLap7nO5VIylARSPJyDuZ9DKN6wZ6t0KwXNLtPQ+JEYkxFIMlp+xoY2RMWjYITzoHOw6DKGfl+av+sxfmuCWT0HvmLx91b1qZH6zpxiSsSZuac852h0DIzM112drbvGBIPzsHst2HcI5C7F1o8DI3+UOghcRm9R7L82fZxCikSTmY2yzmXefDzOiKQ5LF5WTAk7vspUKspdBoEFU/2nUok5akIxL+8XJj+Ckx8CooUhQ4DoMEtGhInkiAqAvFr/YJgPMTqbKjdBjr0h/LVfacSiRQVgfiRsw8+6w9T+kLJcnDFG1DvipgNievesnZMvo9IFKgIJPFWzwqOAtbPDyaEtn0WSleK6Uvo6iCRglMRSOLs+wk+fRqmvwxlqsJ178Op7XynEok8FYEkxvdTYVg32PJ9sFtY68ehZHnfqUQEz0PnzOwqM5tnZnlm9qtrWyUF7NkWXBL69wOTQW8ZEewaphIQSRq+jwjmApcDf/WcQ+Jh0ZhgSNzOtXBBN2jeB4qX8p1KRA7itQiccwsATNsJppZdG2H0AzD3X3B8Xbj2Hah+ru9UInIIvo8ICszMugBdANLT0z2nkXw5B3M/gtG9gn0DmveBpj2gaHHfyUTkN8S9CMxsPFA1nw895JwbWtDv45wbAgyBYNZQjOJJrGxbHQyJWzwm+Nd/58Fw/Om+U4lIAcS9CJxzreL9GuJRXh7MfgvG/SnYMazNM9DoTiiS5juZiBSQhrlEQNz27920FN7uFCwIVz8H7voczu+qEhAJGd+Xj15mZquA84GRZjbWZ55UFfP9e3NzYNogeOUCWPNNsGfwzcOgwkmxfR0RSQjfVw19AnziM4MU0rp5wXiIH2fDqZdC+35QrprvVCJyFEJz1ZB4lrMXpvaDqS8EN4Nd+SaccXnMhsSJiD8qAjm8VdnBUcCGBVD/6gND4ir6TiUiMaIiSDEx3b933y6YeGBIXLkT4Pp/Qp1LYhlXRJKA9iyOgCPav3fZZBj+R9iyHDLvgFaPBfsGiEhoac9iKZjdWyHrkWAD+Qonw62jIKOJ71QiEkcqAvl/C0fCiJ6waz006Q7NH4Rix/hOJSJxpiIQ2LkhmA8072OoUg+uew+qN/CdSkQSREUQAYfcv9c5+OZDGPNAsDDc4mFocg+kFUtsQBHxSkUQAfleHbR1ZTAa4rssqNEQOr8ElU9NfDgR8U5FEDV5eZD9Box/DFxecE9Awy6aDyQSYSqCKNn4XbBv8A+fw0nNoeMgOK6W71Qi4pmKIApyc+CLF+HTP0OxktDpJTjnRo2HEBFARZD61n4LQ7vCmjlwWgdo/wKUzW+fIBGJKhVBqtq/B6b0hWkD4Jjj4Kq34IzLfKcSkSSkIkhFP8wI1gI2LoKzroc2T0OpCr5TiUiSUhGkkr07YeKTMOOvUL4G3PAR1NZOoSLy21QEqeK7CTD8Hti2Ehr+Hlr+CUqU9Z1KREJARRB2u7fA2Ifg63ehYm24bTTUOt93KhEJERVBmM0fBqPug10b4cJ7oVmv4PJQEZFCUBGE0Y51QQEsGAZV68MN/4RqZ/lOJSIhpSIIE+dgznsw5kHYvxtaPgoXdNOQOBE5KiqCsNiyAkbcA0snQs3GwZC4SoeYKioiUggqgmSXlwczXw+GxJnBpX8Jto4sUsR3MhFJESqCZLZhcXBj2MrpcHJL6DgAjk33nUpEUoyKIBnl7odpA2Hyc1CsFPzuVTjrWg2JE5G4UBEkmzVzgiFxa7+Fup2hXV8oW8V3KhFJYSqCZLF/T3AEMG0glK4E17wDp3f0nUpEIkBFkAxWfAHD7oZN38HZN0Kbp4KJoSIiCaAi8GnvDpjwBHz5GhxbE276BE5u4TuViESMisCXJeOD+wK2rYJGd0KLh6FEGd+pRCSCvBaBmfUFOgL7gKXAbc65rT4zxd1Pm2Fsn+AO4Uqnwh3joGZD36lEJMJ835WUBdRzzp0JLAYe9JwnfpyDef8LgxvCt/+EZvfDnVNVAiLindcjAufcuJ89nA5c6StLXO1YCyPvhYUjoNrZwVpA1fq+U4mIAMm1RnA78MGhPmhmXYAuAOnpIbm71jn46p1gv4DcvdD6CWjcFdKS6X92EYm6uP9GMrPxQNV8PvSQc27ogc95CMgB3j3U93HODQGGAGRmZro4RI2tLctheHdYNgnSL4BOL0KlU3ynEhH5lbgXgXPuNzfNNbNbgA5AS+dc8v+CP5y8XPhySHBZqKVB+35w7m0aEiciScv3VUNtgQeAi5xzP/nMEhPrFwY3hq2aCae0DobEla/hO5WIyG/yfbL6JaAEkGXBQLXpzrk7/UY6Ajn7YNoAmNIXipeBy1+D+ldpSJyIhILvq4bCf9J89exgVPS6uXDG5dDueShT2XcqEZEC831EEF77d8Onz8AXL0Hp4+Haf8Bp7X2nEhEpNBXBkVj+WXAUsHkZNLgZWj8JxxzrO5WIyBFRERTGnu0w/lHIfhOOrQU3D4WTmvtOJSJyVFQEBbV4HIzoATt+hPPvhov7QPHSvlOJiBw1FcHh7NoEYx+Ebz6AyqfB1VlQI9N3KhGRmFERHIpzMO9jGNUL9myDi3rDhT2haAnfyUREYkpFkJ/ta2BkT1g0Ck44BzoPgypn+E4lIhIXKoKfcw5mvw3jHgmGxF3yFDT6g4bEiUhK02+4f9u8LBgS9/0UqNUUOg2Ciif7TiUiEncqgrxcmP4KTHwKihSFDgOgwS0aEicikRHtIlg3PxgSt3oW1GkbTAotX913KhGRhIpmEeTsg8/6wZS/QMlycMUbUO8KDYkTkUiKXhGsmhUcBayfD/WuhHbPQelKvlOJiHgTrSKY3BcmPQNlqsJ1H8CpbX0nEhHxLlpFUOHEYCG49eNQsrzvNCIiSSFaRVD/yuCPiIj8h66RFBGJOBWBiEjEqQhERCJORSAiEnEqAhGRiFMRiIhEnIpARCTiVAQiIhFnzjnfGQrNzDYAK3znOAKVgI2+QyRQ1N4v6D1HRVjfcy3nXOWDnwxlEYSVmWU75zJ950iUqL1f0HuOilR7zzo1JCIScSoCEZGIUxEk1hDfARIsau8X9J6jIqXes9YIREQiTkcEIiIRpyIQEYk4FYEHZnafmTkzS/nNks2sr5ktNLNvzOwTMzvWd6Z4MbO2ZrbIzL4zs96+88SbmdU0s0/NbIGZzTOz7r4zJYKZpZnZV2Y2wneWWFERJJiZ1QRaAz/4zpIgWUA959yZwGLgQc954sLM0oDBQDugLnCdmdX1myrucoB7nXOnA42BrhF4zwDdgQW+Q8SSiiDx+gO9gEis0jvnxjnncg48nA7U8JknjhoC3znnljnn9gHvA509Z4or59wa59zsA3/fQfDLsbrfVPFlZjWA9sDrvrPEkooggcysE7DaOTfHdxZPbgdG+w4RJ9WBlT97vIoU/6X4c2aWAZwDzPCbJO4GEPxDLs93kFiK1ub1CWBm44Gq+XzoIaAPcEliE8Xfb71n59zQA5/zEMGphHcTmS2BLJ/nInHUZ2ZlgI+Ae5xz233niRcz6wCsd87NMrPmvvPEkoogxpxzrfJ73szqAycCc8wMglMks82soXNubQIjxtyh3vO/mdktQAegpUvdG1dWATV/9rgG8KOnLAljZsUISuBd59zHvvPEWROgk5ldCpQEypnZO865Gz3nOmq6ocwTM1sOZDrnwjjBsMDMrC3QD7jIObfBd554MbOiBIvhLYHVwEzgeufcPK/B4siCf9H8HdjsnLvHd55EOnBEcJ9zroPvLLGgNQKJt5eAskCWmX1tZq/6DhQPBxbE7wbGEiyafpjKJXBAE+AmoMWBn+3XB/61LCGjIwIRkYjTEYGISMSpCEREIk5FICIScSoCEZGIUxGIiEScikBEJOJUBCIiEaciEImBA3P5Wx/4+1NmNsh3JpGC0qwhkdh4FHjCzI4nmMLZyXMekQLTncUiMWJmk4EyQPMD8/lFQkGnhkRi4MB02WrAXpWAhI2KQOQomVk1gn0WOgO7zKyN50gihaIiEDkKZlYK+Jhg794FwJPAY15DiRSS1ghERCJORwQiIhGnIhARiTgVgYhIxKkIREQiTkUgIhJxKgIRkYhTEYiIRNz/ATtmiY0NO+6SAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# define a test set\n",
    "Xtest = np.linspace(-5,5,100).reshape(-1,1) # 100 x 1 vector of test inputs\n",
    "\n",
    "# predict the function values at the test points using the maximum likelihood estimator\n",
    "ml_prediction = predict_with_estimate(Xtest, theta_ml)\n",
    "\n",
    "# plot\n",
    "plt.figure()\n",
    "plt.plot(X, y, '+', markersize=10)\n",
    "plt.plot(Xtest, ml_prediction)\n",
    "plt.xlabel(\"$x$\")\n",
    "plt.ylabel(\"$y$\");"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Questions\n",
    "1. Does the solution above look reasonable?\n",
    "2. Play around with different values of $\\theta$. How do the corresponding functions change?\n",
    "3. Modify the training targets $\\mathcal Y$ and re-run your computation. What changes?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let us now look at a different training set, where we add 2.0 to every $y$-value, and compute the maximum likelihood estimate"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYQAAAEGCAYAAABlxeIAAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAP1UlEQVR4nO3dfYxldX3H8feny1ap2JJ0p3ELq9OkS9NqBezNqrFpiAstioE21QTT+NQ2ROPDSGzqRhOoWBNME7coiXQbqNBQqxFqqAuNI2qVP0DvbhYEF5eNwbBKyggF3GppVr/9Yw5xdvbOPszOOWfu3Pcruck59/zm3s/J7M4n5zlVhSRJv9B3AEnS6mAhSJIAC0GS1LAQJEmAhSBJapzSd4Dl2rBhQ01PT/cdQ5LGyq5du35YVVOjlo1tIUxPTzMcDvuOIUljJcn3llrmLiNJEmAhSJIaFoIkCbAQJGnsbJ/d18rnWgiSNGauufOhVj7XQpAkARaCJKlhIUiSgDG+ME2SJsH22X0jjxlMb9t52PzM1s1cfsFZJ/VdGdcH5AwGg/JKZUmTaHrbTh6++qJl/WySXVU1GLXMXUaSJMBCkCQ1LARJEmAhSNLYmdm6uZXPtRAkacyc7NlES7EQJEmAhSBJarReCEmem+QbSe5N8kCSD40Y89Ykc0n2NK+/bDuXJOlwXVyp/Azw6qo6mGQ9cFeSO6rq7kXjPlNV7+ogjyRphNYLoeYvhT7YzK5vXuN5ebQkrWGdHENIsi7JHuAxYLaq7hkx7E+T3Jfkc0k2LfE5lyUZJhnOzc21mlmSJk0nhVBVP62qc4AzgS1JXrJoyL8D01X1UuBLwI1LfM6OqhpU1WBqaqrd0JI0YTo9y6iqngS+Cly46P3Hq+qZZvYfgd/rMpckqZuzjKaSnN5MnwqcDzy4aMzGBbMXA3vbziVJOlwXZxltBG5Mso75AvpsVX0hyVXAsKpuA96T5GLgEPAE8NYOckmSFvB5CJI0QXwegiTpmCwESRJgIUiSGhaCJAmwECRJDQtBkgRYCJKkhoUgSQIsBElSw0KQJAEWgiSpYSFIkgALQZLUsBAkSYCFIElqWAiSJMBCkCQ1LARJEmAhSJIaFoIkCbAQJEkNC0GSBFgIkqSGhSBJAiwESVKj9UJI8twk30hyb5IHknxoxJjnJPlMkv1J7kky3XYuSdLhuthCeAZ4dVWdDZwDXJjkFYvG/AXw31X1m8B24KMd5JIkLdB6IdS8g83s+uZVi4ZdAtzYTH8O2JokbWeTJP1cJ8cQkqxLsgd4DJitqnsWDTkDeASgqg4BTwG/OuJzLksyTDKcm5trO7YkTZROCqGqflpV5wBnAluSvGTRkFFbA4u3IqiqHVU1qKrB1NRUG1ElaWJ1epZRVT0JfBW4cNGiA8AmgCSnAL8CPNFlNkmadF2cZTSV5PRm+lTgfODBRcNuA97STL8e+HJVHbGFIElqzykdfMdG4MYk65gvoM9W1ReSXAUMq+o24Hrgn5PsZ37L4NIOckmSFmi9EKrqPuDcEe9fsWD6f4E3tJ1FkrQ0r1SWJAEWgiSpYSFIkgALQZLUsBAkSYCFIElqWAiSJMBCkCQ1LARJEmAhSJIaFoIkCbAQJEkNC0HSkrbP7us7gjpkIUha0jV3PtR3BHXIQpAkARaCJKlhIUiSgG4eoSlpDGyf3TfymMH0tp2Hzc9s3czlF5zVVSx1KOP6LPvBYFDD4bDvGNKaNr1tJw9ffVHfMbSCkuyqqsGoZe4ykiQBFoIkqWEhSJIAC0HSUcxs3dx3BHXIQpC0JM8mmiwWgiQJsBAkSY3WCyHJpiRfSbI3yQNJZkaMOS/JU0n2NK8r2s4lSTpcF1cqHwLeV1W7kzwf2JVktqq+vWjc16vqdR3kkSSN0PoWQlU9WlW7m+kfAXuBM9r+XknSien0GEKSaeBc4J4Ri1+Z5N4kdyR58RI/f1mSYZLh3Nxci0klafJ0VghJTgNuAd5bVU8vWrwbeFFVnQ18Avj8qM+oqh1VNaiqwdTUVLuBJWnCdFIISdYzXwY3V9Wti5dX1dNVdbCZvh1Yn2RDF9kkSfO6OMsowPXA3qr62BJjXtCMI8mWJtfjbWeTJP1cF2cZvQp4E/CtJHua9z4AvBCgqq4DXg+8I8kh4CfApTWu9+WWpDHVeiFU1V1AjjHmWuDatrNIkpbmlcqSJMBCkCQ1LARJEmAhSJIaFoIkCbAQJEkNC0GSBFgIkqSGhSBJAiwESVLDQpAkAcdRCEm+lOTsLsJIkvpzPFsIfw1sT/JPSTa2HUiS1I9jFkJV7a6qVwNfAP4jyZVJTm0/miSpS8d1DKF5eM13gE8C7wYeSvKmNoNJ42z77L6+I0gn7HiOIdwFfB/YDpwBvBU4D9iSZEeb4aRxdc2dD/UdQTphx/OAnLcDD4x4gtm7k+xtIZMkqQfHLISquv8oiy9awSySpB6d1HUIVfXdlQoiSepX689Ulta67bP7Rh4zmN6287D5ma2bufyCs7qKJZ2wHHloYDwMBoMaDod9x5BGmt62k4evdo+qVp8ku6pqMGqZt66QJAEWgiSpYSFIkgALQWrFzNbNfUeQTljrhZBkU5KvJNmb5IEkMyPGJMnHk+xPcl+Sl7WdS2qTZxNpHHVx2ukh4H1VtTvJ84FdSWar6tsLxrwG2Ny8Xs78PZNe3kE2SVKj9S2Eqnq0qnY30z8C9jJ/T6SFLgFuqnl3A6d7q21J6lanxxCSTAPnAvcsWnQG8MiC+QMcWRokuSzJMMlwbm6urZiSNJE6K4QkpwG3AO+tqqcXLx7xI0dcMVdVO6pqUFWDqampNmJK0sTqpBCSrGe+DG6uqltHDDkAbFowfybwgy6ySZLmdXGWUYDrgb1V9bElht0GvLk52+gVwFNV9Wjb2SRJP9fFWUavAt4EfCvJnua9DwAvBKiq64DbgdcC+4EfA2/rIJckaYHWC6Gq7mL0MYKFYwp4Z9tZJElL80plSRJgIUiSGhaCJAmwECRJDQtBkgRYCJKkhoUgSQIsBElSw0KQJAEWgiSpYSFIkgALQZLUsBAkSYCFIElqWAiSJMBCkCQ1LARJEmAhSJIaFoIkCbAQJEkNC0GSBFgIkqSGhSBJAiwESVLDQpAkARaCJKnReiEkuSHJY0nuX2L5eUmeSrKneV3RdiZJ0pFO6eA7PgVcC9x0lDFfr6rXdZBFkrSE1rcQquprwBNtf48k6eSslmMIr0xyb5I7krx4qUFJLksyTDKcm5vrMp8krXmroRB2Ay+qqrOBTwCfX2pgVe2oqkFVDaampjoLKEmToPdCqKqnq+pgM307sD7Jhp5jSdLE6b0QkrwgSZrpLcxnerzfVJI0eVo/yyjJp4HzgA1JDgBXAusBquo64PXAO5IcAn4CXFpV1XYuSdLhWi+EqnrjMZZfy/xpqZKkHvW+y0iStDpYCJIkwELQKrJ9dl/fEaSJZiFo1bjmzof6jiBNNAtBkgRYCJKkhoUgSQK6uf21dITts/tGHjOY3rbzsPmZrZu5/IKzuoolTbSM60XBg8GghsNh3zG0gqa37eThqy/qO4a0piXZVVWDUcvcZSRJAiwESVLDQpAkARaCVpGZrZv7jiBNNAtBq4ZnE0n9shAkSYCFIElqWAiSJMBCkCQ1LARJEmAhSJIaFoIkCbAQJEkNC0GSBFgIkqSGhSBJAjoohCQ3JHksyf1LLE+SjyfZn+S+JC9rO5Mk6UhdbCF8CrjwKMtfA2xuXpcBn+wgkyRpkdYLoaq+BjxxlCGXADfVvLuB05NsbDuXJOlwq+EYwhnAIwvmDzTvHSHJZUmGSYZzc3OdhJOkSbEaCiEj3qtRA6tqR1UNqmowNTXVcixJmiyroRAOAJsWzJ8J/KCnLJI0sVZDIdwGvLk52+gVwFNV9WjfoSRp0pzS9hck+TRwHrAhyQHgSmA9QFVdB9wOvBbYD/wYeFvbmSRJR2q9EKrqjcdYXsA7284hSTq61bDLSJK0ClgIkiTAQpAkNSayELbP7us7giStOhNZCNfc+VDfESRp1ZnIQpAkHclCkCQBFoIkqdH6hWl92z67b+Qxg+ltOw+bn9m6mcsvOKurWJK06mT+QuHxMxgMajgcLutnp7ft5OGrL1rhRJK0+iXZVVWDUcvcZSRJAiwESVLDQpAkARNaCDNbN/cdQZJWnYksBM8mkqQjTWQhSJKOZCFIkgALQZLUGNsL05LMAd9b5o9vAH64gnH65LqsTmtlXdbKeoDr8qwXVdXUqAVjWwgnI8lwqSv1xo3rsjqtlXVZK+sBrsvxcJeRJAmwECRJjUkthB19B1hBrsvqtFbWZa2sB7guxzSRxxAkSUea1C0ESdIiFoIkCZjgQkjy4ST3JdmT5ItJfr3vTMuV5O+SPNisz78lOb3vTMuV5A1JHkjysyRjd4pgkguTfCfJ/iTb+s6zXEluSPJYkvv7znKykmxK8pUke5t/WzN9Z1qOJM9N8o0k9zbr8aEV/45JPYaQ5Jer6ulm+j3A71TV23uOtSxJ/hD4clUdSvJRgKp6f8+xliXJbwM/A/4B+KuqWt5j8XqQZB2wD7gAOAB8E3hjVX2712DLkOQPgIPATVX1kr7znIwkG4GNVbU7yfOBXcAfj9vvJUmA51XVwSTrgbuAmaq6e6W+Y2K3EJ4tg8bzgLFtxqr6YlUdambvBs7sM8/JqKq9VfWdvnMs0xZgf1V9t6r+D/hX4JKeMy1LVX0NeKLvHCuhqh6tqt3N9I+AvcAZ/aY6cTXvYDO7vnmt6N+tiS0EgCQfSfII8GfAFX3nWSF/DtzRd4gJdQbwyIL5A4zhH561LMk0cC5wT79JlifJuiR7gMeA2apa0fVY04WQ5EtJ7h/xugSgqj5YVZuAm4F39Zv26I61Ls2YDwKHmF+fVet41mVMZcR7Y7vludYkOQ24BXjvoj0EY6OqflpV5zC/F2BLkhXdnXfKSn7YalNV5x/n0H8BdgJXthjnpBxrXZK8BXgdsLVW+YGhE/i9jJsDwKYF82cCP+gpixZo9rnfAtxcVbf2nedkVdWTSb4KXAis2IH/Nb2FcDRJFj5H82Lgwb6ynKwkFwLvBy6uqh/3nWeCfRPYnOQ3kvwicClwW8+ZJl5zMPZ6YG9VfazvPMuVZOrZMwiTnAqczwr/3Zrks4xuAX6L+TNavge8vaq+32+q5UmyH3gO8Hjz1t1jfMbUnwCfAKaAJ4E9VfVH/aY6fkleC/w9sA64oao+0nOkZUnyaeA85m+z/F/AlVV1fa+hlinJ7wNfB77F/P93gA9U1e39pTpxSV4K3Mj8v61fAD5bVVet6HdMaiFIkg43sbuMJEmHsxAkSYCFIElqWAiSJMBCkCQ1LARJEmAhSJIaFoK0Qpp77l/QTP9tko/3nUk6EWv6XkZSx64Erkrya8zfUfPinvNIJ8QrlaUVlOQ/gdOA85p770tjw11G0gpJ8rvARuAZy0DjyEKQVkDzmMabmX9C2v8kGZsb8knPshCkk5Tkl4BbgfdV1V7gw8Df9BpKWgaPIUiSALcQJEkNC0GSBFgIkqSGhSBJAiwESVLDQpAkARaCJKnx/8YDRQrCx2s4AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "ynew = y + 2.0\n",
    "\n",
    "plt.figure()\n",
    "plt.plot(X, ynew, '+', markersize=10)\n",
    "plt.xlabel(\"$x$\")\n",
    "plt.ylabel(\"$y$\");"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[0.499]]\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEGCAYAAABo25JHAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAa/ElEQVR4nO3dd5jU5b3+8fcHWDqIwCIILIuICoKArogg0m0gxMJPxXjMz5xgzNFgNFFsMYooioooRsUSMWqMMRpzxBSEpYmUpYn0XgQpAtJhy3P+eNZzUBdYYGaemfner+viunZm1t17FPfeb/k8jznnEBGR6CoTOoCIiISlIhARiTgVgYhIxKkIREQiTkUgIhJx5UIHOBa1a9d22dnZoWOIiKSUmTNnbnHOZX7/+ZQsguzsbPLy8kLHEBFJKWa2uqTndWpIRCTiVAQiIhGnIhARiTgVgYikrGFjloSOkBZUBCKSsoaPXRo6QlpQEYiIRJyKQEQk4lQEIiIRl5IDZSISPcPGLCnxmkD2wNHfeTygW1N+1eO0RMVKC5aKG9Pk5OQ4TRaLSPbA0awa0jN0jJRhZjOdcznff16nhkREIk5FICIScSoCEZGIUxGISMoa0K1p6AhpQUUgIilLdwfFhopARCTiVAQiIhGnIhARiTgVgYhIxKkIREQiTkUgIhJxKgIRkYhTEYiIRJyKQEQk4lQEIiIRpyIQEYk4FYGISMQFLwIzq2hm081srpnNN7OHQmcSEYmSZNizeD/Q1Tm3y8wygMlm9g/n3NTQwUREoiB4ETi/afKu4ocZxX9SbyNlEZEUFfzUEICZlTWzOcAmYIxzbloJn9PfzPLMLG/z5s2JDykikqaSogicc4XOudZAA6CtmbUo4XNGOudynHM5mZmZiQ8pIpKmkqIIvuWc2w6MBy4JHEVEJDKCF4GZZZpZjeKPKwHdgUVhU4mIREfwi8VAPWCUmZXFF9O7zrmPAmcSEYmM4EXgnPscaBM6h0iqGzZmiTZzl2MS/NSQiMTG8LFLQ0eQFKUiEBGJOBWBiEjEqQhERCIu+MViETl6w8YsKfGaQPbA0d95PKBbU11AliMyv9RPasnJyXF5eXmhY4gkleyBo1k1pGfoGJLEzGymcy7n+8/r1JCISMSpCEREIk5FICKSKooK4/JlVQQiaWJAt6ahI0i8FBbAzFEwvDVsWhjzL6+7hkTShO4OSkNFRbDgAxg3GLYuh/o5UHgg5t9GRSAikmycg6VjYNzD8NU8yGwG174Np18GZjH/dioCEZFksnoKjH0Y1nwGNRrBFSOh5dVQpmzcvqWKQEQkGWyYC2MHwbIxULUu9HwK2vwHlCsf92+tIhARCWnLMsh9BOZ/ABVrQPffQduboXzlhEVQEYiIhPDNOhg/BOa8DeUqwoW/gfa3QcUTEh5FRSASQ9ocRo5o9xaY9BTMeMU/btsfOt4JVTODRVIRiMTQ8LFLVQRSsn074LMR8NnzkL8HWveDTndDjazQyVQEIiJxlb8Xpr8Mk5+GvdugeR/ocj9kJs8vDCoCEZF4KMyH2W/ChCdg53o4tTt0vR9OTr4t2lUEIiKxVFQEX/wVcgfDtpXQ8Dy46mXIviB0skNSEYgcI20OI9/hHCz5p58F2DQfTmoB/d6FphfFZRo4lrQxjUgMaXOYiFo5yU8Dr5sONU+BLvfBmVdCmeRa1/NQG9MEPyIws4bAG0BdoAgY6ZwbHjaViEgprJ/tC2D5OKh2MvR6Btr8GMpmhE52VIIXAVAA3Omcm2Vm1YCZZjbGObcgdDARkRJtXgzjHoGFf4dKNeGiR+Dc/4SMSqGTHZPgReCc2wBsKP54p5ktBOoDKgIRSS7b18D4x2Hu25BRGToNhPP/CypWD53suAQvgoOZWTbQBphWwmv9gf4AWVnhBzBESqLNYdLUrk1+GjjvNcDgvFug4x1QpXboZDGRNBeLzawqMAEY7Jx7/3Cfq4vFIpIQe7fDlGdh6gtQsN+f/+90N5xQP3SyY5K0F4sBzCwD+Cvw1pFKQEQk7g7sgekvweRnYN92aHEVdL4Xap8aOllcBC8CMzPgVWChc+7p0HlEJMIKDsCsUTBxKOza6GcAuj4A9c4KnSyughcB0AG4AZhnZnOKn7vXOfdxwEwiEiVFhTDvL5D7KGxfDVntoe8oaHR+6GQJEbwInHOTgeQeuxOR9OQcLBrtbwXdvBDqtoTr3/PrAiX5NHAsBS8CEZEgVoz3w2BfzoRaTaHv69CsT9JNAyeCikBEomVdni+AlROgegPo/Ry06gdlo/vjMLrvXESiZdNCfwpo0UdQuRZc/Bjk3AQZFUMnC05FICLpbdsqyH0MPv8zVKjmF4Rrd4v/WAAVgYikq50b/W2gM1+HMmWh/a1wwR1QuWboZElHRSAi6WXP1uJp4BehKB/a3ACd7oLqJ4dOlrRUBCKSHvbvgmkvwKfPwf4d0LIvdB4ItZqETpb0VAQiktoK9vvTPxOHwu7NcNqlfm/gui1CJ0sZKgIRSU1FhTD3HRg/BL5ZA9kd4dq3oWHb0MlSjopARFKLc35DmHGDYctiqNcaLn8GmnSN1DRwLKkIRCQ1OAcrcv0w2PrZUPs0+H9vQLPeKoDjFL1ZaomEYWOWhI4gsbR2Ooy6HP54Bez+Gvr8Hm75DJr3UQnEgI4IJC0NH7uUX/U4LXQMOV5ffeGngZf8A6pkwqVPwDk/gXIVQidLKyoCEUk+W1f4JaHnvQcVqvs9AdrdAuWrhE6WllQEIpI8dqyHCU/A7D9CmQy44HZo/0tNA8eZikBEwtuzFSY/DdNfhqICf/rnwt9Atbqhk0WCikBS3rAxSxg+dukPns8eOPo7jwd0a6rrBslm/06/MfyU5/zHra71m8PXbBw6WaSYcy50hqOWk5Pj8vLyQseQJJY9cDSrhvQMHUMOJX8f5L0Gk56CPVvgjF5+GrhOs9DJ0pqZzXTO5Xz/eR0RiEjiFBbA3Ldh/OOwYx007gTdHoQG54ROFmkqAhGJv6IiWPA3yB0MXy+D+jnwo9/DKZ1CJxNUBCIST87Bsk/8NPBXn0NmM78e0OmXaRAsiagIJC0N6NY0dARZMxU+eQjWTIEajeCKkdDyar9JjCQVFYGkJd0dFNCGz2HcIFj6b6h6Elz2JJx9I5QrHzqZHEJSFIGZvQb0AjY557SIuEgq2rLMXwOY/z5UrAHdfwdtb4bylUMnkyNIiiIAXgdGAG8EziEiR+ubdTDhcZj9FpSrCB1/De1vg0o1QieTUkqKInDOTTSz7NA5ROQo7N4Ck56GGa8ADtr+DDreCVXrhE4mRykpiqA0zKw/0B8gKysrcBqRCNu3Az4bAZ89D/l7oFU/6Hw31ND/l6kqZfYjcM6NdM7lOOdyMjMzQ8dJKVqbX2Iif69fCmJ4K38q6NRu8Iup8KPnVQIpLmWOCOTYaW1+OS6F+TD7Tb8q6M71fkvIbr+Fk9uETiYxoiIQkZIVFfk7gHIH+/0BGrSFK0dC446hk0mMJUURmNmfgM5AbTNbBzzonHs1bCqRiHIOlvzLzwJs/AJOagHXvQOnXaJp4DSVFEXgnLsudAYRAVZN9stBrJ0GJzaGq16FM6+EMilzOVGOQVIUgcSO1uaXY7J+NowdBMvHQrV60OsZaPNjKJsROpkkgPYjiACtzS+HtHkJ5D4CCz6ESifCBXf4eYCMSqGTSRxoPwIR+T/b1/g9Aea+DRmV/a5g598KFauHTiYBqAhEomTXZr8rWN6rgMF5t8AFv4Kqms2JMhWBSBTs3e6Hwaa+AAX7oM31/ijghAahk0kSUBFEgNbmj7ADe2D6SzD5Gdi33d8B1OU+qH1q6GSSRI5YBGb2CXCnc25uAvJIHOjuoAgqOACz34AJQ2HXV9D0Ir85fL1WoZNJEirNEcFdwDAzWw3c65zbEOdMInKsigph3nsw/lHYtgqyzoe+f4BG7UMnkyR2xCJwzs0CuprZVcA/zex94Ann3N64pxOR0nEOFn8M4x6BTQugbku4/j04tbumgeWISjUuaGYGLAZeAG4DlprZDfEMJiKltGICvNId3ukHBfvh6teg/0Ro2kMlIKVSmmsEk4FTgPnAVOAnwCJggJl1dM71j2tCESnZupkw9iFYOQGq14fez/m9AcrqHhA5OqX5G/NzYL774QjybWa2MA6ZRORwNi30p4AWfQSVa8HFj0LOTyGjYuhkkqJKc43gi8O8rHULRBJl2yoYPwTmvgPlq0Lne+H8X0CFaqGTSYo7rmNI59yKWAURkUPYuREmDoWZr0OZstD+Vr8mUOWaoZNJmtDJRJFktXcbfDocpr4IhQfg7Bv8NHD1k0MnkzSjIhBJNgd2w7QXfQns+wZaXA1d7oVaTUInkzSlIhBJFgX7/emfiU/C7k1+R7Cu9/uZAJE4UhGIhFZUCJ//GXIfg2/WQKML4Jo3Ieu80MkkIlQEIqE4Bwv/298KumUx1GsNlz8DTbpqEEwSSkUgkmjOwYpcvzfw+tlQ+zToOwqa91EBSBAqApFEWjvDTwOvmgQnNIQ+z8NZ12oaWILS3z6RRNg4358CWvwxVMmES5+Ac34C5SqETiaiIhCJq60r/EXgeX+BCtX9XUDn3QIVqoZOJvK/kqIIzOwSYDhQFnjFOTckcCSR47NjA0x8Ama9AWUyoMMA/0fTwJKEgheBmZUFngd6AOuAGWb2d+fcgrDJRI7Bnq0weRhMHwlFBf70z4W/gWp1QycTOaTgRQC0BZZ9u26Rmb0D9AFUBJI69u/yG8NPeRb274SzroHOA6Fm49DJRI4oGYqgPrD2oMfrgB9M0phZf6A/QFZWVmKSiRxJ/j6Y+Qc/DbxnC5zRy18HqNMsdDKRUkuGIijpxunv732Ac24kMBIgJyfnB6+LJFRhAcz9k18Wesc6aNwJuv0WGuSETiZy1JKhCNYBDQ963ABYHyiLyOEVFcHCD2HcYPh6KdQ/B370PJzSOXQykWOWDEUwA2hqZo2BL4FrgX5hI4l8j3OwbCyMexg2zIXMZnDNW3BGT00DS8oLXgTOuQIzuxX4F/720decc/MDxxL5P2umwicPwZopUKMRXPEStOzrN4kRSQPBiwDAOfcx8HHoHCLf8dU8GDsIlv4Lqp4Elz0JZ98I5cqHTiYSU0lRBCJJ5evlkDsYvvgrVDwBuj0I590M5auETiYSFyoCkW998yVMeBxmv+nXAOp4J7T/JVSqETqZSFypCER2fw2Tn4bpLwMO2v7Ml0DVOqGTiSSEikCia98O+Ox5+GwE5O+BVtf5aeAaGliUaFERSPTk74UZr8Ckp2HvVmjW208DZ54eOplIECoCiY7CfJjzFox/HHau91tCdn0A6p8dOplIUCoCSX9FRTD/fX8n0NYV0OBcuHIkNO4YOplIUlARSPpyDpb+288CbJwHdc6E696B0y7RNLDIQVQEkp5Wfeo3h187FU5sDFe+DC2uhjJlQicTSToqAkkv6+f4Alg+FqrVg17DoM0NUDYjdDKRpKUikPSweYm/BrDgb1DpROgxyM8DZFQKnUwk6akIJLVtXwsThsCct6FcJbjwLmh/q18aQkRKRUUgqWnXZpj0FOS96h+f93O44A6omhk2l0gKUhFIatn3DUwZ4SeCC/ZC6+uh091Qo+GR/1kRKZGKQFLDgT0wfSRMHgb7tsOZV0CX+6B209DJRFKeikCSW8EBmP0GTBgKu76CU3tAtwegXqvQyUTShopAklNRod8PIHcwbFsFDdtB3z9Ao/ahk4mkHRWBJBfnYPE/YNwg2LQATmoJ/f4CTXtoGlgkTlQEkjxWTvTDYOtmQM0mcPVr0PwKTQOLxJmKQML7cqYvgBXjodrJcPlwfzeQpoFFEkJFIOFsWuRPAS36CCrXgosfhZyfQkbF0MlEIkVFIIm3bTWMfww+/zNkVIHO90C7X0DF6qGTiUSSikASZ+dGmPQk5P0BrIz/4X/BHVClVuhkIpEWtAjMrC/wO6AZ0NY5lxcyj8TJ3m3w6bMw7UUo2A9n3+DXBDqhfuhkIkL4I4IvgCuBlwLnkHg4sBumvQSfPuOXhmhxlZ8GrtUkdDIROUjQInDOLQQw3R+eXgoOwMzXYeJQ2L0Jml7sp4HrtgydTERKEPqIoNTMrD/QHyArKytwGilRUaG/ADz+Mdi+Bhp1gGv+CFntQicTkcOIexGY2SdA3RJeus8592Fpv45zbiQwEiAnJ8fFKJ7EgnOw8L9h3COwZbFfB6jXMGjSTdPAIikg7kXgnOse7+8hAS3P9cNg62dBrabQdxQ076MCEEkhKXNqSJLMujwY+5BfFqJ6A+g9AlpdB2X1V0ok1YS+ffQK4DkgExhtZnOccxeHzCRHsHGBPwW0eDRUrg2XDIGcm6BchdDJROQYhb5r6APgg5AZpJS2riyeBn4XKlSDLvdDu1ugQtXQyUTkOOk4Xg5vxwZ/G+isUVAmAzr8EjrcDpVrhk4mIjGiIpCS7dnqB8GmjYSifDjnJ9Dx11C9XuhkIhJjKgL5rv27YOoLMOVZ2L8TzroGOg+Emo1DJxOROFERiFewH/Jeg4lPwp4tcHpP6Ho/nNQ8dDIRiTMVQdQVFsDn78D4IfDNWmh8IXR7EBrkhE4mIgmiIogq52DBh35z+C1L4OSzofdz0KRL6GQikmAqgqhxDpaPhbGDYMMcyDwDrnkLzuipaWCRiFIRRMmaaX45iNWToUYWXPEStOwLZcqGTiYiAakIouCreX4aeMk/oUoduOxJOPtGKFc+dDIRSQIqgnT29XLIfRS+eA8qnuAvAp93M5SvEjqZiCQRFUE62rEeJjwOs/7o1wDqeCe0vw0qnRg6mYgkIRVBOtn9NUx+Gqa/DK4Izv2pnwaudlLoZCKSxFQE6WDfDpj6e5gyAvJ3w1nX+mngExuFTiYiKUBFkMry98KMV2HSU7B3KzS73K8KWueM0MlEJIWoCFJRYT7MeQvGPw4718MpXaDbb6H+2aGTiUgKUhGkkqIiWPABjBsMW5dDg3Phypf8shAiIsdIRZAKnIOlY2Dcw34moE5zuPZPcPqlmgYWkeOmIkh2q6f4aeA1n8GJ2XDly9DiKk0Di0jMqAiS1fo5MG4QLPsEqtaFnk9Bm//QNLCIxJyKINlsWeqXg1jwNz8A1uNhOPdnUL5y6GQikqZUBMli+1qYMATmvA3lKsGFd0H7W/3SECIicaQiCG33Fj8HMOMV/7jtzX5JiKqZYXOJSGSoCELZ942fBJ76e8jfA637QaeBUKNh6GQiEjFBi8DMhgKXAweA5cD/d85tD5kp7vL3wvSRMHkY7N0GzX8EXe6DzNNCJxORiAp9RDAGuMc5V2BmjwP3AHcHzhQfhfkw6w2YOBR2boBTu0PXB+Dk1qGTiUjEBS0C59y/D3o4Fbg6VJa4KSry+wHkDoZtq6BhO7jqVcjuEDqZiAgQ/ojgYDcBfz7Ui2bWH+gPkJWVlahMx845vyPY2EGwaT6c1BL6vQtNL9I0sIgklbgXgZl9AtQt4aX7nHMfFn/OfUAB8Nahvo5zbiQwEiAnJ8fFIWrsrJzkp4HXTYeap/gjgDOvhDJlQicTEfmBuBeBc6774V43sxuBXkA351xy/4A/ki9n+QJYkQvVTobLh0Pr66FsRuhkIiKHFPquoUvwF4c7Oef2hMxyXDYv9tPAC/8OlWrCRY/Auf8JGZVCJxMROaLQ1whGABWAMebPm091zv08bKSjsG213xt47p8gowp0vgfa/QIqVg+dTESk1ELfNXRqyO9/zHZtgolPQt5rYGX8D/8L7oAqtUInExE5aqGPCFLL3u0w5VmY+gIU7Ic2P4ZOd8MJ9UMnExE5ZiqC0jiwG6a9BJ8+45eGaHGVnwau1SR0MhGR46YiOJyCAzBrlJ8G3rXRzwB0fQDqnRU6mYhIzKgISlJUCPP+ArmPwvbVkNUe+o6CRueHTiYiEnMqgoM5B4s+8reCbl4Edc+C69/z6wJpGlhE0pSK4FvLc/0w2PpZUKsp9H0dmvXRNLCIpD0Vwbo8GPsQrJwI1RtA7xHQ6jooq381IhIN0f1pt3GBPwW0eDRUrg2XDIGcm6BchdDJREQSKnpFsHUljH8MPn8XKlSDLvdDu1ugQtXQyUREgohWEUwY6jeIL1MOOvwSOtwOlWuGTiUiElS0iqBGFpx9I1z4G6heL3QaEZGkEK0iaHWN/yMiIv9L90aKiEScikBEJOJUBCIiEaciEBGJOBWBiEjEqQhERCJORSAiEnEqAhGRiDPnXOgMR83MNgOrQ+c4BrWBLaFDJFDU3i/oPUdFqr7nRs65zO8/mZJFkKrMLM85lxM6R6JE7f2C3nNUpNt71qkhEZGIUxGIiESciiCxRoYOkGBRe7+g9xwVafWedY1ARCTidEQgIhJxKgIRkYhTEQRgZr82M2dmtUNniTczG2pmi8zsczP7wMxqhM4UL2Z2iZktNrNlZjYwdJ54M7OGZpZrZgvNbL6ZDQidKRHMrKyZzTazj0JniRUVQYKZWUOgB7AmdJYEGQO0cM6dBSwB7gmcJy7MrCzwPHAp0By4zsyah00VdwXAnc65ZkA74L8i8J4BBgALQ4eIJRVB4g0D7gIicZXeOfdv51xB8cOpQIOQeeKoLbDMObfCOXcAeAfoEzhTXDnnNjjnZhV/vBP/w7F+2FTxZWYNgJ7AK6GzxJKKIIHMrDfwpXNubugsgdwE/CN0iDipD6w96PE60vyH4sHMLBtoA0wLmyTunsH/IlcUOkgsRWvz+gQws0+AuiW8dB9wL3BRYhPF3+Hes3Puw+LPuQ9/KuGtRGZLICvhuUgc9ZlZVeCvwO3OuR2h88SLmfUCNjnnZppZ59B5YklFEGPOue4lPW9mLYHGwFwzA3+KZJaZtXXOfZXAiDF3qPf8LTO7EegFdHPpO7iyDmh40OMGwPpAWRLGzDLwJfCWc+790HnirAPQ28wuAyoC1c3sTefcjwPnOm4aKAvEzFYBOc65VFzBsNTM7BLgaaCTc25z6DzxYmbl8BfDuwFfAjOAfs65+UGDxZH532hGAVudc7eHzpNIxUcEv3bO9QqdJRZ0jUDibQRQDRhjZnPM7MXQgeKh+IL4rcC/8BdN303nEijWAbgB6Fr833ZO8W/LkmJ0RCAiEnE6IhARiTgVgYhIxKkIREQiTkUgIhJxKgIRkYhTEYiIRJyKQEQk4lQEIjFQvC5/j+KPHzGzZ0NnEiktrTUkEhsPAg+bWR38Kpy9A+cRKTVNFovEiJlNAKoCnYvX5xdJCTo1JBIDxavL1gP2qwQk1agIRI6TmdXD77PQB9htZhcHjiRyVFQEIsfBzCoD7+P37l0IDAJ+FzSUyFHSNQIRkYjTEYGISMSpCEREIk5FICIScSoCEZGIUxGIiEScikBEJOJUBCIiEfc/c5JtPkO1ueAAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# get maximum likelihood estimate\n",
    "theta_ml = max_lik_estimate(X, ynew)\n",
    "print(theta_ml)\n",
    "\n",
    "# define a test set\n",
    "Xtest = np.linspace(-5,5,100).reshape(-1,1) # 100 x 1 vector of test inputs\n",
    "\n",
    "# predict the function values at the test points using the maximum likelihood estimator\n",
    "ml_prediction = predict_with_estimate(Xtest, theta_ml)\n",
    "\n",
    "# plot\n",
    "plt.figure()\n",
    "plt.plot(X, ynew, '+', markersize=10)\n",
    "plt.plot(Xtest, ml_prediction)\n",
    "plt.xlabel(\"$x$\")\n",
    "plt.ylabel(\"$y$\");"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Question:\n",
    "1. This maximum likelihood estimate doesn't look too good: The orange line is too far away from the observations although we just shifted them by 2. Why is this the case?\n",
    "2. How can we fix this problem?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let us now define a linear regression model that is slightly more flexible:\n",
    "$$\n",
    "y = \\theta_0 + \\boldsymbol x^T \\boldsymbol\\theta_1 + \\epsilon\\,,\\quad \\epsilon\\sim\\mathcal N(0,\\sigma^2)\n",
    "$$\n",
    "Here, we added an offset (bias) parameter $\\theta_0$ to our original model."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Question:\n",
    "1. What is the effect of this bias parameter, i.e., what additional flexibility does it offer?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If we now define the inputs to be the augmented vector $\\boldsymbol x_{\\text{aug}} = \\begin{bmatrix}1\\\\\\boldsymbol x\\end{bmatrix}$, we can write the new linear regression model as \n",
    "$$\n",
    "y = \\boldsymbol x_{\\text{aug}}^T\\boldsymbol\\theta_{\\text{aug}} + \\epsilon\\,,\\quad \\boldsymbol\\theta_{\\text{aug}} = \\begin{bmatrix}\n",
    "\\theta_0\\\\\n",
    "\\boldsymbol\\theta_1\n",
    "\\end{bmatrix}\\,.\n",
    "$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "N, D = X.shape\n",
    "X_aug = np.hstack([np.ones((N,1)), X]) # augmented training inputs of size N x (D+1)\n",
    "theta_aug = np.zeros((D+1, 1)) # new theta vector of size (D+1) x 1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let us now compute the maximum likelihood estimator for this setting.\n",
    "_Hint:_ If possible, re-use code that you have already written"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "## EDIT THIS FUNCTION\n",
    "def max_lik_estimate_aug(X_aug, y):\n",
    "    \n",
    "    theta_aug_ml = max_lik_estimate(X_aug, y) ## <-- SOLUTION\n",
    "    \n",
    "    return theta_aug_ml"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "theta_aug_ml = max_lik_estimate_aug(X_aug, y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, we can make predictions again:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEGCAYAAABo25JHAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAb0UlEQVR4nO3dd5RUVbr+8e9LS5AgIA2CQNOIoANiwBZQUJEgoIQRdcSA8Tdc70UFTCQZc2QUUHEUw0+9MqZRLzk0qCQFbFCUIDkIkjMITYd9/zg4Fwna0FW1q+o8n7V6SVf16npKtJ8+Z5/zbnPOISIi4VXEdwAREfFLRSAiEnIqAhGRkFMRiIiEnIpARCTkTvAd4Hikpqa69PR03zFERBLK7NmzNzvnKh76eEIWQXp6OllZWb5jiIgkFDNbdaTHdWpIRCTkVAQiIiGnIhARCTkVgYhIyKkIRERCTkUgIhJyKgIRkZBTEYiIJIJftsLY3rBvR8S/tYpARCSeOQfzP4MhDeGb12HVVxF/iYS8s1hEJBR2roMx98OPo6DKudDlM6hcP+IvoyIQEYk3zsG3/w3jH4K8bGj1GDTuBinR+ZGtIhARiSdbV8DI7rBiMtRoAh1eggq1ovqSKgIRkXiQnwczX4PPHwdLgXYDocGtUCT6S7kqAhER3zYuhOF3wdosqN06KIGyVWP28ioCERFfcvfD9EEw+TkoXgY6vQH1rwGzmMZQEYiI+LB2Ngy/GzbOh7OuhrbPQalUL1FUBCIisbT/F/jyafj6ZShdGa7/AM5o6zWSikBEJFZWTIWR98DW5dDgFrj8cShR1ncqFYGISNTt2wmZf4PZ/x/K14RbRkLNS3yn+jcVgYhINC0aB6N6wu71cOFdcFk/KFbSd6rfUBGIiETDns0wthfM+xdUqgvXvQfVzved6ohUBCIikeQczPsExj4YnBJq1hea9oQTivlOdlQqAhGRSNn5c3AaaPE4qHo+dHgZTqnrO9UfUhGIiBSWczDnHZjQH/JyoPVT0OhOKJLiO1mBqAhERApj63IYcQ+snArpF0OHF+Hk03ynOiYqAhGR45GfBzNegc+fhJSi0P5FaHBzzMdDRIKKQETkWG1YAMO7wc9zoE5baPcCnHSq71THTUUgIlJQufth6vPBR4mycM1bUK9TQh4FHExFICJSEGuyglHRmxZC/WuhzbNQqoLvVBHhvQjMrDrwLlAZyAeGOucG+00lInLA/j3BOsCMV4LTPzd8BHVa+04VUd6LAMgF7nPOzTGzMsBsM8t0zi3wHUxEQm755GBI3LaVkHE7tHwUSpzkO1XEeS8C59w6YN2BP+8ys4VAVUBFICJ+7N0Omf1hzrvBpaC3job0pr5TRY33IjiYmaUD5wEzj/BcV6ArQFpaWkxziUiI/DgGRt8LuzdAk+7QrA8UPdF3qqiKmyIws9LAJ0AP59zOQ593zg0FhgJkZGS4GMcTkWS3e1MwH2j+p1CpHnT+J1Rt4DtVTMRFEZhZUYISGOac+9R3HhEJEefgh4+DSaH7d8NlDwVHAnE8JC7SvBeBmRnwJrDQOfeC7zwiEiI71sCoe2HJeKh2QTAkrtKZvlPFnPciAJoAXYAfzOy7A4/1dc6N8ZhJRJJZfn6wW1jmw+DyoPXT0Og/EmZIXKR5LwLn3DQgsW/LE5HEsWVZMCRu1TQ4rRm0Hwzl0z2H8st7EYiIxEReLswYAl88BSnFg9NA592U8OMhIkFFICLJb/0PwXiIdd/Bme3gir/DSVV8p4obKgIRSV652TBlAEwbCCeWh2vfhrp/1lHAIYr4DiAicrwGZi4++pM/zYJXLw6KoP610G0W1LtKJXAEKgIRSViDJy05/MHs3TC2N7x5OeT8Ajd+Ale9CiVPjn3ABKFTQyKSPJZ9EQyJ274aLvgrtHwYipfxnSruqQhEJPHt3QYTHoJv34MKp8NtY6HGRb5TJQwVgYgktoUjYfR9sGczNL0XLu0FRUv4TpVQVAQikhAGZi7+zZpARbYzpOjb8OEsFuTX4IGcx5g/sSbd3Wp6tqrjLWciMucSb5BnRkaGy8rK8h1DRHxwDuZ+AON6k713D8Vb9oGL7oGUor6TxT0zm+2cyzj0cR0RiEji2L4aRvaAZZOgemOuWHo1ky7u6jtVwtPloyIS//LzYdbrMKQxrJ4BbQfAbWNZ5qr6TpYUdEQgIvFt8xIYcTes/hpqNYd2g6B8Dd+pkoqKQETiU14OfPUSfPlMsFXkn/8B51z/mzuDu7eo7TFg8lARiEj8WTc3GBK3/nv4U3u44nkoc8phX6argyJDRSAi8SNnH0x5DqYNgpIV4C/vQt2OvlMlPRWBiMSH1TOCo4AtS+CcG6D1k5oPFCMqAhHxK3sXTHosuCqobHW46VM4vYXvVKGiIhARf5ZODO4L2LEm2DO4eX8oXtp3qtBREYhI7P2yFcb3hbnvQ2oduH0cpDX2nSq0VAQiElvz/wfG3B+UwcX3wyUPaEicZyoCEYmNXeuDAlg4EqqcE6wFVDnbdypBRSAi0eYcfDcsOBWUmw0tH4EL74YU/fiJF/qbEJHo2bYKRnaH5V9A2kXQ4SVIPd13KjmEikBEIi8/L7gcdNJjwUiIK5+H82+HIppzGY9UBCISWZsWBUPifpoJp7eCdgOhXHXfqeR3qAhEJDLycmD6IJj8HBQrBVcNhbP/8pshcRKfVAQiUng/fwvD74YNP0C9q4L9AkpX9J1KCkhFICLHL2dvMCb6q5egVEW4bhj8qZ3vVHKMVAQicnxWfRUMidu6DBrcDK0ehxPL+U4lx0FFICLHZt9OmPgIZL0J5WrAzcPhtGaeQ0lhxEURmNlbQDtgo3PuLN95ROQolmQGQ+J2roXG3aB5v2BhWBJavFzU+zbQxncIETmKPVvg064w7JpgOugdmdDmKZVAkoiLIwLn3BQzS/edQ0QO4RzM/wzGPAD7tsMlD8Il98MJxX0nkwiKiyIoCDPrCnQFSEtL85xGJAR2roPR98Gi0XDqedBhOFTWmdtklDBF4JwbCgwFyMjIcJ7jiCQv52DOuzChP+Rlw+VPQKP/1JC4JKa/WRH5P1tXwMh7YMUUqNEUOrwIFWr5TiVRpiIQkWBI3MzX4PPHwVKC+UANbtWQuJCIiyIws/eBZkCqma0BHnbOvek3lUhIbFwY3Bi2Ngtqtw5KoGxV36kkhuKiCJxz1/vOIBI6ufth2kCYMgCKl4FOb0D9azQkLoTioghEJMbWzg6GxG2cD2ddDW2fg1KpvlOJJyoCkTDZ/wt8+RR8PQRKV4brP4Az2vpOJZ5pJUgkSQzMXPz7X7BiKrzaJJgUel4X6DZDJSCAikAkaQyetOTIT+zbEcwHeqddcI/ALSODy0JLlI1tQIlbOjUkkswWjYNRPWH3erjwLrisHxQr6TuVxBkVgUgy2rMZxvaCef+CSnXhuveg2vm+U0mcUhGIJBPnYN4nMPbBYN+AZn2g6b1wQjHfySSOqQhEEtDAzMWHrQlUZgsT/3YZLVO+5dv803kwpxdX5FxGT5WA/AFzLvHmt2VkZLisrCzfMUTiQ34+zHmbXSP7UqYo0KI/NLoTiqT4TiZxxsxmO+cyDn1cRwQiiWzLMhjZHVZO5fv8ejT5r2Fwck3fqSTBqAhEElF+XnBT2BdPQkoxaD+YGz9OZaVKQI6DikAk0WxYAMO7wc9zoE5baPcCnHQqfDzadzJJUCoCkUSRmw1TX4Cpz0OJk+Cat6Bep38PieveorbngJKoVAQiiWBNVjAqetNCOPs6aP00lKrwmy/p2aqOp3CS6FQEIvFs/x74/EmY8Upw+ueGj6BOa9+pJMmoCETi1fLJwbaR21ZCxh3Q8pHglJBIhKkIROLN3u2Q2T/YQP7k0+DW0ZDe1HcqSWIqApF48uMYGH0v7N4ATboHIyKKnug7lSQ5FYFIPNi9KZgPNP9TOOUs6PxPqNrAdyoJCRWBiE/OwfcfwbhewcJw84egSQ9IKeo7mYSIikDElx1rgr0ClkyAahdAh5eh0pm+U0kIqQhEYi0/H2a/BZkPg8uHNs9Cw79qSJx4oyIQiaUty2DE3bBqOpzWDNoPhvLpnkNJ2GnPYpEIOuoG8nm5MH0w/OMi2DAPOg6BLv+jEpC4oCIQiaAjbiC//gd4owVk/g1ObwndZsF5N/17RpCIbzo1JBItudkwZQBMGwgnlodr34G6HVUAEndUBCLR8NOsYEjc5kVwdmdo8zSUPNl3KpEjUhGIRFBJ9sHY3jDzVShbDW78BGq39B1L5Hf9YRGY2UTgPufc3BjkEUkYh24g37TID4wv9gbM3MS7ua14dkNn9ryZTfcWizUiWuLaH25eb2YNgL8Dq4C+zrl1sQj2e7R5vcSVvdtgwkPw7Xssy69CrTveghoX+U4lcpijbV7/h1cNOefmOOeaA6OAcWb2sJlpCpYIwIIRMKQRfPc+NO3JFfufVglIwinQ5aNmZsAi4B/A3cASM+sSqRBm1sbMFpnZUjPrHanvKxI1uzbAh13goy5QuhL89XNo+QjZFPOdTOSYFWSNYBpwGjAfmAHcCvwIdDezi51zXQsTwMxSgCFAK2AN8I2ZjXDOLSjM9xWJCudg7vswrg/k7IXm/YNx0RoSJwmsIFcN3QnMd4cvJtxtZgsjkKEhsNQ5txzAzD4AOgIqAokv21fDyB6wbBJUbwwdXoKKv10E1gbykoj+sAicc/N+5+krI5ChKvDTQZ+vARod+kVm1hXoCpCWlhaBlxUpoPx8+OZ1mPho8HnbAXDB/4Mih59Z1dVBkogKdR/Br7/FF9KRbrM87FIm59xQYCgEVw1F4HVF/timxcGQuJ9mQK3mwZC4cvpFRJJLPNxQtgaoftDn1YCfPWURCeTlwFcvwpfPQtES0PEVOPcGjYeQpBQPRfANUNvMagJrgc7ADX4jSaitmxuMh1j/fTAbqO0AKHOK71QiUeO9CJxzuWZ2FzAeSAHecs7N9xxLwihnH0x+NhgXXSoV/vLfULeD71QiUee9CACcc2OAMb5zSIit+jpYC9iyBM69CVo/EUwMFQmBuCgCEW+ydwVXA33zOpRNgy6fBYvCIiGiIpDwWjIRRnaHnWuh0Z3BzWHFS/tOJRJzKgIJn1+2wvi+wR3CqXXg9vGQdtitKyKhoSKQ8HAOFgyHMfcHE0MveSD4OKG472QiXqkIJBx2rYfR98GPo6DKucFaQOX6vlOJxAUVgSQ35+C7YcGpoNxsaPkoXHgXpOg/fZFf6f8GSV7bVgaLwcu/hBpNoP2LkHq671QicUdFIMknPw9mDYVJj4GlwJUvwPm3HXFInIioCCTZbFoUjIdYMwtObwXtBwWbyIvIUakIJDnk5cD0QTD5OShWGjq9DvWv1ZA4kQJQEUji+/nb4Chgwzyo1wnaPgelK/pOJZIwVASSuHL2wpdPw1cvQalK0PmfcGYk9koSCRcVgSSmldODIXFbl0GDm6HV43BiOd+pRBKSikASy76dMPERyHoTytWAm4fDac08hxJJbCoCSRyLJ8ConsGQuMbdoHk/KFbKdyqRhKcikPi3ZwuM7wPffwgVz4Q7MqH6Bb5TiSQN3WEj8cs5mPcJDGkY/PPSXvAfUwpUAgMzF8cgoEhyUBFIfNq5Dj64Af51O5SrHhTAZX0LPCl08KQlUQ4okjx0akjii3Mw512Y0B/ysuHyJ6DRf2pInEgU6f8uiR9blwdD4lZMgRpNocOLUKGW71QiSU9FIP7l58GMf8DnT0CRE6DdIGhwi4bEicSIikD82rgwGA+xNgtqt4Z2A6Fs1WP6FgMzFx9xTSC99+jffN69RW16tqpTqLgiycicc74zHLOMjAyXlZXlO4YURu5+mDYQpgyAEicF84HOujpiQ+LSe49m5TMaNyFyMDOb7ZzLOPRxHRFI7K2dHRwFbFwQTAht8wyUSvWdSiS0VAQSO/t/gS+ehBmvQOnKcP0HcEZb36lEQk9FILGxYmowJG7bimC3sFaPQomyvlOJCCoCibZ9OyDzbzD7bShfE24ZBTUvjvrLdm9RO+qvIZIsVAQSPYvGBUPidq+Hi+6GZn2hWMmYvLSuDhIpOBWBRN6ezTC2F8z7F1SqC53fg6rn+04lIkehIpDI+XVI3NgHg30DmvWFpj3hhGK+k4nI7/B666aZXWtm880s38wOu7ZVEsiOtfB+Z/jkDiifDndOhWa9VAIiCcD3EcE8oBPwmucccrzy82HOO8GQuPxcaP0UNLoTiqT4TiYiBeS1CJxzCwEsQneTSoxtWRYMiVs5FWpeAu1fhJNr+k4lIsfI9xFBgZlZV6ArQFpamuc0IZeXG9wU9sWTkFIsKIAGN0dsPISIxFbUi8DMJgKVj/BUP+fc8IJ+H+fcUGAoBLOGIhRPjtWG+cF4iJ/nwBlXwJXPw0mn+k4lIoUQ9SJwzrWM9mtIDORmw9QXYOrzwR3B17wF9TrpKEAkCWjgewgUev/eNVnw2qUw+RmodxV0mxXRSaEi4pfvy0evMrM1wIXAaDMb7zNPsjru/Xv374FxfeGNlpC9E274GK5+HUpViGxAEfHK91VDnwGf+cwgR7F8Moy8B7athIw7oOUjwb4BIpJ0EuaqIYmRvdshs3+wgfzJteDWMZDexHcqEYkiFYH8nx9Hw6h7Yc9GaNIdmvWBoif6TiUiUaYiSDLHtX/v7k3BfKD5n8IpZ8H170PVBrGIKyJxQHsWh8BR9+91Dr7/CMb1ChaGL30QmvSAlKKxDykiUac9i+W3tv8U7BWwNBOqNYSOL0PFM3ynEhEPVARhk58PWW/CxEfA5QcbxzfsqiFxIiGmIgiTzUuDfYNXfwWnNQtmBJWv4TuViHimIgiBHs1rwrSB8MXTULQEdHgZzrtJdwaLCKAiSH7rf6DHim6wbi6c2S4YElfmSDMARSSsVATJKmcfTBkA0wfBieXh2reDOUEiIodQESSj1TODtYDNi+CcG6D1k1DyZN+pRCROqQiSSfZu+PxxmPkalK0GN34CtTUFXER+n4ogWSydBCN7wI6foOFfocXfoHgZ36lEJAGoCBLd3m0wvh98Nwwq1IbbxkKNC32nEpEEoiJIZAtGwJj7Yc9muPg+uOTB4PJQEZFjoCJIRLs2BAWwcARUrg83fgxVzvGdSkQSlIogkTgHc9+HcX0gZy+0eBguultD4kSkUFQEiWLbKhjVA5Z9DtUbB0PiUmv7TiUiSUBFEO/y8+GbN4IhcWZwxd+DrSOLeN1uWkSSiIognm1aHNwY9tMMqNUC2g+Ccmm+U4lIklERxKO8HJg+GCY/C0VLwp9fhXM6a0iciESFiiDerJsLw7vB+h+gbkdoOwDKnOI7lYgkMRVBvMjZFxwBTB8MpVLhuvfgT+19pxKREFARxINVX8OIu2DLUjj3Jmj9RDAxVEQkBlQEPmXvgkmPwazXoVx16PIZ1GruO5WIhIyKwJclE4P7AnasgUZ3QvOHoHhp36lEJIRUBLH2y1YY3ze4Qzj1DLhjAlRv6DuViISYiiBWnIMFw4MZQXu3wSUPBB8nFPedTERCTkUQC7vWw+j74MdRUOXcYC2gcn3fqUREABVBdDkH374X7BeQlw2tHoPG3SBF/9pFJH7oJ1K0bFsJI7vD8i8h7SLo8BKknu47lYjIYbwWgZkNANoD+4FlwG3Oue0+MxVafh7MGhpcFmopcOULcP5tGhInInHL90+nTOAs59zZwGKgj+c8hbPxR3irNYzrDTWaQLcZcIEmhYpIfPN6ROCcm3DQpzOAa3xlKZTc/TB9EEwZAMVKQ6fXof61GhInIgkhntYIbgc+PNqTZtYV6AqQlhZHo5jXzglGRW+YB/U6QdvnoHRF36lERAos6kVgZhOBykd4qp9zbviBr+kH5ALDjvZ9nHNDgaEAGRkZLgpRj03OXvjiKfj6ZShVCTr/E8680ncqEZFjFvUicM61/L3nzewWoB3Qwjnn/wd8QaycFhwFbF0ODW6GVo/DieV8pxIROS6+rxpqA/QCLnXO/eIzS4Hs2wkTH4ast6BcDbh5OJzWzHcqEZFC8b1G8DJQHMi0YGF1hnPuTr+RjmLxBBjVE3b9DBfeBZf1hWKlfKcSESk031cNxf8dVnu2wPg+8P2HUPFM+EsmVMvwnUpEJGJ8HxHEL+dg/qcw5kHYtwMu7Q0X36shcSKSdFQER7JzHYy+FxaNgVMbQMeX4ZR6vlOJiESFiuBgzsGcd2FCf8jbD5c/AY3/C4qk+E4mIhI1KoJfbV0OI+6BlVMh/WJoPxgq1PKdSkQk6lQE+Xkw81WY9DikFIV2g6DBLZoPJCKhEe4i2LAguDFsbRbUaRNMCi1b1XcqEZGYCmcR5O6HaS/AlL9DiZPg6jfhrKs1JE5EQil8RbBmNoy4CzYuCCaEtnkGSqX6TiUi4k24imDyAPjyKShdGa7/EM5o4zuRiIh34SqCk2sGC8GtHoUSZX2nERGJC+EqgvrXBB8iIvJvukZSRCTkVAQiIiGnIhARCTkVgYhIyKkIRERCTkUgIhJyKgIRkZBTEYiIhJw553xnOGZmtglY5TvHcUgFNvsOEUNhe7+g9xwWifqeazjnKh76YEIWQaIysyznXIbvHLEStvcLes9hkWzvWaeGRERCTkUgIhJyKoLYGuo7QIyF7f2C3nNYJNV71hqBiEjI6YhARCTkVAQiIiGnIvDAzO43M2dmSb9ZspkNMLMfzex7M/vMzMr5zhQtZtbGzBaZ2VIz6+07T7SZWXUz+8LMFprZfDPr7jtTLJhZipl9a2ajfGeJFBVBjJlZdaAVsNp3lhjJBM5yzp0NLAb6eM4TFWaWAgwB2gJ1gevNrK7fVFGXC9znnPsT0BjoFoL3DNAdWOg7RCSpCGJvIPAgEIpVeufcBOdc7oFPZwDVfOaJoobAUufccufcfuADoKPnTFHlnFvnnJtz4M+7CH44VvWbKrrMrBpwJfCG7yyRpCKIITPrAKx1zs31ncWT24GxvkNESVXgp4M+X0OS/1A8mJmlA+cBM/0mibpBBL/I5fsOEknh2rw+BsxsIlD5CE/1A/oCl8c2UfT93nt2zg0/8DX9CE4lDItlthiyIzwWiqM+MysNfAL0cM7t9J0nWsysHbDROTfbzJr5zhNJKoIIc861PNLjZlYfqAnMNTMITpHMMbOGzrn1MYwYcUd7z78ys1uAdkALl7w3rqwBqh/0eTXgZ09ZYsbMihKUwDDn3Ke+80RZE6CDmV0BlABOMrP3nHM3ec5VaLqhzBMzWwlkOOcScYJhgZlZG+AF4FLn3CbfeaLFzE4gWAxvAawFvgFucM7N9xosiiz4jeYdYKtzrofvPLF04IjgfudcO99ZIkFrBBJtLwNlgEwz+87MXvUdKBoOLIjfBYwnWDT9KJlL4IAmQBeg+YG/2+8O/LYsCUZHBCIiIacjAhGRkFMRiIiEnIpARCTkVAQiIiGnIhARCTkVgYhIyKkIRERCTkUgEgEH5vK3OvDnJ8zsRd+ZRApKs4ZEIuNh4DEzq0QwhbOD5zwiBaY7i0UixMwmA6WBZgfm84skBJ0aEomAA9NlqwDZKgFJNCoCkUIysyoE+yx0BPaYWWvPkUSOiYpApBDMrCTwKcHevQuBx4FHvIYSOUZaIxARCTkdEYiIhJyKQEQk5FQEIiIhpyIQEQk5FYGISMipCEREQk5FICIScv8LYd+ONIPcsaYAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# define a test set (we also need to augment the test inputs with ones)\n",
    "Xtest_aug = np.hstack([np.ones((Xtest.shape[0],1)), Xtest]) # 100 x (D + 1) vector of test inputs\n",
    "\n",
    "# predict the function values at the test points using the maximum likelihood estimator\n",
    "ml_prediction = predict_with_estimate(Xtest_aug, theta_aug_ml)\n",
    "\n",
    "# plot\n",
    "plt.figure()\n",
    "plt.plot(X, y, '+', markersize=10)\n",
    "plt.plot(Xtest, ml_prediction)\n",
    "plt.xlabel(\"$x$\")\n",
    "plt.ylabel(\"$y$\");"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It seems this has solved our problem! \n",
    "#### Question:\n",
    "1. Play around with the first parameter of $\\boldsymbol\\theta_{\\text{aug}}$ and see how the fit of the function changes.\n",
    "2. Play around with the second parameter of $\\boldsymbol\\theta_{\\text{aug}}$ and see how the fit of the function changes."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Nonlinear Features\n",
    "So far, we have looked at linear regression with linear features. This allowed us to fit straight lines. However, linear regression also allows us to fit functions that are nonlinear in the inputs $\\boldsymbol x$, as long as the parameters $\\boldsymbol\\theta$ appear linearly. This means, we can learn functions of the form\n",
    "$$\n",
    "f(\\boldsymbol x, \\boldsymbol\\theta) = \\sum_{k = 1}^K \\theta_k \\phi_k(\\boldsymbol x)\\,,\n",
    "$$\n",
    "where the features $\\phi_k(\\boldsymbol x)$ are (possibly nonlinear) transformations of the inputs $\\boldsymbol x$.\n",
    "\n",
    "Let us have a look at an example where the observations clearly do not lie on a straight line:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYAAAAEGCAYAAABsLkJ6AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAOAElEQVR4nO3de4yld13H8feHXRDKJWA6JrXtOphsiQ1e0Em9YEjDslpp02qisUQJqMmGP4ClwcgoiVWQpEbDUo0hbqCIoUK0rZGwiB0KFUmkMlurtCzdbWpLtyBd1IaLxlr79Y85yO50S2fn9psz3/cr2XTOM2fO832y2/Oe53LOSVUhSernKaMHkCSNYQAkqSkDIElNGQBJasoASFJTO0cPcCbOPvvsmp2dHT2GJE2Vw4cPf7mqZpYvn6oAzM7Osri4OHoMSZoqSe4/3XIPAUlSUwZAkpoyAJLUlAGQpKYMgCQ11SYABxaOjh5BkraUDQ9AkuuSPJTkzpOWfXuShSTHJv993kbPce0txzZ6FZI0VTZjD+BPgEuWLZsHbqmq3cAtk9uSpE204S8Eq6pPJJldtvgK4OLJ1+8FbgXetN7rPrBw9JTf/GfnDwGwf89urtp7wXqvTpKmSjbjA2EmAfhQVb1wcvvhqnruSd//j6o67WGgJPuAfQC7du36ofvvP+0L2p7U7Pwh7rvm0lX9rCRNsySHq2pu+fItfxK4qg5W1VxVzc3MPO6tLCRJqzQqAF9Kcg7A5L8PbfQK9+/ZvdGrkKSpMioAHwReNfn6VcBfbfQKPeYvSafajMtA3w/8PfCCJMeT/ApwDbA3yTFg7+S2JGkTbcZVQK94gm/t2eh1S5Ke2JY/CSxJ2hgGQJKaMgCS1JQBkKSmDIAkNWUAJKkpAyBJTRkASWrKAEhSUwZAkpoyAJLUlAGQpKYMgCQ1ZQAkqSkDIElNGQBJasoASFJTBkCSmjIAktSUAZCkpgyAJDVlACSpKQMgSU0ZAElqygBIUlMGQJKaGhqAJFcluSvJnUnen+TpI+eRpK3owMLRDXncYQFIci7wemCuql4I7ACuHDWPJG1V195ybEMed/QhoJ3AM5LsBM4CvjB4HklqY+eoFVfVg0l+H/g88F/AzVV18/L7JdkH7APYtWvX5g4pSYMcWDh6ym/+s/OHANi/ZzdX7b1gXdaRqlqXBzrjFSfPA24Efh54GPgL4Iaqet8T/czc3FwtLi5u0oSStDXMzh/ivmsuXfXPJzlcVXPLl488BPQy4F+q6kRV/Q9wE/BjA+eRpFZGBuDzwI8kOStJgD3AkYHzSNKWtH/P7g153GEBqKrbgBuA24HPTGY5OGoeSdqq1uuY/3LDTgIDVNXVwNUjZ5CkrkZfBipJGsQASFJTBkCSmjIAktSUAZCkpgyAJDVlACSpKQMgSU0ZAElqygBIUlMGQJKaMgCS1JQBkKSmDIAkNWUAJKkpAyBJTRkASWrKAEhSUwZAkpoyAJLUlAGQpKYMgCQ1ZQAkqSkDIElNGQBJasoASFJTQwOQ5LlJbkjyuSRHkvzoyHkkqZOdg9d/LfCRqvrZJE8Dzho8jyS1MSwASZ4DvAR4NUBVPQI8MmoeSepm5CGg7wZOAO9J8o9J3pXkmcvvlGRfksUkiydOnNj8KSVpmxoZgJ3ADwLvrKoXAV8H5pffqaoOVtVcVc3NzMxs9oyStG2NDMBx4HhV3Ta5fQNLQZAkbYJhAaiqfwUeSPKCyaI9wGdHzSNJ3Yy+Cuh1wPWTK4DuBX5p8DyS1MbQAFTVHcDcyBkkqStfCSxJTRkASWrKAEhSUwZAkpoyAJLUlAGQpKYMgCQ1ZQAkqSkDIElNGQBJasoASFJTBkCSmjIAktSUAZCkpgyAJDVlACSpKQMgSU0ZAElqygBIUlMGQJKaMgCS1NSTBiDJR5N8/2YMI0naPCvZA/g14ECS9yQ5Z6MHkiRtjicNQFXdXlUvBT4EfCTJ1UmesfGjSZI20orOASQJcDfwTuB1wLEkr9zIwSRJG2sl5wA+CTwIHADOBV4NXAxclOTgRg4nSdo4O1dwn9cAd1VVLVv+uiRH1jpAkh3AIvBgVV221seTJK3MSs4B3HmaJ/9vuHQdZtgPrDkkkqQzs6bXAVTVvWv5+STnsRSRd63lcSRJZ270C8HewdJlpo890R2S7EuymGTxxIkTmzeZJG1zwwKQ5DLgoao6/K3uV1UHq2ququZmZmY2aTpJ2v5G7gG8GLg8yX3AB4CXJnnfwHkkqZVhAaiqX6+q86pqFrgS+FhV/eKoeSSpm9HnACRJg6zkdQAbrqpuBW4dPIYkteIegCQ1ZQAkqSkDIElNGQBJasoASFJTBkCSmjIAktSUAZCkpgyAJDVlACSpKQMgSU0ZAElqygBIUlMGQJKaMgCS1JQBkKSmDIAkNWUAJKkpAyBJTRkASWrKAEhSUwZAkpoyAJLUlAGQpKYMgCQ1ZQAkqalhAUhyfpKPJzmS5K4k+0fNIkkd7Ry47keBN1bV7UmeDRxOslBVnx04kyS1MWwPoKq+WFW3T77+KnAEOHfUPJLUzZY4B5BkFngRcNtpvrcvyWKSxRMnTmz2aJK0bQ0PQJJnATcCb6iqryz/flUdrKq5qpqbmZnZ/AElaZsaGoAkT2Xpyf/6qrpp5CyS1M3Iq4ACvBs4UlVvHzWHJHU1cg/gxcArgZcmuWPy5+UD55GkVoZdBlpVnwQyav2S1N3wk8Dq68DC0dEjSK0ZAA1z7S3HRo8gtWYAJKmpkW8FoYYOLBw95Tf/2flDAOzfs5ur9l4waiyppVTV6BlWbG5urhYXF0ePoXUyO3+I+665dPQY0raX5HBVzS1f7iEgSWrKAGiY/Xt2jx5Bas0AaBiP+UtjGQBJasoASFJTBkCSmjIAktSUAZCkpgyAJDVlACSpKQMgSU0ZAElqygBIUlMGQJKaMgCS1JQBkKSmDIAkNWUAJKkpAyBJTRkASWrKAEhSU0MDkOSSJHcnuSfJ/MhZJC05sHB09AjaJMMCkGQH8EfATwEXAq9IcuGoeSQtufaWY6NH0CYZuQdwEXBPVd1bVY8AHwCuGDiPJLWyc+C6zwUeOOn2ceCHl98pyT5gH8CuXbs2ZzKpmQMLR0/5zX92/hAA+/fs5qq9F4waSxtsZABymmX1uAVVB4GDAHNzc4/7vqS1u2rvBf//RD87f4j7rrl08ETaDCMPAR0Hzj/p9nnAFwbNIkntjAzAp4HdSZ6f5GnAlcAHB84jiaXDPuph2CGgqno0yWuBvwF2ANdV1V2j5pG0xGP+fYw8B0BVfRj48MgZJKkrXwksSU0ZAElqygBIUlMGQJKaMgCS1JQBkKSmDIAkNWUAJKkpAyBJTRkASWrKAEhSUwZAkpoyAJLUlAGQ1sGBhaOjR5DOmAGQ1sHJn6crTQsDIElNDf1AGGmaHVg4espv/rPzh4Clj1T0U7U0DVJVo2dYsbm5uVpcXBw9hvQ4s/OHuO+aS0ePIZ1WksNVNbd8uYeAJKkpAyCtg/17do8eQTpjBkBaBx7z1zQyAJLUlAGQpKYMgCQ1ZQAkqSkDIElNTdULwZKcAO5f5Y+fDXx5HccZabtsy3bZDnBbtqrtsi1r3Y7vqqqZ5QunKgBrkWTxdK+Em0bbZVu2y3aA27JVbZdt2ajt8BCQJDVlACSpqU4BODh6gHW0XbZlu2wHuC1b1XbZlg3ZjjbnACRJp+q0ByBJOokBkKSmWgUgyVuT/HOSO5LcnOQ7R8+0Wkl+L8nnJtvzl0meO3qm1Ujyc0nuSvJYkqm8XC/JJUnuTnJPkvnR86xWkuuSPJTkztGzrEWS85N8PMmRyb+t/aNnWq0kT0/yD0n+abItv72uj9/pHECS51TVVyZfvx64sKpeM3isVUnyE8DHqurRJL8LUFVvGjzWGUvyPcBjwB8Dv1pVU/WRb0l2AEeBvcBx4NPAK6rqs0MHW4UkLwG+BvxpVb1w9DyrleQc4Jyquj3Js4HDwE9P6d9JgGdW1deSPBX4JLC/qj61Ho/fag/gG0/+E88EprZ+VXVzVT06ufkp4LyR86xWVR2pqrtHz7EGFwH3VNW9VfUI8AHgisEzrUpVfQL499FzrFVVfbGqbp98/VXgCHDu2KlWp5Z8bXLzqZM/6/a81SoAAEneluQB4BeA3xw9zzr5ZeCvRw/R1LnAAyfdPs6UPtlsR0lmgRcBt42dZPWS7EhyB/AQsFBV67Yt2y4AST6a5M7T/LkCoKreXFXnA9cDrx077bf2ZNsyuc+bgUdZ2p4taSXbMcVymmVTu2e5nSR5FnAj8IZle/9Tpar+t6p+gKW9/IuSrNvhuZ3r9UBbRVW9bIV3/TPgEHD1Bo6zJk+2LUleBVwG7KktfDLnDP5OptFx4PyTbp8HfGHQLJqYHC+/Ebi+qm4aPc96qKqHk9wKXAKsy4n6bbcH8K0kOfmTuy8HPjdqlrVKcgnwJuDyqvrP0fM09mlgd5LnJ3kacCXwwcEztTY5cfpu4EhVvX30PGuRZOYbV/gleQbwMtbxeavbVUA3Ai9g6aqT+4HXVNWDY6danST3AN8G/Ntk0aem8YqmJD8D/CEwAzwM3FFVPzl2qjOT5OXAO4AdwHVV9bbBI61KkvcDF7P01sNfAq6uqncPHWoVkvw48HfAZ1j6fx3gN6rqw+OmWp0k3we8l6V/W08B/ryq3rJuj98pAJKkb2p1CEiS9E0GQJKaMgCS1JQBkKSmDIAkNWUAJKkpAyBJTRkAaQ0m7zu/d/L17yT5g9EzSSu17d4LSNpkVwNvSfIdLL3r5OWD55FWzFcCS2uU5G+BZwEXT95/XpoKHgKS1iDJ9wLnAP/tk7+mjQGQVmny0YPXs/QJYF9PMlVvYicZAGkVkpwF3AS8saqOAG8FfmvoUNIZ8hyAJDXlHoAkNWUAJKkpAyBJTRkASWrKAEhSUwZAkpoyAJLU1P8Bb+BW42QZKkMAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "y = np.array([10.05, 1.5, -1.234, 0.02, 8.03]).reshape(-1,1)\n",
    "plt.figure()\n",
    "plt.plot(X, y, '+')\n",
    "plt.xlabel(\"$x$\")\n",
    "plt.ylabel(\"$y$\");"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Polynomial Regression\n",
    "One class of functions that is covered by linear regression is the family of polynomials because we can write a polynomial of degree $K$ as\n",
    "$$\n",
    "\\sum_{k=0}^K \\theta_k x^k = \\boldsymbol \\phi(x)^T\\boldsymbol\\theta\\,,\\quad\n",
    "\\boldsymbol\\phi(x)= \n",
    "\\begin{bmatrix}\n",
    "x^0\\\\\n",
    "x^1\\\\\n",
    "\\vdots\\\\\n",
    "x^K\n",
    "\\end{bmatrix}\\in\\mathbb{R}^{K+1}\\,.\n",
    "$$\n",
    "Here, $\\boldsymbol\\phi(x)$ is a nonlinear feature transformation of the inputs $x\\in\\mathbb{R}$.\n",
    "\n",
    "Similar to the earlier case we can define a matrix that collects all the feature transformations of the training inputs:\n",
    "$$\n",
    "\\boldsymbol\\Phi = \\begin{bmatrix}\n",
    "\\boldsymbol\\phi(x_1) & \\boldsymbol\\phi(x_2) & \\cdots & \\boldsymbol\\phi(x_n)\n",
    "\\end{bmatrix}^T \\in\\mathbb{R}^{N\\times K+1}\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let us start by computing the feature matrix $\\boldsymbol \\Phi$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "## EDIT THIS FUNCTION\n",
    "def poly_features(X, K):\n",
    "    \n",
    "    # X: inputs of size N x 1\n",
    "    # K: degree of the polynomial\n",
    "    # computes the feature matrix Phi (N x (K+1))\n",
    "    \n",
    "    X = X.flatten()\n",
    "    N = X.shape[0]\n",
    "    \n",
    "    #initialize Phi\n",
    "    Phi = np.zeros((N, K+1))\n",
    "    \n",
    "    # Compute the feature matrix in stages\n",
    "    for k in range(K+1):\n",
    "        Phi[:,k] = X**k ## <-- SOLUTION\n",
    "    return Phi"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "With this feature matrix we get the maximum likelihood estimator as\n",
    "$$\n",
    "\\boldsymbol \\theta^\\text{ML} = (\\boldsymbol\\Phi^T\\boldsymbol\\Phi)^{-1}\\boldsymbol\\Phi^T\\boldsymbol y\n",
    "$$\n",
    "For reasons of numerical stability, we often add a small diagonal \"jitter\" $\\kappa>0$ to $\\boldsymbol\\Phi^T\\boldsymbol\\Phi$ so that we can invert the matrix without significant problems so that the maximum likelihood estimate becomes\n",
    "$$\n",
    "\\boldsymbol \\theta^\\text{ML} = (\\boldsymbol\\Phi^T\\boldsymbol\\Phi + \\kappa\\boldsymbol I)^{-1}\\boldsymbol\\Phi^T\\boldsymbol y\n",
    "$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "## EDIT THIS FUNCTION\n",
    "def nonlinear_features_maximum_likelihood(Phi, y):\n",
    "    # Phi: features matrix for training inputs. Size of N x D\n",
    "    # y: training targets. Size of N by 1\n",
    "    # returns: maximum likelihood estimator theta_ml. Size of D x 1\n",
    "    \n",
    "    kappa = 1e-08 # 'jitter' term; good for numerical stability\n",
    "    \n",
    "    D = Phi.shape[1]  \n",
    "    \n",
    "    # maximum likelihood estimate\n",
    "    Pt = Phi.T @ y # Phi^T*y\n",
    "    PP = Phi.T @ Phi + kappa*np.eye(D) # Phi^T*Phi + kappa*I\n",
    "        \n",
    "    # maximum likelihood estimate\n",
    "    C = scipy.linalg.cho_factor(PP)\n",
    "    theta_ml = scipy.linalg.cho_solve(C, Pt) # inv(Phi^T*Phi)*Phi^T*y \n",
    "    \n",
    "    return theta_ml"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we have all the ingredients together: The computation of the feature matrix and the computation of the maximum likelihood estimator for polynomial regression. Let's see how this works.\n",
    "\n",
    "To make predictions at test inputs $\\boldsymbol X_{\\text{test}}\\in\\mathbb{R}$, we need to compute the features (nonlinear transformations) $\\boldsymbol\\Phi_{\\text{test}}= \\boldsymbol\\phi(\\boldsymbol X_{\\text{test}})$ of $\\boldsymbol X_{\\text{test}}$ to give us the predicted mean\n",
    "$$\n",
    "\\mathbb{E}[\\boldsymbol y_{\\text{test}}] = \\boldsymbol \\Phi_{\\text{test}}\\boldsymbol\\theta^{\\text{ML}}\n",
    "$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYkAAAEGCAYAAACQO2mwAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nO3deXxU1f3G8c8Xwia7EhDZZRcX1ADuoAiioghq64Z70Votdalrf9rWaqnWYrVWpYriviNUUIko4oosArKDLBJAkgiyb0nO748z2IgZTCAz507meb9eeQ3JhLlPqckz955zzzHnHCIiIiWpFDqAiIhEl0pCRETiUkmIiEhcKgkREYlLJSEiInFlhA5Qnho0aOBatmwZOoaISEqZOnVqvnMus6TnKlRJtGzZkilTpoSOISKSUsxsWbzndLlJRETiUkmIiEhcKgkREYlLJSEiInGpJEREJC6VhIiIxKWSEBGRuFQSIiKpbsIQWPxhQl5aJSEikso25fuSWD4pIS+vkhARSWWLxgMO2pyckJdXSYiIpLJF2VAzExp3TsjLqyRERFJVUaE/k2jdEyol5te5SkJEJFWtnA5b1iTsUhOoJEREUteibMCg9UkJO4RKQkQkVS3MhiZHQs39EnYIlYSISCravAZWTIW2vRJ6GJWEiEgq+vp9Ejn1dSeVhIhIKlqYDTX2hQMOT+hhVBIiIqmmqAi+Hg9tekKlygk9lEpCRCTVrJgKm/Kgbe+EHyp4SZhZMzP7wMzmmtlsMxsc+/q+ZpZtZgtjj/VDZxURiYR5b0GljIQPWkMESgIoAG50znUEjgJ+Y2YHAbcC451zbYHxsc9FRGT+WGh5HNRI/Hvn4CXhnFvlnJsW+/MGYC7QBOgHjIh92wjgrDAJRUQiJG8B5C+ADn2TcrjgJVGcmbUEDgcmAY2cc6vAFwnQMFwyEZGImD/GP7Y/NSmHi0xJmFkt4HXgd8659WX4e4PMbIqZTcnLy0tcQBGRKJg3xq/4WrdpUg4XiZIwsyr4gnjeOfdG7Murzaxx7PnGQG5Jf9c5N8w5l+Wcy8rMzExOYBGREDZ8CzmTk3apCSJQEmZmwJPAXOfcP4o9NRq4JPbnS4BRyc4mIhIp88f6xw6nJ+2QGUk7UnzHAgOBr8xseuxrtwNDgFfM7ArgG+DcQPlERKJh3lio3woadkzaIYOXhHPuY8DiPN0zmVlERCJr63pY8iF0HQQW71dm+Qt+uUlEREph/lgo3A4dz0zqYVUSIiKpYNYbULcZNO2S1MOqJEREom7zGr+gX6f+CdvLOh6VhIhI1M39LxQVwMFnJ/3QKgkRkaib9TrseyA0Pizph1ZJiIhE2YbVsPQjfxaRxFlNO6kkRESibM4ocEVBLjWBSkJEJNpmvwEND0rqDXTFqSRERKLq++XwzWfQaUCwCCoJEZGomvmSfzzknGARVBIiIlHkHEx/AVocB/u2ChZDJSEiEkXffA5rFkPnC4LGUEmIiETR9OehSk04qF/QGCoJEZGo2b4JZo+ETmdBtVpBo6gkRESiZu5/YftG6Hxh6CQqCRGRyPnyOajfElocEzqJSkJEJFLWLvPLcHS+MMgyHLtSSYiIRMnUp8EqwWHnh04CqCRERKKjYBtMewbanQr1moVOA0SkJMxsuJnlmtmsYl/7o5mtMLPpsY/TQmYUEUm4OaNhcz50uSJ0kh9EoiSAp4E+JXx9qHOuc+xjbJIziYgk1+T/+H0jDjwxdJIfRKIknHMTgTWhc4iIBLNqJiyfBF2uTPoWpbsTnSQlu9bMZsYuR9UPHUZEJGGmPAkZNYIvw7GrKJfEo0BroDOwCnigpG8ys0FmNsXMpuTl5SUzn4hI+di6Dma+AoecDTWi9X44siXhnFvtnCt0zhUB/wG6xvm+Yc65LOdcVmZmZnJDioiUh2nPwo7N/lJTxES2JMyscbFP+wOz4n2viEjKKtwBnz/qlwQ/4PDQaX4iI3QAADN7EegBNDCzHOAuoIeZdQYcsBS4KlhAEZFEmfUGrM+Bvv8InaREkSgJ51xJtxY+mfQgIiLJ5Bx8+hBkdoA2vUKnKVFkLzeJiFR4X78Pq2fBMddFatprcdFMJSKSDj59CGrtD4ecGzpJXCoJEZEQVs2ExRPgqKsho1roNHGpJEREQph4H1SrA0deFjrJbqkkRESSbdVMv/vcUb+GGvVCp9ktlYSISLJ9+DeoVheOuiZ0kp+lkhARSaZVM2DeW3D0NZE/iwCVhIhIck0YAtXr+ktNKUAlISKSLCu/hPlj4ehrfVGkAJWEiEgyOAfv/RGq14NuqbPKkEpCRCQZFo3390V0vyVlziJAJSEikniFBTDuD1C/VSSXA9+dSCzwJyJSoU1/HvLmwi+egYyqodOUic4kREQSadtG+OAeaNYNOp4ZOk2Z6UxCRCSRPnkQNq6GXz4PZqHTlJnOJEREEiV/EXzyT7/Ka7MuodPsEZWEiEgiOAdjb4SMGtD7ntBp9pguN6Wy7Zth5TTYmAtb1sK2DVCzAdRpAvWa+5kUEd3IRKTCm/W6n/J62t+hdqPQafaYSiLVbPkeZrwIC7Nh2SdQsDX+99bMhNY9oW0vaHcKVKudvJwi6WzrOnj3dmjcGbIuD51mr0SiJMxsONAXyHXOHRz72r7Ay0BLYCnwC+fc2lAZQxmavYDre7XzZwmfPwafPgzb1kGD9pB1BRzYA+o1gxr1oWot2JwP63JgzWJYMhEWjoOZL0HV2nD4RdD1V7Bf69D/s0QqtvF3+zP881+CSpVDp9kr5pwLnQEzOwHYCDxTrCTuA9Y454aY2a1AfefcLbt7naysLDdlypTEB06ilreOYenAHTDmBtj8HbQ/HU68DfY/pHQvUFQIy7+AKcNh9kgoKoBO/eHku6B+y4RmF0lLSybCiDOg29Vw6t9CpykVM5vqnMsq8bkolASAmbUE3ipWEvOBHs65VWbWGJjgnGu/u9eocCVRWMCwuy5mUMYYaHIknHo/ND1yz19vw7fwxTD47N/gCv36McfflBLLFYukhG0b4NFjoFIGXP0JVN0ndKJS2V1JRHlUs5FzbhVA7LFh4DxJMzR7AUfe+gKT/nQsgzLG8HRBb9p+PZihc/dyTKH2/tDzTvjtNDjkF/Dpv+CRbjD/7fIJLpLusu+E75fDWY+mTEH8nCifSXzvnKtX7Pm1zrn6Jfy9QcAggObNmx+5bNmy5AROpC1r4em+8N3XDN58Gf+896+JOc7KL2HUtbB6Fhx6HvT5K+yzb2KOJVLRff0+PNvfLwN+SmpNeU3VM4nVsctMxB5zS/om59ww51yWcy4rMzMzqQETYttGeP5cyF8A57/AqKLjEnesAw6HX33gV6Wc9Ro8drwfvxCRstmUD29eAw3awUl/CJ2mXEW5JEYDl8T+fAkwKmCW5NixFV66AFZMg3OGQ+uTGNyzbWKPmVEVTrwdrnwPKmfAU6fCZ4/4G4FE5OcVFcHIq2HzGjj7SahSI3SichWJkjCzF4HPgPZmlmNmVwBDgF5mthDoFfu8YhtzIyz5EPo9Ah3PAPDTX5PhgMNh0IfQro+f3/3Kxf6sRkR277OHYVE29LkXGh8aOk25i8R9Es658+M81TOpQUL66jWY/hyc8HvoHO+fI8Fq1INfPufPJLL/D55a4ud5120aJo9I1C2fDOP/DAf18/ctVUCROJNIe2uXwlvXQ9Ou0P3WsFnM4Jhr4YJXYe0yGHai/0EQkR/bmAuvXuqXwTnjoZRc4bU0VBKhFe6A168EDM5+wo8LREHbk+GKbD+Nb0RfmPtW6EQi0VGwHV4e6G9w/eWzFfpeI5VEaB89ADmT4YyhUL9F6DQ/1rADXDkeGnWCVwbC5CdDJxIJzzkYexMs/xzOegQaHxY6UUKpJEJauww++gd0GgAHnx06TclqNoBL/gttevmlQT64VzOfJK2Nf/ZemDYCjrshuj+35UglEdK4P4BVgt53h06ye1VrwnkvwOED4cO/wdu3+Gl/Iulm/jt0//rv0PaUCnc/RDwRuQCehpZMhLmj4cQ7UmP2UOUMOPNhqF4XPvuXX6PmzIejM4Yikmg5U+DVS5njWnDoOcNTfnXX0tJPeAiFBfDObVC3ORxzXeg0pWcGvf/ii+KDe2D7Rn/zUEbV0MlEEuqp0dmcOfUyNrraXL79ZvLv+hCAwT3bJu9epkBUEiFMG+HXSzp3ROrdnWkG3W/2e1e8e5sf0P7FM5BRLXQykcRYl8Nli2+AmlXZ74p3yL9/HkuHnB46VdJoTCLZCrbBxL9D86P9DTip6uhr4PQHYME7fimRHVtCJxIpf+tX+b0htnwPF76Wlht2qSSSbcZLsGGlfzee6jffdLnSj0ssGg8v/NLvuS1SUWzM9QWxMRcueh2aHAGQ+PXUIkYlkUyFBfDxUL9O0oEnhk5TPo642K+dv2QivHieikIqhg3fwogzYf0KuPBVaNb1h6cq+hjErlQSyTTnTVi7xM+vTvWziOI6nw/9H1NRSMWwdhkM7wPffwMXvAwtjgmdKCiVRLI452+ca9AOOvQNnab8HXaeikJSX94Cv1z+ljVw8ShodULoRMGpJJJlwbuQO9ufRVSqoP/sxYtCg9mSanKmwFN9/Hpql46FZl1CJ4qECvrbKoI+fcjfF3HIOaGTJNZh58FZ/4bFE+Dli/xGSiJRN2cUPH06VKsNl78D+x8cOlFkqCSSIW8BLPsEulwBlauETpN4nS+IzXp6z99HUbAtdCKRkjkHnz4Mr1wC+x/qF7RMw2muu6OSSIZpI6BSFeh8YegkyXPEQOj7ICwc538AC7aHTiTyYzu2wqjf+DXUDuoHl4z2C1rKj6gkEm3HVpj+AnQ4HWplhk6TXFmXxW64exteu8xf6xWJgnU5fvxh+vN+o69znkq91Q+SRMtyJNq8t/xMiSMvDZ0kjC5XQlEhvH0zvHY5nDM8PS65SXQtnuA3+tqx1a9u3CF9ltjYE5EvCTNbCmwACoEC51xW2ERlNPVpqNcCWnUPnSScblf5onj3NnjjVzAgQjvwSfooLPBL3U+8309Fv/RZyGwfOlXkpcpP6onOufzQIcosfxEs/Qh63llxp72W1tHXgCv83x4a/YepKCR51uXAG1fBso+h80Vw2n1+nxT5WT/7U2pm7wE3OudmJCFPxTLtaaiU4f+jFL8selEhvHdXrCgeT5s1+SWgma/CmBuhqMD/N3fYeaETpZTSvJW7GRhqZsuA251zqxKcaVcOGGdmDnjcOTcsycffM0WFMONlaNcHajcKnSY6jvudP6MY/2dfFGc9qqKQxNi8xpfD7DegaVd/o6emt5bZz5aEc24acJKZnQ28Y2ZvAPc555J1O+2xzrmVZtYQyDazec65iTufNLNBwCCA5s2bJylSKSz9GDblVvyb5/bE8Tf6+env3+0f+z+mopDyNftNGHsTbFnrtxk99npd3txDpbpQbmYGzAceBa4DFprZwEQG28k5tzL2mAuMBLru8vww51yWcy4rMzNCU0xnj4Qq+/i9cOWnTrjJj9V89QqMvMoPKorsrQ2r4eWB8OolUOcAGDQBTvi9CmIvlGZM4mPgQGA28DlwKTAPGGxmxzvnBiUqnJnVBCo55zbE/twb+HOijlduCgv8/tXt+kDVfUKnia7jb/SXnN77o788N2CYpsfKnikqhCnDYfzdULAVet4Fx/xW5VAOSvMveDUw2znndvn6dWY2NwGZimsEjPQnMmQALzjn3knwMffekg9h83dw8IDQSaLvuOvBKkP2/0Hhdn9Tk/bMlrJY+SW8dQOsnAYH9oDTHoAGbUKnqjBKMyYxazdPJ/QuFOfcYuCwRB4jIWaPhKq1oU2v0ElSw7G/hcpV4Z1b/FpP546AKtVDp5Ko25TvJ0BMewZqZsLZT8LBZ1esvVoiYK/OxWK/xKW4gu0w97/Q4TT9oiuLo672l5rG3OD3ozjvec1jl5IVbIfJT8CEIbBjExz9G78dcPW6oZNVSLpgV94WT4Ct30MnXWoqsy5XQEZ1GH0tPNsfLngFatQLnUqiwjmYN8ZfmlyzGFqfBH2G6K7pBFNJlLfZb0C1utC6guxhnWyHX+jPIF6/Ekb0hYFvamVOgeWTIftO+OZTaNAeLnwN2pysS0tJoJIoT4UFMH+sXzAso1roNKmr01m+KF6+CIafAgNHQr0I3QMjyZO3AMb/yS+UWbOhX1X4iEs1aymJ0nxBoXK2fBJsXQft+4ROkvra9vJnEZvy4MnesHpO6ESSTGuXwZvXwL+7+Uu4J94Bv/3SryqsgkgqlUR5WjjOby50oC41lYsWR8Nlb/s/P9UHln0WNo8k3roVfimNh4+Er16Dbr+G3073A9PVaoVOl5ZUyeVp4Tj/i616ndBJKo5GneDyd+G5AfBMPxjwOHTqHzqVlLf1q+DjoX5pfVcIh18EJ9wMdZuETpb2VBLl5fvlkDsHev8ldJKKp34LuHwcvHQBvHqp/7c+5joNWlYE61bAJw/C1BF+ldbOF/glW+q3DJ1MYlQS5WXhOP+otZoSo+Z+cPEov85T9v/B2qVw6t+0jEeqWrvMl8OXz4Er8st3H38j7Htg6GSyC5VEeVk4zu9A16Bt6CQVV5XqftmO8S3gk3/Cdwv93dn77Bs6mZRW3gL4+B8w8xW/btfhF8JxN/izRYkklUR52LEVFn8IRwzUJZBEq1QJev0ZMjvAfwfDEz3h/Jd0Q1XUrZjqxxzmvuVvmOx2FRx9rcYcUoBmN5WHpR9DwRZo2zt0kvTR+QK45C3YtgH+0xPmjP7hqaHZCwIGkx84B4vegxFnwH9OgiUT/XjD9bOgz19VEClCJVEeFr4LGTWg5XGhk6SX5t38fgGZ7fzCgNl3QmEB/xy/MHSy9Fa4w+/K+Nhx8NzZkL/QT+i4frbfAEh30KcUXW4qDwvHQasToEqN0EnST92m/l6Kd27z4xQrptEQ7WEcxNZ1fgrrpMdh/QrI7Aj9/g2HnKvl31OYSmJvrV3qP466JnSS9JVRjaHVruab7VW5Z8lw3qn2JZffvoT3i45gcM+2XN+rXeiEFduaxTBpGHz5LGzfCC2Ph75D/VL5lXSxItWpJPbWko/8Y8vjw+ZIc9f3age9hkD+FSx56GyGV/07dLsaepwUOlrF5Bws/cifNcwb4/co7zTAL9t9QOfQ6aQcqST21tKPYZ8G0LBj6CQC0KAtA7b/ifndP4dJj8Gi8dD/MWiaFTpZxbB9s9+XfNLj/ubRGvv6+xu6XAl1GodOJwmgktgbO99NtTxOU18j5OqenaBXf7/H+Ojr4Mlefr/jHrdq3GhPffc1TH4Spj/nxx4aHQL9HvE7wenftEJTSeyNNYv9AF2rG0InkWJ+GINofSL8+lN493Z/d++cN/1S021ODhswVRTugPlvw9Sn4Ov3oVIGdDzTnzW0OEZvjNJE5EvCzPoA/wQqA08454YEjvQ/S3eOR5wQNofEV70O9PuXn2Ez5gY/JbPTAOh9t58ZJT+1ZolfLuPL52Djt1CnCfS4HY68BGrvHzqdJFmkS8LMKgOPAL2AHGCymY12zkVjc4ElH0GtRlqKIxUc2N2fVXz8IHz0gH+HfMx1cOzgtF+Cemj2Aq7v3tQPQH/5LCz50C+Z0aYXZD3oH7WHQ9qK+v/zXYFFzrnFAGb2EtAPCF8SGo9IPRnVoMct0Pl8eO9PMPE+mDYCjr/Jv0tOt90Ei4pg+SQaffg3mDIVtq33OwCe+Ad/R7vuiBaiXxJNgOXFPs8BugXK8mP5C2Hjak19TUX1msM5T/opstl3wtu/92MWx98AnS/yCwlWZKvnwFev+k191n3DWZWrQYezfTG0OFb3NsiPRL0kSnqL7n70DWaDgEEAzZsncR/kpRP9YyuNR6SsZl3gsrH+8soH9/od0SYMgS6/gi5XVJzlI5yDvHkw+02YPRLy51NEZSYWHsybhX0ZV5TF5knVYdJ6BvdcpJsP5UeiXhI5QLNinzcFVhb/BufcMGAYQFZW1o8KJKGWfAS1D9D696nODA7sAa26+wXoPvsXTLjXL2fdqT8ccTE0Pzr1LikWFkDOZJg/xo81rFkMmL882vVXVDqoHz1qNaQH0PLWMSwdcnrgwBJVUS+JyUBbM2sFrADOAy4IG4nYeMTH0KZn6v3ykJKZ+cHtA7tD3nx/I95Xr8GMF2G/NnDIL6DTWdFeknxdjl+yflG2n7K6dZ3fc73VCX5Z7g6na3aSlFmkS8I5V2Bm1wLv4qfADnfOzQ4cC75bBJvz/fVbqXgy2/u1h3r/BeaM8lNBJ/zVn2FkdoT2p0Lrk6BZt3AL1zkHa5fA8i/gm8/8WdCaxf65Wo2gQ19o28vnrF53ty81uKdm50l85lzyrtAkWlZWlpsyZUriD/Tl8zDqGrjmcy3HkS7Wr4K5o31pLJ/k92OuUhOadfUfTbtA48OgZmbcs8uh2Qv27Hp/4Q5/70LePPh2JqyaASunw6Zc/3y1Ov7mtlbdodXx0LCTBp+lTMxsqnOuxLVrIn0mEVk5k6FaXWgQ4UsPUr7qNPa7qXW7Crau99Ofv34fvpkEE+/3+zSDX8sos4Mfq6rbBOoc4N/ZV6/LO+/P4Pojq/jF8Kwy4KBgG+zYAts3webv/MemXH/p6Pvl8P0yf4ZQVOBf3yr712/T0xdT86P855UqB/unkYpNZxJ74tFjoVZDGDgy8ceS6Nu2EVZOg9Wz/bv93Hn+l/uGb9llMl7p1djX3xFer7m/WbNBe7+5UsODtFaSlDudSZSnbRv86pcd+oZOIlFRrZYfHN51OnThDp4Y+ymjP51BbdtMbbawD1upbEX07tCAXgc18jsaVqnuL13tsy/ss5+felu1Zpj/LSK7UEmU1Yqp/tJC0y6hk0jUVa7ClWd058ozugOaaiqpSaNbZbV8sn9semTYHCIiSaCSKKucL/z14Rr1QyeRFKOpppKKVBJl4Zyf2dRMl5qk7LTchaQilURZfLcItqyFpl1DJxERSQqVRFnkxMYjmqkkRCQ9qCTKYvkXuolORNKKSqIscib7WU1a8kBE0oR+25XWzpvoNB4hImlEJVFaq2b6m+iaHBE6iYhI0qgkSmvVDP/YuHPYHCIiSaSSKK1VM6DW/lC7UegkIiJJo5IorVUz4ACdRYhIelFJlMb2TZA/328qIyKSRlQSpbF6th+0VkmISJpRSZTGD4PWKgkRSS+RLQkz+6OZrTCz6bGP04KFWTUd9mkAdZoEiyAiEkLUNx0a6pz7e+gQrJzhzyLibHAvIlJRRfZMIjJ2bIW8ubrUJCJpKeolca2ZzTSz4WYWZpef3DlQVKCSEJG0FLQkzOw9M5tVwkc/4FGgNdAZWAU8EOc1BpnZFDObkpeXV/4hNWgtImks6JiEc+7k0nyfmf0HeCvOawwDhgFkZWW58ksXs2o6VK8L9VuW+0uLiERdZC83mVnjYp/2B2YFCbJKg9Yikr6iPLvpPjPrDDhgKXBV0hMU7vA30nVL/qFFRKIgsiXhnBsYOgN586BwO+yv8QgRSU+RvdwUCavn+Mf9Dw6bQ0QkEJXE7uTOgUpVYL82oZOIiAShktid3DnQoB1UrhI6iYhIECqJ3Vk9BxodFDqFiEgwKol4tq6D9TnQUCUhIulLJRFP7lz/qJIQkTSmkohn9Wz/qMtNIpLGVBLx5M6FqrWhbrPQSUREglFJxJM7Bxp21HIcIpLWVBIlcc6XhC41iUiaU0mUZMO3sGUtNOwUOomISFAqiZLkxgatG3YMm0NEJDCVREk0/VVEBFBJlGz1HKjVCGruFzqJiEhQKomS5M7WWYSICCqJnyoqhLz50EiD1iIiKoldrVkCBVs1aC0igkrip/Lm+cdMlYSIiEpiV/kL/GODtmFziIhEQNCSMLNzzWy2mRWZWdYuz91mZovMbL6ZnZK0UPkLoXZjqF4naYcUEYmqjMDHnwUMAB4v/kUzOwg4D+gEHAC8Z2btnHOFCU+Uv0BnESIiMUHPJJxzc51z80t4qh/wknNum3NuCbAI6JqEQP5MokG7hB9KRCQVRHVMogmwvNjnObGvJdbGXNi2TiUhIhKT8MtNZvYesH8JT93hnBsV76+V8DUX5/UHAYMAmjdvvkcZf6BBaxGRH0l4STjnTt6Dv5YDFN/tpymwMs7rDwOGAWRlZZVYJKX2Q0noTEJEBKJ7uWk0cJ6ZVTOzVkBb4IuEHzV/IVSpCbUPSPihRERSQegpsP3NLAc4GhhjZu8COOdmA68Ac4B3gN8kb2ZTG6gU1e4UEUmuoFNgnXMjgZFxnrsHuCepgfIXQvNuST2kiEiU6S3zTts3w7rlGo8QESlGJbHTmq8Bp5lNIiLFqCR20swmEZGfUEnslL8QMNi3degkIiKRoZLYKX8B1G8BVaqHTiIiEhkqiZ3yF+hSk4jILlQSAEVFkL9IJSEisguVBMD6HCjYoplNIiK7UEmAZjaJiMShkgCoWgvan66SEBHZReid6aKh+VH+Q0REfkRnEiIiEpdKQkRE4lJJiIhIXCoJERGJSyUhIiJxqSRERCQulYSIiMSlkhARkbjMORc6Q7kxszxg2R7+9QZAfjnGKS9RzQXRzaZcZaNcZVMRc7VwzmWW9ESFKom9YWZTnHNZoXPsKqq5ILrZlKtslKts0i2XLjeJiEhcKgkREYlLJfE/w0IHiCOquSC62ZSrbJSrbNIql8YkREQkLp1JiIhIXCoJERGJSyVRAjO7ycycmTUInQXAzO42s5lmNt3MxpnZAaEzAZjZ/WY2L5ZtpJnVC50JwMzONbPZZlZkZsGnKppZHzObb2aLzOzW0Hl2MrPhZpZrZrNCZ9nJzJqZ2QdmNjf2/+Hg0JkAzKy6mX1hZjNiuf4UOlNxZlbZzL40s7fK+7VVErsws2ZAL+Cb0FmKud85d6hzrjPwFnBn6EAx2cDBzrlDgQXAbYHz7DQLGABMDB3EzCoDjwCnAgcB55vZQWFT/eBpoE/oELsoAG50znUEjgJ+E5F/r23ASc65w4DOQB8zi9J2loOBuYl4YZXETw0FbgYiM6LvnFtf7NOaRCSbc26cc64g9unnQNOQeXZyzs11zs0PnSOmK7DIObfYObcdeAnoFzgTAM65icCa0DmKc86tcs5Ni/15A5/2G20AAALoSURBVP4XX5OwqcB5G2OfVol9ROLn0MyaAqcDTyTi9VUSxZjZmcAK59yM0Fl2ZWb3mNly4EKicyZR3OXA26FDRFATYHmxz3OIwC+9VGBmLYHDgUlhk3ixSzrTgVwg2zkXiVzAg/g3tkWJePGMRLxolJnZe8D+JTx1B3A70Du5ibzd5XLOjXLO3QHcYWa3AdcCd0UhV+x77sBfJng+GZlKmysirISvReIdaJSZWS3gdeB3u5xJB+OcKwQ6x8beRprZwc65oOM5ZtYXyHXOTTWzHok4RtqVhHPu5JK+bmaHAK2AGWYG/tLJNDPr6pz7NlSuErwAjCFJJfFzuczsEqAv0NMl8aabMvx7hZYDNCv2eVNgZaAsKcHMquAL4nnn3Buh8+zKOfe9mU3Aj+eEHvQ/FjjTzE4DqgN1zOw559xF5XUAXW6Kcc595Zxr6Jxr6Zxrif/hPiIZBfFzzKxtsU/PBOaFylKcmfUBbgHOdM5tDp0noiYDbc2slZlVBc4DRgfOFFnm36E9Ccx1zv0jdJ6dzCxz5+w9M6sBnEwEfg6dc7c555rGfmedB7xfngUBKolUMcTMZpnZTPzlsEhMCwT+BdQGsmPTcx8LHQjAzPqbWQ5wNDDGzN4NlSU2sH8t8C5+EPYV59zsUHmKM7MXgc+A9maWY2ZXhM6Ef2c8EDgp9t/U9Ni75NAaAx/EfgYn48ckyn26aRRpWQ4REYlLZxIiIhKXSkJEROJSSYiISFwqCRERiUslISIicakkREQkLpWEiIjEpZIQSaDY3gi9Yn/+i5k9FDqTSFmk3dpNIkl2F/BnM2uIX9H0zMB5RMpEd1yLJJiZfQjUAnrE9kgQSRm63CSSQLHVhRsD21QQkopUEiIJYmaN8Xts9AM2mdkpgSOJlJlKQiQBzGwf4A38fs1zgbuBPwYNJbIHNCYhIiJx6UxCRETiUkmIiEhcKgkREYlLJSEiInGpJEREJC6VhIiIxKWSEBGRuP4f/KgcUqgmcFwAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "K = 5 # Define the degree of the polynomial we wish to fit\n",
    "Phi = poly_features(X, K) # N x (K+1) feature matrix\n",
    "\n",
    "theta_ml = nonlinear_features_maximum_likelihood(Phi, y) # maximum likelihood estimator\n",
    "\n",
    "# test inputs\n",
    "Xtest = np.linspace(-4,4,100).reshape(-1,1)\n",
    "\n",
    "# feature matrix for test inputs\n",
    "Phi_test = poly_features(Xtest, K)\n",
    "\n",
    "y_pred = Phi_test @ theta_ml # predicted y-values\n",
    "\n",
    "plt.figure()\n",
    "plt.plot(X, y, '+')\n",
    "plt.plot(Xtest, y_pred)\n",
    "plt.xlabel(\"$x$\")\n",
    "plt.ylabel(\"$y$\");"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Experiment with different polynomial degrees in the code above.\n",
    "#### Questions:\n",
    "1. What do you observe?\n",
    "2. What is a good fit?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Evaluating the Quality of the Model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let us have a look at a more interesting data set"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYwAAAEGCAYAAAB2EqL0AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAQ7klEQVR4nO3df4wc513H8c8HJyGFAgXsENd2uCDOqFZ/pNXJSpV/ojgHTlPZtBDJEZQEWlmVmmIsULlgqYEUJJdKHKmIWpk2NEBJWpVWMbmAe3FSIgQpWYcktev6bKwUXx3ItYHyI5DIzZc/ds5d782dn/Pt7jMz+35JJ9/sTPY+SuL77MwzzzOOCAEAcD7fkzsAAKAeKAwAQBIKAwCQhMIAACShMAAASS7KHaBfVq9eHSMjI7ljAECtHDp06JsRsaZsX2MLY2RkRK1WK3cMAKgV219fbB+XpAAASSgMAEASCgMAkITCAAAkoTAAAEkoDKAPJqdnckcAeo7CAPrgroPHc0cAeo7CAAAkaezEPWDQJqdnzjmzGJmYkiTt2jKq3eMbc8UCesZNfYDS2NhYMNMbuYxMTOnZvTfmjgEsm+1DETFWto9LUgCAJBQG0Ae7tozmjgD0HIUB9AFjFmgiCgMAkITCAAAkoTAAAEkoDABAEgoDAJCEwgAAJKEwAABJKAwAQBIKAwCQpBKFYfse28/bPrzIftv+qO0Ttp+x/ZZBZwSAYVeJwpD0KUlbl9h/g6TR4munpI8NIBMAoEMlCiMiHpP0whKHbJf0p9H2uKTX2F47mHQAAKkihZFgnaRTHduzxWvnsL3Tdst2a25ubmDhAGAY1KUwXPLagic/RcS+iBiLiLE1a9YMIBYADI+6FMaspA0d2+slnc6UBQCGUl0KY7+kXyrulrpa0rcj4rncoQBgmFyUO4Ak2b5P0rWSVtuelXSHpIslKSI+LukhSW+TdELSi5J+OU9SABhelSiMiLj5PPtD0vsGFAcAUKIul6QAAJlRGACAJBQGACAJhQEASEJhAACSUBgAgCQUBgAgCYUBAEhCYQAVNDk9kzsCsACFAVTQXQeP544ALEBhAACSVGItKQDty1CdZxYjE1OSpF1bRrV7fGOuWMBZbq/r1zxjY2PRarVyx0BNTU7PZP0lPTIxpWf33njB/3zu/Kgv24ciYqxsH5ekgBJ1H0Ooe35UE4UBVNCuLaO5IwALcEkKKHSPIcyryxhC3fOjGpa6JEVhACVWOoaQW93zIx/GMAAAK0ZhACXqPoZQ9/yoJi5JAQDO4pIUAGDFKAwAQBIKAwCQhMIAACShMAAASSgMAEASCgMAkITCAAAkoTAAAEkoDABAEgoDAJCEwgAAJKEwAABJKAwAQJJKFIbtrbaP2T5he6Jk/62252w/VXy9J0dOABhm2QvD9ipJd0u6QdImSTfb3lRy6Gci4qri6xMDDYnamZyeyR0BaJzshSFps6QTEXEyIl6WdL+k7ZkzoebuOng8dwSgcapQGOsknerYni1e6/Zztp+x/TnbG8reyPZO2y3brbm5uX5kBYChdVHuAJJc8lr3c2P/StJ9EfGS7fdKulfSdQv+oYh9kvZJ7Ue09jooqm1yeuacM4uRiSlJ7edb7x7fmCsW0BhVKIxZSZ1nDOslne48ICK+1bH5x5I+PIBcqJnd4xvPFsPIxJSe3Xtj5kRAs1ThktQTkkZtX2n7Ekk7JO3vPMD22o7NbZKODjAfAEAVOMOIiDO2b5N0QNIqSfdExBHbd0pqRcR+Sb9qe5ukM5JekHRrtsCohV1bRnNHABrHEc281D82NhatVit3DACoFduHImKsbF8VLkkBAGqAwgAAJKEwAABJKAwAQBIKAwCQhMJAJbF4IFA9FAYqicUDgeqhMAAASbLP9AbmsXggUG3M9EYlsXggkAczvQEMFDctNBOFgUpi8cB646aFZqIwUEmMWQDVw6A3gJ7gpoXmY9AbQM9x00J9MegNAFgxCgNAz3HTQjNRGAB6jjGLZqIwACzAPAqUoTAALMA8CpShMAAASZiHAUAS8yhwfszDALAA8yiGF/MwAAArRmEAWIB5FChDYQBYgDELlKEwAABJKAwAQBIKAwCQhMIAACShMAAASSgMAECS8xaG7Ydtv2kQYQAA1ZVyhvEBSZO2/8T22n6EsL3V9jHbJ2xPlOz/XtufKfZ/2fZIP3IAABZ33sKIiCcj4jpJD0r6G9t32H5VrwLYXiXpbkk3SNok6Wbbm7oOe7ekf4+In5Q0KenDvfr5AIA0SWMYti3pmKSPSXq/pOO239WjDJslnYiIkxHxsqT7JW3vOma7pHuL7z8naUuRCQAwICljGH8n6Rtqf7JfJ+lWSddK2mx7Xw8yrJN0qmN7tnit9JiIOCPp25J+tAc/GwCQKOV5GO+VdCQWroP+fttHe5Ch7Eyh+2elHCPbOyXtlKQrrrhi5ckAAGeljGEcLimLeb1YMH9W0oaO7fWSTi92jO2LJP2QpBe63ygi9kXEWESMrVmzpgfRAADzVjQPIyJO9iDDE5JGbV9p+xJJOyTt7zpmv6Rbiu9/XtIjS5QYAKAPsj+iNSLO2L5N0gFJqyTdExFHbN8pqRUR+yV9UtKf2T6h9pnFjnyJAWA4ZS8MSYqIhyQ91PXaBzu+/z9JNw06FwDgu1gaBACQhMIAACShMAAASSgMAEASCgMAkITCAAAkoTAAAEkoDABAEgoDAJCEwgAAJKEwAABJKAwAQBIKAwCQhMJAX0xOz+SOAKDHKAz0xV0Hj+eOAKDHKAwAQJJKPEAJzTA5PXPOmcXIxJQkadeWUe0e35grFoAecVMfjT02NhatVit3jKE1MjGlZ/femDsGgGWyfSgixsr2cUkKAJCEwkBf7NoymjsCgB6jMNAXjFkAzUNhAACSUBgAgCQUBgAgCYUBAA3Tr6V5KAwAaJh+Lc1DYQAAkrA0CAA0wCCW5mFpkEVMTs8wlwBALa1kaR6WBrkALM8NAOeiMACgYfq1NA+XpDp0XwOcx/LcAIbFUpekKIxFsDw3gGHEGAYAYMWyFobtH7E9bft48ecPL3Lcd2w/VXztH0Q2lucGgHPlPsOYkHQwIkYlHSy2y/xvRFxVfG0bRDDGLADgXLkLY7uke4vv75X0sxmzAKiIfq2FhJXJXRg/FhHPSVLx52WLHHep7Zbtx20vWiq2dxbHtebm5vqRF8AAMA+qmvq+NIjthyVdXrJrzzLe5oqIOG37JyQ9YvsrEfHP3QdFxD5J+6T2XVIXFBgAUKrvhRER1y+2z/a/2V4bEc/ZXivp+UXe43Tx50nbX5L0ZkkLCgNAfQ1iLSSsTO7FB/dLukXS3uLPB7oPKO6cejEiXrK9WtI1kn5/oCkB9N3u8Y1ni4F5UNWUewxjr6Rx28cljRfbsj1m+xPFMa+T1LL9tKRHJe2NiK9mSQsAQyzrGUZEfEvSlpLXW5LeU3z/95LeMOBoADJiHlQ15T7DAIAFGLOoJgoDAJCEwgAAJKEw+oSZqgCahsLoE2aqAmgaCgMAkCT3xL1GYaYqgCbjiXt9wkxVAHXEE/cAACtGYfQJM1UBNA2F0SeMWQBoGgoDAJCEwgAAJKEwAABJKAwAjcPSPP1BYQBoHJbm6Q8KA6X4hAagG4WBUnxCQ91MTs9oZGLq7JI8899fyIcfPjCVYy0pAI2we3zj2flPK12a566Dx5lLVYLCwFksnghgKSw+iFIsnog6m5yeWfaHnO4PTPOG7QPTUosPcoYBoHEu5Bd8Ly9pNRWD3ijF4okAulEYDbXSuzyG6RQc6MYHpnIURkNxWyxw4fjAVI7CAAAkYdC7QbgtFkA/cVttQ3GXB4ALwTO9AaBGqro0CYXRUNzlAdRXVW9aoTAaijELAL3GoDcAVEAdblph0BsAKibnTSsMegMAVixrYdi+yfYR26/YLm204ritto/ZPmF7YpAZAWDQqnrTSu4zjMOS3inpscUOsL1K0t2SbpC0SdLNtjcNJh4ADF5Vxiy6ZR30joijkmR7qcM2SzoRESeLY++XtF3SV/seEABwVu4zjBTrJJ3q2J4tXlvA9k7bLdutubm5gYQDgGHR9zMM2w9Lurxk156IeCDlLUpeK721KyL2Sdonte+SSg4JADivvhdGRFy/wreYlbShY3u9pNMrfE8A6JsLeURsHdThktQTkkZtX2n7Ekk7JO3PnAkAFlXVpT1WKvdtte+wPSvprZKmbB8oXn+t7YckKSLOSLpN0gFJRyV9NiKO5MoMAMOKmd4A0APdS3vMq9LSHimWmulNYQBAj9X5eTQsDVJDVV0PH8DwojAqqqmDZsAwqOrSHitFYQBAj9VpzGI5eB5GhdRhPXwAw4tB74qq86AZgPpi0BsAsGIURkU1ddAMQH1RGBXFmAWAqqEwAABJKAwAQBIKAwCQhMIAACShMAAASRo7cc/2nKSvr+AtVkv6Zo/i9BK5lodcy0Ou5Wlirh+PiDVlOxpbGCtlu7XYbMecyLU85Foeci3PsOXikhQAIAmFAQBIQmEsbl/uAIsg1/KQa3nItTxDlYsxDABAEs4wAABJKAwAQBIK4zxs/4btsL06d5Z5tj9k+xnbT9n+ou3XViDTR2x/rcj1BduvyZ1pnu2bbB+x/YrtrLdA2t5q+5jtE7YncmbpZPse28/bPpw7SyfbG2w/avto8d9wV+5MkmT7Utv/aPvpItfv5M40z/Yq2/9k+8FevzeFsQTbGySNS/qX3Fm6fCQi3hgRV0l6UNIHcweSNC3p9RHxRkkzkm7PnKfTYUnvlPRYzhC2V0m6W9INkjZJutn2ppyZOnxK0tbcIUqckfTrEfE6SVdLel9F/p29JOm6iHiTpKskbbV9deZM83ZJOtqPN6YwljYp6QOSKnVnQET8Z8fm96sC+SLiixFxpth8XNL6nHk6RcTRiDiWO4ekzZJORMTJiHhZ0v2StmfOJEmKiMckvZA7R7eIeC4iniy+/y+1fxGuy5tKirb/LjYvLr6y/z20vV7SjZI+0Y/3pzAWYXubpG9ExNO5s5Sx/Xu2T0n6BVXjDKPTr0j669whKmidpFMd27OqwC+/urA9IunNkr6cN0lbcennKUnPS5qOiCrk+kO1P+S+0o83v6gfb1oXth+WdHnJrj2SfkvSTw820XctlS0iHoiIPZL22L5d0m2S7sidqThmj9qXET7d7zzLzVYBLnkt+6fSOrD9akl/KenXus6ws4mI70i6qhiv+4Lt10dEtjEg22+X9HxEHLJ9bT9+xlAXRkRcX/a67TdIulLS07al9uWVJ21vjoh/zZmtxF9ImtIACuN8mWzfIuntkrbEgCf4LOPfV06zkjZ0bK+XdDpTltqwfbHaZfHpiPh87jzdIuI/bH9J7TGgnDcNXCNpm+23SbpU0g/a/vOI+MVe/QAuSZWIiK9ExGURMRIRI2r/RX/LoMrifGyPdmxuk/S1XFnm2d4q6TclbYuIF3PnqagnJI3avtL2JZJ2SNqfOVOluf2J7ZOSjkbEH+TOM8/2mvk7AW2/StL1yvz3MCJuj4j1xe+sHZIe6WVZSBRGXe21fdj2M2pfNqvCrYZ/JOkHJE0Xt/t+PHegebbfYXtW0lslTdk+kCNHcVPAbZIOqD14+9mIOJIjSzfb90n6B0k/ZXvW9rtzZypcI+ldkq4r/r96qvgEndtaSY8WfwefUHsMo+e3sVYNS4MAAJJwhgEASEJhAACSUBgAgCQUBgAgCYUBAEhCYQAAklAYAIAkFAYwQMWzHcaL73/X9kdzZwJSDfVaUkAGd0i60/Zlaq+8ui1zHiAZM72BAbP9t5JeLena4hkPQC1wSQoYoGIl5LWSXqIsUDcUBjAgtteq/ZyQ7ZL+x/bPZI4ELAuFAQyA7e+T9Hm1n099VNKHJP121lDAMjGGAQBIwhkGACAJhQEASEJhAACSUBgAgCQUBgAgCYUBAEhCYQAAkvw/OflzCMK/364AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "def f(x):   \n",
    "    return np.cos(x) + 0.2*np.random.normal(size=(x.shape))\n",
    "\n",
    "X = np.linspace(-4,4,20).reshape(-1,1)\n",
    "y = f(X)\n",
    "\n",
    "plt.figure()\n",
    "plt.plot(X, y, '+')\n",
    "plt.xlabel(\"$x$\")\n",
    "plt.ylabel(\"$y$\");"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, let us use the work from above and fit polynomials to this dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYwAAAEGCAYAAAB2EqL0AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nOy9d3hb5f33/7olech7yXslHtl7sFdCWAHCLKOUUUZbZmmfb39p+7QUWijj6TcthUKhbCij0LIChCQkhUAgeztxHNvx3luSh6z798fxObFi2ZYT79yv6/IV65z7HN0GSW99tpBSolAoFApFf5hGegMKhUKhGBsowVAoFAqFTyjBUCgUCoVPKMFQKBQKhU8owVAoFAqFT1hGegNDSUxMjExPTx/pbSgUCsWYYevWrTVSSpu3c+NaMNLT09myZctIb0OhUCjGDEKIw72dGxUuKSHEi0KIKiHEnl7Ony2EaBRC7Oj6+e1w71GhUChOdEaLhfEy8BTwah9rvpJSXjw821EoFArF0YwKC0NK+SVQN9L7UCgUCkXvjArB8JFThBA7hRCfCiGm9bZICHGHEGKLEGJLdXX1cO5PoVAoxjVjRTC2AWlSylnAX4H3e1sopXxOSjlfSjnfZvMa6FcoFArFMTAmBENK2SSlbOn6/RPATwgRM8LbUigUihOKMSEYQoh4IYTo+n0h2r5rR3ZXCoVCcWIxKrKkhBBvAmcDMUKIEuABwA9ASvkscBXwEyGEC3AC10rVl10xBmhub2Z98XouybhkpLeiUBw3o0IwpJTX9XP+KbS0W4ViTLEyfyUPf/cwc+PmkhSSNNLbUSiOizHhklIoxioV9goA6lvrR3gnCsXxowRDoRhCqp1aarcSDMV4QAmGQjGEVDoqAWhoa+h1jZSSdUXraO9sH65tKRTHhBIMhWIIqXJUAX0LxqaKTdy77l4+K/xsuLalUBwTSjAUiiGk2tG/S2r14dUAHKg7MCx7UiiOFSUYCsUQ4ehw0NLRAkBjW6PXNW7pZm3RWgAO1h8ctr0pFMeCEgyFYojQ4xcA9W3eLYwdVTuocdYQERDBwQYlGIrRjRIMhWKI0N1RZmHuNYax+vBq/E3+XDf5OmqcNSqbSjGqUYKhUAwRuoWRHpbuVTCklKwpWsOpiacy2zYbgLyGvGHdo0IxEJRgKBRDhJ4hlRWZRUNrT8HYU7OHCnsF56adS2ZkJgC59bnDukeFYiAowVAohogqRxWyM4CkkCQa2ho4uv3Z6qLVWISFs1POxma1ER4QriwMxahGCYZCMcisWJ1L+vKVvLJpF25XGE+vLafD3cETn+8y1kgpWXN4DQsTFhIeEI4QgqyILJUppRjVKMFQKAaBndU7+fvOvwNw/5JsCh9dyrwME9IVzhNXnALA9adGG+sPNRyiuLmYc9PONY5lRmSS15DXwxJRKEYLSjAUikHg3dx3eWrHUx71FlWOKmRHGBEBEYBnLUZ+Yz4AM2JmGMeyIrOwd9gpt5cP064VioGhBEOhGAQKGwsByKnLAbSCvBpHDfOS0wzB6F6LoYtCQnCCcSwrMgtQBXyK0YsSDIViEChoKgAgp1YTjLrWOlzSxdKpkw3B6J5aW2GvwGqxEuYfZhzLjNAypVQBn2K0ogRDoThO6lvrDXeTLhh6Sm1cUByRgZEAHqm1FfYK4oPj6Zo8DECofygJwQkeFsaXJV9S61TTiBWjAyUYCsVxUtCoWRehfqHsq9sHHBGM2KBYQv1DMQmTh0uqwl7h4Y7SyYzINCyMdw68w11r7+LN/W8O9Z+gUPiEEgyF4jgpbCoE4Ny0czncdJiW9hZDMGxBNkzCRLh/uEfQu9xeTnxwfI97ZUVmUdBYwJclX/LId48AUNRcNPR/hELhA6NCMIQQLwohqoQQe3o5L4QQTwoh8oQQu4QQc4d7jwpFbxQ0FuBv8jdSZHPqcqhyVGESJmKsMQBEBEYYfaLaO9upba3tVTBcbhc/XfdTJkZMZEbMDEpbSofvj1Eo+mBUCAbwMnBBH+cvBLK6fu4AnhmGPSkUAOQ35BtpsN4obCwkNSyVqdFTAS2OUeWoIjowGovJAkBEQIRhYVTatR5T8UFeBCNCy5QK8w/j6UVPkxWZRWmzEgzF6MAy0hsAkFJ+KYRI72PJMuBVqVU0fSuEiBBCJEgpVcK6Ysj53cbf4ZZuXr/oda/nC5oKyI7MJsYaQ2xQLDl1OdS31RMbFGusiQiIMCyFCkcFAAkh3mMY102+jssyLyMhJIGkkCRqW2txdDgI8gsagr9OofCd0WJh9EcSUNztcUnXsR4IIe4QQmwRQmyprq4els0pxjelLaXk1ufilu4e5zo6OyhpLiE9LB2AqVFT2Ve7jypHFbYgm7EuIiDCyJLSazC8WRhmk5lfnfQrw1pJDkkGoKylbFD/JoXiWBgrgiG8HPPaP0FK+ZyUcr6Ucr7NZvO2RKHwGZfbRY2zBqfL6dU1VNxcTKfsZEL4BACmRk+loLGA0uZS4oLijHURgRFGA8IKu2ZheIthHE1SqPa9qKSlZDD+HIXiuBgrglECpHR7nAyor1yKIafGWWNYFt5aj+sFe7pgTImegkTicDk8XFKRAZG0u9txupyU28uJDIgk0BLY7/PrFoYKfCtGA2NFMD4EbuzKljoZaFTxC8VwoKfHAuQ2eBGMrhoM3SU1JWqKce7oGAZo1d560Z4vRAVGYbVYKWlWFoZi5BkVQW8hxJvA2UCMEKIEeADwA5BSPgt8AlwE5AEO4JaR2aliPLO7ejexQbHEBR9xJelT88zC7LXHU2FjITarjRD/EEATiajAKOpa64i19hSM+rZ6KuwVpISm9LiXN4QQJIUkKZeUYlQwKgRDSnldP+clcNcwbUdxgnLvuns5M/lMHjz1QeOYngI7yzbLq2AUNBWQHp5uPBZCMDV6KhtKN3haGIFdHWtbG6mwV7AwfqHP+0oOSfbZJfVZwWc4XU4uz7rc5/srFL4yVlxSCsWQ4pZu6lrrjK6zOpWOSvxN/ixMWEhRcxFOl9M4J6WksLGQCWETPK6ZFj0NgfCwVHQLo7i5mJaOFp9dUqAFvkubS32ak/Hc7ud4cc+LPt9boRgIo8LCUChGGnuHHbd0U9xc7HG80lFJXHAc2ZHZuKWb/IZ8psVMA7SOtE3tTR4WBsAPpv6A2bGzCfUPNY7pgrG/fj+A1z5SvZEckozD5aC+rZ6owKhe17V1tlHQUIDZZEZK6dHYUKEYDJSFoVAATe1NAFQ7qz2siEp7JbFBsWRHZgOemVJ6Dyk9Q0onPCCc05NO9zgW5h+GQLC/VhOMAVkYIVpqbX8V34caDuGSLto626htVR1uFYOPEgyFAmhqazJ+756RVOmoJC4ojuSQZALNgR6CcXSGVF+YTWbCAsKM63sTjBWre2ZiJYdqqbX9Bb4P1B0wfleFfoqhQAmGQsERCwMw3FJSSqocVcQFx2E2mcmIyPAYbnSo4RAB5gCf3Ut6LYZZmLFZvReV/mVtz8C6YWH0E/jWp/2BEgzF0KAEQ6HAu2DUt9XT4e4wKrbbnXFGplRjWyMfHPqAkxJOwmwy+/QcehwjNijW52sAgvyCiAqM6rUWQ7dKDtQdMFxnqtBPMRQowVAo8HRJ6YKhp9TqgrE7P4i61jpqnDW8sPsFWtpbuHfOvT4/hy4YR7ujVqzOJX35StKXrwQwfu/unkoOSe7VJfWXtQdxSzcH6g8wN3Yu4QHhysJQDAkqS0qhABrbtdbj6WHpxjd5vWhPFwx3m/ZB/1XJV7yR8waXZFzCpKhJPj+HXotxtGDcvySb+5dolkH68pUUPrq0x7VJIUnsrtnd671Lm0uxd9iZEj2FndU7KbMrwVAMPkowFAo0C8MszEyKmtRjLvelf9mNdBUhzNoH/W++ehiz2c1dswdWS6pbGANJqdVJDk3m88Of43K7sJgsrFid6xHvOOevb2BNhq25VpJCkvqc36FQHCvKJaVQoMUwwgPCSQlNoaylDJfbRYW9ArMwk/fQNRQ+uhTZGUJ0YDTC3MYPpn6fxJDEAT1Hby6p7ty3OMvr8aSQJDplp2H13L8km8JHlxrWyL0XBmEWZn57wSISQhIoaynzqdBPoRgISjAUCjTBCPMPIzU0FZd0UW4vp9JRSYw1xiNAPTlqMqF+odw247YBP4cvFobumjoavc15b7UYB+oPMCF8AgHmAJJCkmjtbKW+rX7Ae1Qo+kK5pBQKNJdUmH+YUfNQ3FyspdR2m2lx3+IsrjzpV9g77EY8YiCkhqViEiYmhk8c8LV6m/OSlhIW4tmH6r7FWXxct9/oT5UYrFk+ZS1lfVaGKxQDRVkYCgWahREaEGp0kS1pLjHagujcvySb1LBUpkRP6e02fbIgfgFrr15LaljqgK+ND47HLMwUNRX1OHfTGTFUOaqYHDUZwHCVqdRaxWCjBEOh4IhLKjYoFn+TP8XNxVTYKzwsjMEgxhpzTNdZTBZm2Waxvnh9j9jE/jqt3YguGPqscJVaqxhslGAoFBwRDJMwkRyaTE5dDk6X06NF+UizdOJSDjUeMgRCR28JMilSS/EN8w8j1D9UCYZi0FGCoTjhcUs3ze3NhPmHAZASmsLOqp0Ag25hHA/np5+PxWTh4/yPPY7vr9tPfHC8R1wlKSRJ1WIoBh0lGIoTHr21eXhAOKAJRmtnK4BHDGOkCQ8I58ykM/mk4BNcbhegtVjfULqBGTEzPNYmBCcoC0Mx6CjBUJzwNLZpVd66haFnSsHosjAALsm4hBpnDZvKNwGwYusKHB0O7px1p8e6pJAkSlt8G7qkUPiKSqtVnPDojQfDAo64pHRGUwwD4IzkMwj1D+Xj/I8JtATyft77/HD6D8mMzPRYlxiSiNPlpLGt8ZhSgBUKb4wKC0MIcYEQ4oAQIk8IsdzL+ZuFENVCiB1dPwOvmlIoesEQDH9PwYgKjMLf7D9i+/JGgDmA89LOY03RGn7/7e9JCE7gRzN/1GOdkVprV6m1isFjxAVDCGEGngYuBKYC1wkhpnpZ+raUcnbXzz+GdZOKcUf3TrB6p1pdMJJCkjAJ06hzR+lcknEJTpeTvIY8li9cTpBfUI813Yv3FIrBYsQFA1gI5Ekp86WU7cBbwLIR3pNinNO9cZ9uYehBb3+zPwnBCaMq4N2dObFzSA9LZ1HKIs5JOcfrGt3CUIKhGExGQwwjCSju9rgEOMnLuiuFEGcCucD9UspiL2sQQtwB3AGQmjrwilrFicfRLimAh09/2OPxaMIkTLx98dv4mf0QQnhdE+YfRohfiBIMxaAyGgTD2yv+6NSOj4A3pZRtQogfA68Ai7zdTEr5HPAcwPz581WKiMLg6Jbg+sCi0xYUYhEWrBarcW5e3Lxh399A8OaG6o4QgsSQRIqaizhQd4CcuhzC/cM5J9W7RaJQ+MJoEIwSIKXb42TA42uRlLK228PngceGYV+KcUZvg4oe2riVkqKwXr+tj1USQxJZX7yeDaUbADALM2uuXnPM7UkUitEgGJuBLCHEBKAUuBa4vvsCIUSClLK86+GlQA4KxSChtwUZb/xw+g+ZGD6RSZGTCPUP5c61d7IyfyU3TbtppLemGKOMuGBIKV1CiLuBVYAZeFFKuVcI8RCwRUr5IXCvEOJSwAXUATeP2IYV44Lug4qa2pqMGozxxJzYOcyJnWM8nhkzk/fz3ufGqTeOO2tKMTyMhiwppJSfSCmzpZQZUsqHu479tksskFL+Uko5TUo5S0p5jpRyf993VCj6pvugosb2xnFpYRzNssxl5DXkkVOnDHTFsTHiFoZCMdI0tTWRHpZ+7Ddwu6GlEprKwF6t/Thqob0F2u3aj7sTZCdIN5gsYPYDsz8EhEJAGASGQ7ANQuIgJBbCErU1g8j56efz2KbH+CDvA6ZGeyt1Uij6RgmGYsyzYnVur6NNvVHcVEx8SDx+Ju0D2ecYRrsdKvdqPzUHoeYA1OVDYwl0tnu5QGiC4BekiYTJpB1zd4K7A1yt0NaiCUmPS80QngSR6RCdBbFTwDYZ4qeDNdLnv7U74QFaltTKgpX8fP7PR10Vu2L0owRDMeb5y9qDPgtGQ2sDyz5Yxs/m/Ywbpt5wpLX50TEMtxtqcqH4Wyj6Dsq2aY+lWztvsUJMFiTMgimXQHgKhCdDcCyE2MAaBf7B0F+sQErocICzQbNMWqqguRwaiqDhMNQVwO53oatBIgCREyBxDiQvgLRTIG4GmH17Ky/LWMaqwlV8WfIl56ad69M1CoWOEgzFuKTCXsGftvyJn8//OfHB8cbxvbV76XB3sK1qGzdMvYGWjhYkUrMwGorh0BeQvx4K/qu5lQCCYiB5Pky9TBOIuGmaQJgGIQQohCYs/sGaReENKTURqdoH5bugbDuUbIa9/9bO+4dC2qmQsQgyF0N0Zq9CdUriKdisNj7I+0AJhmLAKMFQjEl6K8K7b3EWdy1K52frf8bumt3MiZ3D9VOOZGnvq90HwJ6aPSAlTcXfAhD29dNQdq+2KCQess6D9NMh5WSIzujfUhhKhNBiGmGJkNntQ76xFIo2wuGvNZE7uEo7HpEGky+GyUsh9WQwmY1LLCYLl2Zcyot7XuSDvA9Ylul7F56Buv4U4w8lGIoxSW9FeAAPbnyQ3TW7CTQHsrN6p1fBKLeXU/vXWTS1lEFSAmF+QXDeHyBzCdgmjaxA+Ep4Esy4SvsBzX11aC3kroLNz8O3T2uB9KmXaWuSF4LJxI9m/Yg9tXv4zde/wS3dXJ51uU9PNxDXn2J8ogTjKNo723lq+1PMjp3NolSv3Uf6pNJeSaWjkpm2mUOwO0V/vJf7Hu/mvsttM26jsLGQXdW7tBP2Gtj1DvsK1xLrdlFlsbA3IgH/2d+DgrcJu/AJiF8wsps/XqImQNRtsOA2aGuGvDWw933Y/pomIBGpMPsGrLOv56lFT3HvF/fywDcPIJFckXXFSO9eMUisK1pHYVMhN027CZMY3MqJUVGHMZrwM/nxUf5HrD68+piuf3bXs9zzxT2DvCtFX+hFeIWNhTz83cOcmngqd8++m1m2mZS0lFD7zg3wp8k0rP41ZSbJ5fGnIhDsnXkZTakLgSOdascNAaEw7XL43ivwP3lw+d8haiKsfwT+PIPAN6/nyeSlnJJwMg9tfIj61nqvt1mxOpf0X75P5sN/BDpJX76S9OUrPdrDK0YXHx76kHcOvDPoYgHKwuiBEILZttlsr9p+TNeXNpdS11pHe2e7SlscJnQ3ybridXS4O3hw4a8wb3+dmZv+BkGwq+QbzllwG/tSZ8OWh5k/93bWfFfLnto9xARpfZXGdeFeQCjMulb7qT8MO/4J214l8J0buSk6lW/CIK9yBwvSejYmvOPsJPa6H+e7iu9wll5H3q9+NQJ/gMJXpJRsq9rG6UmnD8n9lYXhhdmxsyltKaXaUT3gaysdlQDUOmv7WakYbLaUbSTdEkb83xfBR/cyVfhjwcSuM+6ECx9ln7sFgClRU5gWM409NXt6DE8a90SmwTm/5M8z3oPvvUpGqJaZlff+rfDZr7RMsS4aWhu4bdVtbKncgkmYMAVUjNSuFT5S1FxEXWsds2NnD8n9lWB4Qe+/s6N6x4Cuk1JSYdfeVDXOmkHfl6IXmivpXPV/2Vb6NfPrSiFpHtz0EYE/2sCk6CnsrNMC3ftq95Eckkx4QDjTY6ZT11pHbn0uFpNna/MTgT9/UQBTlxF706eEWKwcsk2E756FJ2fDB3fRVLmLmz+7mdz6XP58zp+ZEDaBrOSWkd72Cc+KrSt4I+eNXs9vq9wGwNzYuUPy/EowvDAlagoB5oABu6VaOlpwuByAEoxhwV4Dq34Nf5nJgW3P0WIyMf/0X8H334EJZ4IQzLTNZE/NHlxuF/tq9xktMaZHTwfg2/JvCfMff63NfUUIwcTILPKjU+G+nTD/Vtj9Lv997UIONR5ixfzlnJ1yNpmRmbgt5f3fUDGkfJz/MZ/kf9Lr+e1V2wkPCGdC+IQheX4Vw/CCn9mPadHT2FE1MAtDty4AalqVYAwZrY3wzVPw7d+0KumZ17A5KQtyXmZ+tmddwSzbLN7c/ybbKrdR2lLK1dlXAzApahIWk4W61rrj6yM1huitdmXePBsl7u0QkQIXPQ5n/Iy9n92BtSWf0/71Y5j3LZm2RFa1rMLR4eh3eJNiaOh0d1LrrKXdaxsaje1V25ljmzMkAW9QFkavzImdQ05tDk6X0+dr9PgFKAtjSOjsgE3Pw5Nz4MvHtSK2O7+Fy59lS0shqaGpPeZw6+nNbx14C4Ap0VMAbW53dqQWLB+Prc29cf+SbAofXWrUrOi/Xzp1DnWtdUcypULj2RsczhTbDMxzb4StL5P11ZMA5NfsHantn/DUttbSKTtpaGugsXurmC7qWusobCpkTtwcL1cPDkowemFO7Bxc0sXeAbxBdAtDIFTQe7A5uBqeORU++T9gmwK3r9NSRm2TcEs32yq3MT9+fo/LkkOSiQqM4ouiLwCYGnWkS6vuljphAt69kBGRAcChhkMAuNwu9tftZ1rcHLh4Bdy1iYzkUwDIe+9GLcvK7R6x/Z6oVDmqjN+Lmop6nNdd6EMVvwAlGL2iZxkMJPBd6ahEIEgLS1MWxmBRVwBvXgdvXKU1/rv2Tbj5Y0g68qY4WH+QpvYm5sf1FAzRFcfolJ0khSQRERhhnJseownGuKvB8IHuA6QywjXByG/MBzThaO1sZVr0NG1BdAYpV7+Bv8lCXlAovP8TeGEJlA3MZas4Prp7MIqavQhG5Xb8Tf5D2rpeCUYvhAeEMzF84oAC3xX2CmKsMcQHxyvBOF5cbbD+MXj6JMj/L5z7IPxkI0y+qEfbjs0VmwG8CgZocQygxxtpWoz2gXgiWhjdW3zEB8cTZAkyLIy9tZpVrQsqgNlkJiMii7yUOXDZs1on3efOho9/Bk7vRX+KwaV7mn9vFsb0mOlDWv+lBKMPZsfOZkfVDtzSN/O70l5JfHA8MdYYJRjHQ+HX8OzpWlXy5KVwzxY4/adg8f5G2FK5haSQJBJCErye700wMsIzSAhOML5hn6gIIciIyOBQY5dg1Owl1C+UlNAUj3WZEZkcbMiD2dfB3VvgpB/B1pfgqYVaCxIpR2L7JwxVjirMwkxcUByHmw97nHO6nOyr3ecxkncoUILRB7Nts2lqb6KwsdCn9RWOCuKC4oixxlDrrEWqN9DAaG2Cj+6Dly/Shgt9/z24+iWtS2svuKWbrZVbe7UuQPv/eP3k67lowkUex80mM59e8SnXTL5m0P6EscrE8InkN2guqb21e5kaPbVHpk1mZCZVjiqa2pvAGgEXPqbFkkLj4V83wVvXa1MHFUNCpaOSGGsM6eHpPSyMPTV7cEkXc+OGLn4BSjD6RFdrX9xSetGebmG0drZi77AP9RbHD3lr4W+nwLZX4ZS7teynrP7nNRyoO0BDW4PXgLeOn9mPX570SxJDegqPuVvr7xOZjIgMqp3V1DhrOFB/wHDXdSczIhM4EhwHIHG2JhpLfg+H1sHTJ8OON5W1MQRUOaqIC4ojNTS1RwxD/4zSremhYlQIhhDiAiHEASFEnhBiuZfzAUKIt7vOfyeESB+OfaWFpRFrjeXJ7U/yRs4bfeY/N3c043Q5iQ+OJ9oaDajUWp9oa9GsitevAP8g+OHncP7D2kChPuhwd/Davte4ddWtBJoDOTnh5GHa8PhEz5T6rOAzXG7XkYB3N3TBOFh/0POE2QKn3Qs/+RripsL7P9asjebKHvdQHDtVjipig2JJC0ujsa3RI7V2a+VWMiMyhzyBY8QFQwhhBp4GLgSmAtcJIY4O898K1EspM4EVwGPDtDeeWvwUmRGZPLrpUS7+z8VsKt/kdW2lXXtz6C4pUILRLyVb4O9nwNZX4NR74EdfQcqRFuN1rXXG/IruHKg7wJUfXsnjmx9npm0mb1/8tsdUPcXAmRg+EdA6nQJeLYyE4ASCLEHkNeR5v0l0Bty8Es5/RJtc+MwpsL/3qmTFwNAFIzU0FYDDTVoco6Ozg22V21gYv3DI9zDiggEsBPKklPlSynbgLeDoMWDLgFe6fn8XWCyGqZfDlOgpvHD+Czx/3vO4pZu/7fyb13V6DUZ8cDwxgV2Coaq9vePuhP8+Di+cpxXj3fyxNrzIL9Bj2TM7nuG6ldd5tJovbynnJ2t+gr3dzlOLnuKZc59hYsTE4f4Lxh2JIYlYLVZy6nKIDIgkMbin+04IQWZkZu+CAdp0v1Pugh99qcWe3roOPr4f2h1DuPvxj6PDQUtHiyYYYZpg6G6pXTW7aO1sZWHCiSEYSUBxt8clXce8rpFSuoBGINrbzYQQdwghtgghtlRXD7zbbG+cnHAypyae2msAXM+R7m5hqOI9LzSVw6vLYN3DMP0KzY2R7r0Vc3FLMW7p5hdf/oKvSr6iqb2JO9feSaurlb8v+TtnpZx1wvaAGmxMwmT0H5oaM7XX/65ZEVnk1fchGDq2SXDbWs1y3PKiloJblTOIOz6x0D9fYoNiSQ5NRiCMwPem8k0IRJ+JH4PFaBAMb6/MoyNmvqzRDkr5nJRyvpRyvs1mO+7NdWdC+ARqW2u9luVX2CswCRMxQTGEBYRhMVmUS+poDq6GZ0+D0q1w2TNw5T8gsHefa3lLOSfFn0RWRBb3r7+f2z+/ncKmQlacs4LMyMxh3PiJgZ5e7C1+YayJyKC+rd63L0OWADjvD3x6wW/Y6GqA586Bba+pgPgxoFd5xwXFEWAOICE4wXBJbarYxJToKcNSgDoaBKME6J7wnQwcnZtnrBFCWIBwoG5YdtcNvUldYVNhj3OVjkpiAmPwM/lhEiaiA6OVYOi4O+GLh7Vq7dAEuOO/MPv6Pi+RUlJuLycrMotnlzxLUkgS+2r38dCpD3FSwknDtPETC921p7dM8YYR+G442Oua7rR1tvHAoXd4JCUTmTwPPrwb/vNj5aJCayb41+1/9Wha2hu6YMQGxQKQEpZCcXMxTpeTndU7OSl+eN4To0EwNgNZQogJQgh/4Frgw6PWfAjc1PX7VcAXcgSKHHSTvaCxoMc5PaVWRxXvdWGvhdev1JoFzr4BblsDtux+L2tsa8TpcpIQnEBUYBSvXPAKr174KpdkXDIMm+Y2RXAAACAASURBVD4xOSPpDKZHT+8zl39q9FQCzAH8M+efPtUZfVP6DU6Xk8KWYnIvfgLOWg673tZai9TlD+b2xxz7avfx3K7n+Dj/437XHi0YaaFpHG46zI6qHXS4O4YlfgGjQDC6YhJ3A6uAHOAdKeVeIcRDQohLu5a9AEQLIfKAnwE9Um+Hg6TQJCwmi1fBqHRUenRK1Yv3TmjKdsBzZ8Hhb+DSv8JlT4Ofb4OKyu3a7AW9diIiMGLIq1hPdCZFTeLNi9/s07URHhDO3bPvZl3xOlYdXtXvPdcUrSHELwSzMPPp4c/hnF/C9/8FjSVaXCO3/3uMV3LqtJhOjzRlL1Q5qgj1CzVay6eGpdLU3sTnhz/HIixD2nCwOyMuGABSyk+klNlSygwp5cNdx34rpfyw6/dWKeXVUspMKeVCKeWIfDXxM/mRGpraI/CtF+3FBXkKxgltYex+F168QPNX37oK5t44oMvL7JpXMiHYe7sPxchxw9QbmBY9jT9+90caWht6Xdfh7mB98XoWpS7ipIST+KzwM80qyVoCP/ovRKTCP6+BDStOyLiGIRg+uPeqHFXYgo7EZPXU2k/yP2F6zPRhm1EyKgRjLDEhfAIFTZ4WRveiPZ1oazR1rXV0ujuHe4sji7sTVj8A790KiXPgjvXavwNE9+v21h9KMXJYTBYePPVBmtqaeHzz472u21yxmab2JhanLuaC9AsobSllT80e7WRkulakOf0KWPM7+Pft0OH77JnxwP7a/YDm4u5wd/S5Vq/B0EkLSwPA4XIMmzsKlGAMmAnhEyhuKvb4H6x/uB3tktKHnZwwtLXA2zfA13+GebfAjR9AyLFlqpW1lBFoDiQyIHKQN6kYDCZFTeLWGbfyUf5HbCzb6HXN2sNrsVqsnJp4KotSF2ExWfis8LMjC/yD4MoXYPFvNYv0pQuhuf8A8Higw91Bbn0usdZYXG5Xv/3qKh2VHoKhp9YCwxbwBiUYAyY9LB2XdFHSXGIc06u844O6WRiBJ1h7kMZSeOkCyP0MLnwcLvlzr91lfaHcXk58cLyqsxjF3DHzDqICo3g3990e5zrdnawtWssZSWcQaAkkPCCc0xNPZ1XhKs/uz0LAGT+Ha/8J1bnw/GKo2DOMf8XIUNBYQLu7naUZ2vTDvuIYne5Oapw1Hi5vf7M/CcEJ+Jv8mRU7tP2juqMEY4B4y5SqcByp8tY5oYr3ynfBPxZDXSFc97bW9vp4b9lSruIXoxx/sz9L0pbwZcmXODo802R31eyitrWWc9OONJA8f8L5VDoq2VHlZfDS5Ivgh5+C7NRiXwdX91wzjsip1eIXSycsxSIsfcYx6lrr6JSdHhYGwKzYWZyZfCYB5oAh3Wt3lGAMkPTwdMBTMCrtlVrRXpdIwBHBGPftQQ59obkShEkLbmefNyi3LbeXe+0uqxhdnJd2Hq2drXxV+pXH8dWHV+Nn8uOMpDOMY+eknEOAOcDTLdWdhFlw+xcQla4Fw7e9OoQ7H1n21+3HarGSGZFJenh6nxbG0Sm1Oo+d8Rh/OvtPQ7rPo1GCMUDC/MOIscZ4FO+VtZQRY43BYrIYx06IBoQ73oQ3rtYCmLetgbjeK4R9ZcXqXNo626htrVUNBccA8+LmERUYxeeFnxvH2jrbWFW4ipMTTibEP8Q4HuwXzOzY2eyt2dv7DcMS4ZbPYOLZ8OE92tTFcZhBta92H9mR2ZhNZrIisnwSjO4uKdB6ex09s2SoUYJxDEwIn2BYGPYOO+uK1/Xo4xLkF4TVYh2/grHhz1ob67TT4JZP+hxyNBD+svagkUSgLIzRj9lk7uGWeiPnDaocVdw4rWcqdYw1hrrWfpo0BITA9W/DrOu1qYsf3adl340T3NLNgfoDTI6aDEBWZBZl9jJa2lu8ru/NwhgJlGAcA+lh6RQ0FiCl5P2892npaOGGKTf0WDcuazGkhM//L6x5AKZfCd9/t89+UMdCWYuqwRhLnJ9+vuGWqmut4/ldz3NW8lleZ5REBUb1LxgAZj+47G9aQHzbK/Cvm7U57+OA4uZi7B12Y2RwVmQWgEcX4O7p+JWOSszCTFRg1PBu1AtKMI6BCeETaGpvora1ljdy3mC2bTYzbDN6rBt31d6dLvjgbvjmr7DgdrjiH8eVCaWzYnUu6ctXkr58JQA/eEWr/v10e+tx31sx9MyNnUt0YDSfF37OMzuewely8rP5P/O6NiowCofLQavLh/+3Qmgpt+f/EXI+hH9+T0vdHuPoBXvdLQyA3PpcQPNaXPTvi7hn7T00tTdR5agixhozKqZDKsE4BvRMqVf2vkJxczHfn/p9r+vGlYXhaod3b4Edr8PZv4SLngDT4Lx87l+STeGjSyl8VEsxvP9CGwLBL89XU/TGAmaTmXPTzuW/Jf/lX7n/4qrsq4yBTEejf0uub633/QlOuVPrblzwldYa3zHsfUcHlZzaHCwmi9HIMTE4kWC/YCOO8dq+1yizl7GhdAPXfnwte2v3jgp3FCjBOCZ0wXh93+vEB8dzbqr32dPjpmNth1MbuZnzofZt7+zl2re/IaK8pRxbkA0/k9+QPYdicDk//XzaOtuwWqzcOfvOXtfphZh1bQP80J99PVzzGlTsglcuBfvYfV/tr9tPVkQW/mbNOhdCkBmRycGGgzS2NfLK3ldYlLKIFy94EafLSV5DnhKMsUxCcAIB5gBc0sV1k6/zyI7qTnp4Ok3tTcagkzFJW4uWCZW3Bi75i/Ztbwi5b3GWllLrZeKbYvQyN3Yuc2Pncv+8+/v0tUdZtXN1zmOwEiYvhevegto8eOmiMVkVLqUkpzbHcEfpZEVmkdeQx0t7XsLeYefuOXczJ3YOb1/8NkvSlrAkbckI7diTfgVDCLFGCDF8pYRjAJMwkRaWhtVi5cqsK3tdd3bK2QB8UfTFMO1skGlr1mZYHP4GrngO5t085E95/5Jsyu2qaG+sYTaZeeXCV/jepO/1uS4qoMsl1TYAl1R3MhfDDe9q3W5fulD7dxSzYnWux+NKRyX1bfVMiZ7icTwrIovGtkZe3fcqF0640IhrxAbF8r9n/y9LJy4dtj33hS8Wxi+AFUKIl4QQ6l3cxe0zb+fXJ/26z1bQSSFJTI6azNqitcO4s0GitUmbY1G8Ca56AWb2/UEwWLilmwp7hWo6OE45LgtDJ/10uPF9zS318sWjWjT+stazvkLvuzXbNtvjuC4Qbunu06U30vQrGFLKbVLKRcDHwGdCiAeEEL4NNRjHXJB+Acsyl/W7blHqInZU76DaMXjzxYccXSxKt8LVL8G0ywf9KT7I+4Dndj3Xoz12rbOWDneHsjDGKUGWIPxN/gOPYRxNykK44d/gqB31otGdL4q+ICE4oYdLKjsyG7MwsyxzmdGJdjTiUwxDaB3gDgDPAPcAB4UQPxjKjY0X9ID4uuJ1Pq0/2oQddtpaNDdU2Ta46iWY2r8oDhSX28Vjmx7jr9v/ynnvncdjmx4zGjjqczBU0d74RAhBlDXq+CwMnZQFR4lG6fHfcxA4Ok1c//3xVbvYWL6Rc1LO6dFUMzwgnFcvfJXlC0dkNpzP+BLD2ACUAiuAJOBm4GxgoRDiuaHc3HggMyKT1NBUn91SR5uww0q7A968Fkq2wFUvwtRL+7/mGNhZvZPmjmbum3sfS9KW8Nb+t7jqo6vYUbXDmLSn2oKMXyIDIo89hnE0umjYa+DVS6G5cnDuexwcnSau/z5vSjVtnW2ck3qO1+tm2mZitYxu540vFsaPgSQp5RIp5W+klB9LKfOklPcAZ/R38YmOEILFqYvZVL6Jpvamkd5O73S0wlvXQeEGLcA9BJaFzlclX2ERFq6ZdA0Pn/4w/1n2H8IDwrl11a38O/ffACpLahwTFThIFoZOygJt7GtTmVanYR+dxbJfFH1BqH8o8+LmjfRWjhlfYhh7ZO/T3kdH6H6Usyh1ES7p4suSL72e782EHTb3VGcH/OsmyF8Py56GGVcN6dNtKN3A7NjZhPqHAlr68WsXvsbU6KlsLN9IqF+oR9M6xfgiKjBq8CwMnbRTtJTb+gJ4bRk4B/n+x8h9i7Vgtsutvf/PTD5zTNcXHVcdxvHO1hZCRAkhVgshDnb963W8mhCiUwixo+vnw+N5zpFgpm0mNqut1/Ta3kzY+5dkD/3m3J3wnx9rg48u+n8wx3vV+mBRaa/kQP0Bzkj2NE4jAyN5/rznuXjixZyWdNqQ7kExskQGRvrWT2qgTDwLrnkDqvZr7dHb7YP/HANEfw/vqNpBQ1sD56R4d0eNFUa6cG85sFZKmQWs7XrsDaeUcnbXz9A41ocQkzCxKHURG0o3jK7eUlLCyp/Dnndh8QOw8PYhf8qvy74G4PSk03ucC7QE8scz/sgTZz0x5PtQDA/erOSowCicLmePoUuDQta5cOU/oGQzvP0DraXNMNKbM2Zd8Tr8TH5eX/djiZEWjGXAK12/vwJcNoJ7GVKunXQtLreL3238Xa8vKqfLyVkLd2LvGKZvRmsfhK0vwen3wxnem8UNNhtKNxAXFEdWRNawPJ9iZPGWxGH0kxpst5TOtMu0rgSH1sK/bx+21uhri9Zy2lun0dze7HFcSskXRV9wUsJJBPsFD8tehoqRFow4KWU5QNe/vTVMCRRCbBFCfCuE6FNUhBB3dK3dUl09emofMiMz+encn7K+eD3vHXzP65o397/JtuY3+br066Hf0Ma/wYYVMO8WzboYBjrcHWws28jpSaerWd0nMMfUgHCgzL0RzvsD7Htfs6KHYQjTd+Xf0dzeTH6jp6f+UMMhSlpKxrw7CsB7E6RBRAixBvCWI/nrAdwmVUpZJoSYCHwhhNgtpTzkbaGU8jngOYD58+ePqlFdN0y9gS9Lv+TxzY+zIH6BR4FOq6uVV/dqIymrnUMsdLvegVW/hCmXwtI/DWkjwe7sqNpBS0dLj/iFYnyxYnWuh2WhJ3PctziL+5dkExnY1YBwKOIY3Tn1Hq1GY8MKCE2As/+/AV2+5vAamtubuTzLt8JVvT15UVMRs2xHuinp7cznx8/3et1YYsgFQ0rpvZUrIISoFEIkSCnLu9qOVPVyj7Kuf/OFEOuBOYBXwRjNmISJP5z2B6788EqWf7mcVy98FT+zljHxQd4H1LZq8Y0hrQrPWwPv/wTSz4Arnodh7LH/VelXWEwWr4N1FOOH+5dkG8He9OUrjWQOHd3C6C4YZS1lVNgrmBs3d3A3s/gBaKnSJveFxML8W3y+9OW9L1PYVMiyzGX9jkKVUhqCcbjpsMe5gsYCLMJCSmjKwPc/yhhpl9SHwE1dv98EfHD0AiFEpBAioOv3GOA0YN+w7XCQiQ+O53en/o49tXv49de/xi3ddLg7eGnvS8yyzSIuKG7oLIzynfDOTWCbAte+AX6BQ/M8vbCxbCNzY+eOeT+u4vjwJhh/3vZnfrrup4P/ZEJo8Yys82Dlz2D/Sp8vLW8pp7Gtsc952zqVjkojdnF0d+rCpkKSQ5PHdDqtzkgLxqPAEiHEQWBJ12OEEPOFEP/oWjMF2CKE2AmsAx6VUo5ZwQBYkraEn879KZ8WfMoTm5/g04JPKW0p5bYZt2mCMUALw6d6jYYirU25NVIrchrksar9IaXkcNPhHj10FOMbvQ6hO1aLlUBzoEcMI6c2h/q2+l7nWh8XZj+4+mVInAPv3golW/u9pL2znSqn5vDYVLGp3/W6dRHmH0ZhU6HHuYLGAtLD0we661HJiAqGlLJWSrlYSpnV9W9d1/EtUsrbun7/Rko5Q0o5q+vfF0Zyz4PFD6f/kBum3MDrOa/zyHePkBWZxZnJZxJjjRmwhdFvOxFnPbx+FbhaNbEIG9rGfnn1eThdTo9jTe1NOF1O4oLihvS5FaMLb7VEQgiPWgxHh8Nw4+i9xAYDp8vJBe9dwIbSDeAfDNe9DaFx2qjXuoI+r9Vb1IBvgnGg7gCgjTQoai4yMiE73Z0UNRUxIWzCcfwlo4eRtjBOWIQQ/M+C/+HC9Auxd9i5dfqtmIQJW5CNKofXUM6x4WqHt27QKmCv/SfETun/muOgqb2J7338Pd7c/6bH8UqH1uMnNnh0TA5TjCxRgVGGYOQ15CHRPmDLW8r7umxAFDcXU9pSyu7q3dqBEBt8/12QnVqDzT5GvZa2aI0M08PS2Vqxlc5+UnNz63NJCkliStQU7B12Ix5ZZi+j3d1uTOkc6yjBGEFMwsTDpz/Myxe8zEUTLgK0gSlN7U20ulr7vNandiJSwkf3weENsOxv2hyBIWZf7T463B09An96N9r4INVUUKFVe+suqf11+43jg2lhVNi1iXweY5JjsuDaN6GhGN76PrjavF5b1qLtY1nmMpo7mtlfv9/rOp3c+lyyI7NJDUsFjsQxCho1S0a5pBSDgp/Zj3lx84y6BJvVBvSfWutTO5Gv/gQ7/wln/xJmXj00f8BR7KnZAxx5w+noFoZySSnA08I4UHeAEL8Q/E3+PV43x4MuGD3eS2mnwGV/g6Jv4KOfeq3RKGspwyIsXDzxYgA2l2/u9XnaOtsobCokOzKb9LB04EimVGFjIYBySSmGBluQJhge34r6oLGtEURHzxN7/g1f/B5mXgNnec8/r2utMz7gB4t9tVo+QncfMECVowqBICYoZlCfTzE2iQqMor61HiklB+oPMClqEgkhCYMqGPpr0Ot7acZVcPavtC9UG/63x+kyexlxwXHEB8eTHpbeZxzjUMMh3NJNdmQ2iSGJWISFouYuC6OpgIiACCICIwbnjxphlGCMMnQLw9c4xrUfX8tp87d7HizdptVapJwMl/6118K8l/e8zK2rbu21VcmxoAtQeUu5x30rHZXEWGPGRWqh4viJCoyitbMVe4ed3PpcJkdNJjE4sccXjePBq0uqO2f9AmZcDWsfgr3ve5wqaykzhngtiF/AtqptuNwur7fRM6SyI7OxmCwkhSZ5WBjjJX4BSjBGHbFBWlDYl9TalvYWSlpKiInsNua0qRzeuh6CY7VaC0tAr9dXOatwuBy0dAxOKmOts5ZyezmJwYm0u9uNwB9oMQzljlLo6NXeO6t34nQ5mRQ5icSQRCPYPBh0tzC8fikSAi59CpIXah2by3cap0pbSo2ZLAvjF2LvsBvW89Hk1ucSaA40CvNSQ1MNwShoLDDcVOMBJRijjIiACCwmi5ED3hf6m8vw0XY4NbFoa4br34Lgvt0/DW2a0AxWB929tXsBrc4EPDNeKh2VhhgqFHrx3jdl3wBoLqngBOpa6/pN+PAV3cLocHdorltv+AXCNa9DUBS8eT20VNHR2UG1o5qkkCTgSEuP3txSufW5ZEZkYu7qmpAWlkZxczGNbY3UttYqC0MxdAghsFlt1Dj6j2GUtGiD76ud1Vrg7sN7oGy71vIjblq/1ze0aoIxWD199tbuRSBYlLoIgOc3bjPOVdoriQtWFoZCQxeMjeUbsQgLGREZhgtoMNxSbumm0lFJckgy0HdMsMZixn7VC1rfqbdvoKKpCIkkIUSrV4qxxpAZkcnmip6BbykluXW5ZEcdSTZJC0vD6XKypWILgLIwFEOLLcjmk4VR0qwJRo2jBvfXT8Luf8Gi/wuTL/LpeQwLo3WQLIyavUwIn0BGRAYAn+7XTHhHh4PmjmblklIY6IJxsP4gEyImEGAOMASje+A7pzaHP3z7BzrcXhI7uvDW6aDWWYvL7WJ6zHSg76zDWz67hT8c/hAuexqKv6N07W8BDAsDYJZtlmFBd6fGWUN9Wz3ZkUcEQ0+tXV+yHkBZGIqhJdYa61MMQ3dJuaSL+nUPaXO4z/i5z8+j58EPxnxlKSV7avYwPWY6Yf5hBPsFY/LTBMlIqVUWhqILPYYB4G5NJH35Sq76q1brcPNrq4yaovcOvsfbB97mw7zeB21663SgWym6YPRmYVQ5qihsKuSbsm+Q066AM35OWeE6AEPAQLMSGtsae7i2uge8dfQu1F+WfIlFaEHw8YISjFFIjDXGJ8HQLQyAGlumVpznY6vyts42HC5t4tlgWBiVjkpqW2upqrEx4Zef0NQcgvBrIH35Ss79i/ZmVxaGQsdqsWK1WAG4fNoCCh9dysGHrkFKE/ecF2PUFO2o2gHAMzufoa3Te5GdN/T4xYyYGUDvgqFXgde11mlzLM75NaWx2ZilJK6m0FiXEqYFtIubiz2u1wWj+0Cw+KB4/Ex+1LXWkRKWMq4yA5VgjEJig2Jp7mju0Y/paEqbi4l2awJRddb/gYAQn59Dj1/A4MQw9tZo5vrdp51D4aNLOTsjG5NfPYWPLuV/r08HlGAoPNHdUpOiJgFgMVmQHeGG5dzS3sLBhoMsjF9IpaOSt/e/bVzbX6cD3cLIiMjAarH26pLaWbMTgfYe2lyxGUxmylPnEysFlndv0bIOgbRQzWo4uoPBoYZDxFhjPOoszCazkTE1nuIXoARjVGIU73UFvjvcHfx8/c/ZVb3LWCPdbkqbDjPXqY1zrbYMbK6FHr8A37Ok+uqKu7d2LxZhYVKk9uZPDEns4ZJSWVKK7hiC0fWaAe11o3/Y76rehVu6uXXGrZyccDL/2P0PY3xxf50OKuwVWC1WwvzDiLHG9JpEsrNqJzNiZhAXFMeWSi1IXeqsJjF6CrS1wDs3gqudlLAUBMIoyNPJb8wnIzyjx311t9R4il+AEoxRSaxV+2DVA9+7qnfx+eHP+aTgE2NNzca/0IZkVvKZwMCHLumCYTFZfLYw+uqKu6dmD5mRmQRatBkbCcEJCLMTe4edKkcVEQERxjmFAiA6MJq4oDiPeMbClAwj6L29ejsmYWKWbRb3zb2P+rZ6Xt33qk/3rrBXaK9BPeuwtadgdLg72Fe7j5m2mSyIX8Dmis1IKSmzl5EUlaUFwUs2werfEGAOIC44zmPWhZSS/MZ8r6KgC4ayMBRDjt4+QxeB78q/A464fSj6jpINjwMwcc4thAeED7glen2bFvBOD0s/rhiGy+0ivyGfvbV7mRZ9JJXXSJFsKVdFewqv3DXnLh45/RGPY4khiVQ5tFqI7VXbyY7MJtgvmOkx01mcuphX9r5iWBk63mZuVNgriA/WGl1GW6O9fqE6WH+Q1s5WZtlmsSB+AXWtdeTW51LlqNJev9Muh5Pvgu+ehd3vkhaa5mFhVDoqsXfYjazA7uiZUuPNwhjyEa2KgaNbGLoI6IKxv24/rqYKLP+6iZJQGyBJCkvGZh14S3Q9hjExfCIbyzb2uq63+cw/PieR3a7/R05djpHyOC9unrEuIVjLYS+zl1HpUDUYip54G6aVGJyIRPuWv7t6N5dmXGqc+96k77G2aC27qndxSuIpxnFvMzfK7eVGbMRmtXl9jesu3pm2mUb78pX5K3FLt1HlzZIHoWwbfHgPzolLKRJHBCO/MR/Q3kNHc17aeTS0NhhB9/GCsjBGIeEB4fiZ/Kh2VOPocLCrehdJIUm0draS/59bwFFHycwrAC1XPDYo1udmhTq6hZERkUFzR3OvGSi9+YpPmdrErppdXJpxKY+c/gj/uuRfRmdPOCIYFfYKVeWt8Bm9WG598XocLgdzYucY5/QU2f4aZrZ3am1p9C8ptiAbLR0tPZJIdlbvJMYaQ0JwAsmhycQFxfFR/kdAt5Rasx9c9RL4hzC37Bsa2hqM1Nr8hi7BiOgpGOEB4dw+83aj+nu8oARjFCKEIDYolipnFVsrt+KSLm6apo0+31u9Ey56glJcxFpjCTAHEGONOSYLI9Qv1HAVdR+X6Qv7avchEPxiwS+4JOMSJkdNNlq0g/YmtZgsFDYVUtdap1xSCp9ICtZqFvR43ezY2ca5MP8w0sPS2V2zu8976LNX9C8t0YHRQM/U2l3Vu5hlm4UQAiEEC+IXGGu612AQlgBXvchMl/YeKe6KY+Q35hMeEG7c/0RACcYoRa/F2FSxCT+TH8tMkQS73exNnAZzb6SkpYTkUK3tQWxQLLXOWtzS7fP9G9oaiAiMMDJVfMmU6u4r3le3j/TwdIL8gryuNQkTcUFx7KzWGropwVD4QnxwPALBvtp9xAbFGh/6OtNjpvdrYVQ4Kox7wZGsw+6v8frWeoqai5hpm2kcmx+n9YwyCZMx6MtI3/17E+tbtZY3j7/wCCtW53Ko4RATwyd6fFEa74yoYAghrhZC7BVCuIUQ8/tYd4EQ4oAQIk8IsXw49zhSxAbFUu2s5rvy75gdNZWg9+9iqtvCvrAoEILSllKjdYHNatOqvQdgJTS0NRAZEEm0Vft25Evgu7uvOKc2h6nRU/tcnxiSaHT4VDEMhS/4mf2MFv9zYuf0+DCeHjOdame1YUV4Q0/L1cXG21Ay3UqZGXNEMBbELzDW+5m1YrvuLtl/tl6JAE6xfsv9050UNBZ4jV+MZ0bawtgDXAF82dsCIYQZeBq4EJgKXCeE6PuTahxgs9oobS5lf91+TqouhA4H07KWcqA+D0eHg0p7pYeFAf1P6etOfWv9gC0MnVpnLZWOSqZE9T0fPCE4wZghoEazKnxFdwd1j1/o6EHkvqwMvcpbt2r1L0XdM6V2VO3ALMweX3pSQlOIDYr16CHVHSkDiLPGUhQYRN27N1HfVq8EYziRUuZIKQ/0s2whkCelzJdStgNvAcuGfncjiy3IRru7HYnkpPIDcNETTEs5k3Z3O1+WfolEGi/sGKtnGq4vNLQ1EBFwRDAGUu2tz2D2xcLQURaGwlf0wHf3+IXOpKhJWEyWHnGM7u7Ycns5kQGRRt1PVGAUZmH2iGHsqtlFdmS2h0tVCMFDpz7EvXPv9bqv+xZnkRY+gaLYDPL1SnIlGKOOJKB7A5eSrmNeEULcIYTYIoTYUl09sNqE0YRuRge73UzPWgazv2/UOXxe+DlwpJvmsVgYumAE+QVhtVgHVIuhu5n0tMXe0F0CwX7BBPsF+3x/xYnNlKgpxFhjPCrAdQLMAWRHZntYGHtr9nL222fzWeFngGcNBmgxiejAaEMw2jrbCi+IFQAAHxZJREFUjID30ZyWdJpHenh37l+STUpYCkVt9eRPuwSAiaW7vK4drwy5YAgh1ggh9nj58dVK8BZR6nWmqJTyOSnlfCnlfJvNdmybHgXYTNq3o3mdZiyXrAAhSA5NJtQ/lK9KvgIwXFK6heFrplSrqxWny2lU2EYHRg/IwsipyyElNIUw/7A+1+mCoQLeioFw49QbWXn5Siwm72ViM2JmsKd2j2FVvLDnBerb6vnNht+wr3ZfD8GAruK9ri9UG0o34HQ5OSf1nAHvLS00jYa2BnZE2AiSgvg1v4ea3jsgjDeGXDCklOdKKad7+fnAx1uUACndHicDgzcpfjQiJfHf/h2Ak6ddDwGhgGYyT4ueRmtnK34mP8Oy8Df7ExEQ4XMtht4WJCJAa5gWZY3qM4bR1N7k8Xhf7b5+3VFwxCWlBEMxEMwmc6/Zd6AFvu0ddgobCyluKmbN4TVcmXUlkYGR3PvFvZS2lPbIrrIF2YzX+KrCVbhdQSyMXzjgvelda78q3cCEyCyEJRD+dQt0DM6UwNHOWHBJbQayhBAThBD+wLVA783xxwNbXmRC7lqeiD+Xqxbc53FKd0slhSRhEkf+99mCfK/21gUjMqB/C+Ozws84860zjarYxrZGSltK+w14w5G0RhW/UAwmRuC7dg+v5byG2WTmztl38uSiJ2lqb8LpcvawMGKsMVQ7q2l1tbK+eD2u5um9WjB9oXetbWhrICN6Mlz2DFTuhrUPHv8fNgYY6bTay4UQJcApwEohxKqu44lCiE8ApJQu4G5gFZADvCOl7Dn6arxQtR9W/QqRsZgLlvzJmBmgMy3miGB0x9ehS3CkSE9vyRwV6N3C6HR38vT2p+mUnTy1/SngSMB7SnT/ghFgDuCq7KtYnLrYp30pFL6QHpZOsF8wG0o38H7e+yydsJTYoFgmR03m4dMfRiB69HeKscZQ66xj+mMrcLqcuJpm9miJ7gt611ro6hM16QJYeAd8+zc4uGZQ/87RyIj2kpJS/gf4j5fjZcBF3R5/Anxy9LpxR0crvHcr+Ido31xMPfVctzD0+IVOjDWGvIY8n56mh4Vhjaa+rR63dHtYLasKV1HYVMhJ8SexsXwjWyu3GgFvXywMgAdOecCndQqFr5hNWjrspwWfAnDjtBuNc0vSlrD+mvXGa1vHZrUhcXPm/IPk1UfS7JhgtLsZCHrX2gp7xZG25ksegsIN8P5P4CffQMjYjZ32x1hwSZ04fPF7qNwDl/0NQr27cRKCE7g6+2rOTz/f47jeT8qXam/dwggPCAc0C8Mt3R4zMjrdnfx919/JjMjkyUVPEmON4antT7Gvdh8JwQkeLakViuFG7yt1WuJpHuNRQXs9H13wpyeGbK7YzLlp5wLH3uNJd0sZVoyfFa78B7Q2wgd3gew1J2fMowRjtJC/HjY+BQtug+zze10mhOC3p/zWqErVsQXZ6JSdfWY76W0OfrdyEwDzHvya9OUr+fZgO+A523v14dXkN+bzo1k/IsgviNtm3MaWyi38t+S/PgW8FYqhRG/jcfP0m31arwsGwPnp53ttie4r6eHpBJr///buPa6qKm3g+O8RVEy8o3kXNO/KRQVn1ExSFA3R1LSpJs1XmZwcrclqnJpuZjlmvXnrQ1r5TqOVlZqRpmWpeRcIREWtVCzSzDRTFFRkvX9sQJSLB+ScfYDn+/nwkcNZ7P1shPOctfZaz/K6elj45o5WT+O7tRD3domP7e60vLk7SP8NVkyAeq0hbFqJDpFTEv3X9F+v+uPIzMpkbsJcvDy9eCRsAo+EteHFHbt4d281UmZYpaNjf45lw1qrPMgt3EKWyeKNpDdoVasV/Vv0B2BEmxEs2rPIoRXeSjnbrU1uJWZoDL61fB1qn1NPqq5XXbre3JXujUr+0je+83jCfcPzV6INibISxudPgd9t4HNLic/hrrSHYTdj4NO/w7lfYPhCqFL4dMKi5Gy6lHem1PlL55n01STe3vM2i/Ysyi3TcTrjNCbzykK6nGqbOb2Tr374iu9Pf89fAv6Se0+jqkdVovyjgCvDAUrZRUQcThZg9TA8xZOwFmElmh2V183Vb6ZbwwJK31WqBEPmg0cVWBEFly/d0HnckSYMu+3+CPYuhz5ToXH+2jmOyt10KXum1InzJxizZgxbjm7h9ma3k56ZzoHfrCosv134jYbeV0oyX1tP6r3979G4euPc3kWOEW1G8Ea/N67avEapsqCqR1XeCHuDvwX9zbknqtkYBr8GP8XDplecey4baMKw05mjsPpRaBoCvR65oUPlDEMdOXuEN3e/yZCVQ0g5k8Lc2+cytftUABKOJwDWLKkON19Z2FSzak08xdrb+9Dvh9j5807uantXvi53JalEjyY9rppJpVRZEdIoJHeih1N1vBP8R8HGmVbiKEf0L98uxsAnf7O6rXdGww3uzFXZozJ1qtZh0Z5FzP5mNkENgnh30Lv0btqbhtUb0sS7Cd/88g1wpVJtjkpSiTpedTiZcZIPD3yIZyVPht4y9IbiUapCGzgTajS07k1eSr9++zJCb3rbJX4RfL8OBs2Cevk3kS+JPs36cPz8cSYETMhX6TOoQRDbj23HGJO7F0Ze9arV46e0n0g+mUy/5v2uunGulCqmarUhci4sHgZfvQADptsdUanQhGGHU4dg7VPQMtSaRltKnu/5fKHPBTUI4tNDn/Ld6e+4cPlCvq55Xa+6bDu6DYNhZNuRpRaTUhXWLX2h2//AtvnQ7g5o0cPuiG6YDkm5WlYWrJwIlTxhyDxw0faOXRp0AeDLH74EyLfwrp5XPQyGlrVa5s5xV0rdoLDnoU4LaxX4hTS7o7lhmjBcLXYhHNkC4S9CrabXb19KWtZuSc0qNVn/w3rgSqXaHDkzpUa2HVmh9ihWyqmqZpf5+e0IrHvW7mhumCYMVzp1yPqluSUMAu916akrSSWCGgSx79Q+IH8Po23dttTzqsfgVoNdGpdS5V6LHtD9QevNYspmu6O5IZowXCV3KKoyDJ7tsqGovPLukXxtD2Nwq8F8NfKr626KpJQqgb7/gjp+Vq2pi+fsjqbE9Ka3q+QMRQ2ZD7UK3WHWqfJuPXntLCnguusrLl26RGpqKhkZFWOzGOX+vLy8aNq0KZUrV7Y7lKJVqW797f/fIPjyeRj4b7sjKhFNGK7w2xFY9xy06uvyoai8OtTrQJVKVcg0mdSoUqPY35+amkqNGjXw9fXV+xzKdsYYTp48SWpqKn5+fnaHc32+PSHkL7AjGjoMKZOzpnRIytmMgZhJ1hCUTUNROap4VKGTTydqVqmZv3CaAzIyMqhXr54mC+UWRIR69eqVrR5vv2egdnNreLoMLujThOFsCYut0uX9noXaza7T2PnGdBzD6I6jS/z9miyUOylzv49VqsPgOXDqIGwse8NSmjCc6cwxWPsktOhpLeBxA6HNQxnXufQWCyqliqlVKATdB1vmwNFEu6MpFk0YzrR6Cly+YL2jKGC71YqiOHsmF8ezzz7LrFmzCn3+448/Jjk52SnnVuqG9H8BqvvAJxPLVBl0W1/FROQuEdkrIlkiUujyYhFJEZHdIpIoInGujLHE9sXA/k/htifK5UYqxTH7y+9sOa8mDOW2qtWBO16Bn3fD1jl2R+Mwu9/27gGGAV870DbUGBNojHH/uhUZv8Pqx+DmztDDyfX3K5jp06fTtm1b+vXrx4ED1v4eCxcuJDg4mICAAIYPH8758+fZunUrn3zyCY899hiBgYEcPHiwwHZK2ab9YOtj40w4edDuaBxia8IwxuwzxhywMwanWPccpB2HyNng4ebzw50kZ/9w33+sAsj9/EaGp+Lj43n//fdJSEhg+fLlxMbGAjBs2DBiY2PZtWsX7du356233qJHjx5ERkby8ssvk5iYSKtWrQpsp5StBs60duj79BFrRqWbKyvrMAzwuYgY4A1jzAK7AyrUD9sh7i34w1+hSdfrty+nHglrwyNhbQArWaTMuOOGj7lp0ybuvPNObrrJ2sY2MtLak3zPnj089dRTnD59mrS0NAYMGFDg9zvaTimXqdnYmmq76lHY9T4E/snuiIrk9B6GiKwTkT0FfAwpxmF6GmO6AAOBh0SkdxHnixKROBGJO3HixA3HXyyZFyFmMtRqBqFPuvbcFURB0yjHjBnDvHnz2L17N88880yh8/IdbaeUS3UdC826w9p/wrlf7Y6mSE5PGMaYfsaYTgV8rCzGMY5m//sLsAIIKaLtAmNMN2NMt/r169/4BRTHtrlwYr+1KVJVb9ee241N7tu6VI7Tu3dvVqxYQXp6OmfPniUmJgaAs2fP0qhRIy5dusSSJUty29eoUYOzZ8/mPi6snVK2qlTJWtR74Qx8/pTd0RTJ7pve1yUi1UWkRs7nQH+sm+Xu5dRh6+ZV+8HQNtzuaNxKztDUjerSpQujRo0iMDCQ4cOHc+uttwIwbdo0unfvTlhYGO3atcttf/fdd/Pyyy8TFBTEwYMHC22nlO0atIcek2DXe3DYkTlA9hBj440WEbkTmAvUB04DicaYASLSGHjTGDNIRFpi9SrAuufyrjHGof0Ou3XrZuLiXDAL1xhYPBx+3AkTd1rjkuXQvn37aN++vd1hKHWVcvN7efE8vP4H6yb4hC3gWdWWMEQkvrDZqHbPklphjGlqjKlqjLnZGDMg++tHjTGDsj8/ZIwJyP7o6GiycKm9y+Hgl3D7U+U2WSilnKzKTdbajJPfWavA3ZDbD0m5vYwzsOaf0CgQQsbbHY1SqixrHQYdhsLXL7vl2gxNGDdqw0vWmouIV6EEFWCVUuoq4TOsYanVj7nd2gxNGDfiWJJV277bAxV6zYVSqhTVbASh/7SGuffF2B3NVTRhlFRWlrXYplpd6Pu03dEopcqTkCi4uROsmepWW7pqwiipxMWQuhP6T7MKiSmlVGnx8LRugJ9JtabruwlNGCVx/hR88Qw0/yMEuPdSflU0b29rgeXRo0cZMWJEkW1fe+21qwoWDho0iNOnTzs1PlWBNf+DtaXztnlwwj1K7mnCKIn10yHjtLWiu6zt+FUBXL58udjf07hxYz766KMi21ybMFavXk3t2rWLfS6lHNbvOWuXvtVT3OIGeFkpPug+ju2CuLcheDw07GR3NPb57B9WLf/S1LAzDJxRZJOUlBTCw8Pp3r07CQkJtGnThnfeeYcOHTowduxYPv/8cyZOnEhwcDAPPfQQJ06c4KabbmLhwoW0a9eOw4cPc88995CZmUl4ePhVx42IiGDPnj1cvnyZJ554grVr1yIijB8/HmMMR48eJTQ0FB8fH9avX4+vry9xcXH4+Pjw6quv8vbbbwMwbtw4Hn74YVJSUhg4cCC9evVi69atNGnShJUrV1KtWrXS/bmp8su7Ptz+LythJK+EjkNtDUd7GMWRlWVNdatW15rFoGxx4MABoqKiSEpKombNmrz++usAeHl5sXnzZu6++26ioqKYO3cu8fHxzJo1i7/+9a8ATJ48mQkTJhAbG0vDhg0LPP6CBQs4fPgwCQkJJCUlce+99zJp0iQaN27M+vXrWb9+/VXt4+PjWbRoETt27GD79u0sXLiQhIQEAL777jseeugh9u7dS+3atVm2bJkTfzKqXOr6gLW3ztonbb8Brj2M4khaCj/ugCHzoVoFH4q4Tk/AmZo1a0bPnj0BuO+++5gzx1oVO2rUKADS0tLYunUrd911V+73XLhwAYAtW7bkvmj/+c9/5oknnsh3/HXr1vHggw/i6Wn9edStW7fIeDZv3sydd95J9erVAWt/jk2bNhEZGYmfnx+BgYEAdO3alZSUlJJetqqoPDxh0ExYNBA2vQp9/2VbKJowHJVxBr54Gpp0g4B77I6mQru2xHnO45wX7KysLGrXrk1iYqJD338tY8x121zbvjBVq16pB+Th4UF6errDx1UqV4se0HmktZ1r4D1Qr5UtYeiQlKO+ngnnTliZvpL+2Oz0ww8/sG3bNgDee+89evXqddXzNWvWxM/Pjw8//BCwXtB37doFQM+ePXn//fcBCi1x3r9/f6Kjo8nMzATg1KlTQP5y6Tl69+7Nxx9/zPnz5zl37hwrVqzIraSrVKkJe95aAb7WvuFwfeVzxK/fw/ZoCLpXV3S7gfbt2/Of//wHf39/Tp06xYQJE/K1WbJkCW+99RYBAQF07NiRlSut7Vdmz57N/PnzCQ4O5vfffy/w+OPGjaN58+b4+/sTEBDAu+++C0BUVBQDBw4kNDT0qvZdunRhzJgxhISE0L17d8aNG0dQUFApX7Wq8Go2gt6Pwbdr4Pt1toRga3lzZyu18uZL7rK2Xv1bPHg3uPHjlVHuUEY672wmpcA9fi9dJvOCVQK9kidM2AoelUv9FG5b3rxM+HYtfPc53PZEhU4WSik34FkVBrwIv34LOxe6/PSaMIqSedGq5VKvtVXbRdnO19dXexeqYmsTDq36woYZLt8DXBNGUWIXwqmDVkb3rGJ3NEopZVWXCH8JLp2Dr6a59NSaMApz7iRs+Dfc0g/a9Lc7GqWUuqJ+W6vaxDfvwM+u63FrwijM+ulwMQ36u9+OsEopxW2Pg1cta5qtiyYvacIoyPFkiF8Ewf8DDdrZHY1SSuV3U13oMxUOb7Sm2rqArQlDRF4Wkf0ikiQiK0SkwHobIhIuIgdE5HsR+YdTgzIG1k6FqjWs/wyl8tiwYQMRERH5vp6YmMjq1atLdMwXX3wx9/OUlBQ6dSp5UcsxY8Zct+quK2gpeBfpNhZ82lh1pjIvOv10dvcwvgA6GWP8gW+BfK/QIuIBzAcGAh2AP4lIB6dFlPE7pJ+2ksVNRdcQUu4pZ4W2KxWVMK4XT96E4Y5KUi5eS8G7iEdla9j81EGIfdPpp7O1lpQx5vM8D7cDBe1gEwJ8b4w5BCAi7wNDgGSnBFWtNoxfDybLKYcvL/6989/sP7W/VI/Zrm47ngjJXwwwr2nTprFkyRKaNWuGj48PXbt2ZcqUKfTp04cePXqwZcsWIiMjGTFiBGPHjuXEiRPUr1+fRYsW0bx5c8aMGUNERETuZkne3t6kpaWxYcMGnn32WXx8fNizZw9du3Zl8eLFiAhr1qzh4YcfxsfHhy5duuSL6eLFizz99NOkp6ezefNmpk6dyr59+zh69CgpKSn4+PjQv39/4uLimDdvHgARERFMmTKFNWvWkJ6eTmBgIB07dmT69OlcvnyZ8ePHF1kS/ciRIwVeH1jFE2fPns3x48d59dVXiYiIYO/evTzwwANcvHiRrKwsli1bRuvWrVm8eDFz5szh4sWLdO/enddffx0PDw+8vb35+9//ztq1a4mIiGD37t188MEHgNXLeuWVV4iJicmt/Juens6IESN47rnnmDNnTqmWgp8zZw7R0dF4enrSoUOH3NIuKlvrMGh1O2ycAQF3O/WNrt09jLzGAp8V8PUmwI95Hqdmf61AIhIlInEiEnfixImSRVKpklUhUrmVuLg4li1bRkJCAsuXL+faVfynT59m48aNPProo0ycOJH777//qvLk15OQkMBrr71GcnIyhw4dYsuWLWRkZDB+/HhiYmLYtGkTP//8c77vq1KlCs8//zyjRo0iMTExt2pufHw8K1euzC0tUpAZM2ZQrVo1EhMTc2tbOVISvajrS0lJYePGjaxatYoHH3yQjIwMoqOjmTx5MomJicTFxdG0aVP27dvH0qVL2bJlC4mJiXh4eOTGcO7cOTp16sSOHTuYOnUq27dv59w5q7T20qVLc69x+vTpxMXFkZSUxMaNG0lKSir1UvAzZszILTUfHR193f/HCkcE+r8AF87C17OceiqnvyqKyDqgoI0HnjTGrMxu8ySQCRRUDa6gsqGFTgkwxiwAFoBVGqTYASuHXK8n4AybN29myJAhue+2Bw8efNXzOS9iANu2bWP58uWAVcb88ccfv+7xQ0JCaNq0KQCBgYGkpKTg7e2Nn58frVu3Bqxy6gsWLHAo3sjIyBJtluRISfSirm/kyJFUqlSJ1q1b07JlS/bv388f//hHpk+fTmpqKsOGDaN169Z8+eWXxMfHExwcDEB6ejoNGljVDDw8PBg+fDgAnp6ehIeHExMTw4gRI1i1ahUzZ1r7TH/wwQcsWLCAzMxMjh07RnJyMv7+/oVeW0lKwfv7+3PvvfcydOhQhg61dwMht3VzRwi6D3YusCbrOKmardMThjGmX1HPi8hoIALoawoubJUKNMvzuClwtPQiVGXF9eqe5bwIFSSnXLmnpydZWVm5x7t48cqNwmtLkefceyhOqfPC4sl7XoCMjIxCv68kJdHzxlhQ+fd77rmH7t27s2rVKgYMGMCbb76JMYbRo0fz0ksv5Tuel5cXHh4euY9HjRrF/PnzqVu3LsHBwdSoUYPDhw8za9YsYmNjqVOnDmPGjCnyuqBkpeBXrVrF119/zSeffMK0adPYu3dv7l4lKo/QJ2H3MvjyORj5jlNOYfcsqXDgCSDSGHO+kGaxQGsR8RORKsDdwCeuilG5j169ehETE0NGRgZpaWmsWrWq0LY9evS4qox5Tgl0X19f4uPjAVi5ciWXLl0q8pw527oePHgQsMqpF6Sw0uc5fH19SUxMJCsrix9//JGdO3fmPle5cuXrxnGtwq4P4MMPPyQrK4uDBw9y6NAh2rZty6FDh2jZsiWTJk0iMjKSpKQk+vbty0cffcQvv/wCWGXcjxw5UuD5+vTpwzfffMPChQtze3JnzpyhevXq1KpVi+PHj/PZZ1dGlEurFHzOzys0NJSZM2dy+vRp0tLSivWzqjBqNISek62tXH/Y7pRT2H0PYx5QA/hCRBJFJBpARBqLyGoAY0wmMBFYC+wDPjDG7LUrYGWf4OBgIiMjCQgIYNiwYXTr1o1atWoV2HbOnDksWrQIf39//vvf/zJ79mwAxo8fz8aNGwkJCWHHjh1F9krAeqe9YMEC7rjjDnr16kWLFi0KbBcaGkpycjKBgYEsXbo03/M9e/bEz8+Pzp07M2XKlKtunkdFReUOuziqsOsDaNu2LbfddhsDBw4kOjoaLy8vli5dSqdOnQgMDGT//v3cf//9dOjQgRdeeIH+/fvj7+9PWFgYx44dK/B8Hh4eRERE8Nlnn+VOKw4ICCAoKIiOHTsyduzY3F0Qc66pNErBX758mfvuu4/OnTsTFBTEI488orOtitJjItRoZE2zdcJiPi1vrhzmDmWk09LS8Pb25vz58/Tu3ZsFCxYUOHNJVRzu8HvpVhLfg9RYGDAdKhf/HlpR5c11IFCVKVFRUSQnJ5ORkcHo0aM1WSh1rcA/WR9OoAlDlSlFTVFVSjmX3fcwVBlTnocwVdmjv4+upQlDOczLy4uTJ0/qH6lyC8YYTp48iZeXl92hVBg6JKUc1rRpU1JTUynxCnqlSpmXl1fuYkvlfJowlMMqV66Mn5+f3WEopWyiQ1JKKaUcoglDKaWUQzRhKKWUcki5XuktIieAgovjuC8f4Fe7g3AxveaKQa+5bGhhjKlf0BPlOmGURSISV9iy/PJKr7li0Gsu+3RISimllEM0YSillHKIJgz349h2buWLXnPFoNdcxuk9DKWUUg7RHoZSSimHaMJQSinlEE0YbkxEpoiIEREfu2NxNhF5WUT2i0iSiKwQkXK5D6eIhIvIARH5XkT+YXc8ziYizURkvYjsE5G9IjLZ7phcRUQ8RCRBRD61O5bSognDTYlIMyAM+MHuWFzkC6CTMcYf+BaYanM8pU5EPID5wECgA/AnEelgb1ROlwk8aoxpD/wBeKgCXHOOycA+u4MoTZow3Nf/Ao8DFWJWgjHmc2NMZvbD7UB5rFkdAnxvjDlkjLkIvA8MsTkmpzLGHDPGfJP9+VmsF9Am9kblfCLSFLgDeNPuWEqTJgw3JCKRwE/GmF12x2KTscBndgfhBE2AH/M8TqUCvHjmEBFfIAjYYW8kLvEa1hu+LLsDKU26H4ZNRGQd0LCAp54E/gn0d21EzlfUNRtjVma3eRJrGGOJK2NzESngaxWiByki3sAy4GFjzBm743EmEYkAfjHGxItIH7vjKU2aMGxijOlX0NdFpDPgB+wSEbCGZr4RkRBjzM8uDLHUFXbNOURkNBAB9DXlc4FQKtAsz+OmwFGbYnEZEamMlSyWGGOW2x2PC/QEIkVkEOAF1BSRxcaY+2yO64bpwj03JyIpQDdjTFmreFksIhIOvArcZowpl3vAiogn1g39vsBPQCxwjzFmr62BOZFY73r+A5wyxjxsdzyult3DmGKMibA7ltKg9zCUu5gH1AC+EJFEEYm2O6DSln1TfyKwFuvm7wflOVlk6wn8Gbg9+/81MfudtyqDtIehlFLKIdrDUEop5RBNGEoppRyiCUMppZRDNGEopZRyiCYMpZRSDtGEoZRSyiGaMJRSSjlEE4ZSLpS9N0RY9ucviMgcu2NSylFaS0op13oGeF5EGmBVbo20OR6lHKYrvZVyMRHZCHgDfbL3iFCqTNAhKaVcKLsacSPggiYLVdZowlDKRUSkEdY+H0OAcyIywOaQlCoWTRhKuYCI3AQsx9rfeh8wDXjW1qCUKia9h6GUUsoh2sNQSinlEE0YSimlHKIJQymllEM0YSillHKIJgyllFIO0YShlFLKIZowlFJKOeT/AWPCyS3CqDqeAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "## EDIT THIS CELL\n",
    "K = 2 # Define the degree of the polynomial we wish to fit\n",
    "\n",
    "Phi = poly_features(X, K) # N x (K+1) feature matrix\n",
    "\n",
    "theta_ml = nonlinear_features_maximum_likelihood(Phi, y) # maximum likelihood estimator\n",
    "\n",
    "# test inputs\n",
    "Xtest = np.linspace(-5,5,100).reshape(-1,1)\n",
    "ytest = f(Xtest) # ground-truth y-values\n",
    "\n",
    "# feature matrix for test inputs\n",
    "Phi_test = poly_features(Xtest, K)\n",
    "\n",
    "y_pred = Phi_test @ theta_ml # predicted y-values\n",
    "\n",
    "# plot\n",
    "plt.figure()\n",
    "plt.plot(X, y, '+')\n",
    "plt.plot(Xtest, y_pred)\n",
    "plt.plot(Xtest, ytest)\n",
    "plt.legend([\"data\", \"prediction\", \"ground truth observations\"])\n",
    "plt.xlabel(\"$x$\")\n",
    "plt.ylabel(\"$y$\");"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Questions:\n",
    "1. Try out different degrees of polynomials. \n",
    "2. Based on visual inspection, what looks like the best fit?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let us now look at a more systematic way to assess the quality of the polynomial that we are trying to fit. For this, we compute the root-mean-squared-error (RMSE) between the $y$-values predicted by our polynomial and the ground-truth $y$-values. The RMSE is then defined as\n",
    "$$\n",
    "\\text{RMSE} = \\sqrt{\\frac{1}{N}\\sum_{n=1}^N(y_n - y_n^\\text{pred})^2}\n",
    "$$\n",
    "Write a function that computes the RMSE."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    "## EDIT THIS FUNCTION\n",
    "def RMSE(y, ypred):\n",
    "    rmse = np.sqrt(np.mean((y-ypred)**2)) ## SOLUTION\n",
    "    return rmse"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now compute the RMSE for different degrees of the polynomial we want to fit."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEGCAYAAABo25JHAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nO3deXhcd33v8fd3ZrRb1uKREy/aHJwEB+LYVkMpWyBAHUoSoCzOhTZAuWmekqbQQhsecnO5ubd9nkJXSgpNIQ1d0gRoAoYaQspSCBCwnNhJ7CwoXmJ5SeRFlm1Zy2i+949zZE+U0TLyHI1m5vN6nnnmLL9z5qszo/nO2b4/c3dERKR8xQodgIiIFJYSgYhImVMiEBEpc0oEIiJlTolARKTMJQodQK6SyaR3dHQUOgwRkaKyZcuWQ+7ekm1e0SWCjo4Ouru7Cx2GiEhRMbM9k83ToSERkTKnRCAiUuaUCEREypwSgYhImVMiEBEpc5EmAjNbb2ZPmVmPmd2UZf5fm9nW8PG0mfVHGY+IiLxYZJePmlkcuA14E9ALbDazje6+Y7yNu380o/3vA2uiikdERLKL8j6CS4Eed98JYGZ3A1cDOyZpfw3wv6MKZvPuI/z46T5iMSNuRixmJGJGPGbELHwO58VjEDMjET8zb0FVgteubCEWs6hCFBEpiCgTwTJgb8Z4L/CKbA3NrB3oBL4/yfzrgOsA2traZhXMw3uO8tnv98xq2XH/9IFf4fUXLD6rdYiIzDdRJoJsP50n6wVnA/A1dx/LNtPdbwduB+jq6ppVTzq/+7rz+N3XnUc67Yy5M5Z20u6k0h5MC6en04TPwbRU2hlJpbnqcw+yedcRJQIRKTlRJoJeoDVjfDmwf5K2G4APRxjLabGYEcOoiOe23EXLGujeczSaoERECijKq4Y2AyvNrNPMKgm+7DdObGRmFwBNwM8ijOWsdbU3sW1vPyOpdKFDERHJq8gSgbungBuA+4EngK+4+3Yzu9XMrspoeg1wt8/zzpPXtTcxnEqzff+xQociIpJXkVYfdfdNwKYJ026ZMP6pKGPIl672JgC27DnKmramAkcjIpI/urN4hhYvrKa1uYYtOk8gIiVGiSAHXe3NdO85yjw/iiUikhMlghysbW+i7/gwvUdPFToUEZG8USLIwfh5gu49RwociYhI/igR5OD8c+qpr0rQvVvnCUSkdCgR5CAeMy5pa9QJYxEpKUoEOepqb+ap544zMDRa6FBERPJCiSBHXR1NuMMjz6rrBBEpDUoEOVrd2kjM0OEhESkZSgQ5WlCV4KVLFrJFVw6JSIlQIpiFrvYmHnm2n9SYCtCJSPFTIpiFte1NDI6M8eTB44UORUTkrCkRzEJXRzOg8wQiUhqUCGZhWWMNSxqq1VGNiJQEJYJZWtvexMNKBCJSApQIZqmrvYl9/ac4cEwF6ESkuCkRzFJXe3CeQHWHRKTYKRHM0oVL6qmpiOuEsYgUPSWCWaqIx7ikVQXoRKT4KRGcha6OJnYcGODkcKrQoYiIzJoSwVlY297EWNrZ1qsCdCJSvJQIzsLatibMYItOGItIEYs0EZjZejN7ysx6zOymSdq828x2mNl2M7srynjyraGmgvMX1+vGMhEpaomoVmxmceA24E1AL7DZzDa6+46MNiuBTwCvcvejZrY4qniisq6jiW9u20867cRiVuhwRERyFuUewaVAj7vvdPcR4G7g6glt/idwm7sfBXD35yOMJxLr2po4PpTil8+fKHQoIiKzEmUiWAbszRjvDadlOh8438x+YmYPmdn6bCsys+vMrNvMuvv6+iIKd3a6OpoA6Fb/BCJSpKJMBNmOk/iE8QSwErgMuAb4opk1vmgh99vdvcvdu1paWvIe6Nloa64luaBKJ4xFpGhFmQh6gdaM8eXA/ixtvuHuo+6+C3iKIDEUDTNjXXsjW55VIhCR4hRlItgMrDSzTjOrBDYAGye0+TrwegAzSxIcKtoZYUyR6GpvZs/hQfqODxc6FBGRnEWWCNw9BdwA3A88AXzF3beb2a1mdlXY7H7gsJntAH4AfNzdD0cVU1TWhecJ1I+xiBSjyC4fBXD3TcCmCdNuyRh24A/DR9G6aOlCKhMxtuw5yvqXLSl0OCIiOdGdxXlQlYizenmDbiwTkaKkRJAn69qbeXzfMYZGxwodiohITpQI8mRdexOjY85j+44VOhQRkZwoEeTJuvbwxjLdTyAiRUaJIE+a6ypZ0VKnK4dEpOgoEeTRurYmtuw5SnAxlIhIcVAiyKOujiaODo6y89DJQociIjJjSgR5tK69GVBHNSJSXJQI8mhFso7G2gp1aC8iRUWJII9iMWNdW5NKUotIUVEiyLN1HU0803eSoydHCh2KiMiMKBHk2bq24H6Ch1WWWkSKhBJBnq1ubaQibqo7JCJFQ4kgz6or4ly0tEFXDolI0VAiiEBXexPbevsZSaULHYqIyLSUCCKwrr2J4VSa7ftVgE5E5j8lggic6bFMh4dEZP5TIojA4vpq2pprlQhEpCgoEURkXXsT3SpAJyJFQIkgIuvam+g7PszeI6cKHYqIyJSUCCLSNX6e4FmVmxCR+U2JICIrF9dTX5VQj2UiMu9FmgjMbL2ZPWVmPWZ2U5b57zezPjPbGj4+FGU8cykeM9a0N+mEsYjMe5ElAjOLA7cBVwCrgGvMbFWWpve4+yXh44tRxVMIXe1NPPXccY6dGi10KCIik4pyj+BSoMfdd7r7CHA3cHWErzfvrGtvwh227u0vdCgiIpOKMhEsA/ZmjPeG0yb6TTN71My+Zmat2VZkZteZWbeZdff19UURayQuaW0kHjO27NYJYxGZv6JMBJZl2sSL6r8JdLj7xcB/AV/OtiJ3v93du9y9q6WlJc9hRqeuKsFLl9SrEqmIzGtRJoJeIPMX/nJgf2YDdz/s7sPh6D8C6yKMpyDWtTWxdW8/qTEVoBOR+SnKRLAZWGlmnWZWCWwANmY2MLMlGaNXAU9EGE9BrOtoZnBkjCcPHi90KCIiWSWiWrG7p8zsBuB+IA7c4e7bzexWoNvdNwI3mtlVQAo4Arw/qngKpas9uLFsw+0PUV0RJxEz4uHjBcNxIx6LETdIxGIZ04zfXLucK1cvLfBfIiKlyoqtFk5XV5d3d3cXOoyc3PHgLnYfPkkq7aTTTirtjJ1+TpMayxx3Uuk0Y+Fwz/MneMniBdz7e68q9J8hIkXMzLa4e1e2eZHtEcgZH3x156yX/cS9j/Gdxw/kMRoRkRdSiYl5bkWyjqODo/QPjhQ6FBEpUUoE81xnsg6AXYdOFjgSESlVSgTzXGeLEoGIREuJYJ5rbaolHjMlAhGJjBLBPFeZiLG8qYadSgQiEhElgiLQmaxjV58SgYhEQ4mgCHQm69h9+KT6PxaRSCgRFIEVyToGR8Z4/vjw9I1FRHKkRFAEOpMLANipw0MiEgElgiKgS0hFJEpKBEVgycJqqhIxdh06UehQRKQEKREUgVjM6FhUpz0CEYmEEkGR6EwqEYhINJQIikRnSx3PHhlUT2cikndKBEWiM1nH6Jizr/9UoUMRkRKjRFAkVoRVSFVqQkTyTYmgSHSMl6PWvQQikmdKBEViUV0l9dUJdh9WIhCR/FIiKBJmxgpdOSQiEZgyEZjZGzKGOyfMe0dUQUl2nck6lZkQkbybbo/gLzKG/2PCvJunW7mZrTezp8ysx8xumqLdO83MzaxrunWWs87kAvYfO8XQ6FihQxGREjJdIrBJhrONv3CmWRy4DbgCWAVcY2arsrSrB24Efj5ttGWus6UOd9hzeLDQoYhICZkuEfgkw9nGJ7oU6HH3ne4+AtwNXJ2l3f8FPg0MTbO+ste5aLz4nGoOiUj+JKaZv8LMNhL8+h8fJhzvnHwxAJYBezPGe4FXZDYwszVAq7t/y8w+NtmKzOw64DqAtra2aV62dHUkawHYdUh7BCKSP9Mlgsxf8H8xYd7E8YmyHTo6vRdhZjHgr4H3T7Me3P124HaArq6usu2mq766gpb6Ku0RiEheTZkI3P2/M8fNrAJ4GbDP3Z+fZt29QGvG+HJgf8Z4fbiuH5oZwLnARjO7yt27ZxZ++VHxORHJt+kuH/2CmV0UDjcA24B/Bh4xs2umWfdmYKWZdZpZJbABGD+0hLsfc/eku3e4ewfwEKAkMA3dSyAi+TbdyeLXuPv2cPgDwNPu/nJgHfDHUy3o7ingBuB+4AngK+6+3cxuNbOrzjLustWZrOPQiRGOnRotdCgiUiKmO0cwkjH8JuCrAO5+MDycMyV33wRsmjDtlknaXjbtCuV0zaHdh06yurWxwNGISCmYbo+g38zeGl7d8yrgOwBmlgBqog5OXmy8CqlqDolIvky3R/C7wGcJTuR+xN0PhtMvB/4zysAku7ZFtZihUhMikjfTXTX0NLA+y/T7CY79yxyrSsRZ3lSjE8YikjdTJgIz++xU8939xvyGIzPRmVygRCAieTPdoaHrgceBrxDcAzD9GWKJ3IpkHV/bcxR3ZyYn7UVEpjJdIlgCvAt4D5AC7gH+w92PRh2YTK5jUS0nhlMcOjFCS31VocMRkSI35VVD7n7Y3b/g7q8nKAXRCGw3s9+ai+Aku86WBQA6PCQieTGjHsrMbC3wEeB9wLeBLVEGJVMbv4RUNYdEJB+mO1n8f4C3EtwZfDfwifCOYSmgpY01VMZj7NQegYjkwXTnCP4XsBNYHT7+LDw5aYC7+8XRhifZxGNG+6JaduleAhHJg+kSwXR9DkiBqAqpiOTLdDeU7ck2PeyGcgOQdb5ErzNZxw+f7mMs7cRjuoRURGZvujLUC83sE2b2OTN7swV+n+Bw0bvnJkTJpjNZx0gqzf7+U4UORUSK3HSHhv4FOAr8DPgQ8HGgErja3bdGHJtMofP0lUMnaW2uLXA0IlLMpu2zOOx/ADP7InAIaHP345FHJlPqbDmTCF57fkuBoxGRYjbdfQSnez9x9zFgl5LA/NCyoIoFVQmdMBaRszbdHsFqMxsIhw2oCcfHLx9dGGl0Mikz05VDIpIX0101FJ+rQCR3Hck6tu3tL3QYIlLkZlRiQuanzmQdvUcHGU6NFToUESliSgRFbEWyjrTD3iODhQ5FRIqYEkERG7+EVN1WisjZUCIoYh0Z9xKIiMxWpInAzNab2VNm1mNmN2WZf72ZPWZmW83sQTNbFWU8paahpoJFdZXsPqxEICKzF1kiCOsR3QZcAawCrsnyRX+Xu7/c3S8BPg38VVTxlKrOZJ0ODYnIWYlyj+BSoMfdd7r7CEF/BldnNnD3gYzROsAjjKck6V4CETlbUSaCZcDejPHecNoLmNmHzewZgj2CG7OtyMyuM7NuM+vu6+uLJNhi1dlSx/PHhzkxrP6CRGR2okwE2Wojv+gXv7vf5u7nAX8C3JxtRe5+u7t3uXtXS4vq6mQa77Zyt/YKRGSWokwEvUBrxvhyYP8U7e8G3hZhPCWpMxl0ZK9uK0VktqJMBJuBlWbWaWaVBB3ZbMxsYGYrM0Z/A/hlhPGUpPZFQQlq7RGIyGxNV3Ru1tw9ZWY3APcDceAOd99uZrcC3e6+EbjBzN5IUOX0KHBtVPGUquqKOMsaa3TCWERmLbJEAODum4BNE6bdkjH8B1G+frnoTNbp0JCIzJruLC4Bnck6dvWdwF1X34pI7pQISkBnso6BoRRHTo4UOhQRKUJKBCVgvNtKlZoQkdlQIigBnYtUhVREZk+JoAQsb6ohETNdOSQis6JEUAIS8Rhti2qVCERkVpQISsQKFZ8TkVlSIigR41VI02ldQioiuVEiKBGdyQUMp9IcHBgqdCgiUmSUCEpERzKoOaTDQyKSKyWCErFCVUhFZJaUCErEOQurqKmIs0v3EohIjpQISoSZhSeMTxQ6FBEpMkoEJaSzpY7dhwcLHYaIFBklghKyIlnHs0cGGR1LFzoUESkiSgQlpGNRHWNpZ+8R7RWIyMwpEZSQ8SqkuoRURHKhRFBCViSVCEQkd0oEJaSxtpKm2grdSyAiOVEiKDGdyTp2KxGISA6UCEpMZ3KBDg2JSE6UCEpMZ7KWA8eGGBxJFToUESkSkSYCM1tvZk+ZWY+Z3ZRl/h+a2Q4ze9TMvmdm7VHGUw46w5pDuw/pElIRmZnIEoGZxYHbgCuAVcA1ZrZqQrNHgC53vxj4GvDpqOIpF526ckhEchTlHsGlQI+773T3EeBu4OrMBu7+A3cf/+n6ELA8wnjKwply1Ko5JCIzE2UiWAbszRjvDadN5neAb2ebYWbXmVm3mXX39fXlMcTSU1uZYElDNbt0aEhEZijKRGBZpmXtR9HM3gd0AZ/JNt/db3f3LnfvamlpyWOIpUlVSEUkF1Emgl6gNWN8ObB/YiMzeyPwSeAqdx+OMJ6y0aGO7EUkB1Emgs3ASjPrNLNKYAOwMbOBma0B/oEgCTwfYSxlZUWyjqODoxw9OVLoUESkCESWCNw9BdwA3A88AXzF3beb2a1mdlXY7DPAAuCrZrbVzDZOsjrJwekrhw5rr0BEppeIcuXuvgnYNGHaLRnDb4zy9cvVeCLYfegka9uaChyNiMx3urO4BLU21xKPmc4TiMiMKBGUoIp4jLbmWlUhFZEZUSIoUR2LatnVp0QgItNTIihR41VI3bPeuiEicpoSQYnqbKnj1OgYzw3o1gwRmZoSQYlSt5UiMlNKBCVKVUhFZKaUCErUuQurqa6IqeaQiExLiaBExWJGxyLVHBKR6UV6Z7EUVmeyjgd7DnHDXQ9TV5mgtir+wufKOAuqEtRWJairjFNbmaCu6sxzTUUcs2xFZEWklCgRlLC3rVlG79FT7DgwwODwGCdHUpwcTpGe4RWllfEY55+7gFVLFnLR0gZWLV3IhefWU19dEW3gIjKnrNiuM+/q6vLu7u5Ch1G03J3hVJqTwykGR8aTwxiDYZI4PTwyxuETwzx58Djb9w9wJKOSaceiWlYtDZPDkoVctHQhLfVV2nsQmcfMbIu7d2Wbpz2CMmNmVFfEqa6Is2iGy7g7zw0Ms+PAMbbvG2DHgQEe3zfApscOnm6TXFDJqozEsGrpQjoX1RGLKTmIzHdKBDItM+PchmrObajmDReec3r6wNAoTx44zvb9x9ixf4Dt+wf40jM7GR0L9jLrqxJc3NrAxcsbWb28kUtaGzm3obpQf4aITEKJQGZtYXUFl3Y2c2ln8+lpI6k0Pc+f4PH9x3i0t59He4/xxR+fSQ6L66tY3RokhdXLG3n58gYaanTOQaSQlAgkryoTMVaFh4be3RX0VDo0OsYTBwbYtrefbb3H2La3nwd2PHd6mRXJOla3NrJ6eQMXtzayaslCqivihfoTRMqOEoFErroizpq2JtZkdJJz7NQoj/UeY1tvP1v39vOTnkPc98g+ABIxo746gZlhBIemzAiHIZYxHSAWA+NMm6pEnPrqBAtrKqivTgTD1RXUV1ewsCZBfXXF6WkLqxOnp+tyWSlXSgRSEA01Fbx6ZZJXr0yennbw2BBb9/bzaG8/x4dSOI47OMEJa3fCcSedMYxD2j1sF+yBHB9K8fzxIZ7pSzFwapTjQylS01w3G48ZtZVx4rEg0cTMMDNidiYBxcKklO05ZhCPxUjEjHjMqIgHz4lYjETcTk9PxDPaxGLEw3nj649ZcENg5nozY4lZsOzE+cFrGbHYmfUl4kY8FiOeMT8+4dHWXMvSxppI32+Z35QIZN44t6Ga9Q3nsv5l5+Z93e7OqTBBHB8a5dip4Pn4UIqB8edTowyOjOEeJprTCcdJp8+Mp8OkNPF5LO2MuTOWdkbH0oylndSYM5hKhdOCeal0mlQ4L3M8nT6zvomvMz4tChVx48Y3rOT6y86jIq5iA+VIiUDKgplRW5mgtjLBOQuL98qldNpflCjGxhNF2kmlg+SS+XjRNHfG0mlSY8G8u37xLH/5wNN8+/GDfOZdF3PR0oZC/5kyx5QIRIpILGbEyO95jFe9JMmVFx/g5q8/ztWf+wm/d9l53PCGlVQmtHdQLiJ9p81svZk9ZWY9ZnZTlvmvNbOHzSxlZu+MMhYRmdz6ly3hgY++jitXL+Wz3+/hyr97kEd7+wsdlsyRyBKBmcWB24ArgFXANWa2akKzZ4H3A3dFFYeIzExTXSV//Z5L+NK1XfSfGuHtf/9T/vw7TzI0Olbo0CRiUe4RXAr0uPtOdx8B7gauzmzg7rvd/VEgHWEcIpKDy196Dt/96Ot4x5plfP6Hz/Abn/0xW/YcLXRYEqEoE8EyYG/GeG84LWdmdp2ZdZtZd19fX16CE5HJNdRU8Jl3rebOD/wKp0bGeOcXfsr/+9YOTo1o76AURZkIsp3RmtUFcO5+u7t3uXtXS0vLWYYlIjN12QWLuf+jr+WaS9v44oO7uOJvf8Qvdh0pdFiSZ1Emgl6gNWN8ObA/wtcTkQjUV1fwZ29/OXd96BWk0s57bv8Zn9q4nZPDqUKHJnkSZSLYDKw0s04zqwQ2ABsjfD0RidCvvSTJ/R95Lde+soM7f7qb9X/7Ix7Y8Rw7+07w3MAQJ4ZTpKO6600iFWnHNGb2FuBvgDhwh7v/qZndCnS7+0Yz+xXgPqAJGAIOuvtFU61THdOIFN4vdh3hj7+2jd2HB180r64yTm1VggVVZ7o+DYYTLDjdFWowXF9dEdR8qglrP9WcqQOl+xjya6qOadRDmYjMyqmRMX628xADp1KcGE4xOJLixPBY2PvdmeGTw6nTPeGdGE4xOBz0gDed6orYi5LDwpozhQIr4+FpyNNFCMPRsABhMBxMzywmaAYdi+p4w4WLy6rKrXooE5G8q6mMv6Cjolyk087g6BjHh0YZODVe7+nM8MCpUQaGzhQMHBgapX9whGePDIbzRkmFtZlmq746wW+8fAnvWLucrvamsu5NT4lAROZcLGYsCA8fLclDaaPxIxvj1WrHp41XpAXOVLN12LLnKPc+0svGbfu5e/NeljfV8I41y3j72uV0JuvOPqAio0NDIlK2Tg6nuH/7Qe57ZB8P9hzCHda0NfKOtcu58uIlNNZWFjrEvNE5AhGRaRw8NsQ3tu7j3of38dRzx6mIG2+4cDFvX7Oc11/YQlWiuM8nKBGIiMyQu7PjwAD3PryPb2zdz6ETwzTWVvDWi4PzCWtaG4uyJzslAhGRWUiNpflxzyHue3gf928/yHAqzTkLq1iRXED7olraFtXS3lx3enhhdcVZv+b4SfE9hwd59sggzx4eZM+Rk+w9coqP//oFvG3NrCr16KohEZHZSMRjvP6Cxbz+gsUcHxrl248d5KfPHGLPkUEe2PEch0+OvKB9Y20F7c21tC2qC59raW+upX1RHYvrq4jFjLG0c+DYKZ4Nv+j3hF/2wZf/SQaGXnjHdnJBFe2LanlFZ3NknSppj0BEZJaOD43y7JFB9oa/4Pdk/ILf3z/EWMad1lWJGMkFVTx/fIjRsTPTK+LG8qZaWpvHk0Y4vKiWtuZaaivz83tdewQiIhGor67goqUNWbv3HB1Ls7//VEaCOEnf8WHObaihPdxTaG2uZWljDfEC38OgRCAiEoGKeIz2RXW0L5r/9yWomIeISJlTIhARKXNKBCIiZU6JQESkzCkRiIiUOSUCEZEyp0QgIlLmlAhERMpc0ZWYMLM+YM8sF08Ch/IYTr4ortwortzN19gUV27OJq52d2/JNqPoEsHZMLPuyWptFJLiyo3iyt18jU1x5SaquHRoSESkzCkRiIiUuXJLBLcXOoBJKK7cKK7czdfYFFduIomrrM4RiIjIi5XbHoGIiEygRCAiUuZKMhGY2Xoze8rMeszspizzq8zsnnD+z82sYw5iajWzH5jZE2a23cz+IEuby8zsmJltDR+3RB1X+Lq7zeyx8DVf1A+oBT4bbq9HzWztHMR0QcZ22GpmA2b2kQlt5mx7mdkdZva8mT2eMa3ZzB4ws1+Gz02TLHtt2OaXZnZtxDF9xsyeDN+n+8yscZJlp3zPI4rtU2a2L+P9essky075/xtBXPdkxLTbzLZOsmwk22yy74Y5/Xy5e0k9gDjwDLACqAS2AasmtPk94Avh8AbgnjmIawmwNhyuB57OEtdlwLcKsM12A8kp5r8F+DZgwK8CPy/Ae3qQ4IaYgmwv4LXAWuDxjGmfBm4Kh28C/jzLcs3AzvC5KRxuijCmNwOJcPjPs8U0k/c8otg+BXxsBu/1lP+/+Y5rwvy/BG6Zy2022XfDXH6+SnGP4FKgx913uvsIcDdw9YQ2VwNfDoe/BlxuZpF2GuruB9z94XD4OPAEsCzK18yjq4F/9sBDQKOZLZnD178ceMbdZ3tH+Vlz9x8BRyZMzvwcfRl4W5ZFfx14wN2PuPtR4AFgfVQxuft33T0Vjj4ELM/Ha+Vqku01EzP5/40krvA74N3Av+fr9WYY02TfDXP2+SrFRLAM2Jsx3suLv3BPtwn/aY4Bi+YkOiA8FLUG+HmW2a80s21m9m0zu2iOQnLgu2a2xcyuyzJ/Jts0ShuY/J+zENtr3DnufgCCf2ZgcZY2hdx2HyTYk8tmuvc8KjeEh63umORQRyG312uA59z9l5PMj3ybTfhumLPPVykmgmy/7CdeIzuTNpEwswXAfwAfcfeBCbMfJjj8sRr4O+DrcxET8Cp3XwtcAXzYzF47YX4ht1clcBXw1SyzC7W9clGQbWdmnwRSwL9N0mS69zwKnwfOAy4BDhAchpmoYJ814Bqm3huIdJtN890w6WJZpuW8vUoxEfQCrRnjy4H9k7UxswTQwOx2Y3NiZhUEb/S/ufu9E+e7+4C7nwiHNwEVZpaMOi533x8+Pw/cR7B7nmkm2zQqVwAPu/tzE2cUantleG78EFn4/HyWNnO+7cIThm8F3uvhgeSJZvCe5527P+fuY+6eBv5xktcsyGct/B54B3DPZG2i3GaTfDfM2eerFBPBZmClmXWGvyY3ABsntNkIjJ9dfyfw/cn+YfIlPP74JeAJd/+rSdqcO36uwswuJXh/DkccV52Z1Y8PE5xsfHxCs43Ab1vgV4Fj47usc2DSX2mF2F4TZH6OrgW+kaXN/cCbzawpPBTy5nBaJMxsPfAnwFXuPjhJm5m851HElnle6e2TvOZM/n+j8EbgSXfvzTYzym02xZnlN4sAAAUKSURBVHfD3H2+8n0GfD48CK5yeZrg6oNPhtNuJfjnAKgmONTQA/wCWDEHMb2aYJftUWBr+HgLcD1wfdjmBmA7wZUSDwG/NgdxrQhfb1v42uPbKzMuA24Lt+djQNccvY+1BF/sDRnTCrK9CJLRAWCU4FfY7xCcV/oe8MvwuTls2wV8MWPZD4aftR7gAxHH1ENwzHj8MzZ+ddxSYNNU7/kcbK9/CT8/jxJ8yS2ZGFs4/qL/3yjjCqffOf65ymg7J9tsiu+GOft8qcSEiEiZK8VDQyIikgMlAhGRMqdEICJS5pQIRETKnBKBiEiZUyKQeSGsTPmxQscxFTO7MKw8+YiZnXeW67rTzN6Zr9hyfO1NNklV0ow2u+f45jwpICUCKSlmFo9w9W8DvuHua9z9mQhfJ1Lu/hZ37y90HDJ/KBFIwZjZJ8O68/8FXJAx/Twz+05Y3OvHZnZhxvSHzGyzmd1qZifC6ZeF9dzvIrhhCTN7n5n9IvwF/w/jCcLM3mxmPzOzh83sq2F9l4lxXRK+znhN/yYLaud/BPiQmf0gyzInzOwvw/V+z8xaJlvXhOUuN7P7MsbfZGb3ZqzzTy0oqveQmZ0TTm8PX+PR8LktnH6nmX0+3BY7zex1FhR3e8LM7sx4jdO/9s3s6+F23m5zW3xO5pN831Gohx4zeQDrCL60a4GFBHdFfiyc9z1gZTj8CoISIADfAq4Jh68HToTDlwEngc5w/KXAN4GKcPzvgd8GksCPgLpw+p+QpfY8wR2erwuHbwX+Jhz+FJPU0ye4M/S94fAtwOemWdedBOVNDHgSaAmn3wVcmbHO8eFPAzeHw98Erg2HPwh8PWOdd4frvBoYAF5O8INvC3BJ2G43YV19ztytWkNQMmHRxDZ6lP5DewRSKK8B7nP3QQ8qLW6E0xUYfw34qgU9Rf0DQccdAK/kTBXSuyas7xfuviscvpwg0WwO13E5QYmAXyXo8OMn4fRrgfbMlZhZA9Do7v8dTvoyQWcm00lzpmDZvwKvnsm63N0JSi+8Lzxu/0rOlI4eIUh+EHyRd4TDr8z4+/+FoETBuG+G63yMoKTyYx4UeduesXymG81svERHK7ByBn+rlJhEoQOQspatvkkM6Hf3S3Jc18mMYQO+7O6fyGxgZlcSdOJxTY7rno1carf8E8Gv/CHgq36mY5nR8EsdYIzJ/18zX2s4fE5nDI+Pv2B5M7uMoNjaK9190Mx+SFCHS8qM9gikUH4EvN3MasKqjldCUFoa2GVm74LT/SWvDpd5CPjNcHjDFOv+HvBOM1scrqPZzNrD5V9lZi8Jp9ea2fmZC7r7MeComb0mnPRbwH8zvRjBoR6A/wE8ONN1eVDeeD9wM8Hhnen8lDN//3uBB2ewTDYNwNEwCVxIsMckZUh7BFIQ7v6wmd1DUGlxD/DjjNnvBT5vZjcDFQTHvbcRnKz9VzP7I+A/CXqWy7buHeGy3zWzGEGlyQ+7+0Nm9n7g382sKmx+M0Gly0zXAl8ws1qCPmA/MIM/6SRwkZltCeN6T47r+jeC8wQ7ZvBaNwJ3mNnHgb4ZxpfNd4DrzexR4CmCRCllSNVHpWiEX6an3N3NbAPBieO89Wd7NszshLu/6AqkHJb/HPCIu38pj2GJzIj2CKSYrAM+Z2YG9BNcMVP0wr2Ik8AfFToWKU/aIxARKXM6WSwiUuaUCEREypwSgYhImVMiEBEpc0oEIiJl7v8DGEluhlAJaqsAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "## EDIT THIS CELL\n",
    "K_max = 20\n",
    "rmse_train = np.zeros((K_max+1,))\n",
    "\n",
    "for k in range(K_max+1):\n",
    "    \n",
    "     \n",
    "    # feature matrix\n",
    "    Phi = poly_features(X, k)\n",
    "    \n",
    "    # maximum likelihood estimate\n",
    "    theta_ml = nonlinear_features_maximum_likelihood(Phi, y)\n",
    "    \n",
    "    # predict y-values of training set\n",
    "    ypred_train = Phi @ theta_ml\n",
    "    \n",
    "    # RMSE on training set\n",
    "    rmse_train[k] = RMSE(y, ypred_train)\n",
    "    \n",
    "\n",
    "plt.figure()\n",
    "plt.plot(rmse_train)\n",
    "plt.xlabel(\"degree of polynomial\")\n",
    "plt.ylabel(\"RMSE\");"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Question: \n",
    "1. What do you observe?\n",
    "2. What is the best polynomial fit according to this plot?\n",
    "3. Write some code that plots the function that uses the best polynomial degree (use the test set for this plot). What do you observe now?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEGCAYAAABo25JHAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nO3dd3xUVfr48c9J6IQaQigJhA4SIMEACgJBQFERLCiouLh8WdvqIqu4rGXXXcuX309/YFsLiqIrigqCKDZAEbAAoSMlFIOEmgCBEFqSOb8/TiYGSEJIZubcmfu8Xy9eFyYz9z4Dwzz3tOcorTVCCCHcK8x2AEIIIeySRCCEEC4niUAIIVxOEoEQQricJAIhhHC5SrYDKI8GDRrouLg422EIIURQWblyZabWOursx4MyEcTFxZGSkmI7DCGECCpKqZ3FPS5dQ0II4XKSCIQQwuUkEQghhMs5ZoxAKRUOpAC7tdaDbccjRCDk5uaSnp7OyZMnbYciQki1atWIiYmhcuXKZXq+YxIBMBbYBNS2HYgQgZKenk6tWrWIi4tDKWU7HBECtNYcPHiQ9PR0WrRoUabXOKJrSCkVA1wDvGk7FiEC6eTJk0RGRkoSED6jlCIyMvKCWpmOSATA88DDgKekJyil7lRKpSilUjIyMgIXmRB+JklA+NqFfqasJwKl1GDggNZ6ZWnP01pP0Vonaa2ToqLOWQ9RNqlfw5JJ5XutEEKEKOuJAOgFDFFKpQEzgMuVUu/55Uo7FsHi50D2YBCiWE888QTPPfdciT+fM2cOGzduDGBEIhCsJwKt9d+11jFa6zhgBPCt1nqkXy5WJxZyc+DEYb+cXohAmTw/1cp1JRGEJuuJIKDqxJjjkV124xCigl5YuNVn53r66adp164dAwYMYMuWLQC88cYbdOvWjS5dunDjjTdy/PhxfvzxR+bOncv48eNJSEhg+/btxT5PBB9HJQKt9SK/riGoG2uOWZIIhABYuXIlM2bMYPXq1XzyySesWLECgBtuuIEVK1awdu1aOnTowNSpU+nZsydDhgzh2WefZc2aNbRq1arY54ng46R1BP5XpyARHEm3G4cQ5TB5fuoZLYG4CfMAGNu/DeMGti3XOZcsWcL1119PjRo1ABgyZAgAGzZs4LHHHiMrK4tjx45x5ZVXFvv6sj5POJu7EkGNSKhUXbqGRFAaN7Bt4Rd+3IR5pE28xifnLW6q4R133MGcOXPo0qUL06ZNY9GiRcW+tqzPE87mqK4hv1PKjBNIIhACgD59+jB79mxOnDhBdnY2n332GQDZ2dk0btyY3Nxcpk+fXvj8WrVqkZ2dXfjnkp4ngou7EgGYcQIZIxBBbmz/Nj45T9euXRk+fDgJCQnceOON9O7dG4Ann3ySHj16MHDgQNq3b1/4/BEjRvDss8+SmJjI9u3bS3yeCC5KB+Gc+qSkJF3ujWnm3g9bvoLxvpt1IUR5bdq0iQ4dOtgOQ4Sg4j5bSqmVWuuks5/rvhZBnWaQcwBypdqjEEKAKxNBwVqCo7vtxiGEEA7hvkRQuJbgN7txCCGEQ7gvERSuLpa1BEIIAW5MBLWbAkqmkAohRAH3JYLwylCrsUwhFUKIAu5LBGDGCaRFIITfzJ07l4kTJ1q7fnJyMt4p5ldffTVZWVmkpaURHx9f5nMUfX5KSgp/+ctfgPOX6i6vks6bkZFBjx49SExMZMmSJYXvJysri1deecUn13ZXiQmvOrGwu5zrEIQQ5zVkyJDCukW2ffHFFwBkZWWV+xxJSUkkJZ0z/T4gFi5cSPv27XnnnXcAChf9paWl8corr3DvvfdW+BrubBHUiYEju8FT4s6YQrhCWloa7du3Z8yYMcTHx3PbbbexYMECevXqRZs2bVi+fDkAy5cvp2fPniQmJtKzZ8/CctWTJk1i9OjRAKxfv574+HiOHz/OtGnTuO+++wBTj+iee+6hX79+tGzZku+//57Ro0fToUMH7rjjjsJYIiIiCn8/c+bMwp+V9fUliYuLIzMz84zHduzYQWJiIitWrCA/P5/x48fTrVs3OnfuzOuvv37OORYtWsTgwb8XRt64cSPJycm0bNmSF198sfDxSZMmER8fT3x8PM8///x5Hy+uBHhRa9as4eGHH+aLL74gISGBEydOFL6fCRMmsH37dhISEhg/fvx5/x5K484WQd1Y8OTCsf1Qu7HtaIQwvpwA+9b79pyNOsFVpXfRbNu2jY8//pgpU6bQrVs33n//fZYuXcrcuXN55plnmDNnDu3bt2fx4sVUqlSJBQsW8MgjjzBr1iweeOABkpOTmT17Nk8//TSvv/56YSXTog4fPsy3337L3Llzufbaa/nhhx9488036datG2vWrCEhIaHUGCv6+qK2bNnCiBEjePvtt0lISGDKlCnUqVOHFStWcOrUKXr16sUVV1xR6r6/mzdv5rvvviM7O5t27dpxzz33sG7dOt5++22WLVuG1poePXrQt29fPB5PiY97S4Dn5eXRtWtXLr744jOuk5CQwL///W9SUlJ4+eWXz/jZxIkT2bBhA2vWrCnzey+JOxNB0XLUkgiEy7Vo0YJOnToB0LFjR/r3749Sik6dOpGWlgbAkSNHGDVqFFu3bkUpRW5uLgBhYWFMmzaNzp07c9ddd9GrV69ir3HttdcWnjM6OvqM66WlpZ33i7yir/fKyMhg6NChzJo1i44dOwLwzTffsG7dOmbOnFn4Xrdu3UrbtiWX9r7mmmuoWrUqVatWpWHDhuzfv5+lS5dy/fXXU7NmTcDs6bBkyRK01sU+7vF4ii0BboPLE8FvENvNbixCeJ3nzt1fqlatWvj7sLCwwj+HhYWRl5cHwOOPP06/fv2YPXs2aWlpJCcnF75m69atREREsGfPnvNeo+j5z75G0TvwkydPXvDry6JOnTrExsbyww8/FCYCrTUvvfTSOXspeJNgae8HIDw8nLy8PEqq21ZaPbfSWh2B5N4xApBFZUKU0ZEjR2jatCkA06ZNO+PxsWPHsnjxYg4ePFh4V10e0dHRbNq0CY/Hw+zZsysacrGqVKnCnDlzePfdd3n//fcBuPLKK3n11VcLWzmpqank5ORc8Ln79OnDnDlzOH78ODk5OcyePZvevXuX+nhxJcDL6uyS4BXhzhZBtdpQrY6sJRCijB5++GFGjRrFpEmTuPzyywsfHzduHPfeey9t27Zl6tSp9OvXjz59+pTrGhMnTmTw4MHExsYSHx/PsWPHfBX+GWrWrMnnn3/OwIEDqVmzJmPGjCEtLY2uXbuitSYqKoo5c+Zc8Hm7du3KHXfcQffu3QEYM2YMiYmJACU+7i0B3rx588LZQGUVGRlJr169iI+P56qrruLZZ5+94Ji93FeG2uvVXqaL6NYZvglKiHKQMtTCX6QMdVnUkUVlQggBrk4EMdI1JIQQuDkR1G0Gp47AifKvNhTCF4Kxe1Y424V+ptybCOrFmWPWTqthCHerVq0aBw8elGQgfEZrzcGDB6lWrVqZX+POWUMA9Zqb4+E0aNzFaijCvWJiYkhPTycjI8N2KCKEVKtWjZiYmDI/38WJIM4cD0uLQNhTuXJlWrRoYTsM4XLu7RqqVgeq1TUtAiGEcDH3JgIwrQIZIxBCuJzLE0FzaREIIVzP5YkgDrJ+k30JhBCuZj0RKKWqKaWWK6XWKqV+UUr9K2AXr9sc8k9D9t6AXVIIIZzGeiIATgGXa627AAnAIKXUJQG5sqwlEEII+4lAG94yg5ULfgVmdU3hFNK0gFxOCCHK7dQx2LoAThz2+amtJwIApVS4UmoNcACYr7VeVsxz7lRKpSilUny2+KZOLKAkEQghnG//Bph+I+xa4fNTOyIRaK3ztdYJQAzQXSkVX8xzpmitk7TWSVFRUb65cKUqpvicLCoTQjhdZqo5Nmjj81M7IhF4aa2zgEXAoIBdtK5MIRVCBIHMVAivagpm+pj1RKCUilJK1S34fXVgALA5YAHIojIhRDDI3AqRrSEs3OendkKtocbAO0qpcExi+khr/XnArl6vuZk+mnsCKlcP2GWFEOKCZKZCo85+ObX1RKC1XgckWgugcArpLohqay0MIYQoUd4p04UdP8wvp7feNWRd3SLlqIUQwokO7QDtgQb+uVmVRCCLyoQQTufHGUMgiQAiGkKl6tIiEEI4lzcRRLb2y+klESglVUiFEM6WuRVqx0DVCL+cXhIBmO4hWVQmhHCqzFS/dQuBJALDu6hMNhAXQjiN1qZF4KeBYpBEYNSLg9PZcPyQ7UiEEOJM2Xvh9DFpEfhd/ZbmeGi73TiEEOJs3oHiqHZ+u4QkAoDIVuZ4UBKBEMJhMreao3QN+Vnd5qDCpEUghHCezFSoWhsiov12CUkEYMpR120uLQIhhPN4Zwwp5bdLSCLwimwlLQIhhPP4ecYQSCL4Xf1WcHCHTCEVQjjHqWw4utuvM4ZAEsHvIluZKaQ5PtoGUwghKurgNnOUFkGA1PfOHNpmNw4hhPDK2GKOkggCJLJgLYEMGAshnOLARgiv8vtaJz+RROBVpxmEVZIBYyGEcxzYZFoD4ZX9ehlJBF7hlUypCWkRCCGc4sBmiGrv98tIIiiqfiuzE5AQQth2KhuO/AYNO/j9UpIIioosSAQyhVQIYZt3oFgSQYDVbwm5x021PyGEsOnARnOURBBgUnxOCOEUBzaZbXTrxvn9UpIIivKuJZCZQ0II2w5sMqWnw/z/NS2JoKg6MWbOrrQIhBC2HdgEDS8KyKUkERQVFg71WkgiEELYdfwQHNsXkPEBkERwLqlCKoSwLWOzOUoisCSyFRz6FTwe25EIIdwqgDOGQBLBueq3gvxTcGSX7UiEEG51YLPZlax204BcThLB2bxV/rz7hAohRKAd2GRKS/hxV7KiJBGcLaqdOWZusRuHEMKdtDZdQwHqFgIHJAKlVKxS6jul1Cal1C9KqbFWA6rZAKrX/315txBCBFJOBpw4FNBEUClgVypZHvCg1nqVUqoWsFIpNV9rvdFaRFHtzIbRQggRaAEeKAYHtAi01nu11qsKfp8NbAICM0JSkgZtzfQtKT4nhAi0/d5EEJjFZOCARFCUUioOSASWFfOzO5VSKUqplIwMP+8rHNUOThyGnEz/XkcIIc62bz1ERENEw4Bd0jGJQCkVAcwCHtBaHz3751rrKVrrJK11UlRUlH+DkQFjIYQt+9dDo04BvaQjEoFSqjImCUzXWn9iOx4aFCQCGTAWQgRS3mmzhiA6PqCXtZ4IlFIKmAps0lpPsh0PYIrPVa4pA8ZCiMDK3AKeXFe2CHoBtwOXK6XWFPy62mpESkGDNtIiEEIE1r4N5tioc0Ava336qNZ6KRCY5XMXIqod/LrEdhRCCDfZt95sRuPdJCtAnNAicKaodpC9B06eM24thBD+sW8dRF9kSuIHkCSCkngHjKXmkBAiELSG/RsCPj4AkghKJlNIhRCBdHS3Wb8kicBB6rWAsMoyYCyECIx9680xWhKBc4RXMgM2MoVUCBEI+zYAyowRBJgkgtJEtft9yzghhPCnfeugfkuoWivgl5ZEUJoG7eBwGuSetB2JECLU7VsPjQK7othLEkFpotqB9sDBbbYjEUKEslPZcPhXKwPFIImgdN564Ac22Y1DCBHa9v9ijgFeUewliaA0kW3MzKH9G2xHIoQIZYUzhqRryHkqVTHdQwfsbZYmhHCBPauhZhTUbmLl8pIIzie64+/NNiGE8Ic9q6FJV1Pw0gJJBOcT3dGs+Dt+yHYkQohQdDrHTFNvkmgtBEkE5xPd0Ryle0gI4Q9715nZiZIIHMw7eCPdQ0IIf9iz2hwlEThYRDTUiJSZQ0II/9izCmo3hVrR1kI4byJQSi1QSnUJRDCOpJQMGAsh/GfPaqutAShbi+BhYLJS6m2lVGN/B+RI0fFmUZkn33YkQohQciLLVC5okmA1jPMmAq31Kq315cDnwFdKqX8qpar7PzQHie4IucdN3SEhhPCVvWvNsUlXq2GUaYxAKaWALcCrwP3AVqXU7f4MzFEaFpSFlXECIYQvOWCgGMo2RrAU2A1MBpoCdwDJQHel1BR/BucYUe1Bhck4gRDCt/asgrrNoUZ9q2FUKsNz7gZ+0Vrrsx6/XynljmpsVWpA/VaSCIQQvrVnNTS92HYUZRoj2FBMEvC6xsfxOJfMHBJC+FJOJmT9Zr1bCCq4jkBrvcNXgThedLypF37qmO1IhANMni9bmIoK2rPGHIM9EbiKlJoQRbywcKvtEESw27PKHBvbX6YliaCsvDsHead7CSFERexabiaiVKtjO5IyDRYLgDoxUKPB78054TqT56ee0RKImzAPgLH92zBuYFtbYYlg5PFA+nK4aKjtSABJBGWnlFn9t1cSgVuNG9i28As/bsI80ia6Z66E8LGDW+HkEYjtYTsSQLqGLkyTRFNqIveE7UiEEMFs1zJzlEQQhJokgs6HfbLC2O3G9m9jOwQRzHYtg+r1ILK17UgAhyQCpdRbSqkDSilnf8M2LigM5V0WLlxLxgREhexabloDlramPJsjEgEwDRhkO4jzqt0EajaUcQIhRPkdPwSZqRDTzXYkhRyRCLTWiwHnbwrsHTCWFoEQorzSV5ijQ8YHwCGJoCyUUncqpVKUUikZGRn2AmmSaDaaPp1jLwYhRPDatRxUODS1W3q6qKBJBFrrKVrrJK11UlRUlL1AGieYjaZlwFgIUR67lpkFqlVq2o6kUNAkAsfw1gWR7iEhxIXKz4PdKx3VLQSyoOzC1W4MEY1kwNhtTueY8iJ715qKkUd3Q/Y+yDsFFBTnrVobajWGWo3MtMAmidCwA4RXthq6cJD9G8xuh7HdbUdyBkckAqXUB5jNbhoopdKBf2qtp9qNqhQyYBz6tDbJfvM8SP3KlCDXHvOzKhG/f+FXq/v7a05mmWZ/9j7IP2UeC69qZoe0u8r8imwV+PcinGPXcnOURHAurfUttmO4IE0SYes3piR11Qjb0QhfOroXVr0Dq9+DI7vMznTNekLvh8wGIk27QkTD0s/h8ZiS5XtWm1/bv4NvHjW/ouPh4jug001QvW7p5xGhZ9fP5iaiTqztSM7giEQQdAoHjNdD80ttRyN8Yc9qWDoZNn1u/m1b94d+j0CbK6Fm5IWdKyzM3PlHtoJOw8xjh3fyxHPP8UTYOvjiIfjmceh8M1w2Duq38P37Ec6jNfy6BFr2dcxCMi8ZLC4P77Sv3Sl24xAVt28D214cClOSYcf3cOmf4S+rYOQsSLj1wpNASeo1Z1r+ILjre7hzkUkCa2fASxfDnD/DoeL3eJINcEJI5lbIOQBxl9mO5BySCMojoiHUi/u9cJQIPjkH4dP74LVeNDy4DJIfgQfWwxVPQv2WPrvM5PmpxE2YV1iyOm7CPOJe3MPk6vfB2LXQ/U7YMBNe7g7z/wGnss94vWyAE0LSlphjXG+7cRRDuobKK7YH7FhkmnsOa+aJUng8sPpdWPCE+dLteT+XfduJdcnD/XK585auvmoiXPYALHwSfnjBtBKueMqMIcjnKrSkLYVaTXx6o+Er0iIor9gecGw/ZO20HYkoq6xd8O4Q+Gws6ZXjGHjiGeK+vZSjRBTetVvpiqnVCK77D4xZCLWbwid/YsE/+tFtwnsAdmMTvqG1SQRxlzkywUuLoLy8C0J+W2a6iYSzrfsY5j1oyogPeYmYxNuZX/AfMlCbzJy3dHVMkkkGy15jwMJ/MaDmo/zl6EhefOZpv8cm/Cwz1bHjAyAtgvJr2MEsIJJxAmfLPQGz74FPxkDD9nD3Uuj6Byt3ZWUqXR0WBpfea+Js0IYXq7xsxjJkM6Tg5h0faOG88QGQRFB+YeHmDs67QEQ4z+E0mHoFrP0A+k6AO74odqqmIzeZadAGRn/NzzGjYfV/4c0BkLnNdlSivNKWmm6/es6cKiyJoCJie8CBX+DkUduRiLNt/85MCc3aCbd+CP3+DuHF94Q6dpOZsHAuGTMZbpsFR/fAlL6w5UvbUYkL5fDxAZBEUDGx3c3iI1lP4Cyr/gvTh5kVnH/6DtpeaTuiimkzAO5eYuoXfXAL/PCi+XIRwSFjC+RkOHLaqJckgopommRKEEj3kDNoDd8+BXPvgxZ9YPTXoVPbp04M/PFLuGgIzH/cjBvknbYdlSiLwvUDzhwoBkkEFVOtNjTsCL/9bDsSkZ8Hn/4ZFj8LibfDrR+Zf59QUqUGDJsGfR6GNe/BB8NNvSvhbL8uhtoxjp5dKImgomK7Q3oKePJtR+Jeeadg5h9hzXR+ir0ThrwUuqWfw8Lg8kdhyMtmQeM710JOpu2oREny80zpklbJjh0fAEkEFRfbA05nw4FNtiNxp9PHYcatsGkuXPm/3LI12dH/4Xym6+0wfDoc2AhvXWkWywnn2b0STh2B1gNsR1IqSQQV1cy7sOwnu3G40ekceP9m2LbQtAIuvdd2RIHV/mq4fQ4cy4BpV5vpssJZti0w44gtk21HUipJBBVVt7np//MOCInAyD0BH4yAnT/wRdt/E/dR5JmF3dxSkqH5pTDqU1M36e2r4eB22xGJorYtMBsTVa9nO5JSSYmJilLKzFBJ/coUNAuT3Op3uSdNd9CvS+D617i6ywjSbjU/ClS5CEdpkgijPoN3h5pkMOoziHLo2gg3yTlo9rlI/rvtSM5LvrV8oWVfOHHI7EcqfKbYO/r8XPjoD7D9Wxj6MnQZEfjAnKhRJ7NyWuebwnol7G8gAmjHd4B2/PgASCLwjRZ9zPHXxXbjCDHn1OL3eGDOPbD1axg8GRJHnvMaR5aLCJSG7eEPc80sqneGQNZvtiNyt20LoHp9s8e5w0ki8IXaTSCyDfz6ve1IQpfW8NXfYP3H0P+fkDS62Kc5tlxEoERfBH+YA6eOmqmlR/fYjsidPB4ziaFVP1OXzOEkEfhKiz6w80fTdSHKrdgdvSbM46e3xsPyKdDzfrPPryhZ4y4wcrbpo/7v9XD8UKlPd8WgeqDt32DKTgdBtxBIIvCdln3h9DHYvcp2JEFt3MC2pE28pnDAN23iNaTdlMGlu96AhJEw8El3rBOoqJiL4ZYP4NCvMP2mUlcgy3aYfrBtgTm2utxuHGUkicBXvAWlZJzAt1K/gc//Cq0HwrXPSxK4EC16w7C3YM8q+HCkGTsQgbFtAUR3MrvPBQFJBL5So76ZuSHjBD7zTPdc+HgUNIqHm6aFbtkIHzqnm6fDYLPYbsd3ZqDd4yl8XnFdcNJN5AM5mWaBafurbUdSZrKOwJda9IXlb5jFTpWr244muGX9xq3bx0ONBnDrx1A1wnZEQeGFhVvPHTBPHGnKIC94wkxsuOIpxg1sW/g8V6698KctX5jy9O2D5+9UWgS+1KIv5J+S7Ssr6lQ2vD/CdGXc9jHUirYdUfDr9QB0+xP8+BL8/JrtaELb5nlQpxk06mw7kjKTFoEvNe8JYZXM7lgtk21HE5w8+TDzfyBjM4ycaebGi1JNnp96xoCvt7tnbP82v7cOlIKr/g9k74WvJkDtxnDR0MLnCR85lW3+/3f7n6Aaz5JE4EtVI0wySP0aBv7LdjTB6ZvHzIKxayYFzYwL28rczRMWDje+aUpRfHIn1GoCsd1k7YUvbVtgegXaD7YdyQWRriFfazsIMjZJJcjyWPkO/PwK9Ljb3FEJ36tcHUZ8YLbx/GCEmV4qfGfT51AjEppdYjuSCyKJwNfaDjLH1K/txhFsdv4E8x40rYArnrYdTdAqUzdPzUi4bSZ48kwZ7xOH/R+YG+Sdhq3fQLurgmI1cVGOSARKqUFKqS1KqW1KqQm246mQyFam3MSWL21HEjyydpl57nWbmXnv4dJjWV5l7uZp0BpGvG9aBB/eLivifeHXxaa0R/trbUdywawnAqVUOPAf4CrgIuAWpdRFdqOqoHaDIG2pGTgSpfPuMJZ/Gm6Z4fi67SElrpep4Jq2xLTGtLYdUXDb/BlUiQjKiSLWEwHQHdimtd6htT4NzACGWo6pYtpeBZ5cUypZlExrmHs/7FsPN06VGvo2dBkBvR+EVQXjM6J88nPN+EDrAVC5mu1oLpgTEkFToOiGq+kFj51BKXWnUipFKZWSkZERsODKJbYHVKsLW76yHYmz/fgSbJgJ/R+HtlfYjsa9+j0GHYbA14/KZ7a8diyC45nQ+WbbkZSLExJBcZNtz2mjaq2naK2TtNZJUVFRAQirAsIrQZuBZuDIk287Gmfa/i0s+KeZy37ZX21H425hYXD9a6Zq6az/gf2/2I4o+Kz70HRrth5oO5JycUIiSAdii/w5Bgj+IuptB5k7hN0rbUfiPIfTYOZoiGoPQ18JqoU3IatKTVOttEqEmVaak2k7ouBxKtt0C3W8HipVsR1NuTghEawA2iilWiilqgAjgLmWY6q41v1BhZu6I+J3p4/DjJGmFsvw96SGkJPUbgK3vA/HDhRUKz1tO6LgsHke5J2AzsNtR1Ju1hOB1joPuA/4GtgEfKS1Dv62afV6ZrOaX2bLbAwvreGzsWbTjhunmqm2wlmaXgxD/2OqZ84bJ5/dslj3oZn6HNvDdiTlZj0RAGitv9Bat9Vat9Jah85qok7DTDeIdA8Zy16H9R9Bv0fMGIpwpk7DoM94WP2ezCQ6n+z9ZqC4081B3cXpiEQQstoPhvAqsH6m7Ujs2/kjfPMotLsaej9kOxpxPsmPQIdrC2o/LbAdjXNtmGW6OYN0tpCXJAJ/ql4X2lwBv3zi7tlDR/fAR6M4VKWJmZ0SJh87xwsLg+tfh4YdYeYfIUM2rCnWuhnQOAGi2tmOpELkf6S/dRoGx/ablcZulHcKPvoDnM7h5iP3QbU6tiMSZVWlphk8rlQVPhgOxw/ZjshZdq+CvWsh4TbbkVSYJAJ/azvITMnb4NLuoa/+Dukr4LpX2KZjbEcjLlTdZjB8OhxJh4/vkJpERaVMhco1oUvwzhbykkTgb5Wrmy3rNs513+bhq9+DlKm8lnctcf81+w3L3rhBqFkPuPYFsx/3V8FdE9JnThyG9bOg800h0cqVMo+BED/MTDHbtjCoNrSukN0r4fO/Qou+3D1yGneHV5K9cYNZwq1wYBP8+KJZCNj9T7YjsmvNB2btQFJo7JshLYJAaNUPqtc3ycANjmWY0jfPgW8AAA0rSURBVMYR0TDsbSkrHSoGPGG6Or/8m9mO0a08Hg5//yrEdIfGwbMvcWkkEQRCeGXocgts/tzMOw5l+bmmL/n4QRjxntkEpYDsjRvkwsLhhjfMDJmPRrl3JtGv31Pv5G/QbYztSHxGEkGgJI02O0Ktftd2JP71zWOwc6npU27c5Ywfyd64IaBabbNvRHhls7uZG2cSpUzlkI4wBRNDhCSCQGnQmiX58ZAyLXTXFKyeDstegx73mDr3IjTVa252Nzu6x3QBuqQm0eT5qfT++1vkbZzHR/n9iHt8YchMfJBEEEDv5Q+Eo+mhuZ9x+kr4fJypr3TFU7ajEf7WrIepSbRzqakf5YKaROMGtmVJz7VUqlSJqXmDSJt4DWkTrwmJlq6M4vnZ5PmpvLBwKwDhdGWvrs+W6RNZ3ad1SHyAAMjeBx/eBrWiYdg0GRx2i843waEdsOgZqN8S+o63HZF/Hd0Da96HxJFkLA2tLVWlReBn4wa2LbxzyCecxv3uJjl8LeMuDpEvy9yTMOM2OHnEdBcUGRwWoauwO6Tvw9B5BHz3FKz72G5Q/vbjS6Zbt9fYkJv4IIkg0Lr+wexTkPKW7UgqTmuYex/sTjF1aRp1sh2RCBBvKxelYMiL0Pwy+PRe+HWJ3cD8JScTVk4zxeXqxYVOa76AJIIAGtu/DdRubGYbrHzHrE4MZkv+H6z/GC5/DC4aYjsaYUulqjD8v1CvhWkd7t9oOyLf+/lVyD0Bl42zHYlfSCIIoMK7iD4Pwamj5sMVrDbOhW+fhE43SVlpl5g8P7WwRAicVS6kRn0YOdOUVJk+DI7sthytDx0/BMvfMGW5g7zKaEkkEdgQ3RE6DDGJoJRWgWOnpaWnwCd/gphuMOTloN6QQ5Rd0fEu4NxZM3WbmWRw8qhJBsHe4vVaNBFOZ0Py321H4jeSCGzp+7eCVsFrJT6lsB/WSQ7tgPeHQ61GMOIDqFzNdkTCSRp1MivKM7fC9JvhdI7tiComIxVWvAldR0H0Rbaj8RtJBLY0ijc7mP38KpzIsh1N2Rw/BNNvAp0Pt82CiCjbEQlLSp010zIZhk01kwg+HBncVXe/eczsy9DvUduR+JUkApv6/g1OHTF7+RYotR/WptM58MEIyNplWgINWtuNR1h13lkzFw2Fa1+E7d+absRgXE2//VvY+jX0fjDkb3qUDsIVgUlJSTolJcV2GL4x4zaz+fV9K6B2kzN+5JiyzXmnYcYt5j/GTdNCqsaK8LMfXzZ7VXceAde9YgrXBYP8PHi9t7kBum+FmRkVApRSK7XWSWc/Li0C2654yhSj+8qhA1EeD8y5B7YtgMHPSxIQF6bnfWZ68boZbHz19uBpGSydDAc2mv+fIZIESiOJwLb6LUzTc+Mc82VbhPXVi1rDFw+abTYHPAEXjyr8kfWuKhE8+oyHfo9yUcY8mHu/85PB3rXw/USIv9E162MkEThBr7EQ2Rq+GG9KNhSwunrR44F5D5oV0L0eOGchjSNnNAnn6vswk3NvhDXTYfZdZd77OOA3HLknYfbdUKMBXP1cYK9tkSQCJ6hU1XzoDu0wTVLbPB744iGzOXevsaY1IEQ5FJ388EL+jfzf3OGw/mN2vHydWal7HgG/4fjuadMlNPRls0jOJSQROEWrfmZv4yXPwW8/24vDkw/zxhVJAv8qXDDm2BlNwrHOXoT28NNT4JpJtDz8A7x3oylW6BRbF5jCchffAW0G2o4moGTWkJOcyIIpyeZO6a7FpqxzAEyen2q6oXJPwKwxZkvNy/4K/f9R4qphx8xoEkHjjM/M+pmmiyiyNdz6IdSLK3xe0dLtRY3t38Z/3aX7f4GpV0L9OPjjV1A1wj/XsUxmDQWD6nVh+HvmLmnmH8vcj1pRLyzcasoB/Pd62DwPBk2EAf+U0hHCp86Y/NBpGIz8xOxl8cblsPOnwh+dt5SFr2XvN6vlq0bALR+GbBIojSQCp2kUb/b73fkDzP9HQC7ZSu2GqVfA7pUw7C245J7zvsb6jCYRdM75Im/ZF8YshGp14d0hkPJ24Hc6O53D3tevh+MH4ZYPoE7TwF7fIUJkd5QQ02U47FkFP79iBpL7+/7u3Nv8vjrsZz6tMoXMjCr8+fTfuGRvPOPiz//6UKvHLixp0BrGLICZo+HzB+DX782NULU6gJ9vOI4fgvdvpmH2RlMfqUmi/67lcFbHCJRSNwFPAB2A7lrrMnX8h+wYQVEej5nDn/IWdL/LdNeE+bABd/o4LPw3LHuVlZ42XPzQ3HNWNgsRMB4P/PA8fPsU1ImBG6ZAs0v8d72sXfDeDXB4J3eduIfXn/mX/67lIE4dI9gA3AAsthyH84SFwTWT4NL7YPnr8OmfzZe3L2z/Dl69FJa9Ct3vYsTpxyUJCLvCwqD3X+GPX5ruobeuNIvPjh/y/bXSU8h+5XKOZuxi+PGH+drT3fUz4Kx2DWmtNwEoGZQsnlJmiXvVWrDof2HXz6b+f1yv8p3v4Hb4/v/CuhlQvxWM+hxa9Obeyu788AsHatYD7v3JrOz96RUzeaHfI5AwsuIlz3NPmBbHz69Qq1YTGD2fDxt1khlw2G8RiPNRCpInwKjPQHtg2tXw2QOQua3s59i3wUwLfTkJfpltSlrc8wO06A1If79wmKoR5gbo7iXQoJ1Z4f5CF1PA7tSxUl9a7B19fi5s+AReuwx+etnsLXDvT7LHdhF+HyNQSi0AGhXzo0e11p8WPGcR8FBpYwRKqTuBOwGaNWt28c6dO/0QrcOdzjF3NMteN3sCxPaAzsMhOh7qNYeIaDiVDccOQNZOUy009Ss4uA0q14Ruo01XU63i/jmEcCCtzQDy4ucgbQlUqm4We3W8Dlr1N1Ouiyi8u/d4IDPVrIlZMRWy95hW8OBJZr+EIgrX0bhASWMEjlhQVpZEUJQrBotLk70P1s4wdVsyi9wBqTDTavAKrwIt+kDbQaaAlouWzIsQlJ4Caz8w+2XnHDCP1W5q9hGu1QR0PnNW7eS61pVg92qzvSRAy37Q426TQIKlDLafSCIIRVqbu/1Dv5oWwNE9PP3dPh69Odnc9Te92JWLY0SI8+SbMiy7lkHGFvZvX4Pn2AHydDh5hHGEmqzztKJhh14Muuo6U+FXAA5NBEqp64GXgCggC1ijtb7yfK+TRFAyGfgSbiaf/9KVlAhszxqaDcy2GUMoOLs2i7conF9rswghQoasLA4B4wa2LfzClzsi4WZS+qR8ZPqoECJkSAu4fCQRhBi5IxJCXChJBCFG7oiEEBdKEoEQQricJAIhhHA5SQRCCOFykgiEEMLlJBEIIYTLOaLW0IVSSmUAwVh+tAGQaTuIAHLb+wV5z24RrO+5udY66uwHgzIRBCulVEpxdT5CldveL8h7dotQe8/SNSSEEC4niUAIIVxOEkFgTbEdQIC57f2CvGe3CKn3LGMEQgjhctIiEEIIl5NEIIQQLieJwAKl1ENKKa2UamA7Fn9TSj2rlNqslFqnlJqtlKprOyZ/UUoNUkptUUptU0pNsB2PvymlYpVS3ymlNimlflFKjbUdUyAopcKVUquVUp/bjsVXJBEEmFIqFhgI/GY7lgCZD8RrrTsDqcDfLcfjF0qpcOA/wFXARcAtSqmL7Ebld3nAg1rrDsAlwJ9d8J4BxgKbbAfhS5IIAm8y8DDgilF6rfU3Wuu8gj/+DMTYjMePugPbtNY7tNangRnAUMsx+ZXWeq/WelXB77MxX45N7UblX0qpGOAa4E3bsfiSJIIAUkoNAXZrrdfajsWS0cCXtoPwk6bAriJ/TifEvxSLUkrFAYnAMruR+N3zmBs5j+1AfEk2r/cxpdQCoFExP3oUeAS4IrAR+V9p71lr/WnBcx7FdCVMD2RsAaSKecwVrT6lVAQwC3hAa33Udjz+opQaDBzQWq9USiXbjseXJBH4mNZ6QHGPK6U6AS2AtUopMF0kq5RS3bXW+wIYos+V9J69lFKjgMFAfx26C1fSgdgif44B9liKJWCUUpUxSWC61voT2/H4WS9giFLqaqAaUFsp9Z7WeqTluCpMFpRZopRKA5K01sFYwbDMlFKDgElAX611hu14/EUpVQkzGN4f2A2sAG7VWv9iNTA/UuaO5h3gkNb6AdvxBFJBi+AhrfVg27H4gowRCH97GagFzFdKrVFKvWY7IH8oGBC/D/gaM2j6USgngQK9gNuBywv+bdcU3C2LICMtAiGEcDlpEQghhMtJIhBCCJeTRCCEEC4niUAIIVxOEoEQQricJAIhhHA5SQRCCOFykgiE8IGCuvwDC37/lFLqRdsxCVFWUmtICN/4J/BvpVRDTBXOIZbjEaLMZGWxED6ilPoeiACSC+rzCxEUpGtICB8oqC7bGDglSUAEG0kEQlSQUqoxZp+FoUCOUupKyyEJcUEkEQhRAUqpGsAnmL17NwFPAk9YDUqICyRjBEII4XLSIhBCCJeTRCCEEC4niUAIIVxOEoEQQricJAIhhHA5SQRCCOFykgiEEMLl/j+Bqwv/Mi1yLAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# WRITE THE PLOTTING CODE HERE\n",
    "plt.figure()\n",
    "plt.plot(X, y, '+')\n",
    "\n",
    "# feature matrix\n",
    "Phi = poly_features(X, 5)\n",
    "\n",
    "# maximum likelihood estimate\n",
    "theta_ml = nonlinear_features_maximum_likelihood(Phi, y)   \n",
    "\n",
    "# feature matrix for test inputs\n",
    "Phi_test = poly_features(Xtest, 5)\n",
    "\n",
    "ypred_test = Phi_test @ theta_ml\n",
    "\n",
    "plt.plot(Xtest, ypred_test) \n",
    "plt.xlabel(\"$x$\")\n",
    "plt.ylabel(\"$y$\")\n",
    "plt.legend([\"data\", \"maximum likelihood fit\"]);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The RMSE on the training data is somewhat misleading, because we are interested in the generalization performance of the model. Therefore, we are going to compute the RMSE on the test set and use this to choose a good polynomial degree."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYoAAAEGCAYAAAB7DNKzAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nO3deXxU9bn48c+TyQYJCavKFoiKyCI7KIoWFxQUpFVUrPZiXdCqdblXf9XbVlu9vfXWVq2iIirFlcUdFJW6IC4om4IssqMEkFUIAbJM5vn9cU7CECYbmZkzOXner9e85qzf8+QkmWe+33PO9yuqijHGGFOZJK8DMMYYk9gsURhjjKmSJQpjjDFVskRhjDGmSpYojDHGVCnZ6wBioWXLltqxY0evwzDGmHpj4cKFO1S1VaR1vkwUHTt2ZMGCBV6HYYwx9YaIfF/ZOmt6MsYYUyVLFMYYY6rkq0QhIiNEZMKePXu8DsUYY3zDV9coVHUGMKNfv37XVVxXUlJCXl4ehYWFHkTW8KSnp9OuXTtSUlK8DsUYU0e+ShRVycvLo0mTJnTs2BER8TocX1NVdu7cSV5eHrm5uV6HY4ypI181PVWlsLCQFi1aWJKIAxGhRYsWVnszxid8lSiqu0ZhSSJ+7Fwb4x++anqq6hqFMcbUO6EQlOx3XsX73Pf9ULLPfd8PxQUHl5UcgLP+CFH+ouarRJHIdu/ezcsvv8yNN95Y633PP/98Xn75ZZo2bVrpNvfccw9nnHEG55xzTl3CrLU333yTE044ga5du8b1uMb40py/w8LnDiaC4IHa7S8BOONOSGkU1bAsUcTJ7t27eeKJJyImitLSUgKBQKX7zpw5s9ry77vvvjrFd6TefPNNhg8fbonCmLrauxU++Rsc3RXaDIHUxpCS4b43htSMCu8R1gdSo16bAJw7VPz26tu3r1a0fPnyw5bF02WXXabp6enas2dPveOOO/Tjjz/WwYMH6+WXX65dunRRVdWRI0dqnz59tGvXrvrUU0+V79uhQwfdvn27rl+/Xk888US99tprtWvXrjpkyBDdv3+/qqqOGTNGX3nllfLt77nnHu3du7d2795dV6xYoaqq27Zt03POOUd79+6tY8eO1ZycHN2+ffshcQaDQR0zZox269ZNu3fvrg899JCqqq5Zs0bPO+887dOnjw4aNEhXrFihn3/+uTZr1kw7duyoPXv21DVr1hxSltfn3Jh6ZdYfVf/UVHXHmuq3jQFggVbymZrwNQoRSQLuB7JwfpDn6lrmn2csY/nm/DrHFq5rmyzuHdGt0vUPPPAAS5cu5ZtvvgFg9uzZzJs3j6VLl5bfQjpx4kSaN2/OgQMH6N+/PxdffDEtWrQ4pJzVq1czefJknn76aS699FJee+01rrzyysOO17JlSxYtWsQTTzzB3//+d5555hn+/Oc/c9ZZZ3H33Xfz3nvvMWHChMP2++abb9i0aRNLly4FnJoQwNixYxk/fjydOnXiq6++4sYbb+Sjjz7iwgsvZPjw4YwaNerITpwxBg7shvkTodsvoMVxXkdzGE/uehKRiSKyTUSWVlg+VERWisgaEbnLXTwSaAuUAHnxjjWWBgwYcMhzBo8++ig9e/bklFNOYePGjaxevfqwfXJzc+nVqxcAffv2ZcOGDRHLvuiiiw7b5rPPPmP06NEADB06lGbNmh2237HHHsu6dev47W9/y3vvvUdWVhYFBQV88cUXXHLJJfTq1Yvrr7+eLVu21OVHN8aEm/80FO+FQbd7HUlEXtUoJgHjgOfLFohIAHgcGIKTEOaLyHSgMzBXVZ8SkVeBD+t68Kq++cdTRkZG+fTs2bP54IMPmDt3Lo0bN2bw4MERn0NIS0srnw4EAhw4EPliV9l2gUCAYDAIOM2M1WnWrBmLFy/m/fff5/HHH2fatGk88sgjNG3atLw2ZIyJouL98OWT0OlcOOYkr6OJyJMaharOAXZVWDwAWKOq61S1GJiCU5vIA35ytymtrEwRGSsiC0Rkwfbt22MRdp00adKEvXv3Vrp+z549NGvWjMaNG/Pdd9/x5ZdfRj2GQYMGMW3aNABmzZrFTz/9dNg2O3bsIBQKcfHFF3P//fezaNEisrKyyM3N5ZVXXgGchLN48eIa/VzGmGp8/QLs3wmD/tPrSCqVSA/ctQU2hs3nucteB84TkceAOZXtrKoTgD8Di1JTU2MZ5xFp0aIFp512Gt27d+fOO+88bP3QoUMJBoP06NGDP/7xj5xyyilRj+Hee+9l1qxZ9OnTh3fffZfWrVvTpEmTQ7bZtGkTgwcPplevXlx11VX89a9/BeCll17i2WefpWfPnnTr1o233noLgNGjR/Pggw/Su3dv1q5dG/WYjfG1YDF8/ijkDIQOA72OplJSk+aImBxYpCPwtqp2d+cvAc5T1Wvd+V8BA1T1t7Utu1+/flpx4KIVK1bQpUuXuoZdrxUVFREIBEhOTmbu3Ln85je/iWlzkp1zY6rx9Uvw1o1wxavQaYinoYjIQlXtF2ldIt31lAe0D5tvB2yuTQEiMgIYcfzxx0czLt/44YcfuPTSSwmFQqSmpvL00097HZIxDVcoBJ8/4lyXOD6+D8rWViIlivlAJxHJBTYBo4FfehuSv3Tq1Imvv/7a6zCMMQDfvQ07VsGoibF5SC6KvLo9djIwF+gsInkico2qBoGbgfeBFcA0VV1Wm3JVdYaqjs3Ozo5+0MYYEy2q8NlD0PxY6Ppzr6Oplic1ClW9vJLlM4Hq+6uohDU9GWPqhXUfw+avYcQ/Iany7nsSRSLd9VRnVqMwxtQLnz4ETVpDz4jfmROOrxKFjZltjEl4eQtgw6cw8GZITqt++wTgq0SRyDWKst5jj9QjjzzC/v376xzH7Nmz+eKLL+pcjjHmCH36EDRqBn2v8jqSGvNVokhkliiMMWxbASvfgQHXQ1qm19HUmK8SRSI3Pd11112sXbuWXr16lT+Z/eCDD9K/f3969OjBvffeC8C+ffu44IIL6NmzJ927d2fq1Kk8+uijbN68mTPPPJMzzzwzYtldu3alR48e3HHHHQBs376diy++mP79+9O/f38+//xzNmzYwPjx43n44Yfp1asXn376afxOgDEGPnvEGUPi5Ou9jqRWEuk5ijrTmg6F+u5d8OO30T34MSfBsAcqXV2xm/FZs2axevVq5s2bh6py4YUXMmfOHLZv306bNm145513AKcPqOzsbB566CE+/vhjWrZseUi5u3bt4o033uC7775DRMq7Bb/11lu5/fbbGTRoED/88APnnXceK1as4IYbbiAzM7M8oRhj4uSnDfDtK3DyDdC4udfR1IqvEkV9MmvWLGbNmkXv3r0BKCgoYPXq1Zx++unccccd/O53v2P48OGcfvrpVZaTlZVFeno61157LRdccAHDhw8H4IMPPmD58uXl2+Xn51vnfcZ46YvHQJJg4E1eR1JrvkoUNX6Ooopv/vGiqtx9991cf/3hVdCFCxcyc+ZM7r77bs4991zuueeeSstJTk5m3rx5fPjhh0yZMoVx48bx0UcfEQqFmDt3Lo0aRXfsXGPMESjYBl+/CD1HQ3Zbr6OpNV9do0jku54qdsd93nnnMXHiRAoKCgCn19Zt27axefNmGjduzJVXXskdd9zBokWLIu5fpqCggD179nD++efzyCOPlDdtnXvuuYwbN658u7Ll1i24MR748gkoLU7YgYmq46saRSIL72Z82LBhPPjgg6xYsYKBA52uhTMzM3nxxRdZs2YNd955J0lJSaSkpPDkk08CzlCkw4YNo3Xr1nz88cfl5e7du5eRI0dSWFiIqvLwww8Dzmh5N910Ez169CAYDHLGGWcwfvx4RowYwahRo3jrrbd47LHHqm3aMsbUUeEemP8sdB2ZkMOc1oRn3YzHknUznhjsnBsDfPoP+PA+uH4OtO7pdTSVqqqbcV81PRljTEIp3g9zn3C6EU/gJFEdXyWKRH6OwhjTAH39IuzfkdDDnNaErxJFdRez/djMlqjsXJsGr7QEvngU2p8MHU71Opo68VWiqEp6ejo7d+60D7A4UFV27txJenq616EY451vX4U9G+H0/0r4gYmq02DuemrXrh15eXls377d61AahPT0dNq1a+d1GMZ4IxSCzx6Go7tDp3O9jqbOEj5RiMhg4H5gGTBFVWcfSTkpKSnk5uZGMTJjjKnEypmwYyVc/Gy9r02Ad0OhThSRbSKytMLyoSKyUkTWiMhd7mIFCoB0IC/esRpjTK2oOrfENutYL4Y5rQmvrlFMAoaGLxCRAPA4MAzoClwuIl2BT1V1GPA74M9xjtMYY2pn/SeweRGcdisEEr7RpkY8SRSqOgfYVWHxAGCNqq5T1WJgCjBSVUPu+p+A+jEclDGm4frsYcg8Gnr+0utIoiaR0l1bYGPYfB5wsohcBJwHNAXGRdoRQETGAmMBcnJyYhimMcZUYtd6WDcbzvoDpPjnrr9EShSRrvioqr4OvF7dzqo6QUS2ACNSU1P7Rj06Y4ypzpJpgECP0V5HElWJ9BxFHtA+bL4dsNmjWIwxpnZUYfFk6DgImravfvt6JJESxXygk4jkikgqMBqYXpsCErmbcWOMz+XNh5/WO2NO+IxXt8dOBuYCnUUkT0SuUdUgcDPwPrACmKaqy2pZrvX1ZIzxxuLJkNwIulzodSRR12C6GTfGmJgJFsHfT3B6iR31rNfRHJEG08241SiMMZ5YPQsKd0PPy72OJCZ8lSjsGoUxxhOLp0DGUXDsYK8jiQlfJQqrURhj4m7/Llj1PvS41DdPYlfkq0RhNQpjTNwtfQ1CJdDjMq8jiRlfJQqrURhj4m7JVDiqKxxzkteRxIyvEoXVKIwxcbVzrfP8RM/RvuhOvDK+ShTGGBNXi6cAAidd4nUkMeWrRGFNT8aYuAmFYMkU506nrDZeRxNTvkoU1vRkjImbjV/C7h982WVHRb5KFMYYEzeLp0BKBpw43OtIYs4ShTHG1FZJISx7E7qMgLRMr6OJOUsUxhhTW6vehaI90NO/z06E81WisIvZxpi4WDwVmrSG3J95HUlc+CpR2MVsY0zM7dsBa/7t3BKbFPA6mrjwVaIwxpiYW/oahIK+7Sk2EksUxhhTG4snO911HN3V60jixhKFMcbU1PZVsPnrBlWbgHqSKEQkQ0QWioj/b1g2xiSuJVNAkqD7KK8jiSuvxsyeKCLbRGRpheVDRWSliKwRkbvCVv0OmBbfKI0xJkwoBEumwXFnQZOjvY4mrryqUUwChoYvEJEA8DgwDOgKXC4iXUXkHGA5sDXeQRpjTLnvP4c9GxtcsxOAJ8MxqeocEelYYfEAYI2qrgMQkSnASCATyMBJHgdEZKaqhiqWKSJjgbEAOTk5sQveGNMwLZ4CqU2g8/leRxJ3iTRuX1tgY9h8HnCyqt4MICJXATsiJQkAVZ0gIluAEampqX1jHawxpgEp3g/L34KuIyG1sdfRxF0iXcyONOqHlk+oTlLVt6sqwB64M8bExMqZULy3wXTZUVEiJYo8oH3YfDtgc20KsC48jDExsXgKZLWDDoO8jsQTiZQo5gOdRCRXRFKB0cB0j2MyxjR0e7fC2o+gx6WQlEgfmfHj1e2xk4G5QGcRyRORa1Q1CNwMvA+sAKap6rLalGtNT8aYqFv6KmhpgxigqDJe3fUU8f4yVZ0JzDzSckVkBDDi+OOPP9IijDHmUIunQJve0Kqz15F4xlf1KKtRGGOiauty+HEJ9Gi4tQnwWaKwi9nGmKhaMgWSkqH7xV5H4ilfJQqrURhjoiZUCktegePPgcxWXkfjKV8lCmOMiZr1c2Dv5gZ9EbuMrxKFNT0ZY6JmyVRIy4YThnkdied8lSis6ckYExXF+2D5dOg2ElLSvY7Gc75KFMYYExUr3oaSfQ2yp9hIfJUorOnJGBMVS6ZA0xxof4rXkSQEXyUKa3oyxtTJT9/Du7+DdbOdZycaaJcdFSVSN+PGGOONzd/AF4/CsjedoU57jIZTb/Y6qoRhicIY0zCpwtoP4fNHYf0nzqBEA2+Ek38D2W29ji6hWKIwxjQspSWw9DX44jHYuhSatIYh90HfqyDdmq0j8VWisE4BjTGVKsyHRc/Bl09C/iZo1QVGPgEnXQLJqV5Hl9BEVavfqp7p16+fLliwwOswjDGJIH8LfPUkLJgERXug4+lw6i3QaQhIpIE1GyYRWaiq/SKt81WNwhhjym37zmleWjLVGU+iy4Vw2i3Qtq/XkdU7liiMMf4RKoXVs2DeBGdUuuRGzrWHgTdB81yvo6u3Ej5RiEgX4FagJfChqj7pcUjGmESzfxcseh4WPAu7f4AmbeDM30O/ayCjhdfR1XueJAoRmQgMB7apavew5UOBfwIB4BlVfUBVVwA3iEgS8LQX8RpjEtTmb2De085wpcFC6DAIhtwPJ14AgRSvo/MNr2oUk4BxwPNlC0QkADwODAHygPkiMl1Vl4vIhcBd7j7GmIYsWAzL33Kal/LmQUpjp0+mAdfB0d28js6XvBoze46IdKyweACwRlXXAYjIFGAksFxVpwPTReQd4OV4xmqMSRD5m2HBv2DhJNi3DZofB0MfcJJEo6ZeR+driXSNoi2wMWw+DzhZRAYDFwFpwMzKdhaRscBYgJycnNhFaYyJH1X4/nOneWnFDNAQnHCeU3s49izriylOEilRRLqhWVV1NjC7up1VdYKIbAFGpKam2v1vxtRHxfthxyrYvhK2fwer3odtyyC9qdO9Rr9r7O4lD1SZKETkLFX9yJ3OVdX1YesuUtXXoxhLHtA+bL4dsDmK5RtjEkVh/qEJoex99w+A+xBwUjIc0wMufAy6j4LUxp6G3JBV+WS2iCxS1T4VpyPN1/rAzjWKt8vuehKRZGAVcDawCZgP/FJVl9W2bHsy25gEUVIIW745NBlsX+l0oVEmkAotT4BWnaHViQffmx9rdy7FUV2ezJZKpiPN1yagycBgoKWI5AH3quqzInIz8D7O7bETa5skrK8nYxKIKrzwC/jhC2c+pbGTEDqeDq1OcJPCidC0AwQSqRXcVFTdb0crmY40X2OqGnF8QVWdSRUXrI0x9UjefCdJnH4H9PkPyG5vF5/rqeoSxbEiMh2n9lA2jTufcFeUVHUGMKNfv37XeR2LMQ3eV09BWjYMuh3SMr2OxtRBdYliZNj03yusqzjvOWt6MiZB7P0Rlr8JA8ZakvCBKhOFqn4SPi8iKUB3YJOqbotlYEfCahTGJIiFz0EoCP2v9ToSEwVVNhiKyHgR6eZOZwOLcbrd+FpEIl5nMMY0cKUlsGAiHD8EWhzndTQmCqq7snR62J1HvwZWqepJQF/g/8U0siMgIiNEZMKePXu8DsWYhmvFdCj40Wl2Mr5QXaIoDpseArwJoKo/xiyiOlDVGao6Njvbxr01xjPznoZmHeH4c7yOxERJdYlit4gMF5HewGnAe1D+cFyjWAdnjKlntiyBH+ZC/+vsVlgfqe6up+uBR4FjgNvCahJnA+/EMrAjYXc9GeOx+U87D9b1vsLrSEwUVdmFR31lXXgY44H9u+ChrtDzMhjxT6+jMbV0xF14iMijVa1X1VvqEpgxxke+fhGCB5xmJ+Mr1TU93QAsBabh9OR6xP07GWN8LFQK85+BDqfBMd2r397UK9UlitbAJcBlQBCYCrymqj/FOjBjTD2y+t+w+3sYcp/XkZgYqPK2BFXdqarjVfVM4CqgKbBMRH4Vj+Bqy56jMMYj856CJm3gxAu8jsTEQI3uXxORPsBtwJXAu8DCWAZ1pOw5CmM8sGM1rP0I+l1t40f4VHUXs/8MDAdWAFOAu1U1GI/AjDH1xPxnICkF+o7xOhITI9Vdo/gjsA7o6b7+V0TAuaitqtojtuEZYxJa0V745mXo9gvIPMrraEyMVJcoEm7MCWNMAlk8BYry4eTrvY7ExFB13Yx/H2m5iASA0UDE9dEkIj8HLgCOAh5X1VmxPqYxpgZUnX6d2vSGtn29jsbEUHXdjGeJyN0iMk5EzhXHb3Gaoy490oOKyEQR2SYiSyssHyoiK0VkjYjcBaCqb6rqdTh3XV12pMc0xkTZ+jmwY6XTS6zYI1Z+Vt1dTy8AnYFvgWuBWcAoYKSqjqxqx2pMAoaGL3BrKY8Dw4CuwOUi0jVskz+4640xiWDeBGjcArpd5HUkJsaqHTPbHX8CEXkG2AHkqOreuhxUVeeISMcKiwcAa1R1nXu8KcBIEVkBPAC8q6qLKitTRMYCYwFycnLqEp4xpjq7f4CVM+G02yAl3etoTIxVlyhKyiZUtVRE1tc1SVShLbAxbD4POBn4LXAOkC0ix6vq+Eg7q+oEEdkCjEhNTbUGU2NiacFE573f1d7GYeKiukTRU0Ty3WkBGrnzZbfHZkUxlkiNnKqqj+J0dV4tGzPbmDgoKXTGxO58PjRt73U0Jg6qu+spEK9AcGoQ4X917XA6IqwxG4/CmDhY9joc2GVDnTYgiTQE1Xygk4jkikgqzu230z2OyRgTThW+egpanQi5Z3gdjYkTTxKFiEwG5gKdRSRPRK5xuwa5GXgfp8uQaaq6rDblWl9PxsRY3gLY8g0MuM5uiW1AqrtGEROqenkly2cCM4+0XGt6MibG5k2AtCzoMdrrSEwcJVLTU51ZjcKYGCrYBsvegF6/hLRMr6MxceSrRGHjURgTQwsnQajEhjptgHyVKKxGYUyMlJY4z04cdza0tKbdhsZXicJqFMbEyHdvw94tdktsA+WrRGE1CmNiZN7T0LQDdBridSTGA75KFMaYGPhxKXz/OfS/FpLi+QyuSRSWKIwxVZs3AZIbQe8rvY7EeMRXicKuURgTRbvWw1s3w9cvQo9LoHFzryMyHvFVorBrFMZEwa518OZN8FhfWDLNaXIacr/XURkPefJktjEmAe1cC5/+wxkHO5DidNNx2m2Q1drryIzHLFEY09DtXAtz/g5LpjoJ4uTr4bRbockxXkdmEoSvEoX19WRMLexYA3MehG+nQSANTr7BTRBHex2ZSTC+ShQ2cJExNbBjtZsgXnESxCk3wqm3WIIwlfJVojDGVGH7KidBLH0VktNh4E1Ogsg8yuvITIKzRGGMn4VCzsNyC/8FS1+HlEYw8GY3QbTyOjpTT1iiMMaPti53Lk5/+wrkb4LUTDjtFidBZLT0OjpTzyR8ohCRY4HfA9mqOsrreIxJWPmb4dtXnWcftn4LEoDjz4Eh90Hn8yG1sdcRmnrKk0QhIhOB4cA2Ve0etnwo8E8gADyjqg+o6jrgGhF51YtYjUlohfmwYoZTe1g/B1Bo2w+GPQjdfmHNSyYqvKpRTALGAc+XLRCRAPA4MATIA+aLyHRVXe5JhMYkqmAxrP3QSQ4r34VgITTLhZ/9DnpcCi2O8zpC4zNejZk9R0Q6Vlg8AFjj1iAQkSnASKBGiUJExgJjAXJycqIWqzEJQRXy5jvJYenrcGAXNGoOvX8FPS6Ddv1AxOsojU8l0jWKtsDGsPk84GQRaQH8BegtIner6l8j7ayqE4AJAP369dNYB2tM3OzbCW/dBKvedW5r7Xy+kxyOP9t5ktqYGEukRBHp65Cq6k7ghhoVYE9mG79Z+zG8cYNTgxhyP/S9CtKzvI7KNDCJlCjygPZh8+2AzR7FYoy3gsXw0f3wxaPQsjNc+Socc5LXUZkGKpG6GZ8PdBKRXBFJBUYD02tTgHUzbnxhxxp4doiTJPpdDWNnW5IwnvIkUYjIZGAu0FlE8kTkGlUNAjcD7wMrgGmquqyW5drARab+UnUGCXrqDNj9PVz2Igx/2J5/MJ7z6q6nyytZPhOYGedwjPHegd3w9u2w7HXoeDr84inIbut1VMYAidX0VGfW9GTqpe/nwvhBsGI6nH0v/MdbliRMQkmki9l1Znc9mXqlNOj05jrnb9A0B66eBe36eh2VMYexGoUxXtj9A0y6AD55wHkm4vpPLUmYhOWrGoUx9cLS12DG7aAhuOgZ6HGJ1xEZUyVfJQprejIJqzQIBVvh4/+Fb16Edv3hoqehea7XkRlTLV8lChsK1cSVKhTuhoJtThIof996+LJ9OwAFBM640+nAz7rfMPWErxKFMTEVLIYP/ww/zD2YAEqLD98ukAaZRztDjDbt4NQeyubb9YPWPeMfuzF14KtEYU1PJmaK9sLUX8G6jyH3DOdZh8yjDiaAzKMPTqdnW0+uxld8lSis6cnERMF2eGkU/PgtjHwCel/hdUTGxJWvEoUxUbdrPbx4EeRvgcsnwwnneR2RMXFnicKYymxZDC+OglAJjJkO7Qd4HZExnvDVA3fGRM26T+BfF0AgFa5+35KEadB8lSis91gTFUtfhxcvhqbt4dp/Q6vOXkdkjKd8lSisCw9TZ189Ba9e7dzS+uuZkNXG64iM8ZxdozAGnIfnPrwPPnsIThwOFz8DKY28jsqYhGCJwpjSIMy41elao+9VcMFDkBTwOipjEoYlCtOwFe+HV38Nq96Dn90Fg++yh+WMqSDhE4WIZABPAMXAbFV9yeOQjF/s3wUvXwabFji1iP7XeB2RMQnJqzGzJ4rINhFZWmH5UBFZKSJrROQud/FFwKuqeh1wYdyDNf60eyNMHOo8K3HJc5YkjKmCVzWKScA44PmyBSISAB4HhgB5wHwRmQ60A751NyuNb5gmIeVvgcWTnemURpCc7r6nQXIjSEl33pPTKqxPd1471zi3vxbvg1+9AR1P8/bnMSbBeZIoVHWOiHSssHgAsEZV1wGIyBRgJE7SaAd8QxU1IBEZC4wFyMnJiX7QfqMKwSIo2Q+hUkCdgXQ05KzTUNiysPeKy5q2h7Qm8Yt5yVR49/9BYR2flck8Bq5+F47uFp3YjPGxRLpG0RbYGDafB5wMPAqME5ELgBmV7ayqE0RkCzAiNTU1sceULBvHYM8m2JMH+XlOx3PoEZYXgmAhlByAkkLnwz/ovpccOPQVDJs+0uOFS28KZ/0B+v4aAjH8c9q7Fd6+DVbOhPYnw8jHIbud+zO5P3uwyP35Cp33YFHk9arQ83InyRljqpVIiSLSrSaqqvuAX8c7mDopOQD5m2HPxkOTwZ68g/Ml+6J4QDnYvJLS2Gl6KZtOTodGzd1ljcOaYtzplEaQlOzc6SzOvmQAABQgSURBVCNJTlmSdHA+4jJxlmkIFj0PM++ABRNh6ANw7M+i+HPhfKh/+wrMvNP5wD/3L3DKbw7evmrPOhgTc4mUKPKA8K947YDNtSmgrt2Mf/qPX9Ko8EcABEXcb9xOBgubVxAp+zaulE02SSrkaN1Bo5KfDi884yjnG3CrE+C4s5zp7LaQ3R6y2jrjGNTHe/e7XwzfvQ3v/zc8fyF0uRDO/R9o1qHuZRdsg7dvd8pv1x9+/iS07FT3co0xtZJIiWI+0ElEcoFNwGjgl7UpoK4DFzVN2keGFHAwNYCWfXvGSRVly8tSSfj8j8EsPi/KYbO24EDj1hzT/jg6n9CF3t27kpWZeUQxJTwR6DICjj8HvhjnPNm8ehacegsMug1SM2pfpiosfc2pRRTvgyH3wcCb62ciNcYHRDUK7dS1PajIZGAw0BLYCtyrqs+KyPnAI0AAmKiqfzmS8vv166cLFiyIVri1snHXfuas3s6cVdv5fM1OCoqCBJKEPjlNOaNTK844oRUntc0mKcmnD3Xt2QT/vgeWvurUlIbc59Q6avoQW8F2eOd2WDED2vZ1ahHWKZ8xMSciC1W1X8R1XiSKWAmrUVy3evVqr8OhpDTE1z/s5pNV25izagffbnLu1Gmekcqg41vysxNacfoJLTmqSbrHkcbA93Odu5N+XAI5p8KwB6ofK3rp6871jqK9cOZ/w8DfxvYCuTGmXINJFGW8rFFUZUdBEZ+t3sGcVduZs3o7OwqKAejSOosBHZuRlnKwaeWQ798SPnlwpuxLenpygCtOyaFlZloMoz8CoVL4+gWns739u6DvGDjrj5DR8tDt9u2Ad/4Llr8JbXo7tYijungTszENVINJFIlWo6hKKKQs35LPJ6ucZqqlm/YQcn8VGnbbaviv55DfVNhMcWmIDi0a8/zVA+jQ4giuCcTagd3wyf/BvAnONYvBd0P/ayGQAsvfgrf/03ku4sy74dRbrRZhjAcaTKIok6g1ilhZ9MNPXDNpPoEk4V9XDeCkdgk6Hsf2lfDeXbD2I2h1onMH04oZ0LqXU4s4uqvXERrTYFWVKHw1cFFDHeGuT04zXv3NqaQlB7hswlw+WbXd65Aia9UZrnwdRr/sPBOx8j048w9w7QeWJIxJYFaj8JGt+YVc9a/5rN66l7+N6sFFfdp5HVLlgsVw4CdocrTXkRhjaEA1iobu6Kx0pl5/CgNym/Of0xYz/pO1JOwXgeRUSxLG1BOWKHwmKz2Ff/26P8N7tOaBd7/jvreXEwolaLIwxtQLvrq9pK5PZvtFWnKAR0f35qgm6Uz8fD3b9hbx0KU9SUu2J5uNMbXnqxqFqs5Q1bHZ2Ql6108cJSUJfxzehf8+/0TeWbKFMRPnkV9Y4nVYxph6yFeJwhxKRBh7xnE8clkvFmz4iUvHz2VrfqHXYRlj6hlLFA3Az3u3ZeJV/dm4az8XPfEFa7YVeB2SMaYe8VWiaKjPUdTEGSe0Yur1AykKljJq/Bcs/D5CV+jGGBOBrxKFXaOoWve22bz2m1Np2iiFK575kg+Wb/U6JGNMPeCrRGGq16FFBq/+5lQ6H92EsS8sYMq8H7wOyRiT4OzJ7AZqX1GQG19axCerttO2aSOSA0JAhKSksPckCIgQSHJeSRGm05KTyEhLJjMtmYy0wMHp1ORDlmemJdM4LZnMVGc+OWDfUYxJJFU9me2r5yhMzWWkJfPMmH48OXstG3buIxRSgiElpEppSCkNUT4dUiVYqpSqUhwMUapavn1RMMS+oiAFRUH2FQWp6bN9aclJNElPpkl6Clll742SaZLmvh+yPMXdNpms9BSy0lPITE+mbOwnqemgSMaYI5LwiUJEjgV+D2Sr6iiv4/GTlEASt5wdvTGoVZ3EUZY0nPfSQxLJvmJnfl9RkL1FQfYWBsk/UMLewhJ+zC9kb2EJ+QeCHCgprXM8ZflDDlnmzAWShBS3dpQSSCI5ICQnlb2HTQeS3Hlnu7IaVXi5B/OURDzuwXlBBPfljCwiIiQJ5dMiznZJZdshJCW5e8uhZUqE41VMmhW3E8KOL2ExhW0rFeJJEuH4ozLp37E5rZok2JgnJi5imihEZCIwHNimqt3Dlg8F/okz5OkzqvpAZWWo6jrgGhF5NZaxmroTEdJTAqSnBOo8iFJJaYi9hcHyxLG3sIT8whLy3cSyr6gURasYr0MPW6Zh430EQ0ppqfNeUhqiNKSUlCrBUIhgSAmWhgi664OhECWlyoGSUoKlIUJ6cMyQ8jLLyy6bP7RqVTYbUmdPVSf2sumyMlVxX4cuc2pqWulx9JBYwn72sO21/NhOuSF1VpQft8I2kRzbKoMBHZszINd5tWvWOPKGxldiXaOYBIwDni9bICIB4HFgCJAHzBeR6ThJ468V9r9aVbfFOEaTgFICSTTPSKV5RqrXoTRYZcmsJBRi+eZ85q3fxfwNu5j57RamzN8IQNumjRiQ25z+bvI4rlWGNQX6UMwvZotIR+DtshqFiAwE/qSq57nzdwOoasUkUbGcV6tqehKRscBYgJycnL7ff/99VOI3xhwqFFJWbt3LvPW7mLd+F1+t38WOgiIAWmSklieNAbnN6dI6q7ypziS2RLuY3RbYGDafB5xc2cYi0gL4C9BbRO6uLKGo6gRgAjh3PUUvXGNMuKQkoUvrLLq0zmLMqR1RVTbs3M+89Tv5yk0e7y37EYAmacnktGhMciCJVPc6UEpyEinuNZ/w6eSAuyxQNu/sk5qcRFpygNTkJFIDSc67+0pzX6mBwGHLUwJJhFduIl2rOny5+zO6zajG4UWiiPT1otIPdlXdCdxQo4Kt91hj4k5EyG2ZQW7LDC7rnwPA5t0HmL/BqW1syy+kpNS5FlRSGuLAgVJK3GtAJaUhSkIhSoIH15ddKyop9fb73jFZ6XRrk0W3Nll0bZNNtzZZtGvWqEE2rXmRKPKA9mHz7YDNHsRhjImRNk0bMbJXW0b2anvEZagqxW7iKCoppbg0RHHQeRUFQxSXhigqCR2yvLi09OD6YKiScsOmw76jhi8vKQ2xelsByzfn8/HKbeW3fWc3SqFr67LkkUW3Ntkc1yrD988FeZEo5gOdRCQX2ASMBn7pQRzGmAQmIqQlB0hLhsw07+7kP1Bcync/5rNss/NavnkPL3z5PUVuIkpLTuLEY5qU1zq6tckip3ljmjZOjdv1GVUl/0CQXfuLyW2ZEfXyY3oxW0QmA4OBlsBW4F5VfVZEzgcewbnTaaKq/iWax7Uns40xsRQsDbFuxz6Wbd7Dsk1lSWQP+YXB8m2ShPI791pkpNEiM5WWmWm0yEilRWbZ/MF1mWnJ5c1aqsr+4lJ2FhSzc19R+fuOgmJ27StmZ0ERO/cVu/NF7NpXXN5Ut+p/hpGaXPsaTlUXs33VhUfYNYrrVq9e7XU4xpgGRFXJ++kAy7fk8+OeQnYWFLGj7EO9oNj9YC9ib1gyCZcaSKJFZipJIuzcV0RhSeSms4zUQHmiaRGWhFpkptEyM5Vh3VtboqgJq1EYYxJVUbDUrRU4yaMskezYV8SOvcWoKi2bpLk1EbcWknmwZtIoNTZ3YyXa7bExY3c9GWMSXVpygNbZjWid3cjrUGrMV5fqbTwKY4yJPl8lCmOMMdHnq0RhQ6EaY0z0+SpRWNOTMcZEn68ShTHGmOjzVaKwpidjjIk+XyUKa3oyxpjo81WiMMYYE32+fDJbRLYDRzpyUUtgRxTDiRaLq3YsrtqxuGrHj3F1UNVWkVb4MlHUhYgsqOwxdi9ZXLVjcdWOxVU7DS0ua3oyxhhTJUsUxhhjqmSJ4nATvA6gEhZX7VhctWNx1U6DisuuURhjjKmS1SiMMcZUyRKFMcaYKjXYRCEiQ0VkpYisEZG7IqxPE5Gp7vqvRKRjHGJqLyIfi8gKEVkmIrdG2GawiOwRkW/c1z2xjss97gYR+dY95mHDB4rjUfd8LRGRPnGIqXPYefhGRPJF5LYK28TlfInIRBHZJiJLw5Y1F5F/i8hq971ZJfuOcbdZLSJj4hDXgyLynft7ekNEmlayb5W/8xjE9ScR2RT2uzq/kn2r/N+NQVxTw2LaICLfVLJvLM9XxM+GuP2NqWqDewEBYC1wLJAKLAa6VtjmRmC8Oz0amBqHuFoDfdzpJsCqCHENBt724JxtAFpWsf584F1AgFOArzz4nf6I89BQ3M8XcAbQB1gatuxvwF3u9F3A/0XYrzmwzn1v5k43i3Fc5wLJ7vT/RYqrJr/zGMT1J+COGvyeq/zfjXZcFdb/A7jHg/MV8bMhXn9jDbVGMQBYo6rrVLUYmAKMrLDNSOA5d/pV4GwRkVgGpapbVHWRO70XWAG0jeUxo2gk8Lw6vgSaikjrOB7/bGCtqh7pE/l1oqpzgF0VFof/DT0H/DzCrucB/1bVXar6E/BvYGgs41LVWaoadGe/BNpF63h1iauGavK/G5O43P//S4HJ0TpeTVXx2RCXv7GGmijaAhvD5vM4/AO5fBv3n2oP0CIu0QFuU1dv4KsIqweKyGIReVdEusUpJAVmichCERkbYX1Nzmksjabyf2AvzhfA0aq6BZx/dOCoCNt4fd6uxqkJRlLd7zwWbnabxCZW0ozi5fk6HdiqqqsrWR+X81XhsyEuf2MNNVFEqhlUvE+4JtvEhIhkAq8Bt6lqfoXVi3CaV3oCjwFvxiMm4DRV7QMMA24SkTMqrPfyfKUCFwKvRFjt1fmqKS/P2++BIPBSJZtU9zuPtieB44BewBacZp6KPDtfwOVUXZuI+fmq5rOh0t0iLKvVOWuoiSIPaB823w7YXNk2IpIMZHNkVeVaEZEUnD+El1T19YrrVTVfVQvc6ZlAioi0jHVcqrrZfd8GvIHTBBCuJuc0VoYBi1R1a8UVXp0v19ay5jf3fVuEbTw5b+4FzeHAFeo2ZFdUg995VKnqVlUtVdUQ8HQlx/PqfCUDFwFTK9sm1uerks+GuPyNNdREMR/oJCK57rfR0cD0CttMB8ruDhgFfFTZP1S0uG2gzwIrVPWhSrY5puxaiYgMwPkd7oxxXBki0qRsGudi6NIKm00H/kMcpwB7yqrEcVDpNz0vzleY8L+hMcBbEbZ5HzhXRJq5TS3nustiRkSGAr8DLlTV/ZVsU5PfebTjCr+m9YtKjleT/91YOAf4TlXzIq2M9fmq4rMhPn9jsbhCXx9eOHfprMK5g+L37rL7cP55ANJxmjLWAPOAY+MQ0yCcKuES4Bv3dT5wA3CDu83NwDKcuz2+BE6NQ1zHusdb7B677HyFxyXA4+75/BboF6ffY2OcD/7ssGVxP184iWoLUILzDe4anGtaHwKr3ffm7rb9gGfC9r3a/TtbA/w6DnGtwWmzLvsbK7u7rw0ws6rfeYzjesH921mC8wHYumJc7vxh/7uxjMtdPqnsbyps23ier8o+G+LyN2ZdeBhjjKlSQ216MsYYU0OWKIwxxlTJEoUxxpgqWaIwxhhTJUsUxhhjqmSJwtQbbu+id3gdR1VE5ES399CvReS4OpY1SURGRSu2Wh57plTSq2zYNhvi+PCi8ZAlCtPgiEgghsX/HHhLVXur6toYHiemVPV8Vd3tdRwmMViiMAlNRH7vjj3wAdA5bPlxIvKe2wHbpyJyYtjyL0VkvojcJyIF7vLBbn/+L+M81IWIXCki89wawFNlCUREzhWRuSKySERecfvXqRhXL/c4ZWM6NBNn/ITbgGtF5OMI+xSIyD/ccj8UkVaVlVVhv7NF5I2w+SEi8npYmX8Rp9PDL0XkaHd5B/cYS9z3HHf5JBF50j0X60TkZ+J0wLdCRCaFHaO8tiAib7rneZnEr3NAk0ii+fSgvewVzRfQF+dDvTGQhfNU6R3uug+BTu70yThdrAC8DVzuTt8AFLjTg4F9QK473wWYAaS4808A/wG0BOYAGe7y3xFh/AGcJ2R/5k7fBzziTv+JSsZUwHmy9gp3+h5gXDVlTcLpPkaA74BW7vKXgRFhZZZN/w34gzs9AxjjTl8NvBlW5hS3zJFAPnASzpfGhUAvd7sNuGMrcPBp30Y43VK0qLiNvfz9shqFSWSnA2+o6n51esqcDuU9aJ4KvCLOaGNP4QzsAjCQg73IvlyhvHmqut6dPhsnEc13yzgbpxuGU3AGhPncXT4G6BBeiIhkA01V9RN30XM4A95UJ8TBTuVeBAbVpCx1PpVfAK50rxsM5GDX4MU4yRGcD/qO7vTAsJ//BZwuIMrMcMv8Fqfb7G/V6YhvWdj+4W4RkbIuUNoDnWrwsxofSfY6AGOqEamPmSRgt6r2qmVZ+8KmBXhOVe8O30BERuAM8nJ5Lcs+ErXpP+dfOLWEQuAVPTjwUIn7oQ9QSuX/0+HHKnLfQ2HTZfOH7C8ig3E6xBuoqvtFZDZOP2imAbEahUlkc4BfiEgjt2fOEeB0HQ6sF5FLoHy87p7uPl8CF7vTo6so+0NglIgc5ZbRXEQ6uPufJiLHu8sbi8gJ4Tuq6h7gJxE53V30K+ATqpeE05QE8Evgs5qWpU4X1puBP+A0H1XnCw7+/FcAn9Vgn0iygZ/cJHEiTo3LNDBWozAJS1UXichUnJ4yvwc+DVt9BfCkiPwBSMFpd1+MczH5RRH5L+AdnJEJI5W93N13logk4fQWepOqfikiVwGTRSTN3fwPOL2VhhsDjBeRxjhjEP+6Bj/SPqCbiCx047qslmW9hHOdYnkNjnULMFFE7gS21zC+SN4DbhCRJcBKnERqGhjrPdb4ivthe0BVVURG41zYjtqYynUhIgWqetgdVLXYfxzwtao+G8WwjKmW1SiM3/QFxomIALtx7vip99xayD7gv7yOxTQ8VqMwxhhTJbuYbYwxpkqWKIwxxlTJEoUxxpgqWaIwxhhTJUsUxhhjqvT/AdIKGDLO18n+AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "## EDIT THIS CELL\n",
    "K_max = 20\n",
    "rmse_train = np.zeros((K_max+1,))\n",
    "rmse_test = np.zeros((K_max+1,))\n",
    "\n",
    "for k in range(K_max+1):\n",
    "    \n",
    "    # feature matrix\n",
    "    Phi = poly_features(X, k)\n",
    "    \n",
    "    # maximum likelihood estimate\n",
    "    theta_ml = nonlinear_features_maximum_likelihood(Phi, y)\n",
    "    \n",
    "    # predict y-values of training set\n",
    "    ypred_train = Phi @ theta_ml\n",
    "    \n",
    "    # RMSE on training set\n",
    "    rmse_train[k] = RMSE(y, ypred_train)    \n",
    "    \n",
    "    # feature matrix for test inputs\n",
    "    Phi_test = poly_features(Xtest, k)\n",
    "    \n",
    "    # prediction\n",
    "    ypred_test = Phi_test @ theta_ml\n",
    "    \n",
    "    # RMSE on test set\n",
    "    rmse_test[k] = RMSE(ytest, ypred_test)\n",
    "    \n",
    "\n",
    "plt.figure()\n",
    "plt.semilogy(rmse_train) # this plots the RMSE on a logarithmic scale\n",
    "plt.semilogy(rmse_test) # this plots the RMSE on a logarithmic scale\n",
    "plt.xlabel(\"degree of polynomial\")\n",
    "plt.ylabel(\"RMSE\")\n",
    "plt.legend([\"training set\", \"test set\"]);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Questions:\n",
    "1. What do you observe now?\n",
    "2. Why does the RMSE for the test set not always go down?\n",
    "3. Which polynomial degree would you choose now?\n",
    "4. Plot the fit for the \"best\" polynomial degree."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEGCAYAAABo25JHAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nO3dd3xUVfr48c9J6IQaQigJhA4SIMEACgJBQFERLCiouLh8WdvqIqu4rGXXXcuX309/YFsLiqIrigqCKDZAEbAAoSMlFIOEmgCBEFqSOb8/TiYGSEJIZubcmfu8Xy9eFyYz9z4Dwzz3tOcorTVCCCHcK8x2AEIIIeySRCCEEC4niUAIIVxOEoEQQricJAIhhHC5SrYDKI8GDRrouLg422EIIURQWblyZabWOursx4MyEcTFxZGSkmI7DCGECCpKqZ3FPS5dQ0II4XKSCIQQwuUkEQghhMs5ZoxAKRUOpAC7tdaDbccjRCDk5uaSnp7OyZMnbYciQki1atWIiYmhcuXKZXq+YxIBMBbYBNS2HYgQgZKenk6tWrWIi4tDKWU7HBECtNYcPHiQ9PR0WrRoUabXOKJrSCkVA1wDvGk7FiEC6eTJk0RGRkoSED6jlCIyMvKCWpmOSATA88DDgKekJyil7lRKpSilUjIyMgIXmRB+JklA+NqFfqasJwKl1GDggNZ6ZWnP01pP0Vonaa2ToqLOWQ9RNqlfw5JJ5XutEEKEKOuJAOgFDFFKpQEzgMuVUu/55Uo7FsHi50D2YBCiWE888QTPPfdciT+fM2cOGzduDGBEIhCsJwKt9d+11jFa6zhgBPCt1nqkXy5WJxZyc+DEYb+cXohAmTw/1cp1JRGEJuuJIKDqxJjjkV124xCigl5YuNVn53r66adp164dAwYMYMuWLQC88cYbdOvWjS5dunDjjTdy/PhxfvzxR+bOncv48eNJSEhg+/btxT5PBB9HJQKt9SK/riGoG2uOWZIIhABYuXIlM2bMYPXq1XzyySesWLECgBtuuIEVK1awdu1aOnTowNSpU+nZsydDhgzh2WefZc2aNbRq1arY54ng46R1BP5XpyARHEm3G4cQ5TB5fuoZLYG4CfMAGNu/DeMGti3XOZcsWcL1119PjRo1ABgyZAgAGzZs4LHHHiMrK4tjx45x5ZVXFvv6sj5POJu7EkGNSKhUXbqGRFAaN7Bt4Rd+3IR5pE28xifnLW6q4R133MGcOXPo0qUL06ZNY9GiRcW+tqzPE87mqK4hv1PKjBNIIhACgD59+jB79mxOnDhBdnY2n332GQDZ2dk0btyY3Nxcpk+fXvj8WrVqkZ2dXfjnkp4ngou7EgGYcQIZIxBBbmz/Nj45T9euXRk+fDgJCQnceOON9O7dG4Ann3ySHj16MHDgQNq3b1/4/BEjRvDss8+SmJjI9u3bS3yeCC5KB+Gc+qSkJF3ujWnm3g9bvoLxvpt1IUR5bdq0iQ4dOtgOQ4Sg4j5bSqmVWuuks5/rvhZBnWaQcwBypdqjEEKAKxNBwVqCo7vtxiGEEA7hvkRQuJbgN7txCCGEQ7gvERSuLpa1BEIIAW5MBLWbAkqmkAohRAH3JYLwylCrsUwhFUKIAu5LBGDGCaRFIITfzJ07l4kTJ1q7fnJyMt4p5ldffTVZWVmkpaURHx9f5nMUfX5KSgp/+ctfgPOX6i6vks6bkZFBjx49SExMZMmSJYXvJysri1deecUn13ZXiQmvOrGwu5zrEIQQ5zVkyJDCukW2ffHFFwBkZWWV+xxJSUkkJZ0z/T4gFi5cSPv27XnnnXcAChf9paWl8corr3DvvfdW+BrubBHUiYEju8FT4s6YQrhCWloa7du3Z8yYMcTHx3PbbbexYMECevXqRZs2bVi+fDkAy5cvp2fPniQmJtKzZ8/CctWTJk1i9OjRAKxfv574+HiOHz/OtGnTuO+++wBTj+iee+6hX79+tGzZku+//57Ro0fToUMH7rjjjsJYIiIiCn8/c+bMwp+V9fUliYuLIzMz84zHduzYQWJiIitWrCA/P5/x48fTrVs3OnfuzOuvv37OORYtWsTgwb8XRt64cSPJycm0bNmSF198sfDxSZMmER8fT3x8PM8///x5Hy+uBHhRa9as4eGHH+aLL74gISGBEydOFL6fCRMmsH37dhISEhg/fvx5/x5K484WQd1Y8OTCsf1Qu7HtaIQwvpwA+9b79pyNOsFVpXfRbNu2jY8//pgpU6bQrVs33n//fZYuXcrcuXN55plnmDNnDu3bt2fx4sVUqlSJBQsW8MgjjzBr1iweeOABkpOTmT17Nk8//TSvv/56YSXTog4fPsy3337L3Llzufbaa/nhhx9488036datG2vWrCEhIaHUGCv6+qK2bNnCiBEjePvtt0lISGDKlCnUqVOHFStWcOrUKXr16sUVV1xR6r6/mzdv5rvvviM7O5t27dpxzz33sG7dOt5++22WLVuG1poePXrQt29fPB5PiY97S4Dn5eXRtWtXLr744jOuk5CQwL///W9SUlJ4+eWXz/jZxIkT2bBhA2vWrCnzey+JOxNB0XLUkgiEy7Vo0YJOnToB0LFjR/r3749Sik6dOpGWlgbAkSNHGDVqFFu3bkUpRW5uLgBhYWFMmzaNzp07c9ddd9GrV69ir3HttdcWnjM6OvqM66WlpZ33i7yir/fKyMhg6NChzJo1i44dOwLwzTffsG7dOmbOnFn4Xrdu3UrbtiWX9r7mmmuoWrUqVatWpWHDhuzfv5+lS5dy/fXXU7NmTcDs6bBkyRK01sU+7vF4ii0BboPLE8FvENvNbixCeJ3nzt1fqlatWvj7sLCwwj+HhYWRl5cHwOOPP06/fv2YPXs2aWlpJCcnF75m69atREREsGfPnvNeo+j5z75G0TvwkydPXvDry6JOnTrExsbyww8/FCYCrTUvvfTSOXspeJNgae8HIDw8nLy8PEqq21ZaPbfSWh2B5N4xApBFZUKU0ZEjR2jatCkA06ZNO+PxsWPHsnjxYg4ePFh4V10e0dHRbNq0CY/Hw+zZsysacrGqVKnCnDlzePfdd3n//fcBuPLKK3n11VcLWzmpqank5ORc8Ln79OnDnDlzOH78ODk5OcyePZvevXuX+nhxJcDL6uyS4BXhzhZBtdpQrY6sJRCijB5++GFGjRrFpEmTuPzyywsfHzduHPfeey9t27Zl6tSp9OvXjz59+pTrGhMnTmTw4MHExsYSHx/PsWPHfBX+GWrWrMnnn3/OwIEDqVmzJmPGjCEtLY2uXbuitSYqKoo5c+Zc8Hm7du3KHXfcQffu3QEYM2YMiYmJACU+7i0B3rx588LZQGUVGRlJr169iI+P56qrruLZZ5+94Ji93FeG2uvVXqaL6NYZvglKiHKQMtTCX6QMdVnUkUVlQggBrk4EMdI1JIQQuDkR1G0Gp47AifKvNhTCF4Kxe1Y424V+ptybCOrFmWPWTqthCHerVq0aBw8elGQgfEZrzcGDB6lWrVqZX+POWUMA9Zqb4+E0aNzFaijCvWJiYkhPTycjI8N2KCKEVKtWjZiYmDI/38WJIM4cD0uLQNhTuXJlWrRoYTsM4XLu7RqqVgeq1TUtAiGEcDH3JgIwrQIZIxBCuJzLE0FzaREIIVzP5YkgDrJ+k30JhBCuZj0RKKWqKaWWK6XWKqV+UUr9K2AXr9sc8k9D9t6AXVIIIZzGeiIATgGXa627AAnAIKXUJQG5sqwlEEII+4lAG94yg5ULfgVmdU3hFNK0gFxOCCHK7dQx2LoAThz2+amtJwIApVS4UmoNcACYr7VeVsxz7lRKpSilUny2+KZOLKAkEQghnG//Bph+I+xa4fNTOyIRaK3ztdYJQAzQXSkVX8xzpmitk7TWSVFRUb65cKUqpvicLCoTQjhdZqo5Nmjj81M7IhF4aa2zgEXAoIBdtK5MIRVCBIHMVAivagpm+pj1RKCUilJK1S34fXVgALA5YAHIojIhRDDI3AqRrSEs3OendkKtocbAO0qpcExi+khr/XnArl6vuZk+mnsCKlcP2GWFEOKCZKZCo85+ObX1RKC1XgckWgugcArpLohqay0MIYQoUd4p04UdP8wvp7feNWRd3SLlqIUQwokO7QDtgQb+uVmVRCCLyoQQTufHGUMgiQAiGkKl6tIiEEI4lzcRRLb2y+klESglVUiFEM6WuRVqx0DVCL+cXhIBmO4hWVQmhHCqzFS/dQuBJALDu6hMNhAXQjiN1qZF4KeBYpBEYNSLg9PZcPyQ7UiEEOJM2Xvh9DFpEfhd/ZbmeGi73TiEEOJs3oHiqHZ+u4QkAoDIVuZ4UBKBEMJhMreao3QN+Vnd5qDCpEUghHCezFSoWhsiov12CUkEYMpR120uLQIhhPN4Zwwp5bdLSCLwimwlLQIhhPP4ecYQSCL4Xf1WcHCHTCEVQjjHqWw4utuvM4ZAEsHvIluZKaQ5PtoGUwghKurgNnOUFkGA1PfOHNpmNw4hhPDK2GKOkggCJLJgLYEMGAshnOLARgiv8vtaJz+RROBVpxmEVZIBYyGEcxzYZFoD4ZX9ehlJBF7hlUypCWkRCCGc4sBmiGrv98tIIiiqfiuzE5AQQth2KhuO/AYNO/j9UpIIioosSAQyhVQIYZt3oFgSQYDVbwm5x021PyGEsOnARnOURBBgUnxOCOEUBzaZbXTrxvn9UpIIivKuJZCZQ0II2w5sMqWnw/z/NS2JoKg6MWbOrrQIhBC2HdgEDS8KyKUkERQVFg71WkgiEELYdfwQHNsXkPEBkERwLqlCKoSwLWOzOUoisCSyFRz6FTwe25EIIdwqgDOGQBLBueq3gvxTcGSX7UiEEG51YLPZlax204BcThLB2bxV/rz7hAohRKAd2GRKS/hxV7KiJBGcLaqdOWZusRuHEMKdtDZdQwHqFgIHJAKlVKxS6jul1Cal1C9KqbFWA6rZAKrX/315txBCBFJOBpw4FNBEUClgVypZHvCg1nqVUqoWsFIpNV9rvdFaRFHtzIbRQggRaAEeKAYHtAi01nu11qsKfp8NbAICM0JSkgZtzfQtKT4nhAi0/d5EEJjFZOCARFCUUioOSASWFfOzO5VSKUqplIwMP+8rHNUOThyGnEz/XkcIIc62bz1ERENEw4Bd0jGJQCkVAcwCHtBaHz3751rrKVrrJK11UlRUlH+DkQFjIYQt+9dDo04BvaQjEoFSqjImCUzXWn9iOx4aFCQCGTAWQgRS3mmzhiA6PqCXtZ4IlFIKmAps0lpPsh0PYIrPVa4pA8ZCiMDK3AKeXFe2CHoBtwOXK6XWFPy62mpESkGDNtIiEEIE1r4N5tioc0Ava336qNZ6KRCY5XMXIqod/LrEdhRCCDfZt95sRuPdJCtAnNAicKaodpC9B06eM24thBD+sW8dRF9kSuIHkCSCkngHjKXmkBAiELSG/RsCPj4AkghKJlNIhRCBdHS3Wb8kicBB6rWAsMoyYCyECIx9680xWhKBc4RXMgM2MoVUCBEI+zYAyowRBJgkgtJEtft9yzghhPCnfeugfkuoWivgl5ZEUJoG7eBwGuSetB2JECLU7VsPjQK7othLEkFpotqB9sDBbbYjEUKEslPZcPhXKwPFIImgdN564Ac22Y1DCBHa9v9ijgFeUewliaA0kW3MzKH9G2xHIoQIZYUzhqRryHkqVTHdQwfsbZYmhHCBPauhZhTUbmLl8pIIzie64+/NNiGE8Ic9q6FJV1Pw0gJJBOcT3dGs+Dt+yHYkQohQdDrHTFNvkmgtBEkE5xPd0Ryle0gI4Q9715nZiZIIHMw7eCPdQ0IIf9iz2hwlEThYRDTUiJSZQ0II/9izCmo3hVrR1kI4byJQSi1QSnUJRDCOpJQMGAsh/GfPaqutAShbi+BhYLJS6m2lVGN/B+RI0fFmUZkn33YkQohQciLLVC5okmA1jPMmAq31Kq315cDnwFdKqX8qpar7PzQHie4IucdN3SEhhPCVvWvNsUlXq2GUaYxAKaWALcCrwP3AVqXU7f4MzFEaFpSFlXECIYQvOWCgGMo2RrAU2A1MBpoCdwDJQHel1BR/BucYUe1Bhck4gRDCt/asgrrNoUZ9q2FUKsNz7gZ+0Vrrsx6/XynljmpsVWpA/VaSCIQQvrVnNTS92HYUZRoj2FBMEvC6xsfxOJfMHBJC+FJOJmT9Zr1bCCq4jkBrvcNXgThedLypF37qmO1IhANMni9bmIoK2rPGHIM9EbiKlJoQRbywcKvtEESw27PKHBvbX6YliaCsvDsHead7CSFERexabiaiVKtjO5IyDRYLgDoxUKPB78054TqT56ee0RKImzAPgLH92zBuYFtbYYlg5PFA+nK4aKjtSABJBGWnlFn9t1cSgVuNG9i28As/bsI80ia6Z66E8LGDW+HkEYjtYTsSQLqGLkyTRFNqIveE7UiEEMFs1zJzlEQQhJokgs6HfbLC2O3G9m9jOwQRzHYtg+r1ILK17UgAhyQCpdRbSqkDSilnf8M2LigM5V0WLlxLxgREhexabloDlramPJsjEgEwDRhkO4jzqt0EajaUcQIhRPkdPwSZqRDTzXYkhRyRCLTWiwHnbwrsHTCWFoEQorzSV5ijQ8YHwCGJoCyUUncqpVKUUikZGRn2AmmSaDaaPp1jLwYhRPDatRxUODS1W3q6qKBJBFrrKVrrJK11UlRUlL1AGieYjaZlwFgIUR67lpkFqlVq2o6kUNAkAsfw1gWR7iEhxIXKz4PdKx3VLQSyoOzC1W4MEY1kwNhtTueY8iJ715qKkUd3Q/Y+yDsFFBTnrVobajWGWo3MtMAmidCwA4RXthq6cJD9G8xuh7HdbUdyBkckAqXUB5jNbhoopdKBf2qtp9qNqhQyYBz6tDbJfvM8SP3KlCDXHvOzKhG/f+FXq/v7a05mmWZ/9j7IP2UeC69qZoe0u8r8imwV+PcinGPXcnOURHAurfUttmO4IE0SYes3piR11Qjb0QhfOroXVr0Dq9+DI7vMznTNekLvh8wGIk27QkTD0s/h8ZiS5XtWm1/bv4NvHjW/ouPh4jug001QvW7p5xGhZ9fP5iaiTqztSM7giEQQdAoHjNdD80ttRyN8Yc9qWDoZNn1u/m1b94d+j0CbK6Fm5IWdKyzM3PlHtoJOw8xjh3fyxHPP8UTYOvjiIfjmceh8M1w2Duq38P37Ec6jNfy6BFr2dcxCMi8ZLC4P77Sv3Sl24xAVt28D214cClOSYcf3cOmf4S+rYOQsSLj1wpNASeo1Z1r+ILjre7hzkUkCa2fASxfDnD/DoeL3eJINcEJI5lbIOQBxl9mO5BySCMojoiHUi/u9cJQIPjkH4dP74LVeNDy4DJIfgQfWwxVPQv2WPrvM5PmpxE2YV1iyOm7CPOJe3MPk6vfB2LXQ/U7YMBNe7g7z/wGnss94vWyAE0LSlphjXG+7cRRDuobKK7YH7FhkmnsOa+aJUng8sPpdWPCE+dLteT+XfduJdcnD/XK585auvmoiXPYALHwSfnjBtBKueMqMIcjnKrSkLYVaTXx6o+Er0iIor9gecGw/ZO20HYkoq6xd8O4Q+Gws6ZXjGHjiGeK+vZSjRBTetVvpiqnVCK77D4xZCLWbwid/YsE/+tFtwnsAdmMTvqG1SQRxlzkywUuLoLy8C0J+W2a6iYSzrfsY5j1oyogPeYmYxNuZX/AfMlCbzJy3dHVMkkkGy15jwMJ/MaDmo/zl6EhefOZpv8cm/Cwz1bHjAyAtgvJr2MEsIJJxAmfLPQGz74FPxkDD9nD3Uuj6Byt3ZWUqXR0WBpfea+Js0IYXq7xsxjJkM6Tg5h0faOG88QGQRFB+YeHmDs67QEQ4z+E0mHoFrP0A+k6AO74odqqmIzeZadAGRn/NzzGjYfV/4c0BkLnNdlSivNKWmm6/es6cKiyJoCJie8CBX+DkUduRiLNt/85MCc3aCbd+CP3+DuHF94Q6dpOZsHAuGTMZbpsFR/fAlL6w5UvbUYkL5fDxAZBEUDGx3c3iI1lP4Cyr/gvTh5kVnH/6DtpeaTuiimkzAO5eYuoXfXAL/PCi+XIRwSFjC+RkOHLaqJckgopommRKEEj3kDNoDd8+BXPvgxZ9YPTXoVPbp04M/PFLuGgIzH/cjBvknbYdlSiLwvUDzhwoBkkEFVOtNjTsCL/9bDsSkZ8Hn/4ZFj8LibfDrR+Zf59QUqUGDJsGfR6GNe/BB8NNvSvhbL8uhtoxjp5dKImgomK7Q3oKePJtR+Jeeadg5h9hzXR+ir0ThrwUuqWfw8Lg8kdhyMtmQeM710JOpu2oREny80zpklbJjh0fAEkEFRfbA05nw4FNtiNxp9PHYcatsGkuXPm/3LI12dH/4Xym6+0wfDoc2AhvXWkWywnn2b0STh2B1gNsR1IqSQQV1cy7sOwnu3G40ekceP9m2LbQtAIuvdd2RIHV/mq4fQ4cy4BpV5vpssJZti0w44gtk21HUipJBBVVt7np//MOCInAyD0BH4yAnT/wRdt/E/dR5JmF3dxSkqH5pTDqU1M36e2r4eB22xGJorYtMBsTVa9nO5JSSYmJilLKzFBJ/coUNAuT3Op3uSdNd9CvS+D617i6ywjSbjU/ClS5CEdpkgijPoN3h5pkMOoziHLo2gg3yTlo9rlI/rvtSM5LvrV8oWVfOHHI7EcqfKbYO/r8XPjoD7D9Wxj6MnQZEfjAnKhRJ7NyWuebwnol7G8gAmjHd4B2/PgASCLwjRZ9zPHXxXbjCDHn1OL3eGDOPbD1axg8GRJHnvMaR5aLCJSG7eEPc80sqneGQNZvtiNyt20LoHp9s8e5w0ki8IXaTSCyDfz6ve1IQpfW8NXfYP3H0P+fkDS62Kc5tlxEoERfBH+YA6eOmqmlR/fYjsidPB4ziaFVP1OXzOEkEfhKiz6w80fTdSHKrdgdvSbM46e3xsPyKdDzfrPPryhZ4y4wcrbpo/7v9XD8UKlPd8WgeqDt32DKTgdBtxBIIvCdln3h9DHYvcp2JEFt3MC2pE28pnDAN23iNaTdlMGlu96AhJEw8El3rBOoqJiL4ZYP4NCvMP2mUlcgy3aYfrBtgTm2utxuHGUkicBXvAWlZJzAt1K/gc//Cq0HwrXPSxK4EC16w7C3YM8q+HCkGTsQgbFtAUR3MrvPBQFJBL5So76ZuSHjBD7zTPdc+HgUNIqHm6aFbtkIHzqnm6fDYLPYbsd3ZqDd4yl8XnFdcNJN5AM5mWaBafurbUdSZrKOwJda9IXlb5jFTpWr244muGX9xq3bx0ONBnDrx1A1wnZEQeGFhVvPHTBPHGnKIC94wkxsuOIpxg1sW/g8V6698KctX5jy9O2D5+9UWgS+1KIv5J+S7Ssr6lQ2vD/CdGXc9jHUirYdUfDr9QB0+xP8+BL8/JrtaELb5nlQpxk06mw7kjKTFoEvNe8JYZXM7lgtk21HE5w8+TDzfyBjM4ycaebGi1JNnp96xoCvt7tnbP82v7cOlIKr/g9k74WvJkDtxnDR0MLnCR85lW3+/3f7n6Aaz5JE4EtVI0wySP0aBv7LdjTB6ZvHzIKxayYFzYwL28rczRMWDje+aUpRfHIn1GoCsd1k7YUvbVtgegXaD7YdyQWRriFfazsIMjZJJcjyWPkO/PwK9Ljb3FEJ36tcHUZ8YLbx/GCEmV4qfGfT51AjEppdYjuSCyKJwNfaDjLH1K/txhFsdv4E8x40rYArnrYdTdAqUzdPzUi4bSZ48kwZ7xOH/R+YG+Sdhq3fQLurgmI1cVGOSARKqUFKqS1KqW1KqQm246mQyFam3MSWL21HEjyydpl57nWbmXnv4dJjWV5l7uZp0BpGvG9aBB/eLivifeHXxaa0R/trbUdywawnAqVUOPAf4CrgIuAWpdRFdqOqoHaDIG2pGTgSpfPuMJZ/Gm6Z4fi67SElrpep4Jq2xLTGtLYdUXDb/BlUiQjKiSLWEwHQHdimtd6htT4NzACGWo6pYtpeBZ5cUypZlExrmHs/7FsPN06VGvo2dBkBvR+EVQXjM6J88nPN+EDrAVC5mu1oLpgTEkFToOiGq+kFj51BKXWnUipFKZWSkZERsODKJbYHVKsLW76yHYmz/fgSbJgJ/R+HtlfYjsa9+j0GHYbA14/KZ7a8diyC45nQ+WbbkZSLExJBcZNtz2mjaq2naK2TtNZJUVFRAQirAsIrQZuBZuDIk287Gmfa/i0s+KeZy37ZX21H425hYXD9a6Zq6az/gf2/2I4o+Kz70HRrth5oO5JycUIiSAdii/w5Bgj+IuptB5k7hN0rbUfiPIfTYOZoiGoPQ18JqoU3IatKTVOttEqEmVaak2k7ouBxKtt0C3W8HipVsR1NuTghEawA2iilWiilqgAjgLmWY6q41v1BhZu6I+J3p4/DjJGmFsvw96SGkJPUbgK3vA/HDhRUKz1tO6LgsHke5J2AzsNtR1Ju1hOB1joPuA/4GtgEfKS1Dv62afV6ZrOaX2bLbAwvreGzsWbTjhunmqm2wlmaXgxD/2OqZ84bJ5/dslj3oZn6HNvDdiTlZj0RAGitv9Bat9Vat9Jah85qok7DTDeIdA8Zy16H9R9Bv0fMGIpwpk7DoM94WP2ezCQ6n+z9ZqC4081B3cXpiEQQstoPhvAqsH6m7Ujs2/kjfPMotLsaej9kOxpxPsmPQIdrC2o/LbAdjXNtmGW6OYN0tpCXJAJ/ql4X2lwBv3zi7tlDR/fAR6M4VKWJmZ0SJh87xwsLg+tfh4YdYeYfIUM2rCnWuhnQOAGi2tmOpELkf6S/dRoGx/ablcZulHcKPvoDnM7h5iP3QbU6tiMSZVWlphk8rlQVPhgOxw/ZjshZdq+CvWsh4TbbkVSYJAJ/azvITMnb4NLuoa/+Dukr4LpX2KZjbEcjLlTdZjB8OhxJh4/vkJpERaVMhco1oUvwzhbykkTgb5Wrmy3rNs513+bhq9+DlKm8lnctcf81+w3L3rhBqFkPuPYFsx/3V8FdE9JnThyG9bOg800h0cqVMo+BED/MTDHbtjCoNrSukN0r4fO/Qou+3D1yGneHV5K9cYNZwq1wYBP8+KJZCNj9T7YjsmvNB2btQFJo7JshLYJAaNUPqtc3ycANjmWY0jfPgW8AAA0rSURBVMYR0TDsbSkrHSoGPGG6Or/8m9mO0a08Hg5//yrEdIfGwbMvcWkkEQRCeGXocgts/tzMOw5l+bmmL/n4QRjxntkEpYDsjRvkwsLhhjfMDJmPRrl3JtGv31Pv5G/QbYztSHxGEkGgJI02O0Ktftd2JP71zWOwc6npU27c5Ywfyd64IaBabbNvRHhls7uZG2cSpUzlkI4wBRNDhCSCQGnQmiX58ZAyLXTXFKyeDstegx73mDr3IjTVa252Nzu6x3QBuqQm0eT5qfT++1vkbZzHR/n9iHt8YchMfJBEEEDv5Q+Eo+mhuZ9x+kr4fJypr3TFU7ajEf7WrIepSbRzqakf5YKaROMGtmVJz7VUqlSJqXmDSJt4DWkTrwmJlq6M4vnZ5PmpvLBwKwDhdGWvrs+W6RNZ3ad1SHyAAMjeBx/eBrWiYdg0GRx2i843waEdsOgZqN8S+o63HZF/Hd0Da96HxJFkLA2tLVWlReBn4wa2LbxzyCecxv3uJjl8LeMuDpEvy9yTMOM2OHnEdBcUGRwWoauwO6Tvw9B5BHz3FKz72G5Q/vbjS6Zbt9fYkJv4IIkg0Lr+wexTkPKW7UgqTmuYex/sTjF1aRp1sh2RCBBvKxelYMiL0Pwy+PRe+HWJ3cD8JScTVk4zxeXqxYVOa76AJIIAGtu/DdRubGYbrHzHrE4MZkv+H6z/GC5/DC4aYjsaYUulqjD8v1CvhWkd7t9oOyLf+/lVyD0Bl42zHYlfSCIIoMK7iD4Pwamj5sMVrDbOhW+fhE43SVlpl5g8P7WwRAicVS6kRn0YOdOUVJk+DI7sthytDx0/BMvfMGW5g7zKaEkkEdgQ3RE6DDGJoJRWgWOnpaWnwCd/gphuMOTloN6QQ5Rd0fEu4NxZM3WbmWRw8qhJBsHe4vVaNBFOZ0Py321H4jeSCGzp+7eCVsFrJT6lsB/WSQ7tgPeHQ61GMOIDqFzNdkTCSRp1MivKM7fC9JvhdI7tiComIxVWvAldR0H0Rbaj8RtJBLY0ijc7mP38KpzIsh1N2Rw/BNNvAp0Pt82CiCjbEQlLSp010zIZhk01kwg+HBncVXe/eczsy9DvUduR+JUkApv6/g1OHTF7+RYotR/WptM58MEIyNplWgINWtuNR1h13lkzFw2Fa1+E7d+absRgXE2//VvY+jX0fjDkb3qUDsIVgUlJSTolJcV2GL4x4zaz+fV9K6B2kzN+5JiyzXmnYcYt5j/GTdNCqsaK8LMfXzZ7VXceAde9YgrXBYP8PHi9t7kBum+FmRkVApRSK7XWSWc/Li0C2654yhSj+8qhA1EeD8y5B7YtgMHPSxIQF6bnfWZ68boZbHz19uBpGSydDAc2mv+fIZIESiOJwLb6LUzTc+Mc82VbhPXVi1rDFw+abTYHPAEXjyr8kfWuKhE8+oyHfo9yUcY8mHu/85PB3rXw/USIv9E162MkEThBr7EQ2Rq+GG9KNhSwunrR44F5D5oV0L0eOGchjSNnNAnn6vswk3NvhDXTYfZdZd77OOA3HLknYfbdUKMBXP1cYK9tkSQCJ6hU1XzoDu0wTVLbPB744iGzOXevsaY1IEQ5FJ388EL+jfzf3OGw/mN2vHydWal7HgG/4fjuadMlNPRls0jOJSQROEWrfmZv4yXPwW8/24vDkw/zxhVJAv8qXDDm2BlNwrHOXoT28NNT4JpJtDz8A7x3oylW6BRbF5jCchffAW0G2o4moGTWkJOcyIIpyeZO6a7FpqxzAEyen2q6oXJPwKwxZkvNy/4K/f9R4qphx8xoEkHjjM/M+pmmiyiyNdz6IdSLK3xe0dLtRY3t38Z/3aX7f4GpV0L9OPjjV1A1wj/XsUxmDQWD6nVh+HvmLmnmH8vcj1pRLyzcasoB/Pd62DwPBk2EAf+U0hHCp86Y/NBpGIz8xOxl8cblsPOnwh+dt5SFr2XvN6vlq0bALR+GbBIojSQCp2kUb/b73fkDzP9HQC7ZSu2GqVfA7pUw7C245J7zvsb6jCYRdM75Im/ZF8YshGp14d0hkPJ24Hc6O53D3tevh+MH4ZYPoE7TwF7fIUJkd5QQ02U47FkFP79iBpL7+/7u3Nv8vjrsZz6tMoXMjCr8+fTfuGRvPOPiz//6UKvHLixp0BrGLICZo+HzB+DX782NULU6gJ9vOI4fgvdvpmH2RlMfqUmi/67lcFbHCJRSNwFPAB2A7lrrMnX8h+wYQVEej5nDn/IWdL/LdNeE+bABd/o4LPw3LHuVlZ42XPzQ3HNWNgsRMB4P/PA8fPsU1ImBG6ZAs0v8d72sXfDeDXB4J3eduIfXn/mX/67lIE4dI9gA3AAsthyH84SFwTWT4NL7YPnr8OmfzZe3L2z/Dl69FJa9Ct3vYsTpxyUJCLvCwqD3X+GPX5ruobeuNIvPjh/y/bXSU8h+5XKOZuxi+PGH+drT3fUz4Kx2DWmtNwEoGZQsnlJmiXvVWrDof2HXz6b+f1yv8p3v4Hb4/v/CuhlQvxWM+hxa9Obeyu788AsHatYD7v3JrOz96RUzeaHfI5AwsuIlz3NPmBbHz69Qq1YTGD2fDxt1khlw2G8RiPNRCpInwKjPQHtg2tXw2QOQua3s59i3wUwLfTkJfpltSlrc8wO06A1If79wmKoR5gbo7iXQoJ1Z4f5CF1PA7tSxUl9a7B19fi5s+AReuwx+etnsLXDvT7LHdhF+HyNQSi0AGhXzo0e11p8WPGcR8FBpYwRKqTuBOwGaNWt28c6dO/0QrcOdzjF3NMteN3sCxPaAzsMhOh7qNYeIaDiVDccOQNZOUy009Ss4uA0q14Ruo01XU63i/jmEcCCtzQDy4ucgbQlUqm4We3W8Dlr1N1Ouiyi8u/d4IDPVrIlZMRWy95hW8OBJZr+EIgrX0bhASWMEjlhQVpZEUJQrBotLk70P1s4wdVsyi9wBqTDTavAKrwIt+kDbQaaAlouWzIsQlJ4Caz8w+2XnHDCP1W5q9hGu1QR0PnNW7eS61pVg92qzvSRAy37Q426TQIKlDLafSCIIRVqbu/1Dv5oWwNE9PP3dPh69Odnc9Te92JWLY0SI8+SbMiy7lkHGFvZvX4Pn2AHydDh5hHGEmqzztKJhh14Muuo6U+FXAA5NBEqp64GXgCggC1ijtb7yfK+TRFAyGfgSbiaf/9KVlAhszxqaDcy2GUMoOLs2i7conF9rswghQoasLA4B4wa2LfzClzsi4WZS+qR8ZPqoECJkSAu4fCQRhBi5IxJCXChJBCFG7oiEEBdKEoEQQricJAIhhHA5SQRCCOFykgiEEMLlJBEIIYTLOaLW0IVSSmUAwVh+tAGQaTuIAHLb+wV5z24RrO+5udY66uwHgzIRBCulVEpxdT5CldveL8h7dotQe8/SNSSEEC4niUAIIVxOEkFgTbEdQIC57f2CvGe3CKn3LGMEQgjhctIiEEIIl5NEIIQQLieJwAKl1ENKKa2UamA7Fn9TSj2rlNqslFqnlJqtlKprOyZ/UUoNUkptUUptU0pNsB2PvymlYpVS3ymlNimlflFKjbUdUyAopcKVUquVUp/bjsVXJBEEmFIqFhgI/GY7lgCZD8RrrTsDqcDfLcfjF0qpcOA/wFXARcAtSqmL7Ebld3nAg1rrDsAlwJ9d8J4BxgKbbAfhS5IIAm8y8DDgilF6rfU3Wuu8gj/+DMTYjMePugPbtNY7tNangRnAUMsx+ZXWeq/WelXB77MxX45N7UblX0qpGOAa4E3bsfiSJIIAUkoNAXZrrdfajsWS0cCXtoPwk6bAriJ/TifEvxSLUkrFAYnAMruR+N3zmBs5j+1AfEk2r/cxpdQCoFExP3oUeAS4IrAR+V9p71lr/WnBcx7FdCVMD2RsAaSKecwVrT6lVAQwC3hAa33Udjz+opQaDBzQWq9USiXbjseXJBH4mNZ6QHGPK6U6AS2AtUopMF0kq5RS3bXW+wIYos+V9J69lFKjgMFAfx26C1fSgdgif44B9liKJWCUUpUxSWC61voT2/H4WS9giFLqaqAaUFsp9Z7WeqTluCpMFpRZopRKA5K01sFYwbDMlFKDgElAX611hu14/EUpVQkzGN4f2A2sAG7VWv9iNTA/UuaO5h3gkNb6AdvxBFJBi+AhrfVg27H4gowRCH97GagFzFdKrVFKvWY7IH8oGBC/D/gaM2j6USgngQK9gNuBywv+bdcU3C2LICMtAiGEcDlpEQghhMtJIhBCCJeTRCCEEC4niUAIIVxOEoEQQricJAIhhHA5SQRCCOFykgiE8IGCuvwDC37/lFLqRdsxCVFWUmtICN/4J/BvpVRDTBXOIZbjEaLMZGWxED6ilPoeiACSC+rzCxEUpGtICB8oqC7bGDglSUAEG0kEQlSQUqoxZp+FoUCOUupKyyEJcUEkEQhRAUqpGsAnmL17NwFPAk9YDUqICyRjBEII4XLSIhBCCJeTRCCEEC4niUAIIVxOEoEQQricJAIhhHA5SQRCCOFykgiEEMLl/j+Bqwv/Mi1yLAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# WRITE THE PLOTTING CODE HERE\n",
    "plt.figure()\n",
    "plt.plot(X, y, '+')\n",
    "k = 5\n",
    "# feature matrix\n",
    "Phi = poly_features(X, k)\n",
    "\n",
    "# maximum likelihood estimate\n",
    "theta_ml = nonlinear_features_maximum_likelihood(Phi, y)   \n",
    "\n",
    "# feature matrix for test inputs\n",
    "Phi_test = poly_features(Xtest, k)\n",
    "\n",
    "ypred_test = Phi_test @ theta_ml\n",
    "\n",
    "plt.plot(Xtest, ypred_test) \n",
    "plt.xlabel(\"$x$\")\n",
    "plt.ylabel(\"$y$\")\n",
    "plt.legend([\"data\", \"maximum likelihood fit\"]);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Question\n",
    "If you did not have a designated test set, what could you do to estimate the generalization error (purely using the training set)?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2. Maximum A Posteriori Estimation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We are still considering the model\n",
    "$$\n",
    "y = \\boldsymbol\\phi(\\boldsymbol x)^T\\boldsymbol\\theta + \\epsilon\\,,\\quad \\epsilon\\sim\\mathcal N(0,\\sigma^2)\\,.\n",
    "$$\n",
    "We assume that the noise variance $\\sigma^2$ is known."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Instead of maximizing the likelihood, we can look at the maximum of the posterior distribution on the parameters $\\boldsymbol\\theta$, which is given as\n",
    "$$\n",
    "p(\\boldsymbol\\theta|\\mathcal X, \\mathcal Y) = \\frac{\\overbrace{p(\\mathcal Y|\\mathcal X, \\boldsymbol\\theta)}^{\\text{likelihood}}\\overbrace{p(\\boldsymbol\\theta)}^{\\text{prior}}}{\\underbrace{p(\\mathcal Y|\\mathcal X)}_{\\text{evidence}}}\n",
    "$$\n",
    "The purpose of the parameter prior $p(\\boldsymbol\\theta)$ is to discourage the parameters to attain extreme values, a sign that the model overfits. The prior allows us to specify a \"reasonable\" range of parameter values. Typically, we choose a Gaussian prior $\\mathcal N(\\boldsymbol 0, \\alpha^2\\boldsymbol I)$, centered at $\\boldsymbol 0$ with variance $\\alpha^2$ along each parameter dimension."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The MAP estimate of the parameters is\n",
    "$$\n",
    "\\boldsymbol\\theta^{\\text{MAP}} = (\\boldsymbol\\Phi^T\\boldsymbol\\Phi + \\frac{\\sigma^2}{\\alpha^2}\\boldsymbol I)^{-1}\\boldsymbol\\Phi^T\\boldsymbol y\n",
    "$$\n",
    "where $\\sigma^2$ is the variance of the noise."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [],
   "source": [
    "## EDIT THIS FUNCTION\n",
    "def map_estimate_poly(Phi, y, sigma, alpha):\n",
    "    # Phi: training inputs, Size of N x D\n",
    "    # y: training targets, Size of D x 1\n",
    "    # sigma: standard deviation of the noise \n",
    "    # alpha: standard deviation of the prior on the parameters\n",
    "    # returns: MAP estimate theta_map, Size of D x 1\n",
    "    \n",
    "    D = Phi.shape[1] \n",
    "    \n",
    "    # SOLUTION\n",
    "    PP = Phi.T @ Phi + (sigma/alpha)**2 * np.eye(D)\n",
    "    theta_map = scipy.linalg.solve(PP, Phi.T @ y)\n",
    "    \n",
    "    return theta_map"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [],
   "source": [
    "# define the function we wish to estimate later\n",
    "def g(x, sigma):\n",
    "    p = np.hstack([x**0, x**1, np.sin(x)])\n",
    "    w = np.array([-1.0, 0.1, 1.0]).reshape(-1,1)\n",
    "    return p @ w + sigma*np.random.normal(size=x.shape) "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYwAAAEGCAYAAAB2EqL0AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAATD0lEQVR4nO3df4xl513f8fcHuwkQ0hLqTex41x2nWVexjAkwWooslYT1OhvbeEtpKrsqdQhoFQnDEjWCSVciyCGSUaRuUxFBtinIAiMnolh2sw72rPkl/jD1bOqEmI3Hy8rBmwUywSlEpWBt8u0fc+2Ox/fOnvHOPc+9c98vaTT3nPPsme8dy/O55zzneZ5UFZIknc83tC5AkjQdDAxJUicGhiSpEwNDktSJgSFJ6uTi1gWM0yWXXFJzc3Oty5CkqXHixIkvV9WOYce2dWDMzc2xtLTUugxJmhpJvjDqmLekJEmdGBiSpE4MDElSJwaGJKkTA0OS1ImBMSOOLC63LkHSlDMwZsSHH3mqdQmSppyBIUnqZCICI8n+JE8mOZVkYcjxdyZZSfL44OvHWtQ5bY4sLjO3cIy5hWMAL7z29pSklyOtF1BKchGwDOwDzgCPAbdV1Z+safNOYL6q7tjMuefn58uR3qvmFo7x9F03tS5D0oRLcqKq5ocdm4QrjD3Aqao6XVXPAfcCBxrXJElaZxIC43LgmTXbZwb71vuhJJ9N8ptJdo06WZKDSZaSLK2srGx1rVPr0N7drUuQNOUmITAyZN/6+2T/A5irqmuB48Ddo05WVUerar6q5nfsGDrh4kx6z76rWpcgacpNQmCcAdZeMewEzq5tUFV/VVV/P9j8r8B391TbVLATW1IfJiEwHgN2J7kyySuAW4EH1jZIctmazVuAkz3WN/EcYyGpD83Xw6iqc0nuAB4CLgJ+paqeSHInsFRVDwA/meQW4BzwLPDOZgVL0oxq/ljtOG3nx2qPLC4PvbI4tHe3/RWSXraNHqs1MLYBx1hI2iqTPg5DkjQFDIxtwDEWkvpgYGwD9llI6oOBIUnqxMCQJHViYEiSOjEwJDXn9DbTwcCQ1JzT20wHA0OS1EnzuaQkzab109s8v5Sw09tMLqcGkdSc09tMDqcGkSRdMANDUnNObzMdDAxJzdlnMR0MDElSJwaGJKmTiQiMJPuTPJnkVJKFIcdfmeTjg+N/lGSu/yolabY1D4wkFwEfAd4OXA3cluTqdc1+FPhKVb0ROAL8Qr9VSpKaBwawBzhVVaer6jngXuDAujYHgLsHr38T2JskPdYoSTNvEgLjcuCZNdtnBvuGtqmqc8BfA/942MmSHEyylGRpZWVlDOVK0myahMAYdqWwfvh5lzarO6uOVtV8Vc3v2LHjgouTJK2ahMA4A+xas70TODuqTZKLgX8EPNtLdZIkYDIC4zFgd5Irk7wCuBV4YF2bB4DbB6//NfA7tZ0nwZKkCdR8ttqqOpfkDuAh4CLgV6rqiSR3AktV9QDw34BfS3KK1SuLW9tVLEmzqXlgAFTVg8CD6/b97JrXfwe8o++6JEn/3yTckpIkTQEDQ5LUiYEhSerEwJAkdWJgSJI6MTAkSZ0YGJKkTgwMSVInBoYkqRMDQ5LUiYEhSerEwJAkdWJgbJEji8utS5CksTIwtsiHH3mqdQmSNFYGhiSpk4lYD2NaHVlcftGVxdzCMQAO7d3Ne/Zd1aosSRqLtFzpNMm3AR8H5oCngX9TVV8Z0u5rwB8PNv+sqm7pcv75+flaWlrammLPY27hGE/fdVMvP0uSxiXJiaqaH3as9S2pBeCRqtoNPDLYHub/VtWbB1+dwkKStLVaB8YB4O7B67uBf9mwlgtyaO/u1iVI0li1DozXVdWfAwy+v3ZEu29MspTk0SQTGSr2WUja7sbe6Z3kOHDpkEOHN3GaK6rqbJI3AL+T5I+r6k9H/LyDwEGAK664YtP1SpKGG/sVRlVdX1XXDPm6H/jLJJcBDL5/acQ5zg6+nwZ+D/jODX7e0aqar6r5HTt2bPn7kbYTB5xqM1rfknoAuH3w+nbg/vUNkrwmySsHry8BrgP+pLcKpW3MAafajNaBcRewL8lTwL7BNknmk3xs0OZNwFKSzwC/C9xVVQaGJPWs6TiMcetzHIY0LdYPOH2eA04FG4/DMDCkGeaAU603yQP3JElTwsCQZpgDTrUZBoY0w+yz0GYYGJKkTgwMSVInBoYkqRMDQ5LUiYEhSerEwJAkdWJgSJI6MTCkDpwGXDIwpE6cBlwyMCRJHY19iVZpWq2fBnxu4RjgNOAtHFlc9nc+AQwMaYT37LvqhT9STgPe1ocfecrAmADekpIkddI0MJK8I8kTSb6eZOiCHYN2+5M8meRUkoU+a5TAacBbOLK4zNzCsRduBT7/2ifW2mm64l6SNwFfBz4KvLeqXrI8XpKLgGVW1/w+AzwG3NZlXW9X3JO2B28J9mejFfea9mFU1UmAJBs12wOcqqrTg7b3AgeA8waGJGnrTEMfxuXAM2u2zwz2DZXkYJKlJEsrKytjL07S+HlLcDKM/QojyXHg0iGHDlfV/V1OMWTfyPtoVXUUOAqrt6Q6FSlpovmE1GQYe2BU1fUXeIozwK412zuBsxd4TknSJk3DLanHgN1JrkzyCuBW4IHGNUnSzGn9WO0PJjkDfC9wLMlDg/2vT/IgQFWdA+4AHgJOAp+oqida1SxJs6ppYFTVfVW1s6peWVWvq6q3Dfafraob17R7sKquqqp/WlUfbFexJE2+cY1VmYZbUpKkTRjX7MoGhiSpEycflKQptXYW3z5mVzYwJGlKrZ3Ft4/Zlb0lJUnqxCuMbcDFZaTZ0eXW07imUmk6W+24zcpstc7kKc2mcfy/v9Fstd6SkiR14i2pKeV605L6nsXXW1LbgLekJG0Vb0lJki6YgbENuLiMpD6cNzCSHE/yHX0Uo5fHPgtJfehyhfHTwJEkv5rksnEXJEmaTOcNjKr6dFV9P/BJ4LeTvD/JN42/NEmaDOOaLnzadOrDSBLgSeCXgJ8Ankryw+MsTJImxbimC582Xfow/hD4InAEuBx4J/AWYE+So+MsTtL4+KlZm9Vl4N67gSfqpQM2fiLJyQv54UneAfwc8CZgT1UNHTSR5Gngq8DXgHOjnhGW1N3amU71Ug6OfanzBkZVfW6Dwxc6WuxzwL8CPtqh7Vur6ssX+PMkqZM+pgufNhc0NUhVnb7Af38SYLWLRNK4+alZF2Ja5pIq4OEkBXy0qkb2nSQ5CBwEuOKKK3oqT5oOfmp+eRwcu2rsgZHkOHDpkEOHq+r+jqe5rqrOJnktsJjk81X1B8MaDsLkKKzOJfWyipakNbz6WjX2wKiq67fgHGcH37+U5D5gDzA0MCR146dmbdbEzyWV5FVJXv38a+AGVjvLJV0APzVrs5oGRpIfTHIG+F7gWJKHBvtfn+TBQbPXAX+Y5DPA/wSOVdVvj7s2n1GXpBdr2uldVfcB9w3Zfxa4cfD6NND75Ic+oy5JLzbxt6QkSZNhWh6r7YXPqEvSaC7ROoLPqEuaRS7RKkm6YAbGCD6jLkkvZmCMYJ+FJL2YgSFNKMcCadIYGNKEcpU3TRoDQy/wE62kjTgOQy9wdHt7jgXSJDMwZsCRxWX/2EwJ16vQJDMwZsBGVw5+opXUlYEx4/xEO7kcC6RJY2BsU145TD//O2nSGBjb1Mu5cvATraSN+FitXuAnWkkbab3i3oeSfD7JZ5Pcl+RbR7Tbn+TJJKeSLPRd57TzykHSVmh9hbEIXFNV1wLLwPvWN0hyEfAR4O3A1cBtSa7utcop55WDpK3QNDCq6uGqOjfYfBTYOaTZHuBUVZ2uqueAe4EDfdUoSVrV+gpjrXcBnxqy/3LgmTXbZwb7hkpyMMlSkqWVlZUtLlGSZtfYn5JKchy4dMihw1V1/6DNYeAccM+wUwzZN3KZwKo6ChyF1RX3Nl2wJGmosQdGVV2/0fEktwM3A3tr+HqxZ4Bda7Z3Ame3rkJJUhetn5LaD/wMcEtV/e2IZo8Bu5NcmeQVwK3AA33VKEla1boP4xeBVwOLSR5P8ssASV6f5EGAQaf4HcBDwEngE1X1RKuCtfWcVl2aDk1HelfVG0fsPwvcuGb7QeDBvupSv5xWXZoOra8wJElTwrmk1ISTI0rTJ8MfTNoe5ufna2lpqXUZOg+nVZcmR5ITVTU/7Ji3pCRJnRgYas7JEaXpYGCoOfsspOlgYEiSOjEwJEmdGBiSpE4MDElSJwaGJKkTA0OS1ImBIUnqxMCQJHViYEiSOjEwJEmdGBiSpE6aroeR5EPADwDPAX8K/EhV/e8h7Z4Gvgp8DTg3aupdSdL4tL7CWASuqaprgWXgfRu0fWtVvdmwkKQ2mgZGVT1cVecGm48CO1vWI0karfUVxlrvAj414lgBDyc5keTgRidJcjDJUpKllZWVLS9SkjZyZHG5dQljM/bASHI8yeeGfB1Y0+YwcA64Z8Rprquq7wLeDvx4kn8x6udV1dGqmq+q+R07dmzpe5Gk81m7Vv12M/ZO76q6fqPjSW4Hbgb21ogFxqvq7OD7l5LcB+wB/mCra5Ukjdb6Kan9wM8A31dVfzuizauAb6iqrw5e3wDc2WOZkrShI4vLL7qymFs4BqwuP7ydVpTMiA/1/fzw5BTwSuCvBrserap3J3k98LGqujHJG4D7BscvBn6jqj7Y5fzz8/O1tLS05XVL0ihzC8d4+q6bWpfxsiU5Mepp1KZXGFX1xhH7zwI3Dl6fBr6jz7okSS81SU9JSdLUO7R3d+sSxsbAkKQttJ36LNYzMCRJnRgYkqRODAxJUicGhiSpEwNDktSJgSFJ6sTAkCR1YmBIkjoxMCRJnRgYkqRODAxJUicGhiSpEwNDktSJgSFJ6qR5YCT5QJLPJnk8ycOD1faGtbs9yVODr9v7rlOSZl3zwAA+VFXXVtWbgU8CP7u+QZJvA94PfA+wB3h/ktf0W6YkzbbmgVFVf7Nm81XAsEXG3wYsVtWzVfUVYBHY30d9kqRVTdf0fl6SDwL/Hvhr4K1DmlwOPLNm+8xgnySpJ71cYSQ5nuRzQ74OAFTV4araBdwD3DHsFEP2DbsSIcnBJEtJllZWVrbuTUjSjOslMKrq+qq6ZsjX/eua/gbwQ0NOcQbYtWZ7J3B2xM86WlXzVTW/Y8eOrXkDmkpHFpdblyBtK837MJLsXrN5C/D5Ic0eAm5I8ppBZ/cNg33SSB9+5KnWJUjbyiT0YdyV5J8BXwe+ALwbIMk88O6q+rGqejbJB4DHBv/mzqp6tk25kjSbUjW0K2BbmJ+fr6WlpdZlqEdHFpeHXlkc2rub9+y7qkFF0nRJcqKq5oceMzC0Xc0tHOPpu25qXYY0VTYKjOZ9GJKk6WBgaNs6tHf3+RtJ6szA0LZln4W0tQwMSVInBoYkqRMDQ5LUiYGh3jllhzSdDAz1zik7pOlkYEiSOpmEuaQ0A9ZP2TG3cAxwyg5pmjg1iHrnlB3S5HJqEEnSBTMw1Dun7JCmk4Gh3tlnIU0nA0OS1ImBIUnqxMCQJHViYEiSOjEwJEmdbOuBe0lWgC+0rmMDlwBfbl1EQ77/2X7/4O9gEt//P6mqHcMObOvAmHRJlkaNqJwFvv/Zfv/g72Da3r+3pCRJnRgYkqRODIy2jrYuoDHfv2b9dzBV798+DElSJ15hSJI6MTAkSZ0YGBMiyXuTVJJLWtfSpyQfSvL5JJ9Ncl+Sb21dUx+S7E/yZJJTSRZa19OnJLuS/G6Sk0meSHKodU0tJLkoyf9K8snWtXRlYEyAJLuAfcCfta6lgUXgmqq6FlgG3te4nrFLchHwEeDtwNXAbUmubltVr84B/6Gq3gT8c+DHZ+z9P+8QcLJ1EZthYEyGI8BPAzP3BEJVPVxV5wabjwI7W9bTkz3Aqao6XVXPAfcCBxrX1Juq+vOq+vTg9VdZ/aN5eduq+pVkJ3AT8LHWtWyGgdFYkluAL1bVZ1rXMgHeBXyqdRE9uBx4Zs32GWbsD+bzkswB3wn8UdtKevefWf2Q+PXWhWzGxa0LmAVJjgOXDjl0GPiPwA39VtSvjd5/Vd0/aHOY1VsV9/RZWyMZsm/mri6TfAvw34Gfqqq/aV1PX5LcDHypqk4keUvrejbDwOhBVV0/bH+SbweuBD6TBFZvx3w6yZ6q+oseSxyrUe//eUluB24G9tZsDAw6A+xas70TONuoliaS/ANWw+Keqvqt1vX07DrgliQ3At8I/MMkv15V/65xXeflwL0JkuRpYL6qJm32yrFJsh/4T8D3VdVK63r6kORiVjv49wJfBB4D/m1VPdG0sJ5k9dPR3cCzVfVTretpaXCF8d6qurl1LV3Yh6HWfhF4NbCY5PEkv9y6oHEbdPLfATzEaofvJ2YlLAauA34Y+P7Bf/PHB5+2NeG8wpAkdeIVhiSpEwNDktSJgSFJ6sTAkCR1YmBIkjoxMCRJnRgYkqRODAypR4N1IPYNXv98kv/SuiapK+eSkvr1fuDOJK9ldZbWWxrXI3XmSG+pZ0l+H/gW4C2D9SCkqeAtKalHgxmKLwP+3rDQtDEwpJ4kuYzV9T4OAP8nydsalyRtioEh9SDJNwO/xepa1ieBDwA/17QoaZPsw5AkdeIVhiSpEwNDktSJgSFJ6sTAkCR1YmBIkjoxMCRJnRgYkqRO/h9nQU2Hc524lwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Generate some data\n",
    "sigma = 1.0 # noise standard deviation\n",
    "alpha = 1.0 # standard deviation of the parameter prior\n",
    "N = 20\n",
    "\n",
    "np.random.seed(42)\n",
    "\n",
    "X = (np.random.rand(N)*10.0 - 5.0).reshape(-1,1)\n",
    "y = g(X, sigma) # training targets\n",
    "\n",
    "plt.figure()\n",
    "plt.plot(X, y, '+')\n",
    "plt.xlabel(\"$x$\")\n",
    "plt.ylabel(\"$y$\");"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXIAAAD8CAYAAABq6S8VAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nOzdd3zN1//A8dfnZu9EEjMhdhBEJCiqQqm9a6VKW+NbquqrRnV8W0p1ma2WalGltEZqq1GzVozYW2IlRPa4N3ed3x8hPyPIuPdmOM/HIw/J9fmc874R75x77jnvowghkCRJkoovVWEHIEmSJBWMTOSSJEnFnEzkkiRJxZxM5JIkScWcTOSSJEnFnEzkkiRJxZzJErmiKFaKohxTFGW9qdqUJEmSns2UI/JRwFkTtidJkiTlgkkSuaIoPkBHYIEp2pMkSZJyz9pE7cwExgEuubnYy8tL+Pn5mahrSZKk58ORI0fuCiG8H328wIlcUZROwB0hxBFFUVo+5bqhwFCAihUrEhERUdCuJUmSniuKokTn9LgpplaaAV0URYkClgOtFEX57dGLhBDzhRDBQohgb+/HfqFIkiRJ+VTgRC6E+EAI4SOE8AP6AjuEEK8VODJJkiQpV+Q6ckmSpGLOVG92AiCE2AnszM+9Op2OGzduoNFoTBmSVMLZ29vj4+ODjY1NYYciSYXGpIm8IG7cuIGLiwt+fn4oilLY4UjFgBCC+Ph4bty4QeXKlQs7HEkqNEVmakWj0eDp6SmTuJRriqLg6ekpX8VJz70ik8gBmcSlPJM/M5JUxBJ5UfLpp5/yzTffPPHvw8PDOXPmjAUjkiSpWEu5BTs+h7uXTN60TOT5JBO5JEl5khgFu7+G5Gsmb7rYJ/IZWy+YrK0pU6ZQs2ZNXn75Zc6fPw/ATz/9REhICPXr16dnz55kZGTw77//snbtWsaOHUtgYCCXL1/O8TpJkqRsmalZf9q5mbzpYp/IZ22/aJJ2jhw5wvLlyzl27BirV6/m8OHDAPTo0YPDhw8TGRlJrVq1+Pnnn2natCldunTh66+/5vjx41StWjXH6yRJkrJpUrL+tHc1edNFZvlhYduzZw/du3fH0dERgC5dugBw6tQpPvroI5KSkkhLS+OVV17J8f7cXidJ0nMqMznrTzuZyIGs6ZQHR+J+EzYAMKp1dUa3qZHvdnNaATFo0CDCw8OpX78+ixYtYufOnTnem9vrJEl6TplxRF4sp1ZGt6lB1LSORE3rCJD9eUGSeIsWLVizZg1qtZrU1FTWrVsHQGpqKuXKlUOn07F06dLs611cXEhNTc3++knXSZIkAZCZAiobsLY3edPFMpGbQ1BQEH369CEwMJCePXvy4osvAjB58mQaN25MmzZt8Pf3z76+b9++fP311zRo0IDLly8/8TpJkiQga0Ru7wpm2PugCCFM3uizBAcHi0frkZ89e5ZatWrlua0ZWy8UaCQuFX/5/dmRJItaNRhuRMCo4/luQlGUI0KI4EcfL/YjcpnEJUkqFu6PyM2g2CdySZKkYiEzxSwrVkAmckmSJMvQyEQuSZJUvGWmyqkVSZKkYi0zWY7IJUmSii0h5IhcejI/Pz/u3r0LQNOmTZ967aJFi7h161b214MHD5YVHCXJErRpIIxyRP480ev1+brv33//ferfP5rIFyxYQO3atfPVlyRJeWDG7flggkSuKIq9oiiHFEWJVBTltKIon5kiMEuLiorC39+fwYMHExAQQFhYGNu2baNZs2ZUr16dQ4cOAXDo0CGaNm1KgwYNaNq0aXa520WLFtG1a1fatWtHzZo1+eyznL8Nzs7OjBkzhqCgIFq3bk1cXBwALVu2ZOLEibz00kvMmjWLuLg4evbsSUhICCEhIezbtw+A+Ph42rZtS4MGDRg2bBgPbuhydnbO/vyrr76ibt261K9fnwkTJrBy5UoiIiIICwsjMDAQtVpNy5Ytub8x6/fff6du3boEBAQwfvz4h9r88MMPqV+/Pk2aNOH27dsm/K5L0nMi814iN9OIHCFEgT4ABXC+97kNcBBo8rR7GjZsKB515syZxx6zpKtXrworKytx4sQJYTAYRFBQkHjjjTeE0WgU4eHhomvXrkIIIZKTk4VOpxNCCLF161bRo0cPIYQQCxcuFGXLlhV3794VGRkZok6dOuLw4cOP9QOI3377TQghxGeffSZGjBghhBDipZdeEm+//Xb2df369RN79uwRQggRHR0t/P39hRBCjBw5Unz22WdCCCHWr18vABEXFyeEEMLJyUkIIcTGjRvFCy+8INLT04UQQsTHx2f38WBM97++efOm8PX1FXfu3BE6nU6EhoaKNWvWZMe7du1aIYQQY8eOFZMnTy7Ad9k8CvtnR5KeKfqAEP9zFeLi1gI1A0SIHHJqgasf3ms87d6XNvc+Crbvf9MEiD1ZwMgeUbYutJ/21EsqV65M3bp1AahTpw6tW7dGURTq1q1LVFQUAMnJyQwcOJCLFy+iKAo6nS77/jZt2uDp6Qlk1THfu3cvwcEP76ZVqVT06dMHgNdee40ePXpk/939xwG2bdv20Px1SkoKqamp7N69m9WrVwPQsWNHPDw8Hnse27Zt44033sguyVuqVKmnPu/Dhw/TsmVLvL29AQgLC2P37t1069YNW1tbOnXqBEDDhg3ZunXrU9uSJCkHZh6Rm6SMraIoVsARoBrwvRDiYA7XDAWGAlSsWNEU3ZqcnZ1d9ucqlSr7a5VKlT1v/fHHHxMaGsqaNWuIioqiZcuW2fc8WgY3NwcDP3iNk5NT9udGo5H9+/fj4ODw1HtyIoTI06HE4in1dmxsbLLbsrKyyvf8vSQ91zTmq0UOJkrkQggDEKgoijuwRlGUACHEqUeumQ/Mh6yiWU9t8Bkj58KUnJxMhQoVgKx58Qdt3bqVhIQEHBwcCA8P55dffnnsfqPRyMqVK+nbty/Lli2jefPmOfbTtm1bvvvuO8aOHQvA8ePHCQwMpEWLFixdupSPPvqITZs2kZiYmOO9kyZNon///jg6OpKQkECpUqUeK717X+PGjRk1ahR3797Fw8OD33//nZEjR+b1WyNJ0pPcP+atqL7Z+SAhRBKwE2hnynaLknHjxvHBBx/QrFkzDAbDQ3/XvHlzBgwYkF0K99FpFcgadZ8+fZqGDRuyY8cOPvnkkxz7mT17NhEREdSrV4/atWvz448/AvC///2P3bt3ExQUxN9//53jq5t27drRpUsXgoODCQwM5JtvvgGyDr/4z3/+k/1m533lypXjiy++IDQ0lPr16xMUFETXrl3z/T2SJOkRZp5aKXAZW0VRvAGdECJJURQH4G/gSyHE+ifdY8oytkXFokWLiIiI4Lvvvnvqdc7OzqSlpT31GilvivvPjvQc2D4Z9s6AT+ILVI/8SWVsTTG1Ug5YfG+eXAX88bQkLkmS9NzJTAE7F7McKgEmSORCiBNAAxPEUqwNGjSIQYMGPfM6ORqXpOeQGWuRg9zZKUmSZH6ZKWDnZrbmZSKXJEkyNzkilyRJKuYyk7PmyM1EJnJJkiRzM+PpQCATeZGyc+fO7O3wDzp+/DgbN27MV5tTp07N/jwqKoqAgIBn3hMXF0fjxo1p0KABe/bsyVe/OZFldKXnVqacWilSCmOL+tMS+bPieTCR59b27dvx9/fn2LFjvPjii3m+/0lkGV3puXT/UAk5IreMyZMn4+/vT5s2bejXr1/2jshHS8xGR0fTunVr6tWrR+vWrbl27RqQtQRx5cqV2e3dLyu7c+dOWrZsSa9evfD39ycsLCy7vsnmzZvx9/enefPm2cWwHqTVavnkk09YsWIFgYGBrFixgk8//ZShQ4fStm1bXn/9dRYtWsQ777yTfU+nTp3YuXMnEyZMQK1WExgYSFhYGAAGg4EhQ4ZQp04d2rZt+9AOT8j6pTFu3Dg2btyYvQP0wfK4K1euzF5mOWjQIN59912aNm1KlSpVHnrusoyuJN2jU4NRL0fklhAREcGqVas4duwYq1ev5tGdp0lJSezatYsxY8bwzjvv8Prrr3PixAnCwsJ49913n9n+sWPHmDlzJmfOnOHKlSvs27cPjUbDkCFDWLduHXv27CE2Nvax+2xtbZk0aRJ9+vTh+PHj2RUSjxw5wl9//cWyZcue2Oe0adNwcHDg+PHjLF26FICLFy8yYsQITp8+jbu7O6tWrXronsDAwIf6y6lo14NiYmLYu3cv69evZ8KECQBs2rSJ8PBwDh48SGRkJOPGjaNXr14EBwezdOnSx9q9desW48ePZ8eOHRw/fpzDhw8THh4OQHp6Ok2aNCEyMpIWLVrw008/PfN7LUlFirlrkWOiolmm9uWhLzmXcM6kbfqX8md8o/FP/Pu9e/fStWvX7ATTuXPnh/7+wRKz+/fvzx49DxgwgHHjxj2z/0aNGuHj4wNkJcuoqCicnZ2pXLky1atXB7LK2s6fPz9Xz6dLly7PTLI5qVy5MoGBgUBWWdr75Xnzq1u3bqhUKmrXrp09WpZldCXpAfdOBzLiQObp09hW8sPK2ekZN+WNHJHf86yaMw+WmH3U/TKv1tbWGI3G7Pa0Wm32NQ+WyH2wHGxeys0+KZ4H+wXQaDRPvO9JcTzNgzE+2vaD7d3/HsoyupL0gHsj8syYNKJ69kJ9JOIZN+RdkRyRP23kbC7Nmzdn2LBhfPDBB+j1ejZs2MCQIUNyvLZp06YsX76cAQMGsHTp0uxStH5+fhw5coTevXvz119/PXToRE78/f25evUqly9fpmrVqvz+++85Xvek8rP3+fn5MXfuXIxGIzdv3sw+lg6yEqFOp8PGxuZZ34InKlOmDGfPnqVmzZqsWbMGF5enr4eVZXQl6QH3apHrNVmDLatnvELNDzkivyckJIQuXbpQv359evToQXBwMG5uOW+pnT17NgsXLqRevXosWbKEWbNmATBkyBB27dpFo0aNOHjw4FNH8QD29vbMnz+fjh070rx5cypVqpTjdaGhoZw5cyb7zc5HNWvWLPt0o/fff5+goKDsvxs6dCj16tXLfrMzP6ZNm0anTp1o1aoV5cqVe+b1soyuJD3g3ojckJ71atIqh1O9CqrAZWzzo6iWsU1LS8PZ2ZmMjAxatGjB/PnzH0qKUtFUFH52JOmJjiyGde8S7/UJd75bQI2IiHzPkZuzjG2JMXToUM6cOYNGo2HgwIEyiUuSVHD3R+RpGhRbW1ROjibvQibyBzxtKZ8kSVK+aFIABX1yOlYeHvle4PA0co5ckiTJnDJTwc4FQ1KSWebHQSZySZIk88rMKphlSEzEupRM5JIkScWPJhnsXdEnJmDlLhO5JElS8ZM9Ik8yyxpyMEEiVxTFV1GUfxRFOasoymlFUUaZIrCSYu3atUybNq3Q+n+wMFWHDh1ISkrKdTnb+x68PiIiIru2zKeffpq9RtyUzNWuJBUKTQrCxgVjSgpWHu5m6cIUq1b0wBghxFFFUVyAI4qibBVCyELTZNVE6dKlS2GHAZBdCjcpKSnfbQQHBxMc/NgyVkmSniQzBYNtRcA8m4HABCNyIUSMEOLovc9TgbNAhYK2a2lRUVH4+/szePBgAgICCAsLY9u2bTRr1ozq1atnb3s/dOgQTZs2pUGDBjRt2pTz588DMH36dN58800ATp48SUBAABkZGQ+VmB00aBBvv/02oaGhVKlShV27dvHmm29Sq1at7NKwwFPLxubm/ifx8/Pj7t27Dz125coVGjRowOHDhzEYDIwdO5aQkBDq1avHvHnzHmvj0cMvzpw5Q8uWLalSpQqzZ8/Ofnz69OkEBAQQEBDAzJkzn/n4lClTqFmzJi+//HL291SSSgRNCnq9LQDWZppaMek6ckVR/IAGwEFTtmsply5d4s8//2T+/PmEhISwbNky9u7dy9q1a5k6dSrh4eH4+/uze/durK2t2bZtGxMnTmTVqlW89957tGzZkjVr1jBlyhTmzZuXXf3vQYmJiezYsYO1a9fSuXNn9u3bx4IFCwgJCeH48ePZlQmfpKD3P+j8+fP07duXhQsXEhgYyPz583Fzc+Pw4cNkZmbSrFkz2rZt+9R1r+fOneOff/4hNTWVmjVr8vbbb3PixAkWLlzIwYMHEULQuHFjXnrpJYxG4xMfX758OceOHUOv1xMUFETDhg1z/TwkqUjLTMGgzap1ZK43O02WyBVFcQZWAe8JIVJy+PuhwFCAihUrPrWt2KlTyTxr2jK2drX8KTtx4lOvuV+vBKBOnTq0bt0aRVGoW7dudrnX5ORkBg4cyMWLF1EUJbswlkqlYtGiRdSrV49hw4bRrFmzHPvo3LlzdptlypR5qL+oqKhnJuKC3n9fXFwcXbt2ZdWqVdSpUweAv//+mxMnTmQfEJGcnMzFixepUaPGE9vp2LEjdnZ22NnZUbp0aW7fvs3evXvp3r17dq2ZHj16sGfPHoQQOT5uNBrp3r179i++ojIVJUkFptOAQYsh0woowlMrAIqi2JCVxJcKIR4/5gYQQswXQgQLIYLv150uah4syapSqbK/VqlU2eVTP/74Y0JDQzl16hTr1q17qKzrxYsXcXZ2fug4syf18WD7j/aRm7KxT7s/N9zc3PD19WXfvn3ZjwkhmDNnDsePH+f48eNcvXqVtm3bPrWdnMriPql+z9Pq+phjt5skFbr72/Mz75W6NtM68gKPyJWs/4E/A2eFENMLHhLPHDkXpuTkZCpUyHoLYNGiRQ89PmrUKHbv3s0777zDypUr6dWrV776yGvZ2PywtbUlPDycV155BWdnZ/r3788rr7zCDz/8QKtWrbCxseHChQvZzzUvWrRowaBBg5gwYQJCCNasWcOSJUsQQjzzcb1ez7p16xg2bJjJn7MkWVxmVtlmfYYBACv3ortqpRkwADipKMrxe49NFELk79j3Im7cuHEMHDiQ6dOn06pVq+zHR48ezfDhw6lRowY///wzoaGhtGjRIl993C8b6+vrS0BAAGlpaaYK/yFOTk6sX7+eNm3a4OTkxODBg4mKiiIoKAghBN7e3tlHruVFUFAQgwYNolGjRgAMHjyYBg0aADzx8T59+hAYGEilSpVMeuCzJBWqe7XIDRl6VK6uKAU4F+BpZBlbqdiTPztSkXVlJ/zalZux3VBfjaXali0Fau5JZWzlzk5JkiRzUWft2TCkqrE204oVkIlckiTJfNQJAOhT1Wbbng8ykUuSJJlPRjwAhuRUsy09hCKWyAtjvl4q3uTPjFSkZSQibJwxJCaarc4KFKFEbm9vT3x8vPyPKeWaEIL4+Hjs7e0LOxRJyllGPMLGA6HVmm17PhSho958fHy4ceMGcXFxhR2KVIzY29vj4+NT2GFIUs7UCehxA8xXixyKUCK3sbGhcuXKhR2GJEmS6WQkYBAuQAJWZtrVCUVoakWSJKnEyYjHYMiqIWT9vLzZKUmSVKKoE9DrsuoRmXPVSpGZWpEkSSpRDHrQJGPQZ6VZuY5ckiSpuFEnAmDQKGBtjeqBA2NMTSZySZIkc7i3q9OgNmLl4W7WUs0ykUuSJJnDvV2d+gw91h7mm1YBmcglSZLMI+PeiDxNY9Y3OkEmckmSJPO4P7WSkm7WNeQgE7kkSZJ5PFAwy5xryEEmckmSJPPISECo7DEkp5h1ez7IRC5JkmQe6gQMqlIghFnXkINM5JIkSeaRkYBBcQMwawlbMFEiVxTlF0VR7iiKcsoU7UmSJBV7GQkYjFmbgIrLHPkioJ2J2pIkSSr+1Ano9Q4AWHl6mrUrkyRyIcRuIMEUbUmSJJUIGfHoNVl1VqxLlzZrV3KOXJIkydSMRlAnolerwMYGK/diMEeeG4qiDFUUJUJRlAh5CpAkSSWaJgmEEX2aARtvb7PWWQELJnIhxHwhRLAQItjb29tS3UqSJFnevcqH+lSd2adVQE6tSJIkmd69Oiu6pIzik8gVRfkd2A/UVBTlhqIob5miXUmSpGLpfuXDxBSsLTADYZITgoQQ/UzRjiRJUomgTsCoVzCmFaMRuSRJkvSAjISsFSuYf+khyEQuSZJkehnx6DNtAbAubf6pFZnIJUmSTE2dgN7gCoCNHJFLkiQVQxkJ6HSOgGWmVkzyZqckSZL0gIwE9Fp7FDtQubqavTs5Ii9iZmy9UNghSJJUUOqsNzutS5c2+65OkIm8yJm1/WJhhyBJUkFlxKNPt8y0CshELkmSZFpCZE2tpOktshkI5Bx5kTBj64WHRuJ+EzYAMKp1dUa3qVFYYUmSlB/aNDDq0KdocLLA0kOQidziZmy98FhyHt2mRvZjfhM2EDWtY2GEJkmSKWTEY9ApGDU6iyw9BDm1YnFyDlySSjgL7+oEmciLnFGtqxd2CJIkFYQ6Ab3aCrBcIpdTKxaQlzlwOScuScVchkzkJZKcA5ek50jabfQay06tyESeE6MRTv4JO7+AlJugss768AmBl8ZDxcaFHaEkSUVVaiz6TDsURwdUTk4W6VLOkT/q6m6Y/xKsGQp2LtBkOAS/CfV6Q0wk/NIWlnTP+jwf5By4VJTJncUmkBqDXueIjbdldnWCTOQPu7gNfu0K6iSMHeeS6DqcuLOe3L3iQ3xMLdKb/oJ4eRLEnICfX4ELf+e5CzkHLhVlclWVCaTGotPYWGwzEMiplf93+wz8OQiDey0SbfqTMGouhoSExy6zr10bz9em4HL7e5Tl/aDrXKjfpxACliSpSEqNQZ8BDhaaHweZyLOk3oZlvclUOxG9QcEQ/z1OL76I19v/waFBAzAYMKrVpGzaRMIvC7k58TPsa/vjE9oImzVDQZMMjYcW9rOQnjOZhkxO3z3NxcSLRKdGcy3lGvHqeNL16aTr0jEKI/ZW9jjYOOBu546Psw++Lr74uflRz6seZZzKAHJnsUkJgUi5jT61lMXe6ASZyMGgh+X90cUlcG1vFRACv+W/4xAY+P/XWFtj5eKCR+/euPfqRcrGTcT+739cXeGAb9eWOGwaB6UqQ/U2hfc8pBJPCMGFxAtsu7aNQzGHOHX3FFqjFgB7K3t8XX0p7VCa8s7lcbRxxEqxQq1Xo9arSdQksufmHu6q72a3V9apLA1KN6Bl1ZacfKkFLrYuclVVQWWmYszIQOjcil8iVxSlHTALsAIWCCGmmaJdizj+G/rLR7l2sA5GtZZKixdhX7v2Ey9XVCrcOnXEvmYNrg8fQfSSKMq1qo7bqrdgyD/gWdWCwUvPg9j0WFZdXMXmq5uJSolCpaio41mHfv79CCoTRG3P2pR2LI1KefZbXhm6DC4nXSYyLpLIuEgOxhxk09VNWKusaVy2MdYuFdEa2mBrZWuBZ1YCpcZafA05gCKEKFgDimIFXADaADeAw0A/IcSZJ90THBwsIiIiCtSvSejUGKc3IGqDLdpUayr+vADHoKBc365PTOTmyHfJOHYU31A1zrXLwFtbwc7ZjEFLzwMhBBG3I1h2dhn/XP8HozDSqGwj2vq1pVXFVng5eJmkH4PRwMm7J9lxbQebozYTkx6Du507Xap2IaxWGOWdy5ukn+fGlV2kf9WLazu9qPjrYpwaNTJp84qiHBFCBD/6uClG5I2AS0KIK/c6Wg50BZ6YyIuMQ/NJOJJK5h1XfObOzFMSB7D28MB33o9Evz6QG3suUkl1CYd1o6DXz2YKWHoeHIo5xPfHv+fonaO427kzsM5AetfsTQXnCibvy0plRWDpQAJLBzIqaBQHYg6w6uIqlp1dxrKzy+hQpQNvBrxJVXf5SjNXUmPR3R+RF7NVKxWA6w98fQN4bMeMoihDgaEAFStWNEG3BaROQrdlBnfPuuP8cigurVrlqxmVkxO+834kqm8/ru9X4We7BttanaFONxMHLJV0Z+PP8nXE1xyOPUxph9JMbDyR7tW6Y29tb5H+rVRWNKvQjGYVmhGbHsvi04tZdXEVay+vpWOVjoxsMNIsv0xKlNQYdOlWoCjYlLfcqxlTrCPPacX7Y/M1Qoj5QohgIUSwtwV/Uz3RvlnEHRKAijLjxhWoKWsvL3znzweVHdf3lcf41/uQ8fjSRUnKSZImicn7J9NnfR8uJ11mQqMJbOy5kX7+/SyWxB9V1qks4xuNZ0vPLbwZ8CbborfReU1nvjr8FcmZyYUSU7GQdhtdhj3WZcqgsrOzWLemSOQ3AN8HvvYBbpmgXfNJj0e9bj7JUY6UeuNNbE3wCsGuSmUqzJqFNslI7F49bP7ABIFKJZkQgnWX19EpvBOrLq4irFYY67qvI6xWGHZWlksCT+Nh78HohqNZ3309nap0YunZpXQJ78Lay2sp6PtrJVJqDFq1PTY+ln3lYopEfhiorihKZUVRbIG+wFoTtGs24vhyYg/bY+3pgedQ063/dmrSGM//DCP5igPJ68LztfNTej7cybjDyB0jmbh3IlXcqrCy80rGNxqPq635T1zPj7JOZZnUbBIrOq3Ax8WHD/d+yKDNg7iSdKWwQytaUmPRpSnY+vg++1oTKnAiF0LogXeALcBZ4A8hxOmCtms2QpC+bhGaBFu8//s+Vs6mLWrjPWIEDvXrE3ukFNplo0CbbtL2peJvS9QWuv3VjYMxBxkXMo6Fryykmke1wg4rV/xL+bOk/RI+a/oZl5Mv8+q6V/nl1C8YjIbCDq1IMCbeQp9qwMbXx6L9mqTWihBioxCihhCiqhBiiinaNJuYSBIPx2Hl6ohb504mb16xtqb8t9+CjQO3tmoRu6ebvA+peNLoNXy2/zPe3/U+lV0rs7LLSgbUHoCVyqqwQ8sTlaKiR/UehHcN50WfF5lxZAavb3qdq8lXCzu0wiUEutg4AGx9imEiL050OxeQdsse9169UGzNs+nB1qcCZT+dhDrelsSFP0HSNbP0IxUfV5Ov0m9DP1ZeWMmbAW+yqP0iKrlWKuywCsTLwYsZLWfwVYuviE6Nps/6Pqy8sPL5nTvXJKNLznplYuNbzKZWihV9JklrNwMK7v0HmLUr104dcW7WmDvHHdH+Md6sfUlF287rO+m/oT/x6nh+fPlHRjccjY3KprDDMglFUWhfuT2ru6ymvnd9Ptv/GaN3jiZJk1TYoVleaiza9KxXVzZyRG4+4vQ6ks6pcAqqbfaXPoqiUHbyVBRrG2KXHUBE7TNrf1LRYxRGfoj8gZJxwEcAACAASURBVJE7RlLRtSIrOq2gWYVmhR2WWZR2LM28NvMY03AMu27sovf63pyIO1HYYVlWagy6NGsUW8uWsIXnLJGn/jkfvcYKjzeHW6SAvk358nj/97+kx9qTMvO9rJOHpOeCRq9h7K6xzD0+l85VOrO43WLKOZcr7LDMSqWoGBQwiN/a/4ZKUTFw80B+O/Pb8zPVkhqLLt0Km/JlLXagxH3PTyJPjSVp7yWsPRxxbtnSYgX0PV57HYcavtzekYzhwK8W6VMqXPHqeN7a8hZbo7cypuEYpjSfUmgbewpDHa86rOi0guYVmvPl4S8Zs2sM6brnYPVWagzaNGtsfS3/3sdzk8i1e34nPdYO9x5dUKwst0pAsbKi7FezMeisiPv2a9BrLda3ZHlXkq8QtjGMC4kXmNFyBoMCBll8dFYUuNm5MTt0Nv9t+F+2X9tO2IYwopKjCjsssxKpsejSrbGpKBO5SeiNejJ0GSRqEonLiCNDl0HKpvUAdLxWObtwvt+EDfhN2GD2aRZ7f388Or5E4hkDmvBvzNqXVHgi4yJ5fdPrqPVqFrZbSOtKrQs7pEKlKApvBLzBvDbziNfE029DP3Ze31nYYZmN4c4NjDrF4mvIoZgfLJGhy+B0/GlOxJ3gfMJ5bqbd5GbaTeI18Y9d+9k5Pc6lFaq120wbt8r8uT+TJa+9Sj3vejhYO5g9Vu+PppGyvRmxc36lUqeRKPYuZu9TspzdN3YzZucYvB29mffyPHxdLbv8rChrUq4JKzqtYPTO0by7411GNhjJ4LqDS9wrFd2NmwDYWnjpIRTDRH499To7ru1gx7UdRMZFYhBZ6zYrOFfAx8WHlr4tKeNYBgdrB2ytbLFWWZN56TD+N9Zxto0vjtaOHIg5gF3pOwz+ewvWKmsCPANo4dOCNpXa4OfmZ5a4rdzd8f7PQGJnLCRlzljcxv5oln4ky1t3eR0f7/uYGh41mPvy3HzXCp+x9UKJPVqtvHN5FrdbzCf/fsLsY7O5kHiBSc0mWWQQZSm62KzTlyy99BBMcLBEfuT3YIkpB6aw/PxyAGp41KCFTwsalG5AXa+6eNh7PPG+pEmvEbPsCH7Ll+AQmFWTfdqWYzSrk8aR20c4FHuIk3dPZrfbuUpnulXrhru9ez6e3ZMJg4Gol0PQJ6VTdet2VF6W/weXTOuP838w+cBkGpVtxKzQWTjb5v9QkefhmDUhBL+c+oVZR2fhX8qf2a1mU9apbGGHVXBCcLdfJeKOO1EjIsLkpT/ue9LBEsUqke++sZuo5ChCK4bi65L7ly83OgSgjlOodujEE1/OxabHsv3adjZd3URkXCS2Klte8XuFsNph1PGsk+dYnyRj+xqiR0zEq30dvGesNFm7kuUtPLWQ6Uem85LPS3zb8tsCVyw0VSIXOh3a69fRXruGPvY2utuxGOITMGZkYExPR2i1YG2FYmWNYmeLtUcprDxLYe3tjV2VKthWrYq1x5MHRqaw6/ouxu8Zj4O1A7NDZ1PXu65Z+zO7jARiegWSGudFjcNHzdZNiUjk+WGMOceFNl1xa1GfcnP/yNU9FxIv8Mf5P1h/ZT3punSaV2jOsHrDCCwd+Oybc+Hmq81JPXOXqmtXYVPVdL8kJMsQQvBD5A/8EPkD7fzaMfXFqfneqfnoCfb35fYEe6HVojl3DnXkCdSRkWSeP0dmVDTodP9/kUqFlYcHKicnVE5OKLY2YDAijAaEWoM+IQFj8sM1xq28vXBsEIRjo0Y4NgrBrnp1k89pX0q8xDs73uGu+i6Tmk6iQ5UOJm3fom6f4VrvThhcqlJ5/VazdfPcJvK0+R9wfXo4vt9+hnPH3nm6N1WbyorzK/j19K8kZibSrHwz/hv8X2p4FGweUxu5hyv9huAa7Ef5XzcXqC3JsoQQzDk2h59O/kS3at349IVPTVb0KjcjciEEmRcukr5vH+n795MREYFQq4Gsw37t69TBrlo17KpVxbZSJazLlcPa0xPF+ulvhwmtFt2dOLRXLpN5+QqZ586Sfvgw+lsxANhUqohru/a4dmiPfc2aJnm+AAmaBEb/M5qjd44yrN4whgcOz9Uh0kXOpe1c6vMfHEKaUuHHxWbrxpxndhZpqf/sQmUDjm3yfvSai60Lg+sOpr9/f1acX8FPJ3/i1XWv0r1ad0YEjsDbMX/bcG3rv0ip5r7E747G49BuHBq1yFc7kmUJIZhxZAYLTy+kV41efNzkY4skHWE0oj56lNSt20jdvh3djRsA2FapgnuPHjiGBOMQGIhN2TzONRsNoNeAUY9iNGDrbottcADOjQLBygas7dHevEX6vn2kbtlM/E8/ET9vHvb161FqwOu4vtIWxaZgNWNK2ZdiQdsFTDowiXkn5nEl+QpTmk8pdm+CiqRb6DKscK3oVyj9l+gRudCkcemFhjjU8MFnxfYCt5ecmcyPkT+y/Pxy7K3seTfoXXrX6J2vEZkh+gSXu7yKnY83FdfvKXFLsUoaIQRfR3zNkjNL6FuzLxMbTzT5v9mjq1YyL14kee1akjdsQH8rBsXWFscXmuDSqhXOTYKwcbMDTQpkpoAm+d7Hvc8zH/g6MwUyUyEzLetPbRro1GDIfHpAKhuwdwV7d3Atj966LCkXdCTuuYI25i7WZcrgOWQIHr1fLXAlUSEEi08vZvqR6dTyrMXs0NmUcSpToDYtSbvyIy5/tIqyn36MR9/+ZuvnuZxaUW/4magx31B+zOu4DTHd0WvRKdFMOTCF/TH7CfAM4JMXPqGWZ608t5M4oSux4Rfw+XoSLp1fNVl8kmk9mMRfq/Ua40LGmS6Ja9MhNRbS7kB6HMb4m6TsPUrSrjOooxJAAafKDrhVV+FcLgMrkZqVjB8/Fvdh1g5ZSdjO9d6fLlkfti5g6wS2jmDjCNZ2WQnbygYUVdYo3agHgzarH00yqBMh5SYkXYfUGIQQpMXYkXDenYzbVtiUdsd75Ahce/ZHURXsFcrO6zsZv3s8TjZOzGk1hzpexeM9pPRv+nBtwQkqLlqIU5MmZuvnuUzk8RP6cSf8ONX/+RvrcqZdpC+EYNPVTdmH0Q6tN5TB9Qbn6U0vEXeJKx3bg707VXbsf+Y8pvQ4c6+9FkLwbcS3LD6zmLBaYYwPGZ+3JC4EpNyCuLNw9yIkRt37iM56/N5Bxto0KxIvOpF0xRGjToWtmwH3Wta41XXHupQH2Ls98HE/Qd/73N7tga/vfW5tnlr76DRw5zTERCJuRJC+6x/u7NeRmWSDg48d5Ua/hV2bt7J+UeTThcQLjNw+knhNPJ83/5x2fu1M+ATMI3FUM2K3JFB12zZszXhe53OZyK93bIg2MZOq/54yWx/JmclMOzSN9VfWU6tULaY0n0J1j+q5vj91ah9u/HqCsh+MxmOg6c4PfV6Yc+31g3Pi/fz78UGjD56dxFNj4dp+uHkUbh2DmBPZyRoAW2fwqAwelcC1POoEa+K3XiD18FlAwbV1i6xCa42aFI/pNiEQsadJXjib2yv2IHQCr/o6PAf2R2kxGhxL5avZeHU8o3eO5tidY8XiTdDbvWqSeM6amscjzToge+4SudBquNCwPq7BlSm30PwrQ7ZHb2fSgUmkadMYEzyGfv79cvUfUSREEd21NVqNC9V27kPlZJ6NBCWVuRK5EILZx2az4OQC+tTsw4eNP8z531Onhis74dI2uLoH7p7PetzKFsoEQPkGULpW1odXTXDyAkUh48gR7v7wI+l796JydcWjT288+vfHplzRLXX7rFc/+jt3iP3wv6TuOYJ9KS0+oTps2gyHJsOzXjnkkdagZfKByYRfCqeVbyu+ePELHG3yP9I3m8w0rrWti8GuApX/3mvWrsySyBVFeRX4FKgFNBJC5Co7WyKRa3at5uqwDyn/7qu4DZ9k1r7ui1fH8/G+j9lzcw8tfVoyqdmkp+44vU8953Wivj+M15DX8R5jurn8kqqga69z4/vj3/Nj5I+8WuNVPm7y8cNJXKeGcxvg9Bq4vAN0GWDjBJVegMotoFJzKBuQNf/8CPWJE8TNnEn6v/uxKlWKUoMG4dG/H1bO+d8Raim5/aWZsnkzMR9+CIZMyofcwcXfEzrPhBqv5LlPIQRLzy7l64ivqepelVmhs/K0GdAiYk9yoV1PnJuEUP7HpWbt6kmJHCFEvj/ISuA1gZ1AcG7va9iwoTC3+ElDxJma/kJ7/pjZ+3qQ0WgUS04vEQ1+bSBarWgljt4++uybEqLE9Zf9xNm6dYTuzh3zB1mCVBq/3uRt/nD8BxGwKEB8su8TYTAash40GoW4dlCI8BFCTPUR4n+uQnxTU4h1o4W4uE0IXeZT29RcvCiuvT1cnKnpL843eUHcXbhQGDIyTB67OeXle50ZFSUud+suztT0F7dfqy+Mn7gKsWqoEOnx+ep73819oumypqLpsqZi7429+WrDXHT7l4kzNf3F3RmTzd4XECFyyKkFmnQSQpwVQpwvSBvmknHsFDbOYFPDNLsxc0tRFF6r/RpLOyzFztqONze/yeLTi59+SopHJUr3a43Q6Ymb8ZXlgpUe8/PJn/n++Pd0qdqF/73wP1QGPUSugJ9C4ec2cGo1+HeCgetg9BnoNB2qtX7im4v6+HhiPvuMK127kXHoEN6j3qXq1q14DhqEyqHor5WesfVCdrlnyH3pZ9tKlfBb/jtuPXsQfziTW1eaYoxcCfNawK3jeY6jafmmLO+4nDJOZRi+fTgLTi4oMicPZZ7K2pJvV69R4QWRU3bP6wdFbERuNBjE+fo1xc0+L5q1n2dJyUwRo3aMEgGLAsR7O94Tadq0J1+cGC1iOlcSZ2rVEprLly0XZDE3/e/zJmtr8anFImBRgBi3a5zQZ6YLcXC+EN/Wyhp9z26Y9bUmNVdtGbVacffnX8S5oIbiTO06ImbSZKFLSDBZrIUhP69+jEajiPtxnjhT019EvdpN6KfUEmJyaSEi/8hXDOnadDFm5xgRsChAvLv9XZGcmZyvdkwpfmzXrFf/t2+bvS/yOyJXFGWboiincvjompdfGIqiDFUUJUJRlIi4uLh8/dLJLe2JfRg0Cg5BDczaz7O42Lowo+UM3g9+n3+u/0P/Df25mnw154vdK+LVrwMqlYE7X06xbKDFmKnmxH8/9ztfR3xN24ptmGJfDavvgmHj++DmC2GrYMQhaDQE7J49l51+8BBXunfnzldf4dAwiCrr1lL244/MXoiqKFIUBa9hQyn/zTeoz14meq8fevdAWD0Ytn2atTwzDxxtHPm6xdeMCxnH7hu76bu+L+cSzpkn+FzKjI7BykFl8QOXH/TMRC6EeFkIEZDDx1956UgIMV8IESyECPY28xPO2LEWAMfQwi8JqigKA+sMZF6beSRqEum/of8TT0mxbv8BnnXUpO36l4wjRywb6HPsj/N/MPXgVFqVqsu0M3uxvp/AX/8L3twM1V+GXGx00ScmcnPcOK4NHIhQa/CZ+z2+8+ZhV6WKBZ6F+Y1qnftltY9y69QR35/mo42JJXqjFbrq/WHvDNgwJs+HkiuKwoDaA1jYbiGZhkzCNoSx4tyKQptq0cSkYFfWhZnbLHMOcE6K7sLMAlBHHMHKXmAbVHSO2mpcrjHLOy3H18WXd3e8y08nfnr8B8/dl1J9e2DtYODOF58XmTnAkmz1xdVMPjCZlxQnvjmyARuDDvoszUrgVVpCbpaQCkHyX39xpX0HUjZuwvM/w6iyYT0urVoVj7XguVTQVz9OTZpQcf48dLG3ubYkCl2doRDxM6x9J2tHaR4Flg7kz85/ElIuhM8Pfs57/7xHkiapQDHmldCqyUwwYlepnMUOdM9JgRK5oijdFUW5AbwAbFAUZYtpwiqYjIuxOFZ2teghy7lR3rk8v7b/lfaV2zP72GzG7R6HWq9+6BpVq7F411ejPnWO1C1F4ttZYv11YTWf/vs/mmm0TL9xHZu2n8Pwg1CrU64SOIDu9m2uDxvGrfETsK1UicqrV1H6vfdQ2dubOfriyTEkhIoLfkIfF8e1n0+hD3oPji+FNcPyPDKHrKJbc1vPZWzwWHbf3E3PdT05EHPADJE/7P6bvbqzhxF6FXY18v9qxRQKumpljRDCRwhhJ4QoI4TI+0JRE9NdPoMuVeBYr3Zhh5Ije2t7pr04jfeC3mNL1BYGbhpIbHrs/1/gVgG3Xn2xc9Nx56svMWq1hRdsCbbu2Dw+/vd/NFarmelcH9sRB6HpyFxvbRdCkBQezpVOnck4HEGZiROptGwp9jVK5lFtpuQYFITvT/PRxcZyfdEpDE0nwMk/YcvEPM+ZA6gUFa/XeZ2lHZbiaO3IkL+HMOXAFDJ0GWaIPsv90ffu8E0AfHo+K5Va6kD3R5W4qZWMHeEAODZ/uZAjydmMrRdQFIW36r7FnFZzuJZ6jX4b+nEy7mT2NcpLYyjdUI3uViyJy5YVYrQlkNHIur//y4eRc2ikNTC76RTs+68At9wfu6dPTOTGyJHETPgAu+rVqRK+hlKvDyhyrwCLMsegICrMnIHm/HluLD2PMWgoHPwB/p2T7zZre9bmz85/MqD2AFacX0HPtT05FHPIhFE/LjAzaxA2ffJwAKKmdSRqWkeLn71a4hK5JvIoipXArmnhv9GZkwfn0V7yfYnf2v+GnZUdb2x5g01Xs36741IW525v4lRWw93vv8eQZNl5vxIrLY4Nv7Xlo1t/00jlxJxXN+FQr0+up1EA0vbu40qXLqTv2k3psWOptORXbCtVMmPQJZdLy5aU/2IqGQcPcmurBlGrG2z9GE78me827a3tGRcyjl9e+SVrwPT3W0zcM5EETUKB481pTf25Q2exdjZi5V24u01LXiK/FI2dlxWKU/FY6lXNoxrLOi6jjmcdxu0ex3fHvsMojNB8NKVDDBjT0rj7w4+FHWbxF72fdYtaMNEYS7CTL3P6/YODR+4TsNBquT3tS64PHoyVmxt+f/6B51tvylF4Abl16ULpCeNJ3bqNuOjaWeUN/hoONwq2aiu4bDCru6xmSN0hbIraROc1nfn93O/ojLpn3/wEo9vUyB5xQ9bou5xGg31pe1CUAq3qKagSlciF0YjmVhr2lQpvPWdOnrU77v4pKd2qdWPeiXmM3TUWta0D9h2H4145nYSlv6GNji7Mp1B8CQH7ZhO+8lU+dLYmxDOAOd1W4ZCH4kva69eJ6h9GwqJFePTvR+WVK7H39zdj0M+XUgMH4t63D/G/LCLJoS84l4U/Xof0uwVq19466/CXVZ1XUbNUTaYenEqPv3rwz7V/TLIizKjVoo3PxM7HCzDdnoZ8yWmXkLk/zLWzM/NspDhT018kTBlqlvZN4Wm744xGo1h4cqGou6iu6L2ut7idcEloP/ETZwNqievvvGPBKEuIzHQh/hgkVn5dTtRdFCCGbn5TqHXqPDWRvGmzONcwWJwLaSSSt2wxU6CSUasV0W+8Ic4E1BXpG5cKMclbiEWdhNDrTNO+0Sh2RO8QnVZ3EgGLAkTYhjCx+/puYTQa89Xe9L/PC/XpU+JMTX+RPO0Nk8SYG5ij1kpRozmQdXq1fdALhRxJ/iiKwqCAQcxuNZuo5Cj6bR3KhRav4VUrmdSt20g/YP5lVSVG8g1Y2I6l17fyqbcnzSu8yOyX52JvnbtlgUKn4/YXX3DzvfewrVqFyqtX49q2rdnCtfQqh6JGsbGhwsyZ2Pr6cuOzOegafQJXd8OOyUDBvz+KohBaMZTVXVfzcZOPuZNxh+Hbh9N3Q182X92MzpC3KZfRbWqQGXkQALtaAQWKzRRKViKPPAIqgX2TQl8F+US5mUdr6duSX9v/irXKmoExm4h4wRUbNytuT/0CoddbIMpi7kYEzG/JT5m3mObpQeuKrZkZOhM7q8fLyuZEd/s20QMHkbD4VzwGDMBvyRKznvoCFOpmkqLCytUV37nfI/R6bszfibHeANg3Ey5tN9n3x0ZlQ++avdnQfQOTmmadHzB291jarGzD7KOzuZ5yPddtaU4cRVEJbGvlvhRIhi4DQz42Pz1LyUrkF69iX0pBcS+6xflzO49Ws1RNlnVcRm3P2owtZc3RxulkXrhA0p/5f0f/uXBmLWJRR2a6OjDbzYFOVTrxzUvfYGuVu/XhGRERXO3RE825c1SY/i1lP5xY4IOFpdyz9fOj/FdfoTl9mthDTgivmhD+Nh6kmLQfGysbulfvzrru65jbei51very86mf6bCmA73X9eanEz9xIfFC1sKDJ8g4fhoHTy1K6af/n07QJLDm4hpGbh9JixUtOHH3hEmfC5SgE4KEEFxsUAeXWh6U+32fSdsuTFqDlsn7JxF+KZwZS434prpSbcsWrNzcCju0okUIODAX/ZYP+bxSDVap1Lxa41U+avJRro4IE0KQuGwZt7+Yhq2PDz7fzcGuWjWzhmyJAzKKq7jZs7k79wf+DGzJhJor2G4M4m3de4Bitu9PbHosW6K2sDV6K5FxkQC42bkRVDqIet71qOxaGT83P8o7l8dGY+BiSAheAWq8V1wBlQqD0cCdjDvcTLtJVEoUJ++eJPJOJFeSryAQlHcqT2jFUPrW7Iufm1++YizxR73poi5zqV0nyvZthMeni03admETQrB09yesOLCaaYuM2L3alWqTphV2WEWH0Qh/f0jmwbmMq1KXHcZkBtcdzLsN3s1VrRNjZiaxn35G8po1OIeGUv6rL7FycbFA4P/PnGePFkfCYOD622+Tvv8AfuO74HBpDnT5DoIGWKT/2PRYDsYc5MjtI0TcjuB66sNTLg2vqhi/XMv3fW04VasUWoOWDH0GBvH/0ybudu7U865Hfe/6tPBpQU2PmgWuvfOkRF5ijm3fumQZlQH7BoVY3N1MFEXhtRaTqBq1n50NbtHyz7/YH1qXF0LDCju0wqfXQvjbpJxZxbvVAzmiS2BCowmE1crd90Z35w43R76LOjISrxEj8BoxHCUXlQ4l81KsrCj/5Zdc7dGTm4sOcqdhLZpuGp91lF4e1v/nV1mnsnSt1pWu1bKqdadp04hOieZqylVi02Mpe2o7RtUxSlUpS6uKzbGzssPB2oHyzuUp71weX2dffFx8LFY0rcQk8rQjB0AR2DVqU9ihmIei8EKbb6hwuTW3z5chbvIUfvJI5636g4v06eJmlZkGK17jxrXdjKhWl2uGFL5q8RXtK7fP1e3qkye58c5IDCkpVJg9y6yrUp6lMDeTFFXWHh5UmP4t0QNex3ClLiLwBsq6d2FAeJ5245qCs60zdbzqUMerDgBRVzeAh45J9d6AwH4WjSUnJSYDVEy6jZ27QFW2BP+HKB9IxaA+VKqXQvVbglOLZzHqn1EkZyYXdmSWl5EAv3Yh8tZ+wvyqc1cxMr/N/Fwn8ZSNG4l+Las+it/vywo1iUMhbyYpwhwbNKD06NF4nTxGIl3hyk449luhxmRUq1GfvYBj6Uwob9mjJJ+kWCfy7B2T49fjmphOips9fh9sLNlrcl/+FO/qCg4VHRm8x47IC3vos74Pp++eLuzILCclBhZ2YGPKJd4qXx4nR09+6/AbIWVDnnmrMBqJm/MdN/87BvuAAPxW/il3aRZxpd4YhHPLltz+fTdquxDY8mHWz0AhUUdGgsGIYzkFvIrGL+Bincjv1z64+F4QSiZUbFirUCqPWZRzaZTQCZStHYV1RiZzzzfBIAwM2DSApWeXlvzDKBKuov+lLd+Iu4z3cqOOdz2WdlhKZbfKz7zVqFZzc8wY7n7/PW7du1Nx4S9YlyqVp+5L9CChiFJUKsp9MRVrLy9ubjdi0Ghhw3/zVfLWFDIOR4ACDgH+oCoatXaKdSK/T7P/3o7O+o+9mVsyNRqKfbWqeNZXsNq0i988x/JC+ReYdmgaI7aPIF4dX9gRmsedcyQubMfbjnoWuzjQz78fC15ZgIf9swuk6ePiiH59IKmbt1D6/TGUmzoFVT7Wh8uNO4XD2sODCl9/he7WbW5fawznN8KZPJ02aTIZEYex99BjVTmoUPrPSYlI5JnHs7bK2jcqOke7mdOMHVeh/Zd4VbuObRlXUj//itmNv2RCowkcjDlIz7U92XNjT2GHaVq3jhOxtBO9PGw4Ym/PpKaTmNh4IjYqm2feqjl3jqu9+5B56RI+c2bjOXhwiTqC7XnhGBKC1/DhJO+/QHJiDdg8ATSm3Sj0LEatFvWx4zh6a6Bc0Zgfh5KSyC9fwsbJgMq3bmGHUmC5eek+a/tFqBqKql4PytW7hu7WLeJmzSasVhi/d/odD3sPhm8fzif7PiFVm2qBqM3LEL2fH1b14i0PexxcK/Bbx6V0r949V/em7txJdP8wMBrxW/obLi/n/cCRZ1WvlCzH6+3/4BgcTMwuHZk342DH5xbtX3PqFEKrxdFbC+VzvzXf3EpGIr8Zh62XHVgV/9WUeXrp3v4rHH3s8Ah0IvG338g4eowaHjVY3mk5bwW8xV+X/6L7X93ZfWO3+QI2s6hTK3hjyxvMdXWgg28rVnRdTW3PZx/jJ4Qg4dcl3Bg+Als/P/z++AP72vk7/i+nOtQl/r2YIkqxsqL8N1+jsrPnZmQ1jPvnw82jFus/49BhABzKW4NX0VkhV9DDl79WFOWcoignFEVZoyiKu6kCyy1hMKCN02TXBC6pchwVfn6IzT7v4V3lIjaeLtz6YALG9HTsrOx4r+F7LO2wFGcbZ0ZsH8Hof0Y/fDZoEWcwGlj8zwR6HZ7MZVsbpoZMYGqrWTjZOD3zXqHXc3vyZG5PnYpzq1Aq/bYEmzKlLRC1ZAk2ZctS7osvyLyZQtzZMrD+PTBYpphc6rZt2JexxdqvbpF5oxMKPiLfCgQIIeoBF4APCh5S3uiiLyMMYFeliqW7NpncvHR/0qiwXf9RWPm3olxQDLpr17k97f+37gd4BfBH5z8YFTSKvTf30iW8CwtOLkCj11j2CeZRZFwkYas68M21DbyAHeGdV9O5dliu5rUNqalcH/YfEpf9sHQXyQAAEqFJREFUTqm33sRn9mxUjrk/ROJZ5MadosGlVSgeAwaQcEpFasQ5OLzA7H1mXrmC5tQpXH2Si9T8OBRwZ6cQ4u8HvjwA9CpYOHmXeSyrQJZd7XqW7tpkRrepkf0yPc81NxQFOs/E6VoTPJu4E//nSpxatMC1TdYOV1srWwbXHUz7yu2Zdmgas47OYvm55YwIHEGXql2wKkKjiriMOGYencnay2sprdfzpaoM7futRbF3zdX92uvXuf7222ijoin3+WTce5n+x1FOpxQdpce+T0ZEBDER57EvMxWbOt3ApazZ+ktetw5UqqxEXkQ2At1nyjnyN4FNJmwvVzJPHwPAtl5TS3ddaB4bFbpXhPbT8PY5jX1FL2I//gTd7TsPXVLBuQJzWs3hl1d+wdvBm0/+/YSea3uy7vK6Ap1jaArx6ni+OfwNHVZ3YOPl9byVlMw6+wA6vLYp10k848gRonr3QR93l4oLFpgliUtFi8rWlgrTv0Vgx82dtoiNE8zWlxCClLXrcArww8bBWKTe6IRcJHJFUf6vvTuPjqrKFjj826lUUhkJgZAACSE4RMSoICKoNCjiA0UccEBtG+f5tdqwFORpg3bbLaggou1A+5yYxAlRURkERQRFBYTFYAIkJAQTQkLIVFWpOu+PQpsnJCmSqlRusr+1shaV1L13X5K116l9z9lnqYhsOsrXpYe9ZyJQC8yu5zy3i8g6EVlXXFwcmOgBV0424VEebOnWHZEfzp+P7kcdFfa+ATn1CrpkbcVbXcWehx466iYUZ6acyZyL5zB10FREhIdXPcyI90Ywe8tsyl3NO5UrrzyPJ799kuHvDefNLW9yob0jH+7O5/7U/yJ69BywR/l1nrL33if3xpuwxcfTfd5cYvqfFeTIVUsRmZFB5yf+TvU+O0VzlkHOF0G5TvWPP+IuKCC+VyzYY6BDcFscH6smt7EVkTHAncAQY0yVP8cEso3tzvPPwBZWQ7elbWiJel2qy+ClgZRt9VK4wkuH226l09ixdb7da7x8mf8ls36axYbiDThsDi7sfiGXH385vTv1/q3sMm3J9oCVFJweJ6sKVrFg+wK+LviacAlnWPqF3FFUQPfNi+DM22D4FPCjA6HxeCh6+hn2v/oq0QP6kzp9uvZpb6P2PjaZ0jnz6DosgvinvoVw/3aD8lfhpEkc+GAhJ4x2Yks/Ha6dE9Dz+ysobWxFZBjwEDDI3yQeSMYYnMVVJPRJau5Lt0xRCTDq3yS8Oozqvr0oeWUWjqysOhtChUkYg9MGMzhtMJtLNvPu9nf5ZOcnfJjzIYmORM5LO4/BaYN59oviJiXyfdX7WPfLOpbnLWfl7pVU1VaRFJXE3afdzZXdh5G06C+QsxzOfwQGjvWrs52nvJyCceOo/PIr2l93LckTJiD2hhcHqdYpefwEqtd9Q+GynUS+O4nIa/4RsHMbl4vyxZ8Sd3YfbDXvwEkTA3buQGnSiFxEsoFI4Nc14WuMMXc2dFygRuTu3blkDx1GyugzaT/pjSafr9VYPRPv4onkfpeFq6ia7u8s8HtWT5W7ihW7V/DF7i/4quArKt2VAPRo14Osjll0b9edtLg0UmNTiY2IxWFz4Ah34PQ4qXRXUuGqoKCygLzyPHYd2MXGfRvJLc8FfI32h3QbwtD0ofTr3A97RTHMHQ17N8HIGdD7j37F6MzJIf/ue3AVFJDyPxNpP3p04/6fVKviLihg5yXDsNlq6D5/PrYegVlCf3DZMvLvuZe0u88jtnQujMuGmA4BOfexCsqI3BgT0kKRc/2hGSs9rb+iM6AG3ENY8VZSnbPZuaIH+XfdTfq8uYS3b7gnSbQ9mot6XMS2nOPZ+9O52KJysUXlsf1gLtklywgLr/A7jE5RnTi548lcecKV9EnuQ88OPf+zpL5wA8y5BpwH4dp5cKJ/bWQPLlvGngcfQhwO0l9/jegzzvA7HtW62bt2JXXGNHJvv5f8u26h24drkMiml1hK587DlphIjPwI3c4OWRKvj6WXQjo3fQ9AxGn9QxxJCyMCFz+DvXQXqdXfk7fSTf6dd9Httf8lLMq/B4h1TYmsdFeSfzCfPRV7qKytpLq2mpraGiJtkcTYY4ixx9A5pjNpcWlE2+uYv731Y3j3VohKhJs/g5RTGozH1NZS/OyzvnLRKaeQ+twM7J1b7ibbKjSiz72AzreNoPClj/ll7M2kzKxz/oVfKlaupHLVKjrdewuy73HoG7iSTSBZO5Fn/4wt0kN4Dx2VHSE8Aq5+g+jyIXSpLaNgxUYKxo4jdcazSHjjf+0x9hgyEzPJTMw89oO9HljxD/hyKnTp4xuJxyU3eFhtSQkFfxlL1dq1JFx9NckTHyYsACMt1Tol3D8V54+r2L/0B8JnTqPjvQ806jzG5fJtxp2RQeKpdlgOnNQy91W1dK8VV14hkYnhEBG4lXutSnQi3PAB8ZmxJPeroWL5cvZOnozxeo/pNAFZzVi1H2Zf6UvivW+Amxb7lcQrV69mx2WXUb1+PZ2feILOj03WJK7qJ0Knqa8Rn+6keObLlLz6aqNOs//Nt3Dt2kXyhPFI9mJIyWqW/UIbw7KJ3BiDs6iSiC4N133btPbpcOPHJPaOpUOWi7IF71D4yCMYj6fhYw9p8tTDHSvhxYGw62u4ZAZcOhPsjnoPMW43Rc9MI++WW7HFt6P72/NJuMK/jodKSfJJdHnkAeLSqimaMpWS1147puNri4vZ98ILxA4aRGyfnpC3Bk4aEZxgA8Cyibz2l714nYbIjLRQh9LyJXSDGz8m6ZwYOp5aw4F332PP+AlHXTAUUO5qWDwe3hjpS9y3fAZnjGnwMGdODruuu56Sl18m4cpRZCx4G0dmI0o5qk2Ts++h6+hexHVzU/TPJymaNt2vv3mvy0XhpMl4XS6SJ4yH7YsB02LLKmDhGrlrw2oAIjMbflCmgIQ05JYlJMWPQfiJ4kWLMNVVdJkyJaBNpX6Tsxw+eRBKfoZ+t8MFkxssgRmPh/2vv0Hx9OmERUfTdfo04ocNC3xsqm0IC0NGvUjXvedQuD6BkpdeomrtWro89RQRqV2PekhtaSkF//1nqtatI3nCeCLS02HpG5CQDsktN9dYdkTu/Mk3Dz3iVJ2x4re4ZPjTh3S88XqS+xzg4NJl7Lp6FK78gsBdo3QXzLse3rwcvG644X24aGqDSbxmyxZyr7ueoilTiBk4kB4fLdIkrpouIQ255Cm69PqZLjcNwJmdzc7LL2ffiy/hysv77W3GGGq2biV39LVUb9hAl6eeInHMGNj8PuR/B38Y59dCtVBp8hL9xgjEgqC9d43iwKpNnPjtWiSq2dugW9/Gt6l4ZTwFK+xIRCRdn36amMFDG3++4m2wegZsmA82u+8Pv/89DdbCPRUVFM+YQelbs7ElJJA8/iHiL7lEt2JTgWMMLPoz/PAGrgFPUDh3HVVr1gDg6NULcThwbtuGt6ICW0ICqc/P9K1PcNfA82dCZDzc8WWL6D9e14IgyybyvJHn4CkpIePrrQGKqg2q2o9r/gR2/2sZrvJw2g/MoNOESYT18LPpVM0B2P45/LQAfv4MwqN8qzPPfQDaHf2j66+8Lhdl8+az78UX8ZSWkjD6Gjrdf7/2SlFNUmdfII8b3hoFuavhTx/gtmdQ/ulnHFyyBMLCcJyUSeSJmcSeNxh78qHZVKumw9K/wg0fwHHnNe+N1KHVJfLs/lk4kiNIXfh9gKJquzw5ayl+bAKlawuxR9eSckE7Ys8dAJ1Pg6SeEO7wfaz01kJJtm/0vXcj7PzKVz6JTYEzboR+t0FM/Ts1eV0uyhctYt/zL+Des4fos86i07hxRGW13Pqjso56+/lXl8G/h0JFEdzyOSTV8wC9ohie6wPpZ8N184MTbCMEZYl+qJjaWtwH3MSfXv+oT/nHdtxZpLy+nPhvVlL48MPs/nA/UWs/p+NJ84lJcR5ZGgyz+/Yr7H8n9BwJXfs22K2wtqSE0rnzKJ07F09JCY5evUh5/DFizj5byyiqeUQlwHVv+5L5K0N8/X1OueLI9zkr4OMHwFUJQx9v/jgbwZKJ3J2XA0aI6KZTDwMpesAgMj5dTtmCdyiZNYvdK8NwHJdGu8F9iRuQhb1TIiT2gPbdfXXwBnjKyji4bBnlnyymcs0a8HiIGfQHOowZQ/SAAZrAVUBMW7L9/21a/uuWifcNOeHIMktiBtz2Bbxzk+8r92u4YBJExvl+nvsNfHCX76H9kEchyRo7QlmytFLx0Wx2j/sb3SbfSsw1dffbVo1nXC7KFi6k9K3ZOLdtA8CRlUVUVhaRJ55I5AnHExYXR5jDgdjteMrKqC0qwl24l5rNm6hevx5ndg4Ygz0tjfjhw2l32aWW3ltVtXx+b5XoccOyybD6Od/ruM7QLs03QyUhDS77F3Q/N7jBNkKrKq24t28CIKJny9puqTWRiAjaX3UV7a+6CufOnRz8fAkVK1dyYOFCvJWV9R4bFh9P1OmnETd8OLED/4DjlF46+lYti80OF/4NMi+G3FVQsgP25/ie8wx59D8jdIuwZCJ37dqB2Azhxwem37CqX2RGBpF33E7HO27HGIO7YA+uHTl4q6rw1tRgXC5sCQnYO3UiPCmJ8M6dET92+FEq0I65L1D6AN+XxVkzkefvwR4HEq3zx5ubiBCR2rXOlXFKhVKgtiS0GksOm9xFB4joqB0PlVIKLJjIjdeLq9RNRErL26VDKaVCwXKJvLYgD+MBe5pOPVRKKbBgIndv+Q6AiOPaZi1MKaV+r0mJXEQeF5GNIrJeRD4XkS6BCqwurm0/ARCReXqwL6WUUpbQ1BH5VGPMqcaY04GPgEcDEFO9XDuzQQz2nmcG+1JKKWUJTUrkxpjyw17GAEFfJurK34M9FiRWH3YqpRQEoEYuIn8Xkd3A9dQzIheR20VknYisKy4ubvT13EVlRHSov8e1avmmLdke6hCUajUaTOQislRENh3l61IAY8xEY0waMBu4t67zGGNeNsb0Ncb0TUpKalSwxhhc+51EJCc26njVchze5Egp1TQNruw0xlzg57nmAB8Df21SRPXw7NuL1wV27XqolFK/adISfRE5wRjz69BqJBDU7Xrcm78FICLj+GBeRgXJMbUbVUr5ram9Vv4pIpmAF8gF7mx6SHVzbdsIQETmacG8jAqSB4ae+FvC9rvdqFKqQU1K5MaYUYEKxB+uHb7RnP3kfs15WaWUatEstbLTs+8XwmMMYe1TQh1Kq9YcM0qOud2oUqpOlkrkKU/P4vj5r4Q6jFavOWaUaE1cqcCxVj/yhDQkQWesKKXU4ayVyFXQ6IwSpazLkpsvq+DSGSVKtUx1bb5sqRq5UkqpI2kiV0fQGSVKWYsmcnUErYkrZS2ayJVSyuI0kSullMVpIldKKYvTRK6UUhaniVwppSwuJAuCRKQYX9tbq+kI7At1EM2ord0v6D23FVa953RjzBFbrIUkkVuViKw72qqq1qqt3S/oPbcVre2etbSilFIWp4lcKaUsThP5sXk51AE0s7Z2v6D33Fa0qnvWGrlSSlmcjsiVUsriNJE3goiMExEjIh1DHUuwichUEdkqIhtF5H0RSQh1TMEiIsNEZJuIZIvI+FDHE2wikiYiX4jIFhHZLCL3hTqm5iAiNhH5UUQ+CnUsgaKJ/BiJSBowFMgLdSzNZAlwijHmVGA7MCHE8QSFiNiA54HhwMnAtSJycmijCrpaYKwxpifQH7inDdwzwH3AllAHEUiayI/dNOBBoE08XDDGfG6MqT30cg2QGsp4gqgfkG2M2WGMcQHzgEtDHFNQGWMKjTE/HPr3QXzJrWtoowouEUkFLgZmhTqWQNJEfgxEZCRQYIzZEOpYQuRmYHGogwiSrsDuw17n08qT2uFEpDvQG1gb2kiCbjq+gZg31IEEkm6+/DsishRIOcqPJgIPAxc2b0TBV989G2MWHnrPRHwfxWc3Z2zNSI7yvTbxqUtEYoF3gfuNMeWhjidYRGQEUGSM+V5EBoc6nkDSRP47xpgLjvZ9EckCMoANIgK+EsMPItLPGLO3GUMMuLru+VciMgYYAQwxrXe+aj6QdtjrVGBPiGJpNiJix5fEZxtj3gt1PEF2DjBSRC4CHEC8iLxljPljiONqMp1H3kgisgvoa4yxYuMdv4nIMOAZYJAxpjjU8QSLiITje5g7BCgAvgOuM8ZsDmlgQSS+EcnrwH5jzP2hjqc5HRqRjzPGjAh1LIGgNXLVkJlAHLBERNaLyIuhDigYDj3QvRf4DN9Dv7dbcxI/5BzgBuD8Q7/b9YdGq8pidESulFIWpyNypZSyOE3kSillcZrIlVLK4jSRK6WUxWkiV0opi9NErpRSFqeJXCmlLE4TuVJKWdz/AW/J0xulQDAoAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# get the MAP estimate\n",
    "K = 8 # polynomial degree   \n",
    "\n",
    "\n",
    "# feature matrix\n",
    "Phi = poly_features(X, K)\n",
    "\n",
    "theta_map = map_estimate_poly(Phi, y, sigma, alpha)\n",
    "\n",
    "# maximum likelihood estimate\n",
    "theta_ml = nonlinear_features_maximum_likelihood(Phi, y)\n",
    "\n",
    "Xtest = np.linspace(-5,5,100).reshape(-1,1)\n",
    "ytest = g(Xtest, sigma)\n",
    "\n",
    "Phi_test = poly_features(Xtest, K)\n",
    "y_pred_map = Phi_test @ theta_map\n",
    "\n",
    "y_pred_mle = Phi_test @ theta_ml\n",
    "\n",
    "plt.figure()\n",
    "plt.plot(X, y, '+')\n",
    "plt.plot(Xtest, y_pred_map)\n",
    "plt.plot(Xtest, g(Xtest, 0))\n",
    "plt.plot(Xtest, y_pred_mle)\n",
    "\n",
    "plt.legend([\"data\", \"map prediction\", \"ground truth function\", \"maximum likelihood\"]);"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[-1.49712990e+00 -1.08154986e+00]\n",
      " [ 8.56868912e-01  6.09177023e-01]\n",
      " [-1.28335730e-01 -3.62071208e-01]\n",
      " [-7.75319509e-02 -3.70531732e-03]\n",
      " [ 3.56425467e-02  7.43090617e-02]\n",
      " [-4.11626749e-03 -1.03278646e-02]\n",
      " [-2.48817783e-03 -4.89363010e-03]\n",
      " [ 2.70146690e-04  4.24148554e-04]\n",
      " [ 5.35996050e-05  1.03384719e-04]]\n"
     ]
    }
   ],
   "source": [
    "print(np.hstack([theta_ml, theta_map]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, let us compute the RMSE for different polynomial degrees and see whether the MAP estimate addresses the overfitting issue we encountered with the maximum likelihood estimate."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Applications/anaconda3/lib/python3.7/site-packages/ipykernel_launcher.py:13: LinAlgWarning: Ill-conditioned matrix (rcond=1.82839e-17): result may not be accurate.\n",
      "  del sys.path[0]\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<matplotlib.legend.Legend at 0x7f95a1c7c090>"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZ8AAAEGCAYAAAC6i5gfAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nOzdd3hVVdbA4d9KhySEmlBC7xCQLtJRVECKBRUGHRDR0dGxjGXU0RHrZ0XHccSK6KCggiAoIlgQUaSK1BtqgEAKLclNIPXu749zEwOk57Yk632ePLn3nHPPXidiVvY++6wtxhiUUkopT/LzdgBKKaVqHk0+SimlPE6Tj1JKKY/T5KOUUsrjNPkopZTyuABvB1BVNGzY0LRq1crbYSilVJWxadOm48aYRkXt0+RTRq1atWLjxo3eDkMppaoMETlY3D4ddlNKKeVxmnyUUkp5nCYfpZRSHqf3fCohJyeH+Ph4MjMzvR2K8jEhISFER0cTGBjo7VCU8kmafCohPj6e8PBwWrVqhYh4OxzlI4wxnDhxgvj4eFq3bu3tcJTySTrsVgmZmZk0aNBAE486i4jQoEED7RErVQJNPpWkiUcVRf9dKFUyTT5KKaWK9ENsMrNW7SPP4fqldzT5VHEiwo033ljwPjc3l0aNGjFmzJgKnW/JkiU899xzrgqv3IYNG1bwMO/o0aNJSUkhLi6OmJiYMp+j8PEbN27krrvuAmDGjBm89NJLLo/ZXedVytu+2prA7J8P4O/n+p68Tjio4kJDQ9m+fTtnzpyhVq1arFy5kmbNmlX4fOPGjWPcuHEujLDili1bBkBKSkqFz9GnTx/69OnjqpCUqlFsiWl0ahzulnNrz6caGDVqFF999RUA8+bNY9KkSQX71q9fz4ABA+jZsycDBgwgNjYWgJkzZzJt2jQAtm3bRkxMDKdPn2bOnDnceeedAEydOpXbb7+d4cOH06ZNG3788UemTZtG586dmTp1akEbYWFhBa8XLFhQsK+sny9Oq1atOH78+Fnb9u/fT8+ePdmwYQN5eXk88MAD9O3bl+7du/PWW2+dd45Vq1ad1QvcuXMnw4YNo02bNrz22msF22fOnElMTAwxMTG8+uqrpW5/5pln6NixIyNGjCj4mSpVneTmOdiTlO625KM9Hxd5YukOdh5Nc+k5uzStw+Nju5Z63MSJE3nyyScZM2YMW7duZdq0afz0008AdOrUidWrVxMQEMC3337LI488wsKFC7nnnnsYNmwYixYt4plnnuGtt96idu3a55371KlTfP/99yxZsoSxY8fy888/8+6779K3b1+2bNlCjx49Soytsp8vLDY2lokTJ/L+++/To0cP3n77bSIiItiwYQNZWVkMHDiQyy67rMSb/TabjR9++AG73U7Hjh25/fbb2bp1K++//z7r1q3DGMOFF17I0KFDcTgcxW6fP38+v/32G7m5ufTq1YvevXuX+TqUqgriTpwmK9dBp8Z13HJ+TT7VQPfu3YmLi2PevHmMHj36rH2pqalMmTKFPXv2ICLk5OQA4Ofnx5w5c+jevTt/+ctfGDhwYJHnHjt2LCJCt27diIqKolu3bgB07dqVuLi4UpNHZT+f79ixY4wfP56FCxfStauVkFesWMHWrVtZsGBBwbXu2bOHDh06FHueK664guDgYIKDg4mMjCQpKYk1a9Zw1VVXERoaCsDVV1/NTz/9hDGmyO0Oh4OrrrqqIFn7yjClUq5kS7T+mO6oPR/fVpYeijuNGzeO+++/n1WrVnHixImC7Y899hjDhw9n0aJFxMXFMWzYsIJ9e/bsISwsjKNHjxZ73uDgYMBKVvmv89/n5uYCZ08rPvfZlrJ8viwiIiJo3rw5P//8c0HyMcbwn//8h8svv/ysY+Pi4kq9HgB/f39yc3MxpuiZPMVtB51Kraq/2EQ7/n5Cu8iw0g+uAL3nU01MmzaNf/3rXwU9i3ypqakFExDmzJlz1va7776b1atXc+LEiYLeQ0VERUWxa9cuHA4HixYtqvB5ShIUFMTixYv58MMP+fjjjwG4/PLLmTVrVkFvbvfu3WRkZJT73EOGDGHx4sWcPn2ajIwMFi1axODBg0vcvmjRIs6cOYPdbmfp0qUuvValfMGuBDttGoYSEujvlvNrz6eaiI6O5u677z5v+4MPPsiUKVOYOXMmF198ccH2e++9l7/+9a906NCB9957j+HDhzNkyJAKtf3cc88xZswYmjdvTkxMDOnp6RW+jpKEhoby5ZdfcumllxIaGsr06dOJi4ujV69eGGNo1KgRixcvLvd5e/XqxdSpU+nXrx8A06dPp2fPngDFbr/++uvp0aMHLVu2ZPDgwS66QqV8hy0xjV7RYWAMuKGnLyUNLag/9OnTx5y7mNyuXbvo3LmzlyJSvk7/faiqyp6ZQ7cZK1jYfiW9U5bDvTvAr/w9IBHZZIwp8lkHHXZTSil1lt1JdgBa5u6HWvUqlHhKo8lHKaXUWWyJVvKpa98DUWWvLlIemnyUUkqdxZZgp1lwJgHpR6GxJh+llFIeYEtMY0T9ZOtNlHseI9Hko5RSqoAxBluinX61nc//RXUr+QMVpMlHKaVUgaOpmdgzc+nIIQhtBOFRbmlHk08VV9YlFcaPH89FF1101rYZM2bQrFkzevToQUxMDEuWLPFIzEop32VLsMrqNMnc67YhN9DkU+UVXlIBKHJJhZSUFDZv3kxKSgoHDhw4a9+9997Lli1b+Oyzz5g2bRoOh8NjsSulfI8t0Y4/edROcd9MN9DkUy2UtKQCwMKFCxk7diwTJ05k/vz5RZ6jc+fOBAQEnLeEgVKqZrEl2rko4hSSlwWN3XO/B7S8jut8/RAkbnPtORt3g1Glrypa0pIKYCWkxx9/nKioKCZMmMDDDz983jnWrVuHn58fjRo1cuklKKWqFltCGtfWSYRjuLXno8mnGihpSYWkpCT27t3LoEGDEBECAgLYvn17wTLTr7zyCnPnziU8PJxPPvlEqzUrVYNl5eax/3gGF7SKB79AaFj88iSVVaOTj4iEAm8A2cAqY8xHFT5ZGXoo7lTckgqffPIJp06donXr1gCkpaUxf/58nn76acC653P//fd7JWallG/Zm5xOnsPQOu8ANOoIAUFua8ut93xEZLaIJIvI9hKOiRORbSKyRUQ2FndcZdoTkZEiEisie0XkoUK7rgYWGGNuAar0imDFLakwb948li9fTlxcHHFxcWzatKnY+z5KqZrNlmCV1amfvtutQ27g/gkHc4CRZThuuDGmR1HVT0UkUkTCz9nWrqztiYg/8F9gFNAFmCQiXZy7o4HDztd5ZYjTZxW1pEJcXByHDh2if//+Bdtat25NnTp1WLdunadDVEr5uNgkO1EB6QRkJLl1mjW4edjNGLNaRFpV8jRDgdtFZLQxJlNEbgGuAkafe2Ax7fUD9hpj9gOIyHxgPLATiMdKQFsoJhGLyFhgbLt2xeU77ypq7Zxhw4YVrFh65MiR8/Zv3rwZgAsvvNCtsSmlqpZdCWlcXO8Y2HFbTbd8vjDV2gArRGSTiNx63k5jPgOWA/NFZDIwDbiuHOdvxh+9G7ASTv6DMJ8D14jILKDI5SiNMUuNMbdGRESUo0mllKp6bIl2+tdOsN64qaxOPl+YcDDQGHNURCKBlSJiM8asLnyAMeYFZ49lFtDWGFOepTKLmr5lnOfNAG6qaOBKKVVdnEjP4pg9i84ND0JoJIS597ELr/d8jDFHnd+TgUVYw2RnEZHBQIxz/+PlbCIeaF7ofTRwtELBFkFXglVF0X8XqqqJda7h0zRzn9uH3MDLyUdEQvMnEzinPV8GnDtTrSfwDtZ9mpuA+iLydDma2QC0F5HWIhIETARcUsQsJCSEEydO6C8adRZjDCdOnCAkJMTboShVZrZEOwHkEpq21+0z3cDNw24iMg8YBjQUkXjgcWPMeyKyDJgOhACLnA82BgAfG2OWn3Oa2sC1xph9znNOAaaWs707gW8Af2C2MWaHK64vOjqa+Ph4jh075orTqWokJCSE6Ohob4ehVJnZEtPoHXocyct2a1mdfO6e7TapmO2FZ6pdUMo5fj7nfQ5WT6g87S0DlpUYbAUEBgYWPLyplFJVmS3RzuURSXASt0+zBh+456OUUsq78hyG3Ul2egYdAf8gt5bVyafJRymlariDJzLIzHHQJr+sjn+g29vU5KOUUjWczTnTrYEHyurk0+SjlFI1nC3RTiNJJfDMMU0+SimlPMOWkMawusnWGw884wOafJRSqsazJdq5KDTReqM9H6WUUu6WkZXLoZOn6ex3CMIaQ2hDj7SryUcppWqw2CRrskGzLM+U1cmnyUcppWowW4KdQHIJt+/z2JAbaPJRSqkaLTYxjZjgZMSRo8lHKaWUZ+xKtDM0wjnZQIfdlFJKuZsxBltCGr2CnWV1GrT3WNuafJRSqoZKTMskLTOXdo4D0KgT+HtufVFNPkopVUPZEqyZbg0z9npkGYXCNPkopVQNZUu005BUgjKPe3SyAWjyUUqpGsuWmMbgcM9PNgBNPkopVWPZEuz0D0uw3mjPRymllLtl5zrYdyydGP9DEN4Uatf3aPuafJRSqgbadyydXIehefZ+jw+5gSYfpZSqkWyJaVZZnfT9ENXV4+1r8lFKqRrIlmins38C4sj1+P0e0OSjlFI1ki3BzpCCsjqefcYHNPkopVSNZEtMo0/IEfAPhvptPd6+Jh+llKphTmVkk5SWRTtzECI7e7SsTj5NPkopVcPYEu2AIfL0Hq/MdANNPkopVePYEtNoRApBWSchyvP3ewA839fyISISCrwBZAOrjDEfeTkkpZRyu9hEO/1qJ4ADr0yzBjf3fERktogki8j2Uo7zF5HfRORLd7QnIiNFJFZE9orIQ4V2XQ0sMMbcAoyrTNtKKVVV7Eq0MzDUWVanmg67zQFGluG4u4FdRe0QkUgRCT9nW7uytici/sB/gVFAF2CSiHRx7o4GDjtf55UhTqWUqtIcDsPuRDsxgYehTjTUqueVONyafIwxq4GTJR0jItHAFcC7xRwyFPhCREKcx98CvFaO9voBe40x+40x2cB8YLxzXzxWAoJifhYiMlZE3k5NTS3pMpRSqko4dPI0Z3LyaJFzwGtDbuAbEw5eBR7EGn08jzHmM2A5MF9EJgPTgOvKcf5m/NG7ASvhNHO+/hy4RkRmAUuLaX+pMebWiIiIcjSplFK+yZaYRhA5RKQf8NqQG3h5woGIjAGSjTGbRGRYcccZY14QkfnALKCtMSa9PM0UdUrneTOAm8pxLqWUqtJsiXba+x1BjHfK6uTzds9nIDBOROKwhsMuFpG55x4kIoOBGGAR8Hg524gHmhd6Hw0crVC0SilVxdkS7H8sIFdTk48x5mFjTLQxphUwEfjeGHND4WNEpCfwDtZ9mpuA+iLydDma2QC0F5HWIhLkbGeJSy5AKaWqGKuszlEIqAUNPF9WJ5+7p1rPA9YCHUUkXkRudm5fJiJNy3ia2sC1xph9xhgHMAU4WNb2jDG5wJ3AN1gz6j41xuyo3JUppVTVczo7l4MnT9MRZ1kdP3+vxeLWez7GmEnFbB9dxLZVwKoitv98zvscrJ5QedpbBiwrNWCllKrGdielY4wh6sweaOvdRxu9fc9HKaWUh9gS0ogkhaDsFK/e7wFNPkopVWPYEu30DHI+eaLJRymllCfYEtMYWDDTzXsPmIImH6WUqhGMMdgS7XQPjIeI5lCrrlfj0eSjlFI1QLI9i5TTObTK2e/1ITfQ5KOUUjXCroQ0gskm4vRBr5bVyafJRymlagBbop12cgQxeV6/3wOafJRSqkaITbRzUaizspiXVi8tTJOPUkrVALsS0uhX6ygE1ob6rb0djiYfpZSq7nLyHOw7lu4sq9PFq2V18mnyUUqpam7/sQxy8hw0ydznE/d7QJOPUkpVe7bENBpzkqCcVGjs/fs9oMlHKaWqPVuinW4Bh6w3PvCMD2jyUUqpas+WkMaAsCTrTVQX7wbjpMlHKaWqOVuinQsCD0PdFhAS4e1wAE0+SilVraWeziEhNZPWeQd84vmefJp8lFKqGrMlWmV16p455DMz3UCTj1JKVWu2RDsdJB4xDp+o6ZZPk49SSlVjtkQ7vUPirTc+MtMNNPkopVS1ZktM48JaRyEwFOp5v6xOPk0+SilVTTkchthEO539DltTrP1851e+70SilFLKpeJPneF0di5NM/f61JAblJJ8ROTiQq9bn7PvancFpZRSqvJ2JabRlBME5dp9arIBlN7zeanQ64Xn7HvUxbEopZRyIVuCnc7+vlVWJ19pyUeKeV3Ue6WUUj4kNimN/rUTrDc+9IwPlJ58TDGvi3qvlFLKh9gS7PQMiod6rSA43NvhnCWglP1tRGQJVi8n/zXO974zZ08ppdRZzmTnceBEBm3qxvlUWZ18pSWf8YVev3TOvnPfK6WU8hF7ku0EmyzqnTkMUZO8Hc55Skw+xpgfC78XkUAgBjhijEl2Z2BKKaUqzpZgp6McRvCtsjr5Sptq/aaIdHW+jgB+Bz4EfhMR30ul5SQioSLygYi8IyKTvR2PUkq5irWMgu+V1clX2oSDwcaYHc7XNwG7jTHdgN7Ag6WdXERmi0iyiGwvZn+IiKwXkd9FZIeIPFGu6MvYnoiMFJFYEdkrIg8V2nU1sMAYcwswrjJtK6WUL7ElptGv9lEICoO6Lb0dznlKSz7ZhV5fCiwGMMYklvH8c4CRJezPAi42xlwA9ABGikj/wgeISKSIhJ+zrV1Z2xMRf+C/wCigCzBJRPKX8osGDjtf55V2MUopVRUYY7Al2unid9iaYu1DZXXylRZRioiMEZGewEBgOYCIBAC1Sju5MWY1cLKE/cYYk+58G+j8OncK91DgCxEJcbZ9C/BaOdrrB+w1xuw3xmQD8/ljIkU8VgKCYn4WIjJWRN5OTU0t7jKUUsqnHEvP4mRGFs2y9/nkkBuUnnz+AtwJvA/cU6jHcwnwlSsCEBF/EdkCJAMrjTHrCu83xnyGlfTmO+/LTAOuK0cTzfijdwNWwmnmfP05cI2IzAKWFvVhY8xSY8ytERG+sfSsUkqVxpZgpxnHCc5N97mHS/OVNtttN0UMmxljvgG+cUUAxpg8oIeI1AUWiUiMMWb7Oce8ICLzgVlA20K9pbIoqhKDcZ43A+tellJKVRu2xDQ6+znL6jT2vWd8oJTkIyJFDm/lM8bc5apAjDEpIrIKK9mdO2FgMNYU70XA41i9sbKKB5oXeh8NHK1UsEop5cNsiXb6hBwBh0Bkl9I/4AWlDbvdBgzC+mW9Edh0zleliEgjZ48HEakFjABs5xzTE3gH6z7NTUB9EXm6HM1sANqLSGsRCQImAktK+YxSSlVZtgQ7vYKPQP3WEBzm7XCKVFryaQK8DVwO3Ig1IWCJMeYDY8wHpZ1cROYBa4GOIhIvIjc7ty8TkabO8/8gIluxksRKY8yX55ymNnCtMWafMcYBTAEOlrU9Y0wuVk/pG2AX8Gmh6eNKKVWt5OQ52JucTjtHnM/e74HS7/mcAN4E3hSRZsAkYIeI/MMY87/STm6MKfJBVGPMaOfLo0DPUs7x8znvc7B6QuVpbxmwrLR4lVJecuYULLkLek2B9iO8HU2VFnc8A/+809TLioeoG70dTrFKq+0GgIj0wko8lwJf44IhN6WUAiA3C+bfAAfXwOH18LeNPleBuSrZlWino8QjGJ8sq5OvtPI6T4jIJuDvwI9AH2PMzcaYnR6JTilVvTkcsPh2K/EMuAvSE2H1i96OqkqzJaTRtWABuSo67AY8BuwHLnB+PSsiYE1fNsaY7u4NTylVrX37OGxfCCNmwKB74fQJWPsG9LwRGrb3dnRVUmyinfG1j4LU8cmyOvlKSz66Zo9Syj3WvQ2/vAZ9p8PAe6xtI2bArqWw/CGYvABEF0wuL1uinUf8D0Gjrj798ytx2M0Yc7CoL6xnZwZ5JkSlVLWzayl8/SB0vAJGvfDHL8mwSBj2EOz9FmK/9m6MVVDqmRyOpJymefYBnx5yg9Lv+dQRkYdF5HURuUwsf8MaiitPiRullLIcXg8Lp0Oz3nDNu+Dnf/b+frdCo05W7ycn0zsxVlG7k+xEyzGC8jJ8tqZbvtKe8/kf0BHYBkwHVgATgPHGmPElfVAppc5zfC98fD3UaQp/+gSCap9/jH8gjHoeUg5aw3KqzGwJaXQR52OQPlpWJ19p93zaONfvQUTeBY4DLYwxdrdHppSqXtKTYe7V1hDb5AUQ2rD4Y9sMg87j4KeZcMFEqNvCU1FWabZEOxcEHcEgSGRnb4dTotJ6Pjn5L5wFQA9o4lFKlVt2Bnx8nZWA/vQpNGhb+mcuf8b6vuJR98ZWjdgS7fQJPoLUbwNBod4Op0SlJZ8LRCTN+WUHuue/FpE0TwSolKri8nJhwTRI+B0mzIboPmX7XN0WMPjvsPML2L/KrSFWB8YYYhPttDNxPv1wab7SZrv5G2PqOL/CjTEBhV7X8VSQSqkqyhhYdh/sXg6jX4JOo0v/TGED7rKeVfn6H5CXU/rxNVj8qTM4stJpkH0Eonz7fg+U3vNRSqmK++ll2DTHeoC0783l/3xgCIz8Pzhmg/VFlnRUTrZEO53E9ysb5NPko5Ryj9/nw/dPQbfr4OJ/Vfw8HUdD20tg1f9Z94xUkWwJhReQq+LDbkopVSH7foAv7oDWQ2D8f8GvEr9qRKyp1zln4NsnXBdjNWNLstO31hEIjoCI5qV/wMs0+SilXCtxG3xyIzTsCNfPhYCgyp+zYXvofztsmQvxGyt/vmrIlpBGjH+8NeTmw2V18mnyUUq5Tmo8fHSttSTC5M8gJMJ15x76IIQ1hmUPWNWwVYHMnDzijttpkXOgSgy5gSYfpZSrnEmBuROsZ3puWAARzVx7/uBwuPRJOLrZ6gGpAnuT02nGMYIcp32+rE4+TT5KqcrLzYJPboATe62hNnfNtup+HTTvD9/OsFY/VQDsSkijs1SdyQagyUcpVVkOByz+K8T9BFe+AW2Guq8tERj9opV4fvg/97VTxdgS7cQEHMKIHzTy7bI6+TT5KKUq57snYPsCuORxq2dSAZsOnuJfX2zny61HMcaUfHCT7tD7JtjwLiTtqFB71U1sop0+wUeR+m2LLtbqg0orLKqUUsVb/w78/Cr0udl6kLQccvMcLN+RyHtrDvDboRT8/YQP1x7kiy5HeebKGCLrhBT/4YsfhR2fw7IHYeqXVWJ2lzvZEtPoIAeh8YXeDqXMtOejlKoY21fWgnAdRp29IFwpUs/k8PbqfQx9cRV3fvwbpzKyeXJ8V37716U8MroTq3cfY8TMH/ls4+Hie0G168PFj8HBNVYS8hEZWbm8sNzGhc9+y5s/7iMnz/2z8o7Zs8hMT6FBztEqUdkgn/Z8lFLld3gDLLgZmvaECe+Bf+m/Sg6eyOD9n+P4bONhMrLz6N+mPjPGdeWSTpH4+VmJ69YhbRnROYqHFm7jgQVbWbo1gWeviiG6XhFDSb2nWqV7VjwGHUZ6tYqzMYYlvx/l2WW7SErLonOTOjz3tY1Fm4/w7NUx9G5Z321txyba6SiHrTdVoKZbPk0+SqnyObEP5l0P4Y1h0icl/tI3xrD+wEneW3OAlbuSCPATxnZvyrRBrYlpVvQzQG0ahTH/1v58tO4g//e1jctfWc1DozszuV+LgiQFWCugjn4RZl9u1ZC7pBIlfCph+5FUZizZwcaDp+jWLII3Jvemd8t6rNiRyIwlO7hm1lom9WvOP0Z2om5tFzxwew5bYtUqq5NPk49SquzSj8Hca6zXNyyEsEZFHpad6+CrbUd5b80Bth9Jo17tQO4Y1o4bL2pJVEn3cpz8/IQbL2rFsI6RPLJoG48t3s7S34/y/DXdad2wULJr0R+6Xw+//Ad6TC7bOkEucjIjm5dWxDJv/SHq1w7i+Wu6cW3v5gUJ8rKujRnYriGvfrub2T/HsWJHEv+8ojNX9WyGuPAelS3RzkVBzrI6dVz8bJUbSakzSxQAffr0MRs3alkPVYNlZ8CcMZC8y7rJX8S6PKcysvl4/SE+XBtHUloWbRuFcvOgNlzVsxm1gvwr1Kwxhs82xfPUlzvJznVw/2UdmTaoNf75vSB7IvynN7QaZC3N7Wa5eQ4+WneIl1fEkpGdx58vask9IzoQUSuw2M/sPJrGI4u2seVwChe1acDTV8XQtlGYS+IZ85+feMX+AO2bNICbvnLJOV1FRDYZY4pcwEmTTxlp8lE1Wl6u9RDpnm+sh0g7XXHW7n3H0pm95gALN8eTmeNgcPuGTBvUmqHtG509VFYJSWmZ/HPRdr7dlcQFzevy4oTudIgKt3b+/BqsfAz+9Bl0uMwl7RVl7b4TPLF0B7ZEOwPbNeDxsV3/iKEUDofh4/WHeH65jawcB7cNbcNfh7cjJLBiSRmsRNj18a/ZFnQzQX2nWAVYfYgmHxfQ5KNqLGPgq/tg43vWgnD9bnFuNvy89wTvrdnPD7HHCArw46oezZg2qDUdG5ftF3L5QzEs3ZrAjCU7sGfmcNfF7bltWFsCTS7MGgAmD/76KwQEu7TdIylneHbZLr7amkCzurV4bExnLu/auELDZ8n2TJ75ahdfbDlKqwa1eerKGAa3L3r4sjR7k9O5+ZVP+DH47zDudeh1Y4XO4y4lJR+956OUKtn6d6zEM/Ae6HcLmTl5LPn9KLPXHMCWaKdhWBD3jGjPDf1b0jDMtb/0zyUijLugKQPbNmDG0p28vHI3y7Yn8uKE7sSMeh7mXg1rX4fB97mkvcycPN5evZ83Vu3FGLh3RAf+MrRNpXorkeEh/HtiT67t3ZxHF2/jxvfWM+6Cpjw6pjOR4aXfDyvMlliorE4VmmYN2vMpM+35qBopcRu8cwm0GcbxcR8wd91h5v56kOPp2XRqHM60Qa0Zd0HTSv0yrowVOxJ5dPF2TmRk85chbbjv5JP4H/gB7txYqcKmxhi+2ZHE01/tJP7UGa7o1oSHR3cqesr3uXKzIXmnVeCzlCnomTl5vLFqH2+u2kdwoB8Pjux0/qy+Erz0TSzBa57jzoDFyCNHIbBWmT7nKTrs5gI1PfkcPJHB97ZkJvSOJjyk+BurqsPvsTwAACAASURBVBrJzoC3h5F3JpVnmr/D3O2nyc51MLxjI24e1IaB7Rq4dNZWRaWezuGZZTv5dGM8Axuk87/Mv+HXeQxMmF2h8+1JsvPE0p2s2XucjlHhPD6uCwPaNiz9g7lZ8Nv/YM2rkHoY6rexemDdrwf/kv+f2XcsnccWb+eXfSfo0bwuz1wVQ9empS9HMf2Djdx0+BEG1k2BO9eX9RI9RpOPC9TE5GOMYc3e48z5OY7vY5MxBv50YQuevarqPMimKmHJXZjNH3JX4OOsONOJa3pHM21ga9pFumaWlqut3n2Mhz/fxnXp/+PugM/JnLyEkPZlL3KalpnDqyv38MHaOEKD/Lnvso5MvrAFAf6lFILJOQObPrDKDNkTILofdJsAv82FxK0Q0QIG3QM9byjxXpQxhsVbjvD0l7tIOZPDTQNace+lHQgNLr73NOj571mcczsNOw6Aa98v87V6it7zKYaIhAJvANnAKmPMR14OySdkZOXy+eZ4Plh7kL3J6TQMC+JvF7cnOS2TeesPcW3vaHq2qOftMJU77VgEmz/gbceVbPTrzud/7VOmv8S9aUiHRnxz7xBeWVaH+C2ryfr4byRNWsGADo1L/JzDYViwKZ4XvrFxIiObSf1acN+lHWhQ2v2r7AzYONuaaZeRDC0HwVVvQuuhVqmhfrfCnhXw4wvw1d9h9Usw8G7oPaXI4TER4aqe0QzvGMnzy228u+YAX21LYMa4rlze9fxrsGfmkHrqBA1DEqvUw6X53NrzEZHZwBgg2Rhz3k9HRJoDHwKNAQfwtjHm365uT0RGAv8G/IF3jTHPObffCKQYY5aKyCfGmOuLO3dN6PkcPJHBh2sP8unGw9gzc+keHcHUAa24onsTggP8sWfmMGLmjzQMC2bJnYP+eM5CVSvm1EGyXx/AzpzGPBU5kzf/fGHJRT590O4f5tLhxzv4V84UcnrfwsOjO1GniOHizYdOMWPJDrbGp9K7ZT2eGNe12MoLBbLs1iSMta/D6RNWshn6oPWcUVGMgf0/wI8vwqFfIDQSBvwN+kyD4OJ7kZsOnuSfi7ZjS7QzonMUM8Z1Oeue06aDJ3nuzff5LPhJt08xryivDbuJyBAgHfiwmOTTBGhijNksIuHAJuBKY8zOQsdEAmeMMfZC29oZY/aWpT0R8Qd2A5cC8cAGYJIxZqeIPAx8bYzZIiIfG2P+VNy1VNfkc+7Qmr8Io7s1YerAVvRsXve8Mf0vtx7lzo9/Y8bYLkwd2NpLUSt3ycrO4ugrF9Pg9D5ebvMeD/9ppNcmE1SKMeR9OJ7sQ5sZfPpFAsIjefbqGC7uFAVY052f/zqWhZvjiQwP5pHRnRnfo2nJ97DOpMC6t+DXNyAzBdpdaiWd5v3KHlfcGqsndOBHqFUfLrrD6iGF1Cny8Jw8B7PXHODVb/cAcO+l7blpYGsC/f2Y++tBYpfO5KnAOXDvTtevHOsCXht2M8asFpFWJexPABKcr+0isgtoBuwsdNhQ4HYRGW2MyRSRW4CrgNFlbK8fsNcYsx9AROYD451txAPRwBaKqfAtImOBse3atSvtcquUjKxcPv/tCB/8EvfH0NrwdkzuX3L5kyu6NeGT9od5ecVuRndrUuX+IlbFO5GexQ+z7mbCme0s6/g0MyaN9okJBRUigv/oF6k1awDLu//InxInMW3ORq7q2YyOjcN5/fu9ZOXmcfuwttwxvB1hJdxX4fRJK+Gsewuy0qDjaBhyPzTrXf64Wg2yvg6vh9UvwvdPwS+vwYW3Q//boNbZw9mB/n78ZWhbrujehBlLdvDsMhufbz7CM1d1w5aYRo+Aw5ha9ZA6Tcsfi5e5fcKBMxl8WVTPp4jjVgMxxpi0c/Y9CAwAPgPuBC41xqSXpT0RmQCMNMZMd76/EbjQGHOn857P60AmsKakez7Vpedz7tBat2YR3DTwj6G1sjhwPIPLX1nNqG6N+ffEnm6OWHnC7iQ7/5n9Pv/O/BfxLa+kxbQ53g7JNb75J6z9L9nTvuX12Dq88cNech2GSzpF8uiYLmfXiTtXxnGrZtyGdyE7HTqPgyEPWIvZucrR36x7QbYvISgc+k2Hi+6E0PNn1xljWLEziRlLdpCQmklYcACLAh+jfXSkVe7IB/n8hAMRCQMWAvecm3gAjDEvOHsss4C2xSWe4k5fxDbjPG8GcFMFQq5S8p9En/PLAb6z/TG0NmVAK3q1OH9orTStG4Zy27C2vPbdHq7v05wB7cowDVX5rFWxyTz68WoWyitkR7SixeTXvR2S6wz9B2z9lKAVD/H3aSsY070Jx9OzSp46bU+0ks7G2dZMtpirYfD9ENXF9fE17QkTP4LE7fDTS9Y07XVvWfeDBvzNqhzuJCJc7ixW+srK3Xzw8z5a+h+Expe4Pi4P8HryEZFArMTzkTGmyFWhRGQwEAMsAh7H6v2UVTzQvND7aOBoxaKtWs4dWmsQWrahtbL467C2fLHlCI9+sZ2v7x5c5l6T8h3GGOb8EsdTX+5gbtg7RObZkYmLSrwJXuWE1IFLn4DFt8Pv8+jQc3LxtdhSj8DP/4bNH0BeNnS7znpOp1EH98fZOAaunQPDYq3lIX59w5rU0HuKNUMuIrrg0LDgAB4b04W/djMEvZ9Z5Sob5PNq8hHrT+73gF3GmJnFHNMTeAe4AjgAzBWRp40xj5axmQ1AexFpDRwBJgLFTiyoDg6dOM2Ha+P4pNDQ2svXXsCYC8o+tFaakEB/Zozryk3vb+Ddnw5wx/DqdU+susvJczBjyQ4+WneIp6PXM+D4r3D5s9C0h7dDc73uE61ezLePQ+cxEHLObLaUQ7DmFeu5HOOACybCoL97dHmGAo06wtVvWz22Na9YcW98H3pOtpYpr9eq4NAG6butF1FVb5o1uDn5iMg8YBjQUETigceNMe+JyDJgOtAGuBHYJiJbnB97xBizrNBpagPXGmP2Oc85BZhazvbuBL7Bmmo92xizw7VX6n1FDa2N6taEqRUcWiuL4R0jGRXTmNe+28O4C5rSvH4ZSo8or0s9ncNfP97Ez3tP8GhfB5N3vmnN3Lrwdm+H5h5+ftaic28Ph1XPw8hnre0n98NPM+H3eYBYD4EOuhfqtfRquICV+Ma/bs2mW/OqVTlh8/+sxDj4Pmt/0g4Qf2jUydvRVohWOCgjX5twcDo7l33JGexOsrM72c73u5LZ4xxam3xhC5cMrZVFQuoZLnn5Ry5q04B3p/SpurOjaogDxzO4ec4GDp86zfPj2nH1xj9bz6rc/kuxC8NVG0vusno318+FXUtg66fgF1Dk0JbPSTtqPcy66X1rSDDmGjgVZz1zdMc6b0dXLJ+fcKCKdzo7l73J6exOSmdPsp09zu/xp86Q/3dDoL8Q4xxau6J7E48+l9Ekohb3jujAM8t2sXJnEpcV8SS28g2/7DvO7XM34yfw0fT+9NvxFBzbBTcuqv6JB+CSx2HnFzB/EgTUggtvg4F3nXVT32fVaQqjnoPBf3fOwHsPcjIgZoK3I6sw7fmUkbt7PhlZVpLZk5zOniQ7e5LT2Z1kJZl8Qf5+tGkUSrvIMDpEhdMhKox2keG0alC79PpTbpST52DMa2tIz8pl5d+HUDtI/6bxNfPWH+Kxxdtp1TCU2VP60iLpW/j0Rusv/kuf9HZ4nrN7BRzZBH2nV+2Em3ECtnwEbS/26dI6WljUBVyVfPKTzG5ngtmTZGd3UjpHUs5PMu2jwukQGUb7qDDaR4XTsr53kwx5ucWWiN8Qd5Jr31zLbUPb8tCoqjkGXR3lOQzPLtvFe2sOMKRDI17/U0/qZCbCmwOhfluY9g0EBHk7TFVN6bCbl+SvPV+QbIpJMr1b1mNi3+a0jwqnfVSY95JMXi6kHbHGkov6OnMKJs2HjiPP+2jfVvW5tnc07/60n2t6NaN9GZcWVu5jz8zh7vlb+N6WzNQBrXj0is4E4ICPbwWHAya8p4lHeY0mHzcSEV5YHktaZg5tG4XRu2U9JvVrTrtIa8ishTeSzJmU4pNL6mFw5P5xrF8A1G1hTe9seiXs/xG+fgDaDC2yKu9DozqxYmcSjy7ezvxb++vkAy86fPI00z/YyN5j6Tx1ZQw39nfO4Fr1glXc8up3rPVmlPISTT5u9uXfBtEwLMhzSSYvB1Lji08wmSlnH1+7gZVcmvWynuSu1+qPrzrNwK/Q5IUDq+GDsdbUz+EPn9d0g7BgHhrViYc/38ai345wdS8fnj1UjW06eJJbP9xEdp6DOTf1ZXB7572Ng7/Aj89bz710v867QaoaT5OPmzWOcPF0Z2Mg4xicOggpB62EknLQen8qzko8Ju+P4/0CrecW6rWC6D5nJ5e6LYutpluk1kOg69XWw28XTIT651e1vr5Pcz7ZcJhnl+3ikk5RRNTWVU89adFv8fxjwTaa1A3hvSl9/1j47cwpWHiL9d/9ipe8GqNSoMnHN2WmnZ1Q8l+nHLSexs45ffbxoY2sRNK8n/UXbeEEE97k7N5LZV32NOz+xirYOOnj83b7+QlPXxnDuNfX8NKKWJ660ndn4lQnDofh5ZWx/PeHffRvU59Zk3tTL9R5P8cY6xmX9ES4eSUE6/045X2afLwhJ9O6v3LqIKTEFerFOL+fOXX28UHhVu+lfltoe4n1um5L5/cWEFRCZV5Xi2gGQx+Ab2fAnpXQ/tLzDolpFsGUAa2Y80scE3pHc0Hzup6LrwY6nZ3LfZ/+ztfbE5nYtzlPjo8hKKDQMO+mOdZDlZc+ZQ2vKuUDdKp1GVVoqrUxVumOU3FnJxh7As7C2hb/ICuJFCSUwt9bWWt8+NLN+9wsmDXAur6/ri1yXXp7Zg6XvPwjUXVCWHzHQF311AXyHIas3DwycxwF31PP5PDo4m3sOJrGP0d35uZBrc+e6JG8C94eBi0HwOSFVqkZpTxEp1p7iwis/Je1LkidZlZCaTPs/AQT3qRq/VIICIZRz8Pca2Dtf62nrs8RHhLIo2O6cNe83/h43UFuvKiV5+P0ImMMv+4/SULqGbJyHWTmnJ00MnPyyMrNIyvHQeZZ2/KPzX/tICsnj8zcPHLyiv5DMTTIn3f/3IdLOkedvSPnDCy42Rpmu/LNqvVvTFV7mnzc7dYfrXsy1e15inYjoNMYazXG7tcVWRdrbPcmfLrhMC98E8vImCY0Cj+/h1QdGWM92PnOTweK3B/k70dwgB/Bgf6EBPoREuhPcID1PSTQj4hagdb2AH+Cz9l37rHBAf50axZRdFHXFY9B8g6rxxMedf5+pbxIh93KyNcKi/qEUwfhv/2g4yhrLZIi7D+WzshXf+KK7k145fpqWK7/HLl5Dh76fBsLNsXz54tactPA1gVJIv+7R4YgbV/B/D9Zq2Je/oz721OqCCUNu2k/XFVcvZbWuic7FlkPoBahTaMw/jK0DYt+O8LafSc8HKBnZebkcdvczSzYFM89I9rzxLiutG4YSpOIWtQPDaJ2UIBnEk/qEfjiDmjSwyqmqZQP0uSjKmfgXdZ9q68ftB5wLcIdw9vRvH4tHvtiO9m5Dg8H6BlpmTlMmb2e72xJPDm+K/eM6OCdCg+OPPj8VsjNhgmzq99wr6o2NPmoygmsBSOfg2M2WP92kYeEBPrz5LgY9ian8+6a/R4O0P2O2bOY+NavbDp4ilev78GfvTm54qeZcHANXPGyd1biVKqMNPmoyus4yloJ84f/A3tikYcM7xTJ5V2jeO27PcSfOl3kMVXR4ZOnufbNX9h/PJ13p/RhfI9m3gvm0DpY9X/Q7TqrAoVSPkyTj6o8EWvqdV4WrCz+HsPjY7viJ8ITS3d6MDj3iU20c82sXzh1OoePpvdnWMdI7wVzJgUWToe6za1ejy89F6ZUETT5KNdo0BYG/A22zoeDa4s8pGndWtx9SXtW7kzi251JHg7QtTYdPMm1b/4CwKd/uYjeLet5LxhjYOndYD8K17xXvnp9SnmJJh/lOoPvsx6mXfaAdeO7CNMGtaZDVBiPL9nBmeyij/F1P8QmM/ndddQPDWLh7QPo2NjLtdJ++x/sXAwXP2oVj1WqCtDko1wnKNR6piRpG2ycXeQhgf5+PH1lN46knOH1H/Z4OMDK+2LLEW75YCNtGobx2W0Din6405OO7Yav/2FVzhhwt3djUaocNPko1+pypbX0wvdPWWWFitCvdX2u6RXN26v3szfZ7uEAK+6DX+K455Mt9GpZj/l/6e+9ig3GwJFNVqXqt4dZMw6vekvL56gqRf+1KtcSgVEvQnYGfPdEsYc9PLoTtYMCeGzxDny9yoYxhldW7ubxJTu4pFMUH07rR50QL6xTlJkKG96FtwbDOxfDts8g5iqYugzCG3s+HqUqQWu7KdeL7AQX3mYVHe01FaJ7n3dIw7BgHhzZkX8u2s6S3496d4pyCRwOw4ylO/hw7UEm9I7muau7eXbpc2MgfgNs+gB2fG6t5dS4O1wxE7pNgJAIz8WilAtpbbcy0tpu5ZSZBq/3sSYgTP+uyCGhPIfh6lm/cOTUGb67bygRtXxr1dPsXAf3f/Y7S34/yq1D2vDwqE6eq1pw5hT8/gls/gCSd0JQmJVsek+Fpj09E4NSlaS13ZTnhdSxFi87uhm2zC3yEH8/4ZkrYziZkcXMFbEeDrBkp7NzueXDjSz5/SgPjerEI6M7uz/xGAMHf7HK47zcCZb/AwJCYOxrcF8sjP23Jh5Vbeiwm3Kf7tfBpvetVU87jYHa9c87JKZZBH++qBUfro1jQu/mdIv2/jBSyulsps3ZwJbDKTx3dTcm9mvh3gYzTliLDm7+AI7vhuA60PMG6DUFmnR3b9tKeYn2fJT7iMDoF60hpB+eLfawv1/WgfqhwTy6eBt5Du8OAyemZnLdW2vZfiSNNyb3cl/icTisSuALpsHMTrDin9aKtePfgPtsVpUCTTyqGtOej3Kvxt2g73RrllavPxf5C7VOSCCPjenM3fO3MG/9IW7o39ILgcKB4xnc+N46TmVkM+emvgxo19D1jaQnw5aPYPOHcHI/hNSFPjdbP5uoLq5vTykfpclHud/wR2D7QqvywbTlRdYdG3dBUz7ZcJgXltsYGdOYhmGefYZm+5FUpr6/HoeBebf2p3t0Xded3OGA/T/ApjkQuwwcudByIAx7GDqPtZ7TUaqG0dluZaSz3Spp84ew5G9w1dtwwfVFHrI3OZ1R/15NnZBAOjYOp22jMNpFhhV8j6oT7Jab/r/uP8EtH2wkPCSA/02/kLaNwlxz4rQEa7LF5g8h5RDUqg89/mTdy2nUwTVtKOXDSprtpsmnjDT5VJLDAe+NgNR4uHNjscUvf7Al89W2BPYdS2dvcjr2zNyCfaFB/rSNDKNdozDaFiSlUFrUDyUooGK3L1fuTOKOjzfTon5tPpzWj6Z1XdALMcYqL7T8YavSd+sh1hTpTmMgwEtVEZTyAk0+LqDJxwWObIJ3LoGL7rBqwJXCGMOx9Cz2Jqez71gG+5LTC5JSQmpmwXEBfkKLBrXP6ym1aRRaYiWCzzYe5qHPtxHTLII5U/tSL9QFq35m2WHpPbB9AbQbAaNe0EXdVI1VUvLRez7Kc5r1tm6s/zrLmkoc2bnEw0WEyPAQIsNDGND27Jv/6Vm57D9mJaN9yRnOBJXOqthkcvL++IMqMjz4rISU/33p70d5ZtkuBrdvyJs39CY02AX/KyTtgE+nwMl9cPFjMOjvWm9NqWJoz6eMtOfjIhkn4D+9rFlvf17i8kXPcvIcHD55uqC3lJ+U9iWnY8/KPevYK7o1Yeb1FxAc4F/5hn/7CL66zxpOvOY9aD248udUqorTno/yHaENrHVnlt1vrUHT9SqXnj7Q3482jcJoc86kAWMMx+xZ7HUmIn8/P67v2xx/v0omv+zT8PUD8NtcaDXYSjzhUZU7p1I1QI3u+YhIKPAGkA2sMsZ8VNyx2vNxIUcevD0UTp+EO9ZDsItml3na8T3WMFvyDhjygDV12s8FvSilqgmv1XYTkdkikiwi2ytzTGXbE5GRIhIrIntF5KFCu64GFhhjbgHGVbZ9VUZ+/jD6JUg7Aj+97O1oKmb7QmstHXsCTF5o9eY08ShVZu6+GzoHGFmZY0QkUkTCz9nWrqznEhF/4L/AKKALMElE8h8ljwYOO19XzTWdq6oW/eGCSfDLf+D4Xm9HU3a5WfDV/VZZnKiucNtP0H6Et6NSqspxa/IxxqwGTlbymKHAFyISAiAitwCvleNc/YC9xpj9xphsYD4w3rkvHisBQTE/CxEZKyJvp6amlnQZqiJGPGFVbV7+D+vZGF93Kg5mXw4b3oGL7oSpX0FEdKkfU0qdz+fngRpjPgOWA/NFZDIwDbiuHKdoxh+9G7ASTv7KZZ8D14jILGBpMe0vNcbcGhHh/WrL1U54FAx/GPZ+C7FfezuaktmWwVtD4MR+uP4j6zklf99af0ipqqRKzHYzxrwgIvOBWUBbY0x6OT5e1HQm4zxvBnCTC0JUFdXvVtj8P1j+ELQd7nt1zvJyrOXAf/kPNOkB186B+q29HZVSVZ7P93wARGQwEAMsAh4v58fjgeaF3kcDR10Umqos/0AY/QKkHISf/+3taM6WegTmjLEST9/pMO0bTTxKuYjPJx8R6Qm8g3Wf5iagvog8XY5TbADai0hrEQkCJgJLXB+pqrDWQ6Dr1bDmFeu+ii/Y+x28NRgSt1nP7lzxMgSGeDsqpaoNd0+1ngesBTqKSLyI3OzcvkxEmpZ0TCG1gWuNMfuMMQ5gCnCwrO0ZY3KBO4FvgF3Ap8aYHa6/WlUplz0N4gff/NO7cTjy4PtnYO41EBYFt66CbhO8G5NS1VCNfsi0PPQhUw/4aaZ1f2X0S9Dhcoho7vLyOyVKT4aFN8OB1dBjshVHUG3Pta9UNaNVrV1Ak48H5GbB28OtigEAwXWsZ2kKvmKsYqTB4SWfpyLi1ljP7mSmWkNsPW9wfRtK1TBa201VDQHBcMt31n2WpO1WleikHbD1U8hK++O4eq2sRFQ4KdVrVbEKAw4H/PwKfP801G8DN3wOjWNcdUVKqWJo8lG+JbAWNO9nfeUzxloJND8Z5Sem2GVgHM7P1bZ6RfnJKKorRHaB2vWLb+v0SVj0F9izwprwMO419/SqlFLn0eSjfJ8I1GtpfXUa/cf27NNwzHZ2Utq11Fq2Ol+dZmf3kKK6QoN2cHQLfDYVMpKtezt9p3v2/pJSNZwmH1V1BdWGZr2sr3zGgD3x7B5S0g7Y9z04nOv5+AdZs9oimlnP7hT+vFLKIzT5qOpFBOo0sb4KF/zMzYbju/9ISsYBQ+6HWvW8F6tSNZgmH1UzBARZEwkaxwDXezsapWo8n69woJRSqvrR5KOUUsrjNPkopZTyOE0+SimlPE6Tj1JKKY/T5KOUUsrjNPkopZTyOE0+SimlPE6XVCgjETlGMYvYlUFD4LgLw/Gm6nIt1eU6QK/FF1WX64DKXUtLY0yjonZo8vEAEdlY3JoWVU11uZbqch2g1+KLqst1gPuuRYfdlFJKeZwmH6WUUh6nyccz3vZ2AC5UXa6lulwH6LX4oupyHeCma9F7PkoppTxOez5KKaU8TpOPUkopj9Pk40YiMlJEYkVkr4g85O14KkpEmovIDyKyS0R2iMjd3o6pskTEX0R+E5EvvR1LZYhIXRFZICI253+fi7wdU0WIyL3Of1vbRWSeiIR4O6ayEpHZIpIsItsLbasvIitFZI/ze5VYMreYa3nR+e9rq4gsEpG6rmhLk4+biIg/8F9gFNAFmCQiXbwbVYXlAvcZYzoD/YE7qvC15Lsb2OXtIFzg38ByY0wn4AKq4DWJSDPgLqCPMSYG8AcmejeqcpkDjDxn20PAd8aY9sB3zvdVwRzOv5aVQIwxpjuwG3jYFQ1p8nGffsBeY8x+Y0w2MB8Y7+WYKsQYk2CM2ex8bcf6BdfMu1FVnIhEA1cA73o7lsoQkTrAEOA9AGNMtjEmxbtRVVgAUEtEAoDawFEvx1NmxpjVwMlzNo8HPnC+/gC40qNBVVBR12KMWWGMyXW+/RWIdkVbmnzcpxlwuND7eKrwL+x8ItIK6Ams824klfIq8CDg8HYgldQGOAa87xxCfFdEQr0dVHkZY44ALwGHgAQg1RizwrtRVVqUMSYBrD/egEgvx+Mq04CvXXEiTT7uI0Vsq9Lz2kUkDFgI3GOMSfN2PBUhImOAZGPMJm/H4gIBQC9gljGmJ5BB1RneKeC8HzIeaA00BUJF5AbvRqXOJSL/xBqC/8gV59Pk4z7xQPNC76OpQkMJ5xKRQKzE85Ex5nNvx1MJA4FxIhKHNRR6sYjM9W5IFRYPxBtj8nuhC7CSUVUzAjhgjDlmjMkBPgcGeDmmykoSkSYAzu/JXo6nUkRkCjAGmGxc9HCoJh/32QC0F5HWIhKEdQN1iZdjqhAREaz7CruMMTO9HU9lGGMeNsZEG2NaYf03+d4YUyX/yjbGJAKHRaSjc9MlwE4vhlRRh4D+IlLb+W/tEqrgxIlzLAGmOF9PAb7wYiyVIiIjgX8A44wxp111Xk0+buK8QXcn8A3W/0ifGmN2eDeqChsI3IjVS9ji/Brt7aAUAH8DPhKRrUAP4Fkvx1Nuzp7bAmAzsA3r91KVKU8jIvOAtUBHEYkXkZuB54BLRWQPcKnzvc8r5lpeB8KBlc7/9990SVtaXkcppZSnac9HKaWUx2nyUUop5XGafJRSSnmcJh+llFIep8lHKaWUx2nyUTWaiMwQkfu9HUdJRKSTc4rrbyLStpLnmiMiE1wVWznbXlZaRWQRiRORhp6KSXmPJh+lXMBZxdxdrgS+MMb0NMbsc2M7bmWMGV2FC58qF9Pko2ocEfmnc52lb4GOhba3FZHlIrJJRH4SkU6Ftv8qIhtE5EkRSXduH+Zc5+hjrIcjEZEbRGS9s6fyVn5SEpHLRGStiGwWkc+cdfLOjauHs538dVPqOR/mvQeYLiI/FPGZdBF5VcwonwAAA59JREFU2Xne70SkUXHnOudzl4jIokLvLxWRzwud8xkR+d15jijn9pbONrY6v7dwbp8jIrOcP4v9IjLUuS7MLhGZU6iNgl6NiCx2/px3iMitFfjPqKo6Y4x+6VeN+QJ6YyWK2kAdYC9wv3Pfd0B75+sLsUrvAHwJTHK+vg1Id74ehlXMs7XzfWdgKRDofP8G8GegIbAaCHVu/wfwryJi2woMdb5+EnjV+XpGfoxFfMZg1dsC+BfweinnmgNMwCp8awMaObd/DIwtdM781y8AjzpfLwWmOF//f3t38xJVFMZx/PuTIhTCsAhahC0qgoiENk0pBK5FoqIXC6tVELiRdv4ZQrWwEO1FXBRFEEWQZSGFpVhBLbI2QSs10F4knxbnlFcbmzshM+g8HxjmzJ17Hs/d3Mcz93Ce08DNRMzrMWYj8AXYQfjndhCoied9ANbFdlV8LwdeAWvnn+Ov5f3ymY8rNXXADTObsrAz9y34s2P3HqBX0hBwEdgQ+2SA3ti+Oi/eMzMbje16QnJ7HmPUE8oe7CYUFHwSjzcD1ckgkiqBNWbWFw91Emr15DID9MR2N1CbJpaFO30XcDw+h8kwu1X+D0LChZA8NsV2JnH9XUBtIuTtGHME+GxmI2Y2A7xO9E9qkTRMqA+zEdiS4lrdMrKi2ANwrgiy7SlVBoybWU2esSYTbQGdZjan0qOkBuC+mR3NM/b/yGe/rMuE2cw3oNdmC4ZNx0QC8JOF7xPJv/U9vs8k2r8/z+kvaR9hJ+uMmU1JeggsmbLZbnH4zMeVmkfAfknlklYDDQBxFjQq6RCEnbwl7Yx9BoADsf2v8s4PgIOS1scYVZKqY/+9kjbH4xWStiY7mtkEMCapLh46AfSRWxnhZzSAY0B/2lhm9olQ5qON8NNZLk+Zvf4moD9Fn2wqgbGYeLYRZoauxPjMx5UUM3shqQcYAj4CjxNfNwHnJbUBKwnPMYYJD/y7JbUCd4CJBWK/iX3vSSoDpoGzZjYg6SRwTdKqeHob8G5eiGbggqQK4D1wKsUlTQLbJQ3GcR3OM9YVwnOfNKUYWoBLks4RKqimGV82d4EzCjtxvyUkZ1difFdr53KIN/CvZmaSjhAWHzQWe1wQVqaZ2V8r5/Lo3w68NLOORRyWczn5zMe53HYB7ZIEjBNWei15cbY0CbQWeyyu9PjMxznnXMH5ggPnnHMF58nHOedcwXnycc45V3CefJxzzhWcJx/nnHMF9wvHimyoUwT0fgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "## EDIT THIS CELL\n",
    "\n",
    "K_max = 12 # this is the maximum degree of polynomial we will consider\n",
    "assert(K_max < N) # this is the latest point when we'll run into numerical problems\n",
    "\n",
    "rmse_mle = np.zeros((K_max+1,))\n",
    "rmse_map = np.zeros((K_max+1,))\n",
    "\n",
    "for k in range(K_max+1):\n",
    "   \n",
    "    \n",
    "    # feature matrix\n",
    "    Phi = poly_features(X, k)\n",
    "    \n",
    "    # maximum likelihood estimate\n",
    "    theta_ml = nonlinear_features_maximum_likelihood(Phi, y)\n",
    "    \n",
    "    # predict the function values at the test input locations (maximum likelihood)\n",
    "    y_pred_test = 0*Xtest ## <--- EDIT THIS LINE\n",
    "      \n",
    "    ####################### SOLUTION\n",
    "    # feature matrix for test inputs\n",
    "    Phi_test = poly_features(Xtest, k)\n",
    "    \n",
    "    # prediction\n",
    "    ypred_test_mle = Phi_test @ theta_ml\n",
    "    #######################\n",
    "    \n",
    "    # RMSE on test set (maximum likelihood)\n",
    "    rmse_mle[k] = RMSE(ytest, ypred_test_mle)\n",
    "    \n",
    "    # MAP estimate\n",
    "    theta_map = map_estimate_poly(Phi, y, sigma, alpha)\n",
    "\n",
    "    # Feature matrix\n",
    "    Phi_test = poly_features(Xtest, k)\n",
    "    \n",
    "    # predict the function values at the test input locations (MAP)\n",
    "    ypred_test_map = Phi_test @ theta_map\n",
    "    \n",
    "    # RMSE on test set (MAP)\n",
    "    rmse_map[k] = RMSE(ytest, ypred_test_map)\n",
    "    \n",
    "\n",
    "plt.figure()\n",
    "plt.semilogy(rmse_mle) # this plots the RMSE on a logarithmic scale\n",
    "plt.semilogy(rmse_map) # this plots the RMSE on a logarithmic scale\n",
    "plt.xlabel(\"degree of polynomial\")\n",
    "plt.ylabel(\"RMSE\")\n",
    "plt.legend([\"Maximum likelihood\", \"MAP\"])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Questions:\n",
    "1. What do you observe?\n",
    "2. What is the influence of the prior variance on the parameters ($\\alpha^2$)? Change the parameter and describe what happens."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3. Bayesian Linear Regression"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Test inputs\n",
    "Ntest = 200\n",
    "Xtest = np.linspace(-5, 5, Ntest).reshape(-1,1) # test inputs\n",
    "\n",
    "prior_var = 2.0 # variance of the parameter prior (alpha^2). We assume this is known.\n",
    "noise_var = 1.0 # noise variance (sigma^2). We assume this is known.\n",
    "\n",
    "pol_deg = 3 # degree of the polynomial we consider at the moment"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Assume a parameter prior $p(\\boldsymbol\\theta) = \\mathcal N (\\boldsymbol 0, \\alpha^2\\boldsymbol I)$. For every test input $\\boldsymbol x_*$ we obtain the \n",
    "prior mean\n",
    "$$\n",
    "E[f(\\boldsymbol x_*)] = 0\n",
    "$$\n",
    "and the prior (marginal) variance (ignoring the noise contribution)\n",
    "$$\n",
    "V[f(\\boldsymbol x_*)] = \\alpha^2\\boldsymbol\\phi(\\boldsymbol x_*) \\boldsymbol\\phi(\\boldsymbol x_*)^\\top\n",
    "$$\n",
    "where $\\boldsymbol\\phi(\\cdot)$ is the feature map."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAY8AAAEWCAYAAACe8xtsAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nO3dfXxb+V3o+c9Xz5JlWZYtx3GScZzJ86SZybwyT4W7l6UF2l5g2AXuFnqh7e3eWV5bWNiFF5dS9pYCvcte9gUsCws7S7sUKC1dnjqwQ0vL5XLbMtNJZppJ4onHcZ49jh9lyZZkWZbOb/+wjkbJOInl6Jwj29/36+VX5HOO9PvKjs9Xv2cxxqCUUko1w+d1AEoppTYfTR5KKaWapslDKaVU0zR5KKWUapomD6WUUk3T5KGUUqppmjzUliEifysi7/c6jvslIlER+WsRyYnI/+ty2cMi8m1ulqk2p4DXASh1JyJyFdgBVIEC8DzwE8aY/FrXG2Pe7V50jvoBVt93jzGm4lQhIvIHwLgx5hfsY8aYh5wqT20tWvNQ7e57jDFx4FHgMeAXbr9AVm34/7KIePYh6g5lDwKjTiYOpe6XJg+1KRhj3gD+FjgGICL/SUQ+ISJfB4rAvtqx/7Z23icivyAi10RkWkT+UES6auf2iogRkQ+JyHXgP65Vpoj8GxEZE5GMiDwnIgO1478nIv/bbdd+QUT+p9rjARH5cxGZEZErIvI/NFz3iyLyZyLyxyKyAHzgttf5OPDvgP9GRPK1GH9RRP644Ro7/kDDz+KXReTrIrIoIn8nIr0N13+riPyTiGRF5IaIfEBEngHeB/xsrZy/rl17VUTeWXscFpHfFJGJ2tdviki4du7bRGRcRH669vO9KSIfbCjzPSLyWi2eN0TkZ9b5q1abhCYPtSmIyB7gPcA3Gw7/CPAM0Alcu+0pH6h9/ZfAPiAO/PZt1/xz4AjwXWuU9+3A/wL8S2Bn7fU/Vzv9J6ze3KV2bTfwncDnajWgvwZeBXYB7wB+SkQay3ga+DMgCXymsVxjzMeAfw/8qTEmboz55B1+JLf7YeCDQB8QAn6mFtsDrCbd/wNIA48AZ4wxz9bK/g+1cr5njdf8KPBk7TkPA49za82vH+iqvc8PAb9T+1kAfBL474wxnawm/DUTtNq8NHmodvdXIpIFvgb8I6s3VtsfGGOGjTEVY8zKbc97H/DrxpjLtT6SjwDvva2Z6BeNMQVjzNIa5b4P+JQx5hVjzHLt+U+JyF7gq4AB/lnt2h8AXjDGTLDatJY2xvySMaZsjLkM/N/Aexte+wVjzF8ZY6w7lL0R/48xZrT2ep9n9YZvv4+vGGM+a4xZMcbMGWPOrPM13wf8kjFm2hgzA3yc1YRtW6mdXzHGPA/kgUMN546KSMIYM2+MeeV+36BqL5o8VLv7PmNM0hgzaIz572+72d64y/MGuLU2co3VASI7NvL8WgKaA3aZ1dVEPwf8UO30D/NmDWIQGKg1EWVrie/nmyh3oyYbHhdZrWkB7AEubfA11/oZDjR8P3dbv0xjud/Pak3xmoj8o4g8tcEYVJvS5KE2s7stCT3B6o3c9gBQAaY28nwR6QB6gDdqhz4L/ICIDAJPAH9eO34DuFJLePZXpzHmPessdy0FINbwfX8Tz70BPHiHc/eKY62f4cR6CjXGnDLGPM1qM9pfsVobUluIJg+1VX0W+B9FZEhE4rzZj7DeEUx/AnxQRB6pdRL/e+AbxpirAMaYbwIzwO8DXzLGZGvPewlYEJF/W5uv4ReRYyLy2H28lzPAfyEiD9Q6/T/SxHM/A7xTRP6liAREpEdE7CatKVb7g+7ks8AviEi61gH/74A/vsv1AIhISETeJyJdtebEBVaHW6stRJOH2qo+BfwR8J+BK0AJ+In1PtkY8/fA/8xqjeImq5/e33vbZZ8F3slqorGfVwW+h9U+hyvALKsJpmuD7wNjzJeBPwXOAi8Df9PEc6+z2nz000CG1UT0cO30J1ntl8iKyF+t8fRfAU7Xyj0HvFI7th4/AlytjSj7MeBfrTdmtTmIbgallFKqWVrzUEop1TRNHkoppZqmyUMppVTTNHkopZRq2rZYVbe3t9fs3bvX6zCUUmpTefnll2eNMem1zm2L5LF3715Onz7tdRhKKbWpiMjta8bVabOVUkqppmnyUEop1TRNHkoppZqmyUMppVTTNHkopZRqmiYPpZRSTdPkoZRSqmmaPJRSSjVNk8c9FItFJibWtXmaUkq1FcuyuHLliiOvrcnjHkqlEiMjI5RKJa9DUUqppkxPT3PlyhUsy2r5a2vyWIf5+XmuXbvjLH2llGo7lmXx+uuvUy6XHXl9TR7rEI1GuXLlCsVi0etQlFJqXSYmJigWi4iII6+vyWMd/H4/gUCAy5cvex2KUkrdU6VSYXR0lGQy6VgZmjzWKZlMcv36dRYWFrwORSml7mp8fJyVlRXC4bBjZWjyWCefz0c0GmV0dNTrUJRS6o7K5TIXL16ku7vb0XI0eTShq6uL6elpMpmM16EopdSa7KG5gYCz2zVp8mhSPB7nwoULjgx9U0qp+1EsFrl69arjtQ7Q5NG0eDzO4uIik5OTXoeilFK3GBsbw+/34/M5f2vX5LEBXV1djI6OUqlUvA5FKaUAyGaz3Lhxg1Qq5Up5mjw2IBwOs7y8zI0bN7wORSmlsCyLCxcu0NnZ6VqZmjw2qLu7m7GxMV22RCnluenpaebn54nH466VqcljgwKBAMYYLl265HUoSqltrFKpMDIy4koneSPPk4eIRETkJRF5VUSGReTjteNDIvINEbkoIn8qIqHa8XDt+7Ha+b1exZ5KpXTioFLKUzdu3GB5ednRCYFr8Tx5AMvAtxtjHgYeAd4lIk8C/yvwG8aYA8A88KHa9R8C5o0x+4HfqF3nCXvioA7dVUp5oVQqMTo66nqtA9ogeZhV+dq3wdqXAb4d+LPa8U8D31d7/HTte2rn3yFOrfy1DolEgrm5Oaanp70KQSm1TY2OjtbX3nOb58kDQET8InIGmAa+DFwCssYYeyzsOLCr9ngXcAOgdj4H9Kzxms+IyGkROT0zM+No/N3d3bz22ms6dFcp5Rq3h+beri2ShzGmaox5BNgNPA4cWeuy2r9r1TLMWw4Y86wx5qQx5mQ6nW5dsGsIh8OsrKzonh9KKVdYlsX58+fp6uryLIa2SB42Y0wW+E/Ak0BSROy62G7A3gt2HNgDUDvfBXi+2FRPTw+jo6Pk8/l7X6yUUvdhYmKCxcVFYrGYZzF4njxEJC0iydrjKPBO4ALwD8AP1C57P/CF2uPnat9TO/8fjTFvqXm4zefzEYlEGBkZ8ToUpdQWViqVuHDhgied5I3c72V5q53Ap0XEz2oy+7wx5m9E5DXgcyLyK8A3gU/Wrv8k8EciMsZqjeO9XgS9lq6uLiYnJ5mcnKS/v9/rcJRSW9DY2BjGGEKhkKdxeJ48jDFngRNrHL/Mav/H7cdLwA+6ENqG2J3nqVTK81+uUmpryWQyXL9+vS0+nHrebLXV2J3n9pr6SinVCpVKhfPnz5NIJLwOBdDk4Yienh4uXbqkM8+VUi1z48YNisWip53kjTR5OMDn8xGPxzl37pzOPFdK3bd8Ps/o6KhnczrWosnDIfF4nIWFBa5fv+51KEqpTcyyLIaHhwmFQp7MJL8TTR4O6unp4cKFCxSLRa9DUUptUhMTE8zNzXk6IXAtmjwcFAgEiEQinD9/XpuvlFJNs+d0tFNzlU2Th8O6urrIZDJMTEzc+2KllGowPDyM3+9vy2H/mjxckEwmGR4e1uYrpdS6TUxMMDU1RTKZ9DqUNWnycEEoFCIYDPLaa69p85VS6p5KpRLDw8Nt2Vxl0+ThkmQyyfT0tDZfKaXu6fz584hIWzZX2TR5uCiVSmnzlVLqrsbHx5menm7rWgdo8nBVKBQiFArp5EGl1JqKxWLbN1fZNHm4zB59pRtHKaUaWZbFuXPn6h8y250mDw/09PQwMjKia18ppequXbtGJpNpu8mAd6LJwwOBQICOjg7Onj2r+54rpVhYWGBkZISenh6vQ1k3TR4eicfjFAoFLl686HUoSikPVSoVzpw5Qzweb6u1q+5Fk4eHenp6uHLlCpOTk16HopTyyMjICMvLy3R0dHgdSlM0eXjI5/PR3d3NuXPndPiuUtvQxMQE169f3xSjq27nefIQkT0i8g8ickFEhkXkJ2vHUyLyZRG5WPu3u3ZcROS3RGRMRM6KyKPevoP7Ew6HCQQCOnxXqW0mn89z7tw5UqkUIuJ1OE3zPHkAFeCnjTFHgCeBD4vIUeDngL83xhwA/r72PcC7gQO1r2eA33U/5Nayh+9q/4dS20OlUuHVV18lHA5vimG5a/E8eRhjbhpjXqk9XgQuALuAp4FP1y77NPB9tcdPA39oVr0IJEVkp8tht1w6nebSpUva/6HUNvD666+Tz+fbZj/yjfA8eTQSkb3ACeAbwA5jzE1YTTBAX+2yXcCNhqeN147d/lrPiMhpETk9MzPjZNgtof0fSm0P4+PjXLt2bVMNy11L2yQPEYkDfw78lDHmbrPn1mocNG85YMyzxpiTxpiT6XS6VWE6KhwOEwwG+eY3v6nzP5TaghYWFjh//jypVAqfr21uvxvSFtGLSJDVxPEZY8xf1A5P2c1RtX+na8fHgT0NT98NbJmlahOJBPl8npGREa9DUUq1ULlc5pVXXiEej2/afo5GnicPWR1m8EnggjHm1xtOPQe8v/b4/cAXGo7/aG3U1ZNAzm7e2irS6TTXr1/n6tWrXoeilGoBy7I4e/Ys1WqVWCzmdTgt0Q7TGb8F+BHgnIicqR37eeBXgc+LyIeA68AP1s49D7wHGAOKwAfdDdcdvb29vPbaayQSiU05Blwp9aaLFy8yPT1Nf3+/16G0jOfJwxjzNdbuxwB4xxrXG+DDjgZ1a3lUq1W3iqsLBAJ0d3fz8ssv89RTTxGPx12PQSl1/8bHxxkbG2PHjh2elO9U/6nnzVbtbnl5mampKcrlsutl2x3oZ86c8aR8pdT9yWaznDt3jp6eHk86yBcXF5menmb1M3drafK4B2MMKysr3Lx5k1Kp5Hr5iUSCUqnE2bNndQa6UptIsVjk9OnTdHV1edJBnsvlmJubc+y+ocljHfx+P6FQiKmpKU8SSCqVYmZmRkdgKbVJ2COrAoEAkUjE1bKNMWSzWebn5x3tnNfksU6BQIBwOMzk5CSFQsH18tPpNFevXuXKlSuul62UWj976ZFSqeT6DHJjDJlMhmw2SywWc3TNLE0eTfD7/UQiEWZmZlzfBdDn89VHYE1MbJlpLUptKZZlMTIyQiaTcX2UpGVZzMzMsLi4SEdHh+OLLWryaJLf7ycajTI3N0c2m3WkI+pOAoEAPT09nDlzhkwm41q5Sqn1uXjxItevX8ftVS2q1SrT09OUSiXX9gXR5LEBPp+Pjo4Ostmsox1SawmFQnR3d3Pq1ClyuZxr5Sql7u7KlStcvHiRvr6+e1/cQvaAnpWVFaLRqGvlavLYIBGho6ODQqHA9PS0q3NBwuEwHR0dnDp1inw+71q5Sqm1jY+Pc+HCBfr6+lwdklsqlbh5c3WBDbc75jV53KdYLFbP/G7OxYjFYoRCIU6dOqWr8CrlocnJSV599VV6e3td3YM8n88zOTlJMBgkHA67Vq5Nk0cLRCIRRMT1uSDxeBwR4dSpU54MIVZqu5udneWVV16hp6fHtcRhD8WdmZkhEom4mrAaafJokVAoVB/Ku7Cw4FpHeiKRwLIsTp8+rQlEKRdlMhlOnTpFd3e3a5MAq9VqfbBOR0cHfr/flXLXosmjheyRWJlMhkwm41pHeldXF+VyWROIUi7JZDK89NJLJJNJ15qMVlZWmJqaolgsujIU9140ebSYPRIrn88zNTXl2qZOyWRSE4hSLrBrHF1dXa4ljmKxyMTEBJZluTqi6m40eTgkFotRrVaZmJhw7WauCUQpZ83OzvKNb3yDRCLhSuKw+zempqYIh8OedIzfiSYPB4XDYUKhEDdv3nRtQmEymWRlZYVvfOMbOgpLqRaanJx0tamqWq0yNTXVFv0ba9Hk4TC/31+fUDg1NeXKfJCuri6MMbz44os6D0SpFhgfH+fll18mlUq5kjhKpRITExOsrKy0Rf/GWjR5uMCeULiyssLExARLS0uOl5lIJAgEArzwwgtks1nHy1Nqq7py5Up9HofTo6qMMeRyOSYnJz1ZkbcZmjxcFIlECAaDTE1NMT8/7/horHg8TiwW44UXXmB2dtbRspTaaizL4vXXX2d4eJh0Ok0wGHS0vEqlUr83xGIxz+ZvrJcmD5f5/X5isRgLCwtMTk46Pis9Go2STCZ56aWXGB8fd7QspbaKSqXC+fPnuXTpEv39/Y7fyPP5PG+88UZbN1Pdri2Sh4h8SkSmReR8w7GUiHxZRC7W/u2uHRcR+S0RGRORsyLyqHeRb4yIEIvFMMYwMTHh+KTCcDhMb28vZ8+e5fXXX9cdCZW6i+XlZV5++WVu3rxJf3+/o2tVVatVZmdnmZ2dJRwOt3Uz1e3aInkAfwC867ZjPwf8vTHmAPD3te8B3g0cqH09A/yuSzG2XCgUqk8qdLoWEggE6Ovr4/Lly5w9e9a1+SdKbSb5fJ4XX3yRxcVFx5dVLxQK9T7QdhxNdS9tkTyMMf8ZuH2DiqeBT9cefxr4vobjf2hWvQgkRWSnO5G2nj2p0J4T4mQtxOfzsWPHDmZmZnQor1K3mZ2d5etf/zoi4uhGTnZtY2ZmhmAwuKlqG43aInncwQ5jzE2A2r/2Ivm7gBsN143Xjt1CRJ4RkdMicnpmZsbxYO9XOByu10Ju3rzJ8vKyY2X19PRQLpf5p3/6J91USm17lmVx+fLl+uS/eDzuSDnGmHrfxmatbTRq5+RxJ2v1JL3lo7ox5lljzEljzEm3d/XaKLsWYozh5s2bzM/POzYvpKurqz4S68qVK9oPoralcrnMq6++ysjICDt27HBsDke5XGZqaorZ2VlCodCmrW00auexYFMistMYc7PWLDVdOz4O7Gm4bjewpTb1DoVCBINBFhcXyefzpFIpRzazj0QipNNpLly4wPz8PMeOHXNtdVClvJbNZjlz5gyVSoX+/n5HyqhWqywuLpLNZgkEAq5tEeuGdq55PAe8v/b4/cAXGo7/aG3U1ZNAzm7e2kpEhGg0SjAYZGZmhqmpKUc61AOBAP39/WQyGb7+9a8zPz/f8jKUaieWZXH16lVeeOEFAoGAI/0bxph6h3gulyMWi7XVulSt0BY1DxH5LPBtQK+IjAMfA34V+LyIfAi4Dvxg7fLngfcAY0AR+KDrAbvIXt5keXmZiYkJEokEXV1dLW8rTaVSLC0t8cILL3Do0CGGhoZc3U5TKTeUSiXOnz/P9PS0Yzv/LS8vMz8/T6lUIhKJbOp+jbtpi+RhjPmhO5x6xxrXGuDDzkbUfuxFFvP5PIuLi3R3dxOPx1t6g49Go4TDYUZHR5menuZtb3ubY52HSrltcnKS8+dXp5I50UxVqVTIZrPk83mCweCWaqJaS1skD7U+dlOWZVnMz8+Ty+Xo7u5u6YxUezjvwsICX/3qVzl69Ch79uzRWojatMrlMqOjo1y/fp3u7u6WNx/Z/Rq5XK4+6GU70OSxCfl8vvp+IbOzs2SzWbq7u1vaqZ5IJIjFYrz22mvcvHmTY8eOaS1EbTqTk5MMDw9jjGl5baNarZLP58lms/UPdpthWZFW0eSxidn9IZVKpT7hKJVKEYlEWvKfOBAI3FILOXjwIIODg22/YJtSxWKRCxcuMDk52fJl1KvVKoVCob5HTyQS2ZY1c70LbAGBQIBAIFBflTMQCLS0JpJIJOjo6GBsbIwbN25w7Ngxent7WxC5Uq1VqVS4ceMGr7/+OqFQiJ07W7f4hF3TyOVy2zpp2DR5bCGNSWRmZoZAIEAymSQajd73iA+/3086naZUKvHSSy/R19fH4cOHtSlLtY2ZmRmGh4cplUqkUqmW1ZArlUo9aYgI4XB4WycNmyaPLchOItVqlbm5OXw+X732cL9/UJFIhP7+fnK5HF/96lcZGhpi7969W2LGrNqccrkcIyMjZDIZurq6SCQSLXnd5eXl+kRdn8+37fo07kWTxxZm7x1iWRYLCwvMz88Tj8fp7OwkFArd1x9CV1cXnZ2djI+Pc/36dfbv38/u3bt1hrpyTT6f59KlS4yPj9PR0cGOHTvu+zUty2JpaYlcLke5XK7/DWnSeCtNHtuAz+cjEolgjGFpaak+Dr2rq+u+mrR8Ph+pVIpKpcLFixe5dOkS+/fvZ9euXZpElGPy+TxXr17l+vXrRCKRlvRrrKysUCgUWFxcpFqtEgqFts2Q243S5LGNiEi9ealSqTA3NwdAR0cH8XiccDi8oU9YgUCAdDrNysoKo6OjjI2N8eCDD7Jr164ttySD8s7CwgJXr17ljTfeIBgM0tfXd199D5ZlUSqVWFhYoFQq4fP5tD+jCZo8tim7X6SxNhIIBEgkEvU1tZoVDAZJp9NUKhXGxsa4ePEiDzzwAHv27NGOdbUhlmWRzWa5dOkSMzMz9cU8N3qDN8awvLxMoVCgUChgWZbWMjZIk8c211gbqVarZLNZMpkMoVCIRCJBJBJpupM9EAjQ29uLZVm88cYbXLlyhb6+PoaGhuju7tZPduqeVlZWmJqa4vLlyxSLxfpAjY0wxlAul1laWqo3S/n9/pbNh9quNHmoOr/fTzQaBVabtTKZTP2TWTweb7pGYveJwGqTw0svvUQ0GmVwcJD+/v56WUrZcrkc4+PjTExMUK1WSSQS9PX13fuJt7FrGKVS6ZaEEQqF9MNLi2jyUGuym7XgzQXfMpkMgUCAeDxOJBJp6g8xkUiQSCQol8tcvHiRkZERenp6GBwcJJVKbaiZTG0NxWKRmZkZrl69SrFYJBQKbaiGWq1WKZfLFAoFisUilmVpwnCQJg91T42JpHFzG3s9H3uvgkAgcM9mgFAoVJ+dns/neeWVV+oTEHft2kV3d7cmkm2gWCwyNzfH+Pg42WwWv99PZ2cnnZ2d634Ny7JYWVlheXmZfD7PysoKxhgCgYA2SblAk4dqit/vrw/tNcawsrLC3Nwcxph6s1ckEllXMonH48TjcSzLIpfLMTU1hYiQSqUYGBioL7GiNj/LslhcXCSTyTA+Pl6feBePx9c9P8NOFisrK+TzeZaXlzHG4PP5CAaD+n/FZZo81IaJCKFQqD6nwx76WCgU6n/UkUik3sQVDAbXnFNiz4BPJBJYlkWxWOT8+fNYlkUsFqO/v5+enh46Ozt16O8mYYyhWCyyuLjI9PQ0U1NTVKtVRITOzs57JgxjDNVqtV6zWFpaYnl5GVj9fxcMBnXGt8c0eaiWscfJ2+yaSalUwrIsRKQ+ysUexRUMBvH5fPWbgP1p1B7aWy6XGR8f5/Lly4gIHR0dpNPp+mZY0WhU27PbgL1oYKFQYHZ2ltnZ2fq2yeFwmGQyecfJqMYYKpUKlUqFcrlMqVRieXm5/n/G5/Ntuf2/twJNHsoxds2kUWPt5Pbr7N0S/X4/gUCg3tnZ+BrlcpmbN29y7do1LMsiEAjQ1dVFKpWis7OTWCxGJBLRfhOH2MNe7d/h/Pw88/Pzt/w+w+Ew8Xj8LUO8LcuiWq1SrVapVCr10VCVSgVjTD1R6DDazWHTJg8ReRfwvwN+4PeNMb/qcUhqHW6vncCbTRT5fB7Lsm45ZyeQYDBIMBgkEAgQjUbrW/DaQzKvXbvGysoKIoIxhnA4XO+AtUeH2ckpGAzqjeku7Oaicrlcv8EvLCywuLhYH/Zq//zspJ9Op7Esq54g7Brn8vJyvZ+i8XfbWKPQpWw2p02ZPETED/wO8B3AOHBKRJ4zxrzmbWRqI0TklhFdjRo/pd6eWID6Dcjv99f7VPx+P9Vqlfn5eWZnZ6lWq8BqIhKRepu5XUuxv+wEZb9m47/2lx2vffNspyRkjKn/a3813tBv/+R/ezNRsVis3+ztn7X9s7N/tnbir1QqVKvV+kgnu5xG9s/JThDavLi13DN5iMhXgJ82xrzqQjzr9TgwZoy5DCAinwOeBlqePD72sY/x1a9+VTtq25x9s2zF825PCHf7/vYkcq/nbtTtMTZ+35g07nT9nc6v9V7Wqx0TqHqrHTt28IEPfKDlr7uemsfPAr8hIteAnzfG3Gx5FM3bBdxo+H4ceKLxAhF5BngG4IEHHnAvMuWJxtpAM9ZKHrffZO92fq3Xc8vd3u+9klhjLUCTgNqIeyYPY8wrwLeLyPcDXxSRvwD+gzFmyfHo7myt/+W3/NUaY54FngU4efLkhv+iP/7xj/PFL36R3bt3b/QllAuq1eotzTTAms1cjexr7T6Xxuc03lztJiu7P+X2T+v2HJfGZi37eWtdf6+b9J0SVuMxO+7GmO1rGl/ffl+Nz7f/teO1R0HZzXr2NWs9tp9jd2zbfReqfY2Pjzvyuuvq85DV/zmvA78L/Arwb0TkI8aYP3IkqnsbB/Y0fL8bmPAoFuWwxrb6OzVP2SO0wuFwvd+jcQiofRNtvMEHg0FCoVC9zyMcDtefb7fR269lJ4/GRNJ447xb808rPtGvp9mqMRmu1d9hWdYtfR12h7g9j6JcLt+SRO3XCYVCt6zC3PgztftHbk9a9s/I/vlprWbrWU+fx9eAfcAw8CLwAWAE+EkR+WfGmGccjXBtp4ADIjIEvAG8F/hhD+JQLWJ/+reHbTZ+SrZvXPZkQ/tmbn81LoLXOAs5FArVtyVtXI+r8Wa4Wbhx87V/B3ZiaVyJNp/P3zLSyl6axh7BBtySqOwEZU/yaxyh1TjSaqMbkSnvreev58eAYfPWj3s/ISIXHIjpnowxFRH5ceBLrA7V/ZQxZtiLWFTz7E+sjaOC7NFS8Xj8lrkejaOcYPUGZY/wqVQq9aUpEokEAwMD9WXkda5H8xpHva211EfjHI9isVjf2nh+fr7+ewyHw0Qikbc836452onFrvHYs8btFQkaf++qva2nz+P8XU7/ixbG0hRjzPPA816Vr9bHrgnYzSFA/eZkr391p2VLYPWmUigUWFpawrIsfD4fXV1dPHlTbH8AABjuSURBVPjgg3R1dRGLxXRpd5eISL1pr6urq779q73vdz6frw+PnpmZqdcaOzo66s2BNnsFgcYaZ+O8ksZmRjuhadNXe7mvers9VFYpm12rsP/47eam2+dR3I29rHalUgGgu7ubwcFBurq6iMfj2tTRZnw+Hx0dHXR0dNTXrCqXyywuLjI/P8/NmzfJ5XIA9cmbjX1Fjc2StsaJiqVSqd4kCdyyrI3yzuZp9FVtqTFZ2DeCzs7OerJY741+eXm53qYejUbZvXs36XSaRCKxqfom1KpQKERPTw89PT3s37+fUqlELpdjcnKSqampWzYZW+v3a3e0RyIREolEvaN+ZWWFYrFYr4natZtQKKQ1E5fpX6Vqir3Yod2xbW8OZe8y2EytoFwus7CwQLVaJRaLceDAAXp7e4nH43oj2GLs2ueOHTuoVqv1RDIxMUGlUnlLjeR29ui4xqXX7YERxWKRYrFYH/Fl95kpZ2nyUPdkj2ayaxeRSISuri7C4XDTndL2vg7Ly8tEIhH2799POp1uahMgtbn5/X5SqRSpVIrDhw+TzWaZmJjg5s2bVKtV4vH4uvbmaEwmdme+PZiiVCoB1NcyU62nyUOtqTFh+Hw+YrEYHR0dG/5Ut7y8zMLCAgA7d+5k9+7dJJNJbbfe5ux97lOpFIcOHWJ2dparV68yNTVVXzF5Pc2WjZ35iUSCSqVSX/m3UCjcssaWag1NHqrO/vRmD4G1PwHez6J29h4P0WiUI0eO0N/fr3/Aak3BYJCdO3eyc+dOFhcXmZiY4Pr161iWRSKRaGp9Obs5NR6P1xdwXFxcrC8db4/0UxunPz1VHyYJEIvF6O3tva+EYYxhYWGBpaUlent7OXr0KKlUSmsZat06Ozs5dOgQ+/btY3JykkuXLpHNZut7tjTD7/cTi8WIxWJUq9X6xMdCoYDP59M+kg3S5LFNWZZVX47CHhkTjUbv64/IsiwWFhYolUrs2rWLvXv30tXV1cKo1XYTDAbZs2cPu3btYnZ2ltHRUaampojH4xvaWdDv99drJHZnu/1/1l7eRq2PJo9txp7da+8bbjdL3a9sNkupVGLPnj0MDQ3VJ4Ep1Qo+n4++vj56e3vJZDL1JLKRmogtGAzWl69pbNZqXNtM3Zkmj21ieXmZSqVCKBSit7eXWCzWkj+OhYUFCoUCu3fv5sEHH9SkoRzl8/no7e2lp6eHubk5RkZGmJ6ero/+2wh7BGEkEqG7u5tCocDCwkJ9Lor2jaxNfypbWOOIqY6ODnp7ewmHwy2ZQ1Eqlchms/T29nLixAltnlKuEhF6e3t5+9vfzuTkJKOjo+RyOVKp1H3d7O0RXp2dnSwtLZHL5SgUCvUVmNWbNHlsQfbigcaY+j7erRrrXqlUmJubIxaL8fjjj9Pb29uS11VqI3w+HwMDA/T19TE+Ps7o6CgiQiqVuq8PSfaSK7FYjOXl5XoSsWe9K00eW4plWZRKJUSERCJBZ2dnS0eRZDIZqtUqR44cYc+ePTpCRbWNQCDA3r176e/vZ2xsjOvXr9f79O5HY5OWPVdJk8gqTR5bQGPSSCaTLV880G6iGhgY4PDhw9v+j0a1r0gkwrFjxxgYGGB4eJipqSl6enpa0m8RDodJp9Mkk0lyuRz5fH5bJxFNHpuYMYZSqYQxxpGkYVkW8/PzBAIBbaJSm0oqleKpp57ixo0bjIyMEI1GSSQSLXntYDBIb28viUSChYUF8vn8thzmq8ljE7I7wi3LqnfutboJya5tDA4OcuDAAe0sVJtOIBBgaGiIdDrNuXPnWloLAeojFxOJBPPz8xQKhW21lpYmj03GHnIbj8dJJpOODCPMZDL4fD6tbagtIR6P88QTT3Dt2jVGRkbo6Oho6ZDyUCjEjh07KJVKZDIZCoUCkUhky/cJavLYJKrVKqVSiUgkQjqddqSKXC6XmZubY9euXRw9elRrG2rL8Pl8DA0N0dPTw6uvvsrMzAw9PT0tnQgYiUTYuXMnxWKRTCZTXzl6q0429PRdicgPisiwiFgicvK2cx8RkTEReV1Evqvh+Ltqx8ZE5Ofcj9pdxhiKxSIrKyv09fWxY8cORxLHwsIC2WyWhx9+mEceeUQTh9qSEokETz31FLt372Zqaqq+pluriAgdHR3s2rWLZDJZ3wVxK/K65nEe+K+B/6vxoIgcBd4LPAQMAF8RkYO1078DfAcwDpwSkeeMMa+5F7J77L2cu7u777pRzv2amZmho6ODb/3Wb9UZ4mrLCwQCPPTQQ3R3d3Pu3Ln6Mu6t5PP56OrqIhaL1ftDtlpTlqfJwxhzAVhrMs/TwOeMMcvAFREZAx6vnRuz904Xkc/Vrt1SycNuoopGo6RSKcc64CqVCjMzMwwODnL48GFdhkFtKwMDAyQSCc6cOcPMzAzpdLrlZQSDQfr6+lhaWmJubo7l5WWi0eiW2CmzXRvjdgE3Gr4frx270/G3EJFnROS0iJyemZlxLNBWK5VKrKyskE6n6evrcyxxlEolZmdnOX78OMeOHdPEobaleDzOk08+ya5du5icnKRSqThSTjQaZWBggK6uLorFYsuby7zg+B1DRL4C9K9x6qPGmC/c6WlrHDOsnezMWi9gjHkWeBbg5MmTa17TTux9Bjo7O+nu7na0epvL5bAsi7e//e0kk0nHylFqMwgEAhw7dox4PM7w8DCpVMqRfkWfz0cymSQajTI3N0ehUCAWi23aWojjycMY884NPG0c2NPw/W5govb4Tsc3LbtDrb+/n2g06mhZ9rpUJ06cuO+lG5TaSvbu3Us8HueVV15hZWXFsf6/cDhMf38/i4uLzM/Pb9pFF9u12eo54L0iEhaRIeAA8BJwCjggIkMiEmK1U/05D+O8L9Vqtf7pY2BgwNHEYVkWU1NTpNNpnnjiCU0cSq3BXqnXXl3BKXaH+sDAACJCsVjEmLZvILmF10N1/ysRGQeeAv4/EfkSgDFmGPg8qx3hXwQ+bIypGmMqwI8DXwIuAJ+vXbvplEolyuUyfX199PT0ONpMValUmJqaYt++fRw/flz7N5S6C7sfpKOjA6f7S0OhEP39/fW+EKf6XJzg9WirvwT+8g7nPgF8Yo3jzwPPOxyaYyzLYmlpiVgs5njSgDcn/j300EPs3bvX0bKU2ioikQgnT57k/PnzTExM0NfX59hQebsvJBKJMDMzQ6VS2RSLLbZrs9WWtLKywtLSEqlUinQ67XjiWF5eJpPJcOLECU0cSjUpEAhw/PhxBgcHmZqawrIsR8uLRCL15utCoeB4efdL2y9cYi+ZvnPnTldW3yyVSuRyOZ544gl6enocL0+prcjn83HkyBECgQAXL16kr6/P0WZfv99Pb28vkUiE2dnZtu5M1+ThMLuZKh6POz4E11YsFllaWuKpp57SobhK3Sefz8ehQ4cIhUIMDw87nkBgtd8lFAoxPT1dX9Ou3WizlYMqlQpLS0v09PS40r8BkM/nKZVKPPHEE5o4lGqhoaEhjh8/Xu+XcFooFGLnzp1t24ylNQ+HNM7dcOtTQz6fp1wu8/jjj7d8rR6lFDzwwAP4fD5effVV0um04zUQv99PT08PoVCITCZDOBxum9GS7RHFFlMsFutbVrq1EJqdOJ588kld3FApB+3evRvAtQQiIiQSCUKhUFuNxtJmqxayLItCoUA8Hqevr8+1xGGvlfPEE09o4lDKBbt37+bhhx92rQkL3twvxO/3s7S05EqZd6PJo0Xstans/g23NoCxO8cff/xxOjs7XSlTKbWaQI4dO8b09LRrCSQQCNDf308sFqNQKHg6K12TRwuUy2WWl5fp7+939QZeKpUoFAo89thj2sehlAcGBwd56KGHmJmZca1D2+fz0dPTQzKZpFAoUK1WXSn3LXF4UuoWUiqVMMawc+dOV9shy+UyuVyOxx57TEdVKeWhoaEhDhw4wPT0tGsJRERIJpOk02lKpZIny5po8rgPS0tLBAIBdu7c6epEnkqlwtzcHCdOnCCVSrlWrlJqbQcPHmRoaIjp6WlXy43H4/T397OysuL6HiGaPDbAGEOhUCAajbJjxw5Xt5a0LIvZ2Vkefvhh+vvX2iZFKeWFQ4cOMTAw4PhiirezO9KNMa7ul67Jo0nGGIrFIolEgt7eXtc6xmE1cUxPT3Po0KH6cEGlVHvw+XwcO3aMZDLp6HLuawkGg/WRWG4lEE0eTbAsi2KxSHd3N93d3a7vAGbvN/7ggw+6Wq5San0CgQAnTpwgHA6zsLDgatl+v58dO3YQDAYpFouOl6fJY52q1SrFYpGenh66urpcTxyZTIZ0Os2RI0dcLVcp1ZxQKMSjjz5KpVJx5SbeyO/309fXR0dHB4VCwdGyNHmsg2VZlEol+vr6PJlLkcvliEQiHD9+3NVmMqXUxsRiMU6ePMni4qL7Hdm1obydnZ2OJhC9E62Dz+erZ3O3lUolqtUqjz76aNsuzayUeqtkMsnDDz9MJpNxfVFDESGVSjnaSqLJ4x7C4TB9fX2e7Pltz+U4efKk7jmu1CY0MDDAoUOHXB/CC2/OBent7XUkgWjyuAefz+fJJ37LsshkMrztbW/TSYBKbWJDQ0MMDAwwNzfnetki4tjmc54mDxH5NREZEZGzIvKXIpJsOPcRERkTkddF5Lsajr+rdmxMRH7Om8idNzMzw9DQkA7JVWqTs4fwRqNR10dgOcnrmseXgWPGmOPAKPARABE5CrwXeAh4F/B/iohfRPzA7wDvBo4CP1S7dkvJZDKkUikOHTrkdShKqRYIBAI8+uijrKyssLy87HU4LeFp8jDG/J0xxl6U5UXA/pj9NPA5Y8yyMeYKMAY8XvsaM8ZcNsaUgc/Vrt0yisUiPp+Phx9+WEdWKbWFxGIxTpw44UkHuhPa6e70r4G/rT3eBdxoODdeO3an428hIs+IyGkROe32cgEbValUWFhY4NFHH22LzV6UUq3V29vL4cOHXV/CxAmOJw8R+YqInF/j6+mGaz4KVIDP2IfWeClzl+NvPWjMs8aYk8aYk+l0+n7fhitmZ2fryxsopbamoaEh0uk0mUzG61Dui+Pb0Bpj3nm38yLyfuC7gXeYN3c2GQf2NFy2G5ioPb7T8U1tbm6OgYEBHnjgAa9DUUo5yOfzcfz4cb72ta+xtLRENBr1OqQN8Xq01buAfwt8rzGmcR7/c8B7RSQsIkPAAeAl4BRwQESGRCTEaqf6c27H3WrFYpFQKMTRo1uu718ptQZ7CZNcLufJXhyt4HWfx28DncCXReSMiPwegDFmGPg88BrwReDDxphqrXP9x4EvAReAz9eu3bQqlQqLi4ucOHFCZ5ArtY0kk0mOHDnC7Oys16FsiOPNVndjjNl/l3OfAD6xxvHngeedjMtNdj+HbiOr1PYzODjI3Nwc2Wx20/V1el3z2NYymQx9fX06EVCpbcqeQGhZ1qab/6HJwyOlUqn+H0fncyi1fUUiER555BHm5+c31fwPvWt5wLIs5ufnOX78uM7nUErR29vL0NCQJ+tfbZQmDw/Mzc2xb98+ent7vQ5FKdUmDhw4QDQaJZ/Pex3KumjycFmxWCQajXLgwAGvQ1FKtZFAIMAjjzxCPp/fFMN3NXm4yLIsFhYWOH78OIGApwPdlFJtKJFIcOTIkU3RfKXJw0UzMzMcOnRo0w3JU0q5Z3BwkEQi0fbLt2vycEk+n6ezs5OhoSGvQ1FKtTF7Ve1SqdTWzVeaPFxQqVQoFAocP35ch+Uqpe4pHo9z6NChtm6+0juZC+bm5jh48KDOIldKrVu7N19p8nCY3Vw1ODjodShKqU3E5/Pxtre9jaWlpbacPKjJw0GWZVEoFDh27JiOrlJKNS2RSHDw4MG2bL7S5OEgezKgjq5SSm3U4OAgHR0dFIvFe1/sIk0eDimVSoTDYR588EGvQ1FKbWKBQIBjx46Ry+XaqvlKk4dDstmsNlcppVoimUwyNDTE/Py816HUafJwQCaTYWBgQNeuUkq1zP79+/H7/W2zdLsmjxYrl8sYYzh8+LDXoSiltpBQKMThw4fbpvahyaPF5ufnOXTokC61rpRquYGBAfr6+sjlcl6H4m3yEJFfFpGztf3L/05EBmrHRUR+S0TGaucfbXjO+0XkYu3r/d5F/1b2nA7dGVAp5ZTDhw+3xdIlXtc8fs0Yc9wY8wjwN8C/qx1/N3Cg9vUM8LsAIpICPgY8ATwOfExEul2Peg2WZbG4uKg7AyqlHBWPxzl48KDnzVee3uWMMY3z7jsAU3v8NPCHZtWLQFJEdgLfBXzZGJMxxswDXwbe5WrQd5DNZhkcHNQ5HUopxw0ODhIOhz3tPPf8I7KIfEJEbgDv482axy7gRsNl47Vjdzq+1us+IyKnReT0zMxM6wNvYHeS79+/39FylFIKVud+HD161NPah+PJQ0S+IiLn1/h6GsAY81FjzB7gM8CP209b46XMXY6/9aAxzxpjThpjTqbT6Va8lTvSTnKllNv6+vro7e31rPPc8eRhjHmnMebYGl9fuO3SPwG+v/Z4HNjTcG43MHGX454pFot0dHRoJ7lSynVHjx71bOFEr0dbNW7k/b3ASO3xc8CP1kZdPQnkjDE3gS8B3yki3bWO8u+sHfPMwsKCdpIrpTwRj8d58MEHPWm+8nrtjF8VkUOABVwDfqx2/HngPcAYUAQ+CGCMyYjILwOnatf9kjEm427Ib8rlcvT395NKpbwKQSm1zQ0NDXHjxg3K5TKhUMi1cj1NHsaY77/DcQN8+A7nPgV8ysm41sOyLEqlEo899pjXoSiltrFQKMTBgwc5f/48/f39rpWrbS0bND8/z759+4jH416HopTa5nbv3k08HqdUKrlWpiaPDSiXy8BqdVEppbzm8/k4evQo2WzWvTJdK2kLsYfmutm+qJRSd9Pb20tvb69re55r8mhSqVQiGo0yMDDgdShKKXWLw4cPuzZ0V5NHk7LZLIcPH9ZNnpRSbSeRSLBnzx5Xmq80eTQhn8+TTCbp6+vzOhSllFrTvn37WFlZcbz2ocmjCfl8nsOHD+uEQKVU24rFYuzbt8/xiYN6F1ynXC5HX1+fTghUSrU9eySok3t+aPJYB2MMpVKJgwcPeh2KUkrdUygUYv/+/Y7WPjR5rEOhUGDXrl0kEgmvQ1FKqXXZs2cPwWCQ1QU7Wk+TxzrE43H27dvndRhKKbVugUCAgwcPOtZHq+NN78FeN0aXIVFKbTb9/f0sLi46kkA0edxDPB7XxKGU2pQCgQBHjhxx5LW12UoppVTTNHkopZRqmiYPpZRSTdPkoZRSqmmaPJRSSjVNk4dSSqmmafJQSinVNE0eSimlmiZOrXvSTkRkBrjmdRwb0AvMeh2Ey/Q9bw/6njeHQWNMeq0T2yJ5bFYictoYc9LrONyk73l70Pe8+WmzlVJKqaZp8lBKKdU0TR7t7VmvA/CAvuftQd/zJqd9HkoppZqmNQ+llFJN0+ShlFKqaZo8NgkR+RkRMSLS63UsThORXxORERE5KyJ/KSJJr2Nygoi8S0ReF5ExEfk5r+NxmojsEZF/EJELIjIsIj/pdUxuERG/iHxTRP7G61haRZPHJiAie4DvAK57HYtLvgwcM8YcB0aBj3gcT8uJiB/4HeDdwFHgh0TkqLdROa4C/LQx5gjwJPDhbfCebT8JXPA6iFbS5LE5/Abws8C2GN1gjPk7Y0yl9u2LwG4v43HI48CYMeayMaYMfA542uOYHGWMuWmMeaX2eJHVm+kub6NynojsBv4F8Ptex9JKmjzanIh8L/CGMeZVr2PxyL8G/tbrIBywC7jR8P042+BGahORvcAJ4BveRuKK32T1w5/ldSCtFPA6AAUi8hWgf41THwV+HvhOdyNy3t3eszHmC7VrPspqU8dn3IzNJbLGsW1RsxSROPDnwE8ZYxa8jsdJIvLdwLQx5mUR+Tav42klTR5twBjzzrWOi8jbgCHgVRGB1eabV0TkcWPMpIshttyd3rNNRN4PfDfwDrM1JyONA3savt8NTHgUi2tEJMhq4viMMeYvvI7HBd8CfK+IvAeIAAkR+WNjzL/yOK77ppMENxERuQqcNMZstpU5myIi7wJ+HfjnxpgZr+NxgogEWB0M8A7gDeAU8MPGmGFPA3OQrH4C+jSQMcb8lNfxuK1W8/gZY8x3ex1LK2ifh2pHvw10Al8WkTMi8nteB9RqtQEBPw58idWO489v5cRR8y3AjwDfXvu9nql9IlebkNY8lFJKNU1rHkoppZqmyUMppVTTNHkopZRqmiYPpZRSTdPkoZRSqmmaPJRSSjVNk4dSSqmmafJQyiO1vS2+o/b4V0Tkt7yOSan10rWtlPLOx4BfEpE+VleY/V6P41Fq3XSGuVIeEpF/BOLAt9X2uFBqU9BmK6U8Uls1eSewrIlDbTaaPJTygIjsZHWfkqeBgoh8l8chKdUUTR5KuUxEYsBfsLqf9wXgl4Ff9DQopZqkfR5KKaWapjUPpZRSTdPkoZRSqmmaPJRSSjVNk4dSSqmmafJQSinVNE0eSimlmqbJQymlVNP+f1SebJYOtXKXAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "## EDIT THIS CELL\n",
    "\n",
    "# compute the feature matrix for the test inputs\n",
    "Phi_test = poly_features(Xtest, pol_deg) # N x (pol_deg+1) feature matrix SOLUTION\n",
    "\n",
    "# compute the (marginal) prior at the test input locations\n",
    "# prior mean\n",
    "prior_mean = np.zeros((Ntest,1)) # prior mean <-- SOLUTION\n",
    "\n",
    "# prior variance\n",
    "full_covariance = Phi_test @ Phi_test.T * prior_var # N x N covariance matrix of all function values\n",
    "prior_marginal_var =  np.diag(full_covariance)\n",
    "\n",
    "# Let us visualize the prior over functions\n",
    "plt.figure()\n",
    "plt.plot(Xtest, prior_mean, color=\"k\")\n",
    "\n",
    "conf_bound1 = np.sqrt(prior_marginal_var).flatten()\n",
    "conf_bound2 = 2.0*np.sqrt(prior_marginal_var).flatten()\n",
    "conf_bound3 = 2.0*np.sqrt(prior_marginal_var + noise_var).flatten()\n",
    "plt.fill_between(Xtest.flatten(), prior_mean.flatten() + conf_bound1, \n",
    "             prior_mean.flatten() - conf_bound1, alpha = 0.1, color=\"k\")\n",
    "plt.fill_between(Xtest.flatten(), prior_mean.flatten() + conf_bound2, \n",
    "                 prior_mean.flatten() - conf_bound2, alpha = 0.1, color=\"k\")\n",
    "plt.fill_between(Xtest.flatten(), prior_mean.flatten() + conf_bound3, \n",
    "                 prior_mean.flatten() - conf_bound3, alpha = 0.1, color=\"k\")\n",
    "\n",
    "plt.xlabel('$x$')\n",
    "plt.ylabel('$y$')\n",
    "plt.title(\"Prior over functions\");"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, we will use this prior distribution and sample functions from it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Every sampled function is a polynomial of degree 3\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX8AAAEICAYAAAC3Y/QeAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nO2dd3gU1dfHvycFQu81gBQRASuiiKg0CygKNuyCvf7U147YFcGKvWDBAmJvoKKooAICgjRBeg29hRBSSHbP+8d3x91sNskm25Ls+TzPPDs7M3vnzOzMueeee+65oqowDMMw4ouEWAtgGIZhRB9T/oZhGHGIKX/DMIw4xJS/YRhGHGLK3zAMIw4x5W8YhhGHmPKvBIjINBG5JsLnWCIivTzrj4jIuGKOXScip5TxPDeKyDYRyRSRBmUUtyznvV9E3o7W+cJNSf9JGMpvLSIqIkmROkcJ539DRB6MxbkrKzH5I43SIyLrADQB4AKwH8D3AP6nqpnROL+qdo70OUQkGcDzAI5X1YURPE8vAONUtYWzTVWfjNT5KiKe5+0aVf051rIAgKreEGsZKhtm+VcszlLVmgC6ADgWwAMxlifcNAGQAmBJrAWJF4SUaz0gIokh/NYM3CIo13+6ERhV3QTgBwCH+e8TkXYi8quI7BKRnSIyXkTq+uxXETnY5/t7IvKEZ72hiEwSkXQR2S0ifziKIYArJ0VEPhGRfSLyt4gcGUhWEUkQkftEZLVHpk9FpH6A4w4BsNzzNd1zDYVcDb4uLhEZKiLTReRZEdkjImtFpL/PsfVFZKyIbPbs/1pEanjuXXOPaylTRJr7u01E5GyPqyvdc86OPvvWichdIrJIRPZ67kNKSffQ73pDvbY2IvKb5/5PAdDQr/zjRWSmR46FjsvO5zwjRGQGgCwAbf1++yGAVgAmeu7PPT67LxWRDZ5na3hp/2fPsb1EJM3jatvpuZ+X+ux/T0ReF5HvRWQ/gN6+z6nnmGtFZJXnHn8rIs199qmI3CwiKwGsDCSDYcq/QiIiLQGcAWB+oN0ARgJoDqAjgJYAHgmy6DsBpAFoBFrh9wMoKv/HQACfAagP4CMAXwvdNv7cCmAQgJ4emfYAeNX/IFVdAcBxLdVV1T5BytwNrDQaAngawDsiIp59HwKo7im3MYDRqrofQH8Am1W1pmfZ7FugpyKaAOB28F58DyrCKj6HDQbQD0AbAEcAGOrZXpp7GMq1fQRgnmff4wCG+MifCuA7AE+A/89dAL4QkUY+ZV8O4DoAtQCs9z2pql4OYAM8LU1Vfdpn94kAOgDoC+Ahn0oxqP/Zh6Ye2VM9so8RkQ4++y8BMMIj33TfH4pIH/AZHwygmUf+j/3KHwTev07FyBDXmPKvWHwtIungy/AbgEJ+alVdpapTVDVXVXeAPvSeQZafB75MB6lqnqr+oUUnf5qnqp+rap7nHCkAjg9w3PUAhqtqmqrmghXR+RK+5vh6VX1LVV0A3vfI30REmoFK/gZV3eO5nt+CLPNCAN957mMegGcBVANwgs8xL6nqZlXdDWAigKM820tzD8t6ba1At9+Dnv/5d48MDpcB+F5Vv1dVt6pOATAXNBgc3lPVJaqa77nGYHlUVbM9fTILATgtvrL8z478v4GV1WCffd+o6gyP/Dl+v7sUwLuq+rfnXMMAdBeR1j7HjFTV3aqaXYpriytM+VcsBqlqXVU9SFVvCvRgi0hjEflYRDaJSAaAcfBzCRTDMwBWAfhJRNaIyH3FHLvRWVFVN2jtNg9w3EEAvvK4H9IB/At2WjcJUqaS2OojR5ZntSbY4tmtqnvKUGZz+FjDnuvbCFqphc4Luk5qetZLcw9Loqhraw5gj6cV4+BrvR8E4ALnnnvu+4lg5eGwEWWjqOsu7f8cSH7f56c4+fz/n0wAu1Dw/ynr9cUNpvwrHyNBN8MRqlobtALFZ38W6ApxaOqsqOo+Vb1TVdsCOAvAHSLSt4jztHRWPD7tFgA2BzhuI4D+nkrLWVI8/RYl4SiHgPKWwEYA9cWnv8OHkizxzaAyA8BOUfB6S5S5FPcwlGvbAqCep//CoZXP+kYAH/rd8xqqOspX1JIuJUhZfM9Zmv85kPy+z09x5/f/f2oAaICC/4+lKy4BU/6Vj1oAMsFO01QAd/vtXwDgEhFJFJF+8HEJicgAETnYo+wyQMvNVcR5jhGRcz3N+tsB5AKYFeC4NwCMEJGDPOdoJCIDg7kQj9tqE4DLPPJeBaBdkL/dAnbsviYi9UQkWURO9uzeBqCBiNQp4uefAjhTRPp6+jHu9FzfzJLOG+w9DPHa1oNunEdFpIqInAhWNA7jAJwlIqd7yk7xdLK2CFhgYLbBryO4BMryPzvynwRgANiHFAwfAbhSRI4Skaqg+3O2qq4rhbxxjyn/ysejYCjoXtCP+qXf/ttARZEO+k6/9tnXHsDPYOXxJ4DXVHVaEef5BvSN7wE7D88twnf8IoBvQTfIPrCC6FaK67kWrMB2gR23JSpgHy4HffDLAGwHKymo6jKwQ3eNx01RwF2lqsvBFtPLAHaC9+ssVT0QxDlLcw9DubZLwPu4G8DDAD7wkX8j2CF/P4AdoFV+N0r3vo8E8IDn/twVxPGl/Z+3gs/OZgDjwb6ZZcEIpqq/AHgQwBdgK6gdgIuC+a3hRWwyF8MwookEGGRnRB+z/A3DMOIQU/6GYRhxiLl9DMMw4hCz/A3DMOKQCpH0qGHDhtq6detYi2EYhlGhmDdv3k5VbRRoX4VQ/q1bt8bcuXNjLYZhGEaFQkTWF7XP3D6GYRhxiCl/wzCMOMSUv2EYRhxiyt8wDCMOMeVvGIYRh5jyNwzDiENM+RuGYcQhpvwNwzDKK+PHA+PGARFIw2PK3zAMozySkQH83/8B774bkeJN+RuGYZRHnnsO2LEDeOopQKTk40uJKX/DMIzyxtatVP4XXAAce2xETmHK3zAMo7zx0ENAbi4wYkTETmHK3zAMozyxeDHwzjvAzTcD7dtH7DQhK38RSRGROSKyUESWiMijnu1tRGS2iKwUkU9EpIpne1XP91We/a1DlcEwDKNSoArceSdQpw6t/wgSDss/F0AfVT0SwFEA+onI8QCeAjBaVdsD2APgas/xVwPYo6oHAxjtOc4wDMOYPBmYMoWKv379iJ4qZOWvJNPzNdmzKIA+AD73bH8fwCDP+kDPd3j29xWJQFe2YRhGRSI/n1b/wQcDN90U8dOFxecvIokisgDAdgBTAKwGkK6q+Z5D0gCketZTAWwEAM/+vQAaBCjzOhGZKyJzd+zYEQ4xDcMwyi9vvQX8+y/wzDNAlSoRP11YlL+qulT1KAAtABwHoGOgwzyfgaz8QsPXVHWMqnZV1a6NGgWchcwwDKNysHcv8PDDQM+ewMCBUTllWKN9VDUdwDQAxwOoKyLONJEtAGz2rKcBaAkAnv11AOwOpxyGYRgVipEjgZ07geefj8iArkCEI9qnkYjU9axXA3AKgH8BTAVwvuewIQC+8ax/6/kOz/5fVSOQuMIwDKMisHIlMHo0cMUVQJcuUTttOCZwbwbgfRFJBCuTT1V1kogsBfCxiDwBYD6AdzzHvwPgQxFZBVr8F4VBBsMwjIqHKnDbbUBKCjBqVFRPHbLyV9VFAI4OsH0N6P/3354D4IJQz2sYhlHh+fZb4IcfaPk3bRrVU9sIX8MwjFiQnQ3cfjvQuTNH80aZcLh9DMMwjNLy1FPAunXA1KlAcnLUT2+Wv2EYRrRZs4Y+/osuAnr1iokIpvwNwzCize2309p/9tmYiWBuH8MwjGjy3XfAxInA008DqaklHx8hzPI3DMOIFpmZzNvTqRNDPGOIWf6GYRjR4sEHgQ0bgBkzopK/pzjM8jcMw4gGf/0FvPQScOONwAknxFoaU/6GYRgRJy8PuPZaDuQaOTLW0gCIB7fP9u1Ao0ZRS5ZkGIZRiNGjgYULgS+/5Cxd5YDKbfmvWAF06MA82YZhGLFg9Wqmaz7nHC7lhMqt/A8+GOjaFfi//2PmPMMwjGiiCtxwAzt3X3451tIUoHIr/4QE4L33gKpVgcsv5zRphmEY0eKtt4Cff+Zo3hjG9Aeicit/gD3sL7wAzJ4NjBgRa2kMw4gX1q3jnLx9+wLXXx9raQpRuZX/8uXAeecBv/4KXHYZ8PjjrAQMwzAiidsNXHklA03efZdeiHJG+ZMonHTowEEV778PnHgim12XXw7s3x9ryQzDqMy8+iowbRqjfFq1irU0Aancyh8AHngAOPlkNr9GjABWrQLuuCPWUhmGUVlZuRK4917gjDOAq66KtTRFUvmVf1ISMH48p0l77jkq/jFjgM8/j7VkhmFUNlwuYOhQ6pu33irX44sqv/IHgBYtGPWzYAFnzznuOODqq5lT2zAMI1w89xwwcybDOps3j7U0xRIfyh8ABgxgvP9rr1Hxi3AihQMHYi2ZYRiVgXnz6GY+7zzgkkvCU+amTcCePeEpy4/4Uf4AY22POYb+uJEjGQZ6772xlsowjIpOZiZw8cVAkyZ0K4fD3XPgAHD++Zzpy+0OvTw/Qlb+ItJSRKaKyL8iskREbvNsry8iU0Rkpeeznme7iMhLIrJKRBaJSJdQZQiaKlWATz/l+pgxzK73wgvAt99GTQTDMCoh//sf0ziMHw/Urx+eMu+7D5g1i62JCISKhqPEfAB3qmpHAMcDuFlEOgG4D8AvqtoewC+e7wDQH0B7z3IdgNfDIEPwtG0LfPQRkyxlZLAlMHQosH59VMUwDKOS8PHH7FMcPpyRheHg668ZJnrLLcAFF4SnTD9CVv6qukVV//as7wPwL4BUAAMBvO857H0AgzzrAwF8oGQWgLoi0ixUOUpF//7AI4+wlj7rLPbQX3ABkJMTVTEMw6jgrF3L0bsnnAA89FD4yhw6lHnJIjjHb1jbEiLSGsDRAGYDaKKqWwBWEAAaew5LBbDR52dpnm3+ZV0nInNFZO6OHTvCKSZ54AF2Ao8YAQwbRv//zTczEZNhGEZJ5OcDl15K//748QwrD5XsbK+l/+mnzEsWIcKm/EWkJoAvANyuqhnFHRpgWyGNq6pjVLWrqnZt1KhRuMT0kpAAfPghR9+99BIjgd59F3jzzfCfyzCMysewYcCff1JntG4denmq7IecNw/44AOgTZvQyyyGsCh/EUkGFf94Vf3Ss3mb487xfG73bE8D0NLn5y0AbA6HHKWmbl1OrpCezo6Vfv2AW29lnK5hGEZRfPklXTI33wxceGF4ynztNaaiefhh4Oyzw1NmMYQj2kcAvAPgX1V93mfXtwCGeNaHAPjGZ/sVnqif4wHsddxDMeGII1jL/vknULs2cNBBjNPdHJv6yDCMcs6KFfTJd+vGQV3h4I8/gNtvpys6XH0HJRAOy78HgMsB9BGRBZ7lDACjAJwqIisBnOr5DgDfA1gDYBWAtwDcFAYZQuP885nx89NPmY9j3z5uy82NtWSGYZQn9u+ncVilCvDZZ+HxyaelUd+0bQuMGxe1DKAh91Co6nQE9uMDQN8AxyuAm0M9b9gZPhxYtoz+/zvvZI1+3XUM4SrH+TkMw4gSqozsWbIE+PFHoGXLkn9TErm5VPxZWcDUqVGd3ze+RvgWhwjw9ttA9+70vV1/Pd1BTz4Za8kMwygPvP46o3oefRQ49dTQy1MFrr2Wc4y8/z7QqVPoZZaCyq38s7LYRPv77+COT0kBvvoKaNSIo37POYchoc6oYMMw4pOpU4HbbqNbePjw8JQ5YgQjDh97DDj33PCUWQoqt/LfuZNhU337AnPmBPebJk2ASZPo2/v3X3bqXHEFo4EMw4g/Vq+ma+bgg5kdIBw++QkTONHU5ZfTwIwBlVv5t2oF/PYbc22cckrwIZyHH87h1U7K5+bNgYEDOSenYRjxQ0YGswAAwMSJ4fHJz5zJKR5POimmOf8rt/IHGLr5229A06bA6aczpCoYevdmk2zOHPbC5+ayybd7d2TlNQyjfOByMVPnypWc/Ongg0Mvc80aYNAgdhZ/9VVER/CWROVX/gAnc5k2jZ/9+tF/FwyDBzPr5y+/MGHT6tWMw7U5gA2j8nPvvcD333Nilt69Qy9v1y7qj/x8upYbNAi9zBCID+UP0HUzbRqHTJ9xBptwwXDrrXwIJk5kjT17NnNv5OVFVFzDMGLIW28x3PuWW4Abbgi9vP37qfjXrKHF36FD6GWGSPwof4CdudOm0ad/zjkM5QyGkSOBIUMY9XPuucAPP3A2sAhMsGAYRoyZOJEKv18/plUOlbw8dhjPmcOO3p49Qy8zDIQhDV0Fo2FDunHOOYcKfdcuJnUrDmcMQFYWR/UNGMD+gMaNI5py1TCMKDNrFnP1dOnCdz3UTJ1uNzt3J09ma+Kcc8IjZxiIP+UPALVqAd99B1x2GXDHHcCOHYy5La7XPSmJAzxyczkGoE8fNgvr1o1ZqJZhGGFk+XIadqmp1A81a4ZWnir1y/jxHCx6zTXhkTNMxJfbx5eqVTkDz/XX061zzTUl+/GTk+n6Of104NdfGar14IPA009HR2bDMCLDli108yQm0kpv3Ljk35TEE08AL77IhG333Vfy8VEmPi1/h8REDtlu0oSj7NavZ0hX3bpF/6ZqVaZzPfNM4PffOYPPvfeyYijJfWQYRvkjPZ1BIDt2sE+wXbvQyxw1itk5r7iCHoJymB8sfi1/BxHm6hg7luMBevQoeT7f6tXZKdSjB1NBH3ssm3cvvxwdmQ3DCA/79tHiX7qURl3XrqGX+eyznOjlkks4QVSUsnSWlvIpVSwYOpSZ+jZtYkqHv/4q/viaNRn107cvjz3ySIaFvh7d+egNwygj+/ezBT93Lt25p50WepkvvADcfTc7jd9/n96Fcoopf1/69KElX60aw7E++aT442vUYAvgzDOBhQuZle+mm/gAGIZRfsnJ4bidGTPYITtwYOhlvvIKXb/nncdowHDM6RtBTPn707Ejw72OPhq46CI231yuoo9PSWFz8bzz2HTs3JkPwOOP22TwhlEeOXCAcfc//0y3TDimYXz+eeB//2OFMmEC+wDLO6pa7pdjjjlGo05Ojuq116oCqv37q+7ZU/zxeXmql17K4w89lJ933aXqdkdHXsMwSiY7W/Xss/l+vvFG6OW53aoPP8zyLrhANTc39DLDCIC5WoRejbliD2aJifJ3eP111aQk1fbtVZcsKf5Yl0v1//6Pt7VdO35efz23G4YRW/bvVz31VL6Xr7wSenlut/d9v/JK1fz80MsMM8Upf3P7lMQNNzCmf+9e4LjjmM+7KBIS2Px79lkmgWvZEnjzTWYGzMmJnsyGYRQkI4NRPb/8QlfPzSHOJOtycRau0aMZ6PH22+W6czcQpvyD4aSTOBvY0UcDl17KgWHZ2UUff+edrCS2buUYAieSwNJBG0b02b2b0y7OnMnO3SuvDK28nBz2B77zDgd5vvBCuQ3nLI6KJ3GsSE1lC+Dee4ExYzjX78qVRR9/8cUMBc3K4gQQs2ZxQNjatdGT2TDina1bGcW3YAHwxRdU2qGwcyfDuz//nIO3HnusXA7gCoawKH8ReVdEtovIPz7b6ovIFBFZ6fms59kuIvKSiKwSkUUi0iUcMkSF5GSO3Js0Cdi4ETjmGGDcuKKjevr2Zeiok7d740bg+OMZV2wYRmRZtsxrpE2cGHo45+rVNODmzWNr/o47wiNnUaiygnnwwYgUHy7L/z0A/fy23QfgF1VtD+AXz3cA6A+gvWe5DkDFGxV15pnA/PnAEUdwDs6LLwb27Al8bOfOTOXaowdbATk5HEPw2WfRldkw4onp06mos7I4cj/UAVyzZtFw272b/QYXXBAeOYsiI4OTSd11FyuxSKSPL6onuLQLgNYA/vH5vhxAM896MwDLPetvArg40HFFLTGN9imOvDzVJ55gNFCLFqq//FL0sQcOqN54IyMD6tbl5/33WySQYYSbzz9XrVpV9ZBDVFevDr28CRNUU1IYwbdiRejllcTcuTxXYqLqM8+EFC6OaIR6BlD+6X7793g+JwE40Wf7LwC6BijvOgBzAcxt1apVmS8+Kvz1Fx80QPWOO1Szsoo+9rXX+KfWrMnjBwxQTU+PnqyGUVlxu1Wff15VRPWEE1R37gytvPx81bvv5nvao4fq9u3hkbMo3G7VF19UTU6mMfnHHyEXWd6U/3cBlP8xxZVdbi1/XzIzvZb9wQer/vZb0cf++Sf/3MRE1YQE1Q4dVJcti56shlHZyM5WHTKE79+55xZvgAXDrl2qp53G8m68MfKDt3btUh00iOc766zQKy4PxSn/SEb7bBORZgDg+dzu2Z4GoKXPcS0AbI6gHNGhRg3gtdfoD3S56Ne/+WZmDfTn+OMZfXDaafTlrVvHbILFjSEwDCMwmzbxfXv/feCRR9ifVq1a2ctbvJiZeqdN4+xbr70GVKkSLmkL8/PPTAz53XccN/DNN1GZ3D2Syv9bAEM860MAfOOz/QpP1M/xAPaq6pYIyhFd+vThw3PbbczwedhhwPffFz6uQQNGDT35JHON5OdzDME117CTyjCMkpk5k4bT0qWcGP3hh8sec6/KAWDdunEcz7RpkZ19KyuL+YBOPZXG48yZnPglWqGjRTUJSrMAmABgC4A80LK/GkAD0KWz0vNZ33OsAHgVwGoAixHA3++/VAi3TyBmzFDt2JFNuUGDVNetC3zcb7+ptmxJXyXA3/zzT3RlNYyKhNut+vLL9I+3axf6+7J3r+oll/D969NHdfPm8MhZFH/+6e0nvO220N1URYC4ze2Tm8tcHmPHMtomFuTmqo4cqVq9umq1alwP5D9MT1e94gr+JUlJjC545RWLBjIMf3bvVj3nHL4rZ5xBf3kozJ3LfrqEBNXHH49sjp7MTHYiJySotmpVfIRgGIhf5b92repRR/EyW7VSfeklJneKBevWeTt0OnRQnTQpcAjXZ58xFDQhwWuFrF8ffXkNozzy55+qBx1EA+m550IzjvLzGUpZpYpqaqrq77+HTcyATJ6s2ro13+trrolKlF/8Kn9VKtjvv1c98URebqNGqk8+GbvwykmTmCEUUD3lFNWFCwsfs2kTLRqAlUDNmqrvvmvpoY34JT9fddQoKv02bVRnzw6tvBUrGL4JqA4cqLpjR3jkDMS2bV6XUocOxUcChpn4Vv6+/P47c/MDqrVrs/kVC6s6N1f1hRdU69Wjn//qqwv7GN1u1Y8/Vm3QgPICqqefXnS/gWFUVpYvV+3ene/A+eeXPLdGcbhc9ABUq8YW9gcfRM6oOnCA56pbl62LRx7hPCFRxJS/P3//rTp4MK3qxESuz5wZ3nMEw+7dHBSWnMyH8e67C1sgu3YxVzjAiqJqVdWnnopdH4ZhRAuXi0ZStWo0lMaNC01RL1+u2quX/jdBU1pa+GT156efVDt18rbwly6N3LmKwZR/UaxbR4Vbpw5vRbduHModbcW6erXq5ZdTudesqfrgg4Wtm19/9bqLAK7HosIyjGiwYoXqSSfxWT/zTLpCy0p2tupDD9H6rlNH9a23Imftr1jhnSmsXTvVb76JqbvWlH9J7NvHyBpHuTZtqjp8ODuMo8mSJZwKzsn/89BDBUf65eWxGVm9urcSuOSS0F4MwyhPZGV5FXXt2ozUC0V5Tp7snVXvkktUt2wJm6gFSEtTve46b+qWUaOi7uIJhCn/YHG52CE7YABdQiKq/fqpfvUVFW+0+PtvdkIBqjVq0DXk20TdsYPRAk4FkJzMyiozM3oyGka4+eEHr6K++OLQYu1XrVI97zyWdcghqj//HD45fdm1i96DlBS+h7fcEt4K5sCBkIw7U/5lYf16WiCpqbxNzZtTwUYzB8/ixaqXXUZrokoVdgz7RgctXuydkxTwNmnL4VyihlEkq1YxH48TDROKot61S/X226mIa9Rg3H4kLPAdO6gPatemkXj55apr1oRebnY2E7qNGMHcQjVq0P1VRkz5h0JeHv12Z5zhjb3v1k311VfDlnypRNasYXKpatV4/p49Vb/4wtsamTVLtUsXbyXQpInq++/bADGjfLNzJ0e3JifTlfnEE2VX1NnZzOhZrx7f02uuicwo3U2bOGl79epU+uedRyOsrGzbRs/CXXcxoqlKFe97fNhhqjffzP1lxJR/uNi8WfXZZ1UPP1z/c7cMGsT84REanl2AXbtUn36ag1wApoQYOdJbCf34o+qhh3ofnsaNrRIwyh/Z2YxYq1PHq6jL6trIyWF/ndNCP/101UWLwiuvKtNHXHstlXNiIi390kbw5OezX++tt1SHDi0YwFGlCscd3HMPjc0wjTsw5R8JFiygL75JE/3PN3/RRbTII10R5OfTGujTx/vgDB5Mn2l+vuq0ad6RzQDHCjzzTLnogDLimKws5qtv3lz/S81Q1pw8OTmcG6NFC5Z14onhT5Xgcql+953XtZqSonr99cFNEON2s8X+ySe06nv29M7hAag2bMiooKeeUp0+nRViBDDlH0ny8lSnTGFPf8OG3orgwguZqiEjI7LnX7xY9X//U61fn+dOTVUdNowxzX/9RReV88ClpNDK2r07sjIZhi+ZmWwxO4ZSz54MXS4Le/ZQYTqW/gkn8P0LZzjl9u1MHeEkXmvenD744qzxLVtUv/2WYdr9+nl1gWOcHXccXTjvvcd+wyiFf5ryjxZ5eeys8q0IqlRhU/SVVyI7mjgnh+6nM8/09k10785BMrNns1WSnKz/pYzo3r3sL6BhBMPWraoPP8yUKoBq375slZaFtWvZP+BYz3370s0ZLiXqcnFg1uDB3vfk+ONVP/qo4LifvDy6eyZMUL3vPg4Wc1oyzrt12GEcmPn660waF+pEMBGaxlG4v3zTtWtXnTt3bqzFKB35+cCMGcDEicC33wIrV3L7kUcCAwZwIpfjj4/MJBFbtgDjxgEffsi5BQCge3dg0CBg2zbggw+AnTu5vV494IorgMcfB2rVCr8sRvyxYAHw4oucnOjAAeDMM4Fhw4AePUpXjssF/PQTMGYM3yMR4KKLgDvvBI46KjyyLl5MOSdMANavB+rX5/tw9dVAixbAokXAwoXe5Z9/gJwc/jYpCejUie/0UUdxApguXZibP1TWrOE8IN9/D9SuDXz8cb7bWloAACAASURBVJmKEZF5qto14D5T/lFi+XI+wBMnAtOncwavGjU4A9GppwKnnAJ07hz+iRyWLwc+/5yzGy1cyG3dugFHHAH8/Tcwfz5lEQEOOQS47jrgllsiO3ORUfnIyuJz9s47wO+/A9WrA1deCdx6K5+r0rB5MydVefttKuTGjYGrruLMeC1ahC7rqlV8Hz76iMo8MZETwnTqBFStSkPt338ph0PDhlTyvkvHjuF7TzIzgT/+AKZMocJfvpzb27cHLrgAGDGiTMWa8i9vpKdzlqCff+afvWIFtzdtykqgTx9aSe3bh7cyWLkS+OIL4Msvgb/+4rYmTYDmzYG0NGDHDm5LSODDffXVfOlCmRLPqLyoAnPnUuFPmABkZAAHH0wD4ppr2KoMlowMzsQ1fjynQnW7gb59WdagQaEpWbebBte4ccDkycDGjdxeqxZb6NnZ3mNr1qRS79iRlcERR/BdaNYsvO9iTg4waxbw66+83jlzKEvVqkCvXsAZZwD9+1MHhIAp//LOhg2sCJzFUcKNGrEScJZjjgmfpbFtG/Djj8APP/Bzzx5aQA0b0grZv5/HiQBt2gDnnssWwUEHhef8RsVlyRLgk0+4rFhB4+D882ksnHxy8EoyM5PP3iefsEWck8Nn7ZJLgCFDSq/4srPpLlm1iu6c6dP5uXUrKwBf6tfnFKu+ir5jRyA1NTLTKGZm0uCaOZOG3/TpvN6EBLqL+vRhZXfCCWE1tkz5VyTcbjb5ZszgAzJjBh9mAEhJ4YNy7LGsCI45hi9IWecsdcjPp+UxeTIfzNmz6asVAZKTue5QsybPO3gwLbLmzUM7t1H+UaV7cNIk4NNPqfwTEmihXnghlzp1gitr2zYq+q+/pqGTm0uD48ILqfS7dy9a+WZl0VBav9776ayvWlXQTeOQmEiF3qUL3atHHAEceijPGSlUWQn9+SeXmTPZd+BUQIcfTkXfpw8ry2DvXRkw5V/R2baNlcCMGXyQFizwdjrVrAkcfbS3MujShRVCcnLZz5edzSbpb7+xMpg1iy9pIGrXZmV0+ul8uTp3Du3cRvkgI4MuiUmT6IPesoVK+cQTqajPO49uypLIyaERM2UKO28XLOD21q1pPAwcyDJzcqi8N2/muTZvpivSV8k7QQoOIt7O1f37qXSTkuimOe00BlYce2xkn0dH0S9YwApy/ny6wrZv5/6aNRnY0b07l27d2OqIEqb8Kxt5eeyQmjfPuyxc6PVdJiezk61TJypjZzn44LK9CLm5tFzmzGGrYPp0YO3awMeK0D/avTvQrx8rpkMPDU8EhBE59u2jcTF1Kiv8efMYbVO7Niv2AQP4fzZuXHw5u3fTWPjzT5b3559U7ElJfP5at2YZBw7QHeMo/MzMwmVVrw60agU0aMBWr8vF8tetY+UE8Lnq2pXukgi4TQqQkQEsWwYsXepV9AsXemVJTKTr6JhjvMq+c2dujxGm/OOB/Hw+mAsWsFnuLGvX0joBqPjbteNL2K5dwfXWrUvXn7B3r7fi+e03fm7fXti36pCSwkiNo46iu+Dww3nO1NSYvhxxyYED9IX/9Zd3WbKE/11yMq3T3r29gQfJyVTg6elc9uzhZ1oao2VWrKD1W5QS9yclhe7C5s1pKDRvzs7hhAQaNvv2sazly2nkOG7HhAQq027dgOOO42enTqxYwoXbzXOvWMFzO8uyZQXdStWrs4Vx9NFcjjqKfQgpKeGTJQyUS+UvIv0AvAggEcDbqjqqqGPLjfJXpfVR2iVciFBRJiTw03c90LbERL60K1fSWlm6lA/16tX0kWZlectOSKCV1aoV0LIlFXXLlt6lRQv6SYvrDHO5WPaUKQxbmz8f2LSJ5ynuOUtJ4cvfogUro7ZtvRVTaiojkqzlUHqcFqITn75sGf//VatoLABUYqmpDC6oX5//RWZmQUW/Z0/Bfp9AiNDF0bAhgwIOOYRKvWFDr2J3uShTerrXrbNmDZ8Zf5dOy5Y0EA47jMvhh7MFGapyVQV27aI7ae1aLmvWeD/Xry/o4qxVy9sp3LEjZejYkc9nBTBayp3yF5FEACsAnAogDcBfAC5W1aWBji+z8s/IAB59lH9mbi4fYGe9qG3+3/PyvEq8ArSSisW/ohDh4oxPdLu5FFVpidDKSk5mK6FKFYampaTws1o1rvsuyck81/79bObv3Mmme2Ym73VRLYVAJCZ6z12tGpdatbjUqcOlVi26KmrXpjKqXp1yVK/uXa9alWUkJ3PdKdP3XgAFP1Upb35+wc+8PO9nXh63B3qWnH3Ovc3P57Xn53vvvXMeX8PBOY/vuZyyDxygqy8ri5V8To73uNIaHQkJvBdJSVx35MjN9VYUDnXqsLJo2pSWe7NmvIf79nHZs4cKdvdufqanFz5f1aqsHNq2LdwSbduW/11pyM3ls7VzJ6PlduxgBbNpU8Fl8+bCFVm9eowyatuWn23asPLq2DH8IZ5RpjjlH8b2Uqk4DsAqVV0DACLyMYCBAAIq/zKTlwe8+SYfNOeFd9Z9v9esWfQxjvIKZQnXw+OvnJ31QNtK2l/Ub3yVzv79fJmd0M+sLL5kjpLZu5efxVWKIlQmTkXjey+ce+Mov5JwZMvJ8fpZjfDgdheMdy+OvXu5OFFo/jgt1ORkr4unZk1WGvXre10+DRoAdetyn1NRZ2XRBZWVxWcvI8O7ON/T01mpOIp+586i3U1OyyY1lf0BznqLFl5lX7du2e5ZBSdWyj8VwEaf72kAuvkeICLXAbgOAFq1alW2szRoEJwP0ig7qqwYtm/3Lo5ySE8vuJ6eTmtwzx7+JjvbW5EYscG3MnbW/Vs+ZcHt9rZOMjIYsebNgFO2MhMSqMwd91KzZkCHDmyFNGzoXZzvzZqxwqnAlnskiZXyD/RvFHgiVHUMgDEA3T7REMoIElW+0I7V5f/p6yv2dQFkZobPdea0KAK1rnxbL/447qjq1dmPUKOG1yXk77Zy3B+OS8ZpIeXne902zrkct47/p++xvmWoFizD1yXk60IKRVEmJXndfI4Lx7+85GSv68y5F859cLsph+O2clxKvvfVV6H7ug79W5j+9yovr+RrcGT3vVeZmVy2bmU/Ru3aVPItW9Jt1KED5W/UiNdiir9IYqX80wC09PneAkCAERpGVFCldb51K2Ost28vWrE7n4EUK8AX1om+KKmvxBmcFmj0ZWoqxyt07szmeevWfMFdLnZcz53LZdEiyuxQrRojQDp0KOzHbdmy4o1BcKJfHLfHvn2F3R5OuOS2baxk09PZsiqqRZWUROu5dm0qSID//7p1BY/p1InjRpyIliOP5G/CgcvlvRZn8W0hBlocI2LvXm9cv+MSWr6cg8b8SUmhW6dxYz5Tbduy07ZNG1YQjRoxoMC5D3FErDp8k8AO374ANoEdvpeo6pJAx5ebaJ+KRl4eFfmWLV7FHuhz61bvoDF/6tf3NqXr1PFakdnZtOo3b+ZLGchnn5jotSgTErwvO+BNJNepk3fp2JFK23kRc3I4ruD33/k5dy4VnFP2YYdRIXXu7C2jdevQRzxXBtxu/vebNlGpL17MStMZCRsogkeEyr1+fSrM/HxG5OzZ4z2mc2eOSj3pJC7hSLRWFtxuVgJOJ++uXbxeJ4InLY3P9a5dfO6KGqToUKMGO7CbNmVl0KRJwXXfpQJFnpW7aB8AEJEzALwAhnq+q6pFpq0z5R+ArCwmqEpL4+KsO5+bNxcOn3OoX59NZedhd9adz0aNqNw3bGDT2kln6zuwy98/XKsWFUO7dvy+fj1j/51OxNatvakpjjuOFqV/CumsLA4K+u03LrNn86UVYcVw7LEc0NO1K5W+JZwLjZwchuROncqBWcuXB86DI0KlV78+K4xNmwr+ryedxDEB/foFN+o3FrhcrATnz+dYmMWLGfq6bp23EhThM5WUxIrPNxTal5o1C1YGzZt7O5JTU73fw9VKCoFyqfxLQ9wp/5wcKt6NG4tW8L7WmEOjRrTEWrTwDqDxVeqOJVO1asHfpaVR0TrL/Pm0lgC+EE58/86d3vTPxxzDgUAdO9ICmzWLSbocy/6IIziYq1cvDhQqamTomjXAd99xmTqVL2JCAiuHnj25nHhi6TJEGmUnL48V/syZzPU0fbr3P3WUokODBlSWTgc+wMFO/fuzIujRo/zHwjtjUxwDZ9EiVg5O5s/ERLof27b1diDn57OVsW2bd5Ty3r2Fy65Zs2Bl4L+emsp3MoKuSFP+5Y0DB/hwrV1Ly8P/09eH7eAodt8BWI6ib9mSD1IwA2Cys5mmYdYsr7J3Ri5WqcKXt1kz71B2R5bDD+cL3bs3z/fjj8zfPns2rf+mTb1paHv3pmIIhMtFa3PSJCr8Zcu4/ZBDOOnHKadQaUQw2ZVRClQ5MOzXX73ph/fsoQHQoAH/T8cQSUqiSyQjg79r2BA45xxm/Ozdu2L1t2zbVtAgmjPHaxDVq8fWjmPcHHGENzeR73gC//EFmzcX7ugWoWHkDKZ0Bln6Ls2albkSNeUfbdxu/uGrVwdW7ps2FWxaJybyT2/dmh1RrVtzadWKyj1YxR4IJ2+4k7Nl1ixvM7ddOw6RP/ZYtgYWLmS2xW3baNGdeioVcv/+fEg//piTYMyZw9936cKX+8wz6YYpytfucjHPyyefcD6BbdtY0fTsyd+eeSYH9xjlH7eb7jxnhjpngiDHCNm3j30LTgtRhOu1ajEZ3NChVJwVrV/G5aKhMnu21zXpzM5Xty77QXwrg0DK2u1m69m/knBa887iH57epQvveRkw5R8JHAW/ciU70Xw/V68u2IEqwpfDUez+n6mp4ctPkpfHh/PXXwtm5ExIYMRGr15Uuscfz6bqe+8B77/PB7BaNSriwYNpxScmAt98A4wdy5QNbjcfxMGDac05/v2iWLSI5X/8MVsQvuX371/6UZxG+WPDBlYCn33GjnmAhkDXrmwFzJxJxeWrZxo25FSJN95YsSv9TZu8mW+nTfNWBvXqMato//5MileafhBVutl8K4Pq1Xm/yoAp/7Liq+D9lby/gq9alcqwfXs+0O3b87sTYhjJaRHT0jgpy+TJDHfLyCio7Hv1ot+8bl26fT79lNPk/f47j+vXjw/XgAF8YZcuBV57jbMqpadT/iFDuJT0su7ezenxxo7lNJHJyVT4F17I8k3hV17S0lgJTJjAZHGJifzvL7iA+3/4ge4+39HZ7dsDd93FKR8rklsoEJs2eWfomzyZ/QEA38N+/WhQde8e1X6Q4pR/wFndy9tyzDHHlGK++jKQk6O6eLHqp5+qPvqo6sUXqx51lGq1agWHsFStqtqpk+rAgap33qn6xhuqP/+sun69qssVWRl9cblUZ85Uvece1cMO88rXooXqtdeqfvml6p49BX+zcaPq/ferNmjAYw8+WPXJJ1XT0rg/P1/1q69U+/TxXuvFF6v+9BP3FYfbrfrnn6qXXaZapQp/f/TRqi+9pLpjR2TugVG+WbaMz2eTJnwemjVTHT5cdcMG1XnzVP/v/7z7ANXkZNVTT1WdNSvWkocHl0t1/ny+YyefrJqYyOts1Ej16qtVJ05Uzc6OuBgA5moRejXmij2YJWzKPz2dSurdd/lgnnUWlWBCQkElf9BBqv368QF9/fXYKHh/cnNVJ09Wvf561aZNKWdSkmrv3qpPP83Ky+0u/LvZs1UvvJDHiqgOGqT666/eYzMzVZ9/XrV1a28F8uSTqtu3lyxTVhbv5THH8Le1aqnecovqggXhvXaj4nLggOrXX6sOGMDnLymJRoWj5JcsodFQq5b3/WvSRPWJJ/i+VhbS01U/+YTXXrs2r7NGDdXzz1cdP141IyMip41f5b9jh+pNN1FBNmtWUMEnJ6t27syb/8AD/AP+/lt1//6ynSsS5Oaqfvut6qWXqtapQ7mrV1c97zzVceMKW/e+/Pab6imn8Dd16qjecYfqmjXe/bt3qz72mLclcPLJqp9/rpqXV7JcO3eyhdSwIX/bqZPqa69F7AE2KgmrV9OgcpRft25spTpG1S+/cJsI9yckqJ52Go2eklqfFQnHkLvhBq8hV62a6gUX8H6EsUUQv8o/M5OK77jjVIcMUR01SvWbb1SXLw9OycWC/HzVqVPpvqlXj39R/fqqV17JiiArq+jfut1005x0kteCeuaZgkp5xw62emrW5DFnnUUXUjBs2KB6++20WADVM8/kCxuoxWEYRZGRofryy6rt2vE56txZ9aOPvAp+61Y+71Wreo21Ro1orGzZElvZw43LpfrHH6o338xrBFg5DhnCCiJEPRW/yl+14iimhQtpnTdvrv81CS+9VHXSJFoKJTFrFq13QDU1lf5234oiI4PWeq1atKguuojnDIZ16+inTEqi7/Lyy1UXLSrbdRqGQ14eW9ydOvG5bd9edexYr8LLyKDB5usSctxGM2ZUnHc7WPLyVH/8UXXoUG/rqGlT1WHDylxkfCv/8syePXSXOD7z5GTVs89W/fhjtlqCYcUKuq4A1caNVV99lR3YDjk5qi+84LUqzj1XdenS4MpOS6PbLDmZHbm33MKKwDDCicul+sUXDBIAVDt2ZPCBo9wzMtgH4LRWk5P5edRRqm+/HZWO06iTnc17MmgQDbUyYsq/POFysQP5kktUU1L4FxxxhOqLL5YuMmbXLirjpCS2Eh5+uKB7x+3mC9SmDc/Rpw87f4Nh5076ZqtWZfnXX0+Xj2FEErebCu+QQ/jMdu/OviuH3bsZMVStGlugjkHTtKnqyJGVq4PYlxBaOKb8ywN79tACb99e/+uEvekm1blzS/fnulyMsGnYkO6bG24o7AddupRhc44/9aefgis7J0f1uedU69Zl2UOHFuwkNoxokJenOmaM1wU6cGDB53DjRroeAT6rjtuodm32Z23eHDvZyxmm/GPJokW0nKtX5+0+/njVDz4ovuO2KObPVz3hBJZzwgmFQyr37mW/QVISX4qXXgquw8jtZqSP0wF3+ukMHTWMWLJ/v+qIEXx3qlZVfeihgtF4s2bxfXIizk47jUZLSgpbrlu3xk72coIp/2jjctHl0rMnb3FKCqMX5s4tW3n791OpJySwqTt2bOExBxMnMkZfhJFCwcTpq7KV0KuXt5UweXLZZDSMSLFxI/3egGqrVnQNOa1lt1v1ww/5XiQm8tm/7DK+K9Wqqd59d1wPNDTlHy2ys1XffNPrs2zdmgOwdu4se5nTp3tdRddfT7+nL9u2eV+Mww4LfoTk/v2MIkhOZkjp66+X3/BXw1BVnTZN9fDD9T9X0KZN3n27djEiDWA/1zvvMFpOhB3FDz+sum9fzESPFab8I82uXaqPP85oG0C1a1eO5gtFmTrWvghHHP/yS8H9jsXToAEV+KOPBhcSqspWwkEHUdahQ4NvJRhGrMnLU332Wbam69RRfeutgn1m06apdujAZ/uqq1TnzOGgSKdjeMyYuDJyTPlHii1bqKCdQU/9+3OAVqjxx3/95W093Hhj4ZGzu3Z5wzu7d+cQ+WDYscPbSujcWfX330OT0zBixcqVXndl794cPeyQk8NWbUICWwG//86BjE5/WadOqt99FzvZo4gp/3CzebPqbbfR+nAGPYWjg9TlYrRNcjL99/7WvirDRFNTecyoUcEPe//iC7ZMkpPZSgm2lWAY5RWXi5Z87do0wN59t6DhNX26atu2bD3fcw/dsp9/znxeAPMN+VYalRBT/uFi0ybVW2/1Kv0rr6QFEg62b1c94wz+JYMG0br3JSeHmUQBNmvnzQuuXF9rv0sXG5lrVD42bPC2As4/v+C7s2+f6nXX6X/jaZYvp+HzzDPsC3CiiMoSfVcBMOUfKtu309KvWpVK/6qrVFetCl/506YxprlqVdVXXinsNlq71jsK+MYbg08+9+OPzO+TnMwRkgcOhE9mwyhP5OerPvUUn/XU1MKt5kmT2D9Ws6bqhAnclpbGVBFOcMakSdGXO8KY8i8rmZl0kTj5cK66KrzNRLdbdfRoVigdOjCO35/vvmM0Tp06TI0bDLm53lZC587B5/AxjIrOvHl8l0RUH3ywoFt0wwav3/+GG7xpIaZO9Q4Uu+giRtBVEiKm/AFcAGAJADeArn77hgFYBWA5gNN9tvfzbFsF4L5gzhN15X/gAHPuOJNNnHOO6r//hvccWVmMR3bK9+/Uzc9nqmknh0mwLY3ly+necVoJlbQ5axhFsn8/DTWAI919o9kOHGDsvzPhkGPM5eYyYi45mVl0P/igUiSOi6Ty7wigA4BpvsofQCcACwFUBdAGwGoAiZ5lNYC2AKp4julU0nmipvydka5OXP1JJwWf7rg0rF9PBS3CloX/gK2dO725+K+6KngF/uGH7PiqXz/4VoJhVFbefpuu1BYtOImTLxMnskVdv35BF9GSJYygAzih08aN0ZU5zETc7RNA+Q8DMMzn+48AunuWH4s6rqglKsp/4cKCI10nToxMzT99Okcj1q7Nc/jz779Ms1C1Kh/eYMjNZT5wZ1IWZ2pGw4h3/v6bET/JyYX701atorsnMbHgvvx8pkapXp1pUj76KDayh4FYKP9XAFzm8/0dAOd7lrd9tl8O4JUiyrwOwFwAc1u1ahW5u7NjB90jCQnsEHr99cjNGvTpp1TqhxzCOU79+ekn+vYbNw6+xbFxoze/yZ13WqeuYfizezcnLXJGyfu+I3v3MuQTYFSQbwj0ypXed+uiiwpH4FUAQlL+AH4G8E+AZaAWrfxfDaD8z/P0Efgr/5dLkiEiln9eHmv3evVY8996a+HUCeHC7eaoREC1R4/A6R5eeYVyHH548Dnzp05lRVGzpupnn4VVZMOoVLhcHPjlpDf3VeT5+d59J59ccF9eHiPlkpIYkTdlSvRlDwFz+/gzfTrz4AD0rf/zT3jL9yU/n3n3Ac7R6T/xRH6+6v/+p/8NOgl2HtzXXmNl0bFj8JOzGEa88/77nJioffvCre/x47nv0EMLG2Dz5vFdE+GcAhUkRUQslH9nvw7fNZ7O3iTPehufDt/OJZUfNuW/e7d3wEerVuwUjWSPfnY2B2wBqnfdVbhjNyeHFQLAFLTBuJvy8spWWRiGQZx+t7p1C48HmDaN25s1Kxx6vX+/N3nciSdWiM7gSEb7nAMgDUAugG1+Vv1wT2TPcgD9fbafAWCFZ9/wYM4TsvJ3u9lp07gxreW77gp+msSysm+fat++vMUvvVR4/969bH4CHG0YDOnpzLXv+Pcj1TdhGJWdtWvZ2VulCpMw+vLPP4wQqlUrsJtn3DhG1TVoUO4HhkXc8o/0EpLyX7WKkzwAqscdF3ggVbjZs4eDSRIS2Mz0Z+tWhnomJTGeOBhWr+bDmpTETIaGYYTG7t3sgxNhn5svGzey/y0pie4gf5YvVz3ySOqV4cPLrSEWv8p/2TLm4alVi39uNP6g7ds5eCQ5mWMG/FmzhqGc1aurfv99cGXOm8dWS716qr/+Gl55DSOe2b/fGwn0wAMF3cDp6Qz/FmECOX+ys71uoP79IxcwEgLxq/zdbma+9J30IZJs2sROoZQU1R9+KLx/5UrVli2pxP0HnRTFlCmM5mnVKvyjjA3DYD+ao8SvuaagkZiVRcUOqL74YuHfut2qb7xBY69du3I3/Wn8Kv9osmUL4/dr1mSnkT/LljFUrEGDwnPvFsWECXyoDjvMBm4ZRiRxu1Xvv58q8ZJLCkbz5OQwBQugOnJk4N/PmMHJYmrUKFdh16b8I8327RwVXKOG6h9/FN6/ZAkfjEaNgk+p/MIL+l+KiT17wiuvYRiBefJJvneDBxccDHbggDcD6IMPBo4S3LTJmxri8cfLRW4gU/6RZNcuJl5LSQnsj1+8mEq/SZPgZtxyu1UfeUT/S/hmidkMI7o4AzLPOafgiN/8fK976IEHAv82J8ebsPHyy/k9hpjyjxTp6Zyvt0oV5s73Z+lS1YYN6e4JlM7BH7db9b77+LcMGVJuIwgMo9Lz4ot8D886q6ACd7nYLwCojhgR+Ldut+pjj3lb7oFG9EcJU/6RIDOTTbzk5MAJ2tasodJv0kR1xYqSy3O7VW+/Xf/LP+I/IMwwjOjy2mt8H888s3ALwLHun3uu6N9PmMBcXgcfzNDQGGDKP9wcOMAIgIQEzo3rT1oaJ46uXz+43n+Xi5NLAJwxrBz4Cg3DUEbyOKlZfDuB8/K8o/NffbXo38+cSbdvw4aqc+ZEXl4/TPmHE7ebLhkg8GCr7dsZ7lmzZnB/tsvl9SPee68pfsMobzz3nP7nivVtkR84oHr22dz3zjtF/37FCk4TWaNGYPdwBDHlH04cn/xjjxXel57OAV4pKYHDPf1xu5lOurgIAsMwYs+jj/I9vfnmgu9pTg5TriQkFD+B0ubNnEC+qBHDEcKUf7hwOoFuuKGwos7JUe3Zk30AwYzc9fXxm8VvGOUbt9s7/eM99xR8XzMzVbt1o9E3fXrRZaSnM2U0wFDuKGDKPxx88gmHeQ8aVDgKx+XiZA9AcLW6b1SP+fgNo2Lg21J/+umC+3bs4CDPevWKTxGfne0dMBbIexBmTPmHyqxZ7LXv0SNw3L1jEYwaFVx5ThMyUAvCMIzyi8vFAWCBDL21azmYs0WL4tM95+VxDADAUcUR1AGm/ENhwwaGa7Zpw9rdn5df5m286abg/sTnn+fxQ4daOKdhVESys+m+SU4uPB/AggWcn7tz5+ITvfmOF7jjjohVAKb8y8q+fUzbWrt24NG5X35JV9DAgcENyBo3jrf8/PNtAJdhVGR276aCr11bdeHCgvumTuXAzz59ip9T2+XyzvJ3000RMQZN+ZcFl4v+/YSEwB24c+awg6dbN6aFLYnJk9nT37t3zId8G4YRBjZsUE1N5bJhQ8F9H3yg/00KX5xV79uRfO21Ya8ATPmXBWdC50C98ps2cfRu69aM6y+J2bMZ43vUUezxNwyjcrBoEa3/ww7j7Hy+ODpk9Ojiy3C7KGc5cQAADn9JREFUOSFMadzHQWLKv7R89JH+l2bB/4/IyuKMYDVqFG7uBWLZMo7ua9OGaZ8Nw6hcTJnC6WEHDCjoznW5VM89l96D774rvgzfFsDtt4etAjDlXxoWL+YsWyeeWNhf53arXnopb9tXX5Vc1tatbB00asSJXAzDqJy88or+N2bHl8xMTtlaq1bJqV7cboZ+h3Hsjyn/YElPZ6xu06YckefPqFH6X67uksjKYn9AtWqqf/0VflkNwyg/uN3e/Fz+83Knpak2a0ZDMFDEoH85zliChx4KWSxT/sHgdnPwRWKi6u+/F94/cSIjey68sOQa2eVi0ieR4FoIhmFUfA4cYEBHlSqFp2mdM4fbTz215Eg/33xfwY4dKoKIKX8AzwBYBmARgK8A1PXZNwzAKgDLAZzus72fZ9sqAPcFc56oKP+nnuLteP75wvtWrlStU4fNt2Aie5zp4J55JvxyGoZRftm5k3P5NmlSOALorbf0v4FdJeFycTpJQPX118ssTiSV/2kAkjzrTwF4yrPeCcBCAFUBtAGwGkCiZ1kNoC2AKp5jOpV0nogr/19+YafM4MGBO3iPPJLDtteuLbmssWP1v7AtG71rGPHH0qX08R93XOGwbmdgVzAegQMHOJdAr15lHhcUFbcPgHMAjFev1T/MZ9+PALp7lh99thc4rqglosp/yxbVxo1VDz1UNSOj8P4rr+RtKqm3XpWZPJOTVfv2LX5wh2EYlZsvv6TeuPHGgtuzszn7X61awc3ul5UVnLehCIpT/gkIH1cB+MGzngpgo8++NM+2orYXQkSuE5G5IjJ3x44dYRTTB7cbGDIEyMgAPv8cqFWr4P533gHGjgUeeAA444ziy1q/Hjj/fKBtW5aVnBwZmQ3DKP+ccw5wzz3A668DH37o3Z6SAnzxBVC1KnDuuUBmZvHlVKsGVK8eERFLVP4i8rOI/BNgGehzzHAA+QDGO5sCFKXFbC+8UXWMqnZV1a6NGjUq+UrKwvPPAz/9BLzwAtC5c8F98+cDN98MnHIK8MgjxZeTnc0/8sAB4JtvgLp1IyOvYRgVhxEjgF69gOuvBxYt8m5v1Qr4+GNg2TLg2msBDagCI05SSQeo6inF7ReRIQAGAOjraWYAtOhb+hzWAsBmz3pR26PLX38Bw4YB550HXHddwX3p6bTiGzUCPvoISEwsuhxV/n7+fODbb4EOHSIrt2EYFYOkJCr5Ll1oHM6d6zUM+/YFHnuMXoVTTgGuvjr68hXlDwpmASN3lgJo5Le9Mwp2+K4BO3uTPOtt4O3w7VzSecLu89+7lz3yLVsWzrzndjNMMymJ82+WxOjRGq3c3IZhVECmT6c+OfvsgkEg+fnsH6xWrfg5AEIAEYz2WQX68Bd4ljd89g0HI3uWA+jvs/0MACs8+4YHc56wK//LLmN0zx9/FN739tu8LSNHllzOr79yXMCgQZae2TCMonGMRP9cYZs3MwNA586B5woJkYgp/2gtYVX+H37Iy3700cL7/v2XqR369ClZmW/YwJw9HTsGjhIyDMNwcLtVzzqLA73+/rvgvsmT9b9cYmGmOOUfzmif8k9aGnDLLUCPHsDw4QX35eYCF1/M3vUPPwQSirk1eXnAhRfyN19/XThKyDAMwxcR4N13gYYNgYsuKhjlc/rpjAx6803gs8+iJlL8KH9Vdqrk5QHvv1+4E3fYMGDBAv5BzZsXX9awYcCffwJvvw0cckjkZDYMo/LQsCEwbhywciVw660F9z3xBNCtG3DNNcC6dVERJ36U/5tvMqzz2WeBdu0K7ps8GRg9GrjpJuDss4sv55tvgOeeYxjo4MGRk9cwjMpH7970OowdC0yY4N2enMzvqsDQoRyDFGFEYxRjWhq6du2qc+fOLXsBq1cDRx4JnHAC8OOPbII57NwJHHYYwzrnzKHbpyjWrmXYVrt2wIwZHKhhGIZRGvLzgZ49gcWL6W1o29a7b+xY4KqraKTeeWfIpxKRearaNdC+ym/5u1ysSZOSOGLXV/GrAjfeCOzeDYwfX7ziz82lpa9Kv5wpfsMwykJSEscPJSQAl11GHeUwdCgwcCBw//3AP/9EVIzKr/xHjwamTwdeeglo2bLgvk8+YSqGRx4Bjjii+HLuuYeDNN57D2jTJlLSGoYRDxx0EPDqq+w7fOYZ73YRYMwYDga77DJmDYgURYUBlaelzKGeK1aoVq3KOHz/DJubN6vWr8/Me3l5xZfz3XcMxbrttrLJYRiG4Y/brXr++UwG6T8l7DffUOcMGxbSKVBMqGfl9vm7XMDLLzOEs0kT73ZVduz+/DPTMhx6aNFlbNvGVkHTpsDs2UzMZBiGEQ6cPsemTdnnWKWKd9/VV9PT8Mcf7K8sA/Hr809MBG6/vaDiB3hDJ00CRo4sXvGrAldeyayfH31kit8wjPDSsCHw1lvAwoXAo48W3Dd6NJPAXXttRKJ/SkzsVunYsAG47Tb2tvvH2vrz8svADz8Ar7xSOOunYRhGODjrLEb4jBoFDBgAdO/O7bVrs1+yevXiB52Wkcrt9vFHFejfnx3AixYVDLHyZ/Fi4NhjgVNPZbZOCZSN2jAMIwxkZNC9XKUKwz/DlMM/ft0+/nz0EeP8n3yyeMWfnc1+grp1C4eHGoZhhJvatRnjv3Il8NBDUTll/Cj/nTvp/+/WjaNzi+OBB4AlS9g30LhxVMQzDCPO6d2bc4OMHs35RiJM/Cj/O+7gJC1vvVX85Cx//MGbf8MNQL9+0ZPPMAzj6acZ+XPNNcxDFkHiQ/n/9BMzdd53H3D44UUft38/o3taty448MIwDCMa1KnDeX8XLWJFEEEqv/Lfv59zaHboUDiNsz/33cc8QGPHAjVrRkc+wzAMX84+m6lkHnuM8/xGiMqv/B9+mClSx4wpPk7/118Z0umEgRqGYcSKl14CatSg+ydCGT4rt/Jfvpz+++uvB04+uejjMjIYZ9u+PSOBDMMwYkmTJtRdM2bQDRQBKrfyP+QQ4OOPOXiiOO6+G9i4kdE9YYqvNQzDCIkrrgBOO41uaBvhW0pEgAsuKP6YqVPpErrrrjLnzzAMwwg7IgxUqV07IiN8QypRRB4XkUUiskBEfhKR5p7tIiIvicgqz/4uPr8ZIiIrPcuQUC8gJLKzGVfbrl3hvBqGYRixpnHjiOUUC7U6eUZVj1DVowBMAuAMTesPoL1nuQ7A6wAgIvUBPAygG4DjADwsIvVClKHsPPYYsGoVLX9z9xiGEUeEpPxVNcPnaw0ATqKggQA+8KSUngWgrog0A3A6gCmqultV9wCYAiA2I6kWLGAs/1VXAX36xEQEwzCMWBGyz19ERgC4AsBeAL09m1MBbPQ5LM2zrajt0SU/nyFUDRrYYC7DMOKSEi1/EflZRP4JsAwEAFUdrqotAYwHcIvzswBFaTHbA533OhGZKyJzd+zYEdzVBMtLLwHz5jFlc/364S3bMAyjAlCi5a+qpwRZ1kcAvgN9+mkAfCfMbQFgs2d7L7/t04o47xgAYwCmdA5ShpJZuxZ48EHm0C4pEsgwDKOSEmq0T3ufr2cDcMYifwvgCk/Uz/EA9qrqFgA/AjhNROp5OnpP82yLDqrM6JmQwMmTLVWzYRhxSqg+/1Ei0gGAG8B6ADd4tn8P4AwAqwBkAbgSAFR1t4g8DsDJV/qYqu4OUYbg+fprzsz1/PNAy5YlH28YhlFJiZ+ZvPbvBzp2BOrVo78/qXKPbzMMwyhuJq/40YCPP84UDhMmmOI3DCPuqdy5fRyWLgWeew4YOhTo0SPW0hiGYcScyq/8nU7eWrUiPjmCYRhGRaHy+z8mTACmTWNa1EaNYi2NYRhGuaByW/5793Lu3mOPBa69NtbSGIZhlBsqt/LPzga6dwdee634SdsNwzDijMrt9mnaFPjqq1hLYRiGUe6o3Ja/YRiGERBT/oZhGHGIKX/DMIw4xJS/YRhGHGLK3zAMIw4x5W8YhhGHmPI3DMOIQ0z5G4ZhxCEVIp+/iOwAJ4upaDQEsDPWQkQZu+b4wK65YnCQqgZMalYhlH9FRUTmFjWRQmXFrjk+sGuu+JjbxzAMIw4x5W8YhhGHmPKPLGNiLUAMsGuOD+yaKzjm8zcMw4hDzPI3DMOIQ0z5G4ZhxCGm/KOEiNwlIioiDWMtS6QRkWdEZJmILBKRr0SkbqxligQi0k9ElovIKhG5L9byRBoRaSkiU0XkXxFZIiK3xVqmaCEiiSIyX0QmxVqWcGHKPwqISEsApwLYEGtZosQUAIep6hEAVgAYFmN5wo6IJAJ4FUB/AJ0AXCwinWIrVcTJB3CnqnYEcDyAm+Pgmh1uA/BvrIUIJ6b8o8NoAPcAiIvedVX9SVXzPV9nAWgRS3kixHEAVqnqGlU9AOBjAANjLFNEUdUtqvq3Z30fqAxTYytV5BGRFgDOBPB2rGUJJ6b8I4yInA1gk6oujLUsMeIqAD/EWogIkApgo8/3NMSBInQQkdYAjgYwO7aSRIUXQOPNHWtBwknlnsA9SojIzwCaBtg1HMD9AE6LrkSRp7hrVtVvPMcMB10F46MpW5SQANviomUnIjUBfAHgdlXNiLU8kUREBgDYrqrzRKRXrOUJJ6b8w4CqnhJou4gcDqANgIUiAtD98beIHKeqW6MoYtgp6podRGQIgAEA+mrlHEySBqClz/cWADbHSJaoISLJoOIfr6pfxlqeKNADwNkicgaAFAC1RWScql4WY7lCxgZ5RRERWQegq6pWtMyApUJE+gF4HkBPVd0Ra3kigYgkgZ3ZfQFsAvAXgEtUdUlMBYsgQgvmfQC7VfX2WMsTbTyW/12qOiDWsoQD8/kbkeAVALUATBGRBSLyRqwFCjeeDu1bAPwIdnx+WpkVv4ceAC4H0Mfzvy7wWMRGBcQsf8MwjDjELH/DMIw4xJS/YRhGHGLK3zAMIw4x5W8YhhGHmPI3DMOIQ0z5G4ZhxCGm/A3DMOKQ/wdIm30lnmWyIAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "## EDIT THIS CELL\n",
    "\n",
    "# samples from the prior\n",
    "num_samples = 10\n",
    "\n",
    "# We first need to generate random weights theta_i, which we sample from the parameter prior\n",
    "random_weights = np.random.normal(size=(pol_deg+1,num_samples), scale=np.sqrt(prior_var))\n",
    "\n",
    "# Now, we compute the induced random functions, evaluated at the test input locations\n",
    "# Every function sample is given as f_i = Phi * theta_i, \n",
    "# where theta_i is a sample from the parameter prior\n",
    "\n",
    "sample_function = Phi_test @ random_weights # <-- SOLUTION\n",
    "\n",
    "plt.figure()\n",
    "plt.plot(Xtest, sample_function, color=\"r\")\n",
    "plt.title(\"Plausible functions under the prior\")\n",
    "print(\"Every sampled function is a polynomial of degree \"+str(pol_deg));"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we are given some training inputs $\\boldsymbol x_1, \\dotsc, \\boldsymbol x_N$, which we collect in a matrix $\\boldsymbol X = [\\boldsymbol x_1, \\dotsc, \\boldsymbol x_N]^\\top\\in\\mathbb{R}^{N\\times D}$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [],
   "source": [
    "N = 10\n",
    "X = np.random.uniform(high=5, low=-5, size=(N,1)) # training inputs, size Nx1\n",
    "y = g(X, np.sqrt(noise_var)) # training targets, size Nx1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, let us compute the posterior "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [],
   "source": [
    "## EDIT THIS FUNCTION\n",
    "\n",
    "def polyfit(X, y, K, prior_var, noise_var):\n",
    "    # X: training inputs, size N x D\n",
    "    # y: training targets, size N x 1\n",
    "    # K: degree of polynomial we consider\n",
    "    # prior_var: prior variance of the parameter distribution\n",
    "    # sigma: noise variance\n",
    "    \n",
    "    jitter = 1e-08 # increases numerical stability\n",
    "    \n",
    "    Phi = poly_features(X, K) # N x (K+1) feature matrix \n",
    "    \n",
    "    # Compute maximum likelihood estimate\n",
    "    Pt = Phi.T @ y # Phi*y, size (K+1,1)\n",
    "    PP = Phi.T @ Phi + jitter*np.eye(K+1) # size (K+1, K+1)\n",
    "    C = scipy.linalg.cho_factor(PP)\n",
    "    # maximum likelihood estimate\n",
    "    theta_ml = scipy.linalg.cho_solve(C, Pt) # inv(Phi^T*Phi)*Phi^T*y, size (K+1,1)\n",
    "    \n",
    "#     theta_ml = scipy.linalg.solve(PP, Pt) # inv(Phi^T*Phi)*Phi^T*y, size (K+1,1)\n",
    "    \n",
    "    # MAP estimate\n",
    "    theta_map = scipy.linalg.solve(PP + noise_var/prior_var*np.eye(K+1), Pt)\n",
    "    \n",
    "    # parameter posterior\n",
    "    iSN = (np.eye(K+1)/prior_var + PP/noise_var) # posterior precision\n",
    "    SN = scipy.linalg.pinv(noise_var*np.eye(K+1)/prior_var + PP)*noise_var  # posterior covariance\n",
    "    mN = scipy.linalg.solve(iSN, Pt/noise_var) # posterior mean\n",
    "    \n",
    "    return (theta_ml, theta_map, mN, SN)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [],
   "source": [
    "theta_ml, theta_map, theta_mean, theta_var = polyfit(X, y, pol_deg, alpha, sigma)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, let's make predictions (ignoring the measurement noise). We obtain three predictors:\n",
    "\\begin{align}\n",
    "&\\text{Maximum likelihood: }E[f(\\boldsymbol X_{\\text{test}})] = \\boldsymbol \\phi(X_{\\text{test}})\\boldsymbol \\theta_{ml}\\\\\n",
    "&\\text{Maximum a posteriori: } E[f(\\boldsymbol X_{\\text{test}})] = \\boldsymbol \\phi(X_{\\text{test}})\\boldsymbol \\theta_{map}\\\\\n",
    "&\\text{Bayesian: } p(f(\\boldsymbol X_{\\text{test}})) = \\mathcal N(f(\\boldsymbol X_{\\text{test}}) \\,|\\, \\boldsymbol \\phi(X_{\\text{test}}) \\boldsymbol\\theta_{\\text{mean}},\\, \\boldsymbol\\phi(X_{\\text{test}}) \\boldsymbol\\theta_{\\text{var}}  \\boldsymbol\\phi(X_{\\text{test}})^\\top)\n",
    "\\end{align}\n",
    "We already computed all quantities. Write some code that implements all three predictors."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [],
   "source": [
    "## EDIT THIS CELL\n",
    "\n",
    "# predictions (ignoring the measurement/observations noise)\n",
    "\n",
    "Phi_test = poly_features(Xtest, pol_deg) # N x (K+1)\n",
    "\n",
    "# maximum likelihood predictions (just the mean)\n",
    "m_mle_test = Phi_test @ theta_ml\n",
    "\n",
    "# MAP predictions (just the mean)\n",
    "m_map_test = Phi_test @ theta_map\n",
    "\n",
    "# predictive distribution (Bayesian linear regression)\n",
    "# mean prediction\n",
    "mean_blr = Phi_test @ theta_mean\n",
    "# variance prediction\n",
    "cov_blr =  Phi_test @ theta_var @ Phi_test.T"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEICAYAAABS0fM3AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nOy9d5xcV3n//753et2d2V4kq9iyJUuucsF2MLYD2GCHxBh/HZOEbkMCJIFfEto3CemEkEBCQnfK14BNgkKRGwaMsWyMccFNcpO0u9o+s9PLnbnl/P5YnavZ1ZbZ3Vntrva+9dqXpJ07d87M3Huec57yeRQhBA4ODg4O6xd1pQfg4ODg4LCyOIbAwcHBYZ3jGAIHBweHdY5jCBwcHBzWOY4hcHBwcFjnOIbAwcHBYZ2zagyBoiguRVGeUhRl70qPxcHBwWE94V7pAdTw+8ABIDrfga2trWLTpk3LPiAHBweHk4knnngiKYRom/77VWEIFEXpBd4I/DXwofmO37RpE48//viyj8vBwcHhZEJRlP6Zfr9aXEOfBf4YsFZ6IA4ODg7rjRU3BIqiXAuMCyGemOe4WxRFeVxRlMcTicQJGp2Dg4PDyc+KGwLgUuDXFEXpA+4ArlQU5fbpBwkhviyE2C2E2N3WdpyLy8HBwcFhkay4IRBCfFQI0SuE2ATcBPxYCPFbKzwsBwcHh3XDihsCBwcHB4eVZVVkDUmEED8BfrLCw3BwcHBYVzg7AgcHB4d1jmMIHBwcHNY5684Q5HI5TNNc6WE4ODg4rBrWnSHIZrMUi8WVHoaDg4PDqmHdGQLLskilUis9DAcHB4dVw7ozBAClUolKpbLSw3BwcHBYFaxLQ6CqKrlcbqWH4eDg4LAqWJeGwO/3k8lksCxH487BwcFhXRoCVVWxLMsJGjs4ODiwTg0BTO4KJiYmVnoYDg4ODivOujUEHo8HTdPQNG2lh+Lg4OCwoqxbQwCTxiCTyaz0MBwcHBxWlHVtCPx+P9lsFsMwVnooDg4ODivGujYEiqKgqir5fH6lh+Lg4OCwYqxrQwCTu4JUKuWkkjo4OKxb1r0hcLlcGIbhpJI6ODisW1ZVY5qVQqaShsNhFEVZ6eE4OKxphBCYpolhGPaPruv2vy3LmrIDV1UVVVVxu9243W68Xq/9b7fbjcvlWsF3sz5Yd4bAsqzjLiyPx0M+n6dcLhMMBldoZA4OaxPDMKhWq2iaRrFYRNM0hBD24zIWJ38URcHtPjb1CCGwLAtN02wjIQ2FPDYUChEIBPD7/Xg8HmfB1mDWnSFIJpM0NzdPuRABvF4vExMTjiFwcJgHOWmXSiVyuRy6rgOTbla3200wGGzoRG2aJsVikWw2ixACl8tFNBolHA7j9/tR1XXv4V4y684Q6LpOsVgkFApN+b3P5yOXy6FpGn6/f4VG5+CwOjFNk3K5bPfzEELYbpzlvl9cLteUXbxlWeTzedLpNIqi0NzcTCQSwe/3OzuFRbLuDAFMNqdpa2s77qLxeDykUim6u7tXaGQODqsHy7Iol8tkMhkKhQKKouDxeAiFQis64aqqSiAQACbdStIoeDweWlpaCIfDTlxhgaxLQ6DrOpVK5biVTCAQIJfL0dLSgs/nW6HROTisLJqmkcvlyGazWJaF1+slEoms9LBmRFEU2ygYhsHY2BhjY2PE43GamprweDwrPMK1wbo0BKqqUiwWZ9zSejweJiYmnF2Bw7pCqvFOTExQqVSWxde/3LjdbsLhMEIIMpkMExMTNDc3E4vF8Hq9Kz28Vc26NAQ+n49sNkssFjsu0BQIBMjn81QqFWdX4HDSo+s6uVyOdDqNaZr4/f5Vu/qvF0VRCAaDttsok8kQi8WIx+PHJYk4TLIuPxVVVW33kNxW1uJ2u51dgcNJTaVSIZ1Ok8vlUFX1pMy+qTUI2WyWTCZDW1sbTU1NJ917XSrr0hDAsdqBmQyB3BU4GUQOJxuapjExMUE+n18Vgd8TgaIohEIhLMsikUiQTqfp6Og4LnNwPbNuzaLX6yWfz2Oa5oyPezweEonECR6Vg8PyUC6XOXLkCH19fVQqFaLRKIFA4KQ3ArWoqmpnFB05coTh4WG7BmK9s24NgaIoCCEol8szPu73+ykWi5RKpRM8MgeHxlEulxkYGKC/vx/DMIhGo+t+l+vxeIhGo5TLZQ4fPmwXqq1n1q0hgGNB49nw+/2Mj4+v+4vEYe2haRpHjhxhYGAA0zSJRqNO8sM0AoEAgUCAkZERBgcH1/XuYMUNgaIoGxRFeUBRlAOKojyvKMrvn6jX9ng8lMtlqtXqjI97vV4qlYrTr8BhzVCpVBgaGqKvrw9d14lEIo4BmAMpV1GtVjl8+DC5XG6lh7QirIZgsQF8WAjxpKIoEeAJRVHuF0LsPxEvrqoqpVJp1jzjYDDI+Pg4oVDIqVZ0WLXous7ExATZbBa32000Gl3pIa0pAoEApmkyPDxMsVikvb19Xd3vK74jEEKMCCGePPrvPHAA6DlRr+/z+chkMrO6f1wuF0II0un0iRqSg0PdGIZBMpnk0KFDFItFwuHwjJlwDvMjdwfFYpH+/n40TVvpIZ0wVtwQ1KIoyibgXODnMzx2i6IojyuK8ngjs3lkY5q5vvRgMEgymaRSqTTsdR0cloJlWaTTaQ4fPkwmk7ENwHrKAlougsEgqqrS19dHOp1eFzHCVWMIFEUJA98G/kAIcZyjTgjxZSHEbiHE7ra2toa+tsfjmTNorCgKXq+XsbGxdXFROKxehBDkcjkOHz5MIpEgEAisOSmItYDX6yUcDjM2Nsbo6OisaeYnC6vCECiK4mHSCHxdCLHnRL++1+ulUChgGMasx/j9fkqlkhM4dlgxSqUS/f39jIyM4PF4CIfDToXsMqKq6hRX0cnsEVjxq0iZXMp8DTgghPjHFRoDiqJQKBTmPC4UCjE6Orqu08wcTjyVSoXBwUEGBgYAiEQijmbOCUTuuPr6+uadI9Yqq+FquhT4beBZRVF+efR3HxNC3L0cLzZTq0qYXPFns1mamppm3WbLBhnj4+N0d3c723GHZcUwDCYmJshkMnYR1FrGsixM08Q0Tbs95XRXq2xlqaqqfb+thvvM5/PhdrsZHByktbWVlpaWVTGuRrHihkAIsQ84YZ9oKpWasbjG5XJRLpfRNG3OrAvZsyCfz6/5G9NhdWKaJtlslmQyacsirLVJRzasr1Qqdq3OXK7XWoQQU96vx+PB7/cTCATweDx4vd4VcYm5XC4ikQgTExNUq1U6OjpOmhTTFTcEJxrDMCgWizNO4l6vl3Q6PW/6nXQR+f1+R+fcoWFI2eREIoFpmnb2ylrAsix70s/n88f1MfZ4PIsubJNtMguFgr2DCAQCdrHcibwHFUUhEolQLBY5cuQIPT09J0Xzm3VnCAByudyMBSMyaFytVue8uOTFPTIywoYNG9bMzeqweimVSoyNjVGtVgkGg2tipSmb2OdyOUqlEpZlLXnSn4npPYuFEBiGYcu/+Hw+mpqa7B3DiSAUCqFpGv39/fT29q55/aZ1aQiEELPuClwuF7lcjtbW1jnP4ff7KRQKpFKpeY91cJgNTdNIJBJ2x7y10BRG0zQKhQK5XA7LsvB4PCe0hkH2TpaTvmmaJBIJhBAEg0FisdgJaWTv9/vRdZ3+/n66u7vXxHc3G+vSEEixuZnS72TQOBaLzbsqC4VCJJNJ/H4/4XB4OYfscJKh6zrJZJJsNovX61318SbLsiiVSqTTabuV5WopYHO5XHZvgWq1ytDQEB6Ph1gstuzSMB6PB1VVGRoaor29nVgstio+k4WyLg2BqqqYpommaQSDwSmPyS+xUCjQ1NQ053lkB6Th4WE2bdrkxAsc5sUwDLuf7lrQBDIMg0KhMKWV5Wpe9Hi9XrxeL6ZpMj4+jqqqxGKxZU25lUHk8fFxdF2nra1tzbmL16UhgMkLJpPJHGcIYHJXkE6niUQi836h0ic6NDTExo0b14Rv1+HEY5omuVyORCKxJjKBDMOwexkrirLmWlm6XC7C4bAtxZFKpYjH40Sj0WW5RxVFIRqNks1m0XWdrq6uNTUXrJ1vtsF4PB6q1eqMGkMulwvTNOtuSuPz+TBNk9HRUUeCwmEKlmWRzWZtSYhgMLiqJSEMwyCVStHf3082m7XHu5aMQC2qqhIMBgkEAlPel2VZy/J64XDY7gWxlgpP1+2OACZX87lcbsaIv8/nY2Jiou6ersFgkHw+z/j4OO3t7av2Rnc4MdSmghqGseozgSzLIpfLkUqlAFa1sVoMqqrafYuTySSZTIbW1tZleZ/BYNDuDNfb27sm+kGsa0Pg8/koFoszpou63W40TaNcLs/oPpqJSCRCOp3G5XI5mUTrFCEEhUKBRCKBrut2F6zVisygSyaTmKZJIBBYs6v/epAGwTRNRkZGCAQCtLa2NnyyDgQCVKtVO7203jlkpTh5v/E6cblcswrJ+f1+JiYmFuTuiUQiJBIJp3/BOkPuAA4fPszw8DBut3vVawJpmsbQ0BCjo6N4PB5CodBJbQRqkTEEwzA4cuQIqVSq4QqjXq8Xv9/PwMDAqu98tj6+dYllEaxO7WXg8/nI5/Mzlr+73W67WrJeZNBodHTUMQbrgLVoAAzDIJFIMDg4yJ3PFwiHwyfEbSX1haTekPyZSXPoROHz+QiFQmQyGY4cOUKxWGzo+d1uN+FwmOHhYdvtthpZvVfrcvD9D3D5K3u5b/s/2L+SAleFQoHm5ubjniJjBQvJmZZl6KOjowDEYrHGjN9h1WBZFoVCgWQyabuAVnt1qTRayWQSRVEIh8N8+8UR3nr28df9YjFNE8MwME3T1hfSdd3+3Vz3kKqqqKpq6wl5PB7cbjdut3tZDZVMA5fuonA4TEtLS8OqlFVVtdNLDcOgra1t1cVf1pch2P4mAk/dTk/2MVLhX7V/LYvIIpHIcRecx+OhUCjYbQDrRX75sqnFyaZWuF6RaaATExO2T321GwCYlLJOJBK2qGIjJlYp9SCz78rlsr2zlgssqSJaj1Cc3DEYhkGlUpmS2eNyuezsH6/XuyyGQbqLNE1jYGCA9vb2hqX5ysVhOp3GMAw6OztXlRtOWYvpjrt37xaPP/74wp9oWeQ/tR1D9fHY2X8PNV9wuVwmGo3OuCuQq5mNGzcu+KKQq7BYLOZkE61hdF0nm83arQsbNZkuN5ZlkclkSKVSdrHVHc/l+db+43X1b9wR5qadc8skyJV+qVSiVCrZq3ypv7Vcn4lpmui6bvvxZQex5dIXkkJ3gUCA9vb2hr6GlBPp6uo64S5ERVGeEELsnv779bUjUFVeaX0d5w79F7H8AdLRHfZDUnZipl2BzCBajPT0Wi80We9omkYmkyGbzaKq6prKqimXy7Y7ojYN+qadEXvCv/5bI+y5sWvO85imSaVSoVAoUC6XEULUvcpvFNOF50zTnGLgotFoQ43z9N1BW1sbkUikIQu5UChEqVTiyJEj9Pb2rgr10vVlCICB2CXsGN3DKcN3TTEEsiFGPp+fcVcQCATsuoLFXGzhcNhuNdjT07MmcovXK1JXJ5VKUSqV7LaQa2U3Z5omqVSKbDaL3+9f1LUmhKBSqZDP5ymVSgghTri43Fy4XC47LdcwjClxj0gk0jC5F7/fj2VZjI+Pk8/nG7Y7CAaDtnrphg0bVnw+WHeGQFc8HG65gtPH9xIsj1IKdNqPzbUrcLlcCCHIZrPE4/FFvXYwGKRSqdDf309nZ+eq15lZb+i6Tj6ft/24Pp9vzX1HxWKR8fFxgEVpAhmGQalUIpvNYprmqhKXmw0ZUBZC2H3FA4EATU1NDYnfSEmQRscO/H4/1WqVvr4+NmzYsKK1BuvKEHzlma/wnfJ3eF/Lm9mWuJuNo3fzwuZ32o/L4NZcu4JUKkU4HF70ikO2vBseHqZYLM7YF8HhxCGEoFwuk0qlKBaLtvtnNReBzYRsayknwYVeU9L1KT8DmQO/VNxGCX8lidfI4TLLeMwSinUsVdtSPRiuILo7RNXThOZrxVIXd29JTSSYVCGVzaOam5sb8l78fj+maTI2NkahUKC1tXXJuwPpXhsYGKC7u3vFFh7ryhC0B9sZMAf4haefc1svpWf8AQ72vgXdcyxA5vP5yGQyhMPh4wI5iqLgdrtJJpN0dXUtekXgcrmIRqMUi0UOHz5MZ2fnqlZ0PBmpVqvk83kymQyGYeDxeNasnrysZIb6dgHTg8XXf2sEgKt7LX7jtNCixuDR8zQXXiJS7CdcOkKoPEiwMo7brL8GR1LxNFP0d5EPbSIX2kw2vJVioHdKcsd8yMC4NAiBQIBYLLZkl1Ft7ODIkSP27mAp1NYaGIaxaI/DUlhXWUOWsHjD7W8gJ3L8ZeitXPXsx3il90YObrhhynFSnrqlpWXG8xQKhYZN3nIrHo1GaWtrWxWBo5MV+VmnUikqlcqaC/5OR/rGC4XCgnYBMgaSyWT44CMuPn+ZWPB156tM0JJ9llj+BZrzLxIuD9mPlX1tFAK9lPydaL4WNG8rVU8U3RXEcAexlGOv5bKquM0SHqOIV88QqCQIaOOEtGEixT7cVgWAiqeJVHQnyeazScR2o3sWdu9VKhUMwyAajdLU1NSQXbjMLIpEIrS0tCw5A0hmGMbj8WWrNXCyhgBVUbnGfw1fLX2V74uX2RU7j42j99DXfS2m69jWUVYbzxZ0CgQCJBIJ/H7/kr98qUlfLpc5dOgQbW1tDbtQHY7drJlMxq4aXSudwGajVs9IBkjroba5jOwtAHUaASGIFA/Tnn6ctvQTNBUPA6C7QqQjpzPc9moykdPJhTZjuhrkVhMWIW2E5vyLxLPP0ZJ9lq6Jh7EUF6noTkZaL2Gs5VVT7t3ZkL2NC4UChUKBeDxet6DkbMjdgUylbW9vtxvkLIbaWgOZYXiiFinrakcAsGfPHr5d/jb7jf38lf/NvPHApzmw6e0MdL1hynGapuHz+Whvb5/xPKVSiVAoREdHx6LGMRPyRlVV1U5XW6ur1ZVETv7ZbJZisYgQAq/Xu+KZGY1AdjYrFot17wJmMgDyed8/bHLd5tnPESoN0p34KV3JhwhUJxAoZCLbSMTOJ9F8HoVgLygn6BoVgmjxMB0TP6Nz4lGClTF0V4CR1ldzpPO1FIIb6zqNbErl9/uJx+MNyTCS11xTUxPxeHzJC7lisYjX66Wnp6ehtQaz7QjWpSHIiRz/XPxntnu38/mhV/BXEjx07r8g1KkfeLFYpLOzc9ZAUz6fp6urq+H+fXlRuVwuWlpaZsxicpiKrG7NZrN2HwnpJ17NGS/1Uitr7XK56gp+yiwamQVVawDmwqMX6JzYR8/4gzQVD2KhMtF8NqMtryIZO5eqZ2rnPl3opM00OTNHzpr8yZt5KqKCIQwMDPtvN27cihuP4iGoBAmpIcJqmJgrRourhSZXEy6ljmtdCJrzL9I79kM6J36GS+gkms/lYO+byUa2zf98jrmL4vF4w2oEyuUyiqLQ0dGx5IQDea7e3t6GpcM6huAoe/bsQVEU9ln7+EHhB3zEcxVvfenfefbU32O47fIpx+q6jhCC7u7uGS8SWWizYcOGZatulDuEWCxGNBp12mEexbIsKpUKxWKRXC6HrusoimJP/icTlUqFZDJpS6LXs0uUmVC6rtdtAMKlATaO3EN38iFcVpV88BSG2i5npPUyqt5mLGGRNJP06/2MGWOMGWOMG+OkzBSCqfOIGzcBNWBP/G7FjQsXJia60NGFTtEqYjBV7FFFpd3dTre7m25PN5s8m+j19OJWZl8Ve/QcG8Z+yCkjd+E18kxEd/LSKTeTC58673u2LItyuYzP52tIFhBMxm40TaO5ubmu3udzIY1Vb29vQzLZHENwFGkI3AE3f5/4e4JqkDsHh3AjePjsfzhum1ssFmltbZ111a9pGh6PZ1Zj0Qgsy0LTNFvbJhaLrfpGJ43Gsiyq1SqVSoVcLjelwtXn852Un0WtPITH46nLtVWpVEin07Zrc163grBoSz/BKSP30JJ7DlPxMNL2Kwx0vp50cAOHq4c5VD3EgD7AgD5AWUxmAblw0eZuo93dToerg1Z3K01qE1FXlIgawa/4570fhBBURZWCVSBtppkwJ5gwJxgxRhjWh8laWQA8eNjo3cjp3tPZ7ttOp7tzxnO7TI3esR+yefi7+PQsw62/wssbfxPNN39vEDnhtra2Ljl2IN+b3NW3t7cvaRLXdR1N0+ju7l5ybMsxBEeRhiAYDPLL8i+5PXs771TO5g8PfZ9fbvsQYy0XTzleaqv09PTMOtkUi0VisdgJSfuSk6GiKIRCIbtoZjXLHi8GqVqpadoUaYOTddU/nVKpNKW72XwTk2EYZDIZCoWCrd45F4ow6Uw+wpahPYTLQ5S9LQx0vo4n4mfzvDXIS5WXOFg9iI6OgkKnu5NTPKew0bORjZ6NtLnb6nPhLIG8madP7+NQ9RAHqwcZNoYBiLvi7PLt4vzA+XS5j0/jdhkltgx/h1OG7wJF4WDvDfR1X4eYZ7xydxAOh5e8kpcYhkG5XCYejxOLxRYd8zNNk2KxSEdHx5LUjB1DcJRaQyCE4IvpLzKsD/M/ozlieHnkrE8dtysol8uEQqFZ00lll6cTWQ8ghKBardruK1kFu5zqjMuFnPSlmNl0FUuPx4PH4zkpfP3zUa1WmZiYsIXJ5jPwpmna9RD1xA4Uy6A7+VM2D32HkDZKLrCBH/a8hp/4FJ6tPEfSTALQ5mrjdN/pbPNuY4t3C3515RVWs2aWA5UDPFd5jpcqL2Fh0enu5MLAhVwQuICAOnXV7a8kOOPwf9CR/gW50Gae2/pe8qHN876OpmkoikJbW1tDEgxkrMbtdi9pd2BZFvl8ng0bNix6nnEMwVFqDQHAuDHOZ5Kf4SKlky8d+hlPbfsw4y0XTXmO/CLnChzLAG9vb++KyBJLOWCpBik7TgWDQdxuNx6PZ0WNg5Qslnr1lUqFUqlEpVKZolO/Gsa6EpimSTabJZVK4Xa7572G5OJDqqH6/fO4YoRFV3Ifpx65k0AlweNNm7izdSuPMk7WyqKicqr3VHb6d7Ldt52Ya3X30ChaRZ7WnuaJ8hP06/14FA/n+8/nstBldLo7pxzbMfEo2w/fhkfPcaj3Bg72Xj9vppNcmMzlFl4otbuD5ubmRV3jUo2gqalp/oNnwDEER5luCADuy9/H/cX7+eyEwQWGh5/NsCvQdR3Lsuju7p51eycn40ZG+ReLXGXXtt9TVdXOp5b+Y5fLZUtryJ96V95SP17+XfsjbyS5azEMw3btwDE1yfWy0p8NWROQTCaxLKsuN5CmaaRSKarV6vyBYCFozTzFtoFvUK4M8e14L9+LRBkUWVy4OMN3Brv8u9jh20FQXd19dWdjSB/i4dLDPFl+EgODXb5d/Gr4V+nx9NjHuI0C2w/fRndyH6nIdp457YNUfDPv8CUy7TYajS7JrVPLUmMHJ7UhUBTlauBzgAv4qhDi7+Y6vtGGwBAGn0l+Bswid/W9yAvbPsx4/MLjnlsqlWhubp7zS6hWq3am0WqrEpZtAg3DsCdsRVHsNoHy37KpCHDcxS+bhci/ayf3WmoNi9vtduohpiEnhNom9/OtEHVdJ51OUyqV7A5ecxEtvMJpff/FC0Yf32huYZ/fhQBO8ZzC+YHzOcd/zpqd/GeiaBV5qPgQ+0r70ITGmb4zeWPkjbS7j9UCdSV+yo5DX8FSPTx76gdIxs6d85zye/L5fLS1tTVspyozixZad3DSGgJFUVzAS8BrgUHgF8BvCiH2z/acRhsCgFcqr/DF9Bf5nYLJrUW5K5g6wUkXUVdX15y+w0plsix+NRqDehFCzNpHVlGUdb2KXwpCCDRNI5lMUqlUliUO4K1m6Bi4nccqT3FHU5Qht4uwEuai4EXsDuymzd3WyLe06ihbZfaV9vGT4k/Qhc5FgYt4Xfh1RFyTGTfB8ghnv/RPREr9vLTxZvq6f21eHSNN0wDo6Oho2G5fGhlVVWlvb69LfXS5DMFqSDW5EHhFCHEIQFGUO4A3AbMaguXgVN+pXBC4gK/zOG/KDNGeeuy4WIH0vU9MTMzZas7n86FpGsPDw3R1da24m2gxOJN9Y5E3/cTEBJVKBZ/PN6/veXocYD63kWIZmKP/zf2lh7kv5KMajrHFfQpvDV3GLv+uOXPxTyYCaoDXhl/LxYGLub94P4+WHuUp7SneEHkDFwcuphTo4rGdf8GZB7/A6QNfJ1rs47mt78Vyzb648/v96LrO8PBw3ZP2fNT2Sh4eHm6YZtFiWA1XRg9wpOb/g8BF0w9SFOUW4BaAjRvrKyVfKNdGrmW/tp8/a+/ki0fuYDx+wXGxAq/XS6lUIpfLzShVLfH7/VQqFYaGhujq6loTfW0dGo9lWRSLRbu4qx4DAJMr0ImJiboKwoQQjKbu46eF+/mFTyEQ8nOR72wujPwqXZ65u4+dzERcEa6PXs9lwcvYk9vDntweHi8/zg3RG+j2dPPMaX9APrSJ0wbuwF9J8NQZH5lTzM7j8aCqKuPj43bFfyOQmkXFYpFSqURbW9sJVyNeDYZgpiXOcT4JIcSXgS/DpGtoOQYSUkNsqFzLc947+a4nx7mJnzLc/prjjgsEAmQyGfx+/5wTvM/nQ9d1BgcH6erqWpIglcPaQja5kQ1eAoFAXamI0+MAc10zpjB5vvAI+3J3cchl0OqGG11nsyv+ZgKuk8f3v1Ta3e3cGruVJ7Un+V7+e3x24rO8Lvw6rghdweGe36Dk7+Ksl/+ZC5//vzyx/eNzFqDJzmjJZBLTNGlqamrYzlnuDkZHRwmFQg2rdK6H1WAIBoENNf/vBYZXaCz84uA5XHT2s3wuvp+vj/w3SuulCHXql6EoCj6fj0QiQXd395yrNbmKGBkZsYtKHJfLyYmsAM9kMrY0iN/vrytQLtNHc7kcLpdrTgOgC53HSj/nofy9JNHYYur8HmewsfPtuBql/LkIZEypNra0WlyMiqJwfuB8tvu2sye3h3sL93KgcoCbmm6Clot53B3l3Bc/xUXPfYLHt3+CYrB31nOpqkooFLJ7WcTj8YYlQyxnr+S5WA3BYjeTweKrgCEmg8U3CyGen+05yxEsltz6gMGnX9YkJToAACAASURBVF3iM4m/Y6uW56O+1zPUde2Mx8oy/nq0w6W/NxAINKzvqcPKI3v7Sn1/y7IWVPlsWRaFQoF0Om2n9852LRnC4Ofln/NA/n4yosA5WoUbq81Ee9+HFuxu5NuykanBMv1XpgvXjnF6ppl8TM4ttRlmtY/Dseyy6c3pl5unyk+xJ7cHE5O3RN/CuYFzCRf72X3gr0FYPH7mn9alZloqlQgEArS2tjY8M06mrwYCAdra2vB6vSdv1hCAoihvAD7LZProbUKIv57r+EYbgu8fNtnbN000K/pLAj138P5Mma2nfWZWzXMpL1HvFyM1g06UpXdoPLKqW8aKdF23NY/qnQxqA8GWZc25czCEwWPlx/hR4UdkrSznaBVuzWpEO29mpP01C+rcNRemadr1MnKMqqrald1er9dOBa6tOak1AnO935lqTqrVqp1KKavJAbvGZDmNQ9bMcnvmdg7rh3lV4FW8Kfomoto4Fzz/SVRh8Isdf0YhNL8xKJfLeL3ehqaX1lKpVNB13Q4kd3Z2npyGYKEsxRB85zvfQdf1WXuD3vqAwZeumGyEfUfyCzxjHOTvjZ2YG94x4/Hyhu7o6Kg7k0Baer/fT2trqxNIXgNItdNyuUwul8MwDHsFv5CbX6aP1qMMKg3Ajws/JmNl2FUVfGBinI3h3by06e3HyUEvBCGEXXAo5wDZqF7WKMiCwxOF3HlIw1AqlWzjIA1toxdOpjC5p3APPyn+hF53L++IvYOuaokL9n8S1dL5xY4/pRA6Zd7zaJqGy+Wio6NjWT4zOWeYpsnOnTtpa1tcCrBjCI7y4IMPMjo6OqvGvzQEMFmg8rnRT9JsVPm99o8g/J3HHQ/HGl10d3cvKFVUVt5Go1Gam5vXZJrpyYqcKCuVCoVCgVKphBACt9ttNxxfKJqmkU6n7fTR2dIELWHxpPYk9+bvJWNl2GaF+MPxPs43/OzfeivJ2HmLfj+1Gk6BQIBAIGCv+Fdj0Z8UfZQZNZZl4XK5Fv0dzMZz2nN8M/tNfIqPt8fezhmGl937J3cGP9/5l5RnufdrkRpFnZ2dy5YCOj4+zo4dOxadOekYgqM88sgjdkBvpn610zs29eUf4/PFb/EGPcyVG/581vPKiuKurq4Frwjktlj2Uz0ZOmmtNeREqeu6PfHX6jYtRQpDXm+aps1bEXywepDv5b7HkDHERrWdP5hI8KuZfoZbX80Lm9+O4a4/rVC6XuT78Pv9hEIhewxrzS1ZG48pFouYplmX0mq9jOgj3Ja5jbyZ56amm7hUtHLh83+G4Qry851/QdU7v/7SchuDZDLJGWec4RgCWLohUBTFVnmsR9vl4cG/53/d47zPfzVbm3911uM0TbMVBhdqDORFLmWHY7HY/EJiDotGrjR1XadYLFIul23JDSl8t9QVZ60BmG/CShgJ9ub38nzleZrUJt5mtPOOgZ9gusM8v+UWEvEL6n5f0uUjM1Cku2c1rvgXi3TVyd4UjXId5c08/5n5T/r0Pq6LXMevWT3s3v9Jyv4OHjvzz+syxJVKBSEEnZ2dDU8KcQxBDY0wBD6fj1QqRT6fn9+3r+f46sifMux288H2j9Psnn1lIDOJlpJFIF1GsrF9KBRydglLQPqddV2nXC5TLpftjma1E3+jjG6tAXC73XN+d0WryP2F+3mk9AgexcPrvbv54OBjdBQOMhq/mP1b3o3umTmeJald+bvd7imT/3pYSEhjns1ONrKpN2V3Ngxh8I3MN3im8gyvCb2G3zY2sPvFvyMTOZ3Ht3/iuJa2M7FcxsAxBDU0yhAIIRgbG7ODdnPhGt3Dx619dLtaeXf7Hx9Xrl/rUpKN7VtaWpZ0I8q8dMuy8Hg8RCIRgsHgSbe6axRS9VSu9svlsv35Sdxu97IEQeWOrt4dgCEMHi49zA8LP0QTGhcFLuQdZS8X9X0L0+XjwOZ3MdpyyZwZQXJHI+sOpNtnPUz+MyGbt2QyGYAFZXFNxxIW/5v7X35W/hkXBC7gg5Uuzjn4eQbbruD5re+tK1OrWq1iWVZDjcFyGYLVUFC2YsjmEyMjI1Sr1TlvXKvjTXzsxUf4RHOKu3Lf401N1095fG+f4LqjPS+CwSDFYhFgScZAVVV7t1KrVy/TX+WNv54Mg+xrIBVUK5UKlUrFTkOszWlfaErnYscjawh0XZ+3GlgIwTOVZ7g7fzcT5gSne0/n1wNX8Lr+PbSnHyfRfC7PbX3vjP7o7x82eeMpCpVKBcuyCAQCxOPxZX+PawWXy2XvoGVV93y1GbOhKirXR68nrIa5v3g//+g/l0/0XM+2oT0UA9309bxp3nN4vV4qlQpjY2PLGkBuBKt3ZCcImfI1MjJi+1VnQiguurrfyW+NfJbbeYRNvq2c7T971vPWGoNGVB7K0nY4FthMJBL2/30+H8Fg0M5GkavetbY6lFLZpmna6YQye6c260VO+LV9DU6k+0w2Ikqn05imic/nm1dCZKA6wPfy36NP76PD3cF7Yu/hVZrOruc/jdfIc2DT2xnovGbG1aZhGOztg9d2TaY+3z0AN2+aXetqPeNyuWhubiYcDpPNZsnn83XJdk9HURReH3k9HsXD3YW7+cvI2fxty6vYNvANSoGuGaXqp+Pz+ahUKoyOjq5qY7A6R3WC8Xg8tLe3Mzo6OmMmkSQd3cFNY2fyjHaQb2Xu4IDWxgOHj2md3/rA5CR17SaF6za7bGNgmiatra0Nc0fM1Le3VqoYjhUDyRtAuiqkcahtSLNcxkJKWcviodqiotqgppzs5Yp++nuVq/sTPdnPhGEYFAoFcrmcXQg2n1sxZaa4J38PT2lPEVbD3BC9gQt953L64LfZPPxdiv4untz+UfKhTcc9V7p/JicxlZ6eHlRV5X9+MMLNZzmGYC7cbjctLS2EQiFSqZRd2b/QRdmV4StRFIW78nfxkbadfLayhV0vf55Hz/pbioGeeZ8v1YjlzmA1dt9b1zGC6chOUXNlEnmraU579kPc3NWKy9PCB1s+SFANTqk/mM5yF5vMRq08gFxpz9VEptZA1P5+Nmqb1Eyf6Od6vemvu5juaCcS6f/P5/MUi0Xb3TDfhKJZGj8u/pifFn8KwOWhy7kidAWxaoazXv5nmguvMNh+JS9sevtxlesyg+yHoz7uHpj5/HtuXLqyaK1hnv4dzvVdyMen6wrJv6dfTyvtupJSHqlUatEppw8WH+T7+e9zgXcn/3pwH6YnwqO7/gazTn2nRswDTozgBBAOhzEMg0wmM+s2v+qNUeh6M58bvYN3dCvcnrmdd8XeNed5/X6/3Z+gvb39hK1qa10ncyFX7vKndmU+30KhVlRMrtzl71bjpL4QpPsnm82i6zput7suBVlTmPy8/HPuK9xH0Spynv88rolcQ8wVoyvxEDsOfxWByi+3fYixlounPFcaAKkv8+5NPt599JDrvzUy5Vj5/xt3hLlp5+ySyLX9omUlsZzEa3da0p3odruP0w6qneTlOaf/LX/kLk/u9GpdevLY6a+13KiqSjQaxe/3k0wmbQ2fhbz25aHL0YXOvYV7+eimV/GPr/yAnQe/wNOn/WFdwWM5DyQSCdrb21fcONbiGIJpNDU1YRiGfaHMxEDn1VyceJCPpYr8Zfwl9ub3cu2mmYXpJLKxxcjICG1tbatKkvpkmLQbSbVapVAoUCgU7PhLPStIIQQvVF9gb34vY8YYmz2buS52HRs9G3GZZba//Hl6kj8lHTmDZ0774BS5Y+kCkgZgtsWC3AVc/62RGXcEctLXdd2epKWLUKYhy/ahtbu/5abWQJimaQf5ZVZXrVFaTj+61+uls7OTbDZrF5UuZHV+VegqSlaJH5V+yl+ecgl/1v8wmfBd9HfPff9L/H4/5XKZZDK5LEJ1i8UxBNNQFIV4PI6u62iaNqP/V6hunt96K2959uM8EzmL75Ye4s0d7cCr5jy3LFJKJBJUKhWam5tXzYWw3pEyIdlslmq1areErNdADuvD7M3v5aXqS7S6Wnlb89vY6duJoig05V/hrJc/R6Ayziu9b+FQ7/UIZXLyMQzDblnZ0tIyZ7zhxh0zFzPJVbcsiAsEAkQiEXw+37JPrPUi6zXkWGoXQrUJAcVikUKhYO9IZFyrkaiqSiwWw+v1kkwmbdmQet/HdZHr0ITG/5Qfo6tzJ+8e+Drp6HZy4a11nSMQCFAqlUin08Tj8VWxCFv5K2QVInuIzpVWmgufSn/XNfz54N0c2nope3J7aFKb2OHfMee5Xa7JIHI+n0fTNFpbWx2NoRVCqojKyUdKSNdOUtMlR6aTM3PcV7iPx8qP4Vf8/Frk17gkeMlknYmw2Dz0XU49cicVT4zHzvwkmegZwDHD43a76ejoqMvoSPePEILrt/ntCVOKF8qdy2qYWBaCNBCBQIDm5mY7LVjTNPL5POVy2Y7rNdKoyfTrsbGxWRd9M6EoCjdEb6BgFfg8B9gUaeHSlz/Hz876VN3xAjkHyAynlcYJFs9BtVplZGRkVoVJl6lx6dMfJu/y8raejYyZ47wv/j42euoL5Eh3QDwedySpTyDS9SdVROeSJ5gtCaAqqjxYfJAHig9gCpNLgpfw2vBrCaqTdR++aopdr/wrLdlnGY1fzPNbb8Fwh+3AsxCCeDxOMBisa1cojZasiI5EIoTD4QWrn65FZEV4LpdD07RFqb7OhWmaJJNJNE1bUC/iilXhC6kvMG6M8l9DgzQ3Xcxzp76/7ufLGpR4PD6rGvJ0nGDxCuD1euno6Jg1rdR0+dm/+T2c/8Lf8Felc/jDQInb0rfx/vj7aXXP3u6u9vxut5tUKkWhUKClpWXF0yNPVuQKXO7EpNthoZ+3VAa9J38PWSvLLt8u3hh545Tvuy31ODsPfgHVqvLcllsZar8SjupbSQn0pqamKRPZHc/lZwz4ynELIQiHw3aywXpyKcr052g0aveByGazlMvlhqQUu1wu2traSKfTtuRMPYsyn+rjnbF38i8T/8L7ujdw55GH6Uqcw0jbZXW9rnTjTUxMzNuVbrlxdgR1MF9a6ZmvfIGexE/47o4P87faXnyqj9+L/x7Nrvq3fHNNEg6LQ7oYamWk6yksmqlREcBlWw6RDN3FkDFEr7uX66LXsdV7zC+sWlW29d/OKaP3kgtu4pltv08x0GNP5j6fz64Ens704K/Um/J4PMRiMYLB4Krw9a8WZF8H2d95oTGd2c6ZzWZJp9OEQqG6zzVqjPL5iX9ho17l30cSPHnWZ6j4Wup+XXl9dHZ2zuuecnYEK8h8aaUvbHobLbnneO2hr5Pa/kH+LXsbX0p9id+N/y4R1+xpfbXICapQKJDP54nFYoTD4XW18msEM+nXS/9zvTf2dZtdtlzIrQ8Y/NWvpNmb38vTledptpq5uelmzvGfg6oc+25CpUHOfvmzREoD9HW9gZc2vhWheuzVfGtra12Ti5QkDwQC9PT0OAq0s1DbT6FarZLL5chkMksyCIqi2AkcqVSq7uKzTncnNze9lX/P3MZfxUP80cEv8OT2j9fdOU66JsfHx+nq6lqRNraOIaiTpqYmu3p3ujEw3UGe2/o+Ltj/F1w18iCVnnfxldRX+HL6y7wv/j7bbzwf8uK2LIt0Ok02m6W5uZlgMOjsEOZApiPKyV/mqS91Ei1aRXwd9/Hp5KN4FA/XhK/h1aFX41FqblQh6B27nzP6/hPDFeCJMz5CMnbe5CqvWCQcDhOLxWb8/u54Ls+39hfs/8u6gN85r5X3XjZ/xarDJF6vl9bWVpqamuy0UPn9Lwbpr5cy9fUYgx3+HVwdvoZ7uIcd2mF2j/+YoY6r6n5Nt9uNaZqMj4+vSPXxujMELpeLUqm0YL+ioijEYrFZ00pTTTvp73wDp4zezQWx8zFi7+Br6a/xpdSXuCV+CyG1fv+fFJszTZNUKkUmkyEajRIOhx2DwDGtJZnvX6lUABa88p8NXeg8XHqYHxV+hDemcWHgIl4ffv1xuzuPXuDMQ1+kI/UYyaazePbU91P1NqNpGgDt7e1zBh9v2hnhzaf7qVQq/M7deR64dYcTI1oCHo+H1tZWotGoHXeTKbQLRRqDVCpVd8zgytCVDOlD/FP8Gf517Jv4m89C89XfUlJKUSQSCTo6Ok7oTnDd+R3a2toIh8MUCgVM01zQc1VVtRtUy8mnlpc23kwh0MuuVz7PTqWdd8TewZgxxhdTXyRv5hc8Vplq6vV6yWazHDlyxM5uWIuxnaUgq3xTqRSDg4MMDw+TSqUQQhAMBu3PaUmy38LiifITfCrxKfbm97LRs5EPt36IG5puOM4IxLL7edUzf0Rb+glePOW3eGL7xyi7IhSLRYLBIN3d3XMaAdM0KRQmdwM9PZOrf8cINAZZNNbT04NlWRSLxUXdL9FolFgsZu8y50NRFP5P0/+hw9XKx1qidBz6N1jg6/r9kwuDdDq94PEuhXUXLO7v70dRFDRNY3x8HFVVF7yFNAyDkZERuxKylnBpgIuf+Sjp6A6e2P5RXq4e5Lb0bcRcMW6N30qTa2kNxyuVit2AJBqN2j1nTzakKJ1sYq7rOnCsSrbRsZOXKy+zN7+XIWOIHncP10au5TTfaccdpwiTrUf+my1D/0vJ38kzp/0+ufAWexfQ1tY25/UkhLDz4ltbWwmHwyiKwlcfHeHdFy9dO8hhKpZlkc1mmZiYOE6osV4ymcycsjPTGTPG+OfkP3KmVuQPg29mrP2KBb2eTCttaWkhEpm6AHEa09TQCEPgdrsxDINEImGv4hYyuVQqFUZGRvD7/ce5a3rHfsiZh77MixvfSl/Pmzh41BgE1SDvib2Hdnf7LGetHxkUlcHQSCSC3+9fkwVFtWqktRO/rC71eDzL5hIb1oe5K38XL1ZfJKbGuCZyzXGBYElAGzsqFvcyg22v4YXN76TKZEA4EonQ3Nw85zhl9Wxzc/OscQOH5aFSqTA+Pk61Wq3b1SMRQtiupnrrDJ4sPcE3ct/kbTmN8zb/zbxd5qZjWRblcvm4TCLHENTQKEMAx1LGksnkgv2JmqbNXGMgBGe//E+0p37BY2d+kmxkG0f0I3w1/VWEELwr9i5O8Z6yqPHPRK1RUFXVzqaQdQqrKfOotrGMLBSSBVa1ejPLPUlmzAz3Fu7lifIT+BU/V4Wv4tLgpVMDwccGTVfyIXYc/hoChf1bbmG09RJ7F9DS0jLnBCFXeFLufLFBTIelIZMw0un0gjWGLMsikUhQrVbr/v6+O/EfPKQ/x1+UOwlu/v8WPF55X3d3d9vzlWMIamikIZBIvXDZ+aleCoUCiUTiuNRAt1HkVc98BNWq8rOzPkXV20zSSPKV9FfImTl+q/m3ONN/5qLew1zIQGqt2qPX67V3C1LxcTmb1tTKUMuVfrVatZU1p3cRO1EKlDCZCfRA8QH2FfchEFwWvIyrwlfNmtnlNkrsOPQVuiYeJhXZzrOnvZ+iO46maXNmBEnkLiAejzvaUquEYrHI6OjovP2kp2OaJqOjo7YQ4XzoQueLI58kL4pcU7iJc0+/aMFjrVQqqKpKZ2cnqqqunCFQFOWHwIeFEE8v6pWXgeUwBICdvlUsFhdUUCKLUKZvOcPFfi567hPkQ5v5xY4/Rahu8maer6W/xpAxxNXhq7kydOWyT4JyMpb9A2q7e8nG7dPlh+fToq9tOFM74ct/1yLPuRy9guulbJV5sPggD5UeoiqqnOs/l6vDVxN3x2d9TnPuBc565V/wVSY4uOEtHOr5DbTK5M6rpaWFcHhmETiJLHRqb29f0OLCYfmpVquMjo5imuaCvhupIFyvGN5YZYDPpT7HWSXBTZv+DlwLj1GUy2WCwSCtra0ragjOA/4B6Ac+JoQYmfMJJ4DlMgQwOcml02m7oKSeL1s+J5fLHRdQ6kw+zNkvf47+zqt5YfM7gcmVwrey3+Ip7SnO8Z/DjU034lVOvPBc7UReO7nXas7XGoSZmpVMb2KzGpqQ1KJZGvtK+3iw+CBlUeYs31m8Lvw6Oj2dsz5HESZbBr/N1sFvU/a188xpHyATPo1SqWSrhM7lQpQZTk1NTcTjcScWsEqRC79SqbQgeYdZXcKz8FTyDr5uPM57rK2c3v2+RY21WCzS0tJCpVJZmcpiIcSTwJWKorwZuFdRlD3A3wshyosaySpHylB7vV5GR0fx+/3zlvbLGgOZqlbrLx5tvZSmwkE2jewlH9rEUPuVeBQPNzfdTJe7i3sK9zBmjPHbzb/dkCDyQpCT9skoXVAVVR4pPcIDhQcoiiI7fDt4ffj19HjmLtQKaOPsevmfiRVeYqjt1RzY9E404aFaKhGLxYhGo3PulqT7q7Ozc94dg8PK4nK56OzsJJlMks1m6/YC+P1+4vE4qVRqTgNyTKrkzVyw8Wn+K/AKPHyAi7u3zaloOxNSk2i5MgTrihEok5/OmcBlwF8BGvBRIcT/W5ZRzcNy7ghq0TSNkZERW+1wPizLYnx8HF3XpwSUFGFy3oG/IZ7bzxPbP06qaaf92IHKAb6Z+SYmJm+OvpnzAuct6n05TKJZGo+WH+XB4oPkrTzbvNu4Onw1G73zrKCEoDvxINv7/v1oQPg9jLRcYgvUzdUsRlIsFm2hQkdafO1Q6wWo1xgIIey+IvUEj//ioZdwbfkCbQR4V/df4lIWvkuUWY6XX345mzdvXvDzYfYdwbz7GkVR9gFDwD8BPcDbgdcAFyqK8uVFjebYuT+tKMoLiqI8oyjK/yqKsvLC3DX4/X672EdmiMxFbcFZ7fFCcfH0tg9T8ndxzov/QKg0aD+23bedD7V+iC53F9/IfoM7s3dStk7KzdayUrSK3Je/j79O/DV783vpcHfwu/Hf5Zb4LfMaAW81wzkvfppdB/+NXHATj5z9aYZiF9uxou7u7jmNgOyHG41G6enpcYzAGkN6AVpaWuyudPU8p6VlUlhO1rjMxZCxhXeJ03jJVeXnqf9Z1DilDEWxWFzU8+einhjBTuB5McOBiqIcEEJsX/SLK8rrgB8LIQxFUT4FIIT4k/med6J2BBLDMBgdHbVbCc6HaZqMjEyGUmonEH8lwcXPfgxT9fLznX9N1XvM7pnC5AeFH/Dj4o9pUpu4selGtvm2LeCdrU+yZpYHiw/ys/LP0IXOTt9OrgxdOf8O4CgdE4+y49BXcJkaL2/8Tfq73oBWqdpCcfPljRuGgaZptLe3160pfzIg40u1ze5nm0tkK9TaGNJqrXVJp9NMTEzUvTOQ9UTz1SZ8/7DJr2/U+f7AR3jI7+b3Wz9Ep6d7weMbHh7mggsuYOfOnfMfPAPLkj6qKMoWIcShRZ9g6rl+A7hBCPHW+Y490YYAjqWOaZpWV2BJ13VGR0ePqz6OFl7hwuf/nEKgh8d3/BmGe+pE01/t547sHSTMBBcHLuYNkTfULVq3nhjUB3mo+BC/1H6JQHCu/1yuCF0xZxC4Fo9e4Iy+2+hO7iMb2sqzp/4eeX+3naHR0tIyb5BX7vrqkQ9ea9TWe9Rmgc2U+itrVWoTBwA7S82yLPs8sm+xNBy1RkKea6WNRCqVWpCbqFa6ej4CiR/xiepeYq44t3Z8bMEuolVpCBqJoijfB+4UQtw+37ErYQhg8oIeGxurO8tApqjJFE1Ja/pJzn3x02TC23hi+8ewXFPdDrrQuTd/Lw+VHiKgBHhj5I3sDuyesdp1PWEKk2e1Z9lX2kef3odP8bE7sJtXB19Ni7t+/ffW9JOcefBLeI0cB3tv4HDPr1PRJ9NfZVrofBNAsVjE7/fT0dGx5oPttbUnctKXUh6BQMAutHS5XPbPUidraRSkYdA0zS4ulJlrUp75RGehCSGYmJggk8nUFfAXQjA2NnZcbHCWgym99BH+tMnkjaHXc0XktQsa25o1BEfrEGZapn1cCPHdo8d8HNgNXD+TC+roMbcAtwBs3Ljx/P7+/kWNZymGABZuDCqVCqOjo3aVr6Qz+Qhnvfw5ks3n8NTpf4RQjx/PsD7Mntwe+vQ+Nno28sbIG6c0Qlkv5M08j5Uf45HSI2StLC2uFi4LXsbuwG4Cav054G6jxOn9/0Xv+I/JBzbw7GnvJxfcZHe6qqd/tGVZlEolmpqaaGlpWVWpsvUi214ahjGl4b1sfSkb3q/EyrxWWbZYLFIsFu3iyMVqBS12HLKmqB5ZCcMwGBoawu/3z3tNxHL7+Wbi8/w0GOZDbX9cVzdDyZo1BPOhKMrbgPcCVwkhSvU8Z6k7AmBJaVgyO0gGE+dD5h1P1yXqGfsROw99ibH4hTx92h/MaAykIua9hXvJWlm2+7ZzTfgauhfhX1xLmMLkxcqLPFZ+jP2V/VhYbPNu47LgZZzhO2PBu6P2icfYfvhr+PQMh7t/jVc23IhuKbb2TzQanfcGlvUBra2tq6Lh+EIwDINqtYppmiiKQigUIhwOrwl9qmq1iqZpZLNZSqUSiqKcEKNgWRbDw8MYhlGX6092MqxnTuh98W95VyRBr+9U3h1/X92f/0lpCBRFuRr4R+ByIUSi3uctxRCk02nGxsaW3Cx+oTuD2YzBxpG72d73H4zHzuPpbR/CUme+uKuiyr7iPh4oPkBZlNnu285VoavY5N206PewGhnRR3hKe4rHy4+Ts3KE1TDn+8/nouBFi6qz8FVTbD98Gx2px8gFN/H81vcuSC1UIluJdnZ2rmhv2YUgV9aWZeHxeOwmR8uh3nqiMAyDUqlEJpOhXC7b2lrL9X4Mw2BwcHBGpeHpLMRFFCwPc+Tg/+VvW2Pc3HRz3WnjJ6sheAXwARNHf/WoEOK98z1vKYYAJoWbkslkw4yBDDDOx2zGoHf0B5x5+Kskm87mqdP/CGuOMvSSVWJfaR/7ivsoiRKbPZu5JHgJu/y7cCtr01edMBI8rT3NU9pTjBljKCic4TuDCwMXssO3Y1E51wiL3vEfs63/dlRL5+CGt9DXdS2GUOpWC5VomoaiKHR2dq76ngGGYdh+dp/PRywWswUIIHfi0wAAIABJREFUTzaq1Sr5fJ50Om1LRSxHvEbTNAYHB+tSKNZ1naGhobqM07ZDX+Hj6jMcDjTzx20frSsp5KQ0BItlqYYAGmsMRkZGqFardaWWzuomGv8xZx78EunIGTx1xh9huOcOUlWsCj8v/5x9pX2kzBRhNcwFgQs4z38eXZ7VrWtvCYsBfYADlQPsr+xnxJhMtd3i2cI5gXPY5dtVd6/nmQiVjrDj0FeJ5w8wET2T/VtupRTorFsttJa1EBSWjdwNw7Cb3YdCoZNy8p8JWceRTCbt1XijK3Dz+Tyjo6PH9QeY7ViZgjoX3mqGzuf+gN/samV38CLe0vSWec/tGIIaGmEIoHHGQNYN1OtLlMbA5/NNCyA/zK5XPk/J38kT2z9WV5s7S1i8VH2Jn5V+xoHKASwsOtwdnOM/hx2+HXS7u1fc/yuEIGWmOFg9yMHqQV6ovEBRFFFR2ezdzJm+MznLfxbNrqX53d1Gia2D32LjyL2YrgAvbvpthtquwLQsNE2rOy1UjrlYLBKJRGhra1uVrhTZv0FVVZqamohGo/h8vhX/vlcKKfc9Pj5OpVIhGAw21HgnEgny+fy8iwghBCMjIwgh5jXGp/V/nW9rD/L/mqJ8IP7BeetfHENQQ6MMgRCCZDLJxMTEko2BzBqA+loOStlrj8czZfUSzz7HOS/+A6bq48ntHyEfqr+UvGAVeFp7ml+Wf8lh/TAAUTXKNt82tni2sMm7iTZX27JPFJqlMWqMMqQP0af3cah6iKyVBSCkhDjddzo7fDs43Xf6grJ+ZkVYdCd+yraBr+PVcwx2XMXLG25C90Tttp4tLS11+/ZlZlA8HicWi62qibV29e/z+YjH44TD4VVpqFYKIYQtD28YxoKbTs2GZVkMDk6qAsw3wddbaObRC5z7y/fz6z3thHw9fCD+gTkTIRxDUEOjDAEcSxNLp9NLrgyV/sF6Aksw6eMcGxuzc7Yl4dIA5x34WzxGgee3vpfR1ksXPJa8meeF6gu8UHmBlysvUzqakBVUgnR5uuhyd9Hp7iTuihNzxYi5YguKMehCJ2tmSZkp+ydpJBk2hpkwJxBMXlcRNcJW71a2eLewxbOFdnd7Q+shooVDbD/8NZoLL5MOb+OFze8kF96CaZoL3gXAscygjo6OutwAJwrZscqyLJqammhubj7pitgajWxTOT4+jtvtbogUeKVS4ciRI3UZl3Q6TT6fn/d1tx75Hw6k9/Kx9lbeEn0LFwVn71vgGIIaGmkI4JgxyGQyS775K5UKQ0NDx9UNzIau64yNjSGEmHJje6sZznnpH4nlX+Bw17W8fMpbEYsJmjLpPkqYCfqqffTr/YwYI4wZY1RFdcpxPsVHUAkSUAOoqKiKinL0j4mJZmloQqNslTEwpjxXRSXmitHt7qbb002Pu4duTzdNatOyrKj9lSSnHrmT7sRPqXqi3Bm8mY7tl4OiLmoXAMcyg7q7u1dN/wC5O5EKt01NTSdlj+rlRNd1xsfHyefzhEKhJcuCZ7NZEonEvMVmpmkyPDw8b8c9l1HiV576AO/obOOQ18eftP3JrIFjxxDU0GhDAMf8eoVCYcnyweVy2c4cqOeik7ro07WMFMvg9P7/4pTRe0lFd/Dsqe9H89VffDIXlrDImBnSVpq0mSZjZihZJUpWibIoYwoTUfNHQSGgBggoAfyqn4ASIOqKEnfFibviRNXo4rJ7FohHL7B56H/ZOHovAANd13Co53re9ZCXf/2VSUNcT+ew6ay2zCC5M1FVlZaWFqLRqNPXYInIgK9MOV0sQgiGh4frShOtt7Zg09D3EKN3cmNPN5cGL+XXo78+43HLZQhWZxrECiAngaGhIUqlUt1ZJTMRCATo6uqyfYTzbSFlF6uJiQm7klFRFITq5oXN7yQbPpUdh77CJU//Ec9veQ9jrZcsemwSVVGJu+PEmb1D12rCZWpsHL2XzUPfxW2WGG67nFc23Ijmaz0qdjYpV7AY3Z9SqYTX66Wzs3PFM4NqDYAUsnP8/40hEong9/un7A4W89kqikJ7ezsDAwN2n/DZCIVC5HI5dF2fcyd3pPN1vHr4u1xb8XIXj3Bp8FLa3PMnizQKZ0cwDdM0GRwcxDCMJbsHcrkcY2NjdWnXwORKI5PJkM1mj8tDDpZH2fXKv9BceJnh1l/hhU1vQ/ec/GqXLrPMxtH72DT8fbxGnkTzuby08a0UQhtrGn9M5cb/v717j46yPvcF/n3mnft9JpmEZCYhoZUtGCgioIhavKAoFlxba2m7W1h0aWtPW7broC21q16Otbaute3ltD2r3e3SXjZSrfW0eipSwe5KUYsYKIJclJCQBJJwyWSSyWQuv/NH+L3NQDIzycy87yTzfNbKkkzemfm9y5n3eX+355ntxOqm7EN8pbQySA4ByR6Ax+PhAFAk8nvW1dWV11LTcDiMrq6urCMIcpVgtl5BY/sL8B9/BsunN+Iiy79grW/tBcdwj0AjiqIgGAyira0Ng4ODeU3Iud1uJJPJnNPaynFgk8mEnp4emM1m9UM6YJuGty55GDPan8eM9t+j8mwzDk7/DDoCHwVKaFVLoRgT/ag/8TKmd74EcyKCbu+leD90O3pdw6m5k8kkrqsaxC31w9WiVr9wCs/fmfv+iVJZGSSXPMq01x6Ph4eAikx+z6xWK9rb2yd80+dyudDX14dYLJZxONFqtcLpdCIajWa8nrROuwkNHX/Ap6Mm/AL78MHQB5hhnjHudk0E33KMwmg0IhQKQQiBWCyW12v5fD54vd5xFZNwOp2oqalRV75IwmDE+3V3Yufc76DfVos57/8YC/c/DHfk/bzaWErs0U5cfPQX+OjbX8BFbZtx1vUv2DnnMeyetRG9rpnqhTMejyMQCEyoGlgymcTAwACqqqrg9/t1CwLRaBSRSAQejwczZszg+sYas9lsmD59OoxG44SKvRARKisrEY/Hsxaz8Xg8am3wsSQVG47VrMAXTh6Elxz4Y98fcyqSUwjcIxiDyWRCKBRCa2srDAZDXis1KioqkEwmEYlEcl7FYrFYUFtbq84bjBwqitjr8dYlDyPUtQ0fbn0Gi/+xEZ0VV+Jw/WpErbnl4y8pIgV/+F1M7/x/CJzZDUEGdFZehWM1K9DnaFAPGxwcRDKZhM/ng9PpTLto3jk7twl+uTIoGAzqtjJIJlGTQ1LlsgO4FJlMJtTV1eHkyZMIh8M5D+NKci9HtnoEMtdTOBzO+LlrnXYzGjr+iLv7jfiuvQ17Bvdgnm3euM5pIjgQZGCxWNKCwUTv1ogIgUAAiUQC0Wg05wuQoigIBAKIRCJq4Wr1okEGHK++AZ0VV6Kx44+Y3vkiqk+9gROVV+Jo7W2IOHKr0KUna6wHtd2vIdj1GuyxLgwZXfgg+K9onXZTWvU2WRDe6XTC6/WOOqGby5yAXBkUCoV0WRkkywxaLBbU19fntSCBFY7BYFAXCkxkc6m8wCeTyYzXCJfLhXA4nHGCOWG041jNLfjU8efwXx+6DC9FXkKTtanoecQ4EGRhs9kQDAZx/PjxvHZwyg9bR0fHuOYeiAgulwsWiwU9PT0X9A6SRjuO1H8CrdNuRGPHHxE6uRW1Pa+j23sp2qpvQI/30lHTW+vFFO9D4MzbqOn5Kyp694EgcMrdhCN1d+JkxRVp2Vfl3bvNZkNVVVVed8565gySw1kAUFNTA7fbXVK7ldk/b9YURUFXV9e4goGsVd7Z2Zlx4lhRFHi9Xpw5cybjTYDsFXyxX8HXbd14feB1LHUsHe8pjUvpXCFKmByz7+zszCsVhaIoqKmpQXt7e9YJpvPJ5Y2RSASnT5+GoihpwWTI7MPBhs/ig+C/ou7kFtSdeAXzDz6BmMmLjsA1OFGxGGHHDF0mlq2xblSd/juqTv8dvvABGJDCgCWA90N3oD3wUQxa09NLywBgtVpRUVGR14R9KpVCf38/vF6vLoVk5DCQz+dDRUWF7stTWWZyzkiu9sv182K322Gz2bJ+r51OJ3p7ezP2HuImF9qrrsOKk6/g1x++Gq9GXsUi26KilqzlT2WO5GRPV1dXXqkojEYjamtr0d7ejqGhoXHd5RoMBrjdbthsNpw+fVodZhh5cYmbnPggdDuOBm9D5Zl3EOrahukdL6Kx4w+ImivQ5V+I0+4mnHHPQtxUhBQKQsAa64Y/vB++8AH4+g7AMXgCANBnq8PR4G3o8i8cNSjJIaBCBADgn2vyA4EAPJ7i7HAeiwxAFosF06dPL5mdyiw7n88HAOOqWyInjtva2jIW+jEYDPD5fFk3mbXU3oq6E1vwxX4jvmwZxPb+7VjhWjGxE8oBB4Jx8Pl8iMfjeaeiMJlMas+AiMY9EW0ymVBdXY1oNIpTp04hFotdkNZakIJu/wJ0+xeowzHVp98aDgznduX22eoQdjQi4qhHxFaPqDWAQXMFkkoOF2CRgnXoNGyDXbDFuuCMHoer/xicA62wxs8AAOKKA6fds9BWfSO6fZdhwHbh8k65MiuZTMLhcCAQCBRk/F72KmpqajQvJBONRpFIJFBVVQWv18vDQJOQz+dTU8/kGgwsFgs8Hg8ikUjGwG+326EoSsZewaAlgBOVV2LJidcxf+b1eL3/dVxtv3rC55MNB4JxkDsKE4lEzmUqxyJXBclgMJEhA5vNhtraWgwMDODMmTPq3MP5H664yYWOqqXoqFoKSsXhibwPf3g/vH0HURHeh2DPf6cfr9iRMDqQMFjVoEAiBUIKxsQATIkIjMkBEP65tC1FRkRsIZzyzkHY8WGcds9GxB4CxkgwJ0snEhHcbjccDkfBcujInbl1dXWarsiRk8G8Gmhq8Pv9SKVS6OnpyXkUwOv1ore3F0KIvHsFR2tXobbndazrN+LLxiS29W/DIiya0Llkw4FgnIgINTU1aGtrG9cKoNFYrVZ1IjrXvETnMxgMcDqdsNvtagm/wcHB9BVGIwiDCWfdF+Os+2L1MVO8D47ocdhiPbAOnYJl6DSMyQEYk4NQkoMACIIIAgYkbEHEjU7EjU7EzH5ELVWIWgKIWgJZJ6VTqZRaN9dsNqOiomLC5z0auVPY4XCgqqpK0zX5clNYMBgc9xJEVrrk0u/e3t6ccpCZTCb4/X6cPXs244RwLr2CiGM6ur3zcHnndiy6aBl2DuzExbh41GPzxYFgAgwGA4LBIFpbW/PefSyDQUdHx6h38+Npk9PphMPhUAt99/f3g4hgsVgyvm7c5MJZ0yycnehJZJBMJhGPx9UPvMvlUuvmFvp9BgYG1MLyWl2IZQ1dj8eDQCDAk8FTzMhRgFxzkHk8Hpw9ezbjMtHc5wpWYuH+R/DpQQf+DuANegPX4/qJns6YeGfxBMndx/IuNx9yiEdumMoHEcFms2HatGkIBoPw+XzqUFY0GkU8Hs/r9bNJpVKIxWIYGBjAwMAAEokEXC4XampqEAqF4PV6Cx4EYrEYYrEYamtrNU0XMTAwgKGhIYRCIdTU1HAQmKJkQkqj0Zi2038siqLA7/cjGo1mPG5kr2Asp92XIGxvwGUntmOxbTHexbvoHOwc9zlkw5/cPJjNZoRCIRw7dgwGgyGvC4EMBu3t7bBarQW5qMjqZ263G/F4HLFYDJFIRB3GICJ117TBYBj3BTSZTKo/QggIIaAoCmw2G6xWKywWS1Fz58v1+RaLBTU1NZqNycu7Q6/Xq649Z1Oboiiora3FsWPHkEgksn4/XS4Xzpw5k3HoR/YKTp06NXZPgwjHam7BnPd/jDtSNTiMKkQSkXxP5wIcCPJktVpRV1eH1tbWvEsG2mw2hEIhtLe3QwhR0IuoDApOpxOpVArxeFwNDnKteyqVUoOBDBRj/ZeIoCgKLBYLzGazWohHURRN7sjl0lCv1wu/36/Z/gA53BYKhfKuW8EmF7PZrA4JZ/uuy15BtqEfu92eNWB0Vi7BzNbfYO6Jbfi0Zw0ucl6U97mcjwNBAdjtdtTW1qKjoyPv2sdWqxWhUAgdHR05Fb+eCIPBAIvFAovFknYxk3f2qVRKvcM//3myFzGRHkShyCG0adOmaXYxliuCeC6gvNntdlRXV+PkyZNZVxK5XC6cPn06a6/A4/Fk3G0sDCa0Vd+EDx//LTz25Xmfw6jtKMqrliG3243q6mr09fXlnTHQYrEgGAyqhcq1oigKjEYjzGYzLBYLrFZr2o9Mi63VXf/5ZFFyk8mE+vp6zYJANBrF4OAggsEgzwUweL1euFwuNW3IWAwGA/x+f9bvsExRn0qlxjymbdoypMiIWeH/HvOYfHAgKCCfz4fKykpEIvmP4cluqNFozPqBKwfxeBz9/f2oqKhAbW2tJnV7k8kk+vr6YLVa0djYWFLF7Jl+iAjV1dUgoqyLL+QQUqaLvKIocLvdGRedDJk86Ki8Ch/u/zuUofCE2z4WDgQFVlFRAa/XW5BgINNROBwORCIRzXKTlxK5NyCVSiEUCmm2KigajSIajaKmpga1tbXcC2Bp5HczGo1m/F4qigKfz5e1V+ByubLWK2ituQUGkYS9Z++E2z0WDgQFJtcdOxyOCRW7OJ+sXev3+9Hf35/38tLJRPYCfD4f6urq8s49lItUKoVwOAyz2YyGhgbOFMrGZLPZchoBcLlc6tzbWGSvIFMhrD5HAzaHHkFf7VUTbvNYOBAUgdx9bLVaCzKsQ0Tw+/2YNm0aotFo3vsWSl0qlVJ7QKFQSLNVQbFYDP39/aiurkYoFOIUESwrmRwx0wU8116B0+nMeqMXU4qTN4sDQZEYDAZ1LDvbxpJcOZ1O1NfXg4jQ398/JYeKBgcH1R3CoVBIk16AnIQmIjQ0NOhaw5hNLnKz2dDQUMY7fpfLpa7GG4vZbFZTWWuNA0ERKYqCYDAIRVEKFgzkJLLH45lSQ0XxeByRSERN2+z1ejXpBcTjcfT19cHv96O+vl6XymVscrNYLKiqqso4FCw3dma7yHs8HiQSiUI3MauSmAEjog0AngAQEEL0TOQ14vE4jh8/rulyy1wJIdR16IW6uBkMBni9XvUuQ+uCK7mS5z7WHbbcGCYDnJZ5+/v7+6EoCtcLYHmTWUczFabxeDwIhzOv+JG78XPZvVxIugcCIqoDsAxAaz6vc/z4cbhcLjQ0NJRkt17u5gVQ8PYlEgn1YltK5y6EQG9vL7q7uy/oNieTSQwODkJRFFRXV2uasVOmiJDLfTlFBMuXHCI6duzYmIVpLBYLbDZbxoJURASPx4Oenh5NA0Ep3EY+CeB+AHkNeA8ODqKioqKkLoQjyZw+AAo+ti83gQHIOE6pNfmhHvn/JJlMIhKJIB6PIxAIoL6+Pu/d2OMxMlFcdXU1BwFWMLKyXqYFIj6fL+tiD5mMTsvvsq49AiJaCaBdCLEn24WAiO4GcDcA1NfXj3VMoZtYUDIYxOPxjIUrJoKIYDabkUqlkEgkSma4SJ7j0NAQhoaG1OpqDodD0/ZxigimBZ/Ph97e3jGHdmw2G0wmU9a0Ey6XC+FwWLMhy6J/E4noz0S0b5SfVQAeAPDNXF5HCPFTIcQCIcSCQCBQ3EZPwKlTpzBv3jzMmzdPTQEtfx95BzCyZ3B+xL/rrrtw8ODBjO/zk5/8BJs2bRrz7waDQU0FMTJvUDbbt2/H6tWrMx7T3NyMrVu3Zn2tkVKpFFKplJq9UfYAtAwCnCKCaUVRFFRVVY3ZKyCinJeSTqkegRDihtEeJ6I5ABoByN5ACMBuIlokhDhR7HZJT249hHuXzcz7dSoqKtDc3AwAeOihh+B0OrFhw4a0Y2Qit5E9g5HFK372s59lfZ977rknp/bIgJBKpZBMJtUPVT4X4ObmZuzfvx/Lli3LeJw8T1mC02g0orq6esLvO1GyWI3T6URVVZUmaSkYG1kgarTlzw6HA93d3RlHBYxGIxwOR8bJ50LSbexACPEPIUSVEKJBCNEA4DiA+VoGAQD4/quHi/r6R44cQVNTE77whS9g/vz56OzsxN13341FixZh/vz5eOyxx9SL9LXXXos9e/aohc8feOABLFiwANdccw26uroAAA8++CB+8IMfqMc/8MADWLJkCZqamrBz504Aw6thPvGJT2DBggVYs2YNrrnmGhw4cABGo1HtJaRSKfzpT3/CpZdeimXLluHFF19U2/zWW2/huuuuw5VXXokbbrgBR44cQTQaxeOPP47Nmzdj8eLF+P3vf5923PXXX49Dhw6pNQlkamq9EtSdnyKCgwDTiswuIIeAz6coCjweT9alpC6XS7OlpPoPIpeB/fv343Of+xzeeecdBINBPP7449i1axf27NmD7du34+DBgxd0A3t7e3H11Vdj165duPzyy/H000+P+tpCCOzYsQPf/va38a1vfQsA8OMf/xjV1dXYtWsX7rvvPjQ3N6v1A+QFOh6PY/369Xjuuefw8ssvo7OzUw0SM2fOxCuvvIIdO3bg/vvvxyOPPAKr1YqvfvWruPPOO7Fjxw6sWrUKF110EV555RXs3LkTX//61/HYY4/pevEHhnsB4XBYTRTHKSKYHiwWC7xe75j7h3K5yFssFhiNRk32CpXMYOm5XoEmntx6KK0n0PC1lwAA66+/qCDDROf70Ic+hIULF6q/b9q0CT//+c+RSCTQ0dGBQ4cOYdasWWl3DzabDcuXD+cenz9/Pl5//fVRX/u2225Tjzl27BgAYMeOHeqw1Ny5czF79uy05xARDh48iJkzZ+Lii4eLYX/qU5/Cr3/9axiNRkQiEXz+85/HBx98oD5H1iCQw1pEhGg0invuuSftOD3Jymu1tbWarkRibDQVFRXo7e0dtXaxrAeSab8AEcHr9WauYFYgZdkjuHfZTLQ8vgItj68AAPXfxQgCANIqFB0+fBjf//73sW3bNuzduxfLly9HLBZTL64y+o9cZ5yprqkcP1QURb3DmMjy1JFVxx5++GHceOONaG5uxnPPPYdYLAaj0ZhWmIaI8OCDD2LZsmV455138Oyzz+q2mS+RSCAcDsPhcHAvgJUMo9GIysrKMSeOc5k0lquGip1OpiwDgZ7C4TBcLhfcbjc6OzuxZcsWAP+8EBeiK7hkyRI899xzAIB9+/bhwIEDFxwza9YsHDlyBEePHoUQAps3b1b/1tvbi9raWgDAr371K/Vxl8uVlmlxrOO0IlNUx+Nx1NXV8YogVnI8Hg8MBsOo32m73Z5TrQI5aVxMZR8I1l9f+PqfmcyfPx+zZ89GU1MT7rrrLixZsiTt7yM3h030LuCLX/wiOjo6cNlll+F73/seLrnkkgvK6tntdvzwhz/EypUrcd1116GxsVH924YNG7Bx40YsXbo07TlLly7F3r17sWjRIjz//PNjHqeFwcFB9PX1wefzoaGhIWNdWMb0oigKAoHAqL0CmSYml0njYs8T0GTMYLlgwQKxa9eutMcOHDiAWbNm6dSiwpMpKSayMSyRSCCRSMBqteLw4cNYsWIF9u/fr9vd8qFDh+D3+wvyWnJjmKwdy0niWKlLpVJoaWlRl1KPFIvF0NbWlrHsqhAC7e3tMBqNOHnyJBYuXIimpqYJtYWI3hZCLDj/ce5Hlyi5D+D8vQa5iEQiWL58ubrD+Ec/+tGkHzIRQmBgYABEhGAwqGluIsbyYTAYUFlZic7OzgvKnY530rhYJvfVYYojInU7eiKRyDmpnNfrxRtvvKFBC7URjUaRSCRQUVEBn8/H+YHYpONyudDT0zPqBd/r9aKrqyvjzVqxJ43Lfo6g1MkJ5FJMKldssVgM4XAYdrsdjY2NnCmUTVpEhEAgMOq+Ark0NFvtY4fDoWYwLjTuEUwScqhIjv/LJZxT0dDQEGKxGGw2GxoaGjSpUsZYsTmdzlFrDSiKApfLhYGBgYyfdafTyT0C9s+hoqnaOxgaGlILd9TV1aG+vp6DAJsyiAiVlZWj9gpy3WlcrO8D9wgmIUVR1LXJ8sNTCimnJyoWi2FoaAgWiwX19fWw2WxTtrfDypvT6VT3Co0c5rRaraM+PhIRFW3H/OS9epQgIsJnPvMZ9fdEIoFAIIBbb70VAPDUU0/hS1/60gXPa2howJw5c9S01V/5yldyei85dyCDwmRaCiyEQDQaRTgchslkQn19PaZPnw673c5BgE1ZcgXR+b0CWcQp254Cmdm00LhHUEAOhwP79u1DNBqFzWbD1q1bEQwGc3ru9u3bUVlZOe73HJluOpFIFCTddDElEgl1W73X64XH4+G9AKysuFwutXzryO+p3W7PukRUpngptNK8WkxiN998M156aTiJ3aZNm/DJT35Sk/eVAUH2EGSq6VIg9wD09fUhkUiguroaM2bMQFVVFQcBVnYMBgP8fv8FvYKRewq0NjV7BH/6GnDiH4V9zWlzgJsfz3rY6tWr8cgjj+DWW2/F3r17sW7dOvz1r3/N+rxrr71WHRtcs2YN7r333gk1c2QPQfYSRv5NSzIQCSHg9XrhdDp58pcxAG63Gz09PRcUp/F4POju7tZ8A+jUDAQ6mjt3LlpaWrBp0ybccsstOT9vokNDY5Epo2URbFmpTMp1c1quZFWyka8vM5aaTKaCnhtjk53RaITX61X3yUh2u12Xub6pGQhyuHMvppUrV2LDhg147bXXirotPBcytbSiKBdUKBtt6EgGh9GChPyAjvZBle8zMlU1Y2xsXq8XZ86cSXvMaDTCZrMhHo9rWlVvagYCna1btw4ejwdz5szBa6+9pndzVLIXIC/S8i5+rB95zMigMLIewfk/jLHcmc3mUWsbezwenDx5kgPBZBcKhbB+/fpR//bUU0/hhRdeUH+XOYFGzhHMnTsXv/zlL4veTr6AM6Yvv9+Ptra2tEAwMq+QVt9PDgQFNLJoi7R06VI1X//atWuxdu3aC45paWkpbsMYYyXJZrNdkHZiZDEarVbV8UAuY4zpRKadOL9kpdvtLlqCudFwIGCMMR05HA4QUdoiDIvFcsFjxcSBgDHGdKQoCjweT9oGMzk8NDQ0pElIeeq8AAAJaUlEQVQbOBAwxpjOPB7PBXWJtRwe4kDAGGM6s1gssNlsaT0Aq9UKg8GgyfAQBwLGGCsBfr8/LfuowWCA3W7XZHiIA0EBZUtDLa1atQqLFy9Oe+yhhx5CMBjEvHnz0NTUhD/84Q+atJkxVhrsdruaEkbSaniIA0EBjUxDDWDUNNRnz57F7t27cfbsWRw9ejTtb/feey+am5vx7LPPYt26dSWTPZQxVnwGgwFerzdtKalWq4c4EBRYtjTUv/vd7/Cxj30Mq1evxjPPPDPqa8yaNQtGoxE9PT1Fby9jrHS4XK60SeNiF62XpuTO4u+89R28d/q9gr7mxf6L8dVFX816XLY01Js2bcKDDz6I6upq3HHHHdi4ceMFr/Hmm2/CYDAgEAgU9BwYY6VN1iQYmXTO5XKhs7NTrVVeDLr3CIjoy0R0kIjeJaLv6t2efGVKQ33y5EkcOXIEV111FWbOnAmj0Yh9+/apf3/yyScxb948bNiwAZs3b+Y8QIyVofMnjbWo4aFrj4CIrgWwCsBcIUSMiKoK8bq53LkX01hpqDdv3owzZ86gsbERABAOh/HMM8/g0UcfBTA8R7BhwwZd2swYKw2yJrFMOqcoipqaulj07hHcA+BxIUQMAIQQXTq3pyDWrVuHb37zm5gzZ07a45s2bcLLL7+MlpYWtLS04O233x5znoAxVp4URYHb7U6bNHa73UVdRqp3IJgJ4GoiepOI/kJEC8c6kIjuJqJdRLSru7tbwyaO32hpqFtaWtDa2oorrrhCfayxsRFutxtvvvmm1k1kjJUwj8eTVmbWarUWdeVQ0YeGiOjPAKaN8qcHzr2/D8AVABYC+C0RzRCjnLEQ4qcAfgoACxYs0L6WWw6ypaFub2+/4O+7d+8GAFx++eVFbRtjbPKwWq0wGo1IpVJquVeLxYL+/v6ivF/RA4EQ4oax/kZE9wB4/tyF/y0iSgGoBFDat/yMMVZERKSWspQ1jV0uF06cOFGU99N7aOgFANcBABHNBGAGwIvnGWNlz+l0pu0psNlsavGaQtN7H8EvAPyCiPYBGAKwZrRhIcYYKzcWiwVms1mtXmY2m4u2lFTXQCCEGALwb3q2gTHGSpXf70dXVxeMRiOICC6Xqyjvo/fQEGOMsTHY7fa0nGMul0vdZ1BIHAgYY6xEmUymtDoFRqOxKPMEHAgKSFEUzJs3Dx/5yEcwf/58/O1vfwMwvIegqanpguPXrl2LxsZG9Tmvvvqq1k1mjJU4r9db9JoEek8WF01ra2vazrx8Wa1W1NfXZzzGZrOhubkZALBlyxZs3LgRf/nLXzI+54knnsAdd9yB7du34+6778bhw4cL1mbG2ORnt9uLnoZ6ygaCwcFBOJ3Ogr3eaJvFMgmHw/D5fDkfv3jx4lE3nDHGypvRaITD4UhLRFfw9yjaK5ehaDSKefPmYXBwEJ2dndi2bVvOz3355Zdx2223FbF1jLHJyuPxoKOjAwZDcUbzORAU0MihoZ07d+Kzn/1sWprp0dx33324//770dXVhTfeeEOLZjLGJhm73V7USmU8WVwkixcvRk9PD7IlyHviiSdw5MgRPProo1izZo1GrWOMTSaKohS1kD0HgiJ57733kEwmUVFRkfVYg8GA9evXI5VKYcuWLRq0jjE22Xi93rSMpIXEQ0MFJOcIgOGiEk8//TQURQEAHDx4EKFQSD32ySefTHsuEeEb3/gGvvvd7+Kmm27SrtGMsUnBZrPBYrEU5bWnbCCwWq3jXumT7fWyGZkgaqSGhoZRqwt9/OMfT/v99ttvx+233z6xBjLGpjRFUYqWYmLKBoJsa/4ZY2yyGc+S9PGYsoGAMcammmLkGQJ4spgxxsrelAoEXMqg9PD/E8ZK35QJBFarFadOneILTwkRQuDUqVNFK6bBGCuMKTNHEAqFcPz48awbuJi2rFZr2rJZxljpmTKBwGQyobGxUe9mMMbYpDNlhoYYY4xNDAcCxhgrcxwIGGOszNFkXGVDRN0AjundjgmoBNCjdyM0VG7nC/A5l4vJes7ThRCB8x+clIFgsiKiXUKIBXq3Qyvldr4An3O5mGrnzENDjDFW5jgQMMZYmeNAoK2f6t0AjZXb+QJ8zuViSp0zzxEwxliZ4x4BY4yVOQ4EOiCiDUQkiKhS77YUGxE9QUTvEdFeIvo9EXn1blOxENFyIjpIREeI6Gt6t6fYiKiOiLYT0QEiepeI1uvdJi0QkUJE7xDRi3q3pVA4EGiMiOoALAPQqndbNLIVQJMQYi6AQwA26tyeoiAiBcCPANwMYDaATxLRbH1bVXQJAP9TCDELwBUA/kcZnDMArAdwQO9GFBIHAu09CeB+AGUxOSOEeEUIkTj36xsApmoq0kUAjgghPhBCDAF4BsAqndtUVEKITiHE7nP/7sPwxTGob6uKi4hCAFYA+E+921JIHAg0REQrAbQLIfbo3RadrAPwJ70bUSRBAG0jfj+OKX5RHImIGgBcCuBNfVtSdN/D8I1cSu+GFNKUSUNdKojozwCmjfKnBwB8HcCN2rao+DKdsxDi/5475gEMDyX8Rsu2aYhGeawsen1E5ATwOwD/LoQI692eYiGiWwF0CSHeJqKlerenkDgQFJgQ4obRHieiOQAaAewhImB4iGQ3ES0SQpzQsIkFN9Y5S0S0BsCtAK4XU3e98nEAdSN+DwHo0KktmiEiE4aDwG+EEM/r3Z4iWwJgJRHdAsAKwE1EvxZC/JvO7cob7yPQCRG1AFgghJiMiatyRkTLAfwHgI8KIaZs+TgiMmJ4Mvx6AO0A/g7gU0KId3VtWBHR8B3N0wBOCyH+Xe/2aOlcj2CDEOJWvdtSCDxHwIrtfwNwAdhKRM1E9H/0blAxnJsQ/xKALRieNP3tVA4C5ywB8BkA1537f9t87m6ZTTLcI2CMsTLHPQLGGCtzHAgYY6zMcSBgjLEyx4GAMcbKHAcCxhgrcxwIGGOszHEgYIyxMseBgLECOJeXf9m5fz9KRD/Qu02M5YpzDTFWGA8CeISIqjCchXOlzu1hLGe8s5ixAiGivwBwAlh6Lj8/Y5MCDw0xVgDnssvWAIhxEGCTDQcCxvJERDUYrrOwCkA/Ed2kc5MYGxcOBIzlgYjsAJ7HcO3eAwD+F4CHdG0UY+PEcwSMMVbmuEfAGGNljgMBY4yVOQ4EjDFW5jgQMMZYmeNAwBhjZY4DAWOMlTkOBIwxVuY4EDDGWJn7/7spmTSZ1M/NAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# plot the posterior\n",
    "plt.figure()\n",
    "plt.plot(X, y, \"+\")\n",
    "plt.plot(Xtest, m_mle_test)\n",
    "plt.plot(Xtest, m_map_test)\n",
    "var_blr = np.diag(cov_blr)\n",
    "conf_bound1 = np.sqrt(var_blr).flatten()\n",
    "conf_bound2 = 2.0*np.sqrt(var_blr).flatten()\n",
    "conf_bound3 = 2.0*np.sqrt(var_blr + sigma).flatten()\n",
    "\n",
    "plt.fill_between(Xtest.flatten(), mean_blr.flatten() + conf_bound1, \n",
    "                 mean_blr.flatten() - conf_bound1, alpha = 0.1, color=\"k\")\n",
    "plt.fill_between(Xtest.flatten(), mean_blr.flatten() + conf_bound2, \n",
    "                 mean_blr.flatten() - conf_bound2, alpha = 0.1, color=\"k\")\n",
    "plt.fill_between(Xtest.flatten(), mean_blr.flatten() + conf_bound3, \n",
    "                 mean_blr.flatten() - conf_bound3, alpha = 0.1, color=\"k\")\n",
    "plt.legend([\"Training data\", \"MLE\", \"MAP\", \"BLR\"])\n",
    "plt.xlabel('$x$');\n",
    "plt.ylabel('$y$');"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}