labs/lab10-hebbian-learning.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Cvičenie 10: Hebbovo učenie"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Pri trénovaní neurónových sietí je vždy kľúčová otázka spôsob aktualizácie váh. Najčastejšie sa na to použije algoritmus backpropagation alebo niektorý z jeho variantov. Algoritmus je síce účinný a zjednodušuje spôsob trénovania neurónových sietí, má niekoľko nedostatkov a tým hlavným je to, že váhy sa aktualizujú postupne z posledných vrstiev smerom ku vstupnej vrstve. Moderné výpočtové zdroje umožňujú paralelizovať niektoré výpočty, ale efektivita trénovania stále nepribližuje k teoretickým maximálnym možnostiam.\n",
    "\n",
    "Dnes uvidíme, ako funguje alternatívny spôsob aktualizácie váh neurónov, ktorý je založený na teórii učenia biologických neurónov - Hebbovo učenie. Hebbovo učenie sa aplikuje pri nekontrolovanom učení a jeden z najčastejších prípadov použitia je analýza hlavných komponentov, teda získanie prvotných informácií o datasete. Na dnešnom cvičení Hebbovo učenie použijeme na získanie vlastných vektorov nejakej množiny dát."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1. Vlastné hodnoty a vlastné vektory\n",
    "\n",
    "Vlastný vektor (*eigenvector*) je určený pre lineárny operátor, ktorý keď aplikujeme, smer vlastného vektoru sa nemení, mení sa iba jeho veľkosť - vektor teda meníme iba násobkom skalárneho čísla. Vlastná hodnota (*eigenvalue*) je číslo, ktorým potrebujeme násobiť vlastný vektor pri aplikácii operátora.\n",
    "\n",
    "Pre datasety majú vlastné vektory ešte jeden špeciálny význam: sú výstupom analýzy hlavných komponentov. Cieľom takejto analýzy je nájsť množinu kombinácií pôvodných príznakov, ktorá zachováva najväčšie množstvo informácií a zároveň má menej dimenzií ako pôvodná množina príznakov. S vlastnými vektormi to súvisí tak, že vlastný vektor kovariančnej matice popisuje zároveň aj hlavné komponenty."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1.1. Generovanie datasetu\n",
    "\n",
    "Aby sme poukázali na všeobecnosť Hebbovho učenia, nepoužijeme statický dataset, ale náhodne ho vygenerujeme pomocou knižnice `scikit-learn`. Aj keď generovanie bude náhodné, nastavíme niekoľko parametrov aby sme zabezpečili opakovateľnosť našich pokusov.\n",
    "\n",
    "Najprv si naimportujeme všetky potrebné moduly:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "\n",
    "from sklearn.datasets import make_blobs\n",
    "from sklearn.preprocessing import StandardScaler\n",
    "\n",
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Knižnice `numpy` a `matplotlib` už poznáte, a niektoré prvky sme už použili aj zo `scikit-learn`. Dnes najprv použijeme metódu `make_blobs`, ktorá nám vygeneruje náhodné body na základe istých nastavení, a triedu `StandardScaler`, ktorá slúži na škálovanie údajov tak, že vymaže priemerné hodnoty a upraví príznaky na jednotkovú varianciu."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "np.random.seed(1000)\n",
    "\n",
    "X, _ = make_blobs(\n",
    "    n_samples=500,\n",
    "    centers=2,\n",
    "    cluster_std=5.0,\n",
    "    random_state=1000)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Metóda `make_blobs` nám vygeneruje náhodné body, ktoré zorganizuje do zhlukov. Keďže my ale pracujeme s nekontrolovaným učením, očakávané výstupy (zhluky) nepoužijeme. Pri volaní funkcie nastavíme počet vygenerovaných bodov (`n_samples=500`), počet centier zhlukov (`centers=2`), štandardnú deviáciu jednotlivých zhlukov (`cluster_std=5.0`) a nastavíme aj generovanie náhodných čísel pre generovanie bodov (`random_state=1000`). Podrobný popis týchto parametrov nájdete v [dokumentácii](https://scikit-learn.org/stable/modules/generated/sklearn.datasets.make_blobs.html)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1.2. Úprava dát\n",
    "\n",
    "Po vygenerovaní datasetu potrebujeme ho ešte upraviť a prvou úlohou je škálovanie údajov pomocou už spomínaného `StandardScaler`. Podrobný popis triedy nájdete znova v [dokumentácii](https://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.StandardScaler.html)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "scaler = StandardScaler(with_std=False)\n",
    "Xs = scaler.fit_transform(X)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1.3. Výpočet vlastnej hodnoty a vlastných vektorov\n",
    "\n",
    "Ďalšou úlohou je výpočet vlastných vektorov, ktoré potom chceme aproximovať pomocou Hebbovho učenia. Pre výpočet týchto vektorov najprv určíme kovariančnú maticu (pomocou knižnice `numpy`) a následne určíme vlastnú hodnotu a vlastný vektor."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[24.5106037  48.99234467]\n",
      "[[-0.75750566 -0.6528286 ]\n",
      " [ 0.6528286  -0.75750566]]\n"
     ]
    }
   ],
   "source": [
    "Q = np.cov(Xs.T)\n",
    "eigu, eigv = np.linalg.eig(Q)\n",
    "print(eigu)\n",
    "print(eigv)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Kód by mal vygenerovať dve hodnoty a dva vektory, pričom vektory sa vypisujú v stĺpcoch. Hebbovo učenie nám pomôže pri získaní toho hlavného komponentu, teda vektoru s vyššou vlastnou hodnotou (v našom prípade je to druhý vektor `[-0.6528286, -0.75750566]`).\n",
    "\n",
    "Vektory vieme vizualizovať aj pomocou `matplotlib`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXkAAAD4CAYAAAAJmJb0AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAgAElEQVR4nO2df5Bc1XXnv2daT6JHGI0IExs6GqRQWBhZYSZMQLY2VMRiREyAMRjLxOz6DyeErO0tFGqSwWEtyUWMylpWVHkdbJIldpVtLH6YiWQRi9gQe5cY7JFnhJgYJfwUNBQoRqMYzUjqmbn7R/dtvX597333/er3+vX5VKk086b7vft+nXvu+UlCCDAMwzD5pCvtATAMwzDJwUKeYRgmx7CQZxiGyTEs5BmGYXIMC3mGYZgcsyDtAbg544wzxPLly9MeBsMwTFuxd+/efxdC9Kr+likhv3z5coyNjaU9DIZhmLaCiF7R/Y3NNQzDMDmGhTzDMEyOYSHPMAyTY1jIMwzD5BgW8gzDMDkmU9E1DMN0DqPjZWzbcwCvT83grJ4ihtevxNBAKe1h5Q4W8gzDtJzR8TJu++5+zFTmAADlqRnc9t39AMCCPmbYXMMwTMvZtudAXcBLZipz2LbnQEojyi+syTMMkzhe00x5akb5udc125nwsJBnGCZRVKYZAqBqV3RWT7GlY+sEWMgzDBOYIE5TlWlGAE2CvugUMLx+ZWJj7lRYyDMME4igTlOdCUYAKPUUObomYWIR8kR0H4A/APCWEOL9tW2bAfwxgEO1j31OCPFoHMdjGCY9TE5TlZDW2eBLPUU8OXJpYuNkqsQVXfN1AFcotm8XQvTX/rGAZ5gcoNPMdduH169E0Sk0bGPTTOuIRZMXQvyYiJbHsS+GMZHVBJqsjisJdJq5zmkqr0OnXJ+skbRN/jNE9F8BjAG4VQhx2PsBIroJwE0A0NfXl/BwmHYmqwk0WR1XUgyvX9lwvoC/Zj40UMrltWgHkkyGugfAOQD6AbwB4C7Vh4QQ9wohBoUQg729ysYmDAMguwk0WR1XUgwNlHDntatR6imCULWt33ntahbiGSUxTV4I8ab8mYj+BsD3kjoW0xkEtQW3iqyOK0lYM28fEtPkiehM168fAfBsUsdiOgOdzTftBJqsjothgJiEPBHdD+AnAFYS0WtE9CkAXyKi/UT0DIB1ADbGcSymc8lqlEZWx+VldLyMtVsfx4qR3Vi79XGMjpfTHhLTAuKKrrlBsfn/xLFvhpGkEaVhEzWTpegR3Xg7zTnMnISEUFWQSIfBwUExNjaW9jAYBkBz1AxQ1dCz6mQ0jXfbngOckJRjiGivEGJQ9TcuNcwwGtotasY03k50DjNVuHYNw2hoN8GoG1d5agYFIswpVu3t7BzupAS0KLAmzzAa2i1qRjcuApQCPovOYVukaao8NQOBkz4GdiY3w0KeYTS0S9SMRDVeXd32AlFmfQs2tJspLU3YXMMwGrIUNWODary6DkzzQmT2PGxoN1NamrCQZxgDtpmdWbEPe8e7duvjgYqJtQtBi6R1MmyuYZiIZNk+3G4mJ1vyel5JwJo8w0QkaBONVtIqk1OrVzLtZkpLExbyDBORIPbhNMw6SRcTSyublouk2cHmGoaJiG2oZZbNOlHgSJdsw0KeYSIwOl7G9InZpu0q+3BehSFHumQbFvIMExKpmR+erjRs7yk6yhj0vArDdksa6zTYJs8wIVFp5gCweNECpa3YNuwvK+GYOrzjW3deLx7eWw7UDpBpHazJM0xIgmrmNmF/Wbfbq8b38N4yrruwxO0AMwpr8gwTkqAJOTZhf0mHY0ZdJejG98Rzh7hkcUZhId9B+L3gWTcTZI3h9SuV9dtNZgq/sD9TJcm1Wx+PdE/iCHU0rV74+ckmbK7pEPzMAFk3E2SRoYES7rx2daxmCpOzMuo9iSO6Rze+nm6Hn5+MwkK+Q/B7wfMa3pc0QwMlPDlyKV7aeiWeHLk0suaqstu7iXJP4oju0fkVhAA/PxmFzTUdgp8ZQFetsN3D+1pBHGYK9z6WFB2c4nQ1hWZKwt6TOIp66fwKG3dMxDpWJj5iEfJEdB+APwDwlhDi/bVtpwPYAWA5gJcBfEwIcTiO4zHBMZWdLU/NaOuOc6yzmTjs3N59TM1UhTsRoGrBHPaeqHwIADB9Yhaj42Xr8ar8Croesvz8pE9c5pqvA7jCs20EwA+FEOcC+GHtdyYl/MwAAtUGE2441tkfWzPX6HgZa7c+juUju3HObY9i+churN36eF2DV8XbqwR8lHsifQg9Radh++HpSmT7OVeFzC6xaPJCiB8T0XLP5msA/F7t528A+CcAfxHH8ZjguJfZOo1eoOo8zGt0RBLRHzZ2bq+mLlvxSa1fJeDdFIgwL4RxzLbnNjRQwrY9B+qrBUnYME2VmWlqupLL56ddSdIm/24hxBsAIIR4g4h+XfUhIroJwE0A0NfXl+BwGLnM1tngSz3F3MY6J1Up0cbOrdPUgapw1TXZlswLgZe2Xqn9e9BzC+qA1U0gKjNT0Slg+4Z+Fu4ZIvXoGiHEvUKIQSHEYG9vb9rD6Qh0vUClEzaPYW9JRQ/ZmCn8nI9zQhhNaX527aDnFqTWjCm0liOy2oMkhfybRHQmANT+fyvBYzEBcMd3A43NnvMa35xUcTCbWHk/IS2/47WVS44enzXejyTKK0hMgjyvBdfyRpJCfieAT9Z+/iSAv0/wWExAZHx3qafYFFWTR20szUqJJqe3FK5DAyVMbLocd2/ox9LuRmE/NWN2jAY9tyBJXCZBztUn24NYhDwR3Q/gJwBWEtFrRPQpAFsBfIiI/g3Ah2q/MxmjU7SxpKI/bDKJpTZcoGr8kvxfJVyHBkroXtjsKjNNvLpJ5PDR49qJwTaJyyTIOaKmPYgruuYGzZ/+cxz7Z5KjHbrexxEVk1RPUD+7tDeqpugUfEsfBJ14hwZKGHvlbXzzqYMN26cr8xh+aF/9M2Guo6k+T7v3We2UWjuc8drhhCmylRSqlw5AbFExSfQENQnksBUlw0y8Tzx3SLm9MieUE47tdfQT5O3aZzWtvrRpwEK+w0lbG5OC3Zt1K1+6RQu6Ei29GxWTQA5rCgsz8Zr2GWXCAdpXkJtIuqRzlmAhz6T2Enu1KZUDWBdfnhWfgUkgh031DzPxmspWRJlw8konXQ8W8kxqmJKE/MiKz8BPIIc1hQWdeIfXr8Twg/tQmW+cKp0CRZpw8ko7+KLigoU8kxo2WtPSbgfHKvO+grJVTjTdcXQlBIDgpjBThqnJNg4Am3dO1ksWLO12sOmqVZEnnCBjbBey5ItKGhKGdOpWMzg4KMbGxtIeBtMiTCWOAdQjUQCzoPSafdzfjVPwtOI4umP8dt8S/PMLbzeYtIIeOy7BHPU6ZGWCyMo44oCI9gohBpV/YyHPpIVKWEjnaynAS6ebLJZ2OxACWs02KLrj9BQdTGy6vGFbWAHiN/F5SaPeUJTaR62akDsNk5Bncw2TGH6CLq7IHp3Zx9t04/B0pSFu3HacfseZmqk01GOPEp4X1PGXhqMwitOyk6JasgILeSYRbAVdHJE9psgSLzJuPIxANh3HvU+TIBt75W3c//SrmBMCBSLccPEy3DG0OtS5AMASTb2buHFPhF2aqpk2Tku/CSJPJpSskHoVSiaftLJCoV9DFC9uQRNknLZx6qZWi9986mBdQM4JgW8+dRC3j+5vOIaqQqiOXx2fRf+Wx7DC1YQkCLKZien73tINKgFv67Q0lUkI00zeZvydDmvyTCIkEYc8Ol42Ro+4NcCjx2ebGmNI3IImyDiHBkrYsmtS2XvVvc+g2vj9T79a1+a9zV1krXlde8a5eVE/TxuzkLfJx9ETs6jMCeP3daGuNs1MvPjlFQQx5XRS1moUWMgzdeJsSK1z53s1Odtjjo6Xm+LAvTZ2r53dFDfuHo8pXto7vit/60w8vLdsDL3TCTJdToBXM1aFPNqGRwQRiqpJUPV93URoambiF2qq+lvQZuBs37eDhTwDIJmG1F68wjDIMbftOdAksIFmG7vEJm4cMGuWqvE9vLeM6y4s4YnnDuH1qRn01CJ4Nu6YwLY9B4yFu259YJ/S1CGrUnrPN2yiWBChaPP9oIlD3gm2PDWD4QcbJ2PVMxX0OJ2UtRoFFvIMgHi0IpMQUYVEbt45abSHu4WkyfwhO1qponds6rJ4jyX3s3br48rxPfHcITw5cqnvJOU9tqpSJADccPGypm1RBFVQoaj7vq6uEGC2wW/eOdk0IVfmBTbvnDTej6AJSp2UtRoFdrwyAOLRinSfJaCpZvnoeFlrMy9PzWD4oX0NDjg/onS0krXVt2/oB1DVyk3x6vI8gzqX7xhajRvX9DXUlb9xTV9DdI1EJ6jcOv/ihQU4XY2rAJNQtInE8a5i5DUQrmObmowAajOQabskSDMTILkeAXmDNXkGQDxaUZB9+EXZSGdgEHQrDxu7v0or1zk7u4iwYmS31k5umhjvGFqtFOpeVFotPOOZF8CGi5bVTUd+Po2jJ2abzwXAkm4HU9OVhu/3b3lMeeykk6+ChNSmXUG1XWAhzwCIp5ZHkH2ENUf0FB2jRujdr63dX6WVS+3VK8xVdvWGMXZHj133CjBVbLrbdOTHtj0HlBPnkm4H459vztbVXWPbekOqCCRvW8M4sJ0UOjn+noU8AyAerSjIPoKGGUpk+QCdOcW7arD1NeiEl9Reg4w1SKUQv6Jj8ucVI7uV39eN27tf3finasLYm+ykw2Zlt+mqVRh+aF/DpOIUCJuuWuX73STo9FBLFvJMnTiyT233oTNHmOhx2ZRtVw0mX4NNFqd0GAcZ65GZZsFpU1gtTLbtkqKDtVsf9+2mpcOdhORuU6jDtkwy0DzZA2gaayuEbKeHWrKQb1PaffnpTfrxw+kibL76pCZou2rQCceebsdXsJmSdEyoBKdKgAcRPqqJxukiHD0x25QMpeqmpcLpokDnR1CHigLq59FtRkpTm+70UMvEo2uI6GUi2k9EE0TEJSZjIEz6dxaRUS13b+hvipJwCoSeolOPsth2/QXKWPgnRy7FS1uvbIrekegiMISANovTG9kRRBjYZG9KgmbbeiNPTj1lQZOdfaYy5xvFIjn1lAWBzk/U/nmfN5vnsZVlLryYSil0Aq3S5NcJIf69RcfKPXlbfsbhDwiaYanLrlRlcfr5D4pOF45V5gNlb46OlwMX+vKawnR2elukPT6Mf8T9vNk8j2lq053UIEQFm2vakDwuP6P4A2yTkqSg37bnAIpOF6Yr8037UglYP5v8iVn7ieGsniJuH92Pbz11UB+CeWQGt4/u9w21DOu8djM6Xg7lHwFOPm+mgmwySS3NxKVOD7VshZAXAB4jIgHga0KIe91/JKKbANwEAH19fS0YTvvDmX4nGR0vK8sFuDVJ1SSgQtqovUhhcItGO1dp4zob+tT0CWXWqxshUP+MSdCHFc714+DkOS3tdrBoQReOzFTj5adPzCrDIN3I58002cgJ97oLS741fyRJ+JviCCpoV1qR8bpWCPHbAH4fwKeJ6BL3H4UQ9wohBoUQg729vS0YTvuTVKafbdnZtEu7yjEsH9mNjTsmtNEgpsxUFdJGrWJooKSsMQOoa894beg9RQcg4OgJe4F8/9OvGv8uj6Eblxu/jxyeruDoiVls39CPJ0cuxaarVhnLN7ufN79SzzKe3yabNS/+piyRuCYvhHi99v9bRPQIgIsA/Djp4+aZJJafNtEPWYg39o7BFJIuNU1bM9aUj+Z6w8XLlFr4mt9cqgwNdGuPa7c+bu0QlfglXQEw2v/d2MTuV+YEtuyaVJq4lhQdEKEpM1aOQX7WVArCRpvOm78pCyQq5IloMYAuIcSvaj9fDuALSR6zU4h7+Wnzcuk+c4smrC4JbLVyt6Zpa7v2M3fdMbQaLx16B0++8HZ927m/vhg/P3jEd+IL4y+x0dCBeGzzEreJJmiJAVnULYopMUv+pnYPU5Ykba55N4D/R0T7APwUwG4hxPcTPiYTAj/n2YqR3b6VIE3L6iBmHtNnbV72AlGDKcCmc5S3zrxuXD8/eKRh2/NvHbUKDQzjL3FXpzRdk6CdsZIkqikxK+GOeTIbJSrkhRAvCiEuqP1bJYT4qySPx4THVPVQPuh+6OKeg7wwfp/1e9mLTgF3fawxpt5tH9exeKHeHi/R1bdRUa5l1ErWnRfM37T2nNPrTtfbR/dj444J7TVRxdCHxW7toCdoJUkvNpNEK/xCacb1xw2XGmYA6HuLBq0FqdK0g7wwfp819UA1CRSZOKUTYkcs7OVBTQZuQfzEc4cCffflX56Mp1eFWyYlcILX/mzGJklNhTSPzFTm6qYq7z1tlYadJbNRVDhOngGgduaGsfOqNO0gL4zJbLRiZDfO6ik2dGbyK6/rtalGCT9d4lMB04vbpxFUOMgxmlopuhOrbGvV+E3cUVYBUVDVz5EavF+10CQcs3kKU2Yhn3OCOI+8jjadE01XtEtnew3ywpgEqdTcHt5b9jUB6CKBgsRre9H5QRcvLGhDI+XkpMtu1R6rdg6myUFevyitAt24G4a02uEYtVpo3Bp2nrJk2VyTY6IubU320SC2V1tnnK6xhRcbU4VOaHjjtXuKDk5xuurdoEzXRhdiOX1izqgBC9iFQ3q/s23PAWNtenn9ggg43SgIwJ3XVn0A3mdm444JLA9h/w5iO7cV3q1yzEb1LWQJ1uRziLs3p5cgS1u/eHzbEDvbuH5dYwsV7pdfpXmahIY8rrvJN+Af929akch6OHHYtN1j1bXsKzpd9THGEUIpx21yLquuz+2j+3H/069iTggUiHDDxctwx9DqwDkVtqu9VmrYYcKUsxh2SSKghpEkg4ODYmyMC1VGwftyqSCgqdZKXMcO+4CPjpe1ZQNUyDZ0qvMtOgWc4nQp0/Jt6sPrWtzpjiU1vOUhCoY5BcKpixZox/q6JrLJfQ9t7rkNuo5OqnE9OXIpbh/dr0wOu3FNH5547pDW1Bfm2no/mzVBKsdlew5xQ0R7hRCDqr+xJp8zbOyzSTiPvEW3gmTDjo6XMfzgPu3fvc5Ct+amM8ssWtCFolNQanx+10i3CvBbkRQC2t2B6old+VtnNvkJZOiqbp/uexi0Nr8OGwEPnLw+urIL9z/9KuZ9Sk14CZLFndU6NFnN1mUhnzP87LNJLG39wvxs4s8r82qhQAA+eM7peOrFw3WTwHUXnnzJded7ZKaC7Rv664KvQFQfj58gNE2CJgETWMADqMyLup/APTa5J1MzE9W4dM7yOJHXR3e+c0JoWyaGvbbtQFbDLtnxmjNML1FSziObMD8Tps8IAD8/eKQuUOaEwMN7y3Unns4xeVZPEUMDpbrTV37fTwDaZL7qCBt+WK6VPx5ev7KhxaEbqv3zcxQnnf3qjsDRUaDqNXS6GsORdFU+80JWsnW9sJDPGaqXXHZZkrXUR8fLsWYN2oT5mTB9RmrgbmYqc9iyaxKj42W8c6w5GsctqAOHF0ZwUQ2vXwmnEC5ntDw1g+EH9xnDR7dv6Mfx2Xkcnq5oo6VUUSG6icO2No4q2cwU3VQvx+DZ/ZyoFkBLs4JpkiRVHTYq7HjNIW7HVE+3g3eOzTaYQ5wuAggNkSxRHEQ6EwGhKphsbfJek41TIGO0TY8mpr6n6GBi0+UAqt2Tgj7hOuegDaPjZWzZNWlt3w6CzgTiN16TQ9A2Ish7DNN1fXnrlVZmo6JTsE5saxfScgqbHK+syecQd1p598IFTcKzMi+UvUHDpsnrSg18Yk2fdYjltusvaNA4l3Y72PbRC4wmEJ3W694eZqkc1qYtX/Cp6YpWS17a7YQy6yztdkLbfE0x32GrQ+q+J8/Nxkw3U5nDt546mIsiYJKwJR2ShB2vOSeI0yesgyiO+vYmp1uQ0Eqgmpkq67svKToodBHmvI5dIbQprG4BrdLMgOZzBdCUlu+l6BSw6apVgcMtnQJh01WrtE7jLqJ6yYegESm23aW8vg+/eHXb2H0bZ31Y7TiroZaths01OSdItEUUM4XEayoSAvWWcjoB6ffi9W95LHDDDT9OPT6Nzz1xH3a97xL8pG91k8CXphFv+KbO1LVoQZdyjAUizAvRdK4DX3jMaNJRfc8mHj6M2c2UPCdxm8C835P3ct15vXXTS9A6P278cgBszjHNmPU0MJlrWMjnnCCJMndb2M+jHMspECDQYD6K8sLqEp5sWf+v/4yvPfJFPPvuc/C3vzOE3ef9J1QK+jICYdAlnukSify+5xasuno4YSdrU0KaXwKdzq9iQlcszT1+U/0k0zmG/V67wjb5DsZrjzXZiqNqOFt2TRonk8qcaBICNr4AnU3Zrw+pH3ve+0F8/70fwPvffAF3f+8u/N+vfgp/+tSDOO3YO6H36UVlux4dL+PhvWa7s87m7bb56hKOvLXsbRkaKGkjcbqIjPvcvHMykIBf2u3gE2v6fKNRwvohshqzngZsk28DotoW3fZYnQZ5/pnvUvYptR3H6Hg5tFZt8+LpbMoPjh1saMcXlM9fdjM++PI+nHZiGu955238xY++gc/+83fwwOoP4e8Gr8YrS8+y2k+30wUB0tqo3fiFdeq+573+JpNI2N67m69epVyNzQlh3KfJNOMul9BTdLD56lX1fQyefbrx2Q5b8jdPpYKjwuaajGNrW7SdCEzhjt7SAapmDbpxRMm0jGJeiKMo2B9O/AO+uOcrTdvnQXjsvWvwt78zhLHS+fpawwC6CPjDi/uswgFNTteSYYL1Xn+V+cu7r7DX9dYH9ilNQTofg+mcXo5QJ4lt8nZw7Zo2xqYehqnin9yHXyMQvygH3Ti27JoM1RRD4tZag65YTJm2Qbj/gvW4ZvKfcPFrkw3buyBwxb/+BGccncLnP/SnOPCec7Sp/POi2v3JT6iOjpetbNFeVNe/MifQZchnihIttVFjm3dnDrs1e11xs6WGUsk6vM/BdReWsPuZN+r7X7TA38ocR8RXXmAhn3FsbIsmAexOhAqqabuPofvu4ekKRsfLocrdupfuQUvTescXBUFd+NwVn8Gjf/dZLJprzKD96yv+CGd9cTMerY3RFM5pMx7dxESAMTNSt+95oXdgRjFN2ETHuBWBTVetwvBD+xqijmToZxBUz8GOn73acIJTMxUrc1S718KJi8Qdr0R0BREdIKLniWgkiWO0orFvWuheVAHUz1UnAA5PV6ycYTplUB5bap86ZN0Vb60SPxYvquoY/Vsewy07JgI3TvYTYgR7TfKFX1uG//2BDU3b/9s//h2Gfvo9ACc11rDjAfTCWgBN5jf3M21qHiLQfA+jptNbVjyon8/QQKmevCad49s+ekFgIatbsYRx2DNVEhXyRFQA8BUAvw/gfAA3ENH5cR6jVY1908JUcEqeq0kA6HC/jH5RDn5mkbrgCli2xa9ei3vfqol8eP1K7SFLPUW8tPVKjH/+cty4pk/7uaJTwI1r+lDqKeJraz6KF9+9vPqHDRuARYuAuTng5puBP/9zYH4em65apa1Ps+68Xt9z1k0EBVf0iuqZfufYrLEujgCUGa1h0XXB8uIteRw127MVyXudRtLmmosAPC+EeBEAiOg7AK4B8C9xHSCrNZzjQtfFSKKrne5X98UtwL/11EEsqVU3nJquNNkv/V6ms3qKgbo6SQpEviuNJUWnKXFItqT7xJo+fGJNX1OZY68We8fQ6noUh7tGu9fJOTpextajt+Kr9/x3fHT5Nbjlno/jkuE/wrO/fA+Wbfsallx4IYY2VLX9v3xkf1Nf14f3ljF49una5250vIxpTXtDd/SKUpudF+gpOvjVsdlYY+N147TpSZtE8a0gZr9OjJQJQ9LmmhIAd2eB12rb6hDRTUQ0RkRjhw4dCnyAToiHHRoo1U0bKo7MVBriyJd2O77VFDfvnGzQFqdmKpiqVTf0YnqZ5Ise9Hq7y//qcLoIR0/MKh16AsC3njqIwbNPx/YN/dZaLAF4z5JTcPeG/noDkRUju9G/5TEMP7QPj522Ard+4C/wc5yGP3nxFHzzzn/AZYUnsHbJJF65+GMAavHk3Qub9j1TmcOtD+xTriKldm4KM5XKialG/g0XL4vdNKMap+reOF2Epd1Ooj1PdVVUvabALFR3bBeS1uRV68uGp0cIcS+Ae4FqCGXQA3RKPKxfOV+3k2nt1sd9Y9ZVqwJdVyddfRMi1Bt4mNLiS56Ud7lSMH2nQIRTT1G3xXOPd9ueA76mAZUzb/ihfQ3hh+7rcc+rw/i1t5/B/OLjuOl/vA8zc6fizSPAP3y/arkB9PdDF09uW/LYFAXV0+3g4b3lhheIgIYmKlExjXPDRdX+rUmii4pRbcvDSr0VJC3kXwOwzPX7bwB4Pc4DtLKxb5roXnxVVEYcqxi3yUtnMhLipIlCdx/8tD1dieFtH71AG8bnxjaiReXMU3H89R4cf+10/GrvclR+uRjH3jwVAPDZz54U8IDZrKAyF9reEynAVJPq1EwFXgVboBq6GRcmU0mcxzGhi4pxm9W27TmAjTsmWOBbkLS55mcAziWiFUS0EMDHAeyM8wCmMqp5Ikg5X79VTNEpWEWduAWTzmTkFmi290E6UTfumMCppyxA0Tn5GMoSw7ZlcKNEtKj4j5+tAAD86ufLceyVqiP1qquA7dsbP+fXgcl7TBvnuFRO5LX0lhjQWbder5UxiBph5hdFlQUTaN4DLZIgUU1eCDFLRJ8BsAdAAcB9QohJn68FphPiYYMkd6g0QRlLXepRl8ZV4RWgfv4Pm/vgNZ0cnq6g6BSUxdH8yuDarthsnXmzR4qYPvCehm2nLJ7Dl75UQKFwcvzuKpvHZueUwtd97XQdrNws7XbqZYgB1E1gNpUclxSdwDkGbmyqUALZMIHmPdAiCRJPhhJCPArg0aSP0wnYTmZBJgT5cqvKGngFaBz+jyAvqfc8lhQdEEEZAWTCtmb6f+xdDojGxe2xowW8733AJZcAn9r8Bu58vHGCcroI6GouPey+dqZG5boyBoCd5lx0CiBCaMFnW6U0KybQTgi0iBvOeM0pNhOC+zM2JQWC+D90+7N9Sb3ft2kjaDpPoHHSO3p8tkFLnj++APBRhz0AABpSSURBVO/sW6b8/s03A1u3Ah++5xfa0MbFixZor53unAloCnu0KSXsRrbwUxHWX+GlQJQZE2geAy2Sbm7CQp4BYD8pAP4rBFOJAt1LWnS66lUwvX1pg5ofvOi6O7nH+M6+ZRAnGm3gC05/B2dc8QzW33Q2lizRT1BTMxVjiKvunGXWsrspyJ/tmMB87e9+Ar5Ui6rSmVri8FckXdQrqIDLW6BFmHIeQeEqlExs+Nl2ZSilqVmGiTAJP6ZqhEDNXPX2Mbx+7zpUjtSEYtc8lqx5Hks+8AJowXz9uLaVNp2uavinNCutO68XD+8tG30L111YCnRdZASSrmOUrXA2nZPJlBQHUSpM5iWcMq7mJtwZikmcILZdm3hxFX7diVTYvEQPPFCtYgAAi846jNOveAYLe082DpHHDdP9SH7/g+ecjpd/ORO6HLMXbzu+KH1Q0yrJq7s3BSLc9bHgdW/akRUju7UF64I861xq2JI8aQhJYLo+trbdsAIeCGd31ZkjZPekL33/APZ+eQBdC9+FMy/7NxTe/2JTca6G4waszwNUzTJPvvA2blzTF3oV42VqptLU5EWl+fk902mW5A2aUJZHWuFjYCFfoxW2sXbG7/rY2HajCPiwdldT+OSf7ZjA9GtLUVh8Ar1DP8LC046hoKj5c/T4bF1YBq3P4yYuAS+R56V7Vm2f6bRCkIMmlOWRVvgYuMdrDVNoX1YJmgATJWHG7/qYNA+ZGFUKoJ04BUJPMVydFPd56gqCAcA8gAVLptF77RgWnHas6vAUokmTl/XL4zK1BMWmgrPqWc36Mx00ocxLHkqMtyKZkzX5Gu0Wfxt05RF1peJ3fWzLGtjY7aM4/FTJViYWvOt4w++VefXnZipz9eqVrUK2FBw8+3RtOz433nuU9Wda3l/duZkUhzytvJNeSbEmX0P3QGU1/jaolhZVq/O7PjYaifszOqRDNOxDb1sILAxzQhg1z6XdDu7e0I8b1/RFOg4RcPeGfrx455W4Y6h6DectJpcuogatth2e6aGBEu762AXGfgYq/J7nPGj5ccGafI12i78NmlSkMzXYanU21ydIApYuqiPq9U5SS5UrDL+69PJ/b517a0SzNmpTmsHbf/W6C0tNoZtZfKbDOH/9HOp50fLjgIV8jTSjDMJg45W3CWu01erivj5BEquClJ3VXZel3Q66F57MSl13Xi92/OxVa0eq00WYPjFbr3yoqrXjxt2o5PVaMS1bVPfEVI9IZUaaqczhiecO4c5rV7fFMx3UZGGqyrp55yTXt3HBcfJtik18s1/yjkzC8dZ5j/NFiBKWqjpHp0ANdeDlecjz9rsu7vEsKTo4cqy5fK8N3mvtd562iVQEaEs46I5hE2udt/Dg0fEyNu6YCDR5hsmzaBc4Tj6H2GjCJtOFzD51L+eDLGu9QsPdFEQWEjs8XWkofBZ02WxbB15V+151XbwTgF+FR2/RNt0xbcwDNkXSdKWjJTpt129Vl0fzxdBACbdY9BtwkyVfRCthId/G+C1xdS+/O00/zLJWJTTcMeANjUU83/Xu36Rhhm3qrLsuQZyyRPr67d5j2lTWVE0+qm5ZYYSun78krvK8WVsNlAymuWOV+cz7IloFC/kc4/fyhw2xixrBIvevmiw27pjA2Ctv446h1bE3dbadNJwu/wbj7mPaXsekQuV0E4jsnqQ7kyCTaJqrAd3konu+N121CkD7+NeShoV8hohbU/Iz6YRNqY4awSL3r5osBKqZobqWgjqbvElLGx0vY8uuSSv7bamniGlN83A37mPqruOSotNUeiApQeMtG22Tj5BUH4A4sZlcdM93pwp1LyzkM0JSmpJJewwTNjo6Xraqc67D3ZPWNFls2TWJ8c9XC3BFaeo8Ol7G8EP7rKJoZH33FSO7jZ/zhkwqJ6MuwtETJ2vWt1LztVlpBTVfpJVY5Te5pFWSoZ1gIZ8R0tCUgoZFyokoStancB3XZI6RmrTuJba9JkHqzUjNVjcub+VH71jc11G1GmhVGJ9J8BIQalWRVrOOrGfttgOc8ZoR2uFhjiOb1J3t2gpHmO31c2u2qpoqRaeAzVev0n5/aKCEJ0cuxUtbr8STI5diSmPuacX91AneUk+xPr6gE43umiR9D9shazfrsJDPCHE8zGEKlsnCWwInTQq670UVUKoM2W5H/Qj2FB3l9qDYXD9vezvbolGm693TrR6/bntUvEXZHE9Vs6gCuRWFtFSkNbnkicTMNUS0GcAfAzhU2/S5WlNvRkHUsgq2Nn2/HqImk0KQaBcvuj6hX7z2t5oacThdZNSagzC8fqXRJq9rkKEyE3kTqY6emK3v13u9j2tWPHJ7nE52v6JsPUUHm69eFVkgp2H/brdM9CyStE1+uxDifyZ8jFwQ9WHW2fRvfWBfff9eYaCzres09uH1KwMnoEjmhTDa1sN2NfL7nvx9y67JuvCTSU621S5ldI5beKoSqdwT5LSmnOV0ZR7LR3Y3JYkNP7gPW3ZN1lsGRr33bo7PakprxkiSMfTsXI0GO14zRJSH2abLjq1NXWfiCJNlKOkiwuh4uam0gKmrkUT1+bFX3m4oAGaKXolyXW3DESXlqRms3fq47+e802tlXtQnkbjKQEuSdvjmMaM2TyRtk/8MET1DRPcR0VLVB4joJiIaI6KxQ4cOqT7CWGCyPcuX3Mam7mciWhrSpiwnm9tH9wfyA6j8BsMP7cM3FRUe5colzrKyQZ3NBIQ2abmJowy0G1mdMSwm/0PWm5N0OpGEPBH9gIieVfy7BsA9AM4B0A/gDQB3qfYhhLhXCDEohBjs7e2NMpzYaaea1DZddpZonJkFImtn2qarVlUTkkIwU5nDt54+GLkOvikkUk4mQe+V7l4HcTabat2EIUgZaNO9l4S5LoC/g74dIsM6mUjmGiHEZTafI6K/AfC9KMdqNe22BJVj0nXZkY5CL04XYdv1F1ifk9eG3tPt4MhMBRZVAADo68HEKSiC1MeRf9fdaz9nM9UkexSntI4wZaDLUzPaySas2cYvhyOtGHrGjsTMNUR0puvXjwB4NqljJUE7LkGHBvRddojUGvCppywI/NK7Y8LHP385Tjslelhg3PHQ3vo4JvOQ6V4Pr1/ZFI4ocboI2z/WX489D9LDFqhq/qa/BQkTlPfk5a1XYvuGfu3nwkyaNq0fOcwxuyRpk/8SEe0nomcArAOwMcFjxU7cS9BWmX508cy65Bzd9iAc8SnZa8PR47PKa2JrivBiqo/jnaz97rUuNmXDRcsaJkjVWJ0CKSeJpd0Otm/ox90b+pu+41dy2I+hgZJ2wgkzacbR+pFJj8Sia4QQ/yWpfbeCOJegrTb9qKJJdC0A41hSm7r0qMwG0nnrDUl0V6CUyPPYvHOyKWxRt3+bSpsyCmZ4/Urjvd625wDmNLaoJ55rDBTQhYOqtqnuT1wx8zqzTVjtOq7Wj0w6cGcoDTadl2zRdQWSdd1bQZznY7tvXY/RO69drRTagL4zku4aegWZ1ILlRGHbHcs7TrkfU5/WrHUaUt2HoDkBpn1zQlJ24c5QIYgz0y5qE+04SDJz0LRvd59Tt2ar68okavvx7k93Db0CWKBRw/bryCR7oV53YalBoAsAD+8to6fb0ZYdjtOxaOq0ZXuvdKWbvcpEGIHNmnr7wpp8wph6URaIMC9Ex2lGNv1Oi04hdDE0b29T3arB/XlT5Ul3+QJJ0KgkEzYJVzarLtter0mt6Jj0MGnyXKAsYbbtOaBd7s8JYZUQFJWsxfv7rWAKRJGqXUoN+/bR/di4Y8K3l+tZPUXtmI7MVLDtoxc0JIH1FJ3YBDxgl3DldRar7qlNhFI7Ro0x0WBzTcLYmmSSSj1P2ukbZunvF1MepV69dAiOjpeN9nTv502O6aRNFbbPiKlt4m3f3a/1gbgdpJy41HmwJp8wQey2SbxoJs0tqoYftFSxZHj9Sm2MeLfThQIFy6hVZeyaVlAAmj6fZqy37TPiFxb6xHOHfEMZuT5758GafMKoHH+60L8kXjRTCGFUDT9sN6uhgVJTgTGgGlNemROBNHlCVfP3Ro+YJkxVVFOaJW39nMOAfQN2v1VH1JLWTPvBQj4mdGYLlfBYd16v77I6LnSmEZXdO6jJKMrS/46h1U2RN0ePzyrt5wUi3HDxMjzx3CGUp2ZQqNXB95brdU9Spth93XW2Ncu0ouG6KbomSg4H12fvPDi6JgbCRCy0Ku5YNzad1hgk9tsU/y/t3N7zM523TXSIzbGfHLlUGzPujqEPw+2j+5tWIK2OTkkqQoZj4dsXjpNPmDBmi1bEHcuXdqYyV9d+3QLYVhvUvfy6pf+683qVpqCxV95uWMHYat+qMfmtIkwaa1hhpnPmtqpBtyQJbbzdCvIx9rCQj4EsRiyoukBJk5B8aW1sszYvv1fY6Ca9+59+1dhuMIi92GZCUE2kUYSZyZnb6nsdt5IQ1r/CZB8W8jGQxVKrfi+trTZosx/vdzZqukfpHKqyoUUQDTWoA9Fd08XLTGUOW3ZN+h7XJMjd97odzR5ZVFSYeGAhHwNZjFjwK8xlK4DCvPwmZ69O0Lu1aRuBGGRCsMkoPTxd8W2/Z+PMVa0UVIXXbGjlZJFFRYWJB46Tj4EsllrVvZyyPZ1tXHuYuGpdzPkNFy/Tlg0Ok3Xprmv/5Mil2usdtIWfHM9t332mYZvqvLxlgXX1Y7711MFAeQhhcxDCwjXh8wsL+ZiwFTitQieQdE7DIPvxe/l1k94dQ6tx57V6bVauMuIWZGFNDjOVedw+ur+eNLZxxwROcbrQU3Tq57V9Q3+Dhq47lrvwmg2tLj+QRUWFiQc21+QUlTkjTDXMsJEcOrPL0EDJWDAsiagO07mXDPH5APDtpw82RAQdnq6g6BSU5ZD9jhVksknDRs6VJvMJC/kc431pdbHlfnbXOF/+0fGystesm5nKHG7ZMVFvvxf12OvO6zXGto+Ol3GLxlk8LxAo6mR4/Upt1dEg9m22kTNxweaaDsLG9OKuZzPwhcfQv+WxWKtXbttzQNlrVkUcdujR8TIe3ltuaixy3YUnJ66hgRI0bVy16DTqoYESPrGmr6k2T1D7NtvImbjoGE2+HcPa4sbP9OKNDHE3y4jLjBLU3ODWmsPcQ50j1Nu67w8v7sM3nzrY9P2i04WZSnOX1y4irBjZrRyHqmRD0OeNyw8wcdERZQ24UYIdNs08orYstDmGF9kSMMw9DFIq4fbR/fWELVkvZ/Ds02Np6MFkm3ZXAhNrGkJE1xPRJBHNE9Gg52+3EdHzRHSAiNZHOU5UuFGCHTZadlTH37rzeptMGU6B0FN0lJ8HgJ5uJ/Q9DBICesfQarxw54fx8tYr8cKdH8YdQ6ubok5UZZD5WWpvWh2u2mqi2uSfBXAtgB+7NxLR+QA+DmAVgCsA/DURqQOkWwBn89nR060XtJIojj+dfXzD7yzDxKbLcfeGfjiFZiH6zrHZ0BErcdi23eGx85qVLz9L7UvelcBIQl4I8QshhOpKXAPgO0KI40KIlwA8D+CiKMeKAjdKsMPPchfV8ednHx8aKGHxwmY3UWVeaBuJ2EQGxRn/zc9S/si7EpiU47UE4CnX76/VtqVCFssOZAGvHdLUC9XblCMMNi/TEc0YZIG1MPcwzhDQNJ+ldrcbZ5W8h6v6avJE9AMielbx7xrT1xTblHoiEd1ERGNENHbo0CHVRyLD2XzNqOyQuihC6WyNer1stGDdZ+Q9S/sepvUs5d1unCZ5D1f11eSFEJeF2O9rAJa5fv8NAK9r9n8vgHuBanRNiGNZ0cr67e2gaelMJ97SB3E+7DZasOkzQe9hUvcjjcxQLgWcHHkPV03KXLMTwLeJ6H8BOAvAuQB+mtCxMkErmi7EKbRMNVZKPcVEHnablymuFy5vTTDybjdOmzyXdIgk5InoIwC+DKAXwG4imhBCrBdCTBLRAwD+BcAsgE8LIYKVAWwzkta04hZaOjtk1Dh4P2xepjheuLxpvnm3GzPJEUnICyEeAfCI5m9/BeCvouy/nUha04pbaKXlQHSvRnq6HQhRdbZG0dhVWn/eNF8OHmDC0jFlDZImaU0rbqGVhh0yStkElTAH0LS6GX5wH7bsmtS26dPdj6z7U/JuN2aSg4V8TCStaSUxibTaDunXvEO3MlGZqm7ZMaGsj1+ZFw2Thxvd/WgX+32e7cZMcnAVyphIOrQuD2FeYcsm6CaHIKFYpvuR94xHprNhTT5GktS08rBcNzXUcH/GS1Q7OgFGZ3Le7PcM44aFfBuR9nI9qt1aZdJyo1uZ2EwOJkxlgU37z0LkStZ9BUz2YXMNY0XYjEt3E5ItuybhNrIsXlho6JeqM6eoTFVuCkQgAD1FR1ngbE4I45izagrjLFcmDliT7wDi0AbDhHCaommAamu9zVevsoqbB4Atuyab9uGt5e4+1y4izHmqrqnGnFVTWN5i/Zl0YCGfc+KKHAljtw4bTaNCmqr8Jiy3SWv5yG7rMadtClPBvgImDljI55y4tMEwduskmpDYCuPR8bIyxBLIhq3dhiz7Cpj2gW3yOScubTCM3dpGGCUlsLbtOaBt+6cbs9t/EFfj8iik5SvI2nVgosFCPufE1eQiTB6An8M0SYFlKsCmy6jNmpMzjbLGWbwOTDTYXJNz4szEDWq39jo046hVY4upAJuKVjo5gzjCs5CVzM7e9oaFfM5JO3IkLYdm0MmtVU7OrJdQYGdv/mAh3wFkMXIkaYJObq1ycmZdU2Znb/5gIc/kliCTW6tK+WZdU+aSxvmDhTzDoHVmraxrymmb95j4YSHPMDWSNmuNjpcxfWK2aXvWNOVONO/lGRbyDNMCvA5XSU/RsSrtwDBhYSHPxEZWKiZmZRxudCUeFi9akPrYmHzDQp6JhayEBmZlHF6y7nBl8gtnvDKxkJXuSlkZh5e4Mo8ZJiiRhDwRXU9Ek0Q0T0SDru3LiWiGiCZq/74afahMlsmKppqVcXjJas16Jv9ENdc8C+BaAF9T/O0FIUR/xP0zCRKn7ToroYE93Y6ykXfaGjOHJjJpEUnICyF+AQBEzd14mGwTt+06C0k0o+NlvHOsOUTRKVAmNGYOTWTSIEmb/AoiGieiHxHR7+o+REQ3EdEYEY0dOnQoweEwbuK2XadRMdHLtj0HUJlvLjC8eCFHsDCdi68mT0Q/APAexZ/+Ugjx95qvvQGgTwjxSyK6EMAoEa0SQvyH94NCiHsB3AsAg4ODqhLgTAIkYbtOW1PVjf3ITLP5hmE6BV8hL4S4LOhOhRDHARyv/byXiF4A8F4AY4FHyCRCVmzocZLHc2KYqCRiriGiXiIq1H7+TQDnAngxiWMx4chjtEcez4lhohLJ8UpEHwHwZQC9AHYT0YQQYj2ASwB8gYhmAcwBuFkI8Xbk0TKxkcdojzyeE8NEhYTIjhl8cHBQjI2xRYdhGCYIRLRXCDGo+htnvDIMw+QYFvIMwzA5hoU8wzBMjmEhzzAMk2NYyDMMw+QYrifPMG1EFhuiMNmGhTzDtAlZbYjCZBsW8kyuyZPmayoq167nxCQPC3kmt+RN881qQxQm27DjlcktWW0FGBZuIciEgYU8k1vypvlyATYmDCzkmdySN803C41ZmPaDbfJMbslCS8K4SbsxC9N+sJBncguXHmYYFvJMzmHNl+l02CbPMAyTY1jIMwzD5BgW8gzDMDmGhTzDMEyOYSHPMAyTYzLVyJuIDgF4Je1xeDgDwL+nPQhL2mmsAI83SdpprACPNypnCyF6VX/IlJDPIkQ0puuCnjXaaawAjzdJ2mmsAI83SdhcwzAMk2NYyDMMw+QYFvL+3Jv2AALQTmMFeLxJ0k5jBXi8icE2eYZhmBzDmjzDMEyOYSHPMAyTY1jIKyCi64lokojmiWjQtX05Ec0Q0UTt31fTHKdEN97a324joueJ6AARrU9rjDqIaDMRlV3X9MNpj8kLEV1Ru37PE9FI2uPxg4heJqL9tes5lvZ4vBDRfUT0FhE969p2OhH9IxH9W+3/pWmO0Y1mvJl/biUs5NU8C+BaAD9W/O0FIUR/7d/NLR6XDuV4ieh8AB8HsArAFQD+mogKzV9Pne2ua/po2oNxU7teXwHw+wDOB3BD7bpmnXW165nFWO6vo/o8uhkB8EMhxLkAflj7PSt8Hc3jBTL83LphIa9ACPELIUTbdHs2jPcaAN8RQhwXQrwE4HkAF7V2dG3PRQCeF0K8KIQ4AeA7qF5XJiRCiB8DeNuz+RoA36j9/A0AQy0dlAHNeNsGFvLBWUFE40T0IyL63bQH40MJwKuu31+rbcsanyGiZ2rL4sws02u0yzV0IwA8RkR7ieimtAdjybuFEG8AQO3/X095PDZk+bmt07FCnoh+QETPKv6ZtLQ3APQJIQYA/BmAbxPRaRkeLym2tTxm1mfs9wA4B0A/qtf3rlaPz4dMXMOArBVC/DaqJqZPE9ElaQ8oh2T9ua3Tse3/hBCXhfjOcQDHaz/vJaIXALwXQOLOrTDjRVXrXOb6/TcAvB7PiOyxHTsR/Q2A7yU8nKBk4hoGQQjxeu3/t4joEVRNTir/UpZ4k4jOFEK8QURnAngr7QGZEEK8KX/O6HNbp2M1+TAQUa90XBLRbwI4F8CL6Y7KyE4AHyeiRUS0AtXx/jTlMTVQe6ElH0HViZwlfgbgXCJaQUQLUXVk70x5TFqIaDERvUv+DOByZO+aqtgJ4JO1nz8J4O9THIsvbfDc1ulYTd4EEX0EwJcB9ALYTUQTQoj1AC4B8AUimgUwB+BmIUTqDhndeIUQk0T0AIB/ATAL4NNCiLk0x6rgS0TUj6oJ5GUAf5LucBoRQswS0WcA7AFQAHCfEGIy5WGZeDeAR4gIqL7f3xZCfD/dITVCRPcD+D0AZxDRawA2AdgK4AEi+hSAgwCuT2+EjWjG+3tZfm7dcFkDhmGYHMPmGoZhmBzDQp5hGCbHsJBnGIbJMSzkGYZhcgwLeYZhmBzDQp5hGCbHsJBnGIbJMf8fYWKwWLl7auEAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.scatter(Xs[:, 0], Xs[:, 1])\n",
    "plt.quiver(*[0, 0], *eigv[:, 0], color=['r'], scale=21)\n",
    "plt.quiver(*[0, 0], *eigv[:, 1], color=['b'], scale=21)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2. Hebbovo učenie\n",
    "\n",
    "Hebbovo učenie je model učenia biologických neurónov, ktorý vymyslel Donald Hebb v roku 1949. Jeho aplikácia v trénovaní umelých neurónových sietí má jednoznačnú výhodu oproti backpropagation v tom, že aktualizácia váh závisí iba od predsynaptického a postsynaptického neurónu, teda aktualizáciu vieme úplne paralelizovať, nemusíme vypočítať chybové signály pre každý neurón postupne. \n",
    "\n",
    "Hebbovo učenie zavedie aj ďalšie zjednodušenia do modelu neurónu, z ktorých hlavný je zmazanie prahu (vieme ho ale reprezentovať ako váha nultého vstupu). Takisto nemáme aktivačnú a výstupnú funkciu, tým pádom výstup sa vypočíta iba ako skalárny súčin dvoch vektorov:\n",
    "\n",
    "$y=\\sum_{i=1}^{n} w_{i} x_{i}$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Samotné trénovanie je založené na jednoduchom princípe: ak pred- a postsynaptické neuróny sa správajú rovnakým spôsobom (teda ich výstupy majú rovnaké znamienka), zosilníme synapsiu, v opačnom prípade hodnotu váhy znížime (*neurons that fire together, wire together*). Formálne to vieme napísať nasledovne:\n",
    "\n",
    "$\\Delta w_{i} = \\gamma x_{i} y$,\n",
    "\n",
    "kde $\\gamma$ je učiaci parameter."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Aktualizačné pravidlo pre implementáciu umelej neurónovej siete sformuloval Erkki Oja:\n",
    "\n",
    "$\\Delta w_{i} = \\gamma y (x_{i} - w_{i}y)$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.1. Implementácia Hebbovho učenia pre jeden neurón\n",
    "\n",
    "V tomto kroku implementujeme Hebbovo učenie pomocou Ojovho pravidla pre jeden neurón. Použijeme pritom dataset vygenerovaný v predošlom kroku, teda s dvoma príznakmi a jedným výstupom. Váhy neurónu by mali konvergovať ku prvotnému vlastnému vektoru.\n",
    "\n",
    "Najprv váhy nainicializujeme náhodne a nainicializujeme aj premennú pre uchovávanie predošlých hodnôt váh (pre určenie konvergencie):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "W_oja = np.random.normal(scale=0.25, size=(2, 1))\n",
    "prev_W_oja = np.ones((2, 1))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Ďalej nastavíme parametre učenia: učiaci parameter a toleranciu, ktorá slúži na určenie miery konvergencie. Trénovanie potom ukončíme ak zmena váh je menšia ako tolerancia."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "learning_rate = 0.0001\n",
    "tolerance = 1e-8"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.2. Trénovanie neurónu\n",
    "\n",
    "Počas trénovania použijeme Ojovo pravidlo. Neurčíme počet epoch, ale ukončovaciu podmienku pre mieru zmeny váh (tolerancia). Samotný učiaci proces vieme vyjadriť nasledovne:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "while np.linalg.norm(W_oja - prev_W_oja) > tolerance:\n",
    "    prev_W_oja = W_oja.copy()\n",
    "    for x in Xs:\n",
    "        y = np.dot(x, W_oja)\n",
    "\n",
    "        dW_oja = learning_rate * y * (x - W_oja.T * y).reshape((2, 1))\n",
    "        W_oja += dW_oja"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.3. Vizualizácia výsledku\n",
    "\n",
    "Vlastný vektor nájdeme vo váhach ako stĺpec, preto ho najprv pretransformujeme na riadky a zoberieme prvý člen:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[-0.65980779 -0.75238913]\n"
     ]
    }
   ],
   "source": [
    "oja_vec = W_oja.T[0]\n",
    "print(oja_vec)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Ako vidíme, hodnoty vo vektore sú blízko druhému stĺpcovému vektoru v premennej `eigv`. Vektor aj dataset následne vizualizujeme pomocou `matplotlib`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXkAAAD4CAYAAAAJmJb0AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAgAElEQVR4nO2de5Bc1X3nv79pXYkekahFeeJAR4NkioiyojATJqBEjssiDiImwBiCZQIbbzYFSa1dW5apqR0SypJcZJlYcUQqb7zrsmuNsXiYiWS5ImJD4lo5YM9kRohxUIWnoCFGRBrFaBqpZ+bsH9Ondfv2Oeee++p7+/bvUzU1M/2499zX7/zO70lCCDAMwzD5pCftATAMwzDJwUKeYRgmx7CQZxiGyTEs5BmGYXIMC3mGYZgcsyztAbh5z3veI9auXZv2MBiGYTqKycnJt4UQfar3MiXk165di4mJibSHwTAM01EQ0au699hcwzAMk2NYyDMMw+QYFvIMwzA5hoU8wzBMjmEhzzAMk2MyFV3DMEz3MD5Vwe6DR/HGbBUXlYoY2boew4PltIeVO1jIMwzTdsanKrj7G0dQrS0AACqzVdz9jSMAwII+ZthcwzBM29l98GhDwEuqtQXsPng0pRHlF9bkGYZJHK9ppjJbVX7uDc3rTHhYyDMMkygq0wwBULUruqhUbOvYugEW8gzDBCaI01RlmhFAi6AvOgWMbF2f2Ji7FRbyDMMEIqjTVGeCEQDKpSJH1yRMLEKeiL4E4DcAvCWE+Ln6azsB3AHgeP1jfyCE+FYc+2MYJj1MTlOVkNbZ4MulIg6NXp3YOJkl4oqu+TKAaxWv7xFCDNR/WMAzTA7Qaea610e2rkfRKTS9xqaZ9hGLJi+E+C4RrY1jWwxjIqsJNFkdVxLoNHOd01Seh245P1kjaZv8p4jotwFMALhLCHHS+wEiuhPAnQDQ39+f8HCYTiarCTRZHVdSjGxd33S8gL9mPjxYzuW56ASSTIb6awCXABgA8CaAL6g+JIR4QAgxJIQY6utTNjZhGADZTaDJ6riSYniwjPtu2ohyqQjCkm39vps2shDPKIlp8kKIH8m/ieiLAL6Z1L6Y7iCoLbhdZHVcScKaeeeQmCZPRBe6/v0ogOeS2hfTHehsvmkn0GR1XAwDxCTkieghAP8MYD0RvU5Evwvg80R0hIieBbAFwPY49sV0L1mN0sjquLyMT1WweexJrBs9gM1jT2J8qpL2kJg2EFd0za2Kl/9PHNtmGEkaURo2UTNZih7RjbfbnMPMOUgIVQWJdBgaGhITExNpD4NhALRGzQBLGnpWnYym8e4+eJQTknIMEU0KIYZU73GpYYbR0GlRM6bxdqNzmFmCa9cwjIZOE4y6cVVmqygQYUGxau9k53A3JaBFgTV5htHQaVEzunERoBTwWXQO2yJNU5XZKgTO+RjYmdwKC3mG0dApUTMS1Xh1ddsLRJn1LdjQaaa0NGFzDcNoyFLUjA2q8eo6MC0KkdnjsKHTTGlpwkKeYQzYZnZmxT7sHe/msScDFRPrFIIWSetm2FzDMBHJsn2400xOtuT1uJKANXmGiUjQJhrtpF0mp3avZDrNlJYmLOQZJiJB7MNpmHWSLiaWVjYtF0mzg801DBMR21DLLJt1osCRLtmGhTzDRGB8qoK5s/Mtr6vsw3kVhhzpkm1YyDNMSKRmfnKu1vR6qegoY9DzKgw7LWms22CbPMOERKWZA8DKFcuUtmLbsL+shGPq8I5vy2V9eGyyEqgdINM+WJNnmJAE1cxtwv6ybrdXje+xyQpuvqLM7QAzCmvyDBOSoAk5NmF/SYdjRl0l6Mb31PPHuWRxRmEh30X4PeBZNxNkjZGt65X1201mCr+wP1Mlyc1jT0a6JnGEOppWL3z/ZBM213QJfmaArJsJssjwYBn33bQxVjOFyVkZ9ZrEEd2jG1+p1+H7J6OwkO8S/B7wvIb3Jc3wYBmHRq/Gy2PX4dDo1ZE1V5Xd3k2UaxJHdI/OryAE+P7JKGyu6RL8zAC6aoWdHt7XDuIwU7i3saro4DynpyU0UxL2msRR1EvnV9i+dzrWsTLxEYuQJ6IvAfgNAG8JIX6u/toFAPYCWAvgFQAfE0KcjGN/THBMZWcrs1Vt3XGOdTYTh53bu43Z6pJwJwJULZjDXhOVDwEA5s7OY3yqYj1elV9B10OW75/0ictc82UA13peGwXwHSHEpQC+U/+fSQk/M4DAUoMJNxzr7I+tmWt8qoLNY09i7egBXHL3t7B29AA2jz3Z0OBV8fYqAR/lmkgfQqnoNL1+cq4W2X7OVSGzSyyavBDiu0S01vPyjQA+VP/7KwD+EcD/jGN/THDcy2ydRi+w5DzMa3REEtEfNnZur6YuW/FJrV8l4N0UiLAohHHMtsc2PFjG7oNHG6sFSdgwTZWZaXaulsv7p1NJ0ib/XiHEmwAghHiTiH5K9SEiuhPAnQDQ39+f4HAYuczW2eDLpWJuY52TqpRoY+fWaerAknDVNdmWLAqBl8eu074f9NiCOmB1E4jKzFR0CtizbYCFe4ZIPbpGCPGAEGJICDHU19eX9nC6Al0vUOmEzWPYW1LRQzZmCj/n44IQRlOan1076LEFqTVjCq3liKzOIEkh/yMiuhAA6r/fSnBfTADc8d1Ac7PnvMY3J1UczCZW3k9Iy+94beWS02fmjdcjifIKEpMgz2vBtbyRpJDfB+AT9b8/AeDvEtwXExAZ310uFVuiavKojaVZKdHk9JbCdXiwjOkd1+D+bQNY3dss7GerZsdo0GMLksRlEuRcfbIziEXIE9FDAP4ZwHoiep2IfhfAGIBfI6J/A/Br9f+ZjNEt2lhS0R82mcRSGy7QUvyS/K0SrsODZfQub3WVmSZe3SRy8vQZ7cRgm8RlEuQcUdMZxBVdc6vmrV+NY/tMcnRC1/s4omKS6gnqZ5f2RtUUnYJv6YOgE+/wYBkTr57AV58+1vT6XG0RI48ebnwmzHk01efp9D6r3VJrhzNeu5wwRbaSQvXQAYgtKiaJnqAmgRy2omSYifep548rX68tCOWEY3se/QR5p/ZZTasvbRqwkO9y0tbGpGD3Zt3Kh27Fsp5ES+9GxSSQw5rCwky8pm1GmXCAzhXkJpIu6ZwlWMgzqT3EXm1K5QDWxZdnxWdgEshhU/3DTLymshVRJpy80k3ng4U8kxqmJCE/suIz8BPIYU1hQSfeka3rMfLIYdQWm6dKp0CRJpy80gm+qLhgIc+kho3WtLrXwbu1RV9B2S4nmm4/uhICQHBTmCnD1GQbB4Cd+2YaJQtW9zrYcf2GyBNOkDF2ClnyRSUNCUM6dbsZGhoSExMTaQ+DaROmEscAGpEogFlQes0+7u/GKXjasR/dPn6hfxW+9+KJJpNW0H3HJZijnoesTBBZGUccENGkEGJI+R4LeSYtVMJCOl/LAR463WSxuteBENBqtkHR7adUdDC945qm18IKEL+Jz0sa9Yai1D5q14TcbZiEPJtrmMTwE3RxRfbozD7ephsn52pNceO24/Tbz2y11lSPPUp4XlDHXxqOwihOy26KaskKLOSZRLAVdHFE9pgiS7zIuPEwAtm0H/c2TYJs4tUTeOiZ17AgBApEuPWqNbh3eGOoYwGAVZp6N3Hjngh7NFUzbZyWfhNEnkwoWSH1KpRMPmlnhUK/hihe3IImyDht49RNrRa/+vSxhoBcEAJfffoY7hk/0rQPVYVQHT8+M4+BXU9gnasJSRBkMxPT972lG1QC3tZpaSqTEKaZvM34ux3W5JlESCIOeXyqYowecWuAp8/MtzTGkLgFTZBxDg+WsWv/jLL3qnubQbXxh555raHNe5u7yFrzuvaMC4uicZw2ZiFvk4/TZ+dRWxDG7+tCXW2amXjxyysIYsrppqzVKLCQZxrE2ZBa5873anK2+xyfqrTEgXtt7F47uylu3D0eU7y0d3zX/fyFeGyyYgy90wkyXU6AVzNWhTzahkcEEYqqSVD1fd1EaGpm4hdqqnovaDNwtu/bwUKeAZBMQ2ovXmEYZJ+7Dx5tEdhAq41dYhM3Dpg1S9X4Hpus4OYrynjq+eN4Y7aKUj2CZ/veaew+eNRYuOuuhw8rTR2yKqX3eMMmigURijbfD5o45J1gK7NVjDzSPBmr7qmg++mmrNUosJBnAMSjFZmEiCokcue+GaM93C0kTeYP2dFKFb1jU5fFuy+5nc1jTyrH99Tzx3Fo9GrfScq7b1WlSAC49ao1La9FEVRBhaLu+7q6QoDZBr9z30zLhFxbFNi5b8Z4PYImKHVT1moU2PHKAIhHK9J9loCWmuXjUxWtzbwyW8XIo4ebHHB+ROloJWur79k2AGBJKzfFq8vjDOpcvnd4I27f1N9UV/72Tf1N0TUSnaBy6/wrlxfg9DSvAkxC0SYSx7uKkedAuPZtajICqM1AptclQZqZAMn1CMgbrMkzAOLRioJswy/KRjoDg6BbedjY/VVauc7Z2UOEdaMHtHZy08R47/BGpVD3otJq4RnPogC2XbmmYTry82mcPjvfeiwAVvU6mJ2rNX1/YNcTyn0nnXwVJKQ27QqqnQILeQZAPLU8gmwjrDmiVHSMGqF3u7Z2f5VWLrVXrzBX2dWbxtgbPXbdK8BUselu05Efuw8eVU6cq3odTH22NVtXd45t6w2pIpC8bQ3jwHZS6Ob4exbyDIB4tKIg2wgaZiiR5QN05hTvqsHW16ATXlJ7DTLWIJVC/IqOyb/XjR5Qfl83bu92deOfrQtjb7KTDpuV3Y7rN2Dk0cNNk4pTIOy4foPvd5Og20MtWcgzDeLIPrXdhs4cYaLksinbrhpMvgabLE7pMA4y1lPVVsFpU1gtTLbtqqKDzWNP+nbT0uFOQnK3KdRhWyYZaJ3sAbSMtR1CtttDLVnIdyidvvz0Jv344fQQdt5wThO0XTXohGOp1/EVbKYkHRMqwakS4EGEj2qicXoIp8/OtyRDqbppqXB6KNDxEdShooD6fnSbkdLUprs91DLx6BoieoWIjhDRNBFxickYCJP+nUVkVMv92wZaoiScAqFUdBpRFrtvuVwZC39o9Gq8PHZdS/SORBeBIQS0WZzeyI4gwsAme1MSNNvWG3ly/nnLWuzs1dqCbxSL5PzzlgU6PlH/8d5vNvdjO8tceDGVUugG2qXJbxFCvN2mfeWevC0/4/AHBM2w1GVXqrI4/fwHRacH79YWA2Vvjk9VAhf68prCdHZ6W6Q9Pox/xH2/2dyPaWrT3dQgRAWbazqQPC4/o/gDbJOSpKDfffAoik4P5mqLLdtSCVg/m/zZefuJ4aJSEfeMH8GDTx/Th2CequKe8SO+oZZhndduxqcqofwjwLn7zVSQTSappZm41O2hlu0Q8gLAE0QkAPytEOIB95tEdCeAOwGgv7+/DcPpfDjT7xzjUxVluQC3JqmaBFRIG7UXKQw+rdHOVdq4zoY+O3dWmfXqRgg0PmMS9GGFc2M/OHdMq3sdrFjWg1PVpXj5ubPzyjBIN/J+M002csK9+Yqyb80fSRL+pjiCCjqVdmS8bhZC/AKAXwfwSSL6oPtNIcQDQoghIcRQX19fG4bT+SSV6Wdbdjbt0q5yDGtHD2D73mltNIgpM1WFtFGrGB4sK2vMAOraM14beqnoAAScPmsvkB965jXj+3IfunG58fvIybkaTp+dx55tAzg0ejV2XL/BWL7Zfb/5lXqW8fw22ax58TdlicQ1eSHEG/XfbxHR4wCuBPDdpPebZ5JYftpEP2Qh3tg7BlNIutQ0bc1Ysz6a661XrVFq4Zvet1oZGujWHjePPWntEJX4JV0BMNr/3djE7tcWBHbtn1GauFYVHRChJTNWjkF+1lQKwkabzpu/KQskKuSJaCWAHiHEj+t/XwPgc0nus1uIe/lp83DpPvNpTVhdEthq5W5N09Z27Wfuund4I14+/g4OvXii8dqlP7US/3LslO/EF8ZfYqOhA/HY5iVuE03QEgOyqFsUU2KW/E2dHqYsSdpc814A/4+IDgP4PoADQoi/T3ifTAj8nGfrRg/4VoI0LauDmHlMn7V52AtETaYAm85R3jrzunH9y7FTTa+98NZpq9DAMP4Sd3VK0zkJ2hkrSaKaErMS7pgns1GiQl4I8ZIQ4vL6zwYhxB8luT8mPKaqh/JG90MX9xzkgfH7rN/DXnQK+MLHmmPq3fZxHSuX6+3xEl19GxWVekatZMtlwfxNmy+5oOF0vWf8CLbvndaeE1UMfVjs1g56glaS9GIzSbTDL5RmXH/ccKlhBoC+t2jQWpAqTTvIA+P3WVMPVJNAkYlTOiF2ysJeHtRk4BbETz1/PNB3X/mPc/H0qnDLpARO8NqfrdgkqamQ5pFqbaFhqvJe03Zp2FkyG0WF4+QZAGpnbhg7r0rTDvLAmMxG60YP4KJSsakzk195Xa9NNUr46SqfCphe3D6NoMJBjtHUStGdWGVbq8Zv4o6yCoiCqn6O1OD9qoUm4ZjNU5gyC/mcE8R55HW06ZxouqJdOttrkAfGJEil5vbYZMXXBKCLBAoSr+1F5wddubygDY2Uk5Muu1W7r/oxmCYHef6itAp0424Y0m6HY9RqoXFr2HnKkmVzTY6JurQ12UeD2F5tnXG6xhZebEwVOqHhjdcuFR2c5/Q0ukGZzo0uxHLu7IJRAxawC4f0fmf3waPG2vTy/AURcLpREID7blryAXjvme17p7E2hP07iO3cVni3yzEb1beQJViTzyHu3pxegixt/eLxbUPsbOP6dY0tVLgffpXmaRIacr/uJt+Af9y/aUUi6+HEYdN2j1XXsq/o9DTGGEcIpRy3ybmsOj/3jB/BQ8+8hgUhUCDCrVetwb3DGwPnVNiu9tqpYYcJU85i2CWJgBpGkgwNDYmJCS5UGQXvw6WCgJZaK3HtO+wNPj5V0ZYNUCHb0KmOt+gUcJ7To0zLt6kPr2txp9uX1PDWhigY5hQI569Yph3rG5rIJvc1tLnmNug6OqnGdWj0atwzfkSZHHb7pn489fxxrakvzLn1fjZrglSOy/YY4oaIJoUQQ6r3WJPPGTb22SScR96iW0GyYcenKhh55LD2fa+z0K256cwyK5b1oOgUlBqf3znSrQL8ViSFgHZ3YOnArvv5C1v8BDJ0VbdN9zUMWptfh42AB86dH13ZhYeeeQ2LPqUmvATJ4s5qHZqsZuuykM8ZfvbZJJa2fmF+NvHntUW1UCAAv3zJBXj6pZMNk8DNV5x7yHXHe6paw55tAw3BVyBqjMdPEJomQZOACSzgAdQWRcNP4B6b3JKpmYlqXDpneZzI86M73gUhtC0Tw57bTiCrYZfseM0ZpocoKeeRTZifCdNnBIB/OXaqIVAWhMBjk5WGE0/nmLyoVMTwYLnh9JXf9xOANpmvOsKGH1bq5Y9Htq5vanHohuo/fo7ipLNf3RE4Ogq0dA6dnuZwJF2Vz7yQlWxdLyzkc4bqIZddlmQt9fGpSqxZgzZhfiZMn5EauJtqbQG79s9gfKqCd95tjcZxC+rA4YURXFQjW9fDKYTLGa3MVjHyyGFj+OiebQM4M7+Ik3M1bbSUKipEN3HY1sZRJZuZopsa5Rg8m18QSwXQ0qxgmiRJVYeNCjtec4jbMVXqdfDOu/NN5hCnhwBCUyRLFAeRzkRAWBJMtjZ5r8nGKZAx2qakiakvFR1M77gGwFL3pKB3uM45aMP4VAW79s9Y27eDoDOB+I3X5BC0jQjy7sN0Xl8Zu87KbFR0CtaJbZ1CWk5hk+OVNfkc4k4r712+rEV41haFsjdo2DR5XamB2zb1W4dY7r7l8iaNc3Wvg92/ebnRBKLTet2vh1kqh7Vpywd8dq6m1ZJX9zqhzDqre53QNl9TzHfY6pC678ljszHTVWsLePDpY7koAiYJW9IhSdjxmnOCOH3COojiqG9vcroFCa0EljJTZX33VUUHhR7Cgsaxq8ItoFWaGdB6rABa0vK9FJ0Cdly/IXC4pVMg7Lh+g9Zp3EPUKPkQNCLFtruU1/fhF69uG7tv46wPqx1nNdSy3bC5JucEibaIYqaQeE1FQqDRUk4nIP0evIFdTwRuuBEVaRrxhm/qTF0rlvUox1ggwqIQLcc6+LknjCYd1fds4uHDmN1MyXMStwnM+z15Lbdc1tcwvQSt8+PGLwfA5hjTjFlPA5O5hoV8zgmSKHO/hf08yr6cAgECTeajKA+sLuEpS+gSz3SJRH7fcwtWXT2csJO1KSHNL4FO51cxoSuW5h6/qX6S6RjDfq9TYZt8F+O1x5psxVE1nF37Z4yTSW1BtAgBG1+Azqbs14c0C6hs1+NTFTw2abY762zebpuvLuHIW8veluHBsjYSp4fIuM2d+2YCCfjVvQ5u29TvG40S1g+R1Zj1NGCbfAcQ1bbotsfqNMj3X/gTyj6ltuMYn6qE1qptHjydTfmRiWNN7fjSotfpgQBpbdRu/MI6dd/znn+TSSRs792dN2xQrsYWhDBu02SacZdLKBUd7LxhQ2MbQxdfYLy3w5b8zVOp4KiwkM84toWebCcCXfOK7714wliSwG8cURpYhH3wxqcq+F4GBDwAvDu/iN+6qt8qHNBk+y4bJljv+XcKBKeHlBp02HR6+fm7Hj7cYgqq1hZw18OHsX3vdCBlY+qz12jf88tyDVuQLE+lgqPCQj7j2NTDMAlguQ2/RiB+UQ66cezaPxOqKYbE/eAFXbGYMm3jRNqOTbVpFsXSBOpn7x2fqljZor2ozn9tQaDHkM8UJVpqu8Y2784cdk/yuuJmqw2lknV474ObryjjwLNvNra/Ypm/lTmOiK+8wEI+49jYFk0C2J0IFTT+270P3XdPztUwPlUJVe7WvXQPWprWO764kbLTG91iCue0GY9uYiLAqGXqtr0o9A7MKKYJm+gYtyKw4/oNGHn0cFPUkQz9DILqPtj7g9eaDnC2WrMyR3V6LZy4SFzIE9G1AP4MQAHA/xZCjMW9jzzHw+qEp8BSBIGpdrqtjdxPSJi0TwCNuitBoytWrli6/XQhkn4mB7+JhbAU3x3UV+D0EHbfcnnLfocHy8ZsVhuhqrtWAmbzm+k4BMyVOsNgWfGgcTxxac66FYuXLFR37BQSja4hogKAvwTw6wDeD+BWInp/nPtoV2PftDAVnJLHauoepMMdqeIX5eBnFmkIroBlW/zqtbi3raq1M7J1vXaX5VIRL49dh6nPXoPbN/VrP1d0Crh9U3/T+VAJeMmO6zdo69NsuaxPexwS3URQcEWvqO7pd96dN9bFEYAyozUsui5YXrwlj6Nme7Yjea/bSFqTvxLAC0KIlwCAiL4O4EYAP4xrB1mt4RwXui5GEl3tdL+6L24B/uDTx7CqXt1wdq7WooX5PUwXlYqBujpJCqR2GrpZVXRaEodkS7rbNvXjtk39LWWOvVrsvcMbG1Ec7hrtXien1J63751urE6GB8vAkSPAmjVAqdT47B8+fqSlr+tjkxUMXXyB9r4bn6pgTtPe0B29otRmFwVKRQc/fnc+1th43ThtetIm4cgMYvbrxkiZMCQdJ18G4O4s8Hr9tQZEdCcRTRDRxPHj6sgPE90QDzs8WG6YNlScqtaa4shX9zq+1RR37ptp0hZnqzXM1qsbejE9TPJBD3q+3eV/dTg9hNNn55VmCgHgwaePYejiC7Bn24C1FksAfnrVebh/20Cjgci60QMY2PUERh49jMpsFRefqDRWSU/s/x7w4Q8DH/gAcGwp9HR4sIxS7/KWbcvoE9UqUmrnJtORVE5MNfJvvWpNy6okTmErx6m6Nk4PYXWvk2jPU10VVW/Z4m6NlAlD0pq8an3ZdPcIIR4A8ACwlPEadAfdEg/rV87X7WTaPPakrx1atSrQhVDq6psQodHAw5QWX/akvMuVguk7BSKcf566LZ57vLsPHvU1DaiceSOPHm7Kvm2cDyHwZ/v/BP/jhhGcOu98XPbf7gDefgt46y3g4EHgjjsA6K+HLp7ctuSxKQqq1OvgsclK0wNEQFMTlaiYxrntyqX+rUmis+2rXsvDSr0dJC3kXwewxvX/zwB4I84ddEs8rO7BV0VlxLGKcZu8dCYjIc6ZKHTXwU/b05UY3v2bl2vD+NzYRrTYOPMA4JeOPYvL//3f8F8n9+Pn/v1F9L/9+tIbd93VEPCA2aygMhfaXhMpwFST6my1Bq+CLaDPfQiDyVQS535M6KJiVGY1Fvj+JG2u+QGAS4loHREtB/BxAPvi3IGpjGqeCFLO128VU3QKVvHLbsGkMxm5BZrtdZBO1O17p3H+ectQdM7dhrLEsG0Z3CgRLSp+75lvAAB+Z3I/frFSdx3dcgvw+c83fc6vA5N3nzbOcamcyHPpLTGgs269US9jELUJjIyi0pEFE2jeAy2SIFFNXggxT0SfAnAQSyGUXxJCzMS9n26Ihw0SoqbSBGWIXbmkLo2rwitA/fwfNtfBazo5OVdD0Skoi6P5lcG1XbHZOvPWH38FH3p5sum1d1ZdgPPHxoCensb43VU2351fUApf97nTdbBys7rXaZQhBs5lEdtUclxVdALnGLixqUIJZMMEmvdAiyRIPE5eCPEtAN9Kej/dgO1kFmRCkA+3TZx1HP6PIA+p9zhWFR0QQRkBZMK2Zvqd3/9Gy2vnnzoBXHIJcO21+Obdf4q7/+HVpgnK6SGgp7X0sPvcmRqV68oYAHaac9EpgAihBZ9tldKsmEC7IdAibjjjNafYTAjuz9gklAXxf+i2Z/uQer9v00bQdJxA86R3+sx8k5b80//5Nm744T+1fHeeerDsM9uBnTtx3198XxvauHLFMu250x0zAS1hjzalhN3IFn4qwvorvBSIMmMCzWOgRdLJnCzkGQD2kwLgv0IwlSjQPaRFp6dRBdPblzao+cGLrruTe4y/M7kPzmKzsJu+8FL84dZP4Y7bbsbw+edrheZstWYMcbXJWpZlEz6zdxqL9ff9BHy5HlWlM7XE4a9IutFGUAGXt0CLMOU8gsJNQ5jY8LPtylBKU7MME2ESfkwdgoClCevHP3ob3/ur38H5Z+cAAD9eXsSffPC38X8HP4LFnkJjv7ZdtpyepfBPaVbaclkfHpusGH0LN19RDnReZASSqu6P+xj9BIXpmEympDiI0mlyiwwAABlrSURBVPUpL+GUcTU3MTUNYU2eiQUb225lturbLMNEGLuryQfQiK/fvRuoC/i//9lfws5f/T38+0++p2W/tvV5aouiEd9fma3iwaeP4ZcvuQCv/EdV+UBXawuBJ76Vy5c1OWnlsQYVfGFDX+NAd23uevgwAL0mm6dAi3b4GFjIu8iThpAEpvNja9u1SQjSEcbuqntYZPekPQeew9f/+I/Rs6oPYx/573i8/xfN+w1YnwdYMsscevEEbt/UH3oV42W2Wmtp8qLS/Pzu6TRL8gZNKMsj7fAxsJCv0w7bWCfjd35sbLtRBHxYu6spfPIze6dx/cxTOHDZB7DnA7ehuqJXWfPn9Jn5hrAMWp/HTVwCXiKPy9RIxuaeTkszDppQlkfa4WPgHq91TMv6rBI0ASZKwozf+TFpHjIxqhxAO3EKhFIxXJ0U93HqCoIBwCKAf3zfEO791TtwekXvksNTiJYyu7J+edB6+XFhagwiUd2rWb+ngyaUeYkjASxt2pHMyZp8nU6Lvw268oi6UvE7P7a2XZuY7CgOP1WylYlTxZ9o+r+2qP5ctbZg7AyVBD0E/NZV/Ri6+AJlOz4v3muU9XtaXl/dsZkUhzytvJNeSbEmX0d3Q2U1/jaolhZVq/M7PzYaifszOmRUQdib3rYQWBgWhDBqnqt7Hdy/bQC3b+qPtB8i4P5tA3jpvutw7/DSOVy0mFx6iJq02k64p4cHy/jCxy439jNQ4Xc/50HLjwvW5Ot0Wvxt0KQinanBVquzOT9BErB04XNRz3eSWqpcYfjVpZe/vXXurRGt2qhNaQZv/9Wbryi3hG5m8Z4O4/z1c6jnRcuPAxbyddKMMgiDjVfeJqzRVquL+/wESawKUnZWd15W9zroXX4uK3XLZX3Y+4PXrB2pTg9h7ux8o/KhqtaOG3ejkjfqxbRsUV0TUz0ilRmpWlvAU88fx303beyIezqoycJUlXXnvhmub+OCk6E6FJtEEr/kHZmE463zHueDECUsVXWMToGa6sDL45DH7Xde3ONZVXRw6t3W8r02eM+133HaJlIRoC3hoNvHutED2ubgL49dZzW+TmN8qoLte6cDTZ7u85E3OBkqh9howibThcw+dS/ngyxrvULD3RREFhI7OVdrKnwWdNkcpqmz6bx4JwC/Co+m5uXufdqYB2yKpOlKR0t02q7fqi6P5ovhwTI+bdFvwE2WfBHthIV8B+O3xNU9/O40/TDLWpXQcMeANzUW8XzXu32Thhm2qbPuvARxyhLp67d792lTWVM1+ai6ZYURun7+krjK82ZtNVA2mOberS1m3hfRLljI5xi/hz9siF3UCBa5fdVksX3vNCZePYF7hzfG3tTZdtJwevwbjLv3aXsekwqV000gsnuS7kiCTKJprgZ0k4vu/t5x/QYAneNfSxoW8hkibk3Jz6QTNqU6agSL3L5qshBYygzVtRTU2eRNWtr4VAW79s9Y2W/LpSLmNM3D3bj3qTuPq4pOS+mBpASNewKxrRGfVB+AOLGZXHT3d7cKdS8s5DNCUpqSSXsMEzY6PlWxqnOuw92T1jRZ7No/g6nPXgMgWlPn8akKRh49bBVFI+u7rxs9YPycN2RSORn1EE6fPVezvp2ar81KK6j5Iq3EKr/JJU/FypKChXxGSENTChoWKSeiKFmfwrVfkzlGatK6h9j2nASpNyM1W924SkUH0zuu0Y7FfR5Vq4F2hfGZBC8BoVYVaTXryHrWbifAGa8ZoRNu5jiySd3Zru1whNmeP7dmq6qpUnQK2HnDBu33hwfLODR6NV4euw6HRq/GrMbc047rqRO85VKxMb6gE43unCR9DTshazfrsJDPCHHczGEKlsnCWwLnTAq670UVUKoM2V5HfQuWik6kfUlszp+3vZ1t0SjT+S71qsevez0q3qJsjqeqWVSB3I5CWirSmlzyRGLmGiLaCeAOAMfrL/1Bvak3oyBqWQVbm75fD1GTSSFItIsXXZ/Q/3XTz7c04nB6yKg1B2Fk63qjTV7XIENlJvImUp0+O9/Yrvd8n9GseOTrcTrZ/YqylYoOdt6wIbJATsP+3WmZ6FkkaZv8HiHEnyS8j1wQ9Wa26bLjFQY627pOYx/Zuj5wAopkUQijbT3McdsISvn/rv0zDeEnk5xsq13K6By38FQlUrknyDlNOcu52iLWjh5oSRIbeeQwdu2fabQMjHrt3ZyZ15TWjJEkY+jZuRoNdrxmiCg3s02XHVubus7EESbLUNJDhPGpSktpAVNXI4nq8xOvnmgqAGaKXolyXm3DESWV2So2jz3p+znv9OptGRhHGWhJ0g7fPGbU5omkbfKfIqJniehLRLRa9QEiupOIJoho4vjx46qPMBaYbM/yIbexqfuZiFaHtCnLyeae8SOB/AAqv8HIo4fxVUWFR7lyibOsbFBnMwGhTVpu4igD7UZWZwyLyf+Q9eYk3U4kIU9E3yai5xQ/NwL4awCXABgA8CaAL6i2IYR4QAgxJIQY6uvrizKc2OmkmtQ2XXZWaZyZBSJrZ9qO6zcsJSSFoFpbwIPPHItcB98UEiknk6DXSnetgzibTbVuwhCkDLTp2kvCnBfA30HfCZFh3Uwkc40Q4sM2nyOiLwL4ZpR9tZtOW4LKMem67EhHoRenh7D7lsutj8lrQy/1OjhVrcGiCgAAfT2YOAVFkPo48n3dtfZzNlNdskdxSusIUwa6MlvVTjZhzTZ+ORxpxdAzdiRmriGiC13/fhTAc0ntKwk6cQk6PKjvskOk1oDPP29Z4IfeHRM+9dlr8JPnRQ8LjDse2lsfx2QeMl3rka3rW8IRJU4PYc/HBhqx50F62AJLmr/pvSBhgvKavDJ2HfZsG9B+LsykadP6kcMcs0uSNvnPE9ERInoWwBYA2xPcV+zEvQRtl+lHF8+sS87RvR6EUz4le204fWZeeU5sTRFeTPVxvJO137XWxaZsu3JN0wSpGqtTIOUksbrXwZ5tA7h/20DLd/xKDvsxPFjWTjhhJs04Wj8y6ZFYdI0Q4r8kte12EOcStN2mH1U0ia4FYBxLalOXHpXZQDpvvSGJ7gqUEnkcO/fNtIQt6rZvU2lTRsGMbF1vvNa7Dx7FgsYW9dTzzYECunBQ1Wuq6xNXzLzObBNWu46r9SOTDtwZSoNN5yVbdF2BZF33dhDn8dhuW9dj9L6bNiqFNqDvjKQ7h15BJrVgOVHYdsfyjlNux9SnNWudhlTXIWhOgGnbnJCUXbgzVAjizLSL2kQ7DpLMHDRt293n1K3Z6royifp2vNvTnUOvABZo1rD9OjLJXqg3X1FuEugCwGOTFZR6HW3Z4Tgdi6ZOW7bXSle62atMhBHYrKl3LqzJJ4ypF2WBCItCdJ1mZNPvtOgUQhdD8/Y21a0a3J83VZ50ly+QBI1KMmGTcGWz6rLt9ZrUio5JD5MmzwXKEmb3waPa5f6CEFYJQVHJWry/3wqmQBSp2qXUsO8ZP4Lte6d9e7leVCpqx3SqWsPu37y8KQmsVHRiE/CAXcKV11msuqY2EUqdGDXGRIPNNQlja5JJKvU8aadvmKW/X0x5lHr10iE4PlUx2tO9nzc5ppM2VdjeI6a2iXd/44jWB+J2kHLiUvfBmnzCBLHbJvGgmTS3qBp+0FLFkpGt67Ux4r1ODwoULKNWlbFrWkEBaPl8mrHetveIX1joU88f9w1l5Prs3Qdr8gmjcvzpQv+SeNBMIYRRNfyw3ayGB8stBcaApZjy2oIIpMkTljR/b/SIacJURTWlWdLWzzkM2Ddg91t1RC1pzXQeLORjQme2UAmPLZf1+S6r40JnGlHZvYOajKIs/e8d3tgSeXP6zLzSfl4gwq1XrcFTzx9HZbaKQr0Ovrdcr3uSMsXu686zrVmmHQ3XTdE1UXI4uD5798HRNTEQJmKhXXHHurHptMYgsd+m+H9p5/Yen+m4baJDbPZ9aPRqbcy4O4Y+DPeMH2lZgbQ7OiWpCBmOhe9cOE4+YcKYLdoRdywf2mptoaH9ugWwrTaoe/h1S/8tl/UpTUETr55oWsHYat+qMfmtIkwaa1hhpnPmtqtBtyQJbbzTCvIx9rCQj4EsRiyoukBJk5B8aG1sszYPv1fY6Ca9h555zdhuMIi92GZCUE2kUYSZyZnb7msdt5IQ1r/CZB8W8jGQxVKrfg+trTZosx3vd7ZrukfpHKqyoUUQDTWoA9Fd08VLtbaAXftnfPdrEuTua92JZo8sKipMPLCQj4EsRiz4FeayFUBhHn6Ts1cn6N3atI1ADDIh2GSUnpyr+bbfs3HmqlYKqsJrNrRzssiiosLEA8fJx0AWS63qHk7Zns42rj1MXLUu5vzWq9ZoywaHybp017U/NHq19nwHbeEnx3P3N55tek11XN6ywLr6MQ8+fSxQHkLYHISwcE34/MJCPiZsBU670AkkndMwyHb8Hn7dpHfv8Ebcd5Nem5WrjLgFWViTQ7W2iHvGjzSSxrbvncZ5Tg9KRadxXHu2DTRp6Lp9uQuv2dDu8gNZVFSYeGBzTU5RmTPCVMMMG8mhM7sMD5aNBcOSiOowHXvZEJ8PAF975lhTRNDJuRqKTkFZDtlvX0EmmzRs5FxpMp+wkM8x3odWF1vuZ3eN8+Efn6ooe826qdYW8Om90432e1H3veWyPmNs+/hUBZ/WOIsXBQJFnYxsXa+tOhrEvs02ciYu2FzTRdiYXtz1bAY/9wQGdj0Ra/XK3QePKnvNqojDDj0+VcFjk5WWxiI3X3Fu4hoeLEPTxlWLTqMeHizjtk39LbV5gtq32UbOxEXXaPKdGNYWN36mF29kiLtZRlxmlKDmBrfWHOYa6hyh3tZ9v3VVP7769LGW7xedHlRrrV1ee4iwbvSAchyqkg1B7zcuP8DERVeUNeBGCXbYNPOI2rLQZh9eZEvAMNcwSKmEe8aPNBK2ZL2coYsviKWhB5NtOl0JTKxpCBHdQkQzRLRIREOe9+4moheI6CgRbY2yn6hwowQ7bLTsqI6/LZf1tZgynAKhVHSUnweAUq8T+hoGCQG9d3gjXrzvI3hl7Dq8eN9HcO/wxpaoE1UZZL6XOpt2h6u2m6g2+ecA3ATgu+4Xiej9AD4OYAOAawH8FRGpA6TbAGfz2VHq1QtaSRTHn84+vu0X12B6xzW4f9sAnEKrEH3n3fnQEStx2Lbd4bGLmpUv30udS96VwEhCXgjxr0II1Zm4EcDXhRBnhBAvA3gBwJVR9hUFbpRgh5/lLqrjz88+PjxYxsrlrW6i2qLQNhKxiQyKM/6b76X8kXclMCnHaxnA067/X6+/lgpZLDuQBbx2SFMvVG9TjjDYPEynNGOQBdbCXMM4Q0DTvJc63W6cVfIeruqryRPRt4noOcXPjaavKV5T6olEdCcRTRDRxPHjx1UfiQxn87WiskPqogilszXq+bLRgnWfkdcs7WuY1r2Ud7txmuQ9XNVXkxdCfDjEdl8HsMb1/88AeEOz/QcAPAAsRdeE2JcV7azf3gmals504i19EOfNbqMFmz4T9BomdT3SyAzlUsDJkfdw1aTMNfsAfI2I/hTARQAuBfD9hPaVCdrRdCFOoWWqsVIuFRO52W0eprgeuLw1wci73Tht8lzSIZKQJ6KPAvhzAH0ADhDRtBBiqxBihogeBvBDAPMAPimECFYGsMNIWtOKW2jp7JBR4+D9sHmY4njg8qb55t1uzCRHJCEvhHgcwOOa9/4IwB9F2X4nkbSmFbfQSsuB6F6NlHodCLHkbI2isau0/rxpvhw8wISla8oaJE3SmlbcQisNO2SUsgkqYQ6gZXUz8shh7No/o23Tp7seWfen5N1uzCQHC/mYSFrTSmISabcd0q95h25lojJVfXrvtLI+fm1RNE0ebnTXo1Ps93m2GzPJwVUoYyLp0Lo8hHmFLZugmxyChGKZrkfeMx6Z7oY1+RhJUtPKw3Ld1FDD/RkvUe3oBBidyXmz3zOMGxbyHUTay/WodmuVScuNbmViMzmYMJUFNm0/C5ErWfcVMNmHzTWMFWEzLt1NSHbtn4HbyLJyeaGpX6rOnKIyVbkpEIEAlIqOssDZghDGMWfVFMZZrkwcsCbfBcShDYYJ4TRF0wBLrfV23rDBKm4eAHbtn2nZhreWu/tYe4iw4Km6phpzVk1heYv1Z9KBhXzOiStyJIzdOmw0jQppqvKbsNwmrbWjB6zHnLYpTAX7Cpg4YCGfc+LSBsPYrZNoQmIrjMenKsoQSyAbtnYbsuwrYDoHtsnnnLi0wTB2axthlJTA2n3wqLbtn27Mbv9BXI3Lo5CWryBr54GJBgv5nBNXk4sweQB+DtMkBZapAJsuozZrTs40yhpn8Tww0WBzTc6JMxM3qN3a69CMo1aNLaYCbCra6eQM4gjPQlYyO3s7GxbyOSftyJG0HJpBJ7d2OTmzXkKBnb35g4V8F5DFyJGkCTq5tcvJmXVNmZ29+YOFPJNbgkxu7Srlm3VNmUsa5w8W8gyD9pm1sq4pp23eY+KHhTzD1EnarDU+VcHc2fmW17OmKXejeS/PsJBnmDbgdbhKSkXHqrQDw4SFhTwTG1mpmJiVcbjRlXhYuWJZ6mNj8g0LeSYWshIamJVxeMm6w5XJL5zxysRCVrorZWUcXuLKPGaYoEQS8kR0CxHNENEiEQ25Xl9LRFUimq7//E30oTJZJiuaalbG4SWrNeuZ/BPVXPMcgJsA/K3ivReFEAMRt88kSJy266yEBpZ6HWUj77Q1Zg5NZNIikpAXQvwrABC1duNhsk3ctussJNGMT1XwzrutIYpOgTKhMXNoIpMGSdrk1xHRFBH9ExH9iu5DRHQnEU0Q0cTx48cTHA7jJm7bdRoVE73sPngUtcXWAsMrl3MEC9O9+GryRPRtAD+teOsPhRB/p/namwD6hRD/QURXABgnog1CiP/0flAI8QCABwBgaGhIVQKcSYAkbNdpa6q6sZ+qtppvGKZb8BXyQogPB92oEOIMgDP1vyeJ6EUAPwtgIvAImUTIig09TvJ4TAwTlUTMNUTUR0SF+t/vA3ApgJeS2BcTjjxGe+TxmBgmKpEcr0T0UQB/DqAPwAEimhZCbAXwQQCfI6J5AAsAfl8IcSLyaJnYyGO0Rx6PiWGiQkJkxww+NDQkJibYosMwDBMEIpoUQgyp3uOMV4ZhmBzDQp5hGCbHsJBnGIbJMSzkGYZhcgwLeYZhmBzD9eQZpoPIYkMUJtuwkGeYDiGrDVGYbMNCnsk1edJ8TUXlOvWYmORhIc/klrxpvlltiMJkG3a8Mrklq60Aw8ItBJkwsJBnckveNF8uwMaEgYU8k1vypvlmoTEL03mwTZ7JLVloSRg3aTdmYToPFvJMbuHSwwzDQp7JOaz5Mt0O2+QZhmFyDAt5hmGYHMNCnmEYJsewkGcYhskxLOQZhmFyTKYaeRPRcQCvpj0OD+8B8Hbag7Ckk8YK8HiTpJPGCvB4o3KxEKJP9UamhHwWIaIJXRf0rNFJYwV4vEnSSWMFeLxJwuYahmGYHMNCnmEYJsewkPfngbQHEIBOGivA402SThorwONNDLbJMwzD5BjW5BmGYXIMC3mGYZgcw0JeARHdQkQzRLRIREOu19cSUZWIpus/f5PmOCW68dbfu5uIXiCio0S0Na0x6iCinURUcZ3Tj6Q9Ji9EdG39/L1ARKNpj8cPInqFiI7Uz+dE2uPxQkRfIqK3iOg512sXENE/ENG/1X+vTnOMbjTjzfx9K2Ehr+Y5ADcB+K7ivReFEAP1n99v87h0KMdLRO8H8HEAGwBcC+CviKjQ+vXU2eM6p99KezBu6ufrLwH8OoD3A7i1fl6zzpb6+cxiLPeXsXQ/uhkF8B0hxKUAvlP/Pyt8Ga3jBTJ837phIa9ACPGvQoiO6fZsGO+NAL4uhDgjhHgZwAsArmzv6DqeKwG8IIR4SQhxFsDXsXRemZAIIb4L4ITn5RsBfKX+91cADLd1UAY04+0YWMgHZx0RTRHRPxHRr6Q9GB/KAF5z/f96/bWs8Skiera+LM7MMr1Op5xDNwLAE0Q0SUR3pj0YS94rhHgTAOq/fyrl8diQ5fu2QdcKeSL6NhE9p/gxaWlvAugXQgwC+AyArxHRT2Z4vKR4re0xsz5j/2sAlwAYwNL5/UK7x+dDJs5hQDYLIX4BSyamTxLRB9MeUA7J+n3boGvb/wkhPhziO2cAnKn/PUlELwL4WQCJO7fCjBdLWuca1/8/A+CNeEZkj+3YieiLAL6Z8HCCkolzGAQhxBv1328R0eNYMjmp/EtZ4kdEdKEQ4k0iuhDAW2kPyIQQ4kfy74zetw26VpMPAxH1ScclEb0PwKUAXkp3VEb2Afg4Ea0gonVYGu/3Ux5TE/UHWvJRLDmRs8QPAFxKROuIaDmWHNn7Uh6TFiJaSUQ/If8GcA2yd05V7APwifrfnwDwdymOxZcOuG8bdK0mb4KIPgrgzwH0AThARNNCiK0APgjgc0Q0D2ABwO8LIVJ3yOjGK4SYIaKHAfwQwDyATwohFtIcq4LPE9EAlkwgrwD4vXSH04wQYp6IPgXgIIACgC8JIWZSHpaJ9wJ4nIiApef7a0KIv093SM0Q0UMAPgTgPUT0OoAdAMYAPExEvwvgGIBb0hthM5rxfijL960bLmvAMAyTY9hcwzAMk2NYyDMMw+QYFvIMwzA5hoU8wzBMjmEhzzAMk2NYyDMMw+QYFvIMwzA55v8DZ9SS9Hg/tqwAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.scatter(Xs[:, 0], Xs[:, 1])\n",
    "plt.quiver(*[0, 0], *oja_vec, color=['r'], scale=21)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3. Sangerovo pravidlo\n",
    "\n",
    "Zovšeobecnený Hebbov algoritmus, ktorý umožňuje trénovanie sietí aj s viacerými výstupmi, sa riadi Sangerovým pravidlom:\n",
    "\n",
    "$\\Delta w_{ij} = \\gamma (y_{i}x_{j} - y_{i}\\sum_{k=1}^{i}w_{kj}y_{k})$\n",
    "\n",
    "a pre vektorovú formu:\n",
    "\n",
    "$\\Delta \\bar{w} = \\gamma (\\bar{y} \\cdot \\bar{x}^{T} - tril(\\bar{y} \\cdot \\bar{y}^{T}) \\cdot \\bar{w})$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[ 0.6528286   0.75750566]\n",
      " [-0.75749144  0.6528451 ]]\n"
     ]
    }
   ],
   "source": [
    "W_sanger = np.random.normal(scale=0.1, size=(2, 2))\n",
    "prev_W_sanger = np.ones((2, 2))\n",
    "\n",
    "learning_rate = 0.1\n",
    "nb_iterations = 2000\n",
    "t = 0.0\n",
    "\n",
    "for i in range(nb_iterations):\n",
    "    prev_W_sanger = W_sanger.copy()\n",
    "    dw = np.zeros((2, 2))\n",
    "    t += 1.0\n",
    "\n",
    "    for j in range(Xs.shape[0]):\n",
    "        Ysj = np.dot(W_sanger, Xs[j]).reshape((2, 1))\n",
    "        QYd = np.tril(np.dot(Ysj, Ysj.T))\n",
    "        dw += np.dot(Ysj, Xs[j].reshape((1, 2))) - np.dot(QYd, W_sanger)\n",
    "\n",
    "    W_sanger += (learning_rate / t) * dw\n",
    "    W_sanger /= np.linalg.norm(W_sanger, axis=1).reshape((2, 1))\n",
    "\n",
    "print(W_sanger)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Kódy a riešenia nájdete [tu (Ojovo pravidlo)](sources/lab10/lab10-oja-solution.py) a [tu (Sangerovo pravidlo)](sources/lab10/lab10-sanger-solution.py)."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}