online3.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the <a href=\"more-online-learning.html\">last post</a> I went over AdaGrad and described a method that we can use to \"automate\" choosing a learning rate. The analysis techniques presented were all fairly elementary, essentially just plugging things into Zinkevich's online gradient descent regret bound.\n",
    "\n",
    "However, many modern papers will present results in much more complicated language. It's often possible to rephrase these results in simpler terms, but it's still a good idea to understand the more advanced techniques. For one thing, if you don't understand the abstractions then you can't understand proofs that use them regardless of whether or not the abstractions are necessary. Secondly, it's not always possible to easily sidestep the abstractions. Finally, the math involved is just interesting in its own right.\n",
    "\n",
    "So in this post I'm going to give an overview of the relevant concepts. In the interest of not making this overly long, we probably won't actually prove any regret bounds with these ideas yet - I'll save that for a seperate post. Essentially, this is a primer on some concepts in convex analysis."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Most of this post will be setting up a definition of some kind and working through its immediate consequences. The first of these is the concept of a dual norm:\n",
    "\n",
    "<div class=\"definition\">\n",
    "<h4>Dual Norm</h4>\n",
    "Given a vector space $V$ and a norm $\\|\\cdot\\|:V\\to \\R$, its dual norm $\\|\\cdot\\|_*:V\\to \\R$ is defined by\n",
    "$$\n",
    "\\|x\\|_* = \\sup_{\\|z\\|=1} \\langle z,x\\rangle.\n",
    "$$\n",
    "</div>\n",
    "\n",
    "For a finite-dimensional vector space (which is the only kind we'll be worrying about here), we can replace the $\\sup$ with a $\\max$.\n",
    "\n",
    "Note that here were are using norm in the <a href=https://en.wikipedia.org/wiki/Norm_%28mathematics%29>general sense</a>; it's not necessarily the euclidean norm $\\|x\\|=\\sqrt{\\langle x,x\\rangle}$.\n",
    "\n",
    "It's a good idea to go through and verify that this is in fact a norm; the properties follow fairly readily:\n",
    "$$\n",
    "\\|x\\|_*\\ge \\left\\langle \\frac{x}{\\|x\\|},x\\right\\rangle\\ge 0\n",
    "$$\n",
    "with equality only if $x=0$.\n",
    "$$\n",
    "\\|x+y\\|_*=\\sup_{\\|z\\|=1} \\langle z,x\\rangle+\\langle z,y\\rangle\\le \\sup_{\\|z\\|=1} \\langle z,x\\rangle+\\sup_{\\|z\\|=1}\\langle z,y\\rangle = \\|x\\|_*+\\|y\\|_*\n",
    "$$\n",
    "and for some scalar $c$:\n",
    "$$\n",
    "\\|cx\\|_*=\\sup_{\\|z\\|=1} \\langle z,cx\\rangle = \\sup_{\\|z\\|=1} c\\langle z,x\\rangle=c\\|x\\|_*\n",
    "$$\n",
    "\n",
    "In general, a thing is a \"dual\" to something else when the \"dual of the dual\" is the original thing, so we'll sketch a proof of this. Note that this is actually <i>not easy</i> to do! We'll need a somewhat complicated result called the <a href=https://en.wikipedia.org/wiki/Hahn%E2%80%93Banach_theorem>Hahn-Banach Theorem</a>."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div style=\"margin-left: 3em; margin-right: 6em;\" class=\"aside\">\n",
    "<p>\n",
    "First let's notice that since $\\|\\cdot\\|_{**}$ is a norm, we need only show that $\\|z\\|_{**}=\\|z\\|$ for $\\|z\\|=1$. So:\n",
    "$$\n",
    "\\|z\\|_{**} =\\sup_{\\|x\\|_*=1}\\langle x,z\\rangle \n",
    "$$\n",
    "Now notice that $\\|x\\|_*=1$ implies that $\\sup_{\\|z\\|=1} \\langle x,z\\rangle =1$, so that\n",
    "$$\n",
    "\\|z\\|_{**} =\\sup_{\\|x\\|_*=1}\\langle x,z\\rangle \\le 1\n",
    "$$\n",
    "So that's halfway there - now we need to show that $\\|z\\|_{**}\\ge 1$. Unfortunately, this part gets more hairy. Let $\\phi(x)$ be a linear functional defined on the subspace $(z)$ spanned by $z$ such that\n",
    "$$\n",
    "\\phi(z)=\\|z\\|=1\n",
    "$$\n",
    "Then $\\phi(x)$ is dominated by $p(x)=\\|x\\|$ on the subspace $(z)$ spanned by $z$. By the Hahn-Banach theorem, we can extend $\\phi(x)$ on $(z)$ to a linear functional $\\psi(v)$ on all all of $V$ such that $\\psi(z)=\\phi(z)$ and $\\psi(x)\\le p(x)$ for all $x\\in V$. Now since we're dealing with finite dimensional vector spaces, we can identify linear functionals with inner products. That is, there exists some vector $y$ such that $\\psi(x)=\\langle x,y\\rangle$. The conditions on $\\psi$ imply that $\\|y\\|_*=1$. Further, $\\langle y,z\\rangle = 1$ so that\n",
    "$$\n",
    "\\|z\\|_{**}\\ge \\langle y,z\\rangle = 1\n",
    "$$\n",
    "which shows that $\\|z\\|_{**}=\\|z\\|$.\n",
    "</p>\n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "What's actually happening here is that we're defining a norm on the dual space $V^*$ of linear functionals on $V$:\n",
    "$$\n",
    "\\|f\\|_*=\\sup_{\\|v\\|=1} f(v)\n",
    "$$\n",
    "The existence of an inner-product allows us to identify $V$ with its dual space."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The next concept is the Fenchel conjugate function:\n",
    "\n",
    "<div class=\"definition\">\n",
    "<h4>Fenchel Conjugate</h4>\n",
    "Suppose $f:S\\to \\R$. Then the Fenchel conjugate function to $f$ is\n",
    "$$\n",
    "f^*(\\theta) = \\sup_{u} \\langle u,\\theta\\rangle - f(u)\n",
    "$$\n",
    "</div>\n",
    "\n",
    "When $f$ is a closed convex function, then $f^{**}=f$. Intuitively, the graph of any convex function can be described by an intersection of half-spaces tangent to the graph. The conjugate function gives the (negative) $y$-intercept of the hyperplane tangent to the graph with slope $\\theta$, as you can see from the graphs below:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# <!-- collapse=True -->\n",
    "from matplotlib import pyplot as plt\n",
    "import numpy as np\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAswAAAFFCAYAAAD4hruXAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzs3Xe8HFX5x/HPpAEhQBJAOgT4oTQhSgtNzgWVqiCgIEVA\nFBAE9YcCQRFQpApSRLGAKBDLT5TeZUJHQCTUQBJCCZ0QwAZSnt8fZ26yd7N7d3Z3Zp45O8/79bqv\n3NmdOfM94XIyd/bMecAYY4wxxhhjjDHGGGOMMcYYY4wxxhhjjDHGGGOMMcYYY4wxxhhjjDHGGGOM\nMcYYY4wxxhhjjDHGGGOMMcYYUzlDtQMYM4iVgdcHeX8Z4H3gnZzOvxqwBbAb8A/ghZzOY4wxvabV\n+A02hhtjTNdWAT7fYp9hwHE5ZvgGsCGwCDApx/MYY0wvSTN+g43hxhjT0qLAhwZ5/5SU7WwAfKH7\nOINaE/hBiv0WYf4+bQhMzDyRMcaUV9rxG8o1hhvT1DDtAKYyvgEsATwD3AVsAiwN3AHcWLfvusCs\nlO3eCxwK/CabmA19hnSD7eeA62q2hwDfA+7MI5QxxpRI/xj/EvBsG8eVaQw3pqkh2gFMJSyGv5i8\nHLg9eU0G2X8H4OY22n8F+J/OorX0aeBsYLkU+64APFez/VngJiDKIZcxxpRF7Ri/CBC3eXxZxnBj\nmrI7zKYIGwEPAPfUvPYK/k5y/d1l8B/RndhG+1OA9YDpNa+tAnx5kGPuxg/ug/kMcDT+7sdkBr9D\nsTowtWZ7SeA9fD8XbnEeY4wJWe0YfzTtjd9QjjHcmEHZBbPJ20bA14Dn8YPXn5PXX6D5E8sjGXgH\n+tP4i8/NgYeAbfADX/8F6hzgg3VtPEn6ucPN2v9zTd5WdgR+VLO9M/Bz8p+bZ4wxmvrH+OfwY3z9\n+A1hjOHGGKPuj8Babez/l5rvV2TeR3V/A0bjp2yMrNnn43T+YF2a9lsZCnynZnsCMD75fl/g2A6z\nGWNMCGrH+L/UvRfCGG5MS3aH2RRhDeDRNvZ/t+b7Z5I/l8Kvo/k6cFXd/osBr9W9lvbjvDTtt/Jx\n4Iaa7Q3wg/XWwKbAQvg7IFe02a4xxoSgdox/t+69EMZwY1qyC2aTt6WAVxn8Ib96LwKjgH/i5wYv\nAHwUuDV5fwcGDojLAI/VtZH247w07bcyATi+Zvucmu+Pw/fdLpaNMb2ofoyvHb8hjDHcmJZslQyT\nt43wS8e14xb8+sUAn8QPfhGwIH6O3Mt1+4/v4Bz90rQ/mNH4+XeNfA5/Z/nT+BUzjDGm19SP8bXj\nN5R/DDfGGFXrAefhn5Zet81jRwMnpNx3QeCMNtvP0pfx60kbY0yVNBvj2xm/QX8MNyaVNHeYt8E/\nyToNOLLB+3vil4R5EP8b4jo17z2VvP53Bi4pZnrfe/jiI6/ifz7a8Xpy3BIp9t0d+Fmb7WdpWfxH\nkMaU3QX4ohIPDbLP2fixfgrwkSJCmWA1G+PbGb9Bfww3JhND8esijgOG49dZXKNun43xE/bBX1zf\nXfPeTGBsvhFNj4oY/IEP8IVCdiwgSzOr4D/+MyYEm+MvgptdMG8HXJN8vxEDx3Jj2pFm/Ab9MdyY\nzGzMwFK/RyVfzYxhYEnjmcDiOeQyxhjTvnE0v2A+D9itZnsq/oEuY4ypvFZTMpZjYE34WQxeXnJ/\n5t2hAP/U7E3AfaT7bdMYY4yORuP98kpZjDGmVFotK9fOUmB9wBfx68722xRfzW1JfAnkqcBtAw9b\n+j140VbrMMaEagbzCieELqrbbvBvwJL/hVeGF5LGGGOyl8uYPYGBUzIm0vjBv3Xwc50HC3AscHiD\n1wVkg44TVs9x2gECc5x2gMAcpx0gHHI4yHm0d2NB2zgGn5Kxe812sykZAnJNg9dNc8dpBwjMcdoB\nAnOcdoAwyLdBzqfDMbvVnd37gNXwg+wI/Py2+gIMKwJ/AvbCXzT3Gwkskny/MH6txMEeNjHpjNMO\nEJhx2gECM047QEC2Ba7VDpGhK4AvJN9PwK928FKTfdcD+WAhqXrDOO0AgRmnHSAw47QDlJ8MBw7G\nrwTUkVZTMt4Fvgpcj18x43x8NZ4Dk/d/BnwX/7DfT5PX3sEvWr40/kK6/zyXMLB8cK1tGVgpzRhj\nSkwWwa8ksZN2kjb8FtgCv9zXs/hP/fqnVvwM//zJdvgbH/8C9hukrV/g/204LK+wxhiToZ2BaRC1\nu8xtqQjIGyBLagcJhNMOEBinHSAwTjtAGGRHkJv6N1SjFE9AlgN5DWSx1rsb7P+rdjntAIFx2gHK\nT+4E2bl/QzVKFwTkMpA9tYMYY0w68jOQ/+3fUI1SvKS/8juQr+lGMcaYVmQDkKdB+mdVBDtmC8gB\nIJdoBwmE0w4QGKcdIDBOO0D5SQTyDEh/EadgB98O9V8wbwIyHcRWOWrNaQcIjNMOEBinHaDc5Dcg\n36p9oZNWyjLQXQtsDTJUO4gxxrSwFr4s8FTtIMruAubgn0ExxpgSkqWBT+GfwQte/92Kh0Am6EYx\nxphW5FsgP6l9QS2Kjpr+yt4gzR7mNsYYZXKsn0I38EWVKBnov2A+FcRWyjDGlJzEIDvUvqAWRUft\nBfMCIC+CrKkXxxhjGpERIC+ArF3/hkqcDPRfMDuQe1WThMFpBwiM0w4QGKcdoNxkUZB/gCxc+6Ja\nHB11/ZXj6+64m/k57QCBcdoBAuO0A5ST7FmzmtGANzpprSxzmAHuAFYDaVRZyhhjyuDjwJ0Q/Us7\nSIkkFQJltHYQY4ypcRhdFCopo9qP9y4F+ULzXY0xRpP8AuTr9S+qRNHToL9ysS8VbowxZSATQJ5s\nsphEsGN27QXzl0B+qxfFGGOakQhkFsiH6t9QiaOn0QXzhiAzbaUjY0w5yCSQbzR7s9AoGaq9YF4O\nZHbN4tJmfk47QGCcdoDAOO0A5SXrgMzwF84D31CJo6dJf+VuXwHRNOC0AwTGaQcIjNMOUC6yLMic\nQaaJBT+HGYieA2YBG2onMcaYOtsB10JUtQvktM7Gzxk0xhhNBwGTIHpdO0jW6p+4PgnkBJ0oxhjT\njNwCsl2jNwqPoqvZHeYRIM+DfLjYOMYY008WBHkJZPXBdiosTsbqL5g/BvI3nSjGGNOIjE6WkxvZ\n6M3C4+gapL9yDMjPi4tijDG1ZB+Q61rtVEiUHNRfMA9P5p4srROn9Jx2gMA47QCBcdoBykl2Bbm2\n2ZuFRtE32AXzUsn4vXhxcYLgtAMExmkHCIzTDlAOEoHc3+STwAE7dtJ6yeYwA0TvADcB22gnMcaY\nRDJ/2Qwuegm4AthfO4kxpnI2ARYBWt1hDlajJYr2A/lD8VGMMaaeREl51f9ptkOhcfS16K+sB/K0\nrXZkjCmW/B4kzYPHwY7ZjS6Yl0k+1rMB1xijTD4CMm2wHQqLUg4p+iu3g+ySfxRjjAGQFUBeA1k0\nzc6dnKGEUzIAoheAmcDG2klKyGkHCIzTDhAYpx2ghLYFrtEOERhbYm4gpx0gME47QGCcdoAS+Apw\nEURv5nWCkl4wA36+4LbaIYwxlWfzl9v3Z2AVkPHaQYwxvU4WAr4E/Fg7Sd6arem5KcgDxUYxxpha\nMgbkzWRAbrpTYXHKIWV/ZSLI+flGMcYY+SLI1e0ckFuUnDW7YB6WzEdZrtg4xhjTT3ZLMRAHO/h2\nKO0F8xLJsyhL5hvHGFNdEoFMAflkOwd1cqYST8mI3gVuwJaXq+e0AwTGaQcIjNMOUDI2f7lj0avA\nn4AvaycpAacdIDBOO0BgnHYARR8DRgA35n2iEl8wAzaP2RijRobgxx+bv9y5s4GDfUEqY4zJ3GHA\nORBV4pO+VlWjXrfB1hhTPFkPZGqaHXOPUi5t9lcmg3wulyTGmAqTlUBmg4xq98BOzlbyO8zRS8B0\nfPUWY4wpkq2OkY2zga9phzDG9JxDgAsh+qd2kKK0qhr1PZBTiokSBKcdIDBOO0BgnHaA8pA7QT6R\nZsfco5RLu3eYhyWV/9bPJ04QnHaAwDjtAIFx2gGKJwuDvAqySicHd3LGkt9hBmweszGmcLI4sDZw\nq3aS8EXvAucCh2onMcb0jD2BOyB6UjtIkVrdYR6a/BaxQjFxjDFGPg9yRdqdc41SPh30V8YmS8wt\nlX0cY0y1SATyMMhWnTbQyUEB3GGO3sOWlzPGFKtXV8fYBpgKTAOObPD+EsB1wAPAw8C+2Zw2eg34\nA3BgNu0ZYypsy+TPm1VTKEhxpS97gfw5/yhBcNoBAuO0AwTGaQfQJ0NAXgYZl/aAPNNkaCj+Iepx\nwHD8RfEadfscB5yUfL8EMBsYVrdPh/2VtUGeBxnR2fFBc9oBAuO0AwTGaQcollwOckA3DXRyUAB3\nmAG4HtiyogOtMaZY6wGvQvSUdpCMbYi/YH4KeAf4HbBj3T4vAIsm3y+Kv2B+N5vTRw/j71rvlU17\nxpjqkTWACcAl2kk0pC2z+leQLVvvZ4wx3ZBjQX7YzgG5RcnWrsAvarb3As6p22cIMBl4HvgHjR+4\n7qK/4kCe8M+mGGNMu+Q3IN/utpFODqr/qK3M+lfLqNycFWNMobYFuh2QyyjNPxJH46dqOGBVfLnZ\ndfEXz7UuxN+pBng9OWZysu2SPxtt3wJXvA0PHAt8N8X+tm3btm3bybasAjftCIfuyTxpjh8PjE62\nxxGwtHeYNwR5JN8oQXDaAQLjtAMExmkH0CVLgrwBskA7B+UWJ1sT8A/09ZvI/A/+XQNsWrP9F6B+\n/eQu+yvbgTyUlB6vCqcdIDBOO0BgnHaAYsh5ID/IoqFODgppwLoP+IAvhWiMMbn4JBBD9LZ2kBzc\nB6yGv8MyAtgNqF86byrw8eT7pYAPAVmvc3otfg71pzJu1xjTs2Q54HPAmdpJNLVxpS8XgRyUXxRj\nTLXJJR08fR3KHWbw000exz/8NzF57UDmLfe2BHAlMAV4CNijQRsZ9Fd2AbnHr6dqjDGtyI9Azsiq\nsYzaKVw7F8x7+OVEjDEmax0XSQp28O1QFhfMQ0AeTVl63BhTabIkyGsgy2bVYEbtFK6dC+bFO5hf\n2GucdoDAOO0AgXHaAfTIBD+3tv0DM49Sbhn1V/YGmZxNW6XntAMExmkHCIzTDpAv+YGfv5xdg50c\nFNIcZiCaDTwKbK6dxBjTc3q1ul9Z/RZYEWQz7SDGmLKS0cBBwCnaScqgzSt9OSbDeSzGGJOQe/06\nwe0fmHWSksuwv3IAiP2SYoxpQr4D8uusG824vcK0e8G8Pshj+UQxxlSTLAXyOsjwTg7OPE65ZXnB\nvADILJD1smvTGNMbZBTIyyCrZ9ZkHH+QakzJAOB+YCzIytpBlDjtAIFx2gEC47QDKNka+AtE72gH\nqZbobeCH+IIpvcxpBwiM0w4QGKcdICcHALdANDWT1uJ4eSDu9PAAL5ij9/GL7zcq2WqMMZ2w+ct6\nfgFsBrKmdhBjTFnIgsDhwImZNBfHCwGXAWdl0p6SDm6Ny24gV2UfxRhTPTIMZHayMH5HDWQap/xy\n6K8c7dfZN8YY8DU35OpMmorjiDieRBxfTBxHBDxmd3LBPBbkzeQ3EGOM6YJsAjKlmwYyixKGPC6Y\nF0vWwF41+7aNMWGR4SBPgWycSXNxfCRxfG9ylxkqNIcZiF4DHgS20E6iwGkHCIzTDhAYpx1AwXbA\nNdohqi16A/gpcKR2kpw47QCBcdoBAuO0A2RsD+BJiO7quqU43gE4DNiJvr7/dNNUmgvmbYCpwDQa\nD2Z74suoPgjcAazTxrHduBabx2yM6Z7NXy6Hs4BdQZbXDmKM0SJDgYnAD7puKo7XAC4AdqGv77mu\n22thKDAdGAcMBx4A1qjbZ2NgseT7bYC72zgWOv54Tz4CMg0k6ux4Y4yRZUDmdLic3NxGMosThhz7\nK6eDnJlf+8aYcpPPgtzd9bVdHI8hjqcRx/s0OklXbTexMX5Fin5HJV/NjAFmtXlspxfMEciz9mS1\nMaZzciDIpG4bySRKOPK8YF4W5DWQD+R3DmNMOUkE8gDIp7pqJo6HEcc3EMfNitzlMod5OeDZmu1Z\nyWvN7M+8uYDtHtumSIA/ATtn12YQnHaAwDjtAIFx2gEKtjN+HDGlED0P/A74hnaSjDntAIFx2gEC\n47QDZGT75M9uV0E7DX9RfESX7QzQ6oK5navwPuCLzJurXMRdlypeMBtjMiFjmP+TMKPvVOCA5L+P\nMaYSJAK+DZyY3BDtTBzvh7/w3p2+vnczCgfAsBbvPwesULO9AvOmXNRaB7/4/DbAnDaPBbgQeCr5\n/nX8fOfJybZL/my0fTvcuDJs83m47rcp9u+F7f7XypKn7Nv9r5UlT9m3+18rS548tz8Ff3wQPrt+\nm8ePB0Yn2+MwGYueArkS+CrwfeUwWZmsHSAwk7UDBGaydoAM9OGn9V7acQtxvDFwCrAFfX1zWu2e\ntWHADPw/CiNo/ODeiviH+yZ0cCx0fSdafg5yeHdtGGOqRy4D2TuLhjJoIyQF9Fc+BPIyyKj8z2WM\n0Sd/Adm348PjeHni+DnieLs0J+v4PC1sCzyOvyiemLx2YPIF8EtgNvD35OueFsfW6/aCeRuQO7pr\nIyhOO0BgnHaAwDjtAMWQUUnxoyw+9rcL5nxO84ceuhnitAMExmkHCIzTDtAd2TgpVNLZakVxvBBx\nfB9xnHbOcrBjdrcXzCOSZaGWySZO6TntAIFx2gEC47QDFEN2Bbk+q8YyaicURV0wrwvyfI9UdHXa\nAQLjtAMExmkH6I5cBfKVjg6dv+x1qhN2dK4SyCC4XNzxX7YxpoJkkl9SLpvGMmonFAX2V64EObi4\n8xljiiXjQZ7r+BfjOD6qrux1qpN2dK4SyOKCeWeQG7tvxxjT+2SB5FOppbNqMKN2QlHkBfOErj6q\nNcaUXBdTr+J4h2TecrtLFgc7ZmdxwbwwyBsgY7tvq/ScdoDAOO0AgXHaAfIn24PclmWDGbYVgoL7\n2+XDQOXgtAMExmkHCIzTDtAZWb3jh3vjeE3i+GXiuH7BiVQn7uCYluswByL6F/AXoLvqMMaYKrBi\nJWH5ATARZKh2EGNMpo4CzoHon20dFcdjgcuBb9HXd3cewcoqo7sVsifIFdm0ZYzpTTIM5BWQlbJs\nNMO2QlD0HeYI5E6Q3Yo9rzEmP7IyyGyQ0a33rdG67HWqk3dyUI/cYQbgasCBLKIdxBhTWpsDT0P0\ntHYQk1Yk+LvMRyfVwIwx4TsC+BlEr7d5XC5lr0OR4d0KuRbkc9m1V0pOO0BgnHaAwDjtAPmSc0CO\nzrrRjNsrO4X+SgTyd5Adiz93Jpx2gMA47QCBcdoB2iPLg7wG8oG2Dovj/YjjJ4jjbtfPr/wdZvDz\nEnfWDmGMKSMZAnwGm78coEiAY4AT/bQaY0zAjsffXX459RHzyl7vqFH2uiyyvMP8AZDXe2She2NM\npmQjkEfzaDiHNstMqb8SgdwC8iWd8xtjuidrJytjpJ+73F7Z61QhMmqncBkHl1tAdsi2TWNM+ORU\nkBPyaDiHNstMsb+yIcgsv5SoMSY8chXI11Pv3n7Z61QhOjmo16ZkQO9Py3DaAQLjtAMExmkHyIdE\n2HJyANsAU4FpwJFN9nHA34GHgcmFpEotuge4A0j/D245OO0AgXHaAQLjtAOkIw5YE/hpqt19qevz\n8WPWabnFCkjWd5hXBHm1hytDOe0AgXHaAQLjtAPkQ9YBmZnTKguh3GEeCkwHxgHDgQeANer2GQ08\nAiyfbC/RoB3l/sr/JGP8kro52uK0AwTGaQcIjNMO0JoMAbkX5POpD+ms7HWqMBm3V5gcgss9IFtl\n364xJkxyHMjpeTWeU7tZ2xi4rmb7qOSr1sHA91q0U4L+ytn+yxgTBtkN5L7k4evWOi97nSpMJwf1\n4pQM6P1pGcaY9th0DFgOeLZme1byWq3VgLFADNwH7F1MtLZ9H9jD3202xpSbjABOBI6A6P2Wu8fx\nmsAFwC709T2Xc7jUenV5nj8Bk0EOTfUfJyyO0s0rLDWH/X21w9Fzf1+yGrAkcJd2EmVp7qoMBz4K\nbAWMxP+d3Y2f81zrQuCp5PvX8dM7JifbLvkzx+0IkB/h/xH+Sf7n63p7PHBmifKUfdv+vnrq7+vM\nneHrj0N0c8v9V1ppB+655zw23LC/7HUWecbjp5uBn5IWrJw+3pOHQTbOp21VTjtAYJx2gMA47QDZ\nkyNB0j1k0uEJcmw7SxMYOCVjIvM/+HckcFzN9i+BXev2KUl/ZWSyYsZG2klScNoBAuO0AwTGaQdo\nThYDeQnkwy13zabsdapQObefm7wumL8HYk9VGlN58leQT+R5ghzbztIwYAb+DssIGj/0tzpwE/4B\nwZHAQ/in2muVqL+yf7KUqJXMNqaU5ESQX6XaNY5/RBxfTxznPfuhRGNYe/K6YB4PMsMGUmOqTFYA\nmZ3zqjkhDb7bAo/jV8uYmLx2YPLV75v4lTIeAg5r0EaJ+ivDQB4B+ZR2EmNMPVk+GX9XaLlrdmWv\nUwUr4By5yOuCOQJ5EmTdfNpX47QDBMZpBwiM0w6QLTkU5Nd5nyTn9sumZP2VHZKL5jI/k+O0AwTG\naQcIjNMO0JicD3JSy93ieGPi+GXiuP4Tr7zYKhkDRQJcCuyincQYo2Zn/DhgetfVwCvAvso5jDFz\nydrAp4BTBt0tjpcH/gjsS1/fYwUEC1qOdytkY//wnzGmemRJkDdAsl70fr4T5dx+2ZSwv7IhyHNW\nMtuYskhRAjufstdp2B3mBv4KjAX5kHYQY0zhPg1cD9F/tIOYvEX3ALcTXslsY3pQihLYVva6Iznf\nrZBzQeqrWYXMaQcIjNMOEBinHSA7cg3I7kWcqIBzlElJ+yurJiWzP6CdpAGnHSAwTjtAYJx2gHlS\nlsDOr+x1GnaHuQmr+mdM5chiwGbANdpJTFGiGcAk4BjtJMZU2GfxlYV+33SPON4eOBTYib4++wSw\nDXnfYR6e3HVYMd/zGGPKQ/bwc+iKOVlB5ymLEvdXlkzGeyuZbUzhZESynO+WTXeJ4zWSFTEmFBis\nnt1hbix6B7gS2Ek7iTGmMDvjP10ylRK9ApyBL5ltjCnWQUB/Cez5+TWWrwD6y16bNhVwt0J2AJmc\n/3kK4bQDBMZpBwiM0w7QPRmZrI6xRFEnLOg8ZVHy/payZLbTDhAYpx0gME47QMsS2MWVvU7D7jAP\n4iZgfEkfBjHGZOuTwH0QvaodxGiI/g0cC5xqlV6NKcwRwDUQPdTk/dPwF6pFLyHXUwq6WyG/A/ly\nMecyxuiR34AcUuQJCzxXGQTQXxnm1+C3ktnG5K9FCexiy16nEcAY1lhRF8yfA7m2mHMZY3TICJDX\nQJYr8qQFnqsMAulvECWzjekBcj7IyQ3fKr7sdRqBjGHzK+qCeRGQN0FGF3O+3DjtAIFx2gEC47QD\ndEe2Brmr6JMWfD5tgfRXIv/sinxJOwnB/39VOKcdIDBO79SyNsjLDa+t4nh54vg54ng7hWCDsTnM\ng4v+AUwGtlcOYozJj62OYRJR/3zJ461ktjG5ORk4EaLXB7zqC5JcBpxFX5+th5+RAu9WyL4glxZ3\nPmNMcWRo8pR20WvwBnLHNTOB9Vd+D/Jt7RTG9B5xIE+CLDDg5TiOiONJxPHFSQnssglsDJunyAvm\nxZPlpkYWd05jTDFkc5ApGidWOKemwPpb6pLZxgRqkBLYcXykYtnrNGxKRmvRbOAeYGvtJF1w2gEC\n47QDBMZpB+jCzoB9gmTqRDOAS4DjFUM4xXOHyGkHCIxTOOfn8deQA0tgx/EOwGHAZ3qt7HXFLpgB\nP79xF+0QxpgsSYTNXzbNHQfsBLKxdhBjwidjgR8Ch0D0/tyX/UoYFwC70Nc3SylcTyv44z1ZBmSO\nX37KGNMbZH2QJ5QKVQQ2RaFrgfZXdgd5CGS4dhJjwia/APnxgJfieAxxPI043kcpVDsCHcNUgssd\nINsUf15jTD7kxKbrgBZwcqXzagm0vxL5tfjlKO0kxoRLNk9Kzy8696Vylb1Ow+Ywt+FP+I9vQ+S0\nAwTGaQcIjNMO0D6J8NOsbDqGGUQkwMHAN/2DgIVyBZ8vdE47QGBcMaeRBYCfA4dB9GbNG5Uoe13V\nC+Y/Azv6ZaiMMYFbAxgJ3KcdxJRdNBM4BThPafqOMSE7CngCfw3lxfF++PoWu9PX965SrspQ+nhP\n/gaylc65jTHZkeNBztQMoHhuDYH3V4aDPACyl3YSY8IhqyfLM64w96Vylr1OI9gxTOuC+RsgF+qc\n2xiTDYlAZviH/vRCKJ5bQw/0VzYEedGvzW+MGdzcMvNfm/tSectepxHsGKZ1wbx0slpGaCVTnXaA\nwDjtAIFx2gHaI5uCPKb88Xqwg2+HeqS/cjbIBQWdzBV0nl7htAMExuXbvHwxKVLip7HG8ULE8X3E\n8ZH5njc39tBfe6IXgbuBnbSTGGM69gXgouSBLmPa8R3gE768rzGmMfkAcBLwZYjeS0pdnw88Dpyq\nGq0DAmO0M3RD8R86+TzIdXrnN8Z0ThYEmQ2yonYQ5fMXrYf6KzuBPO5/lowx85OLQU6bu1n+stcN\nib+rso/AiwQ8hmleMI9MpmUso5fBGNMZ2QUk1k5BwINvh3qsv/In/+CoMWYg+QTIzLlTV+N4h2Te\n8vLKwdoisJbArQL3CWxAwGOYcnD5Fcj/6mZoi9MOEBinHSAwTjtAenKZn1unLtjBt0M91l9ZDuQV\nkDyf9Hc5tt2LnHaAwLjsm5SRyQPV2wK+7LVfEWNC9ufKh8AogVMFXhE4RGDovLfal2YO8zbAVGAa\n0GiC9+rAXcBbwOF17z0FPAj8Hbink4AFuAjYWzuEMaYdsjjQB/xRO0lgWo3n/TYA3iXcAk9tiJ4D\njgd+BlLh53qMGeAY4F6IriWOxwBXAN+ir+9u5VwtJdMvPgM8CiwDfDiCcyN4L8/zDgWmA+OA4cAD\n+CIBtZYE1gdOYP4L5pnA2Bbn0L7DPATkWZC1dXMYY9KTg0F+p50iEcod1zTjef9+NwNX4Sso1gul\nv22QoSB3g3xJO4kx+mQdkJdBlg6t7LXAKgJXCTwm/qZKk93a1+q36Q3xA+xTwDvA74Ad6/Z5BV9h\n650mbZRIwPbBAAAgAElEQVS8mlL0PnAJdpfZmJDsjf90yKSXZjwHOBR/5/6VwpKpi94DDgBO9EuO\nGlNVMhRf/vrbyWpiQZS9FlhA/Mo39wJ3AOtGkOkzLq0umJcDnq3ZnpW8lpYAN+EvqL/cXrRCXQTs\nGUipbKcdIDBOO0BgnHaA1mQ1YBXgBu0kgUkzni+Hv4j+abLdg3eTm4keBH4F/CiHxl0ObfYypx0g\nMC7Dtg7C/0J9fihlrwW2wk//3RBYL4KTIvhv1ucZ1jpHVzYFXsBP27gRP3futgb7XYi/6wHwOv6j\nwsnJtkv+zHE7wn/8gGPeHJcCz9/W9viS5Sn7tv19tbcdwN/Xz/eFA34H0TtKecYDo5PtcYQjzXh+\nJnBUsm9E808IL0R1zM5t+3i4fjp88wj44akp9k+7PT6nvL26bX9fKn9fshzc9AP46WHEh24EnMJ3\nv/tNbrtt3Zzzd7QtsMzv4aLrYe2t4YDIz7N2+HG5dv9CxuwJQO06xRNp/qDIscw/hznN+yW5g2Gl\nso0pv1KUwq5XkjGspTTj+ZP4Z09mAv8AXgI+XbdPKP3tkGw9YCktYypDLgX5XtnLXgsMEzg0Wf3i\nJIF2/1/NZQwbBszAX5GPoPlDIgDHMfCCeCSwSPL9wvg5JZ9scFxJBt9gS2UbUyGlKIVdryRjWEvt\njOfgpyc0WiUjlP52QSaBBFfFzJjOyadBHueA6aOTstelnLMssJHA/QKxwJqdN5OPbfElEKfj70gA\nHJh8ASyNnxf3BjAHeAYYhZ9j+EDy9XDNsfVKNPjKtSB7aqdowWkHCIzTDhAYpx1gcPIzkKO1U9Qp\n0RjWUqvxvFaVL5iXSlYJGN9631RcRu1UhdMOEBjX3eGyCMgzDHm/jzieRBxfnJTALg2BsQLnCbwg\nsLd0t6BEsGNYiYIHUSrbaQcIjNMOEBinHaC50pTCrleiMawQFemv7A9yT0YPg7sM2qgSpx0gMK67\nw+VMkAuI46PKVva6tqS1wLkCY7JpNkwlCm6lso0pr9KUwq5XojGsEBXpr0Qgt4Acqp3EmPzI+iAv\n8Ju7P5/MW25nJbRcCaxdV9I6w6bDVLLgwZXKNqYiSlMKu17JxrDcVai/sjrIqyAraCcxJnsyDOTv\nbPXiEWUqez1ISesMTxGmkgWXLUH+rp1iEE47QGCcdoDAOO0AjcniIG+ALKqdpIGSjWG5q1h/5TiQ\nK7t80NRlFKYqnHaAwLjODpMjGPvWzdwcTyOO98k0USdpkpLWAs8IXCT+GbmcThWmkgUvfalspx0g\nME47QGCcdoDGSlUKu17JxrDcVay/sgDI30C+2kUjLqs0FeG0AwTGtX+IbMjQ917imltuI45PzzxR\nu2nSlbTO8HRhKmFwORnkFO0Uxph+chfI9topmijhGJarqvUXkP8BeQXko9pJjOmejAZ5kvPuvYI4\nvp44blXELr8kSUlrgdkCE8UveVnAacNUwuCyFsisjJ6ONsZ0RVYDeQlkuHaSJko4huWqav1NyG4g\n00o6LciYlCQCuZQvzriROH6COM5i1YnOksBWAo8LXCHFVk0NdgwraXC5H2Qr7RQNOO0AgXHaAQLj\ntAPMT44HOUs7xSBKOoblpmr9rSHn+alBbc9ndnmk6WFOO0BgXPpd5RDWmz2Vm+OXiePVc0s0WAJY\nRmCSwFMyfyXRgiKEqaTBS1sq22kHCIzTDhAYpx1gIIlAnqRcpbDrlXQMy03V+ltDFgKZAnJAmwe6\nPNL0MKcdIDAu3W7yUZb6z6vcNPkljbLXdSWtT+6gpHWGUcJU0uCyNMjrWKlsYxTJZpSvFHa9ko5h\nualaf+vI6sl85nW0kxiTnizKgu9O58+3z9Aoe51RSesM44SpxMHlGspfKtuYHiY/A5nYej9VJR7D\nclG1/jYge4NMBRmlncSY1iQiev+3nHX/tKLLXteVtN6ry5LWWQl2DCtx8FKWynbaAQLjtAMExmkH\nmKe0pbDrlXgMy0XV+tuE/Ark1yl3dnkm6UFOO0Bg3OBvy5fZf8YL/CW+r6iy1w1KWo8u4rwpBTuG\nlTh4KUtlO+0AgXHaAQLjtAPMU9pS2PVKPIblomr9bUIWBnkUJE3BB5d3mh7jtAMExjV/Sz7MZi+/\nwY2TXyqq7LXAWjUlrcv4/EmwY1jJg1upbGN0lLYUdr2Sj2GZq1p/ByFrJ/OZtedkGtOAjOJ/3pzB\ndbe8UUTZ67qS1gfnUNI6K8GOYSUPXvpS2cb0oFKXwq5X8jEsc1XrbwuyP8hD/hNJY0pkibcm8cc7\n3si77HVNSeunk5LWS+V5vgwEO4aVPHjpSmU77QCBcdoBAuO0A3ilLoVdr+RjWOaq1t8WJAK5BOQX\ng+zkikrTI5x2gMC4+V5Z4N39OOv+f3LdLWfneeKaktaPSjj/3Toaw4ZknaL3RO8DlwB7aycxpkL2\nBi7SDmFMa5EABwFbgOyhncYYkDU4cMa5rPLPKSzwfi5TSpOS1t8G7gFuB8ZHMDmPc5l5ArhbYaWy\njSlO6Uth1wtgDMtU1fqbkoxP5jN/UDuJqTIZyWeencUVt72YV9lrxZLWWQl2DAskeGlLZRvTY0pf\nCrteIGNYZqrW3zbIwf6ZF1lQO4mpqI+9fDlX3foWN05eI+umS1DSOivBjmGBBC9NqWynHSAwTjtA\nYJzu6YMohV0vkDEsM1XrbxskAvk/kB/XveE00gTMaQcIjANgvdmH8Kfb3+UPd+yaZeMCQ2tKWp+o\nWNI6K8GOYYEEL02pbKd8/tA47QCBcbqnD6IUdr1AxrDMVK2/bZLRyS99tRctTitNoJx2gMA4Pvjm\n2vz83nc4/54zs2xYYMOaktaZ37VWEuwYFlBwK5VtTL6CKIVdL6AxLBNV628HZAOQl0FW0U5iKmDJ\n/yzICQ++xi/uuSerstcCY5KS1s8L7FmSktZZCXYMCyh4KUtlG9MjgimFXS+gMSwTVetvh+TrIPeA\njNBOYnrc4VPv5KK7X+OPd3Rd9rqmpPULAj8uWUnrrAQ7hgUUvBSlsp3iuUPktAMExumdOphS2PUC\nGsMyUbX+dkgikMtBfoSNQ+1y2gGC8dUnTuT4s9/j+Ie6ni5RU9L63pKWtM5KsGNYYMHVS2U7xXOH\nyGkHCIzTO3UwpbDrBTaGda1q/e2CjPXzmU85TTtJYJx2gCDsPfOzXHbb+2xy+CndNCOwsMApAZS0\nzkqwY1hgwa1UtjHZC6oUdr3AxrCuVa2/XZIPgrwAsqN2EtND3EsbcMld7/Ldhzt+yC+ZfrFTQCWt\nsxLsGBZY8Lmlsj+sncSY3hFUKex6gY1hXatafzMg6ycPAW6uncT0gGX+vQJn3v8fzrr/hk6bEFhZ\n4MrASlpnJdgxLMDgcgJIrvXZB+GUzhsqpx0gMK74U0oE8gDI1sWfOxMBjmFdqVp/s+BAPpFcNK+j\nHSYATjtAecloDp/6Cr+5expxPCx50aU+OilpLfCqwFECVXwoNdgxLMDgskLyNP8ohZM7hXOGzGkH\nCIwr/pSyKcg0/+lNkAIcw7pStf5mwfk/ZHeQWSArq6YpP6cdoJxkIT77zONcevscrr61tuy1S3W0\nL2k9VeDyQEtaZyXYMSzQ4PInkIO0UxgTPpkE8g3tFF0IaQzbBpgKTAOObPD+nsAU4EHgDqDR3dCQ\n+ltCcijIEyBLaicxIZFhTHj1Fq669S2uvWXNto7snZLWWQl2DAs0uGwJ8nBgFcmMKRlZOlmqcUzr\nfUsrlDFsKDAdf2dpOPAA81fu2hhYLPl+G+DuBu2E0t8SkxNA7gVZRDuJCYFEjPvnJC67/S2uujX1\nw6M9WNI6K8GOYYEGlwjkUZAtCj6xK/h8oXPaAQLjij2dHAPy82LPmblQxrCNgdrCS0clX82MAWY1\neD2U/paJG7gpEcgvQG60wiYNOe0ApTLqvyfzq7/+i6tvPabJHq7+haSk9d96rKR1Vjoaw0KdM1gC\nkQDnAl/VTmJMmGQ4cBD+/yOTv+WAZ2u2ZyWvNbM/cE2uiSorEuArwD+BXwc8f9/kLXr/UI54/GCW\neutaRr53Qqvdk5LWPwUuA84AtozgsdxzmkIEfLdCFgV5DWSwf3SMMQ3JriC3aafIQChj2C7AL2q2\n9wLOabJvH/Ao/i5zvVD6GwBZCORWkLNsep+Zn+zOPjNf57pbphDHg5a9TtZU/kJS0vrcHi1pnZWO\nxrBhrXcxzUVvgvwWOAA4VjuNMYE5BPixdogKeQ5YoWZ7BRpPuVgHf2G9DTCnSVsXAk8l37+Onw89\nOdl2yZ+23XI7+g+sfiqcfiZs9xJwYrny2bbetgxns1d+yhqXvs+RF53AlCn/abb/Z/1zCfsBC+8H\nx10Ij+P/vyxRf1S3xzPvF4hxBCzwuxWyZlLFqah5aK6g8/QKpx0gMK6Y08jaIM/3yPzNUMawYcAM\n/D8YI2j80N+K+AcDJwzSTij9LRM3+NuyLMhMkC8Vkqb8nHYAXbI+K/9jNjdMnkMcN/1/sb+k9Y3w\nekVKWmcl2DEs2ODzyM1+fc1CuILO0yucdoDAuGJOIz8BOb6Yc+UupDFsW/zdp+nAxOS1A5MvgF8C\ns4G/J1/3NGgjpP6WhWu9i5XQruG0A+iRD7LIf1/k6lufJ473abjHwJLWF28EOxedMnDBjmHBBp9H\ndga5XTuFMWGQxZKl5JbVTpKRHhjD2lK1/hbISmhXmyzL0PdmMumuR4jjMxru4UtaX1XRktZZCXYM\nCzb4PDIM5FmQdbWTGFN+cijIH7RTZKgHxrC2VK2/BZOPg7yEldCuGBkN8iCnPXA7cXx9Tdlr/66V\ntM5SsGNYsMEHku8UtJ6sK+AcvcRpBwiMy7d5iUCmgnws3/MUqkfGsNSq1t8suPZ2l93wJbTH5REm\nAE47QLGS1VL2n3EDN8dPEMcDVqdJUdLaFRKzdwQ7hgUbfCBZqqCKZS7n9nuN0w4QGJdv8/JxkId6\nbAmtHhnDUqtaf7Pg2j9kbgntZTJPU35OO0BxZAGQy9jklRu4OX6ZOJ77IG4bJa1d/jl7SrBjWLDB\n5yeXgHxDO4Ux5SV/Bjmw9X5B6aExLJWq9VeRTAR5CuTD2klMHmRxkFtY4V/XcnP8HHG8HWAlrfMX\n7BgWbPD5ycYg061qkzGNyIogs0FGaSfJWA+NYalUrb/KZI/kQcBttJOYLMlqIE+w8DtncnN8H3F8\nJNBf0vp+K2mdq2DHsGCDz08ikPtzHthcjm33IqcdIDAuv6blRJCz8mtfTQ+NYalUrb9ZcN0dLpuC\nvAjylUzSlJ/TDpAv2RzkRaL3DySOJxHHl7wxcuRYgfOSSn17CbQzbc3llbRHdTSG2Z3QTEUCnIuv\nYGaMmUsWBPYHfqKdxJjwRHcAmwFfBzkdxApUBEv2Ai4F9uHmW0YjstqLO+8cL/rvfz8KvAesEcHF\nkf1iahrosR8KGQnyKsjK2kmMKQ/ZG+QG7RQ56bExrKWq9bdEZCxIDHIZiM1rDYpEIMclc9LXJo53\nGHbjjS9NW3bZuwXuE1hfO2GFBDuGBRu8OfkhyKnaKYwpD7kbZLCnvEPWg2PYoKrW35KRESAXgvyN\n3in+0+NkAZCLQe4BWfqjP/vZeqOuvvpft6299hwraa0i2DEs2ODNyaogr/i1FTPncmizlzntAIFx\n2Tcp6yd3VXr1H4UeHMMGVbX+ZsFl25xEIN8GeYbeLJjltANkR5YAuQ3kj0vxwsgHVl11r3GTJr1z\nwp573iGwVEYncRm1UxW5jWHbAFOBacCRDd5fHbgLeAs4vM1joWcHX7kaZL8cGnY5tNnLnHaAwLjs\nm5RfgTT7/78X9OgY1lTV+psFl0+zsntyc2a7fNpX47QDZEM+CDIN5JQ3GbXq28OGXb35WWf9c60L\nLsi60qnLuL1el8sYNhSYjq8sMxx4gPmXOVkSP/fmBAZeMKc5Fnp28JXtko/MeqlAgzFtksXxBX2W\n0E6Sox4dw5qqWn9LTjYBeQHEHjYvFdkC5KXRvPYVge8IzP7EaafdPuQvf7mhvuy1KVwuY9jGwHU1\n20clX40cy8AL5rTH9ujgK0NAZoBM0E5ijB45AuTX2ily1qNjWFNV628AZBWQx/yyjT079Skg8gWQ\nl8/i0G8JPC5wxVoXXPBN4vnLXhsVuSwrtxzwbM32rOS1NLo5tgdE7+OX0Mr6t36XcXu9zmkHCIzL\nrikZCnwF+HF2bRoTJJdv89GTwCbA2sBlPVAcyGkH6IxEIN9fnmdPeJkl7zmMcw4BvhXF8UmPrLzy\nEcCO9PXNyeHELoc2TZ1WHwt0cyehnWMvBJ5Kvn8dP31jcrLtkj9D3P4V3PQ0rL8T3HdZRu2PzzFv\nL27b31d721n+fW0HV70Fn6pd/irrvBrb44HRyfY4jCmFaA7ItsBPgVtBPgXRc9qpqkMWHMHbFx7B\nqRsdx3Ejh/L+Q8Bukb+j/FdgX/r6HlMOaXI0gYHTKibS/OG9+ikZaY/t8Y/35JcgR2unMKZ4cp3/\naLLn9fgYNp+q9TcwEoFMBHnWpgQWRZbfkpumTGPVOe8y5FaBNQGI44WI55W9NqWRyxg2DJiBv4sy\nguYP7gEcx8AL5rTH9vjgKx9Jlv6xSf6mQmQ1kJeTCn+9rsfHsPlUrb+Bkp2ThwF/CmLzZnMhw1dh\n+nd+yRf/8yaj/vEOQ78wt6R1HEf9Za+JY3v4v1xyG8O2BR7Hr3gxMXntwOQLYGn8XOU3gDnAM8Co\nQY6tV4HBV+4A+UxGjbmM2qkKpx0gMC6bZuRHICdl01bpVWAMG6Bq/c2C0zmtjAH5SXLh/IWAVm1y\n2gFaGcZ/Nz2YHz87mzFvP8/SFwsM/KUkjo8kju8ljvOox1DPFXCOXhLsGBZs8PRkD5CbMmrMZdRO\nVTjtAIFx3TchC4PMBlmp+7aCUIExbICq9TcLTvf0sgHIfSC3gKylmyUVpx2gOVniY0z+051MePtl\nlpjxDkM3mG+XON6BOH6OOF6+oFCuoPP0imDHsGCDpycjQF4EaTadxZgeIgeAXK6dokAVGMMGqFp/\ne4QM9Ws1yysgp/hfbE16MmR5nvnqj/jav95k1L9fZezhDUtax/EaxPHLxLHNHy+vYMewYIO3R74P\nco52CmPyJRHIFJBPaCcpUEXGsLmq1t8eI0uDXAzyNMhOAU3TUDOct8fvw68ef4Gl3n6epa8SPxV1\nfnE8hjieRhzvU3BE055gx7Bgg7dHlgd5DWSRLhtyWaSpEKcdIDCuu8Nlc5DHfeGeyqjIGDZX1fqb\nBacdYH7SlxQ7uRJkZe00dZx2AE8W3ZC7L7iWrd+ezZgX3mb4lk13jeNhxPENxPEZBQbs5xTOGbJc\nCpeYzESzgJuBvbWTGJOjQ4Bzk8I9xpjSimJgXeBO4F6//KksoByqJCRanFf2+B7HPHsTH9/jo9x/\n6ljmrLQA79w8yEGn4S/EjigopKmgCt2tkD6QR+wjMNObZBmQOSCLaScpWIXGMKB6/a0AWTm50zwV\npPld1EqQ1Xbh/+59knFvP8/St0ma4kRxvG8yFcOW7wtDsGNYsMHbJ1FywdynncSY7Mmxfs3XyqnQ\nGAZUr78VIjsmc5svAVlWO02xZOSHmXL6H9j1rTks9to/GZluKdg43jh5yM8e6g9HsGNYsME7IweB\nXNVFAy6rJBXhtAMExnV2mIxM1npdO9M0YajYGFa5/mbBaQdITxYGORnkdb/ajeyiMFXDFXMaGQKy\n+TD++8tvcPq/XmfRt15gqXME0q0gEsfLJ8vHbZ9z0Fac8vlDE+wYFmzwzsiCILNA1uuwAZdlmgpw\n2gEC4zo7TL4O8qdMk4SjYmNY5fqbBacdoH2yCMi+IDcn66qfB7JJQVMKXb7Ny2og3wOZ+TEmz3iO\nZZ77NwveMbekdRrlKnvttAMEJtgxLNjgnZPDKrZOrelpshDI8yAf0U6ipGpjWNX6a5CVkocCp4JM\nB/kuyCraqdojY5NPeO8EeWl5nvnJ06zwf+/D8wJ7zi1pnYaVvQ5dsGNYsME7V/kLDNNTKv8LYNXG\nsKr118wlEb5q4Dn4Aii3gnypvA/6yohkXvYfQd4A+f1Q3tn+3yz4RYEXBc4VGN12s8WWvTbZC3YM\nCzZ4d+TrIH/u4ECXdZIe57QDBMa1t7ssCPJcF1OMekFIY9g2wFRgGtDso+Szk/enAI1+qQ+pv2Xh\ntANkb+7F6KX9F6Mg24MMz6Bx10Wu+ov620C+DDJaYC2BWwXuE1i/o+aLL3udhtMOEJhgx7Bgg3dH\nFkoeklq3zQNdHml6mNMOEBjX3u7yVb8cVaWFMoYNBabjl8kaDjwA1D/Zvx1wTfL9RsDdDdoJpb9l\n4rQD5EsWB/kKyF0gL/nVcuRr+EqC40HGtDn32aU870iQNUC2SaZbnIIvxjJg2ojAKIFTBV4ROLhh\nSes0ylv22mkHCExHY1gZ5t4I5cihQP4X2BSiXbSTGNM+WQCYAewE0X3aaRSFMoZtDByLv8sMcFTy\n58k1+5wHxMDvk+2pwBbASzX7hNJfo0I+CHwKWBlYCf8L2krJm08BTydf9d+/AlHdhYwsVtdG/feL\nAs/UtPM0vkDYnRBJMi95J+BM4Fbgm9HAn+X0/BrL9wAn0Nf3647aMGXR0Rg2LIcgJr3zgG+BrAPR\ng9phjGnT/sCUil8sh2Q54Nma7Vn4u8it9lmeTi8yTAVFTwCnD3xNIvxc4XHMu9hdCdik5vuRIM8A\nzwGLJ/sOY+CF9dPAvTXfv9SsqqjAKvjpRasA+0QwueMuxfEw/C+RV9rFcnXZBbOq6N8gpwPHAJ9N\neZCjm//xq8dhf1/tcKT6+5IF8Hcod801jclS2o8h6++8zH/cusjcR6UWBJbG308EmJn8advztl/E\n398vS57Ct6O0+3+ImXyo7u9rLWCttOcbNh12fxhefRx+uAn8cBl4byhxV/lvPBPW2xQeOuoT7MM3\n2u9/ztuV//lqsf0i8Fay/Tr+6YxAVXw+nCwM8iLpCz64PNP0IKcdIDAu3W5yIMi1uSYJRyhj2ATg\nuprticz/4N95wO4121OBper2CaW/ZeK0AwTGdXKQwJYCUwUulzQlrdPwZa+fKHnZa6cdIDDBjmHB\nBs+OHOGfMDYmBDIC5CmQsj34oiWUMWwYfs75OGAErR/6m4A99GcCILCMwCSBpwQ+nVnDVva6VwU7\nhgUbPDsyKnmyOH2VIWPUyJdBrtdOUSIhjWHbAo/jV8uYmLx2YPLV78fJ+1OAjzZoI6T+mh4mMFTg\n0GT1ixMlbUnrNMpT9tpkL9gxLNjg2ZKjQH6bYkeXd5Ie47QDBMYN/rYMB5kJskkhacJQtTGsav3N\ngtMOEBjXageBDQXuF4hl/k9KulOustdpOO0AgeloDBuSdQrTsXOBrfyaksaU1heA6RDdqR3EGFM9\nAmPEz7W/HDgD2DKCxzI7gS91fT7+k5hTM2vXmAzY3Yq55GiQS7RTGNOYDAd5EmQz7SQlU7UxrGr9\nNSUgfpHmfaSbktZpWNnrKgh2DAs2ePZk0aSU54e0kxgzP9kP5GbtFCVUtTGsav01ysSXtL5Fuilp\nnUY5y16b7AU7hgUbPB9yDMhvBtnBFZWkRzjtAIFxjV+WYUm52S0KTROGqo1hVetvFpx2gMA4AIGF\nBU6Rbktap1HestdpOO0AgbE5zD3ibGBbkNW0gxhTY09gFkS3aAcxxvS2IYD4ktaPAssCa0fwkwje\ny+WEfo3lK4Aj6OtrtJSiMaVgdyvmI8eCXKidwhhPhoFMA+nTTlJSVRvDqtZfUyCBlQWuEnhUirhz\nGsfDiOMbiOMzcj+XKQu7w9xDzgJ2AFlVO4gxwOeBF7AS48aYnAgsIHA0cC9wBzA+KmbM6V8J44gC\nzmVMV+xuRUNyPMgFDd5wRScJnNMOEBg3cFOGgjwOspVKmjBUbQyrWn+z4LQDlJnAVnUlrV0hJw6j\n7HUaTjtAYDoaw4ZlncJk5kxgOsgqED2pHcZU1m7AK4CtjmGMyZTAMsDpwCbAYZGfRwz+ojlfcbwx\n/u7yFvT1zcn9fMZkwO5WNCXfB/mldgpTVTIU5DGQT2gnKbmqjWFV66/JWF1J65MyLWmdRvOy16/5\nePbVI1+v0ViwY1iwwfMnY0Fmg4zTTmKqSHYHuRMk0k5SclUbw6rWX5MhGVjSes3CAwxe9tp+tntL\ns/+ewf53DjZ4MeQHID+vecFpJQmU0w4QGOf/kCEgj4BsrZomDFUbw6rW3yw47QDaJClpLfCCwF4C\ng/0i7nIJEccRcTyJOL4kKYHdIGaQnHaAksr0gtlWySi/M4BdQFbSDmIqZVfgH8AN2kGMMeGSpKQ1\n8Bh+HeU1Irg40rk4PQJYDfgSfX2hXhybCrMf2pbkJJDztFOYqpAhIA+DbKudJBBVG8Oq1l/TIfEl\nrW+VvEtap5Gu7LX9bPcWm5JRPbIEyGsgK2onMVUgu4LcY3OXU6vaGFa1/po2CYwSOFWKKGmdRvqy\n1/az3Vvsgrma5BSQn2BzldrltAOEZXgfyIMg9U+Pm+aqNoZVrb9ZcNoBipBMv/iMwNMCFwks1WFT\nLrNQcTyGOJ5GHO+TYu9Qf7addoCSyvSC2dZhDsfpwFTYIoZbtLOYnnXkZsB/gWu0kxhjwiGwCnA2\nsCqwbwSxciRf9hp+D1xJX9+vteMY061Qf6NTID8EOVc7helVMgTkAZBPaycJTNXGsKr11wxCfEnr\n7wjMFpgoMEI701xx/CPi+PrkwjmNUH+2nwKsGuv87A5zhZ0KPOKnZkSPaIcxPWc/4D/AldpBjDHl\nJ/4i7SfA48B6kb9wK4c43g/YHtiIvr53tePkrL9QRwh2wq+//T7wHHBRm/ulPb4nhfIfuSTOOAvk\nFnsgKzWnHSAMsjjIS7D3l7STBKhqY1jV+psFpx0gSwLLCEwSeEogj0+kXFdHx/HGyUN+a7R5ZKg/\n2y8AW2qHSGEx4G8123cBS6Tcb/E2ju9n6zBX29GXA4sAe2onMT3lROAPcNF07SDGmHKSpKQ18CDw\nNIYisVIAAB7jSURBVLBWBFcoxxrILxv3R2Bf+voe046TsSOBWcCbwFSgr8E+awCTgTnAw8Cnat57\nCjgKeARfNvoCYIGa95cFLgVeBp7E/7fO0seAR2u2p9C4D43227KN43tWqL/RKZKNQF4AGa2dxPQC\n2Qjkeft56ljVxrCq9ddQgpLWacwre31Ehy2U+Wf7Q8AzwNLJ9or4By0BZuIvKIcD0/EXxcPwF5Nv\n4ou1gL9gfhBYDhgD3A58P3lvCP7u7XeSY1cGZgCfbJLnKvxFeaOvZr9EfQU4p2b7ZODoNvY7KOXx\n/WwOs4n+CnIl/gc9698ATaXIUPwcxCMgel07jTGmXMRfWJ0E7Ah8C7hEqUrf4Hyp6/Pxd15Py+Uc\nx2XU7+MGLQvezHv4u8FrAbPxF8/1JgAL4y8kwa9UchWwB3A8/r/bj/FzfwF+gL8APQbYAD+94YTk\nvZnAL4HdaVzxdYcO+jAaeKtm+7/AqDb2k5TH58IumMPj8B+3TAQeBfkVRPerJio3h//7Mo0dhC+B\nfUmy7bC/L2Oy5gjs/yuBCPgCcAr+Y/o1Iijql2pH+39fR+LvpH4st7LXnV3oZmU68HXgOPxF8/XA\n/+LnLy+Y7LMs8GzdcU8nr/erff+ZmvdWSr6fU/P+UODW7qPP9Q/8XOR+CwEvtbHfWymPz4XNYQ5W\nNBv4NvATvxyYMe2SpfCD7yG+3oAxxviS1vgF/w8FdojgkAIvltsXxzvgs+5EX99/tOPk6LfA5viL\nW8H/MlPrOWAFGHBhvxLz7iiDn8pR+/3zyffP4u8qj6n5WpTmd5KvxV/YNvq6uskxMxj4kN4Sddma\n7bd4sl/a43uW/UPdMRkCcheIrWxgOiC/Bsnno8tqqdoYVrX+VkZdSetD1EtapxHHa6Yse51GmX+2\nP4ifp7wAfq3rC4BfJe/VzmGegb/bPhx/p/7N5Fjwc5in4Ocwj8XPYe6fgjEUP4f5CPyd26HA2sD6\nGfZhYeChmu0pwAeS71dl3oV+s/0GO74RK41tasl4vxyYDLa0ijF15GMgz4IUNv+rh1VtDKtaf3te\nhiWtixXHY9soe51GmX+2Pwz8FX8BPBv/YF3/A4D9F8zgH8icjP9E4GH83HNq9jsSv0rGHPwF94I1\n7y8DTMJP83gNuJPsl6vbG/9g4XcZuNrX/cBHUuzX7PVG7IK54tz8L8nZIL8oPEkYnHaA8pHhIA+D\nfLbBm67oND2gamNY1fqbBacdoBmBVQSuEnhUypPTtdwjjocRxzcSx2dkeN5Qf7Zdyv1qL6yroPB1\nmLfBP3U6Df+bSSNnJ+9PYeBvCE/hlzD5O3BPJwFNKscA24Nk8ZGU6X2H4ed9/VE7iCnMWOBG4An8\nE++NlhBcAf9U/SP4O1OHFZbOFK6/pDX+3+bbgfFRWA8mnoav9tbpEnLGZGoo/snMcfj5MA/gF8Wu\ntR1wTfL9RsDdNe/NxA/Ugwn1N7qSkb1A7k+WCTOmCVkO5FWQ1Vrva1IKYQw7lXkXFkcyb9mpWksD\n45PvR+HLHTeqlBZCf80gBLYSmCpwufh/38MSx/sRx08Qx2MybrnXf7btDvPgr3dlY+C6mu2jkq9a\n5wG71WxPZd78p5kMXAKkkV7/AS2IRCCTQb6qncSUmfwe5Put9zNtCGEMqx2Xl062W7kM2KrB6yH0\n1zRQQEnr/HVe9joN+9nuLYVeMO8K1M6N3YuBVVYArgQ2qdm+Cfho8v2T+OkY9wFfbnIO+wFtj2v+\nlqwF8kqyXJjxnHaA8pCPg8wEGTnITq6oND0khDGsdm3VqG67kXH49VsbPRQaQn/LxmmevL+kdbL6\nxYniVxsoM9fw1Thenjh+jjjeLqfzhvqz7bQDlFShlf7SNtpsMe/N8Gv8LYmfPzcVuK3Bfhfi5zuD\nf7LzAebNpXLJn7btjW/+fvQInH8TjPwNsHVJ8mpvD/L3VaVtuQs4Fyb+HE7ecJD97e+r9fZ45s0B\nHkd53Mi8p+ZrfbtuWxh8bB+Fn9/+NeCfTfa5EBuz29ker3X+L8JBV8P/bu+fW/hY5D9p2EArT8rt\n+f++4vivwGX8+c9XcvbZ/2aeMuTV3lb7+Qpku5AxewIDp2RMZP4H/87Dl07sV/vRX61jgcMbvB7q\nb3QlJaNAngHZQjuJKRM5GuQK7RQ9KoQxbCrzLqaXofmUjOH4CmJfH6StEPpbeQJjBM4TeF5gT2l+\nY6v84jgijicRxxcnJbDzYj/bvaXQKRnD8Itgj8MvlN3qob8JzHvobySwSPL9wsAdwCcbnMN+QDMn\nuybLhg3XTmLKQMYlD/qtrJ2kR4Uwhp3KvJsdR9H4ob8I+A3woxZthdDfykrWVN5H4AWBc6Xxiihh\nieOjiON7ieOFcj6T/Wz3lsLXYd4W/7T0dPwdZoADk69+P07en8K8+cur4C+wH8AvUTSRxuwHtD2u\n9S4SgVwP8s3c05Sf0w6gTy4Hqf9YvhmXZ5IeFcIYNhb/fEn9snLLMq+M7Wb4ZboewD978nf8sqL1\nQuhv2bgiTiKwlsCtAvdKthXaiubmfhfHOyTzlpcr4Lyh/mw77QAlZYVLKs6l201WS+4qLp9rmvJz\n2gF0yQ4gj4MskPIAl2eaHlW1Maxq/c2Cy7PxmpLWLwt8JYiS1oNzQNZlr9MI9WfbaQcoKbtgNmnJ\n90D+oJ3CaJGFQJ4E+YR2kh5XtTGsav0trbqS1r8JpqR1GnE8JuOy12nYz3ZvsQtmk9bcC6ZGc8dN\nz7NfmApStTGsav0tJYGVS1jSOhu+7PUNGZe9TsN+tnuLXTBXnGtvd9kB5Ik2PpLvNU47gI6Op+S4\nPNL0uKqNYVXrbxZcVg0lJa2/LfCqwFHiH8jvLUce+X/E8fXEcaulb7MW6s+20w5QUpleMA/pIogJ\nQnQV8BhgDwBWhkT4AkMnQzRLO40xJhviKy9OATYE1o/g5Aj+qxwrW3G8L2PHTgB2p6/vXe04gfgd\njatymh4T6m90AZFxILNtWbGqsGUFC1a1Maxq/VXXEyWt08i37HUaof5szwS21A6R0k7A0fjlLfce\nZL898LU7fs/AWh//3969h89RlQcc/w6RQhIMtyAQEvwJGCBciiIJVzNLkBJNBKuCAoraRksR4bGV\niyCllcrtUcRSpY8SigqoReQqCNjhQQWBSAgQCEJIuEQIl4CgIhR8+8d71p3fupfZ387umZnzfp5n\nn8zszsy+Mzlzfmdmz5x3OfAKsBr4aJfvsi4ZZizqiSukvIPXmwz+nLjmnb4jCUhodVho++tNU0rr\n00uQ0nrsBp/2Oouylu2yNJjXB36Vmr8NmNxiuW2Ao930ZOB5Ghn6FgBb0j1TNViDOXjx2FaTdUDu\nAfmHXKMpvth3AMMjEcjFIN/qYyNxXtEEJLQ6LLT9zUPc6woCMwV+JZAIzBhATMWRJONJkkUkyXHu\nndhTJEUv28cDTwAvotk6a+79J2k0mLdH00I/j+bAmJ9afyV6Z3cpsAZYCKSfb5oC/BB4GniERqM1\nL/OB76Tmzwc+2GK5A4HHU/N3Ah9w072MmpJrg3nYHeqNN9Er+lM9vwBZDNHtviMyufs0+od1T9+B\nGGPGRmBD4HS00fA54OKo+A25sdNU1xegDcCzPUdTZNsCR6EJaZ6i9V3WtYGrgW8B+wH7AFcCuwIP\nuWUORbMu/8EtezLwBfSZtquBHwGHANPQZEcPosmOml0D7NUm1p/RuuvQVOCF1PwLwFtbLPdjNGke\naAbSzdHkeAC7oY38SWgipqvaxFBJ1a0ICkkOdD/Zb+I7EpMn2RNkNchWviMJUGh1WGj7OxSplNZP\nVSaldRbDS3udRceyLSB5vMYY2zZov905aMM4rd4lYx/0bnPaJcC/pJb7ZOqzuTQaorOAR5vWPRG9\nC52XExl9UfRvwJe6rDMPuCI1/77U9N10Pk/sDrPpR3QlyO7ApSB/A9HrviMy/ZJNgR8An4DoEd/R\nGGN6I7AD8A1gAjAvgkWeQxqOJJmH/uw/k1rtZd/hdBPp3U5fHgaOBU5Fy8tPgM8yuoE8hdFdGUAb\nwVNS8+nPH0t99mY3/Xzq83HALX3GnfYSsHFqfjx6EdDOBsDHgMNT712Zmn4e7b6TblAPjA0rVz5x\nDtv4gvv3izlsq+hi3wEMlrwBfYr4AoiuzWGDcQ7bMMaMFrd6U2CiwJlon9PvAbMCaizPQO9evp9a\nbVXTp/HwAyqFS9G7yG9G75Ke6d5f1/27Cu1KkW7Yv9m9X7dl0/Rv3PTj6B3oDVOvSegd3lauQxvA\nrV7t/hYtZ/RDfpObYkuL0P7Wfw/8zu3H4eh5UjcRCGroQft5rzdxPpuRTVzXjAPz2V5hxb4DGCw5\nG+R6kHE5bTDOaTshCa0OC21/8xCnZ1z3i4NcSuvvVCqldRZJslGXtNfxMMNJKXLZno52u1gHTVaz\nELjQfVZ/6G9ttFF6vJuO0QcEp7vlVqLjeG8BbAT8HDjNfTYOHcHiOPTO7zhgR7TPdF4mAvem5pcA\nb3LTWzO6of8ZtO/1Zui447OBvWk83DgBbeBP6PB9NkqGyYvMAnlas8KZ8pH3g6wA2bj7smaAQqvD\nQtvfXFU6pXUWmvb6RpLky75DaaHIZXsn4Ha0Afwc+rDbZu6z9LByM9BfLF5AR8lI3xRbgTaml6Ld\nGS6kcXca9OG6S9AG+BrgVvIfru4j6IOGpwCHpd6/C3ibm94beB34k3u9jjbycescC5yD9rvuxBrM\nJk9yJMi9INUd37OSZDuQZ0B29R2JCa4OC21/c9GU0vrESqa0ziJJzvGU9jqLqpftsozXnBdrMAcu\nzndzEoFc5MbvrWJSk9h3APmT9UDuB/m7AWw8HsA2qy60Oiy0/e3bsfBZgWUCV0kjAUN4kuTjJMmv\nSZINuywZDyOcFspatuOMy1mDufP7hVfawD2J89+kTABZAvLp/LftXew7gHxJBPL9PpOTdBIPaLtV\nFlodFtr+jpm4lNbX61Bx1U1pnUWS7OnSXm+XYel40OG0UdayHWdczhrMnd8vvNIGXi2ytevPbEkv\nCk2OBVkEsm73Zc2QhFaHhba/PQsqpXUWSTKtAGmvs7CyXS3WYDaDIvNAHnfj+prCkX1ccpIR35GY\nUUKrw0Lb3564lNZ3BZHSOoskmdCU9rrIrGxXizWYAxcPdvPyRZDEje9bBbHvAPIhm4OsAjlgwF8U\nD3j7VRRaHRba/mYisKHA+QJPChwuo4fIin3F5VWSRCTJpSTJd10K7KziQYXURVnLduw7gILKtcFs\niUtMs1OBV+mertIMjayNZvL7L4iu9x2NMabBjal8BPAAOvzV9hF8Nypv4ytPx6MpnRdQq9nxMKZP\ndhIVjkwGWQnyt74jMQDyFZBrQewCt5hCq8NC29+2BHYQuEVgkeSb4KH8kmQ+SfIESbJF94ULw8p2\ntViXDDMMspt7CHBb35GETQ4GWQ7SbRgm409odVho+/sXBNYTOMs91PePolnRTF2SzHAjYuzuO5Qe\nBV+2K8YazIGLh/dVsgBkqY77W1qx7wDGTma4i5a3dV82N/EQv6sqQqvDQtvfP3PdL94n8JhLab1Z\n97WAkM6r7mmvs4jzCqdHZS3bse8ACirXBnNVHuwyg/EtYA/9Vz6sfyvMcMgk4HLgOIgW+47GmNAJ\nbAV8DdgaOCKCxHNIxaPZ+74PXE2tdpHvcIypGmuEFZqMB7kL5BjfkYRDIpDLQM73HYnJJLQ6LKj9\ndSmtTxZ4LuiU1lkUO+11FkGV7QBYlwwzbPIWN/7vfN+RVJ9EIKeB3AGyju9oTCah1WHB7K/AHIEH\ng09pnUX2tNdFFkzZDoQ1mAMX+/lamQnyFEg//dJ8iH0HkJ2MA/lPd0ffV/KY2NP3lllodVjl97ee\n0lpgZU4preMctlFcvaW9ziLOaTu9KmvZfgqY4zuIArI+zMaH6A6QGnC9DjsXfdl3RNUifwV8G9gU\nqEH0W88BGRMc0b+JRwKnoM9wLIjg936jKrgkmQZcBnyMWm2Z73ACJZSnsX8QmgHzT8Aq4DttllsO\nTAVeAD6H/n3sZf1KKst/sgFApoHcD3KGdh8w/ZP1QG4AuRxkXd/RmJ6VoQ7bCLgR+DVwA7BBh2XH\nAYuBq9t8Xob97ZnALEtp3aNypb3OoqxlewWwr+8gMlgf+FVq/jZgcptlFwBbMvrGbi/rg2X6M35F\njwP7ADV09Az7laIvMhn4X+Ax4GCI/ug5IFNNJ6AN5unAT918O8cA91PexkNP6imtgSuArwD7Rrr/\nphNNdX0BsAw423M0oTgeeAJ4ET3utRbLbA/cDDwP3Aeknz1aiZ77S4E1wEIg/azMFOCHwNPAI8DR\neQYPvJPR59YSWu8DaMbhx4DXxrh+7qzBXD6x7wAgeg7tLzUV+J+C3xWNfQfQnmwJ/Ay4CVgA0Wtd\nVhiG2HcAZiDeC9SH+boI/VmzlanAu9HuCJX+BWnIKa3jAWzTt0GmvY5z3l4VbAschWaUnATsDzzq\nPqv/DV4b/WXoemATtMF7MfDW1HYOdetujV5An+zeX8utuxhtOM8BjnXLtnIN2ihv9bqqzTr1LhZ1\nLzTFlrYb8Engn2k8R9DL+rmzu4NmjKLfuVEzLkL7NR9o/W57IdsDPwHOgegc39GYytsUWO2mV7v5\nVs5B+wtOGkZQvgjsAHwDmADMj+BOzyGVS5LMRxtjM6nVXvYdztAkST4XBrXaWC5GX0fvBu8APIfe\nfW22OzAROMPNJ2jD9lDgX9GLwfPQvr8A/w78B/AFtIE6GTjNfbYCvXD+ENqNq9m8MezDBkD6V9RX\ngXaJ0X4K/MhN3w3c0uP6ubMGc/nc7DuAhuhVkMPQwfxvBjkAotVdVhq2m30H8JdkFnAlmpTk292W\nHrKbfQdgxuxGWmeeO6lpvt0DQvPQn2IX0/0O33+jP++C3uW5m0bZqa9buHmB9RbCN2+CufvpcTk/\n0i5m8RC+ny6fl2N+9uwjWLToq7zjHXOp1VYN8Pvo8vnw57Whm8f24jGufyx6UTsCXAt8Fr3zXDcF\nPR/T2xdg19QyG6Q+fwy9axsDb3Lrv+SWew19nmFpH/E2z78E7JLa3ni0wdtq+1em5sX928v66fld\naDy3MUKJBdFPrtokAjkF5GGQrXxHU2yyv0t3/R7fkZjclKEOW0ajMb25m2/2JeBx9M7Sk+joEK0u\n6Mqwv6P0kdLapOWT9rrIylK23whcQuP8rD/0tzd67qbvYF+CjvpSX+5Tqc/mAg+76T3Qh4Kzug5t\nwLZ6XdtmnbnoxXbdhcCHWyx3OPCD1Pwd6AV91vXrbBzmwMW+A2hPjgR5AmRn35GkxL4DaJAPuQQw\ne/mOpIPYdwAlVIY67Cy0zynoQz9ndFgWYDYVGSVD4C0C1wg8IEN8QKhJ7Ol785MkbyBJbiRJvjKE\nb4uH8B2tFLlsT0cbxeug2SYXog1G0Ebyvmgf5uXoub42ehxfdOuC/iq0BNgCHTnn5zS6YIxDR6A4\nDr1zOw7YEe0znZeJwL2p+SXonW3QPtX1hv7eNEb9mIA29Cd0Wb8VazAHLvYdQGdysGsU7u07Eif2\nHYCSo9zFxE6+I+ki9h1ACZWhDtsIfbi0eVi5KbS+GzSb9g/ulGF/6ymtTxJ4tgAprWOP352P4aa9\njofwHa0UuWzvBNyONoCfQ8/P+i8l9QYz6JCIN6NdM+4DDkxtYwXamF6KPpx3IY0HBkF/fbrEbW8N\ncCv5D1f3EfRBw1OAw1Lv3wW8LTV/GI0uKLMyrN+KNZhN0cm7XLcDS6Wt3VVOBXlIU4ybCgqtDiv8\n/rqU1ssspXVOqpH2OovCl+0+lWW85rxYg9mUQWlTaeeoEKmuzeCFVocVdn8HkNLaNNJeb+87lCEo\nbNnOiTWYO79feKUN3JPYdwDZyXYgj4L8k8cgYj9fK+uAfB8kASnTEF2x7wBKKLQ6rHD7KzBO4GiB\nZwROF+3rWCSx7wDGJEmmkSS/IUmG/ZByPOTvqytc2c4ozricNZg7v9+RDStnBiha5voy/wRkF+Dz\nLlNgxcks4KtoP7C5lr3PmMERmIlm6vstMNuy9OUkSSag4+CeS63WbtQDUy7WLbDkynpFZzKTSSCn\ngTwHcjrI+r4jGgzZ2t1VfgLkE9olwwQgtDqsEPtbT2kt8KTA4VLxzIRDlSQRSXIpSXKxS4EdikKU\nbZMb65Jhykqmgix0o2h8BsTnU+s5ko1BvuouCE4GKdrPwWawQqvDvO5vPaW1ayifJ40RP0xekuQE\nkmQRSTLedyhDFtq5XHXWYA5c7DuA/snOINe7kSM+oCNJDEw8uE3LuiDHgTwD8vWKPNgX+w6ghEKr\nw7ztr8AOArcI3Cn5jg87aLHvADJLknkkySqSZKrHKGJP31vWczn2HUBBWYM5cLHvAPIj7wJZDHLr\nAJN5xPlvUtYC+Yh7oPFHINt2X6c0Yt8BlFBoddjQ91dgPYEzBZ4WOFI0qUKZxL4DyCRJZrgRMfbw\nHEns6XvLei7HvgMoKGswmyqRtUA+CvIYyOUg07uv45PMccPE/RJkH9/RmEIIrQ4b2v6mUlo/KvBt\ngSr8ilNMjbTXH/MdikehnctVZw1mU0UyHuQEkGdBzgPplO7SA9kR5Mcgy9FshiE9CGM6C60OG8r+\nCmzlUlrfL3YHbbCGm/a6yNag5dte1XitobXS1tmlDdyT2HcAgyWTQc5FG84ngUzoc4Nxn/FsAXIB\nmrnwGCrzoGJbse8ASii0Omyg+yua0vpk0ZTWJ4jflNZ5iX0H0JGmvb5hSGmvs4h9B1Ayse8ASmZM\nddhaGZY5AFgGPITmIG/la+7zJYzOBZ5lXdObXXwHMFjRsxAdA+yO7uuD6IN1+4JsPIYN9ni8JALZ\nEmQ+yNnAPcCzwHSIzoXo1THEUCYVL1+myATmoOfcbsA7Ijgjgiqcc8U9r5Lk48B7gEOo1V7zHY5T\n3ONVTHa8hqDb1eQ44DxgP2AVcCdwFfBAapl3A9sAbwVmAd9AGztZ1jW9C2QIpehh4IMgewCHAfOB\nnUFeQi/M0q+HIHq9zYY6HC9ZF9gB+Oum1x/ddhcDb4fo0Rx2qCwCKV+mSAQ2B74M7Al8JtK/FVVS\nzPMqSfYEzgRmU6s97zuclGIer+Ky4zUE3RrMM4GHgZVu/nvAgYxu9L4XuMhN347+x22GZpTptq4x\nXUS3AbfptETACI2G7SHAl4DNQJYyuhF9D0S/Ta23OaMbxTsDW9H4ZWQJcI3+Gz09hB0zJniif4OO\nBE4BvgksiOD3fqMKRJJMAy4DPk6tZn+XjemiW4N5CyCdyvgJ9C5yt2W2AKZkWNf0bsR3AP5EAqxw\nrysa78skYCcajeHDgR1BnoGDxgNHuQXrDePr0LsqD0D0yrCiL4kR3wGYMMjolNbvjKp9M2XEdwCj\naEKSIqe9HvEdQMmM+A4gBN0azFk7RvczYsDyHr7HqCN8B1AsbYvfxKbP57iX6czKV2+W+w5gyJaQ\nQ53ddNbe3+/2SqA451WtVp/aFTjDYySdFOd4lYMdr+yWjGWlbg3mVcC01Pw09E5xp2WmumXWzrAu\naP9nY4wx5WAPGBljTJM3oHdPRtChfe4Gtm9a5t3Aj9307sAve1jXGGOMMcaY0psLPIg+wHeie+9T\n7lV3nvt8CfD2LusaY4wxxhhjjDHGGGOMMWPzQWAp8Dqj70Y3s6QnaiPgRuDXwA20H29xJTrg/2Lg\njqFEViz9JNgJUbfjFaOjFyx2r5OHFlkxLQRWA/d2WKaq5cvq7N5YnZ2N1dm9sTq7N5Wos7cDpgMJ\n7SvfcWg3jhH04cGQ+z+fBRznpo+n/RPNK9CKOkRZyku6r/0sGn3tQ5TleMVUL3lEP/ZBK9R2lW+V\ny5fV2b2xOrs7q7N7Y3V273Kvs7Okxs7bMvTKu5N0wpT/o5H0JETpxDAXAQd1WLaf4f3KLEt5aZVg\nZ9MhxVc0Wc+vUMtTKz8DOmVCq3L5sjq7N1Znd2d1dm+szu5d7nW2jwZzFu2SoYRoU/RnBdy/7f5D\nBbgJWAQsGEJcRZKlvLRaZuqA4yqqLMdL0DTFS9Cr8BnDCa20Qi9fVmc3WJ3dndXZvbE6O389l69u\n4zCP1Y1oeuxmnweuzrB+aIlM2h2vk5rmhfbHZi/gSWATt71l6BVWCMaaYCe0claXZb/vQsdO/wM6\n2s0V6M/ypr0yly+rs3tjdXZ/rM7ujdXZg9FT+RpUg/ldfa6fJWFKlXQ6XqvRivkpYHPg6TbLPen+\nfQZNeTqTcCrfsSbYWTXguIoqy/F6KTV9HfB1tL/lmsGGVlplL19WZ/fG6uz+WJ3dG6uz81eq8pWg\naTlbsaQnDWfReCL2BFo/QDIBeKObngj8Ath/8KEVRj8JdkKU5XhtSuPqeybady50I2R7gKSq5cvq\n7Gyszu7O6uzeWJ09NiOUvM5+H9pv5GX0Cvw69/4U4NrUcpb0RG2E9nNrHqIofby2Qk+gu4H7CPN4\n9ZNgJ0TdjtdRaFm6G7gVrVBCdinwG+BVtP76BOGUL6uze2N1djZWZ/fG6uzehFxnG2OMMcYYY4wx\nxhhjjDHGGGOMMcYYY4wxxhhjjDHGGGOMMcYYY4wxxhhjjDHGGGOMMcYYY4wxxvTk/wEug2in+p4V\nCgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x110ff08d0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# <!-- collapse=False -->\n",
    "x = np.arange(-1,1.1,0.1)\n",
    "y = x**2\n",
    "\n",
    "lvals = [0,0.25,0.5]\n",
    "lines = [x*2*l-l**2 for l in lvals]\n",
    "ystar = x**2/4\n",
    "\n",
    "fig = plt.figure(figsize=(12,5))\n",
    "ax = fig.add_subplot(1,2,0)\n",
    "ax.plot(x,y)\n",
    "for l in lvals:\n",
    "    ax.plot(x,x*2*l-l**2,label = \"slope = $%.2f$\" % (l))\n",
    "ax.grid(True)\n",
    "ax.legend(loc=4)\n",
    "ax.set_ylim(-0.5,1.0)\n",
    "ax.set_title(\"$f(x)=x^2$\")\n",
    "\n",
    "ax = fig.add_subplot(1,2,1)\n",
    "ax.grid(True)\n",
    "ax.plot(x,ystar)\n",
    "ax.set_title(\"$f^*(x)=x^2/4$\")\n",
    "\n",
    "plt.show()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A similar statement is true in higher dimensions, but it's harder to graph so I won't do that. The conjugate function has a number of interesting properties, so we'll go through them now. \n",
    "\n",
    "<ol>\n",
    "<li> $f^*$ is convex. This follows since $\\langle u,\\theta\\rangle - f(u)$ is convex in $\\theta$ for all $u$ and the maximum of convex functions is convex.</li>\n",
    "\n",
    "<li> Suppose $f$ is differentiable. If $f^*(\\theta) = \\langle u,\\theta\\rangle - f(u)$, then $\\theta=\\nabla f(u)$. To see this, just compute $f^*(\\theta)$ by differentiating the definition. </li>\n",
    "<li>\n",
    "For all $u$,\n",
    "$$\n",
    "f^*(\\theta)\\ge \\langle u,\\theta\\rangle -f(u)\n",
    "$$\n",
    "This follows directly from the definition and is called the <i>Fenchel-Young Inequality</i>. \n",
    "\n",
    "Suppose $\\theta$ is a subgradient of $f$ at $u$. Then\n",
    "$$\n",
    "f(w)\\ge f(u)+\\langle w-u,\\theta\\rangle\\\\\n",
    "\\langle u,\\theta\\rangle - f(u)\\ge \\langle w,\\theta\\rangle -f(w)\n",
    "$$\n",
    "so that\n",
    "$$\n",
    "f^*(\\theta)= \\langle u,\\theta\\rangle - f(u)\n",
    "$$\n",
    "and the Fenchel-Young inequality is an equality. Similarly, if we assume that $f$ is closed and $f^*(\\theta)\\ne \\infty$ so that there exists $u$ with $f^*(\\theta) = \\langle u,\\theta\\rangle - f(u)$, then for all $w$ we have\n",
    "$$\n",
    "\\langle u,\\theta\\rangle - f(u)\\ge \\langle w,\\theta\\rangle - f(w)\\\\\n",
    "f(w)\\ge f(u)+\\langle w-u,\\theta\\rangle\n",
    "$$\n",
    "so that $\\theta$ is a subgradient of $u$.</li>\n",
    "\n",
    "<li> When $z\\in \\partial f(w)$ is a subgradient of $f$ at $w$, then $f(w)+f^*(z) = \\langle z,w\\rangle$. This is just a restatement of the \"equality\" condition of the Fenchel-Young inequality.\n",
    "\n",
    "<li> If $f$ is closed and convex, then $f^{**}=f$. </li>\n",
    "$$\n",
    "f^{**}(u) =\\max_{\\theta} \\langle \\theta,u\\rangle - f^*(\\theta)= \\max_{\\theta}\\min_{w} \\langle \\theta,u-w\\rangle+f(w)\n",
    "$$\n",
    "Now by setting $w=u$, we get $f^{**}(u)\\ge f(u)$. Now clearly if $f^*(\\theta)=\\infty$, then our maximization will not occur at this $\\theta$. However, for any $\\theta$ with $f^*(\\theta)\\ne \\infty$ we have $f^*(\\theta)= \\langle w,\\theta\\rangle - f(w)$ for some $w$ with $\\theta$ a subgradient of $f$ at $u$. Thus\n",
    "$$\n",
    "f^{**}(u)=\\max_{\\theta} \\langle \\theta,u-w\\rangle +f (w)\\le f(u)\n",
    "$$\n",
    "by definition of subgradient and so $f^{**}=f$. </li>\n",
    "<li>\n",
    "If $f$ is differentiable, $\\nabla f^*(\\theta)= w$ where $f^*(\\theta) = \\langle w,\\theta\\rangle - f(w)$. To see this, apply fact 2 to $f=f^{**}$ to get\n",
    "$$\n",
    "f(w)=f^{**}(w)= \\langle w,\\theta\\rangle -f^*(\\theta)\n",
    "$$\n",
    "where $\\nabla f^*(\\theta)=w$. Rearranging, we have\n",
    "$$\n",
    "f^*(\\theta) = \\langle w,\\theta\\rangle - f(w)\n",
    "$$\n",
    "as stated.\n",
    "\n",
    "Note that we now have an interesting inversion phenomenon going on: $\\nabla f(w)=\\theta$ when $\\nabla f^*(\\theta)=w$.\n",
    "</li>\n",
    "</ol>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we'll define the third main concept: Bregman Divergences.\n",
    "\n",
    "<div class = \"definition\">\n",
    "<h4>Bregman Divergence</h4>\n",
    "Let $R:S\\to \\R$ be a differentiable function. Then for any $u,w\\in S$ the Bregman divergence $D_R(w\\|u)$ is given by\n",
    "$$\n",
    "D_R(w\\| u) = R(w)- (R(u)+\\langle \\nabla R(u),w-u\\rangle)\n",
    "$$\n",
    "</div>\n",
    "\n",
    "When $R$ is convex as well as differentiable, then $D_R(w\\| u)$ is always positive. The Bregman divergence is measuring how much error you acquire at point $w$ by approximating $R$ as a linear function at point $u$. Since approximating things with linear functions is basically the standard tactic in all analysis, this seems like a pretty reasonable thing to have a handle on.\n",
    "\n",
    "Intuitively, $D_R(w\\|u)$ is some kind of spiritual metric. Instead of actually measuring a distance though, the distance is decreased by an amount that reflects how easy it is to approximate $R$ by a linear function. \n",
    "\n",
    "For example, suppose $R(x)= x^{2}+x$. Then\n",
    "$$\n",
    "D_R(x\\|0.5)=x^{2}+x-(0.5^{2}+0.5+(2)(x-0.1)\n",
    "$$\n",
    "I've plotted the Bregman divergence of the function $R(x) = x^{1.5}$ below:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXUAAAFECAYAAADY267AAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xl4G9W5x/HvyJu8xE7ITsKSFcgtCQkkJCQFE5oQCmUv\npRjoAiVAoUBbdkpcSlugLbelbC3LLaRiaaEFwnIpBUJZy4UsrKEQthCyktiO48ibzv3jjGLZkWzZ\nljXS6Pd5Hj0eSaOZN4786ui8c84BERERERERERERERERERERERERERERERERERGRHDcqjecaDpSk\n8XwiAAS8DkAkTUYD09N4vg3ARWk8n4hITrk2yf0ujNk+GrgMuAQ4pZPXrAQagXXAqTGPT+1wX0Qk\npx0APAM8AZwGnA3cApzVzeNMAs5Nct8F7s8K4PWYx18GBiV4zfeAXYH8OM/dneR5RVIi3ptQJFO8\nBDQDvwWejHl8MbACeDbJ4xwBPNTNcx8IvBNzfzlwMPDXOPs2AZ8mOM4GYCzwQTfPL9Ij6lOXTFYI\n7A883+HxV7BdI8maSvsEnYyRQE3M/RpgXCfHPwP4MXBkh+eWA/t289wiPaaWumSy/bHJuKHD4ztj\nW8DJKgFMzP0jgVbgy8CbwDzgauC9mH0GAOGY+01AWYLjPw383d1eBvyLtg+EzcD4bsQq0itqqUsm\nm43tU48VACqxXTDJyovZ3hX7QfEYMMf9eT+wqsNr6gAn5n4xsCnB8R+O2d7sxhe1DfuNQyQtlNQl\nk1WyY7/5PGA9sAgYDDwOnAOcANwB9ItznJaY7U+x/dtDgS3YFvWj7PhtYCXtC6ODgNVxjn0ycF/M\n/dIO56sg8YeBiEjOKAZqgWDMY6OwLffYQUQL3Z8O8BS2L7yju2jrOtkTezXMd4Cr3MeOiNk3evVL\nKbZrJmo5MMTdHkNbK34W9hsF2G6ej2g/6Ogc4JA4MYn0CfWpSyaair0ufAtQhU2gFcBA4HjaWr5F\n2K6Ng4FDge8Cn8U53nPANOwHwlxsa34N9gPjGOK3wLcC1wFXYL/RXof9hgD2CpjTgKXAC26M5wO7\nASfSvtW/D3B78v90EW8FgX9ji0PvAL/0NhzJMbOA093t24DyBPv1xxZCk7Gg612SFgSuT+HxRLrU\n2z71MLaVtA8w0d2e1dugRJJ0ILDE3R5D+4JorBpgI4kHD/WVE4E/pPmckuNSUSiNftUsxP5RqSgk\n6bAfcCxtifoNbN92ossHf4ftakmXXbBXwrzX1Y4imSaA7X7Zgu13FMlme3gdgEhvpKJQGsF2v1Rg\nh3JX0v4a4g+wX41FRCR5K7FTTHjqJ9ih0rFMvB2lx6q9DsBHqr0OwGeqvQ7AZ3qUO3vbpz4Ie2UB\n2OuK52Av8xIREQ/0tvtlOHZgR8C9LcTOgyEiIj6l7pfUqvQ6AB+p9DoAn6n0OgCfydjcmbGBiYhk\nME/61EVEJIMoqYuI+IiSuoiIjyipi4j4iJK6iIiPKKmLiPiIkrqIiI8oqYuI+IiSuoiIjyipi4j4\niJK6iIiPKKmLiPiIkrqIiI8oqYuI+IiSuoiIjyipi4j4iJK6iIiPKKmLiPiIkrqIiI8oqYuI+IiS\nuoiIjyipi4j4iJK6iIiPKKmLiPiIkrqIiI8oqYuI+IiSuoiIjyipi4j4SFqSugmRl47ziIjkunS1\n1B8zIQak6VwiIjkrXUn9beBVE+K/0nQ+ERHpIwbAhDjZhFhvQhzrdUAiIlnA9ORFTqqjiMNEz2NC\n7Av8DbgLqHaqiKTh/CIi2Wh77uyOtCZ1ABNiKPBXoBY42amiNg0xiIhkmx4l9bRf0uhUsQ74CvAJ\n8G8TYo90xyAiIj2XsF/IhDjd7Wc/PJ0BiYhkgR71qffWLsCz2Ktb3gJ+EGefTgMzIWaYEJ+ZEJeb\nUFq6g0REsoEnSX0YsI+7XQa8B+zVYZ8uAzMhdjYhXjEhHjAhylIco4hINupRUu9tn/paYJm7XQ+8\nC+zc3YM4VXwOHIQtnr5sQozpZVwiIlnMZMQo/N2xxc+OLe2kP21MCMeEONuEWGdCzEllcCIi2cEM\nKST8Ih5fp14GLAauBh7q8JwBfhpzf7F7S8iEOBC4H/gN8Bunypu+JRGRNKqECd/M56snt1Ba4qZN\nT+qMBcCTwPkJnu9RQjYhdjUhXjch7jEhSnscnYhIFiig8ZR8mprBmOm81IpHhVIHuBv470726XFg\nJkSxCXGXCfG2CTGhp8cREclcJr+MuhvBGDDme/zBbKOoBo+S+iwggi2WLnVv8zrs06vA3H7275oQ\nG0yIU3tzLBGRzGIGllPzEhiTT5O5hfk2s4NnST0ZKQnMhPiSCfGuCfE/6o4RkexnJpayZS0YM4S1\n5l/Miib03EjqACZEmQmxUN0xIpLN8mg+oYDGJjBmP141nzIyNqHnTlIHdceISDYzeaVs+XU0d5/K\nn0wDwY4JPbeS+vaDqjtGRLKK6d+P2sVgTB7N5rf8wER2TOa5m9RB3TEiki3MhFK2fAbGDGSD+Sez\nEyXz3E7qsEN3zLf68lwiIt0VoOWoQsJhMGYSS82H7N5VQs/tpL79JOqOEZGMYgIl1F8dzdPf4F5T\nT0kyCd18Uk4tuZ7UQd0xIpIpTHk5Nf8AYxxazbVc2Fn/+fZbbRHmstmY4OUYlNTdk6k7RkQ8ZcaV\nUfcxGNOfTeYJDu0ymTcHMLfshxnyYwzV7k1JvcNJ1R0jImkWoOX4IrZtA2Mm8Jb5D2M7TeYRMIvG\nY/b6flsyL7+E5VQznR7mzrSvUZouThVvAVOBfOBVE9q+mIeISIqZojLq/hgh76+NBIPH8iCvMJ1x\nfJDwFUuHwVdOha+dBO8OhrJG1gYiHF8XZB+qeSWNwXebp18h3O6YU9y1UC8xITJi8nkR8Qszph+1\nK8CYAhrNDZzTaf/5p+WYU4/GOAtsyzx4OVsLr+ACqinseOCeRJOOuXpNms7TeRAhdgXuwk4VfKpT\nxYcehyQiWS5Ay/EFNC9sJBgcxYfczzeYymtx960rgmtnwvUzIFwAeRFagy3cvLWQBVSzOc5LMiJ3\nxpMxnf0mRMCE+KFbRD1NC12LSM/Y7pZoA/xYHjCbqUi6CFpxCY9QzeiuTpKWf0oPZFxgbhF1mQnx\nsAkxxOt4RCSbJNfdEi2C7tm+CLrMLYImdaI+/Wf0QkYGZkIUmRC/NCHWmBBHeh2PiGS+2KtbRrHS\nvMp+cVvnS4ZhDv5WWzIvu5Q1gSs5jupu9Q5kZO6EDA4MwIT4sgnxoQlxmwnRz+t4RCQTmaIy6m7r\nqrulG0XQpE6a8n9GimRsYFEmRLkJcYcJsdKEmOl1PCKSSbrubmk3ErQak3clLaWXcQPVDOjNiVP2\nT0ixjA2sIxPiKLc75hcm1KNPVhHxka66W3pRBE1GxubOjA0sHhNiqAnxiAmx1IT4L6/jEREvdN7d\nkoIiaFJBpPBYKZWxgSXiDlg63YTYaEJcYEL+HXkrIh113t2yZBhm9qntiqCf96AImlQgPXlRzgw+\n6gkTYgxwNxAGvuNU8anHIYlIH3Jo/UYhTX+KN5hoVTlcMRsWTgLjQLCZhojDFU353EQ1TX0QTsbm\nzqxrqccyIfJMiEvdAUtnq9Uu4kdmQDk1f4vX3dJHRdCkgurj4/dYxgbWHSbEBBPiRfemudpFfMMc\nUkL9RjCmhHpzM2eaSN8XQZMKLE3n6baMDay73GkGznZb7QtMiCKvYxKRnjLFZdTdHG2dT+MV8x7j\nEk2Hm+oiaFIBpvl8ScvYwHrKhBjpTjHwjglxgNfxiEh3mcnRhSzyaDZXcYVpJi+dRdCkguzJi1Qo\n7SF3MrDjgBuAvwOXOlXUeRuViHTO5BXTcHkjRQsi5AX2YAULOYVh5a+luwiaVLBkaO70XUs9lgkx\nwIS43YRYpTlkRDKZGdOP2mXR7pZzuMGsKQrGK4L+Pg1F0GSope4lE6IS+COwDPiBU8VabyMSEcs4\nhTR+D/h9E0WFO7Oa2wPf4pMpT7OgEtaX2b0qwiyqDXI+1Rmz1kKPcqeSegqZEMXAT4DTgUuBO50q\nf39TEclsZmg5taE6Kg4B+Dr3ccz407lqzlZWDLZ7lIdZXhfkzAxcQk5JPVOYEJOA24F64Aynivc9\nDkkk5wRoOaqIxoXbKOlXQQ0XDj+Rp+c+ybOj7PNljaxtKOCcSIC/UZ2RjS8l9UziroX6A+By4NfA\nb5wqmr2NSiQXmH79qLt1C+UnARxQfi/DDjmdv01qADKmCJoMJfVMZELsDtwKDAPOdKoy7iueiI+Y\nWaXUP7CVsqGFRes5aObhPD/jtWTXBM00SuqZyr388STgOuCfwCVOFWu8jUrET0xZGVuu3Urp2SYQ\nYeSUn9Bw8K/YVNoKZGQRNBlK6pnOXVnpcmwh9Trgd04Vjd5GJZLtzLxS6u/eSulgZ/zD7DTnDL4Y\nvAGA8jBv1AWZn4FF0GQoqWcLE2IccD2wJ3C+U8VjHockkoXM4HJqb62j4liGL6Fs7nzqR9kZFbOg\nCJoMJfVsY0IcBvwWWAlc4FTxnschiWQB4+TRckoBzTeHyzeWBg65hMike4DtRdArm/L5fYYXQZOh\npJ6N3GXzzsVe1/4/wM803YBIImZUObUL64qcmcy8lsCMXxMpaMrGImgylNSzmQkxDPgFMA+4DLjb\nqSLibVQimcLkFxG+oDUQ+XnLlLsKnIOvxJRuBLK2CJoMz5L6ncDhwHpg7zjPK6l3gwkxFTtJWAA7\n3cC/PQ5JxGNmnzJq7q0f/8KezLkQBq8Asr4ImgzPkvqXsSMn70ZJPSXc1ZVOBn6JLoGUnGWKS9j6\n823D3znfzL3YYdSzgG+KoMnwtPtld2ARSuop5V4CeQVwGroEUnKKmV1S8dY9DbOvG8qkhQAUNbPN\n2JGgN/qgCJoMJXW/6nAJ5KXAg5ooTPzJ7FRa8uEtW6ffcQIzroeCMIGIEyluMTdvLeRKHxVBk5HR\nSf2nMfcXuzfpJhPiUGyXTARbTH1KyV38wTgU1J1YMPn2O5sPuiZIqTt4aFvg8briyLk+LILGU+ne\nohaQwUldLfUUcfvbjwd+BqwGLtN8MpLVAo17F09Y+MC2g34znsHvAlBaX/Le1rKG71DNyx5H56WM\nbqkrqaeYCZEPfBv7af46cIVTxVueBiXSLWan4NgHbwvPvPFYRi0GoGhreV1TsP50kxd5wOdF0GR4\nltTvBQ4CBmIva7wSO4imV4FJckyIIHA2cDHwJLDAqeIjb6MS6YzJy9v12YvZ7w9XtU68Pw8gL1zS\nGmguqW7ut/FXVOtiAJcGH+UyE6Ic+CF2dOq9wNVaUk8yzrDX5xVOuuPepv3u7E9BGKc1n6LNI+4L\nD/rk7BwrgiZDSV3AhBiMvULmW9h53H/lVFHjbVSS8yo+2j24z62PhKfduXe0CBpcN+b18MBVJ3B1\nUy4UQXtCSV3amBC7Yvvbj8SuvPR7p4oGb6OSnFO2ujh/4l13tk6+60Qz2M5XV7Rxt/WNhS0ncP3q\n5zyOLtMpqcuOTIg9sVfKzHR/3uFU5cTADfGUcZxpv/tR3oS//LJl95fyAQpqhzW2bB12ntl52R9V\nBE2KkrokZkLsi50wbBx2dOqfnCrC3kYlvjTthq8UjfrfvzTu9fgAgMC2cuOs2/uW1t1fvCBHRoKm\nipK6dM2EmIntc5+Cncv9Vk31Kykx41e7FQ179ZHGCYsmUhCG1gIKP57xr6bSzcdx65sbvQ4vCymp\nS/JMiInAJcBcbEH1d04VG7yNSrLSjF8H84Ysvysy/okTotPhFn00fVVjeNhR3P/QUo+jy2ZK6tJ9\nJsQY4ELgBODPwK+dKj71NirJChPuDzDi1Wvz9njoh62DVgYACj7fe1vz2qln8Midf/Y6PB9QUpee\nMyGGAxdgZ4RcBFznVPGOt1FJZjIOM37zg4I9/3pd827/LgTI27xLpHXloddTP+IiFv9URdDUUFKX\n3jMhBgDfxw5iegn4pVPFq95GJRnjS/ccVzjhnjuaJjxaAeBsqyCw8pC/tDbu9G0W3b7N6/B8Rkld\nUseEKAFOB34MvI+dHfJpzQqZo3Z5flbR3nfe1zj53hEUhKGlkMKPZr7Q1Fr2de5bpJHLfUNJXVLP\nhCgATsLOLbMVm9wf0vqpOaLs872C+95wf3jqnXtTth6Aok8nv98YKT+OPz33psfR+Z2SuvQdd8rf\no7CXQ/YHbsFe6675Ovwov2FkcOIfF4Zn3FoZXRO0aP3oLxqbBp3M7a/+r8fR5Qoldel7JoQDHICd\nGfKrwIPATU4VunTNF8ygonEP/L7pgBtPNKPsKP6C2iHhlobh55nhy2/TSNC0UlKX9DIhhmL73ecD\nnwE3A3/VOqrZyPQrHP7iz1qm33RuZOJ9ARxDXrg04tSOvK5l6HsLNBLUE0rq4g13wY4jsFfNTATu\nAP7gVPGJp4FJEkx5QfnKC8y0my9r2f/mQgrCOC35FG3a9f7wkA/P0nS4nlJSF++ZEHsAZwGnAC8A\nNwH/VGE105jBwbxNF7ZMWXh+y0G/KIgWQUu+GPFSw8DVp+TImqCZTkldMocJUYq9aub7QAkqrGYI\ns2sptZeHxz9zWuucy/KiRdDi2oEfbetXcxJXtWq928yhpC6ZR4XVTGEmlFO7YMuw9483cy8OMPoZ\nAIobgl80BsPzIwH+piJoxlFSl8zWobC6AVgI3Kdl9/qSmVZBzdW15VvmMPsKmLQQHENhU16YQOtl\nTfncpCJoxlJSl+xgQuQBs4GTsde+v4ydTOwhp4qtXsbmD8YBDqmg5ue1hXnTmHUNzLgeCsLktRIJ\ntnLT1kIWqAia8ZTUJfu4fe9HYRP8AcAj2Bb8M04VrV7Gln1MIEDr0f3Y8vPaQNmeTLkdp/JKTJmd\nUbkizKLaIOerCJo1lNQlu7ndMydir5zZGbgH24JfrjlnOmMKHSJVZdRftYV+Ixn/GHlzfkTr4P8A\nUB5meV2QM6lGRdDsoqQu/mFC7IVtvVcBW7DJPeRU8ZmngWUUM7iQxjMKaL5gK2UDGbaUornn0Dj6\nJQDKGlnbUMA5KoJmLSV18R93zpmZ2Nb7ccBybPfMg7m5DJ9xgOkV1Py4nrKjWsnPo3wVFbPPoW7S\nIxgHgs00RByuUBE06ympi7+ZEEHgcGwLfjbwL+Bh4FH/X0FjShwi3yyn7pJa+o8FoLCWsbPO4NMZ\nD9BUECEvQmuwhZtVBPUNJXXJHe5iHvOwRdZDgfewRdZHgLf90wdvxpVSf14L+d9tJFgMsFNgLftN\nOYvXKx/hizI7ULc8zKN1Qc5TEdRXlNQlN5kQhcCBwJHYJN9CW4J/wami2cPwesDkAYf3Z/NFNQyY\nGX10Gi8zc/zlPDFnMSsG288sFUF9TUldxB3BOhGb4I8ExgBPYLtpnnSqqPUwvC6YwYU0zi+g+byt\nlA0CCLKNb3Ivhwz7FXfOXcEzo+2eKoLmBCV1kY5MiBHYGSSPAmYBr2Bb8IsyYxZJ4wD7V1Bz4fbC\nJzCGDziLW5hTfge/mV3Lwkl2z6IWGgz8pCmfG1UE9T0ldZHOmBBlwFxsC/5w4HPgGWAx8LxTxaY0\nRjMsj5aTyqg/u5b+YwAcIhzOY3yfm5he9CS/mgnXz4BwASqC5gD3/TkTqAQqnSqmo6Qukhx3qoJp\nuH9AwAxgJTbBL6ZPkrwpdYgc05+as2roP8MQcAAGspHTuIMzuZVdAh9z+xRYUAnry+yrNBLUnzom\ncWBv4HXs++85p4qnUVIX6Rl3ge39SHmSN/nA7Apq5jdQckQzhYUABTTxVR7nZP7METxKEY08Nh4u\nnAMrBttXlod5oy7IfBVB/aGLJL4YeMWpYlvsS1BSF0mN3iV54wD7lFJ/msE5uYHSiugzM3iJU1jI\nCfyFge7LlwyHH8+FZ0fZfVQE9YceJPEdDoGSukjf6CTJvwgsAZZ86eI3t/zns/HfKGbb/Doqdo2+\ndizvcwoLqSLEmJgelFXlcPkhsHCSva8iaPZyu/PGA1Pc2wF0P4nvcFiU1EXSw03yUzfVD6j8bNPI\nr/Ur2rLP0P7rgu+snsCSj6fw/sdjGfrxOg789F9MbX6t3R9AXRFcqyJo1nLHRUygLYFPwV5Guxb3\nAx54FXi5m0l8h1OhpC6SDmYwcPgANlVtod/BLRTkAexUtJGzdrmFb4y6n//a/R0Cuxk71+R64GNo\n/QQWFcIP94SPiu2RNBI0s5kQJdiEPZm2BL4X8BFtCXwJsKwPxkAoqYv0HbNHPs1H92PLSTX03zt6\n5YpDhEoWczJ/5jgepKLjHGP5YEbCG/vDO9Ng9wqYWARftNCUH+D5nfN5CngH25XzoVNFOO3/NIm2\nvncDxgJ70pbERwMraJ/A33CqaEhHWHiU1OcBvwXygNuBazs8r6QuWcjkATOKaTgun5YTtlC+c/SZ\nApqYzTMcySMcySOMZHXCo8QrgjqF/OCLcbxT4Gxv+e2BTSa7Ypf5+wCb5Nv9zM1ZKVPHbXWPxv6u\nx2JHG0d/jgBWY3/f/6Etgb/tVHlW3/AkqedhJ1L6CvYX8n/AN4F3exuYSPqZMmBuBTUnNlJ0WJji\nsugzA9jE4TzGUTzMXP5BOVs6PVJPiqAmRD6wC+2TTfTnGGAr8RP+h8BGp4pIL/7xWc+dImIANnHH\n+x0OxHabxPsdfuJh8k7Ek6Q+A1iAba0DXOL+vCZmHyV1yWBmZ4fI1wawuaqO8gOi/eMAo1nJUTzM\nUTzMTF4kP4nV9fqqCOomrGG0T1SxCb8f8AW2pb/e/Zloez1Qk+kfAu6/uQIY7N6GdPjZcXswsI3E\niXt1li2R6ElSPx477en33PsnA/sD57YPzJwEPAfO5708n0gvmVJgRpBtXylm29Gb2WmP6DMOEabz\nyvZulb14N+k/kJYA3DYFFhwMG0rtY+kcCer2CQ+i86QXu12G/RCIJvpNQBhodG+x2/Fu8Z43QFHM\nLdjhfrxbx30GxMQ5yD1uMh9UG4ANPqtJ9Cip56fgpEmovgegkAu2BPjSS2G+exdK8pIWbUm8hIbD\na2mZ0Ep+IEwxYYoppoE5PMWRPMLhPMYw1nXv6JBwJGjtNekbCep2HXzu3roU8yEQTaA7kTgRD4jz\nWLz9HLpO/NHbNqAmzr41xCRqnyXprlS6t17pbUt9OlBNW/fLpUCE9sVSM4/HeYFZ1NOv3YvLqf3c\n4PxzC+X/i5K8pETHJF4xoZX8QPTZAK1MZimVLKaSxczmGUp6eCnx0mHwo0M1ElT6jCfdL/nYQukh\n2BbCq8QplBqgmXyWMMX9U6pUkpcU6V4Sn8UL9O/l5cQaCSpp4tkljYfRdknjHcAvOzxv4jVZupHk\nn95C+WLs5UXvgKM/mpxlHOy1xJOLCM8oZeth6UjiUQmKoLdsLeRKjQSVPpC5g4+S+R6aTJLPo6W1\nnLqPWsl7pY6Kl3AHAoDTm6G4kpFMABgHTC6mYf8SGmbVUzahkWBJ7F59mcSjWgJoOlzxQnYn9Y5a\nyGMJU3iOg3idfVnCFN5n/A77BWg15dStcjD/V0P/FwyBJcAycDRQI2uYAuzQ68ml1E8vonHmFvqN\nb6awqOOeg9jAvrzOZJYykxf7JIlvj4rERVBNhytp4K+kHk8d/VjOJJYwZfvtXfaiNc5FPOXUrs2n\nZUkN/Z+PkLcEO0psFTjZdJ2qzxgHOwBkLDCxnNoD8midUUf56Fbyd/hPHMkqprCEySzd/j8+gtVp\nedNqOlzJAP5P6vFsI8ib7N0u0b/J3jSxQyOPAK2tZdSvL6TpwzDBt+vp9zZtAxM+BqexD0PNESaA\nncZqDDC2iPD4MuonRgiM20rpiCaKgvFeNYYPtv8PTmYpk1nKEDakNXLYsQgabKYh4qgIKp7IzaQe\nTxMFvMte25P8ciaxkjF8zoiEr3GIUMrWjUU0ftJC/jt1lL9lCERHo60Epz5t/4CMZ/KxBcsxwNhi\nGvYqoeFLLeSP3UrpsBYKEo5/KKeWsXzAXry7vRtlH5b1WRdKsuIUQSPuSFAVQcUrSupdaaCYDxnN\nB4xlJWPa/fyE3YiQl/C1xTTUF9FYk0frF8DaZgpW11P2WYS8dew4um1TdnXzGAc7zLzdqEOHyJBi\nto0IEh4RIDK8lbzBzRQMaKCkIkJeINHRBrOesXzAGFYylg/abQ/ki4yaMyJeEVTT4UqGUFLvjSYK\n+ITd4ib8DxkdtzsnEYeICRLe6n4IbATWNVOwOkzwiyYKG9xunmRH3sU+34QdG5DM6L52zztEivJo\nLS6kqTxIeGcHMzxCYHAzBQO2UVzeSn7iT7Q4RrJqh4Q9hpWMYWWXk11lgk6KoGdSzcueBidiKan3\nlVYCbGAwGxjMeobE3Y69v4mBXofcbaXUM5gNDGF9zL+o/f3o9lDWESR7yw8qgkqWUFLPFM3k8wUD\nd0j29ZQRJkgjRXFvXT3XRCH5tLR7Jkg4wSviP1fMNgaxcYfk3dOh8tlERVDJMkrqIvGoCCpZSkld\nJJaKoJLllNRFQEVQ8Q0ldRFNhys+oqQuuWtVOVwx2xZBjaPpcMUXPFn5SMRTcafDbe79mqAi2Uot\ndclKmg5XcoC6X8T/okXQi+bAu21F0OVuEVTT4YqfKKmLvy0dZkeCPjPa3lcRVHxOSV38qWMR1B0J\nekVTPjepCCo+pkKp+IuKoCLdp5a6ZBwVQUUAdb9ItkswElRFUMlVSuqSveIUQdc0FHCuiqCSw5TU\nJfuoCCqSkAqlkj1UBBXpG2qpS1qpCCqSNHW/SOZSEVSk25TUJTMlWBP03EiAB1UEFUlISV0yS8c1\nQTUdrki3qFAqmSFuEbRFRVCRdFBLXVJGRVCRlFL3i3hDRVCRPqGkLumXoAiq6XBFek9JXdJHRVCR\nPqdCqfQ9FUFFMpta6pKUlgDcNgUWHAwbSu1j5WEerQtynoqgIn1C3S+SegmKoG/WBZlPNS97GpyI\nvympS2qpCCriKSV1SQ0VQUUyQtoLpV8HqoE9ganAkl4cSzKAiqAi2a83Sf1N4BjgDymKRTwSbyRo\ntAi69Rcqgopkk94k9RUpi0I8kaAI+kZdkDPrrlERVCQb6Tr1HJWoCFoXVBFUJJt1ldSfAobFefwy\nYFGyJ6kQFHbaAAAO4klEQVSO2a50b+KNREXQ+iIVQUU8VkkK0mMqrn55FvgRiQuluvolA6gIKpJ1\nPJ0mIB2XRkoPqAgqklt6k4yPAW4ABgG1wFLgsDj7qaXugc6KoBoJKpIVNPhILI0EFfEFJfVc17EI\nGmymIeJoJKhIltLUu7kqThE04hZBr1QRVCS3qKWexTorgmo6XJGsp+6XXKEiqEhOUFLPBQmKoOdG\nAjyoIqiIryip+1knRdCbqKbR2+hEpA+oUOpHKoKKSHeopZ6hOimCnk81Kz0NTkTSQd0vfqAiqIi4\nlNSz3dJh8KNDNRJURAAl9ey1qhyumG2LoMbRmqAiAqhQmn3iTofbrOlwpVObgAFeByEptRnYKVUH\nU0vdA/GKoBVhFtXaIqhGgkpnetR6k4yW6P9ULfVMFy2CXjQH3m0rgi6vC3Jm7TW84mlwIuILaqmn\nydJhdiToM6Pt/bJG1rgjQVUEle5QS91/UtpSV1LvYx2LoO5I0CvckaAqgkp3Kan7j7pfsoGKoCLi\nBSX1FOusCKo1QUWkrympp4iKoCI9Mgr4qJPnh2PXQG7oo/OPA74ETAQWAUv66DxpE/A6AD9YOgy+\ncip87SSb0MsaWROIcHxdkMlUK6GLJDAamN7FPhuAi/owhiOA1cD1wI/78Dxpo5Z6LyQqgtYXqQgq\nkoT5wMVd7NMCPAacCtzdBzH8t/tzAp1/Y8gaSuo9oCKoSJemANVABTYZFwGTgHuA59ztz5I81v8B\n59I3ST3qGODn3dj/+8BNfRRLryipd4OKoCJJWwJsAe4AHnYfOxq4AZvQvwb8vRvH2wCMBT5IYYxR\nR7pxjQDeT/I1g/ogjpRQUk+CiqAiPTID+J67XQicjO27BtiP7rWMlwP70j6pj445fjyv0PaBksgx\nwGXYbwKLuxlTRlJS70KckaBrGwo4py6okaCSiVI51s/pzSCnvbBXrXwZe4XLVOCHwKfu8yXQ7u/n\nSKDV3f9NYB42wa5wn98MjO9wjg+BS5OMJ9Hx/073vjH05lwrOntRqiipJ5CgCPqT+iJNhyuShNnY\nVvKT7v0jsZcnRpN6Xsy+uwLvYFvhVwHXYD8QPo3ZZxu2td8TyRy/K3thi7VRs4BgzP3ngcdTdK5e\nUVLvQEVQyW69al2nUiVwe8z9nbAt9n+791tinosmvKHYfvga4NEOx6vATjscK9nul2SO35V3af+t\nYAHw0zj7peJcvaKk7lIRVCRlHOBA2ifcvbFJeTiwBlgLlAH1wJ7Yq2OmAP9y9z+C9slwODaxxkq2\n+yWZ46dKOs8VV84n9QRrgqoIKtIzE4FvAsXAscCd7uN3YgcaDQfuwl7WOA14BpgL9MMm+yC2eLm6\nw3H3oX3LvzuSOX6qpPNcnjEmQ2+vD8cc/C0M1fZWdilrAldyHNWaBU8yll+K8/2Bq5PcN0jbVTOZ\nIpWjXBP9n/bo/zonW+qryuHyQ2wRFNrWBFURVCRtaoCN2Ou9N3ax74nAH/o8ou65zusAEsmp+dTj\nFkFbVASVrOKn+dQd4HTgtk722QXbP93V9ebZTItkdFdLAG6bAgsOhg2l9jGtCSpZyk9JXSwtkpEs\nFUFFJNf4tqW+ZLgdCfrsKHs/OhJUa4JKllNL3X/U/dKZREXQpnwVQcUXlNT9R90v8agIKiLSu6T+\nK+xIqSZgJfAd7BwHadVZEVQjQUUk1/Tma9wc4Gkggp20BuCSOPv1SfdLgiLoG3VB5msJOfExdb/4\nT8Z0vzwVs/1v4LheHKtbEhVBNR2uiOS6VPWpfxe4N0XHSkgjQUVEOtdVUn8KGBbn8cuARe725dh+\n9XsSHaQ6ZrvSvXWHiqAikgMq6X56TLlvAy/SfrL4jno84VZzAHPzfpjBF7ZNulV+CYuoZnR6/nki\nGSeTuxfrsXOIb8HW2hpi7n/Tw7gSWQyc5nUQZNCEXvOAC4GDgHAvjrODToqgZ9Zdw8upPJeIpExZ\nzPZH2IT5jEexJCOTPyB7LNCL1/4e+5/4FLAUuDkVAS0ZDod8C752kk3oZY2sDUT4el2QfahWQhfJ\nQtOAl7HrjH6OzR0FMc9HgPnAf9x9box5LgD8BtiAXRTjHHf/aO6qAO5wj/sZ8LOY574NvIC9/HqT\n+/p57nM/x64heiP2m8QNKfh3ZoTetNTHpSwKdiyCak1QkR5I5dVfqVtXoAU4D3gNO+viE8DZwO9i\n9jkc2A+bpF/H1uyeBM7AJuJJ2O6cB2jfwv4TdhWlMdhG5qPAKuCP7vPTgP8BBmI/OO4ARmBrgQcA\nC2lbyMMXetNST4m6Irh8Now/1yb0vAiR0iZuDBcwsulqrldCF8l6S4BXsS3sT7AJ96AO+1wD1GET\n8rPYJA5wAvBbbEu8BvglbdduDwUOAy7ALky9wd33xJjjfoJN5Aa4G7vy0pCY5313zb9n0wTEGwla\nHubRuiDnaSSoSA9l5qpd47ErF+0LlGDzzmsd9lkbs91AW//8cGyij/osZns3bDfOmpjHArQt/hzv\nuLjHXu9u+65fPe1JXUVQkZxzC7ZL5RvAVuB8kh+suAbbZRMVu70KaMR2rUR6EJfvEjqkufslQRH0\neBVBRXytDFuMbAD2BM7qYn+Htm6Rv2D743fGrmt6MW3JeA3wD+y3gH7YfDYGODDJuNa5+/tKWpL6\nqnI49RjYd74d2h9spqGwhR/VF7Fb5Coe1NB+EV/7MXASts/8j8B9tG8ld/z7NzGP3YZN3G9gW/uP\nAa20tcxPBQqBd7BXuPyVtgGTsceJd67fAce7r/tt9/9ZmSkt86kHL98+EjTijgS9UiNBRXok1yf0\nOgzbnbO7x3GkUsZM6JW0cIGKoCLSI0FgNra1PhRYAPzN04gEQzUzvA5CxCdyrauyGHs5ZB22D/wO\n2o9c9YOUThOQDhkbmEgW0t+T/6Q0qXs++EhERFJHSV1ExEeU1EVEfERJXUTER5TURUR8REldRMRH\nlNRFRHxESV1EssGoNJ5rOHaK4KykpC4imW40MD2N59sAXJTG82UdjYATSZ1c/Hu6Nsn9LozZPhq4\nDLgEOKWT16zEzsm+DjvjY9TUDvf7kqYJEMlh2fL3dADwDHY90tOwa5LeQtdzqXc0CTg3yX0XuD+j\n65xGvQwMSvCa7wG7En9yw7uTPG9vpTSpe7acnYj42ktAM3ae8idjHl8MrMCuQ5qMI4CHunnuA7Hz\nq0ctBw7GzrXeURPtl7+LtQEYC3zQzfN7SkldxGdMKDWteaeqV/O2FwL7A893ePwVbNdIskl9KvCL\nbp57JHaR6qgaYFwnxy8CyoH/AI/EPLccu66qkrqIeKeXyThV9se2lhs6PL4ztgWcrBLad0MciV35\n6MvAm8A84GrgvZh9BgDhmPtNJJ6u92ng7+72MuBftH0gbMYump1VdPWLiPSF2dg+9VgBoBLbBZOs\nvJjtXbEfFI8Bc9yf92MXoI5VR/sVg4qxS9bF83DM9mY3vqht2G8cWUVJXUT6QiU7drHMA9YDi4DB\nwOPAOcAJ2MUv+sU5TkvM9qfYrpCh2IWsa4BH2fHbwEraF0YHAavjHPtk7HqpUaUdzldB4g+DnJYt\n1XqRbJANf0/FQC12KbqoUdiWe+wgooXuTwd4CtsX3tFdtHWd7Im9GuY7wFXuY0fE7Bu9+qUU2zUT\ntRwY4m6Poa0VPwv7jQJsN89HtB90dA5wSJyYUk1Xv4hIxpqKvS58C1CFTaAVwEDgeNpavkXYro2D\ngUOB7wKfxTnec8A07AfCXGxrfg32A+MY4rfAtwLXAVdgeyOuw35DAHsFzGnAUuAFN8bzgd2AE2nf\n6t8HuD35f3ruyIaWhUi28Mvf0yzgdHf7NuzVJ/H0xxZCk7Gg612SFgSuT+HxOqPl7EQk6x0ILHG3\nx9C+IBqrBthI4sFDfeVE4A9pPmdKKKmLSLrtBxxLW6J+A9u3nejywd9hu1rSZRfslTDvdbVjrvLL\n10WRTKC/p8T28DqAHtLcLyI5TH9P/qM+dRERiU9JXUTER5TURUR8REldRMRHNKJUJLtsRsVSv9ns\ndQDdpTegiEj3pf3ql59hJ8pZhp2TeJdeHEuSV+l1AD5S6XUAPlPpdQDSO7HTZJ5L4olv1FJPrWqv\nA/CRaq8D8JlqrwPwmbS31LfEbJdh52cQEREP9bZQ+nPsNJsNwPTehyMiIr3R1VqGTwHD4jx+GXb1\nkqhLsPMufCfOvh9gZ2ETEZHkrQTGenXyXYG3vDq5iIhYvelTHxezfRR2JREREclSD2DXAVwGPEjb\nGoAiIiIiIpKJvg68DbQCUzrZbx6wAngfuDgNcWWjnbDF6v8A/8Cu1xjPx9jVY5YCr6YlsuySzHvt\nBvf55cDkNMWVrbr6fVYCtdj341LsAtAS353AOmyvRyKevzf3xC5L9SyJk3oe9qqY3YECbBfOXukI\nLstcB1zkbl8MXJNgv4+wHwCyo2Tea18FHne39wdeSVdwWSiZ32cl8Ehao8peX8Ym6kRJvdvvzb6Y\npXEFtmXZmWnYN8bHQDNwH7bYKu0dCdzlbt8FHN3Jvl1dnpqrknmvxf6e/439RjQ0TfFlm2T/dvV+\nTM7zdD6hV7ffm15NvTsCWBVz/zP3MWlvKParGe7PRP+ZBvgn8BrwvTTElU2Sea/F22dkH8eVrZL5\nfRrgAGx3wePAhPSE5kvdfm/2dERpsoOSEtF8MG0S/S4v73DfkPj3NhNYAwx2j7cC2wKQ5N9rHVuW\neo/Gl8zvZQl2gr8G4DDgIWyXrPRMt96bPU3qc3r4uqjVtJ/VcRfsJ1Au6ux3uQ6b8NcCw4H1CfZb\n4/7cAPwd+xVZSd1K5r3WcZ+R7mOyo2R+n7HzQj0B3Iyt+Wzq29B8KaPem88C+yZ4Lh87BHZ3oBAV\nShO5jrarCy4hfqG0hLYZM0uBF4G5fR9a1kjmvRZbjJqOCqWdSeb3OZS21uU0bP+7JLY7yRVKPXtv\nHoPtA9qGbWE+4T6+M/BYzH6HAe9hiy6XpjPALLITtq+84yWNsb/L0dg/rGXYqRr0u9xRvPfafPcW\ndaP7/HI6vxRXuv59fh/7XlwGvIQm++vMvcDnQBM2b34XvTdFRERERERERERERERERERERERERERE\nJIP8P4uRExdGIduqAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x11100a590>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# <!-- collapse=False -->\n",
    "x = np.arange(-1,1.1,0.1)\n",
    "\n",
    "R = np.abs(x)**(2)+x\n",
    "line = 2*x+0.5**2+0.5-2*0.5\n",
    "D = x**(2)+x-0.5**(2)-0.5-2*(x-0.5)\n",
    "fig = plt.figure(figsize=(6,5))\n",
    "ax = fig.add_subplot(1,1,0)\n",
    "ax.plot(x,R,lw = 2,label = \"$R(x) = x^2+x$\")\n",
    "ax.plot(x,line,lw = 2,label = \"Tangent\")\n",
    "ax.plot(x,D,color='orange',label = \"$D_R(x|0.5)$\")\n",
    "ax.fill_between(x,R,line,color = 'red',label=\"$D_R(x,0.5)$\")\n",
    "ax.set_title(\"$D_R(x|0.5)$\")\n",
    "ax.legend(loc= 4)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the above plot, the distance between the blue and green curves is the Bregman divergence, also plotted as the orange line."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The Bregman divergence allows us to succinctly define the concepts of <i>strong smoothness</i> and <i>strong convexity</i>.\n",
    "\n",
    "<div class = \"definition\">\n",
    "<h4>Strong Convexity</h4>\n",
    "A function $R$ is $\\sigma$-strongly convex with respect to a norm $\\|\\cdot \\|$ if\n",
    "$$\n",
    "D_R(w\\|u) \\ge \\frac{\\sigma}{2}\\|w-u\\|^2\n",
    "$$\n",
    "for all $u$ and $w$.\n",
    "</div>\n",
    "\n",
    "Intuitively, strong convexity means that the function is growing <i>faster than a quadratic</i>. So in particular, linear functions are NOT strongly convex, but quadratic functions are.\n",
    "\n",
    "Alternatively, we have the notion of strong smoothness:\n",
    "\n",
    "<div class = \"definition\">\n",
    "<h4>Strong Smoothness</h4>\n",
    "A function $R$ is $\\sigma$-strongly smooth with respect to a norm $\\|\\cdot \\|$ if\n",
    "$$\n",
    "D_R(w\\|u)\\le \\frac{\\sigma}{2}\\|w-u\\|^2\n",
    "$$\n",
    "for all $u$ and $w$.\n",
    "</div>\n",
    "\n",
    "This is just the opposite - now the function is growing <i>slower</i> than a quadratic. Note that it could still be growing faster than a quadratic. For example, $R(x)=\\frac{1}{2}x^2$ is both $1$-strongly convex and $1$-strongly smooth:\n",
    "$$\n",
    "D_R(y\\|x)=\\frac{y^2}{2} - \\left(\\frac{x^2}{2} + x(y-x)\\right)= \\frac{1}{2}(y-x)^2\n",
    "$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXUAAAFCCAYAAAAOgk3dAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJztnXmcHFXxwL+9m2NzhwDhJoFw3/cNZkEuuRUUfhIuBRUQ\nEUROpQHlVhERUeRUDrlvEIGOQDjlvgkQSCDkTiCQO1u/P+pNpnd2Zqd3ume6Z7a+n88kM9M9r6vf\ndlfXq1evCgzDMAzDMAzDMAzDMAzDMAzDMAzDMAzDMAzDMAzDMAzD6AJtwOppC1EDjgCeSluIBDkd\nuDptIQwjSXYAngFmAdOBp4Et3LYjyNYNPBi4Fvgc+BJ4Dzg1BTlGAz8o+M6UutIL8IH3ga+AccA1\nwLBqC5YAw9G/Y1PKcjQs1rHVZyDwAPBHYClgJeAcYH4X2qjl3+kPQF9gHVT2fYEPanj8HJLCMeuF\nO4C9gUPQv9HGwP+AXdIUqot4aQtgGJWyBTCzxLZ1gbnAImA2MMN9fz3wF+Ah1BLb2e072rX1JrBP\nqJ3rgT+jD48vgedob9Huhlrcs9x+/6WjFZzjDWC/Ts6nDfgJMNYd61xgBPCsa/9WoGdo/6PdvtOB\ne4EVQtu2A150v3sB2NZ9/1u0T+ai/XJ56Ng/Qi3UmcAVobaOQEdAl6D9+BGwR2j7INSanQh8CpxH\n/mG5Btons4Cp7hxAFc8fgMnAF8DrwPol+uVI4G3XJx8Cx4S2jXTHPMm1NdHJm2Np4D53jOedbKUs\n9W8Cc1DjoBQruvamo33/w9A2H7gNuMHJ+iawudt2KnB7QVt/dC8o3Ye9gFeA491+zcAY4KzQMf/h\n3o9H/46z3fF3cnJuEDrmUOBrtF8MI3MMAKahincP1FoPczgdb+DrUQWTU3IDUGv5NKAH0IreEGuF\n9p+GPkCagX8Ct7hty6DKYn/0BjwBWAAcVULeq9Eb/QhgzSLb24C7gf7AeuiI4wl0WD0QeAs4zO27\nM6okN0Fv/MtR5QkwBFXM33dyHYwq41z/BEVkbEOV1UBgFWAKsLvbdoQ7rx+gyvjHwGeh396NPij7\nAMuiyjOneG9B/b44Obdz73dHLeCB7vPawPJF+gTgW8Bq7v1OqFLa1H0eCSxElVszsKfbPshtv9W9\n+qAPjU+BJ0sc50K0bzrjSfSB1wu14qeg1wxOhrnotegB56MPZFD3zdfo3xYn60RgK/e5sz5cH/37\nrQOcibobc9b42eSV+jA6ul/+7M4rx89QA8AwMss6wHXABPTmvhe1RqC4//Q6VFHn2BH1cYe5Gb1Z\ncPv+LbRtT+Ad9/4w1GoKM57SSr0FVXD/Q5XkWNpbvG3kHza4/U4Jfb4UtW5BrbrwzdrPtTkMGIWO\nKMI8gz7kQBVXMZ/6dqHP/yLv7z/CyZqjr9t/KLAcMM+dW45D0IcRqNX6Vzpav63oCGdruu4Cuxt9\ngIIq9TkFbUxGlWUz2idrhbb9ltKW+tXkH9jFWAUd5fQLfXc+ek2BKvVHQ9vWc7LleAr92wDsSt71\nVq4PQUci76GW94jQ9z55pT6cjkp9a+CT0Of/AQd2ODMjEuZTrw3vosPzVdBh5orAZWV+82no/Yro\nAyHMJ+57UP/z5NC2ueStrRUL2ipsu5B5wAWo1b80OlS/HZ1AzVF4rPDneeQVygq0v1m/Rm/4ldy2\n8Z2cExT3q08KvZ9De+VVuA20H4ahLqHP0dHBTOAq1NoE+CVqVb6AjlKOdN8HqMX7Z3eOf0VHTcXY\nE31ITXftf4v27oPpqDILy9ffydCD9n/fwn4JM432LqxCVkQt5q8L2gs/sMJ/rzmoos7pgptRZQ3w\nf8BN7n25PgS4EVgVdRt+2ImMhTyPXkcjUQNoBDoiMyrAlHrteQ+1DHM+xFITguHvJ6IPhPDk0jDa\nuxdKMRFYOfTZK/jcGbNRBd+PvGuhHIVyDw997ocquk/dtsJojfA5JTlROgF1Ey2NuneWQl0fG7rt\nk1E3wkqoz/5K8nMSf0IfcOuh1nR4VJKjN3AncDE6MlgKVWxRJgOnopb1qqHvVi2xL8BjqIVfyqc+\nEXVt9Q99tyqdP8jD3IEq15VQl93N7vtyfQjabw+gI7vtQ99LifdhbgAORUcJt6OjF6MCTKlXn7XR\nYWnuJlwFtYRyfszJqJINTy4WKoPnUIvql26/kWj0w60l9g/zEHrj7YdahMdR2i8M8CtUifVCLbif\noVbZe538xit4n/t8C2r1bowqvvPduYwHHkaV5CFOru+hVtoD7reTaT+EL3XcKIrzc9Tl8HvU0m5y\nbe/kth9E/kE3C1U8bWg/bI32+Rx0FLK4SPu93Gua+92e6OR0FBYDd6Euij7ow+NwSiu/x4H/oO6d\nzdC+G4DOIRyJKt9n0Idxb2Aj1NX2z4jyTEUn5K9HJ5tzf/dyfTgKnUM4HHU73UB+FBX+G01F+6jw\nb/tP4NvoHMuNEWU1imBKvfrMRhXD82gky7NoFMXJbvvj6OTiJHRCC/SGDt/UC9Folz3Rm+IK9CZ6\nv8T+hD5PQ5XWxe79uqjPslRIZRvqf52KWs27AHuRd2cUUzaFllju8+PoQ+JO1IJcDZ0QBXVH7I32\nwzTgF+5zLgLoj6hfdQalXVXhY3XWB6BzC73QCJUZqDWYe7htgT5sZqPzHScAH6MTpH9z+3/s5Lyk\niByz3W9uc/seQseJvs5GHsejlvUkdI3AtZ3sC9ovD6FzCrPQiKXNUGWPO/5wtM/vAn5N3vddrp9A\nrfNdyFvpOUr14aroPMph6HVyC3qN/b7IMeegcwZjUGMhNwk7AXgZvf6eLnP+RgJci1pObxTZdjL6\nhxhSU4mMSmlClfU30hbEMAq4Bg2RNWrAjujQqlCprwI8gq5oM6WeXXZDJzp7o7HDn7n3hpEVhqOW\nez2sim0YhtNRqd+O+uxMqWebs1HXwZeo+2fLdMUxjHach7qwTi+3o5Esw2mv1PcjH49sSt0wDCMD\n9Kjwd32BM9DFCTksl4NhGEbKVKrUR6CW+2vu88rAS+hM9pSCfT+gfGiaYRiG0Z4P0bxEVWM4xaNf\noHP3S71k2/PTFiAiftoCRMBPW4CI+GkLEBE/bQEi4qctQET8tAWISEW6M2qc+i3ogoa10HjSIwu2\n14viNgzDaGiiul8OKbO9OxQuMAzDyDy2ojTP6LQFiMjotAWIwOi0BYjI6LQFiMjotAWIyOi0BYjI\n6LQFqHfMNWMYhtF1KtKdlUa/GNklXGjCMIzsM5M6W+djlnptsf42jPoiSvrtyJhP3TAMo4EwpW4Y\nhtFAmFI3DMNoIEypG4ZhNBCm1A3DMBoIC2k0DKMz1kSLpG8E3I+WnDM60q36yULsaov1t5EkP0ez\nrw6gY81SI0+cfko0pLEWZFawBsX626gG66EFo43OqaSfTKkbndJI/b1ame0roAVbqsWawAFoOcDN\nqnicJCjXV1HorD/P7GRbLUniPJOgVF9V0k+m1I1OaZT+Xp3y2UF7UN3c2PXieojSV1Eo1Z/7on2w\nZgLHiENS55kExfqq0n4ypW50SqP090UR99sSOKyagpB910PUvopCYX8eALwI/Ae1QuOwP1oG8zRg\nVIl9PgTmA5Pp+HdN8jxzbAJcWmafUnKH++rbVN5PptSNTqmH/t4MuA/4L/AD4Fjgr8A33PaNgZ92\nob0bE5WuI10dUh+X4LHLtdXVvopCNfpzEFryMsezwDJF9jsaWJWOkXnVOM+TgLuA6zrZp5zcSfSV\n5X4x6p6XgdnA74FrgCuBh4HL3fZ9gCe60N5UqlfLcV9UrpW68JtiyqpSyrW1N13rqyhUoz93At4O\nfX4NaC2y3wJgPLCo4PtqnOfvgXvL7FNO7mpeexVhcepGWmyLWmUAvYBD0ZsMYAu65u54DdgcLXKe\nY/VQ+8V4jvI39AHosPunaGGFLLpgtgTOT7jNYv0Zl5WBWaHPsyjue94S6A0MBN5HR3S575M+TwCv\nzPZyclejr2JhSr1bIQm6ZrxyN0NnrAt8AeyIRjNsiQ6Fx7vtfWk/9NwXWOz2fwPYA1Ww77rtM9H6\nuWE+Ak6PKE+p9u92ryQpdy5dpbCvkpCjWH/GZTAwL/R5AdC/yH6Pk+/zV1EX3RcUP88k+rJc35WT\nuxp9FQtT6kYa7Ixayf92n/dFQ8RySr05tO+q6PD3A+Bc4EL0Jh8f2mcuau1XQpT2y7Eu7Sf1dgBa\nQp+fAh6KeKyobeUI9xXAL4E+JeS8Afg4ghyV9Ge5484Glg591wedDC0kPHqaCYx03xWeZ7lziNIP\nUN5SLyd3nGuvKphS71bEsq6TZCTw99DnIajF/rz7HPan5m7S5dAbbBbwQEF7g9CKT2Giul+itF+O\nd2g/KjgbOKfIflGOFbWtHIW+54vLCRtBjmL9WY5yx/0QdavlWIaOS+kPRR/w33Wf+6GWOHQ8z3Ln\nEKUfoLylXk7uSvqqqphSN2qNh04+hRXuhuiNsQLwOTAJHeJ+BayD+lg3A550++9N+xt4BVQZhonq\nfonSflJU41jhvkpKjmL9mWM/ys9FFONJ2ivazYBT3fsR6N/rY+Aq911fYFnyk6OF55lUXxYzdHLy\nSBm5oTp9FQuLfjFqyUboZFcfNK43x7XANsBu7vN/0UU/uO/2Rm++FnTyckpBu5sAYyqUKUr7SVGN\nY4X7Kik5OuvPYn7wKHyNKsezgF+797lj3u6O+TSqJE9EfeMHA3PcPoXnmURfHg8chY4cz0YnZ8Py\nlJMbqtNXmUckYxMJDU49xKmXYzDwm4j7tpCPmskKv6xhW13pqyiU68/vJ3isrpD0eSZBUn2VaJx6\nrdwvZwBH1OhYRv0zC5iG+i+nldn3YHThUpaI6s9Noq2u9FUUivXnWsCm7v02qH9bgDvJ+7yrTdLn\nmQRZ7auqIwLTRCeujOrTCJY66LC6s4lOgFVQv2V3J0pfRSFKf6ZlqUNy55kESfZVKmkCrkXDeN4I\nfXcJOkHwGrrUdlApwQTOFbi6mgIaS2gUpW5kk++lLUAdEbWvUlHqO6JDirBS35X8ROuF7lUMERgi\nMF1gWBVlNBRT6oZRX6SS++UpdCFAmP8Abe798+hy2qJ4Gq52Ne1DgQzDMIwUGU57Sz3M/cD/ldgm\n7p+hAjOka4mRjK5jlrph1BeZy9J4JpoPodMiAp7Gdl5HsuFehmEYRoi4IY1HAN8Cdimznw+wHMhN\ncKTABZ6uEDMMwzCUke5VM4bT3v2yB/AW5fM9S8GHy0UjZ4zqYO4Xw6gvUol+uQWYiLpZJqBLa8cC\nnwCvuNeVUQQTWNn51petnrjdGlPqhlFf1H85O4GrpDoJ740MXwiGYRSlIZT6cBe3PiQNgRqczF4I\nhmEUJXPRL13G0xSb9wA/S+P4hmEYRuUUfdoIrOFywpRKL2BURne01Fer4bFWQHN9G0ZS1L+lDuBp\nGaqH0KK+hlEpq6PZ8GrFVGythdHNKfm0EVhbYKpZ64nS3Sz1iyLud0ro/f5oOujTgFGd/OZDYD6a\nzC5cN3TLgs+GEYf6nygt2HiDaNURIxkyeyGE2A4tU/Yw8APgWOAvwE+62M7GRB/p5a6xQcBLoe+f\npfRai6PRAsfFFundGPG4hlGOhlPqI5xv3SJhkiGzF0IB/wZ2L/huNNDahTbOBNaPuG9Oqe8D/CP0\n/VXAQSV+c3gn7f0OWCPisQ2jMxrDp57D0yHuPcDJacti1IxewNZo9s8wz6GukahsCbzdxWOvjFbR\nyTELWLOT9o8BfoFWuQ/zGrB5F49tGFWnVuXsynEe8LLAZZ5ORBnVwk/IkveLVmGPytaoMp5T8P2K\ndO3v35f21sy+aMmwHdGUFnugdS3fC+2zFDAv9HkBpQsEPw7c7d6/ilaWzz0QZmK1d40Mkgml7sEn\nAv9CJ7MssqCaxFPGSbEz6lMP04QmMzquC+00h96vij4oPgDORYu2fIGmtQjzJe1dfX3QidBi3Bt6\nP9PJd4/7PBcdcRhGpkjd/RLifOCHAsunLYhRdUYCQcF3e6Dpme9H8wI9BBwPfBe4BhhQpJ1Foffj\nUYW+HDAbtagfoONo4EPaT4wuA3xWpO1DgVtDn/sVHG8QWvzFMLodkYf7An8UuKyawnQDsj5R2ge1\noFtC362GWu7hRUS5yUwPrbJVrLLWDeRdJ+ug0TBHopY6wN6hfXMTpf1on230NWCoez/CHQ9gB3RE\nAermGUf7RUfHUz7ltGFEIdGJ0ky4X0JcALwtcKkHn6YtjJE4W6Jx4bPRSuseavEuDRxI3vLtjbo2\nWtEImaMofj38F9gKfSDshlrzn6MPjAMoboF/DVwMnIWOVC9GRwgAt6Mhlq8ATzsZT0Rr6x5Me6t/\nE+Dv0U/dMBqHLj1tBC4R+HO1hOkGZN1Sj8IOwA/d+6uBgSX2G4xOhEYhybUQLcDvE2zP6N40Vkhj\nES4GDha1jozuyU7Ay+79CNpPiIaZBUyjfKGWpDkY+GuNj2kYkcicUnchjX9Fh8dG92ML4NvkFfXr\nqG+7VPjgH1FXS61YBY2Eea/cjobRqHR5CCEwxK0yHVENgRqcRnC/VIO10xbAMErQWGkCOvmRL3B9\nwrJ0BzJ7IRiGUZRuo9QHuQyOZmF1jcxeCIZhFKV7KHX3wzMFbk5SmG5AZi8EwzCK0q2U+gCByQIb\nJilQg5PZC8EwjKJ0H6Xufnyi6NJxIxqZvRAMwyhKt1PqLaIJv3ZISqAGJ7MXgmEYReleSt01cLjA\n00ImMgxmncxeCIZhFKVbKvVmgTelfYImoziZvRAaCJ/21ZPqndFozhsjHRo+TUAHPC18cAZwgZRe\nMm7UD2uihSoqVYx+jN8mQT0/OH069p1Q3+dkhIiq1K9FCwmEU5YOQVOivg88iiZXqib3owUOvl/l\n4xjV58/AC1RPkXhU11VnbkAjs0RV6tehRQzCnIYq9bXQsl+nlfy1z+qVCBfGUwVwGnCuaGpWoz45\nGM2d8jjlleOpaMrdL4F30RwwewCnA99DU/i+4vYdjWZsHIOm110N2A54EU389QKwbajt0Wje9add\n+/9GUwDnOAz4BE0YdhbwMfn86pB/ID2I5lYP8zqwX5HzaQH+6dqc6WRaNiTPeU7+2cB9aP6bm9D8\n8y/QPsldZ+e2ovv9dGAs+YyXpfoOYDil+8JoUIbT3lJ/F60yA1qt6N0SvxN8bklKCIEHBH6WVHsN\nSJaH0QPRRFgrUt6FsjZazShXCWtVWGIcnA3cWLD/aFTxrosaK8uhivP77vPBaL72pUL7jwXWQJVt\ngObzB1gPVXrbAT2BS9BapjmlHpb9ILRgdo6NUaVdrFbBj1Bl24I+0DYlX9FpNDrqXQ3tp7ecfDuj\nLscb0BEz6Ci5s3N7ErgCzUm/MZovvtVtK9V3H5ToC6P6ZManvhz52o6TySv4YuyEz5YxjhXmDOAM\nKV7ezMg256GFJSZS/oJdjI7I1kcV63jgI7etmHtF0FxB7wBtaNGM91BLtw0tTfcuWpw6t/91qDKb\nB9yGFr4ALdhxH/AMsBD4dSfy3o+OVnPJ50a5Yy0qsu8C1AJe07X3CvrwCMszDrWWH0aV/BOuL25H\nHwIAe3VybqugD6NT3fFeQ/v8MPfbUn13bYm+MOqMpCZKy020nANcnETRY0+Hto8CJ8dtqzsiIEm8\nKjj0Jmj5t1y5wsJr4WFUwc0GDkEVzImoVTwZuAVYocwxwkWmV0QfBGE+cd/nmBR6P5d8abwVaV9p\naS7qyihGTgmOQs/pYEqPQP6BujZuRasyXUR7iz5cAHse+YpMuc9h+Uqd2wqo1f51aNt4YKUSMuUo\n1RdGnRGnnN1kdGg8Cb2QppTc8xxWZls2YDz/RCvZjI5xXFDL6X8CV3qdHdfogJfeJN83UBdeThn1\nR90K66I51Pcs8ptb3GsAmmP/ItTijDJc/QzNyx5mGPrwKMdE2ieS60PnPuYbUJfGGLTk3fMl9luE\n+vHPdbI8hFrc1xbZt7MHZ2fnNhF1z/QHvnLbViX/kMqye667M9K9YhHHUr8PONy9Pxy4p+Sewq/Z\njR/wQzbC56kYxwTA0yHqTcCZcdsyasbfUJ/4xqjVfhU6ybh7if3XQv3JvYH5qKW62G2bhD4gCh9Q\n4c8PuTYOQY2X76HFqR8osX+YO4F90MnHXuhoobOH4bOosryUjv7qMCPRPEbN6IhkIflzKpSns+M9\nTOlz+xR1G12A9t1GaI3Xf7rfRuk7Ix1Go9da7lURUZX6LeiFsjY6xD0SuBDYFfX77ew+d8b96Ez9\nYWX2i8pvgUNFL1Aj+8xFR1VT0FHeV3Tu1uiNKqapaDHpZdDIDVD/Mu63/wv9JmyFzkAXq52MTlz+\nwn2eUWL/sAvxLeCnqJtkIqqAp6APl8J9c9yIKux/UprlnexfAG+jN3HYVVNKnsLt08uc2yHofTER\nuAsd2T7htkXpO4tbNzolf3H4bIPPp/j0Tajhc6Rzy6g7Yjdj8vRHrephnewzCo06MYyuUudpAnzu\nwF9iccVteKBLzbtREu01CJm9EOqMfYC+QD/UVfRSJ/v2RcMaD62BXEbjUfdKfS18puEnUwFe4GcS\nbfKru5DZC6HOuBqNBZ+FLrJbs8R+u6OupLupk7QbRuaoc6UO4HMF/pLQtriN9xIYKxqXbGT4QjAM\noygNodSH4jM9ifQB7gDfFnjdkn0BGb4QDMMoSmZWlFaOzxR0EcpvE2rxbnSYfGRC7RmGYRglKP60\n8emHz8Sk0gcIbCkwUWwlnFnqhlFfNIClDuDzNRpgf2lC6QNeRBMRnRK3LcMwjHol7dn6a9HMcgck\n1N4ZwPFSPs+FYRiGUSGdDyF8dsHnQ/xkcqSLVke6Lom26hRzv1Qfn8YqZ2ekS4O4X3L4PA68iWbk\nS4ILgD3FUodmkePRpenziPfg9bFydoZRlPSVuvIL4BT8JQURKsbTXNTnAL8TS1KUNT5Dc6oXy0qY\nJFbOzui2ZEOp+4xFCxycl1CLV6O5pb+VUHtGMtwN3EvpJF6FNFo5O4Ad0OR4M9E0xLlMp4PQPEZT\n3LHOJP/wOMLJeQmatOsj8uUlv+fOMczP0X4GTYx2qTuXScBf0OpGOdkvDf3uVuCaEnIbxhKiDVV9\nBuMzCX9JdZe4B91b4G2JlzO+HqkH18BvKO9+acRydsPQB8j30IVyQ9z+uPO5G801MwzNs36U23aE\nO/4PUEX/Y3TUA5p35kt3HjleBL7r3v8BTYs9GA33vQ84323LVS9rdf30gTu+UVsaYEVpKXx+jM/o\nJEIcBTyBxwV+EretOiOzF0KI8yiv1NdAFc4uqGIN49PRpx7QPgf1KNorW1AL+fDQ/meEtv2EfA6h\nX6P5+nP0QdPuFlPqLejDIlfO7lK0PmgxTkdztRfS7NpfJ/TdMU5GUKU+NrStL1rGbqj7/A/gV+79\nmqiSz9VB/QrardzelnxZQNBiGxPQFMfblZDbqC6JKvWsWbF/B45FQxzvitOQp2XXTgYeEbjJ+dqN\nIEhG6be2xnnwFvvtw6hrAlSh3UK+nN36qHvkJDS3einSLmd3Djoi+E6JfVemvULNsQz64Pok9F1h\nCbqwrHPc//1Rd83NwO/Qh+X/oRb/PFTp96V9hkmP9m7XB9CH0LvoQ8+oc7Kl1H0W4fNz4G/4PIi/\npChBRXjwqsvgeDokk+637omnjJOi2IOlO5SzmwBsVeT7aWi+9uFo4WxoX4KuHI8By6KunIPJR5JN\nQx9I61H6YfhbtGDHcPfbWyMe08go2ZgoDZN8iONZwNGSHx4b6dGMugV6uPe9KZ2ErRHL2d0EfBP1\nw/dAHxQbo+d1G6pg+6MPn5/TeRWlMAvRikaXonMG/3Hft6FBA5ehSh/U+s9lNN0Jde2Mcv//ifYj\nGcMoSteH+z5rupzrsUMcnQCnS2c1VBuLLPvUfVTRhF+/LrHvhqjF+yXq+riP/KTpEOAp1JedK8kW\nkJ9YzLG92z4LnTwM+4wL9z+c9pWLDqd99Munrj0oPlF7ljuf4SXOJ8cOqK//C9TFMsp9Pxj1jU9x\n359F/kFSKBvogyDsK9/BHf9PBfv1Rh8WH5Ivo3c8OvoZR35CFbQk5SNl5DeSp4EnSsP4XIrP1QkJ\n0CLwQTfJuZ7ZC6GOsXJ2RjXpNko96RDHfQTekY6RFI1GZi+EOsPK2Rm1opsodahGiOPDor7KRiaz\nF0KdYeXsjFrRrZR6D3xew2/n94sjyDoCU0UXXTQqmb0QDMMoSqJKvRbhbRLrOD47onG46+LzVQLC\nXAos5enqvEYkXn8bhlFrSt2zmb2X41uOPjfic1ECsiAw0FVIKhYv3AiYpW4Y9UU3cr/k8Fken6n4\nrJuAPAgcIfCcNKYPNLMXgmEYRWmwfOpR8JmEJoG6IolJUzTG2CMfI2wYhtEQJKHUTwfeAt5Afd+J\nVDAqwp/RVXEHxW3I00UaP0WrJA2M217GmIk+4e1lL3vVx2smGWI4mqAop8j/RT4LXg5J7Gg+O+Iz\nAX9J4qVYCFwnmlbVMAwja1SkO+Na6l+iK+36orks+pLP85w8Pk+hy7t/VW7XiJwOHCntkzcZhmF0\na45BCwpMoXjdyOQsdajGpOnJoouSMhk6ZBhGt6Ui3RlXkY0A7gd2RJMF3Q7cQfsCA4Lmmc4x2r0q\nx+dn6DLuXfHjPTREs/C9DpzmdZ+kX4ZhZI+R7pXjbFIwNr+HFrbIMQqd0AyTrKUOuZWmrye40nRn\ngU+EZHz1hmEYCZCKT/1dYBu0iICH5op+O2ab5fFZBBwH/C6JSVMPnkBTuZZKA2sYhtFt+CX5kMYb\n6JgFMXlLPUeyK02XE80Ls2ES7RmGYcSkerozJtVU6klPmv5Y4Gmpl0VZhmE0Mqm4X9JFV5qeC1yF\nn8i5/A0daRyZQFuGYRg1p76VunIl6tOPrYjdStMfA+eLVng3DMOoK7KfejcKPhujhQw2xGdy3OZE\nC/UO9DrWvDQMw6gV1dedFVIbZ7/PhfjcnERToul5PxWNvzcMw0iDbuhTb8+5wNb47Bm3IU/TH/wc\n+Is0fk1TwzCMLlG7sByf3fAZh0+/uE1JvqbpL5MQzTAMo4t0w5DGYvj8Az+ZzIsCIwSmi2ajNAzD\nqCXd3v0kH+MeAAAgAElEQVSS4yTgMHw2i9uQBx8CfwAujy2VYRhGg1D7IYTPEfj8D58ecZsS6C3w\nrsD+SYhmGIYREbPUQ9yATnYeH7chD+ajsetXCAyK255hGEY1aYw49WL4rAk8C2yOzydxmxP4K4AH\nP4rblmEYRgS6eZx6MXzOwOeBJIpVCwwSmCDQmoRohmEYZTD3SxEuRSNXkihW/QVwLHC1aNk+wzCM\nbkm6sZY+2+IzEZ+lk2hO4BYrVm0YRg2wOPWS+Pw+wRQCQwUmC2yZRHuGYRglMPdLJ5wFbIHPAXEb\n8rTA9knANa6+qWEYRrcifUsdwGf7pNwwLoXAAwK/SkI0wzCMImRDdxYhO4Il64ZZxZW/Wy+J9gzD\nMAow90sEknTDTEAt9WsEmmNLZhiGUSdkx1KHpN0wTQL/FfhZEqIZhmGEyJbuDJE9wZJ1w6wlME1g\ntSTaMwzDcJj7pQsk6YZ5H7gYXZSUySW9hmEYSZI9Sx2SdsP0EHhedMWpYRhGEmRTd5JhwRJ2w6zj\n3DBrJtGeYRjdHnO/VECSbph30TqpN1g0jGEYjUx2LXUIu2GWiduUi4Z5QuC0JEQzDKNbk5ruHAzc\nAbwDvA1sU7A920odwOcSfO5OKEXvqm5R0sZJiGYYRrclNffLH4GHgHWBjVDlXm+chYYkHhW3IQ/G\nA6cANwr0jtueYRhGLRkEfFRmn+xb6gA+6+Mz1VVMioXLDXOvwAVJiGYYRrckFd25CfA8cB3wMnA1\nHQtI1IdSB/A5AZ/n8ekZtymB5QQmCWyXhGiGYXQ7UnG/9AA2A650/39N0UnCDa4GfPcaGfOY1eRP\nwEzUHRMLDyajces3CvSP255hGA3PSPJ60k9LiOWBcaHPOwAPFOwjIGNB6kOx+ayAzyT8ZCxsUaV+\nZRJtGYbRrUjNy/EksJZ77wMXFWwXEAH5W02lioPP/vh8hM/AuE0JDBYYL7B7EqIZhtFtSE2pbwy8\nCLwG3IVOnoYRkHlOse9fc+kqxedqfK5LoimBXQUmCCyVRHuGYXQLMjsfKSAnOqU+DWSFtAWKhE9/\nfMbic2ASzQlcLnCHJf0yDCMimVbqTSCPOsX+MEh9KDafrfGZjM/KcZsSaBF4TeDoJEQzDKPhybJS\nB5AVQaY7xf7TdEXqAj6/wucx/PgLtQTWdUm/rASeYRjlyHpCL28ieSv1YpD1a3fsWFwA9AFOituQ\np6ttTwNuFWiJ255hGI2KrJ22BJ1R8LSRa5y1/ipIfSyj9xnu3DCFeW26jFttepvAFUmIZhhGoyGH\nanBJ5t0vSz72B/nAKfZL0hGpAnz2w+djfIbEbcqFOX4ssF8SohmG0QhIM8hFTjcK9aPUAWQbkEUg\nbSA7116kCtGiGvcllM1xO4HJQvxJWMMw6h0ZCPKAU+iLQI6lvpQ6gJztTuBTkNgl5WqCTy+XGya2\nfx1A4EyB0VZUwzC6M7IGyNtOH04PGbp1p9R7gDzjTuT+OgpzTNK/3iwQSAK5ZgzDqEdkZ5AZTg++\nBTIivDE1scrQiWAyDGSmO6GTaydSTJL1r6/ksjlun4RohmHUA+KBHOdcLTnDtjAtST0qdQDZ153U\nQpBtayNSAqh//d6E/Ov7uolTSyNgGA2P9AK5Kj8hKhfqJGnHHWsuWkQiCCa/cyc3HiS29VsT8v71\nnyfRnMCfBO6yNAKG0cjIUJD/On03D+T7ne1cM7G6SBSl3gvkuTr2r28dtymXRuBFgfpxQxmG0QVk\nC5AJTs9NBNmq3A9qIlYFRBTM/OsCw5x/fackRDMMIyvI4eSz1Y4hWmLDelfqUOf+9fsSyg+zh8Bn\nAvWRzdIwjE6QniB/DPnPr1LPRLQfV1W0GHRRsLr1rz+Dz6+SaE7gbIEnhfi1Ug3DSAtZFmS002cL\nQI7pagPVkCoJuqrU69W/vgI+n+Kzd9ymBJoEHhaonzQKhmGEkM1APnF67HOQSspjNopShzr2r2+L\nzxT8JeX9KkZgaRfm+O0kRDMMo1bIoSBznf56FmTFShtKVKwEqVCwdv71RIpA1wSfo/F5O6H6plsK\nTBHiPyQMw6g20gvk8pD//G/Ey0TbaEodQv71iRFni7OBz1/xuTuhidNjBN4Q6JeEaIZhVANZ0UW1\n5PznP0qi0QTaqApxlHrPUKD+mC7MGqeLT++kJk5d/vXrBW6yhUmGkUXkGyCTnJ6aABJ73Uqu4YTa\nSZyYgsnQUMD+VcmIVAOSnTjtK1rf9NgkRDMMIwnEAzmJfP6WxzXiJYGWYTCNq9QBZMtQ4H79FG7O\nT5zGLk0lsIbzr++QhGiGYcRBBoDcFvKfXwDSI5GW9V7/nMZW6gByRMhfVU8Lk47G552EJk73EPhc\nYHgCkhmGURGyDvn851+CHJBYyzBI4G2Bn9D4Sh1A/lTHE6f3JDRxeqJzxfRPQjTDMLqCHOgUuYC8\nCZJYZJpofYUHBf6c/yqbJKnU63nidAw+v4nblJs4/bvA3UL8h4RhGFHoEK54K0iihpXAJQKPh1aS\ndwelDnU8cToUn4/wOSJuUwK9XBqB2A8JwzDKIcNAng+5f48n4ZXuAkcIjBXaJQZMVak3A68A9xfZ\nVgXB6nbidF2Xqndk3KYElhUYJ/B/CUhmGEZRZC/y5eY+pny63K4fAbZ3QRDrdtyUHicBNwH3FdlW\nJcHqduJ0Z6fY14nblMCGAlMFEr/QDKN7Iz1cREvO3fIAVUgwKJpue6LAnsU3p8PKwGNAKzWz1Jc0\nnZs4/Rxk1eodJ2F8jsLnA3xix7SKlsL7VGClJEQzDENWIJ9dcTHIaSCJz18J9Bd4VShZPS01pX47\nsCnwDWqv1HuCBK7zX6dj4dbs4nO+mzxtiduUwGmiVZP6JiGaYXRfZGeQyaEou29U5SiaifUugWs6\nWSleke6M+/TZG5iC+tM7mzjwQ6+RMY8ZwlsIfAd4D9gQ+FdSCwBqwFnAZ8B1CYQ6XoT2QWcXiGEY\nJZEeIOeiXoehwBPApuD9t0oH9IFlgWO9vPIeSXtdmQrnAxOAcegKqK+BGwv2qcEQQkaATHVP1yuT\nnpmuGj598HkWn/PiNiVa4/Q5SfFiMIz6RFYGedLpjzZV7tJctaPBoS7IYWj5XdMlBfdLu8NsH4qI\nKeWjyh75UMfD4zYlMFTgA4EfJiGaYTQ+shfItNDc3M5VPRrsKjBZYL1ou6fLN6hp9Esx5ODQ03a/\n2h03JvlQx9a4TQms6VIJ7JWEaIbRmEgv8qm9BeQRXQNTxSPCJi50ccfoP8kmNRZMznJ/pK9BNq/t\nsWOgoY5T8NkkblMCW7uLx0IdDaMDsjrIC05PLAL5ZTWiW9odEYa7KLUDu/azbFJrpe6BXB+avV6l\ntsePgc+B+EzEZ824TQns7Sz2NZIQzTAaAzkI5AunHz6pxRoXgSEC7wic0PWfZpMUBJNedRzqeLTz\nsceOOxc42vnYqzqsNIzsI/3Q8nI5d8tdIEtV/ajQR+BpgYsr+3k2SUkwGQLyrvsDPlxHoY7gcxo+\nb+ITewWbwDkuht2yOhrdFNk0pAvmUYXcLUWPqlkX7xS4WSoLHzelXuTQ4VDH6+oo1NHD51IX7hir\nNqloVsdrRFN61s+DzTBiI00gJ6OpRARNlbthTY6s992fRLMuVlp82pR6icNv4yZNBeTSOlPs1+Hz\nb3xipRgW6CnwkGjK3vo4f8OIhawA8mjI3XIFSJ+aHR1+KVr3YFC8ZrJJBgST3UNP69PSliYyPj1c\ncY1b8Ym1GMLlmXhR4NykxDOMbCJ7h0bo00D2renRdXHRJxI/H1MGdGdxMiKYfM/Frwv1la63BZ/R\n+PwFP56V7RYnvSvwi6TEM4zsIC20L2TxH5AVayoBfNtFnUVZXBShuWySIcHkJ+Qzr3UlXjRdfAbi\n81JC6QRWFvhI4NgkRDOMbCCbgLxFPh33L6ode95BAviWWy26aXJNZpOMCbZkcdICkF3TliYymk7g\nXXzOituUwGoC44X4VZgMI12kGeTUkHv1PVJYdCiwi1vwt02yzWaTjAkmHshl7gL4iipUMqkaPisk\nqNjXEvhM4OAkRDOM2iPDySfiyk2G1jz9tGjloqmiqVISbjqbCEGwO0GQoagLaQK50V0I00GS8H/V\nhmQV+wYCkwT2T0I0w6gN4qGVz74kn4hrj1QkgS2chb5bdZrPJkIQvEMQPE0QxE5YlRzSE+R+d1F8\nCjIsbYkik6xi39xdlKncFIbRNWQZkDtD1vmd+l0KksBGzijap3qHyCZCEDQTBIcSBGMJgicIgh3S\nFkqRPqHh24fUV0m8JBX7dk6xj0xAMsOoErKns8rFWemHp7XuRGAd0dqi363uYbJJXrAg6EEQHEkQ\njCMI/k0QbJ2iXA4ZDPKiKXZanWKvnyLeRjdBBtA+b8uT6k9PSRoYITBBiF8DofyhsklHwYKgF0Hw\nI4JgPEHwIEGQ5IxxBZhiBxDY04VkZeBhaxgA0grysbs356NpcqtWlaisNBo5Nk7gx7U5XDYpLVgQ\n9CYIfkIQfEIQPEoQRE0eXwVMsQMI7OUs9p2SEM0wKkP60X4h0UsgG6QqkbpcJtRwjUcdKvUcarn/\nkCD4iCAICIKd04mWqXvF/g4+5yaw8nQXF6JVjRl9wyiDbAcy1t2HC0F+rYENKUqkk6ITa7y2o46V\neo4g6EkQHE4QvE8QjEknFLKuFftQfF7G58oEcsXs4Cz2mubNMLoz0gJyMfl0Hm+AJLU6s3KpNGxx\nksD3an/obNJ1wTRa5hCC4C2C4AWCYH+CoIZLfutasQ/EJ8DnX/gVp/wEUr2YjW6HbEV+mf9ikPNB\nYl2/iUilC4vSMm4aSKnnCIImguA7BMH/XKz7EQRBrDS00alrxd6Cz134/Ac/XnEMgQ1TGHYa3Qbp\n46zzxe5eexckExP1GXBDNqBSzxEEHkGwC0HwHxcxcyJBUINKPu0U+0cgI6p/zITwacbnanxewCfW\n4gyBtWs8QWR0C2QHkPdD1vnF1DDneWdkJGCggZV6mCDYgiC4nSCYQhD4BMHSibbfARlMvvL4JJBN\nqnu8BNFCG+e7yJhYIw0XyvWRWNpeIzbS30W25Hznb5KhHEwCB2YktLebKPUcQbAWQXA1QTCDILic\nIFi9KscB0MUPj7kL8AuQ+gr38/k5PuPxWTdOM6Jpe98T+I1YBSWjImQXkHGhyJZzs+A7zyFwlGg+\n9CwYb91MqecIghUJggsIgmkEwZ0EwXbVOZD0BrmdfPHa/apznCrhMwqfSfjxrA/RQhvPC9woxCuz\nZ3QnZBDtV4W+nKVRr2hN0XPcaHTttOVxdFOlniMI+hMExxMEHxIEzxIEBxEECRdalmaQv4R8gEcm\n236V8dkLn6n48fJVCPQVuEe0qG6cGoxGt0AOAPmM/KrQM9KOOw8j0EvgBoEXBJZLW54QqSj1VYAA\neAt4EzihyD61FUzDIQ9wWSHHuUnVAckdQDyQc0IWxynJtV0DfDbF5xN8zo6zSEmgWbRa+hui14Fh\nFCArgdwduleeJWNprgUGCTwmcJ9Av7TlKSAVpb48ed9Tf+A96OC3TW8IEQRbEwT/IgimEwSXEQRr\nJNe4/DR0sV5CStniKsJneXyecwWtK442cEPWkwU+zYgP0sgE0oSWjvyCfEbF46hxeblyCKzijJIr\nhHiL9apEJtwv9wC7FHyXvmBBMIwguJAgmOoSiO2ezGImOcRN9gjIdSAJu3uqiMay34TPi/jEKs4r\ncJAL/9o9KfGMekXWAxkTMnjuBVk5bakKEdjYhemenOFJ/9R153DgE+iw2CV1wZYQBH0IgqMIglcJ\ngvecDz6ma0Z2B/mafPXypZIRtgZoyOMZ+EzAJ1ZdR5dWYJLAD5ISz6gnpLdzS+ZqhX4O8p0sjmAF\ndnNGSDVzoSdBqrqzP/A/ipdFy45Sz6GLmXYkCG5zrpnLCYIY4X6yNchk8oVv10pO2Brg8203gXpQ\nnGbcIqUPBc4TyNRQ26gmsgu6EjRnnV8FMjhtqYoh8ANnfGSkUE+nVKQ7k3iK9gQeAB4GLiuyXYBz\nQp9Hu1c2CIKV0dzIP0DnBK4C7qa1dX7XGpJhwH3ARsAs4EDwHk9U1mrisynqPrsWOA+ftkqaERgK\n3AVMBw7z4IvkhDSyhawE/J68xfsucAx4T6UnU3Fc+O1lqHt4X0/v9awxkvbVx84mBdeQB9wI/KGT\nfbJnqRdD0/8eSBA8RhBMdj74Li5okv4g9zhrZRFIfS2r1wnUZ/B5AJ+KV+q6ELEr3EKlTEU7GEkg\nPUFOBpntrvU5IKdnaRFRGIEVBcYI3FtnIbip6M4dgDbgVeAV9yosYFwfSj2Mrla91E2sPuKyREac\nBJUmkAtCQ9Er6mwCtRc+l7qwx1il7QQOF02IdGBS4hlpI99Al/Xnru+7yHDRdtEsi58KnFWHLsHM\n6s7MClaWIGhxBbPHEAQT3crViP5yGeUWWtTfBCqAzz74TMbnFPzKbwaBzQU+Frgwo2FjRiRkeZB/\nhJT5ByB7pi1VKVy47XGiOVy+lbY8XcZnKCn61MshNTpOdQmC9YCjgFGoP+4a4A5aW78u/SPZDrgb\n9TO/D+wLXhZ9ecXxGQb8C5gGHI7P9EqaEVgWuEXfcrBHZe0YaSC9gZ8CvwIGAvOBC4CLwJuXpmSl\nEOgD/AXYDPi2Bx+kLFI01HjaBZ3j2xmfwVSgO02pdxXN574XOrG6HXAHquBfoLW1yJO13QTqbOBo\n8P5VK3Fj49MLOB84CDgYn2craUagR6idb3vqqjMyi3jAd4CLgdXclw8BPwXvo9TEKoPAMHSi/n3g\nhx50YnRlBJ9l0XoFPwK+Qh9IN+PzJabUa0wQrAQchlrwC9BJ45tobf20/Y7SH40qyYUM/gU4KauW\nTlF89gH+DlwK/C5GdMx3gT8DZwF/8+rZPdewyFZoVMv27ot3gJOBR9SrkU0E9gGuRh9Ef8j0taUp\nOnZArfK90BH9VcAL+Evkrkh3mlJPAq2juiPqmvkO8DLwD+AuWltn607iAT9BI4V6oZPL3wVvbAoS\nV4a6Y24FvgSOxmd8Jc0IrAPcDExArampyQlpVI6sio6mvu++mAb8GrgavEWpiVUGgb7A79AgjVEe\nPJ2ySKXRgjWjgB+iE7dXATfiM7PI3qbUM0EQ9EEthlGoon8AteAfp7V1MchmwO3A6tSnO6Yn8Evg\nROB04JqQZREZFzd8HnAo8AMPHklUTqMLyADgVNQab0FHnZcB54OX6XUGApsDNwEvAsdncl2EFoHf\nDR3R74q6Y68Bnixz75hSzxxBsCxwMOqiWQm1cm9l7+3f4+ueV1Pf7pgNgetRay6O1d4K3IAufDrV\ng7mJyWiUQVrQ0ePp6GQ26MT46eCNS02sCLhIqlOAk4ATPL23soXP6sCRqL98EqrIb8GP/OAxpZ5p\ngmAdVMEfAvRE+Be/XbeNx4f+Arx6dcckZbUvhQ5D1we+78FricppFCA9UWXzKyCXbOs54GTwnklN\nrIgIrIqOfj3U3VKRQVEVfPoBB6BW+YboKOJafF6voDVT6nWB+t83RhX8wcxvEu5caSCPrDCECX2/\nQmuA/i3LE1Id8NkAtdqnU6HV7jLljUJ9oxcAl3lUNhlrlEKa0evuHCBXRP114EzgwXq45kTlvxyd\nyL3Eg8Upi5Rzr7SiI/J9gTHo6PNefLqYbqQdptTrDlXwWzO36TAWNh3J1N4tPLksvDjkWT7odwgL\ne3yStoiRaW+1nwn8vZIIGdHwuX+ift0fZzRHR50hHrAfOoexgfvyfXQS9HbwMv/wFK3dcBmwKfB/\nHryUskg5Y2YUOrE8GQ2OuAWfyQkdwZR6XfOHl5t5aamzWHrBaWw7vYUFTW1Mbvk3m8w6ix7ySvEY\n+AyiF/rVqM/zBHye62oTLqb9ONQ98CfgQo9YFk83RZrQSfuzgC3cl+NRS/3GLEe05HBL+48BzkVD\nan/jwZzUBPJZCQ3LHYXOQ9wE/AOft6pwNFPqjYEMxZMrWXv2d9hpKnxz8lyWWjiNHnI7cCfwvEbR\nZBhdGfd94ELgceA0fCZ2tRnRMnmXo9W0fuxlKbtnppEeqOI5nbxlPhn4DRqeWBcPSFGf9F/1LT/y\ntGRm7dHFQd9BXT8bAfeio8nR+FVw/+gIfgNaW1/HlHqjIB5LFunI0oz46muOGRew5YzheAxFV/bd\nB/yH1tavUhW1M3wGAGcARwOXAJdV4mMUzdN/OfqAOMXTiBujA9IL9eueRt5n/hna938HL/urK1kS\nd/4rdNX2r4Craz6/okv0D0AV+dZoavFbgUdi+smLEwS9gW+gPvl9gDZaW4djSr3RkKHAlaiVAPAY\nR310LqPGb4L+8bdGJ2XuB+6ntXVCOnKWwWcNdGJrXeDnwINdjZIRGIAOwf8P9d3fmOkVgzVF+qIP\nzl+Qj2b5EB0p/aNeLHMA0QVEVwLPAz/3NBSwNvgMAvZGDaqRqBFxK3q9Jv9ADIJl0GRj+wLfBN4m\ndy/DW7S2tmFKvREJW+0sjfqWNUIkGN2E1gXdF9gTXaH5ILqQ5zlaW7PlM/XZA53sGgecWkmYl1ts\n8jd04dZJnq7e7abIcugy8+PIx5m/ha4Kva0efOY5RBfjXYj6/o+t2WI0da3sixpOO6AuvjuBe7oQ\nTx4NrYu8Mfrg2gt1Lz2OKvEHaW2dUvAL86k3NrIcmtPiMPfFRNRivRk8cfnet0WV+x5oFMnj6M3x\n78xY8Zog7Fh0BeMY4NyuKnc3kXoMOjR/AjjL0wdFN0E2AX6Gjlp6uS9fBH4L3F8P0Sw5XAbPs9CV\nxZejYYrVnQjVQusHoIp8c+DfaBKwB/GZneix1BrfFb0nd0dXvD7iXgGtrZ0tODSl3j2QbdCIkFw0\nwxjgBPDaW6xBsDy6NHkP9/8kcgoextDaml4EAYBPX9TKPIXKlXt/dGn7CWg42W8a198uzahr4ETy\nJc8EnVv5IzC6HuLMcwj0Q8/l56iL4zyPxEIB26PJs9ZHfdX7oLmHHkAV+b/xE1zFHAQ9ga3I33vr\noNZ/zrjqSoZLU+rdB2kCDkeHq7lk+tcAZ4JXOISDIGhGLZKctbARWij8CdSaf5HW1oU1Eb2QZJT7\ncqjVfjDqu78s1bC3RJGB6OrPE1AXBWh61muAP4H3YVqSVYIbZR2F1t98GjizKvnOdUS4E3lF3kTe\nXz0anwWJHEddKhuhedB3QV04HwCPoZOrY2htrfRYptS7HzIIVWY/Q2+WL9Ah+J/BK63UgmAAmmxs\nZ/RCXB14irySf8NN0tQOXV6dU+5PA7/F71rOdYE10fPfHvCB6zyoG79yHvFQa+9o9EHVz20Yh7oo\nrst6oq1C3Irh/dHVwhPRPD8vJnoQ9Y/vjirx3dCFazlF/kYlKSw6oOGGa5BX4q3oSurH0fsnoLU1\nqSIwptS7L7IOmtI3Vx92CnARcFWnyj2H+v1Gohfpzqifcwyq6J8CXophbXSNvHI/EV0ocyVwR1fC\nyEQV4oXovMIfgGvqolgCMhj1LR+DTqLl+C86wXw/eNleo1CAQE/0wfQL/cjpwCOJRC759ED/1nu4\n19pAQG7i0U8gckZHuRuiRlDutQhV4GoEdaifkBim1Ls34qEX9jnAlu7Lrin3HOqPD1/Ea6BWVU7J\nP1f1+Hi9YfdBIzs2QN0Nf+1KXhnRkM9T0Pjfq4A/edonGUI8dGRxNBrl1OI2TEPz6fy9rkogOlwI\n6tHow3ksGiv/79jKXFd07o5e698EPkHniR4BnontVtF48c1R182OaHWzKcCT6LX/JPBJjVZ4m1I3\nwCmJPVH3QzzlniMIBqMX947oxb4JeqM+F3q9XzWXjc86aIrYQ9Eb68/A41Fzyzi3zEmoxfgv4Hee\nyp8ishqasXMUOpmW4zE0zcK99RRfnkNgBdT/fzR6LpfEytOii4C+Qd7dsTzwKKrEH8Xn84rbVlfK\nMGCb0GtD1G2TU+BP09panQnc8phSN8KUVO651YWzKm5arZmNaX8zLIUuGHnO/f9SkbjbePj0R8P4\njkNXHd6CFhx5M4q/VHRS+XjUvfM0Oqk6pnaLmGRZNIf+99GHZI7JaLnDa+pt4jOH6GjqRODbaD6U\nP3jQ9VqmOnG+Pfn5nnWBZ1Gf9ePAKxUvzVfjZFN0BJe7bnHt54yTlzK0StuUulGMosp9Dpq74krw\nksldHgTLkb9ZtkYruX+FWmkvu/9forU1vp9TQ9S2Qt0VB6LnczsRFbwLpzsSVfBNqJvjH54u3koY\n6YdmSPw+OnnXw22Yg+YQuQl4FLx0oo9iILoY7mC0CMQK6KKwK7sUVqoP6m3JjwI3R/Pp5ybtn6to\nWX4QDEGvwc1cm5ujVv5rwAvklfj4DCfLM6VudIZ46JLkk1ArKMcY1J1xJ3jJTYbq0HZ19GYK31jz\nUAX/GprL+w1gbMWrXzsq+LnAbURQ8C4iY2tUKX0XDfO8HrgnXkikDEb7en/3fy56ZTHqOrgJda9k\nxSKMjAtJ3B3ts13RPETXA49Hym2uNTp3ID9fsx7wCvn5mqe7tABIQwqHo2GFG7r/N0cn+18hbFDA\ne5lPhtceU+pGVGRd1Ed9ODDQfTkF9eX+FbzqrD7N+zA3R2++3I24IvAuquDfQJX9W8DELllReQV/\nkHt56MKP3GtcKSUv0Ae1qI9wbdyBFjp4NloyKVkZXW6+Pxrm1iO08VlUkd8GXt0V2XYPvw3Q1cyH\noqGV1wO3eVDajafZOtcl7+rYDs28+Qx5Jf4CPtHKOGqU1gboNZNT4OujMuQMhNdRRT62zhR4MUyp\nG11F+qNugePIh9C1ocPe24B7wKv+Cs0g6I/enIU3awuq7N9FJ69y7z+gtbXzIbkq+DXRUM3caxER\nlLxoPdlR7rU0uojkIeBRb0lhY8kpun1QRb5lqIk2dJLtHtQi/zhCL2QK95BrRUcae6FuqpuAG0oW\nLtE48a3Ju+G2Qo2F8IT6a/idrB3QdBeroZPHha9mNOlVWIG/SWvrzFgnm11SU+q5JE3NaBL7iwq2\nm0M/HzYAAAp/SURBVFLPPOKhQ+JjURdGzspcjPo2a6fgw6hfdG063tzDgE/RlXsfohNyHy5539ra\nMSa9uJIXtDZs+PVhOKrGJZr6FrBXG96On7HS+Ls5YM7f+eGqb7DhsqFLey4akXEPWhouqQUoNUO0\nX/dCz3cntD8edK+3lkwoa1+ugkZBhV9L095f/Tx+Ef+6TrSvhqYHHoH2cfj95+Qf4OHXlAz7v6tB\nKkq9GX1qfxPN2/wiGqb1TlzBjLSQpVHL8yA0+qBQwd+OWp/pxXsHQS+KK4MRqLL4AlXwn6ALmCa4\nV+79DP7bCuqL3ZiOiul14FXmDXyH//24N68dviZT19muL3M3bCVgLx5kLx4EWPwO6340j5bHNuDN\nm0bw0YseCS0/rzKi9+46qDW9JarEh6IPpgfRUclMfJYG1nKvcF/Np+MD8QN8FhMEg1ClvwpaJDr3\nfhj6NxqK/i3CD+Tc/x+knpcoO6Si1LdFczjkVjKe5v6/MLSPKfW6paSCB61CM9q9nsyMr1gnzlZA\nlccwiiuX3qhy/xS1CicBk5jYModnZGU+H7cp88etR8/3V2bpsU0MGQs95sGMNYQ5y0xice83m3rP\nCM4Y99z4H74iK6/yBWs3qWJcHXULvBh6vVfzAg8FOJ/4cFTGLVFFvhkwSeDFKf14+/b1mHDKbiyY\n15M1YclrLVT5vw/eWHoPfY8hW37K8nvOZOB6vdBoktxrBdRttYr7TbGHaU6Rf5q5tNDZJBWlfiA6\nE360+3wo6k/7aVzBjKwhS6MTiQehrouWgh3CSv6pVC35cgTBAOY1rcrLS23L1N7bstjbiGZZg36L\nBjNkAQxZAEstgL6LYW7zIhZ5s2heOIWWWTNYOH0x86f0ZN6kfsybNJgFM5Zi0ey+LJ4/o9+8+VO2\nHff11zuP/Upax87ut96Uhcv3XciA+T2YuKiJ8Z7wUcsi3u/VxkfAx+iE4/Qk4uQFBqOKe/gij9Xm\n9mStNo8RPdoY1msxKy1sYuH7yzR/9t81Bnz1xJr95anV+/ed2b/3sjT1XoYe/ebTa8hMWpafRcvy\nX9N76Hx6LQ09B/TC6zkAz1saHcHMI/cA7PiajD4kJwCzupmbpFqkotS/g1rpptS7FdIbtfhGutf2\ndFTyk9BJrbfcy733ZtRMTMBltFwNnXhdz/2/Pup66FOw8xx04dQYYAxbzHiJS17vAQxBlVruFf48\nCGkbRNvCpaFtMDAAmvrR1KMFEbzF89p6LJpPrwXzvZYF85v6zp/v9Zu3kEFzF8jgOQvos2CB10Zb\nGyIitLW1edLWhrS1ebSJJ4vbkMXiIU3iNTcLzZ54TU14zR5ek0eT1yQ0eTR783v29Gb16yVf9OnF\nVy09vTm9e8u83r3b5vfsLYt69Bbp0dKE19yELJqPLP4amA3eLJp6zsBrngV8iSamyr1mdPjc2ppc\nilojCqko9W3QRS0598vp6FAzPFkqaD6SHKOxAsINhvRGh/Qj3WsbdMVnMSahSn4CMDX0mtL+c7l6\nmtKErmIdisYk515DQ/+vSXHlnWMCGl73DKrIX0usWpCGb/ZGc763LHnJ4hbmfb4U82csw+Kvhwye\n/dXyg+bMH9iycFG/loWLW3ovWtyn16K2Pj0WS+9ebdLSY7H08qBpYZM3b1GzN39hszdvQXPT/AU9\nmubO69k8Z27PHnO/6tPrq+kDB0yUHn1n0Gup6bSsMIOmXnNRy3oe6v/+CphrFnSmGUk+Vz6oa7vm\nSr0HOlG6C5pO8wVsotRQhbsqeat4vdD/pZR9IW107pZoIvp19RkdRw1vx0qVYBjVJ7WQxj3JhzRe\ng+ZLDmNK3XAsUfbroRNrxSzs3PveERqcRXFLP/f+Y0x5G/WLLT4yGgXxUCOh053qLbe4YXSRzOpO\n8+EZhmF0nYp0Z1PSUhiGYRjpYUrdMAyjgTClbhiG0UCYUjcMw2ggTKkbhmE0EKbUDcMwGghT6oZh\nGA2EKXXDMIwGwpS6YRhGA2FK3TAMo4EwpW4YhtFAmFI3DMNoIEypG4ZhNBCm1A3DMBoIU+qGYRgN\nhCl1wzCMBsKUumEYRgNhSt0wDKOBMKVuGIbRQJhSNwzDaCBMqRuGYTQQptQNwzAaCFPqhmEYDYQp\ndcMwjAbClLphGEYDEUepXwK8A7wG3AUMSkQiwzAMIxV2Jf9QuNC9iiG1ESc2I9MWICIj0xYgAiPT\nFiAiI9MWICIj0xYgIiPTFiAiI9MWICIV6c44lvp/gDb3/nlg5RhtZYGRaQsQkZFpCxCBkWkLEJGR\naQsQkZFpCxCRkWkLEJGRaQtQTZLyqR8FPJRQW4ZhGEaF9Ciz/T/A8kW+PwO4370/E1gA3JygXIZh\nGEYFeDF/fwRwNLALMK/EPh8AI2IexzAMo7vxIbBGLQ+4B/AWsEwtD2oYhmGUJo6lPhboBcxwn58F\njo0tkWEYhmEYhmEY1SHqoqQ9gHdRi//U2oi2hINQ19FiYLNO9vsYeB14BXih+mJ1IKqcafYlwBB0\nUv194FFgcIn9Piad/ozSP5e77a8Bm9ZIrkLKyTkS+ALtv1eAs2omWZ5rgcnAG53sk4W+LCfnSNLv\ny1WAAL3H3wROKLFf6v0ZZVFSMzqBOhzoCbwKrFsL4RzrAGuhHdqZshyHKqy0iCJn2n0JcDHwS/f+\nVEovREujP6P0z7fIh+RuDTxXK+FCRJFzJHBfTaXqyI6oYimlLLPQl1BezpGk35fLA5u49/2B90jg\n2qxG7pcoi5K2Qi/gj4GFwK3AflWQpRTvolZlFOJGCMUhipxp9yXAvsAN7v0NwP6d7Fvr/ozSP2H5\nn0dHGsvVSL4cUf+OaV6PAE8BMzvZnoW+hPJyQvp9OQl9eAN8hXo4VizYp8v9We2EXqUWJa0ETAh9\n/tR9lzUEeAz4Hxq6mUWy0JfLoUNd3P+lLro0+jNK/xTbp9YrpKPIKcB26DD8IWC92ojWJbLQl1HI\nWl8OR0cWzxd83+X+LLf4qBRxFyXVIh9MFBnLsT3wObCsa+9d1AJIkrhy1iq3Tik5zyz4LJSWqRb9\nWUjU/im02mqdsyjK8V5G/bBzgD2Be1D3XNZIuy+jkKW+7A/cAfwMtdgL6VJ/VqrUdy2z/QjUF7RL\nie2foR2aYxX0CZQk5WSMwufu/6nA3egQOWklFFfOWvQldC7nZFThTwJWAKaU2K8W/VlIlP4p3Gdl\n910tiSLn7ND7h4Er0TmKGWSHLPRlFLLSlz2BO4F/og+WQjLRn1EWJfVAV0sNR2Pd05jcA52A3LzE\ntr7AAPe+HzAG2K0WQhWhMzmz0JcXk4/WOI3iE6Vp9WeU/glPRm1DOpN7UeRcjrzVthXqf0+D4USb\nKE2rL3MMp7ScWehLD7gR+EMn+2SiP8cCn5APFbrSfb8i8GBovz3R2d4PgNNrKSBwAOqnmotalw+7\n78Myro7eWK+i4Ua1lhGiyQnp9iWohfMYHUMas9KfxfrnR+6V4wq3/TU6j4iqJuXkPA7tu1eBZ9Cb\nvNbcAkxEXasT0HmzLPZlOTmz0Jc7oEElr5LXl3uSzf40DMMwDMMwDMMwDMMwDMMwDMMwDMMwDMMw\nDMMwDMMwDMMwDMMwDKPe+X8ZaLeS19mSngAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x11105e750>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# <!-- collapse=False -->\n",
    "x = np.arange(-2,2.1,0.1)\n",
    "\n",
    "R = x**(2)+x + np.maximum(-x,0)**2-0.5*np.maximum(x-1.0,0)**2\n",
    "line = 2*x+0.5**2+0.5-2*0.5\n",
    "D = R-0.5**(2)-0.5-2*(x-0.5)\n",
    "fig = plt.figure(figsize=(6,5))\n",
    "ax = fig.add_subplot(1,1,0)\n",
    "ax.set_xlim(-2,2.0)\n",
    "ax.plot(x,R,lw = 2,label = \"$R(x) = x^2+x+(-x)_+^2-0.5(x-1.0)_+^2$\")\n",
    "ax.plot(x,D,label = \"$D_R(x|0.5)$\")\n",
    "ax.plot(x,2*(x-0.5)**2,label = \"4-strongly smooth\")\n",
    "ax.plot(x,0.5*(x-0.5)**2,label = \"1-strongly convex\")\n",
    "ax.set_title(\"Strong Smoothness and Convexity\")\n",
    "ax.legend(loc= 1)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The above function $R(x) = x^2 + x +\\max(-x,0)^2-0.5\\max(x-1.0,0)^2$ is $4$-strongly smooth and $1$-strongly convex. This is illustrated by the Bregman divergence $D_R(x\\|0.5)$, which always less than $\\frac{3}{2}(x-0.5)^2$ and greater than $(x-0.5)^2$. Of course, to really check strong convexity/smoothness one needs to check $D_R(x\\|y)$ for all $x,y$, not just fixing $y=0.5$.\n",
    "\n",
    "One important fact about strong convexity and strong smoothness is their effects at minima:\n",
    "\n",
    "<div class=\"theorem\">\n",
    "Suppose $R$ is $\\sigma$-strongly convex with respect to $\\|\\cdot\\|$, and suppose $x_1=\\argmin_x R(x)$. Then\n",
    "\n",
    "$$\n",
    "R(x_2)-R(x_1)\\ge \\frac{\\sigma}{2}\\|x_1-x_2\\|^2\n",
    "$$\n",
    "\n",
    "If $R$ is $\\sigma$-strongly smooth instead, then\n",
    "\n",
    "$$\n",
    "R(x_2)-R(x_1)\\le \\frac{\\sigma}{2}\\|x_1-x_2\\|^2\n",
    "$$\n",
    "</div>\n",
    "\n",
    "These facts are direct applications of the definitions using the fact that $\\nabla R(x_1)=0$ (since $x_1$ minimizes $R$).\n",
    "\n",
    "\n",
    "Now with all of these notions in hand, we can state a very important fact:\n",
    "\n",
    "<div class = \"theorem\">\n",
    "<h4>Strong/Smooth Duality</h4>\n",
    "Suppose $R$ is a closed convex function. Then $R$ is $\\sigma$-strongly convex with respect to a norm $\\|\\cdot \\|$ if and ony if $R^*$ is $\\frac{1}{\\sigma}$-strongly smooth respect to the dual norm $\\|\\cdot \\|_*$.\n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's prove that strong-convexity implies strong smoothness first.\n",
    "\n",
    "We have\n",
    "$$\n",
    "D_{R^*}(w\\|\\theta) = R^*(w)-R^*(\\theta) - \\langle \\nabla R^*(\\theta),(w-\\theta)\\rangle\n",
    "$$\n",
    "Suppose\n",
    "$$\n",
    "R^*(w)=\\langle x,w\\rangle - R(x)\\\\\n",
    "R^*(\\theta) = \\langle y,\\theta\\rangle - R(y)\n",
    "$$\n",
    "Then we also have $\\nabla R(x)=w$, $\\nabla R^*(\\theta) = y$, $\\nabla R(y)=\\theta$, and $\\nabla R^*(w)=x$.\n",
    "Applying this, we have\n",
    "$$\n",
    "\\begin{align*}\n",
    "D_{R^*}(w\\|\\theta) &= \\langle x,w\\rangle - R(x) + R(y)-\\langle y,\\theta\\rangle - \\langle y,w-\\theta\\rangle\\\\\n",
    "&=R(y)-R(x)-\\langle y-x,w\\rangle=R(y)-R(x)-\\langle y-x,\\nabla R(x)\\rangle\\\\\n",
    "&=D_R(y\\|x)\n",
    "\\end{align*}\n",
    "$$\n",
    "\n",
    "Thus to show $D_{R^*}(w\\|\\theta)\\le \\frac{1}{2\\sigma} \\|w-\\theta\\|_*^2$, we can instead show\n",
    "$$\n",
    "D_R(y\\|x)\\le \\frac{1}{2\\sigma} \\|\\nabla R(x)-\\nabla R(y)\\|_*^2\n",
    "$$\n",
    "\n",
    "From strong convexity, we have\n",
    "$$\n",
    "D_R(x\\|y)\\ge \\frac{\\sigma}{2}\\|y-x\\|^2\n",
    "$$\n",
    "$$\n",
    "R(x)-R(y)-\\langle x-y,\\nabla R(y)\\rangle\\ge \\frac{\\sigma}{2}\\|y-x\\|^2\\\\\n",
    "R(y)-R(x) +\\langle x-y,\\nabla R(y)\\rangle \\le -\\frac{\\sigma}{2}\\|y-x\\|^2\\\\\n",
    "D_R(y\\|x) \\le -\\frac{\\sigma}{2}\\|y-x\\|^2 +\\langle x-y,\\nabla R(y)-\\nabla R(x)\\rangle\\\\\n",
    "D_R(y\\|x) \\le -\\frac{\\sigma}{2}\\|y-x\\|^2 + \\|y-x\\| \\|\\nabla R(y)-\\nabla R(x)\\|_*\n",
    "$$\n",
    "Now set $k=\\|y-x\\|$. Then we can see that the LHS is maximized for\n",
    "$$\n",
    "k= \\frac{\\|\\nabla R(y)-\\nabla R(x)\\|_*}{\\sigma}\n",
    "$$\n",
    "which gives\n",
    "$$\n",
    "D_R(y\\|x) \\le \\frac{1}{\\sigma} \\|\\nabla R(y)-\\nabla R(x)\\|_*^2\n",
    "$$\n",
    "as desired.\n",
    "\n",
    "Now for the other direction, suppose that $R^*$ is $\\frac{1}{\\sigma}$-strongly smooth. Then by the same calculation as before, it suffices to show that\n",
    "$$\n",
    "D_{R^*}(w\\|\\theta) \\ge \\frac{\\sigma}{2}\\|\\nabla R^*(w)-\\nabla R^*(\\theta)\\|^2\n",
    "$$\n",
    "From strong smoothness, we have\n",
    "$$\n",
    "-D_{R^*}(\\theta\\|w) = -R^*(\\theta)+R^*(w) + \\langle \\nabla R^*(w),\\theta-w\\rangle\\ge -\\frac{1}{2\\sigma}\\|w-\\theta\\|_*^2\n",
    "$$\n",
    "Now apply a similar rearrangement:\n",
    "$$\n",
    "D_{R^*}(w\\|\\theta) = R^*(w)-R^*(\\theta)-\\langle \\nabla R^*(\\theta),w-\\theta\\rangle \\ge -\\frac{1}{2\\sigma}\\|w-\\theta\\|_*^2 +\\langle w-\\theta,\\nabla R^*(w)-\\nabla R^*(\\theta)\\rangle\\\\\n",
    "= -\\frac{1}{2\\sigma}\\|w-\\theta\\|_*^2 + \\|w-\\theta\\|\\|\\nabla R^*(w)-\\nabla R^*(\\theta)\\|_{**}\n",
    "$$\n",
    "And now we note that $\\|\\cdot \\|_{**}=\\|\\cdot \\|$ and apply the same optimization trick as before to get\n",
    "$$\n",
    "D_{R^*}(w\\|\\theta) \\ge \\frac{\\sigma}{2}\\|\\nabla R^*(w)-\\nabla R^*(\\theta)\\|^2\n",
    "$$\n",
    "and so we are done."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Strong/Smooth duality is very powerful - when we use a strongly-convex regularizer in an online convex optimization then strong/smooth duality causes some amount of stability (smoothness!) in the prediction vectors.\n",
    "\n",
    "Moving on, there's one final fact that seems fairly technical, but is a good exercise in these concepts (and is also useful):\n",
    "\n",
    "<div class=\"theorem\">\n",
    "Suppose $R(x)$ and $f(x)$ are convex functions, and suppose $R(x)+f(x)$ is 1-strongly convex with respec to $\\|\\cdot\\|$. Let $x_1=\\argmin_x R(x)$, and let $x_2=\\argmin_x R(x)+f(x)$. If $z$ is an arbitrary subgradient of $f$ at $x_1$, then\n",
    "\n",
    "$$\n",
    "\\|x_1-x_2\\|\\le \\|z\\|_*\n",
    "$$\n",
    "\n",
    "and\n",
    "\n",
    "$$\n",
    "R(x_1)+f(x_1)-R(x')-f(x')\\le\\frac{1}{2}\\|z\\|^2_*\n",
    "$$\n",
    "\n",
    "for any $x'$.\n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To prove this, first we'll use strong-convexity of $R$:\n",
    "\n",
    "$$\n",
    "\\frac{1}{2}\\|x_1-x_2\\|^2\\le R(x_1)+f(x_1)-R(x_2)+f(x_2)\n",
    "$$\n",
    "\n",
    "Now notice let $R'(x)=R(x)+f(x)-\\langle z,x\\rangle$. Then since $x_1=\\argmin_x R(x)$ and $z$ is a subgradient of $f$ at $x_1$, $0$ is a subgradient of $R'$ at $x_1$ and so $x_1=\\argmin_x R'(x)$. Further, since $\\langle z,x\\rangle$ is a linear function, $R'$ is also 1-strongly convex with respect to $\\|\\cdot\\|$. Therefore\n",
    "\n",
    "$$\n",
    "\\frac{1}{2}\\|x_2-x_1\\|^2 \\le R(x_2)+f(x_2)-\\langle z,x_2\\rangle -R(x_1)-f(x_1)+\\langle z,x_1\\rangle\n",
    "$$\n",
    "\n",
    "Adding these two equations and using the definition of the dual norm, we have:\n",
    "\n",
    "$$\n",
    "\\|x_1-x_2\\|^2 \\le \\langle z,x_1-x_2\\rangle \\le \\|x_1-x_2\\|\\|z\\|_*\n",
    "$$\n",
    "\n",
    "from which we can conclude the first statement.\n",
    "\n",
    "Let's use $R'$ once more to show the second statement. Since $R'$ is strong-convex, $R'^*$ is strongly smooth with respect to $\\|\\cdot\\|_*$. Further,\n",
    "\n",
    "$$\n",
    "R'^*(0)=-R'(x_1)=-(R(x_1)+f(x_1)-\\langle z,x_1\\rangle)\\\\\n",
    "x_1=\\nabla R'^*(0)\n",
    "$$\n",
    "\n",
    "since $x_1=\\argmin_x R'$. Also, since $x_2$ is a minimum point of $R(x)+f(x)$, $-z$ is a subgradient of $R'(x)=R(x)+f(x)-\\langle z,x\\rangle$ at $x_2$. Therefore:\n",
    "\n",
    "$$\n",
    "R'^*(-z) = \\max_{x} \\langle -z,x\\rangle  -R(x)-f(x)+\\langle z,x\\rangle = -\\min R(x)+f(x) = -R(x_2)-f(x_2)\n",
    "$$\n",
    "\n",
    "Therefore\n",
    "\n",
    "$$\n",
    "\\frac{1}{2}\\|z\\|_*^2 \\ge D_{R'^*}(-z,0)= R'^*(-z)-R'^*(0)-\\langle \\nabla R'^*(0),-z\\rangle= R(x_1)+f(x_1)-R(x_2)-f(x_2)\\ge R(x_1)+f(x_1)-R(x')-f(x')\n",
    "$$\n",
    "where the second-to-last equality is simply plugging in our various equations about $R'^*$, and the last is because $x_2=\\argmin_x R(x)+f(x)$."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}