Chapter05 Logistic回归.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 回归：  \n",
    "\n",
    "利用一条直线对一些数据点进行拟合，这个拟合过程我们称为回归，而这条拟合的线我们称为最佳拟合直线。  \n",
    "\n",
    "但是Logistic回归是一种分类算法，其主要思想是：根据现有数据对分类边界线建立回归方程，以此进行分类。\n",
    "\n",
    "![title.png](./picture/05逻辑回归.png)\n",
    "\n",
    "对于逻辑回归进行分类，我们希望的是，<font color=\"red\" size = 4>能接受所有输入然后预测类别</font>\n",
    "\n",
    "例如，对于两种类别的情况下，输出应该为1或2，我们之前接触到的阶跃函数可以对应这种情况。  \n",
    "但是，阶跃函数存在一个问题是，从0到1的瞬时跳跃不够光滑，也即不可导，这样在数学上不容易处理。  \n",
    "因此，我们一般选择另一个具有相同性质且更方便处理的函数——sigmoid函数，如图\n",
    "![title.png](./picture/05阶跃函数.png)\n",
    "\n",
    "为了实现Logistic回归分类器，我们在每个特征上乘以一个回归系数，然后把所有值相加，最后将这个总和代入Sigmoid函数中，进而得到0-1之间的值，  \n",
    "然后取一个阈值进行分类，一般这个阈值我们选择0.5，即大于0.5归于一类，小于0.5归于一类，因此，Logistic可以看成是一类概率估计。\n",
    "\n",
    "现在问题转为<font color=\"red\" size = 4>如何确定最佳回归系数</font>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 基于最优化的方法确定最佳回归系数  \n",
    "\n",
    "#### 1.梯度上升法\n",
    "核心思想：要找到某函数的最大值，最好的方法是沿着该函数的梯度方向探寻。  \n",
    "\n",
    "如果梯度记为▽，则函数f(x, y)的梯度记为：\n",
    "![title.png](./picture/05梯度.png)  \n",
    "梯度其实就是沿着各个方向求偏导数，然后叠加这种效果，但是梯度反映的是变化最快的一个方向\n",
    "![title.png](./picture/05梯度上升.png)  \n",
    "梯度算子纵使沿着指向函数增长最快的方向。同时每次移动量的大小称为步长，用α表示\n",
    "![title.png](./picture/05迭代公式.png)\n",
    "\n",
    "事实上，梯度上升算法和梯度下降算法是一样的，只不过加法变成减法  \n",
    "<font color=\"red\" size = 4> 梯度上升算法用来计算函数的最大值，梯度下降算法用来计算函数的最小值</font>\n",
    "\n",
    "<font face=\"微软雅黑\"> <font color=\"red\" size = 6>Talk is cheap， show me the code.</font>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 函数说明"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "3210Runoob0123\n"
     ]
    }
   ],
   "source": [
    "#strip() 返回移除字符串头尾指定的字符生成的新字符串\n",
    "str = \"00000003210Runoob01230000000\"; \n",
    "print(str.strip( '0' ))  # 去除首尾字符 0"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Logistic 回归梯度上升优化算法\n",
    "import pandas as pd \n",
    "import numpy as np\n",
    "\n",
    "def loadDataSet(): #导入数据，读取文本\n",
    "    dataMat = []; labelMat = []\n",
    "    fr = open('./data/Logistic_testSet.txt')\n",
    "    for line in fr.readlines():\n",
    "        lineArr = line.strip().split()\n",
    "        dataMat.append([1.0, float(lineArr[0]), float(lineArr[1])])\n",
    "        labelMat.append(int(lineArr[2]))\n",
    "    return dataMat, labelMat\n",
    "\n",
    "def sigmoid(inX): #构建sigmoid函数\n",
    "    return 1.0 / (1 + np.exp(-inX))  #注意这里math.exp()用于标量，而np.exp()用于数组，两者存在区别\n",
    "\n",
    "def gradAscent(dataMatIn, classLabels):\n",
    "    dataMatrix = np.mat(dataMatIn)\n",
    "    labelMat = np.mat(classLabels).transpose()\n",
    "    m, n = np.shape(dataMatrix)\n",
    "    alpha = 0.001\n",
    "    maxCycles = 500\n",
    "    weigths = np.ones((n, 1))\n",
    "    for k in range(maxCycles):\n",
    "        h = sigmoid(dataMatrix * weigths) #h这里是一个列向量\n",
    "        error = (labelMat - h)\n",
    "        weigths += alpha * dataMatrix.transpose() * error #更新权重\n",
    "    return weigths   "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 4.12414349],\n",
       "       [ 0.48007329],\n",
       "       [-0.6168482 ]])"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#测试\n",
    "dataArr, labelMat = loadDataSet()\n",
    "gradAscent(dataArr, labelMat)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYoAAAEGCAYAAAB7DNKzAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3de3Bd5Xnv8e8j+QK2tc3FkpF8wVxcY0EwCR45GTcXDoEYSiFtEgbayXEujXGm0KRzepy0nEmaZDJNnfuBFJuc0CQdkpTTBMIkDoHmpENCg7FNbYxtnDgGIyNjy4AtGYNtWc/5Yy/JW1t7r60t7b1u+n1mNNZee0l6dPF69vs87/suc3dERETKaYg7ABERSTYlChERCaVEISIioZQoREQklBKFiIiEmhB3APUwY8YMnzdvXtxhiIikxqZNmw66e3Op5zKZKObNm8fGjRvjDkNEJDXMbE+551R6EhGRUEoUIiISqu6JwszuMbMDZvZ0wbG/N7MXzGxz8HZtmY9dZmY7zWyXmX2y3rGKiMhwUYwovg0sK3H8q+5+WfC2rvhJM2sEvgFcA7QDN5tZe10jFRGRYeqeKNz9UeDlUXxoB7DL3Xe7+3HgB8ANNQ1OREQqirNHcauZPRWUps4s8fwsoLPg8d7gWElmtsLMNprZxu7u7lrHKpI6nYc7uW3dbXR8s4Pb1t1G5+HOyh8kUkJcieIu4ALgMmAf8OUS51iJY2W3unX3u919sbsvbm4uORVYZNzoPNzJojWLWLtpLRu6NrB201oWrVmkZCGjEkuicPf97n7S3fuBb5IvMxXbC8wpeDwb6IoiPpG0W/3Yao4cP8KJ/hMAnOg/wZHjR1j92OqYI5M0iiVRmFlrwcM/AZ4ucdoGYL6ZnWdmk4CbgAejiE8k7da/sH4wSQw40X+CJ154IqaIJM2imB77feA3wAIz22tmHwZWm9lWM3sKuAL46+DcNjNbB+DufcCtwM+BHcB97r6t3vGKZMGSWUuY2DBxyLGJDRPpmFVq8C4SzrJ4h7vFixe7tvCQ8WygRzFQfprYMJFpk6axZeUW5kyfU/kTyLhjZpvcfXGp57QyWySD5kyfw5aVW7jl8lvoaOvglstvUZKQUcvkpoAikk8Wd1x7R9xhSAZoRCGSYFoLIUmgEYVIQhX3GTa/uJl7t96rEpJETiMKkYTSWghJCo0oRBKk83Anqx9bzfoX1rPn0J7EroUojHPJrCWsWrpKo5wMU6IQSYjiUlNDiQF/EtZCqCQ2/qj0JJIQxaWmfvoBsGDbs4G1EKuWrootRlBJbDzSiEIkIUptuwHQPLWZedPn0TGrIxElHm0PMv4oUYgkxJJZS9j84uYhF+GJDRO5sf3GRK2HKBdn3CUxqR+VnkQSYtXSVUybNG1wj6aklJqKpSVOqR0lCpGESMu2G1HEqYWGyaJNAUUkUbShYTy0KaCIpIZmVSWPEoWIJIpmVSWPEoWIJIpuupQ8ShQiUlJcDWXNqkoeNbNFZJi4G8oDe0k98cITNV1oqD2qylMzW0SqEndDeeCmS/92478B8J773jPmUc1A8lu7aS0bujawdtNaFq1ZpKm3I1D3RGFm95jZATN7uuDYF83sGTN7yszuN7Mzynzsc2a21cw2m5mGCCIRSUJDudYX9riTX5pFMaL4NrCs6NgjwCXufinwW+BvQz7+Cne/rNyQSETCjabXkISGcq0v7ElIfmlV90Th7o8CLxcde9jd+4KHjwOz6x2HyHi0fu965t8xnzs33MmGrg2s2bhmRK/Kk9BQrvWFPQnJL62S0KP4EPCzMs858LCZbTKzFWGfxMxWmNlGM9vY3d1d8yBF0qbzcCdv//bbOXby2OCxPu+j91hvxVflSdhOpNYX9iQkv7SKZNaTmc0DfuLulxQdvx1YDPyplwjEzNrcvcvMWsiXq24LRiihNOspnTQjpbZuW3cbd264s+RzHW0drP/I+ogjqk49Zl7VazZVFoTNeootUZjZcmAlcKW7Hx3B5/h74Ii7f6nSuUoU6RP3dMws6vhmBxu6NpR8bvmly2ma3JT4pKwLe3TCEkUs96Mws2XAJ4C3l0sSZjYVaHD33uD9q4HPRhimRCiscZmkezGkyZJZS/ivF/+Lvv6+IccnNUzigZ0PcPTE0cTfynRgmqzEK4rpsd8HfgMsMLO9ZvZh4E6gCXgkmPq6Jji3zczWBR86E/i1mW0BngB+6u4P1TteiYdmpNTeqqWraJrUxISGU68HJzdO5voF1w8mCdA0Uams7iMKd7+5xOFvlTm3C7g2eH83sKiOoUmCpO2uaWnopww0pItLN++57z1KylIV3QpVEmHV0lXcu/XeYT2KJM5IKe6npK10k7akLPFLwvRYkUinY3Ye7uQD93+AmV+cycwvzeQDD3ygqtW+aV/hm6RposWLAdfvXa872yWQNgWUcaXzcCdvuOsNHD52eMjx6ZOns/WjW0eUmMrNJkrDlNMBSZhNVDwym2ATOOknaWxopK+/TzPfIqZNAUUCqx9bTc+xnmHHR7IIbUAWVvgOlKTWf2Q9d1x7RywX4uKRWZ/34fjgLK20jdSyTIlCMmGk+xmtf2E9zvBRdD/9I27mJql0k2alZroVU5M9GZQoJPWq2WV0yawlGDbseAMNIx4RJGF7iywoNTIrlraRWlapRyGpd9u621i7ae2wWTy3XH7LsBk/tehRSG2oR5Es6lFIplWzWG/O9Dls/ehWll+6nJYpLbRMbWH5ouVDkkRctwAdibHGlqTvrXhktnLxSn7z4d+w8vKVGqkljEYUknrVjCgqSfKeU2ONrdQr+MaGRi6acRFvnfvWRC4alOhoRCGZVsvmcpLXSIw1tlKzjI6dPMaW/Vt0W1AJpUQhqVfL5nKS95waa2xhs4ySlBAlebSFh2RCrXYZrcf2FrXaF2qssZX6+EKVkk4a9reS+lCPQlKt1hevWvcoavn5at2jKBbW10ly70ZqQz0KyaRq1k+MVK3XSNSy5zHW2Ao/ftHMRUxunDy4BXmlvk6SezdSfyo9SWrV62ZHtbxZzlj6CuVGS7X63qrZ7ynJvRupPyUKSa04Ll7Vlrram9vZ1LWJfvoHj42krxDFVubVJJ2sbE2uPsvoqPQkqRX15nzVlro6D3fywDMPDEkSAFMmTqk4dTdppZ4s7G9Vj1LleKFEIakV9cWr2ov36sdWc/TE0FvCG8a7F7y74qvYpJV6srC/VdKSb5rUvfRkZvcA1wEH3P2S4NhZwL8C84DngBvd/ZUSH7sM+DrQCPwfd/9CveOV9Ch3q896XbyqvXiXOt9xdhzcUfFrJbHUU8veTRySlnzTJIoRxbeBZUXHPgn8wt3nA78IHg9hZo3AN4BrgHbgZjNrr2+okjZR3leh2lLXWEpjWSj1JE0W7iMSl7onCnd/FHi56PANwHeC978DvLvEh3YAu9x9t7sfB34QfJxILKq9eI/lYh9VqSdJmwTWm5Lv6EWy4M7M5gE/KSg9HXL3Mwqef8Xdzyz6mPcCy9z9L4LH7weWuPutlb6eFtxJvVR7C9Eobjk62pk843ERXRJuAZtUYQvukpwo3ge8qyhRdLj7bWW+xgpgBcDcuXMv37NnTz2+FZFEGcvFvpa77kr6JXFl9n4zawUI/j1Q4py9QOFf+mygq9wndPe73X2xuy9ubm6uabAiSTWWmTzlmrv3bb8vshLUSEpf46k8llRxJYoHgeXB+8uBH5c4ZwMw38zOM7NJwE3Bx0kC6T9zPMIu9pV+F+VuRdr9anck6wtGsq5Bax+Soe6Jwsy+D/wGWGBme83sw8AXgKvM7HfAVcFjzKzNzNYBuHsfcCvwc2AHcJ+7b6t3vFI9/WeOT9jFvtLvYqC521B0GXA8kvUFIxkNae1DMkQx6+lmd29194nuPtvdv+XuL7n7le4+P/j35eDcLne/tuBj17n7H7j7Be7++XrHKqOj/8zxKZ7JM3DRd/K9x3K/i4Gm7pzpc5g0YdKwzxvF+oKRrGt49PlHtfYhAbTXk4yZFjLFp3jR4XOHnuPA0aEtv+LfRXEDvHhEAdGsL6i0qLDzcCc7D+4c9nETbILWPkRMW3jImJVbyLSweWGq+xZp6bsULjq88eIbKy4qKx4BDuxFZdjg+VGsL6i0rmH1Y6s56SeHfVxjQ6PWPkRMNy6SMSs1RXPKxCkAHD1xNJVz9JOyxqDaNRIjibvjmx1s6Now7GNbprYwb/q8SNcXhK1rKBfnopmL2Lxyc91jG2/Cpseq9CRjVmrPpd5jvXzv6e/V/F4RUanXvS6qMZqtxkey/1W5ks+N7TdG/rsJ2z+qXJxvnfvWqMKTgBKF1ETxf/iOb3akum+RhL7LaJNVpc37Vi1dxb1b7x026khaOSctcY4H6lFIXaR9A7YkxF+vZJWWLcPTEmeUDvS8zi93HuAbv9zFVx/5bWRfVz0KqYuk1PhHKwnxj3SLDd21LXtO9jvPHjzCtq4eduzrZfu+HrZ39XDwyLHBcy5uy/HTv6pdGS72vZ6ipkSRDGnfgC3u+EeSrJKQ0GRsjh7vG5IMtu/rYeeLPbx+Ij8bbWKjMb+liYvbcrS35WhvzXFRa47ppw9faDkWShQiKVUpWWljv3Q50Pv6YDIY+PfZg68ycBmefvpEFrY20d46fTAxXNA8jUkT6t8l0KwnkZSq1JhOQtNdhsuXjl4dkhCKS0dzzjqd9tYc1y9q4+K26bS35WibfhpmVt0Xy+Wgt3f48aYm6OkZ43eSp0QhkmI1uWVqBBeaLDt6vI9nXuwdkhCeKVE6umJBMwtbc1zcVuPSUanfXdjxUVCiEEmxmkwhjeBCkxWVSke50ybQ3pbjzzrOpb0tx8LWJua3NEVSOqonJQqRFBvJAjup3sl+57mXXh1MBtu6hpeOZp95qnTU3ppjYWuO2WeeXn3pKAWUKETqIMopq5X6GBKusHS0Y18+MTyzr5fXTuT3mZrYaFzY0sQ7FjTT3poLRgq1n3WUZEoUIjU2mq03JBrdvcfY1nW4Yunopo45+QZza44LW6KZdZRkShQiNZaEfaLGu+LS0cC/3b2nSkezzjid9rYcf3xp2+BU1FlnpLB01NRUfjJCjShRiNRY6qasRnChqafXjp/kmReHJoRSpaO3zW9mYWvT4Ehh+pSMlI4imJmmRCFSYzWZshqlFE2B7e49VrQ24TDPHnyV/qB01HTaBNpbc9zcMffUrKP2eUw6fGj4J9P03xFTohCpMe16mjeWhn41paPrLm0b3Nqi5KyjUkkCNP23CrFt4WFmC4B/LTh0PvApd/9awTnvAH4MPBsc+pG7f7bS59YWHhK3uPeJils1e1C9dvwkO/cPLFg7HCxY6+Xo8XzpaEKDcWHLtMHVy+2tuepKR2E9hwxuYTRaidzCw913ApcBmFkj8AJwf4lTf+Xu10UZm8hYjfcpq+Ua+p/75df48/ZPDpmKurv7yLDS0Y2L5ww2mC9smcbkCY0xfjeSlNLTlcDv3X1P3IGIyNg9vnc93tfMlP7zmeQXMKn/PCb2n8/Dj5/Fw4/nm/oDpaNr39CaTwoZXrCWdklJFDcB3y/z3FvMbAvQBfyNu28rdZKZrQBWAMydO7cuQYrIcKVKRy/vvZ1Z/fnLi9PHCevk+IQtXDq3lf915XLaW3OcMWVSzJHLSMW+zbiZTSKfBC529/1Fz+WAfnc/YmbXAl939/mVPqd6FNmlm/TE6+CRY+wo2NKiVOloYWuOuWcb/7L9HzjSv5Oj/J6JjRbffTK06eGIJPp+FGZ2A/CX7n71CM59Dljs7gfDzlOiyCbdpGcMqrxY9g/MOtoX9BKCpLC/59Sso7bpp9EeNJhLlY7Ge0M/bRLZzC5wM2XKTmZ2DrDf3d3MOsjf4/ulKIOT5NCK5zEI2SF2pLOOll44Y3Cvo/Y3v4EzDr44/PMVJJ7x3tDPklgThZlNAa4Cbik4thLA3dcA7wU+amZ9wGvATR73EChNMjbkTt2K53oY4+/04JTp7Gg5j+0t5+ffZp7H7z/90KnS0eQJLGzLzzoamIo6f2aJWUelkgRobUJGxZoo3P0ocHbRsTUF798J3Bl1XJmRsfsMpG7Fcz2M8Hfa3+/sefnoqVHCez/N9pbz2d906r/brMMHWHhgN9fceCXtbflbb2rWkZSShNKTyIhoxXNpr0+YxM4Z57L9iecHewk79vUMLR01zWDpni20H9jNwgPP0r5/N2e+HiSXH34uxuglDZQoJDV0kx546fQc22eeP7R0dNZs+hsa4UdbmTb51IK1gZvpzJ85jdMm6b+6jF7ss57qQbOeAtq6YNTinobb3+88//LRwX2OBu6hUDjrqLWnm4v376b9QP5t4X8+wpwzp9DQUOL3Xut+lf62qpfwnmHSZz2JJErUNx56/cRJdr7YO2Qq6o59PbwalI4aG4wLm6ex9IIZtH/5M7Tvz5ePBktHA86eWv6L1PpClPKtyWOR4p6hEkWW6T/zqNRzGu5LR46xY1/vkLus7T74KieDaUfTJk9gYWsT7wtKRwN7HZ02MZh19JFfJON3moBXwBIdJYos03/mUanFNNzi0tHAvy/2vD54Tuv002hvzbHsknOCBWvTmX3m6aVLRwP0O5UYKFGIFKl2Gu7rJ07y28EFa+VLR2+54OzBO6wtbM1x1tSQvY4SXs8ekSx8DwKomS0JEncDuTCOcluFTJ0wc8gK5u37evh99/DS0cDtNgdnHU2scpvsLDSLs/A91FLCfx6J3uupHpQo0idp+zjtOfQ8n/nFN9j0/H5aJr2JGZMu49nuE0NKR+fkThu8Z8JAUph7VplZR9Wq9qKSxFfvCb8wRi6Jv6MCNZ/1ZGYfdPd/HltYIqfEuY/T6ydO8rv9RwZHCdsGS0dvA+BIg2HNzlsuOHuwwVyxdBS1FM+oGTcSkAxGa7Q9is8AShRSM1Ht4/TKq8eHNZh3dR8ZUjq66Jwm3nv57GCkML106Sjhrw5jl8vFHYHUUNlEYWZPlXsKmFmfcGS8qvU+Tv39TucrR4ckhO37eth3ePiso6vaZw6WkMouWCumV/Dh9HPIlLARxUzgXcArRccN+M+6RSTj0lj2cSouHeUXrvVy5FgfkJ91dEHzVJacd1Zw74QRzDqKW5bXwGThexhnwhLFT4Bp7r65+Akz+4+6RSTj0kj3capUOpo6qZGFrTn+9E2zBhvMC85pqn7WUdyyXL7K8veWUWGJog14odQT7v5n9QlHxrPCG90MlI5+tnVffluL4PabhaWjmbnJXNw2navaZw7OPKrZrKO0yfIIZCzUS6qJsETxz8DPzew7wGp3PxFyrsiojaR0dP6MU6Wj9tbpLGxt4uxpk2OOPEF00StNvaSaKJso3P0+M/sp8Clgo5n9C9Bf8PxXIohPMuaVV48PjhAGksKuA0foC0pHU4pKR+1tOf5gZgJLR3oFH04/n0ypND32BPAqMBlooiBRiIRxdzpffm3IKGF7Vw9dRaWj9tYc/+2ilvxK5rYc56aldKRX8OH088mUsOmxy4CvAA8CbwpuWyoyzLG+oHRUtNdRb1A6ajC4sGUaHeedxcLWgVlHKh3JKETVc1BvY4iwEcXtwPvcfVu9vriZPQf0AieBvuLl45a/ee/XgWuBo8AH3P3JesUjlY20dPTuN84abDCnctaR1NdoL8RR9RzU2xgirEfx1ohiuMLdD5Z57hpgfvC2BLgr+FfqbCSlo5amyVzcluPKhS2DDeZ5Z09NR+lI4hXVhVi9kppI+jbjNwDf9fzOhY+b2Rlm1uru++IOLEsGS0cFo4QdXUNLRxc0T2PxvLMGVzAvbM0xQ6WjcCpfxE8/55qIO1E48LCZObDW3e8uen4W0FnweG9wbFiiMLMVwAqAuXPn1ifaDDh09PiwbS2KS0cXndOUnNJRmi+2Kl9IRsSdKJa6e5eZtQCPmNkz7v5owfOlahgl9ycOkszdkN9mvPahpou7s/eV19hWmBS6Dg8rHbUHpaOBJnPiZh3pYisSu1gThbt3Bf8eMLP7gQ6gMFHsBQr3cJgNdEUXYTqULB3t66H39VOlo/OD0lE+IeRLR81NKh1JykTVc1BvY4jYEoWZTQUa3L03eP9q4LNFpz0I3GpmPyDfxD483vsTh44eH1y5fKp01MuJk0Wzji6bNdhLWDCzidMnadaRJMhoL8RRlRuTXtaMWJwjipnA/fkZsEwAvufuD5nZSgB3XwOsIz81dhf56bEfjCnWyJUqHe3Y18MLh14bPKc5mHV0xYLmwX6CZh1VKc09kDTTzzZVdCvUBDje18/vDvQOazIXlo7OmzF1cPXywK6o46J0VO/badbz8ysJSYrU/FaoUjv/sG4H9zz27GDp6PSJjVzU2sQNl7XR3ppPDOO6dBRnrTiXG9sFXclAMkKJImaXzj6Dv3jr+bQHTeZzz55Ko0pHp8R5sdXMKhFAiSJ2f3RpK390aWvcYUitqewkGdIQdwAimaT1H5IhShQyvo3TefEi1VCikPFNZSCRipQoREQklBKFSBblcvk1IsVvuVzckUkKKVGI1EO53kdUPRE106WGlChEanVRL3wVX3hBbmrKr/J2V09EUkmJQqSn59SFvPCt2ou6XsWnl0p1oZQoRCRaSbwoK8mHUqIQkWjpopw6ShQiWRR3M10yRYlCJG71KMXUqu8ighKFSO2M9lW8SjGScEoUUloSG45JV49X8Un8+Wfxb0OlulDaZlxK06vcZErCz3+sfxtx3oyqHJXkQsU2ojCzOWb2SzPbYWbbzOxjJc55h5kdNrPNwdun4ohVZNyp56hB/ZPUiXNE0Qf8D3d/0syagE1m9oi7by8671fufl0M8YmMXxpRSoHYRhTuvs/dnwze7wV2ALPiikckNqqDS8IlopltZvOANwLrSzz9FjPbYmY/M7OLQz7HCjPbaGYbu7u76xSpSB2UKsWIJEjsicLMpgE/BD7u7sVFyieBc919EXAH8EC5z+Pud7v7Yndf3NzcXL+AxwvNAolXkn/+9YwtizOqMiDWWU9mNpF8krjX3X9U/Hxh4nD3dWb2T2Y2w90PRhnnuKTGYryS/POvZ2zqjSRSnLOeDPgWsMPdv1LmnHOC8zCzDvLxvhRdlCIZVemVe5JHNBK5OEcUS4H3A1vNbHNw7O+AuQDuvgZ4L/BRM+sDXgNuclcBV2TMKr1yT/KIRiIXW6Jw918DVuGcO4E7o4lIRERKib2ZLVJzaoiK1JQShWSPGqLRq1VyVm8kkbTXk4iMXa2Ss3ojiaQRhch4pFfuUgWNKETGI71ylypoRCESRo1xESUKyaBallXUGBdR6UkySGWV6CXxZkRSM0oUIjJ2Ss6ZptKTSNqobyIRU6IQSRv1TSRiShQiYbTeQEQ9CpFQqr2LaEQhIhFSfyWVlCgkHbJ2gcna9zNS6q+kkhKFpEPWLjBj+X7UN5GIqUchkjbqm0jENKKQocZrSUREylKikKGyVuLJEiVxiUmsicLMlpnZTjPbZWafLPG8mdn/Dp5/yszeFEecUkO62I1eFpK4+iupFFuiMLNG4BvANUA7cLOZtReddg0wP3hbAdwVaZBSe6O92GXtApO172ekenrAffib+i6JFmczuwPY5e67AczsB8ANwPaCc24AvuvuDjxuZmeYWau774s+XIlV1i4kWft+JNPiLD3NAjoLHu8NjlV7DgBmtsLMNprZxu7u7poGKiIynsWZKKzEMR/FOfmD7ne7+2J3X9zc3Dzm4Mat8VoSEZGy4iw97QXmFDyeDXSN4hypJZVEkks3B5KYxDmi2ADMN7PzzGwScBPwYNE5DwL/PZj99GbgsPoTKTUw26kcXewqUyNYYhLbiMLd+8zsVuDnQCNwj7tvM7OVwfNrgHXAtcAu4CjwwbjilTEKm9XkJauJIpIQsW7h4e7ryCeDwmNrCt534C+jjksksXK58uUnjSykTrQyWyRNsrDoTlJHiUJEREIpUYiISCglComG1meIpJbuRyHRUKNVJLU0ohBJE43MJAYaUYikiUZmEgONKERA98kQCaFEIQJanyASQokC9GpS6kd/W5IBShSgV5PjXT0v2vrbkgxQohDRRVsklBKFiIiEUqIQCaP1CSJKFCKhtG5BRIkC0GpXKW+ss5b0tyUZoJXZoFeN413YvajHOmtJf1uSARpRSDrUcz2C7kUtEiqWEYWZfRH4Y+A48Hvgg+5+qMR5zwG9wEmgz90XRxmnJIjWI4jEJq4RxSPAJe5+KfBb4G9Dzr3C3S9TkhARiUcsicLdH3b3vuDh48DsOOIQEZHKktCj+BDwszLPOfCwmW0ysxVhn8TMVpjZRjPb2N3dXfMgZZzSrCWR+vUozOzfgXNKPHW7u/84OOd2oA+4t8ynWeruXWbWAjxiZs+4+6OlTnT3u4G7ARYvXuxj/gZEQA1tEeqYKNz9nWHPm9ly4DrgSncveWF3967g3wNmdj/QAZRMFJJxYVNYRaSuYik9mdky4BPA9e5+tMw5U82saeB94Grg6eiilETRFFaR2MTVo7gTaCJfTtpsZmsAzKzNzNYF58wEfm1mW4AngJ+6+0PxhCsiMn7Fso7C3S8sc7wLuDZ4fzewKMq4JENyufKlKo1CRKqShFlPIrWnBXoiNaNEISIioZQoREQklBKFiIiEUqIQEZFQShSSTdp6Q6RmdOMiySZNgRWpGY0oREQklBKFiIiEUqIQEZFQShQiIhJKiUJEREJZmVtBpJqZdQN74o6jCjOAg3EHMQqKOzppjBkUd9TGEve57t5c6olMJoq0MbON7r447jiqpbijk8aYQXFHrV5xq/QkIiKhlChERCSUEkUy3B13AKOkuKOTxphBcUetLnGrRyEiIqE0ohARkVBKFCIiEkqJIiHM7HNm9pSZbTazh82sLe6YRsLMvmhmzwSx329mZ8QdUyVm9j4z22Zm/WaW+CmQZrbMzHaa2S4z+2Tc8YyEmd1jZgfM7Om4Y6mGmc0xs1+a2Y7gb+RjccdUiZmdZmZPmNmWIObP1PxrqEeRDGaWc/ee4P2/AtrdfWXMYVVkZlcD/8/d+8zsHwHc/RMxhxXKzBYC/cBa4G/cfWPMIZVlZo3Ab4GrgL3ABuBmd98ea2AVmNnbgCPAd939krjjGX4WFXkAAAMpSURBVCkzawVa3f1JM2sCNgHvTvLP28wMmOruR8xsIvBr4GPu/nitvoZGFAkxkCQCU4FUZHB3f9jd+4KHjwOz44xnJNx9h7vvjDuOEeoAdrn7bnc/DvwAuCHmmCpy90eBl+OOo1ruvs/dnwze7wV2ALPijSqc5x0JHk4M3mp6/VCiSBAz+7yZdQJ/Dnwq7nhG4UPAz+IOImNmAZ0Fj/eS8AtXVpjZPOCNwPp4I6nMzBrNbDNwAHjE3WsasxJFhMzs383s6RJvNwC4++3uPge4F7g13mhPqRR3cM7tQB/52GM3kphTwkocS8VoM83MbBrwQ+DjRaP9RHL3k+5+GfkRfYeZ1bTcp1uhRsjd3znCU78H/BT4dB3DGbFKcZvZcuA64EpPSNOrip910u0F5hQ8ng10xRTLuBDU+X8I3OvuP4o7nmq4+yEz+w9gGVCziQQaUSSEmc0veHg98ExcsVTDzJYBnwCud/ejcceTQRuA+WZ2nplNAm4CHow5pswKGsPfAna4+1fijmckzKx5YLahmZ0OvJMaXz806ykhzOyHwALys3H2ACvd/YV4o6rMzHYBk4GXgkOPJ322lpn9CXAH0AwcAja7+7vijao8M7sW+BrQCNzj7p+POaSKzOz7wDvIb3u9H/i0u38r1qBGwMz+EPgVsJX8/0WAv3P3dfFFFc7MLgW+Q/7vowG4z90/W9OvoUQhIiJhVHoSEZFQShQiIhJKiUJEREIpUYiISCglChERCaVEIVIHwS6kz5rZWcHjM4PH55rZQ2Z2yMx+EnecIiOhRCFSB+7eCdwFfCE49AXgbnffA3wReH9csYlUS4lCpH6+CrzZzD4O/CHwZQB3/wXQG2dgItXQXk8ideLuJ8zsfwIPAVcH24SLpI5GFCL1dQ2wD0jNzXtEiilRiNSJmV1G/s50bwb+Orh7mkjqKFGI1EGwC+ld5O9n8Dz5BvaX4o1KZHSUKETq4yPA8+7+SPD4n4CLzOztZvYr4P8CV5rZXjNL7M61IqDdY0VEpAKNKEREJJQShYiIhFKiEBGRUEoUIiISSolCRERCKVGIiEgoJQoREQn1/wF6nC59vNoArAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "def plotBestFit(wei):\n",
    "    import matplotlib.pyplot as plt\n",
    "    weights = wei\n",
    "    dataMat, labelMat = loadDataSet()\n",
    "    dataArr = np.array(dataMat)\n",
    "    n = np.shape(dataArr)[0]\n",
    "    xcord1 = []; ycord1 = []\n",
    "    xcord2 = []; ycord2 = []\n",
    "    for i in range(n):\n",
    "        if int(labelMat[i]) == 1:\n",
    "            xcord1.append(dataArr[i, 1]); ycord1.append(dataArr[i, 2])\n",
    "        else:\n",
    "            xcord2.append(dataArr[i, 1]); ycord2.append(dataArr[i, 2])\n",
    "    fig = plt.figure()\n",
    "    ax = fig.add_subplot(111)\n",
    "    ax.scatter(xcord1, ycord1, s = 30, c = 'red', marker = 's')\n",
    "    ax.scatter(xcord2, ycord2, s = 30, c = 'green')\n",
    "    x = np.arange(-3.0, 3.0, 0.1)\n",
    "    y = (-weights[0] - weights[1] * x) / weights[2]\n",
    "    ax.plot(x, y)\n",
    "    plt.xlabel('X1'); plt.ylabel('Y1')\n",
    "    plt.show()\n",
    "\n",
    "weights = gradAscent(dataArr, labelMat)\n",
    "plotBestFit(weights)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 梯度上升算法每次更新回归系数时需要遍历整个数据集，对于样本较多的情况下并不适用。  \n",
    "\n",
    "改进：随机梯度上升算法—每次仅用一个样本点来更新回归系数  \n",
    "\n",
    "一次处理所有数据称为“批处理”"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[ 1.01702007  0.85914348 -0.36579921]\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYoAAAEGCAYAAAB7DNKzAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deXxU9dX48c8JYd/3JSTsBBHZjEFFRUQE6YK21uK+tKKtUGut2D4+XZ/lV4Ko1I3Fqu0jbn2qrY+y44JiQRZZRJIQ1oSwJAIhIZBtzu+PjDaEzGSbmXvvzHm/XrzI3Lkzc5JJ7pn7/Z7zvaKqGGOMMYHEOR2AMcYYd7NEYYwxJihLFMYYY4KyRGGMMSYoSxTGGGOCinc6gHDo0qWL9u3b1+kwjDHGMzZt2pSvql1rui8qE0Xfvn3ZuHGj02EYY4xniMj+QPfZ0JMxxpigLFEYY4wJKuyJQkReEJGjIvJ5lW2/FZGDIrLF/29KgMdOFpEMEckSkV+EO1ZjjDHnisQZxUvA5Bq2P6GqI/3/llS/U0SaAM8A1wJDgZtEZGhYIzXGGHOOsCcKVV0DHGvAQ1OBLFXdo6qlwGvA1JAGZ4wxplZOzlHMEJFt/qGpjjXcnwBkV7md499mjKmD7IJsZi6ZSeqiVGYumUl2QXbtDzKmBk4liueAAcBI4BAwt4Z9pIZtAZe6FZHpIrJRRDbm5eWFJkpjPCq7IJsR80ewYNMCNuRuYMGmBYyYP8KShWkQRxKFqh5R1QpV9QGLqBxmqi4HSKxyuzeQG+Q5F6pqiqqmdO1aY8+IMTEjbW0aRaVFlPnKACjzlVFUWkTa2jSHIzNe5EiiEJGeVW5eD3xew24bgEEi0k9EmgHTgLcjEZ8xXrf+4Pqvk8RXynxlfHrwU4ciMl4WifLYV4F/AskikiMiPwDSRGS7iGwDxgMP+vftJSJLAFS1HJgBLAd2Am+o6o5wx2tMNBiTMIamcU3P2tY0rimpCTWdvBsTnETjFe5SUlLUlvAwseyrOYqvhp+axjWlTbM2bL1vK4ntE2t/AhNzRGSTqqbUdJ91ZhsThRLbJ7L1vq3ce+G9pPZK5d4L77UkYRosKhcFNCZaZBdkk7Y2jfUH1zMmYQyzxs6q88E+sX0iT015KswRmlhgicIYl6o+fLTl8BYWb19sZwYm4mzoyRiXshJX4xZ2RmGMi1Qdatp/Yr+VuBpXsERhjEtUH2qKq+GE3y0lro2ZOzHeY4nCGJeoPtTkwweAICj6dYnrrLGznAzT5k5ikM1RGOMSNXVTA3Rt3dVVJa42dxJ77IzCGJcYkzCGLYe3nJUsmsY15cahN7qqzNWWB4k9dkZhjEvMGjuLNs3afL30hluGmqqz5UFijyUKY1zCK93UXkloJnRsrSdjTL19VfX06cFPSU1IDXnVk1VVRV6wtZ4sURhjXMUWNHSGLQpojPEMq6pyH0sUxhhXsaoq97FEYYypUXZBNjOXzCR1USozl8yM2PW2rarKfSxRGGPO8dU8wYJNC9iQu4EFmxYwYv6IiCSL6lVVguBTH4WlhY1+faeSn9dF4lKoL4jIURH5vMq2OSKSLiLbROQtEekQ4LH7/JdM3SIiNjttTIQ4OU/wVZnwzcNupok0QRAqtIJXtr/SqGTlZPLzukicUbwETK62bSUwTFWHA5nAL4M8fryqjgw0G2+MCa4hn6KdnidIbJ9I2+ZtiZO4r9e8amyysknyhgv7Eh6qukZE+lbbtqLKzXXADeGOw5hY1NAF/AItJxLJeYJQJyunk5+XuWGO4m5gaYD7FFghIptEZHoEYzImKvzqvV9x4syJen+KdkP3dagntW2SvOEcTRQi8ihQDiwOsMtYVR0NXAvcLyJXBHmu6SKyUUQ25uXlhSFaE2420Rha2QXZvLz9ZZSzm2rr8inaDcuJhDpZuSH5eVVEOrP9Q0/vqOqwKtvuAO4DJqhqcR2e47dAkao+Vtu+1pntPdaNG3ozl8zk2Q3Pfj3G/xVBuH347bRt3tb1S2SEeqmQcC894mWOL+FRPVGIyGTgcWCcqtb48V9EWgNxqlro/3ol8HtVXVbb61mi8J6ZS2ayYNOCc8bE773wXlctse0lqYtS2ZC74ZztccTRtnlbisuKLSmbrzm6hIeIvAr8E0gWkRwR+QHwNNAWWOkvfZ3v37eXiCzxP7Q78LGIbAU+Bd6tS5Iw3mQTjaFX05h8HHEM6jzo6yQBVv1jaheJqqebatj8pwD75gJT/F/vAUaEMTTjIm6osqkPL6xuOmvsLBZvX3zOcF6L+BaWlE29uKHqyRhPTTR6pXEr0IT05UmXW/WPqRdbZty4RqQmGrMLsvnVe79iadZSELh24LX8x/j/qPNreX0+xU2FA9XPzG4dfisvb3vZ1Wdq0crxyexIs0RhAskuyOaC5y6goKTgrO3tm7dn+4+21+mgFGiSOLVXKuvvWR+yWMPJDdU/1RNWvMRToRU0iWtCua/cJtkjzK5HYYxf2to0TpacPGd7YUlhnSdzo6FxK7F9Ik9NeYr196znqSlPOXIgrr6kRrmWoyjlvnLAJtndxBKFiQp1bdZbf3D9OQ1oAD58dZ7M9dJ8ipvVVOlWnU2yu4MlCuN59ZlcHpMwBkHO2R5HXJ3PCNzQtRwNajozq85rZ2rRyuYojOfVZ3I5FHMUJjRsjsJdgs1RhL2Pwphwq0+zXmL7RLb/aHvQqic390g0Nraqjx/aZSgIfJH3hSPf51dnZlUn1b+qerIlNtzFziiM54WyXNVNpaOhjq3646ty0/dpnGFVTyaqhXJy2c0Xt2lsbNUfX5Wbvk/jPpYojOeFcnI5HGtOhWr59MbGVluVUW3PZcvAxy6bozBR4au+gMYK9ZpTDb3CXDhiq+nxVQV7rlB+H8Z77IzCeFqoP+WGukcilENZjY2t+uOrqu253DwkZ8LPziiMZ4XjU25NlTiNqbxpzHBRTRVOjYmt+vd2XpfzQGBn3s5an8uWgY9tliiMZwX7lNuYYahgw1j1LU8d2nUom3I3nXWVuboMFwVLguH63oLx2jLwJrRs6Ml4VqQ/5dZ3efHsgmz+nv73cy5F2qppq1qHi9w21BMty5bYhHzDWKIwnhXpxfnqe/BOW5tGcdnZl4MXhOuSr6t1uMhtQz3RsGyJV64j4kaRuBTqCyJyVEQ+r7Ktk4isFJFd/v87BnjsZBHJEJEsEflFuGM13hLpT7n1PXjXtL+i7MzfWetruXGFWjesONsYbjtL85JInFG8BEyutu0XwGpVHQSs9t8+i4g0AZ4BrgWGAjeJyNDwhmq8JNKfcut78G7MwT5ahnrcxG1naV4S9kShqmuAY9U2TwX+7P/6z8B1NTw0FchS1T2qWgq85n+cMV+L5Kfc+h68G3Owj1QSjKUxezeepXlFRNZ6EpG+wDuqOsx/+4Sqdqhy/3FV7VjtMTcAk1X1h/7btwFjVHVGba9naz2ZcKnvleEicSW5hi4U6OZ1rcIh1r7f+nL8UqgNTBTfAyZVSxSpqjozwGtMB6YDJCUlXbh///5wfCvGuEpjDn5eufZ3KFfzdcMlYN3KjcuMHxGRnqp6SER6Akdr2CcHqPoO9gZyAz2hqi4EFkLlGUUogzW1c/PS3NEs0ATtN175Bi3iWwR9LwKN2b/xxRsRe/9q+70JdVNlqJZ6iTVOlce+Ddzh//oO4B817LMBGCQi/USkGTDN/zjjMlZ26JxAB/vtR7fX6Wp/NS3nkXcqLyLvX11+b6xSqWaqSkFx8MvIhlIkymNfBf4JJItIjoj8APgDMFFEdgET/bcRkV4isgRAVcuBGcByYCfwhqruCHe8pv7sj9k5tV1ONNB7kV2QTWFJIT71nfMYRSPy/tXl92bNgTVWqVTNluwT3LxoPdMWrcPni8zgSdiHnlT1pgB3Tahh31xgSpXbS4AlYQrNhIiVHTpn1thZLN6+OOB1JuDc96LqcE6FVtTpMeFQ2+9NdkE2GfkZ5zwuXuJjslJpd14Rjy3PYOnnh+ncuhkzrhqIT5W4Gq4BH2q21pNpNFsHyDnVF/o7XX6anXk7Kdfyr/ep/l4Eu4BRoMeEQ22/N2lr02pMZE3imsRUP8nhgjM8uSqTv27KoUV8HA9MGMQ9V/SnTfPIHb7tUqim0QJV3iy9ZSkvb3vZsxPcbpigr28MdamCSl2UyobcDQGfI1Jlo7XFGijOEd1HsOW+LWGLyy1OFJfy3Ie7eWntPnyq3DKmDzOuGkiXNs3D8nqOl8dGmiWKyKtednjr8Fu5dvG1nq1Zd0PNfUNjqK0ENFBZ7JAuQ2gZ3zKiZaPBYvVK+W6onS6t4MVP9jL/g90UlpRz/cgEHpw4mMROrcL6upYoTMR5/Y/cDfGHKwY3JMG68EqcoVJW4eONjdnMW7WLo4UlTBjSjZ9PSua8nu0i8vpu7KMwUc7rE9xuiD9cMYT64kzh4pU4G0tVWbL9MI+tyGBv/iku7NORp28eTWq/Tk6H9jVLFCYsvD7B7Yb46xpDQ+ZSvNJ45pU4G2ptVj6zl6WzLaeAwd3b8PztKUw4rxsi4a9kqg8bejJh4fVhAzfEX5cY3BCnqb/tOQXMXpbOx1n5JHRoyYMTB3P9qASaxDmXIGyOwjjC6+vquCH+hk5Me2UuKNbsySti7spM3t12iI6tmnL/+IHcenEfWjRt4nRoliiMiVaBSkhTe6Wy/p71DkRkanLk5Bnmrd7F6xuyaR4fxw8v7889l/ejbYvAXfV11q4dFBaeu71tWzh5ss5PY5PZxkSpkMylhOhAY85VUFzG/DW7eXHtXip8yq1jkphx1SC6tg1hL0RN712w7Q1gicIYD6u+hEeDroQXgQNNrDlTVsFLn+zjuQ92c/JMGVNH9OJnE5NJ6hzeXohwsURhjIfFSgmpV5RX+PjrphyeXJXJkZMlXJnclVmThjC0V2R6IcLFEoUxYRDJ5T+ivYTUC1SVZZ8fZs6KDPbknWJUUgfmTRvFxf07Ox1aSFiiMCbEQn2xHeNun+zOZ/ayDLZmn2BgtzYsuO1Crhna3XW9EI1hicKYEAt2nQX75B89Pj9YQNryDNZk5tGzfQvSvjuc74xOIL5JhK8H17Zt4GKEELFEYUyIuWH5j3qJwIHGUSGu6tqXf4rHVmTwzrZDdGjVlEennMdtlzjYCxGByjRLFMaEmBuW/6iXaC+BDVFV19EqvRBNm8QxY/xApo/rT7tQ9EK4nGOJQkSSgderbOoP/FpVn6yyz5VUXk97r3/Tm6r6+4gFaUwDhKRkNQq44XoeoXDyTBkLPtzNCx/vo6zCx7TURH5y1SC6tWvhdGgR44rObBFpAhwExqjq/irbrwR+rqrfrM/zWWe2cZoblv9wkqvWoAo2qRzk+HemrIK//HMfz36wmxPFZXxrRC8emjiYvl1ahz5GF/BCZ/YEYHfVJGGMl8V6yaqXJ/TLK3y8ufkgT6zK5FDBGa4Y3JVZk5IZltDe6dAc45ZEMQ14NcB9l4jIViCXyrOLHZELyxjTEJ6b0KeyF2L5jiM8tiKDrKNFjEjswNwbR3DpgC5Oh+Y4xxOFiDQDvg38soa7NwN9VLVIRKYAfwcGBXie6cB0gKSkpDBFa5wWLePe0c5VE/p1qOpat+dLZi9L57MDJ+jftTXzbx3NpPN7RFUvRGM4PkchIlOB+1X1mjrsuw9IUdX8YPvZHEV0ctW4t9dEeOE/r7xXO3ILmLM8gw8y8ujRrgU/vXoQN1zYO/K9EC7g9jmKmwgw7CQiPYAjqqoikgrEAV9GMjjjHl4e93ZchBf+c/saVAe+LGbuygz+sSWX9i2b8strh3DHpX1dcV0IN3I0UYhIK2AicG+VbfcBqOp84AbgRyJSDpwGpqnTp0BeEmXLR3tx3Dvk3PKe1iEON07o5xWW8NR7u3hl/QHimwg/unIA940bQPuW0d8L0RiOJgpVLQY6V9s2v8rXTwNPRzquqBFly0e7atzbKW55T90SRx0Vnilj0Zo9PP/xXkrKfXz/okQemDCI7jHUC9EYbhh6MqZOrJHN1NeZsgpeXrefZ97P4nhxGd8Y3pOHJg6mf9c2TofmKZYojGe4fdzbuEeFT3lzcw5PrtrFwROnuXxQFx6elMzw3h2cDs2TLFEYT4nUuHfUleFG+8J/fqrKqp1HmbM8ncwjRQzv3Z60G4YzdqALeiHcMr/UAJYojKkmKq8n4fIDUSh8uvcYs5els2n/cfp1ac0zN49mygUu6oXw2LxOVbFXLBxLAn1ajLJPkaEWrAzXcW55T90SB7Dz0EnufmkDNy74JznHi/nv6y9gxYNX8I3hPd2TJDzOziiiWQx8igwHV5fhuuU9dUEc2ceKeXxlJn/fcpC2zeN5ZPIQ7ry0Ly2bWS9EqFmiMK7hlnkBV5Theng8+2th+h7yi0p4+r0sFq/fT5wI06/oz4/HDaR9K+uFCBfHl/AIB1vCw3vctOSDK2Jp4NLYrhLi76HwTBmLPtrL8x/toaTcx40pvXlgwmB6tPdIL4TL31O3L+FhjKuW5/BkGW40nIEEUFJeweJ1B3j6/SyOnSplygU9eOiaZAZ4rRfCw5VnliiMK7htXsCNy08E5eGKmkAqfMrfPzvI4yszOXjiNJcO6Mwjk4cwItGjvRAeTtiWKIwruGJeoD6i+BN8SLRr1+CHqirvpR8lbVkGGUcKGZbQjv/3nQu4fFAXq2JyiCUK4wqeW54jCj/Bh1QDfw4b9x0jbVkGn+47Rt/OrXj65lFMGdaTuDhLEE6yRGFcwZPzAuHk4fHsWtXwPWQcLmTO8nRW7TxKt7bN+c/rhvH9ixJpGoPXhXAjq3oypiHcVsHi0Xhyjlf2Qrz12UHaNI/nvnEDuHtsv9D1QtgQYZ2FvOpJRO5S1RcbF5YxJmQ8dgbyZVEJT7+fxeJ1B0Dgnsv78+MrB9ChVbPQvpANEYZEQ4eefgdYojDGLTzy6fhU0xY8v2oXiz7aQ3FpOd+7MJGfThxEz/YtnQ7NBBEwUYjItkB3Ad3DE44xHuGxT/ARV+3nUxoXz6sjJ/PU2JvIX5XJ5PN78PNJgxnYzX5eXhDsjKI7MAk4Xm27AJ+E4sVFZB9QCFQA5dXHx6SyFm4eMAUoBu5U1c2heG1jGsUjn+Ad4//5+HzK21tzmbsyg+xjp7m4fycWTR7CqKSODXveSM052NzGWYIlineANqq6pfodIvJBCGMYr6r5Ae67Fhjk/zcGeM7/vzHGxVSVDzLymL0snfTDhQzt2Y4/330BV3zVC9HQA3Gk5hxsbuMswRJFL+BgTXeo6s3hCeccU4G/aGVp1joR6SAiPVX1UIRe3xhTT5v2H2f2snQ+3XuMpE6tmDdtJN8a3uvsXohIHYhtiDAkgiWKF4HlIvJnIE1Vy4Ls21AKrBARBRao6sJq9ycA2VVu5/i3WaIw7hdjwxeZRwqZszyDlV8coUub5vx+6vlMuyiJZvEO9kJE4c/ZCQEThaq+ISLvAr8GNorI/wC+Kvc/HoLXH6uquSLSDVgpIumquqbK/TUVY9dYFC4i04HpAElJSSEIzbiClw+2MTJ8cfDEaZ5Ymcmbm3No3SyehyYO5u7L+tG6ufXzRova3sky4BTQHGhLlUQRCqqa6///qIi8BaQCVRNFDlC1Nbc3kBvguRYCC6Gy4S6UcRoHxcjB1ouOnSrl2fez+Mu6/QDcPbYfPx4/kE6tQ9wLYRwXrDx2MvA48DYwWlWLQ/nCItIaiFPVQv/X1wC/r7bb28AMEXmNyknsApufMMZZxaXlvPDxXhZ8uIdTpeV8d3RvfjpxMAkdItALEak5B5vbOEuwM4pHge+p6o4wvXZ34C3/apDxwCuqukxE7gNQ1fnAEipLY7OoLI+9K0yxmFjl5aGtCCst9/H6hgPMW51FflEJE4d2Z9akZAZ1b8DBs6EH4ki9J/ben8XWejLuFu41jML5/G5bf6mBfD7l/7blMndFJgeOFZParxOPTB7ChX0a2AthXMmucGeMEzw+fKGqfJiZR9qyDL44dJIhPdry4p0XcWVyV7suRIyxRGHczcmDbbt2jRuC8PDwxeYDx0lbls66PcdI7NSSJ74/gqkjEuy6EDHKEoVxNycPto2prPLo3EfW0cpeiOU7jtClTTN+9+3zuSnV4V4I4zhLFMaEg8fKenNPnObJVZn876YcWjWL58GrB/PDy60XwlSy3wIT2wINbcWIE8WlPPvBbl76ZB8o3HlpP+4fP4DObZo7HZpxEUsUJradPBm8OilKFZeW8+Lafcz/cDdFJeVcPyqBB68eTGKnVk6HZlzIEoUxMaSswsfrG7KZt3oXeYUlXH1eN34+KZkhPdo5HZpxMUsUxkSjapPpPoR3h1zG3HG3s69DT1L6dOTZW0ZzUd9ODgZpvMIShTHh4HQPRZXX/qjvSGaPu5PPewwkOW8fz9+ewoTzulkvhKkzSxTGhIrLSmK39RjI7HF3srbvSBIKjvD4O3OZ+sWHNPnT/RGPxXibJQpjQvXp3yUlsbvzipg79RcsGXIZnU+d4DerFnDzlqU0ryiPaBye4rIk7zaWKIyJkgPB4YIzzFudyRsbc2jRbzQPfPwK92x4izalp50O7WxuPCi7JMm7lSUKYzyuoLiMZz/M4qW1+/CpctvFfZhx01i6FBc4HVrN7KDsOZYojPGo06UVvPjJXuZ/sJvCknKuG5nAzyb6eyGaBLjGmEcWJDTuYonCGKfVcyimrMLHXzfmMG91JkdOlnDVkG48PCmZ83pW6YWIkuE04w6WKIwJlYZOitdxKEZVWbL9MHNXZLAn/xSjkzrw1E2jSe1nvRAmvCxRmJq5ccLR7cLxc/H3OqztM4LZ4+5gW8/BDO7ehkW3p3C1U70Q0fi74XTfi8s5lihEJBH4C9AD8AELVXVetX2uBP4B7PVvelNVq19X24SDTTi6wvbuA0gbdwcf9RtNQsFRHnv3ca7fuoom4b4uRLBk0NjfDTcelL2a4CLEyTOKcuAhVd0sIm2BTSKyUlW/qLbfR6r6TQfiM8Yxezr2Yu7lt/LueVfQsbiAf1+9iNs+e7eyFyISFw8K5wcFOyh7jmOJQlUPAYf8XxeKyE4gAaieKIyJGUfadGLepTfx+ohraF5eyk/Wvso9n75JW7f1QpiY4oo5ChHpC4wC1tdw9yUishXIBX6uqjsiGJox4de2LQWlPuaPuYEXU75FeVw8t3y2lJmfvEbX4hNOR2eM84lCRNoAfwN+qqrVz0k3A31UtUhEpgB/BwYFeJ7pwHSApKSkMEZsTOicKavgz//3Gc9+sJuC02VMHdmLhyYmk9TlOqdDc0Y0TpRHAVFV515cpCnwDrBcVR+vw/77gBRVzQ+2X0pKim7cuDE0QcYq+4MNq/IKH/+7KYcnV+3i8MkzXJnclYcnJXN+r/aVOzj98w9WTRVsMrqxsQV7XQePVbFARDapakpN9zlZ9STAn4CdgZKEiPQAjqiqikgqEAd8GcEwY5clg7BQVZbvOMyc5RnszjvFyMQOPPH9kVwyoPPZO4b7519bIgpnMjCe4+TQ01jgNmC7iGzxb/s3IAlAVecDNwA/EpFy4DQwTZ08BTKmET7ZnU/asgy2ZJ9gQNfWzL/1Qiad392ZXojaqposGZgqnKx6+hgI+heiqk8DT0cmImPC4/ODBaQtz2BNZh4927dg9ncv4LujexPfJM7p0IypE8cns40JOafH9/325Z9i7spM/m9rLu1bNuXfpgzh9kv60qJpk4jFYEwoWKIw0cfhrvKjhWf44+pdvPZpNk2bxHH/+AFMv2IA7Vs2jcjrOyJUydmNXdvGEoUxoXLyTBkLP9zDnz7eS1mFj2mpifzkqkF0a9fC6dDCL1TJ2eZGXMkShTGNdKasgpfX7eeZ97M4XlzGt0b04qGJg+nbpbXToQVmn9xNPViiMKaByit8vPnZQZ5cmUluwRkuH9SFRyYPYVhCe6dDq519cjf1YInCmGBqGHtXYMXw8cy54zdkHS1iRGIHHrtxBJcO6OJMjMaEmSUKE31COaxS7XnWJQ5j9rg7+SxhCP1Vee6W0Uwe1sOZXghjIsQShYk+YRhW2dGtH2nj7uDD/in0KMznD0v/yA2bl1ovxFdsziOqWaIwJoj9HXrw+GW38o/zr6T96UJ++f4L3LH5HVqUl4JTScIlfSJnsTmPqGaJwpga5BWW8NR7u3jlh/OJ95Xzo3/+lfvW/y/tS045HZrjfSIm9liiMKaKwjNlLFqzh+c/3ktJuY/vb1vJA5+8SveiY06HZoxjLFEYA5SUV/A///xXL8Q3LujJQ9cMpv+AaVBkY+8mtlmiMDGtwqe89dlBnliZycETp7lsYBdmTU5meO8OlTvY2HtouXF+xdTKEoXxhhAfYFSVVTuPMmd5OplHirggoT2zvzucywZFqBciVg+YNr/iSZYojDeE8ACzYd8xZi9NZ+P+4/Tr0pqnbx7FlGE9iYuLYC9EY74fK0U1EWaJwsSM9MMnmbMsg9XpR+nWtjn/ff0FfC+lN0291gsRzWccxpUsUZizReGQSPaxYp5YmclbWw7Stnk8syYnc9el/WjZzK4LYUxdOJooRGQyMA9oAjyvqn+odr/4758CFAN3qurmiAcaS6JoDDm/qISn38ti8fr9xIkw/Yr+/HjcQNq38uh1IaIwiRtvcCxRiEgT4BlgIpADbBCRt1X1iyq7XQsM8v8bAzzn/994VQQOdkUl5ZW9EB/t4Uy5jxtTevOTCYPo2b5lSJ7fMdGQxG1+xZOcPKNIBbJUdQ+AiLwGTAWqJoqpwF9UVYF1ItJBRHqq6qHIh2tCoqEHuzocYErKK1i87gBPv5/FsVOlXDusBw9dk8zAbm0aEXCYxOoB0858PMnJRJEAZFe5ncO5Zws17ZMAWKKINUEOMBU+5R9bDvL4ykxyjp/mkv6deeTaIYxM7BDBAOvJDpjGQ5xMFDXVImoD9qncUWQ6MB0gKSmpcZEZT1BV3ks/ypzlGaQfLuT8Xu347+sv4PJBXWzZb2NCyMlEkQMkVrndG8htwD4AqOpCYCFASkpKjcnE1IFHhkQ27T/GH5ams2Hfcfp2bsVTN43iGxdEuBfCmBjhZKLYAAwSkX7AQWAacHO1fd4GZvjnL8YABTY/EcXU0PYAAAyKSURBVGbhGhIJNIldTxmHC5mzPINVO4/QtW1z/vO6YXz/okTv9UI0hEeSuIk+jiUKVS0XkRnAcirLY19Q1R0icp///vnAEipLY7OoLI+9y6l4TSPVZbI6iJzjxTyxchdvfpZDm+bxPDwpmbvG9qVVsxhqBbJ5DeMQR//KVHUJlcmg6rb5Vb5W4P5Ix2UiTAOPFH5ZVMIz7+/m5XX7QeCey/vzo3ED6Ni6WQQDdBHrpTAOiKGPY8ZLTpWU8/xHe1n00R6KS8u54cLe/PTqwfTq4PFeiMaKhl4K4zmWKIyrlJb7ePXTAzz13i7yi0qZdH53Hp6UzMBuNg5vjFMsURhX8PmUt7fmMndlBtnHTnNx/04sun0Io5I6Oh2aMTHPEoWJjAAVO9q2LR9kHCVtWQY7D53kvJ7teOmuYYwb3NV6IYxxCUsUJjJqmGjdfOA4s5ems/7FDSR1asW8aSP51vBe1gthjMtYojARt+tIZS/Eii+O0KVNM3737fO5KTWJZvEx0AvRWNZLYRxgicJEzMETp3lyZSZ/25xDq2bxPDRxMHdf1o/Wze3XsM6sBNY4wP5CTdgdP1XKM+9n8Zd1+0HhrrH9uH/8QDq5qRfC+hOMCcgSBdhBIkyKS8t54eO9LPhwD6dKy/nO6N48OHEwCW7shbD+BGMCskQBdpAIsbIKH699eoB5q7PILyph4tDKXojB3WNwHN0+hJgoYInChIzPp7yz/RBzV2Sw/8tiUvt2YsFto7mwTyenQwuuXbvwPbd9CDFRwBKFaTRVZc2ufNKWpbMj9yRDerTlxTsv4spkj/RC2EHbmKAsUZhG+ezAcWYvS2fdnmP07tiSJ74/gqkjEqwXwpgoYonCNEjW0SLmLE9n+Y4jdG7djN9+ayg3jUmieXwTp0MLLetPMMYSBWBNTPWQe+I081bt4q+bsmnZtAkPXj2YH1zejzbR3AtR0/CZTUabGBLFf931YH/wtTpRXMqzH+zmpU/2gcIdl/ZlxviBdG7T3OnQwquxk9H2IcREAUsUJqjTpRW8sHYv8z/cTVFJOdePSuDBqweT2KmV06GFTrCDeWMnuu1DiIkCjiQKEZkDfAsoBXYDd6nqiRr22wcUAhVAuaqmRDLOWFZW4eP1Ddn8cfUujhaWMGFINx6enMyQHmEsJQ0mnP0IwR7vhaotY8LMqTOKlcAv/dfNng38EngkwL7jVTU/cqHFNp9PWfL5IeauyGRv/ilS+nTkmVtGc1Ffh3shrB/BGMc4kihUdUWVm+uAG5yIw5zto115pC3LYPvBApK7t+X521OYcF43b/RCGGPCxg1zFHcDrwe4T4EVIqLAAlVdGLmwYse2nBPMXpbO2qwvSejQkrnfG8F1oxJoYr0QNhltDGFMFCKyCuhRw12Pquo//Ps8CpQDiwM8zVhVzRWRbsBKEUlX1TUBXm86MB0gKSmp0fHHgj15Rcxdkcm72w/RqXUzfv3NodxycRT2QjSGTUYbE75EoapXB7tfRO4AvglMUFUN8By5/v+PishbQCpQY6Lwn20sBEhJSanx+UylwwVnmLd6F29szKZFfBwPTBjEPVf0j+5eCGNMgzlV9TSZysnrcapaHGCf1kCcqhb6v74G+H0Ew4w6BcVlPPfhbl5cuxefKrdd3IcZVw2kixd6IWwIyBjHOPUR8mmgOZXDSQDrVPU+EekFPK+qU4DuwFv+++OBV1R1mUPxetrp0gpe+mQfz32QRWFJOVNH9OJnE5NJ6uyhXggbAjLGMU5VPQ0MsD0XmOL/eg8wIpJxRZvyCh9vbMxh3upMjpwsYXxyVx6eNIShvRzqhYgkuw6EMSFjg9JRSFVZ+vlhHluewZ78U4xO6sAfp41iTP/OTocWOdZ3YUzIWKKIMmuz8pm9LJ1tOQUM6taGhbddyMSh3a0XwhjTYJYoosT2nALSlqfz0a58erVvwZwbhvOd0b2tF8IY02iWKDxub/4pHluRwbvbDtGxVVP+/RvncevFfWjR1HohjDGhYYnCo46ePMOTq3fx+oZsmsfH8ZOrBvLDK/rTrkVTp0MzxkQZSxQeU3C6jAUf7uaFtXspr1BuGZPEzKsG0bWtB3ohIsn6LowJGUsUHnGmrII/f7KPZz/YTcHpMqaO7MXPJg6mT+fWTofmTlYCa0zIWKJwufIKH3/bnMOTq3ZxqOAM4wZ3ZdbkZM7v1d7p0IwxMcIShUupKst3HGbO8gx2551iZGIHHr9xJJcMiKFeCGOMK1iicKFPducze1kGW7NPMKBra+bfeiGTzrdeCGOMMyxRuMjnBwtIW57Bmsw8erZvQdp3h/Od0QnEN4lzOjRjTAyzROEC+788xdwVmby9NZcOrZry6JTzuO0S64UwxriDJQqH/XH1Lv64ehdNm8Rx//gBTL9iAO1bWi+EMcY9LFE4LLFTS6alJvKTqwbRrV0Lp8MxxphzWKJw2PWjenP9qN5Oh2GMMQHZLKkxxpigLFEYY4wJypFEISK/FZGDIrLF/29KgP0mi0iGiGSJyC8iHacxxhhn5yieUNXHAt0pIk2AZ4CJQA6wQUTeVtUvIhWgMcYYdw89pQJZqrpHVUuB14CpDsdkjDExx8lEMUNEtonICyLSsYb7E4DsKrdz/NuMMcZEUNgShYisEpHPa/g3FXgOGACMBA4Bc2t6ihq2aZDXmy4iG0VkY15eXki+B2OMMWGco1DVq+uyn4gsAt6p4a4cILHK7d5AbpDXWwgsBEhJSQmYUIwxxtSPI5PZItJTVQ/5b14PfF7DbhuAQSLSDzgITANursvzb9q0KV9E9ock2MjoAuQ7HUQDWNyR48WYweKOtMbE3SfQHU5VPaWJyEgqh5L2AfcCiEgv4HlVnaKq5SIyA1gONAFeUNUddXlyVe0anrDDQ0Q2qmqK03HUl8UdOV6MGSzuSAtX3I4kClW9LcD2XGBKldtLgCWRissYY8y53Fwea4wxxgUsUbjDQqcDaCCLO3K8GDNY3JEWlrhF1QqEjDHGBGZnFMYYY4KyRGGMMSYoSxQuISL/4V/SZIuIrPCXCrueiMwRkXR/7G+JSAenY6qNiHxPRHaIiE9EXF8C6cVVlP1L8xwVkZp6pFxLRBJF5H0R2en/HXnA6ZhqIyItRORTEdnqj/l3IX8Nm6NwBxFpp6on/V//BBiqqvc5HFatROQa4D1/38tsAFV9xOGwghKR8wAfsAD4uapudDikgPyrKGdSZRVl4Ca3r6IsIlcARcBfVHWY0/HUlYj0BHqq6mYRaQtsAq5z889bRARorapFItIU+Bh4QFXXheo17IzCJb5KEn6tCbKulZuo6gpVLfffXEflUiuupqo7VTXD6TjqyJOrKKvqGuCY03HUl6oeUtXN/q8LgZ24fDFSrVTkv9nU/y+kxw9LFC4iIv8lItnALcCvnY6nAe4GljodRJSxVZQdIiJ9gVHAemcjqZ2INBGRLcBRYKWqhjRmSxQRVMuKuqjqo6qaCCwGZjgb7b/UFrd/n0eBcipjd1xdYvaIeq2ibEJDRNoAfwN+Wu1s35VUtUJVR1J5Rp8qIiEd7nPyCncxp64r6gKvAO8CvwljOHVWW9wicgfwTWCCumTSqx4/a7er1yrKpvH84/x/Axar6ptOx1MfqnpCRD4AJlPzYqsNYmcULiEig6rc/DaQ7lQs9SEik4FHgG+rarHT8UShr1dRFpFmVK6i/LbDMUUt/8Twn4Cdqvq40/HUhYh0/araUERaAlcT4uOHVT25hIj8DUimshpnP3Cfqh50NqraiUgW0Bz40r9pndurtUTkeuApoCtwAtiiqpOcjSowEZkCPMm/VlH+L4dDqpWIvApcSeWy10eA36jqnxwNqg5E5DLgI2A7lX+LAP/mX6DUlURkOPBnKn8/4oA3VPX3IX0NSxTGGGOCsaEnY4wxQVmiMMYYE5QlCmOMMUFZojDGGBOUJQpjjDFBWaIwJgz8q5DuFZFO/tsd/bf7iMgyETkhIu84HacxdWGJwpgwUNVs4DngD/5NfwAWqup+YA5wm1OxGVNfliiMCZ8ngItF5KfAZcBcAFVdDRQ6GZgx9WFrPRkTJqpaJiIPA8uAa/zLhBvjOXZGYUx4XQscAjxz8R5jqrNEYUyYiMhIKq9MdzHwoP/qacZ4jiUKY8LAvwrpc1Rez+AAlRPYjzkblTENY4nCmPC4Bzigqiv9t58FhojIOBH5CPgrMEFEckTEtSvXGgO2eqwxxpha2BmFMcaYoCxRGGOMCcoShTHGmKAsURhjjAnKEoUxxpigLFEYY4wJyhKFMcaYoP4/wYgbSRLptvgAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "def stocGradAscent0(dataMatrix, classLabels):\n",
    "    m, n = np.shape(dataMatrix)\n",
    "    alpha = 0.01\n",
    "    weights = np.ones(n)\n",
    "    for i in range(m):\n",
    "        h = sigmoid(sum(dataMatrix[i] *weights))\n",
    "        error = classLabels[i] - h\n",
    "        weights += alpha * error * dataMatrix[i]\n",
    "    return weights\n",
    "\n",
    "dataArr, labelMat = loadDataSet()\n",
    "weights = stocGradAscent0(np.array(dataArr), labelMat)\n",
    "print(weights)\n",
    "plotBestFit(weights)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 对比分析\n",
    "表面上看随机梯度上升方法不如批量梯度上升方法好，但这是因为后者在全部数据集上进行迭代，对比不公平  \n",
    "\n",
    "另外根据下图发现，随机梯度上升算法的参数X0，X1收敛较慢，并且还会存在波动，针对这种现象，我们改进算法  \n",
    "![title.png](./picture/05随机梯度上升参数收敛.png)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[13.73692487  1.33658039 -1.96914719]\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYoAAAEGCAYAAAB7DNKzAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3df3DddZ3v8ec7aVpok5YfTUJpEwokoBUpSifVi6w4KJYuC64/GLh33IqupY5wdedqcWXGX3ec8VbXXS+4tHhl0Tuoy66CXK1KdXcHRClt2ZaCBRugJW2wTQu0DYU2P973j3OSJifnfE9O8j3fX3k9ZjI553u+J3mf/Pi+z+fz/vwwd0dERKSUmrgDEBGRZFOiEBGRQEoUIiISSIlCREQCKVGIiEigaXEHUA1z5871hQsXxh2GiEhqbNmy5YC7NxZ7LJOJYuHChWzevDnuMEREUsPMdpd6TF1PIiISSIlCREQCVT1RmNldZrbfzJ4ccexLZrbXzLbmP5aXeO4yM3vGzDrN7HPVjlVERMaKokVxN7CsyPG/d/eL8h/rCx80s1rg28CVwCLgejNbVNVIRURkjKonCnd/CHhpAk/tADrd/Tl3Pw78CLgm1OBERKSsOGsUN5nZE/muqVOLPD4f6Bpxf0/+WFFmttLMNpvZ5p6enrBjFUmdrkNd3Lz+Zjq+08HN62+m61BX+SeJFBFXorgDOBe4CHgR+Lsi51iRYyWXunX3O919ibsvaWwsOhRYZMroOtTF4rWLWbdlHZu6N7FuyzoWr12sZCETEkuicPd97j7g7oPAd8h1MxXaA7SMuL8A6I4iPpG0W/PIGnqP99I32AdA32Afvcd7WfPImpgjkzSKJVGY2bwRd/8SeLLIaZuAdjM728ymA9cBD0QRn0jabdy7cThJDOkb7OOxvY/FFJGkWRTDY38I/B4438z2mNnHgDVmtt3MngDeBfxN/twzzWw9gLv3AzcBvwJ2APe6+1PVjlckC5bOX0pdTd2oY3U1dXTML9Z4FwlmWdzhbsmSJa4lPGQqG6pRDHU/1dXUUT+9nm2rttEyp6X8F5Apx8y2uPuSYo9pZrZIBrXMaWHbqm3cePGNdJzZwY0X36gkIROWyUUBRSSXLG5bflvcYUgGqEUhkmCaCyFJoBaFSEIV1hm2/mkr92y/R11IEjm1KEQSSnMhJCnUohBJkK5DXax5ZA0b925k9yu7EzsXYmScS+cvZfUlq9XKyTAlCpGEKOxqqinS4E/CXAh1iU096noSSYjCrqZBBgGw/LJnQ3MhVl+yOrYYQV1iU5FaFCIJUWzZDYDGWY0snLOQjvkdieji0fIgU48ShUhCLJ2/lK1/2jrqIlxXU8e1i65N1HyIUnHG3SUm1aOuJ5GEWH3Jauqn1w+v0ZSUrqZCaYlTwqNEIZIQaVl2I4o4NdEwWbQooIgkihY0jIcWBRSR1NCoquRRohCRRNGoquRRohCRRNGmS8mjRCEiRcVVUNaoquRRMVtExoi7oDy0ltRjex8LdaKh1qgqTcVsEalI3AXloU2X/vXafwXgA/d+YNKtmqHkt27LOjZ1b2LdlnUsXrtYQ2/HoeqJwszuMrP9ZvbkiGNfN7OnzewJM7vPzE4p8dxdZrbdzLaamZoIIhFJQkE57At73MkvzaJoUdwNLCs4tgG4wN0vBP4I/G3A89/l7heVahKJSLCJ1BqSUFAO+8KehOSXVlVPFO7+EPBSwbEH3b0/f/dRYEG14xCZijbu2Uj7be3cvul2NnVvYu3mteN6V56EgnLYF/YkJL+0SkKN4qPAL0o85sCDZrbFzFYGfREzW2lmm81sc09PT+hBiqRN16Eu3nn3Ozk2cGz4WL/3c+TYkbLvypOwnEjYF/YkJL+0imTUk5ktBH7m7hcUHL8VWAK834sEYmZnunu3mTWR6666Od9CCaRRT+mkESnhunn9zdy+6faij3Wc2cHGj2+MOKLKVGPkVbVGU2VB0Kin2BKFma0AVgGXu/vRcXyNLwG97v6NcucqUaRP3MMxs6jjOx1s6t5U9LEVF66gYUZD4pOyLuzRCUoUsexHYWbLgFuAd5ZKEmY2C6hx9yP521cAX4kwTIlQUOEySXsxpMnS+Uv5zz/9J/2D/aOOT6+Zzv3P3M/RvqOJ38p0aJisxCuK4bE/BH4PnG9me8zsY8DtQAOwIT/0dW3+3DPNbH3+qc3Ab81sG/AY8HN3/2W145V4aERK+FZfspqG6Q1MqznxfnBG7QyuPv/q4SQBGiYq5VW9ReHu1xc5/N0S53YDy/O3nwMWVzE0SZC07ZqWhnrKUEG6sOvmA/d+QElZKqKtUCURVl+ymnu23zOmRpHEESmF9ZS0dd2kLSlL/JIwPFYk0uGYXYe6+Mh9H6H56800f6OZj9z/kYpm+6Z9hm+ShokWTgbcuGejdrZLIC0KKFNK16Eu3nzHmzl07NCo43NmzGH7J7aPKzGVGk2UhiGnQ5IwmqiwZTbNpjHgA9TW1NI/2K+RbxHTooAieWseWcPhY4fHHB/PJLQhWZjhO9QltfHjG7lt+W2xXIgLW2b93o/jw6O00tZSyzIlCsmE8a5ntHHvRpyxrehBBsddzE1S102aFRvpVkhF9mRQopDUq2SV0aXzl2LYmOM11Iy7RZCE5S2yoFjLrFDaWmpZpRqFpN7N629m3ZZ1Y0bx3HjxjWNG/IRRo5BwqEaRLKpRSKZVMlmvZU4L2z+xnRUXrqBpZhNNs5pYsXjFqCQR1xag4zHZ2JL02gpbZquWrOL3H/s9qy5epZZawqhFIalXSYuinCSvOTXZ2Iq9g6+tqeUNc9/Apa2XJnLSoERHLQrJtDCLy0meIzHZ2IqNMjo2cIxt+7ZpW1AJpEQhqRdmcTnJa05NNragUUZJSoiSPFrCQzIhrFVGq7G8RVjrQk02tmLPH6lc0knD+lZSHapRSKqFffEKu0YR5tcLu0ZRKKiuk+TajYRDNQrJpErmT4xX2HMkwqx5TDa2kc9f3LyYGbUzhpcgL1fXSXLtRqpPXU+SWtXa7CjMzXImU1co1VoK67VVst5Tkms3Un1KFJJacVy8Ku3qWtS4iC3dWxhkcPjYeOoKUSxlXknSycrS5KqzTIy6niS1ol6cr9Kurq5DXdz/9P2jkgTAzLqZZYfuJq2rJwvrW1Wjq3KqUKKQ1Ir64lXpxXvNI2s42jd6S3jDeN/57yv7LjZpXT1ZWN8qack3Tare9WRmdwFXAfvd/YL8sdOAfwYWAruAa9395SLPXQZ8C6gF/o+7f63a8Up6lNrqs1oXr0ov3sXOd5wdB3aU/V5J7OoJs3YTh6Ql3zSJokVxN7Cs4NjngN+4ezvwm/z9UcysFvg2cCWwCLjezBZVN1RJmyj3Vai0q2syXWNZ6OpJmizsIxKXqicKd38IeKng8DXA9/K3vwe8r8hTO4BOd3/O3Y8DP8o/TyQWlV68J3Oxj6qrJ0mLBFabku/ERTLhzswWAj8b0fX0irufMuLxl9391ILnfBBY5u5/nb//YWCpu99U7vtpwp1US6VbiEax5ehER/JMxUl0SdgCNqmCJtwlOVF8CHhvQaLocPebS3yPlcBKgNbW1ot3795djZcikiiTudiHuequpF8SZ2bvM7N5APnP+4ucswcY+Ze+AOgu9QXd/U53X+LuSxobG0MNViSpJjOSp1Rx994/3BtZF9R4ur6mUvdYUsWVKB4AVuRvrwB+WuScTUC7mZ1tZtOB6/LPkwTSP3M8gi725X4XpbYi7Xm1J5L5BeOZ16C5D8lQ9URhZj8Efg+cb2Z7zOxjwNeA95jZTuA9+fuY2Zlmth7A3fuBm4BfATuAe939qWrHK5XTP3N8gi725X4XQ8XdmoLLgOORzC8YT2tIcx+SIYpRT9e7+zx3r3P3Be7+XXc/6O6Xu3t7/vNL+XO73X35iOeud/fz3P1cd/9qtWOVidE/c3wKR/IMXfSdXO2x1O9iqKjbMqeF6dOmj/m6UcwvGM+8hodeeEhzHxJAaz3JpGkiU3wKJx3uemUX+4+OLvkV/i4KC+CFLQqIZn5BuUmFXYe6eObAM2OeN82mae5DxLSEh0xaqYlMb2x8Y6rrFmmpu4ycdHjtm64tO6mssAU4tBaVYcPnRzG/oNy8hjWPrGHAB8Y8r7amVnMfIqaNi2TSig3RnFk3E4CjfUdTOUY/KXMMKp0jMZ64O77TwabuTWOe2zSriYVzFkY6vyBoXkOpOBc3L2brqq1Vj22qCRoeq64nmbRiay4dOXaEHzz5g9D3iohKtfa6qMRElhofz/pXpbp8rl10beS/m6D1o0rFeWnrpVGFJ3lKFBKKwn/4ju90pLpukYS6y0STVbnF+1Zfspp7tt8zptWRtO6ctMQ5FahGIVWR9gXYkhB/tZJVWpYMT0ucUTnWP8DTfzrM/9vWzTc3/JFv/Gpsob9aVKOQqkhKH/9EJSH+8S6xoV3bsuW14wM829PLzv1H2Lmvl879uY/dLx1lYDB3va4xePP8Ofz0pneE9n1jX+spakoUyZD2Bdjijn88ySoJCU0mpvdYP537e9m570ju8/5cctjz8msMXZan1RgL586irbGe85rraWtuoL2pnrPnzuKkutpQ41GiEEmpcslKC/sl36GjfbnWQb5lsHN/L537jtB96PXhc6bX1nD23Fm0N9fT3tSQ/1zPWafPYvq0aCoEGvUkklLlCtNJKLoLuDsHXz2e7yoanRR6jhwbPu+kuhramupZes7ptDXV09aUSwitp81kWu0EE8Ls2XDkyNjjDQ1w+PAEX9FoShQiKRbKlqkRXGiywt3Zf+QYO/f1nmgl5G+/fPTE76B+xjTamuq57LzG4VZCW1M98085mZoaCzeoYr+7oOMToEQhkmKhDCGN4EKTNoODTveh19i5v5dn9/eOSgxHXu8fPm/OyXW0NdWz7IIzaGvK1Q/amuqZN+ckzEJOCDFSohBJsfFMsJPSBgadPS8fzSeCXDIYGmV09PiJ5UNOnzWd9uZ63nfRfNqb64e7jRrrZ2QqIZSiYrZIFaRqyGrQhS4j14e+gUF2H3x1OAnszLcSnu3p5Vj/4PB5Z8w+ifbmes5trOe85obhhHDarLEr7CZGSL8/FbNFIjSRpTckHMf6B3j+wKvDLYTO/FyE5w+8Sv/giYvm/FNOpr25nkvackXl9nxSmH3S2L09RIlCJHRJWCcq64YmpXXuHz0xbdfBVxnKBzUGrafNpK2pgXcvaqat8USX0awZGbr0NTSUHowQkgz9tESSIXVDViO40ExUJZPSzj+jgT+/cF5+yGkD5zSGPyktkSIYmaZEIRKyUIasRikBQ2CHJqWdSAbFJ6Wd0ziLxQtO4YNvbaGtKTdbueykNA3/nTQlCpGQadXTnMKC/mf/y2eZOa151OzkneOYlDY05HTCk9I0/HfSYhv1ZGbnA/884tA5wBfc/R9GnHMZ8FPg+fyhn7j7V8p9bY16krjFvU5UnNydrXuf54q7V9B/fC41A/OZ7q3UeSs1fqI7a2hS2lAiaG+up62xgQWnhjwpbQqM6gpD4td6MrNaYC+w1N13jzh+GfAZd7+qkq+nRCFSfeOdlDbAEfpqXmCgppuO1gXcctkK2pvrOWN2RJPSlCjGJQ3DYy8Hnh2ZJEQkGSYyKe2+zjvoPPwwfTVdDPIK+e24OWAd/Nl5n4nplchEJSVRXAf8sMRjbzezbUA3udbFU8VOMrOVwEqA1tbWqgQpkmW5SWlHh+cedPYET0q7dklLyUlpr6yv4Y9bnmYwLQV9CRR715OZTSeXBN7k7vsKHpsNDLp7r5ktB77l7u3lvqa6nrIrVTOeE6rYpLTO/blJaX0DJ64HC049ebh+UOmktETtk6FRT+OS6BqFmV0DfNLdrxjHubuAJe5+IOg8JYpsStTFJwVGTUpb/UV2NpxB5+kt7Dp1HoM1ufkFNYODtDbW5xa0y++B0N7UwLlNs5g5fXIdDlO5oJ9GSa9RXE+JbiczOwPY5+5uZh3k9vg+GGVwkhya8VzcuCalXfQXLHy5m/MO7Oaqpx+m7WAX7Qde4OyX9nJS//HKv+k43qWX20tD0iPWRGFmM4H3ADeOOLYKwN3XAh8EPmFm/cBrwHUedxMoTTLW5E7djOeQHTrax86L3k7nyaezc24rO09voXNuC92zm4bPGZqUdlHLqXzo4pbh4adnnXka0wf7A756hTQ3YUqJNVG4+1Hg9IJja0fcvh24Peq4MiNj/8ypm/E8AaN2SuspMint/V8G4KS+1zn34B6WvvDkcOug7Xe/Lj0pLcwkIVNOErqeRMYlSzOeK90p7Z3nNXJecz3tN1xH24EXmH+4hxoKGteN9RG/CpkqlCgkNdK4SU8lO6Wd11zPsgvmDXcXFZ2U9pwGaUj0lCgkVaIqkFY6DHe8k9Lm1k+nvakh+p3SErxC7JSR4pph7MNjq0HDY/O0dMGEBA3DPaN+/qhJaUP1g+fK7JSWW8eonlMnu1NaUn6nKb7oxSYpv7sSkj48VqpF7yInZM0ja+g99joMnMnMwVbqvIW61xey/FuP8Pqx2WMmpbU11fOOqHZKS8rvVMlgSlGiyDL9M5dVbKe03+xcwry+92DkJqU5A/TbPo7TzV9fehFt+VZCGJPSKqbfqcRAiUKmhEp2Smuc08/OQz/mdXbRV9NFv+1lWq3z/otv5JZlb4gm4Cx07WThNQigRCEJEsY6TpXslDZmUlp+p7SuQ+eweO1n6YtzGG4W5sBk4TUIoGK2JEQl6ziVnZSWd1JdzXAxudKd0mJfp6jSwmcS370nvHgbuST+jkYIvZhtZje4+z9NLiyRE4qu43Ssl6/827dYccHnAielNcyYxrkjJ6U15RLD/FMmvlNa6tYp0rv35EtAMpioiXY9fRlQopBQDA46v9v1NLV9F3LSYCt1voA6b2X6YAsbNtazYeNGYPSktKHWwXnNDTTPrvIchGIS/u4wdrNnxx2BhKhkojCzJ0o9BDRXJxzJsuBJaZ8e/qMa4GX6al7gtWkPs3ThWXz+XTdEMymtEnoHH0w/h0wJalE0A+8FXi44bsDvqhaRpN7IndKGi8pldkprnN3Hlx5ZxeGBnRz3l4ZrFHd8aBstc+bG+GpikpT5EtWQhdcwxQQlip8B9e6+tfABM/uPqkUkqVFsp7Sd+3rZdbD4TmmXlJmUdvVb/iVV6zhVVZa7r7L82jIqKFGcCewt9oC7/9fqhCNJVGxSWuf+XEIYzOeDGoPW02bS1tTAuxc1T2intNQVkJMkyy2QyVAtKRRB/8H/BPzKzL4HrHH3voBzJQOKTUrr3N9L18tHx0xKO/+MBv78wnm0NzfQ3lTP2XNncVJdbbwvYCrTRa841ZJCUTJRuPu9ZvZz4AvAZjP7v8DgiMe/GUF8UgWvHD0+qnYwNBeh2KS0CxfM4QNvXTC8n/LQpLQpT+/gg+nnkynl+gT6gFeBGUADIxKFJFulk9KWnnP6iH0QGmg59eSyk9KmNL2DD6afT6YEDY9dBnwTeAB4a37bUkmYSndKu+y8xnxBOVdDmMykNJHIRVVzUG1jlKAWxa3Ah9z9qWp9czPbBRwBBoD+wunjlhs0/y1gOXAU+Ii7P16teJJs5E5pnSOTQsBOabFPShMpZaIX4qhqDqptjBJUo7g0ohje5e4HSjx2JdCe/1gK3JH/nFkDg07XS0dPTEgb6joas1PaDNqaZp3YKa0x12U0t366EoIkX1QXYtVKQpH01WOvAb7vuZULHzWzU8xsnru/GHdgYXns+Zd49LmDw8XlZ3t6OV5iUtrIxe0mvVOaVJ+6L+Knn3Mo4k4UDjxoZg6sc/c7Cx6fD3SNuL8nf2xMojCzlcBKgNbW1upEWwXrt7/I3b/bNTwp7dL2ucP7KFd1p7S0SPPFVt0XkhFxJ4pL3L3bzJqADWb2tLs/NOLxYn0oRdcnzieZOyG3zHj4oVbHpy5vZ/Wy86PfKS0tdLEViV2s4x/dvTv/eT9wH9BRcMoeYOQaDguA7miii8aps6YrSYiMV6naQtg1h6i+T0rElijMbJaZNQzdBq4Aniw47QHgryznbcChLNUnRKasiV6IDx/ObXpU+BF2N2RU3ycl4nwr2wzclx+hMw34gbv/0sxWAbj7WmA9uaGxneSGx94QU6ySVWmugaSZfrapEluicPfngMVFjq8dcduBT0YZl0wx1ayBaGimZITWaJBki7OveLK7tKn7QjJCVVRJtjgvqhpZJQKoRSFSHbNng9nYD+0lLSmkRCFSDZr/IRmiRCFTmwrLImUpUcjUpsKySFlKFCIiEkiJQiSLVEyXEClRiFRD3GsFqZguIVKiEAnroj7yXfzIC3JDgybbSaopUYiENYNa7+LTS111gZQoRCRaSbwoK8kHUqIQkWjpopw6ShQiWRR3MV0yRYlCJG7V6IrRyrUSIiUKkbBM9F28umIk4ZQopLgkFhyTrhrv4pP488/i34a66gJpPwopTu9ykykJP//J/m0kcec/dckFiq1FYWYtZvbvZrbDzJ4ys08VOecyMztkZlvzH1+II1aRKaearQbVT1InzhZFP/A/3P1xM2sAtpjZBnf/Q8F5D7v7VTHEJzJ1qUUpI8TWonD3F9398fztI8AOYH5c8YjERv3gknCJKGab2ULgLcDGIg+/3cy2mdkvzOxNAV9jpZltNrPNPT09VYpUpAqKdcWIJEjsicLM6oEfA59298JOyseBs9x9MXAbcH+pr+Pud7r7Endf0tjYWL2ApwqNAolXkn/+1YwtiyOqMiDWUU9mVkcuSdzj7j8pfHxk4nD39Wb2j2Y2190PRBnnlKTCYryS/POvZmyqjSRSnKOeDPgusMPdv1ninDPy52FmHeTiPRhdlCIZVe6de5JbNBK5OFsUlwAfBrab2db8sc8DrQDuvhb4IPAJM+sHXgOuc1cHrsiklXvnnuQWjUQutkTh7r8FrMw5twO3RxORiIgUE3sxWyR0KoiKhEqJQrJHBdHohZWcVRtJJK31JCKTF1ZyVm0kkdSiEJmK9M5dKqAWhchUpHfuUgG1KESCqDAuokQhGRRmt4oK4yLqepIMUrdK9JK4GZGERolCRCZPyTnT1PUkkjaqm0jElChE0kZ1E4mYEoVIEM03EFGNQiSQ+t5F1KIQkQipvpJKShSSDlm7wGTt9YyX6iuppEQh6ZC1C8xkXo/qJhIx1ShE0kZ1E4mYWhQy2lTtEhGRkpQoZLSsdfFkiZK4xCTWRGFmy8zsGTPrNLPPFXnczOx/5x9/wszeGkecEiJd7CYuC0lc9ZVUii1RmFkt8G3gSmARcL2ZLSo47UqgPf+xErgj0iAlfBO92GXtApO11zNehw+D+9gP1V0SLc5idgfQ6e7PAZjZj4BrgD+MOOca4Pvu7sCjZnaKmc1z9xejD1dilbULSdZej2RanF1P84GuEff35I9Veg4AZrbSzDab2eaenp5QAxURmcriTBRW5JhP4JzcQfc73X2Juy9pbGycdHBT1lTtEhGRkuLsetoDtIy4vwDonsA5EiZ1iSSXNgeSmMTZotgEtJvZ2WY2HbgOeKDgnAeAv8qPfnobcEj1iZQaGu1Uii525akQLDGJrUXh7v1mdhPwK6AWuMvdnzKzVfnH1wLrgeVAJ3AUuCGueGWSgkY1edHeRBFJiFiX8HD39eSSwchja0fcduCTUcclklizZ5fuflLLQqpEM7NF0iQLk+4kdZQoREQkkBKFiIgEUqKQaGh+hkhqaT8KiYYKrSKppRaFSJqoZSYxUItCJE3UMpMYqEUhAtonQySAEoUIaH6CSAAlCtC7Sake/W1JBihRgN5NTnXVvGjrb0syQIlCRBdtkUBKFCIiEkiJQiSI5ieIKFGIBNK8BRElCkCzXaW0yY5a0t+WZIBmZoPeNU51QXtRT3bUkv62JAPUopB0qOZ8BO1FLRIolhaFmX0d+AvgOPAscIO7v1LkvF3AEWAA6Hf3JVHGKQmi+QgisYmrRbEBuMDdLwT+CPxtwLnvcveLlCREROIRS6Jw9wfdvT9/91FgQRxxiIhIeUmoUXwU+EWJxxx40My2mNnKoC9iZivNbLOZbe7p6Qk9SJmiNGpJpHo1CjP7NXBGkYdudfef5s+5FegH7inxZS5x924zawI2mNnT7v5QsRPd/U7gToAlS5b4pF+ACKigLUIVE4W7vzvocTNbAVwFXO7uRS/s7t6d/7zfzO4DOoCiiUIyLmgIq4hUVSxdT2a2DLgFuNrdj5Y4Z5aZNQzdBq4AnowuSkkUDWEViU1cNYrbgQZy3UlbzWwtgJmdaWbr8+c0A781s23AY8DP3f2X8YQrIjJ1xTKPwt3bShzvBpbnbz8HLI4yLsmQ2bNLd1WpFSJSkSSMehIJnyboiYRGiUJERAIpUYiISCAlChERCaREISIigZQoJJu09IZIaLRxkWSThsCKhEYtChERCaREISIigZQoREQkkBKFiIgEUqIQEZFAVmIriFQzsx5gd9xxVGAucCDuICZAcUcnjTGD4o7aZOI+y90biz2QyUSRNma22d2XxB1HpRR3dNIYMyjuqFUrbnU9iYhIICUKEREJpESRDHfGHcAEKe7opDFmUNxRq0rcqlGIiEggtShERCSQEoWIiARSokgIM/ufZvaEmW01swfN7My4YxoPM/u6mT2dj/0+Mzsl7pjKMbMPmdlTZjZoZokfAmlmy8zsGTPrNLPPxR3PeJjZXWa238yejDuWSphZi5n9u5ntyP+NfCrumMoxs5PM7DEz25aP+cuhfw/VKJLBzGa7++H87f8OLHL3VTGHVZaZXQH8m7v3m9n/AnD3W2IOK5CZvREYBNYBn3H3zTGHVJKZ1QJ/BN4D7AE2Ade7+x9iDawMM/szoBf4vrtfEHc842Vm84B57v64mTUAW4D3JfnnbWYGzHL3XjOrA34LfMrdHw3re6hFkRBDSSJvFpCKDO7uD7p7f/7uo8CCOOMZD3ff4e7PxB3HOHUAne7+nLsfB34EXBNzTGW5+0PAS3HHUSl3f9HdH8/fPgLsAObHG1Uwz+nN363Lf4R6/VCiSBAz+6qZdQH/DfhC3PFMwEeBX8QdRMbMB7pG3N9Dwi9cWWFmC4G3ABvjjaQ8M6s1s6T6JFQAAAJ6SURBVK3AfmCDu4casxJFhMzs12b2ZJGPawDc/VZ3bwHuAW6KN9oTysWdP+dWoJ9c7LEbT8wpYUWOpaK1mWZmVg/8GPh0QWs/kdx9wN0vItei7zCzULv7tBVqhNz93eM89QfAz4EvVjGccSsXt5mtAK4CLveEFL0q+Fkn3R6gZcT9BUB3TLFMCfl+/h8D97j7T+KOpxLu/oqZ/QewDAhtIIFaFAlhZu0j7l4NPB1XLJUws2XALcDV7n407ngyaBPQbmZnm9l04DrggZhjyqx8Yfi7wA53/2bc8YyHmTUOjTY0s5OBdxPy9UOjnhLCzH4MnE9uNM5uYJW77403qvLMrBOYARzMH3o06aO1zOwvgduARuAVYKu7vzfeqEozs+XAPwC1wF3u/tWYQyrLzH4IXEZu2et9wBfd/buxBjUOZvYO4GFgO7n/RYDPu/v6+KIKZmYXAt8j9/dRA9zr7l8J9XsoUYiISBB1PYmISCAlChERCaREISIigZQoREQkkBKFiIgEUqIQqYL8KqTPm9lp+fun5u+fZWa/NLNXzOxncccpMh5KFCJV4O5dwB3A1/KHvgbc6e67ga8DH44rNpFKKVGIVM/fA28zs08D7wD+DsDdfwMciTMwkUporSeRKnH3PjP7LPBL4Ir8MuEiqaMWhUh1XQm8CKRm8x6RQkoUIlViZheR25nubcDf5HdPE0kdJQqRKsivQnoHuf0MXiBXwP5GvFGJTIwShUh1fBx4wd035O//I/AGM3unmT0M/AtwuZntMbPErlwrAlo9VkREylCLQkREAilRiIhIICUKEREJpEQhIiKBlChERCSQEoWIiARSohARkUD/H2dIAm/ScAYZAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "#优化的随机梯度上升算法\n",
    "#将alpha进行动态调整，以及随机选取状态更新\n",
    "\n",
    "import random\n",
    "def stocGradAscent1(dataMatrix, classLabels, numIter = 150):\n",
    "    m, n = np.shape(dataMatrix)\n",
    "#     alpha = 0.01\n",
    "    weights = np.ones(n)\n",
    "    \n",
    "    for j in range(numIter):\n",
    "        dataIndex = range(m)\n",
    "        for i in range(m):\n",
    "            alpha = 4 / (1.0 + j + i) + 0.01 #动态调整alpha\n",
    "            randIndex = int(random.uniform(0, len(dataIndex)))  #随机选取状态更新\n",
    "            h = sigmoid(sum(dataMatrix[randIndex] *weights))\n",
    "            error = classLabels[randIndex] - h\n",
    "            weights += alpha * error * dataMatrix[randIndex]\n",
    "    return weights\n",
    "\n",
    "dataArr, labelMat = loadDataSet()\n",
    "weights = stocGradAscent1(np.array(dataArr), labelMat)\n",
    "print(weights)\n",
    "plotBestFit(weights)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 示例：从疝气病预测病马的死亡率\n",
    "![title.png](./picture/05使用Logistic回归估计马疝病死率.png)\n",
    "\n",
    "#### 01准备数据：处理数据中的缺失值  \n",
    "\n",
    "当数据缺失时，扔掉和重新获取往往是不可取的，一般采用以下方案解决：  \n",
    "·使用可用特征值的均值来填充缺失值；  \n",
    "·使用特殊值来填充缺失值，如-1；  \n",
    "·忽略有缺失值样本；  \n",
    "·使用相似样本均值填补缺失值；  \n",
    "·使用另外的机器学习算法预测缺失值；  \n",
    "\n",
    "本题采用选择实数0来替代所有缺失值，这样的话在更新系数时不会影响系数的值\n",
    "\n",
    " weights += alpha * error * dataMatrix[randIndex]\n",
    " \n",
    "如果dataMatrix的某特征值对应0，那么该特征的系数将不进行更新，即：  \n",
    " weights = weights\n",
    " \n",
    "另外由于sigmoid(0) = 0.5,即对于预测不具有倾向性，因此对于误差也不产生影响\n",
    "\n",
    "原始数据处理后保存为horseColicTest.txt和horseColic-Training.txt\n",
    "\n",
    "#### 02用Logistic回归分类函数"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "<ipython-input-18-3f4057c48aea>:15: RuntimeWarning: overflow encountered in exp\n",
      "  return 1.0 / (1 + np.exp(-inX))\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "the error rate of this test is :0.119403\n",
      "the error rate of this test is :0.417910\n",
      "the error rate of this test is :0.164179\n",
      "the error rate of this test is :0.119403\n",
      "the error rate of this test is :0.134328\n",
      "the error rate of this test is :0.149254\n",
      "the error rate of this test is :0.194030\n",
      "the error rate of this test is :0.223881\n",
      "the error rate of this test is :0.283582\n",
      "the error rate of this test is :0.164179\n",
      "after 10 iterations the average error rate is: 0.197015\n"
     ]
    }
   ],
   "source": [
    "def classifyVector(inX, weights):   #用回归系数和特征向量作为输入来计算对应Sigmoid\n",
    "    prob = sigmoid(sum(inX*weights))\n",
    "    if  prob > 0.5 : return 1.0\n",
    "    else: return 0.0\n",
    "def colicTest():\n",
    "    frTrain = open('./data/horseColicTest.txt')\n",
    "    frTest = open('./data/horseColicTest.txt')\n",
    "    trainingSet = []; trainingLabels = []\n",
    "    for line in frTrain.readlines():\n",
    "        currLine = line.strip().split('\\t')\n",
    "        lineArr = []\n",
    "        for i in range(21):\n",
    "            lineArr.append(float(currLine[i]))\n",
    "        trainingSet.append(lineArr)\n",
    "        trainingLabels.append(float(currLine[21]))\n",
    "    trainWeights = stocGradAscent1(np.array(trainingSet), trainingLabels, 500)\n",
    "    errorCount = 0; numTestVec = 0.0\n",
    "    for line in frTest.readlines():\n",
    "        numTestVec += 1.0\n",
    "        currLine = line.strip().split('\\t')\n",
    "        lineArr = []\n",
    "        for i in range(21):\n",
    "            lineArr.append(float(currLine[i]))\n",
    "        if int(classifyVector(np.array(lineArr), trainWeights)) != int(currLine[21]):\n",
    "            errorCount += 1\n",
    "            \n",
    "    errorRate = (float(errorCount) / numTestVec)\n",
    "    print(\"the error rate of this test is :%f\" % errorRate)\n",
    "    return errorRate\n",
    "\n",
    "def multiTest():\n",
    "    numTests = 10; errorSum = 0.0\n",
    "    for k in range(numTests):\n",
    "        errorSum += colicTest()\n",
    "    print(\"after %d iterations the average error rate is: %f\" %(numTests, errorSum / float(numTests)))\n",
    "\n",
    "\n",
    "    \n",
    "multiTest()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 总结一下：__逻辑回归的本质是采用回归的思路拟合一条最优曲线，然后利用这条最优曲线对数据进行分类.__"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}