homework1.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "thinking: \n",
    "1. why for collaborative filtering:\n",
    "协同过滤主要是对用户行为和偏好中发现规律，并基于相应的核心策略进行收集和推荐。 用户数据中包含用户行为，商品喜好等方式进行归类和更新。\n",
    "这一点可以最大程度上对新的“同类型” 用户进行推荐， 不仅在既定用户上的已经做好的分类上进行推荐，当发生更新时，也可以给其他用户提供类似推荐。 但实际上，在解决协同过滤时，（学习过程中） 人为最重要的是消减信息“噪声” ， 并对数据进行归一化， 归一化是为了让信息值上具有相同的“影响力”，减噪视为了用户在使用应用过程中产生的，它可能存在大量的噪音和用户的误操作，可以通过经典的数据挖掘算法过滤掉行为数据中的噪音，这样可以让分析更加精确，对于欧式距离小有了解，其距离可以区分不同类别从而实现其推荐算法的普适度，协同过滤的点子上还可以快速为同类型用户推荐类似但仍然有差异的定制商品或者服务，因此是有必要的。\n",
    "\n",
    "2. need RS? \n",
    " emmm 通常个人，小公司不需要推荐系统，推荐系统通常被应用于较大的平台型商品综合类型的公司， 首先这类公司具有较大用户base, 信息数据量volumn 值高，从而为了使更多用户具有有特征个性化的推荐，从而实现定制化供给服务。\n",
    " \n",
    "3. 视频打标签问题： \n",
    " 对nlp实际上有一定了解，但这个问题问的有一点笼统， 总的来说，视频是一个多媒体的时间序列， 包含图像和音频， 但是nlp的打标签，通常使用的对应 音频的名称进行labeling， 但是忽略了图像时间序列的主要信息包含，可能会出现标签不准确，视频匹配度不高的问题。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>PassengerId</th>\n",
       "      <th>Survived</th>\n",
       "      <th>Pclass</th>\n",
       "      <th>Name</th>\n",
       "      <th>Sex</th>\n",
       "      <th>Age</th>\n",
       "      <th>SibSp</th>\n",
       "      <th>Parch</th>\n",
       "      <th>Ticket</th>\n",
       "      <th>Fare</th>\n",
       "      <th>Cabin</th>\n",
       "      <th>Embarked</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>Braund, Mr. Owen Harris</td>\n",
       "      <td>male</td>\n",
       "      <td>22.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>A/5 21171</td>\n",
       "      <td>7.2500</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Cumings, Mrs. John Bradley (Florence Briggs Th...</td>\n",
       "      <td>female</td>\n",
       "      <td>38.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>PC 17599</td>\n",
       "      <td>71.2833</td>\n",
       "      <td>C85</td>\n",
       "      <td>C</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>2</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>Heikkinen, Miss. Laina</td>\n",
       "      <td>female</td>\n",
       "      <td>26.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>STON/O2. 3101282</td>\n",
       "      <td>7.9250</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>3</td>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Futrelle, Mrs. Jacques Heath (Lily May Peel)</td>\n",
       "      <td>female</td>\n",
       "      <td>35.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>113803</td>\n",
       "      <td>53.1000</td>\n",
       "      <td>C123</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>4</td>\n",
       "      <td>5</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>Allen, Mr. William Henry</td>\n",
       "      <td>male</td>\n",
       "      <td>35.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>373450</td>\n",
       "      <td>8.0500</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   PassengerId  Survived  Pclass  \\\n",
       "0            1         0       3   \n",
       "1            2         1       1   \n",
       "2            3         1       3   \n",
       "3            4         1       1   \n",
       "4            5         0       3   \n",
       "\n",
       "                                                Name     Sex   Age  SibSp  \\\n",
       "0                            Braund, Mr. Owen Harris    male  22.0      1   \n",
       "1  Cumings, Mrs. John Bradley (Florence Briggs Th...  female  38.0      1   \n",
       "2                             Heikkinen, Miss. Laina  female  26.0      0   \n",
       "3       Futrelle, Mrs. Jacques Heath (Lily May Peel)  female  35.0      1   \n",
       "4                           Allen, Mr. William Henry    male  35.0      0   \n",
       "\n",
       "   Parch            Ticket     Fare Cabin Embarked  \n",
       "0      0         A/5 21171   7.2500   NaN        S  \n",
       "1      0          PC 17599  71.2833   C85        C  \n",
       "2      0  STON/O2. 3101282   7.9250   NaN        S  \n",
       "3      0            113803  53.1000  C123        S  \n",
       "4      0            373450   8.0500   NaN        S  "
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "from sklearn.model_selection import GridSearchCV\n",
    "from sklearn.model_selection import cross_validate\n",
    "from sklearn import metrics\n",
    "from sklearn.feature_selection import SelectKBest,SelectPercentile\n",
    "import pylab as plt\n",
    "%matplotlib inline\n",
    "train_data = pd.read_csv(r'train.csv')\n",
    "test_data = pd.read_csv(r'test.csv')\n",
    "train_data.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 294 entries, 0 to 293\n",
      "Data columns (total 35 columns):\n",
      "user_id                     294 non-null int64\n",
      "Age                         294 non-null int64\n",
      "BusinessTravel              294 non-null object\n",
      "DailyRate                   294 non-null int64\n",
      "Department                  294 non-null object\n",
      "DistanceFromHome            294 non-null int64\n",
      "Education                   294 non-null int64\n",
      "EducationField              294 non-null object\n",
      "EmployeeCount               294 non-null int64\n",
      "EmployeeNumber              294 non-null int64\n",
      "EnvironmentSatisfaction     294 non-null int64\n",
      "Gender                      294 non-null object\n",
      "HourlyRate                  294 non-null int64\n",
      "JobInvolvement              294 non-null int64\n",
      "JobLevel                    294 non-null int64\n",
      "JobRole                     294 non-null object\n",
      "JobSatisfaction             294 non-null int64\n",
      "MaritalStatus               294 non-null object\n",
      "MonthlyIncome               294 non-null int64\n",
      "MonthlyRate                 294 non-null int64\n",
      "NumCompaniesWorked          294 non-null int64\n",
      "Over18                      294 non-null object\n",
      "OverTime                    294 non-null object\n",
      "PercentSalaryHike           294 non-null int64\n",
      "PerformanceRating           294 non-null int64\n",
      "RelationshipSatisfaction    294 non-null int64\n",
      "StandardHours               294 non-null int64\n",
      "StockOptionLevel            294 non-null int64\n",
      "TotalWorkingYears           294 non-null int64\n",
      "TrainingTimesLastYear       294 non-null int64\n",
      "WorkLifeBalance             294 non-null int64\n",
      "YearsAtCompany              294 non-null int64\n",
      "YearsInCurrentRole          294 non-null int64\n",
      "YearsSinceLastPromotion     294 non-null int64\n",
      "YearsWithCurrManager        294 non-null int64\n",
      "dtypes: int64(27), object(8)\n",
      "memory usage: 80.5+ KB\n"
     ]
    }
   ],
   "source": [
    "test_data.info()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 1176 entries, 0 to 1175\n",
      "Data columns (total 36 columns):\n",
      "user_id                     1176 non-null int64\n",
      "Age                         1176 non-null int64\n",
      "Attrition                   1176 non-null object\n",
      "BusinessTravel              1176 non-null object\n",
      "DailyRate                   1176 non-null int64\n",
      "Department                  1176 non-null object\n",
      "DistanceFromHome            1176 non-null int64\n",
      "Education                   1176 non-null int64\n",
      "EducationField              1176 non-null object\n",
      "EmployeeCount               1176 non-null int64\n",
      "EmployeeNumber              1176 non-null int64\n",
      "EnvironmentSatisfaction     1176 non-null int64\n",
      "Gender                      1176 non-null object\n",
      "HourlyRate                  1176 non-null int64\n",
      "JobInvolvement              1176 non-null int64\n",
      "JobLevel                    1176 non-null int64\n",
      "JobRole                     1176 non-null object\n",
      "JobSatisfaction             1176 non-null int64\n",
      "MaritalStatus               1176 non-null object\n",
      "MonthlyIncome               1176 non-null int64\n",
      "MonthlyRate                 1176 non-null int64\n",
      "NumCompaniesWorked          1176 non-null int64\n",
      "Over18                      1176 non-null object\n",
      "OverTime                    1176 non-null object\n",
      "PercentSalaryHike           1176 non-null int64\n",
      "PerformanceRating           1176 non-null int64\n",
      "RelationshipSatisfaction    1176 non-null int64\n",
      "StandardHours               1176 non-null int64\n",
      "StockOptionLevel            1176 non-null int64\n",
      "TotalWorkingYears           1176 non-null int64\n",
      "TrainingTimesLastYear       1176 non-null int64\n",
      "WorkLifeBalance             1176 non-null int64\n",
      "YearsAtCompany              1176 non-null int64\n",
      "YearsInCurrentRole          1176 non-null int64\n",
      "YearsSinceLastPromotion     1176 non-null int64\n",
      "YearsWithCurrManager        1176 non-null int64\n",
      "dtypes: int64(27), object(9)\n",
      "memory usage: 330.9+ KB\n"
     ]
    }
   ],
   "source": [
    "#check data value whether null.\n",
    "train_data.info()\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>user_id</th>\n",
       "      <th>Age</th>\n",
       "      <th>DailyRate</th>\n",
       "      <th>DistanceFromHome</th>\n",
       "      <th>Education</th>\n",
       "      <th>EmployeeCount</th>\n",
       "      <th>EmployeeNumber</th>\n",
       "      <th>EnvironmentSatisfaction</th>\n",
       "      <th>HourlyRate</th>\n",
       "      <th>JobInvolvement</th>\n",
       "      <th>...</th>\n",
       "      <th>RelationshipSatisfaction</th>\n",
       "      <th>StandardHours</th>\n",
       "      <th>StockOptionLevel</th>\n",
       "      <th>TotalWorkingYears</th>\n",
       "      <th>TrainingTimesLastYear</th>\n",
       "      <th>WorkLifeBalance</th>\n",
       "      <th>YearsAtCompany</th>\n",
       "      <th>YearsInCurrentRole</th>\n",
       "      <th>YearsSinceLastPromotion</th>\n",
       "      <th>YearsWithCurrManager</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>count</td>\n",
       "      <td>1176.000000</td>\n",
       "      <td>1176.000000</td>\n",
       "      <td>1176.000000</td>\n",
       "      <td>1176.000000</td>\n",
       "      <td>1176.000000</td>\n",
       "      <td>1176.0</td>\n",
       "      <td>1176.000000</td>\n",
       "      <td>1176.000000</td>\n",
       "      <td>1176.000000</td>\n",
       "      <td>1176.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>1176.000000</td>\n",
       "      <td>1176.0</td>\n",
       "      <td>1176.000000</td>\n",
       "      <td>1176.000000</td>\n",
       "      <td>1176.000000</td>\n",
       "      <td>1176.000000</td>\n",
       "      <td>1176.000000</td>\n",
       "      <td>1176.00000</td>\n",
       "      <td>1176.000000</td>\n",
       "      <td>1176.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>mean</td>\n",
       "      <td>736.124150</td>\n",
       "      <td>36.805272</td>\n",
       "      <td>802.033163</td>\n",
       "      <td>9.159864</td>\n",
       "      <td>2.918367</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1026.960034</td>\n",
       "      <td>2.750850</td>\n",
       "      <td>65.130102</td>\n",
       "      <td>2.724490</td>\n",
       "      <td>...</td>\n",
       "      <td>2.714286</td>\n",
       "      <td>80.0</td>\n",
       "      <td>0.805272</td>\n",
       "      <td>11.161565</td>\n",
       "      <td>2.767007</td>\n",
       "      <td>2.764456</td>\n",
       "      <td>6.982143</td>\n",
       "      <td>4.19898</td>\n",
       "      <td>2.160714</td>\n",
       "      <td>4.098639</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>std</td>\n",
       "      <td>419.336964</td>\n",
       "      <td>9.065549</td>\n",
       "      <td>405.946729</td>\n",
       "      <td>8.137224</td>\n",
       "      <td>1.009809</td>\n",
       "      <td>0.0</td>\n",
       "      <td>594.763609</td>\n",
       "      <td>1.096221</td>\n",
       "      <td>20.294326</td>\n",
       "      <td>0.715027</td>\n",
       "      <td>...</td>\n",
       "      <td>1.080583</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.865611</td>\n",
       "      <td>7.747576</td>\n",
       "      <td>1.250756</td>\n",
       "      <td>0.713251</td>\n",
       "      <td>6.094338</td>\n",
       "      <td>3.63124</td>\n",
       "      <td>3.208052</td>\n",
       "      <td>3.564190</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>min</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>18.000000</td>\n",
       "      <td>104.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>30.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>80.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>25%</td>\n",
       "      <td>374.750000</td>\n",
       "      <td>30.000000</td>\n",
       "      <td>463.500000</td>\n",
       "      <td>2.000000</td>\n",
       "      <td>2.000000</td>\n",
       "      <td>1.0</td>\n",
       "      <td>498.750000</td>\n",
       "      <td>2.000000</td>\n",
       "      <td>48.000000</td>\n",
       "      <td>2.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>2.000000</td>\n",
       "      <td>80.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>6.000000</td>\n",
       "      <td>2.000000</td>\n",
       "      <td>2.000000</td>\n",
       "      <td>3.000000</td>\n",
       "      <td>2.00000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>2.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>50%</td>\n",
       "      <td>742.500000</td>\n",
       "      <td>36.000000</td>\n",
       "      <td>805.500000</td>\n",
       "      <td>7.000000</td>\n",
       "      <td>3.000000</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1031.000000</td>\n",
       "      <td>3.000000</td>\n",
       "      <td>65.000000</td>\n",
       "      <td>3.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>3.000000</td>\n",
       "      <td>80.0</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>10.000000</td>\n",
       "      <td>3.000000</td>\n",
       "      <td>3.000000</td>\n",
       "      <td>5.000000</td>\n",
       "      <td>3.00000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>3.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>75%</td>\n",
       "      <td>1101.250000</td>\n",
       "      <td>42.250000</td>\n",
       "      <td>1162.000000</td>\n",
       "      <td>14.000000</td>\n",
       "      <td>4.000000</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1555.250000</td>\n",
       "      <td>4.000000</td>\n",
       "      <td>82.250000</td>\n",
       "      <td>3.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>4.000000</td>\n",
       "      <td>80.0</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>15.000000</td>\n",
       "      <td>3.000000</td>\n",
       "      <td>3.000000</td>\n",
       "      <td>9.000000</td>\n",
       "      <td>7.00000</td>\n",
       "      <td>2.250000</td>\n",
       "      <td>7.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>max</td>\n",
       "      <td>1469.000000</td>\n",
       "      <td>60.000000</td>\n",
       "      <td>1499.000000</td>\n",
       "      <td>29.000000</td>\n",
       "      <td>5.000000</td>\n",
       "      <td>1.0</td>\n",
       "      <td>2068.000000</td>\n",
       "      <td>4.000000</td>\n",
       "      <td>100.000000</td>\n",
       "      <td>4.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>4.000000</td>\n",
       "      <td>80.0</td>\n",
       "      <td>3.000000</td>\n",
       "      <td>40.000000</td>\n",
       "      <td>6.000000</td>\n",
       "      <td>4.000000</td>\n",
       "      <td>40.000000</td>\n",
       "      <td>18.00000</td>\n",
       "      <td>15.000000</td>\n",
       "      <td>17.000000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>8 rows × 27 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "           user_id          Age    DailyRate  DistanceFromHome    Education  \\\n",
       "count  1176.000000  1176.000000  1176.000000       1176.000000  1176.000000   \n",
       "mean    736.124150    36.805272   802.033163          9.159864     2.918367   \n",
       "std     419.336964     9.065549   405.946729          8.137224     1.009809   \n",
       "min       0.000000    18.000000   104.000000          1.000000     1.000000   \n",
       "25%     374.750000    30.000000   463.500000          2.000000     2.000000   \n",
       "50%     742.500000    36.000000   805.500000          7.000000     3.000000   \n",
       "75%    1101.250000    42.250000  1162.000000         14.000000     4.000000   \n",
       "max    1469.000000    60.000000  1499.000000         29.000000     5.000000   \n",
       "\n",
       "       EmployeeCount  EmployeeNumber  EnvironmentSatisfaction   HourlyRate  \\\n",
       "count         1176.0     1176.000000              1176.000000  1176.000000   \n",
       "mean             1.0     1026.960034                 2.750850    65.130102   \n",
       "std              0.0      594.763609                 1.096221    20.294326   \n",
       "min              1.0        1.000000                 1.000000    30.000000   \n",
       "25%              1.0      498.750000                 2.000000    48.000000   \n",
       "50%              1.0     1031.000000                 3.000000    65.000000   \n",
       "75%              1.0     1555.250000                 4.000000    82.250000   \n",
       "max              1.0     2068.000000                 4.000000   100.000000   \n",
       "\n",
       "       JobInvolvement  ...  RelationshipSatisfaction  StandardHours  \\\n",
       "count     1176.000000  ...               1176.000000         1176.0   \n",
       "mean         2.724490  ...                  2.714286           80.0   \n",
       "std          0.715027  ...                  1.080583            0.0   \n",
       "min          1.000000  ...                  1.000000           80.0   \n",
       "25%          2.000000  ...                  2.000000           80.0   \n",
       "50%          3.000000  ...                  3.000000           80.0   \n",
       "75%          3.000000  ...                  4.000000           80.0   \n",
       "max          4.000000  ...                  4.000000           80.0   \n",
       "\n",
       "       StockOptionLevel  TotalWorkingYears  TrainingTimesLastYear  \\\n",
       "count       1176.000000        1176.000000            1176.000000   \n",
       "mean           0.805272          11.161565               2.767007   \n",
       "std            0.865611           7.747576               1.250756   \n",
       "min            0.000000           0.000000               0.000000   \n",
       "25%            0.000000           6.000000               2.000000   \n",
       "50%            1.000000          10.000000               3.000000   \n",
       "75%            1.000000          15.000000               3.000000   \n",
       "max            3.000000          40.000000               6.000000   \n",
       "\n",
       "       WorkLifeBalance  YearsAtCompany  YearsInCurrentRole  \\\n",
       "count      1176.000000     1176.000000          1176.00000   \n",
       "mean          2.764456        6.982143             4.19898   \n",
       "std           0.713251        6.094338             3.63124   \n",
       "min           1.000000        0.000000             0.00000   \n",
       "25%           2.000000        3.000000             2.00000   \n",
       "50%           3.000000        5.000000             3.00000   \n",
       "75%           3.000000        9.000000             7.00000   \n",
       "max           4.000000       40.000000            18.00000   \n",
       "\n",
       "       YearsSinceLastPromotion  YearsWithCurrManager  \n",
       "count              1176.000000           1176.000000  \n",
       "mean                  2.160714              4.098639  \n",
       "std                   3.208052              3.564190  \n",
       "min                   0.000000              0.000000  \n",
       "25%                   0.000000              2.000000  \n",
       "50%                   1.000000              3.000000  \n",
       "75%                   2.250000              7.000000  \n",
       "max                  15.000000             17.000000  \n",
       "\n",
       "[8 rows x 27 columns]"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#current status\n",
    "train_data.describe()\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [],
   "source": [
    "# try seperate the data and lable, normalize the data to numerical classifications\n",
    "from sklearn.preprocessing import LabelEncoder\n",
    "train_keys = []\n",
    "for i in train_data:\n",
    "    train_keys.append(i)\n",
    "\n",
    "test_keys = []\n",
    "for i in test_data:\n",
    "    test_keys.append(i)\n",
    "\n",
    "le = LabelEncoder()\n",
    "var_to_encode_train = train_keys\n",
    "var_to_encode_test = test_keys\n",
    "\n",
    "for i in var_to_encode_train:\n",
    "    train_data[i] = le.fit_transform(train_data[i])\n",
    "\n",
    "for i in var_to_encode_test:\n",
    "    test_data[i] = le.fit_transform(test_data[i])\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 101,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>user_id</th>\n",
       "      <th>Age</th>\n",
       "      <th>BusinessTravel</th>\n",
       "      <th>DailyRate</th>\n",
       "      <th>Department</th>\n",
       "      <th>DistanceFromHome</th>\n",
       "      <th>Education</th>\n",
       "      <th>EducationField</th>\n",
       "      <th>EmployeeCount</th>\n",
       "      <th>EmployeeNumber</th>\n",
       "      <th>...</th>\n",
       "      <th>RelationshipSatisfaction</th>\n",
       "      <th>StandardHours</th>\n",
       "      <th>StockOptionLevel</th>\n",
       "      <th>TotalWorkingYears</th>\n",
       "      <th>TrainingTimesLastYear</th>\n",
       "      <th>WorkLifeBalance</th>\n",
       "      <th>YearsAtCompany</th>\n",
       "      <th>YearsInCurrentRole</th>\n",
       "      <th>YearsSinceLastPromotion</th>\n",
       "      <th>YearsWithCurrManager</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>0</td>\n",
       "      <td>1374</td>\n",
       "      <td>58</td>\n",
       "      <td>Travel_Rarely</td>\n",
       "      <td>605</td>\n",
       "      <td>Sales</td>\n",
       "      <td>21</td>\n",
       "      <td>3</td>\n",
       "      <td>Life Sciences</td>\n",
       "      <td>1</td>\n",
       "      <td>1938</td>\n",
       "      <td>...</td>\n",
       "      <td>3</td>\n",
       "      <td>80</td>\n",
       "      <td>1</td>\n",
       "      <td>29</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>1</td>\n",
       "      <td>1092</td>\n",
       "      <td>45</td>\n",
       "      <td>Travel_Rarely</td>\n",
       "      <td>950</td>\n",
       "      <td>Research &amp; Development</td>\n",
       "      <td>28</td>\n",
       "      <td>3</td>\n",
       "      <td>Technical Degree</td>\n",
       "      <td>1</td>\n",
       "      <td>1546</td>\n",
       "      <td>...</td>\n",
       "      <td>4</td>\n",
       "      <td>80</td>\n",
       "      <td>1</td>\n",
       "      <td>8</td>\n",
       "      <td>3</td>\n",
       "      <td>3</td>\n",
       "      <td>5</td>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>2</td>\n",
       "      <td>768</td>\n",
       "      <td>40</td>\n",
       "      <td>Travel_Rarely</td>\n",
       "      <td>300</td>\n",
       "      <td>Sales</td>\n",
       "      <td>26</td>\n",
       "      <td>3</td>\n",
       "      <td>Marketing</td>\n",
       "      <td>1</td>\n",
       "      <td>1066</td>\n",
       "      <td>...</td>\n",
       "      <td>2</td>\n",
       "      <td>80</td>\n",
       "      <td>1</td>\n",
       "      <td>8</td>\n",
       "      <td>3</td>\n",
       "      <td>2</td>\n",
       "      <td>7</td>\n",
       "      <td>7</td>\n",
       "      <td>7</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>3</td>\n",
       "      <td>569</td>\n",
       "      <td>36</td>\n",
       "      <td>Non-Travel</td>\n",
       "      <td>1434</td>\n",
       "      <td>Sales</td>\n",
       "      <td>8</td>\n",
       "      <td>4</td>\n",
       "      <td>Life Sciences</td>\n",
       "      <td>1</td>\n",
       "      <td>789</td>\n",
       "      <td>...</td>\n",
       "      <td>2</td>\n",
       "      <td>80</td>\n",
       "      <td>0</td>\n",
       "      <td>10</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>10</td>\n",
       "      <td>7</td>\n",
       "      <td>0</td>\n",
       "      <td>9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>4</td>\n",
       "      <td>911</td>\n",
       "      <td>25</td>\n",
       "      <td>Travel_Frequently</td>\n",
       "      <td>599</td>\n",
       "      <td>Sales</td>\n",
       "      <td>24</td>\n",
       "      <td>1</td>\n",
       "      <td>Life Sciences</td>\n",
       "      <td>1</td>\n",
       "      <td>1273</td>\n",
       "      <td>...</td>\n",
       "      <td>4</td>\n",
       "      <td>80</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>4</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>1171</td>\n",
       "      <td>763</td>\n",
       "      <td>34</td>\n",
       "      <td>Travel_Rarely</td>\n",
       "      <td>1333</td>\n",
       "      <td>Sales</td>\n",
       "      <td>10</td>\n",
       "      <td>4</td>\n",
       "      <td>Life Sciences</td>\n",
       "      <td>1</td>\n",
       "      <td>1055</td>\n",
       "      <td>...</td>\n",
       "      <td>4</td>\n",
       "      <td>80</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>1172</td>\n",
       "      <td>835</td>\n",
       "      <td>35</td>\n",
       "      <td>Travel_Rarely</td>\n",
       "      <td>528</td>\n",
       "      <td>Human Resources</td>\n",
       "      <td>8</td>\n",
       "      <td>4</td>\n",
       "      <td>Technical Degree</td>\n",
       "      <td>1</td>\n",
       "      <td>1164</td>\n",
       "      <td>...</td>\n",
       "      <td>2</td>\n",
       "      <td>80</td>\n",
       "      <td>0</td>\n",
       "      <td>6</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>5</td>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>1173</td>\n",
       "      <td>1216</td>\n",
       "      <td>43</td>\n",
       "      <td>Travel_Rarely</td>\n",
       "      <td>1179</td>\n",
       "      <td>Sales</td>\n",
       "      <td>2</td>\n",
       "      <td>3</td>\n",
       "      <td>Medical</td>\n",
       "      <td>1</td>\n",
       "      <td>1706</td>\n",
       "      <td>...</td>\n",
       "      <td>1</td>\n",
       "      <td>80</td>\n",
       "      <td>1</td>\n",
       "      <td>10</td>\n",
       "      <td>3</td>\n",
       "      <td>3</td>\n",
       "      <td>10</td>\n",
       "      <td>9</td>\n",
       "      <td>8</td>\n",
       "      <td>8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>1174</td>\n",
       "      <td>559</td>\n",
       "      <td>38</td>\n",
       "      <td>Travel_Rarely</td>\n",
       "      <td>268</td>\n",
       "      <td>Research &amp; Development</td>\n",
       "      <td>2</td>\n",
       "      <td>5</td>\n",
       "      <td>Medical</td>\n",
       "      <td>1</td>\n",
       "      <td>773</td>\n",
       "      <td>...</td>\n",
       "      <td>2</td>\n",
       "      <td>80</td>\n",
       "      <td>1</td>\n",
       "      <td>6</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>1175</td>\n",
       "      <td>684</td>\n",
       "      <td>40</td>\n",
       "      <td>Travel_Rarely</td>\n",
       "      <td>658</td>\n",
       "      <td>Sales</td>\n",
       "      <td>10</td>\n",
       "      <td>4</td>\n",
       "      <td>Marketing</td>\n",
       "      <td>1</td>\n",
       "      <td>954</td>\n",
       "      <td>...</td>\n",
       "      <td>2</td>\n",
       "      <td>80</td>\n",
       "      <td>1</td>\n",
       "      <td>11</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>1176 rows × 35 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "      user_id  Age     BusinessTravel  DailyRate              Department  \\\n",
       "0        1374   58      Travel_Rarely        605                   Sales   \n",
       "1        1092   45      Travel_Rarely        950  Research & Development   \n",
       "2         768   40      Travel_Rarely        300                   Sales   \n",
       "3         569   36         Non-Travel       1434                   Sales   \n",
       "4         911   25  Travel_Frequently        599                   Sales   \n",
       "...       ...  ...                ...        ...                     ...   \n",
       "1171      763   34      Travel_Rarely       1333                   Sales   \n",
       "1172      835   35      Travel_Rarely        528         Human Resources   \n",
       "1173     1216   43      Travel_Rarely       1179                   Sales   \n",
       "1174      559   38      Travel_Rarely        268  Research & Development   \n",
       "1175      684   40      Travel_Rarely        658                   Sales   \n",
       "\n",
       "      DistanceFromHome  Education    EducationField  EmployeeCount  \\\n",
       "0                   21          3     Life Sciences              1   \n",
       "1                   28          3  Technical Degree              1   \n",
       "2                   26          3         Marketing              1   \n",
       "3                    8          4     Life Sciences              1   \n",
       "4                   24          1     Life Sciences              1   \n",
       "...                ...        ...               ...            ...   \n",
       "1171                10          4     Life Sciences              1   \n",
       "1172                 8          4  Technical Degree              1   \n",
       "1173                 2          3           Medical              1   \n",
       "1174                 2          5           Medical              1   \n",
       "1175                10          4         Marketing              1   \n",
       "\n",
       "      EmployeeNumber  ...  RelationshipSatisfaction StandardHours  \\\n",
       "0               1938  ...                         3            80   \n",
       "1               1546  ...                         4            80   \n",
       "2               1066  ...                         2            80   \n",
       "3                789  ...                         2            80   \n",
       "4               1273  ...                         4            80   \n",
       "...              ...  ...                       ...           ...   \n",
       "1171            1055  ...                         4            80   \n",
       "1172            1164  ...                         2            80   \n",
       "1173            1706  ...                         1            80   \n",
       "1174             773  ...                         2            80   \n",
       "1175             954  ...                         2            80   \n",
       "\n",
       "      StockOptionLevel  TotalWorkingYears  TrainingTimesLastYear  \\\n",
       "0                    1                 29                      2   \n",
       "1                    1                  8                      3   \n",
       "2                    1                  8                      3   \n",
       "3                    0                 10                      1   \n",
       "4                    0                  1                      4   \n",
       "...                ...                ...                    ...   \n",
       "1171                 1                  1                      2   \n",
       "1172                 0                  6                      2   \n",
       "1173                 1                 10                      3   \n",
       "1174                 1                  6                      0   \n",
       "1175                 1                 11                      2   \n",
       "\n",
       "     WorkLifeBalance  YearsAtCompany YearsInCurrentRole  \\\n",
       "0                  2               1                  0   \n",
       "1                  3               5                  4   \n",
       "2                  2               7                  7   \n",
       "3                  3              10                  7   \n",
       "4                  3               1                  0   \n",
       "...              ...             ...                ...   \n",
       "1171               3               1                  1   \n",
       "1172               1               5                  4   \n",
       "1173               3              10                  9   \n",
       "1174               1               1                  0   \n",
       "1175               2               1                  0   \n",
       "\n",
       "      YearsSinceLastPromotion  YearsWithCurrManager  \n",
       "0                           0                     0  \n",
       "1                           0                     3  \n",
       "2                           7                     5  \n",
       "3                           0                     9  \n",
       "4                           1                     0  \n",
       "...                       ...                   ...  \n",
       "1171                        0                     0  \n",
       "1172                        1                     4  \n",
       "1173                        8                     8  \n",
       "1174                        0                     1  \n",
       "1175                        0                     0  \n",
       "\n",
       "[1176 rows x 35 columns]"
      ]
     },
     "execution_count": 101,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "    \n",
    "labels = train_data[\"Attrition\"]\n",
    "features = train_data.drop([\"Attrition\"],axis=1)\n",
    "features"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[False  True False False False False False False False False False False\n",
      " False  True  True False  True  True  True False False False  True False\n",
      " False False False  True  True False False  True  True False  True]\n",
      "[1.03093970e-01 3.74200684e+01 2.94692928e-02 5.05795946e+00\n",
      " 4.05310924e+00 6.14456958e+00 1.33350415e+00 1.67840804e+00\n",
      "            nan 1.03093970e-01 1.08348062e+01 1.70432317e+00\n",
      " 1.42500142e-02 1.55372408e+01 3.61847478e+01 3.05778076e+00\n",
      " 1.15123502e+01 2.83558039e+01 5.00240170e+01 1.42375594e+00\n",
      " 1.65823966e+00            nan 6.28526879e+01 1.98308139e+00\n",
      " 4.32887094e-03 1.43863957e+00            nan 2.01087356e+01\n",
      " 4.04001972e+01 3.69895595e+00 6.45277762e+00 2.51310770e+01\n",
      " 3.45263403e+01 2.15919089e+00 3.23776114e+01]\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/charles/opt/anaconda3/lib/python3.7/site-packages/sklearn/feature_selection/univariate_selection.py:114: UserWarning: Features [ 8 21 26] are constant.\n",
      "  UserWarning)\n",
      "/Users/charles/opt/anaconda3/lib/python3.7/site-packages/sklearn/feature_selection/univariate_selection.py:115: RuntimeWarning: invalid value encountered in true_divide\n",
      "  f = msb / msw\n"
     ]
    }
   ],
   "source": [
    "\n",
    "# use the selector to get the most useable items.\n",
    "selection = SelectPercentile(percentile=33)\n",
    "selection.fit(features,labels)\n",
    "selection.transform(features)\n",
    "print(selection.get_support())\n",
    "print(selection.scores_)\n",
    "\n",
    "features_new = features.loc[:,selection.get_support()]\n",
    "features_new\n",
    "test_data_new = test_data.loc[:,selection.get_support()]\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.model_selection import KFold\n",
    "\n",
    "def score(clf):\n",
    "\n",
    "    kf = KFold(n_splits=10,shuffle=True)\n",
    "    score = 0\n",
    "    count = 0\n",
    "    for train_index,test_index in kf.split(features_new):\n",
    "    #     print(\"TRAIN\",train_index,'\\n','TEST',test_index)\n",
    "        features_train, features_test = features_new.iloc[train_index], features_new.iloc[test_index]\n",
    "        labels_train, labels_test = labels.iloc[train_index], labels.iloc[test_index]\n",
    "        \n",
    "        clf = clf.fit(features_train,labels_train)\n",
    "        score += clf.score(features_test,labels_test)\n",
    "        if count == 9:\n",
    "            mean_score = score/10\n",
    "        count += 1\n",
    "    return mean_score\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.837621324062002"
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.ensemble import AdaBoostClassifier\n",
    "\n",
    "clf = AdaBoostClassifier(n_estimators=1000)\n",
    "clf.fit(features_new,labels)\n",
    "pred = clf.predict(test_data_new)\n",
    "np.savetxt('result.csv',pred,delimiter=',',fmt='%d',header='result')\n",
    "score(clf)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [],
   "source": [
    "def modelfit(alg,features=features_new,labels=labels,test_data=test_data_new,useTrain=True,cv=5,early=50):\n",
    "    if useTrain:\n",
    "        xgb_param = alg.get_xgb_params()\n",
    "        xgtrain = xgb.DMatrix(features.values,label=labels.values)\n",
    "        xgtest = xgb.DMatrix(test_data.values)\n",
    "        cvresult = xgb.cv(xgb_param,xgtrain,num_boost_round=alg.get_params()['n_estimators'],nfold=cv,metrics='auc',early_stopping_rounds=early)\n",
    "        print(cvresult.shape[0])\n",
    "        alg.set_params(n_estimators=cvresult.shape[0])\n",
    "\n",
    "    alg.fit(features,labels,eval_metric='auc')\n",
    "    \n",
    "    dtrain_predictions = alg.predict(features)\n",
    "    dtrain_predprob = alg.predict_proba(features)[:,1]\n",
    "\n",
    "    print('Model Report')\n",
    "    print('Accuracy:%.4g' % metrics.accuracy_score(labels.values,dtrain_predictions))\n",
    "    print('AUC Score(train):%f' % metrics.roc_auc_score(labels,dtrain_predprob))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "36\n",
      "Model Report\n",
      "Accuracy:0.9116\n",
      "AUC Score(train):0.954615\n"
     ]
    }
   ],
   "source": [
    "import xgboost as xgb\n",
    "from xgboost.sklearn import XGBClassifier\n",
    "\n",
    "xgb1 = XGBClassifier(\n",
    "    learning_rate=0.1,\n",
    "    n_estimators=1000,\n",
    "    max_depth=5,\n",
    "    min_child_weight=1,\n",
    "    gamma=0,\n",
    "    subsample=0.8,\n",
    "    colsample_bytree=0.8,\n",
    "    objective='binary:logistic',\n",
    "    nthread=4,\n",
    "    scale_pos_weight=1,\n",
    "    seed=27)\n",
    "modelfit(xgb1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 104,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "294"
      ]
     },
     "execution_count": 104,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pred = xgb1.predict_proba(test_data_new)\n",
    "pre=pred[:,1]\n",
    "len(pre)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 112,
   "metadata": {},
   "outputs": [],
   "source": [
    "\n",
    "\n",
    "# test_data.insert(1,\"Attrition\",pre)\n",
    "test_data.iloc[:,:2].to_csv('result.csv',index=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "GridSearchCV(cv=5, error_score='raise-deprecating',\n",
       "             estimator=XGBClassifier(base_score=0.5, booster='gbtree',\n",
       "                                     colsample_bylevel=1, colsample_bynode=1,\n",
       "                                     colsample_bytree=0.8, gamma=0,\n",
       "                                     learning_rate=0.1, max_delta_step=0,\n",
       "                                     max_depth=5, min_child_weight=1,\n",
       "                                     missing=None, n_estimators=50, n_jobs=1,\n",
       "                                     nthread=4, objective='binary:logistic',\n",
       "                                     random_state=0, reg_alpha=0, reg_lambda=1,\n",
       "                                     scale_pos_weight=1, seed=27, silent=None,\n",
       "                                     subsample=0.8, verbosity=1),\n",
       "             iid=False, n_jobs=4,\n",
       "             param_grid={'max_depth': range(3, 10),\n",
       "                         'min_child_weight': range(0, 10)},\n",
       "             pre_dispatch='2*n_jobs', refit=True, return_train_score=False,\n",
       "             scoring='roc_auc', verbose=0)"
      ]
     },
     "execution_count": 51,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# test paras just try but not make sense.\n",
    "param_test1 = {\n",
    "    'max_depth':range(3,10),\n",
    "    'min_child_weight':range(0,10)\n",
    "}\n",
    "gs1 = GridSearchCV(estimator=XGBClassifier(learning_rate=0.1,n_estimators=50,max_depth=5,\n",
    "                                min_child_weight=1,gamma=0,subsample=0.8,colsample_bytree=0.8,\n",
    "                                objective='binary:logistic',nthread=4,scale_pos_weight=1,seed=27),\n",
    "                param_grid=param_test1,scoring='roc_auc',n_jobs=4,iid=False,cv=5)\n",
    "gs1.fit(features,labels)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}