lab5.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.5.2"
    },
    "colab": {
      "name": "lab5.ipynb",
      "provenance": [],
      "collapsed_sections": [
        "Zpq4QOU5Wg-H",
        "i_7DyyXRWg-K",
        "_JewKs4XU-so",
        "5yiLk1P_xYQ2",
        "VlWxW3e9Wg-m",
        "D39SSh0zWg-r",
        "rhVrgkSaWg_K",
        "XsRf9T_SWg_U",
        "ylKZG2MwWg_f",
        "9hedBdcYWhAH",
        "JrqW55jgWhAR",
        "5QYTwyMtWhAZ",
        "DbJrUpARWhAd",
        "MI18l-l9WhAk",
        "1wrEGqBSWhAr",
        "gStgBJy2WhAx"
      ]
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "UHX9p5jfTySS"
      },
      "source": [
        "## Задание 5.1"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "0EnHNZtbXlH0"
      },
      "source": [
        "Набор данных тут: https://github.com/sismetanin/rureviews, также есть в папке [Data](https://drive.google.com/drive/folders/1YAMe7MiTxA-RSSd8Ex2p-L0Dspe6Gs4L). Те, кто предпочитает работать с английским языком, могут использовать набор данных `sms_spam`."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "bJox-LoonoPx"
      },
      "source": [
        "Применим полученные навыки и решим задачу анализа тональности отзывов. \n",
        "\n",
        "Нужно повторить весь пайплайн от сырых текстов до получения обученной модели.\n",
        "\n",
        "Обязательные шаги предобработки:\n",
        "1. токенизация\n",
        "2. приведение к нижнему регистру\n",
        "3. удаление стоп-слов\n",
        "4. лемматизация\n",
        "5. векторизация (с настройкой гиперпараметров)\n",
        "6. построение модели\n",
        "7. оценка качества модели\n",
        "\n",
        "Обязательно использование векторайзеров:\n",
        "1. мешок n-грамм (диапазон для n подбирайте самостоятельно, запрещено использовать только униграммы).\n",
        "2. tf-idf ((диапазон для n подбирайте самостоятельно, также нужно подбирать параметры max_df, min_df, max_features)\n",
        "3. символьные n-граммы (диапазон для n подбирайте самостоятельно)\n",
        "\n",
        "В качестве классификатора нужно использовать наивный байесовский классификатор. \n",
        "\n",
        "Для сравнения векторайзеров между собой используйте precision, recall, f1-score и accuracy. Для этого сформируйте датафрейм, в котором в строках будут разные векторайзеры, а в столбцах разные метрики качества, а в  ячейках будут значения этих метрик для соответсвующих векторайзеров."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Dw_3jA8P8Hdr"
      },
      "source": [
        "import pandas as pd\n",
        "import numpy as np"
      ],
      "execution_count": 19,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "z4uFjWoe8KwZ",
        "outputId": "2930c16a-f777-4207-8eb4-5ced29110096"
      },
      "source": [
        "from google.colab import drive\n",
        "drive.mount('/content/drive')"
      ],
      "execution_count": 20,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Mounted at /content/drive\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "BURaeY6F8SpJ"
      },
      "source": [
        "sms_spam_tbl = pd.read_csv('/content/drive/MyDrive/data/sms_spam.csv')"
      ],
      "execution_count": 43,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 359
        },
        "id": "FdBHxAZe8kMZ",
        "outputId": "fbba6b3a-b61b-4a66-f345-e4cd85d18348"
      },
      "source": [
        "sms_spam_tbl.head(10)"
      ],
      "execution_count": 44,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>type</th>\n",
              "      <th>text</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>ham</td>\n",
              "      <td>Hope you are having a good week. Just checking in</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>ham</td>\n",
              "      <td>K..give back my thanks.</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>ham</td>\n",
              "      <td>Am also doing in cbe only. But have to pay.</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>spam</td>\n",
              "      <td>complimentary 4 STAR Ibiza Holiday or £10,000 ...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>spam</td>\n",
              "      <td>okmail: Dear Dave this is your final notice to...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>5</th>\n",
              "      <td>ham</td>\n",
              "      <td>Aiya we discuss later lar... Pick u up at 4 is...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>6</th>\n",
              "      <td>ham</td>\n",
              "      <td>Are you this much buzy</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>7</th>\n",
              "      <td>ham</td>\n",
              "      <td>Please ask mummy to call father</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>8</th>\n",
              "      <td>spam</td>\n",
              "      <td>Marvel Mobile Play the official Ultimate Spide...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>9</th>\n",
              "      <td>ham</td>\n",
              "      <td>fyi I'm at usf now, swing by the room whenever</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "   type                                               text\n",
              "0   ham  Hope you are having a good week. Just checking in\n",
              "1   ham                            K..give back my thanks.\n",
              "2   ham        Am also doing in cbe only. But have to pay.\n",
              "3  spam  complimentary 4 STAR Ibiza Holiday or £10,000 ...\n",
              "4  spam  okmail: Dear Dave this is your final notice to...\n",
              "5   ham  Aiya we discuss later lar... Pick u up at 4 is...\n",
              "6   ham                             Are you this much buzy\n",
              "7   ham                    Please ask mummy to call father\n",
              "8  spam  Marvel Mobile Play the official Ultimate Spide...\n",
              "9   ham     fyi I'm at usf now, swing by the room whenever"
            ]
          },
          "metadata": {},
          "execution_count": 44
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "bVh4l33U9Ihj",
        "outputId": "5218197a-aea6-42e3-a591-ae87174268f0"
      },
      "source": [
        "sms_spam_tbl.shape"
      ],
      "execution_count": 45,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(5559, 2)"
            ]
          },
          "metadata": {},
          "execution_count": 45
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "XxRh5ycN9V4d"
      },
      "source": [
        "sms_spam_tbl['numType'] = sms_spam_tbl['type'].map({'ham':0, 'spam':1})\n",
        "sms_spam_tbl.drop(columns=['type'], inplace=True)"
      ],
      "execution_count": 46,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "0fiJsYE6-5dd"
      },
      "source": [
        "sms_spam_tbl['count']=0\n",
        "for i in np.arange(0,len(sms_spam_tbl.text)):\n",
        "    sms_spam_tbl.loc[i,'count'] = len(sms_spam_tbl.loc[i,'text'])"
      ],
      "execution_count": 47,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "JjuZ0s8GXXE0"
      },
      "source": [
        "import lightgbm as lgb\n",
        "from sklearn.model_selection import RandomizedSearchCV\n",
        "lgbmodel_bst = lgb.LGBMClassifier(max_depth=6, n_estimators=200, num_leaves=40)\n",
        "param_grid = {\n",
        "    'num_leaves': list(range(8, 92, 4)),\n",
        "    'min_data_in_leaf': [10, 20, 40, 60, 100],\n",
        "    'max_depth': [3, 4, 5, 6, 8, 12, 16, -1],\n",
        "    'learning_rate': [0.1, 0.05, 0.01, 0.005],\n",
        "    'bagging_freq': [3, 4, 5, 6, 7],\n",
        "    'bagging_fraction': np.linspace(0.6, 0.95, 10),\n",
        "    'reg_alpha': np.linspace(0.1, 0.95, 10),\n",
        "    'reg_lambda': np.linspace(0.1, 0.95, 10),\n",
        "    \"min_split_gain\": [0.0, 0.1, 0.01],\n",
        "    \"min_child_weight\": [0.001, 0.01, 0.1, 0.001],\n",
        "    \"min_child_samples\": [20, 30, 25],\n",
        "    \"subsample\": [1.0, 0.5, 0.8],\n",
        "}\n",
        "model = RandomizedSearchCV(lgbmodel_bst, param_grid, random_state=42)\n",
        "# search = model.fit(X_train, y_train)\n",
        "# search.best_params_"
      ],
      "execution_count": 72,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "oxNxV7qt0dHs",
        "outputId": "5dc2f264-d90e-4e71-db0d-fe894f91b52c"
      },
      "source": [
        "import nltk\n",
        "from nltk.tokenize import word_tokenize \n",
        "from nltk.corpus import stopwords         \n",
        "from nltk.stem import WordNetLemmatizer \n",
        "nltk.download('wordnet')\n",
        "nltk.download('punkt')\n",
        "nltk.download('stopwords')"
      ],
      "execution_count": 59,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "[nltk_data] Downloading package wordnet to /root/nltk_data...\n",
            "[nltk_data]   Package wordnet is already up-to-date!\n",
            "[nltk_data] Downloading package punkt to /root/nltk_data...\n",
            "[nltk_data]   Package punkt is already up-to-date!\n",
            "[nltk_data] Downloading package stopwords to /root/nltk_data...\n",
            "[nltk_data]   Unzipping corpora/stopwords.zip.\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "True"
            ]
          },
          "metadata": {},
          "execution_count": 59
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "MistKP0mzfQ5"
      },
      "source": [
        "wnl = WordNetLemmatizer()\n",
        "sms_spam_lem = sms_spam_tbl.copy()\n",
        "stop_words = set(stopwords.words('english'))\n",
        "sms_spam_lem['text'] = sms_spam_lem['text'].apply(lambda x: ' '.join(t for t in x.split() if t.lower() not in stop_words))\n",
        "sms_spam_lem['text'] = sms_spam_lem['text'].apply(lambda x: ' '.join(wnl.lemmatize(t.lower()) for t in word_tokenize(x)))\n"
      ],
      "execution_count": 65,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 204
        },
        "id": "qkZQBVYFOixm",
        "outputId": "68133946-c505-4de7-94d5-b82f2f6cd0c1"
      },
      "source": [
        "sms_spam_lem.head()"
      ],
      "execution_count": 66,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>text</th>\n",
              "      <th>numType</th>\n",
              "      <th>count</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>hope good week . checking</td>\n",
              "      <td>0</td>\n",
              "      <td>49</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>k..give back thanks .</td>\n",
              "      <td>0</td>\n",
              "      <td>23</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>also cbe only . pay .</td>\n",
              "      <td>0</td>\n",
              "      <td>43</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>complimentary 4 star ibiza holiday £10,000 cas...</td>\n",
              "      <td>1</td>\n",
              "      <td>149</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>okmail : dear dave final notice collect 4* ten...</td>\n",
              "      <td>1</td>\n",
              "      <td>161</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "                                                text  numType  count\n",
              "0                          hope good week . checking        0     49\n",
              "1                              k..give back thanks .        0     23\n",
              "2                              also cbe only . pay .        0     43\n",
              "3  complimentary 4 star ibiza holiday £10,000 cas...        1    149\n",
              "4  okmail : dear dave final notice collect 4* ten...        1    161"
            ]
          },
          "metadata": {},
          "execution_count": 66
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "xevNoqryxOI-"
      },
      "source": [
        "# different vectorizers\n",
        "from sklearn.feature_extraction.text import TfidfVectorizer\n",
        "from sklearn.decomposition import TruncatedSVD\n",
        "# TF-IDF\n",
        "tfidf_model = TfidfVectorizer()\n",
        "sms_spam_tfidf = sms_spam_lem.copy()\n",
        "tfidf_text = tfidf_model.fit_transform(sms_spam_tfidf['text'])\n",
        "\n",
        "svd = TruncatedSVD(n_components=500, n_iter=5)\n",
        "tfidf_svd = svd.fit_transform(tfidf_text)\n",
        "tfidf_svd = pd.DataFrame(tfidf_svd)"
      ],
      "execution_count": 74,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "CyT5PsUH1QsC"
      },
      "source": [
        "tfidf_svd['count'] = (sms_spam_lem['count'] - sms_spam_lem['count'].min()) / (sms_spam_lem['count'].max() - sms_spam_lem['count'].min())\n"
      ],
      "execution_count": 75,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "mVWCEFnUG8KG"
      },
      "source": [
        "from sklearn.model_selection import train_test_split\n",
        "X_train,X_test,y_train,y_test = train_test_split(tfidf_svd, sms_spam_lem['numType'],test_size=.2, random_state=42)"
      ],
      "execution_count": 94,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "UJ5ogGEZ3NTD",
        "outputId": "155c239f-9610-4c2d-97ad-9ff04ca50c0c",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "source": [
        "search = model.fit(X_train, y_train)\n",
        "search.best_params_"
      ],
      "execution_count": 76,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "{'bagging_fraction': 0.7166666666666667,\n",
              " 'bagging_freq': 6,\n",
              " 'learning_rate': 0.05,\n",
              " 'max_depth': 16,\n",
              " 'min_child_samples': 25,\n",
              " 'min_child_weight': 0.1,\n",
              " 'min_data_in_leaf': 100,\n",
              " 'min_split_gain': 0.01,\n",
              " 'num_leaves': 28,\n",
              " 'reg_alpha': 0.8555555555555555,\n",
              " 'reg_lambda': 0.3833333333333333,\n",
              " 'subsample': 0.5}"
            ]
          },
          "metadata": {},
          "execution_count": 76
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "4kB0WLD45inD",
        "outputId": "c67253f1-f68a-4cce-ecaa-dcd8c6aea450",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "source": [
        "best_model = lgb.LGBMClassifier(bagging_fraction=0.7166666666666667, bagging_freq=6,\n",
        "               boosting_type='gbdt', class_weight=None, colsample_bytree=1.0,\n",
        "               importance_type='split', learning_rate=0.05, max_depth=16,\n",
        "               min_child_samples=25, min_child_weight=0.1, min_data_in_leaf=100,\n",
        "               min_split_gain=0.01, n_estimators=100, n_jobs=-1, num_leaves=28,\n",
        "               objective=None, random_state=None, reg_alpha=0.8555555555555555,\n",
        "               reg_lambda=0.3833333333333333, silent=True, subsample=0.5,\n",
        "               subsample_for_bin=200000, subsample_freq=0)\n",
        "best_model.fit(X_train,y_train)"
      ],
      "execution_count": 95,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "LGBMClassifier(bagging_fraction=0.7166666666666667, bagging_freq=6,\n",
              "               boosting_type='gbdt', class_weight=None, colsample_bytree=1.0,\n",
              "               importance_type='split', learning_rate=0.05, max_depth=16,\n",
              "               min_child_samples=25, min_child_weight=0.1, min_data_in_leaf=100,\n",
              "               min_split_gain=0.01, n_estimators=100, n_jobs=-1, num_leaves=28,\n",
              "               objective=None, random_state=None, reg_alpha=0.8555555555555555,\n",
              "               reg_lambda=0.3833333333333333, silent=True, subsample=0.5,\n",
              "               subsample_for_bin=200000, subsample_freq=0)"
            ]
          },
          "metadata": {},
          "execution_count": 95
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "y6qAspnq6ED1",
        "outputId": "a62124ad-d451-4d75-c2d8-b455b58b78fd",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "source": [
        "from sklearn.metrics import f1_score, precision_score, recall_score\n",
        "prediction = best_model.predict(X_test)\n",
        "print(f'F1 score is: {f1_score(prediction, y_test)}')\n",
        "print(f'Precision score is: {precision_score(prediction, y_test)}')\n",
        "print(f'Recall score is: {recall_score(prediction, y_test)}')"
      ],
      "execution_count": 97,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "F1 score is: 0.9045936395759717\n",
            "Precision score is: 0.8590604026845637\n",
            "Recall score is: 0.9552238805970149\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 338
        },
        "id": "YNt9ee7-YDJz",
        "outputId": "0e5af548-7d12-406c-aaae-2e418c25002a"
      },
      "source": [
        "# Count\n",
        "from sklearn.feature_extraction.text import CountVectorizer\n",
        "ngram_model = CountVectorizer(binary=True, ngram_range=(2, 3))\n",
        "sms_spam_count = sms_spam_lem.copy()\n",
        "count_text = ngram_model.fit_transform(sms_spam_tfidf['text'])\n",
        "\n",
        "count_svd = svd.fit_transform(count_text)\n",
        "count_svd = pd.DataFrame(count_svd)\n",
        "count_svd['count'] = (sms_spam_lem['count'] - sms_spam_lem['count'].min()) / (sms_spam_lem['count'].max() - sms_spam_lem['count'].min())\n",
        "count_svd.head()\n"
      ],
      "execution_count": 81,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>0</th>\n",
              "      <th>1</th>\n",
              "      <th>2</th>\n",
              "      <th>3</th>\n",
              "      <th>4</th>\n",
              "      <th>5</th>\n",
              "      <th>6</th>\n",
              "      <th>7</th>\n",
              "      <th>8</th>\n",
              "      <th>9</th>\n",
              "      <th>10</th>\n",
              "      <th>11</th>\n",
              "      <th>12</th>\n",
              "      <th>13</th>\n",
              "      <th>14</th>\n",
              "      <th>15</th>\n",
              "      <th>16</th>\n",
              "      <th>17</th>\n",
              "      <th>18</th>\n",
              "      <th>19</th>\n",
              "      <th>20</th>\n",
              "      <th>21</th>\n",
              "      <th>22</th>\n",
              "      <th>23</th>\n",
              "      <th>24</th>\n",
              "      <th>25</th>\n",
              "      <th>26</th>\n",
              "      <th>27</th>\n",
              "      <th>28</th>\n",
              "      <th>29</th>\n",
              "      <th>30</th>\n",
              "      <th>31</th>\n",
              "      <th>32</th>\n",
              "      <th>33</th>\n",
              "      <th>34</th>\n",
              "      <th>35</th>\n",
              "      <th>36</th>\n",
              "      <th>37</th>\n",
              "      <th>38</th>\n",
              "      <th>39</th>\n",
              "      <th>...</th>\n",
              "      <th>461</th>\n",
              "      <th>462</th>\n",
              "      <th>463</th>\n",
              "      <th>464</th>\n",
              "      <th>465</th>\n",
              "      <th>466</th>\n",
              "      <th>467</th>\n",
              "      <th>468</th>\n",
              "      <th>469</th>\n",
              "      <th>470</th>\n",
              "      <th>471</th>\n",
              "      <th>472</th>\n",
              "      <th>473</th>\n",
              "      <th>474</th>\n",
              "      <th>475</th>\n",
              "      <th>476</th>\n",
              "      <th>477</th>\n",
              "      <th>478</th>\n",
              "      <th>479</th>\n",
              "      <th>480</th>\n",
              "      <th>481</th>\n",
              "      <th>482</th>\n",
              "      <th>483</th>\n",
              "      <th>484</th>\n",
              "      <th>485</th>\n",
              "      <th>486</th>\n",
              "      <th>487</th>\n",
              "      <th>488</th>\n",
              "      <th>489</th>\n",
              "      <th>490</th>\n",
              "      <th>491</th>\n",
              "      <th>492</th>\n",
              "      <th>493</th>\n",
              "      <th>494</th>\n",
              "      <th>495</th>\n",
              "      <th>496</th>\n",
              "      <th>497</th>\n",
              "      <th>498</th>\n",
              "      <th>499</th>\n",
              "      <th>count</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>2.097696e-07</td>\n",
              "      <td>1.552595e-06</td>\n",
              "      <td>2.138911e-07</td>\n",
              "      <td>2.490319e-06</td>\n",
              "      <td>7.858619e-06</td>\n",
              "      <td>3.219337e-06</td>\n",
              "      <td>1.875915e-06</td>\n",
              "      <td>-1.076652e-06</td>\n",
              "      <td>4.357234e-06</td>\n",
              "      <td>1.502624e-06</td>\n",
              "      <td>2.744853e-05</td>\n",
              "      <td>1.712693e-05</td>\n",
              "      <td>1.757276e-03</td>\n",
              "      <td>4.471953e-05</td>\n",
              "      <td>3.133917e-06</td>\n",
              "      <td>1.198259e-05</td>\n",
              "      <td>9.852387e-06</td>\n",
              "      <td>3.467911e-06</td>\n",
              "      <td>5.203996e-05</td>\n",
              "      <td>-3.132545e-06</td>\n",
              "      <td>8.751124e-05</td>\n",
              "      <td>-1.940539e-06</td>\n",
              "      <td>-6.010212e-06</td>\n",
              "      <td>-8.508948e-06</td>\n",
              "      <td>1.237719e-05</td>\n",
              "      <td>1.372762e-05</td>\n",
              "      <td>5.659176e-07</td>\n",
              "      <td>1.230278e-05</td>\n",
              "      <td>7.383484e-06</td>\n",
              "      <td>5.170928e-05</td>\n",
              "      <td>1.544727e-05</td>\n",
              "      <td>6.905023e-05</td>\n",
              "      <td>1.262060e-05</td>\n",
              "      <td>4.163107e-05</td>\n",
              "      <td>1.584765e-05</td>\n",
              "      <td>2.658645e-05</td>\n",
              "      <td>1.663303e-06</td>\n",
              "      <td>-1.476843e-05</td>\n",
              "      <td>-8.390584e-06</td>\n",
              "      <td>7.570636e-05</td>\n",
              "      <td>...</td>\n",
              "      <td>0.043482</td>\n",
              "      <td>0.009371</td>\n",
              "      <td>9.784203e-03</td>\n",
              "      <td>0.002003</td>\n",
              "      <td>0.020357</td>\n",
              "      <td>-1.922356e-02</td>\n",
              "      <td>3.034544e-03</td>\n",
              "      <td>-0.000646</td>\n",
              "      <td>3.539007e-02</td>\n",
              "      <td>1.481289e-02</td>\n",
              "      <td>0.000970</td>\n",
              "      <td>-0.018494</td>\n",
              "      <td>0.001005</td>\n",
              "      <td>-0.014508</td>\n",
              "      <td>0.030927</td>\n",
              "      <td>6.084611e-02</td>\n",
              "      <td>8.219020e-03</td>\n",
              "      <td>0.015295</td>\n",
              "      <td>0.007246</td>\n",
              "      <td>0.073083</td>\n",
              "      <td>-0.018132</td>\n",
              "      <td>-0.017889</td>\n",
              "      <td>-0.014504</td>\n",
              "      <td>-0.012402</td>\n",
              "      <td>-0.009816</td>\n",
              "      <td>-0.033111</td>\n",
              "      <td>0.015429</td>\n",
              "      <td>-0.006119</td>\n",
              "      <td>0.011565</td>\n",
              "      <td>-0.008338</td>\n",
              "      <td>-0.009027</td>\n",
              "      <td>-0.036728</td>\n",
              "      <td>-0.024045</td>\n",
              "      <td>0.004453</td>\n",
              "      <td>0.012792</td>\n",
              "      <td>-6.264789e-03</td>\n",
              "      <td>0.013066</td>\n",
              "      <td>-0.019998</td>\n",
              "      <td>-0.023405</td>\n",
              "      <td>0.051762</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>2.765201e-08</td>\n",
              "      <td>4.606746e-08</td>\n",
              "      <td>8.984553e-08</td>\n",
              "      <td>2.193636e-07</td>\n",
              "      <td>-4.451388e-08</td>\n",
              "      <td>3.526507e-07</td>\n",
              "      <td>-4.281100e-08</td>\n",
              "      <td>1.135240e-07</td>\n",
              "      <td>-2.161888e-07</td>\n",
              "      <td>-2.093590e-08</td>\n",
              "      <td>-7.485903e-08</td>\n",
              "      <td>7.698865e-08</td>\n",
              "      <td>6.332972e-07</td>\n",
              "      <td>3.663309e-07</td>\n",
              "      <td>2.808749e-07</td>\n",
              "      <td>-3.784598e-07</td>\n",
              "      <td>7.741332e-06</td>\n",
              "      <td>1.045611e-07</td>\n",
              "      <td>3.645099e-08</td>\n",
              "      <td>-7.049310e-07</td>\n",
              "      <td>2.922706e-06</td>\n",
              "      <td>-7.805591e-07</td>\n",
              "      <td>-1.623914e-06</td>\n",
              "      <td>1.182005e-06</td>\n",
              "      <td>1.002589e-06</td>\n",
              "      <td>7.836369e-07</td>\n",
              "      <td>2.273399e-06</td>\n",
              "      <td>2.005388e-06</td>\n",
              "      <td>-7.781898e-07</td>\n",
              "      <td>5.042939e-06</td>\n",
              "      <td>-2.784225e-06</td>\n",
              "      <td>-7.586874e-07</td>\n",
              "      <td>-1.468564e-07</td>\n",
              "      <td>-4.417874e-07</td>\n",
              "      <td>8.678385e-07</td>\n",
              "      <td>-1.155073e-06</td>\n",
              "      <td>-9.419038e-07</td>\n",
              "      <td>3.924526e-08</td>\n",
              "      <td>-3.636916e-06</td>\n",
              "      <td>1.414112e-05</td>\n",
              "      <td>...</td>\n",
              "      <td>0.003119</td>\n",
              "      <td>0.005411</td>\n",
              "      <td>-3.357364e-03</td>\n",
              "      <td>-0.002116</td>\n",
              "      <td>0.000829</td>\n",
              "      <td>2.678155e-03</td>\n",
              "      <td>1.053067e-03</td>\n",
              "      <td>-0.003084</td>\n",
              "      <td>2.357363e-03</td>\n",
              "      <td>1.544547e-03</td>\n",
              "      <td>0.003821</td>\n",
              "      <td>0.003478</td>\n",
              "      <td>-0.001477</td>\n",
              "      <td>-0.003238</td>\n",
              "      <td>-0.000658</td>\n",
              "      <td>2.792125e-03</td>\n",
              "      <td>2.000401e-04</td>\n",
              "      <td>0.002266</td>\n",
              "      <td>0.001617</td>\n",
              "      <td>0.000946</td>\n",
              "      <td>-0.003597</td>\n",
              "      <td>0.002841</td>\n",
              "      <td>0.000383</td>\n",
              "      <td>0.005869</td>\n",
              "      <td>0.000147</td>\n",
              "      <td>0.003328</td>\n",
              "      <td>-0.004740</td>\n",
              "      <td>-0.003905</td>\n",
              "      <td>-0.007770</td>\n",
              "      <td>-0.005241</td>\n",
              "      <td>0.003822</td>\n",
              "      <td>0.000604</td>\n",
              "      <td>0.001700</td>\n",
              "      <td>-0.004547</td>\n",
              "      <td>0.000849</td>\n",
              "      <td>4.452507e-03</td>\n",
              "      <td>0.000523</td>\n",
              "      <td>-0.006332</td>\n",
              "      <td>0.000995</td>\n",
              "      <td>0.023128</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>2.867426e-11</td>\n",
              "      <td>-6.639903e-12</td>\n",
              "      <td>-7.883944e-11</td>\n",
              "      <td>-7.950413e-11</td>\n",
              "      <td>1.520481e-11</td>\n",
              "      <td>1.823723e-10</td>\n",
              "      <td>1.592245e-10</td>\n",
              "      <td>-1.113614e-10</td>\n",
              "      <td>3.471506e-10</td>\n",
              "      <td>-4.000615e-10</td>\n",
              "      <td>4.897428e-10</td>\n",
              "      <td>2.449179e-10</td>\n",
              "      <td>5.178418e-10</td>\n",
              "      <td>4.568748e-10</td>\n",
              "      <td>-4.281801e-10</td>\n",
              "      <td>-4.394713e-10</td>\n",
              "      <td>8.687718e-10</td>\n",
              "      <td>-8.733943e-10</td>\n",
              "      <td>-2.067250e-10</td>\n",
              "      <td>-1.123522e-09</td>\n",
              "      <td>-4.674181e-10</td>\n",
              "      <td>1.695185e-10</td>\n",
              "      <td>1.860616e-09</td>\n",
              "      <td>7.767027e-10</td>\n",
              "      <td>2.601390e-09</td>\n",
              "      <td>4.763778e-09</td>\n",
              "      <td>4.285622e-10</td>\n",
              "      <td>1.379876e-09</td>\n",
              "      <td>7.539128e-10</td>\n",
              "      <td>1.953613e-09</td>\n",
              "      <td>4.655050e-09</td>\n",
              "      <td>-1.678955e-09</td>\n",
              "      <td>2.475055e-09</td>\n",
              "      <td>2.115408e-09</td>\n",
              "      <td>6.717008e-11</td>\n",
              "      <td>-3.944155e-09</td>\n",
              "      <td>-3.648521e-09</td>\n",
              "      <td>2.953663e-09</td>\n",
              "      <td>-7.761209e-11</td>\n",
              "      <td>-4.006860e-09</td>\n",
              "      <td>...</td>\n",
              "      <td>-0.000005</td>\n",
              "      <td>0.000003</td>\n",
              "      <td>1.739377e-07</td>\n",
              "      <td>0.000005</td>\n",
              "      <td>-0.000003</td>\n",
              "      <td>5.809459e-07</td>\n",
              "      <td>1.687603e-07</td>\n",
              "      <td>-0.000005</td>\n",
              "      <td>2.772638e-07</td>\n",
              "      <td>2.775997e-07</td>\n",
              "      <td>0.000006</td>\n",
              "      <td>0.000003</td>\n",
              "      <td>-0.000003</td>\n",
              "      <td>0.000005</td>\n",
              "      <td>0.000002</td>\n",
              "      <td>-2.846019e-07</td>\n",
              "      <td>-5.640066e-07</td>\n",
              "      <td>-0.000004</td>\n",
              "      <td>0.000008</td>\n",
              "      <td>-0.000001</td>\n",
              "      <td>0.000007</td>\n",
              "      <td>-0.000003</td>\n",
              "      <td>-0.000003</td>\n",
              "      <td>-0.000005</td>\n",
              "      <td>0.000005</td>\n",
              "      <td>-0.000010</td>\n",
              "      <td>-0.000008</td>\n",
              "      <td>0.000004</td>\n",
              "      <td>-0.000001</td>\n",
              "      <td>0.000004</td>\n",
              "      <td>-0.000006</td>\n",
              "      <td>-0.000001</td>\n",
              "      <td>-0.000008</td>\n",
              "      <td>0.000005</td>\n",
              "      <td>0.000003</td>\n",
              "      <td>6.968276e-07</td>\n",
              "      <td>-0.000001</td>\n",
              "      <td>-0.000006</td>\n",
              "      <td>-0.000003</td>\n",
              "      <td>0.045154</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>2.816110e-05</td>\n",
              "      <td>2.505973e-07</td>\n",
              "      <td>1.217910e-06</td>\n",
              "      <td>2.338123e-03</td>\n",
              "      <td>1.487615e-04</td>\n",
              "      <td>1.522361e-01</td>\n",
              "      <td>5.753328e-02</td>\n",
              "      <td>2.079618e-03</td>\n",
              "      <td>6.779565e-03</td>\n",
              "      <td>1.158916e-03</td>\n",
              "      <td>7.065134e-03</td>\n",
              "      <td>7.499200e-03</td>\n",
              "      <td>1.640361e-04</td>\n",
              "      <td>1.416280e-01</td>\n",
              "      <td>-6.159122e-04</td>\n",
              "      <td>1.103654e-04</td>\n",
              "      <td>2.239396e-05</td>\n",
              "      <td>5.629691e-06</td>\n",
              "      <td>-5.476934e-06</td>\n",
              "      <td>-1.840657e-01</td>\n",
              "      <td>-2.267535e-04</td>\n",
              "      <td>-1.411134e-03</td>\n",
              "      <td>7.018339e-05</td>\n",
              "      <td>2.092573e-01</td>\n",
              "      <td>1.657434e-01</td>\n",
              "      <td>1.148029e+00</td>\n",
              "      <td>2.267188e-02</td>\n",
              "      <td>3.704388e-04</td>\n",
              "      <td>-9.692172e-03</td>\n",
              "      <td>-1.162964e-04</td>\n",
              "      <td>-8.984737e-05</td>\n",
              "      <td>2.277300e-05</td>\n",
              "      <td>-1.625218e-02</td>\n",
              "      <td>-3.545087e-01</td>\n",
              "      <td>-1.372769e-02</td>\n",
              "      <td>1.475613e-02</td>\n",
              "      <td>1.796780e-02</td>\n",
              "      <td>7.634295e-03</td>\n",
              "      <td>3.146761e-02</td>\n",
              "      <td>-2.139506e-04</td>\n",
              "      <td>...</td>\n",
              "      <td>-0.038867</td>\n",
              "      <td>-0.092931</td>\n",
              "      <td>-8.620495e-02</td>\n",
              "      <td>-0.066302</td>\n",
              "      <td>-0.099958</td>\n",
              "      <td>-4.791931e-02</td>\n",
              "      <td>1.009236e-01</td>\n",
              "      <td>-0.138400</td>\n",
              "      <td>-4.082140e-02</td>\n",
              "      <td>-7.152187e-02</td>\n",
              "      <td>-0.134333</td>\n",
              "      <td>-0.008394</td>\n",
              "      <td>-0.107434</td>\n",
              "      <td>0.014668</td>\n",
              "      <td>0.120629</td>\n",
              "      <td>1.225097e-01</td>\n",
              "      <td>-8.864720e-02</td>\n",
              "      <td>-0.018443</td>\n",
              "      <td>-0.151342</td>\n",
              "      <td>0.149977</td>\n",
              "      <td>-0.244241</td>\n",
              "      <td>-0.004477</td>\n",
              "      <td>-0.021950</td>\n",
              "      <td>0.044366</td>\n",
              "      <td>-0.000062</td>\n",
              "      <td>-0.006344</td>\n",
              "      <td>0.174557</td>\n",
              "      <td>-0.002952</td>\n",
              "      <td>0.032431</td>\n",
              "      <td>0.080364</td>\n",
              "      <td>-0.118220</td>\n",
              "      <td>0.060367</td>\n",
              "      <td>0.016927</td>\n",
              "      <td>-0.163519</td>\n",
              "      <td>-0.106430</td>\n",
              "      <td>6.415258e-02</td>\n",
              "      <td>-0.031137</td>\n",
              "      <td>0.051390</td>\n",
              "      <td>0.003654</td>\n",
              "      <td>0.161894</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>2.691703e-05</td>\n",
              "      <td>9.213285e-07</td>\n",
              "      <td>-9.153333e-07</td>\n",
              "      <td>2.687159e-03</td>\n",
              "      <td>1.930803e-04</td>\n",
              "      <td>2.082442e-01</td>\n",
              "      <td>2.431806e-01</td>\n",
              "      <td>2.139806e-03</td>\n",
              "      <td>-1.845655e-05</td>\n",
              "      <td>6.537373e-03</td>\n",
              "      <td>3.968245e-03</td>\n",
              "      <td>6.847380e-03</td>\n",
              "      <td>1.010897e-04</td>\n",
              "      <td>9.187775e-02</td>\n",
              "      <td>-3.884441e-04</td>\n",
              "      <td>7.354411e-05</td>\n",
              "      <td>1.546438e-06</td>\n",
              "      <td>5.583395e-07</td>\n",
              "      <td>-7.197417e-06</td>\n",
              "      <td>-7.070964e-02</td>\n",
              "      <td>-1.582995e-04</td>\n",
              "      <td>-1.030884e-03</td>\n",
              "      <td>-4.011120e-07</td>\n",
              "      <td>6.630409e-02</td>\n",
              "      <td>6.095542e-02</td>\n",
              "      <td>4.387281e-01</td>\n",
              "      <td>8.062919e-03</td>\n",
              "      <td>1.225718e-04</td>\n",
              "      <td>-2.679889e-03</td>\n",
              "      <td>-8.370008e-06</td>\n",
              "      <td>2.148912e-05</td>\n",
              "      <td>-3.523017e-05</td>\n",
              "      <td>-3.262680e-03</td>\n",
              "      <td>-5.790524e-02</td>\n",
              "      <td>-2.245172e-03</td>\n",
              "      <td>-7.763734e-04</td>\n",
              "      <td>2.628167e-03</td>\n",
              "      <td>-4.029879e-05</td>\n",
              "      <td>4.889094e-05</td>\n",
              "      <td>-2.907453e-05</td>\n",
              "      <td>...</td>\n",
              "      <td>-0.019592</td>\n",
              "      <td>-0.014934</td>\n",
              "      <td>2.969042e-04</td>\n",
              "      <td>-0.029504</td>\n",
              "      <td>-0.063729</td>\n",
              "      <td>-3.968727e-02</td>\n",
              "      <td>7.522807e-03</td>\n",
              "      <td>-0.050426</td>\n",
              "      <td>9.918325e-03</td>\n",
              "      <td>-3.770332e-02</td>\n",
              "      <td>-0.050573</td>\n",
              "      <td>-0.007302</td>\n",
              "      <td>-0.058538</td>\n",
              "      <td>-0.003761</td>\n",
              "      <td>0.039880</td>\n",
              "      <td>2.058976e-02</td>\n",
              "      <td>-4.094193e-02</td>\n",
              "      <td>-0.043875</td>\n",
              "      <td>-0.005294</td>\n",
              "      <td>0.038069</td>\n",
              "      <td>-0.040584</td>\n",
              "      <td>-0.002172</td>\n",
              "      <td>0.001564</td>\n",
              "      <td>0.011201</td>\n",
              "      <td>-0.007095</td>\n",
              "      <td>-0.010002</td>\n",
              "      <td>0.032937</td>\n",
              "      <td>0.000394</td>\n",
              "      <td>0.001071</td>\n",
              "      <td>0.003633</td>\n",
              "      <td>-0.012206</td>\n",
              "      <td>0.032266</td>\n",
              "      <td>-0.022017</td>\n",
              "      <td>-0.025088</td>\n",
              "      <td>-0.004531</td>\n",
              "      <td>-7.759771e-03</td>\n",
              "      <td>-0.026284</td>\n",
              "      <td>-0.004685</td>\n",
              "      <td>-0.028835</td>\n",
              "      <td>0.175110</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>5 rows × 501 columns</p>\n",
              "</div>"
            ],
            "text/plain": [
              "              0             1             2  ...       498       499     count\n",
              "0  2.097696e-07  1.552595e-06  2.138911e-07  ... -0.019998 -0.023405  0.051762\n",
              "1  2.765201e-08  4.606746e-08  8.984553e-08  ... -0.006332  0.000995  0.023128\n",
              "2  2.867426e-11 -6.639903e-12 -7.883944e-11  ... -0.000006 -0.000003  0.045154\n",
              "3  2.816110e-05  2.505973e-07  1.217910e-06  ...  0.051390  0.003654  0.161894\n",
              "4  2.691703e-05  9.213285e-07 -9.153333e-07  ... -0.004685 -0.028835  0.175110\n",
              "\n",
              "[5 rows x 501 columns]"
            ]
          },
          "metadata": {},
          "execution_count": 81
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "3Ld1HfazHJqp"
      },
      "source": [
        "X_train,X_test,y_train,y_test = train_test_split(count_svd, sms_spam_lem['numType'],test_size=.2, random_state=42)"
      ],
      "execution_count": 98,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "hPZf4odR-Xwe",
        "outputId": "b96179e9-2fa6-43d2-a0a0-da423d98e1da",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "source": [
        "search = model.fit(X_train, y_train)\n",
        "search.best_params_"
      ],
      "execution_count": 82,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "{'bagging_fraction': 0.8333333333333333,\n",
              " 'bagging_freq': 6,\n",
              " 'learning_rate': 0.1,\n",
              " 'max_depth': 6,\n",
              " 'min_child_samples': 20,\n",
              " 'min_child_weight': 0.001,\n",
              " 'min_data_in_leaf': 60,\n",
              " 'min_split_gain': 0.01,\n",
              " 'num_leaves': 72,\n",
              " 'reg_alpha': 0.28888888888888886,\n",
              " 'reg_lambda': 0.95,\n",
              " 'subsample': 1.0}"
            ]
          },
          "metadata": {},
          "execution_count": 82
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "vHUdamDM-gNi",
        "outputId": "2a06d229-f019-4ea5-a463-c62b451e04ca",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "source": [
        "best_model = lgb.LGBMClassifier(bagging_fraction=0.8333333333333333, bagging_freq=6,\n",
        "               boosting_type='gbdt', class_weight=None, colsample_bytree=1.0,\n",
        "               importance_type='split', learning_rate=0.1, max_depth=6,\n",
        "               min_child_samples=20, min_child_weight=0.001,\n",
        "               min_data_in_leaf=60, min_split_gain=0.01, n_estimators=100,\n",
        "               n_jobs=-1, num_leaves=72, objective=None, random_state=None,\n",
        "               reg_alpha=0.28888888888888886, reg_lambda=0.95, silent=True,\n",
        "               subsample=1.0, subsample_for_bin=200000, subsample_freq=0)\n",
        "best_model.fit(X_train,y_train)"
      ],
      "execution_count": 99,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "LGBMClassifier(bagging_fraction=0.8333333333333333, bagging_freq=6,\n",
              "               boosting_type='gbdt', class_weight=None, colsample_bytree=1.0,\n",
              "               importance_type='split', learning_rate=0.1, max_depth=6,\n",
              "               min_child_samples=20, min_child_weight=0.001,\n",
              "               min_data_in_leaf=60, min_split_gain=0.01, n_estimators=100,\n",
              "               n_jobs=-1, num_leaves=72, objective=None, random_state=None,\n",
              "               reg_alpha=0.28888888888888886, reg_lambda=0.95, silent=True,\n",
              "               subsample=1.0, subsample_for_bin=200000, subsample_freq=0)"
            ]
          },
          "metadata": {},
          "execution_count": 99
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "DfUAG8hV-pDw",
        "outputId": "2044aa9d-4669-4026-caf5-e6d95b1194b3",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "source": [
        "prediction = best_model.predict(X_test)\n",
        "print(f'F1 score is: {f1_score(prediction, y_test)}')\n",
        "print(f'Precision score is: {precision_score(prediction, y_test)}')\n",
        "print(f'Recall score is: {recall_score(prediction, y_test)}')"
      ],
      "execution_count": 100,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "F1 score is: 0.8951048951048951\n",
            "Precision score is: 0.8590604026845637\n",
            "Recall score is: 0.9343065693430657\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 253
        },
        "id": "M5iD-mjVYC_5",
        "outputId": "0445c2f4-8f35-4464-a9a9-3c6667ddbd31"
      },
      "source": [
        "# Count(analyzer='char_wb')\n",
        "ngram_char_model = CountVectorizer(analyzer='char_wb', ngram_range=(3, 5))\n",
        "sms_spam_char = sms_spam_lem.copy()\n",
        "char_text = ngram_char_model.fit_transform(sms_spam_tfidf['text'])\n",
        "\n",
        "svd = TruncatedSVD(n_components=1000, n_iter=8)\n",
        "char_svd = svd.fit_transform(char_text)\n",
        "char_svd = pd.DataFrame(char_svd) \n",
        "char_svd.head()"
      ],
      "execution_count": 109,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>0</th>\n",
              "      <th>1</th>\n",
              "      <th>2</th>\n",
              "      <th>3</th>\n",
              "      <th>4</th>\n",
              "      <th>5</th>\n",
              "      <th>6</th>\n",
              "      <th>7</th>\n",
              "      <th>8</th>\n",
              "      <th>9</th>\n",
              "      <th>10</th>\n",
              "      <th>11</th>\n",
              "      <th>12</th>\n",
              "      <th>13</th>\n",
              "      <th>14</th>\n",
              "      <th>15</th>\n",
              "      <th>16</th>\n",
              "      <th>17</th>\n",
              "      <th>18</th>\n",
              "      <th>19</th>\n",
              "      <th>20</th>\n",
              "      <th>21</th>\n",
              "      <th>22</th>\n",
              "      <th>23</th>\n",
              "      <th>24</th>\n",
              "      <th>25</th>\n",
              "      <th>26</th>\n",
              "      <th>27</th>\n",
              "      <th>28</th>\n",
              "      <th>29</th>\n",
              "      <th>30</th>\n",
              "      <th>31</th>\n",
              "      <th>32</th>\n",
              "      <th>33</th>\n",
              "      <th>34</th>\n",
              "      <th>35</th>\n",
              "      <th>36</th>\n",
              "      <th>37</th>\n",
              "      <th>38</th>\n",
              "      <th>39</th>\n",
              "      <th>...</th>\n",
              "      <th>960</th>\n",
              "      <th>961</th>\n",
              "      <th>962</th>\n",
              "      <th>963</th>\n",
              "      <th>964</th>\n",
              "      <th>965</th>\n",
              "      <th>966</th>\n",
              "      <th>967</th>\n",
              "      <th>968</th>\n",
              "      <th>969</th>\n",
              "      <th>970</th>\n",
              "      <th>971</th>\n",
              "      <th>972</th>\n",
              "      <th>973</th>\n",
              "      <th>974</th>\n",
              "      <th>975</th>\n",
              "      <th>976</th>\n",
              "      <th>977</th>\n",
              "      <th>978</th>\n",
              "      <th>979</th>\n",
              "      <th>980</th>\n",
              "      <th>981</th>\n",
              "      <th>982</th>\n",
              "      <th>983</th>\n",
              "      <th>984</th>\n",
              "      <th>985</th>\n",
              "      <th>986</th>\n",
              "      <th>987</th>\n",
              "      <th>988</th>\n",
              "      <th>989</th>\n",
              "      <th>990</th>\n",
              "      <th>991</th>\n",
              "      <th>992</th>\n",
              "      <th>993</th>\n",
              "      <th>994</th>\n",
              "      <th>995</th>\n",
              "      <th>996</th>\n",
              "      <th>997</th>\n",
              "      <th>998</th>\n",
              "      <th>999</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>1.714960</td>\n",
              "      <td>-0.130918</td>\n",
              "      <td>1.259888</td>\n",
              "      <td>-0.184210</td>\n",
              "      <td>0.124364</td>\n",
              "      <td>-0.491307</td>\n",
              "      <td>0.538396</td>\n",
              "      <td>0.237396</td>\n",
              "      <td>-0.271196</td>\n",
              "      <td>-0.357786</td>\n",
              "      <td>0.302153</td>\n",
              "      <td>0.137371</td>\n",
              "      <td>0.671720</td>\n",
              "      <td>0.468999</td>\n",
              "      <td>-0.886192</td>\n",
              "      <td>-0.489958</td>\n",
              "      <td>0.262880</td>\n",
              "      <td>0.622010</td>\n",
              "      <td>0.609022</td>\n",
              "      <td>0.550962</td>\n",
              "      <td>1.105368</td>\n",
              "      <td>0.300502</td>\n",
              "      <td>-0.856719</td>\n",
              "      <td>-1.018277</td>\n",
              "      <td>0.773912</td>\n",
              "      <td>-1.123802</td>\n",
              "      <td>0.817168</td>\n",
              "      <td>0.138417</td>\n",
              "      <td>0.054941</td>\n",
              "      <td>0.279321</td>\n",
              "      <td>0.275934</td>\n",
              "      <td>-0.001016</td>\n",
              "      <td>0.281553</td>\n",
              "      <td>-0.683935</td>\n",
              "      <td>0.242442</td>\n",
              "      <td>-0.651878</td>\n",
              "      <td>0.450863</td>\n",
              "      <td>0.108981</td>\n",
              "      <td>0.179686</td>\n",
              "      <td>-0.848024</td>\n",
              "      <td>...</td>\n",
              "      <td>-0.001356</td>\n",
              "      <td>0.044290</td>\n",
              "      <td>0.078820</td>\n",
              "      <td>-0.160322</td>\n",
              "      <td>0.009244</td>\n",
              "      <td>-0.047518</td>\n",
              "      <td>0.104653</td>\n",
              "      <td>-0.011920</td>\n",
              "      <td>0.087705</td>\n",
              "      <td>0.000868</td>\n",
              "      <td>0.052208</td>\n",
              "      <td>-0.158802</td>\n",
              "      <td>-0.154245</td>\n",
              "      <td>-0.040679</td>\n",
              "      <td>-0.005286</td>\n",
              "      <td>-0.066975</td>\n",
              "      <td>0.012730</td>\n",
              "      <td>0.036830</td>\n",
              "      <td>0.084221</td>\n",
              "      <td>-0.039421</td>\n",
              "      <td>-0.031732</td>\n",
              "      <td>0.058918</td>\n",
              "      <td>0.034107</td>\n",
              "      <td>-0.132350</td>\n",
              "      <td>0.000978</td>\n",
              "      <td>0.072296</td>\n",
              "      <td>0.059865</td>\n",
              "      <td>0.059105</td>\n",
              "      <td>0.015642</td>\n",
              "      <td>-0.083454</td>\n",
              "      <td>0.010003</td>\n",
              "      <td>0.045923</td>\n",
              "      <td>-0.036202</td>\n",
              "      <td>-0.021922</td>\n",
              "      <td>-0.118280</td>\n",
              "      <td>0.082544</td>\n",
              "      <td>-0.015420</td>\n",
              "      <td>-0.003367</td>\n",
              "      <td>-0.025317</td>\n",
              "      <td>-0.052082</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>0.916560</td>\n",
              "      <td>-0.298704</td>\n",
              "      <td>0.282819</td>\n",
              "      <td>0.658428</td>\n",
              "      <td>-0.112788</td>\n",
              "      <td>-0.002591</td>\n",
              "      <td>-0.489749</td>\n",
              "      <td>-0.013324</td>\n",
              "      <td>0.110176</td>\n",
              "      <td>-0.369065</td>\n",
              "      <td>-0.245601</td>\n",
              "      <td>0.027738</td>\n",
              "      <td>-0.249268</td>\n",
              "      <td>-0.423099</td>\n",
              "      <td>-0.282976</td>\n",
              "      <td>0.551186</td>\n",
              "      <td>0.280282</td>\n",
              "      <td>-0.322442</td>\n",
              "      <td>-0.305802</td>\n",
              "      <td>0.244530</td>\n",
              "      <td>0.836742</td>\n",
              "      <td>-0.402883</td>\n",
              "      <td>0.126966</td>\n",
              "      <td>0.013096</td>\n",
              "      <td>0.076345</td>\n",
              "      <td>-0.388110</td>\n",
              "      <td>0.449004</td>\n",
              "      <td>0.009330</td>\n",
              "      <td>0.225700</td>\n",
              "      <td>0.092481</td>\n",
              "      <td>-0.002924</td>\n",
              "      <td>0.357967</td>\n",
              "      <td>0.008440</td>\n",
              "      <td>-0.595793</td>\n",
              "      <td>-0.332182</td>\n",
              "      <td>0.503579</td>\n",
              "      <td>-0.361145</td>\n",
              "      <td>0.276901</td>\n",
              "      <td>-0.093682</td>\n",
              "      <td>-0.140152</td>\n",
              "      <td>...</td>\n",
              "      <td>0.086698</td>\n",
              "      <td>0.004417</td>\n",
              "      <td>-0.088896</td>\n",
              "      <td>-0.097971</td>\n",
              "      <td>-0.106093</td>\n",
              "      <td>-0.139587</td>\n",
              "      <td>-0.042003</td>\n",
              "      <td>-0.047951</td>\n",
              "      <td>0.051599</td>\n",
              "      <td>0.120364</td>\n",
              "      <td>-0.141055</td>\n",
              "      <td>-0.038880</td>\n",
              "      <td>0.045243</td>\n",
              "      <td>0.169572</td>\n",
              "      <td>0.004798</td>\n",
              "      <td>0.035776</td>\n",
              "      <td>-0.011539</td>\n",
              "      <td>-0.049465</td>\n",
              "      <td>-0.072505</td>\n",
              "      <td>-0.027311</td>\n",
              "      <td>0.055633</td>\n",
              "      <td>0.038163</td>\n",
              "      <td>0.062145</td>\n",
              "      <td>-0.093137</td>\n",
              "      <td>0.027899</td>\n",
              "      <td>-0.102148</td>\n",
              "      <td>0.036810</td>\n",
              "      <td>0.022978</td>\n",
              "      <td>0.062484</td>\n",
              "      <td>0.059171</td>\n",
              "      <td>0.068484</td>\n",
              "      <td>-0.045465</td>\n",
              "      <td>-0.013959</td>\n",
              "      <td>-0.101099</td>\n",
              "      <td>-0.000082</td>\n",
              "      <td>-0.057984</td>\n",
              "      <td>0.081073</td>\n",
              "      <td>-0.022199</td>\n",
              "      <td>0.057221</td>\n",
              "      <td>0.034223</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>1.154676</td>\n",
              "      <td>-0.509862</td>\n",
              "      <td>0.282723</td>\n",
              "      <td>1.319404</td>\n",
              "      <td>-0.160060</td>\n",
              "      <td>-0.588202</td>\n",
              "      <td>0.007661</td>\n",
              "      <td>0.380039</td>\n",
              "      <td>-0.145732</td>\n",
              "      <td>-0.286529</td>\n",
              "      <td>0.105394</td>\n",
              "      <td>0.119200</td>\n",
              "      <td>0.016516</td>\n",
              "      <td>-0.021766</td>\n",
              "      <td>-0.074911</td>\n",
              "      <td>0.044578</td>\n",
              "      <td>0.087553</td>\n",
              "      <td>-0.208879</td>\n",
              "      <td>-0.141525</td>\n",
              "      <td>0.199658</td>\n",
              "      <td>-0.001627</td>\n",
              "      <td>0.009459</td>\n",
              "      <td>-0.275326</td>\n",
              "      <td>0.082875</td>\n",
              "      <td>0.022838</td>\n",
              "      <td>0.036055</td>\n",
              "      <td>-0.131069</td>\n",
              "      <td>-0.032037</td>\n",
              "      <td>0.056158</td>\n",
              "      <td>-0.005624</td>\n",
              "      <td>-0.040978</td>\n",
              "      <td>-0.068034</td>\n",
              "      <td>-0.117600</td>\n",
              "      <td>-0.122154</td>\n",
              "      <td>0.008197</td>\n",
              "      <td>-0.088750</td>\n",
              "      <td>-0.069664</td>\n",
              "      <td>-0.133958</td>\n",
              "      <td>-0.075819</td>\n",
              "      <td>-0.005841</td>\n",
              "      <td>...</td>\n",
              "      <td>-0.078535</td>\n",
              "      <td>-0.030820</td>\n",
              "      <td>0.104968</td>\n",
              "      <td>-0.022027</td>\n",
              "      <td>-0.003664</td>\n",
              "      <td>-0.046018</td>\n",
              "      <td>-0.005245</td>\n",
              "      <td>0.114014</td>\n",
              "      <td>-0.071472</td>\n",
              "      <td>0.095325</td>\n",
              "      <td>0.067464</td>\n",
              "      <td>-0.003545</td>\n",
              "      <td>-0.071529</td>\n",
              "      <td>0.013980</td>\n",
              "      <td>-0.003763</td>\n",
              "      <td>-0.082350</td>\n",
              "      <td>-0.044677</td>\n",
              "      <td>-0.039442</td>\n",
              "      <td>-0.010351</td>\n",
              "      <td>-0.092727</td>\n",
              "      <td>0.037859</td>\n",
              "      <td>0.054368</td>\n",
              "      <td>-0.027525</td>\n",
              "      <td>-0.055884</td>\n",
              "      <td>0.089554</td>\n",
              "      <td>-0.149298</td>\n",
              "      <td>0.005665</td>\n",
              "      <td>-0.014704</td>\n",
              "      <td>0.181859</td>\n",
              "      <td>0.005122</td>\n",
              "      <td>0.057794</td>\n",
              "      <td>0.005308</td>\n",
              "      <td>-0.164291</td>\n",
              "      <td>0.035635</td>\n",
              "      <td>0.005426</td>\n",
              "      <td>0.080663</td>\n",
              "      <td>0.014686</td>\n",
              "      <td>-0.038568</td>\n",
              "      <td>-0.102498</td>\n",
              "      <td>0.004311</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>3.400197</td>\n",
              "      <td>-1.699236</td>\n",
              "      <td>-2.390335</td>\n",
              "      <td>0.509867</td>\n",
              "      <td>-2.101982</td>\n",
              "      <td>1.783691</td>\n",
              "      <td>3.194849</td>\n",
              "      <td>-3.557769</td>\n",
              "      <td>-1.158486</td>\n",
              "      <td>-1.336057</td>\n",
              "      <td>1.829501</td>\n",
              "      <td>-0.976670</td>\n",
              "      <td>2.388802</td>\n",
              "      <td>1.044521</td>\n",
              "      <td>1.898434</td>\n",
              "      <td>0.180732</td>\n",
              "      <td>-1.534358</td>\n",
              "      <td>-0.509854</td>\n",
              "      <td>-2.037889</td>\n",
              "      <td>0.649599</td>\n",
              "      <td>0.362198</td>\n",
              "      <td>-2.229466</td>\n",
              "      <td>0.463786</td>\n",
              "      <td>0.636345</td>\n",
              "      <td>-0.227703</td>\n",
              "      <td>-0.353464</td>\n",
              "      <td>-0.834887</td>\n",
              "      <td>-1.175866</td>\n",
              "      <td>0.891739</td>\n",
              "      <td>-0.141221</td>\n",
              "      <td>-3.781485</td>\n",
              "      <td>-2.181723</td>\n",
              "      <td>-1.004918</td>\n",
              "      <td>-0.509211</td>\n",
              "      <td>1.040497</td>\n",
              "      <td>-1.329595</td>\n",
              "      <td>0.450708</td>\n",
              "      <td>0.961824</td>\n",
              "      <td>1.466138</td>\n",
              "      <td>1.585731</td>\n",
              "      <td>...</td>\n",
              "      <td>0.117494</td>\n",
              "      <td>-1.084144</td>\n",
              "      <td>0.160615</td>\n",
              "      <td>0.010309</td>\n",
              "      <td>-0.233459</td>\n",
              "      <td>-0.028858</td>\n",
              "      <td>0.393652</td>\n",
              "      <td>-0.158850</td>\n",
              "      <td>-0.931938</td>\n",
              "      <td>0.153792</td>\n",
              "      <td>-0.710515</td>\n",
              "      <td>-0.236269</td>\n",
              "      <td>-0.378885</td>\n",
              "      <td>-0.070866</td>\n",
              "      <td>-0.172704</td>\n",
              "      <td>-0.059613</td>\n",
              "      <td>-0.362567</td>\n",
              "      <td>-0.012259</td>\n",
              "      <td>-0.310517</td>\n",
              "      <td>-0.324667</td>\n",
              "      <td>-0.038180</td>\n",
              "      <td>-0.055773</td>\n",
              "      <td>0.207647</td>\n",
              "      <td>0.620337</td>\n",
              "      <td>0.240602</td>\n",
              "      <td>-0.184049</td>\n",
              "      <td>-0.398546</td>\n",
              "      <td>-0.265352</td>\n",
              "      <td>-0.218030</td>\n",
              "      <td>0.703275</td>\n",
              "      <td>-0.565718</td>\n",
              "      <td>0.017965</td>\n",
              "      <td>0.769156</td>\n",
              "      <td>-0.940334</td>\n",
              "      <td>-0.348995</td>\n",
              "      <td>0.095657</td>\n",
              "      <td>0.323708</td>\n",
              "      <td>0.312861</td>\n",
              "      <td>0.037932</td>\n",
              "      <td>-0.350764</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>3.501736</td>\n",
              "      <td>-2.280575</td>\n",
              "      <td>-3.467966</td>\n",
              "      <td>-0.493000</td>\n",
              "      <td>-3.127493</td>\n",
              "      <td>1.772344</td>\n",
              "      <td>2.006604</td>\n",
              "      <td>-1.347726</td>\n",
              "      <td>-0.363290</td>\n",
              "      <td>-1.372756</td>\n",
              "      <td>1.783914</td>\n",
              "      <td>1.002707</td>\n",
              "      <td>2.083135</td>\n",
              "      <td>0.204312</td>\n",
              "      <td>0.611118</td>\n",
              "      <td>-0.769753</td>\n",
              "      <td>-1.438870</td>\n",
              "      <td>-1.158014</td>\n",
              "      <td>-1.682491</td>\n",
              "      <td>0.570688</td>\n",
              "      <td>0.249760</td>\n",
              "      <td>-1.974686</td>\n",
              "      <td>-0.177602</td>\n",
              "      <td>0.793695</td>\n",
              "      <td>-1.177818</td>\n",
              "      <td>-0.333029</td>\n",
              "      <td>0.324841</td>\n",
              "      <td>-0.060527</td>\n",
              "      <td>0.717768</td>\n",
              "      <td>-0.233069</td>\n",
              "      <td>-2.550899</td>\n",
              "      <td>-1.415018</td>\n",
              "      <td>0.146813</td>\n",
              "      <td>-0.537219</td>\n",
              "      <td>1.446988</td>\n",
              "      <td>-2.718864</td>\n",
              "      <td>0.485503</td>\n",
              "      <td>0.965427</td>\n",
              "      <td>1.305122</td>\n",
              "      <td>2.570015</td>\n",
              "      <td>...</td>\n",
              "      <td>-0.013259</td>\n",
              "      <td>-0.033751</td>\n",
              "      <td>-0.054681</td>\n",
              "      <td>-0.014600</td>\n",
              "      <td>0.069647</td>\n",
              "      <td>-0.070502</td>\n",
              "      <td>0.067944</td>\n",
              "      <td>-0.016869</td>\n",
              "      <td>-0.057396</td>\n",
              "      <td>0.048976</td>\n",
              "      <td>-0.027164</td>\n",
              "      <td>-0.037220</td>\n",
              "      <td>-0.024124</td>\n",
              "      <td>0.158582</td>\n",
              "      <td>-0.118649</td>\n",
              "      <td>0.158648</td>\n",
              "      <td>-0.233218</td>\n",
              "      <td>-0.204824</td>\n",
              "      <td>-0.310360</td>\n",
              "      <td>-0.135280</td>\n",
              "      <td>0.164615</td>\n",
              "      <td>0.230196</td>\n",
              "      <td>0.030063</td>\n",
              "      <td>0.245133</td>\n",
              "      <td>0.090151</td>\n",
              "      <td>0.201276</td>\n",
              "      <td>-0.279134</td>\n",
              "      <td>-0.035670</td>\n",
              "      <td>-0.035241</td>\n",
              "      <td>0.166737</td>\n",
              "      <td>-0.018101</td>\n",
              "      <td>0.305343</td>\n",
              "      <td>0.001423</td>\n",
              "      <td>0.102610</td>\n",
              "      <td>-0.102453</td>\n",
              "      <td>0.094902</td>\n",
              "      <td>0.021427</td>\n",
              "      <td>0.053850</td>\n",
              "      <td>0.064184</td>\n",
              "      <td>-0.165717</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>5 rows × 1000 columns</p>\n",
              "</div>"
            ],
            "text/plain": [
              "        0         1         2    ...       997       998       999\n",
              "0  1.714960 -0.130918  1.259888  ... -0.003367 -0.025317 -0.052082\n",
              "1  0.916560 -0.298704  0.282819  ... -0.022199  0.057221  0.034223\n",
              "2  1.154676 -0.509862  0.282723  ... -0.038568 -0.102498  0.004311\n",
              "3  3.400197 -1.699236 -2.390335  ...  0.312861  0.037932 -0.350764\n",
              "4  3.501736 -2.280575 -3.467966  ...  0.053850  0.064184 -0.165717\n",
              "\n",
              "[5 rows x 1000 columns]"
            ]
          },
          "metadata": {},
          "execution_count": 109
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ti3D58ONHSlu"
      },
      "source": [
        "X_train,X_test,y_train,y_test = train_test_split(char_svd, sms_spam_lem['numType'],test_size=.2, random_state=42)"
      ],
      "execution_count": 110,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "JaCIBWhN_ElV",
        "outputId": "15b95a91-e97b-4d4f-8955-569f0261cafb",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "source": [
        "search = model.fit(X_train, y_train)\n",
        "search.best_params_"
      ],
      "execution_count": 111,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "{'bagging_fraction': 0.8333333333333333,\n",
              " 'bagging_freq': 6,\n",
              " 'learning_rate': 0.1,\n",
              " 'max_depth': 6,\n",
              " 'min_child_samples': 20,\n",
              " 'min_child_weight': 0.001,\n",
              " 'min_data_in_leaf': 60,\n",
              " 'min_split_gain': 0.01,\n",
              " 'num_leaves': 72,\n",
              " 'reg_alpha': 0.28888888888888886,\n",
              " 'reg_lambda': 0.95,\n",
              " 'subsample': 1.0}"
            ]
          },
          "metadata": {},
          "execution_count": 111
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "x7KC-Zg4DbHT",
        "outputId": "5ef733a0-5899-4d31-dc44-91bcb699da5f",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "source": [
        "best_model = lgb.LGBMClassifier(bagging_fraction=0.8333333333333333, bagging_freq=6,\n",
        "               boosting_type='gbdt', class_weight=None, colsample_bytree=1.0,\n",
        "               importance_type='split', learning_rate=0.1, max_depth=6,\n",
        "               min_child_samples=20, min_child_weight=0.001,\n",
        "               min_data_in_leaf=60, min_split_gain=0.01, n_estimators=100,\n",
        "               n_jobs=-1, num_leaves=72, objective=None, random_state=None,\n",
        "               reg_alpha=0.28888888888888886, reg_lambda=0.95, silent=True,\n",
        "               subsample=1.0, subsample_for_bin=200000, subsample_freq=0)\n",
        "best_model.fit(X_train,y_train)"
      ],
      "execution_count": 112,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "LGBMClassifier(bagging_fraction=0.8333333333333333, bagging_freq=6,\n",
              "               boosting_type='gbdt', class_weight=None, colsample_bytree=1.0,\n",
              "               importance_type='split', learning_rate=0.1, max_depth=6,\n",
              "               min_child_samples=20, min_child_weight=0.001,\n",
              "               min_data_in_leaf=60, min_split_gain=0.01, n_estimators=100,\n",
              "               n_jobs=-1, num_leaves=72, objective=None, random_state=None,\n",
              "               reg_alpha=0.28888888888888886, reg_lambda=0.95, silent=True,\n",
              "               subsample=1.0, subsample_for_bin=200000, subsample_freq=0)"
            ]
          },
          "metadata": {},
          "execution_count": 112
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "27ZMEfeQ_QO8",
        "outputId": "5e786aa9-b2c8-4df5-fec4-4654d3b2d769",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "source": [
        "prediction = best_model.predict(X_test)\n",
        "print(f'F1 score is: {f1_score(prediction, y_test)}')\n",
        "print(f'Precision score is: {precision_score(prediction, y_test)}')\n",
        "print(f'Recall score is: {recall_score(prediction, y_test)}')"
      ],
      "execution_count": 113,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "F1 score is: 0.9225352112676056\n",
            "Precision score is: 0.8791946308724832\n",
            "Recall score is: 0.9703703703703703\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "hLWIeeTnHuj-"
      },
      "source": [
        "## Сonclusion\n",
        "By all metrics **CountVectorizer** with **char with word boundaries** is perfomming better when **TF-IDF** and ordinary  **CountVectorizer**. All of them performed quite well, so the results could be combine for the better outcome. If I wanted to improve the model's score, I would use the Ensemble of these models."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5QYTwyMtWhAZ"
      },
      "source": [
        "## Задание 5.2 Регулярные выражения\n",
        "\n",
        "Регулярные выражения - способ поиска и анализа строк. Например, можно понять, какие даты в наборе строк представлены в формате DD/MM/YYYY, а какие - в других форматах. \n",
        "\n",
        "Или бывает, например, что перед работой с текстом, надо почистить его от своеобразного мусора: упоминаний пользователей, url и так далее.\n",
        "\n",
        "Навык полезный, давайте в нём тоже потренируемся.\n",
        "\n",
        "Для работы с регулярными выражениями есть библиотека **re**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "VaUW5S4gWhAb"
      },
      "source": [
        "import re"
      ],
      "execution_count": 2,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "D6aYh7Osl8xr"
      },
      "source": [
        "В регулярных выражениях, кроме привычных символов-букв, есть специальные символы:\n",
        "* **?а** - ноль или один символ **а**\n",
        "* **+а** - один или более символов **а**\n",
        "* **\\*а** - ноль или более символов **а** (не путать с +)\n",
        "* **.** - любое количество любого символа\n",
        "\n",
        "Пример:\n",
        "Выражению \\*a?b. соответствуют последовательности a, ab, abc, aa, aac НО НЕ abb!"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "q7zOFFA3l_KQ"
      },
      "source": [
        "Рассмотрим подробно несколько наиболее полезных функций:"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "DbJrUpARWhAd"
      },
      "source": [
        "### findall\n",
        "возвращает список всех найденных непересекающихся совпадений.\n",
        "\n",
        "Регулярное выражение **ab+c.**: \n",
        "* **a** - просто символ **a**\n",
        "* **b+** - один или более символов **b**\n",
        "* **c** - просто символ **c**\n",
        "* **.** - любой символ\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "2athHzKuWhAd",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "4c390b10-f3a3-4200-d912-3c5868a1bd81"
      },
      "source": [
        "result = re.findall('ab+c.', 'abcdefghijkabcabcxabc') \n",
        "print(result)"
      ],
      "execution_count": 16,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "['abcd', 'abca']\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "A9FpIw5RWhAf"
      },
      "source": [
        "Вопрос на внимательность: почему нет abcx?"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ytS6XM1LxlW5"
      },
      "source": [
        " Оно пересекается с 'abca'\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "B5ttzoxEWhAg"
      },
      "source": [
        "**Задание**: вернуть список первых двух букв каждого слова в строке, состоящей из нескольких слов."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "7ZR2AEq3WhAg",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "7ca509f4-6f60-4e8e-b5d7-a05986d30a28"
      },
      "source": [
        "text = \"AV is largest Analytics community of India\"\n",
        "result = re.findall(r'\\b\\w{1,2}', text) \n",
        "print(result)"
      ],
      "execution_count": 29,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "['AV', 'is', 'la', 'An', 'I', 'co', 'of', 'In']\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "MI18l-l9WhAk"
      },
      "source": [
        "### split\n",
        "разделяет строку по заданному шаблону\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "sVKdRoc1WhAl",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "78b5d289-8e8c-4621-fe3c-34aa130c727c"
      },
      "source": [
        "result = re.split(',', 'itsy, bitsy, teenie, weenie') \n",
        "print(result)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "['itsy', ' bitsy', ' teenie', ' weenie']\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "10u5efuSWhAm"
      },
      "source": [
        "можно указать максимальное количество разбиений"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "9U9EQZMwWhAn",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "45699604-9950-4185-cd4f-0e3ad6655629"
      },
      "source": [
        "result = re.split(',', 'itsy, bitsy, teenie, weenie', maxsplit=2) \n",
        "print(result)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "['itsy', ' bitsy', ' teenie, weenie']\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "0EMcMyflWhAp"
      },
      "source": [
        "**Задание**: разбейте строку, состоящую из нескольких предложений, по точкам, но не более чем на 3 предложения."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "dVgPSjEOWhAp",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "98b06c9a-4461-407b-9638-377099782f0e"
      },
      "source": [
        "result = re.split(r'\\.', 'itsy. bitsy. teenie. weenie.', maxsplit=2) \n",
        "print(result)"
      ],
      "execution_count": 32,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "['itsy', ' bitsy', ' teenie. weenie.']\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "1wrEGqBSWhAr"
      },
      "source": [
        "### sub\n",
        "ищет шаблон в строке и заменяет все совпадения на указанную подстроку\n",
        "\n",
        "параметры: (pattern, repl, string)"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "az3KxKWwWhAr",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "93f260be-ce79-405c-b21a-5ca8345ff4b7"
      },
      "source": [
        "result = re.sub('a', 'b', 'abcabc')\n",
        "print (result)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "bbcbbc\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "qD0n7_HPWhAt"
      },
      "source": [
        "**Задание**: напишите регулярное выражение, которое позволит заменить все цифры в строке на \"DIG\"."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "s_Sdu7xlWhAu",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "46e7953d-b757-4bea-bea6-99f783a3e4fb"
      },
      "source": [
        "result = re.sub(r'\\d', 'DIG', '132')\n",
        "print(result)"
      ],
      "execution_count": 35,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "DIGDIGDIG\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "b8__oi1PWhAv"
      },
      "source": [
        "**Задание**: напишите  регулярное выражение, которое позволит убрать url из строки."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "KwNS9zt4WhAv",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "1faf9bfe-68dc-4690-8ff7-83138b2df69d"
      },
      "source": [
        "result = re.sub(r'\\b\\S+://\\S+\\b', '', 'URL could have the form http://www.example.com/index.html, which indicates a protocol ( http )')\n",
        "print(result)"
      ],
      "execution_count": 7,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "URL could have the form , which indicates a protocol ( http )\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "gStgBJy2WhAx"
      },
      "source": [
        "### compile\n",
        "компилирует регулярное выражение в отдельный объект"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "JstTupisWhAy",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "f60e13a6-9d93-46e5-f329-620f6072715f"
      },
      "source": [
        "# Пример: построение списка всех слов строки:\n",
        "prog = re.compile('[А-Яа-яё\\-]+')\n",
        "prog.findall(\"Слова? Да, больше, ещё больше слов! Что-то ещё.\")"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "['Слова', 'Да', 'больше', 'ещё', 'больше', 'слов', 'Что-то', 'ещё']"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 147
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "WXEXc3G0WhA2"
      },
      "source": [
        "**Задание**: для выбранной строки постройте список слов, которые длиннее трех символов."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "nFvnIWbUWhA2",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "5bfd291d-1be2-4fd6-8734-875620f459ca"
      },
      "source": [
        "prog = re.compile(r'[А-Яа-яё\\-]{3,}')\n",
        "prog.findall(\"Слова? Да, больше, ещё больше слов! Что-то ещё.\")"
      ],
      "execution_count": 9,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "['Слова', 'больше', 'ещё', 'больше', 'слов', 'Что-то', 'ещё']"
            ]
          },
          "metadata": {},
          "execution_count": 9
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "SQDNZ3HQWhA3"
      },
      "source": [
        "**Задание**: вернуть список доменов (@gmail.com) из списка адресов электронной почты:\n",
        "\n",
        "```\n",
        "abc.test@gmail.com, xyz@test.in, test.first@analyticsvidhya.com, first.test@rest.biz\n",
        "```"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "RHcQ-aMrepbT",
        "outputId": "42677a5c-abff-4bdf-b2cc-d8aadb857e2d"
      },
      "source": [
        "text = \"abc.test@gmail.com, xyz@test.in, test.first@analyticsvidhya.com, first.test@rest.biz\"\n",
        "prog = re.compile(r'@\\S+\\b')\n",
        "prog.findall(text)"
      ],
      "execution_count": 10,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "['@gmail.com', '@test.in', '@analyticsvidhya.com', '@rest.biz']"
            ]
          },
          "metadata": {},
          "execution_count": 10
        }
      ]
    }
  ]
}