keras_main.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "toc-hr-collapsed": false
   },
   "source": [
    "# 目标检测：口罩佩戴检测  \n",
    "\n",
    "<br>\n",
    "<hr>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "toc-hr-collapsed": true
   },
   "source": [
    "## 1.实验介绍"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1.1 实验背景  \n",
    "\n",
    "今年一场席卷全球的新型冠状病毒给人们带来了沉重的生命财产的损失。  \n",
    "有效防御这种传染病毒的方法就是积极佩戴口罩。  \n",
    "我国对此也采取了严肃的措施，在公共场合要求人们必须佩戴口罩。  \n",
    "在本次实验中，我们要建立一个目标检测的模型，可以识别图中的人是否佩戴了口罩。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1.2 实验要求\n",
    "\n",
    "1）建立深度学习模型，检测出图中的人是否佩戴了口罩，并将其尽可能调整到最佳状态。  \n",
    "2）学习经典的模型 MTCNN 和 MobileNet 的结构。  \n",
    "3）学习训练时的方法。  \n",
    "<br>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1.3 实验环境\n",
    "\n",
    "可以使用基于 Python 的 OpenCV 、PIL 库进行图像相关处理，使用 Numpy 库进行相关数值运算，使用 Keras 等深度学习框架训练模型等。\n",
    "<br>\n",
    "<br>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1.4 注意事项  \n",
    "+ Python 与 Python Package 的使用方式，可在右侧 `API文档` 中查阅。\n",
    "+ 当右上角的『Python 3』长时间指示为运行中的时候，造成代码无法执行时，可以重新启动 Kernel 解决（左上角『Kernel』-『Restart Kernel』）。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1.5 参考资料\n",
    "+ 论文 Joint Face Detection and Alignment using Multi-task Cascaded Convolutional Networks：https://kpzhang93.github.io/MTCNN_face_detection_alignment/\n",
    "+ OpenCV：https://opencv-python-tutroals.readthedocs.io/en/latest/py_tutorials/py_tutorials.html\n",
    "+ PIL：https://pillow.readthedocs.io/en/stable/\n",
    "+ Numpy：https://www.numpy.org/\n",
    "+ Scikit-learn： https://scikit-learn.org/\n",
    "+ tensorflow：https://www.tensorflow.org/api_docs/python/tf?hl=zh-cn\n",
    "+ keras：https://keras.io/zh/"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1.6 实验思路\n",
    "\n",
    "针对目标检测的任务，可以分为两个部分：目标识别和位置检测。  \n",
    "通常情况下，特征提取需要由特有的特征提取神经网络来完成，如 VGG、MobileNet、ResNet 等，这些特征提取网络往往被称为 Backbone 。而在 BackBone 后面接全连接层(FC)就可以执行分类任务。  \n",
    "但 FC 对目标的位置识别乏力。经过算法的发展，当前主要以特定的功能网络来代替 FC 的作用，如 Mask-Rcnn、SSD、YOLO 等。  \n",
    "我们选择充分使用已有的人脸检测的模型，再训练一个识别口罩的模型，从而提高训练的开支、增强模型的准确率。\n",
    "\n",
    "**常规目标检测：**  \n",
    "\n",
    "<img src=\"https://imgbed.momodel.cn/20200914162156.png\" width=500px/>\n",
    "\n",
    "\n",
    "\n",
    "**本次案例：**   \n",
    "\n",
    "\n",
    "<img src=\"https://imgbed.momodel.cn/20200918102630.png\" width=500px/>\n",
    "\n",
    "<br>\n",
    "<br>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2.数据集介绍"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.1 导入 Python 第三方库（包）"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "select": true
   },
   "outputs": [],
   "source": [
    "import warnings\n",
    "# 忽视警告\n",
    "warnings.filterwarnings('ignore')\n",
    "import os\n",
    "import matplotlib\n",
    "import cv2 as cv\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "from tensorflow.keras.callbacks import ModelCheckpoint, ReduceLROnPlateau, EarlyStopping\n",
    "from tensorflow.keras.applications.imagenet_utils import preprocess_input\n",
    "from tensorflow.keras import backend as K\n",
    "from tensorflow.keras.optimizers import Adam\n",
    "from keras.utils import np_utils,get_file\n",
    "\n",
    "K.image_data_format() == 'channels_last'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.2 导入已经写好的 Python 文件"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "select": true
   },
   "outputs": [],
   "source": [
    "from keras_py.utils import get_random_data\n",
    "from keras_py.face_rec import mask_rec\n",
    "from keras_py.face_rec import face_rec\n",
    "from keras_py.mobileNet import MobileNet"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.3 数据集介绍\n",
    "\n",
    "数据信息存放在 `/datasets/5f680a696ec9b83bb0037081-momodel/data` 文件夹下。    \n",
    "该文件夹主要有文件夹 `image`、文件 `train.txt` 、文件夹 `keras_model_data` 和文件夹 `mindspore_model_data`共四部分：\n",
    "+ **image 文件夹**：图片分成两类，戴口罩的和没有戴口罩的  \n",
    "+ **train.txt**：  存放的是 image 文件夹下对应图片的标签 （keras 框架专用文件）\n",
    "+ **keras_model_data** 文件夹：存放 keras 框架相关预训练好的模型 （keras 框架专用文件夹）\n",
    "+ **mindspore_model_data** 文件夹：存放 mindspore 框架相关预训练好的模型（mindspore 框架专用文件夹）"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 数据集路径\n",
    "basic_path = \"./datasets/5f680a696ec9b83bb0037081-momodel/data/\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "现在我们尝试读取数据集中戴口罩的图片及其名称"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "以下是训练集中的正样本："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "mask_num = 4\n",
    "fig = plt.figure(figsize=(15, 15))\n",
    "for i in range(mask_num):\n",
    "    sub_img = cv.imread(basic_path + \"/image/mask/mask_\" + str(i + 101) + \".jpg\")\n",
    "    sub_img = cv.cvtColor(sub_img, cv.COLOR_RGB2BGR)\n",
    "    ax = fig.add_subplot(4, 4, (i + 1))\n",
    "    ax.set_xticks([])\n",
    "    ax.set_yticks([])\n",
    "    ax.set_title(\"mask_\" + str(i + 1))\n",
    "    ax.imshow(sub_img)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "以下是训练集中的负样本："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "nomask_num = 4\n",
    "fig1 = plt.figure(figsize=(15, 15))\n",
    "for i in range(nomask_num):\n",
    "    sub_img = cv.imread(basic_path + \"/image/nomask/nomask_\" + str(i + 130) + \".jpg\")\n",
    "    sub_img = cv.cvtColor(sub_img, cv.COLOR_RGB2BGR)\n",
    "    ax = fig1.add_subplot(4, 4, (i + 1))\n",
    "    ax.set_xticks([])\n",
    "    ax.set_yticks([])\n",
    "    ax.set_title(\"nomask_\" + str(i + 1))\n",
    "    ax.imshow(sub_img)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.4 调整图片尺寸"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def letterbox_image(image, size):\n",
    "    \"\"\"\n",
    "    调整图片尺寸\n",
    "    :param image: 用于训练的图片\n",
    "    :param size: 需要调整到网络输入的图片尺寸\n",
    "    :return: 返回经过调整的图片\n",
    "    \"\"\"\n",
    "    new_image = cv.resize(image, size, interpolation=cv.INTER_AREA)\n",
    "    return new_image"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "查看图片尺寸调整前后的对比"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "read_img = cv.imread(\"test1.jpg\")\n",
    "print(\"调整前图片的尺寸:\", read_img.shape)\n",
    "read_img = letterbox_image(image=read_img, size=(50, 50))\n",
    "print(\"调整前图片的尺寸:\", read_img.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.5 制作训练时所需的批量数据集"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "图片生成器 [ImageDataGenerator](https://keras.io/preprocessing/image/): keras.preprocessing.image 模块中的图片生成器，主要用以生成一个 batch 的图像数据，支持实时数据提升。训练时该函数会无限生成数据，直到达到规定的 epoch 次数为止。同时也可以在 batch 中对数据进行增强，扩充数据集大小，增强模型的泛化能力，比如进行旋转，变形，归一化等等。\n",
    "    \n",
    "图片生成器的主要方法：\n",
    "+ fit(x, augment=False, rounds=1)：计算依赖于数据的变换所需要的统计信息(均值方差等)。  \n",
    "\n",
    "+ flow(self, X, y, batch_size=32, shuffle=True, seed=None, save_to_dir=None, save_prefix='', save_format='png')：接收 Numpy 数组和标签为参数,生成经过数据提升或标准化后的 batch 数据，并在一个无限循环中不断的返回 batch 数据。  \n",
    "\n",
    "\n",
    "+ flow_from_directory(directory): 以文件夹路径为参数，会从路径推测 label，生成经过数据提升/归一化后的数据，在一个无限循环中无限产生 batch 数据。\n",
    "\n",
    "英文参考链接：https://keras.io/preprocessing/image/  \n",
    "中文参考链接：https://keras-cn.readthedocs.io/en/latest/preprocessing/image/\n",
    "\n",
    "以上只是对图片生成器进行简单的介绍，详细信息请参考中英文链接。  \n",
    "根据上面的介绍和我们数据集的特性，我们主要运用 `ImageDataGenerator()` 和 `flow_from_directory()` 方法。我们将数据处理过程封装成为一个函数："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 导入图片生成器\n",
    "from tensorflow.keras.preprocessing.image import ImageDataGenerator\n",
    "\n",
    "\n",
    "def processing_data(data_path, height, width, batch_size=32, test_split=0.1):\n",
    "    \"\"\"\n",
    "    数据处理\n",
    "    :param data_path: 带有子目录的数据集路径\n",
    "    :param height: 图像形状的行数\n",
    "    :param width: 图像形状的列数\n",
    "    :param batch_size: batch 数据的大小，整数，默认32。\n",
    "    :param test_split: 在 0 和 1 之间浮动。用作测试集的训练数据的比例，默认0.1。\n",
    "    :return: train_generator, test_generator: 处理后的训练集数据、验证集数据\n",
    "    \"\"\"\n",
    "\n",
    "    train_data = ImageDataGenerator(\n",
    "            # 对图片的每个像素值均乘上这个放缩因子，把像素值放缩到0和1之间有利于模型的收敛\n",
    "            rescale=1. / 255,  \n",
    "            # 浮点数，剪切强度（逆时针方向的剪切变换角度）\n",
    "            shear_range=0.1,  \n",
    "            # 随机缩放的幅度，若为浮点数，则相当于[lower,upper] = [1 - zoom_range, 1+zoom_range]\n",
    "            zoom_range=0.1,\n",
    "            # 浮点数，图片宽度的某个比例，数据提升时图片水平偏移的幅度\n",
    "            width_shift_range=0.1,\n",
    "            # 浮点数，图片高度的某个比例，数据提升时图片竖直偏移的幅度\n",
    "            height_shift_range=0.1,\n",
    "            # 布尔值，进行随机水平翻转\n",
    "            horizontal_flip=True,\n",
    "            # 布尔值，进行随机竖直翻转\n",
    "            vertical_flip=True,\n",
    "            # 在 0 和 1 之间浮动。用作验证集的训练数据的比例\n",
    "            validation_split=test_split  \n",
    "    )\n",
    "\n",
    "    # 接下来生成测试集，可以参考训练集的写法\n",
    "    test_data = ImageDataGenerator(\n",
    "            rescale=1. / 255,\n",
    "            validation_split=test_split)\n",
    "\n",
    "    train_generator = train_data.flow_from_directory(\n",
    "            # 提供的路径下面需要有子目录\n",
    "            data_path, \n",
    "            # 整数元组 (height, width)，默认：(256, 256)。 所有的图像将被调整到的尺寸。\n",
    "            target_size=(height, width),\n",
    "            # 一批数据的大小\n",
    "            batch_size=batch_size,\n",
    "            # \"categorical\", \"binary\", \"sparse\", \"input\" 或 None 之一。\n",
    "            # 默认：\"categorical\",返回one-hot 编码标签。\n",
    "            class_mode='categorical',\n",
    "            # 数据子集 (\"training\" 或 \"validation\")\n",
    "            subset='training', \n",
    "            seed=0)\n",
    "    test_generator = test_data.flow_from_directory(\n",
    "            data_path,\n",
    "            target_size=(height, width),\n",
    "            batch_size=batch_size,\n",
    "            class_mode='categorical',\n",
    "            subset='validation',\n",
    "            seed=0)\n",
    "\n",
    "    return train_generator, test_generator"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 数据路径\n",
    "data_path = basic_path + 'image'\n",
    "\n",
    "# 图像数据的行数和列数\n",
    "height, width = 160, 160\n",
    "\n",
    "# 获取训练数据和验证数据集\n",
    "train_generator, test_generator = processing_data(data_path, height, width)\n",
    "\n",
    "# 通过属性class_indices可获得文件夹名与类的序号的对应字典。 (类别的顺序将按照字母表顺序映射到标签值)。\n",
    "labels = train_generator.class_indices\n",
    "print(labels)\n",
    "\n",
    "# 转换为类的序号与文件夹名对应的字典\n",
    "labels = dict((v, k) for k, v in labels.items())\n",
    "print(labels)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "toc-hr-collapsed": false
   },
   "source": [
    "## 3. MTCNN：人脸检测"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.1  MTCNN 解读\n",
    "\n",
    "参考文献：《Joint Face Detection and Alignment using Multi-task Cascaded Convolutional Networks》  \n",
    "文献与代码地址：https://kpzhang93.github.io/MTCNN_face_detection_alignment/  \n",
    "  \n",
    "论文的主要贡献：  \n",
    "1）**三阶段的级联（cascaded）架构**  \n",
    "2）**coarse-to-fine 的方式**  \n",
    "3）**new online hard sample mining 策略**  \n",
    "4）**同时进行人脸检测和人脸对齐**  \n",
    "5）**state-of-the-art 性能**  \n",
    "\n",
    "<img src=\"https://imgbed.momodel.cn/20200918102724.png\"/>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "toc-hr-collapsed": false
   },
   "source": [
    "### 3.2 MTCNN 的使用\n",
    "\n",
    "这里直接使用现有的表现较好的 MTCNN 的三个权重文件，它们已经保存在 `datasets/5f680a696ec9b83bb0037081-momodel/data/keras_model_data` 文件夹下，路径如下：\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "pnet_path = \"./datasets/5f680a696ec9b83bb0037081-momodel/data/keras_model_data/pnet.h5\"\n",
    "rnet_path = \"./datasets/5f680a696ec9b83bb0037081-momodel/data/keras_model_data/rnet.h5\"\n",
    "onet_path = \"./datasets/5f680a696ec9b83bb0037081-momodel/data/keras_model_data/onet.h5\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "通过搭建 MTCNN 网络实现人脸检测（搭建模型py文件在 keras_py 文件夹） 。 \n",
    "+ keras_py/mtcnn.py  文件是在搭建 MTCNN 网络。  \n",
    "+ keras_py/face_rec.py  文件是在绘制人脸检测的矩形框。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 读取测试图片\n",
    "img = cv.imread(\"test.jpg\")\n",
    "# 转换通道\n",
    "img = cv.cvtColor(img, cv.COLOR_RGB2BGR)\n",
    "# 加载模型进行识别口罩并绘制方框\n",
    "detect = face_rec(pnet_path,rnet_path,onet_path)\n",
    "detect.recognize(img)\n",
    "# 展示结果\n",
    "fig = plt.figure(figsize = (8,8))\n",
    "ax1 = fig.add_subplot(111)\n",
    "ax1.set_xticks([])\n",
    "ax1.set_yticks([])\n",
    "ax1.set_title('mask_1')\n",
    "ax1.imshow(img)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "toc-hr-collapsed": false
   },
   "source": [
    "## 4. 口罩识别\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4.1 预训练模型 MobileNet"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 加载 MobileNet 的预训练模型权重\n",
    "weights_path = basic_path + 'keras_model_data/mobilenet_1_0_224_tf_no_top.h5'\n",
    "# 图像数据的行数和列数\n",
    "height, width = 160, 160\n",
    "model = MobileNet(input_shape=[height,width,3],classes=2)\n",
    "model.load_weights(weights_path,by_name=True)\n",
    "print('加载完成...')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "toc-hr-collapsed": false
   },
   "source": [
    "### 4.2 准备训练模型 Tip"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### **4.2.1 加载和保存模型**(可调参）\n",
    "\n",
    "为了避免训练过程中遇到断电等突发事件，导致模型训练成果无法保存。  \n",
    "我们可以通过 ModelCheckpoint 规定在固定迭代次数后保存模型。  \n",
    "同时，我们设置在下一次重启训练时，会检查是否有上次训练好的模型，如果有，就先加载已有的模型权重。  \n",
    "这样就可以在上次训练的基础上继续模型的训练了。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def save_model(model, checkpoint_save_path, model_dir):\n",
    "    \"\"\"\n",
    "    保存模型，每迭代3次保存一次\n",
    "    :param model: 训练的模型\n",
    "    :param checkpoint_save_path: 加载历史模型\n",
    "    :param model_dir: \n",
    "    :return: \n",
    "    \"\"\"\n",
    "    if os.path.exists(checkpoint_save_path):\n",
    "        print(\"模型加载中\")\n",
    "        model.load_weights(checkpoint_save_path)\n",
    "        print(\"模型加载完毕\")\n",
    "    checkpoint_period = ModelCheckpoint(\n",
    "        # 模型存储路径\n",
    "        model_dir + 'ep{epoch:03d}-loss{loss:.3f}-val_loss{val_loss:.3f}.h5',\n",
    "        # 检测的指标\n",
    "        monitor='val_acc',\n",
    "        # ‘auto’，‘min’，‘max’中选择\n",
    "        mode='max',\n",
    "        # 是否只存储模型权重\n",
    "        save_weights_only=False,\n",
    "        # 是否只保存最优的模型\n",
    "        save_best_only=True,\n",
    "        # 检测的轮数是每隔2轮\n",
    "        period=2\n",
    "    )\n",
    "    return checkpoint_period"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "checkpoint_save_path = \"./results/temp.h5\"\n",
    "model_dir = \"./results/\"\n",
    "checkpoint_period = save_model(model, checkpoint_save_path, model_dir)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### **4.2.2 手动调整学习率**(可调参）\n",
    "\n",
    "学习率的手动设置可以使模型训练更加高效。  \n",
    "这里我们设置当模型在两轮迭代后，准确率没有上升，就调整学习率。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 学习率下降的方式，acc三次不下降就下降学习率继续训练\n",
    "reduce_lr = ReduceLROnPlateau(\n",
    "                        monitor='acc',  # 检测的指标\n",
    "                        factor=0.5,     # 当acc不下降时将学习率下调的比例\n",
    "                        patience=2,     # 检测轮数是每隔两轮\n",
    "                        verbose=2       # 信息展示模式\n",
    "                    )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### **4.2.3 早停法**(可调参）\n",
    "\n",
    "当我们训练深度学习神经网络的时候通常希望能获得最好的泛化性能。    \n",
    "但是所有的标准深度学习神经网络结构如全连接多层感知机都很容易过拟合。    \n",
    "当网络在训练集上表现越来越好，错误率越来越低的时候，就极有可能出现了过拟合。  \n",
    "早停法就是当我们在检测到这一趋势后，就停止训练，这样能避免继续训练导致过拟合的问题。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "early_stopping = EarlyStopping(\n",
    "                            monitor='val_loss',  # 检测的指标\n",
    "                            min_delta=0,         # 增大或减小的阈值\n",
    "                            patience=10,         # 检测的轮数频率\n",
    "                            verbose=1            # 信息展示的模式\n",
    "                        )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4.3 训练模型"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 一次的训练集大小\n",
    "batch_size = 8\n",
    "# 图片数据路径\n",
    "data_path = basic_path + 'image'\n",
    "# 图片处理\n",
    "train_generator,test_generator = processing_data(data_path, height=160, width=160, batch_size=batch_size, test_split=0.1)\n",
    "# 编译模型\n",
    "model.compile(loss='binary_crossentropy',  # 二分类损失函数   \n",
    "              optimizer=Adam(lr=0.1),            # 优化器\n",
    "              metrics=['accuracy'])        # 优化目标\n",
    "# 训练模型\n",
    "history = model.fit(train_generator,    \n",
    "                    epochs=3, # epochs: 整数，数据的迭代总轮数。\n",
    "                    # 一个epoch包含的步数,通常应该等于你的数据集的样本数量除以批量大小。\n",
    "                    steps_per_epoch=637 // batch_size,\n",
    "                    validation_data=test_generator,\n",
    "                    validation_steps=70 // batch_size,\n",
    "                    initial_epoch=0, # 整数。开始训练的轮次（有助于恢复之前的训练）。\n",
    "                    callbacks=[checkpoint_period, reduce_lr])\n",
    "# 保存模型\n",
    "model.save_weights(model_dir + 'temp.h5')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4.4 展示模型训练过程"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "plt.plot(history.history['loss'],label = 'train_loss')\n",
    "plt.plot(history.history['val_loss'],'r',label = 'val_loss')\n",
    "plt.legend()\n",
    "plt.show()\n",
    "\n",
    "plt.plot(history.history['accuracy'],label = 'acc')\n",
    "plt.plot(history.history['val_accuracy'],'r',label = 'val_acc')\n",
    "plt.legend()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4.5 检测图片中人数及戴口罩的人数"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import cv2 as cv\n",
    "# 读取图片\n",
    "img = cv.imread(\"./test1.jpg\")\n",
    "img = cv.cvtColor(img, cv.COLOR_RGB2BGR)\n",
    "\n",
    "# 最佳模型路径\n",
    "model_path = \"results/temp.h5\"\n",
    "\n",
    "# 加载训练模型并进行口罩识别\n",
    "detect = mask_rec(model_path)\n",
    "img, all_num, mask_num = detect.recognize(img)\n",
    "\n",
    "# 展示图片口罩识别结果\n",
    "fig = plt.figure(figsize=(8, 8))\n",
    "ax1 = fig.add_subplot(111)\n",
    "ax1.set_xticks([])\n",
    "ax1.set_yticks([])\n",
    "ax1.set_title('test_mask')\n",
    "ax1.imshow(img)\n",
    "print(\"图中的人数有：\" + str(all_num) + \"个\")\n",
    "print(\"戴口罩的人数有：\" + str(mask_num) + \"个\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 5.作业"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 5.1 训练模型"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "口罩佩戴检测模型训练流程, 包含数据处理、创建模型、训练模型、模型保存、评价模型等。训练模型可以参考第 4.3 部分训练模型代码  \n",
    "如果对训练出来的模型不满意, 你可以通过调整模型的参数等方法重新训练模型, 直至训练出你满意的模型。  \n",
    "如果你对自己训练出来的模型非常满意, 则可以提交作业!  \n",
    "\n",
    "注意：\n",
    "\n",
    "1. 你可以在我们准好的接口中实现深度学习模型（若使用可以修改函数接口），也可以自己实现深度学习模型。\n",
    "2. 写好代码后可以在 Py 文件中使用 [离线任务](https://momodel.cn/docs/#/zh-cn/%E5%9C%A8GPU%E6%88%96CPU%E8%B5%84%E6%BA%90%E4%B8%8A%E8%AE%AD%E7%BB%83%E6%9C%BA%E5%99%A8%E5%AD%A6%E4%B9%A0%E6%A8%A1%E5%9E%8B) 进行模型训练。\n",
    "3. **使用离线训练模型必须保存在 results 文件夹**。    \n",
    "4. 将自己认为最佳模型保存在 result 文件夹，其余模型备份在项目中其它文件夹，方便您加快测试通过。\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "inputHidden": false
   },
   "source": [
    "===========================================  实现自己的深度学习模型代码答题区域  ===========================================\n",
    "\n",
    "双击下方区域开始编写  **数据处理**、**创建模型**、**训练模型**、**保存模型**  和  **评估模型**  等部分的代码，请勿在别的位置作答"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 1.加载数据并进行数据处理\n",
    "\n",
    "# 2.如果有预训练模型，则加载预训练模型；如果没有则不需要加载\n",
    "\n",
    "# 3.创建模型和训练模型，训练模型时尽量将模型保存在 results 文件夹\n",
    "\n",
    "# 4.评估模型，将自己认为最佳模型保存在 result 文件夹，其余模型备份在项目中其它文件夹，方便您加快测试通过。\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 5.2 提交作业"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**作业要求及注意事项**：    \n",
    "\n",
    "1.使用上述学到的方法，训练自己的口罩识别模型，尽可能提高准确度。将训练好的模型保存在 results 文件夹下。             \n",
    "2.点击左侧栏提交作业后点击【生成文件】则需要勾选与预测 predict() 函数的 cell相关的其它cell ，并将其转化成为 main.py 文件。                       \n",
    "3.请导入必要的包和第三方库以及该模型所依赖的 py 文件 (包括此文件中曾经导入过的)。             \n",
    "4.请加载你认为训练最佳的模型，即请按要求填写模型路径。              \n",
    "5.predict() 函数的输入输出及函数名称请不要改动。\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "===========================================  **模型预测代码答题区域**  ===========================================  \n",
    "在下方的代码块中编写 **模型预测** 部分的代码，请勿在别的位置作答"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "select": true
   },
   "outputs": [],
   "source": [
    "from keras_py.utils import get_random_data\n",
    "from keras_py.face_rec import mask_rec\n",
    "from keras_py.face_rec import face_rec\n",
    "from keras_py.mobileNet import MobileNet\n",
    "\n",
    "# -------------------------- 请加载您最满意的模型 ---------------------------\n",
    "# 加载模型(请加载你认为的最佳模型)\n",
    "# 加载模型,加载请注意 model_path 是相对路径, 与当前文件同级。\n",
    "# 如果你的模型是在 results 文件夹下的 dnn.h5 模型，则 model_path = 'results/temp.h5'\n",
    "model_path = None\n",
    "# ---------------------------------------------------------------------------\n",
    "\n",
    "def predict(img):\n",
    "    \"\"\"\n",
    "    加载模型和模型预测\n",
    "    :param img: cv2.imread 图像\n",
    "    :return: 预测的图片中的总人数、其中佩戴口罩的人数\n",
    "    \"\"\"\n",
    "    # -------------------------- 实现模型预测部分的代码 ---------------------------\n",
    "    \n",
    "    detect = mask_rec(model_path)\n",
    "    img, all_num, mask_num = detect.recognize(img)\n",
    "    \n",
    "    # -------------------------------------------------------------------------\n",
    "    return all_num,mask_num\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 输入图片路径和名称\n",
    "img = cv.imread(\"test1.jpg\")\n",
    "img = cv.cvtColor(img, cv.COLOR_BGR2RGB)\n",
    "all_num,mask_num = predict(img)\n",
    "# 打印预测该张图片中总人数以及戴口罩的人数\n",
    "print(all_num, mask_num)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}