notebooks/train_and_quantize.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "ea2f3ec5",
   "metadata": {},
   "source": [
    "### load the imports and intel extension"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "352c8d66",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-10-17T11:10:24.131333Z",
     "iopub.status.busy": "2023-10-17T11:10:24.130990Z",
     "iopub.status.idle": "2023-10-17T11:10:24.992917Z",
     "shell.execute_reply": "2023-10-17T11:10:24.992018Z"
    },
    "papermill": {
     "duration": 0.870243,
     "end_time": "2023-10-17T11:10:24.994980",
     "exception": false,
     "start_time": "2023-10-17T11:10:24.124737",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/u136698/.local/lib/python3.9/site-packages/torchvision/io/image.py:13: UserWarning: Failed to load image Python extension: ''If you don't plan on using image functionality from `torchvision.io`, you can ignore this warning. Otherwise, there might be something wrong with your environment. Did you have `libjpeg` or `libpng` installed before building `torchvision` from source?\n",
      "  warn(\n"
     ]
    }
   ],
   "source": [
    "import pandas as pd\n",
    "import torch\n",
    "import numpy as np\n",
    "############# code changes ###############\n",
    "import intel_extension_for_pytorch as ipex\n",
    "############# code changes ###############"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e0363e62",
   "metadata": {},
   "source": [
    "### load the files"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "e848e121",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-10-17T11:10:25.006843Z",
     "iopub.status.busy": "2023-10-17T11:10:25.006483Z",
     "iopub.status.idle": "2023-10-17T11:10:28.616210Z",
     "shell.execute_reply": "2023-10-17T11:10:28.615217Z"
    },
    "papermill": {
     "duration": 3.618249,
     "end_time": "2023-10-17T11:10:28.618449",
     "exception": false,
     "start_time": "2023-10-17T11:10:25.000200",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "train_df = pd.read_csv(\"../data/train.csv\")\n",
    "test_df = pd.read_csv(\"../data/test.csv\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "ff48e9ca",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-10-17T11:10:28.630017Z",
     "iopub.status.busy": "2023-10-17T11:10:28.629745Z",
     "iopub.status.idle": "2023-10-17T11:10:31.826524Z",
     "shell.execute_reply": "2023-10-17T11:10:31.825234Z"
    },
    "papermill": {
     "duration": 3.204627,
     "end_time": "2023-10-17T11:10:31.828403",
     "exception": false,
     "start_time": "2023-10-17T11:10:28.623776",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Is it a large country? No\n",
      "Malawi (, or ; or [maláwi]), officially the Republic of Malawi, is a landlocked country in southeast Africa that was formerly known as Nyasaland. It is bordered by Zambia to the northwest, Tanzania to the northeast, and Mozambique on the east, south and west. Malawi is over with an estimated population of 16,777,547 (July 2013 est.). Its capital is Lilongwe, which is also Malawi's largest city; the second largest is Blantyre, the third is Mzuzu and the fourth largest is its old capital Zomba. The name Malawi comes from the Maravi, an old name of the Nyanja people that inhabit the area. The country is also nicknamed \"The Warm Heart of Africa\". \n",
      "\n",
      "Malawi is among the smallest countries in Africa. Lake Malawi takes up about a third of Malawi's area. \n",
      "\n",
      "The area of Africa now known as Malawi was settled by migrating Bantu groups around the 10th century. Centuries later in 1891 the area was colonised by the British. In 1953 Malawi, then known as Nyasaland, a protectorate of the United Kingdom, became a protectorate within the semi-independent Federation of Rhodesia and Nyasaland. The Federation was dissolved in 1963. In 1964 the protectorate over Nyasaland was ended and Nyasaland became an independent country under Queen Elizabeth II with the new name Malawi. Two years later it became a republic. Upon gaining independence it became a one-party state under the presidency of Hastings Banda, who remained president until 1994, when he lost an election. Arthur Peter Mutharika is the current president. Malawi has a democratic, multi-party government. The country has a Malawian Defence Force that includes an army, a navy and an air wing. Malawi's foreign policy is pro-Western and includes positive diplomatic relations with most countries and participation in several international organisations, including the United Nations, the Commonwealth of Nations, the Southern African Development Community (SADC), the Common Market for Eastern and Southern Africa (COMESA), and the African Union (AU).\n",
      "--------------------------\n",
      "Besides breaking or staling a computer what other harm could come to them? unknown\n",
      "Computer security, also known as cybersecurity or IT security, is the protection of information systems from theft or damage to the hardware, the software, and to the information on them, as well as from disruption or misdirection of the services they provide. It includes controlling physical access to the hardware, as well as protecting against harm that may come via network access, data and code injection, and due to malpractice by operators, whether intentional, accidental, or due to them being tricked into deviating from secure procedures. \n",
      "\n",
      "Denial of service attacks are designed to make a machine or network resource unavailable to its intended users. Attackers can deny service to individual victims, such as by deliberately entering a wrong password enough consecutive times to cause the victim account to be locked, or they may overload the capabilities of a machine or network and block all users at once. While a network attack from a single IP address can be blocked by adding a new firewall rule, many forms of Distributed denial of service (DDoS) attacks are possible, where the attack comes from a large number of points – and defending is much more difficult. Such attacks can originate from the zombie computers of a botnet, but a range of other techniques are possible including reflection and amplification attacks, where innocent systems are fooled into sending traffic to the victim.\n",
      "--------------------------\n",
      "0.36222245575055173\n"
     ]
    }
   ],
   "source": [
    "count = 0\n",
    "stop = False\n",
    "for _, row in train_df.iterrows():\n",
    "    if row[\"Answer\"] not in row[\"Story\"]:\n",
    "        count += 1\n",
    "        if not stop:\n",
    "            print(row[\"Question\"], row[\"Answer\"])\n",
    "            print(row[\"Story\"])\n",
    "            print(\"--------------------------\")\n",
    "    if count == 2:\n",
    "        stop = True\n",
    "print(count/len(train_df))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "48d2b8dd",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-10-17T11:10:31.841360Z",
     "iopub.status.busy": "2023-10-17T11:10:31.841117Z",
     "iopub.status.idle": "2023-10-17T11:10:31.844965Z",
     "shell.execute_reply": "2023-10-17T11:10:31.844091Z"
    },
    "papermill": {
     "duration": 0.012169,
     "end_time": "2023-10-17T11:10:31.846658",
     "exception": false,
     "start_time": "2023-10-17T11:10:31.834489",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "# sjrhuschlee/flan-t5-large-squad2\n",
    "# mrm8488/t5-base-finetuned-squadv2"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2cf76ac8",
   "metadata": {},
   "source": [
    "### model to finetune"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "4ffea439",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-10-17T11:10:31.859008Z",
     "iopub.status.busy": "2023-10-17T11:10:31.858674Z",
     "iopub.status.idle": "2023-10-17T11:10:31.862701Z",
     "shell.execute_reply": "2023-10-17T11:10:31.861759Z"
    },
    "papermill": {
     "duration": 0.012273,
     "end_time": "2023-10-17T11:10:31.864552",
     "exception": false,
     "start_time": "2023-10-17T11:10:31.852279",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "model_checkpoint = \"mrm8488/t5-base-finetuned-squadv2\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "011b87d4",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-10-17T11:10:31.965232Z",
     "iopub.status.busy": "2023-10-17T11:10:31.964535Z",
     "iopub.status.idle": "2023-10-17T11:10:31.982929Z",
     "shell.execute_reply": "2023-10-17T11:10:31.982005Z"
    },
    "papermill": {
     "duration": 0.026584,
     "end_time": "2023-10-17T11:10:31.984687",
     "exception": false,
     "start_time": "2023-10-17T11:10:31.958103",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "train_df = train_df.sample(frac=1).reset_index(drop=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "3fbff59e",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-10-17T11:10:31.997053Z",
     "iopub.status.busy": "2023-10-17T11:10:31.996829Z",
     "iopub.status.idle": "2023-10-17T11:10:32.003628Z",
     "shell.execute_reply": "2023-10-17T11:10:32.002797Z"
    },
    "papermill": {
     "duration": 0.01505,
     "end_time": "2023-10-17T11:10:32.005259",
     "exception": false,
     "start_time": "2023-10-17T11:10:31.990209",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "train_data = train_df.iloc[:60000].reset_index(drop=True)\n",
    "valid_data = train_df.iloc[60000:].reset_index(drop=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "c509ac50",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-10-17T11:10:32.018134Z",
     "iopub.status.busy": "2023-10-17T11:10:32.017401Z",
     "iopub.status.idle": "2023-10-17T11:10:35.408479Z",
     "shell.execute_reply": "2023-10-17T11:10:35.407443Z"
    },
    "papermill": {
     "duration": 3.399553,
     "end_time": "2023-10-17T11:10:35.410661",
     "exception": false,
     "start_time": "2023-10-17T11:10:32.011108",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "The `xla_device` argument has been deprecated in v4.4.0 of Transformers. It is ignored and you can safely remove it from your `config.json` file.\n",
      "The `xla_device` argument has been deprecated in v4.4.0 of Transformers. It is ignored and you can safely remove it from your `config.json` file.\n",
      "You are using the default legacy behaviour of the <class 'transformers.models.t5.tokenization_t5.T5Tokenizer'>. This is expected, and simply means that the `legacy` (previous) behavior will be used so nothing changes for you. If you want to use the new behaviour, set `legacy=False`. This should only be set if you understand what it means, and thouroughly read the reason why this was added as explained in https://github.com/huggingface/transformers/pull/24565\n",
      "The `xla_device` argument has been deprecated in v4.4.0 of Transformers. It is ignored and you can safely remove it from your `config.json` file.\n"
     ]
    }
   ],
   "source": [
    "from transformers import AutoTokenizer\n",
    "    \n",
    "tokenizer = AutoTokenizer.from_pretrained(model_checkpoint)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7f319dcc",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-10-17T11:10:35.425648Z",
     "iopub.status.busy": "2023-10-17T11:10:35.424930Z",
     "iopub.status.idle": "2023-10-17T11:10:35.429505Z",
     "shell.execute_reply": "2023-10-17T11:10:35.428716Z"
    },
    "papermill": {
     "duration": 0.013687,
     "end_time": "2023-10-17T11:10:35.431176",
     "exception": false,
     "start_time": "2023-10-17T11:10:35.417489",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "ea07f51c",
   "metadata": {},
   "source": [
    "### preprocess the data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "40ee8c5f",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-10-17T11:10:35.445167Z",
     "iopub.status.busy": "2023-10-17T11:10:35.444841Z",
     "iopub.status.idle": "2023-10-17T11:10:36.108960Z",
     "shell.execute_reply": "2023-10-17T11:10:36.108107Z"
    },
    "papermill": {
     "duration": 0.673664,
     "end_time": "2023-10-17T11:10:36.111119",
     "exception": false,
     "start_time": "2023-10-17T11:10:35.437455",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "train_data[\"input\"] = train_data.apply(lambda x: f'question: {x[\"Question\"]}  context: {x[\"Story\"]}', axis=1)\n",
    "valid_data[\"input\"] = valid_data.apply(lambda x: f'question: {x[\"Question\"]}  context: {x[\"Story\"]}', axis=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "08362099",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-10-17T11:10:36.138197Z",
     "iopub.status.busy": "2023-10-17T11:10:36.137926Z",
     "iopub.status.idle": "2023-10-17T11:10:36.734838Z",
     "shell.execute_reply": "2023-10-17T11:10:36.733941Z"
    },
    "papermill": {
     "duration": 0.60596,
     "end_time": "2023-10-17T11:10:36.736885",
     "exception": false,
     "start_time": "2023-10-17T11:10:36.130925",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "from datasets import load_dataset, load_metric, Dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "293a7b83",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-10-17T11:10:36.764277Z",
     "iopub.status.busy": "2023-10-17T11:10:36.763559Z",
     "iopub.status.idle": "2023-10-17T11:10:37.088328Z",
     "shell.execute_reply": "2023-10-17T11:10:37.087406Z"
    },
    "papermill": {
     "duration": 0.334564,
     "end_time": "2023-10-17T11:10:37.090711",
     "exception": false,
     "start_time": "2023-10-17T11:10:36.756147",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Load datasets\n",
    "train_dataset = Dataset.from_pandas(train_data[[\"input\", \"Answer\"]])\n",
    "valid_dataset = Dataset.from_pandas(valid_data[[\"input\", \"Answer\"]])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9c2d9b4e",
   "metadata": {
    "papermill": {
     "duration": 0.007483,
     "end_time": "2023-10-17T11:10:37.105265",
     "exception": false,
     "start_time": "2023-10-17T11:10:37.097782",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "63dfda2d",
   "metadata": {},
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "2e58c749",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-10-17T11:10:37.119513Z",
     "iopub.status.busy": "2023-10-17T11:10:37.119249Z",
     "iopub.status.idle": "2023-10-17T11:10:37.124444Z",
     "shell.execute_reply": "2023-10-17T11:10:37.123478Z"
    },
    "papermill": {
     "duration": 0.014228,
     "end_time": "2023-10-17T11:10:37.126107",
     "exception": false,
     "start_time": "2023-10-17T11:10:37.111879",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "max_input_length = 1024\n",
    "max_target_length = 128\n",
    "\n",
    "def preprocess_function(examples):\n",
    "    inputs = examples[\"input\"]\n",
    "    targets = examples[\"Answer\"]\n",
    "\n",
    "    model_inputs = tokenizer(inputs, max_length=max_input_length, truncation=True)\n",
    "    labels = tokenizer(text_target=targets, max_length=max_target_length, truncation=True)\n",
    "\n",
    "    model_inputs[\"labels\"] = labels[\"input_ids\"]\n",
    "    return model_inputs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "5c2b99dd",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-10-17T11:10:37.152352Z",
     "iopub.status.busy": "2023-10-17T11:10:37.151609Z",
     "iopub.status.idle": "2023-10-17T11:11:44.391344Z",
     "shell.execute_reply": "2023-10-17T11:11:44.390238Z"
    },
    "papermill": {
     "duration": 67.248482,
     "end_time": "2023-10-17T11:11:44.393114",
     "exception": false,
     "start_time": "2023-10-17T11:10:37.144632",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "a2872080af1d4f4f8f152d2a41c4fca7",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Map:   0%|          | 0/60000 [00:00<?, ? examples/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "74b73b0924fb480fb9e9a0e294a299a5",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Map:   0%|          | 0/6611 [00:00<?, ? examples/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "train_tokenized_datasets = train_dataset.map(preprocess_function, batched=True)\n",
    "valid_tokenized_datasets = valid_dataset.map(preprocess_function, batched=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "09fc513b",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-10-17T11:11:44.408118Z",
     "iopub.status.busy": "2023-10-17T11:11:44.407793Z",
     "iopub.status.idle": "2023-10-17T11:12:06.425496Z",
     "shell.execute_reply": "2023-10-17T11:12:06.424517Z"
    },
    "papermill": {
     "duration": 22.02833,
     "end_time": "2023-10-17T11:12:06.428422",
     "exception": false,
     "start_time": "2023-10-17T11:11:44.400092",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Intel(R) Extension for Scikit-learn* enabled (https://github.com/intel/scikit-learn-intelex)\n",
      "The `xla_device` argument has been deprecated in v4.4.0 of Transformers. It is ignored and you can safely remove it from your `config.json` file.\n"
     ]
    }
   ],
   "source": [
    "from transformers import AutoModelForSeq2SeqLM, DataCollatorForSeq2Seq, Seq2SeqTrainingArguments, Seq2SeqTrainer\n",
    "model = AutoModelForSeq2SeqLM.from_pretrained(model_checkpoint)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4c616d8a",
   "metadata": {},
   "source": [
    "## optimize the model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "97004b89",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/u136698/.local/lib/python3.9/site-packages/intel_extension_for_pytorch/frontend.py:611: UserWarning: Conv BatchNorm folding failed during the optimize process.\n",
      "  warnings.warn(\n",
      "/home/u136698/.local/lib/python3.9/site-packages/intel_extension_for_pytorch/frontend.py:618: UserWarning: Linear BatchNorm folding failed during the optimize process.\n",
      "  warnings.warn(\n"
     ]
    }
   ],
   "source": [
    "#################### code changes #################\n",
    "model = model.to(\"xpu\")\n",
    "model = ipex.optimize(model)\n",
    "#################### code changes #################"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "21ad4ef5",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-10-17T11:12:06.449627Z",
     "iopub.status.busy": "2023-10-17T11:12:06.449319Z",
     "iopub.status.idle": "2023-10-17T11:12:06.456228Z",
     "shell.execute_reply": "2023-10-17T11:12:06.455348Z"
    },
    "papermill": {
     "duration": 0.016948,
     "end_time": "2023-10-17T11:12:06.458266",
     "exception": false,
     "start_time": "2023-10-17T11:12:06.441318",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "trainable model parameters: 222903552\n",
      "all model parameters: 222903552\n",
      "percentage of trainable model parameters: 100.00%\n"
     ]
    }
   ],
   "source": [
    "def print_number_of_trainable_model_parameters(model):\n",
    "    trainable_model_params = 0\n",
    "    all_model_params = 0\n",
    "    for _, param in model.named_parameters():\n",
    "        all_model_params += param.numel()\n",
    "        if param.requires_grad:\n",
    "            trainable_model_params += param.numel()\n",
    "    return f\"trainable model parameters: {trainable_model_params}\\nall model parameters: {all_model_params}\\npercentage of trainable model parameters: {100 * trainable_model_params / all_model_params:.2f}%\"\n",
    "\n",
    "print(print_number_of_trainable_model_parameters(model))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3550cdc0",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-10-17T11:12:06.494048Z",
     "iopub.status.busy": "2023-10-17T11:12:06.493776Z",
     "iopub.status.idle": "2023-10-17T11:12:09.201049Z",
     "shell.execute_reply": "2023-10-17T11:12:09.199397Z"
    },
    "papermill": {
     "duration": 2.718507,
     "end_time": "2023-10-17T11:12:09.204825",
     "exception": false,
     "start_time": "2023-10-17T11:12:06.486318",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "1446d13a",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-10-17T11:12:09.251174Z",
     "iopub.status.busy": "2023-10-17T11:12:09.250673Z",
     "iopub.status.idle": "2023-10-17T11:12:09.256552Z",
     "shell.execute_reply": "2023-10-17T11:12:09.255328Z"
    },
    "papermill": {
     "duration": 0.035403,
     "end_time": "2023-10-17T11:12:09.263303",
     "exception": false,
     "start_time": "2023-10-17T11:12:09.227900",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "batch_size = 6"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "065a1c56",
   "metadata": {},
   "outputs": [],
   "source": [
    "model.generation_config.max_length = max_target_length"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e34f3c82",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "05e84baa",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-10-17T11:12:09.378877Z",
     "iopub.status.busy": "2023-10-17T11:12:09.377973Z",
     "iopub.status.idle": "2023-10-17T11:12:09.517227Z",
     "shell.execute_reply": "2023-10-17T11:12:09.515954Z"
    },
    "papermill": {
     "duration": 0.166531,
     "end_time": "2023-10-17T11:12:09.521028",
     "exception": false,
     "start_time": "2023-10-17T11:12:09.354497",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "args = Seq2SeqTrainingArguments(\n",
    "    f\"new_model\",\n",
    "    evaluation_strategy = \"steps\",\n",
    "    per_device_train_batch_size=batch_size,\n",
    "    per_device_eval_batch_size=batch_size,\n",
    "    save_total_limit=3,\n",
    "    num_train_epochs=4,\n",
    "    predict_with_generate=True,\n",
    "    fp16=True,\n",
    "    eval_steps=10,\n",
    "    save_steps=10,\n",
    "    report_to=\"none\",\n",
    "    load_best_model_at_end=True\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "e7670851",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-10-17T11:12:09.564956Z",
     "iopub.status.busy": "2023-10-17T11:12:09.564531Z",
     "iopub.status.idle": "2023-10-17T11:12:09.569761Z",
     "shell.execute_reply": "2023-10-17T11:12:09.568833Z"
    },
    "papermill": {
     "duration": 0.032251,
     "end_time": "2023-10-17T11:12:09.574982",
     "exception": false,
     "start_time": "2023-10-17T11:12:09.542731",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "data_collator = DataCollatorForSeq2Seq(tokenizer, model=model)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3983abc2",
   "metadata": {},
   "source": [
    "### adding eval metric and bleu score"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "90cda9c9",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-10-17T11:12:09.617587Z",
     "iopub.status.busy": "2023-10-17T11:12:09.617200Z",
     "iopub.status.idle": "2023-10-17T11:12:09.635810Z",
     "shell.execute_reply": "2023-10-17T11:12:09.634777Z"
    },
    "papermill": {
     "duration": 0.043221,
     "end_time": "2023-10-17T11:12:09.639003",
     "exception": false,
     "start_time": "2023-10-17T11:12:09.595782",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "from sklearn.metrics import f1_score\n",
    "\n",
    "def postprocess_text(preds, labels):\n",
    "    preds = [pred.strip() for pred in preds]\n",
    "    labels = [label.strip() for label in labels]\n",
    "    return preds, labels\n",
    "\n",
    "def compute_metrics(eval_preds):\n",
    "    preds, labels = eval_preds\n",
    "    if isinstance(preds, tuple):\n",
    "        preds = preds[0]\n",
    "    decoded_preds = tokenizer.batch_decode(preds, skip_special_tokens=True)\n",
    "    # Replace -100 in the labels as we can't decode them.\n",
    "    labels = np.where(labels != -100, labels, tokenizer.pad_token_id)\n",
    "    decoded_labels = tokenizer.batch_decode(labels, skip_special_tokens=True)\n",
    "\n",
    "    # Some simple post-processing\n",
    "    decoded_preds, decoded_labels = postprocess_text(decoded_preds, decoded_labels)\n",
    "\n",
    "    # Calculate Exact Match (EM) and F1 score\n",
    "    em_score = 0\n",
    "    f1_score_value = 0\n",
    "    for pred, label in zip(decoded_preds, decoded_labels):\n",
    "        em_score += int(pred == label)\n",
    "        f1_score_value += f1_score([label], [pred], average='macro')\n",
    "\n",
    "    em_score = em_score / len(decoded_preds)  # EM is a ratio\n",
    "    f1_score_value = f1_score_value / len(decoded_preds)  # F1 score is an average\n",
    "\n",
    "    result = {\n",
    "        \"exact_match\": round(em_score, 4),\n",
    "        \"f1_score\": round(f1_score_value, 4)\n",
    "    }\n",
    "\n",
    "    prediction_lens = [np.count_nonzero(pred != tokenizer.pad_token_id) for pred in preds]\n",
    "    result[\"gen_len\"] = np.mean(prediction_lens)\n",
    "\n",
    "    return result"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "id": "7307465e",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-10-17T11:12:09.680902Z",
     "iopub.status.busy": "2023-10-17T11:12:09.680530Z",
     "iopub.status.idle": "2023-10-17T11:12:17.083119Z",
     "shell.execute_reply": "2023-10-17T11:12:17.082051Z"
    },
    "papermill": {
     "duration": 7.425755,
     "end_time": "2023-10-17T11:12:17.085298",
     "exception": false,
     "start_time": "2023-10-17T11:12:09.659543",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/u136698/.local/lib/python3.9/site-packages/torch/cuda/amp/grad_scaler.py:120: UserWarning: torch.cuda.amp.GradScaler is enabled, but CUDA is not available.  Disabling.\n",
      "  warnings.warn(\"torch.cuda.amp.GradScaler is enabled, but CUDA is not available.  Disabling.\")\n"
     ]
    }
   ],
   "source": [
    "trainer = Seq2SeqTrainer(\n",
    "    model,\n",
    "    args,\n",
    "    train_dataset=train_tokenized_datasets,\n",
    "    eval_dataset=valid_tokenized_datasets,\n",
    "    data_collator=data_collator,\n",
    "    tokenizer=tokenizer,\n",
    "    compute_metrics=compute_metrics\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9e1edbef",
   "metadata": {},
   "source": [
    "### train and save the model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "bccf8818",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-10-17T11:12:17.101362Z",
     "iopub.status.busy": "2023-10-17T11:12:17.101002Z",
     "iopub.status.idle": "2023-10-17T22:40:16.415620Z",
     "shell.execute_reply": "2023-10-17T22:40:16.414551Z"
    },
    "papermill": {
     "duration": 41279.324595,
     "end_time": "2023-10-17T22:40:16.417800",
     "exception": false,
     "start_time": "2023-10-17T11:12:17.093205",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "trainer.train()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "id": "a5945d1e",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-10-17T22:40:16.441580Z",
     "iopub.status.busy": "2023-10-17T22:40:16.441299Z",
     "iopub.status.idle": "2023-10-17T22:40:17.369946Z",
     "shell.execute_reply": "2023-10-17T22:40:17.368901Z"
    },
    "papermill": {
     "duration": 0.940841,
     "end_time": "2023-10-17T22:40:17.372262",
     "exception": false,
     "start_time": "2023-10-17T22:40:16.431421",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "trainer.save_model(\"../T5-trained\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "id": "3bb4be01",
   "metadata": {
    "papermill": {
     "duration": 0.009072,
     "end_time": "2023-10-17T22:40:17.390047",
     "exception": false,
     "start_time": "2023-10-17T22:40:17.380975",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...\n",
      "To disable this warning, you can either:\n",
      "\t- Avoid using `tokenizers` before the fork if possible\n",
      "\t- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)\n",
      "huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...\n",
      "To disable this warning, you can either:\n",
      "\t- Avoid using `tokenizers` before the fork if possible\n",
      "\t- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)\n",
      "huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...\n",
      "To disable this warning, you can either:\n",
      "\t- Avoid using `tokenizers` before the fork if possible\n",
      "\t- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)\n"
     ]
    }
   ],
   "source": [
    "from neural_compressor.config import PostTrainingQuantConfig\n",
    "from neural_compressor import quantization"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "200ae109",
   "metadata": {},
   "source": [
    "## quantize the trained model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "id": "21a517a2",
   "metadata": {},
   "outputs": [],
   "source": [
    "conf = PostTrainingQuantConfig(\n",
    "    device=\"gpu\",\n",
    "    backend=\"itex\",\n",
    "    approach=\"dynamic\"\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "id": "b75fb868",
   "metadata": {},
   "outputs": [],
   "source": [
    "calib_dataloader = trainer.get_eval_dataloader()\n",
    "conf.use_bf16 = False"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "id": "4b5a1974",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2023-10-20 10:45:14 [INFO] Start auto tuning.\n",
      "2023-10-20 10:45:14 [INFO] Quantize model without tuning!\n",
      "2023-10-20 10:45:14 [INFO] Quantize the model with default configuration without evaluating the model.                To perform the tuning process, please either provide an eval_func or provide an                    eval_dataloader an eval_metric.\n",
      "2023-10-20 10:45:14 [INFO] Adaptor has 5 recipes.\n",
      "2023-10-20 10:45:14 [INFO] 0 recipes specified by user.\n",
      "2023-10-20 10:45:14 [INFO] 3 recipes require future tuning.\n",
      "2023-10-20 10:45:14 [INFO] *** Initialize auto tuning\n",
      "2023-10-20 10:45:14 [INFO] {\n",
      "2023-10-20 10:45:14 [INFO]     'PostTrainingQuantConfig': {\n",
      "2023-10-20 10:45:14 [INFO]         'AccuracyCriterion': {\n",
      "2023-10-20 10:45:14 [INFO]             'criterion': 'relative',\n",
      "2023-10-20 10:45:14 [INFO]             'higher_is_better': True,\n",
      "2023-10-20 10:45:14 [INFO]             'tolerable_loss': 0.01,\n",
      "2023-10-20 10:45:14 [INFO]             'absolute': None,\n",
      "2023-10-20 10:45:14 [INFO]             'keys': <bound method AccuracyCriterion.keys of <neural_compressor.config.AccuracyCriterion object at 0x145c4cec5a30>>,\n",
      "2023-10-20 10:45:14 [INFO]             'relative': 0.01\n",
      "2023-10-20 10:45:14 [INFO]         },\n",
      "2023-10-20 10:45:14 [INFO]         'approach': 'post_training_dynamic_quant',\n",
      "2023-10-20 10:45:14 [INFO]         'backend': 'itex',\n",
      "2023-10-20 10:45:14 [INFO]         'calibration_sampling_size': [\n",
      "2023-10-20 10:45:14 [INFO]             100\n",
      "2023-10-20 10:45:14 [INFO]         ],\n",
      "2023-10-20 10:45:14 [INFO]         'device': 'gpu',\n",
      "2023-10-20 10:45:14 [INFO]         'diagnosis': False,\n",
      "2023-10-20 10:45:14 [INFO]         'domain': 'auto',\n",
      "2023-10-20 10:45:14 [INFO]         'example_inputs': None,\n",
      "2023-10-20 10:45:14 [INFO]         'excluded_precisions': [\n",
      "2023-10-20 10:45:14 [INFO]         ],\n",
      "2023-10-20 10:45:14 [INFO]         'framework': 'pytorch',\n",
      "2023-10-20 10:45:14 [INFO]         'inputs': [\n",
      "2023-10-20 10:45:14 [INFO]         ],\n",
      "2023-10-20 10:45:14 [INFO]         'model_name': '',\n",
      "2023-10-20 10:45:14 [INFO]         'ni_workload_name': 'quantization',\n",
      "2023-10-20 10:45:14 [INFO]         'op_name_dict': None,\n",
      "2023-10-20 10:45:14 [INFO]         'op_type_dict': None,\n",
      "2023-10-20 10:45:14 [INFO]         'outputs': [\n",
      "2023-10-20 10:45:14 [INFO]         ],\n",
      "2023-10-20 10:45:14 [INFO]         'quant_format': 'default',\n",
      "2023-10-20 10:45:14 [INFO]         'quant_level': 'auto',\n",
      "2023-10-20 10:45:14 [INFO]         'recipes': {\n",
      "2023-10-20 10:45:14 [INFO]             'smooth_quant': False,\n",
      "2023-10-20 10:45:14 [INFO]             'smooth_quant_args': {\n",
      "2023-10-20 10:45:14 [INFO]             },\n",
      "2023-10-20 10:45:14 [INFO]             'layer_wise_quant': False,\n",
      "2023-10-20 10:45:14 [INFO]             'layer_wise_quant_args': {\n",
      "2023-10-20 10:45:14 [INFO]             },\n",
      "2023-10-20 10:45:14 [INFO]             'fast_bias_correction': False,\n",
      "2023-10-20 10:45:14 [INFO]             'weight_correction': False,\n",
      "2023-10-20 10:45:14 [INFO]             'gemm_to_matmul': True,\n",
      "2023-10-20 10:45:14 [INFO]             'graph_optimization_level': None,\n",
      "2023-10-20 10:45:14 [INFO]             'first_conv_or_matmul_quantization': True,\n",
      "2023-10-20 10:45:14 [INFO]             'last_conv_or_matmul_quantization': True,\n",
      "2023-10-20 10:45:14 [INFO]             'pre_post_process_quantization': True,\n",
      "2023-10-20 10:45:14 [INFO]             'add_qdq_pair_to_weight': False,\n",
      "2023-10-20 10:45:14 [INFO]             'optypes_to_exclude_output_quant': [\n",
      "2023-10-20 10:45:14 [INFO]             ],\n",
      "2023-10-20 10:45:14 [INFO]             'dedicated_qdq_pair': False,\n",
      "2023-10-20 10:45:14 [INFO]             'rtn_args': {\n",
      "2023-10-20 10:45:14 [INFO]             },\n",
      "2023-10-20 10:45:14 [INFO]             'awq_args': {\n",
      "2023-10-20 10:45:14 [INFO]             },\n",
      "2023-10-20 10:45:14 [INFO]             'gptq_args': {\n",
      "2023-10-20 10:45:14 [INFO]             },\n",
      "2023-10-20 10:45:14 [INFO]             'teq_args': {\n",
      "2023-10-20 10:45:14 [INFO]             }\n",
      "2023-10-20 10:45:14 [INFO]         },\n",
      "2023-10-20 10:45:14 [INFO]         'reduce_range': None,\n",
      "2023-10-20 10:45:14 [INFO]         'TuningCriterion': {\n",
      "2023-10-20 10:45:14 [INFO]             'max_trials': 100,\n",
      "2023-10-20 10:45:14 [INFO]             'objective': [\n",
      "2023-10-20 10:45:14 [INFO]                 'performance'\n",
      "2023-10-20 10:45:14 [INFO]             ],\n",
      "2023-10-20 10:45:14 [INFO]             'strategy': 'basic',\n",
      "2023-10-20 10:45:14 [INFO]             'strategy_kwargs': None,\n",
      "2023-10-20 10:45:14 [INFO]             'timeout': 0\n",
      "2023-10-20 10:45:14 [INFO]         },\n",
      "2023-10-20 10:45:14 [INFO]         'use_bf16': False\n",
      "2023-10-20 10:45:14 [INFO]     }\n",
      "2023-10-20 10:45:14 [INFO] }\n",
      "2023-10-20 10:45:15 [INFO] Pass query framework capability elapsed time: 30.36 ms\n",
      "2023-10-20 10:45:15 [INFO] Do not evaluate the baseline and quantize the model with default configuration.\n",
      "2023-10-20 10:45:15 [INFO] Quantize the model with default config.\n",
      "2023-10-20 10:45:16 [WARNING] None of the submodule got qconfig applied. Make sure you passed correct configuration through `qconfig_dict` or by assigning the `.qconfig` attribute directly on submodules.\n",
      "2023-10-20 10:45:16 [INFO] |******Mixed Precision Statistics******|\n",
      "2023-10-20 10:45:16 [INFO] +---------------------+----------------+\n",
      "2023-10-20 10:45:16 [INFO] |       Op Type       |     Total      |\n",
      "2023-10-20 10:45:16 [INFO] +---------------------+----------------+\n",
      "2023-10-20 10:45:16 [INFO] +---------------------+----------------+\n",
      "2023-10-20 10:45:16 [INFO] Pass quantize model elapsed time: 1329.17 ms\n",
      "2023-10-20 10:45:16 [INFO] Save tuning history to /home/u136698/Training/oneAPI_final/notebooks/nc_workspace/2023-10-20_10-45-08/./history.snapshot.\n",
      "2023-10-20 10:45:16 [INFO] [Strategy] Found the model meets accuracy requirements, ending the tuning process.\n",
      "2023-10-20 10:45:16 [INFO] Specified timeout or max trials is reached! Found a quantized model which meet accuracy goal. Exit.\n",
      "2023-10-20 10:45:16 [INFO] Save deploy yaml to /home/u136698/Training/oneAPI_final/notebooks/nc_workspace/2023-10-20_10-45-08/deploy.yaml\n"
     ]
    }
   ],
   "source": [
    "q_model = quantization.fit(\n",
    "    model,\n",
    "    conf,\n",
    "    calib_dataloader=calib_dataloader\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "29b96d4c",
   "metadata": {},
   "source": [
    "### save the quantized model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "id": "0c58d28a",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2023-10-20 10:45:24 [INFO] Save config file and weights of quantized model to /home/u136698/Training/oneAPI_final/quantized_model.\n"
     ]
    }
   ],
   "source": [
    "q_model.save(\"../quantized_model\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9951d309",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.16"
  },
  "papermill": {
   "default_parameters": {},
   "duration": 41412.439883,
   "end_time": "2023-10-17T22:40:21.679694",
   "environment_variables": {},
   "exception": null,
   "input_path": "__notebook__.ipynb",
   "output_path": "__notebook__.ipynb",
   "parameters": {},
   "start_time": "2023-10-17T11:10:09.239811",
   "version": "2.4.0"
  },
  "widgets": {
   "application/vnd.jupyter.widget-state+json": {
    "state": {
     "0034909014fb459096d298c048afc4fe": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "02c057ee0c384345b919f9e23de61b61": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "FloatProgressModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "FloatProgressModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "ProgressView",
       "bar_style": "success",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_5e98974ea72745669eb55b1b744d3481",
       "max": 25,
       "min": 0,
       "orientation": "horizontal",
       "style": "IPY_MODEL_9ec4120f4ba24bc5b91112fc2fd752fd",
       "value": 25
      }
     },
     "03d5f846016f4b0fbabe6d1b87569c79": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HTMLModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HTMLModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HTMLView",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_f5b66925e12c41749fcac2e39a38fe69",
       "placeholder": "​",
       "style": "IPY_MODEL_9c7691475da744c083a9b6473004625c",
       "value": " 1.79k/1.79k [00:00&lt;00:00, 161kB/s]"
      }
     },
     "05450f031da64bfea0805f9c1a891fc2": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "0561808015a1492e8dc4da39dd39ae75": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HTMLModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HTMLModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HTMLView",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_cdef8019932a4de1867743ab43151712",
       "placeholder": "​",
       "style": "IPY_MODEL_2201f0c157ec4373a7a67aaa5d827049",
       "value": " 792k/792k [00:00&lt;00:00, 4.35MB/s]"
      }
     },
     "0621aa140d8f44348686579bebd7fcef": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HTMLModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HTMLModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HTMLView",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_3dcb9c574d3b4e1cb53dc0b732a7a466",
       "placeholder": "​",
       "style": "IPY_MODEL_b1d5e7bb958b45fa9c2f0fe9e1b15e73",
       "value": "100%"
      }
     },
     "0715825e1ca849aa8d40147c924e1362": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "FloatProgressModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "FloatProgressModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "ProgressView",
       "bar_style": "success",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_a8d113ad362d4147b5c1c544aca08f16",
       "max": 60,
       "min": 0,
       "orientation": "horizontal",
       "style": "IPY_MODEL_d44a472c612d4e4faf27c57c368491c7",
       "value": 60
      }
     },
     "08db67a21c954856930d37efeb791a09": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "DescriptionStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "DescriptionStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "description_width": ""
      }
     },
     "0a919e4f3aa74b7083f328c58ee92742": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "0ea44e9510b048d4989ea9068d681be3": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "130da211b7dd4142849b4b7cf27d1cb1": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "139bade04bdc4010b739b1ecdb0a7484": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "ProgressStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "ProgressStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "bar_color": null,
       "description_width": ""
      }
     },
     "154419c3009e4c37bb2c91d467add5f8": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "DescriptionStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "DescriptionStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "description_width": ""
      }
     },
     "15cab10542384001b1a8668a85392323": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "1b6000b96b794c24b130c5b18cbcb470": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "FloatProgressModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "FloatProgressModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "ProgressView",
       "bar_style": "success",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_15cab10542384001b1a8668a85392323",
       "max": 791656,
       "min": 0,
       "orientation": "horizontal",
       "style": "IPY_MODEL_25fb1dfd939648f3b1c1b3089cfedcfc",
       "value": 791656
      }
     },
     "1c084ac612944f2297e60abbcab04230": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "ProgressStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "ProgressStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "bar_color": null,
       "description_width": ""
      }
     },
     "1e3ca82e9b764fc996abc3aa2690a16f": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "DescriptionStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "DescriptionStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "description_width": ""
      }
     },
     "20439dc8141b4810b7960b910c1aa425": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "2201f0c157ec4373a7a67aaa5d827049": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "DescriptionStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "DescriptionStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "description_width": ""
      }
     },
     "25fb1dfd939648f3b1c1b3089cfedcfc": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "ProgressStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "ProgressStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "bar_color": null,
       "description_width": ""
      }
     },
     "2927a5df63dd40309a02949f9de72c05": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HBoxModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HBoxModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HBoxView",
       "box_style": "",
       "children": [
        "IPY_MODEL_4852d70ea5e8485d87d74bd26830da86",
        "IPY_MODEL_3cbfaa4797ef43699e16fe0759528933",
        "IPY_MODEL_03d5f846016f4b0fbabe6d1b87569c79"
       ],
       "layout": "IPY_MODEL_0ea44e9510b048d4989ea9068d681be3"
      }
     },
     "2992c51668a1453d85ea626293683db4": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HTMLModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HTMLModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HTMLView",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_e8309a1b0ec843698dee23177c4b5533",
       "placeholder": "​",
       "style": "IPY_MODEL_154419c3009e4c37bb2c91d467add5f8",
       "value": "Downloading (…)lve/main/config.json: 100%"
      }
     },
     "2ff9d32df1ef4c5bb4337d450b8a574d": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "3684101a6bd04494bb801500766d4782": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "3751fbfe11db4ce89dc4898b5e089678": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "DescriptionStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "DescriptionStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "description_width": ""
      }
     },
     "3771553bbc534d84beafda70bbd80d45": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HTMLModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HTMLModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HTMLView",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_2ff9d32df1ef4c5bb4337d450b8a574d",
       "placeholder": "​",
       "style": "IPY_MODEL_1e3ca82e9b764fc996abc3aa2690a16f",
       "value": " 1.19G/1.19G [00:04&lt;00:00, 288MB/s]"
      }
     },
     "3c6e1d07768540bd822478957610d8e8": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HTMLModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HTMLModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HTMLView",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_20439dc8141b4810b7960b910c1aa425",
       "placeholder": "​",
       "style": "IPY_MODEL_60c58df0a9034ffab7d1a3aa8b7fddb9",
       "value": " 1.23k/1.23k [00:00&lt;00:00, 93.1kB/s]"
      }
     },
     "3cbfaa4797ef43699e16fe0759528933": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "FloatProgressModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "FloatProgressModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "ProgressView",
       "bar_style": "success",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_0034909014fb459096d298c048afc4fe",
       "max": 1786,
       "min": 0,
       "orientation": "horizontal",
       "style": "IPY_MODEL_7833cc72e9994d6abc6c44fe631416db",
       "value": 1786
      }
     },
     "3dcb9c574d3b4e1cb53dc0b732a7a466": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "455e4aaf11d2459cbfdae0ac4570ca4d": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HTMLModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HTMLModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HTMLView",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_89fd3131f950499ca0ada9dddf96de0e",
       "placeholder": "​",
       "style": "IPY_MODEL_3751fbfe11db4ce89dc4898b5e089678",
       "value": "Downloading model.safetensors: 100%"
      }
     },
     "4607a412d3284311b9f0271c0c0e57ef": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HBoxModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HBoxModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HBoxView",
       "box_style": "",
       "children": [
        "IPY_MODEL_8669410b4b1b4eb683dd668a851e382d",
        "IPY_MODEL_1b6000b96b794c24b130c5b18cbcb470",
        "IPY_MODEL_0561808015a1492e8dc4da39dd39ae75"
       ],
       "layout": "IPY_MODEL_0a919e4f3aa74b7083f328c58ee92742"
      }
     },
     "4852d70ea5e8485d87d74bd26830da86": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HTMLModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HTMLModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HTMLView",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_81aa9b8284bc4714974d9098341007e0",
       "placeholder": "​",
       "style": "IPY_MODEL_eba911cfa02c47b2872c6127e71ffbba",
       "value": "Downloading (…)cial_tokens_map.json: 100%"
      }
     },
     "4899228965d743eb88dd8519cbbc5e51": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "4a256df5d00d45eaa5d04f58a2166935": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "DescriptionStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "DescriptionStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "description_width": ""
      }
     },
     "4d672b361d924c2dba7ba76f6ac984e1": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "FloatProgressModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "FloatProgressModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "ProgressView",
       "bar_style": "success",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_8bf64ec2f770473f985b78af8c691c9d",
       "max": 7,
       "min": 0,
       "orientation": "horizontal",
       "style": "IPY_MODEL_96c99474aef94c10bb30fc72f3450cbb",
       "value": 7
      }
     },
     "54f149dd3a624648b1581b7892d79aa5": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "57fa9846aa6c4bcaa0858c3bf44b996a": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HBoxModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HBoxModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HBoxView",
       "box_style": "",
       "children": [
        "IPY_MODEL_a972491538404370a2d259440b182d17",
        "IPY_MODEL_02c057ee0c384345b919f9e23de61b61",
        "IPY_MODEL_f342788584454082b4f319ebadb5b3d1"
       ],
       "layout": "IPY_MODEL_c837b8bbd5e44900a48051a51647609a"
      }
     },
     "5e98974ea72745669eb55b1b744d3481": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "60c58df0a9034ffab7d1a3aa8b7fddb9": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "DescriptionStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "DescriptionStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "description_width": ""
      }
     },
     "735858b024bd4ee3942e96ff216ee5cb": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "DescriptionStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "DescriptionStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "description_width": ""
      }
     },
     "7833cc72e9994d6abc6c44fe631416db": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "ProgressStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "ProgressStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "bar_color": null,
       "description_width": ""
      }
     },
     "7bd5ed808e624821b02646f32563be4b": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "DescriptionStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "DescriptionStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "description_width": ""
      }
     },
     "7c3dad841fd9426bb86658b41a892920": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "81aa9b8284bc4714974d9098341007e0": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "844b705361324af8b1f845bb153ea885": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HBoxModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HBoxModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HBoxView",
       "box_style": "",
       "children": [
        "IPY_MODEL_0621aa140d8f44348686579bebd7fcef",
        "IPY_MODEL_4d672b361d924c2dba7ba76f6ac984e1",
        "IPY_MODEL_984bb84517cf4d63a0a293d04eda0198"
       ],
       "layout": "IPY_MODEL_aab5889e6147448b9839a99342a5f533"
      }
     },
     "8669410b4b1b4eb683dd668a851e382d": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HTMLModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HTMLModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HTMLView",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_3684101a6bd04494bb801500766d4782",
       "placeholder": "​",
       "style": "IPY_MODEL_ba36429e22fc40398b8bb794bbdbdd02",
       "value": "Downloading (…)ve/main/spiece.model: 100%"
      }
     },
     "89fd3131f950499ca0ada9dddf96de0e": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "8bf64ec2f770473f985b78af8c691c9d": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "96c99474aef94c10bb30fc72f3450cbb": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "ProgressStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "ProgressStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "bar_color": null,
       "description_width": ""
      }
     },
     "97082494b66e419d8eb0f9ade7e627a7": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "FloatProgressModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "FloatProgressModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "ProgressView",
       "bar_style": "success",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_7c3dad841fd9426bb86658b41a892920",
       "max": 1230,
       "min": 0,
       "orientation": "horizontal",
       "style": "IPY_MODEL_139bade04bdc4010b739b1ecdb0a7484",
       "value": 1230
      }
     },
     "984bb84517cf4d63a0a293d04eda0198": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HTMLModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HTMLModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HTMLView",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_130da211b7dd4142849b4b7cf27d1cb1",
       "placeholder": "​",
       "style": "IPY_MODEL_735858b024bd4ee3942e96ff216ee5cb",
       "value": " 7/7 [00:06&lt;00:00,  1.23ba/s]"
      }
     },
     "9c7691475da744c083a9b6473004625c": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "DescriptionStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "DescriptionStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "description_width": ""
      }
     },
     "9d519ec58fea4a1f91c27959e0ad08d0": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "9ec4120f4ba24bc5b91112fc2fd752fd": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "ProgressStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "ProgressStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "bar_color": null,
       "description_width": ""
      }
     },
     "a8d113ad362d4147b5c1c544aca08f16": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "a972491538404370a2d259440b182d17": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HTMLModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HTMLModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HTMLView",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_05450f031da64bfea0805f9c1a891fc2",
       "placeholder": "​",
       "style": "IPY_MODEL_08db67a21c954856930d37efeb791a09",
       "value": "Downloading (…)okenizer_config.json: 100%"
      }
     },
     "aab5889e6147448b9839a99342a5f533": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "ab92abe23e134827b49f30b554f01e30": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "b1d5e7bb958b45fa9c2f0fe9e1b15e73": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "DescriptionStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "DescriptionStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "description_width": ""
      }
     },
     "b28e5aabcd5f45adb457ab4e28d14f67": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "b4419d15bf5a4b9e9f56f5518263740f": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HBoxModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HBoxModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HBoxView",
       "box_style": "",
       "children": [
        "IPY_MODEL_455e4aaf11d2459cbfdae0ac4570ca4d",
        "IPY_MODEL_e50d64f58c864a508eb11a9dc4c6d76a",
        "IPY_MODEL_3771553bbc534d84beafda70bbd80d45"
       ],
       "layout": "IPY_MODEL_fed456b4374d442c949f2d6c8c384097"
      }
     },
     "ba36429e22fc40398b8bb794bbdbdd02": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "DescriptionStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "DescriptionStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "description_width": ""
      }
     },
     "baf442d85e2c46449dbcc5f813e1f748": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HBoxModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HBoxModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HBoxView",
       "box_style": "",
       "children": [
        "IPY_MODEL_2992c51668a1453d85ea626293683db4",
        "IPY_MODEL_97082494b66e419d8eb0f9ade7e627a7",
        "IPY_MODEL_3c6e1d07768540bd822478957610d8e8"
       ],
       "layout": "IPY_MODEL_4899228965d743eb88dd8519cbbc5e51"
      }
     },
     "c837b8bbd5e44900a48051a51647609a": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "ca0124290e1a46bb9b41f833034ffad9": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "DescriptionStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "DescriptionStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "description_width": ""
      }
     },
     "cc15e0fb6b5642919525581d50088938": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HTMLModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HTMLModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HTMLView",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_9d519ec58fea4a1f91c27959e0ad08d0",
       "placeholder": "​",
       "style": "IPY_MODEL_7bd5ed808e624821b02646f32563be4b",
       "value": "100%"
      }
     },
     "cc3e7417509a42558e97760ec9ea7510": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HBoxModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HBoxModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HBoxView",
       "box_style": "",
       "children": [
        "IPY_MODEL_cc15e0fb6b5642919525581d50088938",
        "IPY_MODEL_0715825e1ca849aa8d40147c924e1362",
        "IPY_MODEL_f7c2d48f7dba425181366f3c746a1d01"
       ],
       "layout": "IPY_MODEL_54f149dd3a624648b1581b7892d79aa5"
      }
     },
     "cdef8019932a4de1867743ab43151712": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "d44a472c612d4e4faf27c57c368491c7": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "ProgressStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "ProgressStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "bar_color": null,
       "description_width": ""
      }
     },
     "e50d64f58c864a508eb11a9dc4c6d76a": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "FloatProgressModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "FloatProgressModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "ProgressView",
       "bar_style": "success",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_b28e5aabcd5f45adb457ab4e28d14f67",
       "max": 1187738336,
       "min": 0,
       "orientation": "horizontal",
       "style": "IPY_MODEL_1c084ac612944f2297e60abbcab04230",
       "value": 1187738336
      }
     },
     "e8309a1b0ec843698dee23177c4b5533": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "eba911cfa02c47b2872c6127e71ffbba": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "DescriptionStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "DescriptionStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "description_width": ""
      }
     },
     "f342788584454082b4f319ebadb5b3d1": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HTMLModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HTMLModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HTMLView",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_ab92abe23e134827b49f30b554f01e30",
       "placeholder": "​",
       "style": "IPY_MODEL_4a256df5d00d45eaa5d04f58a2166935",
       "value": " 25.0/25.0 [00:00&lt;00:00, 2.14kB/s]"
      }
     },
     "f5b66925e12c41749fcac2e39a38fe69": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "f62ec8933f654d16967818ec27ce07a3": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "f7c2d48f7dba425181366f3c746a1d01": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HTMLModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HTMLModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HTMLView",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_f62ec8933f654d16967818ec27ce07a3",
       "placeholder": "​",
       "style": "IPY_MODEL_ca0124290e1a46bb9b41f833034ffad9",
       "value": " 60/60 [01:00&lt;00:00,  1.02ba/s]"
      }
     },
     "fed456b4374d442c949f2d6c8c384097": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     }
    },
    "version_major": 2,
    "version_minor": 0
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}