{ "cells": [ { "cell_type": "markdown", "metadata": { "id": "C_jdZ5vHJ4A9" }, "source": [ "# Task description\n", "- Classify the speakers of given features.\n", "- Main goal: Learn how to use transformer.\n", "- Baselines:\n", " - Easy: Run sample code and know how to use transformer.\n", " - Medium: Know how to adjust parameters of transformer.\n", " - Strong: Construct [conformer](https://arxiv.org/abs/2005.08100) which is a variety of transformer. \n", " - Boss: Implement [Self-Attention Pooling](https://arxiv.org/pdf/2008.01077v1.pdf) & [Additive Margin Softmax](https://arxiv.org/pdf/1801.05599.pdf) to further boost the performance.\n", "\n", "- Other links\n", " - Competiton: [link](https://www.kaggle.com/t/49ea0c385a974db5919ec67299ba2e6b)\n", " - Slide: [link](https://docs.google.com/presentation/d/1LDAW0GGrC9B6D7dlNdYzQL6D60-iKgFr/edit?usp=sharing&ouid=104280564485377739218&rtpof=true&sd=true)\n", " - Data: [link](https://github.com/googly-mingto/ML2023HW4/releases)\n", "\n", "# Download dataset\n", "- Data is [here](https://drive.google.com/drive/folders/1vI1kuLB-q1VilIftiwnPOCAeOOFfBZge?usp=sharing)" ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "gtKxUzSgXKj3", "outputId": "3f59402c-95a7-4fbd-a39c-57606590a89c" }, "outputs": [], "source": [ "#!wget https://github.com/googly-mingto/ML2023HW4/releases/download/data/Dataset.tar.gz.partaa\n", "#!wget https://github.com/googly-mingto/ML2023HW4/releases/download/data/Dataset.tar.gz.partab\n", "#!wget https://github.com/googly-mingto/ML2023HW4/releases/download/data/Dataset.tar.gz.partac\n", "#!wget https://github.com/googly-mingto/ML2023HW4/releases/download/data/Dataset.tar.gz.partad\n", "\n", "#!cat Dataset.tar.gz.part* > Dataset.tar.gz\n", "#!rm Dataset.tar.gz.partaa\n", "#!rm Dataset.tar.gz.partab\n", "#!rm Dataset.tar.gz.partac\n", "#!rm Dataset.tar.gz.partad\n", "# unzip the file\n", "#!tar zxf Dataset.tar.gz\n", "#!rm Dataset.tar.gz" ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "U6Y1cfpDfpON", "outputId": "6ba26637-5c7b-48a9-be0b-1f10ba76590a" }, "outputs": [], "source": [ "#!tar zxf Dataset.tar.gz" ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "id": "E6burzCXIyuA" }, "outputs": [], "source": [ "import numpy as np\n", "import torch\n", "import random\n", "from common_function import train_start_log, train_log\n", "\n", "def set_seed(seed):\n", " np.random.seed(seed)\n", " random.seed(seed)\n", " torch.manual_seed(seed)\n", " if torch.cuda.is_available():\n", " torch.cuda.manual_seed(seed)\n", " torch.cuda.manual_seed_all(seed)\n", " torch.mps.manual_seed(seed)\n", " torch.mps.seed(seed)\n", " torch.backends.cudnn.benchmark = False\n", " torch.backends.cudnn.deterministic = True\n", "\n", "set_seed(8787)" ] }, { "cell_type": "markdown", "metadata": { "id": "k7dVbxW2LASN" }, "source": [ "# Data\n", "\n", "## Dataset\n", "- Original dataset is [Voxceleb2](https://www.robots.ox.ac.uk/~vgg/data/voxceleb/vox2.html).\n", "- The [license](https://creativecommons.org/licenses/by/4.0/) and [complete version](https://www.robots.ox.ac.uk/~vgg/data/voxceleb/files/license.txt) of Voxceleb2.\n", "- We randomly select 600 speakers from Voxceleb2.\n", "- Then preprocess the raw waveforms into mel-spectrograms.\n", "\n", "- Args:\n", " - data_dir: The path to the data directory.\n", " - metadata_path: The path to the metadata.\n", " - segment_len: The length of audio segment for training. \n", "- The architecture of data directory \\\\\n", " - data directory \\\\\n", " |---- metadata.json \\\\\n", " |---- testdata.json \\\\\n", " |---- mapping.json \\\\\n", " |---- uttr-{random string}.pt \\\\\n", "\n", "- The information in metadata\n", " - \"n_mels\": The dimention of mel-spectrogram.\n", " - \"speakers\": A dictionary. \n", " - Key: speaker ids.\n", " - value: \"feature_path\" and \"mel_len\"\n", "\n", "\n", "For efficiency, we segment the mel-spectrograms into segments in the traing step." ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "id": "KpuGxl4CI2pr" }, "outputs": [], "source": [ "import os\n", "import json\n", "import torch\n", "import random\n", "from pathlib import Path\n", "from torch.utils.data import Dataset\n", "from torch.nn.utils.rnn import pad_sequence\n", "from common_function import myDataset" ] }, { "cell_type": "markdown", "metadata": { "id": "668hverTMlGN" }, "source": [ "## Dataloader\n", "- Split dataset into training dataset(90%) and validation dataset(10%).\n", "- Create dataloader to iterate the data." ] }, { "cell_type": "code", "execution_count": 3, "metadata": { "id": "B7c2gZYoJDRS" }, "outputs": [], "source": [ "import torch\n", "from torch.utils.data import DataLoader, random_split\n", "from torch.nn.utils.rnn import pad_sequence\n", "\n", "\n", "from common_function import collate_batch\n", "\n", "\n", "def get_dataloader(data_dir, batch_size, n_workers):\n", " \"\"\"Generate dataloader\"\"\"\n", " dataset = myDataset(data_dir)\n", " speaker_num = dataset.get_speaker_number()\n", " # Split dataset into training dataset and validation dataset\n", " trainlen = int(0.9 * len(dataset))\n", " lengths = [trainlen, len(dataset) - trainlen]\n", " trainset, validset = random_split(dataset, lengths)\n", "\n", " train_loader = DataLoader(\n", " trainset,\n", " batch_size=batch_size,\n", " shuffle=True,\n", " drop_last=True,\n", " num_workers=n_workers,\n", " pin_memory=True,\n", " collate_fn=collate_batch,\n", " )\n", " valid_loader = DataLoader(\n", " validset,\n", " batch_size=batch_size,\n", " num_workers=n_workers,\n", " drop_last=True,\n", " pin_memory=True,\n", " collate_fn=collate_batch,\n", " )\n", "\n", " return train_loader, valid_loader, speaker_num" ] }, { "cell_type": "markdown", "metadata": { "id": "5FOSZYxrMqhc" }, "source": [ "# Model\n", "- TransformerEncoderLayer:\n", " - Base transformer encoder layer in [Attention Is All You Need](https://arxiv.org/abs/1706.03762)\n", " - Parameters:\n", " - d_model: the number of expected features of the input (required).\n", "\n", " - nhead: the number of heads of the multiheadattention models (required).\n", "\n", " - dim_feedforward: the dimension of the feedforward network model (default=2048).\n", "\n", " - dropout: the dropout value (default=0.1).\n", "\n", " - activation: the activation function of intermediate layer, relu or gelu (default=relu).\n", "\n", "- TransformerEncoder:\n", " - TransformerEncoder is a stack of N transformer encoder layers\n", " - Parameters:\n", " - encoder_layer: an instance of the TransformerEncoderLayer() class (required).\n", "\n", " - num_layers: the number of sub-encoder-layers in the encoder (required).\n", "\n", " - norm: the layer normalization component (optional)." ] }, { "cell_type": "code", "execution_count": 4, "metadata": { "id": "iXZ5B0EKJGs8" }, "outputs": [], "source": [ "import torch\n", "import torch.nn as nn\n", "import torch.nn.functional as F\n", "import torchaudio\n", "\n", "class Classifier(nn.Module):\n", " def __init__(self, d_model=160, n_spks=600, dropout=0.5):\n", " super().__init__()\n", " # Project the dimension of features from that of input into d_model.\n", " self.prenet = nn.Linear(40, d_model)\n", " # TODO:\n", " # Change Transformer to Conformer.\n", " # https://arxiv.org/abs/2005.08100\n", " #self.encoder_layer = nn.TransformerEncoderLayer(\n", " # d_model=d_model, dim_feedforward=256, nhead=8\n", " #)\n", " #self.encoder = nn.TransformerEncoder(self.encoder_layer, num_layers=2)\n", " self.encoder = torchaudio.models.Conformer(d_model, 8, 256, 2, 31, dropout=dropout) # From torchaudio\n", " \n", " # Project the the dimension of features from d_model into speaker nums.\n", " self.pred_layer = nn.Linear(d_model, n_spks)\n", "\n", " def forward(self, mels):\n", " \"\"\"\n", " args:\n", " mels: (batch size, length, 40)\n", " return:\n", " out: (batch size, n_spks)\n", " \"\"\"\n", " # out: (batch size, length, d_model)\n", " out = self.prenet(mels)\n", " # out: (length, batch size, d_model)\n", " #out = out.permute(1, 0, 2)\n", " # The encoder layer expect features in the shape of (length, batch size, d_model).\n", " device = \"cuda\" if torch.cuda.is_available() else 'mps' if torch.backends.mps.is_available() else 'cpu'\n", " #lengths = torch.randint(out.size(1), (int(out.size(0)),)).to(device)\n", " #lengths[torch.argmax(lengths)] = out.size(1)\n", " lengths = torch.full((int(out.size(0)),), out.size(1)).to(device)\n", " out, _ = self.encoder(out, lengths)\n", " # out: (batch size, length, d_model)\n", " #out = out.transpose(0, 1)\n", " # mean pooling\n", " #stats = out.mean(dim=1)\n", "\n", " # out: (batch, n_spks)\n", " stats = out.mean(dim=1)\n", " out = self.pred_layer(stats)\n", " return out, _" ] }, { "cell_type": "markdown", "metadata": { "id": "W7yX8JinM5Ly" }, "source": [ "# Learning rate schedule\n", "- For transformer architecture, the design of learning rate schedule is different from that of CNN.\n", "- Previous works show that the warmup of learning rate is useful for training models with transformer architectures.\n", "- The warmup schedule\n", " - Set learning rate to 0 in the beginning.\n", " - The learning rate increases linearly from 0 to initial learning rate during warmup period." ] }, { "cell_type": "code", "execution_count": 5, "metadata": { "id": "ykt0N1nVJJi2" }, "outputs": [], "source": [ "import math\n", "\n", "import torch\n", "from torch.optim import Optimizer\n", "from torch.optim.lr_scheduler import LambdaLR\n", "\n", "\n", "def get_cosine_schedule_with_warmup(\n", " optimizer: Optimizer,\n", " num_warmup_steps: int,\n", " num_training_steps: int,\n", " num_cycles: float = 0.5,\n", " last_epoch: int = -1,\n", "):\n", " \"\"\"\n", " Create a schedule with a learning rate that decreases following the values of the cosine function between the\n", " initial lr set in the optimizer to 0, after a warmup period during which it increases linearly between 0 and the\n", " initial lr set in the optimizer.\n", "\n", " Args:\n", " optimizer (:class:`~torch.optim.Optimizer`):\n", " The optimizer for which to schedule the learning rate.\n", " num_warmup_steps (:obj:`int`):\n", " The number of steps for the warmup phase.\n", " num_training_steps (:obj:`int`):\n", " The total number of training steps.\n", " num_cycles (:obj:`float`, `optional`, defaults to 0.5):\n", " The number of waves in the cosine schedule (the defaults is to just decrease from the max value to 0\n", " following a half-cosine).\n", " last_epoch (:obj:`int`, `optional`, defaults to -1):\n", " The index of the last epoch when resuming training.\n", "\n", " Return:\n", " :obj:`torch.optim.lr_scheduler.LambdaLR` with the appropriate schedule.\n", " \"\"\"\n", " def lr_lambda(current_step):\n", " # Warmup\n", " if current_step < num_warmup_steps:\n", " return float(current_step) / float(max(1, num_warmup_steps))\n", " # decadence\n", " progress = float(current_step - num_warmup_steps) / float(\n", " max(1, num_training_steps - num_warmup_steps)\n", " )\n", " return max(\n", " 0.0, 0.5 * (1.0 + math.cos(math.pi * float(num_cycles) * 2.0 * progress))\n", " )\n", "\n", " return LambdaLR(optimizer, lr_lambda, last_epoch)" ] }, { "cell_type": "markdown", "metadata": { "id": "-LN2XkteM_uH" }, "source": [ "# Model Function\n", "- Model forward function." ] }, { "cell_type": "code", "execution_count": 6, "metadata": { "id": "N-rr8529JMz0" }, "outputs": [], "source": [ "import torch\n", "\n", "\n", "def model_fn(batch, model, criterion, device):\n", " \"\"\"Forward a batch through the model.\"\"\"\n", "\n", " mels, labels = batch\n", " mels = mels.to(device)\n", " labels = labels.to(device)\n", "\n", " outs, outs_length = model(mels)\n", " \n", " loss = criterion(outs, labels)\n", "\n", " # Get the speaker id with highest probability.\n", " preds = outs.argmax(1)\n", " # Compute accuracy.\n", " accuracy = torch.mean((preds == labels).float())\n", "\n", " return loss, accuracy" ] }, { "cell_type": "markdown", "metadata": { "id": "cwM_xyOtNCI2" }, "source": [ "# Validate\n", "- Calculate accuracy of the validation set." ] }, { "cell_type": "code", "execution_count": 7, "metadata": { "id": "YAiv6kpdJRTJ" }, "outputs": [], "source": [ "from tqdm import tqdm\n", "import torch\n", "\n", "\n", "def valid(dataloader, model, criterion, device): \n", " \"\"\"Validate on validation set.\"\"\"\n", "\n", " model.eval()\n", " running_loss = 0.0\n", " running_accuracy = 0.0\n", " pbar = tqdm(total=len(dataloader.dataset), ncols=0, desc=\"Valid\", unit=\" uttr\")\n", "\n", " for i, batch in enumerate(dataloader):\n", " with torch.no_grad():\n", " loss, accuracy = model_fn(batch, model, criterion, device)\n", " running_loss += loss.item()\n", " running_accuracy += accuracy.item()\n", "\n", " pbar.update(dataloader.batch_size)\n", " pbar.set_postfix(\n", " loss=f\"{running_loss / (i+1):.2f}\",\n", " accuracy=f\"{running_accuracy / (i+1):.2f}\",\n", " )\n", "\n", " pbar.close()\n", " model.train()\n", "\n", " return running_accuracy / len(dataloader)" ] }, { "cell_type": "markdown", "metadata": { "id": "g6ne9G-eNEdG" }, "source": [ "# Main function" ] }, { "cell_type": "code", "execution_count": 10, "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "Usv9s-CuJSG7", "outputId": "f4f6a983-3559-4f36-efae-402bbf790473" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "[Info]: Use mps now!\n", "[Info]: Finish loading data!\n", "[Info]: Finish creating model!\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "Train: 100% 2000/2000 [04:36<00:00, 7.24 step/s, accuracy=0.19, loss=3.68, step=2000] \n", "Valid: 100% 5664/5667 [00:52<00:00, 107.38 uttr/s, accuracy=0.23, loss=3.66]\n", "Train: 100% 2000/2000 [04:33<00:00, 7.30 step/s, accuracy=0.47, loss=2.42, step=4000] \n", "Valid: 100% 5664/5667 [00:55<00:00, 101.57 uttr/s, accuracy=0.40, loss=2.75]\n", "Train: 100% 2000/2000 [04:53<00:00, 6.82 step/s, accuracy=0.53, loss=1.94, step=6000] \n", "Valid: 100% 5664/5667 [00:54<00:00, 103.31 uttr/s, accuracy=0.51, loss=2.22]\n", "Train: 100% 2000/2000 [05:36<00:00, 5.94 step/s, accuracy=0.69, loss=1.35, step=8000] \n", "Valid: 100% 5664/5667 [00:52<00:00, 108.07 uttr/s, accuracy=0.58, loss=1.88]\n", "Train: 100% 2000/2000 [04:40<00:00, 7.13 step/s, accuracy=0.75, loss=0.93, step=1e+4] \n", "Valid: 100% 5664/5667 [00:53<00:00, 106.50 uttr/s, accuracy=0.63, loss=1.69]\n", "Train: 0% 0/2000 [00:00 best_accuracy:\n", " best_accuracy = valid_accuracy\n", " best_state_dict = model.state_dict()\n", "\n", " pbar = tqdm(total=valid_steps, ncols=0, desc=\"Train\", unit=\" step\")\n", "\n", " # Save the best model so far.\n", " if (step + 1) % save_steps == 0 and best_state_dict is not None:\n", " torch.save(best_state_dict, save_path)\n", " pbar.write(f\"Step {step + 1}, best model saved. (accuracy={best_accuracy:.4f})\")\n", " train_log({\n", " 'status': 'best_model_save',\n", " 'step': step+1,\n", " 'accuracy': f\"{best_accuracy:.6f}\",\n", " })\n", "\n", " pbar.close()\n", " train_log({\n", " 'status': 'completed'\n", " })\n", " \n", "if __name__ == \"__main__\":\n", " main(**parse_args())" ] }, { "cell_type": "code", "execution_count": 11, "metadata": {}, "outputs": [], "source": [ "import gc\n", "\n", "#del train_loader, valid_loader\n", "gc.collect()" ] }, { "cell_type": "markdown", "metadata": { "id": "NLatBYAhNNMx" }, "source": [ "# Inference\n", "\n", "## Dataset of inference" ] }, { "cell_type": "code", "execution_count": 8, "metadata": { "id": "efS4pCmAJXJH" }, "outputs": [], "source": [ "import os\n", "import json\n", "import torch\n", "from pathlib import Path\n", "from torch.utils.data import Dataset\n", "\n", "\n", "from common_function import InferenceDataset, inference_collate_batch\n" ] }, { "cell_type": "markdown", "metadata": { "id": "tl0WnYwxNK_S" }, "source": [ "## Main funcrion of Inference" ] }, { "cell_type": "code", "execution_count": 9, "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 103, "referenced_widgets": [ "6786c2b0e2614ad389620246cb2178f2", "0d592098920140dab61aac5410568c36", "da681e3cc353420cb142d56df0fce231", "401ae722b95c4ff59b836422dbe71edc", "4efbfb7c7cb54276862e5321209d57fa", "f6dcb3ec9c624171966bb889808dbcb3", "100abf072991474abfcee871d2b83f37", "decc12da6f5742ec8b7ec7789ee434ab", "0117ac88c98440b29ab1f452107cbe1a", "3d64942a0eaa409a93df049bb594c062", "8c8721e504cf434eb9e263fb9983969a" ] }, "id": "i8SAbuXEJb2A", "outputId": "3808f409-19c9-426c-dc15-1b88b0c21645" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "[Info]: Use mps now!\n", "[Info]: Finish loading data!\n", "[Info]: Finish creating model!\n" ] }, { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "4a12f429161a47e2832ec39d1fe0b251", "version_major": 2, "version_minor": 0 }, "text/plain": [ " 0%| | 0/8000 [00:00 2\u001b[0m \u001b[38;5;28;01mdel\u001b[39;00m dataloader\n\u001b[1;32m 3\u001b[0m gc\u001b[38;5;241m.\u001b[39mcollect()\n", "\u001b[0;31mNameError\u001b[0m: name 'dataloader' is not defined" ] } ], "source": [ "import gc\n", "del dataloader\n", "gc.collect()" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] } ], "metadata": { "accelerator": "GPU", "colab": { "provenance": [] }, "gpuClass": "standard", "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.10.5" }, "widgets": { "application/vnd.jupyter.widget-state+json": { "0117ac88c98440b29ab1f452107cbe1a": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "ProgressStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "ProgressStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "bar_color": null, "description_width": "" } }, "0d592098920140dab61aac5410568c36": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_f6dcb3ec9c624171966bb889808dbcb3", "placeholder": "​", "style": "IPY_MODEL_100abf072991474abfcee871d2b83f37", "value": "100%" } }, "100abf072991474abfcee871d2b83f37": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "3d64942a0eaa409a93df049bb594c062": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "401ae722b95c4ff59b836422dbe71edc": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_3d64942a0eaa409a93df049bb594c062", "placeholder": "​", "style": "IPY_MODEL_8c8721e504cf434eb9e263fb9983969a", "value": " 8000/8000 [00:33<00:00, 256.07it/s]" } }, "4efbfb7c7cb54276862e5321209d57fa": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "6786c2b0e2614ad389620246cb2178f2": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HBoxModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HBoxModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HBoxView", "box_style": "", "children": [ "IPY_MODEL_0d592098920140dab61aac5410568c36", "IPY_MODEL_da681e3cc353420cb142d56df0fce231", "IPY_MODEL_401ae722b95c4ff59b836422dbe71edc" ], "layout": "IPY_MODEL_4efbfb7c7cb54276862e5321209d57fa" } }, "8c8721e504cf434eb9e263fb9983969a": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "da681e3cc353420cb142d56df0fce231": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "FloatProgressModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "FloatProgressModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "ProgressView", "bar_style": "success", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_decc12da6f5742ec8b7ec7789ee434ab", "max": 8000, "min": 0, "orientation": "horizontal", "style": "IPY_MODEL_0117ac88c98440b29ab1f452107cbe1a", "value": 8000 } }, "decc12da6f5742ec8b7ec7789ee434ab": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "f6dcb3ec9c624171966bb889808dbcb3": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } } } } }, "nbformat": 4, "nbformat_minor": 1 }