{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "Berikut adalah dokumentasi dalam format Markdown untuk proyek klasifikasi sampah menggunakan ResNet, diikuti oleh implementasi kode.\n", "\n", "---\n", "\n", "# Klasifikasi Sampah Menggunakan ResNet dengan TensorFlow\n", "\n", "## Pendahuluan\n", "Klasifikasi sampah menggunakan deep learning adalah metode yang dapat membantu otomatisasi dalam pemilahan dan pengelolaan limbah secara efisien. Model berbasis arsitektur Residual Network (ResNet) mampu mengenali jenis sampah dengan menggunakan dataset gambar yang dilabeli, yang berpotensi memperbaiki proses daur ulang otomatis. ResNet sering digunakan dalam pengenalan gambar karena kemampuannya mengatasi masalah vanishing gradient melalui blok residual yang mempertahankan informasi penting dari lapisan sebelumnya [(He et al., 2016)](https://consensus.app/papers/resnet-deep-learning-he/9c586abb395db4c2a24d1a4e18388f23/).\n", "\n", "## Lingkungan dan Pengaturan Awal\n", "- **Library yang Digunakan**: TensorFlow digunakan sebagai framework utama untuk deep learning, bersama pustaka `pandas`, `numpy`, dan `matplotlib` untuk manipulasi data dan visualisasi. `wandb` digunakan untuk pelacakan metrik model dan logging eksperimen, yang membantu dalam memantau performa model dan stabilitas selama pelatihan [(Biewald, 2020)](https://consensus.app/papers/weights-biases-open-source-biewald/e1289d2819c3a3d8cb70d44f6a5587b5/).\n", "- **Dataset TrashNet**: Dataset gambar dari berbagai jenis sampah, seperti kertas, plastik, dan kaca, digunakan untuk melatih model. TrashNet menyediakan data yang beragam dan relevan untuk kebutuhan klasifikasi sampah otomatis [(Yang & Thung, 2017)](https://consensus.app/papers/classification-trash-using-deep-learning-yang-thung/e65627e5d9095e9232ff2f193fef10a7/).\n", "\n", "## Arsitektur Model ResNet\n", "ResNet menggunakan struktur blok residual untuk memungkinkan sinyal informasi melewati jaringan dengan minimal gangguan, mengatasi masalah vanishing gradient yang umum pada jaringan dalam [(He et al., 2016)](https://consensus.app/papers/resnet-deep-learning-he/9c586abb395db4c2a24d1a4e18388f23/).\n", "\n", "- **Blok Residual**: Blok residual memungkinkan shortcut connections yang membawa input asli dari satu lapisan ke lapisan berikutnya, menjaga integritas informasi dari lapisan awal [(Szegedy et al., 2015)](https://consensus.app/papers/going-deeper-convolutions-szegedy/1a4e1e4b0f45310d711eaac8ae945a17/).\n", "- **Arsitektur Model**: Model dimulai dengan lapisan konvolusi besar, diikuti dengan pooling dan beberapa blok residual. Global Average Pooling digunakan di akhir jaringan, yang memberikan stabilitas saat menangani data gambar dalam jumlah besar dan bervariasi.\n", "\n", "## Pembagian Data\n", "Untuk melatih model dengan data yang seimbang, dataset dibagi menjadi tiga set: pelatihan, validasi, dan pengujian. Pembagian manual memastikan data terdistribusi secara merata, meminimalkan risiko bias [(James et al., 2013)](https://consensus.app/papers/introduction-statistical-learning-james-et-al/09d17878513a84a3e95c748c8b6e5e93/).\n", "\n", "## Preprocessing dan Augmentasi Data\n", "- **Normalisasi dan Deteksi Blur**: Gambar normalisasi dan deteksi blur diterapkan pada setiap gambar agar kualitas gambar menjadi lebih baik sebelum dimasukkan ke model. Normalisasi dapat meningkatkan akurasi dan stabilitas model [(Krizhevsky et al., 2012)](https://consensus.app/papers/image-net-classification-deep-convolutional-krizhevsky-sutskever-hinton/2e5c8ac2f47d97cf9fc9b515f9aa2b24/).\n", "- **Augmentasi Data**: Augmentasi seperti flip horizontal, rotasi, dan penyesuaian kecerahan digunakan untuk memperkaya data gambar. Teknik ini terbukti meningkatkan kemampuan generalisasi model pada data yang belum pernah dilihat sebelumnya [(Shorten & Khoshgoftaar, 2019)](https://consensus.app/papers/survey-image-data-augmentation-deep-shorten-khoshgoftaar/c2c52f2dd34891f128cc27ff18bc9e48/).\n", "\n", "## Pelatihan Model\n", "Model dilatih pada data pelatihan dan dievaluasi pada data validasi untuk menguji akurasi dan stabilitas. Hanya model yang mencapai akurasi dan stabilitas tertentu yang disimpan, untuk menghindari model overfitting atau underfitting [(Goodfellow et al., 2016)](https://consensus.app/papers/deep-learning-goodfellow-bengio-courville/90a54b117d70e0309978f7d0af87a4ef/).\n", "\n", "## Evaluasi dan Prediksi Model\n", "Evaluasi model menggunakan data pengujian untuk menghitung loss dan akurasi, memberikan gambaran yang lebih baik tentang performa model pada data baru. Akurasi tinggi menunjukkan kemampuan model dalam mengklasifikasikan sampah dengan baik [(Russakovsky et al., 2015)](https://consensus.app/papers/imagenet-large-scale-visual-recognition-challenge-russakovsky-deng/66e0a771282f27a07b05f98f315d2be3/).\n", "\n", "---" ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Error in callback > (for pre_run_cell), with arguments args (,),kwargs {}:\n" ] }, { "ename": "TypeError", "evalue": "_WandbInit._resume_backend() takes 1 positional argument but 2 were given", "output_type": "error", "traceback": [ "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m", "\u001b[0;31mTypeError\u001b[0m Traceback (most recent call last)", "\u001b[0;31mTypeError\u001b[0m: _WandbInit._resume_backend() takes 1 positional argument but 2 were given" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Pengaturan memori GPU berhasil.\n", "Error in callback > (for post_run_cell), with arguments args ( result=None>,),kwargs {}:\n" ] }, { "ename": "TypeError", "evalue": "_WandbInit._pause_backend() takes 1 positional argument but 2 were given", "output_type": "error", "traceback": [ "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m", "\u001b[0;31mTypeError\u001b[0m Traceback (most recent call last)", "\u001b[0;31mTypeError\u001b[0m: _WandbInit._pause_backend() takes 1 positional argument but 2 were given" ] } ], "source": [ "import tensorflow as tf # type: ignore\n", "import numpy as np\n", "import random\n", "from PIL import Image\n", "from tensorflow.keras.preprocessing import image # type: ignore\n", "from tensorflow.keras.applications import ResNet50 # type: ignore\n", "from tensorflow.keras import layers, models # type: ignore\n", "import wandb # type: ignore\n", "import tensorflow as tf # type: ignore\n", "import sys\n", "import numpy as np\n", "import random\n", "import wandb\n", "import os\n", "from tensorflow.keras.optimizers import Adam # type: ignore\n", "from tensorflow.keras.callbacks import ModelCheckpoint # type: ignore\n", "\n", "\n", "# Memeriksa dan mengatur GPU\n", "gpus = tf.config.list_physical_devices('GPU')\n", "if not gpus:\n", " print(\"Tidak ada GPU yang terdeteksi. Program ini memerlukan GPU untuk dijalankan.\")\n", " sys.exit(1) # Menghentikan program dengan kode status 1\n", "\n", "try:\n", " tf.config.experimental.set_memory_growth(gpus[0], True)\n", " print(\"Pengaturan memori GPU berhasil.\")\n", "except Exception as e:\n", " print(f\"Kesalahan saat mengatur memori GPU: {e}\")\n", " sys.exit(1) # Menghentikan program jika terjadi kesalahan saat mengatur GPU\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "\u001b[34m\u001b[1mwandb\u001b[0m: \u001b[33mWARNING\u001b[0m If you're specifying your api key in code, ensure this code is not shared publicly.\n", "\u001b[34m\u001b[1mwandb\u001b[0m: \u001b[33mWARNING\u001b[0m Consider setting the WANDB_API_KEY environment variable, or running `wandb login` from the command line.\n", "\u001b[34m\u001b[1mwandb\u001b[0m: Appending key for api.wandb.ai to your netrc file: /root/.netrc\n" ] }, { "data": { "text/html": [ "wandb version 0.18.6 is available! To upgrade, please run:\n", " $ pip install wandb --upgrade" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "Tracking run with wandb version 0.15.10" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "Run data is saved locally in /notebooks/wandb/run-20241109_012221-3ywsslxm" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "Syncing run super-plant-13 to Weights & Biases (docs)
" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ " View project at https://wandb.ai/kerjta-kkkk/trash-classification" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ " View run at https://wandb.ai/kerjta-kkkk/trash-classification/runs/3ywsslxm" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "name": "stdout", "output_type": "stream", "text": [ "Epoch 1/500\n", "31/32 [============================>.] - ETA: 0s - loss: 1.1242 - accuracy: 0.4647" ] } ], "source": [ "# Fungsi untuk memproses dan melakukan augmentasi pada gambar\n", "import os\n", "import random\n", "import numpy as np\n", "from tensorflow.keras.preprocessing import image # type: ignore\n", "from tensorflow.keras.applications.resnet50 import ResNet50 # type: ignore\n", "from tensorflow.keras.layers import GlobalAveragePooling2D, Dense # type: ignore\n", "from tensorflow.keras.models import Model # type: ignore\n", "from tensorflow.keras.optimizers import Adam # type: ignore\n", "from datasets import load_dataset # type: ignore\n", "from tensorflow.keras.models import save_model # type: ignore\n", "\n", "import wandb\n", "\n", "def preprocess_and_augment_images(dataset, num_images=0, target_size=(224, 224)):\n", " train_images = []\n", " train_labels = []\n", "\n", " for i in range(min(num_images, len(dataset['train']))):\n", " try:\n", " img = dataset['train'][i]['image'].convert('RGB').resize(target_size)\n", " label = dataset['train'][i]['label']\n", " img_array = np.array(img) / 255.0\n", " \n", " # Deteksi blur dan augmentasi gambar\n", " if np.std(img_array) < 0.05:\n", " img_array *= 2 # Tingkatkan kontras\n", " \n", " augment_type = random.choice(['flip', 'rotate', 'brightness'])\n", " if augment_type == 'flip':\n", " img_array = np.fliplr(img_array)\n", " elif augment_type == 'rotate':\n", " img_array = np.rot90(img_array)\n", " elif augment_type == 'brightness':\n", " factor = random.uniform(0.5, 1.5)\n", " img_array = np.clip(img_array * factor, 0, 1)\n", "\n", " train_images.append(img_array)\n", " train_labels.append(label)\n", "\n", " except Exception as e:\n", " print(f\"Error processing image {i + 1}: {e}\")\n", "\n", " return np.array(train_images), np.array(train_labels)\n", "\n", "# Fungsi untuk membuat model ResNet50\n", "def create_resnet50(input_shape, num_classes):\n", " base_model = ResNet50(weights='imagenet', include_top=False, input_shape=input_shape)\n", " x = base_model .output\n", " x = GlobalAveragePooling2D()(x)\n", " outputs = Dense(num_classes, activation='softmax')(x)\n", "\n", " model = Model(inputs=base_model.input, outputs=outputs)\n", "\n", " for layer in base_model.layers:\n", " layer.trainable = False # Membekukan lapisan dasar\n", "\n", " return model\n", "\n", "# Fungsi untuk melatih dan menyimpan model dengan kondisi tertentu\n", "def train_and_save_model(train_images, train_labels, val_images, val_labels, num_classes, epochs=10, b=20):\n", " # Membuat model menggunakan ResNet50\n", " model = create_resnet50(input_shape=(224, 224, 3), num_classes=num_classes)\n", "\n", " # Mengompilasi model dengan optimizer dan loss function\n", " model.compile(optimizer=Adam(), loss='sparse_categorical_crossentropy', metrics=['accuracy'])\n", "\n", " # Melatih model dan menyimpan history\n", " history = model.fit(train_images, train_labels, validation_data=(val_images, val_labels), epochs=epochs, batch_size=b)\n", "\n", " if history.history['val_accuracy'][-1] >= 0.80 and history.history['accuracy'][-1] >= 0.80 and abs(history.history['val_accuracy'][-1]-history.history['accuracy'][-1]) >= 0.01 :\n", " # Log ke wandb\n", " wandb.log({\n", " \"train_accuracy\": history.history['accuracy'][-1],\n", " \"val_accuracy\": history.history['val_accuracy'][-1],\n", " \"train_loss\": history.history['loss'][-1],\n", " \"val_loss\": history.history['val_loss'][-1]\n", " })\n", " model.save('notebooks/simpan_resnet50_model_best.h5')\n", " \n", " save_model(model, 'notebooks/simpan_resnet50_model_best.h5')\n", " # Memeriksa akurasi pada epoch terakhir\n", " \n", " model.save('notebooks/simpan_resnet50_model.h5') # Simpan model jika akurasi validasi <= 80%\n", " save_model(model, 'notebooks/simpan_resnet50_model.h5')\n", " return model\n", "\n", "# Fungsi untuk mengevaluasi model pada data uji\n", "def evaluate_model(model, test_images, test_labels):\n", " loss, accuracy = model.evaluate(test_images, test_labels)\n", " print(f\"Test Loss: {loss:.4f}, Test Accuracy: {accuracy:.4f}\")\n", "\n", "# Fungsi untuk prediksi gambar\n", "def predict_image(model, img_path):\n", " img = image.load_img(img_path, target_size=(224, 224))\n", " img_array = np.array(img) / 255.0\n", " pred = model.predict(np.expand_dims(img_array, axis=0))\n", " predicted_class = np.argmax(pred, axis=-1)\n", " return predicted_class\n", "\n", "# Fungsi utama untuk menjalankan pelatihan dan evaluasi\n", "def main():\n", " # Inisialisasi wandb dan login menggunakan API Key\n", " wandb.login(key='967fafd5558eeeff3ce5681cf55c71633438428d') # Ganti dengan API Key kamu\n", " \n", " wandb.init(project=\"trash-classification\")\n", " # Load dataset TrashNet dari Hugging Face\n", " dataset = load_dataset(\"garythung/trashnet\")\n", "\n", " # Preprocessing dan augmentasi data\n", " train_images, train_labels = preprocess_and_augment_images(dataset, num_images=1000)\n", " val_images, val_labels = preprocess_and_augment_images(dataset, num_images=200)\n", " \n", " # Melatih dan menyimpan model\n", " modelVal = train_and_save_model(train_images, train_labels, val_images, val_labels, num_classes=6, epochs=500, b=32)\n", " \n", " # Evaluasi model pada data uji\n", " test_images, test_labels = preprocess_and_augment_images(dataset, num_images=100)\n", " #evaluate_model(modelVal, test_images, test_labels)\n", "\n", " # Prediksi gambar\n", " #predicted_class = predict_image(modelVal, \"/workspaces/aitask/notebooks/image/sampah.jpg\")\n", " #print(predicted_class)\n", "\n", " # Logout wandb setelah eksperimen selesai\n", " wandb.finish()\n", "\n", "# Menjalankan fungsi utama\n", "if __name__ == \"__main__\":\n", " main()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Jika Hadi" ] } ], "metadata": { "colab": { "provenance": [] }, "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "widgets": { "application/vnd.jupyter.widget-state+json": { "1227005b375c43a59f8149320bb7a1f9": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HBoxModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HBoxModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HBoxView", "box_style": "", "children": [ "IPY_MODEL_f8f8a9de0bec46f9b2f9af48caa5d6a2", "IPY_MODEL_3ebd114674ab430783025b2d96d7a981", "IPY_MODEL_cfa232bb399a43199ec0fe6fbede1034" ], "layout": "IPY_MODEL_61d4c3e65b494f95946b67d12926cc9c" } }, "15a15bd732a94df9a52d13ff44efcdb9": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HBoxModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HBoxModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HBoxView", "box_style": "", "children": [ "IPY_MODEL_658624743f4d4ba0a4674c82297cde22", "IPY_MODEL_2d16cfba17e149c9b09505bb7efc7de9", "IPY_MODEL_ee837e3c5d41400d918f6cc216c735dc" ], "layout": "IPY_MODEL_cd8a8dad899c479ebdd968b1296c5d44" } }, "1889c07918a2421d97d31ba6af6e0b4a": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "1f17ad2fe522472a9ac50a9ccdf44f2e": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "29c82d3e265046bb8d00f5c690bb592d": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "2b8fea821b1e4609abc51fca84f220cb": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "ProgressStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "ProgressStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "bar_color": null, "description_width": "" } }, "2d16cfba17e149c9b09505bb7efc7de9": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "FloatProgressModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "FloatProgressModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "ProgressView", "bar_style": "success", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_61da9c6fd65c4622b5bebc164ba3cb7b", "max": 5054, "min": 0, "orientation": "horizontal", "style": "IPY_MODEL_9030c6ae6bae4d43a5ee6fd64699bde0", "value": 5054 } }, "2d8db0b1575d488f8cceaf7818f9d55f": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "3ebd114674ab430783025b2d96d7a981": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "FloatProgressModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "FloatProgressModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "ProgressView", "bar_style": "success", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_f545237abd674bb787be6aa3f4ed6f9c", "max": 3634187193, "min": 0, "orientation": "horizontal", "style": "IPY_MODEL_af08f90e17144df7ae429731c53c2cc5", "value": 3634187193 } }, "55164fa51e6044d8a0b9ae25b8887304": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "60f363d5dff24c8d84a8aa539727491e": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "61d4c3e65b494f95946b67d12926cc9c": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "61da9c6fd65c4622b5bebc164ba3cb7b": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "658624743f4d4ba0a4674c82297cde22": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_efeaa58b1cca4e51954e581b5da14b7b", "placeholder": "​", "style": "IPY_MODEL_8b18bb0b8c6b46b3a66c7b3b968e8ba1", "value": "Generating train split: 100%" } }, "70a83c6df7f748cf89d794795a27242e": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "7df7de58d4814c02ae20920edee3bc9e": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HBoxModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HBoxModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HBoxView", "box_style": "", "children": [ "IPY_MODEL_c70c52247ce0471b8eac8d3d3548ac75", "IPY_MODEL_a6e2d3dd99b540e4bcfe352118ceaf9a", "IPY_MODEL_d20b9f47b626480f9b0d73b0fce88b4b" ], "layout": "IPY_MODEL_60f363d5dff24c8d84a8aa539727491e" } }, "8a2f024fb2e64b78be8a8e061001e4d7": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "8b18bb0b8c6b46b3a66c7b3b968e8ba1": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "9030c6ae6bae4d43a5ee6fd64699bde0": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "ProgressStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "ProgressStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "bar_color": null, "description_width": "" } }, "9ce879d1f1e24cb7aad74d8557413895": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "a6e2d3dd99b540e4bcfe352118ceaf9a": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "FloatProgressModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "FloatProgressModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "ProgressView", "bar_style": "success", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_1f17ad2fe522472a9ac50a9ccdf44f2e", "max": 42834870, "min": 0, "orientation": "horizontal", "style": "IPY_MODEL_2b8fea821b1e4609abc51fca84f220cb", "value": 42834870 } }, "af08f90e17144df7ae429731c53c2cc5": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "ProgressStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "ProgressStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "bar_color": null, "description_width": "" } }, "c70c52247ce0471b8eac8d3d3548ac75": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_9ce879d1f1e24cb7aad74d8557413895", "placeholder": "​", "style": "IPY_MODEL_e289a91e30c84bdbbf8d4115a495eae8", "value": "dataset-resized.zip: 100%" } }, "cd8a8dad899c479ebdd968b1296c5d44": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "cfa232bb399a43199ec0fe6fbede1034": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_1889c07918a2421d97d31ba6af6e0b4a", "placeholder": "​", "style": "IPY_MODEL_8a2f024fb2e64b78be8a8e061001e4d7", "value": " 3.63G/3.63G [01:18<00:00, 36.5MB/s]" } }, "d20b9f47b626480f9b0d73b0fce88b4b": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_dac04a166f7946a3bced900875dee264", "placeholder": "​", "style": "IPY_MODEL_55164fa51e6044d8a0b9ae25b8887304", "value": " 42.8M/42.8M [00:01<00:00, 35.3MB/s]" } }, "dac04a166f7946a3bced900875dee264": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "e289a91e30c84bdbbf8d4115a495eae8": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "ee837e3c5d41400d918f6cc216c735dc": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_70a83c6df7f748cf89d794795a27242e", "placeholder": "​", "style": "IPY_MODEL_fc4b6974d57341208150ec3780c8946c", "value": " 5054/5054 [00:01<00:00, 5525.20 examples/s]" } }, "efeaa58b1cca4e51954e581b5da14b7b": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "f545237abd674bb787be6aa3f4ed6f9c": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "f8f8a9de0bec46f9b2f9af48caa5d6a2": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_2d8db0b1575d488f8cceaf7818f9d55f", "placeholder": "​", "style": "IPY_MODEL_29c82d3e265046bb8d00f5c690bb592d", "value": "dataset-original.zip: 100%" } }, "fc4b6974d57341208150ec3780c8946c": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } } } } }, "nbformat": 4, "nbformat_minor": 4 }