Instructions to use eltorio/IDEFICS3_ROCO with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- PEFT
How to use eltorio/IDEFICS3_ROCO with PEFT:
from peft import PeftModel from transformers import AutoModelForCausalLM base_model = AutoModelForCausalLM.from_pretrained("HuggingFaceM4/Idefics3-8B-Llama3") model = PeftModel.from_pretrained(base_model, "eltorio/IDEFICS3_ROCO") - Notebooks
- Google Colab
- Kaggle
æLtorio commited on
up
Browse files
README.md
CHANGED
|
@@ -14,6 +14,12 @@ pipeline_tag: image-to-text
|
|
| 14 |
|
| 15 |
[](https://colab.research.google.com/#fileId=https://huggingface.co/eltorio/IDEFICS3_ROCO/blob/main/ROCO-idefics3.ipynb)
|
| 16 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 17 |
## A Fine-tuned Radiology-focused Model based on Hugging Face's Idefics3 Model
|
| 18 |
|
| 19 |
This repository contains a fine-tuned version of the Hugging Face [Idefics3-8B-Llama3](https://huggingface.co/HuggingFaceM4/Idefics3-8B-Llama3) model, built on top of the Meta Llama 3.1 8B architecture. Our model, `IDEFICS3_ROCO`, has been fine-tuned on the [Radiology Objects in Context (ROCO)](https://huggingface.co/datasets/eltorio/ROCO-radiology) dataset, a large-scale medical and multimodal imaging collection.
|
|
@@ -27,7 +33,7 @@ This repository contains a fine-tuned version of the Hugging Face [Idefics3-8B-L
|
|
| 27 |
|
| 28 |
### Training Progress Status
|
| 29 |
|
| 30 |
-
* Current checkpoint:
|
| 31 |
* Estimated remaining GPU time: ~57 hours
|
| 32 |
* Hardware requirements: T4 GPU with >16GB VRAM
|
| 33 |
* Last update: november, 8th 2024
|
|
@@ -60,7 +66,7 @@ If you use this model in your work, please cite the original Idefics3 model and
|
|
| 60 |
|
| 61 |
2. **Getting Started**
|
| 62 |
* Fork the repository
|
| 63 |
-
* Resume from checkpoint
|
| 64 |
* Follow instructions in [ROCO-idefics3.ipynb](https://huggingface.co/eltorio/IDEFICS3_ROCO/blob/main/ROCO-idefics3.ipynb) [](https://colab.research.google.com/#fileId=https://huggingface.co/eltorio/IDEFICS3_ROCO/blob/main/ROCO-idefics3.ipynb)
|
| 65 |
|
| 66 |
3. **Contact**
|
|
@@ -68,10 +74,15 @@ If you use this model in your work, please cite the original Idefics3 model and
|
|
| 68 |
|
| 69 |
### Docker Image
|
| 70 |
|
| 71 |
-
A AI training docker image is available for this model. The image and includes all necessary dependencies to run the fine-tuning process.
|
|
|
|
|
|
|
|
|
|
|
|
|
| 72 |
|
| 73 |
```bash
|
| 74 |
-
|
|
|
|
| 75 |
```
|
| 76 |
|
| 77 |
The Dockerfile is available in the [IDEFICS_ROCO repository](https://huggingface.co/eltorio/IDEFICS3_ROCO/blob/main/Dockerfile).
|
|
|
|
| 14 |
|
| 15 |
[](https://colab.research.google.com/#fileId=https://huggingface.co/eltorio/IDEFICS3_ROCO/blob/main/ROCO-idefics3.ipynb)
|
| 16 |
|
| 17 |
+
## Star the project
|
| 18 |
+
|
| 19 |
+
**If you appreciate my work, please consider giving it a star! 🤩**
|
| 20 |
+
**I'm also looking for donations of free GPU time to complete the fine-tuning process.**
|
| 21 |
+
**Please contact me if you can help! 🙏**
|
| 22 |
+
|
| 23 |
## A Fine-tuned Radiology-focused Model based on Hugging Face's Idefics3 Model
|
| 24 |
|
| 25 |
This repository contains a fine-tuned version of the Hugging Face [Idefics3-8B-Llama3](https://huggingface.co/HuggingFaceM4/Idefics3-8B-Llama3) model, built on top of the Meta Llama 3.1 8B architecture. Our model, `IDEFICS3_ROCO`, has been fine-tuned on the [Radiology Objects in Context (ROCO)](https://huggingface.co/datasets/eltorio/ROCO-radiology) dataset, a large-scale medical and multimodal imaging collection.
|
|
|
|
| 33 |
|
| 34 |
### Training Progress Status
|
| 35 |
|
| 36 |
+
* Current checkpoint: 3000/12267 (~25% completed)
|
| 37 |
* Estimated remaining GPU time: ~57 hours
|
| 38 |
* Hardware requirements: T4 GPU with >16GB VRAM
|
| 39 |
* Last update: november, 8th 2024
|
|
|
|
| 66 |
|
| 67 |
2. **Getting Started**
|
| 68 |
* Fork the repository
|
| 69 |
+
* Resume from checkpoint 3000/12267
|
| 70 |
* Follow instructions in [ROCO-idefics3.ipynb](https://huggingface.co/eltorio/IDEFICS3_ROCO/blob/main/ROCO-idefics3.ipynb) [](https://colab.research.google.com/#fileId=https://huggingface.co/eltorio/IDEFICS3_ROCO/blob/main/ROCO-idefics3.ipynb)
|
| 71 |
|
| 72 |
3. **Contact**
|
|
|
|
| 74 |
|
| 75 |
### Docker Image
|
| 76 |
|
| 77 |
+
A AI training docker image is available for this model. The image and includes all necessary dependencies to run the fine-tuning process.
|
| 78 |
+
You need to set the `HF_TOKEN` environment variable to your Hugging Face API token.
|
| 79 |
+
You also need to have NVidia Docker container runtime installed.
|
| 80 |
+
Finnaly, you need to run the container with GPU support with `--gpus all` option.
|
| 81 |
+
The image is available on Docker Hub:
|
| 82 |
|
| 83 |
```bash
|
| 84 |
+
export HF_TOKEN=hf_some_token
|
| 85 |
+
docker run --gpus all --user=42420:42420 -e HF_TOKEN=$HF_TOKEN -it sctg/roco-idefics3:latest bash -i /start.sh $HF_TOKEN
|
| 86 |
```
|
| 87 |
|
| 88 |
The Dockerfile is available in the [IDEFICS_ROCO repository](https://huggingface.co/eltorio/IDEFICS3_ROCO/blob/main/Dockerfile).
|