| --- |
| license: apache-2.0 |
| datasets: |
| - IraGia/gprMax_Train |
| metrics: |
| - accuracy |
| base_model: |
| - unsloth/Meta-Llama-3.1-8B-Instruct-bnb-4bit |
| --- |
| Fine-Tuned |
|
|
| ```markdown |
| --- |
| language: en |
| license: llama2 |
| base_model: Meta-Llama-3.1-8B-Instruct-bnb-4bit |
| datasets: |
| - gprmax_train |
| tags: |
| - fine-tuned |
| - gguf |
| - unsloth |
| - 4-bit |
| --- |
|
|
| # Meta-Llama-3.1-8B-Instruct-bnb-4bit Fine-Tuned on gprmax_train (GGUF Format) |
| |
| This repository contains the GGUF format of the fine-tuned `Meta-Llama-3.1-8B-Instruct-bnb-4bit` model, optimized for the `gprmax_train` dataset. The model was fine-tuned using the `unsloth` library for memory and time efficiency and has been converted to GGUF format for compatibility with popular inference frameworks like `llama.cpp`. |
|
|
| ## Model Details |
| - **Base Model**: `Meta-Llama-3.1-8B-Instruct-bnb-4bit` |
| - **Fine-Tuning Dataset**: `gprmax_train` |
| - **Fine-Tuning Framework**: `unsloth` |
| - **Quantization**: 4-bit (bnb) |
| - **GGUF Conversion**: The model has been converted to GGUF format for efficient inference. |
|
|
| ## Download the Model |
| You can download the GGUF model file from the "Files and versions" section of this repository or use the following command: |
|
|
| ```bash |
| huggingface-cli download sriyaflows/gprmax_8 --include "*.gguf" |
| ``` |
|
|
| ## Usage with `llama.cpp` |
|
|
| To run the model using `llama.cpp`, follow these steps: |
|
|
| ### 1. Install `llama.cpp` |
| Clone and build `llama.cpp`: |
|
|
| ```bash |
| git clone https://github.com/ggerganov/llama.cpp |
| cd llama.cpp |
| make |
| ``` |
|
|
| ### 2. Download the GGUF Model |
| Download the GGUF model file from this repository: |
|
|
| ```bash |
| wget https://huggingface.co/sriyaflows/gprmax_8/resolve/main/gprmax_8.gguf |
| ``` |
|
|
| ### 3. Run Inference |
| Use the `main` executable from `llama.cpp` to run inference: |
|
|
| ```bash |
| ./main -m gprmax_8.gguf -p "Your input text here" -n 512 |
| ``` |
|
|
| ### Example |
| ```bash |
| ./main -m gprmax_8.gguf -p "What is the output for this input?" -n 512 |
| ``` |
|
|
| ## Usage with Python |
| You can also use the GGUF model with Python libraries like `llama-cpp-python`: |
|
|
| ### 1. Install `llama-cpp-python` |
| ```bash |
| pip install llama-cpp-python |
| ``` |
|
|
| ### 2. Load and Run the Model |
| ```python |
| from llama_cpp import Llama |
| |
| # Load the GGUF model |
| llm = Llama(model_path="gprmax_8.gguf") |
| |
| # Generate text |
| output = llm("Your input text here", max_tokens=512) |
| print(output["choices"][0]["text"]) |
| ``` |
|
|
| ## Fine-Tuning Details |
| The model was fine-tuned on the `gprmax_train` dataset using the `unsloth` library, which optimized memory usage and training time. The fine-tuning process leveraged an L4 GPU with 32GB of RAM and 8 vCPUs. |
|
|
| ## License |
| This model is licensed under the same terms as the base `Meta-Llama-3.1-8B-Instruct-bnb-4bit` model. Refer to the [Hugging Face model card](https://huggingface.co/Meta-Llama-3.1-8B-Instruct-bnb-4bit) for details. |
|
|
| ## Acknowledgments |
| - Thanks to Hugging Face for the `transformers` library. |
| - Thanks to the `unsloth` team for their memory-efficient fine-tuning tools. |
| - Thanks to the creators of the `gprmax_train` dataset. |
| - Thanks to `ggerganov` for `llama.cpp` and the GGUF format. |
| ``` |
