README.md

metadata

license: apache-2.0
datasets:
  - IraGia/gprMax_Train
metrics:
  - accuracy
base_model:
  - unsloth/Meta-Llama-3.1-8B-Instruct-bnb-4bit

Fine-Tuned

---
language: en
license: llama2
base_model: Meta-Llama-3.1-8B-Instruct-bnb-4bit
datasets:
  - gprmax_train
tags:
  - fine-tuned
  - gguf
  - unsloth
  - 4-bit
---

# Meta-Llama-3.1-8B-Instruct-bnb-4bit Fine-Tuned on gprmax_train (GGUF Format)

This repository contains the GGUF format of the fine-tuned `Meta-Llama-3.1-8B-Instruct-bnb-4bit` model, optimized for the `gprmax_train` dataset. The model was fine-tuned using the `unsloth` library for memory and time efficiency and has been converted to GGUF format for compatibility with popular inference frameworks like `llama.cpp`.

## Model Details
- **Base Model**: `Meta-Llama-3.1-8B-Instruct-bnb-4bit`
- **Fine-Tuning Dataset**: `gprmax_train`
- **Fine-Tuning Framework**: `unsloth`
- **Quantization**: 4-bit (bnb)
- **GGUF Conversion**: The model has been converted to GGUF format for efficient inference.

## Download the Model
You can download the GGUF model file from the "Files and versions" section of this repository or use the following command:

```bash
huggingface-cli download sriyaflows/gprmax_8 --include "*.gguf"

Usage with llama.cpp

To run the model using llama.cpp, follow these steps:

1. Install llama.cpp

Clone and build llama.cpp:

git clone https://github.com/ggerganov/llama.cpp
cd llama.cpp
make

2. Download the GGUF Model

Download the GGUF model file from this repository:

wget https://huggingface.co/sriyaflows/gprmax_8/resolve/main/gprmax_8.gguf

3. Run Inference

Use the main executable from llama.cpp to run inference:

./main -m gprmax_8.gguf -p "Your input text here" -n 512

Example

./main -m gprmax_8.gguf -p "What is the output for this input?" -n 512

Usage with Python

You can also use the GGUF model with Python libraries like llama-cpp-python:

1. Install llama-cpp-python

pip install llama-cpp-python

2. Load and Run the Model

from llama_cpp import Llama

# Load the GGUF model
llm = Llama(model_path="gprmax_8.gguf")

# Generate text
output = llm("Your input text here", max_tokens=512)
print(output["choices"][0]["text"])

Fine-Tuning Details

The model was fine-tuned on the gprmax_train dataset using the unsloth library, which optimized memory usage and training time. The fine-tuning process leveraged an L4 GPU with 32GB of RAM and 8 vCPUs.

License

This model is licensed under the same terms as the base Meta-Llama-3.1-8B-Instruct-bnb-4bit model. Refer to the Hugging Face model card for details.

Acknowledgments

• Thanks to Hugging Face for the transformers library.
• Thanks to the unsloth team for their memory-efficient fine-tuning tools.
• Thanks to the creators of the gprmax_train dataset.
• Thanks to ggerganov for llama.cpp and the GGUF format.