---
license: apache-2.0
datasets:
- IraGia/gprMax_Train
metrics:
- accuracy
base_model:
- unsloth/Meta-Llama-3.1-8B-Instruct-bnb-4bit
---
Fine-Tuned

```markdown
---
language: en
license: llama2
base_model: Meta-Llama-3.1-8B-Instruct-bnb-4bit
datasets:
  - gprmax_train
tags:
  - fine-tuned
  - gguf
  - unsloth
  - 4-bit
---

# Meta-Llama-3.1-8B-Instruct-bnb-4bit Fine-Tuned on gprmax_train (GGUF Format)

This repository contains the GGUF format of the fine-tuned `Meta-Llama-3.1-8B-Instruct-bnb-4bit` model, optimized for the `gprmax_train` dataset. The model was fine-tuned using the `unsloth` library for memory and time efficiency and has been converted to GGUF format for compatibility with popular inference frameworks like `llama.cpp`.

## Model Details
- **Base Model**: `Meta-Llama-3.1-8B-Instruct-bnb-4bit`
- **Fine-Tuning Dataset**: `gprmax_train`
- **Fine-Tuning Framework**: `unsloth`
- **Quantization**: 4-bit (bnb)
- **GGUF Conversion**: The model has been converted to GGUF format for efficient inference.

## Download the Model
You can download the GGUF model file from the "Files and versions" section of this repository or use the following command:

```bash
huggingface-cli download sriyaflows/gprmax_8 --include "*.gguf"
```

## Usage with `llama.cpp`

To run the model using `llama.cpp`, follow these steps:

### 1. Install `llama.cpp`
Clone and build `llama.cpp`:

```bash
git clone https://github.com/ggerganov/llama.cpp
cd llama.cpp
make
```

### 2. Download the GGUF Model
Download the GGUF model file from this repository:

```bash
wget https://huggingface.co/sriyaflows/gprmax_8/resolve/main/gprmax_8.gguf
```

### 3. Run Inference
Use the `main` executable from `llama.cpp` to run inference:

```bash
./main -m gprmax_8.gguf -p "Your input text here" -n 512
```

### Example
```bash
./main -m gprmax_8.gguf -p "What is the output for this input?" -n 512
```

## Usage with Python
You can also use the GGUF model with Python libraries like `llama-cpp-python`:

### 1. Install `llama-cpp-python`
```bash
pip install llama-cpp-python
```

### 2. Load and Run the Model
```python
from llama_cpp import Llama

# Load the GGUF model
llm = Llama(model_path="gprmax_8.gguf")

# Generate text
output = llm("Your input text here", max_tokens=512)
print(output["choices"][0]["text"])
```

## Fine-Tuning Details
The model was fine-tuned on the `gprmax_train` dataset using the `unsloth` library, which optimized memory usage and training time. The fine-tuning process leveraged an L4 GPU with 32GB of RAM and 8 vCPUs.

## License
This model is licensed under the same terms as the base `Meta-Llama-3.1-8B-Instruct-bnb-4bit` model. Refer to the [Hugging Face model card](https://huggingface.co/Meta-Llama-3.1-8B-Instruct-bnb-4bit) for details.

## Acknowledgments
- Thanks to Hugging Face for the `transformers` library.
- Thanks to the `unsloth` team for their memory-efficient fine-tuning tools.
- Thanks to the creators of the `gprmax_train` dataset.
- Thanks to `ggerganov` for `llama.cpp` and the GGUF format.
```