README.md

---
tags:
- gguf
- llama.cpp
- unsloth
license: mit
datasets:
- b-mc2/sql-create-context
language:
- en
metrics:
- accuracy
base_model:
- microsoft/Phi-3-mini-4k-instruct
---
This is a specialized **Text-to-SQL** model fine-tuned from the **Microsoft Phi-3-mini-4k-instruct** architecture. It has been optimized using **Unsloth** to provide high-accuracy SQL generation while remaining lightweight enough to run on consumer hardware.

## Key Features
- **Architecture:** Phi-3-mini (3.8B parameters)
- **Quantization:** Q4_K_M GGUF & Q5_K_M
- **Training Technique:** Fine-tuned using Lora with [Unsloth](https://github.com/unslothai/unsloth).
- **Format:** GGUF (Ready for Ollama, LM Studio, and llama.cpp)
  - **phi-3-mini-4k-instruct.Q4_K_M.gguf** 
  - **phi-3-mini-4k-instruct.Q5_K.gguf**

## Usage Instructions

### Ollama (Recommended)
To deploy locally:

1. Download the `.gguf` file (Q4 or Q5).

2. Create the Modelfile with the following instructions

```Dockerfile
FROM ./phi-3-mini-4k-instruct.Q4_K_M.gguf

TEMPLATE """<s><|user|>
Schema: {{ .System }}
Question: {{ .Prompt }}<|end|>
<|assistant|>
"""

# Parameters for SQL stability
PARAMETER stop "<|end|>"
PARAMETER stop "<s>"
PARAMETER stop "</s>"
PARAMETER temperature 0.0
```

3. Run ```ollama create phi3-sql-expert -f Modelfile```

5. Run ```ollama run phi3-sql-expert "schema: CREATE TABLE table_name_7 (nba_draft VARCHAR, school VARCHAR) question: What was the NBA draft status for Northeast High School?"```

6. The answer should be ```SELECT nba_draft FROM table_name_7 WHERE school = "Northeast"```

## Evaluation Data
The model was fine-tuned on the [sql-create-context dataset](https://huggingface.co/datasets/b-mc2/sql-create-context), focusing on:

- Mapping natural language to SQL queries with SELECT, WHERE, and JOIN statements.
- Understanding table schemas provided in the prompt.
- Maintaining strict SQL syntax in the response.

## Recommended Settings
Temperature: 0.0 or 0.1 (SQL requires deterministic output).

Stop Tokens: Ensure <|end|> is set as a stop sequence to prevent "infinite looping" generation.

Context Window: 2048 tokens.

**Model Developer**: [msquared](https://github.com/mrcmilano)

Base Model: [Phi-3-mini-4k-instruct](https://huggingface.co/microsoft/Phi-3-mini-4k-instruct)