File size: 2,283 Bytes
24310d8 30e743b fe24383 7b0adef fe24383 7b0adef c82b290 7b0adef c82b290 7b0adef c82b290 7b0adef c82b290 7b0adef c82b290 7b0adef c82b290 7b0adef c82b290 7b0adef c82b290 7b0adef c82b290 7b0adef c82b290 66d2627 7b0adef 66d2627 7b0adef 66d2627 7b0adef 66d2627 7b0adef 66d2627 7b0adef | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 | ---
tags:
- gguf
- llama.cpp
- unsloth
license: mit
datasets:
- b-mc2/sql-create-context
language:
- en
metrics:
- accuracy
base_model:
- microsoft/Phi-3-mini-4k-instruct
---
This is a specialized **Text-to-SQL** model fine-tuned from the **Microsoft Phi-3-mini-4k-instruct** architecture. It has been optimized using **Unsloth** to provide high-accuracy SQL generation while remaining lightweight enough to run on consumer hardware.
## Key Features
- **Architecture:** Phi-3-mini (3.8B parameters)
- **Quantization:** Q4_K_M GGUF & Q5_K_M
- **Training Technique:** Fine-tuned using Lora with [Unsloth](https://github.com/unslothai/unsloth).
- **Format:** GGUF (Ready for Ollama, LM Studio, and llama.cpp)
- **phi-3-mini-4k-instruct.Q4_K_M.gguf**
- **phi-3-mini-4k-instruct.Q5_K.gguf**
## Usage Instructions
### Ollama (Recommended)
To deploy locally:
1. Download the `.gguf` file (Q4 or Q5).
2. Create the Modelfile with the following instructions
```Dockerfile
FROM ./phi-3-mini-4k-instruct.Q4_K_M.gguf
TEMPLATE """<s><|user|>
Schema: {{ .System }}
Question: {{ .Prompt }}<|end|>
<|assistant|>
"""
# Parameters for SQL stability
PARAMETER stop "<|end|>"
PARAMETER stop "<s>"
PARAMETER stop "</s>"
PARAMETER temperature 0.0
```
3. Run ```ollama create phi3-sql-expert -f Modelfile```
5. Run ```ollama run phi3-sql-expert "schema: CREATE TABLE table_name_7 (nba_draft VARCHAR, school VARCHAR) question: What was the NBA draft status for Northeast High School?"```
6. The answer should be ```SELECT nba_draft FROM table_name_7 WHERE school = "Northeast"```
## Evaluation Data
The model was fine-tuned on the [sql-create-context dataset](https://huggingface.co/datasets/b-mc2/sql-create-context), focusing on:
- Mapping natural language to SQL queries with SELECT, WHERE, and JOIN statements.
- Understanding table schemas provided in the prompt.
- Maintaining strict SQL syntax in the response.
## Recommended Settings
Temperature: 0.0 or 0.1 (SQL requires deterministic output).
Stop Tokens: Ensure <|end|> is set as a stop sequence to prevent "infinite looping" generation.
Context Window: 2048 tokens.
**Model Developer**: [msquared](https://github.com/mrcmilano)
Base Model: [Phi-3-mini-4k-instruct](https://huggingface.co/microsoft/Phi-3-mini-4k-instruct) |