Instructions to use Sam-2610/Llama-3.2-3B-Text2SQL-QLoRA with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- PEFT
How to use Sam-2610/Llama-3.2-3B-Text2SQL-QLoRA with PEFT:
from peft import PeftModel from transformers import AutoModelForCausalLM base_model = AutoModelForCausalLM.from_pretrained("unsloth/Llama-3.2-3B-Instruct") model = PeftModel.from_pretrained(base_model, "Sam-2610/Llama-3.2-3B-Text2SQL-QLoRA") - Notebooks
- Google Colab
- Kaggle
Updated Readme File
Browse files
README.md
CHANGED
|
@@ -1,209 +1,90 @@
|
|
| 1 |
---
|
| 2 |
-
|
|
|
|
| 3 |
library_name: peft
|
| 4 |
pipeline_tag: text-generation
|
|
|
|
|
|
|
|
|
|
|
|
|
| 5 |
tags:
|
| 6 |
-
-
|
|
|
|
|
|
|
| 7 |
- lora
|
| 8 |
-
-
|
| 9 |
-
-
|
| 10 |
-
-
|
| 11 |
---
|
| 12 |
|
| 13 |
-
#
|
| 14 |
-
|
| 15 |
-
|
| 16 |
-
|
| 17 |
-
|
| 18 |
-
|
| 19 |
-
|
| 20 |
-
|
| 21 |
-
|
| 22 |
-
|
| 23 |
-
|
| 24 |
-
|
| 25 |
-
|
| 26 |
-
|
| 27 |
-
|
| 28 |
-
|
| 29 |
-
|
| 30 |
-
|
| 31 |
-
|
| 32 |
-
|
| 33 |
-
|
| 34 |
-
|
| 35 |
-
|
| 36 |
-
|
| 37 |
-
|
| 38 |
-
|
| 39 |
-
|
| 40 |
-
|
| 41 |
-
|
| 42 |
-
|
| 43 |
-
|
| 44 |
-
|
| 45 |
-
|
| 46 |
-
|
| 47 |
-
|
| 48 |
-
|
| 49 |
-
|
| 50 |
-
|
| 51 |
-
|
| 52 |
-
|
| 53 |
-
|
| 54 |
-
|
| 55 |
-
|
| 56 |
-
|
| 57 |
-
|
| 58 |
-
|
| 59 |
-
|
| 60 |
-
|
| 61 |
-
|
| 62 |
-
|
| 63 |
-
|
| 64 |
-
|
| 65 |
-
|
| 66 |
-
|
| 67 |
-
|
| 68 |
-
|
| 69 |
-
|
| 70 |
-
|
| 71 |
-
|
| 72 |
-
|
| 73 |
-
|
| 74 |
-
|
| 75 |
-
|
| 76 |
-
|
| 77 |
-
|
| 78 |
-
|
| 79 |
-
|
| 80 |
-
|
| 81 |
-
|
| 82 |
-
|
| 83 |
-
|
| 84 |
-
|
| 85 |
-
### Training Data
|
| 86 |
-
|
| 87 |
-
<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
|
| 88 |
-
|
| 89 |
-
[More Information Needed]
|
| 90 |
-
|
| 91 |
-
### Training Procedure
|
| 92 |
-
|
| 93 |
-
<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
|
| 94 |
-
|
| 95 |
-
#### Preprocessing [optional]
|
| 96 |
-
|
| 97 |
-
[More Information Needed]
|
| 98 |
-
|
| 99 |
-
|
| 100 |
-
#### Training Hyperparameters
|
| 101 |
-
|
| 102 |
-
- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
|
| 103 |
-
|
| 104 |
-
#### Speeds, Sizes, Times [optional]
|
| 105 |
-
|
| 106 |
-
<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
|
| 107 |
-
|
| 108 |
-
[More Information Needed]
|
| 109 |
-
|
| 110 |
-
## Evaluation
|
| 111 |
-
|
| 112 |
-
<!-- This section describes the evaluation protocols and provides the results. -->
|
| 113 |
-
|
| 114 |
-
### Testing Data, Factors & Metrics
|
| 115 |
-
|
| 116 |
-
#### Testing Data
|
| 117 |
-
|
| 118 |
-
<!-- This should link to a Dataset Card if possible. -->
|
| 119 |
-
|
| 120 |
-
[More Information Needed]
|
| 121 |
-
|
| 122 |
-
#### Factors
|
| 123 |
-
|
| 124 |
-
<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
|
| 125 |
-
|
| 126 |
-
[More Information Needed]
|
| 127 |
-
|
| 128 |
-
#### Metrics
|
| 129 |
-
|
| 130 |
-
<!-- These are the evaluation metrics being used, ideally with a description of why. -->
|
| 131 |
-
|
| 132 |
-
[More Information Needed]
|
| 133 |
-
|
| 134 |
-
### Results
|
| 135 |
-
|
| 136 |
-
[More Information Needed]
|
| 137 |
-
|
| 138 |
-
#### Summary
|
| 139 |
-
|
| 140 |
-
|
| 141 |
-
|
| 142 |
-
## Model Examination [optional]
|
| 143 |
-
|
| 144 |
-
<!-- Relevant interpretability work for the model goes here -->
|
| 145 |
-
|
| 146 |
-
[More Information Needed]
|
| 147 |
-
|
| 148 |
-
## Environmental Impact
|
| 149 |
-
|
| 150 |
-
<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
|
| 151 |
-
|
| 152 |
-
Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
|
| 153 |
-
|
| 154 |
-
- **Hardware Type:** [More Information Needed]
|
| 155 |
-
- **Hours used:** [More Information Needed]
|
| 156 |
-
- **Cloud Provider:** [More Information Needed]
|
| 157 |
-
- **Compute Region:** [More Information Needed]
|
| 158 |
-
- **Carbon Emitted:** [More Information Needed]
|
| 159 |
-
|
| 160 |
-
## Technical Specifications [optional]
|
| 161 |
-
|
| 162 |
-
### Model Architecture and Objective
|
| 163 |
-
|
| 164 |
-
[More Information Needed]
|
| 165 |
-
|
| 166 |
-
### Compute Infrastructure
|
| 167 |
-
|
| 168 |
-
[More Information Needed]
|
| 169 |
-
|
| 170 |
-
#### Hardware
|
| 171 |
-
|
| 172 |
-
[More Information Needed]
|
| 173 |
-
|
| 174 |
-
#### Software
|
| 175 |
-
|
| 176 |
-
[More Information Needed]
|
| 177 |
-
|
| 178 |
-
## Citation [optional]
|
| 179 |
-
|
| 180 |
-
<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
|
| 181 |
-
|
| 182 |
-
**BibTeX:**
|
| 183 |
-
|
| 184 |
-
[More Information Needed]
|
| 185 |
-
|
| 186 |
-
**APA:**
|
| 187 |
-
|
| 188 |
-
[More Information Needed]
|
| 189 |
-
|
| 190 |
-
## Glossary [optional]
|
| 191 |
-
|
| 192 |
-
<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
|
| 193 |
-
|
| 194 |
-
[More Information Needed]
|
| 195 |
-
|
| 196 |
-
## More Information [optional]
|
| 197 |
-
|
| 198 |
-
[More Information Needed]
|
| 199 |
-
|
| 200 |
-
## Model Card Authors [optional]
|
| 201 |
-
|
| 202 |
-
[More Information Needed]
|
| 203 |
-
|
| 204 |
-
## Model Card Contact
|
| 205 |
-
|
| 206 |
-
[More Information Needed]
|
| 207 |
-
### Framework versions
|
| 208 |
-
|
| 209 |
-
- PEFT 0.21.2
|
|
|
|
| 1 |
---
|
| 2 |
+
license: llama3.2
|
| 3 |
+
base_model: meta-llama/Llama-3.2-3B-Instruct
|
| 4 |
library_name: peft
|
| 5 |
pipeline_tag: text-generation
|
| 6 |
+
language:
|
| 7 |
+
- en
|
| 8 |
+
datasets:
|
| 9 |
+
- gretelai/synthetic_text_to_sql
|
| 10 |
tags:
|
| 11 |
+
- text-to-sql
|
| 12 |
+
- sql
|
| 13 |
+
- qlora
|
| 14 |
- lora
|
| 15 |
+
- peft
|
| 16 |
+
- llama
|
| 17 |
+
- llama-3.2
|
| 18 |
---
|
| 19 |
|
| 20 |
+
# Llama-3.2-3B-Text2SQL-QLoRA
|
| 21 |
+
|
| 22 |
+
**Built with Llama.** A LoRA adapter for Llama-3.2-3B-Instruct that turns a question and a database schema into a SQLite query. It was trained with QLoRA (4-bit) on 600 examples in about 16 minutes on a free Colab T4 GPU.
|
| 23 |
+
|
| 24 |
+
- Code, training scripts and evaluation: https://github.com/Sam-2610/LocalText2SQL
|
| 25 |
+
- This repository contains only the adapter (about 97 MB). You need the base model to use it.
|
| 26 |
+
|
| 27 |
+
## Results
|
| 28 |
+
Scored by **execution accuracy**: the correct and the predicted query are both run on a SQLite database built from the schema and sample rows, and the returned rows are compared. Test set: 150 questions from the dataset's test split (50 easy, 50 medium, 50 hard), none of which were used for training.
|
| 29 |
+
|
| 30 |
+
| | Untuned model | This adapter |
|
| 31 |
+
|---|---|---|
|
| 32 |
+
| **Execution accuracy (all)** | 53% (79/150) | **69% (104/150)** |
|
| 33 |
+
| Easy | 78% | 86% |
|
| 34 |
+
| Medium | 50% | 78% |
|
| 35 |
+
| Hard | 30% | 44% |
|
| 36 |
+
| Valid SQL | 87% | 99% |
|
| 37 |
+
|
| 38 |
+
Of the questions where the two models differ, the adapter fixed 28 and broke 3. Reloading this published adapter in a fresh session reproduced the 69% result exactly.
|
| 39 |
+
|
| 40 |
+
## How to use
|
| 41 |
+
Tested with transformers 5.22.2, peft 0.21.2 and trl 1.17.1 on a Colab T4. 4-bit loading needs an NVIDIA GPU.
|
| 42 |
+
|
| 43 |
+
```python
|
| 44 |
+
import torch
|
| 45 |
+
from peft import PeftModel
|
| 46 |
+
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
|
| 47 |
+
|
| 48 |
+
adapter = "Sam-2610/Llama-3.2-3B-Text2SQL-QLoRA"
|
| 49 |
+
base_id = "unsloth/Llama-3.2-3B-Instruct" # ungated mirror; meta-llama/Llama-3.2-3B-Instruct also works
|
| 50 |
+
|
| 51 |
+
bnb = BitsAndBytesConfig(load_in_4bit=True, bnb_4bit_quant_type="nf4",
|
| 52 |
+
bnb_4bit_compute_dtype=torch.float16, bnb_4bit_use_double_quant=True)
|
| 53 |
+
tokenizer = AutoTokenizer.from_pretrained(adapter)
|
| 54 |
+
model = AutoModelForCausalLM.from_pretrained(base_id, quantization_config=bnb,
|
| 55 |
+
device_map="auto", dtype=torch.float16)
|
| 56 |
+
model = PeftModel.from_pretrained(model, adapter)
|
| 57 |
+
model.eval()
|
| 58 |
+
|
| 59 |
+
schema = ("CREATE TABLE employees (id INT, name TEXT, department TEXT, salary REAL); "
|
| 60 |
+
"INSERT INTO employees VALUES (1,'Asha','Sales',50000),(2,'Ravi','Sales',60000),(3,'Meera','IT',70000);")
|
| 61 |
+
question = "Who earns the most?"
|
| 62 |
+
|
| 63 |
+
messages = [
|
| 64 |
+
{"role": "system", "content": "You are an expert in SQLite. Given a database schema, write one SQLite query "
|
| 65 |
+
"that answers the question. Output only the SQL query."},
|
| 66 |
+
{"role": "user", "content": f"Schema:\n{schema}\n\nQuestion: {question}"},
|
| 67 |
+
]
|
| 68 |
+
prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
|
| 69 |
+
inputs = tokenizer(prompt, return_tensors="pt", add_special_tokens=False).to(model.device)
|
| 70 |
+
with torch.no_grad():
|
| 71 |
+
out = model.generate(**inputs, max_new_tokens=200, do_sample=False)
|
| 72 |
+
print(tokenizer.decode(out[0, inputs["input_ids"].shape[1]:], skip_special_tokens=True))
|
| 73 |
+
```
|
| 74 |
+
Use the same system prompt and message format as above, because the adapter was trained with it.
|
| 75 |
+
|
| 76 |
+
## Training
|
| 77 |
+
- **Data:** 600 examples (200 easy, 200 medium, 200 hard) from [gretelai/synthetic_text_to_sql](https://huggingface.co/datasets/gretelai/synthetic_text_to_sql). Every example runs on SQLite and returns real rows. Easy is basic SQL, medium is aggregation and single joins, hard is subqueries, window functions, multiple joins and set operations.
|
| 78 |
+
- **Method:** QLoRA. 4-bit NF4 base model with double quantization, LoRA r=16, alpha 32, dropout 0.05 on the attention and MLP projection layers (24.3M trainable parameters, 0.75%). Only the SQL answer contributes to the loss.
|
| 79 |
+
- **Settings:** 3 epochs, learning rate 2e-4 with a cosine schedule, effective batch size 16, fp16, max length 1024.
|
| 80 |
+
- **Run:** 15 min 32 s on a free Colab T4. Training loss fell from 0.51 to 0.08.
|
| 81 |
+
|
| 82 |
+
## Limitations
|
| 83 |
+
- The test questions come from the same synthetic dataset as the training data, so the results show the model learned this style of question. They do not show it is a general SQL expert. Public benchmarks such as Spider or BIRD would be a harder test.
|
| 84 |
+
- Hard questions are still only 44% correct, and each difficulty group has only 50 test questions, so the per-group numbers are rough.
|
| 85 |
+
- Trained and tested for SQLite and short schemas. It can produce wrong or invalid queries, so check generated SQL before running it, and never run it with write access on data you care about.
|
| 86 |
+
- Not evaluated for any use outside this demo.
|
| 87 |
+
|
| 88 |
+
## License and credits
|
| 89 |
+
- Built with Llama. Llama 3.2 is licensed under the Llama 3.2 Community License, Copyright (c) Meta Platforms, Inc. All Rights Reserved. Use of this adapter is subject to that license and Meta's Acceptable Use Policy.
|
| 90 |
+
- Training data: gretelai/synthetic_text_to_sql by Gretel.ai, licensed under Apache 2.0.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|