Sam-2610 commited on
Commit
82d53f7
·
verified ·
1 Parent(s): 2410f62

Updated Readme File

Browse files
Files changed (1) hide show
  1. README.md +83 -202
README.md CHANGED
@@ -1,209 +1,90 @@
1
  ---
2
- base_model: unsloth/Llama-3.2-3B-Instruct
 
3
  library_name: peft
4
  pipeline_tag: text-generation
 
 
 
 
5
  tags:
6
- - base_model:adapter:unsloth/Llama-3.2-3B-Instruct
 
 
7
  - lora
8
- - sft
9
- - transformers
10
- - trl
11
  ---
12
 
13
- # Model Card for Model ID
14
-
15
- <!-- Provide a quick summary of what the model is/does. -->
16
-
17
-
18
-
19
- ## Model Details
20
-
21
- ### Model Description
22
-
23
- <!-- Provide a longer summary of what this model is. -->
24
-
25
-
26
-
27
- - **Developed by:** [More Information Needed]
28
- - **Funded by [optional]:** [More Information Needed]
29
- - **Shared by [optional]:** [More Information Needed]
30
- - **Model type:** [More Information Needed]
31
- - **Language(s) (NLP):** [More Information Needed]
32
- - **License:** [More Information Needed]
33
- - **Finetuned from model [optional]:** [More Information Needed]
34
-
35
- ### Model Sources [optional]
36
-
37
- <!-- Provide the basic links for the model. -->
38
-
39
- - **Repository:** [More Information Needed]
40
- - **Paper [optional]:** [More Information Needed]
41
- - **Demo [optional]:** [More Information Needed]
42
-
43
- ## Uses
44
-
45
- <!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
46
-
47
- ### Direct Use
48
-
49
- <!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
50
-
51
- [More Information Needed]
52
-
53
- ### Downstream Use [optional]
54
-
55
- <!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
56
-
57
- [More Information Needed]
58
-
59
- ### Out-of-Scope Use
60
-
61
- <!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
62
-
63
- [More Information Needed]
64
-
65
- ## Bias, Risks, and Limitations
66
-
67
- <!-- This section is meant to convey both technical and sociotechnical limitations. -->
68
-
69
- [More Information Needed]
70
-
71
- ### Recommendations
72
-
73
- <!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
74
-
75
- Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
76
-
77
- ## How to Get Started with the Model
78
-
79
- Use the code below to get started with the model.
80
-
81
- [More Information Needed]
82
-
83
- ## Training Details
84
-
85
- ### Training Data
86
-
87
- <!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
88
-
89
- [More Information Needed]
90
-
91
- ### Training Procedure
92
-
93
- <!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
94
-
95
- #### Preprocessing [optional]
96
-
97
- [More Information Needed]
98
-
99
-
100
- #### Training Hyperparameters
101
-
102
- - **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
103
-
104
- #### Speeds, Sizes, Times [optional]
105
-
106
- <!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
107
-
108
- [More Information Needed]
109
-
110
- ## Evaluation
111
-
112
- <!-- This section describes the evaluation protocols and provides the results. -->
113
-
114
- ### Testing Data, Factors & Metrics
115
-
116
- #### Testing Data
117
-
118
- <!-- This should link to a Dataset Card if possible. -->
119
-
120
- [More Information Needed]
121
-
122
- #### Factors
123
-
124
- <!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
125
-
126
- [More Information Needed]
127
-
128
- #### Metrics
129
-
130
- <!-- These are the evaluation metrics being used, ideally with a description of why. -->
131
-
132
- [More Information Needed]
133
-
134
- ### Results
135
-
136
- [More Information Needed]
137
-
138
- #### Summary
139
-
140
-
141
-
142
- ## Model Examination [optional]
143
-
144
- <!-- Relevant interpretability work for the model goes here -->
145
-
146
- [More Information Needed]
147
-
148
- ## Environmental Impact
149
-
150
- <!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
151
-
152
- Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
153
-
154
- - **Hardware Type:** [More Information Needed]
155
- - **Hours used:** [More Information Needed]
156
- - **Cloud Provider:** [More Information Needed]
157
- - **Compute Region:** [More Information Needed]
158
- - **Carbon Emitted:** [More Information Needed]
159
-
160
- ## Technical Specifications [optional]
161
-
162
- ### Model Architecture and Objective
163
-
164
- [More Information Needed]
165
-
166
- ### Compute Infrastructure
167
-
168
- [More Information Needed]
169
-
170
- #### Hardware
171
-
172
- [More Information Needed]
173
-
174
- #### Software
175
-
176
- [More Information Needed]
177
-
178
- ## Citation [optional]
179
-
180
- <!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
181
-
182
- **BibTeX:**
183
-
184
- [More Information Needed]
185
-
186
- **APA:**
187
-
188
- [More Information Needed]
189
-
190
- ## Glossary [optional]
191
-
192
- <!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
193
-
194
- [More Information Needed]
195
-
196
- ## More Information [optional]
197
-
198
- [More Information Needed]
199
-
200
- ## Model Card Authors [optional]
201
-
202
- [More Information Needed]
203
-
204
- ## Model Card Contact
205
-
206
- [More Information Needed]
207
- ### Framework versions
208
-
209
- - PEFT 0.21.2
 
1
  ---
2
+ license: llama3.2
3
+ base_model: meta-llama/Llama-3.2-3B-Instruct
4
  library_name: peft
5
  pipeline_tag: text-generation
6
+ language:
7
+ - en
8
+ datasets:
9
+ - gretelai/synthetic_text_to_sql
10
  tags:
11
+ - text-to-sql
12
+ - sql
13
+ - qlora
14
  - lora
15
+ - peft
16
+ - llama
17
+ - llama-3.2
18
  ---
19
 
20
+ # Llama-3.2-3B-Text2SQL-QLoRA
21
+
22
+ **Built with Llama.** A LoRA adapter for Llama-3.2-3B-Instruct that turns a question and a database schema into a SQLite query. It was trained with QLoRA (4-bit) on 600 examples in about 16 minutes on a free Colab T4 GPU.
23
+
24
+ - Code, training scripts and evaluation: https://github.com/Sam-2610/LocalText2SQL
25
+ - This repository contains only the adapter (about 97 MB). You need the base model to use it.
26
+
27
+ ## Results
28
+ Scored by **execution accuracy**: the correct and the predicted query are both run on a SQLite database built from the schema and sample rows, and the returned rows are compared. Test set: 150 questions from the dataset's test split (50 easy, 50 medium, 50 hard), none of which were used for training.
29
+
30
+ | | Untuned model | This adapter |
31
+ |---|---|---|
32
+ | **Execution accuracy (all)** | 53% (79/150) | **69% (104/150)** |
33
+ | Easy | 78% | 86% |
34
+ | Medium | 50% | 78% |
35
+ | Hard | 30% | 44% |
36
+ | Valid SQL | 87% | 99% |
37
+
38
+ Of the questions where the two models differ, the adapter fixed 28 and broke 3. Reloading this published adapter in a fresh session reproduced the 69% result exactly.
39
+
40
+ ## How to use
41
+ Tested with transformers 5.22.2, peft 0.21.2 and trl 1.17.1 on a Colab T4. 4-bit loading needs an NVIDIA GPU.
42
+
43
+ ```python
44
+ import torch
45
+ from peft import PeftModel
46
+ from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
47
+
48
+ adapter = "Sam-2610/Llama-3.2-3B-Text2SQL-QLoRA"
49
+ base_id = "unsloth/Llama-3.2-3B-Instruct" # ungated mirror; meta-llama/Llama-3.2-3B-Instruct also works
50
+
51
+ bnb = BitsAndBytesConfig(load_in_4bit=True, bnb_4bit_quant_type="nf4",
52
+ bnb_4bit_compute_dtype=torch.float16, bnb_4bit_use_double_quant=True)
53
+ tokenizer = AutoTokenizer.from_pretrained(adapter)
54
+ model = AutoModelForCausalLM.from_pretrained(base_id, quantization_config=bnb,
55
+ device_map="auto", dtype=torch.float16)
56
+ model = PeftModel.from_pretrained(model, adapter)
57
+ model.eval()
58
+
59
+ schema = ("CREATE TABLE employees (id INT, name TEXT, department TEXT, salary REAL); "
60
+ "INSERT INTO employees VALUES (1,'Asha','Sales',50000),(2,'Ravi','Sales',60000),(3,'Meera','IT',70000);")
61
+ question = "Who earns the most?"
62
+
63
+ messages = [
64
+ {"role": "system", "content": "You are an expert in SQLite. Given a database schema, write one SQLite query "
65
+ "that answers the question. Output only the SQL query."},
66
+ {"role": "user", "content": f"Schema:\n{schema}\n\nQuestion: {question}"},
67
+ ]
68
+ prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
69
+ inputs = tokenizer(prompt, return_tensors="pt", add_special_tokens=False).to(model.device)
70
+ with torch.no_grad():
71
+ out = model.generate(**inputs, max_new_tokens=200, do_sample=False)
72
+ print(tokenizer.decode(out[0, inputs["input_ids"].shape[1]:], skip_special_tokens=True))
73
+ ```
74
+ Use the same system prompt and message format as above, because the adapter was trained with it.
75
+
76
+ ## Training
77
+ - **Data:** 600 examples (200 easy, 200 medium, 200 hard) from [gretelai/synthetic_text_to_sql](https://huggingface.co/datasets/gretelai/synthetic_text_to_sql). Every example runs on SQLite and returns real rows. Easy is basic SQL, medium is aggregation and single joins, hard is subqueries, window functions, multiple joins and set operations.
78
+ - **Method:** QLoRA. 4-bit NF4 base model with double quantization, LoRA r=16, alpha 32, dropout 0.05 on the attention and MLP projection layers (24.3M trainable parameters, 0.75%). Only the SQL answer contributes to the loss.
79
+ - **Settings:** 3 epochs, learning rate 2e-4 with a cosine schedule, effective batch size 16, fp16, max length 1024.
80
+ - **Run:** 15 min 32 s on a free Colab T4. Training loss fell from 0.51 to 0.08.
81
+
82
+ ## Limitations
83
+ - The test questions come from the same synthetic dataset as the training data, so the results show the model learned this style of question. They do not show it is a general SQL expert. Public benchmarks such as Spider or BIRD would be a harder test.
84
+ - Hard questions are still only 44% correct, and each difficulty group has only 50 test questions, so the per-group numbers are rough.
85
+ - Trained and tested for SQLite and short schemas. It can produce wrong or invalid queries, so check generated SQL before running it, and never run it with write access on data you care about.
86
+ - Not evaluated for any use outside this demo.
87
+
88
+ ## License and credits
89
+ - Built with Llama. Llama 3.2 is licensed under the Llama 3.2 Community License, Copyright (c) Meta Platforms, Inc. All Rights Reserved. Use of this adapter is subject to that license and Meta's Acceptable Use Policy.
90
+ - Training data: gretelai/synthetic_text_to_sql by Gretel.ai, licensed under Apache 2.0.