| ---
|
| base_model: meta-llama/Llama-3.1-8B-Instruct
|
| library_name: peft
|
| license: llama3.1
|
| pipeline_tag: text-generation
|
| tags:
|
| - lora
|
| - peft
|
| - text-to-sql
|
| - tinyforge
|
| - low-vram
|
| ---
|
|
|
| # tinyforge: Llama-3.1-8B-Instruct LoRA for text-to-SQL, trained on a 4 GB GPU
|
|
|
| A 14 MB LoRA adapter (r=16) trained with [tinyforge](https://github.com/rajratangm/tinyforge) on a **4 GB RTX 3050 Ti laptop
|
| GPU** using layer streaming: 200 steps in 9.5 minutes. Training data: 1,800 question/SQL pairs over six public seaborn
|
| tables (titanic, mpg, tips, car_crashes, planets, healthexp), each answer verified by executing the SQL.
|
| Use under the Llama 3.1 Community License.
|
|
|
| ## Results (read the caveats)
|
|
|
| 100 questions about a table the model never saw (penguins), served with llama.cpp (adapter f16 on a Q4_K_M base), greedy decoding.
|
|
|
| | Model | Execution accuracy |
|
| |---|---|
|
| | This adapter | **100%** |
|
| | Base told to "reply with exactly one SQLite query" | 93% |
|
|
|
| Caveats, honestly:
|
| - 100% is a ceiling: the instructed base is already at 93%, so this test cannot show how much headroom tuning adds.
|
| All 7 base misses are one malformed `MAX`/`MIN` call, so the gain is **consistency, not new capability**.
|
| - Questions use the same 12 templates as training, on a new schema: it measures transfer to new tables, not new kinds of questions.
|
| - On harder question shapes the same 8B model scored 98% vs 97% (no meaningful gain over a well-prompted base).
|
| - n=100, one run, no confidence interval. Raw numbers: `eval_results.json`.
|
|
|
| ## Use
|
|
|
| ```python
|
| from transformers import AutoModelForCausalLM
|
| from peft import PeftModel
|
| base = AutoModelForCausalLM.from_pretrained("meta-llama/Llama-3.1-8B-Instruct", device_map="auto")
|
| model = PeftModel.from_pretrained(base, "RajGMore/tinyforge-llama3.1-8b-sql-lora")
|
| ```
|
|
|
