README.md

---
base_model: unsloth/granite-4.0-350m-unsloth-bnb-4bit
library_name: peft
pipeline_tag: text-generation
tags:
- base_model:adapter:unsloth/granite-4.0-350m-unsloth-bnb-4bit
- lora
- sft
- transformers
- trl
- unsloth
- text-to-sql
license: apache-2.0
datasets:
- b-mc2/sql-create-context
---

# Model Card for granite_sql (checkpoint-6000)

LoRA adapter fine-tuning IBM Granite 4.0 350M for text-to-SQL generation: given a
`CREATE TABLE` schema and a natural-language question, generate the corresponding SQL query.

## Model Details

### Model Description

This checkpoint (step 6000 of 6432, epoch 2.80/3) is the best-performing checkpoint
of the run — lowest eval loss among all saved checkpoints (see Results).

- **Model type:** LoRA adapter (PEFT) on a causal LM
- **Language(s):** SQL (generation), English (instructions)
- **License:** Apache 2.0 (inherited from base model)
- **Finetuned from model:** [unsloth/granite-4.0-350m-unsloth-bnb-4bit](https://huggingface.co/unsloth/granite-4.0-350m-unsloth-bnb-4bit) (IBM Granite 4.0, 350M, GraniteMoeHybrid)

## How to Get Started with the Model

```python
from unsloth import FastLanguageModel

model, tokenizer = FastLanguageModel.from_pretrained(
    model_name="/workspace/outputs/granite_sql_train/checkpoint-6000",
    load_in_4bit=False,
    device_map="cuda:0",
)
FastLanguageModel.for_inference(model)

instruction = """Generate ONLY the SQL query for the following database.

Do not explain your answer.
Do not include markdown.
Do not include any additional text.

Schema:
{schema}

Question:
{question}

SQL:"""

messages = [{"role": "user", "content": [{"type": "text", "text": instruction.format(schema=schema, question=question)}]}]
input_text = tokenizer.apply_chat_template(messages, add_generation_prompt=True, tokenize=False)
inputs = tokenizer(input_text, add_special_tokens=False, return_tensors="pt").to("cuda")
output = model.generate(**inputs, max_new_tokens=256, use_cache=True, temperature=0.7, top_p=0.8, top_k=20)
```

## Training Details

### Training Data

[b-mc2/sql-create-context](https://huggingface.co/datasets/b-mc2/sql-create-context) —
schema + natural-language-question + SQL-answer triples. Split via
`train_test_split(test_size=10000, seed=42, shuffle=True)`: remaining rows for train, 10,000 held out for eval.

### Training Procedure

LoRA (r=16, alpha=32, dropout=0, bias=none, no rslora) applied to
`q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj, shared_mlp.input_linear, shared_mlp.output_linear`,
trained with Unsloth + TRL SFTTrainer.

#### Training Hyperparameters

- **Training regime:** bf16 mixed precision
- Epochs: 3 | Per-device train batch size: 32 | Per-device eval batch size: 64 | Grad accumulation: 1
- Learning rate: 2e-4, linear schedule, 50 warmup steps
- Optimizer: adamw_8bit, weight decay 0.001, max grad norm 1.0
- Seed: 3407 | Eval every 200 steps | Save every 500 steps

#### Speeds, Sizes, Times

Full run: 6432 steps / 3 epochs, train_runtime ≈ 2567s. This adapter checkpoint: ~26.7MB (`adapter_model.safetensors`).

## Evaluation

### Testing Data & Metrics

10,000-row held-out split of `b-mc2/sql-create-context` (see Training Data), evaluated by SFT eval loss (cross-entropy) every 200 steps.

### Results

| Checkpoint | Step | Eval loss |
|---|---|---|
| checkpoint-4000 | 4000 | 0.02830 |
| checkpoint-5000 | 5000 | 0.02811 |
| **checkpoint-6000** | **6000** | **0.02673 (best saved)** |
| checkpoint-6432 (final) | 6432 | ~0.02746 (nearest eval at step 6400) |

Lowest eval_loss observed during training was 0.02630 at step 5200, but no checkpoint was
saved at that exact step (checkpoints every 500 steps, eval every 200), so checkpoint-6000
is the closest usable minimum. `load_best_model_at_end` was not enabled, so this checkpoint
was selected manually by comparing `eval_loss` across saved checkpoints.

## Compute Infrastructure

### Hardware

- 1× NVIDIA A100 40GB

### Software

- Unsloth
- Transformers
- PyTorch

---

## Model Card Authors

- Yian

## Contact

- https://github.com/Siruenyian