---
license: cc-by-nc-4.0
base_model: CohereLabs/tiny-aya-global
library_name: peft
pipeline_tag: text-generation
language:
- en
tags:
- text-to-sql
- agent
- tool-use
- lora
- qlora
- agenttune
- auditkit
datasets:
- satish9177/tiny-aya-sql-agent-data
- trl-lab/SQaLe-2-text-to-SQL-Queries
---

# Tiny Aya SQL agent (A1500, supervised fine-tuning only)

A LoRA adapter for Cohere's **Tiny Aya Global**, trained with QLoRA on 1,500 verified multi-turn SQL
agent episodes through Lexsi **AgentTune**.
- **Its role:** the plain fine-tuning model, the starting point for the submitted model
  **[B](https://huggingface.co/satish9177/tiny-aya-sql-agent)**. B adds one round of self-training.
- **Which to use:** use B for the submission's results, and this one to compare plain fine-tuning with B.

- Write-up: [Teaching Tiny Aya to query databases (HF Article)](https://huggingface.co/blog/satish9177/teaching-tiny-aya-to-query-databases-what-fine-tun)
- Code, plans and full results: [github.com/satish9177/aya-db-agent](https://github.com/satish9177/aya-db-agent)
- Data, trajectories and logs: [satish9177/tiny-aya-sql-agent-data](https://huggingface.co/datasets/satish9177/tiny-aya-sql-agent-data)

## Results (strict)

| Set | Base (untuned) | **A1500 (this model)** | B |
|---|---:|---:|---:|
| SQaLe dev (95) | 2 | **37** | 40 |
| SQaLe final (144, unseen databases) | 2.8% | **31.3%** | 30.6% |
| BIRD Mini-Dev (493) | 0.0% | **1.6%** | 1.4% |
| Shop retention, dev (40) | 0 | **40** | 40 |
| Shop retention, multi-hop (40) | 0 | **1** | 0 |

- **On the final sets:** A1500 and B are the same within noise.
- **Behaviour:** the same strengths and limits as B. It is a clean tool user for one- and two-table
  questions. It is weak on 3+ tables, multi-step questions and large schemas, and it never recovers
  after a failed query.

## Training

| | |
|---|---|
| Data | 1,500 verified episodes: 1,265 SQaLe questions + 235 shop-database episodes (1,425 train / 75 validation) |
| How the episodes were made | the reference query path replayed against the real database; no teacher model |
| Quantization | 4-bit NF4, double quantization, fp16 compute |
| LoRA | rank 16, alpha 32, dropout 0.05; q, k, v, o, gate, up and down projections |
| Optimization | 2 epochs, learning rate 2e-4, effective batch 16, 180 updates; inputs up to 4,096 tokens |
| Loss | the model's own turns only; tool outputs are never trained |
| Hardware and cost | one Tesla T4; training plus dev check about $2.36 by rate |

Training records are in `training/`.

## How to load

Use the same code as on the [B model card](https://huggingface.co/satish9177/tiny-aya-sql-agent), with
`"satish9177/tiny-aya-sql-agent-a1500"` as the adapter. The base model is pinned at revision
`af89d219b53ed9b13b8a4645f9c8028973510324`. Load the tokenizer from its `tokenizer.json`, as that card
explains.

## Licence and credits

- **Licence:** CC-BY-NC-4.0, following **Tiny Aya Global** by Cohere Labs. Non-commercial use only.
- **AgentTune and AuditKIT:** Lexsi Labs, source-available and non-commercial.
- **SQaLe:** Wolff, Gomm and Hulsebos (2026). Its card declares MIT.