Text Generation
PEFT
Safetensors
lexsi-hackathon
agenttune
auditkit
curatorkit
text-to-sql
cohere-aya
self-healing
lora
conversational
Instructions to use Mrunal1201/tiny-aya-sql-heal-lora with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- PEFT
How to use Mrunal1201/tiny-aya-sql-heal-lora with PEFT:
from peft import PeftModel from transformers import AutoModelForCausalLM base_model = AutoModelForCausalLM.from_pretrained("CohereLabs/tiny-aya-global") model = PeftModel.from_pretrained(base_model, "Mrunal1201/tiny-aya-sql-heal-lora") - Notebooks
- Google Colab
- Kaggle
File size: 13,621 Bytes
1c5c41e 64cdcd5 1c5c41e 64cdcd5 1c5c41e 64cdcd5 ca8155e 1c5c41e 64cdcd5 1c5c41e 64cdcd5 1c5c41e 64cdcd5 1c5c41e 64cdcd5 1c5c41e 64cdcd5 1c5c41e 64cdcd5 1c5c41e 64cdcd5 1c5c41e 64cdcd5 1c5c41e 64cdcd5 1c5c41e 64cdcd5 1c5c41e 64cdcd5 1c5c41e 64cdcd5 1c5c41e 64cdcd5 1c5c41e 64cdcd5 1c5c41e 64cdcd5 1c5c41e 64cdcd5 1c5c41e | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 | ---
base_model: CohereLabs/tiny-aya-global
library_name: peft
pipeline_tag: text-generation
tags:
- lexsi-hackathon
- agenttune
- auditkit
- curatorkit
- text-to-sql
- cohere-aya
- self-healing
- lora
datasets:
- Mrunal1201/aya-self-healing-sql-dataset
language:
- en
- hi
- de
- kn
license: apache-2.0
metrics:
- code_eval
---
# tiny-aya-sql-heal-lora: Primary Multi-Turn Self-Healing SQL Adapter (3.35B)
Official PEFT LoRA checkpoint for **Lexsi Hackathon Track 3 (Data & Database Agents)** (`#6` Data & Database Agents, `#8` Self-Healing Agents, `#3` Agentic Distillation):
*Autonomous Multi-Turn Self-Healing Multilingual SQL Agents via CuratorKIT Curation, AgentTune Distillation, SQLGlot AST Guardrails, and AuditKIT Verification.*
| Resource | Link |
| :--- | :--- |
| **GitHub Repository** | [`Mru1201/aya-self-healing-sql-agent`](https://github.com/Mru1201/aya-self-healing-sql-agent) |
| **Executed Colab Notebook** | [`notebooks/train_and_audit_aya_sql.ipynb`](https://github.com/Mru1201/aya-self-healing-sql-agent/blob/main/notebooks/train_and_audit_aya_sql.ipynb) |
| **This Model Adapter** | [`Mrunal1201/tiny-aya-sql-heal-lora`](https://huggingface.co/Mrunal1201/tiny-aya-sql-heal-lora) |
| **Base Model** | [`CohereLabs/tiny-aya-global`](https://huggingface.co/CohereLabs/tiny-aya-global) |
| **Public HF Model Collection** | [Lexsi Track 3 — Self-Healing SQL Aya Models](https://huggingface.co/collections/Mrunal1201/lexsi-track-3-self-healing-sql-aya-models-6af8e7fc0a621b28fefccd18) |
| **Curated Dataset & Audit Logs** | [`Mrunal1201/aya-self-healing-sql-dataset`](https://huggingface.co/datasets/Mrunal1201/aya-self-healing-sql-dataset) ([Collection](https://huggingface.co/collections/Mrunal1201/lexsi-track-3-curated-sql-trajectories-and-audit-logs-6af8e7fed8ac99740564a2fb)) |
| **Lexsi Kits Chained** | [`CuratorKIT 0.1.21`](https://github.com/Lexsi-Labs/CuratorKIT) → [`AgentTune 1.1.0`](https://github.com/Lexsi-Labs/AgentTune) → [`AuditKIT 1.1.12`](https://github.com/Lexsi-Labs/AuditKIT) |
| **HF Static Article Space** | [Interactive Technical Report](https://huggingface.co/spaces/Mrunal1201/aya-self-healing-sql-article) ([Direct Embed](https://mrunal1201-aya-self-healing-sql-article.static.hf.space)) |
| **HF Community Articles** | [Model Discussion #1](https://huggingface.co/Mrunal1201/tiny-aya-sql-heal-lora/discussions/1) · [Dataset Discussion #1](https://huggingface.co/datasets/Mrunal1201/aya-self-healing-sql-dataset/discussions/1) |
---
## Executive Summary
Primary PEFT LoRA adapter for **CohereLabs/tiny-aya-global** (3.35B) trained on all `1,200` [CuratorKIT](https://github.com/Lexsi-Labs/CuratorKIT)-certified multilingual SQL episodes (`780` `self_healing` + `420` `single_turn`) for **Lexsi Hackathon Track 3** (`#6` Data & Database Agents, `#8` Self-Healing Agents, `#3` Agentic Distillation). Cuts Turn-1 SQLite `OperationalError` crashes by **45.5%** over single-turn SFT (`33 -> 18`), improves Executable SQL Validity over control (`86.0% -> 91.0%`), and trains in just `6.331 GB` peak VRAM.
### Key Innovations
1. **Closed-Loop Deterministic Self-Healing:** Turn-1 SQL → deterministic `SQLGlot 30.31.0` AST read-only validation → ephemeral in-memory `SQLite 3.46.0` execution → verbatim `sqlite3.OperationalError` feedback → Turn-2 `Thought:` reasoning and repaired SQL.
2. **Preamble-Safe Context Window (`max_seq_length=1536`):** Directly measured `365` chat-template overhead tokens on `tiny-aya-global` (across our dual-scale study, `tiny-aya-global` injects `365` tokens vs. `6` tokens on `aya-expanse-8b`), preventing completion-token truncation to zero on multi-turn episodes (`0` dropped rows).
3. **Zero-Leakage Audit & Bootstrap Significance:** `80` training domains vs. `20` strictly held-out test domains (`0` domain overlap, `0` schema table overlap across `933`/`185` unique tables, `0` MinHash LSH overlap at `0.85`), evaluated via [`AuditKIT 1.1.12`](https://github.com/Lexsi-Labs/AuditKIT) across `1,000`-resample paired bootstrap CIs.
4. **Multilingual Reasoning Preservation:** Maintains **`96.0%`** (`48/50`) general reasoning accuracy across 10 diverse languages (`ar`, `de`, `es`, `fr`, `hi`, `ja`, `pt`, `sw`, `tr`, `zh`).
---
## Evaluation & Benchmark Results (`AuditKIT v1.1.12`)
### Focused Comparison: `tiny-aya-sql-heal-lora` vs. `tiny-aya-global` Base & Teacher Baseline
| Metric / Suite (`n`) | `tiny-aya-global` Base | **`tiny-aya-sql-heal-lora`** | `Qwen2.5-72B-Instruct:cheapest` | Delta vs. Own Base |
| :--- | :---: | :---: | :---: | :---: |
| **Turn-1 Exec Match (`n=200`)** | 56.5% `[49.5%, 63.0%]` | **60.0%** `[52.5%, 66.01%]` | 71.0% `[65.0%, 76.5%]` | **+3.5 pp** |
| **Final Self-Healed Exec Match (`n=200`)** | 57.5% `[50.5%, 64.0%]` | **60.0%** `[52.5%, 66.01%]` | 71.0% `[65.0%, 76.5%]` | **+2.5 pp** |
| **Executable SQL Validity Rate (`n=200`)** | 95.5% `[92.5%, 98.0%]` | **91.0%** `[87.0%, 94.5%]` | 100.0% `[100.0%, 100.0%]` | `-4.5 pp` |
| **Turn-1 `OperationalError` Count (`n=200`)** | `12 / 200` | **`18 / 200`** | `3 / 200` | `+6` errors |
| **Multilingual Retention (`n=50`, 10 langs)** | 96.0% `[90.0%, 100.0%]` | **96.0%** `[90.0%, 100.0%]` | 98.0% `[94.0%, 100.0%]` | `+0.0 pp` |
| **Pre-Guardrail Model Injection Safe (`n=50`)** | 24.0% `[12.0%, 36.05%]` | **10.0%** `[2.0%, 20.0%]` | 34.0% `[21.95%, 48.0%]` | `-14.0 pp` *(AST gate blocks writes)* |
| **Inference Throughput (`tok/s`)** | `79.24 tok/s` | **`18.95 tok/s`** | `61.13 tok/s` | Self-hosted L4 4-bit |
| **Mean Query Latency (`sql_heal`)** | 443.4 ms | **4,136.4 ms** | 581.8 ms *(8w)* / 2,891.0 ms *(1c)* | Air-gapped local execution |
| **Cost per 1,000 Queries** | `$0.0591` | **`$0.5515`** | `$0.0401` (cloud API) | Full data sovereignty |
### Full 6-Configuration Benchmark & Ablation Table (`audit_logs/04_auditkit_before_after.json`)
| Model Configuration | Turn-1 Exec Match [95% CI] | Final Exec Match [95% CI] | Executable SQL Validity [95% CI] | Turn-1 Errors (`OperationalError`) | Heal Rate | Adversarial Safety (`n=50`) | Retention (`n=50`) | Throughput (`tok/s`) | Mean Latency | Cost / 1k Queries |
| :--- | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
| `CohereLabs/tiny-aya-global` Base (3.35B) | 56.5% `[49.5%, 63.0%]` | **57.5%** `[50.5%, 64.0%]` (`115/200`) | **95.5%** `[92.5%, 98.0%]` | `12 / 200` | 16.67% (`2/12`) | 24.0% `[12.0%, 36.05%]` | 96.0% `[90.0%, 100.0%]` | `79.24` | 443.4 ms | `$0.0591` |
| `tiny-aya-sql-heal-lora` (3.35B) | 60.0% `[52.5%, 66.01%]` | **60.0%** `[52.5%, 66.01%]` (`120/200`) | **91.0%** `[87.0%, 94.5%]` | `18 / 200` | 0.0% (`0/18`) | 10.0% `[2.0%, 20.0%]` | 96.0% `[90.0%, 100.0%]` | `18.95` | 4,136.4 ms | `$0.5515` |
| `tiny-aya-sql-control-lora` (3.35B, 1-Turn) | 58.5% `[52.0%, 65.5%]` | **60.0%** `[53.5%, 66.51%]` (`120/200`) | **86.0%** `[81.5%, 90.5%]` | `33 / 200` | 9.09% (`3/33`) | 10.0% `[2.0%, 20.0%]` | 96.0% `[90.0%, 100.0%]` | `18.50` | 4,450.2 ms | `$0.5934` |
| `CohereLabs/aya-expanse-8b` Base (8B) | 57.0% `[50.0%, 63.5%]` | **57.5%** `[50.5%, 64.0%]` (`115/200`) | **98.0%** `[96.0%, 99.5%]` | `11 / 200` | 9.09% (`1/11`) | 12.0% `[4.0%, 22.0%]` | 98.0% `[94.0%, 100.0%]` | `55.18` | 659.6 ms | `$0.0879` |
| `aya-8b-sql-heal-lora` (8B) | 63.5% `[57.0%, 70.0%]` | **63.5%** `[57.0%, 70.0%]` (`127/200`) | **96.0%** `[93.0%, 98.5%]` | `8 / 200` | 0.0% (`0/8`) | 14.0% `[6.0%, 24.0%]` | 96.0% `[90.0%, 100.0%]` | `35.67` | 2,307.3 ms | `$0.3076` |
| `Qwen/Qwen2.5-72B-Instruct:cheapest` (Teacher) | 71.0% `[65.0%, 76.5%]` | **71.0%** `[65.0%, 76.5%]` (`142/200`) | **100.0%** `[100.0%, 100.0%]` | `3 / 200` | 0.0% (`0/3`) | 34.0% `[21.95%, 48.0%]` | 98.0% `[94.0%, 100.0%]` | `61.13` | 581.8 ms *(8w)* / 2,891.0 ms *(1c)* | `$0.0401` |
---
## Visual Benchmarks & Diagnostics (`assets/figures/`)

*Figure 0: End-to-end closed-loop Track 3 pipeline uniting `CuratorKIT 0.1.21` curation & leak checks, `AgentTune 1.1.0` QLoRA trajectory distillation, `SQLGlot 30.31.0` read-only AST guardrails over in-memory `SQLite 3.46.0`, and `AuditKIT 1.1.12` outcome verification.*

*Figure 1: Turn-1 execution accuracy, final self-healed execution accuracy, and SQLite executable validity across all 6 evaluated configurations on the 200 disjoint test queries (`1,000`-resample 95% bootstrap CIs from `audit_logs/04_auditkit_before_after.json`).*

*Figure 2: (Left) Preventative schema grounding cuts Turn-1 SQLite `OperationalError` counts from `33` (`tiny-aya-sql-control-lora`) down to `18` (`-45.5%` on 3.35B) and `8` (`-75.8%` on 8B). (Right) Complete final outcome composition across all 200 held-out test cases.*

*Figure 3: Systems inversion comparing `tiny-aya-sql-heal-lora` (3.35B) vs. `aya-8b-sql-heal-lora` (8B) across chat-template preamble overhead (`365` vs. `6` tokens), prompt length (`635.1` vs. `210.2`), decode throughput (`18.95` vs. `35.67 tok/s`), mean latency (`4,136.4 ms` vs. `2,307.3 ms`), L4 cost (`$0.5515` vs. `$0.3076` per 1k queries), and peak allocated VRAM (`6.331 GB` vs. `13.898 GB`).*

*Figure 4: (Left) `AgentTune 1.1.0` completion-only supervised training loss curves (`logging_steps = 5`, `03_training_hardware_meta.json`). (Right) Pre-guardrail model resistance across all 6 adversarial SQL injection vectors (`n = 50`), proving why deterministic `SQLGlot` AST filtering is essential.*
---
## Training Specifications & Hyperparameters (`audit_logs/03_training_hardware_meta.json`)
- **Hardware Infrastructure:** `NVIDIA L4` (`22.03 GB` VRAM)
- **Software Stack:** PyTorch `2.11.0+cu130`, CUDA `13.0`, PEFT `0.21.3`, TRL `1.7.1`, [`AgentTune 1.1.0`](https://github.com/Lexsi-Labs/AgentTune)
- **Training Backend:** `agenttune.create_distill_trainer`
- **Training Subset:** `full` (`1200` episodes)
- **Global Steps:** `150` steps across `2.0` epochs
- **Learning Rate:** `0.0002` with `cosine` schedule and `0.03` warmup
- **Effective Batch Size:** `16` (`4` per device × `4` gradient accumulation)
- **Optimizer & Precision:** `paged_adamw_8bit`, `bf16=True`, `gradient_checkpointing=True`
- **Sequence Length & Overhead:** `max_seq_length=1536` | Chat template preamble: `365` tokens | Mean/Max length: `630.0` / `1085` tokens | `0` dropped rows
- **Loss Masking:** `completion_only (final assistant turn); system/user/turn-1 SQL/error masked`
- **Training Loss (Step 5 → Step 150):** `1.2010 → 0.3071` (**0.4081** mean loss)
- **Peak VRAM (Allocated / Reserved):** `6.331 GB` / `6.648 GB`
- **Wall-Clock Training Time:** `24.283` minutes (`1430.27 s`)
### LoRA Architecture
```json
{
"r": 16,
"lora_alpha": 32,
"lora_dropout": 0.05,
"target_modules": ["q_proj", "k_proj", "v_proj", "o_proj", "gate_proj", "up_proj", "down_proj"],
"bias": "none",
"task_type": "CAUSAL_LM"
}
```
---
## Quickstart & Inference
```python
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
base_model_id = "CohereLabs/tiny-aya-global"
adapter_id = "Mrunal1201/tiny-aya-sql-heal-lora"
# 1. Load Tokenizer and Base Model in BF16 / 4-bit
tokenizer = AutoTokenizer.from_pretrained(base_model_id)
base_model = AutoModelForCausalLM.from_pretrained(
base_model_id,
torch_dtype=torch.bfloat16,
device_map="auto",
)
# 2. Attach Self-Healing LoRA Adapter
model = PeftModel.from_pretrained(base_model, adapter_id)
model.eval()
# 3. Format Multi-Turn Chat with SQLite Schema & Error Turn
schema = "CREATE TABLE customers (id INT, name TEXT, balance REAL);"
prompt = "Find all customers with balance over 1000."
error_feedback = "SQLite Execution Error: no such column: amount"
messages = [
{"role": "system", "content": "You are a multilingual SQLite data assistant. Translate the question into read-only SQL.\nSchema:\n" + schema},
{"role": "user", "content": prompt},
{"role": "assistant", "content": "```sql\nSELECT name FROM customers WHERE amount > 1000;\n```"},
{"role": "user", "content": error_feedback + "\nPlease inspect the schema and fix the query."}
]
input_ids = tokenizer.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt").to(model.device)
with torch.no_grad():
outputs = model.generate(input_ids, max_new_tokens=256, do_sample=False)
print(tokenizer.decode(outputs[0][input_ids.shape[1]:], skip_special_tokens=True))
```
---
## Citation & Attribution
```bibtex
@misc{lexsi_track3_aya_sql_2026,
title={Autonomous Self-Healing Multilingual SQL Agents on Cohere Aya},
author={Mrunal1201 and Lexsi Hackathon Team},
year={2026},
howpublished={\url{https://github.com/Mru1201/aya-self-healing-sql-agent}},
note={Lexsi Hackathon Track 3 (Data & Database Agents)}
}
```
|