Text Generation
PEFT
Safetensors
lexsi-hackathon
agenttune
auditkit
curatorkit
text-to-sql
cohere-aya
self-healing
lora
conversational
Instructions to use Mrunal1201/tiny-aya-sql-heal-lora with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- PEFT
How to use Mrunal1201/tiny-aya-sql-heal-lora with PEFT:
from peft import PeftModel from transformers import AutoModelForCausalLM base_model = AutoModelForCausalLM.from_pretrained("CohereLabs/tiny-aya-global") model = PeftModel.from_pretrained(base_model, "Mrunal1201/tiny-aya-sql-heal-lora") - Notebooks
- Google Colab
- Kaggle
Update tiny-aya-sql-heal-lora Model Card with official Lexsi links, exact AuditKIT table, and benchmark figures
ca8155e verified |
Download README.md from Mrunal1201/tiny-aya-sql-heal-lora: direct link, hf CLI and curl.
- Browser
- Download file 13.6 kB
-
https://huggingface.co/Mrunal1201/tiny-aya-sql-heal-lora/resolve/main/README.md
- Command line
-
hf download hf://Mrunal1201/tiny-aya-sql-heal-lora/README.md
-
curl -L -o README.md https://huggingface.co/Mrunal1201/tiny-aya-sql-heal-lora/resolve/main/README.md
13.6 kB
| base_model: CohereLabs/tiny-aya-global | |
| library_name: peft | |
| pipeline_tag: text-generation | |
| tags: | |
| - lexsi-hackathon | |
| - agenttune | |
| - auditkit | |
| - curatorkit | |
| - text-to-sql | |
| - cohere-aya | |
| - self-healing | |
| - lora | |
| datasets: | |
| - Mrunal1201/aya-self-healing-sql-dataset | |
| language: | |
| - en | |
| - hi | |
| - de | |
| - kn | |
| license: apache-2.0 | |
| metrics: | |
| - code_eval | |
| # tiny-aya-sql-heal-lora: Primary Multi-Turn Self-Healing SQL Adapter (3.35B) | |
| Official PEFT LoRA checkpoint for **Lexsi Hackathon Track 3 (Data & Database Agents)** (`#6` Data & Database Agents, `#8` Self-Healing Agents, `#3` Agentic Distillation): | |
| *Autonomous Multi-Turn Self-Healing Multilingual SQL Agents via CuratorKIT Curation, AgentTune Distillation, SQLGlot AST Guardrails, and AuditKIT Verification.* | |
| | Resource | Link | | |
| | :--- | :--- | | |
| | **GitHub Repository** | [`Mru1201/aya-self-healing-sql-agent`](https://github.com/Mru1201/aya-self-healing-sql-agent) | | |
| | **Executed Colab Notebook** | [`notebooks/train_and_audit_aya_sql.ipynb`](https://github.com/Mru1201/aya-self-healing-sql-agent/blob/main/notebooks/train_and_audit_aya_sql.ipynb) | | |
| | **This Model Adapter** | [`Mrunal1201/tiny-aya-sql-heal-lora`](https://huggingface.co/Mrunal1201/tiny-aya-sql-heal-lora) | | |
| | **Base Model** | [`CohereLabs/tiny-aya-global`](https://huggingface.co/CohereLabs/tiny-aya-global) | | |
| | **Public HF Model Collection** | [Lexsi Track 3 — Self-Healing SQL Aya Models](https://huggingface.co/collections/Mrunal1201/lexsi-track-3-self-healing-sql-aya-models-6af8e7fc0a621b28fefccd18) | | |
| | **Curated Dataset & Audit Logs** | [`Mrunal1201/aya-self-healing-sql-dataset`](https://huggingface.co/datasets/Mrunal1201/aya-self-healing-sql-dataset) ([Collection](https://huggingface.co/collections/Mrunal1201/lexsi-track-3-curated-sql-trajectories-and-audit-logs-6af8e7fed8ac99740564a2fb)) | | |
| | **Lexsi Kits Chained** | [`CuratorKIT 0.1.21`](https://github.com/Lexsi-Labs/CuratorKIT) → [`AgentTune 1.1.0`](https://github.com/Lexsi-Labs/AgentTune) → [`AuditKIT 1.1.12`](https://github.com/Lexsi-Labs/AuditKIT) | | |
| | **HF Static Article Space** | [Interactive Technical Report](https://huggingface.co/spaces/Mrunal1201/aya-self-healing-sql-article) ([Direct Embed](https://mrunal1201-aya-self-healing-sql-article.static.hf.space)) | | |
| | **HF Community Articles** | [Model Discussion #1](https://huggingface.co/Mrunal1201/tiny-aya-sql-heal-lora/discussions/1) · [Dataset Discussion #1](https://huggingface.co/datasets/Mrunal1201/aya-self-healing-sql-dataset/discussions/1) | | |
| --- | |
| ## Executive Summary | |
| Primary PEFT LoRA adapter for **CohereLabs/tiny-aya-global** (3.35B) trained on all `1,200` [CuratorKIT](https://github.com/Lexsi-Labs/CuratorKIT)-certified multilingual SQL episodes (`780` `self_healing` + `420` `single_turn`) for **Lexsi Hackathon Track 3** (`#6` Data & Database Agents, `#8` Self-Healing Agents, `#3` Agentic Distillation). Cuts Turn-1 SQLite `OperationalError` crashes by **45.5%** over single-turn SFT (`33 -> 18`), improves Executable SQL Validity over control (`86.0% -> 91.0%`), and trains in just `6.331 GB` peak VRAM. | |
| ### Key Innovations | |
| 1. **Closed-Loop Deterministic Self-Healing:** Turn-1 SQL → deterministic `SQLGlot 30.31.0` AST read-only validation → ephemeral in-memory `SQLite 3.46.0` execution → verbatim `sqlite3.OperationalError` feedback → Turn-2 `Thought:` reasoning and repaired SQL. | |
| 2. **Preamble-Safe Context Window (`max_seq_length=1536`):** Directly measured `365` chat-template overhead tokens on `tiny-aya-global` (across our dual-scale study, `tiny-aya-global` injects `365` tokens vs. `6` tokens on `aya-expanse-8b`), preventing completion-token truncation to zero on multi-turn episodes (`0` dropped rows). | |
| 3. **Zero-Leakage Audit & Bootstrap Significance:** `80` training domains vs. `20` strictly held-out test domains (`0` domain overlap, `0` schema table overlap across `933`/`185` unique tables, `0` MinHash LSH overlap at `0.85`), evaluated via [`AuditKIT 1.1.12`](https://github.com/Lexsi-Labs/AuditKIT) across `1,000`-resample paired bootstrap CIs. | |
| 4. **Multilingual Reasoning Preservation:** Maintains **`96.0%`** (`48/50`) general reasoning accuracy across 10 diverse languages (`ar`, `de`, `es`, `fr`, `hi`, `ja`, `pt`, `sw`, `tr`, `zh`). | |
| --- | |
| ## Evaluation & Benchmark Results (`AuditKIT v1.1.12`) | |
| ### Focused Comparison: `tiny-aya-sql-heal-lora` vs. `tiny-aya-global` Base & Teacher Baseline | |
| | Metric / Suite (`n`) | `tiny-aya-global` Base | **`tiny-aya-sql-heal-lora`** | `Qwen2.5-72B-Instruct:cheapest` | Delta vs. Own Base | | |
| | :--- | :---: | :---: | :---: | :---: | | |
| | **Turn-1 Exec Match (`n=200`)** | 56.5% `[49.5%, 63.0%]` | **60.0%** `[52.5%, 66.01%]` | 71.0% `[65.0%, 76.5%]` | **+3.5 pp** | | |
| | **Final Self-Healed Exec Match (`n=200`)** | 57.5% `[50.5%, 64.0%]` | **60.0%** `[52.5%, 66.01%]` | 71.0% `[65.0%, 76.5%]` | **+2.5 pp** | | |
| | **Executable SQL Validity Rate (`n=200`)** | 95.5% `[92.5%, 98.0%]` | **91.0%** `[87.0%, 94.5%]` | 100.0% `[100.0%, 100.0%]` | `-4.5 pp` | | |
| | **Turn-1 `OperationalError` Count (`n=200`)** | `12 / 200` | **`18 / 200`** | `3 / 200` | `+6` errors | | |
| | **Multilingual Retention (`n=50`, 10 langs)** | 96.0% `[90.0%, 100.0%]` | **96.0%** `[90.0%, 100.0%]` | 98.0% `[94.0%, 100.0%]` | `+0.0 pp` | | |
| | **Pre-Guardrail Model Injection Safe (`n=50`)** | 24.0% `[12.0%, 36.05%]` | **10.0%** `[2.0%, 20.0%]` | 34.0% `[21.95%, 48.0%]` | `-14.0 pp` *(AST gate blocks writes)* | | |
| | **Inference Throughput (`tok/s`)** | `79.24 tok/s` | **`18.95 tok/s`** | `61.13 tok/s` | Self-hosted L4 4-bit | | |
| | **Mean Query Latency (`sql_heal`)** | 443.4 ms | **4,136.4 ms** | 581.8 ms *(8w)* / 2,891.0 ms *(1c)* | Air-gapped local execution | | |
| | **Cost per 1,000 Queries** | `$0.0591` | **`$0.5515`** | `$0.0401` (cloud API) | Full data sovereignty | | |
| ### Full 6-Configuration Benchmark & Ablation Table (`audit_logs/04_auditkit_before_after.json`) | |
| | Model Configuration | Turn-1 Exec Match [95% CI] | Final Exec Match [95% CI] | Executable SQL Validity [95% CI] | Turn-1 Errors (`OperationalError`) | Heal Rate | Adversarial Safety (`n=50`) | Retention (`n=50`) | Throughput (`tok/s`) | Mean Latency | Cost / 1k Queries | | |
| | :--- | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | | |
| | `CohereLabs/tiny-aya-global` Base (3.35B) | 56.5% `[49.5%, 63.0%]` | **57.5%** `[50.5%, 64.0%]` (`115/200`) | **95.5%** `[92.5%, 98.0%]` | `12 / 200` | 16.67% (`2/12`) | 24.0% `[12.0%, 36.05%]` | 96.0% `[90.0%, 100.0%]` | `79.24` | 443.4 ms | `$0.0591` | | |
| | `tiny-aya-sql-heal-lora` (3.35B) | 60.0% `[52.5%, 66.01%]` | **60.0%** `[52.5%, 66.01%]` (`120/200`) | **91.0%** `[87.0%, 94.5%]` | `18 / 200` | 0.0% (`0/18`) | 10.0% `[2.0%, 20.0%]` | 96.0% `[90.0%, 100.0%]` | `18.95` | 4,136.4 ms | `$0.5515` | | |
| | `tiny-aya-sql-control-lora` (3.35B, 1-Turn) | 58.5% `[52.0%, 65.5%]` | **60.0%** `[53.5%, 66.51%]` (`120/200`) | **86.0%** `[81.5%, 90.5%]` | `33 / 200` | 9.09% (`3/33`) | 10.0% `[2.0%, 20.0%]` | 96.0% `[90.0%, 100.0%]` | `18.50` | 4,450.2 ms | `$0.5934` | | |
| | `CohereLabs/aya-expanse-8b` Base (8B) | 57.0% `[50.0%, 63.5%]` | **57.5%** `[50.5%, 64.0%]` (`115/200`) | **98.0%** `[96.0%, 99.5%]` | `11 / 200` | 9.09% (`1/11`) | 12.0% `[4.0%, 22.0%]` | 98.0% `[94.0%, 100.0%]` | `55.18` | 659.6 ms | `$0.0879` | | |
| | `aya-8b-sql-heal-lora` (8B) | 63.5% `[57.0%, 70.0%]` | **63.5%** `[57.0%, 70.0%]` (`127/200`) | **96.0%** `[93.0%, 98.5%]` | `8 / 200` | 0.0% (`0/8`) | 14.0% `[6.0%, 24.0%]` | 96.0% `[90.0%, 100.0%]` | `35.67` | 2,307.3 ms | `$0.3076` | | |
| | `Qwen/Qwen2.5-72B-Instruct:cheapest` (Teacher) | 71.0% `[65.0%, 76.5%]` | **71.0%** `[65.0%, 76.5%]` (`142/200`) | **100.0%** `[100.0%, 100.0%]` | `3 / 200` | 0.0% (`0/3`) | 34.0% `[21.95%, 48.0%]` | 98.0% `[94.0%, 100.0%]` | `61.13` | 581.8 ms *(8w)* / 2,891.0 ms *(1c)* | `$0.0401` | | |
| --- | |
| ## Visual Benchmarks & Diagnostics (`assets/figures/`) | |
|  | |
| *Figure 0: End-to-end closed-loop Track 3 pipeline uniting `CuratorKIT 0.1.21` curation & leak checks, `AgentTune 1.1.0` QLoRA trajectory distillation, `SQLGlot 30.31.0` read-only AST guardrails over in-memory `SQLite 3.46.0`, and `AuditKIT 1.1.12` outcome verification.* | |
|  | |
| *Figure 1: Turn-1 execution accuracy, final self-healed execution accuracy, and SQLite executable validity across all 6 evaluated configurations on the 200 disjoint test queries (`1,000`-resample 95% bootstrap CIs from `audit_logs/04_auditkit_before_after.json`).* | |
|  | |
| *Figure 2: (Left) Preventative schema grounding cuts Turn-1 SQLite `OperationalError` counts from `33` (`tiny-aya-sql-control-lora`) down to `18` (`-45.5%` on 3.35B) and `8` (`-75.8%` on 8B). (Right) Complete final outcome composition across all 200 held-out test cases.* | |
|  | |
| *Figure 3: Systems inversion comparing `tiny-aya-sql-heal-lora` (3.35B) vs. `aya-8b-sql-heal-lora` (8B) across chat-template preamble overhead (`365` vs. `6` tokens), prompt length (`635.1` vs. `210.2`), decode throughput (`18.95` vs. `35.67 tok/s`), mean latency (`4,136.4 ms` vs. `2,307.3 ms`), L4 cost (`$0.5515` vs. `$0.3076` per 1k queries), and peak allocated VRAM (`6.331 GB` vs. `13.898 GB`).* | |
|  | |
| *Figure 4: (Left) `AgentTune 1.1.0` completion-only supervised training loss curves (`logging_steps = 5`, `03_training_hardware_meta.json`). (Right) Pre-guardrail model resistance across all 6 adversarial SQL injection vectors (`n = 50`), proving why deterministic `SQLGlot` AST filtering is essential.* | |
| --- | |
| ## Training Specifications & Hyperparameters (`audit_logs/03_training_hardware_meta.json`) | |
| - **Hardware Infrastructure:** `NVIDIA L4` (`22.03 GB` VRAM) | |
| - **Software Stack:** PyTorch `2.11.0+cu130`, CUDA `13.0`, PEFT `0.21.3`, TRL `1.7.1`, [`AgentTune 1.1.0`](https://github.com/Lexsi-Labs/AgentTune) | |
| - **Training Backend:** `agenttune.create_distill_trainer` | |
| - **Training Subset:** `full` (`1200` episodes) | |
| - **Global Steps:** `150` steps across `2.0` epochs | |
| - **Learning Rate:** `0.0002` with `cosine` schedule and `0.03` warmup | |
| - **Effective Batch Size:** `16` (`4` per device × `4` gradient accumulation) | |
| - **Optimizer & Precision:** `paged_adamw_8bit`, `bf16=True`, `gradient_checkpointing=True` | |
| - **Sequence Length & Overhead:** `max_seq_length=1536` | Chat template preamble: `365` tokens | Mean/Max length: `630.0` / `1085` tokens | `0` dropped rows | |
| - **Loss Masking:** `completion_only (final assistant turn); system/user/turn-1 SQL/error masked` | |
| - **Training Loss (Step 5 → Step 150):** `1.2010 → 0.3071` (**0.4081** mean loss) | |
| - **Peak VRAM (Allocated / Reserved):** `6.331 GB` / `6.648 GB` | |
| - **Wall-Clock Training Time:** `24.283` minutes (`1430.27 s`) | |
| ### LoRA Architecture | |
| ```json | |
| { | |
| "r": 16, | |
| "lora_alpha": 32, | |
| "lora_dropout": 0.05, | |
| "target_modules": ["q_proj", "k_proj", "v_proj", "o_proj", "gate_proj", "up_proj", "down_proj"], | |
| "bias": "none", | |
| "task_type": "CAUSAL_LM" | |
| } | |
| ``` | |
| --- | |
| ## Quickstart & Inference | |
| ```python | |
| import torch | |
| from transformers import AutoModelForCausalLM, AutoTokenizer | |
| from peft import PeftModel | |
| base_model_id = "CohereLabs/tiny-aya-global" | |
| adapter_id = "Mrunal1201/tiny-aya-sql-heal-lora" | |
| # 1. Load Tokenizer and Base Model in BF16 / 4-bit | |
| tokenizer = AutoTokenizer.from_pretrained(base_model_id) | |
| base_model = AutoModelForCausalLM.from_pretrained( | |
| base_model_id, | |
| torch_dtype=torch.bfloat16, | |
| device_map="auto", | |
| ) | |
| # 2. Attach Self-Healing LoRA Adapter | |
| model = PeftModel.from_pretrained(base_model, adapter_id) | |
| model.eval() | |
| # 3. Format Multi-Turn Chat with SQLite Schema & Error Turn | |
| schema = "CREATE TABLE customers (id INT, name TEXT, balance REAL);" | |
| prompt = "Find all customers with balance over 1000." | |
| error_feedback = "SQLite Execution Error: no such column: amount" | |
| messages = [ | |
| {"role": "system", "content": "You are a multilingual SQLite data assistant. Translate the question into read-only SQL.\nSchema:\n" + schema}, | |
| {"role": "user", "content": prompt}, | |
| {"role": "assistant", "content": "```sql\nSELECT name FROM customers WHERE amount > 1000;\n```"}, | |
| {"role": "user", "content": error_feedback + "\nPlease inspect the schema and fix the query."} | |
| ] | |
| input_ids = tokenizer.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt").to(model.device) | |
| with torch.no_grad(): | |
| outputs = model.generate(input_ids, max_new_tokens=256, do_sample=False) | |
| print(tokenizer.decode(outputs[0][input_ids.shape[1]:], skip_special_tokens=True)) | |
| ``` | |
| --- | |
| ## Citation & Attribution | |
| ```bibtex | |
| @misc{lexsi_track3_aya_sql_2026, | |
| title={Autonomous Self-Healing Multilingual SQL Agents on Cohere Aya}, | |
| author={Mrunal1201 and Lexsi Hackathon Team}, | |
| year={2026}, | |
| howpublished={\url{https://github.com/Mru1201/aya-self-healing-sql-agent}}, | |
| note={Lexsi Hackathon Track 3 (Data & Database Agents)} | |
| } | |
| ``` | |