imagine-v10 / README.md
Interchained's picture
Update README.md
a910fe3 verified
|
Raw History Blame Contribute Delete
7.09 kB
---
license: apache-2.0
language:
- en
library_name: transformers
pipeline_tag: text-generation
tags:
- text-to-sql
- postgresql
- code-generation
base_model: Interchained/imagine-v9
---
<div align="center">
# 🧠 imagine-v10
### 100% on held-out. Zero hallucinations. Analytical SQL that actually works.
**Natural language in. Correct PostgreSQL out. ~1B parameters. No GPU. No API key. No metered inference.**
[![eval](https://img.shields.io/badge/heldout_eval-40%2F40_100%25-3ecf8e?style=flat-square)](#evaluation)
[![gate](https://img.shields.io/badge/execution_gate-68%2F68-3ecf8e?style=flat-square)](#execution-gate)
[![protocol](https://img.shields.io/badge/protocol_compliance-100%25-3ecf8e?style=flat-square)](#evaluation)
[![hallucinations](https://img.shields.io/badge/invented_columns-0-3ecf8e?style=flat-square)](#evaluation)
[![runtime](https://img.shields.io/badge/runtime-CPU_only-f5a524?style=flat-square)](#local-first)
[![licence](https://img.shields.io/badge/license-Apache--2.0-8957e5?style=flat-square)](#status)
*Part of project `imagine` — Interchained*
</div>
---
## The failure that built v10
v9 was asked for a LEFT JOIN analytical query:
```sql
-- "List every customer, their total completed spending, sort by spending desc"
```
It returned:
```sql
SELECT c.id, c.name, COALESCE(o.total, 0) AS total_spending
FROM customers c
LEFT JOIN orders o ON c.id
-- ...truncated. Dead.
```
The correct answer:
```sql
SELECT c.id, c.name, COALESCE(SUM(o.total), 0) AS total_spending
FROM customers c
LEFT JOIN orders o ON c.id = o.customer_id AND o.status = 'completed'
GROUP BY c.id, c.name
ORDER BY total_spending DESC, c.id ASC;
```
v9 couldn't do multi-clause analytical queries. **v10 can.**
---
## 🎯 What v10 adds
Where v9 added write capability (`INSERT`/`UPDATE`/`DELETE`), v10 adds **complex analytical queries**:
- **LEFT JOIN + GROUP BY + COALESCE** — the exact v9 failure mode, now handled
- **HAVING filters** — post-aggregation conditions
- **Subquery comparisons** — nested analytical logic
- **Complex ORDER BY** — multi-column, expression-based sorting
- **Multi-table JOINs** — beyond simple two-table patterns
---
## 📊 The numbers
| Model | Held-out (telemetry) | Protocol | Execute rate | Invented cols |
|---|---|---|---|---|
| imagine-v8 | 50.0% (20/40) | 100% | — | — |
| imagine-v9 | 90.0% (36/40) | 100% | 97.5% | 0 |
| **imagine-v10** | **100.0% (40/40)** | **100%** | **100%** | **0** |
**Perfect score.** Zero wrong answers. Zero hallucinations. Zero refusals.
---
## 🧬 How v10 was built
Two-stage fine-tuning from `Interchained/imagine-v9`:
### Stage 1 — Full fine-tune (smoke)
- **Corpus:** 3,216 admitted pairs (99.94% gate admit rate)
- **New:** Analytical query templates (LEFT JOIN aggregations, HAVING, subqueries)
- **Epochs:** 3, LR 1e-5, cosine schedule
- **Final loss:** 0.0052 (v9 was 0.0320 — **6x better**)
- **Time:** 4.1 min on H200, 26k tok/s
### Stage 2 — LoRA refinement
- **Base:** v10-smoke checkpoint
- **Rank:** 16, LR 5e-6
- **Epochs:** 3
- **Final loss:** 0.0244
- **Time:** 5.4 min on H200
- Adapter folded into full checkpoint
### The corpus
3,218 candidates forged, 3,216 admitted:
| Schema | Templates | Writes | Total |
|---|---|---|---|
| shop | 356 | 388 | 744 |
| clinic | 545 | 290 | 835 |
| library | 481 | 257 | 738 |
| fleet | 297 | 226 | 523 |
| audit | 252 | 126 | 378 |
**Nothing enters training that a live database hasn't agreed with.**
---
## 🔬 The execution gate
Every candidate goes through L0–L4:
```
candidate SQL ──► L0 real PostgreSQL parser not a regex
L1 read-only + bounded no writes, no sleeps
L2 EXPLAIN on live schema hallucinations die HERE
L3 execute, timed, capped real rows
L4 SAME ANSWER as ref? ◄── the one that matters
▼
admitted to corpus
```
68/68 gate tests passing. The gate is the truth authority — not a bigger model, not vibes.
---
## 📦 Output protocol
SQL wrapped in sentinel blocks:
```
<<<SQL>>>
SELECT c.id, c.name, COALESCE(SUM(o.total), 0) AS total_spending
FROM customers c
LEFT JOIN orders o ON c.id = o.customer_id AND o.status = 'completed'
GROUP BY c.id, c.name
ORDER BY total_spending DESC, c.id ASC;
<<<END>>>
```
Can also refuse:
| block | meaning |
|---|---|
| `<<<SQL>>>` | here is your query |
| `<<<UNANSWERABLE>>>` | this schema cannot answer that |
| `<<<CLARIFY>>>` | ambiguous — here's what's missing |
> ⚠️ **A truncated generation is not an answer.** An unterminated block extracts to **nothing**.
---
## 💻 Loading v10
```python
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch
model_id = "Interchained/imagine-v10"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
model_id,
torch_dtype=torch.bfloat16,
device_map="auto",
)
# Ask for an analytical query
prompt = """Schema:
customers(id, name)
orders(id, customer_id, total, status)
List every customer ID, name, and total completed spending.
Include customers with no completed orders (show 0).
Sort by spending descending, then ID ascending."""
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
with torch.no_grad():
output = model.generate(inputs["input_ids"], max_new_tokens=256, do_sample=False)
print(tokenizer.decode(output[0], skip_special_tokens=True))
```
---
## 🎯 Identity
The deployed model identity is **Imagine**.
Built and fine-tuned by **Interchained**.
DeepSeek-Coder is part of the upstream lineage (via v8 → v9 → v10), but the deployed identity is Imagine.
---
## ⚡ Local-first
Runs on your hardware. No API key. No metered inference. No cloud dependency.
---
## 📐 The rules
**1 · Don't write a verifier — the engine already shipped one.**
Real parser. Real planner. Real rows.
**2 · Assert the property, not a proxy.**
Execution accuracy, not string similarity.
**3 · Schema goes in the prompt, not in the weights.**
The model learns *"read the schema you were handed"* — not memorise ours.
---
## ⚠️ Limitations
v10 is a research checkpoint. It may still:
- generate incorrect SQL on novel patterns
- misunderstand ambiguous requests
- produce writes with wrong `WHERE` clauses — **always review before executing**
Generated SQL should be reviewed before use in production. This applies doubly to writes.
---
## 🔒 Security
Do not rely on model behavior alone for database safety. Production systems should enforce:
- least-privilege database roles
- statement timeouts and row limits
- query validation and schema restrictions
- application-level authorization
- audit logging
- **human approval for all write statements**
---
<div align="center">
**Built by [Interchained](https://Interchained.org)** · ownership at every layer, including the model
`3 > 1` — Mark drives, oracle points, Muse builds
</div>