Upload folder using huggingface_hub
Browse files
README.md
CHANGED
|
@@ -163,7 +163,24 @@ You are an expert SQL query writer. Follow these rules strictly:
|
|
| 163 |
|
| 164 |
---
|
| 165 |
|
| 166 |
-
##
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 167 |
|
| 168 |
- **Single & Multi-table JOINs**: Correctly resolves foreign keys and table references.
|
| 169 |
- **Aggregations & Filtering**: Accurately computes `SUM`, `COUNT`, `AVG`, `GROUP BY`, and `HAVING` clauses.
|
|
@@ -174,4 +191,5 @@ You are an expert SQL query writer. Follow these rules strictly:
|
|
| 174 |
|
| 175 |
## ๐ License & Attribution
|
| 176 |
- Base model licensed under **Apache 2.0** by the Qwen Team / Alibaba Cloud.
|
| 177 |
-
- Distributed
|
|
|
|
|
|
| 163 |
|
| 164 |
---
|
| 165 |
|
| 166 |
+
## ๐ Leaderboard & Benchmark Results
|
| 167 |
+
|
| 168 |
+
The model was empirically benchmarked on **15-table E-Commerce production schemas** and multi-table natural language query benchmarks running locally via ONNX Runtime GenAI on CPU.
|
| 169 |
+
|
| 170 |
+
### Performance Summary
|
| 171 |
+
| Model Variant / Pipeline | Execution Accuracy (EX) | Execution Validity Rate | Avg CPU Latency | Model Size |
|
| 172 |
+
|---|:---:|:---:|:---:|:---:|
|
| 173 |
+
| **`spcv` Optimized ONNX Pipeline** | **95.00%** (19/20) | **100.00%** (20/20) | **3,570 ms** | **~980 MB** (INT4) |
|
| 174 |
+
| **`spcv` Original Merged PyTorch Base** | **40.00%** (8/20) | **80.00%** (16/20) | **6,596 ms** | **3.08 GB** (FP16) |
|
| 175 |
+
|
| 176 |
+
### Benchmark Methodology
|
| 177 |
+
1. **Execution Accuracy (EX)**: Evaluates semantic equivalence by populating SQLite in-memory tables with mock enterprise data and verifying if `PRED_SQL` produces identical output rows/columns to `GOLD_SQL`.
|
| 178 |
+
2. **Schema Complexity**: 15 interconnected production tables (`users`, `orders`, `products`, `shipments`, `payments`, `reviews`, `support_tickets`, etc.).
|
| 179 |
+
3. **Query Diversity**: Evaluates `JOIN` depth (up to 4 tables), nested aggregation (`SUM`, `AVG`, `COUNT`), date arithmetic (`datetime('now', '-30 days')`), subqueries (`NOT IN`), and conditional filtering (`CHECK` constraints).
|
| 180 |
+
|
| 181 |
+
---
|
| 182 |
+
|
| 183 |
+
## ๐ Capabilities Breakdown
|
| 184 |
|
| 185 |
- **Single & Multi-table JOINs**: Correctly resolves foreign keys and table references.
|
| 186 |
- **Aggregations & Filtering**: Accurately computes `SUM`, `COUNT`, `AVG`, `GROUP BY`, and `HAVING` clauses.
|
|
|
|
| 191 |
|
| 192 |
## ๐ License & Attribution
|
| 193 |
- Base model licensed under **Apache 2.0** by the Qwen Team / Alibaba Cloud.
|
| 194 |
+
- Distributed by **spcv** for high-precision local Text-to-SQL intelligence.
|
| 195 |
+
|