nl2sql / performance.md
JayaHareeswarReddy's picture
Update performance.md
ea8aa45 verified
|
Raw History Blame Contribute Delete
1.19 kB
# NL2SQL – Performance Analysis
## 1. Model Performance Comparison
The following results compare the original Qwen2.5-Coder-1.5B-Instruct model against the QLoRA fine-tuned model.
| Evaluation Metric | Base Model | Fine-tuned Model | Improvement |
|---|---:|---:|---:|
| Exact Match | 23.5% | 31.0% | +7.5 pp |
| Execution Match | 66.7% | 71.5% | +4.8 pp |
## 2. Performance Visualization
### Exact Match
- Base Model: 23.5%
- Fine-tuned Model: 31.0%
### Execution Match
- Base Model: 66.7%
- Fine-tuned Model: 71.5%
## 3. Key Observations
1. Fine-tuning improved exact SQL query matching.
2. Execution Match increased, indicating better query execution performance on the evaluation sample.
3. QLoRA enabled parameter-efficient adaptation of the base model.
4. Both metrics showed measurable improvements after fine-tuning.
## 4. Conclusion
The experiment demonstrates that QLoRA fine-tuning improved the performance of Qwen2.5-Coder-1.5B-Instruct on the evaluated Text-to-SQL task.
Further testing with larger datasets and diverse database schemas is needed to assess generalization.
## Performance Visualization
![NL2SQL Model Performance](performance_comparison.png)