Spaces:
Running on Zero
Running on Zero
|
Download performance.md from JayaHareeswarReddy/nl2sql: direct link, hf CLI and curl.
- Browser
- Download file 1.19 kB
-
https://huggingface.co/spaces/JayaHareeswarReddy/nl2sql/resolve/main/performance.md
- Command line
-
hf download hf://spaces/JayaHareeswarReddy/nl2sql/performance.md
-
curl -L -o performance.md https://huggingface.co/spaces/JayaHareeswarReddy/nl2sql/resolve/main/performance.md
1.19 kB
A newer version of the Gradio SDK is available: 6.32.0
NL2SQL – Performance Analysis
1. Model Performance Comparison
The following results compare the original Qwen2.5-Coder-1.5B-Instruct model against the QLoRA fine-tuned model.
| Evaluation Metric | Base Model | Fine-tuned Model | Improvement |
|---|---|---|---|
| Exact Match | 23.5% | 31.0% | +7.5 pp |
| Execution Match | 66.7% | 71.5% | +4.8 pp |
2. Performance Visualization
Exact Match
- Base Model: 23.5%
- Fine-tuned Model: 31.0%
Execution Match
- Base Model: 66.7%
- Fine-tuned Model: 71.5%
3. Key Observations
- Fine-tuning improved exact SQL query matching.
- Execution Match increased, indicating better query execution performance on the evaluation sample.
- QLoRA enabled parameter-efficient adaptation of the base model.
- Both metrics showed measurable improvements after fine-tuning.
4. Conclusion
The experiment demonstrates that QLoRA fine-tuning improved the performance of Qwen2.5-Coder-1.5B-Instruct on the evaluated Text-to-SQL task.
Further testing with larger datasets and diverse database schemas is needed to assess generalization.
