Spaces:
Running on Zero
Running on Zero
|
Download performance.md from JayaHareeswarReddy/nl2sql: direct link, hf CLI and curl.
- Browser
- Download file 1.19 kB
-
https://huggingface.co/spaces/JayaHareeswarReddy/nl2sql/resolve/main/performance.md
- Command line
-
hf download hf://spaces/JayaHareeswarReddy/nl2sql/performance.md
-
curl -L -o performance.md https://huggingface.co/spaces/JayaHareeswarReddy/nl2sql/resolve/main/performance.md
1.19 kB
| # NL2SQL – Performance Analysis | |
| ## 1. Model Performance Comparison | |
| The following results compare the original Qwen2.5-Coder-1.5B-Instruct model against the QLoRA fine-tuned model. | |
| | Evaluation Metric | Base Model | Fine-tuned Model | Improvement | | |
| |---|---:|---:|---:| | |
| | Exact Match | 23.5% | 31.0% | +7.5 pp | | |
| | Execution Match | 66.7% | 71.5% | +4.8 pp | | |
| ## 2. Performance Visualization | |
| ### Exact Match | |
| - Base Model: 23.5% | |
| - Fine-tuned Model: 31.0% | |
| ### Execution Match | |
| - Base Model: 66.7% | |
| - Fine-tuned Model: 71.5% | |
| ## 3. Key Observations | |
| 1. Fine-tuning improved exact SQL query matching. | |
| 2. Execution Match increased, indicating better query execution performance on the evaluation sample. | |
| 3. QLoRA enabled parameter-efficient adaptation of the base model. | |
| 4. Both metrics showed measurable improvements after fine-tuning. | |
| ## 4. Conclusion | |
| The experiment demonstrates that QLoRA fine-tuning improved the performance of Qwen2.5-Coder-1.5B-Instruct on the evaluated Text-to-SQL task. | |
| Further testing with larger datasets and diverse database schemas is needed to assess generalization. | |
| ## Performance Visualization | |
|  |