File size: 1,185 Bytes
a4fb495
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
ea8aa45
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
# NL2SQL – Performance Analysis

## 1. Model Performance Comparison

The following results compare the original Qwen2.5-Coder-1.5B-Instruct model against the QLoRA fine-tuned model.

| Evaluation Metric | Base Model | Fine-tuned Model | Improvement |
|---|---:|---:|---:|
| Exact Match | 23.5% | 31.0% | +7.5 pp |
| Execution Match | 66.7% | 71.5% | +4.8 pp |

## 2. Performance Visualization

### Exact Match
- Base Model: 23.5%
- Fine-tuned Model: 31.0%

### Execution Match
- Base Model: 66.7%
- Fine-tuned Model: 71.5%

## 3. Key Observations

1. Fine-tuning improved exact SQL query matching.
2. Execution Match increased, indicating better query execution performance on the evaluation sample.
3. QLoRA enabled parameter-efficient adaptation of the base model.
4. Both metrics showed measurable improvements after fine-tuning.

## 4. Conclusion

The experiment demonstrates that QLoRA fine-tuning improved the performance of Qwen2.5-Coder-1.5B-Instruct on the evaluated Text-to-SQL task.

Further testing with larger datasets and diverse database schemas is needed to assess generalization.

## Performance Visualization

![NL2SQL Model Performance](performance_comparison.png)