nl2sql / performance.md
JayaHareeswarReddy's picture
Update performance.md
ea8aa45 verified
|
Raw History Blame Contribute Delete
1.19 kB

A newer version of the Gradio SDK is available: 6.32.0

Upgrade

NL2SQL – Performance Analysis

1. Model Performance Comparison

The following results compare the original Qwen2.5-Coder-1.5B-Instruct model against the QLoRA fine-tuned model.

Evaluation Metric Base Model Fine-tuned Model Improvement
Exact Match 23.5% 31.0% +7.5 pp
Execution Match 66.7% 71.5% +4.8 pp

2. Performance Visualization

Exact Match

  • Base Model: 23.5%
  • Fine-tuned Model: 31.0%

Execution Match

  • Base Model: 66.7%
  • Fine-tuned Model: 71.5%

3. Key Observations

  1. Fine-tuning improved exact SQL query matching.
  2. Execution Match increased, indicating better query execution performance on the evaluation sample.
  3. QLoRA enabled parameter-efficient adaptation of the base model.
  4. Both metrics showed measurable improvements after fine-tuning.

4. Conclusion

The experiment demonstrates that QLoRA fine-tuning improved the performance of Qwen2.5-Coder-1.5B-Instruct on the evaluated Text-to-SQL task.

Further testing with larger datasets and diverse database schemas is needed to assess generalization.

Performance Visualization

NL2SQL Model Performance