Text Generation
PEFT
Safetensors
English
pyspark
apache-spark
qwen2.5-coder
qlora
lora
code-generation
data-engineering
fine-tuned
Instructions to use AaronTekle/PySparkQwen with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- PEFT
How to use AaronTekle/PySparkQwen with PEFT:
from peft import PeftModel from transformers import AutoModelForCausalLM base_model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-Coder-1.5B-Instruct") model = PeftModel.from_pretrained(base_model, "AaronTekle/PySparkQwen") - Notebooks
- Google Colab
- Kaggle
Update README.md
Browse files
README.md
CHANGED
|
@@ -1,3 +1,20 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
# SparkQwen (Spark-QLoRA, [Qwen/Qwen3-1.7B](https://huggingface.co/Qwen/Qwen3-1.7B) )
|
| 2 |
|
| 3 |
QLoRA fine-tuning of [**Qwen/Qwen3-1.7B**](https://huggingface.co/Qwen/Qwen3-1.7B) for generating Spark/PySpark code.
|
|
@@ -216,4 +233,4 @@ print(generated_code)
|
|
| 216 |
- [Hu, E. J. et al. LoRA: Low-Rank Adaptation of Large Language Models (2021)](https://arxiv.org/abs/2106.09685)
|
| 217 |
- [Dettmers, T. et al. QLoRA: Efficient Finetuning of Quantized LLMs (2023)](https://arxiv.org/abs/2305.14314)
|
| 218 |
- [Hugging Face. Parameter-Efficient Fine-Tuning](https://huggingface.co/docs/transformers/peft)
|
| 219 |
-
- [Hugging Face. PEFT LoRA](https://huggingface.co/docs/peft/en/package_reference/lora)
|
|
|
|
| 1 |
+
---
|
| 2 |
+
language:
|
| 3 |
+
- en
|
| 4 |
+
base_model:
|
| 5 |
+
- Qwen/Qwen3-1.7B
|
| 6 |
+
pipeline_tag: text-generation
|
| 7 |
+
library_name: peft
|
| 8 |
+
tags:
|
| 9 |
+
- pyspark
|
| 10 |
+
- apache-spark
|
| 11 |
+
- qwen3
|
| 12 |
+
- qlora
|
| 13 |
+
- lora
|
| 14 |
+
- code-generation
|
| 15 |
+
- fine-tuned
|
| 16 |
+
- qwen
|
| 17 |
+
---
|
| 18 |
# SparkQwen (Spark-QLoRA, [Qwen/Qwen3-1.7B](https://huggingface.co/Qwen/Qwen3-1.7B) )
|
| 19 |
|
| 20 |
QLoRA fine-tuning of [**Qwen/Qwen3-1.7B**](https://huggingface.co/Qwen/Qwen3-1.7B) for generating Spark/PySpark code.
|
|
|
|
| 233 |
- [Hu, E. J. et al. LoRA: Low-Rank Adaptation of Large Language Models (2021)](https://arxiv.org/abs/2106.09685)
|
| 234 |
- [Dettmers, T. et al. QLoRA: Efficient Finetuning of Quantized LLMs (2023)](https://arxiv.org/abs/2305.14314)
|
| 235 |
- [Hugging Face. Parameter-Efficient Fine-Tuning](https://huggingface.co/docs/transformers/peft)
|
| 236 |
+
- [Hugging Face. PEFT LoRA](https://huggingface.co/docs/peft/en/package_reference/lora)
|