AaronTekle commited on
Commit
caecdc8
·
verified ·
1 Parent(s): c8c87e0

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +4 -5
README.md CHANGED
@@ -15,11 +15,11 @@ tags:
15
  - fine-tuned
16
  - qwen
17
  ---
18
- # SparkQwen (Spark-QLoRA, [Qwen/Qwen3-1.7B](https://huggingface.co/Qwen/Qwen3-1.7B) )
19
 
20
  QLoRA fine-tuning of [**Qwen/Qwen3-1.7B**](https://huggingface.co/Qwen/Qwen3-1.7B) for generating Spark/PySpark code.
21
 
22
- SparkQwen is a lightweight coding language model for day-to-day data engineering tasks, including DataFrame manipulations, joins, Spark SQL, structured streaming, Delta Lake operations, optimization techniques, MLlib, and more.
23
 
24
  given:
25
 
@@ -213,11 +213,10 @@ question = (
213
 
214
  # Define coding assistant behavior
215
  system_prompt = (
216
- "You are a Spark/PySpark data engineering coding assistant. "
217
- "Return only executable Spark/PySpark code with necessary imports. "
218
  "Prefer Spark DataFrame APIs over pandas and RDDs. "
219
  "Use the provided schemas and names without inventing tables or columns. "
220
- "Do not add Markdown fences or prose."
221
  )
222
 
223
  messages = [
 
15
  - fine-tuned
16
  - qwen
17
  ---
18
+ # PySparkQwen (PySparkQwen-QLoRA, [Qwen/Qwen3-1.7B](https://huggingface.co/Qwen/Qwen3-1.7B) )
19
 
20
  QLoRA fine-tuning of [**Qwen/Qwen3-1.7B**](https://huggingface.co/Qwen/Qwen3-1.7B) for generating Spark/PySpark code.
21
 
22
+ PySparkQwen is a lightweight coding language model for day-to-day data engineering tasks, including DataFrame manipulations, joins, Spark SQL, structured streaming, Delta Lake operations, optimization techniques, MLlib, and more.
23
 
24
  given:
25
 
 
213
 
214
  # Define coding assistant behavior
215
  system_prompt = (
216
+ "You are a PySpark data engineering coding assistant. "
217
+ "Return executable PySpark code with necessary imports. "
218
  "Prefer Spark DataFrame APIs over pandas and RDDs. "
219
  "Use the provided schemas and names without inventing tables or columns. "
 
220
  )
221
 
222
  messages = [