README.md

---
base_model: Qwen/Qwen2.5-Coder-1.5B-Instruct
library_name: peft
license: apache-2.0
language:
- en
tags:
- code
- python
- lora
- peft
- qwen2
- code-generation
datasets:
- iamtarun/python_code_instructions_18k_alpaca
---

# my-python-coder

A LoRA fine-tune of [Qwen2.5-Coder-1.5B-Instruct](https://huggingface.co/Qwen/Qwen2.5-Coder-1.5B-Instruct) specialized for Python code generation.

This model was fine-tuned as a learning project to demonstrate the full workflow of taking a base model, training it on a custom dataset, and publishing it to the Hugging Face Hub.

## Training Details

| Parameter | Value |
|---|---|
| **Base model** | `Qwen/Qwen2.5-Coder-1.5B-Instruct` |
| **Dataset** | `iamtarun/python_code_instructions_18k_alpaca` (first 1,500 examples) |
| **Method** | LoRA (r=16, alpha=32, target_modules=`all-linear`) |
| **Training steps** | 200 |
| **Learning rate** | 2e-4 |
| **Effective batch size** | 8 (batch=2 × grad_accum=4) |
| **Max sequence length** | 1024 |
| **Hardware** | Google Colab (NVIDIA T4, 16 GB VRAM) |
| **Training time** | ~33 minutes |

## Prompt Format

This model was trained with the following instruction format. Using the same format at inference time will give the best results:

```
### Instruction:
<your task description>

### Response:
<model's answer>
```

## Usage

```python
import torch
from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel

# Load base model and LoRA adapter
base_model = AutoModelForCausalLM.from_pretrained(
    "Qwen/Qwen2.5-Coder-1.5B-Instruct",
    dtype=torch.float16,
    device_map="auto",
)
model = PeftModel.from_pretrained(base_model, "SathishKumar89/my-python-coder")
tokenizer = AutoTokenizer.from_pretrained("SathishKumar89/my-python-coder")

# Prepare a prompt
prompt = """### Instruction:
Write a Python function that checks if a number is prime.

### Response:
"""

inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=200, do_sample=False)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
```

## Example Output

**Prompt:**
```
### Instruction:
Write a Python function that checks if a number is prime.

### Response:
```

**Model output:**
```python
def is_prime(num):
    # Check for 0 and 1
    if num <= 1:
        return False

    # Check for even numbers greater than 2
    elif num == 2:
        return True
    elif num % 2 == 0:
        return False

    # Check for odd numbers greater than 3
    else:
        for i in range(3, int(num**0.5) + 1, 2):
            if num % i == 0:
                return False
        return True
```

## Limitations

- Trained on a **small subset** (1,500 of 18,612 examples) for only 200 steps — this is a proof-of-concept, not a production model.
- May not generalize well to complex Python tasks (large refactors, multi-file projects, advanced libraries).
- Inherits any biases or limitations present in the base model and training dataset.
- Not evaluated against standard benchmarks.

## Future Improvements

- Train on the full dataset for multiple epochs
- Increase LoRA rank for greater capacity
- Evaluate on HumanEval or MBPP benchmarks

## Acknowledgements

- Base model: [Qwen2.5-Coder-1.5B-Instruct](https://huggingface.co/Qwen/Qwen2.5-Coder-1.5B-Instruct) by the Qwen team
- Dataset: [iamtarun/python_code_instructions_18k_alpaca](https://huggingface.co/datasets/iamtarun/python_code_instructions_18k_alpaca)
- Training framework: Hugging Face `transformers`, `peft`, `trl`
```