README.md

metadata

base_model: Qwen/Qwen2.5-Coder-1.5B-Instruct
library_name: peft
license: apache-2.0
language:
  - en
tags:
  - code
  - python
  - lora
  - peft
  - qwen2
  - code-generation
datasets:
  - iamtarun/python_code_instructions_18k_alpaca

my-python-coder

A LoRA fine-tune of Qwen2.5-Coder-1.5B-Instruct specialized for Python code generation.

This model was fine-tuned as a learning project to demonstrate the full workflow of taking a base model, training it on a custom dataset, and publishing it to the Hugging Face Hub.

Training Details

Parameter	Value
Base model	Qwen/Qwen2.5-Coder-1.5B-Instruct
Dataset	iamtarun/python_code_instructions_18k_alpaca (first 1,500 examples)
Method	LoRA (r=16, alpha=32, target_modules=all-linear)
Training steps	200
Learning rate	2e-4
Effective batch size	8 (batch=2 × grad_accum=4)
Max sequence length	1024
Hardware	Google Colab (NVIDIA T4, 16 GB VRAM)
Training time	~33 minutes

Prompt Format

This model was trained with the following instruction format. Using the same format at inference time will give the best results:

### Instruction:
<your task description>

### Response:
<model's answer>

Usage

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel

# Load base model and LoRA adapter
base_model = AutoModelForCausalLM.from_pretrained(
    "Qwen/Qwen2.5-Coder-1.5B-Instruct",
    dtype=torch.float16,
    device_map="auto",
)
model = PeftModel.from_pretrained(base_model, "SathishKumar89/my-python-coder")
tokenizer = AutoTokenizer.from_pretrained("SathishKumar89/my-python-coder")

# Prepare a prompt
prompt = """### Instruction:
Write a Python function that checks if a number is prime.

### Response:
"""

inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=200, do_sample=False)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Example Output

Prompt:

### Instruction:
Write a Python function that checks if a number is prime.

### Response:

Model output:

def is_prime(num):
    # Check for 0 and 1
    if num <= 1:
        return False

    # Check for even numbers greater than 2
    elif num == 2:
        return True
    elif num % 2 == 0:
        return False

    # Check for odd numbers greater than 3
    else:
        for i in range(3, int(num**0.5) + 1, 2):
            if num % i == 0:
                return False
        return True

Limitations

• Trained on a small subset (1,500 of 18,612 examples) for only 200 steps — this is a proof-of-concept, not a production model.
• May not generalize well to complex Python tasks (large refactors, multi-file projects, advanced libraries).
• Inherits any biases or limitations present in the base model and training dataset.
• Not evaluated against standard benchmarks.

Future Improvements

• Train on the full dataset for multiple epochs
• Increase LoRA rank for greater capacity
• Evaluate on HumanEval or MBPP benchmarks

Acknowledgements

• Base model: Qwen2.5-Coder-1.5B-Instruct by the Qwen team
• Dataset: iamtarun/python_code_instructions_18k_alpaca
• Training framework: Hugging Face transformers, peft, trl