README.md

---
library_name: peft
base_model: codellama/CodeLlama-7b-Instruct-hf
tags:
- instruction-tuning
- qlora
- code-llama
- text-generation
language:
- en
datasets:
- mingyue0101/prompt_code_parquet
- mingyue0101/prompts_modi
license: apache-2.0
---

# Model Card for codellama-7b-matplotlib-assistant

This model is a fine-tuned version of `codellama/CodeLlama-7b-Instruct-hf` designed to enhance instruction-following capabilities. It was developed as part of a Master's thesis project.

## Model Details

### Model Description

The `codellama-7b-matplotlib-assistant` model is a large language model fine-tuned using the QLoRA (4-bit Quantization + LoRA) technique. The goal of this model was to adapt the base CodeLlama model to better follow user instructions while maintaining its coding and reasoning capabilities.

- **Developed by:** mingyue0101
- **Model type:** Causal Language Model (Fine-tuned with PEFT/LoRA)
- **Language(s) (NLP):** English, Chinese
- **License:** Apache-2.0 (inherited from CodeLlama)
- **Finetuned from model:** codellama/CodeLlama-7b-Instruct-hf

### Model Sources

- **Repository:** https://huggingface.co/mingyue0101/codellama-7b-matplotlib-assistant
- **Dataset:** https://huggingface.co/datasets/mingyue0101/prompt_code_parquet

## Uses

### Direct Use

The model can be used for text generation, code assistance, and general-purpose instruction following. It is particularly suited for tasks where a balance of technical coding knowledge and conversational instruction following is required.

### Out-of-Scope Use

The model should not be used for high-stakes decision-making, generating malicious code, or any application that violates the safety guidelines of the base CodeLlama model.

## Bias, Risks, and Limitations

This model may inherit biases present in the training data or the base model. Since it was fine-tuned on a specific dataset (`parquet02`), it might exhibit limitations when handling domains outside of its training distribution. Users should expect potential hallucinations in complex reasoning tasks.

### Recommendations

Users are encouraged to use safety filters when deploying this model in production and to perform domain-specific evaluation before use.

## How to Get Started with the Model

Use the code below to load the model in 4-bit precision:

```python
import os
import torch
from datasets import load_dataset
from transformers import (
    AutoModelForCausalLM,
    AutoTokenizer,
    BitsAndBytesConfig,
    TrainingArguments,
    pipeline,
    logging,
)
from peft import LoraConfig
from trl import SFTTrainer

# ==========================================
# 1. Global Parameter Configuration
# ==========================================
base_model = "codeparrot/codeparrot"          # Base model ID on Hugging Face
new_dataset = "mingyue0101/prompts_modi"      # Fine-tuning dataset ID
new_model = "codeparrot_ming03"              # Directory name for saving the fine-tuned model

# ==========================================
# 2. Dataset Loading
# ==========================================
dataset = load_dataset(new_dataset, split="train")

# ==========================================
# 3. QLoRA 4-bit Quantization Configuration
# ==========================================
compute_dtype = getattr(torch, "float16")
quant_config = BitsAndBytesConfig(
    load_in_4bit=True,                        # Enable 4-bit quantization storage
    bnb_4bit_quant_type="nf4",                # Use NormalFloat4 for better precision than FP4
    bnb_4bit_compute_dtype=compute_dtype,     # Cast to Float16 during matrix multiplication
    bnb_4bit_use_double_quant=False,          # Disable double quantization
)

# ==========================================
# 4. Load Base Model with Optimizations
# ==========================================
model = AutoModelForCausalLM.from_pretrained(
    base_model,
    quantization_config=quant_config,
    device_map={"": 0}                       # Force load the model onto the first GPU (GPU 0)
)
model.config.use_cache = False                # Must disable KV cache during training to avoid backprop conflicts
model.config.pretraining_tp = 1               # Set tensor parallelism to 1 for single-GPU training

# ==========================================
# 5. Tokenizer Configuration & Alignment
# ==========================================
tokenizer = AutoTokenizer.from_pretrained(base_model, trust_remote_code=True)
tokenizer.pad_token = tokenizer.eos_token     # Causal LMs usually have no pad_token; reuse eos_token
tokenizer.padding_side = "right"              # Pad on the right to maintain proper causal attention masks

# ==========================================
# 6. PEFT (Lora) Adapter Hyperparameters
# ==========================================
peft_params = LoraConfig(
    r=64,                                     # LoRA rank, controlling the number of trainable parameters
    lora_alpha=16,                            # Scaling factor for LoRA weights
    lora_dropout=0.1,                         # Dropout probability to prevent overfitting in the adapter
    bias="none",                              # Do not train bias parameters
    task_type="CAUSAL_LM",                    # Explicitly declare the task type as Causal LM
    fan_in_fan_out="True"
)

# ==========================================
# 7. Training Arguments
# ==========================================
training_params = TrainingArguments(
    output_dir="./results",                   # Output directory for checkpoints and logs
    num_train_epochs=1,                       # Number of training epochs
    per_device_train_batch_size=4,            # Batch size per device during training
    gradient_accumulation_steps=1,            # Number of updates steps to accumulate gradients
    optim="paged_adamw_32bit",                # Use QLoRA paged optimizer to prevent Out-Of-Memory (OOM)
    save_steps=25,                            # Save checkpoint every 25 steps
    logging_steps=25,                         # Log training metrics every 25 steps
    learning_rate=2e-4,                       # Initial learning rate
    weight_decay=0.001,                       # Weight decay coefficient
    fp16=False,                               # Disable standard fp16 (handled by the quantization kernel)
    bf16=False,
    max_grad_norm=0.3,                        # Max gradient norm for gradient clipping
    max_steps=-1,                             # Rely on epochs instead of max_steps to control training length
    warmup_ratio=0.03,                        # Linear warmup ratio over training steps
    group_by_length=True,                     # Group sequences of similar lengths into batches to speed up training
    lr_scheduler_type="constant",             # Learning rate schedule type
    report_to="tensorboard"                   # Use TensorBoard to log training progress
)

# ==========================================
# 8. Start Supervised Fine-Tuning (SFT) & Save
# ==========================================
trainer = SFTTrainer(
    model=model,
    train_dataset=dataset,
    peft_config=peft_params,
    dataset_text_field="column0",             # Name of the column containing text data in the dataset
    max_seq_length=None,                      # Use default maximum sequence length
    tokenizer=tokenizer,
    args=training_params,
    packing=False,                            # Disable sample packing (combining multiple examples into one sequence)
)

# Launch the training process
trainer.train()

# Save the trained LoRA adapter weights and tokenizer files
trainer.model.save_pretrained(new_model)
trainer.tokenizer.save_pretrained(new_model)
print(f"Training complete! Finetuned weights successfully saved to: {new_model}")
```
## Training Details
### Training Data
The model was trained on the `mingyue0101/parquet02` dataset. This dataset contains instruction-response pairs formatted for Supervised Fine-Tuning (SFT).

### Training Procedure
**Training Hyperparameters**
- Training regime: QLoRA 4-bit (NF4) mixed precision (fp16)
- Learning rate: 2e-4
- Optimizer: paged_adamw_32bit
- Batch size: 4
- Epochs: 1
- LoRA Rank (r): 64
- LoRA Alpha: 16
- LoRA Dropout: 0.1
- LR Scheduler: constant
- Warmup Ratio: 0.03

## Technical Specifications
### Model Architecture and Objective
Based on the Llama 2 architecture, this model utilizes grouped-query attention (GQA) and rotary positional embeddings (RoPE), fine-tuned with a causal language modeling objective.

### Compute Infrastructure
### Software

- PEFT 0.10.0
- Transformers
- Bitsandbytes
- TRL (SFTTrainer)