--- library_name: peft base_model: codellama/CodeLlama-7b-Instruct-hf tags: - instruction-tuning - qlora - code-llama - text-generation language: - en datasets: - mingyue0101/prompt_code_parquet - mingyue0101/prompts_modi license: apache-2.0 --- # Model Card for codellama-7b-matplotlib-assistant This model is a fine-tuned version of `codellama/CodeLlama-7b-Instruct-hf` designed to enhance instruction-following capabilities. It was developed as part of a Master's thesis project. ## Model Details ### Model Description The `codellama-7b-matplotlib-assistant` model is a large language model fine-tuned using the QLoRA (4-bit Quantization + LoRA) technique. The goal of this model was to adapt the base CodeLlama model to better follow user instructions while maintaining its coding and reasoning capabilities. - **Developed by:** mingyue0101 - **Model type:** Causal Language Model (Fine-tuned with PEFT/LoRA) - **Language(s) (NLP):** English, Chinese - **License:** Apache-2.0 (inherited from CodeLlama) - **Finetuned from model:** codellama/CodeLlama-7b-Instruct-hf ### Model Sources - **Repository:** https://huggingface.co/mingyue0101/codellama-7b-matplotlib-assistant - **Dataset:** https://huggingface.co/datasets/mingyue0101/prompt_code_parquet ## Uses ### Direct Use The model can be used for text generation, code assistance, and general-purpose instruction following. It is particularly suited for tasks where a balance of technical coding knowledge and conversational instruction following is required. ### Out-of-Scope Use The model should not be used for high-stakes decision-making, generating malicious code, or any application that violates the safety guidelines of the base CodeLlama model. ## Bias, Risks, and Limitations This model may inherit biases present in the training data or the base model. Since it was fine-tuned on a specific dataset (`parquet02`), it might exhibit limitations when handling domains outside of its training distribution. Users should expect potential hallucinations in complex reasoning tasks. ### Recommendations Users are encouraged to use safety filters when deploying this model in production and to perform domain-specific evaluation before use. ## How to Get Started with the Model Use the code below to load the model in 4-bit precision: ```python import os import torch from datasets import load_dataset from transformers import ( AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig, TrainingArguments, pipeline, logging, ) from peft import LoraConfig from trl import SFTTrainer # ========================================== # 1. Global Parameter Configuration # ========================================== base_model = "codeparrot/codeparrot" # Base model ID on Hugging Face new_dataset = "mingyue0101/prompts_modi" # Fine-tuning dataset ID new_model = "codeparrot_ming03" # Directory name for saving the fine-tuned model # ========================================== # 2. Dataset Loading # ========================================== dataset = load_dataset(new_dataset, split="train") # ========================================== # 3. QLoRA 4-bit Quantization Configuration # ========================================== compute_dtype = getattr(torch, "float16") quant_config = BitsAndBytesConfig( load_in_4bit=True, # Enable 4-bit quantization storage bnb_4bit_quant_type="nf4", # Use NormalFloat4 for better precision than FP4 bnb_4bit_compute_dtype=compute_dtype, # Cast to Float16 during matrix multiplication bnb_4bit_use_double_quant=False, # Disable double quantization ) # ========================================== # 4. Load Base Model with Optimizations # ========================================== model = AutoModelForCausalLM.from_pretrained( base_model, quantization_config=quant_config, device_map={"": 0} # Force load the model onto the first GPU (GPU 0) ) model.config.use_cache = False # Must disable KV cache during training to avoid backprop conflicts model.config.pretraining_tp = 1 # Set tensor parallelism to 1 for single-GPU training # ========================================== # 5. Tokenizer Configuration & Alignment # ========================================== tokenizer = AutoTokenizer.from_pretrained(base_model, trust_remote_code=True) tokenizer.pad_token = tokenizer.eos_token # Causal LMs usually have no pad_token; reuse eos_token tokenizer.padding_side = "right" # Pad on the right to maintain proper causal attention masks # ========================================== # 6. PEFT (Lora) Adapter Hyperparameters # ========================================== peft_params = LoraConfig( r=64, # LoRA rank, controlling the number of trainable parameters lora_alpha=16, # Scaling factor for LoRA weights lora_dropout=0.1, # Dropout probability to prevent overfitting in the adapter bias="none", # Do not train bias parameters task_type="CAUSAL_LM", # Explicitly declare the task type as Causal LM fan_in_fan_out="True" ) # ========================================== # 7. Training Arguments # ========================================== training_params = TrainingArguments( output_dir="./results", # Output directory for checkpoints and logs num_train_epochs=1, # Number of training epochs per_device_train_batch_size=4, # Batch size per device during training gradient_accumulation_steps=1, # Number of updates steps to accumulate gradients optim="paged_adamw_32bit", # Use QLoRA paged optimizer to prevent Out-Of-Memory (OOM) save_steps=25, # Save checkpoint every 25 steps logging_steps=25, # Log training metrics every 25 steps learning_rate=2e-4, # Initial learning rate weight_decay=0.001, # Weight decay coefficient fp16=False, # Disable standard fp16 (handled by the quantization kernel) bf16=False, max_grad_norm=0.3, # Max gradient norm for gradient clipping max_steps=-1, # Rely on epochs instead of max_steps to control training length warmup_ratio=0.03, # Linear warmup ratio over training steps group_by_length=True, # Group sequences of similar lengths into batches to speed up training lr_scheduler_type="constant", # Learning rate schedule type report_to="tensorboard" # Use TensorBoard to log training progress ) # ========================================== # 8. Start Supervised Fine-Tuning (SFT) & Save # ========================================== trainer = SFTTrainer( model=model, train_dataset=dataset, peft_config=peft_params, dataset_text_field="column0", # Name of the column containing text data in the dataset max_seq_length=None, # Use default maximum sequence length tokenizer=tokenizer, args=training_params, packing=False, # Disable sample packing (combining multiple examples into one sequence) ) # Launch the training process trainer.train() # Save the trained LoRA adapter weights and tokenizer files trainer.model.save_pretrained(new_model) trainer.tokenizer.save_pretrained(new_model) print(f"Training complete! Finetuned weights successfully saved to: {new_model}") ``` ## Training Details ### Training Data The model was trained on the `mingyue0101/parquet02` dataset. This dataset contains instruction-response pairs formatted for Supervised Fine-Tuning (SFT). ### Training Procedure **Training Hyperparameters** - Training regime: QLoRA 4-bit (NF4) mixed precision (fp16) - Learning rate: 2e-4 - Optimizer: paged_adamw_32bit - Batch size: 4 - Epochs: 1 - LoRA Rank (r): 64 - LoRA Alpha: 16 - LoRA Dropout: 0.1 - LR Scheduler: constant - Warmup Ratio: 0.03 ## Technical Specifications ### Model Architecture and Objective Based on the Llama 2 architecture, this model utilizes grouped-query attention (GQA) and rotary positional embeddings (RoPE), fine-tuned with a causal language modeling objective. ### Compute Infrastructure ### Software - PEFT 0.10.0 - Transformers - Bitsandbytes - TRL (SFTTrainer)