-
upload nemo-chatbot-v2 model
# 🧠 Fine-Tuning Spec Sheet (Gemma 3 4B + LoRA)
## 📌 Base Model
- **Model ID:** `google/gemma-3-4b-it`
- **Model Type:** Causal Language Model
- **Precision:** float32
- **Device Mapping:** `auto`
- **Attention Implementation:** `eager`
- **Tokenizer Padding:** `right`
- **Special Token Handling:**
- `pad_token` is set to `eos_token` if undefined
---
## 🔧 LoRA Configuration
- **r (rank):** 16
- **lora_alpha:** 64
- **lora_dropout:** 0.05
- **bias:** `"none"`
- **target_modules:** `["q_proj", "k_proj", "v_proj", "o_proj", "gate_proj"]`
- **task_type:** `TaskType.CAUSAL_LM`
- **Framework:** PEFT (`get_peft_model`)
---
## 🏋️ SFT Training Configuration
- **output_dir:** `./outputs`
- **per_device_train_batch_size:** 1
- **gradient_accumulation_steps:** 8
- **num_train_epochs:** 4
- **logging_steps:** 1
- **learning_rate:** 2e-5
- **warmup_steps:** 50
- **optim:** `adamw_torch`
- **save_strategy:** `steps`
- **save_steps:** 100
- **report_to:** `"none"` (no external logger)
- **fp… -
initial commit