--- language: - id - en license: gemma library_name: transformers base_model: google/gemma-3n-E4B-it tags: - gemma - gemma-3n - unsloth - lora - fine-tuned - indonesian - english - text-generation --- # Orbit-4-Gemma-E4B-it ## Model Description Orbit-4-Gemma-E4B-it is a language model fine-tuned from Gemma 3n E4B-it, developed to enhance comprehension and knowledge capabilities in the Indonesian language. The fine-tuning focuses on strengthening the model's ability to follow instructions in Indonesian, understand local context, and generate more natural and relevant responses. The training process utilized a large-scale Indonesian instruction dataset and spanned a full epoch without sampling. This model retains the multimodal architecture of the Gemma 3n base model, meaning it structurally still supports text, image, audio, and video processing. However, the fine-tuning process was performed exclusively on the text layers, so multimodal performance was not specifically reinforced through this training. ## Model Specifications | Item | Detail | |---|---| | Base model | google/gemma-3n-E4B-it | | Fine-tuning method | LoRA (r=16, alpha=16) | | Epoch | 1 full epoch (approx. 29,807 steps) | | Effective batch size | 4 (batch size 1 x gradient accumulation 4) | | Learning rate | 2e-4 (linear schedule) | | Warmup | 3% of total steps | | Max sequence length | 2048 tokens | | Optimizer | adamw_8bit | | Weight decay | 0.01 | | LoRA dropout | 0 | | Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | | Merge precision | float32 (performed on CPU) | | Training hardware | Tesla T4 16GB | | Framework | Unsloth, TRL, Transformers | | Prompt format | Gemma 3n chat template | | Seed | 3407 | ### Prompt Format The model uses the Gemma 3n chat template as follows: ```text user {instruction} model {output} ``` ## Model Capabilities The model possesses the following capabilities: * Comprehension of Indonesian-language instructions, including complex and multi-step instructions * General knowledge and local Indonesian context, covering history, culture, geography, and everyday topics * Generation of natural Indonesian text for various needs, such as explanations, summaries, and narratives * Creative writing tasks in Indonesian, such as poetry (*puisi* and *pantun*), and short stories * Translation between Indonesian and English * Sentiment analysis and Indonesian text processing * Basic mathematical reasoning and simple problem-solving * Inherent multimodal capabilities (text, image, audio, video input; text output), although not reinforced through this fine-tuning ### Limitations * The model is optimized for Indonesian text; performance on other languages or non-text modalities may decrease compared to the base model * The model has not been comprehensively tested for multimodal tasks within the Indonesian context * The model may produce hallucinations; verification of critical facts remains necessary * The model has not undergone additional safety tuning; deployment in production applications should be accompanied by supervision * Maximum context is 2048 tokens in accordance with training settings ## Credits * Base model: Google DeepMind (Gemma 3n) * Fine-tuning framework: Unsloth * Dataset: AksaraLLM * Fine-tuning and training: OpenPathAI ## License This model follows Google's Gemma license. Commercial use is subject to the Gemma Terms of Use.