---
base_model: google/gemma-4-E4B-it
library_name: peft
pipeline_tag: text-generation
tags:
- gemma4
- lora
- peft
- reasoning
- math
- agent
- tool-use
- function-calling
- multimodal
- russian
- unsloth
license: apache-2.0
language:
- ru
- en
datasets:
- open-r1/OpenR1-Math-220k
- nvidia/Nemotron-Post-Training-Dataset-v2
- nvidia/Nemotron-SFT-Agentic-v2
- HuggingFaceTB/smoltalk2
- Vikhrmodels/GrandMaster-PRO-MAX
---
# Cerberus-E4B: Multi-Task Reasoning & Agentic Model
**Cerberus-E4B** is an open-weights, multi-task reasoning and agentic language model fine-tuned on top of Google's [`gemma-4-E4B-it`](https://huggingface.co/google/gemma-4-E4B-it) architecture (~7.5B total parameters with frozen vision and audio towers, 4B active language backbone).
Cerberus is designed for advanced bilingual step-by-step reasoning (English & Russian), precise function calling via native Gemma 4 tool schemas, and versatile dual-channel generation.
Looking for quantized GGUF weights?
👉 **[`Cerberus-E4B-GGUF`](https://huggingface.co/rrlabs-ai/Cerberus-E4B-GGUF)**
---
## Highlights & Capabilities
* **Dual-Channel Generation:** Seamlessly toggles between long, deliberate step-by-step thought traces (`<|channel>thought\n...`) and concise, direct responses (`instant`).
* **Advanced Mathematical Reasoning:** Shows a **+16.7%** accuracy improvement over the Google Gemma 4 base model on challenging competition-level mathematics (MATH-500).
* **Native Tool Calling (Agentic):** Trained with strict adherence to Gemma 4's native function calling protocol (`<|turn>model\n<|tool_call>...<|call:function_name{...}>`), ensuring structured execution for multi-turn agents.
* **Deep Russian Language Alignment:** Native-grade Russian math and logical reasoning without falling back to clumsy machine translation.
* **Multimodal Architecture Compatibility:** Trained with LoRA targeting the language reasoning backbone while keeping vision (ViT SigLIP, 280 soft tokens) and audio (Conformer, 16 kHz) encoders intact for drop-in GGUF `mmproj` support.
---
## Benchmark Results (vs. Base Model)
All evaluations were conducted in greedy / deterministic mode comparing Cerberus-E4B directly against `google/gemma-4-E4B-it`:
| Benchmark | Base Model (`Gemma-4-E4B-it`) | Cerberus-E4B | Delta (vs. Base) | Generation Mode |
| :--- | :---: | :---: | :---: | :--- |
| **MATH-500** | 43.3% | **60.0%** | **+16.7%** | Thinking Mode (``) |
| **MATH-500** | 35.0% | **48.0%** | **+13.0%** | Instant Mode (Direct calculation) |
| **MMLU-Redux** | 62.0% | **63.0%** | **+1.0%** | Multi-discipline academic knowledge |
| **IFEval** | 90.8% | **90.5%** | **Parity (~90%)** | Complex system constraint following |
---
## Quickstart (Hugging Face Transformers)
Cerberus-E4B is released as an official PEFT LoRA adapter for `google/gemma-4-E4B-it`.
```python
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
base_model_id = "google/gemma-4-E4B-it"
adapter_model_id = "rrlabs-ai/Cerberus-E4B"
# Load tokenizer and base model
tokenizer = AutoTokenizer.from_pretrained(base_model_id)
base_model = AutoModelForCausalLM.from_pretrained(
base_model_id,
torch_dtype=torch.bfloat16,
device_map="auto"
)
# Apply Cerberus LoRA adapter
model = PeftModel.from_pretrained(base_model, adapter_model_id)
# 1. Thinking Mode Prompt (Step-by-step reasoning)
prompt = "Find the number of positive integers n <= 1000 such that gcd(n, 36) = 1."
formatted_input = f"<|turn>user\n{prompt}\n<|turn>model\n<|channel>thought\n"
inputs = tokenizer(formatted_input, return_tensors="pt").to(model.device)
outputs = model.generate(
**inputs,
max_new_tokens=2048,
temperature=0.8,
top_p=0.95,
do_sample=True
)
response = tokenizer.decode(outputs[0][inputs.input_ids.shape[1]:], skip_special_tokens=False)
print(response)
```
---
## Recommended Generation Presets
To ensure optimal generation quality across different tasks:
* **For Deliberate Reasoning (`Thinking` mode):**
* Sampling: `do_sample=True`, `temperature=0.8`, `top_p=0.95`, `top_k=64`.
* For `llama.cpp` / GGUF: Enable the dynamic DRY sampler (`--dry-multiplier 0.8 --dry-base 1.75 --dry-allowed-length 2`) to eliminate n-gram loops during long thinking traces.
* **For Direct Answers, Code & Function Calling (`Instant` mode):**
* Sampling: Greedy decoding (`do_sample=False`, `temperature=0.0`) for maximum determinism and syntax reliability.
---
## GGUF & Quantized Models
Looking for standalone quantized GGUF weights for local execution on llama.cpp, LM Studio, or Ollama?
Pre-quantized GGUF files (including `Q4_K_M`, `Q5_K_M`, `Q8_0`, `BF16`, and multimodal `mmproj`) are available at:
👉 **[`rrlabs-ai/Cerberus-E4B-GGUF`](https://huggingface.co/rrlabs-ai/Cerberus-E4B-GGUF)**
---
## 📱 Recommended Web & Mobile Interface: rrchat
Cerberus-E4B pairs seamlessly with **[rrchat](https://github.com/leftik1/rrchat)** — an open-source, mobile-ready AI chat interface:
> **Use your local PC's llama.cpp from your phone.** Clean mobile-ready AI chat interface with vision, web search, and privacy.
* 🌐 **Project Repository:** [github.com/leftik1/rrchat](https://github.com/leftik1/rrchat)
* 📷 **Camera & Vision Ready:** Upload photos or snapshots directly from your mobile camera and get instant multimodal visual analysis from Cerberus-E4B.
* 🔍 **Web Search & Privacy:** Built-in web search capabilities while keeping your inference and chat history 100% private and hosted on your local hardware.
---
## Dataset Attribution & Licensing
Cerberus was trained on a carefully curated, decontaminated multi-source dataset. In accordance with dataset terms and open research best practices, we gratefully acknowledge the following data creators:
1. **NVIDIA Nemotron:**
* Contains synthetic data derived from [`nvidia/Nemotron-Post-Training-Dataset-v2`](https://huggingface.co/datasets/nvidia/Nemotron-Post-Training-Dataset-v2) and [`nvidia/Nemotron-SFT-Agentic-v2`](https://huggingface.co/datasets/nvidia/Nemotron-SFT-Agentic-v2).
* *Attribution Notice:* This work contains content derived from NVIDIA Nemotron Post-Training and SFT Agentic datasets, developed by NVIDIA Corporation and licensed under the NVIDIA Open Synthetic Data License.
2. **Hugging Face Open-R1:**
* Mathematical reasoning traces sourced from [`open-r1/OpenR1-Math-220k`](https://huggingface.co/datasets/open-r1/OpenR1-Math-220k) (Apache-2.0 License).
3. **Hugging Face SmolTalk:**
* General dialogue and structured conversation samples from [`HuggingFaceTB/smoltalk2`](https://huggingface.co/datasets/HuggingFaceTB/smoltalk2) (Apache-2.0 License).
4. **Vikhr Models (GrandMaster):**
* Russian-language STEM and complex instructions sourced from [`Vikhrmodels/GrandMaster-PRO-MAX`](https://huggingface.co/datasets/Vikhrmodels/GrandMaster-PRO-MAX) (Apache-2.0 License).
5. **Base Model:**
* Built on [`google/gemma-4-E4B-it`](https://huggingface.co/google/gemma-4-E4B-it), developed by Google DeepMind and governed by the Gemma Terms of Use.
---
## License
This model adapter is released under the **[Apache-2.0 License](https://www.apache.org/licenses/LICENSE-2.0)**. Use of the underlying base model remains subject to the [Gemma Terms of Use](https://ai.google.dev/gemma/terms).