Upload README.md with huggingface_hub

README.md

@@ -0,0 +1,146 @@
+---
+base_model: google/gemma-4-E4B-it
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- gemma4
+- lora
+- peft
+- reasoning
+- math
+- agent
+- tool-use
+- function-calling
+- multimodal
+- russian
+- unsloth
+license: apache-2.0
+language:
+- ru
+- en
+datasets:
+- open-r1/OpenR1-Math-220k
+- nvidia/Nemotron-Post-Training-Dataset-v2
+- nvidia/Nemotron-SFT-Agentic-v2
+- HuggingFaceTB/smoltalk2
+- Vikhrmodels/GrandMaster-PRO-MAX
+---
+
+<div align="center">
+  <img src="cerberusbanner.png" alt="Cerberus 4B Banner" width="100%" style="border-radius: 10px; margin-bottom: 20px;">
+</div>
+
+# Cerberus 4B: Multi-Task Reasoning & Agentic Model (Preview)
+
+**Cerberus 4B** is an open-weights, multi-task reasoning and agentic language model fine-tuned on top of Google's [`gemma-4-E4B-it`](https://huggingface.co/google/gemma-4-E4B-it) architecture (~7.5B total parameters with frozen vision and audio towers, 4B active language backbone). 
+
+Cerberus is designed for advanced bilingual step-by-step reasoning (English & Russian), precise function calling via native Gemma 4 tool schemas, and versatile dual-channel generation.
+
+---
+
+## Highlights & Capabilities
+
+* **Dual-Channel Generation:** Seamlessly toggles between long, deliberate step-by-step thought traces (`<|channel>thought\n...<channel|>`) and concise, direct responses (`instant`).
+* **Advanced Mathematical Reasoning:** Shows a **+16.7%** accuracy improvement over the Google Gemma 4 base model on challenging competition-level mathematics (MATH-500).
+* **Native Tool Calling (Agentic):** Trained with strict adherence to Gemma 4's native function calling protocol (`<|turn>model\n<|tool_call>...<|call:function_name{...}>`), ensuring structured execution for multi-turn agents.
+* **Deep Russian Language Alignment:** Native-grade Russian math and logical reasoning without falling back to clumsy machine translation.
+* **Multimodal Architecture Compatibility:** Trained with LoRA targeting the language reasoning backbone while keeping vision (ViT SigLIP, 280 soft tokens) and audio (Conformer, 16 kHz) encoders intact for drop-in GGUF `mmproj` support.
+
+---
+
+## Benchmark Results (vs. Base Model)
+
+All evaluations were conducted in greedy / deterministic mode comparing Cerberus 4B directly against `google/gemma-4-E4B-it`:
+
+| Benchmark | Base Model (`Gemma-4-E4B-it`) | Cerberus 4B | Delta (vs. Base) | Generation Mode |
+| :--- | :---: | :---: | :---: | :--- |
+| **MATH-500** | 43.3% | **60.0%** | **+16.7%** | Thinking Mode (`<thought>`) |
+| **MATH-500** | 35.0% | **48.0%** | **+13.0%** | Instant Mode (Direct calculation) |
+| **MMLU-Redux** | 62.0% | **63.0%** | **+1.0%** | Multi-discipline academic knowledge |
+| **IFEval** | 90.8% | **90.5%** | **Parity (~90%)** | Complex system constraint following |
+
+---
+
+## Quickstart (Hugging Face Transformers)
+
+Cerberus 4B is released as an official PEFT LoRA adapter for `google/gemma-4-E4B-it`.
+
+```python
+import torch
+from transformers import AutoModelForCausalLM, AutoTokenizer
+from peft import PeftModel
+
+base_model_id = "google/gemma-4-E4B-it"
+adapter_model_id = "rrchat/cerberus-4b-preview"
+
+# Load tokenizer and base model
+tokenizer = AutoTokenizer.from_pretrained(base_model_id)
+base_model = AutoModelForCausalLM.from_pretrained(
+    base_model_id,
+    torch_dtype=torch.bfloat16,
+    device_map="auto"
+)
+
+# Apply Cerberus LoRA adapter
+model = PeftModel.from_pretrained(base_model, adapter_model_id)
+
+# 1. Thinking Mode Prompt (Step-by-step reasoning)
+prompt = "Find the number of positive integers n <= 1000 such that gcd(n, 36) = 1."
+formatted_input = f"<|turn>user\n{prompt}<turn|>\n<|turn>model\n<|channel>thought\n"
+
+inputs = tokenizer(formatted_input, return_tensors="pt").to(model.device)
+outputs = model.generate(
+    **inputs,
+    max_new_tokens=2048,
+    temperature=0.8,
+    top_p=0.95,
+    do_sample=True
+)
+
+response = tokenizer.decode(outputs[0][inputs.input_ids.shape[1]:], skip_special_tokens=False)
+print(response)
+```
+
+---
+
+## Recommended Generation Presets
+
+To ensure optimal generation quality across different tasks:
+
+* **For Deliberate Reasoning (`Thinking` mode):**
+  * Sampling: `do_sample=True`, `temperature=0.8`, `top_p=0.95`, `top_k=64`.
+  * For `llama.cpp` / GGUF: Enable the dynamic DRY sampler (`--dry-multiplier 0.8 --dry-base 1.75 --dry-allowed-length 2`) to eliminate n-gram loops during long thinking traces.
+* **For Direct Answers, Code & Function Calling (`Instant` mode):**
+  * Sampling: Greedy decoding (`do_sample=False`, `temperature=0.0`) for maximum determinism and syntax reliability.
+
+---
+
+## GGUF & Quantized Models
+
+Looking for standalone quantized GGUF weights for local execution on llama.cpp, LM Studio, or Ollama?
+Pre-quantized GGUF files (including `Q4_K_M`, `Q5_K_M`, `Q8_0`, `BF16`, and multimodal `mmproj`) are available at:
+👉 **[`rrchat/Cerberus-4B-Preview-GGUF`](https://huggingface.co/rrchat/Cerberus-4B-Preview-GGUF)**
+
+---
+
+## Dataset Attribution & Licensing
+
+Cerberus was trained on a carefully curated, decontaminated multi-source dataset. In accordance with dataset terms and open research best practices, we gratefully acknowledge the following data creators:
+
+1. **NVIDIA Nemotron:**
+   * Contains synthetic data derived from [`nvidia/Nemotron-Post-Training-Dataset-v2`](https://huggingface.co/datasets/nvidia/Nemotron-Post-Training-Dataset-v2) and [`nvidia/Nemotron-SFT-Agentic-v2`](https://huggingface.co/datasets/nvidia/Nemotron-SFT-Agentic-v2).
+   * *Attribution Notice:* This work contains content derived from NVIDIA Nemotron Post-Training and SFT Agentic datasets, developed by NVIDIA Corporation and licensed under the NVIDIA Open Synthetic Data License.
+2. **Hugging Face Open-R1:**
+   * Mathematical reasoning traces sourced from [`open-r1/OpenR1-Math-220k`](https://huggingface.co/datasets/open-r1/OpenR1-Math-220k) (Apache-2.0 License).
+3. **Hugging Face SmolTalk:**
+   * General dialogue and structured conversation samples from [`HuggingFaceTB/smoltalk2`](https://huggingface.co/datasets/HuggingFaceTB/smoltalk2) (Apache-2.0 License).
+4. **Vikhr Models (GrandMaster):**
+   * Russian-language STEM and complex instructions sourced from [`Vikhrmodels/GrandMaster-PRO-MAX`](https://huggingface.co/datasets/Vikhrmodels/GrandMaster-PRO-MAX) (Apache-2.0 License).
+5. **Base Model:**
+   * Built on [`google/gemma-4-E4B-it`](https://huggingface.co/google/gemma-4-E4B-it), developed by Google DeepMind and governed by the Gemma Terms of Use.
+
+---
+
+## License
+
+This model adapter is released under the **[Apache-2.0 License](https://www.apache.org/licenses/LICENSE-2.0)**. Use of the underlying base model remains subject to the [Gemma Terms of Use](https://ai.google.dev/gemma/terms).