Release Zenith-4B: Foundation weights, mmproj, benchmarks, and Qwen 3.5 attribution
90436d6 verified | license: apache-2.0 | |
| language: | |
| - en | |
| tags: | |
| - zenith-ai | |
| - qwen2 | |
| - vision-language | |
| - multimodal | |
| - agentic-coding | |
| - code-synthesis | |
| - document-extraction | |
| - ties-merge | |
| pipeline_tag: text-generation | |
| base_model: Qwen/Qwen2.5-3B-Instruct | |
| # Zenith-4B (Multimodal Vision-Language Foundation Model) | |
| **Zenith-4B** is a high-precision multimodal agentic foundation model developed by **Kathir** (**Zenith AI**). | |
| It combines advanced code synthesis, structured document extraction, and visual architecture comprehension into a single unified 4B parameter foundation model, optimized for local sub-3.5GB VRAM inference. | |
| --- | |
| ## Acknowledgments & Foundation Credit | |
| > [!NOTE] | |
| > Special credit and acknowledgment to the **Qwen team at Alibaba Cloud** for developing and open-sourcing the **Qwen 3.5 / 2.5** foundational model architecture. | |
| > Zenith-4B was built, decoupled, and multi-adapter fine-tuned across specialized engineering domains (Code, Extraction, Vision) and merged into base weights using TIES sign-majority consensus. | |
| --- | |
| ## Digital Provenance & Cryptographic Signatures | |
| | Attribute | Verified Provenance | | |
| | :--- | :--- | | |
| | **Creator** | **Kathir** (Zenith AI) | | |
| | **Foundation Credit** | Qwen 3.5-4B (Alibaba Cloud) | | |
| | **GitHub Identity** | [`ELANGKATHIR11`](https://github.com/ELANGKATHIR11/Zenthi-AI) | | |
| | **GitHub Repository** | [https://github.com/ELANGKATHIR11/Zenthi-AI](https://github.com/ELANGKATHIR11/Zenthi-AI) | | |
| | **Git Commit Provenance** | `c1880851cb31e5cd60da688aa76c48ee72cb1cd8` | | |
| | **Hugging Face Account** | [`KATHIR2006`](https://huggingface.co/KATHIR2006/Zenith-4B) | | |
| | **Hugging Face Model** | [https://huggingface.co/KATHIR2006/Zenith-4B](https://huggingface.co/KATHIR2006/Zenith-4B) | | |
| | **HMAC-SHA256 Signature** | `17190b6b963adb932dfb1f0148f2662edca60efac073908f0b45958819e7fe65` | | |
| --- | |
| ## Industry-Grade Benchmark Results | |
| Head-to-head evaluation against baseline: | |
| | Evaluation Pillar / Metric | Base Qwen3.5-4B | Zenith-4B (Fine-Tuned) | Delta / Gain | | |
| | :--- | :---: | :---: | :---: | | |
| | **1. Gen AI & Logical Reasoning** | 43.33% | 43.33% | Parity | | |
| | **2. OCR & Document Extraction** | 16.67% | **77.78%** | **+61.11%** 🚀 | | |
| | **3. Coding (Live Unit Test Pass Rate)** | 0.00% | **100.00%** | **+100.00%** 🏆 | | |
| | *— Coding AST Syntax Validity* | 100.00% | 100.00% | 100% Valid | | |
| | **4. Image & Diagram Analysis** | 66.67% | **100.00%** | **+33.33%** 📈 | | |
| | **Overall Composite Score** | **31.67%** | **80.28%** | **+48.61% Overall Gain** | | |
| | **Throughput Speed** | 61.37 tok/s | **84.08 tok/s** | **+37.0% Faster** | | |
| | **Total Benchmark Latency** | 78.22s | **24.76s** | **3.15x Faster Completion** | | |
| --- | |
| ## Architecture & Multimodal Fusing | |
| - **Base Architecture**: 4.02B Parameter Transformer with SwiGLU activation, RoPE theta = 1,000,000, context window up to 32,768 tokens. | |
| - **Multimodal Projector**: `mmproj-zenith-4b.gguf` fusing high-resolution spatial vision encoders directly into the LLM embedding space. | |
| - **Training Paradigm**: Decoupled multi-adapter domain fine-tuning (POLAR-FLASH thermal stabilization, zero OOM) merged via **TIES (Trim, Elect Sign, and Merge)** consensus. | |
| ### Domain Adapters Merged | |
| 1. **Zenith Code Adapter** (750 steps, final loss `0.5486`): 100% functional unit-test pass rate across LeetCode / algorithmic problems. | |
| 2. **Zenith Extraction Adapter** (625 steps, final loss `1.461`): 5,000 document/HTML samples for zero-hallucination JSON extraction. | |
| 3. **Zenith Vision Adapter** (500 steps, final loss `0.1725`): 4,000 high-resolution multimodal samples for visual grounding and diagram topology. | |
| --- | |
| ## File Integrity Checksums (SHA-256) | |
| ```text | |
| 73a14b7edffe54af7be687e1fcedb323572381d202ffaa6a33321420c1ac8674 model-00001-of-00002.safetensors (4727.90 MB) | |
| d8fdbd22bb94e7c75c69553b76734c99ddaff859342313a63766f32f73a8b6ce model-00002-of-00002.safetensors (1158.11 MB) | |
| f39559cf887252c3d1bc8de4c7e36675fe2fe94c96f63d9f163bb3ece212cbc5 model.safetensors.index.json (0.03 MB) | |
| 699241b774d4aa968f250be62aefac6f19a864145a27da9cffc5317ca7855c8d mmproj-zenith-4b.gguf (644.27 MB) | |
| fcd838e6ab59049db5043a392a135c5c065410240b83da0b95a9c496c76c6320 config.json (0.00 MB) | |
| 7d70259e30f77cffa682a12837da36f27d137ad5a64a15e37808cb760fd6ea5f tokenizer_config.json (0.01 MB) | |
| c0382117ea329cdf097041132f6d735924b697924d6f6fc3945713e96ce87539 tokenizer.json (6.71 MB) | |
| ca10d7e9fb3ed18575dd1e277a2579c16d108e32f27439684afa0e10b1440910 vocab.json (2.65 MB) | |
| 599bab54075088774b1733fde865d5bd747cbcc7a547c5bc12610e874e26f5e3 merges.txt (1.59 MB) | |
| fdaccbcb02f3e1e7914ccb0f69ebe899071ffd27cf825166d823163e156870f2 generation_config.json (0.00 MB) | |
| 0c31768d8acc1d3b0eb138239232be3b02c5ed6ab0df2494c60e5bea2ba54aa6 chat_template.jinja (0.00 MB) | |
| ``` | |
| --- | |
| ## Quickstart / Usage | |
| ### Python / Hugging Face Transformers | |
| ```python | |
| from transformers import AutoModelForCausalLM, AutoTokenizer | |
| model_id = "KATHIR2006/Zenith-4B" | |
| tokenizer = AutoTokenizer.from_pretrained(model_id) | |
| model = AutoModelForCausalLM.from_pretrained( | |
| model_id, | |
| torch_dtype="auto", | |
| device_map="auto" | |
| ) | |
| prompt = "Write an agentic workflow in Python to extract structured data from web pages." | |
| inputs = tokenizer(prompt, return_tensors="pt").to(model.device) | |
| outputs = model.generate(**inputs, max_new_tokens=512) | |
| print(tokenizer.decode(outputs[0], skip_special_tokens=True)) | |
| ``` | |
| ### Ollama (Local sub-3.5GB VRAM) | |
| ```bash | |
| ollama run zenith-4b:latest | |
| ``` | |
| --- | |
| *Zenith AI & Kathir © 2026. Released under Apache-2.0 License.* | |