Update README.md
Browse files
README.md
CHANGED
|
@@ -1,3 +1,53 @@
|
|
| 1 |
---
|
| 2 |
license: apache-2.0
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 3 |
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
---
|
| 2 |
license: apache-2.0
|
| 3 |
+
language:
|
| 4 |
+
- en
|
| 5 |
+
base_model: google/gemma-2
|
| 6 |
+
pipeline_tag: image-text-to-text
|
| 7 |
+
tags:
|
| 8 |
+
- gemma
|
| 9 |
+
- gguf
|
| 10 |
+
- vision
|
| 11 |
+
- llama.cpp
|
| 12 |
+
- quantized
|
| 13 |
---
|
| 14 |
+
|
| 15 |
+
# 🏔️ Winter LLM (Gemma Multimodal GGUF)
|
| 16 |
+
|
| 17 |
+
Welcome to **Winter LLM**, a lightweight, high-performance GGUF build based on Google's Gemma architecture with multimodal (vision + text) capabilities. Designed for fast, efficient local execution on consumer hardware, PCs, and edge setups across any region—from Kigali to anywhere in the world! 🌍✨
|
| 18 |
+
|
| 19 |
+
---
|
| 20 |
+
|
| 21 |
+
## 🚀 Overview
|
| 22 |
+
|
| 23 |
+
* **Base Architecture:** Gemma Multimodal
|
| 24 |
+
* **Format:** Ready-to-run `.gguf`
|
| 25 |
+
* **Features:**
|
| 26 |
+
* 💬 **Text Generation & Chat:** Fast local inference.
|
| 27 |
+
* 👁️ **Vision Capability:** Powered by the included `mmproj` vision adapter for reading and understanding images.
|
| 28 |
+
* ⚡ **Optimized Memory Usage:** Quantized to run efficiently on low-VRAM GPUs or standard CPUs.
|
| 29 |
+
|
| 30 |
+
---
|
| 31 |
+
|
| 32 |
+
## 📂 Included Files
|
| 33 |
+
|
| 34 |
+
| File Name | Size | Purpose |
|
| 35 |
+
| :--- | :--- | :--- |
|
| 36 |
+
| `winter llm.gguf` | ~6 GB | Main LLM model weights & tokenizer |
|
| 37 |
+
| `mmproj-gemma-4-12B-it-QAT-BF16.gguf` | ~100 MB | Multimodal vision projector (for image inputs) |
|
| 38 |
+
|
| 39 |
+
---
|
| 40 |
+
|
| 41 |
+
## 💻 How to Run Locally
|
| 42 |
+
|
| 43 |
+
You can run **Winter LLM** offline using any GGUF-compatible software:
|
| 44 |
+
|
| 45 |
+
### 1. LM Studio / Jan AI
|
| 46 |
+
1. Download both `winter llm.gguf` and the `mmproj-...gguf` file from the **Files** tab.
|
| 47 |
+
2. Drag and drop `winter llm.gguf` into LM Studio.
|
| 48 |
+
3. Attach the `mmproj` projector in settings if you wish to upload images to the chat!
|
| 49 |
+
|
| 50 |
+
### 2. Ollama / llama.cpp
|
| 51 |
+
Run via terminal using `llama.cpp`:
|
| 52 |
+
```bash
|
| 53 |
+
./main -m "winter llm.gguf" --mmproj "mmproj-gemma-4-12B-it-QAT-BF16.gguf" -p "User: Describe this image: [image] Assistant:"
|