iabru commited on
Commit
03e57d6
·
verified ·
1 Parent(s): 1873f45

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +50 -0
README.md CHANGED
@@ -1,3 +1,53 @@
1
  ---
2
  license: apache-2.0
 
 
 
 
 
 
 
 
 
 
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  license: apache-2.0
3
+ language:
4
+ - en
5
+ base_model: google/gemma-2
6
+ pipeline_tag: image-text-to-text
7
+ tags:
8
+ - gemma
9
+ - gguf
10
+ - vision
11
+ - llama.cpp
12
+ - quantized
13
  ---
14
+
15
+ # 🏔️ Winter LLM (Gemma Multimodal GGUF)
16
+
17
+ Welcome to **Winter LLM**, a lightweight, high-performance GGUF build based on Google's Gemma architecture with multimodal (vision + text) capabilities. Designed for fast, efficient local execution on consumer hardware, PCs, and edge setups across any region—from Kigali to anywhere in the world! 🌍✨
18
+
19
+ ---
20
+
21
+ ## 🚀 Overview
22
+
23
+ * **Base Architecture:** Gemma Multimodal
24
+ * **Format:** Ready-to-run `.gguf`
25
+ * **Features:**
26
+ * 💬 **Text Generation & Chat:** Fast local inference.
27
+ * 👁️ **Vision Capability:** Powered by the included `mmproj` vision adapter for reading and understanding images.
28
+ * ⚡ **Optimized Memory Usage:** Quantized to run efficiently on low-VRAM GPUs or standard CPUs.
29
+
30
+ ---
31
+
32
+ ## 📂 Included Files
33
+
34
+ | File Name | Size | Purpose |
35
+ | :--- | :--- | :--- |
36
+ | `winter llm.gguf` | ~6 GB | Main LLM model weights & tokenizer |
37
+ | `mmproj-gemma-4-12B-it-QAT-BF16.gguf` | ~100 MB | Multimodal vision projector (for image inputs) |
38
+
39
+ ---
40
+
41
+ ## 💻 How to Run Locally
42
+
43
+ You can run **Winter LLM** offline using any GGUF-compatible software:
44
+
45
+ ### 1. LM Studio / Jan AI
46
+ 1. Download both `winter llm.gguf` and the `mmproj-...gguf` file from the **Files** tab.
47
+ 2. Drag and drop `winter llm.gguf` into LM Studio.
48
+ 3. Attach the `mmproj` projector in settings if you wish to upload images to the chat!
49
+
50
+ ### 2. Ollama / llama.cpp
51
+ Run via terminal using `llama.cpp`:
52
+ ```bash
53
+ ./main -m "winter llm.gguf" --mmproj "mmproj-gemma-4-12B-it-QAT-BF16.gguf" -p "User: Describe this image: [image] Assistant:"