Instructions to use innerloop-dev/gemma3-4b-text with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use innerloop-dev/gemma3-4b-text with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf innerloop-dev/gemma3-4b-text:Q4_K_M # Run inference directly in the terminal: llama cli -hf innerloop-dev/gemma3-4b-text:Q4_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf innerloop-dev/gemma3-4b-text:Q4_K_M # Run inference directly in the terminal: llama cli -hf innerloop-dev/gemma3-4b-text:Q4_K_M
Use pre-built binary
# Download pre-built binary from: # https://github.com/ggerganov/llama.cpp/releases # Start a local OpenAI-compatible server with a web UI: ./llama-server -hf innerloop-dev/gemma3-4b-text:Q4_K_M # Run inference directly in the terminal: ./llama-cli -hf innerloop-dev/gemma3-4b-text:Q4_K_M
Build from source code
git clone https://github.com/ggerganov/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-server llama-cli # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama-server -hf innerloop-dev/gemma3-4b-text:Q4_K_M # Run inference directly in the terminal: ./build/bin/llama-cli -hf innerloop-dev/gemma3-4b-text:Q4_K_M
Use Docker
docker model run hf.co/innerloop-dev/gemma3-4b-text:Q4_K_M
- LM Studio
- Jan
- vLLM
How to use innerloop-dev/gemma3-4b-text with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "innerloop-dev/gemma3-4b-text" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "innerloop-dev/gemma3-4b-text", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/innerloop-dev/gemma3-4b-text:Q4_K_M
- Ollama
How to use innerloop-dev/gemma3-4b-text with Ollama:
ollama run hf.co/innerloop-dev/gemma3-4b-text:Q4_K_M
- Unsloth Desktop
- Docker Model Runner
How to use innerloop-dev/gemma3-4b-text with Docker Model Runner:
docker model run hf.co/innerloop-dev/gemma3-4b-text:Q4_K_M
- Lemonade
How to use innerloop-dev/gemma3-4b-text with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull innerloop-dev/gemma3-4b-text:Q4_K_M
Run and chat with the model
lemonade run user.gemma3-4b-text-Q4_K_M
List all available models
lemonade list
- Atomic Chat
File size: 6,523 Bytes
31153ca 5a78c55 31153ca | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 | ---
license: gemma
license_name: gemma-terms-of-use
license_link: https://ai.google.dev/gemma/terms
base_model: google/gemma-3-4b-it
pipeline_tag: text-generation
library_name: gguf
language:
- en
tags:
- gemma3
- gguf
- text-only
- ollama
- breadcrumb
---
Google's Gemma 3 4B, as Ollama builds it, with the picture-reading part taken out.
The text weights are the same bytes. It writes text. It cannot look at an image.
## What it is
This is Ollama's own `gemma3:4b` — the Q4_K_M build of `google/gemma-3-4b-it` — with the
vision tower and the multimodal projection removed and nothing else changed.
- Source file: the model layer of `gemma3:4b`,
sha256 `aeda25e63ebd698fab8638ffb778e68bed908b960d39d0becc650fa981609d25`,
3,338,792,448 bytes.
- What came out: 439 tensors whose names begin `v.` (the SigLIP vision tower) or `mm.`
(the multimodal input projection), and 9 metadata keys — `gemma3.mm.tokens_per_image`
and the eight `gemma3.vision.*` keys.
- What stayed: all 444 text tensors, byte for byte. Same names, same ggml quantization
types, same shapes, same bytes. Nothing was requantized, converted, retrained or
fine-tuned. The tokenizer is untouched, all 262,145 entries, so token numbers do not
move and the model writes the same words.
- What was added: one metadata key, `general.description`, carrying the notice that this
file is a modified Gemma file.
- Result: 2,498,332,864 bytes,
sha256 `199388f8f8cbec06b80bd63b0b1a774103e00c993c107bf1d480e58047420532`.
The template and the sampling parameters are the ones Ollama ships with `gemma3:4b`,
unchanged: stop `<end_of_turn>`, temperature 1, top_k 64, top_p 0.95.
## Why
[Breadcrumb](https://innerloop.works/breadcrumb), by Innerloop, uses this model to give screens, chapters and meetings short names. It only
ever sends it text. The vision tower was 840 MB of weights that were read off disk,
paged in and held in memory for work that never happened.
Measured on the same machine, one model resident at a time, with `footprint` against the
running `llama-server`:
| | on disk | held while answering |
|---|---|---|
| `gemma3:4b` | 3.3 GB | 4.28 GB |
| this model | 2.5 GB | 2.73 GB |
That is about 1.6 GB less memory while it is working, and 0.8 GB less disk. On an 8 GB
Mac that is the difference between naming a screen and swapping.
The names it writes are the same names. On 300 pinned items from a real store — 150
screen moments, 75 chapters, 75 meetings — this file and `gemma3:4b` wrote 300 identical
titles out of 300, at the same settings, and both were stable across a second pass.
## It cannot see
This is a text-only model. There is no vision tower in the file and no projector layer in
the manifest. `ollama show` reports one capability, `completion`, where `gemma3:4b`
reports `completion` and `vision`. If you send it an image it has nothing to look at.
If you need Gemma 3 to read pictures, use `gemma3:4b` instead.
## Gemma notice
Gemma is provided under and subject to the Gemma Terms of Use found at
ai.google.dev/gemma/terms
This distribution contains a modified Gemma file. The full Gemma Terms of Use, as last
modified April 1, 2026, travel with the model: the LICENSE file in this repository is that complete copy, and NOTICE carries the modification notice.
Use of this model is subject to Google's Gemma Prohibited Use Policy at
ai.google.dev/gemma/prohibited_use_policy. That policy is part of the Gemma Terms and
applies to you whether you got this model from us or from anywhere else. If you pass this
model on, or anything you build from it, you pass these terms on with it and you tell the
people you pass it to that the Gemma use restrictions apply.
Google claims no rights in what you generate with it.
Everything in this distribution that is not Gemma is © Innerloop, [innerloop.works](https://innerloop.works).
## How to verify it yourself
You do not have to take our word that only the vision parts came out. The check needs
nothing but a GGUF reader.
1. Get Ollama's `gemma3:4b` and find its model layer in your blob store. It should hash
to `aeda25e63ebd698fab8638ffb778e68bed908b960d39d0becc650fa981609d25`.
2. Get this model and find its model layer. It should hash to
`199388f8f8cbec06b80bd63b0b1a774103e00c993c107bf1d480e58047420532` and be
2,498,332,864 bytes.
3. Open both with a GGUF reader and list the tensors. The original has 883 tensors and
this one has 444 tensors. Every one of the 439 missing tensors has a name beginning
`v.` or `mm.`. Nothing else is missing.
4. The 444 that remain are `token_embd.weight`, `output_norm.weight`, and thirteen
tensors for each of the 34 blocks: `attn_q.weight`, `attn_k.weight`, `attn_v.weight`,
`attn_output.weight`, `attn_q_norm.weight`, `attn_k_norm.weight`, `attn_norm.weight`,
`post_attention_norm.weight`, `ffn_gate.weight`, `ffn_up.weight`, `ffn_down.weight`,
`ffn_norm.weight`, `post_ffw_norm.weight`. By quantization: 205 Q4_K, 34 Q6_K,
205 F32, the same counts as in the original's text half.
5. For each of those 444, compare the ggml type, the shape, the byte count and the
sha256 of the tensor's own bytes against the same tensor in the original. All 444
match. Any single mismatch means the weights were touched, and they were not.
6. The metadata: 27 keys here against 35 in the original. The 9 that are gone are the
vision keys listed above. One key is new, `general.description`, which says the file
was modified. `general.architecture` is still `gemma3`, and the tokenizer keys are
identical.
The script that does the surgery lives in a private repository, so the recipe is written
out above rather than linked. It is short enough to re-implement: copy every tensor whose
name does not start with `v.` or `mm.`, copy every metadata key that does not start with
`gemma3.vision.` or `gemma3.mm.`, add `general.description`, write the file. Then hash
what you wrote. If you get `199388f8…` you have reproduced it exactly.
## What we did not do
No fine-tuning. No requantization. No distillation. No change to the tokenizer, the chat
template or the sampling defaults. No system prompt. This is the same model, minus a part
it was not being asked to use.
## Get it
- Ollama: `ollama run innerloop/gemma3-4b-text` (the same file, published at https://ollama.com/innerloop/gemma3-4b-text)
- This repository: the GGUF is `gemma-3-4b-it-text-Q4_K_M.gguf`, 2,498,332,864 bytes, sha256 `199388f8f8cbec06b80bd63b0b1a774103e00c993c107bf1d480e58047420532`.
|