README.md

---
license: apache-2.0
library_name: mlx
pipeline_tag: text-generation
base_model:
- google/gemma-4-E4B-it
tags:
- mlx
- gemma4
- text-only
- 4-bit
language:
- en
- ar
- ckb
---

# Gemma 4 E4B Text for MLX (4-bit)

[google/gemma-4-E4B-it](https://huggingface.co/google/gemma-4-E4B-it) as a **text-only** model for Apple's [MLX](https://github.com/ml-explore/mlx), compressed to 4 bits. It runs on device on Apple silicon: Mac, and iPhone or iPad with 8 GB of memory.

Gemma 4 E4B can also read images and audio. This version keeps only the language model, so it is **4.2 GB instead of 5.2 GB** and writes exactly the same text.

All credit for the model goes to Google. This repository only changes its file format.

## At a glance

| Detail | Value |
|---|---|
| Base model | google/gemma-4-E4B-it (instruction-tuned) |
| Input and output | Text only |
| Version | 4-bit: weights compressed from 16 to 4 bits (affine, groups of 64) |
| Size | 4.2 GB (the multimodal 4-bit MLX version is 5.2 GB) |
| Model type | `gemma4_text` |
| Runtime | [mlx-lm](https://github.com/ml-explore/mlx-lm) (Python) or [mlx-swift-lm](https://github.com/ml-explore/mlx-swift-lm) |
| License | Apache 2.0, the same as the original model |

## Usage

```bash
pip install -U mlx-lm
```

```python
from mlx_lm import load, generate

model, tokenizer = load("UIDUser-NSB/Gemma4-E4B-Text-MLX")
messages = [{"role": "user", "content": "Summarize this meeting in three points: ..."}]
prompt = tokenizer.apply_chat_template(messages, add_generation_prompt=True, tokenize=False, enable_thinking=False)
print(generate(model, tokenizer, prompt=prompt, max_tokens=512))
```

Pass `enable_thinking=False` for a direct answer. Without it, Gemma 4 writes its reasoning first.

## Check

With greedy decoding, this model and [mlx-community/gemma-4-e4b-it-4bit](https://huggingface.co/mlx-community/gemma-4-e4b-it-4bit) (the multimodal version) write the same text for the same prompt.

## How it was converted

1. **Source:** google/gemma-4-E4B-it, revision `ee0ef6023621cff504d758262d4e04895a5af4a2`.
2. **Text only:** the language model's weights are kept and renamed from `model.language_model.*` to `model.*`. The vision and audio encoders and their embedders are removed.
3. **Config:** `model_type` is `gemma4_text`. The text model's settings (`text_config`) are also written at the top level, where mlx-lm and mlx-swift-lm read them for this model type.
4. **Compression:** `python -m mlx_lm convert -q --q-bits 4 --q-group-size 64` with mlx-lm 0.32.0.

## Credits

Gemma 4 by Google DeepMind. See the [original model card](https://huggingface.co/google/gemma-4-E4B-it) for training data, evaluations, intended use and limitations.