gemma-4-e2b-it-lite-mlx

Text-only, lightweight 4-bit MLX quantization of google/gemma-4-e2b-it, optimized for on-device inference on iPhone (8GB devices) and Apple Silicon Macs.

~2.05 GB โ€” compared to 3.58 GB for the official mlx-community/gemma-4-e2b-it-4bit (which bundles the audio/vision towers in BF16).

What was changed (modification notice)

  • Converted from the original bf16 checkpoint with mlx-lm 0.31.3 (mlx_lm.convert).
  • Text-only: audio_tower, vision_tower, embed_audio, embed_vision, multi_modal_projector weights are not included. This checkpoint works with text-only Gemma 4 runtimes (e.g. mlx-lm / mlx-swift-lm MLXLLM).
  • Quantization recipe: 4-bit / group size 64 (affine) for all layers, except embed_tokens_per_layer (per-layer embeddings, ~1.3 GB at 4-bit) which is quantized to 2-bit / group size 64. The per-layer override is recorded in config.json (quantization section).
  • KV-shared layers (15-34) do not include k_proj / v_proj / k_norm (same layout as the official MLX conversion).

Japanese smoke tests (greeting / Q&A / no repetition loops) show quality on par with the official 4-bit conversion on this recipe. A uniform 3-bit recipe of the same size was clearly worse and was rejected.

Attribution

Gemma 4 is developed by Google DeepMind and released under the Apache License 2.0. This repository redistributes a quantized derivative under the same license.

Usage (mlx-lm)

pip install mlx-lm
mlx_lm.generate --model mouri45/gemma-4-e2b-it-lite-mlx --prompt "ใ“ใ‚“ใซใกใฏ!"

Created as part of the AppleSiliconLLM project (Issue0012).

Downloads last month
23
Safetensors
Model size
5B params
Tensor type
U32
ยท
BF16
ยท
MLX
Hardware compatibility
Log In to add your hardware

4-bit

Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support