File size: 1,270 Bytes
7a29883 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 | ---
license: apache-2.0
base_model: google/gemma-4-E2B-it-qat-q4_0-unquantized
pipeline_tag: text-generation
tags:
- gemma4
- int8
- cpu
---
# Gemma 4 E2B int8 for gemma4.c
This is the ready-to-run model file for [gemma4.c](https://github.com/ryanssenn/gemma4.c), a small educational implementation of Gemma 4 E2B inference on CPU.
The project's `exporter.py` converts the original [Google Gemma 4 E2B QAT checkpoint](https://huggingface.co/google/gemma-4-E2B-it-qat-q4_0-unquantized) into one 4.7 GiB file containing the model configuration, tokenizer, and weights. During export, matrix weights are quantized to int8 with FP16 scales.
## Download
```bash
hf download QmogAI/gemma4-e2b-int8 gemma4-E2B-int8.bin --local-dir .
```
## Run
```bash
git clone https://github.com/ryanssenn/gemma4.c
cd gemma4.c
make
./run -m ./gemma4-E2B-int8.bin -t 1.0 -n 256 "Why is the sky blue?"
```
## Export your own
If you would rather create the file from the original checkpoint:
```bash
python3 -m pip install -r requirements.txt
python3 exporter.py /path/to/gemma-4-E2B-it-qat-q4_0-unquantized -o ./gemma4-E2B-int8.bin
```
The runtime is made for x86-64 CPUs with AVX2 and FMA. AVX-512 VNNI is used when available. It is intended for learning and experimentation.
|