gemma-4-E4B-it-BaseQuant-V1

A quantized build of google/gemma-4-E4B-it. It is closer to the BF16 model than the unsloth quant of the same size.

Size	baseRT bundle	Same size as
Medium	`gemma-4-E4B-it-BaseQuant-V1-Medium.base` (4.98 GB of quantized text weights)	unsloth `Q4_K_M` (4.98 GB)

The bundle also carries the vision and audio towers at 16-bit (0.96 GB; 5.97 GB file). The size comparison is text weights against text weights: unsloth's GGUFs hold the text model only.

Usage

hf download basecompute/gemma-4-E4B-it-BaseQuant-V1 gemma-4-E4B-it-BaseQuant-V1-Medium.base --local-dir .
basert serve ./gemma-4-E4B-it-BaseQuant-V1-Medium.base

Results

KL divergence of each quant's next-token distribution from the BF16 model. Lower is closer to BF16. The percentage over each pair is our change against unsloth.

Public prompts

The prompts come from seven public datasets (OpenHands, SWE-agent, Glaive function calling, OpenOrca, Tulu 3, OpenHermes, UltraChat), published as basecompute/quant-fidelity-eval. The BF16 model writes each continuation, and every quant is scored along it. BaseQuant is 81% closer to BF16 than unsloth's quant of the same size, and closer on 139 of the 140 prompts.

Source	unsloth Q4_K_M (4.98 GB)	unsloth UD-Q4_K_XL (5.13 GB)	BaseQuant Medium (4.98 GB)	Change vs Q4_K_M
OpenHands	0.03348	0.02431	0.00391	-88%
SWE-agent	0.03219	0.02290	0.00506	-84%
Glaive function calling	0.03453	0.02990	0.00609	-82%
OpenOrca	0.03141	0.02451	0.00561	-82%
Tulu 3	0.02444	0.02085	0.00450	-82%
OpenHermes	0.02445	0.01978	0.00498	-80%
UltraChat	0.04787	0.03760	0.01003	-79%
All seven	0.03191	0.02551	0.00597	-81%
All but UltraChat	0.02807	0.02261	0.00499	-82%

Scored along the dataset's own text instead of the model's continuations, BaseQuant is 77% closer than Q4_K_M and 69% closer than UD-Q4_K_XL:

Dataset text	unsloth Q4_K_M (4.98 GB)	unsloth UD-Q4_K_XL (5.13 GB)	BaseQuant Medium (4.98 GB)	Change vs Q4_K_M
OpenHands	0.03237	0.02386	0.00589	-82%
SWE-agent	0.02503	0.01755	0.00422	-83%
Glaive function calling	0.03132	0.01973	0.00948	-70%
Tulu 3	0.01768	0.01452	0.00540	-69%
OpenHermes	0.03764	0.02977	0.00947	-75%
UltraChat	0.05279	0.03862	0.01172	-78%
All six	0.03194	0.02394	0.00734	-77%

Mean KL over every scored token; each source is position-weighted. BaseQuant runs in baseRT and unsloth's quants in llama.cpp, all against the same BF16 references. Bootstrapping over prompts (10,000 resamples), BaseQuant's KL is 0.19x unsloth Q4_K_M's [0.18, 0.20] on the model's continuations and 0.23x [0.21, 0.26] on dataset text.

Evaluation sets

On the model's own continuations and on written text, BaseQuant is 73-88% closer to BF16 than unsloth's quants, including UD-Q4_K_XL, which is 150 MB larger.

Evaluation set	unsloth Q4_K_M (4.98 GB)	unsloth UD-Q4_K_XL (5.13 GB)	BaseQuant Medium (4.98 GB)
Model's own continuations	0.02097	0.01614	0.00339
Written text	0.03336	0.02654	0.00721
Tool use	0.01276	0.00996	0.00175
Agentic coding	0.02462	0.01774	0.00294
Top-1 agreement, 32-token greedy continuations	97.31%	98.19%	99.07%