arunb74/Jeff_Geema-GGUF

Jeff-Gemma4-E2B — GGUF (Q4_K_M)

This repository contains an unofficial GGUF quantization of Jeff-Gemma4-E2B, converted for use with llama.cpp and compatible tools.

All credit for the model itself goes to the original authors (ORIGINAL_AUTHOR). I did not train or fine-tune this model. I only converted and quantized their released weights. Please see the original model card for the full details, benchmarks, and intended use.

Files

File	Quantization	Size	Notes
jeff_geema_Q4_K_M.gguf	Q4_K_M (4-bit)	~3.4 GB	Recommended for local use
jeff_geema_f16.gguf	F16 (16-bit, unquantized)	~9.3 GB	Highest quality; slower and needs more memory. Use it as a source for your own quantizations

Important limitations of this GGUF

• Text backbone only. The original release includes a separate readout head (readout.safetensors) and decision_config.json, which the original model uses to return calibrated probabilities for each option in a single forward pass. These files are not included in this GGUF, and llama.cpp does not run that head. This GGUF therefore behaves like a regular Gemma 4 text model (it generates text), not like the original zero-shot decision model.
• If you want the decision/classification behavior described by the original authors, use the original model and its code.
• This is a general-purpose conversion for experimentation. I have only checked that the model loads and generates coherent text. I have not benchmarked the quantized model or compared its quality against the original.

About the original model

Summarized from the original model card. See it for the authoritative description.

The Jeff models are fine-tunes of Qwen3.5 and Gemma 4 for zero-shot classification: small, fast decision models meant to be slotted into code. You describe a situation and list the options in plain words, and the model returns a calibrated probability for each option from a single forward pass, with no generated text to parse. The options can be anything (support queues, user intents, moderation labels, voice commands, game moves), and they do not need to appear in the training data.

According to the original authors, the models are very small, so their reasoning is limited compared with much larger models, and a short fine-tune on your own examples can improve accuracy substantially. The models were built entirely on local hardware, with synthetic training data written by an open model. The project is independent and not affiliated with or endorsed by TypeSafe, the makers of Jev. The training code starts from the open-source AutoJev recipe.

How this GGUF was made

• Source: ORIGINAL_ORG/Jeff-Gemma4-E2B
• Converted with llama.cpp convert_hf_to_gguf.py to F16, then quantized with llama-quantize to Q4_K_M.
• The checkpoint is a text-only Gemma 4 export, so the converter needed small local changes to map its per-layer tensor names. These changes affect tensor naming only, not the weights.
• Tested with llama.cpp build b11476 (Gemma 4 support is recent, so use a current build).

Usage

llama-cli -m jeff_geema_Q4_K_M.gguf -p "Hello, who are you?"

License

This model is a derivative of the original Jeff-Gemma4-E2B, which is built on Gemma 4. The original license and the Gemma terms of use apply. See the LICENSE and NOTICE files and the original repository for the full terms.