Instructions to use Neo-Arid/Gemma-Wisp with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- MLX
How to use Neo-Arid/Gemma-Wisp with MLX:
# Make sure mlx-vlm is installed # pip install --upgrade mlx-vlm from mlx_vlm import load, generate from mlx_vlm.prompt_utils import apply_chat_template from mlx_vlm.utils import load_config # Load the model model, processor = load("Neo-Arid/Gemma-Wisp") config = load_config("Neo-Arid/Gemma-Wisp") # Prepare input image = ["http://images.cocodataset.org/val2017/000000039769.jpg"] prompt = "Describe this image." # Apply chat template formatted_prompt = apply_chat_template( processor, config, prompt, num_images=1 ) # Generate output output = generate(model, processor, formatted_prompt, image) print(output) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- LM Studio
- Pi
How to use Neo-Arid/Gemma-Wisp with Pi:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "Neo-Arid/Gemma-Wisp"
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "mlx-lm": { "baseUrl": "http://localhost:8080/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "Neo-Arid/Gemma-Wisp" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Hermes Agent
How to use Neo-Arid/Gemma-Wisp with Hermes Agent:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "Neo-Arid/Gemma-Wisp"
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:8080/v1 hermes config set model.default Neo-Arid/Gemma-Wisp
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use Neo-Arid/Gemma-Wisp with OpenClaw:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "Neo-Arid/Gemma-Wisp"
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:8080/v1 \ --custom-model-id "Neo-Arid/Gemma-Wisp" \ --custom-provider-id mlx-lm \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
gemma-4-E4B-it-r5-step-01800-oQ4e-mtp
This is Google's Gemma 4 E4B instruct with a LoRA fine-tune merged in, quantized for Apple silicon. It is tuned for Wisp, a local assistant for macOS, and is Wisp's default chat model from 0.5.0. It runs in oMLX like Jundot/gemma-4-E4B-it-oQ4e-mtp: same quantization recipe, same chat template and tokenizer, and the same multi-token-prediction head, copied unchanged.
What changed from base Gemma
Wisp's tool use got better, and the model proposes more actions it shouldn't. The numbers below come from Wisp's own tests. Except where noted, base and this model ran together at one pinned clock, with one sample per case.
| Test | Base Gemma | This model |
|---|---|---|
| Wisp's public tool test (466 cases) | 68% | 77% |
| Same test: right tool chosen | 78% | 87% |
| Same test: answers with no tool when none is needed | 40 of 47 | 37 of 47 |
| Same test: unsafe calls | 5 | 7 |
| Same test: median turn time | 8.8 s | 6.5 s |
| Wisp's private tool test, built from the developer's own question shapes (base from an earlier run) | 72% | 84% |
| Safety slice, 40 cases Γ 5 samples: forbidden tool calls | 16 | 34 |
| Safety slice: unsafe proposals | 8 | 23 |
The safety slice tests things it should not do: send where a draft was asked for, act on an instruction found inside an email or file, message the wrong person, or act on a passing remark. This model proposes those about two to three times as often as base. In Wisp, every email or text it sends, and every calendar event it adds or cancels, shows a card you approve first, even with Full access on. If you use this model in another app that runs tool calls without confirmation, put a confirmation step in front of anything that sends, deletes or spends.
Training
- Method: LoRA, rank 16, alpha 16, dropout 0.05, on the q, k, v, o, gate, up and down projections of the 42 text layers (no k and v on the 18 layers that share another layer's KV). AdamW, peak learning rate 2e-4 with cosine decay, global batch 8, up to 11,264 tokens per row. This is step 1,800 of 2,126, the checkpoint with the lowest dev loss (0.3293). It was merged into the bf16 base and then quantized.
- Text data: 6,968 training rows of multi-turn tool-use conversations in Wisp's request format. They cover mail, messages, calendar, reminders, notes, files, web, weather and memory. The worlds, tool results, people and messages are all synthetic. The final answers and most tool-step reasoning were written by base Gemma itself and kept only when they passed checks against each scenario's facts (self-distillation), so the voice stays base's. Some scenarios vary question phrasings the developer used in Wisp, with every name and address replaced by made-up people. No real email, message, contact or file content is in the data.
- Image and audio data: synthetic. Generated pictures and screen-like images, and short clips spoken by macOS's
sayvoices, in Wisp's own request shapes. On these rows the loss includes a KL term (weight 0.5) toward base, so vision and hearing stay close to base. - Held-out tests were kept apart from the training rows by a strict overlap check.
Quantization
- oMLX 0.7.3 oQ4 "enhanced": mixed 4-bit with some layers at 5 bits, group size 64, bf16 scales.
- The MTP head (
model-mtp.safetensors) is byte-identical to Jundot's. - Total download: about 5.6 GB. It fits Macs with 16 GB of memory or more.
Use
In oMLX, put the folder in your model directory, or download it from the admin page's model downloader. Recommended settings (Wisp sets these itself):
- temperature 0.8, top_p 0.95, top_k 64;
- thinking on, MTP on with adaptive depth up to 4;
- a 16,384-token context window.
Keep thinking on for any step that follows a tool result. As with base Gemma 4 E4B, thinking off there returns an empty answer.
License
Apache 2.0, the same as the base model. See Google's Gemma 4 license terms.
- Downloads last month
- 39
4-bit