Instructions to use ericrisco/gemma-maia-e4b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- PEFT
How to use ericrisco/gemma-maia-e4b with PEFT:
from peft import PeftModel from transformers import AutoModelForCausalLM base_model = AutoModelForCausalLM.from_pretrained("unsloth/gemma-4-E4B-it-unsloth-bnb-4bit") model = PeftModel.from_pretrained(base_model, "ericrisco/gemma-maia-e4b") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use ericrisco/gemma-maia-e4b with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf ericrisco/gemma-maia-e4b:Q4_K_M # Run inference directly in the terminal: llama cli -hf ericrisco/gemma-maia-e4b:Q4_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf ericrisco/gemma-maia-e4b:Q4_K_M # Run inference directly in the terminal: llama cli -hf ericrisco/gemma-maia-e4b:Q4_K_M
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf ericrisco/gemma-maia-e4b:Q4_K_M # Run inference directly in the terminal: ./build/bin/llama cli -hf ericrisco/gemma-maia-e4b:Q4_K_M
Use Docker
docker model run hf.co/ericrisco/gemma-maia-e4b:Q4_K_M
- Llama new
- LM Studio
- Jan
- vLLM
How to use ericrisco/gemma-maia-e4b with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "ericrisco/gemma-maia-e4b" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "ericrisco/gemma-maia-e4b", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/ericrisco/gemma-maia-e4b:Q4_K_M
- Ollama
How to use ericrisco/gemma-maia-e4b with Ollama:
ollama run hf.co/ericrisco/gemma-maia-e4b:Q4_K_M
- Unsloth Desktop
- Pi
How to use ericrisco/gemma-maia-e4b with Pi:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf ericrisco/gemma-maia-e4b:Q4_K_M
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "llama-cpp": { "baseUrl": "http://localhost:9931/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "ericrisco/gemma-maia-e4b:Q4_K_M" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Docker Model Runner
How to use ericrisco/gemma-maia-e4b with Docker Model Runner:
docker model run hf.co/ericrisco/gemma-maia-e4b:Q4_K_M
- Lemonade
How to use ericrisco/gemma-maia-e4b with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull ericrisco/gemma-maia-e4b:Q4_K_M
Run and chat with the model
lemonade run user.gemma-maia-e4b-Q4_K_M
List all available models
lemonade list
- Hermes Agent
How to use ericrisco/gemma-maia-e4b with Hermes Agent:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf ericrisco/gemma-maia-e4b:Q4_K_M
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:9931/v1 hermes config set model.default ericrisco/gemma-maia-e4b:Q4_K_M
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use ericrisco/gemma-maia-e4b with OpenClaw:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf ericrisco/gemma-maia-e4b:Q4_K_M
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:9931/v1 \ --custom-model-id "ericrisco/gemma-maia-e4b:Q4_K_M" \ --custom-provider-id llama-cpp \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
|
Download README.md from ericrisco/gemma-maia-e4b: direct link, hf CLI and curl.
- Browser
- Download file 5.08 kB
-
https://huggingface.co/ericrisco/gemma-maia-e4b/resolve/main/README.md
- Command line
-
hf download hf://ericrisco/gemma-maia-e4b/README.md
-
curl -L -o README.md https://huggingface.co/ericrisco/gemma-maia-e4b/resolve/main/README.md
5.08 kB
| base_model: unsloth/gemma-4-E4B-it-unsloth-bnb-4bit | |
| library_name: peft | |
| pipeline_tag: text-generation | |
| license: apache-2.0 | |
| language: | |
| - ca | |
| tags: | |
| - maia | |
| - andorra | |
| - lora | |
| - qlora | |
| - unsloth | |
| - sft | |
| # gemma-maia-e4b | |
| LoRA adapter of **maia**, a model specialised in Andorra built from Gemma 4. | |
| Experimental: **not a final model yet**. | |
| This card describes the latest version, **`20261102-1437-e4b-k-canon`**. | |
| Every training run is a tagged version of this repo: | |
| - `20261102-1437-e4b-k-canon` | |
| - `20261028-2208-e4b-k` | |
| - `20261027-2015-e4b-k` | |
| Load a specific version with `revision="<tag>"`. | |
| - Base model: `unsloth/gemma-4-E4B-it-unsloth-bnb-4bit` | |
| - Data: variant **K-canon** of the `training-data` export | |
| (conversations in Andorran Catalan) | |
| - Code: [github.com/ericrisco/maia](https://github.com/ericrisco/maia), with a | |
| [training guide](https://github.com/ericrisco/maia/blob/main/training/README.md) | |
| - The whole process, including what failed: | |
| [how maia was trained](https://ericrisco.github.io/maia-bench/training.html) | |
| ## Evaluation | |
| Both models were run as Q4_K_M GGUF through Ollama, at temperature 0, under the same conditions. | |
| **Fact recall** on 338 held-out questions about facts of the augmented canon (asked in wording | |
| never seen in training; a correct answer must contain the expected keywords): | |
| | | Gemma 4 E4B (base) | maia (`20261102-1437-e4b-k-canon`) | | |
| |---|---:|---:| | |
| | Facts recalled | 25 / 338 (7 %) | **121 / 338 (36 %)** | | |
| maia recalls 103 facts the base model misses and misses 7 the base model knew. | |
| **[maia-bench](https://github.com/ericrisco/maia-bench)** (926 questions about Andorra, judge | |
| Gemini 3.8 Flash; the benchmark questions and the articles it reserves are never trained on): | |
| | Question type | Gemma 4 E4B (base) | maia | Difference (95 % CI, paired bootstrap) | | |
| |---|---:|---:|---| | |
| | All | 30.3 % | **32.9 %** | +2.6 pts [−0.6, +5.9] | | |
| | Multiple choice (604) | 42.4 % | 46.2 % | +3.8 pts [−1.0, +8.6] | | |
| | Open (231) | 9.1 % | 9.3 % | +0.2 pts [−3.2, +3.7] | | |
| | Trap (91) | 3.8 % | 5.0 % | +1.1 pts [−3.8, +6.0] | | |
| maia improves on every question type and regresses on none, but the benchmark gain alone is not | |
| statistically significant (p ≈ 0.05): the benchmark deliberately asks about facts that are not in | |
| the training data, so most of the gain shows up in fact recall. | |
| ## Training | |
| QLoRA with Unsloth on 1 NVIDIA L4. The loss is computed on the model's responses only. | |
| | Parameter | Value | | |
| |---|---| | |
| | LoRA r / alpha | 32 / 32 | | |
| | Epochs | 2 | | |
| | Learning rate | 0.0002 (cosine, 3 % warmup) | | |
| | Effective batch | 16 (4 x 4) | | |
| | Max length | 2048 tokens | | |
| | Chat template | `gemma-4` | | |
| | Seed | 3407 | | |
| Final validation loss: **1.6060** | |
| | Step | Epoch | Validation loss | | |
| |---:|---:|---:| | |
| | 200 | 0.11 | 1.8957 | | |
| | 400 | 0.22 | 1.7958 | | |
| | 600 | 0.33 | 1.7506 | | |
| | 800 | 0.45 | 1.7123 | | |
| | 1000 | 0.56 | 1.6821 | | |
| | 1200 | 0.67 | 1.6637 | | |
| | 1400 | 0.78 | 1.6394 | | |
| | 1600 | 0.89 | 1.6271 | | |
| | 1800 | 1.00 | 1.6266 | | |
| | 2000 | 1.12 | 1.6423 | | |
| | 2200 | 1.23 | 1.6382 | | |
| | 2400 | 1.34 | 1.6218 | | |
| | 2600 | 1.45 | 1.6203 | | |
| | 2800 | 1.56 | 1.6143 | | |
| | 3000 | 1.67 | 1.6076 | | |
| | 3200 | 1.78 | 1.6036 | | |
| | 3400 | 1.90 | 1.6059 | | |
| | 3586 | 2.00 | 1.6060 | | |
| ## System message | |
| The model was always trained with this system message (in Catalan). Use it at inference time too: | |
| > Ets maia, un assistent que coneix a fons Andorra: la seva història, les institucions, la geografia, la cultura i la gent. Respons en català, amb naturalitat i amb la parla d'Andorra. | |
| ## Samples | |
| Generated at the end of training (greedy decoding). Unedited and not fact-checked: **some of these | |
| answers are wrong or invented**, so do not take them as facts about Andorra. | |
| ## Què és el Consell de la Terra? | |
| És el nom que el Consell General tenia abans de la Constitució del 1993. | |
| ## Quines són les set parròquies d'Andorra? | |
| Canillo, Encamp, Ordino, la Massana, Andorra la Vella, Sant Julià de Lòria i Escaldes-Engordany. | |
| ## Explica'm què era la Querimònia. | |
| Era un acte de gràcia que el Consell General concedia a un veí que havia fet una feina útil per a la comunitat. No era un salari: el Consell pagava una quantitat simbòlica, i el beneficiari podia fer servir aquella quantitat per pagar el seu propi deute. Per això, la recompensa ajudava a alliberar-lo d’una obligació, no pas a guanyar diners per a ell. | |
| ## Què es menja per Sant Jordi a Andorra? | |
| Per Sant Jordi es menja la coca massegada, una coca dolça amb llet, farina, sucre, ous i mantega. | |
| ## Com va anar la reforma constitucional del 1993? | |
| El Consell General va aprovar la Constitució el 2 de febrer de 1993, i el poble la va ratificar en referèndum el 14 de març. | |
| ## Usage | |
| ```python | |
| from unsloth import FastModel | |
| model, tokenizer = FastModel.from_pretrained("ericrisco/gemma-maia-e4b", revision="20261102-1437-e4b-k-canon", load_in_4bit=True) | |
| ``` | |
| Quantized GGUF (Q4_K_M) for llama.cpp or Ollama: `maia-e4b-Q4_K_M.gguf`. With Ollama, use a | |
| Modelfile with `FROM ./maia-e4b-Q4_K_M.gguf`, `RENDERER gemma4` and `PARSER gemma4`, and keep the | |
| system message above. | |