Instructions to use Aurumdev95/medx-gemma4-e4b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use Aurumdev95/medx-gemma4-e4b with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf Aurumdev95/medx-gemma4-e4b:BF16
# Run inference directly in the terminal:
llama cli -hf Aurumdev95/medx-gemma4-e4b:BF16

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf Aurumdev95/medx-gemma4-e4b:BF16
# Run inference directly in the terminal:
llama cli -hf Aurumdev95/medx-gemma4-e4b:BF16

Build from source code

git clone https://github.com/ggml-org/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-app
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama serve -hf Aurumdev95/medx-gemma4-e4b:BF16
# Run inference directly in the terminal:
./build/bin/llama cli -hf Aurumdev95/medx-gemma4-e4b:BF16

Use Docker

docker model run hf.co/Aurumdev95/medx-gemma4-e4b:BF16

Llama new
LM Studio
Jan

vLLM

How to use Aurumdev95/medx-gemma4-e4b with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Aurumdev95/medx-gemma4-e4b"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Aurumdev95/medx-gemma4-e4b",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/Aurumdev95/medx-gemma4-e4b:BF16

Ollama
How to use Aurumdev95/medx-gemma4-e4b with Ollama:
```
ollama run hf.co/Aurumdev95/medx-gemma4-e4b:BF16
```
Unsloth Desktop

How to use Aurumdev95/medx-gemma4-e4b with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf Aurumdev95/medx-gemma4-e4b:BF16

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "Aurumdev95/medx-gemma4-e4b:BF16"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use Aurumdev95/medx-gemma4-e4b with Docker Model Runner:
```
docker model run hf.co/Aurumdev95/medx-gemma4-e4b:BF16
```

Lemonade

How to use Aurumdev95/medx-gemma4-e4b with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull Aurumdev95/medx-gemma4-e4b:BF16

Run and chat with the model

lemonade run user.medx-gemma4-e4b-BF16

List all available models

lemonade list

Hermes Agent

How to use Aurumdev95/medx-gemma4-e4b with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf Aurumdev95/medx-gemma4-e4b:BF16

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default Aurumdev95/medx-gemma4-e4b:BF16

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use Aurumdev95/medx-gemma4-e4b with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf Aurumdev95/medx-gemma4-e4b:BF16

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "Aurumdev95/medx-gemma4-e4b:BF16" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

medx-gemma4-e4b / README.md

Aurumdev95

medx Gemma 4 E4B: shipped top-up (Q8_0 GGUF) and both LoRA stages

7ca9f55 verified 21 days ago

preview code

Raw History Blame Contribute Delete

4.08 kB

---
license: apache-2.0
base_model: google/gemma-4-E4B-it
language:
- en
- fr
- rw
library_name: gguf
pipeline_tag: text-generation
tags:
- clinical-nlp
- information-extraction
- structured-output
- json
- gguf
- lora
- gemma4
---
# medx-gemma4-e4b

A fine-tune of [Gemma 4 E4B-it](https://huggingface.co/google/gemma-4-E4B-it) that reads one clinical note and returns the
facts as a single flat JSON object. The schema goes in the system prompt: the default schema has 44 fields
(demographics, vitals, social history, symptoms, history, medications, labs, assessment), and a deployer can rename
fields, take a subset, add typed custom fields or ask for nulls without retraining. Notes may be in English, French,
Kinyarwanda or a mix. The model answers directly, with thinking off.

**Research model, not a medical device.** It was trained on model-generated notes whose labels no clinician has
reviewed. Do not use its output for clinical decisions without human verification.

## Files

| File | What it is |
|---|---|
| `medx-gemma4-e4b.Q8_0.gguf` | The shipped model, merged and quantized to Q8_0 for llama.cpp (8.0 GB) |
| `medx-gemma4-e4b.BF16-mmproj.gguf` | Vision projector exported with it; not needed for text-only extraction |
| `adapters/run2_brand/` | LoRA adapter (rank 32, alpha 32) for `google/gemma-4-E4B-it`: stage 1, 30,000 English rows with brand-name drug mentions |
| `adapters/topup/` | LoRA adapter (rank 32, alpha 32) for the **merged stage-1 model**: stage 2, 7,980 rows of French, Kinyarwanda, long English dictations and fresh English notes |

To rebuild the model in Transformers: load `google/gemma-4-E4B-it`, apply `adapters/run2_brand`, merge it, then apply
`adapters/topup` and merge again. The GGUF is the result of exactly that.

## Usage (llama.cpp)

```bash
llama-server -m medx-gemma4-e4b.Q8_0.gguf -c 8192 --jinja \
  --chat-template-kwargs '{"enable_thinking": false}' --temp 0
```

Send the schema as the system message and the note as the user message. Thinking must stay off: the model was
trained without reasoning and the chat template otherwise opens a thought channel. Use greedy decoding.

## Evaluation

Micro-F1 over gold facts on six held-out sets (no note or schema in them was trained on). Real is 150 de-identified
MTSamples transcriptions; the other five sets are model-generated. Labels come from model annotators with model
adjudication.

| Set | Notes | Micro-F1 | Precision | Recall |
|---|---|---|---|---|
| Synthetic (English, default schema) | 987 | 0.945 | 0.947 | 0.943 |
| Hard cases (English) | 250 | 0.939 | 0.942 | 0.936 |
| Real transcriptions (English) | 150 | 0.734 | 0.768 | 0.702 |
| Custom held-out schemas (English) | 1,000 | 0.950 | 0.948 | 0.952 |
| French, held-out schemas | 300 | 0.928 | 0.927 | 0.929 |
| Kinyarwanda, held-out schemas | 300 | 0.910 | 0.906 | 0.915 |

Every output on these sets was valid JSON. Scores are from greedy decoding of the bf16 weights; the Q8_0 GGUF was not
scored separately.
## Training
Unsloth 16-bit LoRA on all attention and MLP projections of the language layers, answer-only loss, 8-bit AdamW,
cosine schedule, effective batch 32, context 4,096, one NVIDIA GB10 (DGX Spark). Stage 1: one epoch on 30,000 rows,
learning rate 1e-4, with half of the generic drug mentions in each note rewritten as brand names. Stage 2: one epoch
on 7,980 rows at 5e-5, starting from the merged stage-1 model.
Every training note was generated from a seeded specification and kept only if each gold fact had a verbatim quote in
the note and every number was reproducible from it; a second model extracted the facts blind and a third adjudicated
disagreements.
## Limitations
- No clinician has reviewed the training or evaluation labels.
- Real-note accuracy (0.734) is far below accuracy on generated notes; the real set has only 150 English notes from
  one public source, and no real French or Kinyarwanda notes have been scored.
- The model can map brand names to the wrong generic drug.
- A larger successor (Qwen3.6-35B-A3B fine-tune) scores higher on every set above.

	---
	license: apache-2.0
	base_model: google/gemma-4-E4B-it
	language:
	- en
	- fr
	- rw
	library_name: gguf
	pipeline_tag: text-generation
	tags:
	- clinical-nlp
	- information-extraction
	- structured-output
	- json
	- gguf
	- lora
	- gemma4
	---

	# medx-gemma4-e4b

	A fine-tune of [Gemma 4 E4B-it](https://huggingface.co/google/gemma-4-E4B-it) that reads one clinical note and returns the
	facts as a single flat JSON object. The schema goes in the system prompt: the default schema has 44 fields
	(demographics, vitals, social history, symptoms, history, medications, labs, assessment), and a deployer can rename
	fields, take a subset, add typed custom fields or ask for nulls without retraining. Notes may be in English, French,
	Kinyarwanda or a mix. The model answers directly, with thinking off.

	Research model, not a medical device. It was trained on model-generated notes whose labels no clinician has
	reviewed. Do not use its output for clinical decisions without human verification.

	## Files

	\| File \| What it is \|
	\|---\|---\|
	\| `medx-gemma4-e4b.Q8_0.gguf` \| The shipped model, merged and quantized to Q8_0 for llama.cpp (8.0 GB) \|
	\| `medx-gemma4-e4b.BF16-mmproj.gguf` \| Vision projector exported with it; not needed for text-only extraction \|
	\| `adapters/run2_brand/` \| LoRA adapter (rank 32, alpha 32) for `google/gemma-4-E4B-it`: stage 1, 30,000 English rows with brand-name drug mentions \|
	\| `adapters/topup/` \| LoRA adapter (rank 32, alpha 32) for the merged stage-1 model: stage 2, 7,980 rows of French, Kinyarwanda, long English dictations and fresh English notes \|

	To rebuild the model in Transformers: load `google/gemma-4-E4B-it`, apply `adapters/run2_brand`, merge it, then apply
	`adapters/topup` and merge again. The GGUF is the result of exactly that.

	## Usage (llama.cpp)

	```bash
	llama-server -m medx-gemma4-e4b.Q8_0.gguf -c 8192 --jinja \
	--chat-template-kwargs '{"enable_thinking": false}' --temp 0
	```

	Send the schema as the system message and the note as the user message. Thinking must stay off: the model was
	trained without reasoning and the chat template otherwise opens a thought channel. Use greedy decoding.

	## Evaluation

	Micro-F1 over gold facts on six held-out sets (no note or schema in them was trained on). Real is 150 de-identified
	MTSamples transcriptions; the other five sets are model-generated. Labels come from model annotators with model
	adjudication.

	\| Set \| Notes \| Micro-F1 \| Precision \| Recall \|
	\|---\|---\|---\|---\|---\|
	\| Synthetic (English, default schema) \| 987 \| 0.945 \| 0.947 \| 0.943 \|
	\| Hard cases (English) \| 250 \| 0.939 \| 0.942 \| 0.936 \|
	\| Real transcriptions (English) \| 150 \| 0.734 \| 0.768 \| 0.702 \|
	\| Custom held-out schemas (English) \| 1,000 \| 0.950 \| 0.948 \| 0.952 \|
	\| French, held-out schemas \| 300 \| 0.928 \| 0.927 \| 0.929 \|
	\| Kinyarwanda, held-out schemas \| 300 \| 0.910 \| 0.906 \| 0.915 \|

	Every output on these sets was valid JSON. Scores are from greedy decoding of the bf16 weights; the Q8_0 GGUF was not
	scored separately.

	## Training

	Unsloth 16-bit LoRA on all attention and MLP projections of the language layers, answer-only loss, 8-bit AdamW,
	cosine schedule, effective batch 32, context 4,096, one NVIDIA GB10 (DGX Spark). Stage 1: one epoch on 30,000 rows,
	learning rate 1e-4, with half of the generic drug mentions in each note rewritten as brand names. Stage 2: one epoch
	on 7,980 rows at 5e-5, starting from the merged stage-1 model.

	Every training note was generated from a seeded specification and kept only if each gold fact had a verbatim quote in
	the note and every number was reproducible from it; a second model extracted the facts blind and a third adjudicated
	disagreements.

	## Limitations

	- No clinician has reviewed the training or evaluation labels.
	- Real-note accuracy (0.734) is far below accuracy on generated notes; the real set has only 150 English notes from
	one public source, and no real French or Kinyarwanda notes have been scored.
	- The model can map brand names to the wrong generic drug.
	- A larger successor (Qwen3.6-35B-A3B fine-tune) scores higher on every set above.