Instructions to use evalengine/unbound-e2b-gguf with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use evalengine/unbound-e2b-gguf with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf evalengine/unbound-e2b-gguf:Q4_K_M # Run inference directly in the terminal: llama cli -hf evalengine/unbound-e2b-gguf:Q4_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf evalengine/unbound-e2b-gguf:Q4_K_M # Run inference directly in the terminal: llama cli -hf evalengine/unbound-e2b-gguf:Q4_K_M
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf evalengine/unbound-e2b-gguf:Q4_K_M # Run inference directly in the terminal: ./build/bin/llama cli -hf evalengine/unbound-e2b-gguf:Q4_K_M
Use Docker
docker model run hf.co/evalengine/unbound-e2b-gguf:Q4_K_M
- Llama new
- LM Studio
- Jan
- vLLM
How to use evalengine/unbound-e2b-gguf with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "evalengine/unbound-e2b-gguf" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "evalengine/unbound-e2b-gguf", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker
docker model run hf.co/evalengine/unbound-e2b-gguf:Q4_K_M
- Ollama
How to use evalengine/unbound-e2b-gguf with Ollama:
ollama run hf.co/evalengine/unbound-e2b-gguf:Q4_K_M
- Unsloth Desktop
- Pi
How to use evalengine/unbound-e2b-gguf with Pi:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf evalengine/unbound-e2b-gguf:Q4_K_M
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "llama-cpp": { "baseUrl": "http://localhost:9931/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "evalengine/unbound-e2b-gguf:Q4_K_M" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Docker Model Runner
How to use evalengine/unbound-e2b-gguf with Docker Model Runner:
docker model run hf.co/evalengine/unbound-e2b-gguf:Q4_K_M
- Lemonade
How to use evalengine/unbound-e2b-gguf with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull evalengine/unbound-e2b-gguf:Q4_K_M
Run and chat with the model
lemonade run user.unbound-e2b-gguf-Q4_K_M
List all available models
lemonade list
- Hermes Agent
How to use evalengine/unbound-e2b-gguf with Hermes Agent:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf evalengine/unbound-e2b-gguf:Q4_K_M
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:9931/v1 hermes config set model.default evalengine/unbound-e2b-gguf:Q4_K_M
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use evalengine/unbound-e2b-gguf with OpenClaw:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf evalengine/unbound-e2b-gguf:Q4_K_M
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:9931/v1 \ --custom-model-id "evalengine/unbound-e2b-gguf:Q4_K_M" \ --custom-provider-id llama-cpp \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
README: 3-quant lineup, split-file UX, wllama browser support
Browse files
README.md
CHANGED
|
@@ -24,17 +24,23 @@ pipeline_tag: text-generation
|
|
| 24 |
> use it and for complying with all applicable laws.
|
| 25 |
|
| 26 |
GGUF quantizations of [`evalengine/unbound-e2b`](https://huggingface.co/evalengine/unbound-e2b)
|
| 27 |
-
for on-device deployment via Ollama, llama.cpp, LM Studio,
|
|
|
|
| 28 |
|
| 29 |
Built by [Chromia](https://x.com/Chromia) and [Eval Engine](https://x.com/eval_engine).
|
| 30 |
|
| 31 |
## Available quants
|
| 32 |
|
| 33 |
-
|
| 34 |
-
|
| 35 |
-
|
|
|
|
| 36 |
|
| 37 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
| 38 |
|
| 39 |
## Recommended sampling
|
| 40 |
|
|
@@ -51,7 +57,7 @@ through.
|
|
| 51 |
|
| 52 |
## Default sampling (Ollama)
|
| 53 |
|
| 54 |
-
When you `ollama pull hf.co/evalengine/unbound-e2b-
|
| 55 |
`Modelfile` sets these defaults, tuned for factual recall:
|
| 56 |
|
| 57 |
- `temperature = 0.6` (lower than Gemma's training default of 1.0 β keeps
|
|
@@ -63,7 +69,7 @@ When you `ollama pull hf.co/evalengine/unbound-e2b-gguf`, the bundled
|
|
| 63 |
**To override per-session in Ollama:**
|
| 64 |
|
| 65 |
```
|
| 66 |
-
ollama run hf.co/evalengine/unbound-e2b-
|
| 67 |
>>> /set parameter temperature 1.0 # creative / open-ended
|
| 68 |
>>> /set parameter temperature 0.3 # max factual / brand questions
|
| 69 |
```
|
|
@@ -74,20 +80,39 @@ include the SYSTEM line from the `Modelfile` as your `--system` argument.
|
|
| 74 |
## Run with Ollama
|
| 75 |
|
| 76 |
```bash
|
| 77 |
-
ollama pull hf.co/evalengine/unbound-e2b-
|
| 78 |
-
ollama run hf.co/evalengine/unbound-e2b-
|
| 79 |
```
|
| 80 |
|
|
|
|
|
|
|
| 81 |
## Run with llama.cpp
|
| 82 |
|
| 83 |
```bash
|
| 84 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 85 |
```
|
| 86 |
|
| 87 |
## About the base
|
| 88 |
|
| 89 |
See [`evalengine/unbound-e2b`](https://huggingface.co/evalengine/unbound-e2b)
|
| 90 |
for the full model card, benchmarks, intended use, and the merged HF weights.
|
|
|
|
| 91 |
## Acknowledgements
|
| 92 |
|
| 93 |
- Fine-tuned with [Unsloth](https://github.com/unslothai/unsloth) + Huggingface's [TRL](https://github.com/huggingface/trl).
|
|
|
|
| 24 |
> use it and for complying with all applicable laws.
|
| 25 |
|
| 26 |
GGUF quantizations of [`evalengine/unbound-e2b`](https://huggingface.co/evalengine/unbound-e2b)
|
| 27 |
+
for on-device deployment via Ollama, llama.cpp, LM Studio, [wllama](https://github.com/ngxson/wllama)
|
| 28 |
+
(in-browser), and similar runtimes.
|
| 29 |
|
| 30 |
Built by [Chromia](https://x.com/Chromia) and [Eval Engine](https://x.com/eval_engine).
|
| 31 |
|
| 32 |
## Available quants
|
| 33 |
|
| 34 |
+
All quants ship as **split multi-part GGUFs** (`*-00001-of-0000N.gguf` ...) so
|
| 35 |
+
they work in browsers (wllama's 2 GB ArrayBuffer cap) and let desktop
|
| 36 |
+
runtimes parallel-download chunks. Ollama, llama.cpp, and LM Studio
|
| 37 |
+
auto-stitch on the first part β same UX as a single file.
|
| 38 |
|
| 39 |
+
| Quant | Parts | Total | Largest part | wllama (browser) | Desktop (Ollama/llama.cpp/LM Studio) | Notes |
|
| 40 |
+
|---------|-------|--------|--------------|------------------|--------------------------------------|-------|
|
| 41 |
+
| Q4_K_M | 3 | 3.2 GB | 1.79 GB | β
| β
| Recommended on-device default β best size/quality |
|
| 42 |
+
| Q6_K | 4 | 3.6 GB | 1.79 GB | β
| β
| Higher fidelity, still browser-safe |
|
| 43 |
+
| Q8_0 | 4 | 4.6 GB | **2.32 GB** | β (over 2 GB) | β
| Highest fidelity; one tensor exceeds the browser ArrayBuffer limit, so desktop runtimes only |
|
| 44 |
|
| 45 |
## Recommended sampling
|
| 46 |
|
|
|
|
| 57 |
|
| 58 |
## Default sampling (Ollama)
|
| 59 |
|
| 60 |
+
When you `ollama pull hf.co/evalengine/unbound-e2b-GGUF`, the bundled
|
| 61 |
`Modelfile` sets these defaults, tuned for factual recall:
|
| 62 |
|
| 63 |
- `temperature = 0.6` (lower than Gemma's training default of 1.0 β keeps
|
|
|
|
| 69 |
**To override per-session in Ollama:**
|
| 70 |
|
| 71 |
```
|
| 72 |
+
ollama run hf.co/evalengine/unbound-e2b-GGUF
|
| 73 |
>>> /set parameter temperature 1.0 # creative / open-ended
|
| 74 |
>>> /set parameter temperature 0.3 # max factual / brand questions
|
| 75 |
```
|
|
|
|
| 80 |
## Run with Ollama
|
| 81 |
|
| 82 |
```bash
|
| 83 |
+
ollama pull hf.co/evalengine/unbound-e2b-GGUF
|
| 84 |
+
ollama run hf.co/evalengine/unbound-e2b-GGUF
|
| 85 |
```
|
| 86 |
|
| 87 |
+
(Defaults to Q4_K_M. Ollama auto-stitches the split parts on load.)
|
| 88 |
+
|
| 89 |
## Run with llama.cpp
|
| 90 |
|
| 91 |
```bash
|
| 92 |
+
# point at the FIRST part β llama.cpp follows the chain automatically
|
| 93 |
+
./llama-cli -m unbound-e2b-Q4_K_M-00001-of-00003.gguf -p "your prompt"
|
| 94 |
+
```
|
| 95 |
+
|
| 96 |
+
## Run in the browser (wllama)
|
| 97 |
+
|
| 98 |
+
[wllama](https://github.com/ngxson/wllama) is a WebAssembly port of llama.cpp
|
| 99 |
+
that runs entirely in the browser β no server, no install. Use Q4_K_M or
|
| 100 |
+
Q6_K (Q8_0 has a tensor above the 2 GB ArrayBuffer limit):
|
| 101 |
+
|
| 102 |
+
```js
|
| 103 |
+
import { Wllama } from '@wllama/wllama';
|
| 104 |
+
const wllama = new Wllama(/* β¦ */);
|
| 105 |
+
await wllama.loadModelFromHF(
|
| 106 |
+
'evalengine/unbound-e2b-GGUF',
|
| 107 |
+
'unbound-e2b-Q4_K_M-00001-of-00003.gguf' // wllama follows the chain
|
| 108 |
+
);
|
| 109 |
```
|
| 110 |
|
| 111 |
## About the base
|
| 112 |
|
| 113 |
See [`evalengine/unbound-e2b`](https://huggingface.co/evalengine/unbound-e2b)
|
| 114 |
for the full model card, benchmarks, intended use, and the merged HF weights.
|
| 115 |
+
|
| 116 |
## Acknowledgements
|
| 117 |
|
| 118 |
- Fine-tuned with [Unsloth](https://github.com/unslothai/unsloth) + Huggingface's [TRL](https://github.com/huggingface/trl).
|