Instructions to use fableforge-ai/ShellWhisperer-1.5B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use fableforge-ai/ShellWhisperer-1.5B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="fableforge-ai/ShellWhisperer-1.5B")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("fableforge-ai/ShellWhisperer-1.5B")
model = AutoModelForCausalLM.from_pretrained("fableforge-ai/ShellWhisperer-1.5B", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

llama.cpp

How to use fableforge-ai/ShellWhisperer-1.5B with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf fableforge-ai/ShellWhisperer-1.5B:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf fableforge-ai/ShellWhisperer-1.5B:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf fableforge-ai/ShellWhisperer-1.5B:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf fableforge-ai/ShellWhisperer-1.5B:Q4_K_M

Build from source code

git clone https://github.com/ggml-org/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-app
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama serve -hf fableforge-ai/ShellWhisperer-1.5B:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama cli -hf fableforge-ai/ShellWhisperer-1.5B:Q4_K_M

Use Docker

docker model run hf.co/fableforge-ai/ShellWhisperer-1.5B:Q4_K_M

Llama new
LM Studio
Jan

vLLM

How to use fableforge-ai/ShellWhisperer-1.5B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "fableforge-ai/ShellWhisperer-1.5B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "fableforge-ai/ShellWhisperer-1.5B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/fableforge-ai/ShellWhisperer-1.5B:Q4_K_M

SGLang

How to use fableforge-ai/ShellWhisperer-1.5B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "fableforge-ai/ShellWhisperer-1.5B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "fableforge-ai/ShellWhisperer-1.5B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "fableforge-ai/ShellWhisperer-1.5B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "fableforge-ai/ShellWhisperer-1.5B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Ollama
How to use fableforge-ai/ShellWhisperer-1.5B with Ollama:
```
ollama run hf.co/fableforge-ai/ShellWhisperer-1.5B:Q4_K_M
```
Unsloth Desktop

How to use fableforge-ai/ShellWhisperer-1.5B with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf fableforge-ai/ShellWhisperer-1.5B:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "fableforge-ai/ShellWhisperer-1.5B:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use fableforge-ai/ShellWhisperer-1.5B with Docker Model Runner:
```
docker model run hf.co/fableforge-ai/ShellWhisperer-1.5B:Q4_K_M
```

Lemonade

How to use fableforge-ai/ShellWhisperer-1.5B with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull fableforge-ai/ShellWhisperer-1.5B:Q4_K_M

Run and chat with the model

lemonade run user.ShellWhisperer-1.5B-Q4_K_M

List all available models

lemonade list

Hermes Agent

How to use fableforge-ai/ShellWhisperer-1.5B with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf fableforge-ai/ShellWhisperer-1.5B:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default fableforge-ai/ShellWhisperer-1.5B:Q4_K_M

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use fableforge-ai/ShellWhisperer-1.5B with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf fableforge-ai/ShellWhisperer-1.5B:Q4_K_M

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "fableforge-ai/ShellWhisperer-1.5B:Q4_K_M" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

King3Djbl commited on Jul 6

Commit

d3edb2b

verified ·

1 Parent(s): 0825836

Viral card: model-tree metadata, quant table, Ollama-from-source, demo funnel

Browse files

Files changed (1) hide show

README.md +57 -76

README.md CHANGED Viewed

@@ -2,111 +2,92 @@
 license: apache-2.0
 language:
   - en
 base_model: Qwen/Qwen2.5-1.5B-Instruct
 tags:
-  - shell
   - commands
-  - terminal
   - devops
-  - system-admin
-  - uncensored
   - fableforge
-  - ollama
-  - lightweight
   - gguf
-  - 1.5b
-  - conversational
-library_name: transformers
-pipeline_tag: text-generation
-widget:
-  - text: "Find all files larger than 100MB in the current directory"
-    example_title: "Disk usage"
-  - text: "Kill all processes using more than 500MB of memory"
-    example_title: "Process management"
 ---
-# ShellWhisperer-1.5B — Ultra-Fast Shell Command Assistant
-<div align="center">
-**1,100+ downloads · 986MB · 20+ tok/s · Runs on anything**
-[![HuggingFace](https://img.shields.io/badge/HuggingFace-fableforge--ai%2FShellWhisperer--1.5B-FFD21E?logo=huggingface)](https://huggingface.co/fableforge-ai/ShellWhisperer-1.5B)
-[![Size](https://img.shields.io/badge/size-986MB-success)](#quantizations)
-[![License](https://img.shields.io/badge/license-Apache--2.0-blue)](#license)
-[![Speed](https://img.shields.io/badge/speed-20%2B%20tok%2Fs-brightgreen)](#hardware-requirements)
-</div>
-## What Is This?
-**ShellWhisperer is a fast shell command assistant** focused exclusively on shell tasks. Fine-tuned from Qwen2.5-1.5B-Instruct on 100K+ shell command traces, it predicts bash, zsh, PowerShell, and DevOps commands.
-Unlike general models that *kind of* know shell commands, ShellWhisperer is purpose-built for terminal productivity.
-## Quick Start
-### Ollama
 ```bash
-ollama run FableForge-AI/shellwhisperer
 ```
-### llama.cpp
-```bash
-./llama-cli -m shellwhisperer-1.5b-Q4_K_M.gguf --prompt "find all files larger than 100MB" -n 512
-```
-### Python
-```python
-from transformers import AutoModelForCausalLM, AutoTokenizer
-model = AutoModelForCausalLM.from_pretrained("fableforge-ai/ShellWhisperer-1.5B")
 ```
-## Quantizations
-| File | Size | Best For |
-|------|------|----------|
-| `IQ2_XXS.gguf` | 488 MB | Extreme compression |
-| `Q2_K.gguf` | 645 MB | Phone, Raspberry Pi |
-| `IQ3_XXS.gguf` | 638 MB | Ultra low-resource |
-| `Q3_K_M.gguf` | 786 MB | Low-end devices |
-| `Q4_0.gguf` | 895 MB | Fast inference |
-| **`Q4_K_M.gguf`** | **940 MB** | **Recommended** |
-| `IQ4_XS.gguf` | 860 MB | Quality + compression |
-| `Q5_K_M.gguf` | 1.0 GB | High quality |
-| `Q6_K.gguf` | 1.2 GB | Pro quality |
-| `Q8_0.gguf` | 1.5 GB | Near-lossless |
-| `f16.gguf` | 2.9 GB | Full precision |
-## Hardware Requirements
-| Hardware | Best Quant |
-|----------|------------|
-| Phone (2GB+ RAM) | Q2_K |
-| Raspberry Pi | Q2_K |
-| Old laptop (4GB RAM) | Q4_K_M |
-| Gaming PC (RTX 3060+) | Q5_K_M |
-| Server (32GB+) | Q8_0 / F16 |
-## Training Details
-| Parameter | Value |
-|-----------|-------|
-| Base Model | Qwen2.5-1.5B-Instruct |
-| Training Data | 100K+ shell command traces |
-| Method | QLoRA |
-| Precision | bfloat16 |
-## Limitations
-- **Single-domain focus**: Best at shell/terminal tasks; general knowledge may be limited
-- **Quantization impact**: Smaller quants may show quality degradation
-- **Uncensored**: Model does not refuse harmful requests — use responsibly
-## License
-Apache 2.0 — commercial use allowed.
 ---
-*Part of the [FableForge AI](https://huggingface.co/fableforge-ai) ecosystem.*

 license: apache-2.0
 language:
   - en
+library_name: transformers
+pipeline_tag: text-generation
 base_model: Qwen/Qwen2.5-1.5B-Instruct
+base_model_relation: finetune
 tags:
+  - 1.5b
   - commands
   - devops
   - fableforge
   - gguf
+  - imatrix
+  - llama.cpp
+  - lm-studio
+  - ollama
+  - shell
+  - sysadmin
+  - terminal
+  - text-generation
+  - uncensored
 ---
+# ShellWhisperer-1.5B
+> **Natural language → precise bash commands. Tiny, fast, uncensored.**
+[![Downloads](https://img.shields.io/badge/dynamic/json?url=https%3A%2F%2Fhuggingface.co%2Fapi%2Fmodels%2Ffableforge-ai%2FShellWhisperer-1.5B&query=downloads&label=downloads&color=blue)](https://huggingface.co/fableforge-ai/ShellWhisperer-1.5B) [![Demos](https://img.shields.io/badge/🤗-Live%20Demos-orange)](https://huggingface.co/spaces/fableforge-ai/fableforge-nexus)
+Base model: [Qwen/Qwen2.5-1.5B-Instruct](https://huggingface.co/Qwen/Qwen2.5-1.5B-Instruct)  ·  Part of the **FableForge** ecosystem.
+---
+## ⚡ Quickest start — Ollama (pulls straight from this repo → credits the source)
 ```bash
+ollama run hf.co/fableforge-ai/ShellWhisperer-1.5B:Q4_K_M
 ```
+## 🦙 llama.cpp
+```bash
+llama-cli -hf fableforge-ai/ShellWhisperer-1.5B:Q4_K_M -p "Hello!"
 ```
+## 💠 LM Studio
+Search **`fableforge-ai/ShellWhisperer-1.5B`** and pick a quant below.
+---
+## 📦 Provided quants (download one file, not the whole repo)
+| File | Quant | Size | Notes |
+| ---- | ----- | ---- | ----- |
+| [shellwhisperer-1.5b-f16.gguf](https://huggingface.co/fableforge-ai/ShellWhisperer-1.5B/blob/main/shellwhisperer-1.5b-f16.gguf) | `F16` | 3.09 GB | Full precision (unquantized). Reference / conversion. |
+| [shellwhisperer-1.5b-Q8_0.gguf](https://huggingface.co/fableforge-ai/ShellWhisperer-1.5B/blob/main/shellwhisperer-1.5b-Q8_0.gguf) | `Q8_0` | 1.65 GB | Extremely high quality — usually overkill. |
+| [shellwhisperer-1.5b-Q6_K.gguf](https://huggingface.co/fableforge-ai/ShellWhisperer-1.5B/blob/main/shellwhisperer-1.5b-Q6_K.gguf) | `Q6_K` | 1.27 GB | Very high quality, near-perfect. **Recommended** if you have the RAM. |
+| [shellwhisperer-1.5b-Q5_K_M.gguf](https://huggingface.co/fableforge-ai/ShellWhisperer-1.5B/blob/main/shellwhisperer-1.5b-Q5_K_M.gguf) | `Q5_K_M` | 1.13 GB | High quality. **Recommended.** |
+| [shellwhisperer-1.5b-Q4_K_M.gguf](https://huggingface.co/fableforge-ai/ShellWhisperer-1.5B/blob/main/shellwhisperer-1.5b-Q4_K_M.gguf) | `Q4_K_M` | 0.99 GB | ⭐ **Best size/quality balance — default pick for most users.** |
+| [shellwhisperer-1.5b-IQ4_XS.gguf](https://huggingface.co/fableforge-ai/ShellWhisperer-1.5B/blob/main/shellwhisperer-1.5b-IQ4_XS.gguf) | `IQ4_XS` | 0.90 GB | imatrix — great quality, smaller than Q4_K_S. **Recommended for low RAM.** |
+| [shellwhisperer-1.5b-Q4_0.gguf](https://huggingface.co/fableforge-ai/ShellWhisperer-1.5B/blob/main/shellwhisperer-1.5b-Q4_0.gguf) | `Q4_0` | 0.93 GB | Legacy format — prefer Q4_K_M. |
+| [shellwhisperer-1.5b-Q3_K_M.gguf](https://huggingface.co/fableforge-ai/ShellWhisperer-1.5B/blob/main/shellwhisperer-1.5b-Q3_K_M.gguf) | `Q3_K_M` | 0.82 GB | Lower quality; usable when RAM is tight. |
+| [shellwhisperer-1.5b-IQ3_XXS.gguf](https://huggingface.co/fableforge-ai/ShellWhisperer-1.5B/blob/main/shellwhisperer-1.5b-IQ3_XXS.gguf) | `IQ3_XXS` | 0.67 GB | imatrix very low quality. |
+| [shellwhisperer-1.5b-Q2_K.gguf](https://huggingface.co/fableforge-ai/ShellWhisperer-1.5B/blob/main/shellwhisperer-1.5b-Q2_K.gguf) | `Q2_K` | 0.68 GB | Very low quality — surprisingly usable. |
+| [shellwhisperer-1.5b-IQ2_XXS.gguf](https://huggingface.co/fableforge-ai/ShellWhisperer-1.5B/blob/main/shellwhisperer-1.5b-IQ2_XXS.gguf) | `IQ2_XXS` | 0.51 GB | Tiny imatrix — last resort. |
+**Not sure?** Take **`Q4_K_M`**. Low RAM → **`IQ4_XS`**. Max quality → **`Q6_K`**.
+## 🧠 Prompt format (ChatML)
+```
+<|im_start|>system
+{system_prompt}<|im_end|>
+<|im_start|>user
+{prompt}<|im_end|>
+<|im_start|>assistant
+```
+## 🌌 See it live — the FableForge demos
+This family powers a galaxy of free, interactive HF Spaces:
+- 🧭 [FableForge Nexus](https://huggingface.co/spaces/fableforge-ai/fableforge-nexus)
+- 🍺 [Infinite NPC](https://huggingface.co/spaces/fableforge-ai/infinite-npc)
+- 🎭 [Dual-GM](https://huggingface.co/spaces/King3Djbl/dual-gm-simulator)
+- 💻 [ShellWhisperer](https://huggingface.co/spaces/King3Djbl/shellwhisperer-terminal)
+- 👻 [Ghost Writer](https://huggingface.co/spaces/fableforge-ai/ghost-writer)
 ---
+*⭐ Like & share — it helps people find the source instead of a mirror.*