Text Generation
GGUF
English
tool-routing
function-calling
prism-coder
qwen3.5
synalux
prompt-engineering
imatrix
conversational
Instructions to use dcostenco/prism-coder-4b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use dcostenco/prism-coder-4b with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf dcostenco/prism-coder-4b:Q3_K_M # Run inference directly in the terminal: llama cli -hf dcostenco/prism-coder-4b:Q3_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf dcostenco/prism-coder-4b:Q3_K_M # Run inference directly in the terminal: llama cli -hf dcostenco/prism-coder-4b:Q3_K_M
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf dcostenco/prism-coder-4b:Q3_K_M # Run inference directly in the terminal: ./build/bin/llama cli -hf dcostenco/prism-coder-4b:Q3_K_M
Use Docker
docker model run hf.co/dcostenco/prism-coder-4b:Q3_K_M
- Llama new
- LM Studio
- Jan
- vLLM
How to use dcostenco/prism-coder-4b with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "dcostenco/prism-coder-4b" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "dcostenco/prism-coder-4b", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/dcostenco/prism-coder-4b:Q3_K_M
- Ollama
How to use dcostenco/prism-coder-4b with Ollama:
ollama run hf.co/dcostenco/prism-coder-4b:Q3_K_M
- Unsloth Desktop
- Pi
How to use dcostenco/prism-coder-4b with Pi:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf dcostenco/prism-coder-4b:Q3_K_M
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "llama-cpp": { "baseUrl": "http://localhost:9931/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "dcostenco/prism-coder-4b:Q3_K_M" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Docker Model Runner
How to use dcostenco/prism-coder-4b with Docker Model Runner:
docker model run hf.co/dcostenco/prism-coder-4b:Q3_K_M
- Lemonade
How to use dcostenco/prism-coder-4b with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull dcostenco/prism-coder-4b:Q3_K_M
Run and chat with the model
lemonade run user.prism-coder-4b-Q3_K_M
List all available models
lemonade list
- Hermes Agent
How to use dcostenco/prism-coder-4b with Hermes Agent:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf dcostenco/prism-coder-4b:Q3_K_M
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:9931/v1 hermes config set model.default dcostenco/prism-coder-4b:Q3_K_M
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use dcostenco/prism-coder-4b with OpenClaw:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf dcostenco/prism-coder-4b:Q3_K_M
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:9931/v1 \ --custom-model-id "dcostenco/prism-coder-4b:Q3_K_M" \ --custom-provider-id llama-cpp \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
Update model card: prompt-engineered (not stock), Ollama tag live
Browse files
README.md
CHANGED
|
@@ -8,16 +8,24 @@ tags:
|
|
| 8 |
- qwen3.5
|
| 9 |
- synalux
|
| 10 |
- prompt-engineering
|
|
|
|
| 11 |
base_model: Qwen/Qwen3.5-4B
|
| 12 |
pipeline_tag: text-generation
|
| 13 |
---
|
| 14 |
|
| 15 |
# prism-coder:4b β 17-Tool Memory Agent (Verifier + Mobile Tier)
|
| 16 |
|
| 17 |
-
|
| 18 |
Primary deployment: **mobile / 8 GB+ devices** via Ollama β the lightweight verifier tier.
|
| 19 |
|
| 20 |
-
**
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 21 |
|
| 22 |
## BFCL Benchmark β 100% Γ 3 Seeds
|
| 23 |
|
|
@@ -40,67 +48,62 @@ Primary deployment: **mobile / 8 GB+ devices** via Ollama β the lightweight ve
|
|
| 40 |
|
| 41 |
115 test cases, 3 shuffled runs (seeds 2027/2028/2029), temperature=0, 0 hallucinations across all runs.
|
| 42 |
|
| 43 |
-
##
|
| 44 |
|
| 45 |
-
Qwen3.5-4B's linear attention architecture + native thinking capability makes it uniquely suited to tool routing via prompt engineering
|
| 46 |
|
| 47 |
-
Key
|
|
|
|
|
|
|
|
|
|
| 48 |
|
| 49 |
-
|
| 50 |
|
| 51 |
-
|
| 52 |
-
|
| 53 |
-
|
| 54 |
-
|
|
|
|
|
|
|
|
|
|
| 55 |
|
| 56 |
-
|
| 57 |
|
| 58 |
## Tools
|
| 59 |
|
| 60 |
-
Routes to
|
| 61 |
|
| 62 |
| Tool | Trigger |
|
| 63 |
|------|---------|
|
| 64 |
| `session_load_context` | Load/resume project context |
|
| 65 |
| `session_save_ledger` | Log/record completed work |
|
| 66 |
| `session_save_handoff` | Create handoff note for next session |
|
| 67 |
-
| `session_search_memory` | Recall prior discussions |
|
| 68 |
-
| `session_forget_memory` | Delete a memory entry |
|
| 69 |
-
| `session_health_check` | Check session system health |
|
| 70 |
| `session_compact_ledger` | Compact/prune session ledger |
|
| 71 |
-
| `
|
| 72 |
-
| `session_task_route` | Route task: local vs cloud |
|
| 73 |
-
| `session_save_experience` | Save a notable experience |
|
| 74 |
-
| `session_synthesize_edges` | Build knowledge graph edges |
|
| 75 |
-
| `session_backfill_links` | Backfill missing memory links |
|
| 76 |
| `knowledge_search` | Search accumulated knowledge |
|
| 77 |
-
|
|
| 78 |
-
| `knowledge_upvote` | Upvote useful knowledge |
|
| 79 |
-
| `knowledge_downvote` | Downvote unhelpful knowledge |
|
| 80 |
-
| `knowledge_set_retention` | Set knowledge retention policy |
|
| 81 |
-
| *(no tool)* | General questions, greetings, code, math |
|
| 82 |
|
| 83 |
## Fleet Position
|
| 84 |
|
| 85 |
| Model | Ollama tag | Size | Role | Tier |
|
| 86 |
|---|---|---|---|---|
|
| 87 |
-
| prism-coder:1.7b | `prism-coder:1b7` | 1.1 GB | Always-fits fallback | Free |
|
| 88 |
-
| **
|
| 89 |
-
| prism-coder:14b | `prism-coder:14b` | 8.4 GB | Default router | Standard+ |
|
| 90 |
-
| prism-coder:32b | `prism-coder:32b` | 16 GB | Complex tasks | Advanced+ |
|
| 91 |
|
| 92 |
-
##
|
| 93 |
|
| 94 |
-
| | prism-coder:1b7/14b/32b |
|
| 95 |
|---|---|---|
|
| 96 |
-
|
|
| 97 |
-
|
|
| 98 |
-
| Thinking | Disabled (empty think) | Enabled (500 token budget) |
|
| 99 |
-
| GGUF hosting | Custom GGUF on HuggingFace | Stock from
|
| 100 |
| Accuracy | 100% | 100% |
|
| 101 |
|
| 102 |
## Links
|
| 103 |
|
|
|
|
| 104 |
- [Prism MCP Server](https://github.com/dcostenco/prism-coder) β the MCP server that uses this model
|
| 105 |
-
- [Ollama model page](https://ollama.com/dcostenco/prism-coder) β all fleet models
|
| 106 |
- [Qwen3.5-4B base](https://huggingface.co/Qwen/Qwen3.5-4B) β upstream base model
|
|
|
|
| 8 |
- qwen3.5
|
| 9 |
- synalux
|
| 10 |
- prompt-engineering
|
| 11 |
+
- gguf
|
| 12 |
base_model: Qwen/Qwen3.5-4B
|
| 13 |
pipeline_tag: text-generation
|
| 14 |
---
|
| 15 |
|
| 16 |
# prism-coder:4b β 17-Tool Memory Agent (Verifier + Mobile Tier)
|
| 17 |
|
| 18 |
+
Prompt-engineered [Qwen3.5-4B](https://huggingface.co/Qwen/Qwen3.5-4B) for full Prism Memory tool routing in the [Prism Coder](https://ollama.com/dcostenco/prism-coder) system.
|
| 19 |
Primary deployment: **mobile / 8 GB+ devices** via Ollama β the lightweight verifier tier.
|
| 20 |
|
| 21 |
+
**Fine-tuned via system prompt engineering.** Qwen3.5-4B achieves 100% routing accuracy through a structured routing system prompt + thinking enabled (500 token budget). No LoRA or weight modification β the system prompt IS the specialization, baked into the [Ollama Modelfile](https://ollama.com/dcostenco/prism-coder:4b).
|
| 22 |
+
|
| 23 |
+
## Quick Start
|
| 24 |
+
|
| 25 |
+
```bash
|
| 26 |
+
ollama pull dcostenco/prism-coder:4b
|
| 27 |
+
ollama run dcostenco/prism-coder:4b
|
| 28 |
+
```
|
| 29 |
|
| 30 |
## BFCL Benchmark β 100% Γ 3 Seeds
|
| 31 |
|
|
|
|
| 48 |
|
| 49 |
115 test cases, 3 shuffled runs (seeds 2027/2028/2029), temperature=0, 0 hallucinations across all runs.
|
| 50 |
|
| 51 |
+
## How It Works
|
| 52 |
|
| 53 |
+
Qwen3.5-4B's linear attention architecture + native thinking capability makes it uniquely suited to tool routing via prompt engineering. The model reasons through a structured routing ruleset in its think chain before selecting a tool β achieving the same 100% accuracy as LoRA-fine-tuned models in the fleet.
|
| 54 |
|
| 55 |
+
Key design decisions:
|
| 56 |
+
- **Thinking enabled** (not empty `<think>` blocks) β Qwen3.5 needs to reason through routing rules in its think chain. The forced empty-think pattern from Qwen3 breaks it.
|
| 57 |
+
- **500 token thinking budget** β enough to reason, not enough to overthink. Higher budgets (800+) cause analysis paralysis on compound queries.
|
| 58 |
+
- **Structured routing rules** β top-to-bottom priority matching with disambiguation rules and a no-tool override section.
|
| 59 |
|
| 60 |
+
SFT was attempted (3 rounds) but consistently hurt accuracy β the base model + prompt outperforms every SFT variant.
|
| 61 |
|
| 62 |
+
## System Prompt
|
| 63 |
+
|
| 64 |
+
The full routing system prompt is baked into the Ollama Modelfile. It includes:
|
| 65 |
+
- 6 tool definitions with exact names
|
| 66 |
+
- 13 priority-ordered routing rules
|
| 67 |
+
- Disambiguation rules for edge cases (knowledge_search vs session_search_memory, code requests vs tool calls)
|
| 68 |
+
- No-tool override section for categories that should never trigger tools
|
| 69 |
|
| 70 |
+
See the [Ollama model page](https://ollama.com/dcostenco/prism-coder:4b) for the complete Modelfile.
|
| 71 |
|
| 72 |
## Tools
|
| 73 |
|
| 74 |
+
Routes to 6 Prism Memory tools + knows when NOT to call any tool:
|
| 75 |
|
| 76 |
| Tool | Trigger |
|
| 77 |
|------|---------|
|
| 78 |
| `session_load_context` | Load/resume project context |
|
| 79 |
| `session_save_ledger` | Log/record completed work |
|
| 80 |
| `session_save_handoff` | Create handoff note for next session |
|
|
|
|
|
|
|
|
|
|
| 81 |
| `session_compact_ledger` | Compact/prune session ledger |
|
| 82 |
+
| `session_search_memory` | Recall prior discussions |
|
|
|
|
|
|
|
|
|
|
|
|
|
| 83 |
| `knowledge_search` | Search accumulated knowledge |
|
| 84 |
+
| *(no tool)* | General questions, greetings, code, math, AAC, translation |
|
|
|
|
|
|
|
|
|
|
|
|
|
| 85 |
|
| 86 |
## Fleet Position
|
| 87 |
|
| 88 |
| Model | Ollama tag | Size | Role | Tier |
|
| 89 |
|---|---|---|---|---|
|
| 90 |
+
| prism-coder:1.7b | `dcostenco/prism-coder:1b7` | 1.1 GB | Always-fits fallback | Free |
|
| 91 |
+
| **prism-coder:4b** | **`dcostenco/prism-coder:4b`** | **3.4 GB** | **Verifier + mobile** | **Free** |
|
| 92 |
+
| prism-coder:14b | `dcostenco/prism-coder:14b` | 8.4 GB | Default router | Standard+ |
|
| 93 |
+
| prism-coder:32b | `dcostenco/prism-coder:32b` | 16 GB | Complex tasks | Advanced+ |
|
| 94 |
|
| 95 |
+
## Comparison with SFT Models
|
| 96 |
|
| 97 |
+
| | prism-coder:1b7/14b/32b | prism-coder:4b (this) |
|
| 98 |
|---|---|---|
|
| 99 |
+
| Base model | Qwen3 | Qwen3.5 |
|
| 100 |
+
| Fine-tuning | LoRA SFT (weight modification) | System prompt engineering |
|
| 101 |
+
| Thinking mode | Disabled (empty think block) | Enabled (500 token budget) |
|
| 102 |
+
| GGUF hosting | Custom GGUF on HuggingFace | Stock weights from Qwen3.5-4B |
|
| 103 |
| Accuracy | 100% | 100% |
|
| 104 |
|
| 105 |
## Links
|
| 106 |
|
| 107 |
+
- [Ollama model page](https://ollama.com/dcostenco/prism-coder:4b) β pull and run
|
| 108 |
- [Prism MCP Server](https://github.com/dcostenco/prism-coder) β the MCP server that uses this model
|
|
|
|
| 109 |
- [Qwen3.5-4B base](https://huggingface.co/Qwen/Qwen3.5-4B) β upstream base model
|