dcostenco commited on
Commit
28ff1d0
Β·
verified Β·
1 Parent(s): 2ff352f

Update model card: prompt-engineered (not stock), Ollama tag live

Browse files
Files changed (1) hide show
  1. README.md +39 -36
README.md CHANGED
@@ -8,16 +8,24 @@ tags:
8
  - qwen3.5
9
  - synalux
10
  - prompt-engineering
 
11
  base_model: Qwen/Qwen3.5-4B
12
  pipeline_tag: text-generation
13
  ---
14
 
15
  # prism-coder:4b β€” 17-Tool Memory Agent (Verifier + Mobile Tier)
16
 
17
- Stock [Qwen3.5-4B](https://huggingface.co/Qwen/Qwen3.5-4B) with prompt engineering for full Prism Memory tool routing in the [Prism Coder](https://ollama.com/dcostenco/prism-coder) system.
18
  Primary deployment: **mobile / 8 GB+ devices** via Ollama β€” the lightweight verifier tier.
19
 
20
- **No fine-tuning required.** Qwen3.5-4B achieves 100% routing accuracy through system prompt engineering + thinking enabled (500 token budget). This is a drop-in replacement for the retired prism-coder:4b SFT model.
 
 
 
 
 
 
 
21
 
22
  ## BFCL Benchmark β€” 100% Γ— 3 Seeds
23
 
@@ -40,67 +48,62 @@ Primary deployment: **mobile / 8 GB+ devices** via Ollama β€” the lightweight ve
40
 
41
  115 test cases, 3 shuffled runs (seeds 2027/2028/2029), temperature=0, 0 hallucinations across all runs.
42
 
43
- ## Why No Fine-Tuning?
44
 
45
- Qwen3.5-4B's linear attention architecture + native thinking capability makes it uniquely suited to tool routing via prompt engineering alone. The thinking chain reasons through routing rules before selecting a tool, achieving the same 100% accuracy as SFT'd models in the fleet.
46
 
47
- Key insight: enabling thinking mode (not empty `<think>` blocks) is critical β€” Qwen3.5 needs to reason through the routing rules in its think chain.
 
 
 
48
 
49
- ## How to Use
50
 
51
- ```bash
52
- # Pull stock model from Ollama
53
- ollama pull qwen3.5:4b
54
- ```
 
 
 
55
 
56
- The system prompt is injected at runtime by [Prism MCP Server](https://github.com/dcostenco/prism-coder). No custom Modelfile or GGUF needed β€” use the stock Ollama model directly.
57
 
58
  ## Tools
59
 
60
- Routes to 17 Prism Memory tools + knows when NOT to call any tool:
61
 
62
  | Tool | Trigger |
63
  |------|---------|
64
  | `session_load_context` | Load/resume project context |
65
  | `session_save_ledger` | Log/record completed work |
66
  | `session_save_handoff` | Create handoff note for next session |
67
- | `session_search_memory` | Recall prior discussions |
68
- | `session_forget_memory` | Delete a memory entry |
69
- | `session_health_check` | Check session system health |
70
  | `session_compact_ledger` | Compact/prune session ledger |
71
- | `session_export_memory` | Export session data |
72
- | `session_task_route` | Route task: local vs cloud |
73
- | `session_save_experience` | Save a notable experience |
74
- | `session_synthesize_edges` | Build knowledge graph edges |
75
- | `session_backfill_links` | Backfill missing memory links |
76
  | `knowledge_search` | Search accumulated knowledge |
77
- | `knowledge_forget` | Prune bad/old knowledge |
78
- | `knowledge_upvote` | Upvote useful knowledge |
79
- | `knowledge_downvote` | Downvote unhelpful knowledge |
80
- | `knowledge_set_retention` | Set knowledge retention policy |
81
- | *(no tool)* | General questions, greetings, code, math |
82
 
83
  ## Fleet Position
84
 
85
  | Model | Ollama tag | Size | Role | Tier |
86
  |---|---|---|---|---|
87
- | prism-coder:1.7b | `prism-coder:1b7` | 1.1 GB | Always-fits fallback | Free |
88
- | **Qwen3.5-4B** | **`qwen3.5:4b`** | **3.4 GB** | **Verifier + mobile** | **Free** |
89
- | prism-coder:14b | `prism-coder:14b` | 8.4 GB | Default router | Standard+ |
90
- | prism-coder:32b | `prism-coder:32b` | 16 GB | Complex tasks | Advanced+ |
91
 
92
- ## Differences from SFT Models
93
 
94
- | | prism-coder:1b7/14b/32b | Qwen3.5-4B (this) |
95
  |---|---|---|
96
- | Training | LoRA SFT on Qwen3 | None (stock weights) |
97
- | System prompt | Baked into Modelfile | Injected at runtime |
98
- | Thinking | Disabled (empty think) | Enabled (500 token budget) |
99
- | GGUF hosting | Custom GGUF on HuggingFace | Stock from Ollama registry |
100
  | Accuracy | 100% | 100% |
101
 
102
  ## Links
103
 
 
104
  - [Prism MCP Server](https://github.com/dcostenco/prism-coder) β€” the MCP server that uses this model
105
- - [Ollama model page](https://ollama.com/dcostenco/prism-coder) β€” all fleet models
106
  - [Qwen3.5-4B base](https://huggingface.co/Qwen/Qwen3.5-4B) β€” upstream base model
 
8
  - qwen3.5
9
  - synalux
10
  - prompt-engineering
11
+ - gguf
12
  base_model: Qwen/Qwen3.5-4B
13
  pipeline_tag: text-generation
14
  ---
15
 
16
  # prism-coder:4b β€” 17-Tool Memory Agent (Verifier + Mobile Tier)
17
 
18
+ Prompt-engineered [Qwen3.5-4B](https://huggingface.co/Qwen/Qwen3.5-4B) for full Prism Memory tool routing in the [Prism Coder](https://ollama.com/dcostenco/prism-coder) system.
19
  Primary deployment: **mobile / 8 GB+ devices** via Ollama β€” the lightweight verifier tier.
20
 
21
+ **Fine-tuned via system prompt engineering.** Qwen3.5-4B achieves 100% routing accuracy through a structured routing system prompt + thinking enabled (500 token budget). No LoRA or weight modification β€” the system prompt IS the specialization, baked into the [Ollama Modelfile](https://ollama.com/dcostenco/prism-coder:4b).
22
+
23
+ ## Quick Start
24
+
25
+ ```bash
26
+ ollama pull dcostenco/prism-coder:4b
27
+ ollama run dcostenco/prism-coder:4b
28
+ ```
29
 
30
  ## BFCL Benchmark β€” 100% Γ— 3 Seeds
31
 
 
48
 
49
  115 test cases, 3 shuffled runs (seeds 2027/2028/2029), temperature=0, 0 hallucinations across all runs.
50
 
51
+ ## How It Works
52
 
53
+ Qwen3.5-4B's linear attention architecture + native thinking capability makes it uniquely suited to tool routing via prompt engineering. The model reasons through a structured routing ruleset in its think chain before selecting a tool β€” achieving the same 100% accuracy as LoRA-fine-tuned models in the fleet.
54
 
55
+ Key design decisions:
56
+ - **Thinking enabled** (not empty `<think>` blocks) β€” Qwen3.5 needs to reason through routing rules in its think chain. The forced empty-think pattern from Qwen3 breaks it.
57
+ - **500 token thinking budget** β€” enough to reason, not enough to overthink. Higher budgets (800+) cause analysis paralysis on compound queries.
58
+ - **Structured routing rules** β€” top-to-bottom priority matching with disambiguation rules and a no-tool override section.
59
 
60
+ SFT was attempted (3 rounds) but consistently hurt accuracy β€” the base model + prompt outperforms every SFT variant.
61
 
62
+ ## System Prompt
63
+
64
+ The full routing system prompt is baked into the Ollama Modelfile. It includes:
65
+ - 6 tool definitions with exact names
66
+ - 13 priority-ordered routing rules
67
+ - Disambiguation rules for edge cases (knowledge_search vs session_search_memory, code requests vs tool calls)
68
+ - No-tool override section for categories that should never trigger tools
69
 
70
+ See the [Ollama model page](https://ollama.com/dcostenco/prism-coder:4b) for the complete Modelfile.
71
 
72
  ## Tools
73
 
74
+ Routes to 6 Prism Memory tools + knows when NOT to call any tool:
75
 
76
  | Tool | Trigger |
77
  |------|---------|
78
  | `session_load_context` | Load/resume project context |
79
  | `session_save_ledger` | Log/record completed work |
80
  | `session_save_handoff` | Create handoff note for next session |
 
 
 
81
  | `session_compact_ledger` | Compact/prune session ledger |
82
+ | `session_search_memory` | Recall prior discussions |
 
 
 
 
83
  | `knowledge_search` | Search accumulated knowledge |
84
+ | *(no tool)* | General questions, greetings, code, math, AAC, translation |
 
 
 
 
85
 
86
  ## Fleet Position
87
 
88
  | Model | Ollama tag | Size | Role | Tier |
89
  |---|---|---|---|---|
90
+ | prism-coder:1.7b | `dcostenco/prism-coder:1b7` | 1.1 GB | Always-fits fallback | Free |
91
+ | **prism-coder:4b** | **`dcostenco/prism-coder:4b`** | **3.4 GB** | **Verifier + mobile** | **Free** |
92
+ | prism-coder:14b | `dcostenco/prism-coder:14b` | 8.4 GB | Default router | Standard+ |
93
+ | prism-coder:32b | `dcostenco/prism-coder:32b` | 16 GB | Complex tasks | Advanced+ |
94
 
95
+ ## Comparison with SFT Models
96
 
97
+ | | prism-coder:1b7/14b/32b | prism-coder:4b (this) |
98
  |---|---|---|
99
+ | Base model | Qwen3 | Qwen3.5 |
100
+ | Fine-tuning | LoRA SFT (weight modification) | System prompt engineering |
101
+ | Thinking mode | Disabled (empty think block) | Enabled (500 token budget) |
102
+ | GGUF hosting | Custom GGUF on HuggingFace | Stock weights from Qwen3.5-4B |
103
  | Accuracy | 100% | 100% |
104
 
105
  ## Links
106
 
107
+ - [Ollama model page](https://ollama.com/dcostenco/prism-coder:4b) β€” pull and run
108
  - [Prism MCP Server](https://github.com/dcostenco/prism-coder) β€” the MCP server that uses this model
 
109
  - [Qwen3.5-4B base](https://huggingface.co/Qwen/Qwen3.5-4B) β€” upstream base model