dcostenco commited on
Commit
0d24c05
Β·
verified Β·
1 Parent(s): 41e04f6

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +43 -49
README.md CHANGED
@@ -7,85 +7,79 @@ tags:
7
  - prism-coder
8
  - qwen3.5
9
  - synalux
 
10
  - gguf
11
  base_model: Qwen/Qwen3.5-4B
12
  pipeline_tag: text-generation
13
  ---
14
 
15
- # prism-coder:2b β€” iPhone / Mobile First Gate
16
 
17
- Qwen3.5-4B quantized to Q3_K_M (2.3 GB) for on-device MCP tool routing on iPhone and mobile devices. Part of the [Prism Coder](https://ollama.com/dcostenco/prism-coder) fleet.
18
 
19
- **99.1% routing accuracy** (BFCL, 115 cases × 3 seeds). Single failure: regex→knowledge_search edge case.
 
 
 
 
 
20
 
21
  ## Quick Start
22
 
23
  ```bash
 
24
  ollama pull dcostenco/prism-coder:2b
25
- ollama run dcostenco/prism-coder:2b
26
- ```
27
-
28
- ## Download
29
 
30
- | File | Size | BFCL | SHA256 |
31
- |------|------|------|--------|
32
- | [Qwen3.5-4B-Q3_K_M.gguf](https://huggingface.co/dcostenco/prism-coder-4b/resolve/main/Qwen3.5-4B-Q3_K_M.gguf) | 2.3 GB | 99.1% | `d6981ab4d77ba712b48ef69d69042d75b5e39b9dce5fb5a5b054fd08e06afb95` |
33
 
34
- ## BFCL Benchmark β€” 99.1% Γ— 3 Seeds
35
 
36
- **114/115 Γ— 3 shuffled runs = 99.1%, 1 flaky case, stable across all seeds**
37
 
38
- | Category | Count | Description | Accuracy |
39
- |----------|------:|-------------|:--------:|
40
- | save | 17 | session_save_ledger logging | 100% |
41
- | smem | 17 | Session memory search/recall | 100% |
42
- | aac | 12 | AAC phrase/communication routing | 100% |
43
- | hand | 12 | Agent handoff (session_save_handoff) | 100% |
44
- | irrel | 10 | Irrelevant/general questions β†’ plain text | 90% |
45
- | load | 9 | Context loading (session_load_context) | 100% |
46
- | pred | 8 | Prediction/task routing | 100% |
47
- | know | 7 | Knowledge search/recall | 100% |
48
- | cmpct | 6 | Compact/prune ledger | 100% |
49
- | edge | 6 | Edge cases (self-description, ambiguous) | 100% |
50
- | tran | 6 | Translation routing | 100% |
51
- | info | 5 | Info/capability queries β†’ plain text | 100% |
52
 
53
- Single failure: `"Write a regex to match email addresses"` β†’ knowledge_search instead of plain text. All other 114 cases correct across all 3 seeds (2027/2028/2029).
 
 
 
 
 
 
 
 
 
 
 
 
 
54
 
55
- ### Comparison: Q3_K_M vs Q4_K_M vs old 1.7B
56
 
57
- | Model | GGUF Size | BFCL | Avg Latency | Failures |
58
- |-------|-----------|------|-------------|----------|
59
- | **prism-coder:2b (Q3_K_M)** | **2.3 GB** | **99.1%** | **1.79s** | **1** |
60
- | prism-coder:4b (Q4_K_M) | 3.4 GB | 100% | 1.37s | 0 |
61
- | old prism-coder:1b7 (Qwen3 SFT) | 2.2 GB | 90.4% | 0.30s | 11 |
62
 
63
- The Q3_K_M replaces the old 1.7B SFT: same size (+100 MB), +9 percentage points accuracy, 10 fewer failures.
64
 
65
  ## Architecture
66
 
67
  Qwen3.5-4B uses a hybrid attention architecture:
68
- - **24 linear attention layers** (Gated DeltaNet) β€” O(n) inference, efficient on-device
69
- - **8 full attention layers** (standard softmax) β€” precise routing decisions
70
-
71
- The `2b` tag reflects the memory footprint (2.3 GB), not the parameter count (4B). No fine-tuning β€” the system prompt is baked into the Ollama Modelfile.
72
-
73
- ### Why not SFT?
74
 
75
- Qwen3.5's DeltaNet linear attention layers are incompatible with LoRA fine-tuning β€” every SFT attempt regressed accuracy below the base model. The hybrid architecture means LoRA only reaches 8 of 32 layers. Prompt engineering achieves 99.1-100% without any weight modification.
76
 
77
  ## Fleet Position
78
 
79
  | Model | Ollama tag | Size | BFCL | Role |
80
  |---|---|---|---|---|
81
- | **Qwen3.5-4B Q3_K_M** | **`prism-coder:2b`** | **2.3 GB** | **99.1%** | **iPhone / mobile** |
82
- | Qwen3.5-4B Q4_K_M | `prism-coder:4b` | 3.4 GB | 100% | Verifier / 8 GB+ |
83
- | prism-coder:14b | `prism-coder:14b` | 8.4 GB | 100% | Mac default |
84
- | prism-coder:32b | `prism-coder:32b` | 16 GB | 100% | Complex tasks |
85
 
86
  ## Links
87
 
88
- - [Ollama model page](https://ollama.com/dcostenco/prism-coder:2b)
89
- - [GGUF download](https://huggingface.co/dcostenco/prism-coder-4b/resolve/main/Qwen3.5-4B-Q3_K_M.gguf)
90
- - [Prism MCP Server](https://github.com/dcostenco/prism-coder)
91
- - [Qwen3.5-4B base](https://huggingface.co/Qwen/Qwen3.5-4B)
 
7
  - prism-coder
8
  - qwen3.5
9
  - synalux
10
+ - prompt-engineering
11
  - gguf
12
  base_model: Qwen/Qwen3.5-4B
13
  pipeline_tag: text-generation
14
  ---
15
 
16
+ # prism-coder:4b β€” Prism Memory Tool Router
17
 
18
+ Prompt-engineered [Qwen3.5-4B](https://huggingface.co/Qwen/Qwen3.5-4B) for MCP tool routing in the [Prism Coder](https://ollama.com/dcostenco/prism-coder) system. No fine-tuning β€” the system prompt IS the specialization.
19
 
20
+ ## Downloads
21
+
22
+ | File | Quantization | Size | BFCL Accuracy | Use when |
23
+ |------|-------------|------|---------------|----------|
24
+ | `Qwen3.5-4B-Q3_K_M.gguf` | Q3_K_M | **2.3 GB** | **99.1%** Γ— 3 seeds | **iPhone / mobile first gate** |
25
+ | *(stock via Ollama)* | Q4_K_M | 3.4 GB | **100%** Γ— 3 seeds | Mac / 8 GB+ devices |
26
 
27
  ## Quick Start
28
 
29
  ```bash
30
+ # iPhone-optimized (2.3 GB, 99.1%)
31
  ollama pull dcostenco/prism-coder:2b
 
 
 
 
32
 
33
+ # Full quality (3.4 GB, 100%)
34
+ ollama pull dcostenco/prism-coder:4b
35
+ ```
36
 
37
+ ## BFCL Benchmark
38
 
39
+ ### Q3_K_M (prism-coder:2b) β€” 99.1% Γ— 3 seeds
40
 
41
+ **114/115 Γ— 3 shuffled runs = 99.1%, 1 flaky case**
 
 
 
 
 
 
 
 
 
 
 
 
 
42
 
43
+ | Category | Count | Accuracy |
44
+ |----------|------:|:--------:|
45
+ | save | 17 | 100% |
46
+ | smem | 17 | 100% |
47
+ | aac | 12 | 100% |
48
+ | hand | 12 | 100% |
49
+ | irrel | 10 | 90% |
50
+ | load | 9 | 100% |
51
+ | pred | 8 | 100% |
52
+ | know | 7 | 100% |
53
+ | cmpct | 6 | 100% |
54
+ | edge | 6 | 100% |
55
+ | tran | 6 | 100% |
56
+ | info | 5 | 100% |
57
 
58
+ Single failure: "Write a regex to match email addresses" β†’ knowledge_search instead of plain.
59
 
60
+ ### Q4_K_M (prism-coder:4b) β€” 100% Γ— 3 seeds
 
 
 
 
61
 
62
+ **115/115 Γ— 3 shuffled runs = 100.0%, 0 flaky**
63
 
64
  ## Architecture
65
 
66
  Qwen3.5-4B uses a hybrid attention architecture:
67
+ - **24 linear attention layers** (Gated DeltaNet) β€” O(n) inference
68
+ - **8 full attention layers** (standard softmax) β€” precise retrieval
 
 
 
 
69
 
70
+ This hybrid design is why prompt-only routing works at 4B scale but not smaller. The 8 full-attention layers are sufficient to hold the routing rules when combined with the DeltaNet layers' pattern matching.
71
 
72
  ## Fleet Position
73
 
74
  | Model | Ollama tag | Size | BFCL | Role |
75
  |---|---|---|---|---|
76
+ | **Qwen3.5-4B Q3_K_M** | **`dcostenco/prism-coder:2b`** | **2.3 GB** | **99.1%** | **iPhone / mobile** |
77
+ | Qwen3.5-4B Q4_K_M | `dcostenco/prism-coder:4b` | 3.4 GB | 100% | Verifier / 8 GB+ |
78
+ | Qwen3.5-9B Q4_K_M | `dcostenco/prism-coder:9b` | 5.8 GB | 100% | Default router |
79
+ | prism-coder:32b | `dcostenco/prism-coder:32b` | 19 GB | 100% | Complex tasks |
80
 
81
  ## Links
82
 
83
+ - [Ollama model page](https://ollama.com/dcostenco/prism-coder) β€” pull and run
84
+ - [Prism MCP Server](https://github.com/dcostenco/prism-coder) β€” the MCP server
85
+ - [Qwen3.5-4B base](https://huggingface.co/Qwen/Qwen3.5-4B) β€” upstream model