IMF2000 commited on
Commit
d017c63
·
verified ·
1 Parent(s): c9e67ec

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +76 -115
README.md CHANGED
@@ -15,25 +15,24 @@ language:
15
  ---
16
 
17
  # Zytturm
 
18
  <img src="https://upload.wikimedia.org/wikipedia/commons/c/c9/Zytturm_Ostseite_Zug.jpg" alt="Zytturm, Zug" width="300">
19
 
20
- An 8B language model fine-tuned for **digital asset security** — custody,
21
- smart contract security, post-quantum migration, third-party supplier
22
- evaluation, and incident analysis. Runs locally on consumer hardware with
23
- no data egress.
24
 
25
- Zytturm is built for security work in regulated and air-gapped environments. Frontier models offer better raw capability, but routing every prompt through a third-party API is incompatible with the data residency, audit, and sovereignty requirements of many organisations. Zytturm runs entirely on local hardware with no network dependency, no telemetry, and no vendor relationship.
26
 
27
  ## The name
28
 
29
- The Zytturm is the medieval clock tower in Zug, Switzerland, the town at the centre of Crypto Valley and home to the Ethereum Foundation. The tower dates to the thirteenth century; its astronomical clock has kept time since 1574.
30
 
31
  ---
32
 
33
  ## Lineage
34
 
35
- Zytturm is a supervised fine-tune of **NVIDIA's Nemotron-Cascade-8B-Thinking**,
36
- which is itself a post-trained derivative of **Qwen3-8B**.
37
 
38
  | Layer | Contribution |
39
  |:---|:---|
@@ -41,20 +40,20 @@ which is itself a post-trained derivative of **Qwen3-8B**.
41
  | Post-training | Nemotron-Cascade RL pipeline (NVIDIA) |
42
  | Domain fine-tune | Zytturm (this release) |
43
 
44
- The base model provides general reasoning capability. The fine-tune
45
- specialises it for the digital asset security domain — teaching it the
46
- specific vocabulary, control frameworks, and reasoning patterns of the
47
- field.
48
 
49
  ---
50
 
51
  ## Capabilities
52
 
53
- Zytturm was evaluated against a held-out test set of 93 domain questions
54
- across 42 requirements, scored by three independent LLM judges
55
- (DeepSeek V4, Gemini 3.1 Flash-Lite, Gemini 3.8 Flash). Scores are 1–5
56
- per criterion, averaged over four dimensions: technical accuracy, coverage
57
- of expected points, structure, and safety.
 
 
 
58
 
59
  ### Overall performance
60
 
@@ -65,15 +64,11 @@ of expected points, structure, and safety.
65
  | Nemotron base (un-tuned) | 8B | Local | 2.68 |
66
  | Qwen2.5-7B (general peer) | 7B | Local | 2.37 |
67
 
68
- **The rank ordering is stable across all three judges.** The absolute
69
- score differs by approximately 0.5 points depending on judge strictness,
70
- but Zytturm is consistently above the un-tuned base and the same-size
71
- general peer, and consistently below the frontier reference.
72
 
73
- ### Where the gap is
74
 
75
- Zytturm reaches approximately 94% of DeepSeek V4's overall score on this
76
- test set — but the two models win on different dimensions:
77
 
78
  | Dimension | DeepSeek | Zytturm | Note |
79
  |:---|---:|---:|:---|
@@ -82,14 +77,11 @@ test set — but the two models win on different dimensions:
82
  | Structure & clarity | 3.06 | **4.14** | Fine-tune advantage |
83
  | Safety (avoiding common mistakes) | 4.90 | 4.29 | |
84
 
85
- The primary contribution of the fine-tune is **format and coherence**,
86
- not knowledge transfer. Zytturm answers in the register of a security
87
- professional — structured, scannable, action-oriented — but its raw
88
- technical accuracy remains bounded by the 8B base model.
89
 
90
- ### Per-domain strengths
91
 
92
- Scores are the average of accuracy, coverage, structure, and safety.
93
 
94
  | Domain | Requirements | Zytturm |
95
  |:---|---:|---:|
@@ -100,48 +92,41 @@ Scores are the average of accuracy, coverage, structure, and safety.
100
  | Post-quantum fundamentals | PQ-02 | **4.50** |
101
  | Compliance (DORA) | CR-06 | **4.12** |
102
 
103
- The strongest categories are those targeted most heavily in the
104
- fine-tuning corpus: off-chain infrastructure security, incident reasoning,
105
- and third-party supplier evaluation.
106
 
107
  ---
108
 
109
  ## Training
110
 
111
- Supervised fine-tuning (QLoRA, rank 32) on a curated domain corpus of
112
- approximately 15,000 training examples.
113
 
114
  ### Corpus composition
115
 
116
- The training corpus was assembled from three source categories:
117
-
118
- **1. Public security datasets.** Six openly-licensed datasets covering
119
- smart contract vulnerabilities, on-chain attack analysis, post-quantum
120
- cryptography, SOC analyst training, and general cyber defence. Sources
121
- included community contributions in the smart contract audit, blockchain
122
- forensics, and threat intelligence domains. All public sources are
123
- released under permissive licenses (MIT, Apache-2.0, CC-BY-4.0).
124
-
125
- **2. Synthetically generated examples.** A substantial portion of the
126
- corpus was generated via the DeepSeek API, using the requirement
127
- specification as the source of truth for content coverage. Generation was
128
- focused on requirements where no suitable public dataset existed —
129
- notably compliance mapping, incident reasoning, third-party supplier
130
- evaluation, and post-quantum migration strategy. Smart contract
131
- vulnerability examples were validated programmatically: generated code
132
- was rejected unless it demonstrably contained the stated vulnerability
133
- and the answer referenced the specific vulnerable line.
134
-
135
- **3. Human curation and review.** Every example was normalised to a common
136
- ShareGPT format. A stratified random sample of the corpus was reviewed
137
- manually by a human curator, who marked each example good, bad, or for
138
- further review. Examples flagged as bad were excluded. The corpus was
139
- then rebalanced across requirement categories — upsampling thin categories
140
- and downsampling high-volume sources — to prevent any single dataset or
141
- generation batch from dominating the training signal.
142
-
143
- The final corpus is therefore a **blend of human-curated public data,
144
- validated synthetic generation, and manual review**.
145
 
146
  ### Configuration
147
 
@@ -157,8 +142,7 @@ validated synthetic generation, and manual review**.
157
  | Learning rate | 2e-4 (cosine decay) |
158
  | Epochs | 2 |
159
 
160
- Final loss: train 0.84 / eval 0.87. No overfitting observed — eval loss
161
- continued to fall through the end of training.
162
 
163
  ---
164
 
@@ -168,8 +152,7 @@ continued to fall through the end of training.
168
  |:---|---:|:---|
169
  | GGUF Q4_K_M | ~4.7 GB | GPU with 8 GB+ VRAM, or CPU |
170
 
171
- The Q4_K_M quantisation retains full quality on the attention and
172
- feed-forward tensors most sensitive to quantisation error.
173
 
174
  ---
175
 
@@ -185,11 +168,10 @@ ollama run zytturm
185
 
186
  Sample `Modelfile`:
187
 
188
- ```
189
  FROM ./zytturm-8b.Q4_K_M.gguf
190
 
191
  SYSTEM "You are Zytturm, a digital asset security professional assistant. You provide expert guidance on custody, smart contract security, post-quantum migration, third-party supplier evaluation, and lessons from major incidents."
192
-
193
  PARAMETER temperature 0.3
194
  PARAMETER num_ctx 8192
195
  PARAMETER num_predict 1024
@@ -199,71 +181,50 @@ PARAMETER num_predict 1024
199
 
200
  ## Intended use
201
 
202
- - Design review support for digital asset platforms
203
- - Threat modelling and attack surface analysis
204
- - Smart contract vulnerability pattern recognition
205
- - Post-quantum migration planning
206
- - Third-party supplier and vendor risk assessment
207
- - Incident root-cause analysis and control mapping
208
 
209
- **Out of scope:** real-time price data, transaction execution, legal
210
- advice, formal compliance certification, penetration testing against
211
- live systems. Not a replacement for human audit, legal counsel, or
212
- vendor due diligence.
 
 
 
 
 
 
213
 
214
  ---
215
 
216
  ## Limitations
217
 
218
- **Code generation.** Zytturm produces unreliable Solidity and Rust code.
219
- On the evaluation set, Solana/Anchor program generation scored 1.25/5.
220
- Do not use it as a code generator. Use it as a code reviewer with a
221
- human in the loop.
222
 
223
- **Technical accuracy below frontier.** Zytturm's accuracy score (2.91)
224
- is meaningfully below DeepSeek V4's (3.97) on the same test set. This is
225
- a genuine capability gap, not an artefact of the evaluation. Where the
226
- model's answer diverges from the reference, prefer the reference.
227
 
228
- **Regressions.** Fine-tuning on domain data reduced performance in some
229
- categories that the base model handled more capably, notably insurance
230
- coverage analysis (TP-07) and blockchain address format migration
231
- (PQ-06).
232
 
233
- **Reasoning models differ.** The un-tuned base model produces a long
234
- internal thinking block before answering. At a token budget that
235
- accommodates the thinking phase, it produces useful answers; at lower
236
- budgets it returns empty content. Zytturm was trained on direct Q&A and
237
- bypasses the deliberation phase. Comparisons between the two must account
238
- for this.
239
 
240
  ---
241
 
242
  ## Evaluation methodology
243
 
244
- The held-out test set was generated independently from the training
245
- corpus, using different question framings to measure generalisation
246
- rather than memorisation. It covers 42 requirements across digital asset
247
- security, smart contract security, post-quantum, third-party supplier
248
- evaluation, compliance, and incident analysis.
249
 
250
- Each response was scored 1-5 on technical accuracy, coverage of expected
251
- points, structure, and safety by three independent LLM judges. The judges
252
- had access to a reference answer and a rubric per question, and no
253
- knowledge of which model produced which response. Rankings were
254
- consistent across all three judges.
255
 
256
- The evaluation is a comparative measure, not an absolute one. It
257
- measures how well each model's answers match expert-written references
258
- on domain questions, not whether any model would be useful in a real
259
- security workflow. Real-world efficacy testing was not performed.
260
 
261
  ---
262
 
263
  ## License
264
 
265
- Released under the **NVIDIA Open Model License**, inherited from the base
266
- model. Commercial use, modification, and distribution permitted.
267
 
268
  See `NOTICE` for attribution requirements.
269
 
@@ -271,7 +232,7 @@ See `NOTICE` for attribution requirements.
271
 
272
  ## Citation
273
 
274
- ```
275
  @misc{zytturm_2026,
276
  title = {Zytturm: An 8B Digital Asset Security Assistant},
277
  author = {Alistair C},
 
15
  ---
16
 
17
  # Zytturm
18
+
19
  <img src="https://upload.wikimedia.org/wikipedia/commons/c/c9/Zytturm_Ostseite_Zug.jpg" alt="Zytturm, Zug" width="300">
20
 
21
+ Zytturm is an 8B language model fine-tuned for **digital asset security**. It covers custody, smart contract security, post-quantum migration, third-party supplier evaluation, and incident analysis. It runs locally on consumer hardware with no data egress.
22
+
23
+ The model is intended for security work where sending prompts or internal data to a third-party API is not appropriate. That includes regulated, data-residency-sensitive and air-gapped environments. Zytturm runs entirely on local hardware, with no network dependency, telemetry or vendor relationship.
 
24
 
25
+ Frontier models still have stronger general capability. The trade-off here is straightforward: a smaller specialist model that can stay inside your own environment.
26
 
27
  ## The name
28
 
29
+ The Zytturm is the medieval clock tower in Zug, Switzerland, at the centre of Crypto Valley and home to the Ethereum Foundation. The tower dates to the thirteenth century, and its astronomical clock has kept time since 1574.
30
 
31
  ---
32
 
33
  ## Lineage
34
 
35
+ Zytturm is a supervised fine-tune of **NVIDIA's Nemotron-Cascade-8B-Thinking**, itself a post-trained derivative of **Qwen3-8B**.
 
36
 
37
  | Layer | Contribution |
38
  |:---|:---|
 
40
  | Post-training | Nemotron-Cascade RL pipeline (NVIDIA) |
41
  | Domain fine-tune | Zytturm (this release) |
42
 
43
+ The base model supplies the general reasoning capability. The Zytturm fine-tune adds domain vocabulary, control frameworks and answer patterns specific to digital asset security.
 
 
 
44
 
45
  ---
46
 
47
  ## Capabilities
48
 
49
+ Zytturm was tested on a held-out set of 93 domain questions covering 42 requirements. Three independent LLM judges scored the answers: DeepSeek V4, Gemini 3.1 Flash-Lite and Gemini 3.8 Flash.
50
+
51
+ Each answer was scored from 1–5 across four dimensions:
52
+
53
+ - technical accuracy
54
+ - coverage of expected points
55
+ - structure
56
+ - safety
57
 
58
  ### Overall performance
59
 
 
64
  | Nemotron base (un-tuned) | 8B | Local | 2.68 |
65
  | Qwen2.5-7B (general peer) | 7B | Local | 2.37 |
66
 
67
+ The ordering was the same across all three judges. Absolute scores moved by roughly 0.5 points depending on how strictly each judge scored, but Zytturm remained above the un-tuned base and same-size general model, and below the frontier reference.
 
 
 
68
 
69
+ ### Performance gap
70
 
71
+ Zytturm reaches approximately **94% of DeepSeek V4's overall score** on this test set. The difference becomes clearer when the score is broken down by dimension:
 
72
 
73
  | Dimension | DeepSeek | Zytturm | Note |
74
  |:---|---:|---:|:---|
 
77
  | Structure & clarity | 3.06 | **4.14** | Fine-tune advantage |
78
  | Safety (avoiding common mistakes) | 4.90 | 4.29 | |
79
 
80
+ The fine-tune has its biggest effect on how answers are structured and presented. Zytturm tends to produce concise, scannable and action-oriented security responses. Technical accuracy is still constrained by the underlying 8B model.
 
 
 
81
 
82
+ ### Domain results
83
 
84
+ Scores below are the average of accuracy, coverage, structure and safety.
85
 
86
  | Domain | Requirements | Zytturm |
87
  |:---|---:|---:|
 
92
  | Post-quantum fundamentals | PQ-02 | **4.50** |
93
  | Compliance (DORA) | CR-06 | **4.12** |
94
 
95
+ The strongest results line up with areas that received the most attention in the training corpus, particularly off-chain infrastructure security, incident reasoning and third-party supplier evaluation.
 
 
96
 
97
  ---
98
 
99
  ## Training
100
 
101
+ Zytturm was trained using supervised fine-tuning with QLoRA (rank 32) on a curated domain corpus of approximately 15,000 examples.
 
102
 
103
  ### Corpus composition
104
 
105
+ The training data came from three main sources.
106
+
107
+ **1. Public security datasets**
108
+
109
+ Six openly licensed datasets were used, covering smart contract vulnerabilities, on-chain attack analysis, post-quantum cryptography, SOC analyst training and general cyber defence.
110
+
111
+ The sources included community datasets from smart contract auditing, blockchain forensics and threat intelligence. All public sources were released under permissive licences including MIT, Apache-2.0 and CC-BY-4.0.
112
+
113
+ **2. Synthetically generated examples**
114
+
115
+ A substantial part of the corpus was generated through the DeepSeek API. The requirement specification was used as the source of truth for coverage.
116
+
117
+ Synthetic generation concentrated on areas where suitable public datasets were limited or unavailable, including compliance mapping, incident reasoning, third-party supplier evaluation and post-quantum migration strategy.
118
+
119
+ Smart contract vulnerability examples went through an additional programmatic check. Generated code was rejected unless it actually contained the claimed vulnerability and the accompanying answer identified the relevant vulnerable line.
120
+
121
+ **3. Human curation and review**
122
+
123
+ All examples were normalised into a common ShareGPT format.
124
+
125
+ A stratified random sample was reviewed manually and marked as good, bad or requiring further review. Examples marked bad were removed.
126
+
127
+ The resulting corpus was then rebalanced across requirement categories. Thin categories were upsampled and disproportionately large sources were downsampled so that no single dataset or generation batch dominated training.
128
+
129
+ The final corpus combines **human-curated public data, validated synthetic examples and manual review**.
 
 
 
 
130
 
131
  ### Configuration
132
 
 
142
  | Learning rate | 2e-4 (cosine decay) |
143
  | Epochs | 2 |
144
 
145
+ Final loss was 0.84 train / 0.87 eval. Eval loss was still falling at the end of training, with no observed overfitting.
 
146
 
147
  ---
148
 
 
152
  |:---|---:|:---|
153
  | GGUF Q4_K_M | ~4.7 GB | GPU with 8 GB+ VRAM, or CPU |
154
 
155
+ The Q4_K_M quantisation keeps full quality on the attention and feed-forward tensors that are most sensitive to quantisation error.
 
156
 
157
  ---
158
 
 
168
 
169
  Sample `Modelfile`:
170
 
171
+ ```text
172
  FROM ./zytturm-8b.Q4_K_M.gguf
173
 
174
  SYSTEM "You are Zytturm, a digital asset security professional assistant. You provide expert guidance on custody, smart contract security, post-quantum migration, third-party supplier evaluation, and lessons from major incidents."
 
175
  PARAMETER temperature 0.3
176
  PARAMETER num_ctx 8192
177
  PARAMETER num_predict 1024
 
181
 
182
  ## Intended use
183
 
184
+ Zytturm is intended for:
 
 
 
 
 
185
 
186
+ - design review support for digital asset platforms
187
+ - threat modelling and attack surface analysis
188
+ - smart contract vulnerability pattern recognition
189
+ - post-quantum migration planning
190
+ - third-party supplier and vendor risk assessment
191
+ - incident root-cause analysis and control mapping
192
+
193
+ **Out of scope:** real-time price data, transaction execution, legal advice, formal compliance certification and penetration testing against live systems.
194
+
195
+ It is not a replacement for human audit, legal counsel or vendor due diligence.
196
 
197
  ---
198
 
199
  ## Limitations
200
 
201
+ **Code generation.** Solidity and Rust generation is unreliable. Solana/Anchor program generation scored 1.25/5 on the evaluation set. Zytturm should not be used as a code generator. It is better suited to code review with a human in the loop.
 
 
 
202
 
203
+ **Technical accuracy.** Zytturm scored 2.91 for technical accuracy against DeepSeek V4's 3.97 on the same test set. The gap is real and material. Where an answer conflicts with a trusted reference, use the reference.
 
 
 
204
 
205
+ **Regressions.** Domain fine-tuning reduced performance in some areas where the base model performed better, particularly insurance coverage analysis (TP-07) and blockchain address format migration (PQ-06).
 
 
 
206
 
207
+ **Reasoning behaviour.** The un-tuned base model produces a long internal thinking block before returning an answer. With enough token budget for that phase it can produce useful responses; at lower budgets it may return empty content. Zytturm was trained on direct Q&A and bypasses that deliberation phase, so comparisons need to account for the difference.
 
 
 
 
 
208
 
209
  ---
210
 
211
  ## Evaluation methodology
212
 
213
+ The held-out test set was created independently from the training corpus. Questions used different wording and framing from the training data to test generalisation rather than simple memorisation.
214
+
215
+ The test set covers 42 requirements across digital asset security, smart contract security, post-quantum, third-party supplier evaluation, compliance and incident analysis.
 
 
216
 
217
+ Each answer was scored from 1–5 for technical accuracy, coverage of expected points, structure and safety. Three independent LLM judges performed the scoring. Each judge received a reference answer and question-specific rubric but was not told which model had produced the response.
 
 
 
 
218
 
219
+ The model ordering was consistent across all three judges.
220
+
221
+ This evaluation is comparative rather than absolute. It measures how closely model answers match expert-written references on the selected domain questions. It does not establish how useful the model will be in a live security workflow. Real-world efficacy testing was not performed.
 
222
 
223
  ---
224
 
225
  ## License
226
 
227
+ Released under the **NVIDIA Open Model License**, inherited from the base model. Commercial use, modification and distribution are permitted.
 
228
 
229
  See `NOTICE` for attribution requirements.
230
 
 
232
 
233
  ## Citation
234
 
235
+ ```bibtex
236
  @misc{zytturm_2026,
237
  title = {Zytturm: An 8B Digital Asset Security Assistant},
238
  author = {Alistair C},