Icodetools commited on
Commit
6b417bb
·
verified ·
1 Parent(s): a598205

Update README.md

Browse files

added more benchmarks. added more info.

Files changed (1) hide show
  1. README.md +49 -19
README.md CHANGED
@@ -34,16 +34,20 @@ tags:
34
 
35
  # Vulcan-1-Coder (GGUF)
36
 
37
- A 2.5B coding and reasoning model built by using Integer Space Artificial Selection on **[CherryPit 1.2](https://huggingface.co/DRUPEs/CherryPit-1.2-2B)**, an already excellent model made by
38
- the team at DRUPEs.
39
 
40
- Vulcan is an **Integer Space** model: it was generated and tested as a `.gguf`, using a new technique called
41
- **Integer Space Artificial Selection**. Details of the method will be published later.
 
 
42
 
43
  **Made in an afternoon.** Vulcan was produced in a single afternoon along with about 30 other models (12 in its
44
  class), all generated and tested in integer space, on a laptop with a **6 GB VRAM** GPU, using only my own tools.
45
  No training runs, no cluster.
46
 
 
 
 
47
  Named after Vulcan, the Roman god of fire and the forge, the smith who crafts tools for the gods.
48
 
49
  **Safetensors version: coming soon.**
@@ -57,15 +61,22 @@ Named after Vulcan, the Roman god of fire and the forge, the smith who crafts to
57
  ## Results
58
 
59
  Notes: This model has not been fully tested or trained further after selection. It may have flaws or issues, for example one time it send a bunch of symbols.
60
- But generally it scores high.
 
 
 
 
 
 
61
 
62
  ### AIME 2026
63
 
64
- **23 / 30 = 76.7%** with thinking on, up to 24K tokens, temperature 1.0, 1 sample, Q4_K_M.
65
- Per-problem results: [`aime2026_results.md`](aime2026_results.md).
 
66
 
67
  Run 2:
68
- **26 / 30 = 86.7%** with thinking on, up to 24k tokens, temperature 1.0, 1 sample, Q8, selectre
69
 
70
  ### Selection league
71
 
@@ -99,21 +110,40 @@ llama-server -m Vulcan-1-Coder-Q4_K_M-league.gguf -ngl 99 -c 32768 --jinja
99
 
100
  For questions, contact ryan@filioai.com
101
 
102
- ## Problems I've solved
103
-
104
- 1) Instant generation of a brand new GGUF file (takes seconds to minutes for larger models, done during the trials so there's no downtime.)
105
- 2) Natural selection replicated with great success, in just a few rounds we see success using a specific technique.
106
- 3) All done in quantised space without ever making a safetensors file. After, the GGUF can be reversed to safetensors.
107
- 4) Tensor analysis techniques taking into account the architecture of the model and the shape of it.
108
- 5) Much more that is unrelated to the scope of this model, but it's what made this possible - a lot of analysis helped along with thousands of files of data about models.
109
- 6) **This doesn't add to the model. It replaces parts of the base model. The natural selection writes over the models data, it doesn't add more to it, its genetics mutate.**
110
-
111
  ## Thanks
112
 
113
  - **The team at DRUPEs** for [CherryPit 1.2](https://huggingface.co/DRUPEs/CherryPit-1.2-2B), the base of Vulcan.
114
  - **NANI-Nithin** for the [OpenCodeReasoning-2 LoRA adapter](https://huggingface.co/NANI-Nithin/MiniCPM5-2B-OpenCodeReasoning2-LoRA).
115
- - **NVIDIA** for the [OpenCodeReasoning-2](https://huggingface.co/datasets/nvidia/OpenCodeReasoning-2) dataset that
116
- adapter was trained on ([CC-BY-4.0](https://creativecommons.org/licenses/by/4.0/)).
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
117
 
118
  ## Licence
119
 
 
34
 
35
  # Vulcan-1-Coder (GGUF)
36
 
37
+ ![image](https://cdn-uploads.huggingface.co/production/uploads/6aa007cf4ae94de9b1aaa30c/uzEAciENqItRjhT0bwydS.png)
 
38
 
39
+ A 2.5B coding and reasoning model built by using Integer Space Artificial Selection on **[CherryPit 1.2](https://huggingface.co/DRUPEs/CherryPit-1.2-2B)**,
40
+ an already excellent model made by the team at DRUPEs. This is a technique I created to solve many problems I faced when wanting to create models on a basic
41
+ graphics card. So we work in integer space, rather than weight space. Vulcan is an **Integer Space** model: it was generated and tested as a `.gguf`. Details
42
+ of the method will be published later and released for all. I'm working hard to put everything together.
43
 
44
  **Made in an afternoon.** Vulcan was produced in a single afternoon along with about 30 other models (12 in its
45
  class), all generated and tested in integer space, on a laptop with a **6 GB VRAM** GPU, using only my own tools.
46
  No training runs, no cluster.
47
 
48
+ Every run, lasting just a few hours, resulted in success. The techique can work for all types of models. It's a data-led technique where the training is
49
+ squeezed out rather than training it on more data. If the information exists in the model, we select for it.
50
+
51
  Named after Vulcan, the Roman god of fire and the forge, the smith who crafts tools for the gods.
52
 
53
  **Safetensors version: coming soon.**
 
61
  ## Results
62
 
63
  Notes: This model has not been fully tested or trained further after selection. It may have flaws or issues, for example one time it send a bunch of symbols.
64
+ But generally it scores high. This is with no tools AND quantised.
65
+
66
+ ### AIME 2025
67
+
68
+ Run 1:
69
+ 21 / 30 = 70.0% with thinking on, up to 24K tokens, temperature 1.0, 1 sample, Q8_0. Benchmarking harness. No tools.
70
+ Per-problem results: Will add soon to a site I'm building.
71
 
72
  ### AIME 2026
73
 
74
+ Run 1:
75
+ **23 / 30 = 76.7%** with thinking on, up to 24K tokens, temperature 1.0, 1 sample, Q4_K_M. Benchmarking harness. No tools.
76
+ Per-problem results: Will add soon to a site I'm building.
77
 
78
  Run 2:
79
+ Running now, with thinking on, up to 32k tokens, temperature 1.0, 1 sample, Q8. Benchmarking harness. No tools.
80
 
81
  ### Selection league
82
 
 
110
 
111
  For questions, contact ryan@filioai.com
112
 
 
 
 
 
 
 
 
 
 
113
  ## Thanks
114
 
115
  - **The team at DRUPEs** for [CherryPit 1.2](https://huggingface.co/DRUPEs/CherryPit-1.2-2B), the base of Vulcan.
116
  - **NANI-Nithin** for the [OpenCodeReasoning-2 LoRA adapter](https://huggingface.co/NANI-Nithin/MiniCPM5-2B-OpenCodeReasoning2-LoRA).
117
+ - **NVIDIA** for the [OpenCodeReasoning-2](https://huggingface.co/datasets/nvidia/OpenCodeReasoning-2) dataset that adapter was trained on ([CC-BY-4.0](https://creativecommons.org/licenses/by/4.0/)).
118
+
119
+ ## Things to come - progress
120
+
121
+ #### Paper and files released - 50%
122
+
123
+ Hand writing the paper and hand-making the diagrams without AI so I'll release it as soon as I can as a pre-print.
124
+ I will remove secrets from the folder and clean it up. Folder currently has thousands of files and I'd say about 25% are relevant.
125
+
126
+ #### More benchmarks - ?
127
+
128
+ Whenever I feel up to it, adding some now.
129
+
130
+ #### Quant perplexity-solver
131
+
132
+ Find the quants with the best perplexity
133
+
134
+ #### Further processing - 30%
135
+
136
+ I've discovered another technique that works to post-train one of these models. I'm still perfecting it but I'm already getting good results. I think the models can be optimised further.
137
+
138
+ #### Qwen 27b - 20%
139
+
140
+ Started the analysis process.
141
+
142
+ #### STEM - 50%
143
+
144
+ Static Embedding. **Experimental native runtime** for the `.stem` model format and embedding engine. The current runtime remains intentionally narrow:
145
+ **static embedding tables, token IDs, FP32 or INT8, mean pooling and optional L2**. It is not yet a general-purpose encoder/ONNX/GGUF runner.
146
+ It creates a .STEM format that is faster, smaller, and more accurate.
147
 
148
  ## Licence
149