Hodlforjesus commited on
Commit
e157e7a
路
verified 路
1 Parent(s): 46ecde1

Built at 8a31755eabf1a779d75f65ba6621d585cf116ccd

Browse files
Files changed (4) hide show
  1. README.md +28 -29
  2. config.json +25 -14
  3. model.safetensors +2 -2
  4. state.bin +2 -2
README.md CHANGED
@@ -27,20 +27,20 @@ are fixed operations of the engine; the network only chooses between them.
27
  What it knows is in the tree, where it can be read and changed, and not in the weights. The
28
  network learns only how to shape and read that memory. That is why it is small: it is not a
29
  transformer, it was not trained on a large corpus, and it does not write free text. These files
30
- hold 5,596,839 numbers, three networks of 1,865,613 numbers each that read by vote, and the
31
  permanent state the project's seeds make. It runs on a processor, and in a browser through
32
  WebAssembly, where the Datamine Network chat runs it with nothing sent to a server.
33
 
34
  ## 馃搳 How it compares
35
 
36
- Every model was asked the same 2,669 questions: the questions of the held-out lines of DAM1's
37
- curriculum, which no network was trained on. Each item is a short text and one question about it.
38
  One rule scores every reply. The other models are also given an order and two worked examples,
39
  which DAM1 is not; they run in float16 on an NVIDIA RTX 3080 Ti, DAM1 on the processor.
40
 
41
  | Model | Parameters | Weights | Memory | Answered | Exact | One answer |
42
  |---|---:|---:|---:|---:|---:|---:|
43
- | **DAM1** | **5,596,839** | **22 MB** | **43 MB** | **100.0%** | **100.0%** | **3 ms** |
44
  | Qwen2.5-0.5B-Instruct | 494,032,768 | 988 MB | 1.01 GB | 63.8% | 39.6% | 109 ms |
45
  | Qwen3-0.6B | 596,049,920 | 1.50 GB | 1.22 GB | 52.7% | 41.4% | 80 ms |
46
  | LFM2-350M | 354,483,968 | 709 MB | 727 MB | 52.2% | 44.1% | 53 ms |
@@ -48,7 +48,7 @@ which DAM1 is not; they run in float16 on an NVIDIA RTX 3080 Ti, DAM1 on the pro
48
  | SmolLM2-135M-Instruct | 134,515,008 | 269 MB | 285 MB | 36.0% | 14.5% | 228 ms |
49
 
50
  **Weights** is the file the hub publishes. DAM1's page downloads the same network written
51
- small, 10 MB. **Memory** is what answering one question takes: for DAM1 the one block of
52
  WebAssembly memory that holds the network, the state, the tree and the stack; for the others the
53
  most the card held over the same questions. **Answered** is the share of replies that say the
54
  answer. **Exact** is the share that begin with the answer and put nothing before it. **One answer**
@@ -66,8 +66,7 @@ not. The harness is in
66
  - **Model type:** an LLM built as a reader of one word at a time over a tree, driven by networks
67
  over a stack: an embedding per feature place, a block of weights per stack slot, one hidden
68
  layer of rectified units, an output per step class and a pointer over the words of the
69
- sentence. Three networks of one shape, trained from different starts, read by vote. They are
70
- trained from nothing; there is no base model.
71
  - **Language:** English.
72
  - **Licence:** AGPL-3.0-or-later.
73
  - **Files built at:** the commit `config.json` records under `build`, which the publish writes as it exports.
@@ -144,12 +143,11 @@ Out of scope:
144
  of features hashed into a space of 2^64 places.
145
  5. A network reads the stack, one event per slot of 200. An event's embedding is the sum of the
146
  16-wide embeddings of its places. Each slot has its own weights into 512 hidden rectified
147
- units. The outputs take a softmax over the 92 step classes. A class is a move and its
148
  properties, such as `{grab}`, `{drop @}`, `{children add: @ type: property}` or `{get owner}`.
149
- 6. The three networks vote: the shares each gives every step are added and the step with the
150
- largest sum is taken. When the step takes a word of the sentence (the `@`), the pointers score
151
  every word on the stack the same way and the best one is taken.
152
- 7. The step runs, its two events join the stack, and the networks choose again. `{continue}`
153
  goes on to the next word. A word may take at most 50 steps.
154
  8. What a sentence wrote to the tree stays for the next sentence and the next turn, so a question
155
  is answered from everything the conversation said and from the seeds.
@@ -216,22 +214,22 @@ not recorded, so the card gives no `co2_eq_emissions`.
216
 
217
  | File | Holds |
218
  |---|---|
219
- | `config.json` | The step limit, the network's shape, its voters and its step classes, and the build commit |
220
- | `model.safetensors` | The eight tensors of each of the three voters, listed below |
221
  | `state.bin` | The permanent state every chat starts from: the tree the seeds make |
222
  | `LICENSE`, `NOTICE.md` | The licence and what it covers |
223
 
224
- The tensors of `model.safetensors`, for each voter `N` of 0, 1 and 2:
225
 
226
  | Tensor | Type | Shape |
227
  |---|---|---|
228
- | `network.N.items` | `U64` | 4,240 |
229
- | `network.N.item_table` | `F32` | 4,240 by 16 |
230
  | `network.N.slot_weights` | `F32` | 200 by 512 by 16 |
231
  | `network.N.fill_weights` | `F32` | 200 by 512 |
232
  | `network.N.hidden_bias` | `F32` | 512 |
233
- | `network.N.output_weights` | `F32` | 92 by 512 |
234
- | `network.N.output_bias` | `F32` | 92 |
235
  | `network.N.point_weights` | `F32` | 512 by 16 |
236
 
237
  No file contains code. A tensor runtime cannot run DAM1; it is read by the dam-core crate.
@@ -297,23 +295,24 @@ expected answer.
297
  |---|---:|---:|
298
  | Basics | 446 of 446 (100.0%) | 142 of 142 (100.0%) |
299
  | Grade 1 | 73 of 73 (100.0%) | 24 of 24 (100.0%) |
300
- | Grade 2 | 676 of 676 (100.0%) | 239 of 239 (100.0%) |
301
- | Grade 3 | 246 of 246 (100.0%) | 97 of 97 (100.0%) |
302
- | Grade 4 | 265 of 265 (100.0%) | 88 of 88 (100.0%) |
303
- | Grade 5 | 172 of 172 (100.0%) | 72 of 72 (100.0%) |
304
- | Grade 6 | 156 of 156 (100.0%) | 58 of 58 (100.0%) |
305
- | Grade 7 | 83 of 83 (100.0%) | 35 of 35 (100.0%) |
306
  | Grade 8 | 79 of 79 (100.0%) | 33 of 33 (100.0%) |
307
- | Grade 9 | 149 of 149 (100.0%) | 65 of 66 (98.5%) |
308
  | Grade 10 | 26 of 26 (100.0%) | 9 of 9 (100.0%) |
309
- | Grade 11 | 82 of 82 (100.0%) | 23 of 23 (100.0%) |
310
  | Grade 12 | 39 of 39 (100.0%) | 11 of 11 (100.0%) |
311
  | University | 47 of 47 (100.0%) | 13 of 13 (100.0%) |
312
  | World and debug cases | 36 of 36 (100.0%) | 32 of 32 (100.0%) |
313
- | **All** | **2575 of 2575 (100.0%)** | **941 of 942 (99.9%)** |
314
 
315
- It reads every learned line of the curriculum. The one held-out line it misses is in the lesson
316
- on the parts a kind of thing has, which asks a part of a thing that is not a creature.
 
317
 
318
  The held-out lines use the same sentence shapes and many of the same words as the learned lines.
319
  They measure new words and numbers in taught shapes. They are not an independent benchmark. No
 
27
  What it knows is in the tree, where it can be read and changed, and not in the weights. The
28
  network learns only how to shape and read that memory. That is why it is small: it is not a
29
  transformer, it was not trained on a large corpus, and it does not write free text. These files
30
+ hold 1,870,616 numbers, one network, and the
31
  permanent state the project's seeds make. It runs on a processor, and in a browser through
32
  WebAssembly, where the Datamine Network chat runs it with nothing sent to a server.
33
 
34
  ## 馃搳 How it compares
35
 
36
+ Every model was asked the questions of the held-out lines of DAM1's curriculum, which no network
37
+ was trained on: 2,673 for DAM1, and the 2,669 that existed when the others were run. Each item is a short text and one question about it.
38
  One rule scores every reply. The other models are also given an order and two worked examples,
39
  which DAM1 is not; they run in float16 on an NVIDIA RTX 3080 Ti, DAM1 on the processor.
40
 
41
  | Model | Parameters | Weights | Memory | Answered | Exact | One answer |
42
  |---|---:|---:|---:|---:|---:|---:|
43
+ | **DAM1** | **1,870,616** | **8 MB** | **23 MB** | **99.9%** | **99.9%** | **2 ms** |
44
  | Qwen2.5-0.5B-Instruct | 494,032,768 | 988 MB | 1.01 GB | 63.8% | 39.6% | 109 ms |
45
  | Qwen3-0.6B | 596,049,920 | 1.50 GB | 1.22 GB | 52.7% | 41.4% | 80 ms |
46
  | LFM2-350M | 354,483,968 | 709 MB | 727 MB | 52.2% | 44.1% | 53 ms |
 
48
  | SmolLM2-135M-Instruct | 134,515,008 | 269 MB | 285 MB | 36.0% | 14.5% | 228 ms |
49
 
50
  **Weights** is the file the hub publishes. DAM1's page downloads the same network written
51
+ small, 3.3 MB. **Memory** is what answering one question takes: for DAM1 the one block of
52
  WebAssembly memory that holds the network, the state, the tree and the stack; for the others the
53
  most the card held over the same questions. **Answered** is the share of replies that say the
54
  answer. **Exact** is the share that begin with the answer and put nothing before it. **One answer**
 
66
  - **Model type:** an LLM built as a reader of one word at a time over a tree, driven by networks
67
  over a stack: an embedding per feature place, a block of weights per stack slot, one hidden
68
  layer of rectified units, an output per step class and a pointer over the words of the
69
+ sentence. One network, trained from nothing; there is no base model.
 
70
  - **Language:** English.
71
  - **Licence:** AGPL-3.0-or-later.
72
  - **Files built at:** the commit `config.json` records under `build`, which the publish writes as it exports.
 
143
  of features hashed into a space of 2^64 places.
144
  5. A network reads the stack, one event per slot of 200. An event's embedding is the sum of the
145
  16-wide embeddings of its places. Each slot has its own weights into 512 hidden rectified
146
+ units. The outputs take a softmax over the 104 step classes. A class is a move and its
147
  properties, such as `{grab}`, `{drop @}`, `{children add: @ type: property}` or `{get owner}`.
148
+ 6. The step with the largest share is taken. When the step takes a word of the sentence (the `@`), the pointers score
 
149
  every word on the stack the same way and the best one is taken.
150
+ 7. The step runs, its two events join the stack, and the network chooses again. `{continue}`
151
  goes on to the next word. A word may take at most 50 steps.
152
  8. What a sentence wrote to the tree stays for the next sentence and the next turn, so a question
153
  is answered from everything the conversation said and from the seeds.
 
214
 
215
  | File | Holds |
216
  |---|---|
217
+ | `config.json` | The step limit, the network's shape, its one voter and its step classes, and the build commit |
218
+ | `model.safetensors` | The eight tensors of the network, listed below |
219
  | `state.bin` | The permanent state every chat starts from: the tree the seeds make |
220
  | `LICENSE`, `NOTICE.md` | The licence and what it covers |
221
 
222
+ The tensors of `model.safetensors`, `N` being 0, the one voter:
223
 
224
  | Tensor | Type | Shape |
225
  |---|---|---|
226
+ | `network.N.items` | `U64` | 4,235 |
227
+ | `network.N.item_table` | `F32` | 4,235 by 16 |
228
  | `network.N.slot_weights` | `F32` | 200 by 512 by 16 |
229
  | `network.N.fill_weights` | `F32` | 200 by 512 |
230
  | `network.N.hidden_bias` | `F32` | 512 |
231
+ | `network.N.output_weights` | `F32` | 104 by 512 |
232
+ | `network.N.output_bias` | `F32` | 104 |
233
  | `network.N.point_weights` | `F32` | 512 by 16 |
234
 
235
  No file contains code. A tensor runtime cannot run DAM1; it is read by the dam-core crate.
 
295
  |---|---:|---:|
296
  | Basics | 446 of 446 (100.0%) | 142 of 142 (100.0%) |
297
  | Grade 1 | 73 of 73 (100.0%) | 24 of 24 (100.0%) |
298
+ | Grade 2 | 688 of 688 (100.0%) | 241 of 242 (99.6%) |
299
+ | Grade 3 | 248 of 248 (100.0%) | 96 of 97 (99.0%) |
300
+ | Grade 4 | 273 of 273 (100.0%) | 91 of 91 (100.0%) |
301
+ | Grade 5 | 175 of 175 (100.0%) | 72 of 72 (100.0%) |
302
+ | Grade 6 | 158 of 158 (100.0%) | 58 of 58 (100.0%) |
303
+ | Grade 7 | 85 of 85 (100.0%) | 35 of 35 (100.0%) |
304
  | Grade 8 | 79 of 79 (100.0%) | 33 of 33 (100.0%) |
305
+ | Grade 9 | 150 of 150 (100.0%) | 65 of 66 (98.5%) |
306
  | Grade 10 | 26 of 26 (100.0%) | 9 of 9 (100.0%) |
307
+ | Grade 11 | 84 of 84 (100.0%) | 23 of 23 (100.0%) |
308
  | Grade 12 | 39 of 39 (100.0%) | 11 of 11 (100.0%) |
309
  | University | 47 of 47 (100.0%) | 13 of 13 (100.0%) |
310
  | World and debug cases | 36 of 36 (100.0%) | 32 of 32 (100.0%) |
311
+ | **All** | **2607 of 2607 (100.0%)** | **945 of 948 (99.7%)** |
312
 
313
+ It reads every learned line of the curriculum. The three held-out lines it misses are a statement
314
+ of what we like, a question of whether one letter follows another read backward through the
315
+ alphabet, and the last day of the week asked with no week named.
316
 
317
  The held-out lines use the same sentence shapes and many of the same words as the learned lines.
318
  They measure new words and numbers in taught shapes. They are not an independent benchmark. No
config.json CHANGED
@@ -9,9 +9,9 @@
9
  "slots": 200,
10
  "item": 16,
11
  "hidden": 512,
12
- "items": 4275,
13
  "pointer": true,
14
- "voters": 3,
15
  "classes": [
16
  "{continue}",
17
  "{grab}",
@@ -21,8 +21,7 @@
21
  "{step parent}",
22
  "{give}",
23
  "{setFlag type: a value: true}",
24
- "{setProperty color: @}",
25
- "{setProperty size: @}",
26
  "{children add: @ type: question}",
27
  "{children add: @}",
28
  "{point to: {nothing}}",
@@ -31,19 +30,14 @@
31
  "{getPropertyValue withChild: @}",
32
  "{get children}",
33
  "{get location}",
34
- "{setProperty age: @}",
35
  "{contain}",
36
- "{setProperty material: @}",
37
  "{setFlag type: property value: @}",
38
  "{find next}",
39
- "{setProperty trait: @}",
40
- "{setProperty speed: @}",
41
- "{setProperty temperature: @}",
42
  "{check @}",
43
  "{children add: @ type: relation}",
44
  "{step newest}",
45
  "{step top}",
46
- "{setProperty feeling: @}",
47
  "{hand}",
48
  "{get all with: @}",
49
  "{setFlag type: quantity value: @}",
@@ -54,19 +48,29 @@
54
  "{number set: @}",
55
  "{number add: @}",
56
  "{number say}",
 
57
  "{get subject: @}",
58
  "{setState @}",
59
  "{take}",
60
  "{release}",
61
- "{get past}",
62
  "{get relation: @}",
63
  "{children add: @ type: relation time: past}",
 
64
  "{join}",
 
65
  "{activity}",
 
66
  "{children add: @ type: deed}",
 
67
  "{get name}",
 
 
 
 
 
68
  "{belong}",
69
  "{children add: @ type: property time: past}",
 
70
  "{setFlag type: owned value: @}",
71
  "{possess}",
72
  "{setFlag type: when value: @}",
@@ -82,34 +86,41 @@
82
  "{grabAll}",
83
  "{measure @}",
84
  "{setFlag type: time value: past}",
85
- "{group : @}",
86
  "{get least: @}",
87
  "{role}",
88
  "{children role}",
89
  "{get shifted}",
 
90
  "{get about}",
91
  "{get choice}",
92
  "{get before: @}",
93
  "{get given: @}",
94
  "{get after: @}",
95
  "{number subtract: @}",
 
96
  "{number multiply: @}",
97
  "{number negate}",
98
  "{number divide: @}",
99
  "{number whole}",
100
  "{clock @}",
101
  "{get distance: @}",
 
102
  "{number remainder: @}",
103
  "{number percent: @}",
104
  "{number lessPercent: @}",
105
  "{number addPercent: @}",
106
  "{number root}",
107
  "{regard}",
108
- "{find with: @}"
 
 
 
 
109
  ]
110
  },
111
  "build": {
112
- "commit": "a2f3d0b967c85a203269a1bdb4cbaf9e80045c22",
113
  "network": "data/network/release.bin",
114
  "state": "data/seeds: all"
115
  }
 
9
  "slots": 200,
10
  "item": 16,
11
  "hidden": 512,
12
+ "items": 4235,
13
  "pointer": true,
14
+ "voters": 1,
15
  "classes": [
16
  "{continue}",
17
  "{grab}",
 
21
  "{step parent}",
22
  "{give}",
23
  "{setFlag type: a value: true}",
24
+ "{setProperty property: @}",
 
25
  "{children add: @ type: question}",
26
  "{children add: @}",
27
  "{point to: {nothing}}",
 
30
  "{getPropertyValue withChild: @}",
31
  "{get children}",
32
  "{get location}",
 
33
  "{contain}",
 
34
  "{setFlag type: property value: @}",
35
  "{find next}",
 
 
 
36
  "{check @}",
37
  "{children add: @ type: relation}",
38
  "{step newest}",
39
  "{step top}",
40
+ "{check relation @}",
41
  "{hand}",
42
  "{get all with: @}",
43
  "{setFlag type: quantity value: @}",
 
48
  "{number set: @}",
49
  "{number add: @}",
50
  "{number say}",
51
+ "{get past}",
52
  "{get subject: @}",
53
  "{setState @}",
54
  "{take}",
55
  "{release}",
 
56
  "{get relation: @}",
57
  "{children add: @ type: relation time: past}",
58
+ "{get backward relation: @}",
59
  "{join}",
60
+ "{find stood name: @ orderBy: lastMentioned}",
61
  "{activity}",
62
+ "{get doing relation: @}",
63
  "{children add: @ type: deed}",
64
+ "{activity named}",
65
  "{get name}",
66
+ "{get said relation: @}",
67
+ "{activity under}",
68
+ "{get toward relation: @}",
69
+ "{get location claimed}",
70
+ "{activity done}",
71
  "{belong}",
72
  "{children add: @ type: property time: past}",
73
+ "{get owner every}",
74
  "{setFlag type: owned value: @}",
75
  "{possess}",
76
  "{setFlag type: when value: @}",
 
86
  "{grabAll}",
87
  "{measure @}",
88
  "{setFlag type: time value: past}",
89
+ "group",
90
  "{get least: @}",
91
  "{role}",
92
  "{children role}",
93
  "{get shifted}",
94
+ "{get ranked relation: @}",
95
  "{get about}",
96
  "{get choice}",
97
  "{get before: @}",
98
  "{get given: @}",
99
  "{get after: @}",
100
  "{number subtract: @}",
101
+ "{getPropertyValue why withChild: @}",
102
  "{number multiply: @}",
103
  "{number negate}",
104
  "{number divide: @}",
105
  "{number whole}",
106
  "{clock @}",
107
  "{get distance: @}",
108
+ "{getPropertyValue measure withChild: @}",
109
  "{number remainder: @}",
110
  "{number percent: @}",
111
  "{number lessPercent: @}",
112
  "{number addPercent: @}",
113
  "{number root}",
114
  "{regard}",
115
+ "{get route withChild: @}",
116
+ "{find with: @}",
117
+ "{get through relation: @}",
118
+ "{get location motive}",
119
+ "{check right @}"
120
  ]
121
  },
122
  "build": {
123
+ "commit": "8a31755eabf1a779d75f65ba6621d585cf116ccd",
124
  "network": "data/network/release.bin",
125
  "state": "data/seeds: all"
126
  }
model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:94792089c4b7f17da5f4b16a10c58e2ccbef26fc18af1d2e5868237f89a4038c
3
- size 22492156
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7b6d4e809c611e231b09dc139e48e6ff7508fdbe17881f82e9345afa84e63fcb
3
+ size 7517096
state.bin CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:93fa09e6f138f1b145994f1f918f1454089656613234b0f07187b72521b00a2b
3
- size 212118
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6ac5eba00fe7b50899a72c87729f449d3967b1c48074cb064d7f9542b5471583
3
+ size 212142