File size: 408 Bytes
68333ff
 
 
 
 
c3fc416
 
 
68333ff
c3fc416
68333ff
 
c3fc416
d33f04d
 
c3fc416
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
---
license: apache-2.0
tags:
- fmac
- cognitive-graph
- data-distillation
- cosmopedia
- smoltalk
---
# fMAC-v4: Data-Distilled Cognitive Graph

- **Architecture:** Cellular Nodes with Shared Dense SwiGLU Core
- **Distillation Source:** Cosmopedia-v2 (Mixtral-8x22B) & SmolTalk (Llama-3.1-70B)
- **Training Step:** 2000 / 4000
- **Loss:** 3.5633
- **Active Parameters per token:** ~34M (out of 56.5M total)