File size: 408 Bytes
68333ff c3fc416 68333ff c3fc416 68333ff c3fc416 d33f04d c3fc416 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 | ---
license: apache-2.0
tags:
- fmac
- cognitive-graph
- data-distillation
- cosmopedia
- smoltalk
---
# fMAC-v4: Data-Distilled Cognitive Graph
- **Architecture:** Cellular Nodes with Shared Dense SwiGLU Core
- **Distillation Source:** Cosmopedia-v2 (Mixtral-8x22B) & SmolTalk (Llama-3.1-70B)
- **Training Step:** 2000 / 4000
- **Loss:** 3.5633
- **Active Parameters per token:** ~34M (out of 56.5M total)
|