fmac-lobotomy / README.md
ConceptModels's picture
Upload README.md with huggingface_hub
d33f04d verified
|
Raw History Blame Contribute Delete
408 Bytes
---
license: apache-2.0
tags:
- fmac
- cognitive-graph
- data-distillation
- cosmopedia
- smoltalk
---
# fMAC-v4: Data-Distilled Cognitive Graph
- **Architecture:** Cellular Nodes with Shared Dense SwiGLU Core
- **Distillation Source:** Cosmopedia-v2 (Mixtral-8x22B) & SmolTalk (Llama-3.1-70B)
- **Training Step:** 2000 / 4000
- **Loss:** 3.5633
- **Active Parameters per token:** ~34M (out of 56.5M total)