fmac-lobotomy / README.md
ConceptModels's picture
Upload README.md with huggingface_hub
d33f04d verified
|
Raw History Blame Contribute Delete
408 Bytes
metadata
license: apache-2.0
tags:
  - fmac
  - cognitive-graph
  - data-distillation
  - cosmopedia
  - smoltalk

fMAC-v4: Data-Distilled Cognitive Graph

  • Architecture: Cellular Nodes with Shared Dense SwiGLU Core
  • Distillation Source: Cosmopedia-v2 (Mixtral-8x22B) & SmolTalk (Llama-3.1-70B)
  • Training Step: 2000 / 4000
  • Loss: 3.5633
  • Active Parameters per token: ~34M (out of 56.5M total)