ConceptModels commited on
Commit
c3fc416
·
verified ·
1 Parent(s): 522e4d5

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +8 -10
README.md CHANGED
@@ -3,16 +3,14 @@ license: apache-2.0
3
  tags:
4
  - fmac
5
  - cognitive-graph
6
- - sparse-moe
7
- - deepseek-v3-balancing
8
- - muon
9
  ---
10
- # fMAC: Fractal Multi-Agent Cognitive Graph
11
 
12
  - **Architecture:** Cellular Nodes with Shared Dense SwiGLU Core
13
- - **Sequence Mixer:** Fractal Memory Pyramid with QK-Norm RoPE
14
- - **Training Step:** 4000 / 4000
15
- - **Current Loss:** 3.5641
16
- - **Optimizer:** Hybrid Muon (2D Dense Linears) + AdamW (MoE Nodes, Embeddings)
17
-
18
- ![billede](https://cdn-uploads.huggingface.co/production/uploads/693f140bb1693bd71406e3ea/-83cz8BEOIp0MdFQrh-Ur.png)
 
3
  tags:
4
  - fmac
5
  - cognitive-graph
6
+ - data-distillation
7
+ - cosmopedia
8
+ - smoltalk
9
  ---
10
+ # fMAC-v4: Data-Distilled Cognitive Graph
11
 
12
  - **Architecture:** Cellular Nodes with Shared Dense SwiGLU Core
13
+ - **Distillation Source:** Cosmopedia-v2 (Mixtral-8x22B) & SmolTalk (Llama-3.1-70B)
14
+ - **Training Step:** 2000 / 4000
15
+ - **Loss:** 3.5928
16
+ - **Active Parameters per token:** ~34M (out of 56.5M total)