TejAndrewsACC commited on
Commit
a5feddc
·
verified ·
1 Parent(s): 7e028c1

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +61 -0
README.md CHANGED
@@ -1,3 +1,64 @@
1
  ---
2
  license: apache-2.0
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  license: apache-2.0
3
  ---
4
+
5
+
6
+ language:
7
+ en
8
+ pipeline_tag: conversational
9
+ tags:
10
+ nce
11
+ neural-cognition
12
+ phi-model
13
+ license: other
14
+
15
+ Zemo
16
+ Next-Generation NCE
17
+ Zemo is a Next-Generation Neural Cognition Engine developed under the Algorithmic Computer-generated Consciousness framework by the ACC.
18
+ Model Information
19
+ Property
20
+ Details
21
+ Name
22
+ Zemo
23
+ Model Type
24
+ Next-Generation NCE
25
+ Organization
26
+ ACC
27
+ Framework
28
+ Neural Cognition Engine
29
+ Research Direction
30
+ Neural cognition and PhiModel concepts
31
+ Foundation Model Reference
32
+ Kimi K2.5
33
+ Architecture
34
+ Mixture of Experts (MoE)
35
+
36
+ Architecture
37
+ Zemo is associated with the NCE framework, which explores the integration of multiple neural network architectures, including transformers, recurrent neural networks, convolutional neural networks, feedforward networks, and spiking neural networks.
38
+ The framework also incorporates the PhiModel concept, which investigates the use of the golden ratio as a mathematical principle for neural organization.
39
+ Technical Specifications
40
+
41
+ Specification
42
+ Value
43
+ Total Parameters
44
+ 1 Trillion
45
+ Activated Parameters
46
+ 32 Billion
47
+ Layers
48
+ 61
49
+ Experts
50
+ 384
51
+ Experts per Token
52
+ 8
53
+ Context Window
54
+ 256K tokens
55
+ Vocabulary Size
56
+ 160K
57
+ Attention Mechanism
58
+ Multi-head Latent Attention (MLA)
59
+ Activation Function
60
+ SwiGLU
61
+ Vision Encoder
62
+ MoonViT
63
+ Input Modalities
64
+ Text, images