|
|
| --- |
| language: |
| - en |
| pipeline_tag: image-text-to-text |
| tags: |
| - nce |
| - neural-cognition |
| - phi-model |
| license: other |
| --- |
| |
| <p align="center"> |
| <img src="https://huggingface.co/spaces/TejAndrewsACC/Zemo/resolve/main/IMG_3490.png" alt="Zemo Model Logo" width="300"> |
| </p> |
|
|
| # Zemo |
|
|
| ## Next-Generation NCE |
|
|
| Zemo is a Next-Generation Neural Cognition Engine developed under the Algorithmic Computer-generated Consciousness framework by the ACC. |
|
|
| ## Model Information |
|
|
| | Property | Details | |
| |---|---| |
| | Name | Zemo | |
| | Model Type | Next-Generation NCE | |
| | Organization | ACC | |
| | Framework | Neural Cognition Engine | |
| | Research Direction | Neural cognition and PhiModel concepts | |
| | Foundation Model Reference | [Kimi K2.5](https://huggingface.co/moonshotai/Kimi-K2.5) | |
| | Architecture | Mixture of Experts (MoE) | |
|
|
| ## Architecture |
|
|
| Zemo is associated with the NCE framework, which explores the integration of multiple neural network architectures, including transformers, recurrent neural networks, convolutional neural networks, feedforward networks, and spiking neural networks. |
|
|
| The framework also incorporates the PhiModel concept, which investigates the use of the golden ratio as a mathematical principle for neural organization. |
|
|
| ## Technical Specifications |
|
|
| | Specification | Value | |
| |---|---| |
| | Total Parameters | 1 Trillion | |
| | Activated Parameters | 32 Billion | |
| | Layers | 61 | |
| | Experts | 384 | |
| | Experts per Token | 8 | |
| | Context Window | 256K tokens | |
| | Vocabulary Size | 160K | |
| | Attention Mechanism | Multi-head Latent Attention (MLA) | |
| | Activation Function | SwiGLU | |
| | Vision Encoder | MoonViT | |
| | Input Modalities | Text, images | |
