Add illustrated model card, family links and generation example
Browse files- README.md +5 -5
- manifest.json +2 -2
README.md
CHANGED
|
@@ -21,7 +21,7 @@ datasets:
|
|
| 21 |
|
| 22 |
[Paper](https://arxiv.org/abs/2602.13215) 路 [Code](https://github.com/HaoranZhengRaul/AMOR) 路 [Files](https://huggingface.co/FlyinGodzilla/AMOR-GatedDeltaNet-180M/tree/main) 路 [All checkpoints](#model-family)
|
| 23 |
|
| 24 |
-
|
| 25 |
|
| 26 |
## The idea
|
| 27 |
|
|
@@ -41,8 +41,8 @@ position for future retrieval.
|
|
| 41 |
their own pre-normalization. A shared, embedding-tied LM head supplies all three
|
| 42 |
entropy evaluations and the final output prediction.*
|
| 43 |
|
| 44 |
-
This checkpoint uses the **native entropy gate**.
|
| 45 |
-
deployment routers
|
| 46 |
attention with an output mask, so gating does not reduce training attention FLOPs.
|
| 47 |
|
| 48 |
## This checkpoint
|
|
@@ -142,7 +142,7 @@ stored once and restored by the loader. Optimizer state is excluded.
|
|
| 142 |
| Backbone | 180M | 440M | 1.5B |
|
| 143 |
|---|---|---|---|
|
| 144 |
| Mamba2 | [180M](https://huggingface.co/FlyinGodzilla/AMOR-Mamba2-180M) | [440M](https://huggingface.co/FlyinGodzilla/AMOR-Mamba2-440M) | [1.5B](https://huggingface.co/FlyinGodzilla/AMOR-Mamba2-1.5B) |
|
| 145 |
-
| Gated DeltaNet | [180M](https://huggingface.co/FlyinGodzilla/AMOR-GatedDeltaNet-180M) | [440M](https://huggingface.co/FlyinGodzilla/AMOR-GatedDeltaNet-440M) | 1.5B
|
| 146 |
|
| 147 |
All six checkpoints use the three-block canonical AMOR layout. Choose a backbone
|
| 148 |
and scale to match your experiment; larger models require more memory.
|
|
@@ -158,7 +158,7 @@ storage still grows with context. Training context is 3,072 tokens; long-context
|
|
| 158 |
experiments do not establish reliable performance on every domain.
|
| 159 |
|
| 160 |
## License
|
| 161 |
-
The code and model weights are released under the MIT license; see [LICENSE](LICENSE).
|
| 162 |
|
| 163 |
## Citation
|
| 164 |
```bibtex
|
|
|
|
| 21 |
|
| 22 |
[Paper](https://arxiv.org/abs/2602.13215) 路 [Code](https://github.com/HaoranZhengRaul/AMOR) 路 [Files](https://huggingface.co/FlyinGodzilla/AMOR-GatedDeltaNet-180M/tree/main) 路 [All checkpoints](#model-family)
|
| 23 |
|
| 24 |
+
Looking for the larger checkpoint? [AMOR-Gated DeltaNet 1.5B](https://huggingface.co/FlyinGodzilla/AMOR-GatedDeltaNet-1.5B).
|
| 25 |
|
| 26 |
## The idea
|
| 27 |
|
|
|
|
| 41 |
their own pre-normalization. A shared, embedding-tied LM head supplies all three
|
| 42 |
entropy evaluations and the final output prediction.*
|
| 43 |
|
| 44 |
+
This checkpoint uses the **native entropy gate**. Optional distilled
|
| 45 |
+
deployment routers are not included in this release. Training uses dense
|
| 46 |
attention with an output mask, so gating does not reduce training attention FLOPs.
|
| 47 |
|
| 48 |
## This checkpoint
|
|
|
|
| 142 |
| Backbone | 180M | 440M | 1.5B |
|
| 143 |
|---|---|---|---|
|
| 144 |
| Mamba2 | [180M](https://huggingface.co/FlyinGodzilla/AMOR-Mamba2-180M) | [440M](https://huggingface.co/FlyinGodzilla/AMOR-Mamba2-440M) | [1.5B](https://huggingface.co/FlyinGodzilla/AMOR-Mamba2-1.5B) |
|
| 145 |
+
| Gated DeltaNet | [180M](https://huggingface.co/FlyinGodzilla/AMOR-GatedDeltaNet-180M) | [440M](https://huggingface.co/FlyinGodzilla/AMOR-GatedDeltaNet-440M) | [1.5B](https://huggingface.co/FlyinGodzilla/AMOR-GatedDeltaNet-1.5B) |
|
| 146 |
|
| 147 |
All six checkpoints use the three-block canonical AMOR layout. Choose a backbone
|
| 148 |
and scale to match your experiment; larger models require more memory.
|
|
|
|
| 158 |
experiments do not establish reliable performance on every domain.
|
| 159 |
|
| 160 |
## License
|
| 161 |
+
The code and model weights are released under the MIT license; see [LICENSE](https://huggingface.co/FlyinGodzilla/AMOR-GatedDeltaNet-180M/blob/main/LICENSE).
|
| 162 |
|
| 163 |
## Citation
|
| 164 |
```bibtex
|
manifest.json
CHANGED
|
@@ -4,8 +4,8 @@
|
|
| 4 |
"sha256": "1d0dde41ece0f9620937f1a81a67e295ace0eae180c009d862006e8f3b288458"
|
| 5 |
},
|
| 6 |
"README.md": {
|
| 7 |
-
"bytes":
|
| 8 |
-
"sha256": "
|
| 9 |
},
|
| 10 |
"amor.py": {
|
| 11 |
"bytes": 25015,
|
|
|
|
| 4 |
"sha256": "1d0dde41ece0f9620937f1a81a67e295ace0eae180c009d862006e8f3b288458"
|
| 5 |
},
|
| 6 |
"README.md": {
|
| 7 |
+
"bytes": 7228,
|
| 8 |
+
"sha256": "c925cdaae7ad8e75d0b2bc128e0eaa7f220ba1ae9504c1f50b13b0ec7e5c2837"
|
| 9 |
},
|
| 10 |
"amor.py": {
|
| 11 |
"bytes": 25015,
|