Kye Gomez commited on
Commit ·
f00d10b
1
Parent(s): e522ecf
more references fix
Browse files- README.md +3 -0
- open_mythos/variants.py +0 -3
README.md
CHANGED
|
@@ -289,6 +289,9 @@ Theoretical analysis suggests 2-3x improvements in inference throughput. For a d
|
|
| 289 |
- Looped transformer cyclic trajectories and input injection (rosinality): https://x.com/rosinality/status/2043953033428541853
|
| 290 |
- Parcae scaling laws for stable looped language models — thread (Hayden Prairie): https://x.com/hayden_prairie/status/2044453231913537927
|
| 291 |
- RoPE-like loop index embedding idea to differentiate functions across iterations (davidad): https://x.com/davidad/status/2044453231913537927
|
|
|
|
|
|
|
|
|
|
| 292 |
|
| 293 |
### Papers
|
| 294 |
|
|
|
|
| 289 |
- Looped transformer cyclic trajectories and input injection (rosinality): https://x.com/rosinality/status/2043953033428541853
|
| 290 |
- Parcae scaling laws for stable looped language models — thread (Hayden Prairie): https://x.com/hayden_prairie/status/2044453231913537927
|
| 291 |
- RoPE-like loop index embedding idea to differentiate functions across iterations (davidad): https://x.com/davidad/status/2044453231913537927
|
| 292 |
+
- On the Looped Transformers Controversy by ChrisHayduk: https://x.com/ChrisHayduk/status/2045947623572688943
|
| 293 |
+
- On the Looped Transformers Controversy Summary by @realsigridjin https://x.com/realsigridjin/status/2046012743778766875
|
| 294 |
+
|
| 295 |
|
| 296 |
### Papers
|
| 297 |
|
open_mythos/variants.py
CHANGED
|
@@ -191,6 +191,3 @@ def mythos_1t() -> MythosConfig:
|
|
| 191 |
lora_rank=256,
|
| 192 |
max_output_tokens=131072,
|
| 193 |
)
|
| 194 |
-
|
| 195 |
-
|
| 196 |
-
# fmt: on
|
|
|
|
| 191 |
lora_rank=256,
|
| 192 |
max_output_tokens=131072,
|
| 193 |
)
|
|
|
|
|
|
|
|
|