ykyao commited on
Commit
a29f310
路
verified 路
1 Parent(s): 3cea8f5

Add paper links, Figure 1 and citation

Browse files
Files changed (3) hide show
  1. .gitattributes +1 -0
  2. README.md +17 -1
  3. assets/figure1.png +3 -0
.gitattributes CHANGED
@@ -1 +1,2 @@
1
  *.safetensors filter=lfs diff=lfs merge=lfs -text
 
 
1
  *.safetensors filter=lfs diff=lfs merge=lfs -text
2
+ assets/figure1.png filter=lfs diff=lfs merge=lfs -text
README.md CHANGED
@@ -13,11 +13,13 @@ tags:
13
 
14
  # ARLoop-1.2B
15
 
 
 
16
  ARLoop is a fully-open pretrained looped transformer with attention residuals. This release
17
  contains the checkpoints of the 1.2B, 500B-token training run.
18
  It is a **base pretrained model**, intended for text completion, which can be used for mid-training and post-training.
19
 
20
- Training code: [OSU-NLP-Group/ARLoop](https://github.com/OSU-NLP-Group/ARLoop).
21
 
22
  ## Load and generate
23
 
@@ -97,3 +99,17 @@ where 10000 denotes the number of optimization steps.
97
  | 238,417 (final) | 499,996,688,384 | [main](https://huggingface.co/osunlp/ARLoop-1.2B/tree/main) / [step-238417](https://huggingface.co/osunlp/ARLoop-1.2B/tree/step-238417) |
98
 
99
  All 23 scheduled intermediate checkpoints from 10k through 230k are available.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
13
 
14
  # ARLoop-1.2B
15
 
16
+ 馃摎 [Paper](https://arxiv.org/abs/2611.05234) 路 馃捇 [Code](https://github.com/OSU-NLP-Group/ARLoop)
17
+
18
  ARLoop is a fully-open pretrained looped transformer with attention residuals. This release
19
  contains the checkpoints of the 1.2B, 500B-token training run.
20
  It is a **base pretrained model**, intended for text completion, which can be used for mid-training and post-training.
21
 
22
+ ![Figure 1: ARLoop benchmark results](assets/figure1.png)
23
 
24
  ## Load and generate
25
 
 
99
  | 238,417 (final) | 499,996,688,384 | [main](https://huggingface.co/osunlp/ARLoop-1.2B/tree/main) / [step-238417](https://huggingface.co/osunlp/ARLoop-1.2B/tree/step-238417) |
100
 
101
  All 23 scheduled intermediate checkpoints from 10k through 230k are available.
102
+
103
+ ## Citation
104
+
105
+ ```bibtex
106
+ @misc{yao2026arloop,
107
+ title={{ARLoop}: Fully-Open Pre-trained Looped Transformers with Attention Residuals},
108
+ author={Yuekun Yao and Yupei Du and Dawei Zhu and Huan Sun},
109
+ year={2026},
110
+ eprint={2611.05234},
111
+ archivePrefix={arXiv},
112
+ primaryClass={cs.CL},
113
+ url={https://arxiv.org/abs/2611.05234}
114
+ }
115
+ ```
assets/figure1.png ADDED

Git LFS Details

  • SHA256: 2bd2c2012dc1ad598b4560a07e3008ec28a8f91b091904be781b490945d5ef7e
  • Pointer size: 131 Bytes
  • Size of remote file: 335 kB