Can111 commited on
Commit
360fe96
路
verified 路
1 Parent(s): bea6649

Landing page: paper PDF and project page links, no license, new Figure 1

Browse files
Files changed (3) hide show
  1. README.md +0 -1
  2. figure1.png +2 -2
  3. index.html +10 -10
README.md CHANGED
@@ -5,7 +5,6 @@ colorFrom: blue
5
  colorTo: yellow
6
  sdk: static
7
  pinned: true
8
- license: apache-2.0
9
  short_description: "Scalable elastic pretraining: checkpoints and data"
10
  ---
11
 
 
5
  colorTo: yellow
6
  sdk: static
7
  pinned: true
 
8
  short_description: "Scalable elastic pretraining: checkpoints and data"
9
  ---
10
 
figure1.png CHANGED

Git LFS Details

  • SHA256: 862a06b1ece5306e31ce0b1653b16de6865289105aa385367170d8ec5110049c
  • Pointer size: 131 Bytes
  • Size of remote file: 135 kB

Git LFS Details

  • SHA256: 85e803ffce530adfc1ce0ce3493760a9c436a09221d4d81e814033b66aaa0b87
  • Pointer size: 131 Bytes
  • Size of remote file: 140 kB
index.html CHANGED
@@ -14,13 +14,15 @@
14
  * { box-sizing: border-box; }
15
  body { margin: 0; background: var(--bg); color: var(--fg);
16
  font: 16px/1.6 -apple-system, BlinkMacSystemFont, "Segoe UI", "Noto Sans", Helvetica, Arial, sans-serif; }
17
- main { max-width: 980px; margin: 0 auto; padding: 32px 16px 56px; }
 
18
  a { color: var(--link); text-decoration: none; }
19
  a:hover { text-decoration: underline; }
20
  header { text-align: center; }
21
  h1 { font-size: 2.6em; margin: 0.2em 0 0.3em; padding-bottom: 0.25em; border-bottom: 1px solid var(--line); }
22
  .subtitle { font-size: 1.55em; font-weight: 600; line-height: 1.3; margin: 0.6em 0; }
23
- .authors { font-size: 1.15em; margin: 0.9em 0 0.4em; }
 
24
  .authors b { font-weight: 600; }
25
  .author { white-space: nowrap; }
26
  .affil, .corr { margin: 0.4em 0; }
@@ -45,23 +47,24 @@ code, .repo { font-family: ui-monospace, SFMono-Regular, "SF Mono", Menlo, Conso
45
  </style>
46
  </head>
47
  <body>
48
- <main>
49
  <header>
50
  <h1>ShellElastic</h1>
51
- <div class="subtitle">Scalable Elastic Pretraining with Simultaneous Maximal Updates and Hyperparameter Transfer</div>
52
  <p class="authors"><span class="author"><b>Can Jin</b><sup>1*</sup></span> 路 <span class="author"><b>Yan Kang</b><sup>2</sup></span> 路 <span class="author"><b>Yifan Gong</b><sup>2</sup></span> 路 <span class="author"><b>Chongjian Ge</b><sup>2</sup></span> 路 <span class="author"><b>Mingze Xu</b><sup>2</sup></span> 路 <span class="author"><b>Yuanjun Xiong</b><sup>2</sup></span> 路 <span class="author"><b>Dimitris N. Metaxas</b><sup>1</sup></span> 路 <span class="author"><b>Hongwu Peng</b><sup>2</sup></span></p>
53
  <p class="affil"><sup>1</sup>Rutgers University &emsp; <sup>2</sup>Adobe Research</p>
54
  <p class="corr"><sup>*</sup>Corresponding author: <a href="mailto:can.jin@rutgers.edu">can.jin@rutgers.edu</a>. Work done during an
55
  internship at Adobe Research.</p>
56
  <div class="badges">
57
- <a href="https://github.com/jincan333/ShellElastic/tree/main/paper"><img alt="Paper" src="https://img.shields.io/badge/Paper-PDF-b31b1b?style=for-the-badge"></a>
58
- <a href="http://localhost:8765/"><img alt="Project Page" src="https://img.shields.io/badge/Project-Page-2f6db5?style=for-the-badge"></a>
59
  <a href="https://github.com/jincan333/ShellElastic"><img alt="Code" src="https://img.shields.io/badge/Code-GitHub-181717?style=for-the-badge&amp;logo=github&amp;logoColor=white"></a>
60
  <a href="https://huggingface.co/collections/Can111/shellelastic-checkpoints-and-data-6ac41e3a763d5a3fd0e79111"><img alt="Checkpoints" src="https://img.shields.io/badge/Checkpoints-Hugging%20Face-ffd21e?style=for-the-badge&amp;logo=huggingface&amp;logoColor=white"></a>
61
  <a href="https://huggingface.co/collections/Can111/shellelastic-checkpoints-and-data-6ac41e3a763d5a3fd0e79111"><img alt="Datasets" src="https://img.shields.io/badge/Datasets-Hugging%20Face-ffd21e?style=for-the-badge&amp;logo=huggingface&amp;logoColor=white"></a>
62
- <a href="https://github.com/jincan333/ShellElastic/blob/main/LICENSE"><img alt="License" src="https://img.shields.io/badge/License-Apache--2.0-3da639?style=for-the-badge&amp;logo=apache&amp;logoColor=white"></a>
63
  </div>
64
  </header>
 
 
65
 
66
  <h2>About ShellElastic</h2>
67
  <figure><img src="figure1.png" alt="Figure 1 of the paper: the ShellElastic architecture"></figure>
@@ -90,9 +93,6 @@ read, in the layout that the code reads. Training: 3,136 shards of DCLM-Baseline
90
  (141.32 GB, 25,690,112 sequences of 2,049 tokens); validation: 11 shards of held-out C4
91
  text (1.12 GB, 84,999 sequences).</p>
92
 
93
- <h2>License</h2>
94
- <p>The code is released under the <a href="https://github.com/jincan333/ShellElastic/blob/main/LICENSE">Apache License 2.0</a>. The checkpoints and the data are
95
- released under CC BY 4.0; the data also carry the terms of their sources (see the dataset card).</p>
96
  </main>
97
  </body>
98
  </html>
 
14
  * { box-sizing: border-box; }
15
  body { margin: 0; background: var(--bg); color: var(--fg);
16
  font: 16px/1.6 -apple-system, BlinkMacSystemFont, "Segoe UI", "Noto Sans", Helvetica, Arial, sans-serif; }
17
+ main { max-width: 980px; margin: 0 auto; padding: 8px 16px 56px; }
18
+ .top { max-width: 1140px; margin: 0 auto; padding: 32px 16px 0; }
19
  a { color: var(--link); text-decoration: none; }
20
  a:hover { text-decoration: underline; }
21
  header { text-align: center; }
22
  h1 { font-size: 2.6em; margin: 0.2em 0 0.3em; padding-bottom: 0.25em; border-bottom: 1px solid var(--line); }
23
  .subtitle { font-size: 1.55em; font-weight: 600; line-height: 1.3; margin: 0.6em 0; }
24
+ .authors { font-size: min(16px, calc((100vw - 32px) * 0.0147)); white-space: nowrap; margin: 0.9em 0 0.4em; }
25
+ @media (max-width: 900px) { .authors { font-size: 15px; white-space: normal; } }
26
  .authors b { font-weight: 600; }
27
  .author { white-space: nowrap; }
28
  .affil, .corr { margin: 0.4em 0; }
 
47
  </style>
48
  </head>
49
  <body>
50
+ <div class="top">
51
  <header>
52
  <h1>ShellElastic</h1>
53
+ <div class="subtitle">Scalable Elastic Pretraining<br>with Simultaneous Maximal Updates and Hyperparameter Transfer</div>
54
  <p class="authors"><span class="author"><b>Can Jin</b><sup>1*</sup></span> 路 <span class="author"><b>Yan Kang</b><sup>2</sup></span> 路 <span class="author"><b>Yifan Gong</b><sup>2</sup></span> 路 <span class="author"><b>Chongjian Ge</b><sup>2</sup></span> 路 <span class="author"><b>Mingze Xu</b><sup>2</sup></span> 路 <span class="author"><b>Yuanjun Xiong</b><sup>2</sup></span> 路 <span class="author"><b>Dimitris N. Metaxas</b><sup>1</sup></span> 路 <span class="author"><b>Hongwu Peng</b><sup>2</sup></span></p>
55
  <p class="affil"><sup>1</sup>Rutgers University &emsp; <sup>2</sup>Adobe Research</p>
56
  <p class="corr"><sup>*</sup>Corresponding author: <a href="mailto:can.jin@rutgers.edu">can.jin@rutgers.edu</a>. Work done during an
57
  internship at Adobe Research.</p>
58
  <div class="badges">
59
+ <a href="https://github.com/jincan333/ShellElastic/blob/main/paper/ShellElastic.pdf"><img alt="Paper" src="https://img.shields.io/badge/Paper-PDF-b31b1b?style=for-the-badge"></a>
60
+ <a href="https://jincan333.github.io/ShellElastic_page"><img alt="Project Page" src="https://img.shields.io/badge/Project-Page-2f6db5?style=for-the-badge"></a>
61
  <a href="https://github.com/jincan333/ShellElastic"><img alt="Code" src="https://img.shields.io/badge/Code-GitHub-181717?style=for-the-badge&amp;logo=github&amp;logoColor=white"></a>
62
  <a href="https://huggingface.co/collections/Can111/shellelastic-checkpoints-and-data-6ac41e3a763d5a3fd0e79111"><img alt="Checkpoints" src="https://img.shields.io/badge/Checkpoints-Hugging%20Face-ffd21e?style=for-the-badge&amp;logo=huggingface&amp;logoColor=white"></a>
63
  <a href="https://huggingface.co/collections/Can111/shellelastic-checkpoints-and-data-6ac41e3a763d5a3fd0e79111"><img alt="Datasets" src="https://img.shields.io/badge/Datasets-Hugging%20Face-ffd21e?style=for-the-badge&amp;logo=huggingface&amp;logoColor=white"></a>
 
64
  </div>
65
  </header>
66
+ </div>
67
+ <main>
68
 
69
  <h2>About ShellElastic</h2>
70
  <figure><img src="figure1.png" alt="Figure 1 of the paper: the ShellElastic architecture"></figure>
 
93
  (141.32 GB, 25,690,112 sequences of 2,049 tokens); validation: 11 shards of held-out C4
94
  text (1.12 GB, 84,999 sequences).</p>
95
 
 
 
 
96
  </main>
97
  </body>
98
  </html>