Fernando Gutierrez commited on
Commit ·
89bb222
1
Parent(s): 22be4dd
Add TIBI SAURON training section (671B MoE on 50k TibiBoxes, theoretical scenario)
Browse files- assets/sauron-announcement-en.png +3 -0
- assets/sauron-eye.png +3 -0
- index.html +18 -0
assets/sauron-announcement-en.png
ADDED
|
Git LFS Details
|
assets/sauron-eye.png
ADDED
|
Git LFS Details
|
index.html
CHANGED
|
@@ -18,6 +18,7 @@
|
|
| 18 |
<a href="#how">How it works</a>
|
| 19 |
<a href="#fleet">The fleet</a>
|
| 20 |
<a href="#demo">Demo</a>
|
|
|
|
| 21 |
<a href="#contact">Contact</a>
|
| 22 |
</nav>
|
| 23 |
<a class="btn btn-small" href="https://tibios.ai">tibios.ai</a>
|
|
@@ -88,6 +89,23 @@
|
|
| 88 |
<video class="demo" src="assets/fleet-demo.mp4" autoplay muted loop playsinline controls></video>
|
| 89 |
</section>
|
| 90 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 91 |
<section class="section">
|
| 92 |
<h2>Use the demo fleet today</h2>
|
| 93 |
<div class="cards">
|
|
|
|
| 18 |
<a href="#how">How it works</a>
|
| 19 |
<a href="#fleet">The fleet</a>
|
| 20 |
<a href="#demo">Demo</a>
|
| 21 |
+
<a href="#sauron">Sauron</a>
|
| 22 |
<a href="#contact">Contact</a>
|
| 23 |
</nav>
|
| 24 |
<a class="btn btn-small" href="https://tibios.ai">tibios.ai</a>
|
|
|
|
| 89 |
<video class="demo" src="assets/fleet-demo.mp4" autoplay muted loop playsinline controls></video>
|
| 90 |
</section>
|
| 91 |
|
| 92 |
+
<section id="sauron" class="section">
|
| 93 |
+
<h2>Training TIBI SAURON</h2>
|
| 94 |
+
<p class="lede center">
|
| 95 |
+
Our north star: what if a frontier model could be trained not in a
|
| 96 |
+
data center, but on a network of small machines? TIBI SAURON is a
|
| 97 |
+
planned 671B-parameter MoE model — in the spirit of DeepSeek-V3 and
|
| 98 |
+
Kimi-K3 — trained across thousands of TibiBoxes.
|
| 99 |
+
<em>Theoretical scenario based on our model; not a benchmark.</em>
|
| 100 |
+
</p>
|
| 101 |
+
<div class="cards">
|
| 102 |
+
<div class="card"><h3>671B</h3><p>parameters, MoE architecture inspired by DeepSeek-V3 (671B) and Kimi-K3.</p></div>
|
| 103 |
+
<div class="card"><h3>14.8T tokens</h3><p>~3.3 × 10²⁴ total FLOPs of distributed training.</p></div>
|
| 104 |
+
<div class="card"><h3>50,000 TibiBoxes</h3><p>~12 months to train — 20,000 nodes for training, 30,000 for inference and services.</p></div>
|
| 105 |
+
</div>
|
| 106 |
+
<img class="wide sauron-img" src="assets/sauron-announcement-en.png" alt="TIBI SAURON training scenario: estimated time to train a 671B model on a TibiBox network" />
|
| 107 |
+
</section>
|
| 108 |
+
|
| 109 |
<section class="section">
|
| 110 |
<h2>Use the demo fleet today</h2>
|
| 111 |
<div class="cards">
|