Spaces:
Running
Running
File size: 2,298 Bytes
ad29e43 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 | <!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>One training step of a 1,024-GPU MoE</title>
<meta name="description" content="A 45-layer mixture-of-experts model running one training step on 1,024 GPUs, live in your browser.">
<meta name="twitter:card" content="player">
<meta name="twitter:title" content="One training step of a 1,024-GPU MoE">
<meta name="twitter:description" content="A 45-layer mixture-of-experts model running one training step on 1,024 GPUs, live. Hover a GPU to see what it is doing.">
<meta name="twitter:player" content="https://thomwolf-moe-training-step.static.hf.space/player.html?embed=1&theme=dark">
<meta name="twitter:player:width" content="640">
<meta name="twitter:player:height" content="480">
<meta name="twitter:image" content="https://thomwolf-moe-training-step.static.hf.space/card.png">
<meta name="twitter:image:alt" content="An isometric city of 1,024 GPUs in four pipeline stages, with forward passes in blue and backward passes in orange.">
<meta property="og:type" content="website">
<meta property="og:title" content="One training step of a 1,024-GPU MoE">
<meta property="og:description" content="A 45-layer mixture-of-experts model running one training step on 1,024 GPUs, live in your browser.">
<meta property="og:image" content="https://thomwolf-moe-training-step.static.hf.space/card.png">
<meta property="og:url" content="https://thomwolf-moe-training-step.static.hf.space/share.html">
<style>
html, body { margin: 0; height: 100%; background: #111215; color: #e6e8eb; font: 400 14px/1.4 "IBM Plex Sans", -apple-system, "Helvetica Neue", Arial, sans-serif; }
body { display: flex; flex-direction: column; }
iframe { flex: 1; width: 100%; border: 0; display: block; }
a.post { display: block; padding: 12px 16px; text-align: center; color: inherit; text-decoration: none; border-top: 1px solid #2a2d33; }
a.post:hover { text-decoration: underline; }
</style>
</head>
<body>
<iframe src="player.html?embed=1&theme=dark" title="One training step of a 1,024-GPU MoE"></iframe>
<a class="post" href="https://huggingface.co/spaces/huggingface/moe-blogpost">Read the post: how to train a mixture-of-experts model, knob by knob →</a>
</body>
</html>
|