Schroding18 commited on
Commit
8d9810b
·
verified ·
1 Parent(s): 704c148
Files changed (1) hide show
  1. index.html +2 -2
index.html CHANGED
@@ -1,6 +1,6 @@
1
  <!doctype html><html lang="en"><head><meta charset="utf-8">
2
  <meta name="viewport" content="width=device-width, initial-scale=1">
3
- <title>Free 4–6% lower perplexity: a new MoE architecture</title>
4
  <meta name="description" content="int3 MoE beats full-precision MoE: run each MoE layer twice while its weights are still in L2. 6% lower perplexity, 22% of the expert bytes.">
5
  <style>
6
  body{margin:0;background:#fbfbfa;color:#1c1c1c;font:17px/1.65 -apple-system,BlinkMacSystemFont,"Segoe UI",Roboto,Helvetica,Arial,sans-serif}
@@ -22,7 +22,7 @@ blockquote{margin:1.2em 0 1.4em;padding:18px 22px;background:#f4f0ff;border-left
22
  blockquote h2{margin:0 0 .35em;border:0;padding:0;font-size:1.75em;line-height:1.2;color:#4c1d95}
23
  blockquote p{margin:.3em 0}
24
  </style></head><body><main>
25
- <h1>Free 4–6% lower perplexity: a new MoE architecture</h1>
26
 
27
  <blockquote>
28
  <h2>int3 MoE beats full-precision MoE.</h2>
 
1
  <!doctype html><html lang="en"><head><meta charset="utf-8">
2
  <meta name="viewport" content="width=device-width, initial-scale=1">
3
+ <title>Loop in L2: int3 MoE beats full precision, at no extra memory</title>
4
  <meta name="description" content="int3 MoE beats full-precision MoE: run each MoE layer twice while its weights are still in L2. 6% lower perplexity, 22% of the expert bytes.">
5
  <style>
6
  body{margin:0;background:#fbfbfa;color:#1c1c1c;font:17px/1.65 -apple-system,BlinkMacSystemFont,"Segoe UI",Roboto,Helvetica,Arial,sans-serif}
 
22
  blockquote h2{margin:0 0 .35em;border:0;padding:0;font-size:1.75em;line-height:1.2;color:#4c1d95}
23
  blockquote p{margin:.3em 0}
24
  </style></head><body><main>
25
+ <h1>Loop in L2: int3 MoE beats full precision, at no extra memory</h1>
26
 
27
  <blockquote>
28
  <h2>int3 MoE beats full-precision MoE.</h2>