TheDrainFlorist commited on
Commit
56584d8
·
verified ·
1 Parent(s): cc221eb

6 names the non-rebuildable artifact; 2.6 names the 27B ppl corpus

Browse files
Files changed (1) hide show
  1. index.html +10 -5
index.html CHANGED
@@ -288,7 +288,8 @@ corpora, first 8192 tokens. Deterministic — an artifact reproduces its
288
  total negative log-likelihood to all printed decimals across launches
289
  and machines. <strong>35B and 27B:</strong> KL divergence to the bf16 model's cached
290
  logits, in millinats, with top-1 agreement, plus referee perplexity on
291
- the 27B. Comparator rows were re-verified on a second machine and agree
 
292
  to every reported digit. The gemma-4 family, where we observed similar
293
  size-quality behaviour, is excluded throughout: raw likelihood is
294
  invalid on those instruction-tuned models as a property of the model
@@ -945,10 +946,14 @@ GiB residency on our 128 GB machine and is uncharacterized at other
945
  sizes; we publish ratios within a session, never absolutes.</p>
946
  <p><strong>Costs we pay.</strong> Prefill remains ~0.5x affine at 35B scale even after
947
  the shipped lever. Codebooks beyond threadgroup capacity pay ~19%
948
- decode. One published artifact cannot be rebuilt at all — it predates
949
- the manifests and its build inputs were later overwritten — though
950
- it remains downloadable, its scores reproduce exactly, and its quality
951
- is consistent with the measured draw distribution of its geometry.</p>
 
 
 
 
952
  <h2 id="7-reproducibility">7. Reproducibility</h2>
953
  <p>All artifacts are published under <code>TheDrainFlorist</code> on Hugging Face
954
  with their VQ runtimes bundled in-checkpoint (stock mlx-lm, no
 
288
  total negative log-likelihood to all printed decimals across launches
289
  and machines. <strong>35B and 27B:</strong> KL divergence to the bf16 model's cached
290
  logits, in millinats, with top-1 agreement, plus referee perplexity on
291
+ the 27B — the same wikitext-style prose corpus at its first 2048 tokens,
292
+ a single corpus where the 397B uses two. Comparator rows were re-verified on a second machine and agree
293
  to every reported digit. The gemma-4 family, where we observed similar
294
  size-quality behaviour, is excluded throughout: raw likelihood is
295
  invalid on those instruction-tuned models as a property of the model
 
946
  sizes; we publish ratios within a session, never absolutes.</p>
947
  <p><strong>Costs we pay.</strong> Prefill remains ~0.5x affine at 35B scale even after
948
  the shipped lever. Codebooks beyond threadgroup capacity pay ~19%
949
+ decode. One published artifact cannot be rebuilt at all: the 111.6 GiB
950
+ d4/K256 build (<code>VQ-2.4bpw</code>) predates the manifest system, and the log
951
+ recording its exact fit invocation was overwritten four days after the
952
+ fit, so its remaining unrecoverable inputs are the fit flags themselves.
953
+ It remains downloadable, its scores reproduce exactly on the artifact,
954
+ and its quality sits inside the measured draw distribution of its
955
+ geometry — a favourable but unexceptional draw, not a mystery (§2.6's
956
+ floors are how we know).</p>
957
  <h2 id="7-reproducibility">7. Reproducibility</h2>
958
  <p>All artifacts are published under <code>TheDrainFlorist</code> on Hugging Face
959
  with their VQ runtimes bundled in-checkpoint (stock mlx-lm, no