add sidenote about methodology

app/src/content/chapters/4-analyses.mdx

@@ -61,7 +61,7 @@ article/commentary/discussion/tutorial-1b-hq, tutorial-12b-hq, faq-1b-lq, faq-12
 
 **Semantic diversity helps commonsense and NLU but does not predict aggregate performance.** We added three semantic-diversity metrics computed on a 1000-output sample of each rephrased dataset: the Vendi Score [@vendi] (effective number of distinct samples), mean intra-set cosine similarity, and near-duplicate rate, all computed from [`BAAI/bge-base-en-v1.5`](https://huggingface.co/BAAI/bge-base-en-v1.5) [@bge] embeddings.[^diversity-metric-choice] Vendi Score correlates strongly *positively* with sentence-completion tasks like PIQA (ρ = 0.46, p {'<'} 0.001) and HellaSwag (ρ = 0.36, p {'<'} 0.001), where varied phrasings expose the student to more surface forms. On the overall macro score it is slightly negative but not significant (ρ = −0.21, p = 0.06).<Sidenote>Mean cosine similarity is the inverse measure of the same property: high pairwise similarity is exactly low diversity, so it flips the sign of every Vendi correlation by construction and is not an independent signal.</Sidenote>
 
-**Gradient-space diversity (G-Vendi) is the most consistent diversity signal.** Following @prismatic-synthesis, we compute G-Vendi: instead of embedding each output with a sentence encoder, we backprop the autoregressive cross-entropy loss on each output through a frozen 0.6B proxy LLM ([`Qwen/Qwen3-0.6B`](https://huggingface.co/Qwen/Qwen3-0.6B), an independent model smaller than any of our six rephrasers), sketch the resulting gradient via 1024 random index samples from the last MLP down-projection layer, L2-normalise, and take the Vendi Score over the resulting feature set per dataset. G-Vendi correlates positively with almost every benchmark we care about, including `agg_score_macro` (ρ = 0.26, p {'<'} 0.05) and `agg_score_micro` (ρ = 0.29, p {'<'} 0.01) where semantic Vendi is non-significantly negative. It is the single best predictor for reasoning (`agg_score_RES` ρ = 0.38, p {'<'} 0.001) and XCSQA (ρ = 0.34, p {'<'} 0.01), beating every DCLM, edu-score, and semantic-diversity variant. Across our 83 experiments, G-Vendi consistently beats semantic Vendi as a downstream-performance predictor, confirming the headline finding of @prismatic-synthesis: **gradient-space diversity predicts training value better than embedding-space diversity.**
+**Gradient-space diversity (G-Vendi) is the most consistent diversity signal.** Following @prismatic-synthesis, we compute G-Vendi: instead of embedding each output with a sentence encoder, we backprop the autoregressive cross-entropy loss on each output through a frozen 0.6B proxy LLM ([`Qwen/Qwen3-0.6B`](https://huggingface.co/Qwen/Qwen3-0.6B), an independent model smaller than any of our six rephrasers), sketch the resulting gradient via 1024 random index samples from the last MLP down-projection layer, L2-normalise, and take the Vendi Score over the resulting feature set per dataset.<Sidenote>This is a cheaper approximation of the @prismatic-synthesis pipeline. The paper computes gradients across all proxy parameters and sketches them via Rademacher random projection (a Johnson-Lindenstrauss ±1 transform that preserves pairwise dot products). We restrict gradients to the last MLP `down_proj.weight` (~3M parameters in Qwen3-0.6B) and sketch by sub-sampling 1024 random indices instead, both to make 83 datasets tractable on Apple Silicon.</Sidenote> G-Vendi correlates positively with almost every benchmark we care about, including `agg_score_macro` (ρ = 0.26, p {'<'} 0.05) and `agg_score_micro` (ρ = 0.29, p {'<'} 0.01) where semantic Vendi is non-significantly negative. It is the single best predictor for reasoning (`agg_score_RES` ρ = 0.38, p {'<'} 0.001) and XCSQA (ρ = 0.34, p {'<'} 0.01), beating every DCLM, edu-score, and semantic-diversity variant. Across our 83 experiments, G-Vendi consistently beats semantic Vendi as a downstream-performance predictor, confirming the headline finding of @prismatic-synthesis: **gradient-space diversity predicts training value better than embedding-space diversity.**
 
 **But neither diversity flavor displaces the quality scores.** DCLM-difference is the strongest macro and micro predictor (ρ ≈ 0.59–0.61, p {'<'} 0.001), and the diversity metrics complement it rather than replacing it. If you had to pick a single cheap proxy per benchmark family, the best bets are: DCLM-difference for reading comprehension (ρ = 0.52, p {'<'} 0.001), table understanding (ρ = 0.54, p {'<'} 0.001), and general knowledge (ρ = 0.29, p {'<'} 0.01); DCLM-output for math (ρ = 0.39, p {'<'} 0.001); G-Vendi for reasoning (ρ = 0.38, p {'<'} 0.001); and either edu-output (negative, ρ = −0.27, p {'<'} 0.05) or G-Vendi (ρ = +0.21, n.s.) for NLU, which has the weakest signal of any benchmark family. **Even the strongest of these explains only ~37% of variance, WinoGrande is uncorrelated with everything we tested, and most per-family signals fall below ρ = 0.4.** Treat the cheat sheet as triage for ranking candidate datasets, not as a substitute for training and evaluating.