AwakeningOS commited on
Commit
4f7b45f
·
verified ·
1 Parent(s): 9287d39

Explain development metrics and experiment terminology for new readers

Browse files
Files changed (3) hide show
  1. EVALUATION.md +1 -1
  2. REPRODUCING.md +1 -1
  3. SHA256SUMS +2 -2
EVALUATION.md CHANGED
@@ -24,7 +24,7 @@ Recorded revision: `6f825c291e2c4c78ad33b1935fd64d45f52642dc`.
24
 
25
  The learning-curve chart omits SuperGLUE, Reading and AoA and averages the six zero-shot categories only. Global PIQA contributes once. Its parallel and nonparallel splits receive equal weight. Reading uses the mean of the evaluator's eye-tracking and self-paced reading scores. AoA is a correlation, not MSE or accuracy.
26
 
27
- The score summaries retain the evaluator's reported precision. Public-table comparison uses the saved 2026-09-15 snapshot and does not constitute an official placement. Final scores and all release claims refer to the raw 80M model; task fine-tuning is a separate operation.
28
 
29
  ## Re-running
30
 
 
24
 
25
  The learning-curve chart omits SuperGLUE, Reading and AoA and averages the six zero-shot categories only. Global PIQA contributes once. Its parallel and nonparallel splits receive equal weight. Reading uses the mean of the evaluator's eye-tracking and self-paced reading scores. AoA is a correlation, not MSE or accuracy.
26
 
27
+ The score summaries retain the evaluator's reported precision. Public-table comparison uses the saved 2026-09-15 snapshot and does not constitute an official placement. Final scores and all release claims refer to the 80M-word checkpoint without weight averaging model; task fine-tuning is a separate operation.
28
 
29
  ## Re-running
30
 
REPRODUCING.md CHANGED
@@ -30,7 +30,7 @@ The builder writes objective flags inherited from a shared data format. VISTA's
30
 
31
  ## Checkpoint identity
32
 
33
- `main` uses the same tensors as the raw 80M checkpoint:
34
 
35
  - Original checkpoint SHA-256: `2b4f0555d6a5e614bd81d09555c008ac8f6310e75a6a3b96378e24fd9d5ff018`.
36
  - Exported safetensors SHA-256: `ea395c4df82b7ca0f05d4b4d97b2e1102751ce61e66b3e1345645deb8fd6ce43`.
 
30
 
31
  ## Checkpoint identity
32
 
33
+ `main` uses the same tensors as the 80M-word checkpoint without weight averaging checkpoint:
34
 
35
  - Original checkpoint SHA-256: `2b4f0555d6a5e614bd81d09555c008ac8f6310e75a6a3b96378e24fd9d5ff018`.
36
  - Exported safetensors SHA-256: `ea395c4df82b7ca0f05d4b4d97b2e1102751ce61e66b3e1345645deb8fd6ce43`.
SHA256SUMS CHANGED
@@ -1,10 +1,10 @@
1
  ac1c58d234dbfebd92c729332e5e7ffbda082320dcca5d3994eac066f092899c ARCHITECTURE.md
2
  131b1e127dd02e982903599008a59403be66cfd1f8e5e95a6cdd57ab5bbc33ec CHECKPOINT_TESTS.json
3
- e3689afd11382b2f4a9abd54ae0d4df6c4455de471ae8714ac6a54cef4ccb6cf EVALUATION.md
4
  ce281119447213d9390ce20306ca1c1b4e62716900f25aeee14a83b1b3866cc6 LICENSE
5
  f8cc165cad9cf3d2be69445cbb1e03895d1d1b50b002c7a4c963804349b7ead8 README.md
6
  85a62bfcccea1d85bdf5ec6dc7bce30a3c0ee7319ce654f2770cf9146f5ed594 RELEASE_TESTS.json
7
- 8de8d9626aa47bd92b125a6453fdf6c8fc8d78bae2583b3b704bf6baec9baa0b REPRODUCING.md
8
  ae5f26e74fcfae9b0d3a5fbb1ed6a99a6c4bde7b089d2184f8aadb06c7c7c79b assets/architecture.png
9
  0cb38fff6ceb408d78edeb98f7ba847557881c9e74146c23ed8695d72e036e55 assets/architecture.svg
10
  6ac2b8659671cf0a1938db20e4c9ded2b56bc2d1ec1199c36e102079998b30dc assets/learning_curve.png
 
1
  ac1c58d234dbfebd92c729332e5e7ffbda082320dcca5d3994eac066f092899c ARCHITECTURE.md
2
  131b1e127dd02e982903599008a59403be66cfd1f8e5e95a6cdd57ab5bbc33ec CHECKPOINT_TESTS.json
3
+ eadbbf1800359a071394bcea755b8c80393308295d12a5de1ac4448b1afd5762 EVALUATION.md
4
  ce281119447213d9390ce20306ca1c1b4e62716900f25aeee14a83b1b3866cc6 LICENSE
5
  f8cc165cad9cf3d2be69445cbb1e03895d1d1b50b002c7a4c963804349b7ead8 README.md
6
  85a62bfcccea1d85bdf5ec6dc7bce30a3c0ee7319ce654f2770cf9146f5ed594 RELEASE_TESTS.json
7
+ 878f5d13dfa97d0dc81e7fb7ae75a460cc80037ef4f86b9c4c9cf7582cbbc513 REPRODUCING.md
8
  ae5f26e74fcfae9b0d3a5fbb1ed6a99a6c4bde7b089d2184f8aadb06c7c7c79b assets/architecture.png
9
  0cb38fff6ceb408d78edeb98f7ba847557881c9e74146c23ed8695d72e036e55 assets/architecture.svg
10
  6ac2b8659671cf0a1938db20e4c9ded2b56bc2d1ec1199c36e102079998b30dc assets/learning_curve.png