Commit History

  1. Add 2MB val text sample for Gemma/HF tokenizer notebooks

    Verified signature
    588ff7f
  2. Add 20MB raw text sample for Gemma/HF tokenizer notebooks

    Verified signature
    d9422a5

  1. Add curated training tokens (266M tokens, Chinchilla-optimal)

    Verified signature
    ab3f5b8
  2. Add validation tokens (72M tokens)

    Verified signature
    ff30a4a