Commit History

  1. Add tokenizer_config.json

    Verified signature
    dd82552
  2. Add tokenizer.json (vocab 32000, BPE, fit on fineweb-edu-dedup 1.6M + k-10-12-mixed)

    Verified signature
    acbb711
  3. initial commit

    Verified signature
    f5d9731