midigenai / v3

Commit History

  1. v3 tokenizer

    Verified signature
    71347f0
  2. v3: 113.8M, 24B tokens on corpus_full (7 sources, strict dedup, EOS docs); val 0.7495; serve at t=1.1 top_k=50

    Verified signature
    033937d