Hugging Face's logo

tapiocaTakeshi
/
Qubit-7B-ja-training

Model card Files
xet
 Community
Qubit-7B-ja-training / code /prep_data.py

Commit History

  1. prep: stream FineWeb2 test split (avoid preparing full train split)

    tapiocaTakeshi
    Verified signature
    51b49bd

  2. Add data prep + tokenizer script

    tapiocaTakeshi
    Verified signature
    9730675