pika
🎉 You are looking at pika 5, which is using FineWeb-Edu!
pika is a simple and public domain-like tokenizer.
Special Tokens
- End-of-Sequence token:
<|endoftext|>(ID 0)
Training
pika was trained on a shuffled portion of HuggingFaceFW/fineweb-edu.
Limitations
Extra special tokens aren't present, you'll have to add them manually if needed.
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support