Hugging Face's logo

usrnotfound101
/
anlp-a2-tokenizers

Model card Files
xet
 Community

YAML Metadata Warning:empty or missing yaml metadata in repo card

Check out the documentation for more information.

ANLP A2 tokenizers

  • mt_spm.model - joint en/vi/ja SentencePiece unigram (32000 pieces, byte fallback, special tokens <vi> <ja> <en>)
  • lm_spm.model - English SentencePiece unigram for HAP-E (16384 pieces)
  • tokenizer_stats.json - fertility statistics
Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support