junaid008
/

hindko-tokenizer

Northern Hindko

Southern Hindko

Model card Files Files and versions

hindko-tokenizer

6.36 MB

Ctrl+K

Ctrl+K

1 contributor

History: 11 commits

junaid008's picture

Card: repository is public (no token needed); link code and split manifest

92ea296 verified about 4 hours ago

benchmarks
Add benchmark images (benchmarks/) 4 days ago
eval
Add evaluation tables and charts 4 days ago
examples
Upload quickstart.py 4 days ago
.gitattributes

2.21 kB
Add benchmark images (benchmarks/) 4 days ago
LICENSE

19.3 kB
License: CC BY 4.0; add benchmark gallery to the model card; refresh SHA256SUMS 4 days ago
README.md

48.6 kB
Card: repository is public (no token needed); link code and split manifest about 4 hours ago
SHA256SUMS

2.85 kB
Card: repository is public (no token needed); link code and split manifest about 4 hours ago
gauge.json

6.91 kB
Hindko tokenizer v1.0.0: SentencePiece Unigram 32k (tokenizer files) 4 days ago
sp.model

669 kB
xet

Hindko tokenizer v1.0.0: SentencePiece Unigram 32k (tokenizer files) 4 days ago
sp.vocab

951 kB
Hindko tokenizer v1.0.0: SentencePiece Unigram 32k (tokenizer files) 4 days ago
special_tokens_map.json

1.61 kB
Hindko tokenizer v1.0.0: SentencePiece Unigram 32k (tokenizer files) 4 days ago
tokenizer.json

2.16 MB
Hindko tokenizer v1.0.0: SentencePiece Unigram 32k (tokenizer files) 4 days ago
tokenizer_config.json

14.6 kB
Hindko tokenizer v1.0.0: SentencePiece Unigram 32k (tokenizer files) 4 days ago