junaid008
/

hindko-tokenizer

Northern Hindko

Southern Hindko

Model card Files Files and versions

hindko-tokenizer

6.36 MB

Ctrl+K

Ctrl+K

1 contributor

History: 11 commits

junaid008's picture

Card: repository is public (no token needed); link code and split manifest

92ea296 verified about 2 months ago

benchmarks
Add benchmark images (benchmarks/) about 2 months ago
eval
Add evaluation tables and charts about 2 months ago
examples
Upload quickstart.py about 2 months ago
.gitattributes

2.21 kB
Add benchmark images (benchmarks/) about 2 months ago
LICENSE

19.3 kB
License: CC BY 4.0; add benchmark gallery to the model card; refresh SHA256SUMS about 2 months ago
README.md

48.6 kB
Card: repository is public (no token needed); link code and split manifest about 2 months ago
SHA256SUMS

2.85 kB
Card: repository is public (no token needed); link code and split manifest about 2 months ago
gauge.json

6.91 kB
Hindko tokenizer v1.0.0: SentencePiece Unigram 32k (tokenizer files) about 2 months ago
sp.model

669 kB
xet

Hindko tokenizer v1.0.0: SentencePiece Unigram 32k (tokenizer files) about 2 months ago
sp.vocab

951 kB
Hindko tokenizer v1.0.0: SentencePiece Unigram 32k (tokenizer files) about 2 months ago
special_tokens_map.json

1.61 kB
Hindko tokenizer v1.0.0: SentencePiece Unigram 32k (tokenizer files) about 2 months ago
tokenizer.json

2.16 MB
Hindko tokenizer v1.0.0: SentencePiece Unigram 32k (tokenizer files) about 2 months ago
tokenizer_config.json

14.6 kB
Hindko tokenizer v1.0.0: SentencePiece Unigram 32k (tokenizer files) about 2 months ago