ThisenEkanayake
/

sinhala-tokenizer

Model card Files Files and versions

sinhala-tokenizer

Commit History

Oct 1, 2025

13:19
tokenizer class updated

Thisen Ekanayake

e337c70
13:13
renamed to "tokenizer"

Thisen Ekanayake

6e0c6f7
13:12
renamed

Thisen Ekanayake

49d2aac

Sep 30, 2025

16:51
updated tokenizer class

Thisen Ekanayake

3841b6e
16:14
Add custom Sinhala SentencePiece tokenizer

Thisen Ekanayake

01e5ddf
16:12
initial commit

ThisenEkanayake

Verified signature
48fae19