mengsay's picture
Initial private draft of the Samdei Type release
73c7c04 verified
Raw History Blame Contribute Delete
1.06 kB
Samdei Type Mini
Copyright 2026 The Samdei Type Authors
Licensed under the Apache License, Version 2.0 (see LICENSE). This covers every file in this
repository: the ONNX graphs, onnx/vocab.json, rerank.json and the word language model.
Word language model (lm/wordfull_nowiki_p25.json.gz)
Word unigram and bigram counts of Khmer text, built without Wikipedia, from:
- HPLT v2 (HPLT/HPLT2.0_cleaned, khm_Khmr): CC0 packaging; the Common Crawl Terms of Use
apply to the text.
- FineWeb-2 (HuggingFaceFW/fineweb-2, khm_Khmr): ODC-By 1.0; the Common Crawl Terms of Use
apply to the text.
- Asian Language Treebank (ALT), Khmer side: CC BY 4.0.
- Samdei Type chat sentences (samdei/samdei-type-resources): CC BY 4.0.
Training data
The model was trained on samdei/samdei-type-pretraining-data and
samdei/samdei-type-finetuning-data (CC BY-SA 4.0); their cards list every source. Part of the
fine-tuning pairs were spelled with a table derived from the Chhunneng romanized Khmer
dataset, MIT License, Copyright (c) 2025 Chrea Chanchhunneng.