File size: 1,159 Bytes
7a7ad8f
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
Samdei Type Base
Copyright 2026 The Samdei Type Authors

Licensed under the Apache License, Version 2.0 (see LICENSE). This covers every file in this
repository: the model weights, the ONNX graphs, onnx/vocab.json, the channel model, rerank.json
and the language models.

Language models (lm/char5_nowiki.npz, lm/wordfull_nowiki.json.gz)
  Character and word n-gram counts of Khmer text, built without Wikipedia, from:
  - HPLT v2 (HPLT/HPLT2.0_cleaned, khm_Khmr): CC0 packaging; the Common Crawl Terms of Use
    apply to the text.
  - FineWeb-2 (HuggingFaceFW/fineweb-2, khm_Khmr): ODC-By 1.0; the Common Crawl Terms of Use
    apply to the text.
  - Asian Language Treebank (ALT), Khmer side: CC BY 4.0.
  - Samdei Type chat sentences (samdei/samdei-type-resources): CC BY 4.0.

Training data
  The model and the channel model were trained on samdei/samdei-type-pretraining-data and
  samdei/samdei-type-finetuning-data (CC BY-SA 4.0); their cards list every source. Part of the
  fine-tuning pairs were spelled with tables derived from the Chhunneng romanized Khmer
  dataset, MIT License, Copyright (c) 2025 Chrea Chanchhunneng, and from Wikidata (CC0).