Translation
Transformers
ONNX
Safetensors
PyTorch
Khmer
blenderbot
text2text-generation
transliteration
romanization
romanized-khmer
khmer
character-level
int8
Eval Results (legacy)
Instructions to use samdei/samdei-type-base with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use samdei/samdei-type-base with Transformers:
# Use a pipeline as a high-level helper # Warning: Pipeline type "translation" is no longer supported in transformers v5. # You must load the model directly (see below) or downgrade to v4.x with: # pip install "transformers<5.0.0" from transformers import pipeline pipe = pipeline("translation", model="samdei/samdei-type-base")# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForSeq2SeqLM tokenizer = AutoTokenizer.from_pretrained("samdei/samdei-type-base") model = AutoModelForSeq2SeqLM.from_pretrained("samdei/samdei-type-base", device_map="auto") - Notebooks
- Google Colab
- Kaggle
File size: 717 Bytes
7a7ad8f 231cdd1 7a7ad8f a3cc27d 7a7ad8f | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 | {
"model": "Samdei Type Base",
"backend": "torch",
"beam": 5,
"chunk_max_words": 8,
"chunk_rule": "samdei_type.evaluation.chunk_source: at most 8 words per piece, break after any punctuation mark; pieces are decoded and reranked on their own and joined with a space",
"rerank_weights": {"char": 0.3, "word": 0.3, "oov": 1.0, "channel": 0.3},
"score": "log P(y|x) + char*log P_char(y) + word*log P_word(y) - oov*#OOV(y) + channel*log P_ch(x|y)",
"char_lm": "lm/char5_nowiki.npz",
"word_lm": "lm/wordfull_nowiki.json.gz",
"channel_model": "channel/",
"max_input_chars": 249,
"weights_tuned_on": "closed development set, whole input, 2-fold cross-validated grid search; applied unchanged to 8-word pieces"
}
|