samdei-type-base / rerank.json
mengsay's picture
Release cards: no paper URL yet, takedown contact via Community tab; rerank.json names the backend of the published scores
231cdd1 verified
Raw History Blame Contribute Delete
717 Bytes
{
"model": "Samdei Type Base",
"backend": "torch",
"beam": 5,
"chunk_max_words": 8,
"chunk_rule": "samdei_type.evaluation.chunk_source: at most 8 words per piece, break after any punctuation mark; pieces are decoded and reranked on their own and joined with a space",
"rerank_weights": {"char": 0.3, "word": 0.3, "oov": 1.0, "channel": 0.3},
"score": "log P(y|x) + char*log P_char(y) + word*log P_word(y) - oov*#OOV(y) + channel*log P_ch(x|y)",
"char_lm": "lm/char5_nowiki.npz",
"word_lm": "lm/wordfull_nowiki.json.gz",
"channel_model": "channel/",
"max_input_chars": 249,
"weights_tuned_on": "closed development set, whole input, 2-fold cross-validated grid search; applied unchanged to 8-word pieces"
}