semeru
/

completeformer_java

Model card Files Files and versions

completeformer_java / added_tokens.json

ncoop57

Add java tokenizer

0a6a5c8 over 3 years ago

History Blame Contribute Delete

99 Bytes

	{
	"<BOS>": 32770,
	"<EOS>": 32771,
	"<MASK>": 32768,
	"<NEW_LINE>": 32772,
	"<PAD>": 32769
	}