claudios
/

sptcode

Feature Extraction

Model card Files Files and versions

sptcode / code_tokenizer_fast /special_tokens_map.json

claudios's picture

Upload folder using huggingface_hub

cbcf394 verified over 2 years ago

History Blame Contribute Delete

148 Bytes

	{
	"bos_token": "[SOS]",
	"eos_token": "[EOS]",
	"mask_token": "[MSK]",
	"pad_token": "[PAD]",
	"sep_token": "[SEP]",
	"unk_token": "[UNK]"
	}