p1atdev
/

multi-tokenizers-processor-sample

Transformers

Model card Files Files and versions

xet

Community

multi-tokenizers-processor-sample / processor_config.json

p1atdev

Upload tokenizer

b5e658d verified almost 2 years ago

Raw History Blame

173 Bytes

{
  "auto_map": {
    "AutoProcessor": [
      "processor_multitokenizers.MultiTokenizersProcessor",
      null
    ]
  },
  "processor_class": "MultiTokenizersProcessor"
}

	{
	"auto_map": {
	"AutoProcessor": [
	"processor_multitokenizers.MultiTokenizersProcessor",
	null
	]
	},
	"processor_class": "MultiTokenizersProcessor"
	}