CodeBERT-language-detect / tokenizer_config.json

Commit History

  1. fix: reduce number of languages, add lora

    42edf3c

  1. feat: switch train set -> better perf⁠

    3fda8cb

  1. fix: max len tokenizer

    128954f

  1. add tokenizer

    230bbf3