qyxu1994's picture
Fix eval-env tokenizer loading: set add_prefix_space=true in tokenizer_config.json (transformers 4.x ignores it in tokenizer.json)
fbf7815 verified