Transformers How to use pacifio/untoken-v2 with Transformers:
# pip install -U transformers accelerate
# Load model directly
from transformers import UntokenCompressor
model = UntokenCompressor.from_pretrained("pacifio/untoken-v2", device_map="auto") 
