Instructions to use Yusra/quantize_model with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use Yusra/quantize_model with Transformers:
# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForSeq2SeqLM tokenizer = AutoTokenizer.from_pretrained("Yusra/quantize_model") model = AutoModelForSeq2SeqLM.from_pretrained("Yusra/quantize_model", device_map="auto") - Notebooks
- Google Colab
- Kaggle
File size: 273 Bytes
decd9fc | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 | {
"bos_token_id": 0,
"decoder_start_token_id": 2,
"early_stopping": true,
"eos_token_id": 2,
"length_penalty": 2.0,
"max_length": 512,
"min_length": 100,
"no_repeat_ngram_size": 3,
"num_beams": 4,
"pad_token_id": 1,
"transformers_version": "4.31.0"
}
|