Any-to-Any
Transformers
Safetensors
multilingual
minicpmo
feature-extraction
minicpm-o
omni
vision
ocr
multi-image
video
custom_code
audio
speech
voice cloning
live Streaming
realtime speech conversation
asr
tts
4-bit precision
gptq
Instructions to use openbmb/MiniCPM-o-2_6-int4 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use openbmb/MiniCPM-o-2_6-int4 with Transformers:
# pip install -U transformers accelerate # Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("openbmb/MiniCPM-o-2_6-int4", trust_remote_code=True, device_map="auto") - Notebooks
- Google Colab
- Kaggle
Download quantize_config.json from openbmb/MiniCPM-o-2_6-int4: direct link, hf CLI and curl.
- Browser
- Download file 286 Bytes
-
https://huggingface.co/openbmb/MiniCPM-o-2_6-int4/resolve/main/quantize_config.json
- Command line
-
hf download hf://openbmb/MiniCPM-o-2_6-int4/quantize_config.json
-
curl -L -o quantize_config.json https://huggingface.co/openbmb/MiniCPM-o-2_6-int4/resolve/main/quantize_config.json
286 Bytes
| { | |
| "bits": 4, | |
| "group_size": -1, | |
| "damp_percent": 0.01, | |
| "desc_act": true, | |
| "static_groups": false, | |
| "sym": true, | |
| "true_sequential": true, | |
| "lm_head": false, | |
| "model_name_or_path": null, | |
| "model_file_base_name": null, | |
| "quant_method": "gptq", | |
| "checkpoint_format": "gptq" | |
| } |