Automatic Speech Recognition
Transformers
Safetensors
phi4mm
text-generation
auto-round
w4a16
custom_code
4-bit precision
Instructions to use INCModel4/Phi-4-multimodal-instruct-W4A16-AutoRound with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use INCModel4/Phi-4-multimodal-instruct-W4A16-AutoRound with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("automatic-speech-recognition", model="INCModel4/Phi-4-multimodal-instruct-W4A16-AutoRound", trust_remote_code=True)# pip install -U transformers accelerate # Load model directly from transformers import AutoModelForCausalLM model = AutoModelForCausalLM.from_pretrained("INCModel4/Phi-4-multimodal-instruct-W4A16-AutoRound", trust_remote_code=True, device_map="auto") - Notebooks
- Google Colab
- Kaggle
Download auto-round-block-list-fallback.patch from INCModel4/Phi-4-multimodal-instruct-W4A16-AutoRound: direct link, hf CLI and curl.
- Browser
- Download file 758 Bytes
-
https://huggingface.co/INCModel4/Phi-4-multimodal-instruct-W4A16-AutoRound/resolve/main/auto-round-block-list-fallback.patch
- Command line
-
hf download hf://INCModel4/Phi-4-multimodal-instruct-W4A16-AutoRound/auto-round-block-list-fallback.patch
-
curl -L -o auto-round-block-list-fallback.patch https://huggingface.co/INCModel4/Phi-4-multimodal-instruct-W4A16-AutoRound/resolve/main/auto-round-block-list-fallback.patch
758 Bytes
| --- a/auto_round/compressors/base.py | |
| +++ b/auto_round/compressors/base.py | |
| @property | |
| def quant_block_list(self): | |
| plan = self.__dict__.get("compression_plan") | |
| if plan is None: | |
| return self.__dict__.get("_quant_block_list") | |
| - return [list(group) for group in plan.quant_block_list] if plan.quant_block_list is not None else None | |
| + if plan.quant_block_list is None: | |
| + return self.__dict__.get("_quant_block_list") | |
| + return [list(group) for group in plan.quant_block_list] | |
| @quant_block_list.setter | |
| def quant_block_list(self, value) -> None: | |
| self.__dict__["_quant_block_list"] = value | |
| self._replace_compression_plan(quant_block_list=value) | |