MTP works after adding layer 45 to the quantization exclusion lists
#15
by soonh1618 - opened
Hi, I encountered an issue running MTP with nvidia/GLM-5.3-Flash-NVFP4.
MTP started working after I added entries for the MTP layer (layer 45) to both:
- quantization_config.ignore in config.json
- quantization.exclude_modules in hf_quant_config.json
I only changed these configuration files; I did not modify or re-quantize the weights.
This suggests that the missing exclusion entries for layer 45 may be causing the MTP issue in my setup. Could you confirm whether these entries should be included in the published configuration files?
Thanks @soonh1618 , this is caused by an issue in exporting models that have weights that are not loaded in PTQ, we have fixed the ModelOpt library code and also adopted community fix for the checkpoint. Thanks!