MTP-enabled Arete-27B checkpoint
Could you release an MTP-enabled checkpoint or a compatible MTP head for Arete?
Were the original MTP weights retained and simply omitted from the export? If not, could you advise whether reusing the base Qwen head is appropriate, or whether it needs additional training and validation after Arete’s fine-tuning?
I no longer see any innovation in Western models, beyond their political stances.
Could you release an MTP-enabled checkpoint or a compatible MTP head for Arete?
Were the original MTP weights retained and simply omitted from the export? If not, could you advise whether reusing the base Qwen head is appropriate, or whether it needs additional training and validation after Arete’s fine-tuning?
Using the original qwen head works, in my testing.