Add `{% generation %}` blocks to the chat template for training
#5
by bezzam HF Staff - opened
This PR wraps the assistant turns of the chat template in {% generation %} ... {% endgeneration %} blocks. The rendered text is unchanged (checked on single- and multi-turn conversations, with and without a generation prompt).
These blocks let apply_chat_template(..., return_assistant_tokens_mask=True) return a mask of the assistant tokens, so that only the assistant turn (the transcription) contributes to the loss during training, e.g. with processor.apply_transcription_request(audio, transcription=...) in Transformers, or assistant_only_loss=True in TRL. Without these blocks, Transformers raises an error when an assistant mask is requested (https://github.com/huggingface/transformers/pull/49870).
Same convention as Qwen/Qwen3-ASR-1.7B-hf.