File size: 183,815 Bytes
1586f2d b5b3ca1 faee79b e557193 faee79b b5b3ca1 faee79b b5b3ca1 faee79b b5b3ca1 faee79b b5b3ca1 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 | <!doctype html><meta charset=utf-8><title>transformers radar</title><style>body{max-width:1200px;margin:auto;padding:1em;font-family:system-ui}table{border-collapse:collapse;width:100%}td,th{border-bottom:1px solid #ddd;padding:4px 8px;text-align:left;vertical-align:top}code{font-size:85%}</style><h1>transformers downstream radar · 2026-11-08</h1>
<p>44 repos · last 2 days of GitHub activity · 659 monkey patches tracked</p>
<h2>Act on these</h2>
<table><tr><th></th><th>action</th><th>component</th><th>what</th><th>threads</th></tr></table>
<details><summary>All 0 matching threads</summary>
<table><tr><th>thread</th><th>comments</th><th>title</th></tr></table>
</details>
<h2>New monkey patches since last run (0)</h2>
<p>Something changed on their side or ours: check whether a transformers change forced it.</p>
<p>None.</p>
<h2>Most-patched transformers internals</h2>
<p>Patched by several repos = missing extension point (kernel hub / attention interface / hook).</p>
<table><tr><th>symbol</th><th>#repos</th><th>repos</th></tr><tr><td><code>*Attention.forward</code></td><td>8</td><td>FlagEmbedding, TensorRT-LLM, axolotl, LlamaFactory, ms-swift, unsloth, unsloth-zoo, verl</td></tr><tr><td><code>*ForCausalLM.forward</code></td><td>6</td><td>Automodel, axolotl, Liger-Kernel, ms-swift, unsloth, unsloth-zoo</td></tr><tr><td><code>*Model.forward</code></td><td>6</td><td>lmms-eval, axolotl, ms-swift, unsloth, unsloth-zoo, verl</td></tr><tr><td><code>*MLP.forward</code></td><td>5</td><td>TensorRT-LLM, axolotl, DeepSpeed, unsloth, verl</td></tr><tr><td><code>*DecoderLayer.forward</code></td><td>4</td><td>axolotl, LlamaFactory, unsloth, verl</td></tr><tr><td><code>*FlashAttention2</code></td><td>4</td><td>GPTQModel, ms-swift, sglang, unsloth-zoo</td></tr><tr><td><code>*Experts</code></td><td>3</td><td>axolotl, Liger-Kernel, unsloth-zoo</td></tr><tr><td><code>*Model._create_and_replace</code></td><td>3</td><td>axolotl, ms-swift, unsloth-zoo</td></tr><tr><td><code>*RMSNorm</code></td><td>3</td><td>axolotl, Liger-Kernel, unsloth</td></tr><tr><td><code>PreTrainedModel.from_pretrained</code></td><td>3</td><td>ColossalAI, ms-swift, unsloth</td></tr><tr><td><code>apply_rotary_pos_emb</code></td><td>3</td><td>axolotl, Liger-Kernel, verl</td></tr><tr><td><code>create_causal_mask</code></td><td>3</td><td>GPTQModel, ms-swift, unsloth-zoo</td></tr><tr><td><code>*Experts.forward</code></td><td>2</td><td>unsloth-zoo, verl</td></tr><tr><td><code>*FlashAttention2.forward</code></td><td>2</td><td>LlamaFactory, unsloth</td></tr><tr><td><code>*MLP</code></td><td>2</td><td>axolotl, Liger-Kernel</td></tr><tr><td><code>*Model._postprocess</code></td><td>2</td><td>ROLL, verl</td></tr><tr><td><code>*Model.load_adapter</code></td><td>2</td><td>ms-swift, unsloth-zoo</td></tr><tr><td><code>*Model.save_pretrained</code></td><td>2</td><td>unsloth, unsloth-zoo</td></tr><tr><td><code>*RMSNorm.forward</code></td><td>2</td><td>Automodel, verl</td></tr><tr><td><code>*SdpaAttention.forward</code></td><td>2</td><td>LlamaFactory, unsloth</td></tr><tr><td><code>*SparseMoeBlock</code></td><td>2</td><td>LlamaFactory, verl</td></tr><tr><td><code>CrossEntropyLoss</code></td><td>2</td><td>axolotl, Liger-Kernel</td></tr><tr><td><code>PreTrainedModel.enable_input_require_grads</code></td><td>2</td><td>ms-swift, unsloth</td></tr><tr><td><code>PreTrainedModel.post_init</code></td><td>2</td><td>sglang, unsloth</td></tr><tr><td><code>_init_weights</code></td><td>2</td><td>GPTQModel, ColossalAI</td></tr><tr><td><code>checkpoint</code></td><td>2</td><td>ms-swift, unsloth-zoo</td></tr><tr><td><code>dynamic_module_utils.check_imports</code></td><td>2</td><td>LlamaFactory, ktransformers</td></tr><tr><td><code>get_model_conversion_mapping</code></td><td>2</td><td>unsloth, unsloth-zoo</td></tr><tr><td><code>nn.CrossEntropyLoss</code></td><td>2</td><td>axolotl, Liger-Kernel</td></tr><tr><td><code>nn.LayerNorm</code></td><td>2</td><td>axolotl, Liger-Kernel</td></tr><tr><td><code>*Attention</code></td><td>1</td><td>GPTQModel</td></tr><tr><td><code>*Attention.ATTN_DONE</code></td><td>1</td><td>ColossalAI</td></tr><tr><td><code>*Attention.CU_SEQLENS</code></td><td>1</td><td>ColossalAI</td></tr><tr><td><code>*Attention.DEFAULT_BLOCK_SIZES</code></td><td>1</td><td>diffusers</td></tr><tr><td><code>*Attention.HALF_INDICES</code></td><td>1</td><td>ColossalAI</td></tr><tr><td><code>*Attention.INNER_RING_GROUP</code></td><td>1</td><td>ColossalAI</td></tr><tr><td><code>*Attention.INTER_RING_GROUP</code></td><td>1</td><td>ColossalAI</td></tr><tr><td><code>*Attention.SP_GROUP</code></td><td>1</td><td>ColossalAI</td></tr><tr><td><code>*Attention.SP_STREAM</code></td><td>1</td><td>ColossalAI</td></tr><tr><td><code>*Attention.TOTAL_SEQLEN</code></td><td>1</td><td>ColossalAI</td></tr></table>
<h2>Per repo</h2>
<table><tr><th>repo</th><th>threads</th><th>patches</th><th>new patches</th></tr><tr><td><a href="https://github.com/ggml-org/llama.cpp">ggml-org/llama.cpp</a></td><td>0</td><td>0</td><td>0</td></tr><tr><td><a href="https://github.com/vllm-project/vllm">vllm-project/vllm</a></td><td>0</td><td>6</td><td>0</td></tr><tr><td><a href="https://github.com/unslothai/unsloth">unslothai/unsloth</a></td><td>0</td><td>132</td><td>0</td></tr><tr><td><a href="https://github.com/hiyouga/LlamaFactory">hiyouga/LlamaFactory</a></td><td>0</td><td>8</td><td>0</td></tr><tr><td><a href="https://github.com/deepspeedai/DeepSpeed">deepspeedai/DeepSpeed</a></td><td>0</td><td>11</td><td>0</td></tr><tr><td><a href="https://github.com/hpcaitech/ColossalAI">hpcaitech/ColossalAI</a></td><td>0</td><td>14</td><td>0</td></tr><tr><td><a href="https://github.com/sgl-project/sglang">sgl-project/sglang</a></td><td>0</td><td>4</td><td>0</td></tr><tr><td><a href="https://github.com/huggingface/diffusers">huggingface/diffusers</a></td><td>0</td><td>3</td><td>0</td></tr><tr><td><a href="https://github.com/huggingface/lerobot">huggingface/lerobot</a></td><td>0</td><td>0</td><td>0</td></tr><tr><td><a href="https://github.com/huggingface/open-r1">huggingface/open-r1</a></td><td>0</td><td>0</td><td>0</td></tr><tr><td><a href="https://github.com/m-bain/whisperX">m-bain/whisperX</a></td><td>0</td><td>0</td><td>0</td></tr><tr><td><a href="https://github.com/verl-project/verl">verl-project/verl</a></td><td>0</td><td>48</td><td>0</td></tr><tr><td><a href="https://github.com/huggingface/peft">huggingface/peft</a></td><td>0</td><td>0</td><td>0</td></tr><tr><td><a href="https://github.com/huggingface/trl">huggingface/trl</a></td><td>0</td><td>1</td><td>0</td></tr><tr><td><a href="https://github.com/kvcache-ai/ktransformers">kvcache-ai/ktransformers</a></td><td>0</td><td>5</td><td>0</td></tr><tr><td><a href="https://github.com/huggingface/sentence-transformers">huggingface/sentence-transformers</a></td><td>0</td><td>0</td><td>0</td></tr><tr><td><a href="https://github.com/NVIDIA-NeMo/Automodel">NVIDIA-NeMo/Automodel</a></td><td>0</td><td>10</td><td>0</td></tr><tr><td><a href="https://github.com/modelscope/ms-swift">modelscope/ms-swift</a></td><td>0</td><td>53</td><td>0</td></tr><tr><td><a href="https://github.com/dottxt-ai/outlines">dottxt-ai/outlines</a></td><td>0</td><td>0</td><td>0</td></tr><tr><td><a href="https://github.com/NVIDIA/TensorRT-LLM">NVIDIA/TensorRT-LLM</a></td><td>0</td><td>2</td><td>0</td></tr><tr><td><a href="https://github.com/flairNLP/flair">flairNLP/flair</a></td><td>0</td><td>0</td><td>0</td></tr><tr><td><a href="https://github.com/EleutherAI/lm-evaluation-harness">EleutherAI/lm-evaluation-harness</a></td><td>0</td><td>0</td><td>0</td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl">axolotl-ai-cloud/axolotl</a></td><td>0</td><td>81</td><td>0</td></tr><tr><td><a href="https://github.com/FlagOpen/FlagEmbedding">FlagOpen/FlagEmbedding</a></td><td>0</td><td>2</td><td>0</td></tr><tr><td><a href="https://github.com/OpenRLHF/OpenRLHF">OpenRLHF/OpenRLHF</a></td><td>0</td><td>1</td><td>0</td></tr><tr><td><a href="https://github.com/huggingface/accelerate">huggingface/accelerate</a></td><td>0</td><td>0</td><td>0</td></tr><tr><td><a href="https://github.com/THUDM/slime">THUDM/slime</a></td><td>0</td><td>0</td><td>0</td></tr><tr><td><a href="https://github.com/bitsandbytes-foundation/bitsandbytes">bitsandbytes-foundation/bitsandbytes</a></td><td>0</td><td>0</td><td>0</td></tr><tr><td><a href="https://github.com/ml-explore/mlx-lm">ml-explore/mlx-lm</a></td><td>0</td><td>0</td><td>0</td></tr><tr><td><a href="https://github.com/arcee-ai/mergekit">arcee-ai/mergekit</a></td><td>0</td><td>0</td><td>0</td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel">linkedin/Liger-Kernel</a></td><td>0</td><td>175</td><td>0</td></tr><tr><td><a href="https://github.com/Blaizzy/mlx-vlm">Blaizzy/mlx-vlm</a></td><td>0</td><td>0</td><td>0</td></tr><tr><td><a href="https://github.com/EvolvingLMMs-Lab/lmms-eval">EvolvingLMMs-Lab/lmms-eval</a></td><td>0</td><td>2</td><td>0</td></tr><tr><td><a href="https://github.com/urchade/GLiNER">urchade/GLiNER</a></td><td>0</td><td>0</td><td>0</td></tr><tr><td><a href="https://github.com/vllm-project/llm-compressor">vllm-project/llm-compressor</a></td><td>0</td><td>4</td><td>0</td></tr><tr><td><a href="https://github.com/allenai/open-instruct">allenai/open-instruct</a></td><td>0</td><td>0</td><td>0</td></tr><tr><td><a href="https://github.com/huggingface/optimum">huggingface/optimum</a></td><td>0</td><td>0</td><td>0</td></tr><tr><td><a href="https://github.com/embeddings-benchmark/mteb">embeddings-benchmark/mteb</a></td><td>0</td><td>0</td><td>0</td></tr><tr><td><a href="https://github.com/alibaba/ROLL">alibaba/ROLL</a></td><td>0</td><td>7</td><td>0</td></tr><tr><td><a href="https://github.com/PrimeIntellect-ai/prime-rl">PrimeIntellect-ai/prime-rl</a></td><td>0</td><td>7</td><td>0</td></tr><tr><td><a href="https://github.com/intel/auto-round">intel/auto-round</a></td><td>0</td><td>2</td><td>0</td></tr><tr><td><a href="https://github.com/ModelCloud/GPTQModel">ModelCloud/GPTQModel</a></td><td>0</td><td>15</td><td>0</td></tr><tr><td><a href="https://github.com/huggingface/optimum-quanto">huggingface/optimum-quanto</a></td><td>0</td><td>0</td><td>0</td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo">unslothai/unsloth-zoo</a></td><td>0</td><td>66</td><td>0</td></tr></table>
<details><summary>All 659 patches</summary>
<table><tr><th>where</th><th>patch</th></tr><tr><td><a href="https://github.com/EvolvingLMMs-Lab/lmms-eval/blob/HEAD/lmms_eval/models/simple/cambrians_vsc.py">EvolvingLMMs-Lab/lmms-eval/lmms_eval/models/simple/cambrians_vsc.py</a></td><td><code>CambrianQwenModel.forward = cambrian_qwen2_forward</code></td></tr><tr><td><a href="https://github.com/EvolvingLMMs-Lab/lmms-eval/blob/HEAD/lmms_eval/models/simple/cambrians_vsr.py">EvolvingLMMs-Lab/lmms-eval/lmms_eval/models/simple/cambrians_vsr.py</a></td><td><code>CambrianQwenModel.forward = cambrian_qwen2_forward</code></td></tr><tr><td><a href="https://github.com/FlagOpen/FlagEmbedding/blob/HEAD/research/llm_embedder/src/utils/llama_patch.py">FlagOpen/FlagEmbedding/research/llm_embedder/src/utils/llama_patch.py</a></td><td><code>transformers.models.llama.modeling_llama.LlamaAttention.forward = forward</code></td></tr><tr><td><a href="https://github.com/FlagOpen/FlagEmbedding/blob/HEAD/research/llm_embedder/src/utils/llama_patch.py">FlagOpen/FlagEmbedding/research/llm_embedder/src/utils/llama_patch.py</a></td><td><code>transformers.models.llama.modeling_llama.LlamaModel._prepare_decoder_attention_mask = (</code></td></tr><tr><td><a href="https://github.com/ModelCloud/GPTQModel/blob/HEAD/gptqmodel/models/base.py">ModelCloud/GPTQModel/gptqmodel/models/base.py</a></td><td><code>modeling_utils.check_support_param_buffer_assignment = check_support_param_buffer_assignment</code></td></tr><tr><td><a href="https://github.com/ModelCloud/GPTQModel/blob/HEAD/gptqmodel/models/definitions/intern_s2_preview.py">ModelCloud/GPTQModel/gptqmodel/models/definitions/intern_s2_preview.py</a></td><td><code>modeling_module.create_causal_mask = create_causal_mask_compat</code></td></tr><tr><td><a href="https://github.com/ModelCloud/GPTQModel/blob/HEAD/gptqmodel/utils/hf.py">ModelCloud/GPTQModel/gptqmodel/utils/hf.py</a></td><td><code>PreTrainedModel.__getattr__ = __getattr__</code></td></tr><tr><td><a href="https://github.com/ModelCloud/GPTQModel/blob/HEAD/gptqmodel/utils/hf.py">ModelCloud/GPTQModel/gptqmodel/utils/hf.py</a></td><td><code>PreTrainedModel._gptqmodel_legacy_tied_weights_patch = True</code></td></tr><tr><td><a href="https://github.com/ModelCloud/GPTQModel/blob/HEAD/gptqmodel/utils/hf.py">ModelCloud/GPTQModel/gptqmodel/utils/hf.py</a></td><td><code>PreTrainedModel._gptqmodel_missing_all_tied_weights_patch = True</code></td></tr><tr><td><a href="https://github.com/ModelCloud/GPTQModel/blob/HEAD/gptqmodel/utils/hf.py">ModelCloud/GPTQModel/gptqmodel/utils/hf.py</a></td><td><code>PreTrainedModel.get_expanded_tied_weights_keys = get_expanded_tied_weights_keys</code></td></tr><tr><td><a href="https://github.com/ModelCloud/GPTQModel/blob/HEAD/gptqmodel/utils/hf.py">ModelCloud/GPTQModel/gptqmodel/utils/hf.py</a></td><td><code>PreTrainedModel.get_head_mask = get_head_mask</code></td></tr><tr><td><a href="https://github.com/ModelCloud/GPTQModel/blob/HEAD/gptqmodel/utils/hf.py">ModelCloud/GPTQModel/gptqmodel/utils/hf.py</a></td><td><code>PreTrainedModel.is_parallelizable = False</code></td></tr><tr><td><a href="https://github.com/ModelCloud/GPTQModel/blob/HEAD/gptqmodel/utils/hf.py">ModelCloud/GPTQModel/gptqmodel/utils/hf.py</a></td><td><code>modeling_auto.AutoModelForCausalLM.register = classmethod(patched_register)</code></td></tr><tr><td><a href="https://github.com/ModelCloud/GPTQModel/blob/HEAD/gptqmodel/utils/hf.py">ModelCloud/GPTQModel/gptqmodel/utils/hf.py</a></td><td><code>modeling_llama.LlamaAttention = DeepSeekVLV2CompatLlamaAttention</code></td></tr><tr><td><a href="https://github.com/ModelCloud/GPTQModel/blob/HEAD/gptqmodel/utils/hf.py">ModelCloud/GPTQModel/gptqmodel/utils/hf.py</a></td><td><code>modeling_llama.LlamaFlashAttention2 = DeepSeekVLV2CompatLlamaAttention</code></td></tr><tr><td><a href="https://github.com/ModelCloud/GPTQModel/blob/HEAD/gptqmodel/utils/hf.py">ModelCloud/GPTQModel/gptqmodel/utils/hf.py</a></td><td><code>modeling_llama.LlamaFlashAttention2 = original_llama_attention</code></td></tr><tr><td><a href="https://github.com/ModelCloud/GPTQModel/blob/HEAD/gptqmodel/utils/hf.py">ModelCloud/GPTQModel/gptqmodel/utils/hf.py</a></td><td><code>modeling_utils._init_weights = False</code></td></tr><tr><td><a href="https://github.com/ModelCloud/GPTQModel/blob/HEAD/gptqmodel/utils/hf.py">ModelCloud/GPTQModel/gptqmodel/utils/hf.py</a></td><td><code>modeling_utils._init_weights = original_init_flag</code></td></tr><tr><td><a href="https://github.com/ModelCloud/GPTQModel/blob/HEAD/gptqmodel/utils/hf.py">ModelCloud/GPTQModel/gptqmodel/utils/hf.py</a></td><td><code>modeling_utils.is_flash_attn_2_available = is_flash_attn_2_available</code></td></tr><tr><td><a href="https://github.com/NVIDIA-NeMo/Automodel/blob/HEAD/nemo_automodel/_transformers/auto_model.py">NVIDIA-NeMo/Automodel/nemo_automodel/_transformers/auto_model.py</a></td><td><code>PreTrainedModel._finalize_model_loading = staticmethod(_compat_finalize)</code></td></tr><tr><td><a href="https://github.com/NVIDIA-NeMo/Automodel/blob/HEAD/nemo_automodel/_transformers/model_init.py">NVIDIA-NeMo/Automodel/nemo_automodel/_transformers/model_init.py</a></td><td><code>PreTrainedModel.get_init_context = classmethod(_patched_get_init_context)</code></td></tr><tr><td><a href="https://github.com/NVIDIA-NeMo/Automodel/blob/HEAD/nemo_automodel/_transformers/utils.py">NVIDIA-NeMo/Automodel/nemo_automodel/_transformers/utils.py</a></td><td><code>Qwen3ForCausalLM.forward = _patched_forward # type: ignore[method-assign]</code></td></tr><tr><td><a href="https://github.com/NVIDIA-NeMo/Automodel/blob/HEAD/nemo_automodel/components/models/beam/model.py">NVIDIA-NeMo/Automodel/nemo_automodel/components/models/beam/model.py</a></td><td><code>BeamForCausalLM.parallelizer = PARALLELIZER</code></td></tr><tr><td><a href="https://github.com/NVIDIA-NeMo/Automodel/blob/HEAD/nemo_automodel/components/models/common/utils.py">NVIDIA-NeMo/Automodel/nemo_automodel/components/models/common/utils.py</a></td><td><code>TERMSNorm.forward = _patched_rmsnorm_forward</code></td></tr><tr><td><a href="https://github.com/NVIDIA-NeMo/Automodel/blob/HEAD/nemo_automodel/components/models/deepseek_v4/model.py">NVIDIA-NeMo/Automodel/nemo_automodel/components/models/deepseek_v4/model.py</a></td><td><code>DeepseekV4ForCausalLM.parallelizer = PARALLELIZER</code></td></tr><tr><td><a href="https://github.com/NVIDIA-NeMo/Automodel/blob/HEAD/nemo_automodel/components/models/deepseek_v41/model.py">NVIDIA-NeMo/Automodel/nemo_automodel/components/models/deepseek_v41/model.py</a></td><td><code>DeepseekV41ForCausalLM.parallelizer = PARALLELIZER</code></td></tr><tr><td><a href="https://github.com/NVIDIA-NeMo/Automodel/blob/HEAD/nemo_automodel/components/models/nemotron_v3/model.py">NVIDIA-NeMo/Automodel/nemo_automodel/components/models/nemotron_v3/model.py</a></td><td><code>NemotronHForCausalLM.parallelizer = PARALLELIZER</code></td></tr><tr><td><a href="https://github.com/NVIDIA-NeMo/Automodel/blob/HEAD/nemo_automodel/components/models/qwen3_5/model.py">NVIDIA-NeMo/Automodel/nemo_automodel/components/models/qwen3_5/model.py</a></td><td><code>Qwen3_5ForCausalLM.parallelizer = PARALLELIZER</code></td></tr><tr><td><a href="https://github.com/NVIDIA-NeMo/Automodel/blob/HEAD/nemo_automodel/shared/transformers_patches.py">NVIDIA-NeMo/Automodel/nemo_automodel/shared/transformers_patches.py</a></td><td><code>modeling_t5.T5LayerNorm = _NativeT5LayerNorm</code></td></tr><tr><td><a href="https://github.com/NVIDIA/TensorRT-LLM/blob/HEAD/tensorrt_llm/tools/layer_wise_benchmarks/mark_utils.py">NVIDIA/TensorRT-LLM/tensorrt_llm/tools/layer_wise_benchmarks/mark_utils.py</a></td><td><code>GatedMLP.forward = nvtx.annotate("GatedMLP")(GatedMLP.forward)</code></td></tr><tr><td><a href="https://github.com/NVIDIA/TensorRT-LLM/blob/HEAD/tensorrt_llm/tools/layer_wise_benchmarks/mark_utils.py">NVIDIA/TensorRT-LLM/tensorrt_llm/tools/layer_wise_benchmarks/mark_utils.py</a></td><td><code>KimiKDALinearAttention.forward = nvtx.annotate("KimiKDALinearAttention")(</code></td></tr><tr><td><a href="https://github.com/OpenRLHF/OpenRLHF/blob/HEAD/openrlhf/utils/deepspeed/deepspeed.py">OpenRLHF/OpenRLHF/openrlhf/utils/deepspeed/deepspeed.py</a></td><td><code>transformers.modeling_flash_attention_utils.deterministic_g = True</code></td></tr><tr><td><a href="https://github.com/PrimeIntellect-ai/prime-rl/blob/HEAD/src/prime_rl/inference/patches.py">PrimeIntellect-ai/prime-rl/src/prime_rl/inference/patches.py</a></td><td><code>Glm5NextForCausalLM.process_weights_after_loading = process_weights_after_loading</code></td></tr><tr><td><a href="https://github.com/PrimeIntellect-ai/prime-rl/blob/HEAD/src/prime_rl/inference/patches.py">PrimeIntellect-ai/prime-rl/src/prime_rl/inference/patches.py</a></td><td><code>LoRAModel.from_local_checkpoint = classmethod(_patched_from_local_checkpoint)</code></td></tr><tr><td><a href="https://github.com/PrimeIntellect-ai/prime-rl/blob/HEAD/src/prime_rl/inference/patches.py">PrimeIntellect-ai/prime-rl/src/prime_rl/inference/patches.py</a></td><td><code>MiniMaxM2ForCausalLM.hf_to_vllm_mapper = WeightsMapper(</code></td></tr><tr><td><a href="https://github.com/PrimeIntellect-ai/prime-rl/blob/HEAD/src/prime_rl/inference/vllm/qwen38_weight_loading.py">PrimeIntellect-ai/prime-rl/src/prime_rl/inference/vllm/qwen38_weight_loading.py</a></td><td><code>Qwen4ExpNGramEmbedding.load_weights = load_weights</code></td></tr><tr><td><a href="https://github.com/PrimeIntellect-ai/prime-rl/blob/HEAD/src/prime_rl/trainer/models/gpt_oss/attention.py">PrimeIntellect-ai/prime-rl/src/prime_rl/trainer/models/gpt_oss/attention.py</a></td><td><code>GptOssAttention.compute_attention = ulysses_attention</code></td></tr><tr><td><a href="https://github.com/PrimeIntellect-ai/prime-rl/blob/HEAD/src/prime_rl/trainer/models/layers/ulysses_attn.py">PrimeIntellect-ai/prime-rl/src/prime_rl/trainer/models/layers/ulysses_attn.py</a></td><td><code>AfmoeFlashAttention._compute_attention = _ulysses_compute_attention</code></td></tr><tr><td><a href="https://github.com/PrimeIntellect-ai/prime-rl/blob/HEAD/src/prime_rl/trainer/models/layers/ulysses_attn.py">PrimeIntellect-ai/prime-rl/src/prime_rl/trainer/models/layers/ulysses_attn.py</a></td><td><code>FlashAttention._compute_attention = _ulysses_compute_attention</code></td></tr><tr><td><a href="https://github.com/alibaba/ROLL/blob/HEAD/roll/models/model_providers.py">alibaba/ROLL/roll/models/model_providers.py</a></td><td><code>AutoModelForCausalLMWithValueHead.post_init = no_set_device_hook_post_init</code></td></tr><tr><td><a href="https://github.com/alibaba/ROLL/blob/HEAD/roll/third_party/megatron/mtp_patcher.py">alibaba/ROLL/roll/third_party/megatron/mtp_patcher.py</a></td><td><code>GPTModel._original_postprocess = original_gpt_postprocess</code></td></tr><tr><td><a href="https://github.com/alibaba/ROLL/blob/HEAD/roll/third_party/megatron/mtp_patcher.py">alibaba/ROLL/roll/third_party/megatron/mtp_patcher.py</a></td><td><code>GPTModel._postprocess = patched_gpt_postprocess</code></td></tr><tr><td><a href="https://github.com/alibaba/ROLL/blob/HEAD/roll/third_party/vllm/gdn_patcher.py">alibaba/ROLL/roll/third_party/vllm/gdn_patcher.py</a></td><td><code>GDNAttentionMetadataBuilder._original_build = original_build</code></td></tr><tr><td><a href="https://github.com/alibaba/ROLL/blob/HEAD/roll/third_party/vllm/gdn_patcher.py">alibaba/ROLL/roll/third_party/vllm/gdn_patcher.py</a></td><td><code>GDNAttentionMetadataBuilder.build = patched_build</code></td></tr><tr><td><a href="https://github.com/alibaba/ROLL/blob/HEAD/roll/utils/context_parallel/monkey_patch.py">alibaba/ROLL/roll/utils/context_parallel/monkey_patch.py</a></td><td><code>Qwen2Model._update_causal_mask = _update_causal_mask</code></td></tr><tr><td><a href="https://github.com/alibaba/ROLL/blob/HEAD/roll/utils/context_parallel/monkey_patch.py">alibaba/ROLL/roll/utils/context_parallel/monkey_patch.py</a></td><td><code>Qwen2Model._update_causal_mask = old_update_causal_mask</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/integrations/expert_parallel/experts_fn.py">axolotl-ai-cloud/axolotl/src/axolotl/integrations/expert_parallel/experts_fn.py</a></td><td><code>PreTrainedModel.get_correct_experts_implementation = patched # type: ignore[assignment]</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/integrations/kernels/libs/scattermoe_lora/experts.py">axolotl-ai-cloud/axolotl/src/axolotl/integrations/kernels/libs/scattermoe_lora/experts.py</a></td><td><code>PreTrainedModel.get_correct_experts_implementation = _patched_get_correct</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/integrations/kernels/libs/scattermoe_lora/nvfp4_moe_loading.py">axolotl-ai-cloud/axolotl/src/axolotl/integrations/kernels/libs/scattermoe_lora/nvfp4_moe_loading.py</a></td><td><code>PreTrainedModel._initialize_missing_keys = patched</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/integrations/liger/models/glm4_moe_lite.py">axolotl-ai-cloud/axolotl/src/axolotl/integrations/liger/models/glm4_moe_lite.py</a></td><td><code>modeling_glm4_moe_lite.Glm4MoeLiteMLP.forward = LigerSwiGLUMLP.forward</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/integrations/liger/models/glm4_moe_lite.py">axolotl-ai-cloud/axolotl/src/axolotl/integrations/liger/models/glm4_moe_lite.py</a></td><td><code>modeling_glm4_moe_lite.Glm4MoeLiteRMSNorm = LigerRMSNorm</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/integrations/liger/models/glm4_moe_lite.py">axolotl-ai-cloud/axolotl/src/axolotl/integrations/liger/models/glm4_moe_lite.py</a></td><td><code>modeling_glm4_moe_lite.apply_rotary_pos_emb = liger_rotary_pos_emb</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/integrations/liger/models/glm4_moe_lite.py">axolotl-ai-cloud/axolotl/src/axolotl/integrations/liger/models/glm4_moe_lite.py</a></td><td><code>modeling_glm4_moe_lite.apply_rotary_pos_emb_interleave = (</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/integrations/liger/models/qwen3_5.py">axolotl-ai-cloud/axolotl/src/axolotl/integrations/liger/models/qwen3_5.py</a></td><td><code>modeling_qwen3_5.Qwen3_5ForCausalLM.forward = lce_forward</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/integrations/liger/models/qwen3_5.py">axolotl-ai-cloud/axolotl/src/axolotl/integrations/liger/models/qwen3_5.py</a></td><td><code>modeling_qwen3_5.Qwen3_5MLP = _liger_swiglu_mlp_wrapper</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/integrations/liger/models/qwen3_5.py">axolotl-ai-cloud/axolotl/src/axolotl/integrations/liger/models/qwen3_5.py</a></td><td><code>modeling_qwen3_5.Qwen3_5RMSNorm = LigerRMSNormForQwen3_5</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/integrations/liger/models/qwen3_5.py">axolotl-ai-cloud/axolotl/src/axolotl/integrations/liger/models/qwen3_5.py</a></td><td><code>modeling_qwen3_5.Qwen3_5RMSNormGated = FusedRMSNormGated</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/integrations/liger/models/qwen3_5.py">axolotl-ai-cloud/axolotl/src/axolotl/integrations/liger/models/qwen3_5.py</a></td><td><code>modeling_qwen3_5.nn.LayerNorm = LigerLayerNorm</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/integrations/liger/models/qwen3_5_moe.py">axolotl-ai-cloud/axolotl/src/axolotl/integrations/liger/models/qwen3_5_moe.py</a></td><td><code>modeling_mod.Qwen3_5MoeForCausalLM.forward = lce_forward</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/integrations/liger/models/qwen3_5_moe.py">axolotl-ai-cloud/axolotl/src/axolotl/integrations/liger/models/qwen3_5_moe.py</a></td><td><code>modeling_mod.Qwen3_5MoeMLP = _liger_swiglu_mlp_wrapper</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/integrations/liger/models/qwen3_5_moe.py">axolotl-ai-cloud/axolotl/src/axolotl/integrations/liger/models/qwen3_5_moe.py</a></td><td><code>modeling_mod.Qwen3_5MoeRMSNorm = LigerRMSNormForQwen3_5Moe</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/integrations/liger/models/qwen3_5_moe.py">axolotl-ai-cloud/axolotl/src/axolotl/integrations/liger/models/qwen3_5_moe.py</a></td><td><code>modeling_mod.Qwen3_5MoeRMSNormGated = FusedRMSNormGated</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/integrations/liger/models/qwen3_5_moe.py">axolotl-ai-cloud/axolotl/src/axolotl/integrations/liger/models/qwen3_5_moe.py</a></td><td><code>modeling_mod.nn.LayerNorm = LigerLayerNorm</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/integrations/liger/plugin.py">axolotl-ai-cloud/axolotl/src/axolotl/integrations/liger/plugin.py</a></td><td><code>modeling_cohere_compass.CohereCompassMLP = LigerSwiGLUMLP</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/integrations/liger/plugin.py">axolotl-ai-cloud/axolotl/src/axolotl/integrations/liger/plugin.py</a></td><td><code>modeling_cohere_compass.nn.LayerNorm = LigerLayerNorm</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/integrations/liger/plugin.py">axolotl-ai-cloud/axolotl/src/axolotl/integrations/liger/plugin.py</a></td><td><code>modeling_gemma4_unified.Gemma4UnifiedRMSNorm = (</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/integrations/liger/plugin.py">axolotl-ai-cloud/axolotl/src/axolotl/integrations/liger/plugin.py</a></td><td><code>modeling_gemma4_unified.Gemma4UnifiedTextMLP = _LigerGemma4UnifiedMLP</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/integrations/liger/plugin.py">axolotl-ai-cloud/axolotl/src/axolotl/integrations/liger/plugin.py</a></td><td><code>modeling_gemma4_unified.nn.CrossEntropyLoss = LigerCrossEntropyLoss</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/integrations/liger/plugin.py">axolotl-ai-cloud/axolotl/src/axolotl/integrations/liger/plugin.py</a></td><td><code>modeling_gemma4_unified.nn.LayerNorm = LigerLayerNorm</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/integrations/liger/plugin.py">axolotl-ai-cloud/axolotl/src/axolotl/integrations/liger/plugin.py</a></td><td><code>modeling_jamba.JambaForCausalLM.forward = jamba_lce_forward</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/integrations/liger/plugin.py">axolotl-ai-cloud/axolotl/src/axolotl/integrations/liger/plugin.py</a></td><td><code>modeling_jamba.JambaMLP = LigerSwiGLUMLP</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/integrations/liger/plugin.py">axolotl-ai-cloud/axolotl/src/axolotl/integrations/liger/plugin.py</a></td><td><code>modeling_jamba.JambaRMSNorm = LigerRMSNorm</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/integrations/liger/plugin.py">axolotl-ai-cloud/axolotl/src/axolotl/integrations/liger/plugin.py</a></td><td><code>modeling_jamba.apply_rotary_pos_emb = liger_rotary_pos_emb</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/integrations/liger/plugin.py">axolotl-ai-cloud/axolotl/src/axolotl/integrations/liger/plugin.py</a></td><td><code>modeling_jamba.nn.LayerNorm = LigerLayerNorm</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/integrations/liger/plugin.py">axolotl-ai-cloud/axolotl/src/axolotl/integrations/liger/plugin.py</a></td><td><code>modeling_ministral3.Ministral3MLP = LigerSwiGLUMLP</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/integrations/liger/plugin.py">axolotl-ai-cloud/axolotl/src/axolotl/integrations/liger/plugin.py</a></td><td><code>modeling_ministral3.Ministral3RMSNorm = LigerRMSNorm</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/integrations/liger/plugin.py">axolotl-ai-cloud/axolotl/src/axolotl/integrations/liger/plugin.py</a></td><td><code>modeling_ministral3.apply_rotary_pos_emb = liger_rotary_pos_emb</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/loaders/adapter.py">axolotl-ai-cloud/axolotl/src/axolotl/loaders/adapter.py</a></td><td><code>LoraModel._axolotl_clippable_patched = True</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/loaders/adapter.py">axolotl-ai-cloud/axolotl/src/axolotl/loaders/adapter.py</a></td><td><code>LoraModel._create_and_replace = _patched</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/loaders/model.py">axolotl-ai-cloud/axolotl/src/axolotl/loaders/model.py</a></td><td><code>transformers.integrations.deepspeed.is_deepspeed_zero3_enabled = lambda: (</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/loaders/model.py">axolotl-ai-cloud/axolotl/src/axolotl/loaders/model.py</a></td><td><code>transformers.modeling_utils.is_deepspeed_zero3_enabled = lambda: True</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/monkeypatch/accelerate/fsdp2.py">axolotl-ai-cloud/axolotl/src/axolotl/monkeypatch/accelerate/fsdp2.py</a></td><td><code>PreTrainedModel._adjust_tied_keys_with_tied_pointers = (</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/monkeypatch/accelerate/fsdp2.py">axolotl-ai-cloud/axolotl/src/axolotl/monkeypatch/accelerate/fsdp2.py</a></td><td><code>PreTrainedModel._initialize_missing_keys = _patched_initialize_missing_keys</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/monkeypatch/accelerate/fsdp2.py">axolotl-ai-cloud/axolotl/src/axolotl/monkeypatch/accelerate/fsdp2.py</a></td><td><code>PreTrainedModel._move_missing_keys_from_meta_to_device = _patched_move_missing_keys</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/monkeypatch/attention/flex_attn.py">axolotl-ai-cloud/axolotl/src/axolotl/monkeypatch/attention/flex_attn.py</a></td><td><code>transformers.integrations.flex_attention.WrappedFlexAttention = WrappedFlexAttention</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/monkeypatch/btlm_attn_hijack_flash.py">axolotl-ai-cloud/axolotl/src/axolotl/monkeypatch/btlm_attn_hijack_flash.py</a></td><td><code>modeling_btlm.BTLMAttention._attn = flashattn_attn</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/monkeypatch/llama_attn_hijack_flash.py">axolotl-ai-cloud/axolotl/src/axolotl/monkeypatch/llama_attn_hijack_flash.py</a></td><td><code>transformers.loss.loss_utils.fixed_cross_entropy = fa2_fixed_cross_entropy</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/monkeypatch/llama_attn_hijack_xformers.py">axolotl-ai-cloud/axolotl/src/axolotl/monkeypatch/llama_attn_hijack_xformers.py</a></td><td><code>transformers.models.llama.modeling_llama.LlamaAttention.forward = xformers_forward</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/monkeypatch/loss/chunked.py">axolotl-ai-cloud/axolotl/src/axolotl/monkeypatch/loss/chunked.py</a></td><td><code>transformers.loss.loss_utils.ForCausalLMLoss = for_causal_lm_chunked_loss</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/monkeypatch/mistral_attn_hijack_flash.py">axolotl-ai-cloud/axolotl/src/axolotl/monkeypatch/mistral_attn_hijack_flash.py</a></td><td><code>transformers.models.mistral.modeling_mistral.CrossEntropyLoss = partial(</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/monkeypatch/mixtral/__init__.py">axolotl-ai-cloud/axolotl/src/axolotl/monkeypatch/mixtral/__init__.py</a></td><td><code>MixtralBlockSparseTop2MLP.forward = mlp_forward</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/monkeypatch/models/falcon_h1/modeling.py">axolotl-ai-cloud/axolotl/src/axolotl/monkeypatch/models/falcon_h1/modeling.py</a></td><td><code>FalconH1DecoderLayer.forward = patched_decoder_forward</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/monkeypatch/models/gemma4/fused_attn.py">axolotl-ai-cloud/axolotl/src/axolotl/monkeypatch/models/gemma4/fused_attn.py</a></td><td><code>Gemma4TextAttention.forward = _make_fused_forward(original_forward)</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/monkeypatch/models/gemma4/fused_attn.py">axolotl-ai-cloud/axolotl/src/axolotl/monkeypatch/models/gemma4/fused_attn.py</a></td><td><code>Gemma4TextDecoderLayer.__call__ = patched_call</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/monkeypatch/models/gemma4/fused_attn.py">axolotl-ai-cloud/axolotl/src/axolotl/monkeypatch/models/gemma4/fused_attn.py</a></td><td><code>Gemma4TextDecoderLayer._axolotl_shared_kv_patched = True</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/monkeypatch/models/gemma4_unified/fused_attn.py">axolotl-ai-cloud/axolotl/src/axolotl/monkeypatch/models/gemma4_unified/fused_attn.py</a></td><td><code>Gemma4UnifiedTextAttention._axolotl_fused_attn_patched = True</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/monkeypatch/models/gemma4_unified/fused_attn.py">axolotl-ai-cloud/axolotl/src/axolotl/monkeypatch/models/gemma4_unified/fused_attn.py</a></td><td><code>Gemma4UnifiedTextAttention.forward = _make_fused_forward(original_forward)</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/monkeypatch/models/gemma4_unified/fused_attn.py">axolotl-ai-cloud/axolotl/src/axolotl/monkeypatch/models/gemma4_unified/fused_attn.py</a></td><td><code>Gemma4UnifiedTextDecoderLayer.__call__ = patched_call</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/monkeypatch/models/gemma4_unified/fused_attn.py">axolotl-ai-cloud/axolotl/src/axolotl/monkeypatch/models/gemma4_unified/fused_attn.py</a></td><td><code>Gemma4UnifiedTextDecoderLayer._axolotl_shared_kv_patched = True</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/monkeypatch/models/glm4_moe_lite/fused_attn.py">axolotl-ai-cloud/axolotl/src/axolotl/monkeypatch/models/glm4_moe_lite/fused_attn.py</a></td><td><code>Glm4MoeLiteAttention._axolotl_fused_attn_patched = True</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/monkeypatch/models/glm4_moe_lite/fused_attn.py">axolotl-ai-cloud/axolotl/src/axolotl/monkeypatch/models/glm4_moe_lite/fused_attn.py</a></td><td><code>Glm4MoeLiteAttention.forward = _make_fused_forward(Glm4MoeLiteAttention.forward)</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/monkeypatch/models/llama4/modeling.py">axolotl-ai-cloud/axolotl/src/axolotl/monkeypatch/models/llama4/modeling.py</a></td><td><code>modeling_llama4.Llama4TextExperts = Llama4TextExperts</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/monkeypatch/models/llama4/modeling.py">axolotl-ai-cloud/axolotl/src/axolotl/monkeypatch/models/llama4/modeling.py</a></td><td><code>modeling_llama4.Llama4TextExperts = old_lamma_4_text_experts</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/monkeypatch/models/pixtral/modeling_flash_attention_utils.py">axolotl-ai-cloud/axolotl/src/axolotl/monkeypatch/models/pixtral/modeling_flash_attention_utils.py</a></td><td><code>modeling_flash_attention_utils._is_packed_sequence = fixed_is_packed_sequence</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/monkeypatch/models/pixtral/modeling_flash_attention_utils.py">axolotl-ai-cloud/axolotl/src/axolotl/monkeypatch/models/pixtral/modeling_flash_attention_utils.py</a></td><td><code>modeling_flash_attention_utils._is_packed_sequence = old_fn</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/monkeypatch/models/qwen3/fused_attn.py">axolotl-ai-cloud/axolotl/src/axolotl/monkeypatch/models/qwen3/fused_attn.py</a></td><td><code>Qwen3Attention._axolotl_fused_attn_patched = True</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/monkeypatch/models/qwen3/fused_attn.py">axolotl-ai-cloud/axolotl/src/axolotl/monkeypatch/models/qwen3/fused_attn.py</a></td><td><code>Qwen3Attention.forward = _make_fused_forward()</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/monkeypatch/models/qwen3_5/fused_attn.py">axolotl-ai-cloud/axolotl/src/axolotl/monkeypatch/models/qwen3_5/fused_attn.py</a></td><td><code>Qwen3_5Attention._axolotl_fused_attn_patched = True</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/monkeypatch/models/qwen3_5/fused_attn.py">axolotl-ai-cloud/axolotl/src/axolotl/monkeypatch/models/qwen3_5/fused_attn.py</a></td><td><code>Qwen3_5Attention.forward = _make_fused_forward()</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/monkeypatch/models/qwen3_5_moe/fused_attn.py">axolotl-ai-cloud/axolotl/src/axolotl/monkeypatch/models/qwen3_5_moe/fused_attn.py</a></td><td><code>Qwen3_5MoeAttention._axolotl_fused_attn_patched = True</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/monkeypatch/models/qwen3_5_moe/fused_attn.py">axolotl-ai-cloud/axolotl/src/axolotl/monkeypatch/models/qwen3_5_moe/fused_attn.py</a></td><td><code>Qwen3_5MoeAttention.forward = _make_fused_forward()</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/monkeypatch/models/qwen3_moe/fused_attn.py">axolotl-ai-cloud/axolotl/src/axolotl/monkeypatch/models/qwen3_moe/fused_attn.py</a></td><td><code>Qwen3MoeAttention._axolotl_fused_attn_patched = True</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/monkeypatch/models/qwen3_moe/fused_attn.py">axolotl-ai-cloud/axolotl/src/axolotl/monkeypatch/models/qwen3_moe/fused_attn.py</a></td><td><code>Qwen3MoeAttention.forward = _make_fused_forward()</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/monkeypatch/models/qwen3_next/modeling.py">axolotl-ai-cloud/axolotl/src/axolotl/monkeypatch/models/qwen3_next/modeling.py</a></td><td><code>Qwen3NextDecoderLayer.forward = original_decoder_forward</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/monkeypatch/models/qwen3_next/modeling.py">axolotl-ai-cloud/axolotl/src/axolotl/monkeypatch/models/qwen3_next/modeling.py</a></td><td><code>Qwen3NextDecoderLayer.forward = patched_decoder_forward</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/monkeypatch/models/qwen3_vl/fused_attn.py">axolotl-ai-cloud/axolotl/src/axolotl/monkeypatch/models/qwen3_vl/fused_attn.py</a></td><td><code>Qwen3VLTextAttention._axolotl_fused_attn_patched = True</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/monkeypatch/models/qwen3_vl/fused_attn.py">axolotl-ai-cloud/axolotl/src/axolotl/monkeypatch/models/qwen3_vl/fused_attn.py</a></td><td><code>Qwen3VLTextAttention.forward = _make_fused_forward()</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/monkeypatch/moe_quant.py">axolotl-ai-cloud/axolotl/src/axolotl/monkeypatch/moe_quant.py</a></td><td><code>transformers.core_model_loading.set_param_for_module = _patched_set_param_for_module</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/monkeypatch/moe_quant.py">axolotl-ai-cloud/axolotl/src/axolotl/monkeypatch/moe_quant.py</a></td><td><code>transformers.modeling_utils.caching_allocator_warmup = _noop_warmup</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/monkeypatch/multipack.py">axolotl-ai-cloud/axolotl/src/axolotl/monkeypatch/multipack.py</a></td><td><code>modeling_arch._get_unpad_data = get_unpad_data</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/monkeypatch/peft/nf4.py">axolotl-ai-cloud/axolotl/src/axolotl/monkeypatch/peft/nf4.py</a></td><td><code>LoraModel.merge_and_unload = merge_and_unload</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/monkeypatch/quantized_init.py">axolotl-ai-cloud/axolotl/src/axolotl/monkeypatch/quantized_init.py</a></td><td><code>PreTrainedModel._initialize_weights = _initialize_weights</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/monkeypatch/stablelm_attn_hijack_flash.py">axolotl-ai-cloud/axolotl/src/axolotl/monkeypatch/stablelm_attn_hijack_flash.py</a></td><td><code>modeling_stablelm.Attention.forward = flashattn_attn</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/monkeypatch/stablelm_attn_hijack_flash.py">axolotl-ai-cloud/axolotl/src/axolotl/monkeypatch/stablelm_attn_hijack_flash.py</a></td><td><code>modeling_stablelm.DecoderLayer.forward = decoder_layer_forward</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/monkeypatch/stablelm_attn_hijack_flash.py">axolotl-ai-cloud/axolotl/src/axolotl/monkeypatch/stablelm_attn_hijack_flash.py</a></td><td><code>modeling_stablelm.StableLMEpochModel.forward = stablelm_model_forward</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/monkeypatch/transformers_fa_utils.py">axolotl-ai-cloud/axolotl/src/axolotl/monkeypatch/transformers_fa_utils.py</a></td><td><code>transformers.modeling_flash_attention_utils.fa_peft_integration_check = partial(</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/monkeypatch/transformers_num_parameters.py">axolotl-ai-cloud/axolotl/src/axolotl/monkeypatch/transformers_num_parameters.py</a></td><td><code>PreTrainedModel.resize_token_embeddings = resize_token_embeddings # type: ignore[method-assign]</code></td></tr><tr><td><a href="https://github.com/deepspeedai/DeepSpeed/blob/HEAD/deepspeed/inference/engine.py">deepspeedai/DeepSpeed/deepspeed/inference/engine.py</a></td><td><code>DeepSpeedSelfAttention.num_layers = 0</code></td></tr><tr><td><a href="https://github.com/deepspeedai/DeepSpeed/blob/HEAD/deepspeed/linear/context_manager.py">deepspeedai/DeepSpeed/deepspeed/linear/context_manager.py</a></td><td><code>transformers.AutoModelForCausalLM.from_config = from_config</code></td></tr><tr><td><a href="https://github.com/deepspeedai/DeepSpeed/blob/HEAD/deepspeed/linear/context_manager.py">deepspeedai/DeepSpeed/deepspeed/linear/context_manager.py</a></td><td><code>transformers.AutoModelForCausalLM.from_config = self._orig_causallm_config</code></td></tr><tr><td><a href="https://github.com/deepspeedai/DeepSpeed/blob/HEAD/deepspeed/linear/context_manager.py">deepspeedai/DeepSpeed/deepspeed/linear/context_manager.py</a></td><td><code>transformers.AutoModelForCausalLM.from_pretrained = from_pretrained</code></td></tr><tr><td><a href="https://github.com/deepspeedai/DeepSpeed/blob/HEAD/deepspeed/linear/context_manager.py">deepspeedai/DeepSpeed/deepspeed/linear/context_manager.py</a></td><td><code>transformers.AutoModelForCausalLM.from_pretrained = self._orig_causallm_pretrained</code></td></tr><tr><td><a href="https://github.com/deepspeedai/DeepSpeed/blob/HEAD/deepspeed/ops/transformer/inference/ds_attention.py">deepspeedai/DeepSpeed/deepspeed/ops/transformer/inference/ds_attention.py</a></td><td><code>DeepSpeedSelfAttention._qkv_buffers = [</code></td></tr><tr><td><a href="https://github.com/deepspeedai/DeepSpeed/blob/HEAD/deepspeed/ops/transformer/inference/ds_attention.py">deepspeedai/DeepSpeed/deepspeed/ops/transformer/inference/ds_attention.py</a></td><td><code>DeepSpeedSelfAttention.num_layers = DeepSpeedSelfAttention.num_layers + 1</code></td></tr><tr><td><a href="https://github.com/deepspeedai/DeepSpeed/blob/HEAD/deepspeed/ops/transformer/inference/ds_mlp.py">deepspeedai/DeepSpeed/deepspeed/ops/transformer/inference/ds_mlp.py</a></td><td><code>DeepSpeedMLP._inter_w_buffers = [</code></td></tr><tr><td><a href="https://github.com/deepspeedai/DeepSpeed/blob/HEAD/deepspeed/ops/transformer/inference/triton/attention.py">deepspeedai/DeepSpeed/deepspeed/ops/transformer/inference/triton/attention.py</a></td><td><code>TritonSelfAttention.num_layers = TritonSelfAttention.num_layers + 1</code></td></tr><tr><td><a href="https://github.com/deepspeedai/DeepSpeed/blob/HEAD/deepspeed/runtime/sequence_parallel/ulysses_sp.py">deepspeedai/DeepSpeed/deepspeed/runtime/sequence_parallel/ulysses_sp.py</a></td><td><code>modeling_llama.LlamaMLP.forward = mlp_forward_new</code></td></tr><tr><td><a href="https://github.com/deepspeedai/DeepSpeed/blob/HEAD/deepspeed/runtime/sequence_parallel/ulysses_sp.py">deepspeedai/DeepSpeed/deepspeed/runtime/sequence_parallel/ulysses_sp.py</a></td><td><code>modeling_llama.LlamaMLP.forward = tiled_mlp_forward</code></td></tr><tr><td><a href="https://github.com/hiyouga/LlamaFactory/blob/HEAD/src/llamafactory/extras/misc.py">hiyouga/LlamaFactory/src/llamafactory/extras/misc.py</a></td><td><code>transformers.dynamic_module_utils.check_imports = get_relative_imports</code></td></tr><tr><td><a href="https://github.com/hiyouga/LlamaFactory/blob/HEAD/src/llamafactory/model/model_utils/longlora.py">hiyouga/LlamaFactory/src/llamafactory/model/model_utils/longlora.py</a></td><td><code>LlamaAttention.forward = llama_attention_forward</code></td></tr><tr><td><a href="https://github.com/hiyouga/LlamaFactory/blob/HEAD/src/llamafactory/model/model_utils/longlora.py">hiyouga/LlamaFactory/src/llamafactory/model/model_utils/longlora.py</a></td><td><code>LlamaFlashAttention2.forward = llama_flash_attention_2_forward</code></td></tr><tr><td><a href="https://github.com/hiyouga/LlamaFactory/blob/HEAD/src/llamafactory/model/model_utils/longlora.py">hiyouga/LlamaFactory/src/llamafactory/model/model_utils/longlora.py</a></td><td><code>LlamaSdpaAttention.forward = llama_sdpa_attention_forward</code></td></tr><tr><td><a href="https://github.com/hiyouga/LlamaFactory/blob/HEAD/src/llamafactory/model/model_utils/visual.py">hiyouga/LlamaFactory/src/llamafactory/model/model_utils/visual.py</a></td><td><code>transformers.models.llava.modeling_llava.LlavaMultiModalProjector = LlavaMultiModalProjectorForYiVL</code></td></tr><tr><td><a href="https://github.com/hiyouga/LlamaFactory/blob/HEAD/src/llamafactory/model/patcher.py">hiyouga/LlamaFactory/src/llamafactory/model/patcher.py</a></td><td><code>Qwen3_5DecoderLayer.forward = _patched_decoder_forward</code></td></tr><tr><td><a href="https://github.com/hiyouga/LlamaFactory/blob/HEAD/src/llamafactory/model/patcher.py">hiyouga/LlamaFactory/src/llamafactory/model/patcher.py</a></td><td><code>Qwen3_5MoeDecoderLayer.forward = _patched_decoder_forward</code></td></tr><tr><td><a href="https://github.com/hiyouga/LlamaFactory/blob/HEAD/src/llamafactory/model/patcher.py">hiyouga/LlamaFactory/src/llamafactory/model/patcher.py</a></td><td><code>modeling_qwen3_omni_moe.Qwen3OmniMoeThinkerTextSparseMoeBlock = Qwen3OmniMoeThinkerTextSparseMoeBlock</code></td></tr><tr><td><a href="https://github.com/hpcaitech/ColossalAI/blob/HEAD/colossalai/lazy/pretrained.py">hpcaitech/ColossalAI/colossalai/lazy/pretrained.py</a></td><td><code>PreTrainedModel.from_pretrained = classmethod(PretrainedManager.old_from_pretrained)</code></td></tr><tr><td><a href="https://github.com/hpcaitech/ColossalAI/blob/HEAD/colossalai/lazy/pretrained.py">hpcaitech/ColossalAI/colossalai/lazy/pretrained.py</a></td><td><code>PreTrainedModel.from_pretrained = new_from_pretrained</code></td></tr><tr><td><a href="https://github.com/hpcaitech/ColossalAI/blob/HEAD/colossalai/legacy/inference/quant/smoothquant/models/base_model.py">hpcaitech/ColossalAI/colossalai/legacy/inference/quant/smoothquant/models/base_model.py</a></td><td><code>transformers.modeling_utils._init_weights = False</code></td></tr><tr><td><a href="https://github.com/hpcaitech/ColossalAI/blob/HEAD/colossalai/shardformer/layer/attn.py">hpcaitech/ColossalAI/colossalai/shardformer/layer/attn.py</a></td><td><code>ColoAttention._flash_kernel_dispatch = ColoAttention._flash_kernel_dispatch.load()</code></td></tr><tr><td><a href="https://github.com/hpcaitech/ColossalAI/blob/HEAD/colossalai/shardformer/layer/attn.py">hpcaitech/ColossalAI/colossalai/shardformer/layer/attn.py</a></td><td><code>ColoAttention._flash_kernel_dispatch = FlashAttentionDaoLoader()</code></td></tr><tr><td><a href="https://github.com/hpcaitech/ColossalAI/blob/HEAD/colossalai/shardformer/layer/attn.py">hpcaitech/ColossalAI/colossalai/shardformer/layer/attn.py</a></td><td><code>ColoAttention._kernel_dispatch_map = {</code></td></tr><tr><td><a href="https://github.com/hpcaitech/ColossalAI/blob/HEAD/colossalai/shardformer/layer/attn.py">hpcaitech/ColossalAI/colossalai/shardformer/layer/attn.py</a></td><td><code>RingAttention.ATTN_DONE = torch.cuda.Event()</code></td></tr><tr><td><a href="https://github.com/hpcaitech/ColossalAI/blob/HEAD/colossalai/shardformer/layer/attn.py">hpcaitech/ColossalAI/colossalai/shardformer/layer/attn.py</a></td><td><code>RingAttention.CU_SEQLENS = cu_seqlens</code></td></tr><tr><td><a href="https://github.com/hpcaitech/ColossalAI/blob/HEAD/colossalai/shardformer/layer/attn.py">hpcaitech/ColossalAI/colossalai/shardformer/layer/attn.py</a></td><td><code>RingAttention.HALF_INDICES = (half_idx_front, half_idx_back)</code></td></tr><tr><td><a href="https://github.com/hpcaitech/ColossalAI/blob/HEAD/colossalai/shardformer/layer/attn.py">hpcaitech/ColossalAI/colossalai/shardformer/layer/attn.py</a></td><td><code>RingAttention.INNER_RING_GROUP = inner_ring_group</code></td></tr><tr><td><a href="https://github.com/hpcaitech/ColossalAI/blob/HEAD/colossalai/shardformer/layer/attn.py">hpcaitech/ColossalAI/colossalai/shardformer/layer/attn.py</a></td><td><code>RingAttention.INTER_RING_GROUP = inter_ring_group</code></td></tr><tr><td><a href="https://github.com/hpcaitech/ColossalAI/blob/HEAD/colossalai/shardformer/layer/attn.py">hpcaitech/ColossalAI/colossalai/shardformer/layer/attn.py</a></td><td><code>RingAttention.SP_GROUP = sp_group</code></td></tr><tr><td><a href="https://github.com/hpcaitech/ColossalAI/blob/HEAD/colossalai/shardformer/layer/attn.py">hpcaitech/ColossalAI/colossalai/shardformer/layer/attn.py</a></td><td><code>RingAttention.SP_STREAM = torch.cuda.Stream()</code></td></tr><tr><td><a href="https://github.com/hpcaitech/ColossalAI/blob/HEAD/colossalai/shardformer/layer/attn.py">hpcaitech/ColossalAI/colossalai/shardformer/layer/attn.py</a></td><td><code>RingAttention.TOTAL_SEQLEN = b * sq</code></td></tr><tr><td><a href="https://github.com/huggingface/diffusers/blob/HEAD/examples/research_projects/pytorch_xla/inference/flux/flux_inference.py">huggingface/diffusers/examples/research_projects/pytorch_xla/inference/flux/flux_inference.py</a></td><td><code>FlashAttention.DEFAULT_BLOCK_SIZES = {</code></td></tr><tr><td><a href="https://github.com/huggingface/diffusers/blob/HEAD/examples/research_projects/pytorch_xla/inference/flux/flux_inference_spmd.py">huggingface/diffusers/examples/research_projects/pytorch_xla/inference/flux/flux_inference_spmd.py</a></td><td><code>FlashAttention.DEFAULT_BLOCK_SIZES = {</code></td></tr><tr><td><a href="https://github.com/huggingface/diffusers/blob/HEAD/src/diffusers/pipelines/prx/__init__.py">huggingface/diffusers/src/diffusers/pipelines/prx/__init__.py</a></td><td><code>transformers.T5GemmaEncoder = T5GemmaEncoder</code></td></tr><tr><td><a href="https://github.com/huggingface/trl/blob/HEAD/trl/_compat.py">huggingface/trl/trl/_compat.py</a></td><td><code>transformers.cache_utils.HybridCache = Cache</code></td></tr><tr><td><a href="https://github.com/intel/auto-round/blob/HEAD/auto_round/modeling/unfused_moe/__init__.py">intel/auto-round/auto_round/modeling/unfused_moe/__init__.py</a></td><td><code>transformers.modeling_utils.get_checkpoint_conversion_mapping = get_checkpoint_conversion_mapping_ar</code></td></tr><tr><td><a href="https://github.com/intel/auto-round/blob/HEAD/auto_round/utils/common.py">intel/auto-round/auto_round/utils/common.py</a></td><td><code>transformers.modeling_utils.no_init_weights = no_init_weights</code></td></tr><tr><td><a href="https://github.com/kvcache-ai/ktransformers/blob/HEAD/archive/kt-sft/ktransformers/operators/dynamic_attention.py">kvcache-ai/ktransformers/archive/kt-sft/ktransformers/operators/dynamic_attention.py</a></td><td><code>DynamicScaledDotProductAttention.cpu_infer = CPUInfer(threads_num)</code></td></tr><tr><td><a href="https://github.com/kvcache-ai/ktransformers/blob/HEAD/archive/kt-sft/ktransformers/operators/models.py">kvcache-ai/ktransformers/archive/kt-sft/ktransformers/operators/models.py</a></td><td><code>KLlamaModel.dynamic_sdpa = DynamicScaledDotProductAttention(</code></td></tr><tr><td><a href="https://github.com/kvcache-ai/ktransformers/blob/HEAD/archive/kt-sft/ktransformers/sft/metrics_utils/misc.py">kvcache-ai/ktransformers/archive/kt-sft/ktransformers/sft/metrics_utils/misc.py</a></td><td><code>transformers.dynamic_module_utils.check_imports = get_relative_imports</code></td></tr><tr><td><a href="https://github.com/kvcache-ai/ktransformers/blob/HEAD/archive/ktransformers/operators/dynamic_attention.py">kvcache-ai/ktransformers/archive/ktransformers/operators/dynamic_attention.py</a></td><td><code>DynamicScaledDotProductAttention.cpu_infer = CPUInfer(threads_num)</code></td></tr><tr><td><a href="https://github.com/kvcache-ai/ktransformers/blob/HEAD/archive/ktransformers/operators/models.py">kvcache-ai/ktransformers/archive/ktransformers/operators/models.py</a></td><td><code>KLlamaModel.dynamic_sdpa = DynamicScaledDotProductAttention(</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/llama4_rope.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/llama4_rope.py</a></td><td><code>modeling_module.apply_rotary_emb = liger_llama4_text_rotary_pos_emb</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/llama4_rope.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/llama4_rope.py</a></td><td><code>modeling_module.vision_apply_rotary_emb = liger_llama4_vision_rotary_pos_emb</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>Gemma4TextForCausalLM.forward = causal_forward</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_deepseek_v2.DeepseekV2Experts = LigerExperts</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_deepseek_v2.DeepseekV2ForCausalLM.forward = deepseek_v2_lce_forward</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_deepseek_v2.DeepseekV2MLP = LigerQwen3MoeSwiGLUMLP</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_deepseek_v2.DeepseekV2RMSNorm = LigerRMSNorm</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_deepseek_v3.DeepseekV3Experts = LigerExperts</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_deepseek_v3.DeepseekV3ForCausalLM.forward = deepseek_v3_lce_forward</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_deepseek_v3.DeepseekV3MLP = LigerQwen3MoeSwiGLUMLP</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_deepseek_v3.DeepseekV3RMSNorm = LigerRMSNorm</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_deepseek_v4.DeepseekV4ForCausalLM.forward = deepseek_v4_lce_forward</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_deepseek_v4.DeepseekV4RMSNorm = LigerRMSNorm</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_exaone4.Exaone4ForCausalLM.forward = exaone4_lce_forward</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_exaone4.Exaone4MLP = LigerSwiGLUMLP</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_exaone4.Exaone4RMSNorm = Exaone4LigerRMSNorm</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_exaone4.apply_rotary_pos_emb = liger_rotary_pos_emb</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_falcon_h1.FalconH1ForCausalLM.forward = falcon_h1_lce_forward</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_falcon_h1.FalconH1MLP = LigerFalconH1SwiGLUMLP</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_falcon_h1.FalconH1RMSNorm = LigerRMSNorm</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_falcon_h1.apply_rotary_pos_emb = liger_rotary_pos_emb</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_gemma.GemmaForCausalLM.forward = gemma_lce_forward</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_gemma.GemmaMLP = LigerGEGLUMLP</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_gemma.GemmaRMSNorm = LigerRMSNormForGemma</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_gemma.apply_rotary_pos_emb = liger_rotary_pos_emb</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_gemma2.Gemma2ForCausalLM.forward = gemma2_lce_forward</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_gemma2.Gemma2MLP = LigerGEGLUMLP</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_gemma2.Gemma2RMSNorm = LigerRMSNormForGemma2</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_gemma2.apply_rotary_pos_emb = liger_rotary_pos_emb</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_gemma3.Gemma3ForCausalLM.forward = causal_forward</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_gemma3.Gemma3ForConditionalGeneration.forward = multimodal_forward</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_gemma3.Gemma3MLP = LigerGEGLUMLP</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_gemma3.Gemma3RMSNorm = LigerRMSNormForGemma3</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_gemma3.apply_rotary_pos_emb = liger_rotary_pos_emb</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_gemma3.nn.CrossEntropyLoss = LigerCrossEntropyLoss</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_gemma4.Gemma4ForCausalLM.forward = causal_forward</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_gemma4.Gemma4ForConditionalGeneration.forward = multimodal_forward</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_gemma4.Gemma4RMSNorm = LigerRMSNormForGemma4</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_gemma4.Gemma4TextMLP = LigerGEGLUMLPForGemma4</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_glm4.Glm4ForCausalLM.forward = glm4_lce_forward</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_glm4.Glm4MLP = LigerPhi3SwiGLUMLP</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_glm4.Glm4RMSNorm = LigerRMSNormForGlm4</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_glm4v.Glm4vForConditionalGeneration.forward = glm4v_lce_forward</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_glm4v.Glm4vRMSNorm = LigerRMSNormForGlm4</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_glm4v_moe.Glm4vMoeForConditionalGeneration.forward = glm4v_moe_lce_forward</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_glm4v_moe.Glm4vMoeRMSNorm = LigerRMSNormForGlm4</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_glm4v_moe.Glm4vMoeTextNaiveMoe = LigerExperts</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_glm4v_moe.Glm4vMoeTextRMSNorm = LigerRMSNormForGlm4</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_gpt_oss.GptOssForCausalLM.forward = gpt_oss_lce_forward</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_gpt_oss.GptOssRMSNorm = LigerRMSNorm</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_gpt_oss.apply_rotary_pos_emb = liger_rotary_pos_emb_gpt_oss</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_granite.GraniteMLP = LigerSwiGLUMLP</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_granite.GraniteRMSNorm = LigerRMSNorm</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_granite.apply_rotary_pos_emb = liger_rotary_pos_emb</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_hunyuan_v1_dense.HunYuanDenseV1ForCausalLM.forward = hunyuan_v1_lce_forward</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_hunyuan_v1_dense.HunYuanDenseV1MLP = LigerHunyuanV1SwiGLUMLP</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_hunyuan_v1_dense.HunYuanDenseV1RMSNorm = LigerRMSNorm</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_hunyuan_v1_dense.apply_rotary_pos_emb = liger_rotary_pos_emb</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_hunyuan_v1_moe.HunYuanMoEV1Experts = LigerExperts</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_hunyuan_v1_moe.HunYuanMoEV1ForCausalLM.forward = hunyuan_v1_moe_lce_forward</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_hunyuan_v1_moe.HunYuanMoEV1RMSNorm = LigerRMSNorm</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_hunyuan_v1_moe.apply_rotary_pos_emb = liger_rotary_pos_emb</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_internvl.InternVLForConditionalGeneration.forward = internvl_lce_forward</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_internvl.InternVLVisionRMSNorm = LigerRMSNorm</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_internvl.nn.LayerNorm = LigerLayerNorm</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_llama.LlamaForCausalLM.forward = llama_lce_forward</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_llama.LlamaMLP = LigerSwiGLUMLP</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_llama.LlamaRMSNorm = LigerRMSNorm</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_llama.apply_rotary_pos_emb = liger_rotary_pos_emb</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_llama4.CrossEntropyLoss = LigerCrossEntropyLoss</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_llama4.Llama4ForCausalLM.forward = llama4_lce_forward</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_llama4.Llama4TextMLP = LigerSwiGLUMLP</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_llama4.Llama4TextRMSNorm = LigerRMSNorm</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_llava.LlavaForConditionalGeneration.forward = llava_lce_forward</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_ministral.CrossEntropyLoss = LigerCrossEntropyLoss</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_ministral.MinistralForCausalLM.forward = ministral_lce_forward</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_ministral.MinistralMLP = LigerSwiGLUMLP</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_ministral.MinistralRMSNorm = LigerRMSNorm</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_ministral.apply_rotary_pos_emb = liger_rotary_pos_emb</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_mistral.CrossEntropyLoss = LigerCrossEntropyLoss</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_mistral.MistralForCausalLM.forward = mistral_lce_forward</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_mistral.MistralMLP = LigerSwiGLUMLP</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_mistral.MistralRMSNorm = LigerRMSNorm</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_mistral.apply_rotary_pos_emb = liger_rotary_pos_emb</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_mixtral.MixtralExperts = LigerExperts</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_mixtral.MixtralForCausalLM.forward = mixtral_lce_forward</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_mixtral.MixtralRMSNorm = LigerRMSNorm</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_mixtral.apply_rotary_pos_emb = liger_rotary_pos_emb</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_mllama.MllamaForCausalLM.forward = mllama_lce_forward</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_mllama.MllamaTextMLP = LigerSwiGLUMLP</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_mllama.MllamaTextRMSNorm = LigerRMSNorm</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_mllama.apply_rotary_pos_emb = liger_rotary_pos_emb</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_mllama.nn.LayerNorm = LigerLayerNorm</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_muse_glimmer.MuseGlimmerForConditionalGeneration.forward = muse_glimmer_lce_forward</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_muse_glimmer.MuseGlimmerRMSNorm = LigerRMSNormForMuseGlimmer</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_muse_glimmer.MuseGlimmerTextCenteredRMSNorm = LigerRMSNormForMuseGlimmerTextCentered</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_muse_glimmer.MuseGlimmerTextMLP = LigerSwiGLUMLPForMuseGlimmer</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_muse_glimmer.apply_rotary_pos_emb = liger_rotary_pos_emb</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_nemotron.CrossEntropyLoss = LigerCrossEntropyLoss</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_nemotron.NemotronForCausalLM.forward = nemotron_lce_forward</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_olmo2.Olmo2ForCausalLM.forward = olmo2_lce_forward</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_olmo2.Olmo2MLP = LigerSwiGLUMLP</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_olmo2.Olmo2RMSNorm = LigerRMSNormForOlmo2</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_olmo2.apply_rotary_pos_emb = liger_rotary_pos_emb</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_olmo3.Olmo3ForCausalLM.forward = olmo3_lce_forward</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_olmo3.Olmo3MLP = LigerSwiGLUMLP</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_olmo3.Olmo3RMSNorm = LigerRMSNormForOlmo2 # same as olmo2</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_olmo3.apply_rotary_pos_emb = liger_rotary_pos_emb</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_paligemma.PaliGemmaForConditionalGeneration.forward = lce_forward</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_paligemma.nn.CrossEntropyLoss = LigerCrossEntropyLoss</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_phi3.Phi3ForCausalLM.forward = phi3_lce_forward</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_phi3.Phi3MLP = LigerPhi3SwiGLUMLP</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_phi3.Phi3RMSNorm = LigerRMSNorm # Same as Llama</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_phi3.apply_rotary_pos_emb = liger_rotary_pos_emb # Same as Gemma</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_pixtral.PixtralMLP = LigerSwiGLUMLP</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_pixtral.PixtralRMSNorm = LigerRMSNorm</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_pixtral.apply_rotary_pos_emb = liger_rotary_pos_emb</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_qwen2.Qwen2ForCausalLM.forward = qwen2_lce_forward</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_qwen2.Qwen2MLP = LigerSwiGLUMLP</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_qwen2.Qwen2RMSNorm = LigerRMSNorm</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_qwen2.apply_rotary_pos_emb = liger_rotary_pos_emb</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_qwen2_5_vl.CrossEntropyLoss = LigerCrossEntropyLoss</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_qwen2_5_vl.Qwen2MLP = LigerSwiGLUMLP</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_qwen2_5_vl.Qwen2RMSNorm = LigerRMSNorm</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_qwen2_5_vl.Qwen2_5_VLForConditionalGeneration.forward = qwen2_5_vl_lce_forward</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_qwen2_5_vl.Qwen2_5_VLRMSNorm = LigerRMSNorm</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_qwen2_5_vl.apply_multimodal_rotary_pos_emb = liger_multimodal_rotary_pos_emb</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_qwen2_vl.CrossEntropyLoss = LigerCrossEntropyLoss</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_qwen2_vl.LayerNorm = LigerLayerNorm</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_qwen2_vl.Qwen2MLP = LigerSwiGLUMLP</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_qwen2_vl.Qwen2RMSNorm = LigerRMSNorm</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_qwen2_vl.Qwen2VLForConditionalGeneration.forward = qwen2_vl_lce_forward</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_qwen2_vl.Qwen2VLRMSNorm = LigerRMSNorm</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_qwen2_vl.apply_multimodal_rotary_pos_emb = liger_multimodal_rotary_pos_emb</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_qwen3.Qwen3ForCausalLM.forward = qwen3_lce_forward</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_qwen3.Qwen3MLP = LigerSwiGLUMLP</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_qwen3.Qwen3RMSNorm = LigerRMSNorm</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_qwen3.apply_rotary_pos_emb = liger_rotary_pos_emb</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_qwen3_5.Qwen3_5ForCausalLM.forward = qwen3_5_lce_forward</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_qwen3_5.Qwen3_5ForConditionalGeneration.forward = qwen3_5_lce_forward_for_multimodal</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_qwen3_5.Qwen3_5MLP = LigerQwen3MoeSwiGLUMLP</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_qwen3_5.Qwen3_5RMSNorm = LigerRMSNormForQwen3Next</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_qwen3_5.apply_rotary_pos_emb = liger_rotary_pos_emb</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_qwen3_5.apply_rotary_pos_emb_vision = liger_rotary_pos_emb_vision</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_qwen3_5_moe.Qwen3_5MoeExperts = LigerExperts</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_qwen3_5_moe.Qwen3_5MoeForCausalLM.forward = qwen3_5_moe_lce_forward</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_qwen3_5_moe.Qwen3_5MoeForConditionalGeneration.forward = (</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_qwen3_5_moe.Qwen3_5MoeRMSNorm = LigerRMSNormForQwen3Next</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_qwen3_5_moe.apply_rotary_pos_emb = liger_rotary_pos_emb</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_qwen3_5_moe.apply_rotary_pos_emb_vision = liger_rotary_pos_emb_vision</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_qwen3_moe.Qwen3MoeExperts = LigerExperts</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_qwen3_moe.Qwen3MoeForCausalLM.forward = qwen3_lce_forward</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_qwen3_moe.Qwen3MoeRMSNorm = LigerRMSNorm</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_qwen3_moe.apply_rotary_pos_emb = liger_rotary_pos_emb</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_qwen3_next.Qwen3NextExperts = LigerExperts</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_qwen3_next.Qwen3NextForCausalLM.forward = qwen3_next_lce_forward</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_qwen3_next.Qwen3NextRMSNorm = LigerRMSNormForQwen3Next</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_qwen3_next.apply_rotary_pos_emb = liger_rotary_pos_emb</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_qwen3_vl.Qwen3VLForConditionalGeneration.forward = qwen3_vl_lce_forward</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_qwen3_vl.Qwen3VLTextRMSNorm = LigerRMSNorm</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_qwen3_vl.apply_rotary_pos_emb = liger_rotary_pos_emb</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_qwen3_vl.apply_rotary_pos_emb_vision = liger_rotary_pos_emb_vision</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_qwen3_vl_moe.Qwen3VLMoeForConditionalGeneration.forward = qwen3_vl_moe_lce_forward</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_qwen3_vl_moe.Qwen3VLMoeTextExperts = LigerExperts</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_qwen3_vl_moe.Qwen3VLMoeTextRMSNorm = LigerRMSNorm</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_qwen3_vl_moe.apply_rotary_pos_emb = liger_rotary_pos_emb</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_qwen3_vl_moe.apply_rotary_pos_emb_vision = liger_rotary_pos_emb_vision</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_siglip.nn.LayerNorm = LigerLayerNorm</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_smollm3.SmolLM3ForCausalLM.forward = smollm3_lce_forward</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_smollm3.SmolLM3MLP = LigerSwiGLUMLP</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_smollm3.SmolLM3RMSNorm = LigerRMSNorm</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_smollm3.apply_rotary_pos_emb = liger_rotary_pos_emb</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_smolvlm.SmolVLMForConditionalGeneration.forward = smolvlm_lce_forward</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_smolvlm.SmolVLMRMSNorm = LigerRMSNorm</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_smolvlm.nn.LayerNorm = LigerLayerNorm</code></td></tr><tr><td><a href="https://github.com/modelscope/ms-swift/blob/HEAD/swift/megatron/utils/convert_utils.py">modelscope/ms-swift/swift/megatron/utils/convert_utils.py</a></td><td><code>TEDotProductAttention.forward = forward</code></td></tr><tr><td><a href="https://github.com/modelscope/ms-swift/blob/HEAD/swift/megatron/utils/convert_utils.py">modelscope/ms-swift/swift/megatron/utils/convert_utils.py</a></td><td><code>TEDotProductAttention.forward = new_forward</code></td></tr><tr><td><a href="https://github.com/modelscope/ms-swift/blob/HEAD/swift/model/models/deepseek.py">modelscope/ms-swift/swift/model/models/deepseek.py</a></td><td><code>UnlimitedOCRModel._swift_multi_gpu_patched = True</code></td></tr><tr><td><a href="https://github.com/modelscope/ms-swift/blob/HEAD/swift/model/models/deepseek.py">modelscope/ms-swift/swift/model/models/deepseek.py</a></td><td><code>UnlimitedOCRModel.forward = _patched_forward</code></td></tr><tr><td><a href="https://github.com/modelscope/ms-swift/blob/HEAD/swift/model/models/deepseek.py">modelscope/ms-swift/swift/model/models/deepseek.py</a></td><td><code>modeling_module.torch.cat = _orig_cat</code></td></tr><tr><td><a href="https://github.com/modelscope/ms-swift/blob/HEAD/swift/model/models/deepseek.py">modelscope/ms-swift/swift/model/models/deepseek.py</a></td><td><code>modeling_module.torch.cat = _safe_cat</code></td></tr><tr><td><a href="https://github.com/modelscope/ms-swift/blob/HEAD/swift/model/models/deepseek.py">modelscope/ms-swift/swift/model/models/deepseek.py</a></td><td><code>transformers.models.llama.modeling_llama.LlamaFlashAttention2 = None</code></td></tr><tr><td><a href="https://github.com/modelscope/ms-swift/blob/HEAD/swift/model/models/internlm.py">modelscope/ms-swift/swift/model/models/internlm.py</a></td><td><code>PreTrainedModel._old_enable_input_require_grads = old_enable_input_require_grads</code></td></tr><tr><td><a href="https://github.com/modelscope/ms-swift/blob/HEAD/swift/model/models/internlm.py">modelscope/ms-swift/swift/model/models/internlm.py</a></td><td><code>PreTrainedModel.enable_input_require_grads = patched_enable_input_require_grads</code></td></tr><tr><td><a href="https://github.com/modelscope/ms-swift/blob/HEAD/swift/model/models/llava.py">modelscope/ms-swift/swift/model/models/llava.py</a></td><td><code>LlavaLlamaForCausalLM.__old_forward = forward</code></td></tr><tr><td><a href="https://github.com/modelscope/ms-swift/blob/HEAD/swift/model/models/llava.py">modelscope/ms-swift/swift/model/models/llava.py</a></td><td><code>LlavaLlamaForCausalLM.forward = _new_forward</code></td></tr><tr><td><a href="https://github.com/modelscope/ms-swift/blob/HEAD/swift/model/models/spark.py">modelscope/ms-swift/swift/model/models/spark.py</a></td><td><code>modeling_module._swift_patched = True</code></td></tr><tr><td><a href="https://github.com/modelscope/ms-swift/blob/HEAD/swift/model/models/spark.py">modelscope/ms-swift/swift/model/models/spark.py</a></td><td><code>setattr(modeling_module, name, Spark2_5Loader._compat_mask_fn(getattr(modeling_module, name)))</code></td></tr><tr><td><a href="https://github.com/modelscope/ms-swift/blob/HEAD/swift/model/models/valley.py">modelscope/ms-swift/swift/model/models/valley.py</a></td><td><code>ValleyQwen2ForCausalLM._origin_forward = forward</code></td></tr><tr><td><a href="https://github.com/modelscope/ms-swift/blob/HEAD/swift/model/models/valley.py">modelscope/ms-swift/swift/model/models/valley.py</a></td><td><code>ValleyQwen2ForCausalLM.forward = new_forward</code></td></tr><tr><td><a href="https://github.com/modelscope/ms-swift/blob/HEAD/swift/model/models/xing4_0.py">modelscope/ms-swift/swift/model/models/xing4_0.py</a></td><td><code>modeling_module.Xing4_0HyperConnection.forward = torch.compile(</code></td></tr><tr><td><a href="https://github.com/modelscope/ms-swift/blob/HEAD/swift/model/models/xing4_0.py">modelscope/ms-swift/swift/model/models/xing4_0.py</a></td><td><code>modeling_module.Xing4_0PreTrainedModel._can_set_experts_implementation_cached_value = True</code></td></tr><tr><td><a href="https://github.com/modelscope/ms-swift/blob/HEAD/swift/model/models/xing4_0.py">modelscope/ms-swift/swift/model/models/xing4_0.py</a></td><td><code>modeling_module._swift_patch_options = patch_options</code></td></tr><tr><td><a href="https://github.com/modelscope/ms-swift/blob/HEAD/swift/model/patcher.py">modelscope/ms-swift/swift/model/patcher.py</a></td><td><code>PreTrainedModel._get_no_split_modules = _get_no_split_modules</code></td></tr><tr><td><a href="https://github.com/modelscope/ms-swift/blob/HEAD/swift/model/patcher.py">modelscope/ms-swift/swift/model/patcher.py</a></td><td><code>PreTrainedModel._get_no_split_modules = _new_get_no_split_modules</code></td></tr><tr><td><a href="https://github.com/modelscope/ms-swift/blob/HEAD/swift/model/patcher.py">modelscope/ms-swift/swift/model/patcher.py</a></td><td><code>PreTrainedModel.from_pretrained = _new_from_pretrained</code></td></tr><tr><td><a href="https://github.com/modelscope/ms-swift/blob/HEAD/swift/model/patcher.py">modelscope/ms-swift/swift/model/patcher.py</a></td><td><code>PreTrainedModel.from_pretrained = classmethod(from_pretrained)</code></td></tr><tr><td><a href="https://github.com/modelscope/ms-swift/blob/HEAD/swift/model/patcher.py">modelscope/ms-swift/swift/model/patcher.py</a></td><td><code>transformers.modeling_utils.get_balanced_memory = lambda *args, **kwargs: {}</code></td></tr><tr><td><a href="https://github.com/modelscope/ms-swift/blob/HEAD/swift/model/patcher.py">modelscope/ms-swift/swift/model/patcher.py</a></td><td><code>transformers.modeling_utils.infer_auto_device_map = _infer_auto_device_map_patch</code></td></tr><tr><td><a href="https://github.com/modelscope/ms-swift/blob/HEAD/swift/sequence_parallel/sequence_parallel.py">modelscope/ms-swift/swift/sequence_parallel/sequence_parallel.py</a></td><td><code>modeling_flash_attention_utils._flash_attention_forward = flash_attention_forward</code></td></tr><tr><td><a href="https://github.com/modelscope/ms-swift/blob/HEAD/swift/sequence_parallel/sequence_parallel.py">modelscope/ms-swift/swift/sequence_parallel/sequence_parallel.py</a></td><td><code>modeling_flash_attention_utils._flash_attention_forward_origin = _flash_attention_forward</code></td></tr><tr><td><a href="https://github.com/modelscope/ms-swift/blob/HEAD/swift/template/base.py">modelscope/ms-swift/swift/template/base.py</a></td><td><code>modeling_module._flash_attention_forward = _flash_attention_forward</code></td></tr><tr><td><a href="https://github.com/modelscope/ms-swift/blob/HEAD/swift/template/base.py">modelscope/ms-swift/swift/template/base.py</a></td><td><code>modeling_module._flash_attention_forward = _origin_flash_attention_forward</code></td></tr><tr><td><a href="https://github.com/modelscope/ms-swift/blob/HEAD/swift/template/templates/glm.py">modelscope/ms-swift/swift/template/templates/glm.py</a></td><td><code>modeling_module.create_causal_mask = new_create_causal_mask</code></td></tr><tr><td><a href="https://github.com/modelscope/ms-swift/blob/HEAD/swift/trainers/mixin.py">modelscope/ms-swift/swift/trainers/mixin.py</a></td><td><code>transformers.modeling_utils.checkpoint = _new_checkpoint</code></td></tr><tr><td><a href="https://github.com/modelscope/ms-swift/blob/HEAD/swift/tuners/peft.py">modelscope/ms-swift/swift/tuners/peft.py</a></td><td><code>AdaLoraModel._create_and_replace = _create_and_replace_hook</code></td></tr><tr><td><a href="https://github.com/modelscope/ms-swift/blob/HEAD/swift/tuners/peft.py">modelscope/ms-swift/swift/tuners/peft.py</a></td><td><code>AdaLoraModel._create_and_replace_origin = AdaLoraModel._create_and_replace</code></td></tr><tr><td><a href="https://github.com/modelscope/ms-swift/blob/HEAD/swift/tuners/peft.py">modelscope/ms-swift/swift/tuners/peft.py</a></td><td><code>AdaLoraModel.forward = adalora_forward</code></td></tr><tr><td><a href="https://github.com/modelscope/ms-swift/blob/HEAD/swift/tuners/peft.py">modelscope/ms-swift/swift/tuners/peft.py</a></td><td><code>BOFTModel._create_and_replace = _create_and_replace_hook</code></td></tr><tr><td><a href="https://github.com/modelscope/ms-swift/blob/HEAD/swift/tuners/peft.py">modelscope/ms-swift/swift/tuners/peft.py</a></td><td><code>BOFTModel._create_and_replace_origin = BOFTModel._create_and_replace</code></td></tr><tr><td><a href="https://github.com/modelscope/ms-swift/blob/HEAD/swift/tuners/peft.py">modelscope/ms-swift/swift/tuners/peft.py</a></td><td><code>BoneModel._create_and_replace = _create_and_replace_hook</code></td></tr><tr><td><a href="https://github.com/modelscope/ms-swift/blob/HEAD/swift/tuners/peft.py">modelscope/ms-swift/swift/tuners/peft.py</a></td><td><code>BoneModel._create_and_replace_origin = BoneModel._create_and_replace</code></td></tr><tr><td><a href="https://github.com/modelscope/ms-swift/blob/HEAD/swift/tuners/peft.py">modelscope/ms-swift/swift/tuners/peft.py</a></td><td><code>FourierFTModel._create_and_replace = _create_and_replace_hook</code></td></tr><tr><td><a href="https://github.com/modelscope/ms-swift/blob/HEAD/swift/tuners/peft.py">modelscope/ms-swift/swift/tuners/peft.py</a></td><td><code>FourierFTModel._create_and_replace_origin = FourierFTModel._create_and_replace</code></td></tr><tr><td><a href="https://github.com/modelscope/ms-swift/blob/HEAD/swift/tuners/peft.py">modelscope/ms-swift/swift/tuners/peft.py</a></td><td><code>LoraModel.__init__ = __new_init__</code></td></tr><tr><td><a href="https://github.com/modelscope/ms-swift/blob/HEAD/swift/tuners/peft.py">modelscope/ms-swift/swift/tuners/peft.py</a></td><td><code>LoraModel.__init_origin__ = LoraModel.__init__</code></td></tr><tr><td><a href="https://github.com/modelscope/ms-swift/blob/HEAD/swift/tuners/peft.py">modelscope/ms-swift/swift/tuners/peft.py</a></td><td><code>LoraModel._cast_adapter_dtype = _cast_adapter_dtype_hook</code></td></tr><tr><td><a href="https://github.com/modelscope/ms-swift/blob/HEAD/swift/tuners/peft.py">modelscope/ms-swift/swift/tuners/peft.py</a></td><td><code>LoraModel._cast_adapter_dtype_origin = LoraModel._cast_adapter_dtype</code></td></tr><tr><td><a href="https://github.com/modelscope/ms-swift/blob/HEAD/swift/tuners/peft.py">modelscope/ms-swift/swift/tuners/peft.py</a></td><td><code>LoraModel._create_and_replace = _create_and_replace_hook</code></td></tr><tr><td><a href="https://github.com/modelscope/ms-swift/blob/HEAD/swift/tuners/peft.py">modelscope/ms-swift/swift/tuners/peft.py</a></td><td><code>LoraModel._create_and_replace_origin = LoraModel._create_and_replace</code></td></tr><tr><td><a href="https://github.com/modelscope/ms-swift/blob/HEAD/swift/tuners/peft.py">modelscope/ms-swift/swift/tuners/peft.py</a></td><td><code>PeftModel.activate_adapter = PeftModel.set_adapter</code></td></tr><tr><td><a href="https://github.com/modelscope/ms-swift/blob/HEAD/swift/tuners/peft.py">modelscope/ms-swift/swift/tuners/peft.py</a></td><td><code>PeftModel.create_optimizer_param_groups = create_optimizer_param_groups</code></td></tr><tr><td><a href="https://github.com/modelscope/ms-swift/blob/HEAD/swift/tuners/peft.py">modelscope/ms-swift/swift/tuners/peft.py</a></td><td><code>PeftModel.deactivate_adapter = partial(dummy_function, func='deactivate_adapter')</code></td></tr><tr><td><a href="https://github.com/modelscope/ms-swift/blob/HEAD/swift/tuners/peft.py">modelscope/ms-swift/swift/tuners/peft.py</a></td><td><code>PeftModel.load_adapter = load_adapter</code></td></tr><tr><td><a href="https://github.com/modelscope/ms-swift/blob/HEAD/swift/tuners/peft.py">modelscope/ms-swift/swift/tuners/peft.py</a></td><td><code>PeftModel.load_adapter_origin = PeftModel.load_adapter</code></td></tr><tr><td><a href="https://github.com/modelscope/ms-swift/blob/HEAD/swift/tuners/peft.py">modelscope/ms-swift/swift/tuners/peft.py</a></td><td><code>PeftModel.set_active_adapters = partial(dummy_function, func='set_active_adapters')</code></td></tr><tr><td><a href="https://github.com/modelscope/ms-swift/blob/HEAD/swift/tuners/peft.py">modelscope/ms-swift/swift/tuners/peft.py</a></td><td><code>VeraModel._create_and_replace = _create_and_replace_hook</code></td></tr><tr><td><a href="https://github.com/modelscope/ms-swift/blob/HEAD/swift/tuners/peft.py">modelscope/ms-swift/swift/tuners/peft.py</a></td><td><code>VeraModel._create_and_replace_origin = VeraModel._create_and_replace</code></td></tr><tr><td><a href="https://github.com/sgl-project/sglang/blob/HEAD/python/sglang/multimodal_gen/runtime/layers/attention/layer.py">sgl-project/sglang/python/sglang/multimodal_gen/runtime/layers/attention/layer.py</a></td><td><code>USPAttention._usp_a2a_stream = torch.get_device_module().Stream()</code></td></tr><tr><td><a href="https://github.com/sgl-project/sglang/blob/HEAD/python/sglang/multimodal_gen/runtime/loader/utils.py">sgl-project/sglang/python/sglang/multimodal_gen/runtime/loader/utils.py</a></td><td><code>PreTrainedModel.post_init = lambda self: None</code></td></tr><tr><td><a href="https://github.com/sgl-project/sglang/blob/HEAD/python/sglang/srt/multimodal/processors/qwen_vl.py">sgl-project/sglang/python/sglang/srt/multimodal/processors/qwen_vl.py</a></td><td><code>transformers.models.qwen2_vl.image_processing_qwen2_vl_fast.Qwen2VLImageProcessorFast._preprocess = fast_preprocess_cpu</code></td></tr><tr><td><a href="https://github.com/sgl-project/sglang/blob/HEAD/python/sglang/srt/utils/hf_transformers_patches.py">sgl-project/sglang/python/sglang/srt/utils/hf_transformers_patches.py</a></td><td><code>modeling_llama.LlamaFlashAttention2 = modeling_llama.LlamaAttention</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/blob/HEAD/unsloth_zoo/compiler.py">unslothai/unsloth-zoo/unsloth_zoo/compiler.py</a></td><td><code>modeling_file.__UNSLOTH_PATCHED__ = True</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/blob/HEAD/unsloth_zoo/compiler.py">unslothai/unsloth-zoo/unsloth_zoo/compiler.py</a></td><td><code>modeling_file.__UNSLOTH_SUPPORTS_SDPA__ = False</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/blob/HEAD/unsloth_zoo/compiler.py">unslothai/unsloth-zoo/unsloth_zoo/compiler.py</a></td><td><code>modeling_file.__UNSLOTH_SUPPORTS_SDPA__ = final_supports_sdpa</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/blob/HEAD/unsloth_zoo/compiler.py">unslothai/unsloth-zoo/unsloth_zoo/compiler.py</a></td><td><code>modeling_file.unsloth_mhc_projection = unsloth_mhc_projection</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/blob/HEAD/unsloth_zoo/compiler.py">unslothai/unsloth-zoo/unsloth_zoo/compiler.py</a></td><td><code>modeling_file.unsloth_sinkhorn_knopp = unsloth_sinkhorn_knopp</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/blob/HEAD/unsloth_zoo/compiler.py">unslothai/unsloth-zoo/unsloth_zoo/compiler.py</a></td><td><code>setattr(modeling_file, key, value)</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/blob/HEAD/unsloth_zoo/empty_model.py">unslothai/unsloth-zoo/unsloth_zoo/empty_model.py</a></td><td><code>BitsAndBytesModelLoader._stack_quantization_states = (</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/blob/HEAD/unsloth_zoo/empty_model.py">unslothai/unsloth-zoo/unsloth_zoo/empty_model.py</a></td><td><code>Gemma4ForCausalLM.get_expert_mapping = get_expert_mapping</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/blob/HEAD/unsloth_zoo/empty_model.py">unslothai/unsloth-zoo/unsloth_zoo/empty_model.py</a></td><td><code>SiglipVisionModel._init_weights = SiglipVisionModel._original_initialize_weights</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/blob/HEAD/unsloth_zoo/empty_model.py">unslothai/unsloth-zoo/unsloth_zoo/empty_model.py</a></td><td><code>SiglipVisionModel._init_weights = _init_weights</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/blob/HEAD/unsloth_zoo/empty_model.py">unslothai/unsloth-zoo/unsloth_zoo/empty_model.py</a></td><td><code>SiglipVisionModel._original_initialize_weights = SiglipVisionModel._init_weights</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/blob/HEAD/unsloth_zoo/gradient_checkpointing.py">unslothai/unsloth-zoo/unsloth_zoo/gradient_checkpointing.py</a></td><td><code>transformers.modeling_utils._unsloth_old_checkpoint = current</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/blob/HEAD/unsloth_zoo/gradient_checkpointing.py">unslothai/unsloth-zoo/unsloth_zoo/gradient_checkpointing.py</a></td><td><code>transformers.modeling_utils.checkpoint = (</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/blob/HEAD/unsloth_zoo/gradient_checkpointing.py">unslothai/unsloth-zoo/unsloth_zoo/gradient_checkpointing.py</a></td><td><code>transformers.modeling_utils.checkpoint = getattr(</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/blob/HEAD/unsloth_zoo/gradient_checkpointing.py">unslothai/unsloth-zoo/unsloth_zoo/gradient_checkpointing.py</a></td><td><code>transformers.modeling_utils.checkpoint = shim</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/blob/HEAD/unsloth_zoo/gradient_checkpointing.py">unslothai/unsloth-zoo/unsloth_zoo/gradient_checkpointing.py</a></td><td><code>transformers.modeling_utils.checkpoint = unsloth_checkpoint</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/blob/HEAD/unsloth_zoo/loss_utils.py">unslothai/unsloth-zoo/unsloth_zoo/loss_utils.py</a></td><td><code>UnslothForCausalLMLoss._unsloth_fused_loss = UnslothForCausalLMLoss</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/blob/HEAD/unsloth_zoo/loss_utils.py">unslothai/unsloth-zoo/unsloth_zoo/loss_utils.py</a></td><td><code>transformers.modeling_utils.PreTrainedModel.loss_function = \</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/blob/HEAD/unsloth_zoo/mlx/loader.py">unslothai/unsloth-zoo/unsloth_zoo/mlx/loader.py</a></td><td><code>modeling_llama.LlamaFlashAttention2 = modeling_llama.LlamaAttention</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/blob/HEAD/unsloth_zoo/temporary_patches/common.py">unslothai/unsloth-zoo/unsloth_zoo/temporary_patches/common.py</a></td><td><code>setattr(modeling_module, name, value)</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/blob/HEAD/unsloth_zoo/temporary_patches/compiled_model_identity.py">unslothai/unsloth-zoo/unsloth_zoo/temporary_patches/compiled_model_identity.py</a></td><td><code>PreTrainedModel.is_custom_code = method</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/blob/HEAD/unsloth_zoo/temporary_patches/deepseek_v3_moe.py">unslothai/unsloth-zoo/unsloth_zoo/temporary_patches/deepseek_v3_moe.py</a></td><td><code>DeepseekV3ForCausalLM.__UNSLOTH_SUPPORTS_RETURN_HIDDEN_STATES__ = True</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/blob/HEAD/unsloth_zoo/temporary_patches/deepseek_v3_moe.py">unslothai/unsloth-zoo/unsloth_zoo/temporary_patches/deepseek_v3_moe.py</a></td><td><code>DeepseekV3ForCausalLM.forward = _patched_causal_lm_forward</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/blob/HEAD/unsloth_zoo/temporary_patches/gemma3n.py">unslothai/unsloth-zoo/unsloth_zoo/temporary_patches/gemma3n.py</a></td><td><code>Gemma3nTextAttention.forward = forward</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/blob/HEAD/unsloth_zoo/temporary_patches/gemma4.py">unslothai/unsloth-zoo/unsloth_zoo/temporary_patches/gemma4.py</a></td><td><code>Gemma4AudioAttention.forward = forward</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/blob/HEAD/unsloth_zoo/temporary_patches/gemma4.py">unslothai/unsloth-zoo/unsloth_zoo/temporary_patches/gemma4.py</a></td><td><code>Gemma4TextAttention.forward = _make_gemma4_attention_carrier_forward(Gemma4TextAttention.forward)</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/blob/HEAD/unsloth_zoo/temporary_patches/gemma4.py">unslothai/unsloth-zoo/unsloth_zoo/temporary_patches/gemma4.py</a></td><td><code>LoraModel._create_and_replace = create_and_replace</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/blob/HEAD/unsloth_zoo/temporary_patches/gpt_oss.py">unslothai/unsloth-zoo/unsloth_zoo/temporary_patches/gpt_oss.py</a></td><td><code>GptOssForCausalLM._unsloth_grpo_patched = True</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/blob/HEAD/unsloth_zoo/temporary_patches/gpt_oss.py">unslothai/unsloth-zoo/unsloth_zoo/temporary_patches/gpt_oss.py</a></td><td><code>GptOssForCausalLM.forward = _patched_causal_lm_forward</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/blob/HEAD/unsloth_zoo/temporary_patches/gpt_oss.py">unslothai/unsloth-zoo/unsloth_zoo/temporary_patches/gpt_oss.py</a></td><td><code>GptOssModel.forward = forward</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/blob/HEAD/unsloth_zoo/temporary_patches/gpt_oss.py">unslothai/unsloth-zoo/unsloth_zoo/temporary_patches/gpt_oss.py</a></td><td><code>GptOssPreTrainedModel._unsloth_init_weights_fixed = True</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/blob/HEAD/unsloth_zoo/temporary_patches/gpt_oss.py">unslothai/unsloth-zoo/unsloth_zoo/temporary_patches/gpt_oss.py</a></td><td><code>transformers.generation.utils.create_masks_for_generate = wrap(transformers.generation.utils.create_masks_for_generate)</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/blob/HEAD/unsloth_zoo/temporary_patches/gpt_oss.py">unslothai/unsloth-zoo/unsloth_zoo/temporary_patches/gpt_oss.py</a></td><td><code>transformers.masking_utils.__patched_causal_mask__ = True</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/blob/HEAD/unsloth_zoo/temporary_patches/gpt_oss.py">unslothai/unsloth-zoo/unsloth_zoo/temporary_patches/gpt_oss.py</a></td><td><code>transformers.masking_utils._old_create_causal_mask = compile_mask_builder(transformers.masking_utils.create_causal_mask)</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/blob/HEAD/unsloth_zoo/temporary_patches/gpt_oss.py">unslothai/unsloth-zoo/unsloth_zoo/temporary_patches/gpt_oss.py</a></td><td><code>transformers.masking_utils._old_create_sliding_window_causal_mask = compile_mask_builder(transformers.masking_utils.create_sliding_window_causal_mask)</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/blob/HEAD/unsloth_zoo/temporary_patches/gpt_oss.py">unslothai/unsloth-zoo/unsloth_zoo/temporary_patches/gpt_oss.py</a></td><td><code>transformers.masking_utils.create_causal_mask = wrap(create_causal_mask)</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/blob/HEAD/unsloth_zoo/temporary_patches/gpt_oss.py">unslothai/unsloth-zoo/unsloth_zoo/temporary_patches/gpt_oss.py</a></td><td><code>transformers.masking_utils.create_masks_for_generate = wrap(transformers.masking_utils.create_masks_for_generate)</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/blob/HEAD/unsloth_zoo/temporary_patches/gpt_oss.py">unslothai/unsloth-zoo/unsloth_zoo/temporary_patches/gpt_oss.py</a></td><td><code>transformers.masking_utils.create_sliding_window_causal_mask = wrap(create_sliding_window_causal_mask)</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/blob/HEAD/unsloth_zoo/temporary_patches/gpt_oss.py">unslothai/unsloth-zoo/unsloth_zoo/temporary_patches/gpt_oss.py</a></td><td><code>transformers.models.gpt_oss.modeling_gpt_oss.GptOssExperts = GptOssExpertsBnb4bit</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/blob/HEAD/unsloth_zoo/temporary_patches/gpt_oss.py">unslothai/unsloth-zoo/unsloth_zoo/temporary_patches/gpt_oss.py</a></td><td><code>transformers.models.gpt_oss.modeling_gpt_oss.GptOssExperts = \</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/blob/HEAD/unsloth_zoo/temporary_patches/gpt_oss.py">unslothai/unsloth-zoo/unsloth_zoo/temporary_patches/gpt_oss.py</a></td><td><code>transformers.models.gpt_oss.modeling_gpt_oss.GptOssExperts.forward = experts_forward</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/blob/HEAD/unsloth_zoo/temporary_patches/gpt_oss.py">unslothai/unsloth-zoo/unsloth_zoo/temporary_patches/gpt_oss.py</a></td><td><code>transformers.models.gpt_oss.modeling_gpt_oss.GptOssTopKRouter = GptOssTopKRouter</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/blob/HEAD/unsloth_zoo/temporary_patches/gpt_oss.py">unslothai/unsloth-zoo/unsloth_zoo/temporary_patches/gpt_oss.py</a></td><td><code>transformers.models.gpt_oss.modeling_gpt_oss.GptOssTopKRouter = \</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/blob/HEAD/unsloth_zoo/temporary_patches/gpt_oss.py">unslothai/unsloth-zoo/unsloth_zoo/temporary_patches/gpt_oss.py</a></td><td><code>transformers.models.gpt_oss.modeling_gpt_oss._original_GptOssExperts = \</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/blob/HEAD/unsloth_zoo/temporary_patches/gpt_oss.py">unslothai/unsloth-zoo/unsloth_zoo/temporary_patches/gpt_oss.py</a></td><td><code>transformers.models.gpt_oss.modeling_gpt_oss._original_GptOssTopKRouter = \</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/blob/HEAD/unsloth_zoo/temporary_patches/gpt_oss.py">unslothai/unsloth-zoo/unsloth_zoo/temporary_patches/gpt_oss.py</a></td><td><code>transformers.models.gpt_oss.modeling_gpt_oss.create_causal_mask = transformers.masking_utils.create_causal_mask</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/blob/HEAD/unsloth_zoo/temporary_patches/gpt_oss.py">unslothai/unsloth-zoo/unsloth_zoo/temporary_patches/gpt_oss.py</a></td><td><code>transformers.models.gpt_oss.modeling_gpt_oss.create_sliding_window_causal_mask = transformers.masking_utils.create_sliding_window_causal_mask</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/blob/HEAD/unsloth_zoo/temporary_patches/gpt_oss.py">unslothai/unsloth-zoo/unsloth_zoo/temporary_patches/gpt_oss.py</a></td><td><code>transformers.quantizers.quantizer_mxfp4.Mxfp4HfQuantizer._lazy_import_kernels = lambda *args, **kwargs: triton_kernels</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/blob/HEAD/unsloth_zoo/temporary_patches/gpt_oss.py">unslothai/unsloth-zoo/unsloth_zoo/temporary_patches/gpt_oss.py</a></td><td><code>transformers.quantizers.quantizer_mxfp4.Mxfp4HfQuantizer.is_trainable = lambda *args, **kwargs: True</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/blob/HEAD/unsloth_zoo/temporary_patches/gpt_oss.py">unslothai/unsloth-zoo/unsloth_zoo/temporary_patches/gpt_oss.py</a></td><td><code>transformers.quantizers.quantizer_mxfp4.is_kernels_available = is_kernels_available</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/blob/HEAD/unsloth_zoo/temporary_patches/gpt_oss.py">unslothai/unsloth-zoo/unsloth_zoo/temporary_patches/gpt_oss.py</a></td><td><code>transformers.quantizers.quantizer_mxfp4.is_kernels_available = lambda *args, **kwargs: False</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/blob/HEAD/unsloth_zoo/temporary_patches/misc.py">unslothai/unsloth-zoo/unsloth_zoo/temporary_patches/misc.py</a></td><td><code>HiggsAudioV2Embeddings.forward = forward</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/blob/HEAD/unsloth_zoo/temporary_patches/misc.py">unslothai/unsloth-zoo/unsloth_zoo/temporary_patches/misc.py</a></td><td><code>ModernBertModel._update_attention_mask = _update_attention_mask_contiguous</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/blob/HEAD/unsloth_zoo/temporary_patches/misc.py">unslothai/unsloth-zoo/unsloth_zoo/temporary_patches/misc.py</a></td><td><code>transformers.models.mllama.modeling_mllama.MllamaVisionEncoderLayer = MllamaVisionEncoderLayer</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/blob/HEAD/unsloth_zoo/temporary_patches/misc.py">unslothai/unsloth-zoo/unsloth_zoo/temporary_patches/misc.py</a></td><td><code>transformers.utils.import_utils.is_causal_conv1d_available = lambda: False</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/blob/HEAD/unsloth_zoo/temporary_patches/moe_utils.py">unslothai/unsloth-zoo/unsloth_zoo/temporary_patches/moe_utils.py</a></td><td><code>PeftModel.load_adapter = patched</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/blob/HEAD/unsloth_zoo/temporary_patches/moe_utils.py">unslothai/unsloth-zoo/unsloth_zoo/temporary_patches/moe_utils.py</a></td><td><code>PeftModel.save_pretrained = patched</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/blob/HEAD/unsloth_zoo/temporary_patches/moe_utils_bnb4bit.py">unslothai/unsloth-zoo/unsloth_zoo/temporary_patches/moe_utils_bnb4bit.py</a></td><td><code>modeling_utils.get_model_conversion_mapping = patched_get_model_conversion_mapping</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/blob/HEAD/unsloth_zoo/temporary_patches/mxfp4.py">unslothai/unsloth-zoo/unsloth_zoo/temporary_patches/mxfp4.py</a></td><td><code>PreTrainedModel.save_pretrained = save_pretrained</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/blob/HEAD/unsloth_zoo/temporary_patches/mxfp4.py">unslothai/unsloth-zoo/unsloth_zoo/temporary_patches/mxfp4.py</a></td><td><code>modeling_utils._get_device_map = _get_device_map</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/blob/HEAD/unsloth_zoo/temporary_patches/mxfp4.py">unslothai/unsloth-zoo/unsloth_zoo/temporary_patches/mxfp4.py</a></td><td><code>transformers.integrations.mxfp4.convert_moe_packed_tensors_cpu = convert_moe_packed_tensors_cpu</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/blob/HEAD/unsloth_zoo/temporary_patches/qwen3_5_moe.py">unslothai/unsloth-zoo/unsloth_zoo/temporary_patches/qwen3_5_moe.py</a></td><td><code>transformers.models.qwen3_5_moe.modeling_qwen3_5_moe.Qwen3_5MoeExperts._unsloth_lora_extractor_fn = staticmethod(_qwen3_5_lora_extractor)</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/blob/HEAD/unsloth_zoo/temporary_patches/qwen3_moe.py">unslothai/unsloth-zoo/unsloth_zoo/temporary_patches/qwen3_moe.py</a></td><td><code>transformers.models.qwen3_moe.modeling_qwen3_moe.Qwen3MoeExperts._unsloth_lora_extractor_fn = staticmethod(_qwen3_lora_extractor)</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/blob/HEAD/unsloth_zoo/temporary_patches/qwen3_next_moe.py">unslothai/unsloth-zoo/unsloth_zoo/temporary_patches/qwen3_next_moe.py</a></td><td><code>transformers.models.qwen3_next.modeling_qwen3_next.Qwen3NextExperts._unsloth_lora_extractor_fn = staticmethod(_qwen3_next_lora_extractor)</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/blob/HEAD/unsloth_zoo/temporary_patches/qwen3_vl_moe.py">unslothai/unsloth-zoo/unsloth_zoo/temporary_patches/qwen3_vl_moe.py</a></td><td><code>transformers.models.qwen3_vl_moe.modeling_qwen3_vl_moe.Qwen3VLMoeTextExperts._unsloth_lora_extractor_fn = staticmethod(_qwen3_vl_lora_extractor)</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/blob/HEAD/unsloth_zoo/vllm_utils.py">unslothai/unsloth-zoo/unsloth_zoo/vllm_utils.py</a></td><td><code>Qwen3MoeForCausalLM.is_3d_moe_weight = True</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/studio/backend/core/inference/safetensors_pread.py">unslothai/unsloth/studio/backend/core/inference/safetensors_pread.py</a></td><td><code>modeling_utils.safe_open = _original</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/studio/backend/core/inference/safetensors_pread.py">unslothai/unsloth/studio/backend/core/inference/safetensors_pread.py</a></td><td><code>modeling_utils.safe_open = _with_pread(current)</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/import_fixes.py">unslothai/unsloth/unsloth/import_fixes.py</a></td><td><code>PreTrainedModel.disable_input_require_grads = _patched_single_disable_input_require_grads</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/import_fixes.py">unslothai/unsloth/unsloth/import_fixes.py</a></td><td><code>PreTrainedModel.enable_input_require_grads = _patched_enable_input_require_grads</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/import_fixes.py">unslothai/unsloth/unsloth/import_fixes.py</a></td><td><code>PreTrainedModel.enable_input_require_grads = _patched_single_enable_input_require_grads</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/import_fixes.py">unslothai/unsloth/unsloth/import_fixes.py</a></td><td><code>PreTrainedModel.post_init = post_init</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/import_fixes.py">unslothai/unsloth/unsloth/import_fixes.py</a></td><td><code>modeling_utils.revert_weight_conversion = revert_weight_conversion</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/kernels/rms_layernorm.py">unslothai/unsloth/unsloth/kernels/rms_layernorm.py</a></td><td><code>transformers.models.llama.modeling_llama.LlamaRMSNorm = LlamaRMSNorm</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/kernels/rms_layernorm.py">unslothai/unsloth/unsloth/kernels/rms_layernorm.py</a></td><td><code>transformers.models.llama.modeling_llama.LlamaRMSNorm = Unsloth_LlamaRMSNorm</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/kernels/rms_layernorm.py">unslothai/unsloth/unsloth/kernels/rms_layernorm.py</a></td><td><code>transformers.models.mllama.modeling_mllama.MllamaTextRMSNorm = MllamaTextRMSNorm</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/kernels/rms_layernorm.py">unslothai/unsloth/unsloth/kernels/rms_layernorm.py</a></td><td><code>transformers.models.mllama.modeling_mllama.MllamaTextRMSNorm = Unsloth_MllamaTextRMSNorm</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/_utils.py">unslothai/unsloth/unsloth/models/_utils.py</a></td><td><code>PreTrainedModel._get_key_renaming_mapping = _get_key_renaming_mapping</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/_utils.py">unslothai/unsloth/unsloth/models/_utils.py</a></td><td><code>PreTrainedModel._get_key_renaming_mapping = original</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/_utils.py">unslothai/unsloth/unsloth/models/_utils.py</a></td><td><code>modeling_utils.get_model_conversion_mapping = get_model_conversion_mapping</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/_utils.py">unslothai/unsloth/unsloth/models/_utils.py</a></td><td><code>transformers.integrations.bitsandbytes.should_convert_module = patched_should_convert_module</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/_utils.py">unslothai/unsloth/unsloth/models/_utils.py</a></td><td><code>transformers.trainer.get_model_param_count = get_model_param_count</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/_utils.py">unslothai/unsloth/unsloth/models/_utils.py</a></td><td><code>transformers.trainer_pt_utils.get_model_param_count = get_model_param_count</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/_utils.py">unslothai/unsloth/unsloth/models/_utils.py</a></td><td><code>transformers.utils.import_utils.is_flash_attn_2_available = (</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/_utils.py">unslothai/unsloth/unsloth/models/_utils.py</a></td><td><code>transformers.utils.import_utils.is_flash_attn_2_available = lambda *args, **kwargs: (</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/_utils.py">unslothai/unsloth/unsloth/models/_utils.py</a></td><td><code>transformers.utils.import_utils.is_flash_attn_2_available = lambda *args, **kwargs: False</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/_utils.py">unslothai/unsloth/unsloth/models/_utils.py</a></td><td><code>transformers.utils.is_flash_attn_2_available = lambda *args, **kwargs: False</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/_utils.py">unslothai/unsloth/unsloth/models/_utils.py</a></td><td><code>transformers.utils.is_openai_available = _lazy_openai_import_check(</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/_utils.py">unslothai/unsloth/unsloth/models/_utils.py</a></td><td><code>transformers.utils.quantization_config.BitsAndBytesConfig.__init__ = _BitsAndBytesConfig__init__</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/cohere.py">unslothai/unsloth/unsloth/models/cohere.py</a></td><td><code>CohereAttention.__init__ = eval(init_name)</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/cohere.py">unslothai/unsloth/unsloth/models/cohere.py</a></td><td><code>CohereAttention.forward = CohereAttention_fast_forward</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/cohere.py">unslothai/unsloth/unsloth/models/cohere.py</a></td><td><code>CohereDecoderLayer.forward = CohereDecoderLayer_fast_forward</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/cohere.py">unslothai/unsloth/unsloth/models/cohere.py</a></td><td><code>CohereFlashAttention2.forward = CohereAttention_fast_forward</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/cohere.py">unslothai/unsloth/unsloth/models/cohere.py</a></td><td><code>CohereForCausalLM.forward = CausalLM_fast_forward(CohereModel_fast_forward_inference)</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/cohere.py">unslothai/unsloth/unsloth/models/cohere.py</a></td><td><code>CohereModel.forward = LlamaModel_fast_forward</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/cohere.py">unslothai/unsloth/unsloth/models/cohere.py</a></td><td><code>CohereSdpaAttention.forward = CohereAttention_fast_forward</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/cohere.py">unslothai/unsloth/unsloth/models/cohere.py</a></td><td><code>PeftModelForCausalLM.forward = PeftModel_fast_forward</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/cohere.py">unslothai/unsloth/unsloth/models/cohere.py</a></td><td><code>transformers.models.cohere.modeling_cohere.CohereRotaryEmbedding = LlamaRotaryEmbedding</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/compressed_tensors_int4.py">unslothai/unsloth/unsloth/models/compressed_tensors_int4.py</a></td><td><code>PeftModel.save_pretrained = save_pretrained</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/falcon_h1.py">unslothai/unsloth/unsloth/models/falcon_h1.py</a></td><td><code>FalconH1Attention.__init__ = eval(init_name)</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/falcon_h1.py">unslothai/unsloth/unsloth/models/falcon_h1.py</a></td><td><code>FalconH1Attention.forward = FalconH1Attention_fast_forward</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/falcon_h1.py">unslothai/unsloth/unsloth/models/falcon_h1.py</a></td><td><code>FalconH1DecoderLayer.forward = FalconH1DecoderLayer_fast_forward</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/falcon_h1.py">unslothai/unsloth/unsloth/models/falcon_h1.py</a></td><td><code>FalconH1ForCausalLM.forward = CausalLM_fast_forward(</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/falcon_h1.py">unslothai/unsloth/unsloth/models/falcon_h1.py</a></td><td><code>FalconH1Model.forward = LlamaModel_fast_forward</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/falcon_h1.py">unslothai/unsloth/unsloth/models/falcon_h1.py</a></td><td><code>PeftModelForCausalLM.forward = PeftModel_fast_forward</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/falcon_h1.py">unslothai/unsloth/unsloth/models/falcon_h1.py</a></td><td><code>transformers.models.falcon_h1.modeling_falcon_h1.FalconH1RMSNorm = Unsloth_FalconH1RMSNorm</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/falcon_h1.py">unslothai/unsloth/unsloth/models/falcon_h1.py</a></td><td><code>transformers.models.falcon_h1.modeling_falcon_h1.FalconH1RotaryEmbedding = (</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/gemma.py">unslothai/unsloth/unsloth/models/gemma.py</a></td><td><code>GemmaAttention.__init__ = eval(init_name)</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/gemma.py">unslothai/unsloth/unsloth/models/gemma.py</a></td><td><code>GemmaAttention.forward = LlamaAttention_fast_forward</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/gemma.py">unslothai/unsloth/unsloth/models/gemma.py</a></td><td><code>GemmaDecoderLayer.forward = GemmaDecoderLayer_fast_forward</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/gemma.py">unslothai/unsloth/unsloth/models/gemma.py</a></td><td><code>GemmaFlashAttention2.forward = LlamaAttention_fast_forward</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/gemma.py">unslothai/unsloth/unsloth/models/gemma.py</a></td><td><code>GemmaForCausalLM.forward = CausalLM_fast_forward(GemmaModel_fast_forward_inference)</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/gemma.py">unslothai/unsloth/unsloth/models/gemma.py</a></td><td><code>GemmaModel.forward = LlamaModel_fast_forward</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/gemma.py">unslothai/unsloth/unsloth/models/gemma.py</a></td><td><code>GemmaSdpaAttention.forward = LlamaAttention_fast_forward</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/gemma.py">unslothai/unsloth/unsloth/models/gemma.py</a></td><td><code>PeftModelForCausalLM.forward = PeftModel_fast_forward</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/gemma.py">unslothai/unsloth/unsloth/models/gemma.py</a></td><td><code>transformers.models.gemma.modeling_gemma.GemmaRotaryEmbedding = GemmaFixedRotaryEmbedding</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/gemma2.py">unslothai/unsloth/unsloth/models/gemma2.py</a></td><td><code>Gemma2Attention.__init__ = eval(init_name)</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/gemma2.py">unslothai/unsloth/unsloth/models/gemma2.py</a></td><td><code>Gemma2Attention.forward = Gemma2Attention_fast_forward</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/gemma2.py">unslothai/unsloth/unsloth/models/gemma2.py</a></td><td><code>Gemma2DecoderLayer.forward = Gemma2DecoderLayer_fast_forward</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/gemma2.py">unslothai/unsloth/unsloth/models/gemma2.py</a></td><td><code>Gemma2FlashAttention2.forward = Gemma2Attention_fast_forward</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/gemma2.py">unslothai/unsloth/unsloth/models/gemma2.py</a></td><td><code>Gemma2ForCausalLM.forward = CausalLM_fast_forward(Gemma2Model_fast_forward_inference)</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/gemma2.py">unslothai/unsloth/unsloth/models/gemma2.py</a></td><td><code>Gemma2Model.forward = LlamaModel_fast_forward</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/gemma2.py">unslothai/unsloth/unsloth/models/gemma2.py</a></td><td><code>Gemma2SdpaAttention.forward = Gemma2Attention_fast_forward</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/gemma2.py">unslothai/unsloth/unsloth/models/gemma2.py</a></td><td><code>PeftModelForCausalLM.forward = PeftModel_fast_forward</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/gemma2.py">unslothai/unsloth/unsloth/models/gemma2.py</a></td><td><code>transformers.models.gemma2.modeling_gemma2.Gemma2RotaryEmbedding = GemmaFixedRotaryEmbedding</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/granite.py">unslothai/unsloth/unsloth/models/granite.py</a></td><td><code>GraniteAttention.__init__ = eval(init_name)</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/granite.py">unslothai/unsloth/unsloth/models/granite.py</a></td><td><code>GraniteAttention.forward = GraniteAttention_fast_forward</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/granite.py">unslothai/unsloth/unsloth/models/granite.py</a></td><td><code>GraniteDecoderLayer.forward = GraniteDecoderLayer_fast_forward</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/granite.py">unslothai/unsloth/unsloth/models/granite.py</a></td><td><code>GraniteFlashAttention2.forward = GraniteAttention_fast_forward</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/granite.py">unslothai/unsloth/unsloth/models/granite.py</a></td><td><code>GraniteForCausalLM.__init__ = patched_init(GraniteForCausalLM.__init__)</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/granite.py">unslothai/unsloth/unsloth/models/granite.py</a></td><td><code>GraniteForCausalLM.forward = CausalLM_fast_forward(GraniteModel_fast_forward_inference)</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/granite.py">unslothai/unsloth/unsloth/models/granite.py</a></td><td><code>GraniteModel.forward = LlamaModel_fast_forward</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/granite.py">unslothai/unsloth/unsloth/models/granite.py</a></td><td><code>GraniteSdpaAttention.forward = GraniteAttention_fast_forward</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/granite.py">unslothai/unsloth/unsloth/models/granite.py</a></td><td><code>PeftModelForCausalLM.forward = PeftModel_fast_forward</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/granite.py">unslothai/unsloth/unsloth/models/granite.py</a></td><td><code>transformers.models.granite.modeling_granite.GraniteRotaryEmbedding = GraniteRotaryEmbedding</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/llama.py">unslothai/unsloth/unsloth/models/llama.py</a></td><td><code>FastLlamaModel.from_pretrained = staticmethod(</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/llama.py">unslothai/unsloth/unsloth/models/llama.py</a></td><td><code>FastLlamaModel.get_peft_model = staticmethod(</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/llama.py">unslothai/unsloth/unsloth/models/llama.py</a></td><td><code>LlamaAttention.__init__ = eval(init_name)</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/llama.py">unslothai/unsloth/unsloth/models/llama.py</a></td><td><code>LlamaAttention.forward = LlamaAttention_fast_forward</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/llama.py">unslothai/unsloth/unsloth/models/llama.py</a></td><td><code>LlamaDecoderLayer.forward = LlamaDecoderLayer_fast_forward</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/llama.py">unslothai/unsloth/unsloth/models/llama.py</a></td><td><code>LlamaFlashAttention2.forward = LlamaAttention_fast_forward</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/llama.py">unslothai/unsloth/unsloth/models/llama.py</a></td><td><code>LlamaForCausalLM.forward = CausalLM_fast_forward(LlamaModel_fast_forward_inference)</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/llama.py">unslothai/unsloth/unsloth/models/llama.py</a></td><td><code>LlamaModel.forward = LlamaModel_fast_forward</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/llama.py">unslothai/unsloth/unsloth/models/llama.py</a></td><td><code>LlamaModel_fast_forward_inference_custom._unsloth_graph_attention = (</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/llama.py">unslothai/unsloth/unsloth/models/llama.py</a></td><td><code>LlamaSdpaAttention.forward = LlamaAttention_fast_forward</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/llama.py">unslothai/unsloth/unsloth/models/llama.py</a></td><td><code>PeftModelForCausalLM.forward = PeftModel_fast_forward</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/llama.py">unslothai/unsloth/unsloth/models/llama.py</a></td><td><code>transformers.models.llama.modeling_llama.LlamaLinearScalingRotaryEmbedding = (</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/llama.py">unslothai/unsloth/unsloth/models/llama.py</a></td><td><code>transformers.models.llama.modeling_llama.LlamaRotaryEmbedding = LlamaRotaryEmbedding</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/loader.py">unslothai/unsloth/unsloth/models/loader.py</a></td><td><code>PreTrainedModel.from_pretrained = classmethod(wrapped)</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/mistral.py">unslothai/unsloth/unsloth/models/mistral.py</a></td><td><code>MistralAttention.__init__ = eval(init_name)</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/mistral.py">unslothai/unsloth/unsloth/models/mistral.py</a></td><td><code>MistralAttention.forward = MistralAttention_fast_forward</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/mistral.py">unslothai/unsloth/unsloth/models/mistral.py</a></td><td><code>MistralDecoderLayer.forward = LlamaDecoderLayer_fast_forward</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/mistral.py">unslothai/unsloth/unsloth/models/mistral.py</a></td><td><code>MistralFlashAttention2.forward = MistralAttention_fast_forward</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/mistral.py">unslothai/unsloth/unsloth/models/mistral.py</a></td><td><code>MistralForCausalLM.forward = MistralForCausalLM_fast_forward</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/mistral.py">unslothai/unsloth/unsloth/models/mistral.py</a></td><td><code>MistralModel.forward = LlamaModel_fast_forward</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/mistral.py">unslothai/unsloth/unsloth/models/mistral.py</a></td><td><code>MistralSdpaAttention.forward = MistralAttention_fast_forward</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/mistral.py">unslothai/unsloth/unsloth/models/mistral.py</a></td><td><code>PeftModelForCausalLM.forward = PeftModel_fast_forward</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/mistral.py">unslothai/unsloth/unsloth/models/mistral.py</a></td><td><code>transformers.models.mistral.modeling_mistral.MistralRotaryEmbedding = LlamaRotaryEmbedding</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/qwen2.py">unslothai/unsloth/unsloth/models/qwen2.py</a></td><td><code>PeftModelForCausalLM.forward = PeftModel_fast_forward</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/qwen2.py">unslothai/unsloth/unsloth/models/qwen2.py</a></td><td><code>Qwen2Attention.__init__ = eval(init_name)</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/qwen2.py">unslothai/unsloth/unsloth/models/qwen2.py</a></td><td><code>Qwen2Attention.forward = LlamaAttention_fast_forward</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/qwen2.py">unslothai/unsloth/unsloth/models/qwen2.py</a></td><td><code>Qwen2DecoderLayer.forward = LlamaDecoderLayer_fast_forward</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/qwen2.py">unslothai/unsloth/unsloth/models/qwen2.py</a></td><td><code>Qwen2FlashAttention2.forward = LlamaAttention_fast_forward</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/qwen2.py">unslothai/unsloth/unsloth/models/qwen2.py</a></td><td><code>Qwen2ForCausalLM.forward = CausalLM_fast_forward(LlamaModel_fast_forward_inference)</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/qwen2.py">unslothai/unsloth/unsloth/models/qwen2.py</a></td><td><code>Qwen2Model.forward = LlamaModel_fast_forward</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/qwen2.py">unslothai/unsloth/unsloth/models/qwen2.py</a></td><td><code>Qwen2SdpaAttention.forward = LlamaAttention_fast_forward</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/qwen2.py">unslothai/unsloth/unsloth/models/qwen2.py</a></td><td><code>transformers.models.qwen2.modeling_qwen2.Qwen2RotaryEmbedding = LlamaRotaryEmbedding</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/qwen3.py">unslothai/unsloth/unsloth/models/qwen3.py</a></td><td><code>PeftModelForCausalLM.forward = PeftModel_fast_forward</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/qwen3.py">unslothai/unsloth/unsloth/models/qwen3.py</a></td><td><code>Qwen3Attention.__init__ = eval(init_name)</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/qwen3.py">unslothai/unsloth/unsloth/models/qwen3.py</a></td><td><code>Qwen3Attention.forward = Qwen3Attention_fast_forward</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/qwen3.py">unslothai/unsloth/unsloth/models/qwen3.py</a></td><td><code>Qwen3DecoderLayer.forward = LlamaDecoderLayer_fast_forward</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/qwen3.py">unslothai/unsloth/unsloth/models/qwen3.py</a></td><td><code>Qwen3FlashAttention2.forward = Qwen3Attention_fast_forward</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/qwen3.py">unslothai/unsloth/unsloth/models/qwen3.py</a></td><td><code>Qwen3ForCausalLM.forward = CausalLM_fast_forward(</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/qwen3.py">unslothai/unsloth/unsloth/models/qwen3.py</a></td><td><code>Qwen3Model.forward = LlamaModel_fast_forward</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/qwen3.py">unslothai/unsloth/unsloth/models/qwen3.py</a></td><td><code>Qwen3SdpaAttention.forward = Qwen3Attention_fast_forward</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/qwen3.py">unslothai/unsloth/unsloth/models/qwen3.py</a></td><td><code>transformers.models.qwen3.modeling_qwen3.Qwen3RotaryEmbedding = LlamaRotaryEmbedding</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/qwen3_moe.py">unslothai/unsloth/unsloth/models/qwen3_moe.py</a></td><td><code>PeftModelForCausalLM.forward = PeftModel_fast_forward</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/qwen3_moe.py">unslothai/unsloth/unsloth/models/qwen3_moe.py</a></td><td><code>Qwen3MoeAttention.__init__ = eval(init_name)</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/qwen3_moe.py">unslothai/unsloth/unsloth/models/qwen3_moe.py</a></td><td><code>Qwen3MoeAttention.forward = Qwen3Attention_fast_forward</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/qwen3_moe.py">unslothai/unsloth/unsloth/models/qwen3_moe.py</a></td><td><code>Qwen3MoeDecoderLayer.forward = Qwen3MoeDecoderLayer_fast_forward</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/qwen3_moe.py">unslothai/unsloth/unsloth/models/qwen3_moe.py</a></td><td><code>Qwen3MoeForCausalLM.forward = CausalLM_fast_forward(LlamaModel_fast_forward_inference)</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/qwen3_moe.py">unslothai/unsloth/unsloth/models/qwen3_moe.py</a></td><td><code>Qwen3MoeMLP.forward = fast_swiglu_inference # This is analogous to Dense models' MLP</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/qwen3_moe.py">unslothai/unsloth/unsloth/models/qwen3_moe.py</a></td><td><code>Qwen3MoeModel.forward = LlamaModel_fast_forward</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/qwen3_moe.py">unslothai/unsloth/unsloth/models/qwen3_moe.py</a></td><td><code>transformers.models.qwen3_moe.modeling_qwen3_moe.Qwen3MoeRotaryEmbedding = (</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/sentence_transformer.py">unslothai/unsloth/unsloth/models/sentence_transformer.py</a></td><td><code>AutoModel.from_pretrained = original_model_from_pretrained</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/sentence_transformer.py">unslothai/unsloth/unsloth/models/sentence_transformer.py</a></td><td><code>AutoModel.from_pretrained = return_existing_model</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/sentence_transformer.py">unslothai/unsloth/unsloth/models/sentence_transformer.py</a></td><td><code>modeling_distilbert.DistilBertModel.forward = forward</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/sentence_transformer.py">unslothai/unsloth/unsloth/models/sentence_transformer.py</a></td><td><code>modeling_mpnet.MPNetEncoder.forward = forward</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/sentence_transformer.py">unslothai/unsloth/unsloth/models/sentence_transformer.py</a></td><td><code>modeling_mpnet.MPNetModel._set_gradient_checkpointing = _set_gradient_checkpointing</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/sentence_transformer.py">unslothai/unsloth/unsloth/models/sentence_transformer.py</a></td><td><code>modeling_mpnet.MPNetModel.supports_gradient_checkpointing = True</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/sentence_transformer.py">unslothai/unsloth/unsloth/models/sentence_transformer.py</a></td><td><code>transformers.AutoModelForCausalLM = original_causal</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/sentence_transformer.py">unslothai/unsloth/unsloth/models/sentence_transformer.py</a></td><td><code>transformers.AutoModelForCausalLM = transformers.AutoModel</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/tp_native.py">unslothai/unsloth/unsloth/models/tp_native.py</a></td><td><code>PreTrainedModel.from_pretrained = classmethod(from_pretrained)</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/tp_native.py">unslothai/unsloth/unsloth/models/tp_native.py</a></td><td><code>PreTrainedModel.from_pretrained = descriptor</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/vision.py">unslothai/unsloth/unsloth/models/vision.py</a></td><td><code>FastBaseModel.from_pretrained = staticmethod(</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/vision.py">unslothai/unsloth/unsloth/models/vision.py</a></td><td><code>FastBaseModel.get_peft_model = staticmethod(</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/vision.py">unslothai/unsloth/unsloth/models/vision.py</a></td><td><code>PreTrainedModel.to = original_to</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/vision.py">unslothai/unsloth/unsloth/models/vision.py</a></td><td><code>PreTrainedModel.to = to</code></td></tr><tr><td><a href="https://github.com/verl-project/verl/blob/HEAD/verl/models/mcore/patch.py">verl-project/verl/verl/models/mcore/patch.py</a></td><td><code>GPTModel._postprocess = _patched</code></td></tr><tr><td><a href="https://github.com/verl-project/verl/blob/HEAD/verl/models/mcore/patch.py">verl-project/verl/verl/models/mcore/patch.py</a></td><td><code>MLASelfAttention.get_query_key_value_tensors = patch_get_query_key_value_tensors</code></td></tr><tr><td><a href="https://github.com/verl-project/verl/blob/HEAD/verl/models/mcore/patch.py">verl-project/verl/verl/models/mcore/patch.py</a></td><td><code>MultiLatentAttention.forward = patch_forward</code></td></tr><tr><td><a href="https://github.com/verl-project/verl/blob/HEAD/verl/models/transformers/monkey_patch.py">verl-project/verl/verl/models/transformers/monkey_patch.py</a></td><td><code>AutoModelForCausalLMWithValueHead.state_dict = state_dict</code></td></tr><tr><td><a href="https://github.com/verl-project/verl/blob/HEAD/verl/models/transformers/monkey_patch.py">verl-project/verl/verl/models/transformers/monkey_patch.py</a></td><td><code>Glm4vModel.forward = glm4v_base_forward</code></td></tr><tr><td><a href="https://github.com/verl-project/verl/blob/HEAD/verl/models/transformers/monkey_patch.py">verl-project/verl/verl/models/transformers/monkey_patch.py</a></td><td><code>Glm4vTextAttention.forward = glm4v_attn_forward</code></td></tr><tr><td><a href="https://github.com/verl-project/verl/blob/HEAD/verl/models/transformers/monkey_patch.py">verl-project/verl/verl/models/transformers/monkey_patch.py</a></td><td><code>Qwen2VLAttention.forward = qwen2_vl_attn_forward</code></td></tr><tr><td><a href="https://github.com/verl-project/verl/blob/HEAD/verl/models/transformers/monkey_patch.py">verl-project/verl/verl/models/transformers/monkey_patch.py</a></td><td><code>Qwen2VLModel.forward = qwen2_vl_base_forward</code></td></tr><tr><td><a href="https://github.com/verl-project/verl/blob/HEAD/verl/models/transformers/monkey_patch.py">verl-project/verl/verl/models/transformers/monkey_patch.py</a></td><td><code>Qwen2_5_VLAttention.forward = qwen2_vl_attn_forward</code></td></tr><tr><td><a href="https://github.com/verl-project/verl/blob/HEAD/verl/models/transformers/monkey_patch.py">verl-project/verl/verl/models/transformers/monkey_patch.py</a></td><td><code>Qwen2_5_VLModel.forward = qwen2_vl_base_forward</code></td></tr><tr><td><a href="https://github.com/verl-project/verl/blob/HEAD/verl/models/transformers/monkey_patch.py">verl-project/verl/verl/models/transformers/monkey_patch.py</a></td><td><code>Qwen3VLModel.forward = qwen3_vl_base_forward</code></td></tr><tr><td><a href="https://github.com/verl-project/verl/blob/HEAD/verl/models/transformers/monkey_patch.py">verl-project/verl/verl/models/transformers/monkey_patch.py</a></td><td><code>Qwen3VLMoeModel.forward = qwen3_vl_base_forward</code></td></tr><tr><td><a href="https://github.com/verl-project/verl/blob/HEAD/verl/models/transformers/monkey_patch.py">verl-project/verl/verl/models/transformers/monkey_patch.py</a></td><td><code>Qwen3VLMoeVisionModel.fast_pos_embed_interpolate = fast_pos_embed_interpolate</code></td></tr><tr><td><a href="https://github.com/verl-project/verl/blob/HEAD/verl/models/transformers/monkey_patch.py">verl-project/verl/verl/models/transformers/monkey_patch.py</a></td><td><code>Qwen3VLVisionModel.fast_pos_embed_interpolate = fast_pos_embed_interpolate</code></td></tr><tr><td><a href="https://github.com/verl-project/verl/blob/HEAD/verl/models/transformers/monkey_patch.py">verl-project/verl/verl/models/transformers/monkey_patch.py</a></td><td><code>Qwen3_5DecoderLayer.forward = qwen3_5_decoder_layer_forward</code></td></tr><tr><td><a href="https://github.com/verl-project/verl/blob/HEAD/verl/models/transformers/monkey_patch.py">verl-project/verl/verl/models/transformers/monkey_patch.py</a></td><td><code>Qwen3_5Model.forward = qwen3_5_base_forward</code></td></tr><tr><td><a href="https://github.com/verl-project/verl/blob/HEAD/verl/models/transformers/monkey_patch.py">verl-project/verl/verl/models/transformers/monkey_patch.py</a></td><td><code>Qwen3_5MoeDecoderLayer.forward = qwen3_5_decoder_layer_forward</code></td></tr><tr><td><a href="https://github.com/verl-project/verl/blob/HEAD/verl/models/transformers/monkey_patch.py">verl-project/verl/verl/models/transformers/monkey_patch.py</a></td><td><code>Qwen3_5MoeModel.forward = qwen3_5_base_forward</code></td></tr><tr><td><a href="https://github.com/verl-project/verl/blob/HEAD/verl/models/transformers/monkey_patch.py">verl-project/verl/verl/models/transformers/monkey_patch.py</a></td><td><code>Qwen3_5MoeVisionModel.fast_pos_embed_interpolate = fast_pos_embed_interpolate</code></td></tr><tr><td><a href="https://github.com/verl-project/verl/blob/HEAD/verl/models/transformers/monkey_patch.py">verl-project/verl/verl/models/transformers/monkey_patch.py</a></td><td><code>Qwen3_5VisionModel.fast_pos_embed_interpolate = fast_pos_embed_interpolate</code></td></tr><tr><td><a href="https://github.com/verl-project/verl/blob/HEAD/verl/models/transformers/npu_patch.py">verl-project/verl/verl/models/transformers/npu_patch.py</a></td><td><code>modeling_qwen2.Qwen2MLP.forward = silu_forward_npu</code></td></tr><tr><td><a href="https://github.com/verl-project/verl/blob/HEAD/verl/models/transformers/npu_patch.py">verl-project/verl/verl/models/transformers/npu_patch.py</a></td><td><code>modeling_qwen2.Qwen2RMSNorm.forward = rms_norm_forward_npu</code></td></tr><tr><td><a href="https://github.com/verl-project/verl/blob/HEAD/verl/models/transformers/npu_patch.py">verl-project/verl/verl/models/transformers/npu_patch.py</a></td><td><code>modeling_qwen2.apply_rotary_pos_emb = apply_rotary_pos_emb_npu</code></td></tr><tr><td><a href="https://github.com/verl-project/verl/blob/HEAD/verl/models/transformers/npu_patch.py">verl-project/verl/verl/models/transformers/npu_patch.py</a></td><td><code>modeling_qwen2_5_vl.Qwen2RMSNorm.forward = rms_norm_forward_npu</code></td></tr><tr><td><a href="https://github.com/verl-project/verl/blob/HEAD/verl/models/transformers/npu_patch.py">verl-project/verl/verl/models/transformers/npu_patch.py</a></td><td><code>modeling_qwen2_5_vl.Qwen2_5_VLMLP.forward = silu_forward_npu</code></td></tr><tr><td><a href="https://github.com/verl-project/verl/blob/HEAD/verl/models/transformers/npu_patch.py">verl-project/verl/verl/models/transformers/npu_patch.py</a></td><td><code>modeling_qwen2_5_vl.Qwen2_5_VLRMSNorm.forward = rms_norm_forward_npu</code></td></tr><tr><td><a href="https://github.com/verl-project/verl/blob/HEAD/verl/models/transformers/npu_patch.py">verl-project/verl/verl/models/transformers/npu_patch.py</a></td><td><code>modeling_qwen3.Qwen3MLP.forward = silu_forward_npu</code></td></tr><tr><td><a href="https://github.com/verl-project/verl/blob/HEAD/verl/models/transformers/npu_patch.py">verl-project/verl/verl/models/transformers/npu_patch.py</a></td><td><code>modeling_qwen3.Qwen3RMSNorm.forward = rms_norm_forward_npu</code></td></tr><tr><td><a href="https://github.com/verl-project/verl/blob/HEAD/verl/models/transformers/npu_patch.py">verl-project/verl/verl/models/transformers/npu_patch.py</a></td><td><code>modeling_qwen3.apply_rotary_pos_emb = apply_rotary_pos_emb_npu</code></td></tr><tr><td><a href="https://github.com/verl-project/verl/blob/HEAD/verl/models/transformers/npu_patch.py">verl-project/verl/verl/models/transformers/npu_patch.py</a></td><td><code>modeling_qwen3_5.Qwen3_5RMSNorm.forward = qwen3_next_rms_norm_forward_npu</code></td></tr><tr><td><a href="https://github.com/verl-project/verl/blob/HEAD/verl/models/transformers/npu_patch.py">verl-project/verl/verl/models/transformers/npu_patch.py</a></td><td><code>modeling_qwen3_5.Qwen3_5RMSNormGated.forward = qwen3_next_rms_norm_forward_gated_npu</code></td></tr><tr><td><a href="https://github.com/verl-project/verl/blob/HEAD/verl/models/transformers/npu_patch.py">verl-project/verl/verl/models/transformers/npu_patch.py</a></td><td><code>modeling_qwen3_5.apply_rotary_pos_emb = qwen3_next_apply_rotary_pos_emb_npu</code></td></tr><tr><td><a href="https://github.com/verl-project/verl/blob/HEAD/verl/models/transformers/npu_patch.py">verl-project/verl/verl/models/transformers/npu_patch.py</a></td><td><code>modeling_qwen3_5_moe.Qwen3_5MoeExperts.forward = qwen3_5_moe_experts_forward_npu</code></td></tr><tr><td><a href="https://github.com/verl-project/verl/blob/HEAD/verl/models/transformers/npu_patch.py">verl-project/verl/verl/models/transformers/npu_patch.py</a></td><td><code>modeling_qwen3_5_moe.Qwen3_5MoeRMSNorm.forward = qwen3_next_rms_norm_forward_npu</code></td></tr><tr><td><a href="https://github.com/verl-project/verl/blob/HEAD/verl/models/transformers/npu_patch.py">verl-project/verl/verl/models/transformers/npu_patch.py</a></td><td><code>modeling_qwen3_5_moe.Qwen3_5MoeRMSNormGated.forward = qwen3_next_rms_norm_forward_gated_npu</code></td></tr><tr><td><a href="https://github.com/verl-project/verl/blob/HEAD/verl/models/transformers/npu_patch.py">verl-project/verl/verl/models/transformers/npu_patch.py</a></td><td><code>modeling_qwen3_5_moe.apply_rotary_pos_emb = qwen3_next_apply_rotary_pos_emb_npu</code></td></tr><tr><td><a href="https://github.com/verl-project/verl/blob/HEAD/verl/models/transformers/npu_patch.py">verl-project/verl/verl/models/transformers/npu_patch.py</a></td><td><code>modeling_qwen3_moe.Qwen3MoeRMSNorm.forward = rms_norm_forward_npu</code></td></tr><tr><td><a href="https://github.com/verl-project/verl/blob/HEAD/verl/models/transformers/npu_patch.py">verl-project/verl/verl/models/transformers/npu_patch.py</a></td><td><code>modeling_qwen3_moe.Qwen3MoeSparseMoeBlock.forward = qwen3_moe_sparse_moe_block_forward_npu</code></td></tr><tr><td><a href="https://github.com/verl-project/verl/blob/HEAD/verl/models/transformers/npu_patch.py">verl-project/verl/verl/models/transformers/npu_patch.py</a></td><td><code>modeling_qwen3_moe.apply_rotary_pos_emb = apply_rotary_pos_emb_npu</code></td></tr><tr><td><a href="https://github.com/verl-project/verl/blob/HEAD/verl/models/transformers/npu_patch.py">verl-project/verl/verl/models/transformers/npu_patch.py</a></td><td><code>modeling_qwen3_next.Qwen3NextRMSNorm.forward = qwen3_next_rms_norm_forward_npu</code></td></tr><tr><td><a href="https://github.com/verl-project/verl/blob/HEAD/verl/models/transformers/npu_patch.py">verl-project/verl/verl/models/transformers/npu_patch.py</a></td><td><code>modeling_qwen3_next.Qwen3NextRMSNormGated.forward = qwen3_next_rms_norm_forward_gated_npu</code></td></tr><tr><td><a href="https://github.com/verl-project/verl/blob/HEAD/verl/models/transformers/npu_patch.py">verl-project/verl/verl/models/transformers/npu_patch.py</a></td><td><code>modeling_qwen3_next.Qwen3NextSparseMoeBlock.forward = qwen3_next_sparse_moe_block_forward_npu</code></td></tr><tr><td><a href="https://github.com/verl-project/verl/blob/HEAD/verl/models/transformers/npu_patch.py">verl-project/verl/verl/models/transformers/npu_patch.py</a></td><td><code>modeling_qwen3_next.apply_rotary_pos_emb = qwen3_next_apply_rotary_pos_emb_npu</code></td></tr><tr><td><a href="https://github.com/verl-project/verl/blob/HEAD/verl/models/transformers/npu_patch.py">verl-project/verl/verl/models/transformers/npu_patch.py</a></td><td><code>modeling_qwen3_vl.Qwen3VLTextMLP.forward = silu_forward_npu</code></td></tr><tr><td><a href="https://github.com/verl-project/verl/blob/HEAD/verl/models/transformers/npu_patch.py">verl-project/verl/verl/models/transformers/npu_patch.py</a></td><td><code>modeling_qwen3_vl.Qwen3VLTextRMSNorm.forward = rms_norm_forward_npu</code></td></tr><tr><td><a href="https://github.com/verl-project/verl/blob/HEAD/verl/models/transformers/npu_patch.py">verl-project/verl/verl/models/transformers/npu_patch.py</a></td><td><code>modeling_qwen3_vl_moe.Qwen3VLMoeTextRMSNorm.forward = rms_norm_forward_npu</code></td></tr><tr><td><a href="https://github.com/verl-project/verl/blob/HEAD/verl/models/transformers/npu_patch.py">verl-project/verl/verl/models/transformers/npu_patch.py</a></td><td><code>modeling_qwen3_vl_moe.Qwen3VLMoeTextSparseMoeBlock = NPUQwen3VLMoeTextSparseMoeBlock</code></td></tr><tr><td><a href="https://github.com/verl-project/verl/blob/HEAD/verl/models/transformers/npu_patch.py">verl-project/verl/verl/models/transformers/npu_patch.py</a></td><td><code>modeling_qwen3_vl_moe.apply_rotary_pos_emb = apply_rotary_pos_emb_npu</code></td></tr><tr><td><a href="https://github.com/vllm-project/llm-compressor/blob/HEAD/examples/quantizing_moe/deepseek_v4_example.py">vllm-project/llm-compressor/examples/quantizing_moe/deepseek_v4_example.py</a></td><td><code>DeepseekV4PreTrainedModel._keep_in_fp32_modules_strict = set()</code></td></tr><tr><td><a href="https://github.com/vllm-project/llm-compressor/blob/HEAD/examples/quantizing_moe/deepseek_v4_mixed_mxfp4_mxfp8.py">vllm-project/llm-compressor/examples/quantizing_moe/deepseek_v4_mixed_mxfp4_mxfp8.py</a></td><td><code>DeepseekV4PreTrainedModel._keep_in_fp32_modules_strict = set()</code></td></tr><tr><td><a href="https://github.com/vllm-project/llm-compressor/blob/HEAD/examples/quantizing_moe/deepseek_v4_pro_example.py">vllm-project/llm-compressor/examples/quantizing_moe/deepseek_v4_pro_example.py</a></td><td><code>DeepseekV4PreTrainedModel._keep_in_fp32_modules_strict = set()</code></td></tr><tr><td><a href="https://github.com/vllm-project/llm-compressor/blob/HEAD/src/llmcompressor/modeling/moe/linearize.py">vllm-project/llm-compressor/src/llmcompressor/modeling/moe/linearize.py</a></td><td><code>transformers.integrations.finegrained_fp8.replace_with_fp8_linear = (</code></td></tr><tr><td><a href="https://github.com/vllm-project/vllm/blob/HEAD/vllm/model_executor/models/adapters.py">vllm-project/vllm/vllm/model_executor/models/adapters.py</a></td><td><code>ModelForEmbedding.__name__ = _get_pooling_model_name(cls.__name__, "ForEmbedding")</code></td></tr><tr><td><a href="https://github.com/vllm-project/vllm/blob/HEAD/vllm/model_executor/models/expert_substitution.py">vllm-project/vllm/vllm/model_executor/models/expert_substitution.py</a></td><td><code>ExpertSubstitutionModel.__name__ = model_cls.__name__</code></td></tr><tr><td><a href="https://github.com/vllm-project/vllm/blob/HEAD/vllm/model_executor/models/expert_substitution.py">vllm-project/vllm/vllm/model_executor/models/expert_substitution.py</a></td><td><code>ExpertSubstitutionModel.__qualname__ = model_cls.__qualname__</code></td></tr><tr><td><a href="https://github.com/vllm-project/vllm/blob/HEAD/vllm/v1/attention/backends/composite.py">vllm-project/vllm/vllm/v1/attention/backends/composite.py</a></td><td><code>CompositeAttentionBackend.__module__ = module</code></td></tr><tr><td><a href="https://github.com/vllm-project/vllm/blob/HEAD/vllm/v1/attention/backends/composite.py">vllm-project/vllm/vllm/v1/attention/backends/composite.py</a></td><td><code>CompositeAttentionBackend.__name__ = name</code></td></tr><tr><td><a href="https://github.com/vllm-project/vllm/blob/HEAD/vllm/v1/attention/backends/composite.py">vllm-project/vllm/vllm/v1/attention/backends/composite.py</a></td><td><code>CompositeAttentionBackend.__qualname__ = name</code></td></tr></table></details> |