HyperFlow H3 Flash 1.0.0: INT8 convrot and GGUF for fl2va and ref2va

.gitattributes

@@ -0,0 +1,36 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gguf filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text

LICENSE

@@ -0,0 +1,84 @@
+MiniMax H3 COMMUNITY LICENSE AGREEMENT
+MiniMax H3 release date/License date: August 2, 2026.
+The scope of this License Agreement (this “Agreement”) is expressly limited to the “Applicable Territory” as defined below.
+By clicking to accept, or by using, reproducing, modifying, distributing, running, or displaying any portion or element of the MiniMax H3 Works (including through any Hosted Services) in any manner, you acknowledge and accept the terms of this Agreement, and this Agreement shall take immediate effect upon the occurrence of such act.
+I.  Definitions
+1. “Acceptable Use Policy” means the policy published by MiniMax in Exhibit A.
+2. “Agreement” means the terms and conditions set forth herein that govern the use, reproduction, distribution, modification, running, and display of the MiniMax H3 Works or any portion or element thereof.
+3. “Applicable Territory” means worldwide, excluding the Excluded Territories.
+4. “Documentation” means the specifications, manuals, and documentation concerning MiniMax H3 that are publicly released by MiniMax.
+5. “Excluded Territories” means the European Union, the United Kingdom, the Republic of Korea and the United States of America.  
+6. “MiniMax H3” means the video generation model, together with its software and algorithms, including trained model weights, parameters (including optimizer states), machine-learning model code, inference-supporting code, and other elements thereof made publicly available by Us, as released at https://huggingface.co/MiniMaxAI/MiniMax-H3.
+7. “MiniMax H3 Works” means (i) the Materials, (ii) the Model Derivatives, and (iii) all derivatives thereof.
+8. “Hosted Services” means hosted services provided via application programming interfaces (APIs), web access, or any other electronic or remote means.
+9. “Licensee,” “you,” or “your” means the natural or legal person exercising rights and/or using the MiniMax H3 Works for any purpose in any field of use under this Agreement.
+10. “Materials” means, collectively, MiniMax H3 and the Documentation (and any portion thereof), in each case as made available by MiniMax under this Agreement and proprietary to MiniMax.
+11. “Model Derivatives” means all of the following: (i) any modification of MiniMax H3 or any Model Derivative thereof; (ii) any work based on MiniMax H3 or any Model Derivative thereof; or (iii) any other machine learning model created by transferring the patterns of the weights, parameters, operational patterns, or Outputs of MiniMax H3 or any Model Derivative thereof to another model, such that the latter model exhibits behavior similar to MiniMax H3 or its Model Derivatives, including by distillation methods, methods using intermediate data representations, or methods based on training using synthetic-data Outputs generated by MiniMax H3 or its Model Derivatives. For the avoidance of doubt, Outputs are not deemed Model Derivatives.
+12. “Output” means any result of operating or otherwise using MiniMax H3 or any Model Derivatives (including through Hosted Services).
+13. “Third Party” means any natural or legal person that is not under common control with us or with you.
+14. “Including” means “including but not limited to.”
+15. “We,” “Us”  or  “MiniMax” means Nanonoble Pte. Ltd..
+II.  Grant of Rights
+Solely within the Applicable Territory, we grant you a non-exclusive, non-transferable, royalty-free, limited license to use, reproduce, distribute, create derivative works (including Model Derivatives), and modify the Materials in accordance with the terms of this Agreement and the Acceptable Use Policy, based on the intellectual property and other rights owned by MiniMax that are embodied in or used by the Materials. You shall not violate (or encourage or permit any person to violate) any term of this Agreement or the Acceptable Use Policy.
+We will continuously evaluate the applicable laws, regulations and compliance requirements for the Excluded Territories. In the meantime, should any person in such Excluded Territories be interested in deploying our models, you are welcome to contact us about obtaining a license, which will be granted based on robust controls and guardrails for purposes of complying with the laws, regulations and compliance requirements of the Excluded Territories.
+III.  Distribution and Redistribution
+Subject to and conditioned on your continuing compliance with this Agreement, including its territorial restrictions and the Acceptable Use Policy, and solely within the Applicable Territory, you may distribute or make available the MiniMax H3 Works to Third Parties within the Applicable Territory; provided, that all of the following conditions are met:
+1. You must provide a copy of this Agreement to all such Third Parties who receive the MiniMax H3 Works or use your products or services related thereto;
+2. You must cause any modified files to carry prominent notices stating that you have modified such files;
+3. You are encouraged to:
+  a. display a notice on any product or service developed using MiniMax H3 indicating that the product or service is “Powered by MiniMax H3”; 
+  b. add an AI-generation identifier to files produced using generative AI models including MiniMax H3; and
+  c. publish at least one technical blog post or a public statement describing your experience using MiniMax H3 Works;
+4. All distributions to Third Parties (other than through Hosted Services) must be accompanied by a “NOTICE” text file containing the following notice:
+  “MiniMax H3 is licensed under the MiniMax H3 Community License Agreement, Copyright © 2026 MiniMax. All Rights Reserved.”
+You may add your own copyright notices on your modifications; except as provided in this Section and in Section V, however, you may not impose additional or different terms and conditions on the use, reproduction, or distribution of your modifications or of any aggregate Model Derivatives, and your use, reproduction, modification, distribution, running, and display of the work must otherwise comply with the terms and conditions of this Agreement (including the provisions concerning the Applicable Territory). If you receive the MiniMax H3 Works from a Licensee as part of an integrated end-user product, the provisions of Section III of this Agreement do not apply to you, but Section V and Exhibit A remain applicable.
+IV.  Additional Commercial Terms
+1. You shall obtain a separate, prior written authorization from MiniMax by contacting api@minimax.io with the subject line “MiniMax H3 licensing - authorization request”, if your commercial products and services generate more than 20 million US dollars (or equivalent in other currencies) in yearly revenue.
+2. You shall prominently display “MiniMax H3”on the user interface of commercial product or service that uses MiniMax H3 or MiniMax H3 Works.
+V. Use Restrictions
+1. Your use of the MiniMax H3 Works must comply with applicable laws and regulations (including trade-compliance laws and regulations) and must comply with the Acceptable Use Policy for the MiniMax H3 Works, which is incorporated into this Agreement by reference. 
+2. Before providing access to the MiniMax H3 Works or any product, service, or Hosted Service incorporating them, you must bind each recipient or user to enforceable terms at least as protective as the use restrictions in this Section V and Exhibit A, and you must notify each recipient or user that those restrictions apply.
+3. You may not use the MiniMax H3 Works or any of their Outputs or results to improve any other artificial intelligence model (other than MiniMax H3 or its Model Derivatives).
+4. You may not use, reproduce, modify, distribute, or display the MiniMax H3 Works or any of their Outputs or results outside the Applicable Territory. Any such use outside the Applicable Territory is not authorized by this Agreement.
+5. If you provide or make available to any Third Party a product, service, or Hosted Service that permits the generation of Outputs using MiniMax H3 or any Model Derivative, you must, before making that product or service available and throughout its operation, implement, maintain, test, and periodically review reasonable and proportionate technical and organizational safeguards designed to prevent and mitigate access, uses, and Outputs that violate this Section V or Exhibit A, including uses or Outputs that infringe, misappropriate, or otherwise violate any Third Party’s intellectual-property or other rights. You must not knowingly disable, materially weaken, or permit the circumvention of those safeguards. You must maintain a reasonably accessible mechanism for reporting suspected violations. Upon receiving a good-faith report or otherwise obtaining actual knowledge of a violation, you must promptly investigate and take reasonable steps within your control to stop or mitigate the violation, including removing or disabling access to offending content or services and suspending or terminating repeat violators where appropriate. You are responsible for implementing and enforcing these requirements with respect to your products, services, systems, users, and downstream recipients.
+VI.  Intellectual Property
+1. Subject to MiniMax’s rights in the MiniMax H3 Works (and the intellectual property therein), and to your compliance with the terms and conditions of this Agreement, as between you and MiniMax, you will own the derivative works and modifications of the Materials that you have created or had created, as well as any Model Derivatives.
+2. Except for the limited license expressly granted in this paragraph, no trademark license is granted under this Agreement; with respect to MiniMax H3 Works, the Licensee may not use any name or mark owned by or associated with MiniMax or any of its affiliates, except as reasonably and customarily necessary to describe and distribute the MiniMax H3 Works. MiniMax hereby grants you a license to use the “MiniMax H3” mark (the “Mark”) within the Applicable Territory solely for the purpose of complying with Section III.3; provided, that you comply with all applicable trademark-protection laws. All goodwill arising from your use of the Mark shall inure to the benefit of MiniMax.
+3. If you bring or assert any suit or other legal proceeding (including a cross-claim or counterclaim in any action) against us or any other natural or legal person alleging that the Materials, any Output, or any portion of the foregoing infringes any intellectual property right or other right owned by you or for which you can obtain a license, all licenses granted to you under this Agreement will terminate as of the date such suit or proceeding is filed. You shall defend, indemnify, and hold us harmless against any Third-Party claim arising out of or related to the use or distribution of the MiniMax H3 Works by you or by any Third Party.
+4. MiniMax claims no rights over the Outputs you generate. You and your users are entirely responsible for the Outputs and any subsequent use thereof.
+VII.  Disclaimers and Limitations of Liability
+1. We have no obligation to support, update, provide training for, or develop any further version of the MiniMax H3 Works, or to grant any license with respect thereto.
+2. UNLESS AND ONLY TO THE EXTENT REQUIRED BY APPLICABLE LAW, THE MINIMAX H3 WORKS AND ANY OUTPUT AND RESULTS THEREFROM ARE PROVIDED “AS IS” WITHOUT ANY EXPRESS OR IMPLIED WARRANTIES OF ANY KIND INCLUDING ANY WARRANTIES OF TITLE, MERCHANTABILITY, NONINFRINGEMENT, COURSE OF DEALING, USAGE OF TRADE, OR FITNESS FOR A PARTICULAR PURPOSE. YOU ARE SOLELY RESPONSIBLE FOR DETERMINING THE APPROPRIATENESS OF USING, REPRODUCING, MODIFYING, PERFORMING, DISPLAYING OR DISTRIBUTING ANY OF THE MINIMAX H3 WORKS OR OUTPUTS AND ASSUME ANY AND ALL RISKS ASSOCIATED WITH YOUR OR A THIRD PARTY’S USE OR DISTRIBUTION OF ANY OF THE MINIMAX H3 WORKS OR OUTPUTS AND YOUR EXERCISE OF RIGHTS AND PERMISSIONS UNDER THIS AGREEMENT.
+3. TO THE FULLEST EXTENT PERMITTED BY APPLICABLE LAW, IN NO EVENT SHALL MINIMAX OR ITS AFFILIATES BE LIABLE UNDER ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, TORT, NEGLIGENCE, PRODUCTS LIABILITY, OR OTHERWISE, FOR ANY DAMAGES, INCLUDING ANY DIRECT, INDIRECT, SPECIAL, INCIDENTAL, EXEMPLARY, CONSEQUENTIAL OR PUNITIVE DAMAGES, OR LOST PROFITS OF ANY KIND ARISING FROM THIS AGREEMENT OR RELATED TO ANY OF THE MINIMAX H3 WORKS OR OUTPUTS, EVEN IF MINIMAX OR ITS AFFILIATES HAVE BEEN ADVISED OF THE POSSIBILITY OF ANY OF THE FOREGOING.
+VIII.  Term and Termination
+1. This Agreement is effective from the moment you accept this Agreement or begin accessing the Materials, and, subject to your compliance with its terms and conditions, will remain in effect until terminated as provided herein.
+2. If you breach any term or condition of this Agreement, we have the right to terminate this Agreement. Upon termination, you must immediately cease accessing, using, and distributing the MiniMax H3 Works; delete or destroy all copies within your possession or control; and notify each downstream recipient that your authorization has ended. The obligations in the preceding sentence and Sections VI.1, VI.3, VII, and IX survive termination.
+IX.  Governing Law and Jurisdiction
+1. This Agreement, and any dispute arising out of or related to this Agreement, shall be governed by the laws of the Hong Kong Special Administrative Region of the People’s Republic of China, without regard to its conflict-of-laws rules. The United Nations Convention on Contracts for the International Sale of Goods does not apply to this Agreement.
+2. Any dispute arising out of or related to this Agreement shall be subject to the exclusive jurisdiction of the courts of the Hong Kong Special Administrative Region of the People’s Republic of China with competent jurisdiction. Both MiniMax and the Licensee hereby consent to the exclusive jurisdiction of such courts for any such dispute.
+Additional Note: Please note that the encoder of MiniMax H3 uses Qwen3-VL-32B, which is licensed under Apache 2.0 License: https://github.com/QwenLM/Qwen3-VL/blob/main/LICENSE.
+
+Exhibit A — Acceptable Use Policy
+MiniMax reserves the right to update this Acceptable Use Policy from time to time. 
+Last revised: August 2, 2026.
+MiniMax is committed to promoting the safe and fair use of its tools and features, including MiniMax H3. You agree not to use MiniMax H3, any Model Derivatives, or any Output in any of the following ways:
+1. Use outside the Applicable Territory;
+2. Use in any manner that violates any applicable national, federal, state, local, or international law, regulation, or other legal requirement, or that infringes, misappropriates, or otherwise violates any Third Party’s intellectual-property or other proprietary rights, including through unauthorized reproduction, distribution, public display, public performance, or creation of derivative works;
+3. Use in any manner that may harm yourself or others;
+4. Use to repurpose or distribute the Outputs of MiniMax H3 or any Model Derivatives in order to harm yourself or others;
+5. Use to circumvent or bypass any safety guardrails or safeguards we have implemented;
+6. Use in any manner that exploits or harms, or intends to exploit or harm, minors;
+7. Use to generate or disseminate verifiably false information and/or content for the purpose of harming others or influencing elections;
+8. Use to manufacture or facilitate false online engagement, including fake reviews and other means of false online engagement;
+9. Use to intentionally defame, disparage, or otherwise harass others;
+10. Use to generate and/or disseminate malware (including ransomware) or any other content intended to damage electronic systems;
+11. Use to generate or disseminate personally identifiable information for the purpose of harming others;
+12. Use to generate or disseminate information (including images, code, posts, or articles) in or to any public environment (including via bot tweets or similar means) without clearly and prominently disclosing that such information and/or content is machine-generated;
+13. Use to impersonate another person without that person’s consent, authorization, or lawful right to do so;
+14. Use to make high-risk automated decisions in critical domains that affect individual safety, rights, or well-being (such as law enforcement, immigration, healthcare or medical services, critical-infrastructure management, product-safety components, essential services, credit, employment, housing, education, social scoring, or insurance);
+15. Use in any manner that violates or disregards the social, ethical, or moral standards of other countries or regions;
+16. Use to carry out, assist, threaten, incite, plan, advocate for, or encourage violent extremism or terrorism;
+17. Use for any purpose intended to discriminate against, or harm, individuals or groups based on protected characteristics or categories, online or offline social behavior, or known or predicted personality traits;
+18. Use to intentionally exploit the vulnerabilities of specific populations based on age, social, physical, or psychological characteristics, so as to materially distort the behavior of a member of that group in a manner that causes, or is likely to cause, physical or psychological harm to that person or to others;
+19. Use for military purposes;
+20. Use to engage in any unauthorized or unlicensed professional activity, including but not limited to financial, legal, medical or healthcare, or other professional practice.

NOTICE

@@ -0,0 +1,19 @@
+MiniMax H3 is licensed under the MiniMax H3 Community License Agreement, Copyright © 2026 MiniMax. All Rights Reserved.
+
+The HyperFlow H3 Flash weights in this repository are a Model Derivative of MiniMax-H3
+(https://huggingface.co/MiniMaxAI/MiniMax-H3) as defined in Section I.11 of that Agreement. Video Rebirth created these
+weights (Copyright 2026 Video Rebirth) and distributes them under the same MiniMax H3 Community License Agreement. A
+copy of the Agreement is included in this repository as LICENSE.
+
+Modifications from MiniMax H3 (Section III.2):
+- fl2va_int8_convrot/ and ref2va_int8_convrot/: The transformer weights are pruned, low-rank factored, and quantized to
+  rotated W8A8 INT8.
+- fl2va_gguf/ and ref2va_gguf/: The compressed transformer weights are converted to the stable-diffusion.cpp tensor
+  layout and quantized to GGUF.
+- hyperflow_h3_flash_transformer.py: Modified from the diffusers MiniMax-H3 transformer (paired with
+  hyperflow_h3_flash_int8.py for rotated INT8 linear layers).
+- The upstream text encoder, VAEs, tokenizer, and processor are not redistributed here and remain the official
+  MiniMax-H3 files.
+
+HyperFlow H3 Flash is not an official MiniMax product and is not affiliated with or endorsed by MiniMax.
+Powered by MiniMax H3.

README.md

@@ -0,0 +1,303 @@
+---
+license: other
+license_name: minimax-h3-community-license-agreement
+license_link: LICENSE
+base_model:
+  - MiniMaxAI/MiniMax-H3
+language:
+  - en
+  - zh
+library_name: minimax-h3
+pipeline_tag: image-text-to-video
+tags:
+  - hyperflow-h3
+  - minimax-h3
+  - audio-video-generation
+  - text-to-video
+  - image-to-video
+  - reference-to-video
+  - few-step
+  - int8
+  - gguf
+  - comfyui
+  - stable-diffusion.cpp
+  - unsloth
+---
+
+# HyperFlow H3 Flash (14B)
+
+> By downloading or using these weights, you accept the [MiniMax H3 Community License Agreement](LICENSE). The Agreement restricts territory (not licensed in the EU, UK, South Korea, or the US without prior authorization from MiniMax) and includes an Acceptable Use Policy. See [License](#license). This repository is not an official MiniMax product.
+
+HyperFlow H3 Flash is [Video Rebirth](https://www.videorebirth.com/)'s compressed, few-step version of [MiniMax-H3](https://huggingface.co/MiniMaxAI/MiniMax-H3). It generates video and synchronized stereo audio together.
+
+**Showcase:** See [HyperFlow H3 Flash generated video and audio examples](https://www.videorebirth.com/lp/hyperflow-flash).
+
+- **Output:** a 5.2 to 15 s video at 24 fps (1344×768 by default) with a 32 kHz stereo soundtrack.
+- **Sampling:** 4, 6, or 8 steps without classifier-free guidance (CFG). The default is 6 steps.
+- **Speed (5.2 s clip, 6 steps):** 8.7 s on 4× B200, 36 s on 1× H200, and 92 s on an RTX 5090.
+- **Real time on 4× Blackwell GPUs:** [NVIDIA SOL-H3](https://github.com/NVlabs/Sana/tree/sol-engine/models/minimax_h3/HyperFlow-Flash) runs HyperFlow H3 Flash in real time on 4× B-series GPUs. SOL-H3 is our inference-acceleration partner and supports this model from day 0.
+- **Model size:** 13.7B (`fl2va`) or 13.9B (`ref2va`) parameters, less than half of the ~31B MiniMax-H3 transformer.
+- **Minimum footprint:** `Q4_K_M` is **9.1 GB** (`Q8_0` is **14.9 GB**). It runs on 24 GB and 32 GB GPUs in PCs with 16 GB of system RAM. 16 GB GPUs are not measured yet.
+
+Code, setup guides, and ComfyUI workflows are in [`Video-Rebirth/hyperflow-h3-flash`](https://github.com/Video-Rebirth/hyperflow-h3-flash). For the 8-step LoRA on the full-size MiniMax-H3 transformer, see [`VideoRebirthAI/hyperflow`](https://huggingface.co/VideoRebirthAI/hyperflow) ([GitHub](https://github.com/Video-Rebirth/hyperflow)).
+
+### Supported Tasks
+
+| Task | Input | Hub partition | ComfyUI template | GGUF support | Python INT8 support |
+|---|---|---|---|---|---|
+| `t2va` | Text prompt | `fl2va_*` | `hyperflow_t2v` | Yes | Yes |
+| `fl2va` | Prompt + first keyframe, or first and last keyframes | `fl2va_*` | `hyperflow_i2v`, `hyperflow_flf2v` | Yes | Yes |
+| `ref2va` | Prompt + up to 12 references | `ref2va_*` | `hyperflow_ref2v` | Images only | Images, videos, and audio |
+
+---
+
+## Choose a Weight Format
+
+**Decision rule:** Use **GGUF** on any PC with 64 GB of system RAM or less. Use **INT8 convrot** on data-center GPUs, or on 24 GB+ RTX GPUs with at least 69 GiB of system RAM.
+
+| Feature | GGUF | INT8 convrot |
+|---|---|---|
+| **Hub folders** | `fl2va_gguf/`, `ref2va_gguf/` | `fl2va_int8_convrot/`, `ref2va_int8_convrot/` |
+| **Precision & size** | `Q3_K_M` to `Q8_0` (7.2–15.2 GB) | W8A8 INT8 + low-rank (~16 GB) |
+| **Runtime** | ComfyUI, Unsloth Studio, `stable-diffusion.cpp` | [`hyperflow-h3-flash`](https://github.com/Video-Rebirth/hyperflow-h3-flash) CLI and `diffusers` |
+| **Target GPU** | Consumer GPUs; measured on **24 GB and 32 GB** RTX GPUs (16 GB GPUs not measured yet) | **H100 / H200 / B200 / B300** (1–8 GPUs), or **24 GB+ RTX GPUs** |
+| **System RAM** | **16 GB+** | **~16 GiB** without offload; **≥ 69 GiB** (`fp8` text encoder) or **≥ 91 GiB** (`bf16`) with offload |
+| **`ref2va` inputs** | Images | Images, videos, and audio |
+
+---
+
+## Repository Layout and Companion Files
+
+```text
+fl2va_int8_convrot/    config.json, diffusion_pytorch_model-0000{1..3}-of-00003.safetensors (+ index), model code
+ref2va_int8_convrot/   config.json, diffusion_pytorch_model-0000{1..4}-of-00004.safetensors (+ index), model code
+fl2va_gguf/            minimax_h3_fl2va_hyperflow_flash_14b-<QUANT>.gguf (6 quant levels)
+ref2va_gguf/           minimax_h3_ref2va_hyperflow_flash_14b-<QUANT>.gguf (6 quant levels)
+```
+
+This repository stores only the compressed transformers (`transformer` for `fl2va`, `transformer_ref` for `ref2va`). Each runtime loads the shared text encoder, video VAE, audio VAE, tokenizer, and processor from their upstream repositories:
+
+| Runtime | Companion text encoder and VAEs |
+|---|---|
+| **`hyperflow-h3-flash` (Python)** | Downloads automatically from [`MiniMaxAI/MiniMax-H3`](https://huggingface.co/MiniMaxAI/MiniMax-H3), or from [`Qwen/Qwen3-VL-32B-Instruct-FP8`](https://huggingface.co/Qwen/Qwen3-VL-32B-Instruct-FP8) with `--text-encoder-precision fp8`. Never downloads the 62 GB base transformer. |
+| **ComfyUI & stable-diffusion.cpp** | [`Comfy-Org/MiniMax-H3`](https://huggingface.co/Comfy-Org/MiniMax-H3): `text_encoders/qwen3vl_32b_minimax_h3_*.safetensors` (`nvfp4_awq` 15.7 GB, `int8_convrot` 27.1 GB, or `bf16` 51.5 GB), `vae/minimax_h3_video_vae_fp16.safetensors`, `vae/minimax_h3_audio_vae_fp32.safetensors` |
+| **Unsloth Studio** | Downloads automatically from [`unsloth/MiniMax-H3-GGUF`](https://huggingface.co/unsloth/MiniMax-H3-GGUF) |
+
+---
+
+## GGUF (ComfyUI, Unsloth Studio, stable-diffusion.cpp)
+
+| Quant | `fl2va_gguf/` | `ref2va_gguf/` | Recipe | When to use |
+|---|---:|---:|---|---|
+| **`Q4_K_M`** | **9.1 GB** | **9.2 GB** | `Q4_K` (lifted layers `Q6_K`) | **Recommended default for 24 GB GPUs** |
+| `Q8_0` | 14.9 GB | 15.2 GB | `Q8_0` | Closest to full precision when VRAM and RAM fit |
+| `Q6_K` | 11.6 GB | 11.8 GB | `Q6_K` | High quality, smaller than `Q8_0` |
+| `Q5_K_M` | 10.3 GB | 10.5 GB | `Q5_K` (lifted layers `Q6_K`) | Balanced intermediate quant |
+| `Q4_K_S` | 8.0 GB | 8.1 GB | `Q4_K` | Compact 4-bit quant |
+| `Q3_K_M` | 7.2 GB | 7.4 GB | `Q3_K` (lifted layers `Q5_K`) | Smallest footprint |
+
+Full setup instructions are in [`integrations/README.md`](https://github.com/Video-Rebirth/hyperflow-h3-flash/blob/main/integrations/README.md). Summary:
+
+- **ComfyUI:**
+  1. Clone [ComfyUI-GGUF](https://github.com/city96/ComfyUI-GGUF) and [ComfyUI-HyperFlow-Flash](https://github.com/Video-Rebirth/ComfyUI-HyperFlow-Flash) into `ComfyUI/custom_nodes/`.
+  2. Put the `.gguf` file in `models/diffusion_models/` and download the 15.7 GB `qwen3vl_32b_minimax_h3_nvfp4_awq` text encoder.
+  3. On 16 GB or 32 GB RAM PCs, start ComfyUI with `python main.py --fast-disk`.
+  4. Open **Workflow > Browse Templates > ComfyUI-HyperFlow-Flash** and select `hyperflow_t2v`, `hyperflow_i2v`, `hyperflow_flf2v`, or `hyperflow_ref2v`.
+- **Unsloth Studio:** Point `SD_CLI_PATH` at the patched `sd-cli` binary (`export SD_CLI_PATH=/path/to/stable-diffusion.cpp/build/bin/sd-cli`), run `unsloth studio`, and load `VideoRebirthAI/hyperflow-h3-flash` (`fl2va_gguf/` or `ref2va_gguf/`). Studio downloads the companion files and runs the 6-step schedule at `CFG = 1`.
+- **stable-diffusion.cpp:** Apply [`integrations/stable-diffusion.cpp/hyperflow-two-time.patch`](https://github.com/Video-Rebirth/hyperflow-h3-flash/tree/main/integrations/stable-diffusion.cpp) to upstream commit `3be2378`, build, and run:
+  ```bash
+  sd-cli -M vid_gen \
+    --diffusion-model minimax_h3_fl2va_hyperflow_flash_14b-Q4_K_M.gguf \
+    --llm qwen3vl_32b_minimax_h3_bf16.safetensors \
+    --vae minimax_h3_video_vae_fp16.safetensors \
+    --audio-vae minimax_h3_audio_vae_fp32.safetensors \
+    --sampling-method euler --cfg-scale 1.0 --steps 6 \
+    -W 1344 -H 768 --video-frames 124 --fps 24 -s 42 \
+    -p "A chef slides a steaming bowl of ramen across a wooden counter; rain taps on the window." \
+    -o output.webm
+  ```
+  Add `-i first.png` (and `--end-img last.png`) for keyframes. Use `minimax_h3_ref2va_...` with `--ref-image a.png --ref-image b.png` for reference images. Add `--offload-to-cpu` on smaller GPUs.
+
+---
+
+## INT8 convrot (`hyperflow-h3-flash` & `diffusers`)
+
+### Install
+
+```bash
+pip install git+https://github.com/Video-Rebirth/hyperflow-h3-flash
+pip install "hyperflow-h3-flash[hopper] @ git+https://github.com/Video-Rebirth/hyperflow-h3-flash"  # H100 / H200: FlashAttention-3
+pip install "hyperflow-h3-flash[fp8] @ git+https://github.com/Video-Rebirth/hyperflow-h3-flash"     # --text-encoder-precision fp8
+```
+
+- **Requirements:** Python 3.10+, PyTorch 2.6+ with CUDA and Triton (`pip install triton-windows` on native Windows), an Ampere or newer NVIDIA GPU, and `diffusers >= 0.40`. Multi-GPU (`--gpus` > 1) requires Linux or WSL2.
+- **First run:** Downloads ~90 GB (~68 GB with `--text-encoder-precision fp8`), covering the text encoder, VAEs, and the 16 GB INT8 transformer for the selected task. Triton INT8 kernels compile on first use (~1 min).
+
+### Command Line
+
+```bash
+# Text to video + audio (t2va)
+hyperflow-h3-flash --prompt "A chef slides a steaming bowl of ramen across a wooden counter; rain taps on the window." \
+    --output ramen.mp4
+
+# First keyframe, or first + last keyframe (fl2va)
+hyperflow-h3-flash --prompt "..." --image first.png --last-image last.png --output clip.mp4
+
+# Reference images, videos, and audio clips in prompt order (ref2va)
+hyperflow-h3-flash --prompt "<Picture 1> walks through the scene of <Picture 2> ..." \
+    --reference person.png --reference scene.png --output clip.mp4
+
+# 15 s clip across 4 GPUs; or fp8 text encoder to save ~22 GiB system RAM when offloading
+hyperflow-h3-flash --prompt "..." --num-frames 360 --gpus 4
+hyperflow-h3-flash --prompt "..." --steps 4 --text-encoder-precision fp8
+```
+
+| Option | Values (default) | Description |
+|---|---|---|
+| `--steps` | `4`, `6` (`6`), `8` | `4` for speed, `6` balanced, `8` for quality |
+| `--num-frames` | `124`–`362` (`124`) | 5.2 to 15 s at 24 fps; rounds up to `17n + 5` |
+| `--gpus` | `1`, `2`, `4`, `7`, `8`, `auto` | Context and VAE tile parallelism under `torchrun` |
+| `--offload` | `auto` (`auto`), `none`, `model`, `stream` | `none` needs ≥ 90 GiB VRAM; `model`/`stream` need ≥ 91 GiB RAM (`bf16`) or ≥ 69 GiB (`fp8`) |
+| `--text-encoder-precision` | `bf16` (`bf16`), `fp8` | `fp8` saves ~22 GiB system RAM when offloading (requires `[fp8]`) |
+| `--attention` | `auto` (`auto`), `sdpa`, `fa3` | Uses FlashAttention-3 (`fa3`) on Hopper when `kernels` is installed |
+| `--block-chunk` | `0` (`0`), `16384`, ... | Slice token-wise block ops (`--block-chunk 16384` fits 15 s mixed `ref2va` on 24 GB GPUs) |
+
+### Python API (`load_pipeline`)
+
+```python
+import torch
+from diffusers.utils.export_utils import encode_video
+from hyperflow_h3_flash import load_pipeline
+
+pipe = load_pipeline("t2va")  # "fl2va": pass image=... (and last_image=...); "ref2va": pass references=[...]
+out = pipe(
+    prompt="A chef slides a steaming bowl of ramen across a wooden counter; rain taps on the window.",
+    num_frames=124,
+    num_inference_steps=6,  # 4, 6, or 8
+    generator=torch.Generator().manual_seed(0),
+    output=["videos", "audio", "sampling_rate"],
+)
+encode_video(
+    out["videos"][0],
+    fps=24,
+    output_path="ramen.mp4",
+    audio=out["audio"][0],
+    audio_sample_rate=out["sampling_rate"],
+)
+```
+
+### Standalone Transformer (`AutoModel`)
+
+Each INT8 folder ships `hyperflow_h3_flash_transformer.py` and `hyperflow_h3_flash_int8.py`. `AutoModel` loads the transformer without installing `hyperflow-h3-flash`:
+
+```python
+import torch
+from diffusers import AutoModel
+
+model = AutoModel.from_pretrained(
+    "VideoRebirthAI/hyperflow-h3-flash",
+    subfolder="fl2va_int8_convrot",  # or "ref2va_int8_convrot"
+    trust_remote_code=True,
+    torch_dtype=torch.bfloat16,
+)
+model.pad_lowrank_ranks()  # pad low-rank dimensions to multiples of 64 for aligned GEMMs
+model.to("cuda")
+
+# Optional on Hopper GPUs (requires `pip install kernels`):
+model.set_attention_backend("_flash_3_hub")
+```
+
+> Note: The transformer requires `endpoint_timestep` on every forward pass for two-time `(t, r)` conditioning. Use `load_pipeline` or `hyperflow_h3_flash.flash_blocks` to run the full sampling loop.
+
+---
+
+## Benchmarks and Memory
+
+All benchmarks measure end-to-end generation (text encoding, 6-step transformer, and video and audio VAE decode) at **1344×768 and 6 steps**. Clip lengths: `5.2 s` = 124 frames; `15 s` = 362 frames.
+
+### 1. Consumer RTX GPUs (`5.2 s / 15 s` total time)
+
+| GPU | GGUF `Q4_K_M` (ComfyUI)<br>≥ 64 GB RAM | GGUF `--fast-disk`<br>32 GB PC | GGUF `--fast-disk`<br>16 GB PC | INT8 (`hyperflow-h3-flash`)¹<br>`--offload stream` |
+|---|---|---|---|---|
+| **RTX 5090 (32 GB)** | 99 s / 575 s | 118 s / 618 s | 130 s / 615 s | 92 s / 551 s |
+| **RTX 4090 (24 GB)** | 157 s / 830 s | 151 s / 815 s | 153 s / 812 s | 130 s / 750 s |
+| **RTX 3090 (24 GB)** | 315 s / 1835 s | 318 s / 1869 s | 325 s / 1866 s | 266 s / 1684 s |
+| *Peak VRAM (5.2 s / 15 s)* | *5090: 28 / 27 GiB; 24 GB: 23 / 22 GiB* | *same as ≥ 64 GB* | *same as ≥ 64 GB* | *12.0–12.5 / 21.8–22.4 GiB* |
+
+- **GGUF columns:** Use the 15.7 GB `nvfp4_awq` text encoder. `--fast-disk` streams unresident weights from the OS file page cache instead of pinned RAM, keeping anonymous RSS at ~8–16 GiB.
+- **¹ INT8 column:** Requires ≥ 91 GiB system RAM with the `bf16` text encoder, or ~69 GiB with `--text-encoder-precision fp8`.
+
+**Reference to video (`ref2va`) on one RTX GPU (`5.2 s / 15 s` total time):**
+
+| GPU | INT8, 2 images | INT8, mixed² | GGUF `match`, 2 images | GGUF `max`, 2 images |
+|---|---|---|---|---|
+| **RTX 5090 (32 GB)** | 266 s / 914 s | 503 s / 1318 s | 135 s / 660 s | 295 s / 995 s³ |
+| **RTX 4090 (24 GB)** | 319 s / 1167 s | 630 s / 1702 s⁴ | 194 s / 885 s | 398 s / 1275 s³ |
+| **RTX 3090 (24 GB)** | 726 s / 2763 s | 1441 s / 4033 s⁴ | 401 s / 2059 s | 885 s / 2995 s³ |
+
+- **² Mixed:** One reference image, one 5.2 s reference video with soundtrack, and one audio clip. **³** Start ComfyUI with `--disable-cuda-malloc` for `max` at 15 s. **⁴** Pass `--block-chunk 16384` on 24 GB GPUs for 15 s mixed `ref2va`. On 32 GB / 16 GB PCs with `--fast-disk`, 2-image `ref2v` (`match`) runs in `139 s / 663 s` and `157 s / 684 s` on RTX 5090, `180 s / 866 s` and `178 s / 876 s` on RTX 4090, and `385 s / 2010 s` and `384 s / 2025 s` on RTX 3090.
+
+### 2. Data-Center GPUs (`hyperflow-h3-flash` INT8)
+
+| GPUs | `--offload` | 5.2 s (124 frames) | 15 s (362 frames) | Peak VRAM / GPU (5.2 s / 15 s) | System RAM / GPU |
+|---|---|---|---|---|---|
+| 1× H200 (141 GB) | `none` | 36 s | 229 s | 80 / 91 GiB | 16 GiB |
+| 2× H200 (141 GB) | `none` | 24 s | 119 s | — / 71 GiB | 16 GiB |
+| 4× H200 (141 GB) | `none` | 12.6 s | 66 s | 47 / 59 GiB | 16 GiB |
+| 4× B200 | `none` | 8.7 s | — | 46 GiB | 16 GiB |
+
+The table measures the `hyperflow-h3-flash` package. For real time on 4× B-series GPUs, use [NVIDIA SOL-H3](https://github.com/NVlabs/Sana/tree/sol-engine/models/minimax_h3/HyperFlow-Flash).
+
+---
+
+## Step Schedules and Prompting
+
+### Step Grids
+
+Both partitions and both weight formats share three unshifted sigma grids (`video_shift = 12`, `audio_shift = 3`):
+
+| Steps | Mode | Raw sigmas |
+|---:|---|---|
+| `4` | Speed | `1, 0.681872, 0.205019, 0.036496, 0` |
+| `6` | Balanced (default) | `1, 0.825217, 0.471725, 0.147401, 0.049626, 0.015517, 0` |
+| `8` | Quality | `1, 0.931506, 0.839236, 0.703462, 0.5, 0.296538, 0.160764, 0.068494, 0` |
+
+### Prompts
+
+- Describe the subject, action, setting, camera movement, and sound in one prompt. Do not use a negative prompt (`CFG = 1.0`).
+- For `ref2va`, name inputs in prompt order as `<Picture i>`, `<Video i>`, and `<Audio i>`. Limits: 9 images, 3 videos, 3 audio clips, and 12 references total. Audio references require at least one image or video reference. GGUF runtimes accept image references only; video and audio references require the Python package.
+
+---
+
+## How It Works
+
+1. **Structural pruning and low-rank factorization:** `fl2va` drops 3 of 50 blocks (6, 21, 22) and factors 102 linear layers (13.7B parameters). `ref2va` drops 1 of 50 blocks (9) and factors 114 linear layers (13.9B parameters).
+2. **Rotated W8A8 INT8 (`convrot`):** 180 linear layers are quantized to W8A8 INT8 behind a block-diagonal Hadamard rotation. Kernels dispatch to fused Triton (`sm_80`–`sm_90`, `sm_120`), `cuBLASLt` INT8 (`sm_100`), or BF16 tensor cores (`sm_103` B300).
+3. **Two-time `(t, r)` conditioning:** Every token receives the current noise level `t` and the step endpoint `r`. This enables 4-to-8-step sampling without CFG.
+
+---
+
+## Acknowledgements
+
+Thanks to the MiniMax team for releasing [MiniMax-H3](https://huggingface.co/MiniMaxAI/MiniMax-H3), to [Unsloth](https://huggingface.co/unsloth/MiniMax-H3-GGUF) for the MiniMax-H3 GGUF layout and naming convention, to [`WAS/Minimax-h3_Singularity_v2_pruned`](https://huggingface.co/WAS/Minimax-h3_Singularity_v2_pruned) for pioneering rotated INT8 on MiniMax-H3, and to [stable-diffusion.cpp](https://github.com/leejet/stable-diffusion.cpp), [ComfyUI](https://github.com/Comfy-Org/ComfyUI), and [ComfyUI-GGUF](https://github.com/city96/ComfyUI-GGUF).
+
+## Intended Use and Limitations
+
+- HyperFlow H3 Flash provides fast, reduced-memory joint video and stereo audio generation across data-center and consumer GPUs (4–8 steps, 7.2–16 GB transformer weights).
+- Prompt adherence, supported languages, biases, and failure modes follow the [MiniMax-H3 base model](https://huggingface.co/MiniMaxAI/MiniMax-H3).
+- Every use of these weights is bound by the license terms below, including territorial restrictions and the Acceptable Use Policy.
+
+## License
+
+The transformer weights in this repository (both INT8 convrot and GGUF) are a Model Derivative of MiniMax-H3 and are distributed under the [MiniMax H3 Community License Agreement](LICENSE) with no additional terms. By downloading or using them, you accept that Agreement:
+
+- **Territory:** Licensed worldwide **excluding the European Union, the United Kingdom, the Republic of Korea, and the United States of America**. Individuals and organizations in those jurisdictions must obtain MiniMax's prior authorization before downloading ([application form](https://platform.minimax.io/h3-license); [License Q&A](https://huggingface.co/MiniMaxAI/MiniMax-H3/blob/main/docs/QA-about-License.md)).
+- **Redistribution:** Include a copy of the Agreement and the [`NOTICE`](NOTICE) file, clearly mark modified files, and do not impose additional or different terms (Section III).
+- **AI-Generated Content Disclosure:** Publicly distributed content generated with these weights must be clearly disclosed as AI-generated (Acceptable Use Policy, item 12).
+- **Commercial Use:** Products or services exceeding $20 million USD in annual revenue require prior written authorization from MiniMax (`api@minimax.io`). Products and services built with the weights must prominently display `"MiniMax H3"` on their user interface (Section IV.2).
+- **Hosted Services & Use Restrictions:** Hosted services must implement the safeguards in Section V.5 and bind end users to Section V and the Acceptable Use Policy (Exhibit A). Outputs may not be used to train or improve other AI models.
+- **Upstream Text:** [`LICENSE`](LICENSE) reproduces the August 2, 2026 Agreement text; the version in the [upstream repository](https://huggingface.co/MiniMaxAI/MiniMax-H3/blob/main/LICENSE) governs.
+
+See [`NOTICE`](NOTICE) for the architectural and numerical modifications from MiniMax-H3. The inference code on [GitHub](https://github.com/Video-Rebirth/hyperflow-h3-flash) is licensed separately under Apache-2.0 (ComfyUI custom node under GPL-3.0; `stable-diffusion.cpp` patch under MIT).
+
+HyperFlow H3 Flash is developed by Video Rebirth and is not an official MiniMax product. Powered by MiniMax H3.

fl2va_gguf/minimax_h3_fl2va_hyperflow_flash_14b-Q3_K_M.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f43bf830bb0097046eb6332c7aa95ede02885f0485e989f98031928b7bcd3d06
+size 7215807072

fl2va_gguf/minimax_h3_fl2va_hyperflow_flash_14b-Q4_K_M.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ee251661dffdebfa4a8c2555d2967b642f0efe9f2e350830a6e5c691935e23c3
+size 9053456480

fl2va_gguf/minimax_h3_fl2va_hyperflow_flash_14b-Q4_K_S.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6fc38b0bf62505811bc009cd22345a03f213ee2c8a7ffb342f993036e75024b6
+size 8011480160

fl2va_gguf/minimax_h3_fl2va_hyperflow_flash_14b-Q5_K_M.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a3173754d6b62df6faab50f86b97d7117b525f16db66b5715b33a92e177578e0
+size 10277808224

fl2va_gguf/minimax_h3_fl2va_hyperflow_flash_14b-Q6_K.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9870823daa5238e0e33a1661b82e59f8fad18ed281ae4bc19c5b5945b0556b74
+size 11578681952

fl2va_gguf/minimax_h3_fl2va_hyperflow_flash_14b-Q8_0.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9b7aebc17bd1be998bb82aa65409514c85e97e7b63559b75f206c44db17f7b2b
+size 14929689696

fl2va_int8_convrot/config.json

@@ -0,0 +1,325 @@
+{
+  "_class_name": "HyperFlowH3FlashTransformer3DModel",
+  "_diffusers_version": "0.36.0.dev0",
+  "num_attention_heads": 56,
+  "attention_head_dim": 128,
+  "hidden_size": 5376,
+  "num_layers": 50,
+  "num_refiner_layers": 2,
+  "ffn_dim": 14336,
+  "in_channels": 24,
+  "audio_in_channels": 32,
+  "patch_size": [
+    1,
+    2,
+    2
+  ],
+  "text_dim": 5120,
+  "freq_dim": 256,
+  "time_embed_hidden_dim": 5376,
+  "time_embed_dim": 16,
+  "rope_freq_dim": 16,
+  "rope_theta": 10000.0,
+  "norm_eps": 1e-05,
+  "qk_norm_eps": 1e-05,
+  "final_norm_eps": 1e-05,
+  "hyperflow_gate": 0.25,
+  "lowrank_ranks": {
+    "transformer_blocks.1.attn.to_k": 670,
+    "transformer_blocks.1.attn.to_out.0": 1731,
+    "transformer_blocks.1.attn.to_q": 743,
+    "transformer_blocks.1.attn.to_v": 836,
+    "transformer_blocks.1.ff.net.0.proj": 1642,
+    "transformer_blocks.1.ff.net.2": 2450,
+    "transformer_blocks.10.attn.to_k": 273,
+    "transformer_blocks.10.attn.to_out.0": 388,
+    "transformer_blocks.10.attn.to_q": 265,
+    "transformer_blocks.10.attn.to_v": 448,
+    "transformer_blocks.10.ff.net.0.proj": 335,
+    "transformer_blocks.10.ff.net.2": 1478,
+    "transformer_blocks.11.attn.to_k": 256,
+    "transformer_blocks.11.attn.to_out.0": 410,
+    "transformer_blocks.11.attn.to_q": 264,
+    "transformer_blocks.11.attn.to_v": 445,
+    "transformer_blocks.11.ff.net.0.proj": 361,
+    "transformer_blocks.11.ff.net.2": 456,
+    "transformer_blocks.12.attn.to_k": 145,
+    "transformer_blocks.12.attn.to_out.0": 201,
+    "transformer_blocks.12.attn.to_q": 155,
+    "transformer_blocks.12.attn.to_v": 300,
+    "transformer_blocks.12.ff.net.0.proj": 210,
+    "transformer_blocks.12.ff.net.2": 1082,
+    "transformer_blocks.13.attn.to_k": 559,
+    "transformer_blocks.13.attn.to_out.0": 696,
+    "transformer_blocks.13.attn.to_q": 565,
+    "transformer_blocks.13.attn.to_v": 790,
+    "transformer_blocks.13.ff.net.0.proj": 734,
+    "transformer_blocks.13.ff.net.2": 2,
+    "transformer_blocks.14.attn.to_k": 238,
+    "transformer_blocks.14.attn.to_out.0": 357,
+    "transformer_blocks.14.attn.to_q": 258,
+    "transformer_blocks.14.attn.to_v": 438,
+    "transformer_blocks.14.ff.net.0.proj": 315,
+    "transformer_blocks.14.ff.net.2": 1566,
+    "transformer_blocks.15.attn.to_k": 248,
+    "transformer_blocks.15.attn.to_out.0": 394,
+    "transformer_blocks.15.attn.to_q": 261,
+    "transformer_blocks.15.attn.to_v": 431,
+    "transformer_blocks.15.ff.net.0.proj": 315,
+    "transformer_blocks.15.ff.net.2": 1539,
+    "transformer_blocks.16.attn.to_k": 451,
+    "transformer_blocks.16.attn.to_out.0": 711,
+    "transformer_blocks.16.attn.to_q": 460,
+    "transformer_blocks.16.attn.to_v": 694,
+    "transformer_blocks.16.ff.net.0.proj": 652,
+    "transformer_blocks.16.ff.net.2": 2287,
+    "transformer_blocks.17.attn.to_k": 242,
+    "transformer_blocks.17.attn.to_out.0": 413,
+    "transformer_blocks.17.attn.to_q": 261,
+    "transformer_blocks.17.attn.to_v": 443,
+    "transformer_blocks.17.ff.net.0.proj": 334,
+    "transformer_blocks.17.ff.net.2": 1489,
+    "transformer_blocks.18.attn.to_k": 210,
+    "transformer_blocks.18.attn.to_out.0": 440,
+    "transformer_blocks.18.attn.to_q": 238,
+    "transformer_blocks.18.attn.to_v": 433,
+    "transformer_blocks.18.ff.net.0.proj": 390,
+    "transformer_blocks.18.ff.net.2": 1417,
+    "transformer_blocks.2.attn.to_k": 1551,
+    "transformer_blocks.2.attn.to_out.0": 2092,
+    "transformer_blocks.2.attn.to_q": 1587,
+    "transformer_blocks.2.attn.to_v": 1615,
+    "transformer_blocks.2.ff.net.0.proj": 3137,
+    "transformer_blocks.20.attn.to_k": 814,
+    "transformer_blocks.20.attn.to_out.0": 1209,
+    "transformer_blocks.20.attn.to_q": 831,
+    "transformer_blocks.20.attn.to_v": 1093,
+    "transformer_blocks.20.ff.net.0.proj": 1300,
+    "transformer_blocks.20.ff.net.2": 3061,
+    "transformer_blocks.23.attn.to_k": 70,
+    "transformer_blocks.23.attn.to_out.0": 267,
+    "transformer_blocks.23.attn.to_q": 102,
+    "transformer_blocks.23.attn.to_v": 241,
+    "transformer_blocks.23.ff.net.0.proj": 232,
+    "transformer_blocks.23.ff.net.2": 1121,
+    "transformer_blocks.4.attn.to_k": 182,
+    "transformer_blocks.4.attn.to_out.0": 255,
+    "transformer_blocks.4.attn.to_q": 170,
+    "transformer_blocks.4.attn.to_v": 280,
+    "transformer_blocks.4.ff.net.0.proj": 333,
+    "transformer_blocks.4.ff.net.2": 815,
+    "transformer_blocks.49.ff.net.2": 2932,
+    "transformer_blocks.7.attn.to_k": 186,
+    "transformer_blocks.7.attn.to_out.0": 315,
+    "transformer_blocks.7.attn.to_q": 197,
+    "transformer_blocks.7.attn.to_v": 330,
+    "transformer_blocks.7.ff.net.0.proj": 265,
+    "transformer_blocks.7.ff.net.2": 843,
+    "transformer_blocks.8.attn.to_k": 129,
+    "transformer_blocks.8.attn.to_out.0": 224,
+    "transformer_blocks.8.attn.to_q": 136,
+    "transformer_blocks.8.attn.to_v": 255,
+    "transformer_blocks.8.ff.net.0.proj": 209,
+    "transformer_blocks.8.ff.net.2": 1120,
+    "transformer_blocks.9.attn.to_k": 132,
+    "transformer_blocks.9.attn.to_out.0": 270,
+    "transformer_blocks.9.attn.to_q": 146,
+    "transformer_blocks.9.attn.to_v": 260,
+    "transformer_blocks.9.ff.net.0.proj": 185,
+    "transformer_blocks.9.ff.net.2": 1121
+  },
+  "dropped_blocks": [
+    6,
+    21,
+    22
+  ],
+  "auto_map": {
+    "AutoModel": "hyperflow_h3_flash_transformer.HyperFlowH3FlashTransformer3DModel"
+  },
+  "quantization": {
+    "method": "int8_convrot",
+    "group": 256,
+    "linears": [
+      "transformer_blocks.0.attn.to_k",
+      "transformer_blocks.0.attn.to_out.0",
+      "transformer_blocks.0.attn.to_q",
+      "transformer_blocks.0.attn.to_v",
+      "transformer_blocks.0.ff.net.0.proj",
+      "transformer_blocks.0.ff.net.2",
+      "transformer_blocks.19.attn.to_k",
+      "transformer_blocks.19.attn.to_out.0",
+      "transformer_blocks.19.attn.to_q",
+      "transformer_blocks.19.attn.to_v",
+      "transformer_blocks.19.ff.net.0.proj",
+      "transformer_blocks.19.ff.net.2",
+      "transformer_blocks.2.ff.net.2",
+      "transformer_blocks.24.attn.to_k",
+      "transformer_blocks.24.attn.to_out.0",
+      "transformer_blocks.24.attn.to_q",
+      "transformer_blocks.24.attn.to_v",
+      "transformer_blocks.24.ff.net.0.proj",
+      "transformer_blocks.24.ff.net.2",
+      "transformer_blocks.25.attn.to_k",
+      "transformer_blocks.25.attn.to_out.0",
+      "transformer_blocks.25.attn.to_q",
+      "transformer_blocks.25.attn.to_v",
+      "transformer_blocks.25.ff.net.0.proj",
+      "transformer_blocks.25.ff.net.2",
+      "transformer_blocks.26.attn.to_k",
+      "transformer_blocks.26.attn.to_out.0",
+      "transformer_blocks.26.attn.to_q",
+      "transformer_blocks.26.attn.to_v",
+      "transformer_blocks.26.ff.net.0.proj",
+      "transformer_blocks.26.ff.net.2",
+      "transformer_blocks.27.attn.to_k",
+      "transformer_blocks.27.attn.to_out.0",
+      "transformer_blocks.27.attn.to_q",
+      "transformer_blocks.27.attn.to_v",
+      "transformer_blocks.27.ff.net.0.proj",
+      "transformer_blocks.27.ff.net.2",
+      "transformer_blocks.28.attn.to_k",
+      "transformer_blocks.28.attn.to_out.0",
+      "transformer_blocks.28.attn.to_q",
+      "transformer_blocks.28.attn.to_v",
+      "transformer_blocks.28.ff.net.0.proj",
+      "transformer_blocks.28.ff.net.2",
+      "transformer_blocks.29.attn.to_k",
+      "transformer_blocks.29.attn.to_out.0",
+      "transformer_blocks.29.attn.to_q",
+      "transformer_blocks.29.attn.to_v",
+      "transformer_blocks.29.ff.net.0.proj",
+      "transformer_blocks.29.ff.net.2",
+      "transformer_blocks.3.attn.to_k",
+      "transformer_blocks.3.attn.to_out.0",
+      "transformer_blocks.3.attn.to_q",
+      "transformer_blocks.3.attn.to_v",
+      "transformer_blocks.3.ff.net.0.proj",
+      "transformer_blocks.3.ff.net.2",
+      "transformer_blocks.30.attn.to_k",
+      "transformer_blocks.30.attn.to_out.0",
+      "transformer_blocks.30.attn.to_q",
+      "transformer_blocks.30.attn.to_v",
+      "transformer_blocks.30.ff.net.0.proj",
+      "transformer_blocks.30.ff.net.2",
+      "transformer_blocks.31.attn.to_k",
+      "transformer_blocks.31.attn.to_out.0",
+      "transformer_blocks.31.attn.to_q",
+      "transformer_blocks.31.attn.to_v",
+      "transformer_blocks.31.ff.net.0.proj",
+      "transformer_blocks.31.ff.net.2",
+      "transformer_blocks.32.attn.to_k",
+      "transformer_blocks.32.attn.to_out.0",
+      "transformer_blocks.32.attn.to_q",
+      "transformer_blocks.32.attn.to_v",
+      "transformer_blocks.32.ff.net.0.proj",
+      "transformer_blocks.32.ff.net.2",
+      "transformer_blocks.33.attn.to_k",
+      "transformer_blocks.33.attn.to_out.0",
+      "transformer_blocks.33.attn.to_q",
+      "transformer_blocks.33.attn.to_v",
+      "transformer_blocks.33.ff.net.0.proj",
+      "transformer_blocks.33.ff.net.2",
+      "transformer_blocks.34.attn.to_k",
+      "transformer_blocks.34.attn.to_out.0",
+      "transformer_blocks.34.attn.to_q",
+      "transformer_blocks.34.attn.to_v",
+      "transformer_blocks.34.ff.net.0.proj",
+      "transformer_blocks.34.ff.net.2",
+      "transformer_blocks.35.attn.to_k",
+      "transformer_blocks.35.attn.to_out.0",
+      "transformer_blocks.35.attn.to_q",
+      "transformer_blocks.35.attn.to_v",
+      "transformer_blocks.35.ff.net.0.proj",
+      "transformer_blocks.35.ff.net.2",
+      "transformer_blocks.36.attn.to_k",
+      "transformer_blocks.36.attn.to_out.0",
+      "transformer_blocks.36.attn.to_q",
+      "transformer_blocks.36.attn.to_v",
+      "transformer_blocks.36.ff.net.0.proj",
+      "transformer_blocks.36.ff.net.2",
+      "transformer_blocks.37.attn.to_k",
+      "transformer_blocks.37.attn.to_out.0",
+      "transformer_blocks.37.attn.to_q",
+      "transformer_blocks.37.attn.to_v",
+      "transformer_blocks.37.ff.net.0.proj",
+      "transformer_blocks.37.ff.net.2",
+      "transformer_blocks.38.attn.to_k",
+      "transformer_blocks.38.attn.to_out.0",
+      "transformer_blocks.38.attn.to_q",
+      "transformer_blocks.38.attn.to_v",
+      "transformer_blocks.38.ff.net.0.proj",
+      "transformer_blocks.38.ff.net.2",
+      "transformer_blocks.39.attn.to_k",
+      "transformer_blocks.39.attn.to_out.0",
+      "transformer_blocks.39.attn.to_q",
+      "transformer_blocks.39.attn.to_v",
+      "transformer_blocks.39.ff.net.0.proj",
+      "transformer_blocks.39.ff.net.2",
+      "transformer_blocks.40.attn.to_k",
+      "transformer_blocks.40.attn.to_out.0",
+      "transformer_blocks.40.attn.to_q",
+      "transformer_blocks.40.attn.to_v",
+      "transformer_blocks.40.ff.net.0.proj",
+      "transformer_blocks.40.ff.net.2",
+      "transformer_blocks.41.attn.to_k",
+      "transformer_blocks.41.attn.to_out.0",
+      "transformer_blocks.41.attn.to_q",
+      "transformer_blocks.41.attn.to_v",
+      "transformer_blocks.41.ff.net.0.proj",
+      "transformer_blocks.41.ff.net.2",
+      "transformer_blocks.42.attn.to_k",
+      "transformer_blocks.42.attn.to_out.0",
+      "transformer_blocks.42.attn.to_q",
+      "transformer_blocks.42.attn.to_v",
+      "transformer_blocks.42.ff.net.0.proj",
+      "transformer_blocks.42.ff.net.2",
+      "transformer_blocks.43.attn.to_k",
+      "transformer_blocks.43.attn.to_out.0",
+      "transformer_blocks.43.attn.to_q",
+      "transformer_blocks.43.attn.to_v",
+      "transformer_blocks.43.ff.net.0.proj",
+      "transformer_blocks.43.ff.net.2",
+      "transformer_blocks.44.attn.to_k",
+      "transformer_blocks.44.attn.to_out.0",
+      "transformer_blocks.44.attn.to_q",
+      "transformer_blocks.44.attn.to_v",
+      "transformer_blocks.44.ff.net.0.proj",
+      "transformer_blocks.44.ff.net.2",
+      "transformer_blocks.45.attn.to_k",
+      "transformer_blocks.45.attn.to_out.0",
+      "transformer_blocks.45.attn.to_q",
+      "transformer_blocks.45.attn.to_v",
+      "transformer_blocks.45.ff.net.0.proj",
+      "transformer_blocks.45.ff.net.2",
+      "transformer_blocks.46.attn.to_k",
+      "transformer_blocks.46.attn.to_out.0",
+      "transformer_blocks.46.attn.to_q",
+      "transformer_blocks.46.attn.to_v",
+      "transformer_blocks.46.ff.net.0.proj",
+      "transformer_blocks.46.ff.net.2",
+      "transformer_blocks.47.attn.to_k",
+      "transformer_blocks.47.attn.to_out.0",
+      "transformer_blocks.47.attn.to_q",
+      "transformer_blocks.47.attn.to_v",
+      "transformer_blocks.47.ff.net.0.proj",
+      "transformer_blocks.47.ff.net.2",
+      "transformer_blocks.48.attn.to_k",
+      "transformer_blocks.48.attn.to_out.0",
+      "transformer_blocks.48.attn.to_q",
+      "transformer_blocks.48.attn.to_v",
+      "transformer_blocks.48.ff.net.0.proj",
+      "transformer_blocks.48.ff.net.2",
+      "transformer_blocks.49.attn.to_k",
+      "transformer_blocks.49.attn.to_out.0",
+      "transformer_blocks.49.attn.to_q",
+      "transformer_blocks.49.attn.to_v",
+      "transformer_blocks.49.ff.net.0.proj",
+      "transformer_blocks.5.attn.to_k",
+      "transformer_blocks.5.attn.to_out.0",
+      "transformer_blocks.5.attn.to_q",
+      "transformer_blocks.5.attn.to_v",
+      "transformer_blocks.5.ff.net.0.proj",
+      "transformer_blocks.5.ff.net.2"
+    ]
+  }
+}

fl2va_int8_convrot/diffusion_pytorch_model-00001-of-00003.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:175790b38e171f1b0027b6b7a5cbb9a946a9e98e9a6e9e0f897da681be9e0fa2
+size 5331906296

fl2va_int8_convrot/diffusion_pytorch_model-00002-of-00003.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:47853b57cfc421245d4fa365dafaab474b113c7144a0cef5855849ed4a73d142
+size 5290477648

fl2va_int8_convrot/diffusion_pytorch_model-00003-of-00003.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b5f6516bb503b5ecc64f507037b341de7de42c792904d9f977e3c9c49a11d1c1
+size 5258285344

fl2va_int8_convrot/diffusion_pytorch_model.safetensors.index.json

@@ -0,0 +1,895 @@
+{
+  "metadata": {
+    "total_size": 15880568448
+  },
+  "weight_map": {
+    "audio_proj_in.bias": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "audio_proj_in.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "audio_proj_out.bias": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "audio_proj_out.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "context_embedder.bias": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "context_embedder.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "endpoint_time_embedder.linear_1.bias": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "endpoint_time_embedder.linear_1.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "endpoint_time_embedder.linear_2.bias": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "endpoint_time_embedder.linear_2.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "norm_out.linear.bias": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "norm_out.linear.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "norm_out.norm.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "proj_in.bias": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "proj_in.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "proj_out.bias": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "proj_out.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "time_embedder.linear_1.bias": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "time_embedder.linear_1.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "time_embedder.linear_2.bias": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "time_embedder.linear_2.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "token_refiner.final_norm.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "token_refiner.refiner_blocks.0.attn.norm_k.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "token_refiner.refiner_blocks.0.attn.norm_q.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "token_refiner.refiner_blocks.0.attn.to_k.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "token_refiner.refiner_blocks.0.attn.to_out.0.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "token_refiner.refiner_blocks.0.attn.to_q.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "token_refiner.refiner_blocks.0.attn.to_v.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "token_refiner.refiner_blocks.0.ff.net.0.proj.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "token_refiner.refiner_blocks.0.ff.net.2.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "token_refiner.refiner_blocks.0.norm1.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "token_refiner.refiner_blocks.0.norm2.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "token_refiner.refiner_blocks.1.attn.norm_k.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "token_refiner.refiner_blocks.1.attn.norm_q.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "token_refiner.refiner_blocks.1.attn.to_k.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "token_refiner.refiner_blocks.1.attn.to_out.0.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "token_refiner.refiner_blocks.1.attn.to_q.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "token_refiner.refiner_blocks.1.attn.to_v.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "token_refiner.refiner_blocks.1.ff.net.0.proj.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "token_refiner.refiner_blocks.1.ff.net.2.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "token_refiner.refiner_blocks.1.norm1.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "token_refiner.refiner_blocks.1.norm2.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.0.adaln_proj.linear.bias": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.0.adaln_proj.linear.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.0.attn.norm_k.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.0.attn.norm_q.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.0.attn.to_k.qweight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.0.attn.to_k.wscale": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.0.attn.to_out.0.qweight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.0.attn.to_out.0.wscale": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.0.attn.to_q.qweight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.0.attn.to_q.wscale": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.0.attn.to_v.qweight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.0.attn.to_v.wscale": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.0.ff.net.0.proj.qweight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.0.ff.net.0.proj.wscale": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.0.ff.net.2.qweight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.0.ff.net.2.wscale": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.0.norm1.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.0.norm2.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.1.adaln_proj.linear.bias": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.1.adaln_proj.linear.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.1.attn.norm_k.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.1.attn.norm_q.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.1.attn.to_k.down.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.1.attn.to_k.up.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.1.attn.to_out.0.down.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.1.attn.to_out.0.up.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.1.attn.to_q.down.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.1.attn.to_q.up.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.1.attn.to_v.down.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.1.attn.to_v.up.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.1.ff.net.0.proj.down.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.1.ff.net.0.proj.up.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.1.ff.net.2.down.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.1.ff.net.2.up.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.1.norm1.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.1.norm2.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.10.adaln_proj.linear.bias": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.10.adaln_proj.linear.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.10.attn.norm_k.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.10.attn.norm_q.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.10.attn.to_k.down.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.10.attn.to_k.up.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.10.attn.to_out.0.down.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.10.attn.to_out.0.up.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.10.attn.to_q.down.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.10.attn.to_q.up.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.10.attn.to_v.down.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.10.attn.to_v.up.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.10.ff.net.0.proj.down.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.10.ff.net.0.proj.up.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.10.ff.net.2.down.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.10.ff.net.2.up.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.10.norm1.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.10.norm2.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.11.adaln_proj.linear.bias": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.11.adaln_proj.linear.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.11.attn.norm_k.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.11.attn.norm_q.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.11.attn.to_k.down.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.11.attn.to_k.up.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.11.attn.to_out.0.down.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.11.attn.to_out.0.up.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.11.attn.to_q.down.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.11.attn.to_q.up.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.11.attn.to_v.down.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.11.attn.to_v.up.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.11.ff.net.0.proj.down.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.11.ff.net.0.proj.up.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.11.ff.net.2.down.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.11.ff.net.2.up.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.11.norm1.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.11.norm2.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.12.adaln_proj.linear.bias": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.12.adaln_proj.linear.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.12.attn.norm_k.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.12.attn.norm_q.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.12.attn.to_k.down.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.12.attn.to_k.up.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.12.attn.to_out.0.down.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.12.attn.to_out.0.up.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.12.attn.to_q.down.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.12.attn.to_q.up.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.12.attn.to_v.down.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.12.attn.to_v.up.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.12.ff.net.0.proj.down.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.12.ff.net.0.proj.up.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.12.ff.net.2.down.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.12.ff.net.2.up.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.12.norm1.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.12.norm2.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.13.adaln_proj.linear.bias": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.13.adaln_proj.linear.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.13.attn.norm_k.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.13.attn.norm_q.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.13.attn.to_k.down.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.13.attn.to_k.up.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.13.attn.to_out.0.down.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.13.attn.to_out.0.up.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.13.attn.to_q.down.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.13.attn.to_q.up.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.13.attn.to_v.down.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.13.attn.to_v.up.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.13.ff.net.0.proj.down.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.13.ff.net.0.proj.up.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.13.ff.net.2.down.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.13.ff.net.2.up.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.13.norm1.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.13.norm2.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.14.adaln_proj.linear.bias": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.14.adaln_proj.linear.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.14.attn.norm_k.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.14.attn.norm_q.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.14.attn.to_k.down.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.14.attn.to_k.up.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.14.attn.to_out.0.down.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.14.attn.to_out.0.up.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.14.attn.to_q.down.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.14.attn.to_q.up.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.14.attn.to_v.down.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.14.attn.to_v.up.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.14.ff.net.0.proj.down.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.14.ff.net.0.proj.up.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.14.ff.net.2.down.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.14.ff.net.2.up.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.14.norm1.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.14.norm2.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.15.adaln_proj.linear.bias": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.15.adaln_proj.linear.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.15.attn.norm_k.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.15.attn.norm_q.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.15.attn.to_k.down.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.15.attn.to_k.up.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.15.attn.to_out.0.down.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.15.attn.to_out.0.up.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.15.attn.to_q.down.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.15.attn.to_q.up.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.15.attn.to_v.down.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.15.attn.to_v.up.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.15.ff.net.0.proj.down.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.15.ff.net.0.proj.up.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.15.ff.net.2.down.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.15.ff.net.2.up.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.15.norm1.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.15.norm2.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.16.adaln_proj.linear.bias": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.16.adaln_proj.linear.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.16.attn.norm_k.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.16.attn.norm_q.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.16.attn.to_k.down.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.16.attn.to_k.up.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.16.attn.to_out.0.down.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.16.attn.to_out.0.up.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.16.attn.to_q.down.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.16.attn.to_q.up.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.16.attn.to_v.down.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.16.attn.to_v.up.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.16.ff.net.0.proj.down.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.16.ff.net.0.proj.up.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.16.ff.net.2.down.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.16.ff.net.2.up.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.16.norm1.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.16.norm2.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.17.adaln_proj.linear.bias": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.17.adaln_proj.linear.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.17.attn.norm_k.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.17.attn.norm_q.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.17.attn.to_k.down.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.17.attn.to_k.up.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.17.attn.to_out.0.down.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.17.attn.to_out.0.up.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.17.attn.to_q.down.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.17.attn.to_q.up.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.17.attn.to_v.down.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.17.attn.to_v.up.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.17.ff.net.0.proj.down.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.17.ff.net.0.proj.up.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.17.ff.net.2.down.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.17.ff.net.2.up.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.17.norm1.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.17.norm2.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.18.adaln_proj.linear.bias": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.18.adaln_proj.linear.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.18.attn.norm_k.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.18.attn.norm_q.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.18.attn.to_k.down.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.18.attn.to_k.up.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.18.attn.to_out.0.down.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.18.attn.to_out.0.up.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.18.attn.to_q.down.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.18.attn.to_q.up.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.18.attn.to_v.down.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.18.attn.to_v.up.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.18.ff.net.0.proj.down.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.18.ff.net.0.proj.up.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.18.ff.net.2.down.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.18.ff.net.2.up.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.18.norm1.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.18.norm2.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.19.adaln_proj.linear.bias": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.19.adaln_proj.linear.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.19.attn.norm_k.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.19.attn.norm_q.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.19.attn.to_k.qweight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.19.attn.to_k.wscale": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.19.attn.to_out.0.qweight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.19.attn.to_out.0.wscale": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.19.attn.to_q.qweight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.19.attn.to_q.wscale": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.19.attn.to_v.qweight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.19.attn.to_v.wscale": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.19.ff.net.0.proj.qweight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.19.ff.net.0.proj.wscale": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.19.ff.net.2.qweight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.19.ff.net.2.wscale": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.19.norm1.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.19.norm2.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.2.adaln_proj.linear.bias": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.2.adaln_proj.linear.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.2.attn.norm_k.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.2.attn.norm_q.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.2.attn.to_k.down.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.2.attn.to_k.up.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.2.attn.to_out.0.down.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.2.attn.to_out.0.up.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.2.attn.to_q.down.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.2.attn.to_q.up.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.2.attn.to_v.down.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.2.attn.to_v.up.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.2.ff.net.0.proj.down.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.2.ff.net.0.proj.up.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.2.ff.net.2.qweight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.2.ff.net.2.wscale": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.2.norm1.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.2.norm2.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.20.adaln_proj.linear.bias": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.20.adaln_proj.linear.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.20.attn.norm_k.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.20.attn.norm_q.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.20.attn.to_k.down.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.20.attn.to_k.up.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.20.attn.to_out.0.down.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.20.attn.to_out.0.up.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.20.attn.to_q.down.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.20.attn.to_q.up.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.20.attn.to_v.down.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.20.attn.to_v.up.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.20.ff.net.0.proj.down.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.20.ff.net.0.proj.up.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.20.ff.net.2.down.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.20.ff.net.2.up.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.20.norm1.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.20.norm2.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.23.adaln_proj.linear.bias": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.23.adaln_proj.linear.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.23.attn.norm_k.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.23.attn.norm_q.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.23.attn.to_k.down.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.23.attn.to_k.up.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.23.attn.to_out.0.down.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.23.attn.to_out.0.up.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.23.attn.to_q.down.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.23.attn.to_q.up.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.23.attn.to_v.down.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.23.attn.to_v.up.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.23.ff.net.0.proj.down.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.23.ff.net.0.proj.up.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.23.ff.net.2.down.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.23.ff.net.2.up.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.23.norm1.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.23.norm2.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.24.adaln_proj.linear.bias": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.24.adaln_proj.linear.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.24.attn.norm_k.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.24.attn.norm_q.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.24.attn.to_k.qweight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.24.attn.to_k.wscale": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.24.attn.to_out.0.qweight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.24.attn.to_out.0.wscale": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.24.attn.to_q.qweight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.24.attn.to_q.wscale": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.24.attn.to_v.qweight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.24.attn.to_v.wscale": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.24.ff.net.0.proj.qweight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.24.ff.net.0.proj.wscale": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.24.ff.net.2.qweight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.24.ff.net.2.wscale": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.24.norm1.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.24.norm2.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.25.adaln_proj.linear.bias": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.25.adaln_proj.linear.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.25.attn.norm_k.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.25.attn.norm_q.weight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.25.attn.to_k.qweight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.25.attn.to_k.wscale": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.25.attn.to_out.0.qweight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.25.attn.to_out.0.wscale": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.25.attn.to_q.qweight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.25.attn.to_q.wscale": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.25.attn.to_v.qweight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.25.attn.to_v.wscale": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.25.ff.net.0.proj.qweight": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.25.ff.net.0.proj.wscale": "diffusion_pytorch_model-00001-of-00003.safetensors",
+    "transformer_blocks.25.ff.net.2.qweight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.25.ff.net.2.wscale": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.25.norm1.weight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.25.norm2.weight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.26.adaln_proj.linear.bias": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.26.adaln_proj.linear.weight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.26.attn.norm_k.weight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.26.attn.norm_q.weight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.26.attn.to_k.qweight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.26.attn.to_k.wscale": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.26.attn.to_out.0.qweight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.26.attn.to_out.0.wscale": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.26.attn.to_q.qweight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.26.attn.to_q.wscale": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.26.attn.to_v.qweight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.26.attn.to_v.wscale": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.26.ff.net.0.proj.qweight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.26.ff.net.0.proj.wscale": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.26.ff.net.2.qweight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.26.ff.net.2.wscale": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.26.norm1.weight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.26.norm2.weight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.27.adaln_proj.linear.bias": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.27.adaln_proj.linear.weight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.27.attn.norm_k.weight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.27.attn.norm_q.weight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.27.attn.to_k.qweight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.27.attn.to_k.wscale": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.27.attn.to_out.0.qweight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.27.attn.to_out.0.wscale": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.27.attn.to_q.qweight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.27.attn.to_q.wscale": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.27.attn.to_v.qweight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.27.attn.to_v.wscale": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.27.ff.net.0.proj.qweight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.27.ff.net.0.proj.wscale": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.27.ff.net.2.qweight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.27.ff.net.2.wscale": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.27.norm1.weight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.27.norm2.weight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.28.adaln_proj.linear.bias": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.28.adaln_proj.linear.weight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.28.attn.norm_k.weight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.28.attn.norm_q.weight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.28.attn.to_k.qweight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.28.attn.to_k.wscale": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.28.attn.to_out.0.qweight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.28.attn.to_out.0.wscale": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.28.attn.to_q.qweight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.28.attn.to_q.wscale": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.28.attn.to_v.qweight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.28.attn.to_v.wscale": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.28.ff.net.0.proj.qweight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.28.ff.net.0.proj.wscale": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.28.ff.net.2.qweight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.28.ff.net.2.wscale": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.28.norm1.weight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.28.norm2.weight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.29.adaln_proj.linear.bias": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.29.adaln_proj.linear.weight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.29.attn.norm_k.weight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.29.attn.norm_q.weight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.29.attn.to_k.qweight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.29.attn.to_k.wscale": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.29.attn.to_out.0.qweight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.29.attn.to_out.0.wscale": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.29.attn.to_q.qweight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.29.attn.to_q.wscale": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.29.attn.to_v.qweight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.29.attn.to_v.wscale": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.29.ff.net.0.proj.qweight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.29.ff.net.0.proj.wscale": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.29.ff.net.2.qweight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.29.ff.net.2.wscale": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.29.norm1.weight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.29.norm2.weight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.3.adaln_proj.linear.bias": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.3.adaln_proj.linear.weight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.3.attn.norm_k.weight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.3.attn.norm_q.weight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.3.attn.to_k.qweight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.3.attn.to_k.wscale": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.3.attn.to_out.0.qweight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.3.attn.to_out.0.wscale": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.3.attn.to_q.qweight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.3.attn.to_q.wscale": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.3.attn.to_v.qweight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.3.attn.to_v.wscale": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.3.ff.net.0.proj.qweight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.3.ff.net.0.proj.wscale": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.3.ff.net.2.qweight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.3.ff.net.2.wscale": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.3.norm1.weight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.3.norm2.weight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.30.adaln_proj.linear.bias": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.30.adaln_proj.linear.weight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.30.attn.norm_k.weight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.30.attn.norm_q.weight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.30.attn.to_k.qweight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.30.attn.to_k.wscale": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.30.attn.to_out.0.qweight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.30.attn.to_out.0.wscale": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.30.attn.to_q.qweight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.30.attn.to_q.wscale": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.30.attn.to_v.qweight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.30.attn.to_v.wscale": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.30.ff.net.0.proj.qweight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.30.ff.net.0.proj.wscale": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.30.ff.net.2.qweight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.30.ff.net.2.wscale": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.30.norm1.weight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.30.norm2.weight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.31.adaln_proj.linear.bias": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.31.adaln_proj.linear.weight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.31.attn.norm_k.weight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.31.attn.norm_q.weight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.31.attn.to_k.qweight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.31.attn.to_k.wscale": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.31.attn.to_out.0.qweight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.31.attn.to_out.0.wscale": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.31.attn.to_q.qweight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.31.attn.to_q.wscale": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.31.attn.to_v.qweight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.31.attn.to_v.wscale": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.31.ff.net.0.proj.qweight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.31.ff.net.0.proj.wscale": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.31.ff.net.2.qweight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.31.ff.net.2.wscale": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.31.norm1.weight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.31.norm2.weight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.32.adaln_proj.linear.bias": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.32.adaln_proj.linear.weight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.32.attn.norm_k.weight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.32.attn.norm_q.weight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.32.attn.to_k.qweight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.32.attn.to_k.wscale": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.32.attn.to_out.0.qweight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.32.attn.to_out.0.wscale": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.32.attn.to_q.qweight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.32.attn.to_q.wscale": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.32.attn.to_v.qweight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.32.attn.to_v.wscale": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.32.ff.net.0.proj.qweight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.32.ff.net.0.proj.wscale": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.32.ff.net.2.qweight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.32.ff.net.2.wscale": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.32.norm1.weight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.32.norm2.weight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.33.adaln_proj.linear.bias": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.33.adaln_proj.linear.weight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.33.attn.norm_k.weight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.33.attn.norm_q.weight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.33.attn.to_k.qweight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.33.attn.to_k.wscale": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.33.attn.to_out.0.qweight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.33.attn.to_out.0.wscale": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.33.attn.to_q.qweight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.33.attn.to_q.wscale": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.33.attn.to_v.qweight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.33.attn.to_v.wscale": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.33.ff.net.0.proj.qweight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.33.ff.net.0.proj.wscale": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.33.ff.net.2.qweight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.33.ff.net.2.wscale": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.33.norm1.weight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.33.norm2.weight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.34.adaln_proj.linear.bias": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.34.adaln_proj.linear.weight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.34.attn.norm_k.weight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.34.attn.norm_q.weight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.34.attn.to_k.qweight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.34.attn.to_k.wscale": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.34.attn.to_out.0.qweight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.34.attn.to_out.0.wscale": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.34.attn.to_q.qweight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.34.attn.to_q.wscale": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.34.attn.to_v.qweight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.34.attn.to_v.wscale": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.34.ff.net.0.proj.qweight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.34.ff.net.0.proj.wscale": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.34.ff.net.2.qweight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.34.ff.net.2.wscale": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.34.norm1.weight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.34.norm2.weight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.35.adaln_proj.linear.bias": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.35.adaln_proj.linear.weight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.35.attn.norm_k.weight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.35.attn.norm_q.weight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.35.attn.to_k.qweight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.35.attn.to_k.wscale": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.35.attn.to_out.0.qweight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.35.attn.to_out.0.wscale": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.35.attn.to_q.qweight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.35.attn.to_q.wscale": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.35.attn.to_v.qweight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.35.attn.to_v.wscale": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.35.ff.net.0.proj.qweight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.35.ff.net.0.proj.wscale": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.35.ff.net.2.qweight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.35.ff.net.2.wscale": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.35.norm1.weight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.35.norm2.weight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.36.adaln_proj.linear.bias": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.36.adaln_proj.linear.weight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.36.attn.norm_k.weight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.36.attn.norm_q.weight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.36.attn.to_k.qweight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.36.attn.to_k.wscale": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.36.attn.to_out.0.qweight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.36.attn.to_out.0.wscale": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.36.attn.to_q.qweight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.36.attn.to_q.wscale": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.36.attn.to_v.qweight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.36.attn.to_v.wscale": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.36.ff.net.0.proj.qweight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.36.ff.net.0.proj.wscale": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.36.ff.net.2.qweight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.36.ff.net.2.wscale": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.36.norm1.weight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.36.norm2.weight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.37.adaln_proj.linear.bias": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.37.adaln_proj.linear.weight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.37.attn.norm_k.weight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.37.attn.norm_q.weight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.37.attn.to_k.qweight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.37.attn.to_k.wscale": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.37.attn.to_out.0.qweight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.37.attn.to_out.0.wscale": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.37.attn.to_q.qweight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.37.attn.to_q.wscale": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.37.attn.to_v.qweight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.37.attn.to_v.wscale": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.37.ff.net.0.proj.qweight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.37.ff.net.0.proj.wscale": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.37.ff.net.2.qweight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.37.ff.net.2.wscale": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.37.norm1.weight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.37.norm2.weight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.38.adaln_proj.linear.bias": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.38.adaln_proj.linear.weight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.38.attn.norm_k.weight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.38.attn.norm_q.weight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.38.attn.to_k.qweight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.38.attn.to_k.wscale": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.38.attn.to_out.0.qweight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.38.attn.to_out.0.wscale": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.38.attn.to_q.qweight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.38.attn.to_q.wscale": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.38.attn.to_v.qweight": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.38.attn.to_v.wscale": "diffusion_pytorch_model-00002-of-00003.safetensors",
+    "transformer_blocks.38.ff.net.0.proj.qweight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.38.ff.net.0.proj.wscale": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.38.ff.net.2.qweight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.38.ff.net.2.wscale": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.38.norm1.weight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.38.norm2.weight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.39.adaln_proj.linear.bias": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.39.adaln_proj.linear.weight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.39.attn.norm_k.weight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.39.attn.norm_q.weight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.39.attn.to_k.qweight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.39.attn.to_k.wscale": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.39.attn.to_out.0.qweight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.39.attn.to_out.0.wscale": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.39.attn.to_q.qweight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.39.attn.to_q.wscale": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.39.attn.to_v.qweight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.39.attn.to_v.wscale": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.39.ff.net.0.proj.qweight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.39.ff.net.0.proj.wscale": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.39.ff.net.2.qweight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.39.ff.net.2.wscale": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.39.norm1.weight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.39.norm2.weight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.4.adaln_proj.linear.bias": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.4.adaln_proj.linear.weight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.4.attn.norm_k.weight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.4.attn.norm_q.weight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.4.attn.to_k.down.weight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.4.attn.to_k.up.weight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.4.attn.to_out.0.down.weight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.4.attn.to_out.0.up.weight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.4.attn.to_q.down.weight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.4.attn.to_q.up.weight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.4.attn.to_v.down.weight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.4.attn.to_v.up.weight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.4.ff.net.0.proj.down.weight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.4.ff.net.0.proj.up.weight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.4.ff.net.2.down.weight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.4.ff.net.2.up.weight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.4.norm1.weight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.4.norm2.weight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.40.adaln_proj.linear.bias": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.40.adaln_proj.linear.weight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.40.attn.norm_k.weight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.40.attn.norm_q.weight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.40.attn.to_k.qweight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.40.attn.to_k.wscale": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.40.attn.to_out.0.qweight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.40.attn.to_out.0.wscale": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.40.attn.to_q.qweight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.40.attn.to_q.wscale": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.40.attn.to_v.qweight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.40.attn.to_v.wscale": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.40.ff.net.0.proj.qweight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.40.ff.net.0.proj.wscale": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.40.ff.net.2.qweight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.40.ff.net.2.wscale": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.40.norm1.weight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.40.norm2.weight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.41.adaln_proj.linear.bias": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.41.adaln_proj.linear.weight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.41.attn.norm_k.weight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.41.attn.norm_q.weight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.41.attn.to_k.qweight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.41.attn.to_k.wscale": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.41.attn.to_out.0.qweight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.41.attn.to_out.0.wscale": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.41.attn.to_q.qweight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.41.attn.to_q.wscale": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.41.attn.to_v.qweight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.41.attn.to_v.wscale": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.41.ff.net.0.proj.qweight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.41.ff.net.0.proj.wscale": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.41.ff.net.2.qweight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.41.ff.net.2.wscale": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.41.norm1.weight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.41.norm2.weight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.42.adaln_proj.linear.bias": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.42.adaln_proj.linear.weight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.42.attn.norm_k.weight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.42.attn.norm_q.weight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.42.attn.to_k.qweight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.42.attn.to_k.wscale": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.42.attn.to_out.0.qweight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.42.attn.to_out.0.wscale": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.42.attn.to_q.qweight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.42.attn.to_q.wscale": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.42.attn.to_v.qweight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.42.attn.to_v.wscale": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.42.ff.net.0.proj.qweight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.42.ff.net.0.proj.wscale": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.42.ff.net.2.qweight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.42.ff.net.2.wscale": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.42.norm1.weight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.42.norm2.weight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.43.adaln_proj.linear.bias": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.43.adaln_proj.linear.weight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.43.attn.norm_k.weight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.43.attn.norm_q.weight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.43.attn.to_k.qweight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.43.attn.to_k.wscale": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.43.attn.to_out.0.qweight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.43.attn.to_out.0.wscale": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.43.attn.to_q.qweight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.43.attn.to_q.wscale": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.43.attn.to_v.qweight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.43.attn.to_v.wscale": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.43.ff.net.0.proj.qweight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.43.ff.net.0.proj.wscale": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.43.ff.net.2.qweight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.43.ff.net.2.wscale": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.43.norm1.weight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.43.norm2.weight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.44.adaln_proj.linear.bias": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.44.adaln_proj.linear.weight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.44.attn.norm_k.weight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.44.attn.norm_q.weight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.44.attn.to_k.qweight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.44.attn.to_k.wscale": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.44.attn.to_out.0.qweight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.44.attn.to_out.0.wscale": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.44.attn.to_q.qweight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.44.attn.to_q.wscale": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.44.attn.to_v.qweight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.44.attn.to_v.wscale": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.44.ff.net.0.proj.qweight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.44.ff.net.0.proj.wscale": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.44.ff.net.2.qweight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.44.ff.net.2.wscale": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.44.norm1.weight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.44.norm2.weight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.45.adaln_proj.linear.bias": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.45.adaln_proj.linear.weight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.45.attn.norm_k.weight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.45.attn.norm_q.weight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.45.attn.to_k.qweight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.45.attn.to_k.wscale": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.45.attn.to_out.0.qweight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.45.attn.to_out.0.wscale": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.45.attn.to_q.qweight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.45.attn.to_q.wscale": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.45.attn.to_v.qweight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.45.attn.to_v.wscale": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.45.ff.net.0.proj.qweight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.45.ff.net.0.proj.wscale": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.45.ff.net.2.qweight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.45.ff.net.2.wscale": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.45.norm1.weight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.45.norm2.weight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.46.adaln_proj.linear.bias": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.46.adaln_proj.linear.weight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.46.attn.norm_k.weight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.46.attn.norm_q.weight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.46.attn.to_k.qweight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.46.attn.to_k.wscale": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.46.attn.to_out.0.qweight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.46.attn.to_out.0.wscale": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.46.attn.to_q.qweight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.46.attn.to_q.wscale": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.46.attn.to_v.qweight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.46.attn.to_v.wscale": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.46.ff.net.0.proj.qweight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.46.ff.net.0.proj.wscale": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.46.ff.net.2.qweight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.46.ff.net.2.wscale": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.46.norm1.weight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.46.norm2.weight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.47.adaln_proj.linear.bias": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.47.adaln_proj.linear.weight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.47.attn.norm_k.weight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.47.attn.norm_q.weight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.47.attn.to_k.qweight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.47.attn.to_k.wscale": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.47.attn.to_out.0.qweight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.47.attn.to_out.0.wscale": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.47.attn.to_q.qweight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.47.attn.to_q.wscale": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.47.attn.to_v.qweight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.47.attn.to_v.wscale": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.47.ff.net.0.proj.qweight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.47.ff.net.0.proj.wscale": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.47.ff.net.2.qweight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.47.ff.net.2.wscale": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.47.norm1.weight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.47.norm2.weight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.48.adaln_proj.linear.bias": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.48.adaln_proj.linear.weight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.48.attn.norm_k.weight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.48.attn.norm_q.weight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.48.attn.to_k.qweight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.48.attn.to_k.wscale": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.48.attn.to_out.0.qweight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.48.attn.to_out.0.wscale": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.48.attn.to_q.qweight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.48.attn.to_q.wscale": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.48.attn.to_v.qweight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.48.attn.to_v.wscale": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.48.ff.net.0.proj.qweight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.48.ff.net.0.proj.wscale": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.48.ff.net.2.qweight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.48.ff.net.2.wscale": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.48.norm1.weight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.48.norm2.weight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.49.adaln_proj.linear.bias": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.49.adaln_proj.linear.weight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.49.attn.norm_k.weight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.49.attn.norm_q.weight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.49.attn.to_k.qweight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.49.attn.to_k.wscale": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.49.attn.to_out.0.qweight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.49.attn.to_out.0.wscale": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.49.attn.to_q.qweight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.49.attn.to_q.wscale": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.49.attn.to_v.qweight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.49.attn.to_v.wscale": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.49.ff.net.0.proj.qweight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.49.ff.net.0.proj.wscale": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.49.ff.net.2.down.weight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.49.ff.net.2.up.weight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.49.norm1.weight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.49.norm2.weight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.5.adaln_proj.linear.bias": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.5.adaln_proj.linear.weight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.5.attn.norm_k.weight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.5.attn.norm_q.weight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.5.attn.to_k.qweight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.5.attn.to_k.wscale": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.5.attn.to_out.0.qweight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.5.attn.to_out.0.wscale": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.5.attn.to_q.qweight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.5.attn.to_q.wscale": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.5.attn.to_v.qweight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.5.attn.to_v.wscale": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.5.ff.net.0.proj.qweight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.5.ff.net.0.proj.wscale": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.5.ff.net.2.qweight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.5.ff.net.2.wscale": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.5.norm1.weight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.5.norm2.weight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.7.adaln_proj.linear.bias": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.7.adaln_proj.linear.weight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.7.attn.norm_k.weight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.7.attn.norm_q.weight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.7.attn.to_k.down.weight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.7.attn.to_k.up.weight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.7.attn.to_out.0.down.weight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.7.attn.to_out.0.up.weight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.7.attn.to_q.down.weight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.7.attn.to_q.up.weight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.7.attn.to_v.down.weight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.7.attn.to_v.up.weight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.7.ff.net.0.proj.down.weight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.7.ff.net.0.proj.up.weight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.7.ff.net.2.down.weight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.7.ff.net.2.up.weight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.7.norm1.weight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.7.norm2.weight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.8.adaln_proj.linear.bias": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.8.adaln_proj.linear.weight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.8.attn.norm_k.weight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.8.attn.norm_q.weight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.8.attn.to_k.down.weight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.8.attn.to_k.up.weight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.8.attn.to_out.0.down.weight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.8.attn.to_out.0.up.weight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.8.attn.to_q.down.weight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.8.attn.to_q.up.weight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.8.attn.to_v.down.weight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.8.attn.to_v.up.weight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.8.ff.net.0.proj.down.weight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.8.ff.net.0.proj.up.weight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.8.ff.net.2.down.weight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.8.ff.net.2.up.weight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.8.norm1.weight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.8.norm2.weight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.9.adaln_proj.linear.bias": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.9.adaln_proj.linear.weight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.9.attn.norm_k.weight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.9.attn.norm_q.weight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.9.attn.to_k.down.weight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.9.attn.to_k.up.weight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.9.attn.to_out.0.down.weight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.9.attn.to_out.0.up.weight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.9.attn.to_q.down.weight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.9.attn.to_q.up.weight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.9.attn.to_v.down.weight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.9.attn.to_v.up.weight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.9.ff.net.0.proj.down.weight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.9.ff.net.0.proj.up.weight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.9.ff.net.2.down.weight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.9.ff.net.2.up.weight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.9.norm1.weight": "diffusion_pytorch_model-00003-of-00003.safetensors",
+    "transformer_blocks.9.norm2.weight": "diffusion_pytorch_model-00003-of-00003.safetensors"
+  }
+}

fl2va_int8_convrot/hyperflow_h3_flash_int8.py

@@ -0,0 +1,410 @@
+# Copyright 2026 The HyperFlow authors. Licensed under the Apache License, Version 2.0.
+"""INT8 W8A8 linears with a Hadamard-rotated input dimension.
+
+A dense ``y = x @ W^T`` is stored as ``qweight = round(W R / wscale)`` (int8, one fp32 scale per output channel), where
+``R`` is a block-diagonal regular Hadamard of size ``group``. ``R`` is orthonormal, so ``y = (x R) @ (W R)^T``: at run
+time the activation is rotated and quantized per token, multiplied in int8 with an exact int32 accumulator, and
+rescaled. The rotation spreads outlier channels across the group, which is what makes per-token int8 activations work.
+
+On CUDA the rotation + quantization and the GEMM are Triton kernels (compiled on first use). Every kernel fits the
+99 KiB of shared memory per block of Ampere, Ada and consumer Blackwell GPUs. The int8 GEMM runs as a Triton kernel or
+as cuBLASLt (``torch._int_mm``) plus a Triton epilogue; both use the same int32 accumulator and the same fp32 multiply
+order, so they give bitwise identical outputs. cuBLASLt is faster from compute capability 10 on, Triton on Hopper.
+The B300 (compute capability 10.3) has INT8 tensor cores at about 1/20 of the B200's, so there the int8 codes are
+multiplied on bf16 tensor cores into an fp32 accumulator: products are exact, and the sum matches the int32 one while
+it stays below 2**24, which the rotated activations keep well clear of.
+Elsewhere a plain torch implementation of the same scheme runs.
+
+`KERNEL_OPTIONS` (set by `load_pipeline`, no weight change) picks the kernels at run time:
+
+* ``int8_gemm``: ``"auto"`` (the choice above), ``"triton"``, ``"cublas"``, ``"bf16"``, or ``"sgl"``: CUTLASS
+  ``sgl_kernel.int8_scaled_mm`` with the scales in its epilogue, within 1 ulp of Triton and 1.25 to 1.45x faster on
+  H200 (Triton when sgl_kernel is missing).
+* ``rotate_quant``: ``"dense"`` (the kernel above) or ``"factored"``: a 256-channel group rotated as ``H16 X H16``, two
+  K = 16 dots in 8.5 KiB of shared memory, 1.6 to 2.0x faster on H200; it rounds the rotated activation like the dense
+  kernel up to rare one-code differences.
+"""
+
+from __future__ import annotations
+
+import functools
+from types import SimpleNamespace
+from typing import Any, ClassVar
+
+import torch
+from torch import Tensor, nn
+
+#: Run-time kernel choices.
+INT8_GEMMS = ("auto", "triton", "cublas", "sgl", "bf16")
+ROTATE_QUANTS = ("dense", "factored")
+
+#: Rows below this go through the torch path: `torch._int_mm` needs more than 16 rows, and so few rows are not worth
+#: a kernel launch.
+_MIN_KERNEL_ROWS = 17
+
+#: Run-time kernel choices for every `Int8RotLinear`; `load_pipeline` sets them.
+KERNEL_OPTIONS: dict[str, Any] = {"int8_gemm": "auto", "rotate_quant": "dense"}
+
+
+def set_kernel_options(*, int8_gemm: str | None = None, rotate_quant: str | None = None) -> None:
+    if int8_gemm is not None:
+        if int8_gemm not in INT8_GEMMS:
+            raise ValueError(f"int8_gemm must be one of {INT8_GEMMS}, got {int8_gemm!r}")
+        KERNEL_OPTIONS["int8_gemm"] = int8_gemm
+    if rotate_quant is not None:
+        if rotate_quant not in ROTATE_QUANTS:
+            raise ValueError(f"rotate_quant must be one of {ROTATE_QUANTS}, got {rotate_quant!r}")
+        KERNEL_OPTIONS["rotate_quant"] = rotate_quant
+
+
+@functools.cache
+def _sgl_int8_scaled_mm():
+    """sgl_kernel's ``int8_scaled_mm``, or None when sgl_kernel is not importable."""
+    try:
+        from sgl_kernel import int8_scaled_mm
+    except ImportError:
+        return None
+    return int8_scaled_mm
+
+
+def _has_bf16_mm() -> bool:
+    # `torch.mm(..., out_dtype=)` is PyTorch 2.8+.
+    return tuple(int(v) for v in torch.__version__.split(".")[:2]) >= (2, 8)
+
+
+@functools.cache
+def _auto_int8_gemm(device: torch.device) -> str:
+    capability = torch.cuda.get_device_capability(device)
+    if capability == (10, 3) and _has_bf16_mm():
+        return "bf16"
+    if hasattr(torch, "_int_mm") and capability[0] >= 10:
+        return "cublas"
+    return "triton"
+
+
+def resolve_int8_gemm(device: torch.device) -> str:
+    """The GEMM ``KERNEL_OPTIONS["int8_gemm"]`` runs as on ``device``: ``"triton"``, ``"cublas"``, ``"bf16"`` or
+    ``"sgl"``."""
+    choice = KERNEL_OPTIONS["int8_gemm"]
+    if choice == "auto":
+        return _auto_int8_gemm(device)
+    if (
+        (choice == "sgl" and _sgl_int8_scaled_mm() is None)
+        or (choice == "cublas" and not hasattr(torch, "_int_mm"))
+        or (choice == "bf16" and not _has_bf16_mm())
+    ):
+        return "triton"
+    return choice
+
+
+@functools.cache
+def regular_hadamard(group: int) -> Tensor:
+    """``kron(H4 / 2, ..., H4 / 2)`` of size ``group`` (a power of 4): orthonormal, symmetric, rows sum to one."""
+    h4 = torch.tensor([[1, 1, 1, -1], [1, 1, -1, 1], [1, -1, 1, 1], [-1, 1, 1, 1]], dtype=torch.float64) / 2
+    h, n = torch.ones(1, 1, dtype=torch.float64), 1
+    while n < group:
+        h, n = torch.kron(h, h4), n * 4
+    if n != group:
+        raise ValueError(f"Hadamard group {group} is not a power of 4")
+    return h.float()
+
+
+def hadamard_factors(group: int) -> tuple[int, int] | None:
+    """``(GA, GB)`` with ``regular_hadamard(group) == kron(regular_hadamard(GA), regular_hadamard(GB))``, both at
+    least 16 (the smallest bf16 ``tl.dot`` K); None when ``group`` cannot be split that way."""
+    n = group.bit_length() - 1
+    if group < 256 or group != 1 << n or n % 2:
+        return None
+    ga = 4 ** (n // 4)
+    return ga, group // ga
+
+
+@functools.cache
+def _rotation(group: int, device: torch.device) -> Tensor:
+    return regular_hadamard(group).to(device=device, dtype=torch.bfloat16)
+
+
+class Int8RotLinear(nn.Module):
+    """Bias-free W8A8 linear: ``y = (q(x R) @ qweight^T) * s_x * wscale``, ``R = blockdiag(regular Hadamard)``.
+
+    Linears that read one input (``to_q``, ``to_k``, ``to_v``) quantize it once: a layer with ``keep_input = True``
+    leaves its quantized input for the next layer, which reuses it if it receives the exact same, unmodified tensor.
+    Any other layer clears it, so nothing outlives the projections.
+    """
+
+    _last_input: ClassVar[dict[str, Any]] = {}
+
+    def __init__(self, in_features: int, out_features: int, group: int) -> None:
+        super().__init__()
+        if in_features % group:
+            raise ValueError(f"in_features {in_features} is not a multiple of the Hadamard group {group}")
+        self.in_features, self.out_features, self.group = in_features, out_features, group
+        self.keep_input = False
+        # Parameters, not buffers: diffusers' streamed group offloading moves module buffers on but never back off.
+        self.qweight = nn.Parameter(torch.zeros(out_features, in_features, dtype=torch.int8), requires_grad=False)
+        self.wscale = nn.Parameter(torch.ones(out_features, dtype=torch.float32), requires_grad=False)
+
+    def extra_repr(self) -> str:
+        return f"in_features={self.in_features}, out_features={self.out_features}, group={self.group}"
+
+    def _quantize(self, x: Tensor, x2: Tensor, kernels: SimpleNamespace) -> tuple[Tensor, Tensor]:
+        cache = Int8RotLinear._last_input
+        # Inference tensors (`torch.inference_mode`) have no version counter.
+        version = None if x.is_inference() else x._version
+        hit = cache.get("x") is x and cache.get("version") == version and cache.get("group") == self.group
+        quantized = cache["value"] if hit else kernels.rotate_quant(x2, self.group)
+        cache.clear()
+        if self.keep_input:
+            cache.update(x=x, version=version, group=self.group, value=quantized)
+        return quantized
+
+    def forward(self, x: Tensor) -> Tensor:
+        shape = x.shape
+        x2 = x.reshape(-1, self.in_features).to(torch.bfloat16).contiguous()
+        kernels = _kernels() if x2.is_cuda and x2.shape[0] >= _MIN_KERNEL_ROWS else None
+        if kernels is None:
+            Int8RotLinear._last_input.clear()
+            y = self._forward_torch(x2, _rotation(self.group, x2.device))
+        else:
+            xq, sx = self._quantize(x, x2, kernels)
+            y = kernels.int8_matmul(xq, self.qweight, sx, self.wscale)
+        return y.reshape(*shape[:-1], self.out_features).to(x.dtype)
+
+    def _forward_torch(self, x2: Tensor, rot: Tensor) -> Tensor:
+        """The same scheme in plain torch; the rotated activation is rounded to bf16 before quantization."""
+        g = self.group
+        xr = (x2.reshape(-1, self.in_features // g, g) @ rot).reshape(x2.shape).float()
+        sx = xr.abs().amax(1, keepdim=True).clamp_min(1e-12) / 127.0
+        xq = (xr / sx).round().clamp(-127, 127)
+        return ((xq @ self.qweight.float().t()) * sx * self.wscale[None, :]).to(torch.bfloat16)
+
+
+@functools.cache
+def _kernels() -> SimpleNamespace | None:
+    """``rotate_quant`` and ``int8_matmul`` as Triton kernels; ``None`` when Triton is not installed."""
+    try:
+        import triton
+        import triton.language as tl
+    except ImportError:
+        return None
+
+    ROTATE_BM = 64
+    FACTORED_BM = 16
+
+    @triton.jit
+    def rotate_quant_kernel(
+        x_ptr, h_ptr, q_ptr, s_ptr, M, K, stride_xm, stride_qm,
+        G: tl.constexpr, BM: tl.constexpr, BN: tl.constexpr,
+    ):  # fmt: skip
+        # Two passes over the row block: the per-token scale needs the absmax of the whole rotated row first.
+        # The Hadamard streams through in G x BN column tiles, so shared memory is (BM * G + G * BN) * 2 bytes.
+        rows = tl.program_id(0) * BM + tl.arange(0, BM)
+        rows64 = rows.to(tl.int64)
+        cols = tl.arange(0, G)
+        tile = tl.arange(0, BN)
+        rmask = rows < M
+        amax = tl.zeros((BM,), dtype=tl.float32)
+        for g in range(0, K, G):
+            x = tl.load(x_ptr + rows64[:, None] * stride_xm + (g + cols)[None, :], mask=rmask[:, None], other=0.0)
+            for n in range(0, G, BN):
+                h = tl.load(h_ptr + cols[:, None] * G + (n + tile)[None, :])
+                amax = tl.maximum(amax, tl.max(tl.abs(tl.dot(x, h)), axis=1))
+        scale = tl.maximum(amax, 1e-12) / 127.0
+        tl.store(s_ptr + rows, scale, mask=rmask)
+        inv = 1.0 / scale
+        for g in range(0, K, G):
+            x = tl.load(x_ptr + rows64[:, None] * stride_xm + (g + cols)[None, :], mask=rmask[:, None], other=0.0)
+            for n in range(0, G, BN):
+                h = tl.load(h_ptr + cols[:, None] * G + (n + tile)[None, :])
+                q = tl.extra.cuda.libdevice.rint(tl.dot(x, h) * inv[:, None])
+                q = tl.minimum(tl.maximum(q, -127.0), 127.0).to(tl.int8)
+                tl.store(q_ptr + rows64[:, None] * stride_qm + (g + n + tile)[None, :], q, mask=rmask[:, None])
+
+    @triton.jit
+    def rotate_quant_factored_kernel(
+        x_ptr, ha_ptr, hb_ptr, q_ptr, s_ptr, M, K, stride_xm, stride_qm,
+        GA: tl.constexpr, GB: tl.constexpr, BM: tl.constexpr,
+    ):  # fmt: skip
+        # Channel g + a * GB + b of a group is X[a, b]; the rotated group is H_GA X H_GB. Pass 0 takes the row absmax,
+        # pass 1 rotates again and writes int8. The fp32 input of the second dot is split into three bf16 terms
+        # (8 + 8 + 8 significand bits), so it is exact; a tf32 dot or a two-term split rounds it measurably.
+        # The second dot is batched per row, so its result is already in channel order.
+        G: tl.constexpr = GA * GB
+        rows = tl.program_id(0) * BM + tl.arange(0, BM)
+        rows64 = rows.to(tl.int64)
+        cols = tl.arange(0, G)
+        rmask = rows < M
+        ia = tl.arange(0, GA)
+        ib = tl.arange(0, GB)
+        ha = tl.broadcast_to(tl.load(ha_ptr + ia[:, None] * GA + ia[None, :])[None, :, :], (BM, GA, GA))
+        hb = tl.load(hb_ptr + ib[:, None] * GB + ib[None, :])
+        amax = tl.zeros((BM,), dtype=tl.float32)
+        inv = tl.zeros((BM,), dtype=tl.float32)
+        for p in tl.static_range(2):
+            for g in range(0, K, G):
+                x = tl.load(x_ptr + rows64[:, None] * stride_xm + (g + cols)[None, :], mask=rmask[:, None], other=0.0)
+                z = tl.reshape(tl.dot(tl.reshape(x, (BM * GA, GB)), hb), (BM, GA, GB))
+                z_hi = z.to(tl.bfloat16)
+                z = z - z_hi.to(tl.float32)
+                z_mid = z.to(tl.bfloat16)
+                z_lo = (z - z_mid.to(tl.float32)).to(tl.bfloat16)
+                y = tl.dot(ha, z_lo, acc=tl.dot(ha, z_mid, acc=tl.dot(ha, z_hi)))
+                y = tl.reshape(y, (BM, G))
+                if p == 0:
+                    amax = tl.maximum(amax, tl.max(tl.abs(y), axis=1))
+                else:
+                    q = tl.extra.cuda.libdevice.rint(y * inv[:, None])
+                    q = tl.minimum(tl.maximum(q, -127.0), 127.0).to(tl.int8)
+                    tl.store(q_ptr + rows64[:, None] * stride_qm + (g + cols)[None, :], q, mask=rmask[:, None])
+            if p == 0:
+                scale = tl.maximum(amax, 1e-12) / 127.0
+                tl.store(s_ptr + rows, scale, mask=rmask)
+                inv = 1.0 / scale
+
+    @functools.cache
+    def rotate_tile(device_index: int, group: int) -> int:
+        # At group 256, 128 columns take 96 KiB: that fits every GPU from Ampere on (99 KiB per block).
+        shared = triton.runtime.driver.active.utils.get_device_properties(device_index)["max_shared_mem"]
+        tile = 128 if (ROTATE_BM * group + group * 128) * 2 <= shared else 64
+        return min(tile, group)
+
+    configs = [
+        triton.Config({"BM": bm, "BN": bn, "BK": bk, "GROUP_M": 8}, num_stages=stages, num_warps=warps)
+        for bm, bn, bk, stages, warps in [
+            (128, 256, 128, 3, 8),
+            (256, 128, 128, 3, 8),
+            (128, 128, 128, 4, 8),
+            (128, 256, 64, 4, 8),
+            (256, 128, 64, 4, 8),
+            (128, 128, 64, 5, 4),
+        ]
+    ]
+
+    @triton.autotune(configs=configs, key=["M_BUCKET", "N", "K"])
+    @triton.jit
+    def int8_mm_kernel(
+        a_ptr, b_ptr, c_ptr, sa_ptr, sb_ptr, M, N, K, M_BUCKET, stride_am, stride_bn, stride_cm,
+        BM: tl.constexpr, BN: tl.constexpr, BK: tl.constexpr, GROUP_M: tl.constexpr,
+    ):  # fmt: skip
+        pid = tl.program_id(0)
+        num_m = tl.cdiv(M, BM)
+        num_n = tl.cdiv(N, BN)
+        group = GROUP_M * num_n
+        first_m = (pid // group) * GROUP_M
+        size_m = tl.minimum(num_m - first_m, GROUP_M)
+        pid_m = first_m + (pid % group) % size_m
+        pid_n = (pid % group) // size_m
+        rm = pid_m * BM + tl.arange(0, BM)
+        rn = pid_n * BN + tl.arange(0, BN)
+        rk = tl.arange(0, BK)
+        a_ptrs = a_ptr + rm.to(tl.int64)[:, None] * stride_am + rk[None, :]
+        b_ptrs = b_ptr + rn.to(tl.int64)[None, :] * stride_bn + rk[:, None]
+        acc = tl.zeros((BM, BN), dtype=tl.int32)
+        for k in range(0, K, BK):
+            a = tl.load(a_ptrs, mask=(rm[:, None] < M) & (k + rk[None, :] < K), other=0)
+            b = tl.load(b_ptrs, mask=(rn[None, :] < N) & (k + rk[:, None] < K), other=0)
+            acc += tl.dot(a, b, out_dtype=tl.int32)
+            a_ptrs += BK
+            b_ptrs += BK
+        sa = tl.load(sa_ptr + rm, mask=rm < M, other=0.0)
+        sb = tl.load(sb_ptr + rn, mask=rn < N, other=0.0)
+        c = (acc.to(tl.float32) * sa[:, None] * sb[None, :]).to(tl.bfloat16)
+        mask = (rm[:, None] < M) & (rn[None, :] < N)
+        tl.store(c_ptr + rm.to(tl.int64)[:, None] * stride_cm + rn[None, :], c, mask=mask)
+
+    @triton.jit
+    def rescale_kernel(acc_ptr, sa_ptr, sb_ptr, c_ptr, M, N, BM: tl.constexpr, BN: tl.constexpr):
+        rm = tl.program_id(0) * BM + tl.arange(0, BM)
+        rn = tl.program_id(1) * BN + tl.arange(0, BN)
+        mask = (rm[:, None] < M) & (rn[None, :] < N)
+        offsets = rm.to(tl.int64)[:, None] * N + rn[None, :]
+        acc = tl.load(acc_ptr + offsets, mask=mask, other=0)
+        sa = tl.load(sa_ptr + rm, mask=rm < M, other=0.0)
+        sb = tl.load(sb_ptr + rn, mask=rn < N, other=0.0)
+        tl.store(c_ptr + offsets, (acc.to(tl.float32) * sa[:, None] * sb[None, :]).to(tl.bfloat16), mask=mask)
+
+    def rotate_quant(x: Tensor, group: int) -> tuple[Tensor, Tensor]:
+        M, K = x.shape
+        q = torch.empty(M, K, device=x.device, dtype=torch.int8)
+        s = torch.empty(M, device=x.device, dtype=torch.float32)
+        factors = hadamard_factors(group) if KERNEL_OPTIONS["rotate_quant"] == "factored" else None
+        if factors is not None:
+            ga, gb = factors
+            ha, hb = _rotation(ga, x.device), _rotation(gb, x.device)
+            rotate_quant_factored_kernel[(triton.cdiv(M, FACTORED_BM),)](
+                x, ha, hb, q, s, M, K, x.stride(0), q.stride(0), GA=ga, GB=gb, BM=FACTORED_BM, num_warps=4,
+                num_stages=4,
+            )  # fmt: skip
+            return q, s
+        BN = rotate_tile(x.device.index, group)
+        rotate_quant_kernel[(triton.cdiv(M, ROTATE_BM),)](
+            x, _rotation(group, x.device), q, s, M, K, x.stride(0), q.stride(0), G=group, BM=ROTATE_BM, BN=BN,
+            num_warps=8, num_stages=1,
+        )  # fmt: skip
+        return q, s
+
+    def matmul_triton(a: Tensor, b: Tensor, sa: Tensor, sb: Tensor, c: Tensor) -> None:
+        M, K = a.shape
+        N = b.shape[0]
+        grid = lambda meta: (triton.cdiv(M, meta["BM"]) * triton.cdiv(N, meta["BN"]),)  # noqa: E731
+        m_bucket = 1 << max(M - 1, 1).bit_length()
+        int8_mm_kernel[grid](a, b, c, sa, sb, M, N, K, m_bucket, a.stride(0), b.stride(0), c.stride(0))
+
+    def matmul_cublas(a: Tensor, b: Tensor, sa: Tensor, sb: Tensor, c: Tensor) -> None:
+        # Row chunks keep the int32 accumulator near 1 GiB.
+        M, N = a.shape[0], b.shape[0]
+        chunk = max(128, ((1 << 28) // N) // 128 * 128)
+        for start in range(0, M, chunk):
+            stop = min(M, start + chunk)
+            if stop - start < _MIN_KERNEL_ROWS:
+                matmul_triton(a[start:stop], b, sa[start:stop], sb, c[start:stop])
+                continue
+            acc = torch._int_mm(a[start:stop], b.t())
+            grid = (triton.cdiv(stop - start, 64), triton.cdiv(N, 128))
+            rescale_kernel[grid](acc, sa[start:stop], sb, c[start:stop], stop - start, N, BM=64, BN=128, num_warps=8)
+
+    def matmul_bf16(a: Tensor, b: Tensor, sa: Tensor, sb: Tensor, c: Tensor) -> None:
+        M, N = a.shape[0], b.shape[0]
+        weight = b.to(torch.bfloat16)
+        chunk = max(128, ((1 << 28) // N) // 128 * 128)
+        for start in range(0, M, chunk):
+            stop = min(M, start + chunk)
+            acc = torch.mm(a[start:stop].to(torch.bfloat16), weight.t(), out_dtype=torch.float32)
+            grid = (triton.cdiv(stop - start, 64), triton.cdiv(N, 128))
+            rescale_kernel[grid](acc, sa[start:stop], sb, c[start:stop], stop - start, N, BM=64, BN=128, num_warps=8)
+
+    def int8_matmul(a: Tensor, b: Tensor, sa: Tensor, sb: Tensor) -> Tensor:
+        M, K = a.shape
+        N = b.shape[0]
+        gemm = resolve_int8_gemm(a.device)
+        if gemm == "sgl" and K % 16 == 0 and N % 16 == 0:
+            # b is the [N, K] row-major qweight; its transpose is the column-major [K, N] operand CUTLASS expects.
+            return _sgl_int8_scaled_mm()(a, b.t(), sa, sb, torch.bfloat16)
+        c = torch.empty(M, N, device=a.device, dtype=torch.bfloat16)
+        if gemm == "bf16":
+            matmul_bf16(a, b, sa, sb, c)
+        elif gemm == "cublas" and K % 8 == 0 and N % 8 == 0:
+            matmul_cublas(a, b, sa, sb, c)
+        else:
+            matmul_triton(a, b, sa, sb, c)
+        return c
+
+    jit = SimpleNamespace(
+        rotate_quant=rotate_quant_kernel,
+        rotate_quant_factored=rotate_quant_factored_kernel,
+        int8_mm=int8_mm_kernel,
+        rescale=rescale_kernel,
+    )
+    return SimpleNamespace(rotate_quant=rotate_quant, int8_matmul=int8_matmul, jit=jit, rotate_tile=rotate_tile)
+
+
+__all__ = [
+    "INT8_GEMMS",
+    "KERNEL_OPTIONS",
+    "ROTATE_QUANTS",
+    "Int8RotLinear",
+    "hadamard_factors",
+    "regular_hadamard",
+    "resolve_int8_gemm",
+    "set_kernel_options",
+]

fl2va_int8_convrot/hyperflow_h3_flash_transformer.py

@@ -0,0 +1,403 @@
+# Copyright 2026 The HyperFlow authors. Licensed under the Apache License, Version 2.0.
+#
+# `HyperFlowH3FlashTransformer3DModel.forward` is derived from `MiniMaxH3Transformer3DModel.forward` in diffusers
+# (`models/transformers/transformer_minimax_h3.py`), Copyright 2025 The MiniMax Team and The HuggingFace Team,
+# Apache License 2.0. See https://github.com/Video-Rebirth/hyperflow-h3-flash/blob/main/THIRD_PARTY_NOTICES.md.
+"""The HyperFlow H3 Flash transformer: MiniMax-H3, compressed and conditioned on two times.
+
+Four changes to `MiniMaxH3Transformer3DModel`, all driven by `config.json`:
+
+* **Two-time conditioning.** An `endpoint_time_embedder` embeds the step's endpoint ``r`` next to the current time
+  ``t``; every AdaLN reads ``temb(t, r) = emb_t(t) + hyperflow_gate * (emb_r(r) - emb_t(t))``.
+* **Dropped blocks** (`dropped_blocks`): those blocks are identities and have no weights.
+* **Low-rank linears** (`lowrank_ranks`): ``W ~= up @ down``, stored as `<name>.down.weight` (rank x in) and
+  `<name>.up.weight` (out x rank). Ranks are zero-padded to multiples of 64, in the checkpoint or at load
+  (`pad_lowrank_ranks`), so both GEMMs are aligned; `set_lowrank_rank_multiple` re-pads them at run time.
+* **INT8 linears** (`quantization`): W8A8 with a Hadamard-rotated input, see `int8.py`.
+
+Embeddings, AdaLN, the token refiner and the output heads are the base model's, in bf16 / fp32. The block
+feed-forwards run in token chunks (`ChunkedFeedForward`, `set_ff_chunk`), which bounds their activations;
+`set_block_chunk` slices every token-wise op of a block.
+"""
+
+from __future__ import annotations
+
+import sys
+from typing import Any, ClassVar
+
+import torch
+from diffusers import MiniMaxH3Transformer3DModel
+from diffusers.configuration_utils import register_to_config
+from diffusers.models.embeddings import TimestepEmbedding
+from diffusers.models.modeling_utils import get_parameter_dtype
+from diffusers.models.transformers.transformer_minimax_h3 import (
+    MINIMAX_H3_MODALITY_NUM,
+    MiniMaxH3TransformerOutput,
+)
+from diffusers.utils import apply_lora_scale
+from torch import Tensor, nn
+
+from .hyperflow_h3_flash_int8 import Int8RotLinear
+
+#: Tokens per feed-forward chunk; a 5.2 s clip has about 38k, a 14.4 s clip about 105k.
+FF_CHUNK_TOKENS = 16384
+#: Low-rank ranks are padded to multiples of this at load, so both GEMMs of a low-rank linear are aligned.
+RANK_MULTIPLE = 64
+
+
+class LowRankLinear(nn.Module):
+    """``y = up(down(x))``, the factorized replacement of a bias-free `nn.Linear`."""
+
+    def __init__(self, in_features: int, out_features: int, rank: int) -> None:
+        super().__init__()
+        self.in_features, self.out_features, self.rank = in_features, out_features, rank
+        self.down = nn.Linear(in_features, rank, bias=False)
+        self.up = nn.Linear(rank, out_features, bias=False)
+
+    def forward(self, x: Tensor) -> Tensor:
+        return self.up(self.down(x))
+
+    @torch.no_grad()
+    def pad_rank(self, multiple: int = RANK_MULTIPLE) -> bool:
+        """Zero-pad the rank up to a multiple of ``multiple``: new rows of ``down`` and columns of ``up`` are zero, so
+        ``up @ down`` is unchanged. Returns whether the rank changed."""
+        rank = -(-self.rank // multiple) * multiple
+        if rank == self.rank:
+            return False
+        extra = rank - self.rank
+        down, up = self.down.weight, self.up.weight
+        self.down.weight = nn.Parameter(torch.cat([down, down.new_zeros(extra, down.shape[1])]), requires_grad=False)
+        self.up.weight = nn.Parameter(torch.cat([up, up.new_zeros(up.shape[0], extra)], dim=1), requires_grad=False)
+        self.down.out_features = self.up.in_features = self.rank = rank
+        return True
+
+
+class ChunkedFeedForward(nn.Module):
+    """A block's feed-forward run over at most ``chunk_tokens`` tokens at a time; it keeps the block's ``net``.
+
+    The feed-forward is token-wise, so the chunks compute the same function; only the rounding of bf16 GEMMs can
+    change with their row count. It bounds the SwiGLU input, ``2 * ffn_dim`` wide per token (6 GiB for a 14.4 s
+    clip), to one chunk.
+    """
+
+    def __init__(self, net: nn.ModuleList, out_features: int, chunk_tokens: int) -> None:
+        super().__init__()
+        self.net, self.out_features, self.chunk_tokens = net, out_features, chunk_tokens
+
+    def _forward(self, hidden_states: Tensor) -> Tensor:
+        for module in self.net:
+            hidden_states = module(hidden_states)
+        return hidden_states
+
+    def forward(self, hidden_states: Tensor) -> Tensor:
+        if hidden_states.shape[-2] <= self.chunk_tokens:
+            return self._forward(hidden_states)
+        out = hidden_states.new_empty(*hidden_states.shape[:-1], self.out_features)
+        for start in range(0, hidden_states.shape[-2], self.chunk_tokens):
+            stop = start + self.chunk_tokens
+            out[..., start:stop, :] = self._forward(hidden_states[..., start:stop, :])
+        return out
+
+
+class DroppedBlock(nn.Module):
+    """A removed transformer block: the hidden states go straight to the next block.
+
+    The signature is the block's own: context parallel hooks address `adaln_indices` by name.
+    """
+
+    def forward(self, hidden_states: Tensor, temb=None, adaln_indices=None, rotary_emb=None, attention_mask=None):
+        return hidden_states
+
+
+def _replace_linear(model: nn.Module, name: str, make: type[nn.Module], *args: Any) -> None:
+    owner_name, _, attr = name.rpartition(".")
+    owner = model.get_submodule(owner_name)
+    dense = owner[int(attr)] if isinstance(owner, nn.ModuleList) else getattr(owner, attr)
+    if type(dense) is not nn.Linear or dense.bias is not None:
+        raise TypeError(f"{name} is not a bias-free nn.Linear")
+    replacement = make(dense.in_features, dense.out_features, *args)
+    if isinstance(owner, nn.ModuleList):
+        owner[int(attr)] = replacement
+    else:
+        setattr(owner, attr, replacement)
+
+
+class HyperFlowH3FlashTransformer3DModel(MiniMaxH3Transformer3DModel):
+    """`MiniMaxH3Transformer3DModel` conditioned on ``(t, r)``, with dropped blocks, low-rank and INT8 linears."""
+
+    # Matched against whole dotted name parts at load time, so `time_embedder` does not cover the endpoint embedder.
+    _keep_in_fp32_modules: ClassVar[list[str]] = [
+        *MiniMaxH3Transformer3DModel._keep_in_fp32_modules,
+        "endpoint_time_embedder",
+        "wscale",
+    ]
+
+    @register_to_config
+    def __init__(
+        self,
+        num_attention_heads: int = 56,
+        attention_head_dim: int = 128,
+        hidden_size: int = 5376,
+        num_layers: int = 50,
+        num_refiner_layers: int = 2,
+        ffn_dim: int = 14336,
+        in_channels: int = 24,
+        audio_in_channels: int = 32,
+        patch_size: tuple[int, int, int] = (1, 2, 2),
+        text_dim: int = 5120,
+        freq_dim: int = 256,
+        time_embed_hidden_dim: int = 5376,
+        time_embed_dim: int = 16,
+        rope_freq_dim: int = 16,
+        rope_theta: float = 10000.0,
+        norm_eps: float = 1e-5,
+        qk_norm_eps: float = 1e-5,
+        final_norm_eps: float = 1e-5,
+        hyperflow_gate: float = 0.25,
+        dropped_blocks: list[int] | None = None,
+        lowrank_ranks: dict[str, int] | None = None,
+        quantization: dict[str, Any] | None = None,
+    ) -> None:
+        super().__init__(
+            num_attention_heads=num_attention_heads,
+            attention_head_dim=attention_head_dim,
+            hidden_size=hidden_size,
+            num_layers=num_layers,
+            num_refiner_layers=num_refiner_layers,
+            ffn_dim=ffn_dim,
+            in_channels=in_channels,
+            audio_in_channels=audio_in_channels,
+            patch_size=patch_size,
+            text_dim=text_dim,
+            freq_dim=freq_dim,
+            time_embed_hidden_dim=time_embed_hidden_dim,
+            time_embed_dim=time_embed_dim,
+            rope_freq_dim=rope_freq_dim,
+            rope_theta=rope_theta,
+            norm_eps=norm_eps,
+            qk_norm_eps=qk_norm_eps,
+            final_norm_eps=final_norm_eps,
+        )
+        self.endpoint_time_embedder = TimestepEmbedding(
+            in_channels=freq_dim, time_embed_dim=time_embed_hidden_dim, out_dim=time_embed_dim
+        )
+
+        dropped = set(dropped_blocks or [])
+        for index, block in enumerate(self.transformer_blocks):
+            if index in dropped:
+                self.transformer_blocks[index] = DroppedBlock()
+            else:
+                block.ff = ChunkedFeedForward(block.ff.net, block.ff.net[-1].out_features, FF_CHUNK_TOKENS)
+        for name, rank in (lowrank_ranks or {}).items():
+            if int(name.split(".")[1]) not in dropped:
+                _replace_linear(self, name, LowRankLinear, int(rank))
+        if quantization:
+            if quantization.get("method") != "int8_convrot":
+                raise ValueError(f"unsupported quantization method {quantization.get('method')!r}")
+            for name in quantization["linears"]:
+                _replace_linear(self, name, Int8RotLinear, int(quantization["group"]))
+        for block in self.transformer_blocks:
+            attn = getattr(block, "attn", None)
+            for name in ("to_q", "to_k"):
+                if isinstance(layer := getattr(attn, name, None), Int8RotLinear):
+                    layer.keep_input = True
+
+    def pad_lowrank_ranks(self, multiple: int = RANK_MULTIPLE) -> int:
+        """Zero-pad every low-rank linear to a rank multiple of ``multiple`` (see `LowRankLinear.pad_rank`) for a
+        checkpoint stored at its unpadded ranks. Returns how many linears changed."""
+        ranks = dict(self.config.lowrank_ranks or {})
+        padded = 0
+        for name, module in self.named_modules():
+            if isinstance(module, LowRankLinear) and module.pad_rank(multiple):
+                ranks[name] = module.rank
+                padded += 1
+        if padded:
+            self.register_to_config(lowrank_ranks=ranks)
+        return padded
+
+    def time_embedding(self, timestep: Tensor, endpoint_timestep: Tensor) -> Tensor:
+        """``temb`` of every distinct ``(t, r)`` pair, at the time embedders' precision (fp32)."""
+        if endpoint_timestep.shape != timestep.shape:
+            raise ValueError(f"endpoint_timestep {list(endpoint_timestep.shape)} != timestep {list(timestep.shape)}")
+        dtype = get_parameter_dtype(self.time_embedder)
+        current = self.time_embedder(self.time_proj(timestep).to(dtype))
+        endpoint = self.endpoint_time_embedder(self.time_proj(endpoint_timestep).to(dtype))
+        return current + self.config.hyperflow_gate * (endpoint - current)
+
+    @apply_lora_scale("attention_kwargs")
+    def forward(
+        self,
+        hidden_states: Tensor,
+        audio_hidden_states: Tensor,
+        encoder_hidden_states: Tensor,
+        timestep: Tensor,
+        timestep_indices: Tensor,
+        token_tags: Tensor,
+        position_ids: Tensor,
+        video_indices: Tensor,
+        audio_indices: Tensor,
+        text_indices: Tensor,
+        endpoint_timestep: Tensor | None = None,
+        attention_kwargs: dict[str, Any] | None = None,
+        return_dict: bool = True,
+    ) -> MiniMaxH3TransformerOutput | tuple[Tensor, Tensor]:
+        """The base forward plus ``endpoint_timestep``: ``(timestep[i], endpoint_timestep[i])`` is the ``i``-th pair.
+
+        ``timestep_indices`` maps every row of the packed sequence to its pair. Build the pairs with
+        `hyperflow_h3_flash.schedule.build_row_time_pairs`, or run the model through `flash_blocks(...)`.
+        """
+        if endpoint_timestep is None:
+            raise ValueError(
+                "HyperFlow H3 Flash needs `endpoint_timestep`, the endpoint `r` of every `(t, r)` pair: it is a "
+                "few-step model and has no single-time mode. Run it through `hyperflow_h3_flash.flash_blocks(...)`."
+            )
+        if position_ids.ndim != 2 or position_ids.shape[-1] != 3:
+            raise ValueError(f"`position_ids` must be a `(seq_len, 3)` tensor, got {list(position_ids.shape)}.")
+        sequence_length = position_ids.shape[0]
+        if token_tags.shape != (sequence_length,) or timestep_indices.shape != (sequence_length,):
+            raise ValueError("`token_tags` and `timestep_indices` must both be `(seq_len,)` tensors.")
+
+        rotary_emb = self.rope(position_ids)
+
+        video_embeds = self.proj_in(hidden_states.to(get_parameter_dtype(self.proj_in)))
+        audio_embeds = self.audio_proj_in(audio_hidden_states.to(get_parameter_dtype(self.audio_proj_in)))
+        text_embeds = self.context_embedder(encoder_hidden_states.to(get_parameter_dtype(self.context_embedder)))
+        text_embeds = self.token_refiner(text_embeds)
+
+        hidden_states = text_embeds.new_zeros((text_embeds.shape[0], sequence_length, text_embeds.shape[-1]))
+        hidden_states = hidden_states.index_copy(1, text_indices, text_embeds)
+        hidden_states = hidden_states.index_copy(1, video_indices, video_embeds.to(text_embeds.dtype))
+        hidden_states = hidden_states.index_copy(1, audio_indices, audio_embeds.to(text_embeds.dtype))
+
+        temb = self.time_embedding(timestep, endpoint_timestep)
+        adaln_indices = timestep_indices * MINIMAX_H3_MODALITY_NUM + token_tags
+        for block in self.transformer_blocks:
+            hidden_states = block(hidden_states, temb, adaln_indices, rotary_emb)
+
+        hidden_states = self.norm_out(hidden_states, temb, timestep_indices).to(get_parameter_dtype(self.proj_out))
+        video_output = self.proj_out(hidden_states).index_select(1, video_indices)
+        audio_output = self.audio_proj_out(hidden_states).index_select(1, audio_indices)
+        if not return_dict:
+            return (video_output, audio_output)
+        return MiniMaxH3TransformerOutput(sample=video_output, audio_sample=audio_output)
+
+
+def _used_rank(layer: LowRankLinear) -> int:
+    """The rank without the zero padding (trailing zero rows of ``down`` and columns of ``up``)."""
+    used = (layer.down.weight.detach().abs().amax(1) != 0) | (layer.up.weight.detach().abs().amax(0) != 0)
+    nonzero = used.nonzero()
+    return int(nonzero[-1]) + 1 if len(nonzero) else 1
+
+
+def set_lowrank_rank_multiple(model: nn.Module, multiple: int) -> int:
+    """Re-pad every low-rank linear to a rank multiple of ``multiple`` (0: no padding); returns the layers changed.
+
+    Zero rows and columns add exact zeros, so any multiple computes the same function; the GEMM shapes change, and with
+    them the bf16 rounding (8 is 2 to 7 % faster than 64 on H200). Run time only: the config still lists the stored
+    ranks, so do not `save_pretrained` the re-padded model.
+    """
+    changed = 0
+    for layer in model.modules():
+        if not isinstance(layer, LowRankLinear):
+            continue
+        rank = _used_rank(layer)
+        target = -(-rank // multiple) * multiple if multiple > 0 else rank
+        if target == layer.rank:
+            continue
+        down, up = layer.down.weight.detach(), layer.up.weight.detach()
+        if target < layer.rank:
+            down, up = down[:target], up[:, :target]
+        else:
+            down = torch.cat([down, down.new_zeros(target - layer.rank, down.shape[1])])
+            up = torch.cat([up, up.new_zeros(up.shape[0], target - layer.rank)], dim=1)
+        layer.down.weight = nn.Parameter(down.contiguous(), requires_grad=False)
+        layer.up.weight = nn.Parameter(up.contiguous(), requires_grad=False)
+        layer.rank = layer.down.out_features = layer.up.in_features = target
+        changed += 1
+    return changed
+
+
+def set_ff_chunk(model: nn.Module, tokens: int) -> None:
+    """Run every feed-forward over at most ``tokens`` tokens at a time (0: whole sequence)."""
+    for module in model.modules():
+        if isinstance(module, ChunkedFeedForward):
+            module.chunk_tokens = tokens if tokens > 0 else sys.maxsize
+
+
+def chunk_block(block: nn.Module, chunk: int) -> None:
+    """Run every token-wise op of a transformer block over ``chunk``-token slices; only the attention sees all of q /
+    k / v.
+
+    The unchunked block holds about a dozen ``(seq, hidden)`` tensors at once (each modulation picked per token, the
+    modulated norm, the residual sums), 1.2 GB apiece at 108k tokens. Per-row results are the same up to the bf16 GEMM
+    rounding for the smaller row count. Apply before context parallelism and offloading, which hook the block.
+    """
+    import diffusers.models.transformers.transformer_minimax_h3 as h3
+
+    attn = block.attn
+
+    def forward(hidden_states, temb, adaln_indices, rotary_emb, attention_mask=None):
+        shift_msa, scale_msa, gate_msa, shift_mlp, scale_mlp, gate_mlp = block.adaln_proj(temb)
+        batch, seq, _ = hidden_states.shape
+        cos, sin = rotary_emb
+        qkv = None
+        for start in range(0, seq, chunk):
+            rows = slice(start, min(start + chunk, seq))
+            idx = adaln_indices[rows]
+            h = block.norm1(hidden_states[:, rows])
+            h = h * (1.0 + scale_msa.index_select(0, idx)) + shift_msa.index_select(0, idx)
+            parts = []
+            for proj, norm in ((attn.to_q, attn.norm_q), (attn.to_k, attn.norm_k), (attn.to_v, None)):
+                x = proj(h).unflatten(-1, (attn.heads, -1))
+                if norm is not None:
+                    x = h3._apply_rotary_emb(norm(x), cos[rows], sin[rows])
+                parts.append(x)
+            if qkv is None:
+                qkv = [x.new_empty((batch, seq, *x.shape[2:])) for x in parts]
+            for full, x in zip(qkv, parts, strict=True):
+                full[:, rows] = x
+            del h, parts, x
+        out = h3.dispatch_attention_fn(
+            *qkv,
+            attn_mask=attention_mask,
+            dropout_p=0.0,
+            is_causal=False,
+            backend=attn.processor._attention_backend,
+            parallel_config=attn.processor._parallel_config,
+        )
+        dtype = qkv[0].dtype
+        del qkv
+        result = torch.empty_like(hidden_states)
+        for start in range(0, seq, chunk):
+            rows = slice(start, min(start + chunk, seq))
+            idx = adaln_indices[rows]
+            a = attn.to_out[1](attn.to_out[0](out[:, rows].flatten(2, 3).to(dtype)))
+            h = hidden_states[:, rows] + gate_msa.index_select(0, idx) * a
+            n = block.norm2(h)
+            n = n * (1.0 + scale_mlp.index_select(0, idx)) + shift_mlp.index_select(0, idx)
+            result[:, rows] = h + gate_mlp.index_select(0, idx) * block.ff(n)
+            del a, h, n
+        return result
+
+    block.forward = forward
+
+
+def set_block_chunk(model: nn.Module, tokens: int) -> None:
+    """`chunk_block` on every kept block (0: off). Call once, before context parallelism and offloading."""
+    if tokens > 0:
+        for block in model.transformer_blocks:
+            if not isinstance(block, DroppedBlock):
+                chunk_block(block, tokens)
+
+
+__all__ = [
+    "RANK_MULTIPLE",
+    "ChunkedFeedForward",
+    "DroppedBlock",
+    "HyperFlowH3FlashTransformer3DModel",
+    "LowRankLinear",
+    "chunk_block",
+    "set_block_chunk",
+    "set_ff_chunk",
+    "set_lowrank_rank_multiple",
+]

ref2va_gguf/minimax_h3_ref2va_hyperflow_flash_14b-Q3_K_M.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:50f68efc06e5c27039aef77d958278a7c10dc6ffc9456ea5171d6e3c740565f1
+size 7354188800

ref2va_gguf/minimax_h3_ref2va_hyperflow_flash_14b-Q4_K_M.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a6b3018230cfcc35c552d7c2b2c5533fe37067f03f413ce7f9ddf7fcdea6ec2d
+size 9220352512

ref2va_gguf/minimax_h3_ref2va_hyperflow_flash_14b-Q4_K_S.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:86a5c4bedd0a6c31249ddba92199ff540b2741f3c86ca6d8f127b47a48f4ac29
+size 8138873344

ref2va_gguf/minimax_h3_ref2va_hyperflow_flash_14b-Q5_K_M.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e6c66150645362ac15d8d7972703eeac00a53a3909392bb3ec7a985e46a8e93f
+size 10452388352

ref2va_gguf/minimax_h3_ref2va_hyperflow_flash_14b-Q6_K.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7a50e5f0eea0fb962f1d01f6032e10d18d10d858c7b781f4b5f1da47450c0fa7
+size 11761426432

ref2va_gguf/minimax_h3_ref2va_hyperflow_flash_14b-Q8_0.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fc7c98be8a1197092fe491678d3f71d7514fe037a95a7d9d518bba6f21476a40
+size 15164430848

ref2va_int8_convrot/config.json

@@ -0,0 +1,335 @@
+{
+  "_class_name": "HyperFlowH3FlashTransformer3DModel",
+  "_diffusers_version": "0.36.0.dev0",
+  "num_attention_heads": 56,
+  "attention_head_dim": 128,
+  "hidden_size": 5376,
+  "num_layers": 50,
+  "num_refiner_layers": 2,
+  "ffn_dim": 14336,
+  "in_channels": 24,
+  "audio_in_channels": 32,
+  "patch_size": [
+    1,
+    2,
+    2
+  ],
+  "text_dim": 5120,
+  "freq_dim": 256,
+  "time_embed_hidden_dim": 5376,
+  "time_embed_dim": 16,
+  "rope_freq_dim": 16,
+  "rope_theta": 10000.0,
+  "norm_eps": 1e-05,
+  "qk_norm_eps": 1e-05,
+  "final_norm_eps": 1e-05,
+  "hyperflow_gate": 0.25,
+  "lowrank_ranks": {
+    "transformer_blocks.1.attn.to_k": 990,
+    "transformer_blocks.1.attn.to_out.0": 1872,
+    "transformer_blocks.1.attn.to_q": 1102,
+    "transformer_blocks.1.attn.to_v": 1200,
+    "transformer_blocks.1.ff.net.0.proj": 2516,
+    "transformer_blocks.1.ff.net.2": 3120,
+    "transformer_blocks.10.attn.to_k": 275,
+    "transformer_blocks.10.attn.to_out.0": 362,
+    "transformer_blocks.10.attn.to_q": 269,
+    "transformer_blocks.10.attn.to_v": 471,
+    "transformer_blocks.10.ff.net.0.proj": 359,
+    "transformer_blocks.10.ff.net.2": 1436,
+    "transformer_blocks.11.attn.to_k": 420,
+    "transformer_blocks.11.attn.to_out.0": 572,
+    "transformer_blocks.11.attn.to_q": 426,
+    "transformer_blocks.11.attn.to_v": 673,
+    "transformer_blocks.11.ff.net.0.proj": 604,
+    "transformer_blocks.11.ff.net.2": 557,
+    "transformer_blocks.12.attn.to_k": 171,
+    "transformer_blocks.12.attn.to_out.0": 210,
+    "transformer_blocks.12.attn.to_q": 181,
+    "transformer_blocks.12.attn.to_v": 342,
+    "transformer_blocks.12.ff.net.0.proj": 247,
+    "transformer_blocks.12.ff.net.2": 953,
+    "transformer_blocks.13.attn.to_k": 379,
+    "transformer_blocks.13.attn.to_out.0": 474,
+    "transformer_blocks.13.attn.to_q": 381,
+    "transformer_blocks.13.attn.to_v": 582,
+    "transformer_blocks.13.ff.net.0.proj": 462,
+    "transformer_blocks.13.ff.net.2": 1,
+    "transformer_blocks.14.attn.to_k": 286,
+    "transformer_blocks.14.attn.to_out.0": 376,
+    "transformer_blocks.14.attn.to_q": 305,
+    "transformer_blocks.14.attn.to_v": 505,
+    "transformer_blocks.14.ff.net.0.proj": 374,
+    "transformer_blocks.14.ff.net.2": 1348,
+    "transformer_blocks.15.attn.to_k": 155,
+    "transformer_blocks.15.attn.to_out.0": 261,
+    "transformer_blocks.15.attn.to_q": 167,
+    "transformer_blocks.15.attn.to_v": 311,
+    "transformer_blocks.15.ff.net.0.proj": 201,
+    "transformer_blocks.15.ff.net.2": 1038,
+    "transformer_blocks.16.attn.to_k": 239,
+    "transformer_blocks.16.attn.to_out.0": 400,
+    "transformer_blocks.16.attn.to_q": 246,
+    "transformer_blocks.16.attn.to_v": 440,
+    "transformer_blocks.16.ff.net.0.proj": 335,
+    "transformer_blocks.16.ff.net.2": 1510,
+    "transformer_blocks.17.attn.to_k": 951,
+    "transformer_blocks.17.attn.to_out.0": 1116,
+    "transformer_blocks.17.attn.to_q": 952,
+    "transformer_blocks.17.attn.to_v": 1216,
+    "transformer_blocks.17.ff.net.0.proj": 1325,
+    "transformer_blocks.17.ff.net.2": 2836,
+    "transformer_blocks.18.attn.to_k": 121,
+    "transformer_blocks.18.attn.to_out.0": 279,
+    "transformer_blocks.18.attn.to_q": 140,
+    "transformer_blocks.18.attn.to_v": 287,
+    "transformer_blocks.18.ff.net.0.proj": 234,
+    "transformer_blocks.18.ff.net.2": 1024,
+    "transformer_blocks.20.attn.to_k": 433,
+    "transformer_blocks.20.attn.to_out.0": 725,
+    "transformer_blocks.20.attn.to_q": 445,
+    "transformer_blocks.20.attn.to_v": 656,
+    "transformer_blocks.20.ff.net.0.proj": 698,
+    "transformer_blocks.20.ff.net.2": 2243,
+    "transformer_blocks.21.attn.to_k": 389,
+    "transformer_blocks.21.attn.to_out.0": 704,
+    "transformer_blocks.21.attn.to_q": 434,
+    "transformer_blocks.21.attn.to_v": 669,
+    "transformer_blocks.21.ff.net.0.proj": 711,
+    "transformer_blocks.21.ff.net.2": 2261,
+    "transformer_blocks.22.attn.to_k": 170,
+    "transformer_blocks.22.attn.to_out.0": 459,
+    "transformer_blocks.22.attn.to_q": 199,
+    "transformer_blocks.22.attn.to_v": 385,
+    "transformer_blocks.22.ff.net.0.proj": 378,
+    "transformer_blocks.22.ff.net.2": 1507,
+    "transformer_blocks.23.attn.to_k": 78,
+    "transformer_blocks.23.attn.to_out.0": 261,
+    "transformer_blocks.23.attn.to_q": 103,
+    "transformer_blocks.23.attn.to_v": 227,
+    "transformer_blocks.23.ff.net.0.proj": 221,
+    "transformer_blocks.23.ff.net.2": 1147,
+    "transformer_blocks.4.attn.to_k": 284,
+    "transformer_blocks.4.attn.to_out.0": 398,
+    "transformer_blocks.4.attn.to_q": 265,
+    "transformer_blocks.4.attn.to_v": 403,
+    "transformer_blocks.4.ff.net.0.proj": 546,
+    "transformer_blocks.4.ff.net.2": 1130,
+    "transformer_blocks.5.attn.to_k": 224,
+    "transformer_blocks.5.attn.to_out.0": 424,
+    "transformer_blocks.5.attn.to_q": 224,
+    "transformer_blocks.5.attn.to_v": 413,
+    "transformer_blocks.5.ff.net.0.proj": 991,
+    "transformer_blocks.5.ff.net.2": 2357,
+    "transformer_blocks.6.attn.to_k": 439,
+    "transformer_blocks.6.attn.to_out.0": 673,
+    "transformer_blocks.6.attn.to_q": 428,
+    "transformer_blocks.6.attn.to_v": 607,
+    "transformer_blocks.6.ff.net.0.proj": 836,
+    "transformer_blocks.6.ff.net.2": 2076,
+    "transformer_blocks.7.attn.to_k": 621,
+    "transformer_blocks.7.attn.to_out.0": 815,
+    "transformer_blocks.7.attn.to_q": 626,
+    "transformer_blocks.7.attn.to_v": 864,
+    "transformer_blocks.7.ff.net.0.proj": 986,
+    "transformer_blocks.7.ff.net.2": 1322,
+    "transformer_blocks.8.attn.to_k": 483,
+    "transformer_blocks.8.attn.to_out.0": 609,
+    "transformer_blocks.8.attn.to_q": 479,
+    "transformer_blocks.8.attn.to_v": 704,
+    "transformer_blocks.8.ff.net.0.proj": 802,
+    "transformer_blocks.8.ff.net.2": 2054
+  },
+  "dropped_blocks": [
+    9
+  ],
+  "auto_map": {
+    "AutoModel": "hyperflow_h3_flash_transformer.HyperFlowH3FlashTransformer3DModel"
+  },
+  "quantization": {
+    "method": "int8_convrot",
+    "group": 256,
+    "linears": [
+      "transformer_blocks.0.attn.to_k",
+      "transformer_blocks.0.attn.to_out.0",
+      "transformer_blocks.0.attn.to_q",
+      "transformer_blocks.0.attn.to_v",
+      "transformer_blocks.0.ff.net.0.proj",
+      "transformer_blocks.0.ff.net.2",
+      "transformer_blocks.19.attn.to_k",
+      "transformer_blocks.19.attn.to_out.0",
+      "transformer_blocks.19.attn.to_q",
+      "transformer_blocks.19.attn.to_v",
+      "transformer_blocks.19.ff.net.0.proj",
+      "transformer_blocks.19.ff.net.2",
+      "transformer_blocks.2.attn.to_k",
+      "transformer_blocks.2.attn.to_out.0",
+      "transformer_blocks.2.attn.to_q",
+      "transformer_blocks.2.attn.to_v",
+      "transformer_blocks.2.ff.net.0.proj",
+      "transformer_blocks.2.ff.net.2",
+      "transformer_blocks.24.attn.to_k",
+      "transformer_blocks.24.attn.to_out.0",
+      "transformer_blocks.24.attn.to_q",
+      "transformer_blocks.24.attn.to_v",
+      "transformer_blocks.24.ff.net.0.proj",
+      "transformer_blocks.24.ff.net.2",
+      "transformer_blocks.25.attn.to_k",
+      "transformer_blocks.25.attn.to_out.0",
+      "transformer_blocks.25.attn.to_q",
+      "transformer_blocks.25.attn.to_v",
+      "transformer_blocks.25.ff.net.0.proj",
+      "transformer_blocks.25.ff.net.2",
+      "transformer_blocks.26.attn.to_k",
+      "transformer_blocks.26.attn.to_out.0",
+      "transformer_blocks.26.attn.to_q",
+      "transformer_blocks.26.attn.to_v",
+      "transformer_blocks.26.ff.net.0.proj",
+      "transformer_blocks.26.ff.net.2",
+      "transformer_blocks.27.attn.to_k",
+      "transformer_blocks.27.attn.to_out.0",
+      "transformer_blocks.27.attn.to_q",
+      "transformer_blocks.27.attn.to_v",
+      "transformer_blocks.27.ff.net.0.proj",
+      "transformer_blocks.27.ff.net.2",
+      "transformer_blocks.28.attn.to_k",
+      "transformer_blocks.28.attn.to_out.0",
+      "transformer_blocks.28.attn.to_q",
+      "transformer_blocks.28.attn.to_v",
+      "transformer_blocks.28.ff.net.0.proj",
+      "transformer_blocks.28.ff.net.2",
+      "transformer_blocks.29.attn.to_k",
+      "transformer_blocks.29.attn.to_out.0",
+      "transformer_blocks.29.attn.to_q",
+      "transformer_blocks.29.attn.to_v",
+      "transformer_blocks.29.ff.net.0.proj",
+      "transformer_blocks.29.ff.net.2",
+      "transformer_blocks.3.attn.to_k",
+      "transformer_blocks.3.attn.to_out.0",
+      "transformer_blocks.3.attn.to_q",
+      "transformer_blocks.3.attn.to_v",
+      "transformer_blocks.3.ff.net.0.proj",
+      "transformer_blocks.3.ff.net.2",
+      "transformer_blocks.30.attn.to_k",
+      "transformer_blocks.30.attn.to_out.0",
+      "transformer_blocks.30.attn.to_q",
+      "transformer_blocks.30.attn.to_v",
+      "transformer_blocks.30.ff.net.0.proj",
+      "transformer_blocks.30.ff.net.2",
+      "transformer_blocks.31.attn.to_k",
+      "transformer_blocks.31.attn.to_out.0",
+      "transformer_blocks.31.attn.to_q",
+      "transformer_blocks.31.attn.to_v",
+      "transformer_blocks.31.ff.net.0.proj",
+      "transformer_blocks.31.ff.net.2",
+      "transformer_blocks.32.attn.to_k",
+      "transformer_blocks.32.attn.to_out.0",
+      "transformer_blocks.32.attn.to_q",
+      "transformer_blocks.32.attn.to_v",
+      "transformer_blocks.32.ff.net.0.proj",
+      "transformer_blocks.32.ff.net.2",
+      "transformer_blocks.33.attn.to_k",
+      "transformer_blocks.33.attn.to_out.0",
+      "transformer_blocks.33.attn.to_q",
+      "transformer_blocks.33.attn.to_v",
+      "transformer_blocks.33.ff.net.0.proj",
+      "transformer_blocks.33.ff.net.2",
+      "transformer_blocks.34.attn.to_k",
+      "transformer_blocks.34.attn.to_out.0",
+      "transformer_blocks.34.attn.to_q",
+      "transformer_blocks.34.attn.to_v",
+      "transformer_blocks.34.ff.net.0.proj",
+      "transformer_blocks.34.ff.net.2",
+      "transformer_blocks.35.attn.to_k",
+      "transformer_blocks.35.attn.to_out.0",
+      "transformer_blocks.35.attn.to_q",
+      "transformer_blocks.35.attn.to_v",
+      "transformer_blocks.35.ff.net.0.proj",
+      "transformer_blocks.35.ff.net.2",
+      "transformer_blocks.36.attn.to_k",
+      "transformer_blocks.36.attn.to_out.0",
+      "transformer_blocks.36.attn.to_q",
+      "transformer_blocks.36.attn.to_v",
+      "transformer_blocks.36.ff.net.0.proj",
+      "transformer_blocks.36.ff.net.2",
+      "transformer_blocks.37.attn.to_k",
+      "transformer_blocks.37.attn.to_out.0",
+      "transformer_blocks.37.attn.to_q",
+      "transformer_blocks.37.attn.to_v",
+      "transformer_blocks.37.ff.net.0.proj",
+      "transformer_blocks.37.ff.net.2",
+      "transformer_blocks.38.attn.to_k",
+      "transformer_blocks.38.attn.to_out.0",
+      "transformer_blocks.38.attn.to_q",
+      "transformer_blocks.38.attn.to_v",
+      "transformer_blocks.38.ff.net.0.proj",
+      "transformer_blocks.38.ff.net.2",
+      "transformer_blocks.39.attn.to_k",
+      "transformer_blocks.39.attn.to_out.0",
+      "transformer_blocks.39.attn.to_q",
+      "transformer_blocks.39.attn.to_v",
+      "transformer_blocks.39.ff.net.0.proj",
+      "transformer_blocks.39.ff.net.2",
+      "transformer_blocks.40.attn.to_k",
+      "transformer_blocks.40.attn.to_out.0",
+      "transformer_blocks.40.attn.to_q",
+      "transformer_blocks.40.attn.to_v",
+      "transformer_blocks.40.ff.net.0.proj",
+      "transformer_blocks.40.ff.net.2",
+      "transformer_blocks.41.attn.to_k",
+      "transformer_blocks.41.attn.to_out.0",
+      "transformer_blocks.41.attn.to_q",
+      "transformer_blocks.41.attn.to_v",
+      "transformer_blocks.41.ff.net.0.proj",
+      "transformer_blocks.41.ff.net.2",
+      "transformer_blocks.42.attn.to_k",
+      "transformer_blocks.42.attn.to_out.0",
+      "transformer_blocks.42.attn.to_q",
+      "transformer_blocks.42.attn.to_v",
+      "transformer_blocks.42.ff.net.0.proj",
+      "transformer_blocks.42.ff.net.2",
+      "transformer_blocks.43.attn.to_k",
+      "transformer_blocks.43.attn.to_out.0",
+      "transformer_blocks.43.attn.to_q",
+      "transformer_blocks.43.attn.to_v",
+      "transformer_blocks.43.ff.net.0.proj",
+      "transformer_blocks.43.ff.net.2",
+      "transformer_blocks.44.attn.to_k",
+      "transformer_blocks.44.attn.to_out.0",
+      "transformer_blocks.44.attn.to_q",
+      "transformer_blocks.44.attn.to_v",
+      "transformer_blocks.44.ff.net.0.proj",
+      "transformer_blocks.44.ff.net.2",
+      "transformer_blocks.45.attn.to_k",
+      "transformer_blocks.45.attn.to_out.0",
+      "transformer_blocks.45.attn.to_q",
+      "transformer_blocks.45.attn.to_v",
+      "transformer_blocks.45.ff.net.0.proj",
+      "transformer_blocks.45.ff.net.2",
+      "transformer_blocks.46.attn.to_k",
+      "transformer_blocks.46.attn.to_out.0",
+      "transformer_blocks.46.attn.to_q",
+      "transformer_blocks.46.attn.to_v",
+      "transformer_blocks.46.ff.net.0.proj",
+      "transformer_blocks.46.ff.net.2",
+      "transformer_blocks.47.attn.to_k",
+      "transformer_blocks.47.attn.to_out.0",
+      "transformer_blocks.47.attn.to_q",
+      "transformer_blocks.47.attn.to_v",
+      "transformer_blocks.47.ff.net.0.proj",
+      "transformer_blocks.47.ff.net.2",
+      "transformer_blocks.48.attn.to_k",
+      "transformer_blocks.48.attn.to_out.0",
+      "transformer_blocks.48.attn.to_q",
+      "transformer_blocks.48.attn.to_v",
+      "transformer_blocks.48.ff.net.0.proj",
+      "transformer_blocks.48.ff.net.2",
+      "transformer_blocks.49.attn.to_k",
+      "transformer_blocks.49.attn.to_out.0",
+      "transformer_blocks.49.attn.to_q",
+      "transformer_blocks.49.attn.to_v",
+      "transformer_blocks.49.ff.net.0.proj",
+      "transformer_blocks.49.ff.net.2"
+    ]
+  }
+}

ref2va_int8_convrot/diffusion_pytorch_model-00001-of-00004.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7630fd9f5d2002a9fac4748190d2b865dfad4b715fc973da974970db8b9ca7a9
+size 5338670032

ref2va_int8_convrot/diffusion_pytorch_model-00002-of-00004.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6f26d16bae5a7f5a05121e0e4360f95cf9a0163c18dc54e79fc42cdb89afc033
+size 5325764328

ref2va_int8_convrot/diffusion_pytorch_model-00003-of-00004.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e24ef706ac68eb76b1269ce0051d2b604ce23e8125d00d8c63ed64a5feb75b3e
+size 5342150184

ref2va_int8_convrot/diffusion_pytorch_model-00004-of-00004.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:54b1d4cc02696c749aaa46bdf506675b10c79904019be522179ddd65b2a0f9ba
+size 304661120

ref2va_int8_convrot/diffusion_pytorch_model.safetensors.index.json

@@ -0,0 +1,931 @@
+{
+  "metadata": {
+    "total_size": 16311140480
+  },
+  "weight_map": {
+    "audio_proj_in.bias": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "audio_proj_in.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "audio_proj_out.bias": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "audio_proj_out.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "context_embedder.bias": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "context_embedder.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "endpoint_time_embedder.linear_1.bias": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "endpoint_time_embedder.linear_1.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "endpoint_time_embedder.linear_2.bias": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "endpoint_time_embedder.linear_2.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "norm_out.linear.bias": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "norm_out.linear.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "norm_out.norm.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "proj_in.bias": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "proj_in.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "proj_out.bias": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "proj_out.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "time_embedder.linear_1.bias": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "time_embedder.linear_1.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "time_embedder.linear_2.bias": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "time_embedder.linear_2.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "token_refiner.final_norm.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "token_refiner.refiner_blocks.0.attn.norm_k.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "token_refiner.refiner_blocks.0.attn.norm_q.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "token_refiner.refiner_blocks.0.attn.to_k.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "token_refiner.refiner_blocks.0.attn.to_out.0.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "token_refiner.refiner_blocks.0.attn.to_q.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "token_refiner.refiner_blocks.0.attn.to_v.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "token_refiner.refiner_blocks.0.ff.net.0.proj.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "token_refiner.refiner_blocks.0.ff.net.2.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "token_refiner.refiner_blocks.0.norm1.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "token_refiner.refiner_blocks.0.norm2.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "token_refiner.refiner_blocks.1.attn.norm_k.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "token_refiner.refiner_blocks.1.attn.norm_q.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "token_refiner.refiner_blocks.1.attn.to_k.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "token_refiner.refiner_blocks.1.attn.to_out.0.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "token_refiner.refiner_blocks.1.attn.to_q.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "token_refiner.refiner_blocks.1.attn.to_v.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "token_refiner.refiner_blocks.1.ff.net.0.proj.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "token_refiner.refiner_blocks.1.ff.net.2.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "token_refiner.refiner_blocks.1.norm1.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "token_refiner.refiner_blocks.1.norm2.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.0.adaln_proj.linear.bias": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.0.adaln_proj.linear.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.0.attn.norm_k.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.0.attn.norm_q.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.0.attn.to_k.qweight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.0.attn.to_k.wscale": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.0.attn.to_out.0.qweight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.0.attn.to_out.0.wscale": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.0.attn.to_q.qweight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.0.attn.to_q.wscale": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.0.attn.to_v.qweight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.0.attn.to_v.wscale": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.0.ff.net.0.proj.qweight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.0.ff.net.0.proj.wscale": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.0.ff.net.2.qweight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.0.ff.net.2.wscale": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.0.norm1.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.0.norm2.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.1.adaln_proj.linear.bias": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.1.adaln_proj.linear.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.1.attn.norm_k.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.1.attn.norm_q.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.1.attn.to_k.down.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.1.attn.to_k.up.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.1.attn.to_out.0.down.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.1.attn.to_out.0.up.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.1.attn.to_q.down.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.1.attn.to_q.up.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.1.attn.to_v.down.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.1.attn.to_v.up.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.1.ff.net.0.proj.down.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.1.ff.net.0.proj.up.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.1.ff.net.2.down.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.1.ff.net.2.up.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.1.norm1.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.1.norm2.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.10.adaln_proj.linear.bias": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.10.adaln_proj.linear.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.10.attn.norm_k.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.10.attn.norm_q.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.10.attn.to_k.down.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.10.attn.to_k.up.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.10.attn.to_out.0.down.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.10.attn.to_out.0.up.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.10.attn.to_q.down.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.10.attn.to_q.up.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.10.attn.to_v.down.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.10.attn.to_v.up.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.10.ff.net.0.proj.down.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.10.ff.net.0.proj.up.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.10.ff.net.2.down.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.10.ff.net.2.up.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.10.norm1.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.10.norm2.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.11.adaln_proj.linear.bias": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.11.adaln_proj.linear.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.11.attn.norm_k.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.11.attn.norm_q.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.11.attn.to_k.down.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.11.attn.to_k.up.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.11.attn.to_out.0.down.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.11.attn.to_out.0.up.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.11.attn.to_q.down.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.11.attn.to_q.up.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.11.attn.to_v.down.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.11.attn.to_v.up.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.11.ff.net.0.proj.down.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.11.ff.net.0.proj.up.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.11.ff.net.2.down.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.11.ff.net.2.up.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.11.norm1.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.11.norm2.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.12.adaln_proj.linear.bias": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.12.adaln_proj.linear.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.12.attn.norm_k.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.12.attn.norm_q.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.12.attn.to_k.down.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.12.attn.to_k.up.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.12.attn.to_out.0.down.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.12.attn.to_out.0.up.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.12.attn.to_q.down.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.12.attn.to_q.up.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.12.attn.to_v.down.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.12.attn.to_v.up.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.12.ff.net.0.proj.down.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.12.ff.net.0.proj.up.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.12.ff.net.2.down.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.12.ff.net.2.up.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.12.norm1.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.12.norm2.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.13.adaln_proj.linear.bias": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.13.adaln_proj.linear.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.13.attn.norm_k.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.13.attn.norm_q.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.13.attn.to_k.down.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.13.attn.to_k.up.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.13.attn.to_out.0.down.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.13.attn.to_out.0.up.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.13.attn.to_q.down.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.13.attn.to_q.up.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.13.attn.to_v.down.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.13.attn.to_v.up.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.13.ff.net.0.proj.down.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.13.ff.net.0.proj.up.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.13.ff.net.2.down.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.13.ff.net.2.up.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.13.norm1.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.13.norm2.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.14.adaln_proj.linear.bias": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.14.adaln_proj.linear.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.14.attn.norm_k.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.14.attn.norm_q.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.14.attn.to_k.down.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.14.attn.to_k.up.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.14.attn.to_out.0.down.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.14.attn.to_out.0.up.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.14.attn.to_q.down.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.14.attn.to_q.up.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.14.attn.to_v.down.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.14.attn.to_v.up.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.14.ff.net.0.proj.down.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.14.ff.net.0.proj.up.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.14.ff.net.2.down.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.14.ff.net.2.up.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.14.norm1.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.14.norm2.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.15.adaln_proj.linear.bias": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.15.adaln_proj.linear.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.15.attn.norm_k.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.15.attn.norm_q.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.15.attn.to_k.down.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.15.attn.to_k.up.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.15.attn.to_out.0.down.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.15.attn.to_out.0.up.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.15.attn.to_q.down.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.15.attn.to_q.up.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.15.attn.to_v.down.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.15.attn.to_v.up.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.15.ff.net.0.proj.down.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.15.ff.net.0.proj.up.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.15.ff.net.2.down.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.15.ff.net.2.up.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.15.norm1.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.15.norm2.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.16.adaln_proj.linear.bias": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.16.adaln_proj.linear.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.16.attn.norm_k.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.16.attn.norm_q.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.16.attn.to_k.down.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.16.attn.to_k.up.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.16.attn.to_out.0.down.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.16.attn.to_out.0.up.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.16.attn.to_q.down.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.16.attn.to_q.up.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.16.attn.to_v.down.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.16.attn.to_v.up.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.16.ff.net.0.proj.down.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.16.ff.net.0.proj.up.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.16.ff.net.2.down.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.16.ff.net.2.up.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.16.norm1.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.16.norm2.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.17.adaln_proj.linear.bias": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.17.adaln_proj.linear.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.17.attn.norm_k.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.17.attn.norm_q.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.17.attn.to_k.down.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.17.attn.to_k.up.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.17.attn.to_out.0.down.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.17.attn.to_out.0.up.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.17.attn.to_q.down.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.17.attn.to_q.up.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.17.attn.to_v.down.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.17.attn.to_v.up.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.17.ff.net.0.proj.down.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.17.ff.net.0.proj.up.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.17.ff.net.2.down.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.17.ff.net.2.up.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.17.norm1.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.17.norm2.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.18.adaln_proj.linear.bias": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.18.adaln_proj.linear.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.18.attn.norm_k.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.18.attn.norm_q.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.18.attn.to_k.down.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.18.attn.to_k.up.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.18.attn.to_out.0.down.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.18.attn.to_out.0.up.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.18.attn.to_q.down.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.18.attn.to_q.up.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.18.attn.to_v.down.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.18.attn.to_v.up.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.18.ff.net.0.proj.down.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.18.ff.net.0.proj.up.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.18.ff.net.2.down.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.18.ff.net.2.up.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.18.norm1.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.18.norm2.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.19.adaln_proj.linear.bias": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.19.adaln_proj.linear.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.19.attn.norm_k.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.19.attn.norm_q.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.19.attn.to_k.qweight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.19.attn.to_k.wscale": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.19.attn.to_out.0.qweight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.19.attn.to_out.0.wscale": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.19.attn.to_q.qweight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.19.attn.to_q.wscale": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.19.attn.to_v.qweight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.19.attn.to_v.wscale": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.19.ff.net.0.proj.qweight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.19.ff.net.0.proj.wscale": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.19.ff.net.2.qweight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.19.ff.net.2.wscale": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.19.norm1.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.19.norm2.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.2.adaln_proj.linear.bias": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.2.adaln_proj.linear.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.2.attn.norm_k.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.2.attn.norm_q.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.2.attn.to_k.qweight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.2.attn.to_k.wscale": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.2.attn.to_out.0.qweight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.2.attn.to_out.0.wscale": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.2.attn.to_q.qweight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.2.attn.to_q.wscale": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.2.attn.to_v.qweight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.2.attn.to_v.wscale": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.2.ff.net.0.proj.qweight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.2.ff.net.0.proj.wscale": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.2.ff.net.2.qweight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.2.ff.net.2.wscale": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.2.norm1.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.2.norm2.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.20.adaln_proj.linear.bias": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.20.adaln_proj.linear.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.20.attn.norm_k.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.20.attn.norm_q.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.20.attn.to_k.down.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.20.attn.to_k.up.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.20.attn.to_out.0.down.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.20.attn.to_out.0.up.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.20.attn.to_q.down.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.20.attn.to_q.up.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.20.attn.to_v.down.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.20.attn.to_v.up.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.20.ff.net.0.proj.down.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.20.ff.net.0.proj.up.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.20.ff.net.2.down.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.20.ff.net.2.up.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.20.norm1.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.20.norm2.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.21.adaln_proj.linear.bias": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.21.adaln_proj.linear.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.21.attn.norm_k.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.21.attn.norm_q.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.21.attn.to_k.down.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.21.attn.to_k.up.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.21.attn.to_out.0.down.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.21.attn.to_out.0.up.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.21.attn.to_q.down.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.21.attn.to_q.up.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.21.attn.to_v.down.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.21.attn.to_v.up.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.21.ff.net.0.proj.down.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.21.ff.net.0.proj.up.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.21.ff.net.2.down.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.21.ff.net.2.up.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.21.norm1.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.21.norm2.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.22.adaln_proj.linear.bias": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.22.adaln_proj.linear.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.22.attn.norm_k.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.22.attn.norm_q.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.22.attn.to_k.down.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.22.attn.to_k.up.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.22.attn.to_out.0.down.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.22.attn.to_out.0.up.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.22.attn.to_q.down.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.22.attn.to_q.up.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.22.attn.to_v.down.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.22.attn.to_v.up.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.22.ff.net.0.proj.down.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.22.ff.net.0.proj.up.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.22.ff.net.2.down.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.22.ff.net.2.up.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.22.norm1.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.22.norm2.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.23.adaln_proj.linear.bias": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.23.adaln_proj.linear.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.23.attn.norm_k.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.23.attn.norm_q.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.23.attn.to_k.down.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.23.attn.to_k.up.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.23.attn.to_out.0.down.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.23.attn.to_out.0.up.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.23.attn.to_q.down.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.23.attn.to_q.up.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.23.attn.to_v.down.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.23.attn.to_v.up.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.23.ff.net.0.proj.down.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.23.ff.net.0.proj.up.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.23.ff.net.2.down.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.23.ff.net.2.up.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.23.norm1.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.23.norm2.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.24.adaln_proj.linear.bias": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.24.adaln_proj.linear.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.24.attn.norm_k.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.24.attn.norm_q.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.24.attn.to_k.qweight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.24.attn.to_k.wscale": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.24.attn.to_out.0.qweight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.24.attn.to_out.0.wscale": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.24.attn.to_q.qweight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.24.attn.to_q.wscale": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.24.attn.to_v.qweight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.24.attn.to_v.wscale": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.24.ff.net.0.proj.qweight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.24.ff.net.0.proj.wscale": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.24.ff.net.2.qweight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.24.ff.net.2.wscale": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.24.norm1.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.24.norm2.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.25.adaln_proj.linear.bias": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.25.adaln_proj.linear.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.25.attn.norm_k.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.25.attn.norm_q.weight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.25.attn.to_k.qweight": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.25.attn.to_k.wscale": "diffusion_pytorch_model-00001-of-00004.safetensors",
+    "transformer_blocks.25.attn.to_out.0.qweight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.25.attn.to_out.0.wscale": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.25.attn.to_q.qweight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.25.attn.to_q.wscale": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.25.attn.to_v.qweight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.25.attn.to_v.wscale": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.25.ff.net.0.proj.qweight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.25.ff.net.0.proj.wscale": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.25.ff.net.2.qweight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.25.ff.net.2.wscale": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.25.norm1.weight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.25.norm2.weight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.26.adaln_proj.linear.bias": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.26.adaln_proj.linear.weight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.26.attn.norm_k.weight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.26.attn.norm_q.weight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.26.attn.to_k.qweight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.26.attn.to_k.wscale": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.26.attn.to_out.0.qweight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.26.attn.to_out.0.wscale": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.26.attn.to_q.qweight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.26.attn.to_q.wscale": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.26.attn.to_v.qweight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.26.attn.to_v.wscale": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.26.ff.net.0.proj.qweight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.26.ff.net.0.proj.wscale": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.26.ff.net.2.qweight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.26.ff.net.2.wscale": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.26.norm1.weight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.26.norm2.weight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.27.adaln_proj.linear.bias": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.27.adaln_proj.linear.weight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.27.attn.norm_k.weight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.27.attn.norm_q.weight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.27.attn.to_k.qweight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.27.attn.to_k.wscale": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.27.attn.to_out.0.qweight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.27.attn.to_out.0.wscale": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.27.attn.to_q.qweight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.27.attn.to_q.wscale": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.27.attn.to_v.qweight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.27.attn.to_v.wscale": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.27.ff.net.0.proj.qweight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.27.ff.net.0.proj.wscale": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.27.ff.net.2.qweight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.27.ff.net.2.wscale": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.27.norm1.weight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.27.norm2.weight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.28.adaln_proj.linear.bias": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.28.adaln_proj.linear.weight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.28.attn.norm_k.weight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.28.attn.norm_q.weight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.28.attn.to_k.qweight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.28.attn.to_k.wscale": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.28.attn.to_out.0.qweight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.28.attn.to_out.0.wscale": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.28.attn.to_q.qweight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.28.attn.to_q.wscale": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.28.attn.to_v.qweight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.28.attn.to_v.wscale": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.28.ff.net.0.proj.qweight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.28.ff.net.0.proj.wscale": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.28.ff.net.2.qweight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.28.ff.net.2.wscale": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.28.norm1.weight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.28.norm2.weight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.29.adaln_proj.linear.bias": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.29.adaln_proj.linear.weight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.29.attn.norm_k.weight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.29.attn.norm_q.weight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.29.attn.to_k.qweight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.29.attn.to_k.wscale": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.29.attn.to_out.0.qweight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.29.attn.to_out.0.wscale": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.29.attn.to_q.qweight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.29.attn.to_q.wscale": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.29.attn.to_v.qweight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.29.attn.to_v.wscale": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.29.ff.net.0.proj.qweight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.29.ff.net.0.proj.wscale": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.29.ff.net.2.qweight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.29.ff.net.2.wscale": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.29.norm1.weight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.29.norm2.weight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.3.adaln_proj.linear.bias": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.3.adaln_proj.linear.weight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.3.attn.norm_k.weight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.3.attn.norm_q.weight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.3.attn.to_k.qweight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.3.attn.to_k.wscale": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.3.attn.to_out.0.qweight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.3.attn.to_out.0.wscale": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.3.attn.to_q.qweight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.3.attn.to_q.wscale": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.3.attn.to_v.qweight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.3.attn.to_v.wscale": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.3.ff.net.0.proj.qweight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.3.ff.net.0.proj.wscale": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.3.ff.net.2.qweight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.3.ff.net.2.wscale": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.3.norm1.weight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.3.norm2.weight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.30.adaln_proj.linear.bias": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.30.adaln_proj.linear.weight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.30.attn.norm_k.weight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.30.attn.norm_q.weight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.30.attn.to_k.qweight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.30.attn.to_k.wscale": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.30.attn.to_out.0.qweight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.30.attn.to_out.0.wscale": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.30.attn.to_q.qweight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.30.attn.to_q.wscale": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.30.attn.to_v.qweight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.30.attn.to_v.wscale": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.30.ff.net.0.proj.qweight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.30.ff.net.0.proj.wscale": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.30.ff.net.2.qweight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.30.ff.net.2.wscale": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.30.norm1.weight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.30.norm2.weight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.31.adaln_proj.linear.bias": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.31.adaln_proj.linear.weight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.31.attn.norm_k.weight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.31.attn.norm_q.weight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.31.attn.to_k.qweight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.31.attn.to_k.wscale": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.31.attn.to_out.0.qweight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.31.attn.to_out.0.wscale": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.31.attn.to_q.qweight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.31.attn.to_q.wscale": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.31.attn.to_v.qweight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.31.attn.to_v.wscale": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.31.ff.net.0.proj.qweight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.31.ff.net.0.proj.wscale": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.31.ff.net.2.qweight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.31.ff.net.2.wscale": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.31.norm1.weight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.31.norm2.weight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.32.adaln_proj.linear.bias": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.32.adaln_proj.linear.weight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.32.attn.norm_k.weight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.32.attn.norm_q.weight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.32.attn.to_k.qweight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.32.attn.to_k.wscale": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.32.attn.to_out.0.qweight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.32.attn.to_out.0.wscale": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.32.attn.to_q.qweight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.32.attn.to_q.wscale": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.32.attn.to_v.qweight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.32.attn.to_v.wscale": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.32.ff.net.0.proj.qweight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.32.ff.net.0.proj.wscale": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.32.ff.net.2.qweight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.32.ff.net.2.wscale": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.32.norm1.weight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.32.norm2.weight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.33.adaln_proj.linear.bias": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.33.adaln_proj.linear.weight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.33.attn.norm_k.weight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.33.attn.norm_q.weight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.33.attn.to_k.qweight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.33.attn.to_k.wscale": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.33.attn.to_out.0.qweight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.33.attn.to_out.0.wscale": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.33.attn.to_q.qweight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.33.attn.to_q.wscale": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.33.attn.to_v.qweight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.33.attn.to_v.wscale": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.33.ff.net.0.proj.qweight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.33.ff.net.0.proj.wscale": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.33.ff.net.2.qweight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.33.ff.net.2.wscale": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.33.norm1.weight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.33.norm2.weight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.34.adaln_proj.linear.bias": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.34.adaln_proj.linear.weight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.34.attn.norm_k.weight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.34.attn.norm_q.weight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.34.attn.to_k.qweight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.34.attn.to_k.wscale": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.34.attn.to_out.0.qweight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.34.attn.to_out.0.wscale": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.34.attn.to_q.qweight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.34.attn.to_q.wscale": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.34.attn.to_v.qweight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.34.attn.to_v.wscale": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.34.ff.net.0.proj.qweight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.34.ff.net.0.proj.wscale": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.34.ff.net.2.qweight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.34.ff.net.2.wscale": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.34.norm1.weight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.34.norm2.weight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.35.adaln_proj.linear.bias": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.35.adaln_proj.linear.weight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.35.attn.norm_k.weight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.35.attn.norm_q.weight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.35.attn.to_k.qweight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.35.attn.to_k.wscale": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.35.attn.to_out.0.qweight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.35.attn.to_out.0.wscale": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.35.attn.to_q.qweight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.35.attn.to_q.wscale": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.35.attn.to_v.qweight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.35.attn.to_v.wscale": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.35.ff.net.0.proj.qweight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.35.ff.net.0.proj.wscale": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.35.ff.net.2.qweight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.35.ff.net.2.wscale": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.35.norm1.weight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.35.norm2.weight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.36.adaln_proj.linear.bias": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.36.adaln_proj.linear.weight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.36.attn.norm_k.weight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.36.attn.norm_q.weight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.36.attn.to_k.qweight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.36.attn.to_k.wscale": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.36.attn.to_out.0.qweight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.36.attn.to_out.0.wscale": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.36.attn.to_q.qweight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.36.attn.to_q.wscale": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.36.attn.to_v.qweight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.36.attn.to_v.wscale": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.36.ff.net.0.proj.qweight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.36.ff.net.0.proj.wscale": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.36.ff.net.2.qweight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.36.ff.net.2.wscale": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.36.norm1.weight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.36.norm2.weight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.37.adaln_proj.linear.bias": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.37.adaln_proj.linear.weight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.37.attn.norm_k.weight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.37.attn.norm_q.weight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.37.attn.to_k.qweight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.37.attn.to_k.wscale": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.37.attn.to_out.0.qweight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.37.attn.to_out.0.wscale": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.37.attn.to_q.qweight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.37.attn.to_q.wscale": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.37.attn.to_v.qweight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.37.attn.to_v.wscale": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.37.ff.net.0.proj.qweight": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.37.ff.net.0.proj.wscale": "diffusion_pytorch_model-00002-of-00004.safetensors",
+    "transformer_blocks.37.ff.net.2.qweight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.37.ff.net.2.wscale": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.37.norm1.weight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.37.norm2.weight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.38.adaln_proj.linear.bias": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.38.adaln_proj.linear.weight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.38.attn.norm_k.weight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.38.attn.norm_q.weight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.38.attn.to_k.qweight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.38.attn.to_k.wscale": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.38.attn.to_out.0.qweight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.38.attn.to_out.0.wscale": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.38.attn.to_q.qweight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.38.attn.to_q.wscale": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.38.attn.to_v.qweight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.38.attn.to_v.wscale": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.38.ff.net.0.proj.qweight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.38.ff.net.0.proj.wscale": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.38.ff.net.2.qweight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.38.ff.net.2.wscale": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.38.norm1.weight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.38.norm2.weight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.39.adaln_proj.linear.bias": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.39.adaln_proj.linear.weight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.39.attn.norm_k.weight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.39.attn.norm_q.weight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.39.attn.to_k.qweight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.39.attn.to_k.wscale": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.39.attn.to_out.0.qweight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.39.attn.to_out.0.wscale": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.39.attn.to_q.qweight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.39.attn.to_q.wscale": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.39.attn.to_v.qweight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.39.attn.to_v.wscale": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.39.ff.net.0.proj.qweight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.39.ff.net.0.proj.wscale": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.39.ff.net.2.qweight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.39.ff.net.2.wscale": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.39.norm1.weight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.39.norm2.weight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.4.adaln_proj.linear.bias": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.4.adaln_proj.linear.weight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.4.attn.norm_k.weight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.4.attn.norm_q.weight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.4.attn.to_k.down.weight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.4.attn.to_k.up.weight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.4.attn.to_out.0.down.weight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.4.attn.to_out.0.up.weight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.4.attn.to_q.down.weight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.4.attn.to_q.up.weight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.4.attn.to_v.down.weight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.4.attn.to_v.up.weight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.4.ff.net.0.proj.down.weight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.4.ff.net.0.proj.up.weight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.4.ff.net.2.down.weight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.4.ff.net.2.up.weight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.4.norm1.weight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.4.norm2.weight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.40.adaln_proj.linear.bias": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.40.adaln_proj.linear.weight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.40.attn.norm_k.weight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.40.attn.norm_q.weight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.40.attn.to_k.qweight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.40.attn.to_k.wscale": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.40.attn.to_out.0.qweight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.40.attn.to_out.0.wscale": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.40.attn.to_q.qweight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.40.attn.to_q.wscale": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.40.attn.to_v.qweight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.40.attn.to_v.wscale": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.40.ff.net.0.proj.qweight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.40.ff.net.0.proj.wscale": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.40.ff.net.2.qweight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.40.ff.net.2.wscale": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.40.norm1.weight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.40.norm2.weight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.41.adaln_proj.linear.bias": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.41.adaln_proj.linear.weight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.41.attn.norm_k.weight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.41.attn.norm_q.weight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.41.attn.to_k.qweight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.41.attn.to_k.wscale": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.41.attn.to_out.0.qweight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.41.attn.to_out.0.wscale": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.41.attn.to_q.qweight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.41.attn.to_q.wscale": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.41.attn.to_v.qweight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.41.attn.to_v.wscale": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.41.ff.net.0.proj.qweight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.41.ff.net.0.proj.wscale": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.41.ff.net.2.qweight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.41.ff.net.2.wscale": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.41.norm1.weight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.41.norm2.weight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.42.adaln_proj.linear.bias": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.42.adaln_proj.linear.weight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.42.attn.norm_k.weight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.42.attn.norm_q.weight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.42.attn.to_k.qweight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.42.attn.to_k.wscale": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.42.attn.to_out.0.qweight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.42.attn.to_out.0.wscale": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.42.attn.to_q.qweight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.42.attn.to_q.wscale": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.42.attn.to_v.qweight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.42.attn.to_v.wscale": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.42.ff.net.0.proj.qweight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.42.ff.net.0.proj.wscale": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.42.ff.net.2.qweight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.42.ff.net.2.wscale": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.42.norm1.weight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.42.norm2.weight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.43.adaln_proj.linear.bias": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.43.adaln_proj.linear.weight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.43.attn.norm_k.weight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.43.attn.norm_q.weight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.43.attn.to_k.qweight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.43.attn.to_k.wscale": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.43.attn.to_out.0.qweight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.43.attn.to_out.0.wscale": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.43.attn.to_q.qweight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.43.attn.to_q.wscale": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.43.attn.to_v.qweight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.43.attn.to_v.wscale": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.43.ff.net.0.proj.qweight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.43.ff.net.0.proj.wscale": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.43.ff.net.2.qweight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.43.ff.net.2.wscale": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.43.norm1.weight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.43.norm2.weight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.44.adaln_proj.linear.bias": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.44.adaln_proj.linear.weight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.44.attn.norm_k.weight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.44.attn.norm_q.weight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.44.attn.to_k.qweight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.44.attn.to_k.wscale": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.44.attn.to_out.0.qweight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.44.attn.to_out.0.wscale": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.44.attn.to_q.qweight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.44.attn.to_q.wscale": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.44.attn.to_v.qweight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.44.attn.to_v.wscale": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.44.ff.net.0.proj.qweight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.44.ff.net.0.proj.wscale": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.44.ff.net.2.qweight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.44.ff.net.2.wscale": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.44.norm1.weight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.44.norm2.weight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.45.adaln_proj.linear.bias": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.45.adaln_proj.linear.weight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.45.attn.norm_k.weight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.45.attn.norm_q.weight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.45.attn.to_k.qweight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.45.attn.to_k.wscale": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.45.attn.to_out.0.qweight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.45.attn.to_out.0.wscale": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.45.attn.to_q.qweight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.45.attn.to_q.wscale": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.45.attn.to_v.qweight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.45.attn.to_v.wscale": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.45.ff.net.0.proj.qweight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.45.ff.net.0.proj.wscale": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.45.ff.net.2.qweight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.45.ff.net.2.wscale": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.45.norm1.weight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.45.norm2.weight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.46.adaln_proj.linear.bias": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.46.adaln_proj.linear.weight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.46.attn.norm_k.weight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.46.attn.norm_q.weight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.46.attn.to_k.qweight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.46.attn.to_k.wscale": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.46.attn.to_out.0.qweight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.46.attn.to_out.0.wscale": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.46.attn.to_q.qweight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.46.attn.to_q.wscale": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.46.attn.to_v.qweight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.46.attn.to_v.wscale": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.46.ff.net.0.proj.qweight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.46.ff.net.0.proj.wscale": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.46.ff.net.2.qweight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.46.ff.net.2.wscale": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.46.norm1.weight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.46.norm2.weight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.47.adaln_proj.linear.bias": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.47.adaln_proj.linear.weight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.47.attn.norm_k.weight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.47.attn.norm_q.weight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.47.attn.to_k.qweight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.47.attn.to_k.wscale": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.47.attn.to_out.0.qweight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.47.attn.to_out.0.wscale": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.47.attn.to_q.qweight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.47.attn.to_q.wscale": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.47.attn.to_v.qweight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.47.attn.to_v.wscale": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.47.ff.net.0.proj.qweight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.47.ff.net.0.proj.wscale": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.47.ff.net.2.qweight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.47.ff.net.2.wscale": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.47.norm1.weight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.47.norm2.weight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.48.adaln_proj.linear.bias": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.48.adaln_proj.linear.weight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.48.attn.norm_k.weight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.48.attn.norm_q.weight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.48.attn.to_k.qweight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.48.attn.to_k.wscale": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.48.attn.to_out.0.qweight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.48.attn.to_out.0.wscale": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.48.attn.to_q.qweight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.48.attn.to_q.wscale": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.48.attn.to_v.qweight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.48.attn.to_v.wscale": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.48.ff.net.0.proj.qweight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.48.ff.net.0.proj.wscale": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.48.ff.net.2.qweight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.48.ff.net.2.wscale": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.48.norm1.weight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.48.norm2.weight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.49.adaln_proj.linear.bias": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.49.adaln_proj.linear.weight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.49.attn.norm_k.weight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.49.attn.norm_q.weight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.49.attn.to_k.qweight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.49.attn.to_k.wscale": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.49.attn.to_out.0.qweight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.49.attn.to_out.0.wscale": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.49.attn.to_q.qweight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.49.attn.to_q.wscale": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.49.attn.to_v.qweight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.49.attn.to_v.wscale": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.49.ff.net.0.proj.qweight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.49.ff.net.0.proj.wscale": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.49.ff.net.2.qweight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.49.ff.net.2.wscale": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.49.norm1.weight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.49.norm2.weight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.5.adaln_proj.linear.bias": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.5.adaln_proj.linear.weight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.5.attn.norm_k.weight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.5.attn.norm_q.weight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.5.attn.to_k.down.weight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.5.attn.to_k.up.weight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.5.attn.to_out.0.down.weight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.5.attn.to_out.0.up.weight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.5.attn.to_q.down.weight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.5.attn.to_q.up.weight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.5.attn.to_v.down.weight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.5.attn.to_v.up.weight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.5.ff.net.0.proj.down.weight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.5.ff.net.0.proj.up.weight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.5.ff.net.2.down.weight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.5.ff.net.2.up.weight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.5.norm1.weight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.5.norm2.weight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.6.adaln_proj.linear.bias": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.6.adaln_proj.linear.weight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.6.attn.norm_k.weight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.6.attn.norm_q.weight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.6.attn.to_k.down.weight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.6.attn.to_k.up.weight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.6.attn.to_out.0.down.weight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.6.attn.to_out.0.up.weight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.6.attn.to_q.down.weight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.6.attn.to_q.up.weight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.6.attn.to_v.down.weight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.6.attn.to_v.up.weight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.6.ff.net.0.proj.down.weight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.6.ff.net.0.proj.up.weight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.6.ff.net.2.down.weight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.6.ff.net.2.up.weight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.6.norm1.weight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.6.norm2.weight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.7.adaln_proj.linear.bias": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.7.adaln_proj.linear.weight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.7.attn.norm_k.weight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.7.attn.norm_q.weight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.7.attn.to_k.down.weight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.7.attn.to_k.up.weight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.7.attn.to_out.0.down.weight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.7.attn.to_out.0.up.weight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.7.attn.to_q.down.weight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.7.attn.to_q.up.weight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.7.attn.to_v.down.weight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.7.attn.to_v.up.weight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.7.ff.net.0.proj.down.weight": "diffusion_pytorch_model-00003-of-00004.safetensors",
+    "transformer_blocks.7.ff.net.0.proj.up.weight": "diffusion_pytorch_model-00004-of-00004.safetensors",
+    "transformer_blocks.7.ff.net.2.down.weight": "diffusion_pytorch_model-00004-of-00004.safetensors",
+    "transformer_blocks.7.ff.net.2.up.weight": "diffusion_pytorch_model-00004-of-00004.safetensors",
+    "transformer_blocks.7.norm1.weight": "diffusion_pytorch_model-00004-of-00004.safetensors",
+    "transformer_blocks.7.norm2.weight": "diffusion_pytorch_model-00004-of-00004.safetensors",
+    "transformer_blocks.8.adaln_proj.linear.bias": "diffusion_pytorch_model-00004-of-00004.safetensors",
+    "transformer_blocks.8.adaln_proj.linear.weight": "diffusion_pytorch_model-00004-of-00004.safetensors",
+    "transformer_blocks.8.attn.norm_k.weight": "diffusion_pytorch_model-00004-of-00004.safetensors",
+    "transformer_blocks.8.attn.norm_q.weight": "diffusion_pytorch_model-00004-of-00004.safetensors",
+    "transformer_blocks.8.attn.to_k.down.weight": "diffusion_pytorch_model-00004-of-00004.safetensors",
+    "transformer_blocks.8.attn.to_k.up.weight": "diffusion_pytorch_model-00004-of-00004.safetensors",
+    "transformer_blocks.8.attn.to_out.0.down.weight": "diffusion_pytorch_model-00004-of-00004.safetensors",
+    "transformer_blocks.8.attn.to_out.0.up.weight": "diffusion_pytorch_model-00004-of-00004.safetensors",
+    "transformer_blocks.8.attn.to_q.down.weight": "diffusion_pytorch_model-00004-of-00004.safetensors",
+    "transformer_blocks.8.attn.to_q.up.weight": "diffusion_pytorch_model-00004-of-00004.safetensors",
+    "transformer_blocks.8.attn.to_v.down.weight": "diffusion_pytorch_model-00004-of-00004.safetensors",
+    "transformer_blocks.8.attn.to_v.up.weight": "diffusion_pytorch_model-00004-of-00004.safetensors",
+    "transformer_blocks.8.ff.net.0.proj.down.weight": "diffusion_pytorch_model-00004-of-00004.safetensors",
+    "transformer_blocks.8.ff.net.0.proj.up.weight": "diffusion_pytorch_model-00004-of-00004.safetensors",
+    "transformer_blocks.8.ff.net.2.down.weight": "diffusion_pytorch_model-00004-of-00004.safetensors",
+    "transformer_blocks.8.ff.net.2.up.weight": "diffusion_pytorch_model-00004-of-00004.safetensors",
+    "transformer_blocks.8.norm1.weight": "diffusion_pytorch_model-00004-of-00004.safetensors",
+    "transformer_blocks.8.norm2.weight": "diffusion_pytorch_model-00004-of-00004.safetensors"
+  }
+}

ref2va_int8_convrot/hyperflow_h3_flash_int8.py

@@ -0,0 +1,410 @@
+# Copyright 2026 The HyperFlow authors. Licensed under the Apache License, Version 2.0.
+"""INT8 W8A8 linears with a Hadamard-rotated input dimension.
+
+A dense ``y = x @ W^T`` is stored as ``qweight = round(W R / wscale)`` (int8, one fp32 scale per output channel), where
+``R`` is a block-diagonal regular Hadamard of size ``group``. ``R`` is orthonormal, so ``y = (x R) @ (W R)^T``: at run
+time the activation is rotated and quantized per token, multiplied in int8 with an exact int32 accumulator, and
+rescaled. The rotation spreads outlier channels across the group, which is what makes per-token int8 activations work.
+
+On CUDA the rotation + quantization and the GEMM are Triton kernels (compiled on first use). Every kernel fits the
+99 KiB of shared memory per block of Ampere, Ada and consumer Blackwell GPUs. The int8 GEMM runs as a Triton kernel or
+as cuBLASLt (``torch._int_mm``) plus a Triton epilogue; both use the same int32 accumulator and the same fp32 multiply
+order, so they give bitwise identical outputs. cuBLASLt is faster from compute capability 10 on, Triton on Hopper.
+The B300 (compute capability 10.3) has INT8 tensor cores at about 1/20 of the B200's, so there the int8 codes are
+multiplied on bf16 tensor cores into an fp32 accumulator: products are exact, and the sum matches the int32 one while
+it stays below 2**24, which the rotated activations keep well clear of.
+Elsewhere a plain torch implementation of the same scheme runs.
+
+`KERNEL_OPTIONS` (set by `load_pipeline`, no weight change) picks the kernels at run time:
+
+* ``int8_gemm``: ``"auto"`` (the choice above), ``"triton"``, ``"cublas"``, ``"bf16"``, or ``"sgl"``: CUTLASS
+  ``sgl_kernel.int8_scaled_mm`` with the scales in its epilogue, within 1 ulp of Triton and 1.25 to 1.45x faster on
+  H200 (Triton when sgl_kernel is missing).
+* ``rotate_quant``: ``"dense"`` (the kernel above) or ``"factored"``: a 256-channel group rotated as ``H16 X H16``, two
+  K = 16 dots in 8.5 KiB of shared memory, 1.6 to 2.0x faster on H200; it rounds the rotated activation like the dense
+  kernel up to rare one-code differences.
+"""
+
+from __future__ import annotations
+
+import functools
+from types import SimpleNamespace
+from typing import Any, ClassVar
+
+import torch
+from torch import Tensor, nn
+
+#: Run-time kernel choices.
+INT8_GEMMS = ("auto", "triton", "cublas", "sgl", "bf16")
+ROTATE_QUANTS = ("dense", "factored")
+
+#: Rows below this go through the torch path: `torch._int_mm` needs more than 16 rows, and so few rows are not worth
+#: a kernel launch.
+_MIN_KERNEL_ROWS = 17
+
+#: Run-time kernel choices for every `Int8RotLinear`; `load_pipeline` sets them.
+KERNEL_OPTIONS: dict[str, Any] = {"int8_gemm": "auto", "rotate_quant": "dense"}
+
+
+def set_kernel_options(*, int8_gemm: str | None = None, rotate_quant: str | None = None) -> None:
+    if int8_gemm is not None:
+        if int8_gemm not in INT8_GEMMS:
+            raise ValueError(f"int8_gemm must be one of {INT8_GEMMS}, got {int8_gemm!r}")
+        KERNEL_OPTIONS["int8_gemm"] = int8_gemm
+    if rotate_quant is not None:
+        if rotate_quant not in ROTATE_QUANTS:
+            raise ValueError(f"rotate_quant must be one of {ROTATE_QUANTS}, got {rotate_quant!r}")
+        KERNEL_OPTIONS["rotate_quant"] = rotate_quant
+
+
+@functools.cache
+def _sgl_int8_scaled_mm():
+    """sgl_kernel's ``int8_scaled_mm``, or None when sgl_kernel is not importable."""
+    try:
+        from sgl_kernel import int8_scaled_mm
+    except ImportError:
+        return None
+    return int8_scaled_mm
+
+
+def _has_bf16_mm() -> bool:
+    # `torch.mm(..., out_dtype=)` is PyTorch 2.8+.
+    return tuple(int(v) for v in torch.__version__.split(".")[:2]) >= (2, 8)
+
+
+@functools.cache
+def _auto_int8_gemm(device: torch.device) -> str:
+    capability = torch.cuda.get_device_capability(device)
+    if capability == (10, 3) and _has_bf16_mm():
+        return "bf16"
+    if hasattr(torch, "_int_mm") and capability[0] >= 10:
+        return "cublas"
+    return "triton"
+
+
+def resolve_int8_gemm(device: torch.device) -> str:
+    """The GEMM ``KERNEL_OPTIONS["int8_gemm"]`` runs as on ``device``: ``"triton"``, ``"cublas"``, ``"bf16"`` or
+    ``"sgl"``."""
+    choice = KERNEL_OPTIONS["int8_gemm"]
+    if choice == "auto":
+        return _auto_int8_gemm(device)
+    if (
+        (choice == "sgl" and _sgl_int8_scaled_mm() is None)
+        or (choice == "cublas" and not hasattr(torch, "_int_mm"))
+        or (choice == "bf16" and not _has_bf16_mm())
+    ):
+        return "triton"
+    return choice
+
+
+@functools.cache
+def regular_hadamard(group: int) -> Tensor:
+    """``kron(H4 / 2, ..., H4 / 2)`` of size ``group`` (a power of 4): orthonormal, symmetric, rows sum to one."""
+    h4 = torch.tensor([[1, 1, 1, -1], [1, 1, -1, 1], [1, -1, 1, 1], [-1, 1, 1, 1]], dtype=torch.float64) / 2
+    h, n = torch.ones(1, 1, dtype=torch.float64), 1
+    while n < group:
+        h, n = torch.kron(h, h4), n * 4
+    if n != group:
+        raise ValueError(f"Hadamard group {group} is not a power of 4")
+    return h.float()
+
+
+def hadamard_factors(group: int) -> tuple[int, int] | None:
+    """``(GA, GB)`` with ``regular_hadamard(group) == kron(regular_hadamard(GA), regular_hadamard(GB))``, both at
+    least 16 (the smallest bf16 ``tl.dot`` K); None when ``group`` cannot be split that way."""
+    n = group.bit_length() - 1
+    if group < 256 or group != 1 << n or n % 2:
+        return None
+    ga = 4 ** (n // 4)
+    return ga, group // ga
+
+
+@functools.cache
+def _rotation(group: int, device: torch.device) -> Tensor:
+    return regular_hadamard(group).to(device=device, dtype=torch.bfloat16)
+
+
+class Int8RotLinear(nn.Module):
+    """Bias-free W8A8 linear: ``y = (q(x R) @ qweight^T) * s_x * wscale``, ``R = blockdiag(regular Hadamard)``.
+
+    Linears that read one input (``to_q``, ``to_k``, ``to_v``) quantize it once: a layer with ``keep_input = True``
+    leaves its quantized input for the next layer, which reuses it if it receives the exact same, unmodified tensor.
+    Any other layer clears it, so nothing outlives the projections.
+    """
+
+    _last_input: ClassVar[dict[str, Any]] = {}
+
+    def __init__(self, in_features: int, out_features: int, group: int) -> None:
+        super().__init__()
+        if in_features % group:
+            raise ValueError(f"in_features {in_features} is not a multiple of the Hadamard group {group}")
+        self.in_features, self.out_features, self.group = in_features, out_features, group
+        self.keep_input = False
+        # Parameters, not buffers: diffusers' streamed group offloading moves module buffers on but never back off.
+        self.qweight = nn.Parameter(torch.zeros(out_features, in_features, dtype=torch.int8), requires_grad=False)
+        self.wscale = nn.Parameter(torch.ones(out_features, dtype=torch.float32), requires_grad=False)
+
+    def extra_repr(self) -> str:
+        return f"in_features={self.in_features}, out_features={self.out_features}, group={self.group}"
+
+    def _quantize(self, x: Tensor, x2: Tensor, kernels: SimpleNamespace) -> tuple[Tensor, Tensor]:
+        cache = Int8RotLinear._last_input
+        # Inference tensors (`torch.inference_mode`) have no version counter.
+        version = None if x.is_inference() else x._version
+        hit = cache.get("x") is x and cache.get("version") == version and cache.get("group") == self.group
+        quantized = cache["value"] if hit else kernels.rotate_quant(x2, self.group)
+        cache.clear()
+        if self.keep_input:
+            cache.update(x=x, version=version, group=self.group, value=quantized)
+        return quantized
+
+    def forward(self, x: Tensor) -> Tensor:
+        shape = x.shape
+        x2 = x.reshape(-1, self.in_features).to(torch.bfloat16).contiguous()
+        kernels = _kernels() if x2.is_cuda and x2.shape[0] >= _MIN_KERNEL_ROWS else None
+        if kernels is None:
+            Int8RotLinear._last_input.clear()
+            y = self._forward_torch(x2, _rotation(self.group, x2.device))
+        else:
+            xq, sx = self._quantize(x, x2, kernels)
+            y = kernels.int8_matmul(xq, self.qweight, sx, self.wscale)
+        return y.reshape(*shape[:-1], self.out_features).to(x.dtype)
+
+    def _forward_torch(self, x2: Tensor, rot: Tensor) -> Tensor:
+        """The same scheme in plain torch; the rotated activation is rounded to bf16 before quantization."""
+        g = self.group
+        xr = (x2.reshape(-1, self.in_features // g, g) @ rot).reshape(x2.shape).float()
+        sx = xr.abs().amax(1, keepdim=True).clamp_min(1e-12) / 127.0
+        xq = (xr / sx).round().clamp(-127, 127)
+        return ((xq @ self.qweight.float().t()) * sx * self.wscale[None, :]).to(torch.bfloat16)
+
+
+@functools.cache
+def _kernels() -> SimpleNamespace | None:
+    """``rotate_quant`` and ``int8_matmul`` as Triton kernels; ``None`` when Triton is not installed."""
+    try:
+        import triton
+        import triton.language as tl
+    except ImportError:
+        return None
+
+    ROTATE_BM = 64
+    FACTORED_BM = 16
+
+    @triton.jit
+    def rotate_quant_kernel(
+        x_ptr, h_ptr, q_ptr, s_ptr, M, K, stride_xm, stride_qm,
+        G: tl.constexpr, BM: tl.constexpr, BN: tl.constexpr,
+    ):  # fmt: skip
+        # Two passes over the row block: the per-token scale needs the absmax of the whole rotated row first.
+        # The Hadamard streams through in G x BN column tiles, so shared memory is (BM * G + G * BN) * 2 bytes.
+        rows = tl.program_id(0) * BM + tl.arange(0, BM)
+        rows64 = rows.to(tl.int64)
+        cols = tl.arange(0, G)
+        tile = tl.arange(0, BN)
+        rmask = rows < M
+        amax = tl.zeros((BM,), dtype=tl.float32)
+        for g in range(0, K, G):
+            x = tl.load(x_ptr + rows64[:, None] * stride_xm + (g + cols)[None, :], mask=rmask[:, None], other=0.0)
+            for n in range(0, G, BN):
+                h = tl.load(h_ptr + cols[:, None] * G + (n + tile)[None, :])
+                amax = tl.maximum(amax, tl.max(tl.abs(tl.dot(x, h)), axis=1))
+        scale = tl.maximum(amax, 1e-12) / 127.0
+        tl.store(s_ptr + rows, scale, mask=rmask)
+        inv = 1.0 / scale
+        for g in range(0, K, G):
+            x = tl.load(x_ptr + rows64[:, None] * stride_xm + (g + cols)[None, :], mask=rmask[:, None], other=0.0)
+            for n in range(0, G, BN):
+                h = tl.load(h_ptr + cols[:, None] * G + (n + tile)[None, :])
+                q = tl.extra.cuda.libdevice.rint(tl.dot(x, h) * inv[:, None])
+                q = tl.minimum(tl.maximum(q, -127.0), 127.0).to(tl.int8)
+                tl.store(q_ptr + rows64[:, None] * stride_qm + (g + n + tile)[None, :], q, mask=rmask[:, None])
+
+    @triton.jit
+    def rotate_quant_factored_kernel(
+        x_ptr, ha_ptr, hb_ptr, q_ptr, s_ptr, M, K, stride_xm, stride_qm,
+        GA: tl.constexpr, GB: tl.constexpr, BM: tl.constexpr,
+    ):  # fmt: skip
+        # Channel g + a * GB + b of a group is X[a, b]; the rotated group is H_GA X H_GB. Pass 0 takes the row absmax,
+        # pass 1 rotates again and writes int8. The fp32 input of the second dot is split into three bf16 terms
+        # (8 + 8 + 8 significand bits), so it is exact; a tf32 dot or a two-term split rounds it measurably.
+        # The second dot is batched per row, so its result is already in channel order.
+        G: tl.constexpr = GA * GB
+        rows = tl.program_id(0) * BM + tl.arange(0, BM)
+        rows64 = rows.to(tl.int64)
+        cols = tl.arange(0, G)
+        rmask = rows < M
+        ia = tl.arange(0, GA)
+        ib = tl.arange(0, GB)
+        ha = tl.broadcast_to(tl.load(ha_ptr + ia[:, None] * GA + ia[None, :])[None, :, :], (BM, GA, GA))
+        hb = tl.load(hb_ptr + ib[:, None] * GB + ib[None, :])
+        amax = tl.zeros((BM,), dtype=tl.float32)
+        inv = tl.zeros((BM,), dtype=tl.float32)
+        for p in tl.static_range(2):
+            for g in range(0, K, G):
+                x = tl.load(x_ptr + rows64[:, None] * stride_xm + (g + cols)[None, :], mask=rmask[:, None], other=0.0)
+                z = tl.reshape(tl.dot(tl.reshape(x, (BM * GA, GB)), hb), (BM, GA, GB))
+                z_hi = z.to(tl.bfloat16)
+                z = z - z_hi.to(tl.float32)
+                z_mid = z.to(tl.bfloat16)
+                z_lo = (z - z_mid.to(tl.float32)).to(tl.bfloat16)
+                y = tl.dot(ha, z_lo, acc=tl.dot(ha, z_mid, acc=tl.dot(ha, z_hi)))
+                y = tl.reshape(y, (BM, G))
+                if p == 0:
+                    amax = tl.maximum(amax, tl.max(tl.abs(y), axis=1))
+                else:
+                    q = tl.extra.cuda.libdevice.rint(y * inv[:, None])
+                    q = tl.minimum(tl.maximum(q, -127.0), 127.0).to(tl.int8)
+                    tl.store(q_ptr + rows64[:, None] * stride_qm + (g + cols)[None, :], q, mask=rmask[:, None])
+            if p == 0:
+                scale = tl.maximum(amax, 1e-12) / 127.0
+                tl.store(s_ptr + rows, scale, mask=rmask)
+                inv = 1.0 / scale
+
+    @functools.cache
+    def rotate_tile(device_index: int, group: int) -> int:
+        # At group 256, 128 columns take 96 KiB: that fits every GPU from Ampere on (99 KiB per block).
+        shared = triton.runtime.driver.active.utils.get_device_properties(device_index)["max_shared_mem"]
+        tile = 128 if (ROTATE_BM * group + group * 128) * 2 <= shared else 64
+        return min(tile, group)
+
+    configs = [
+        triton.Config({"BM": bm, "BN": bn, "BK": bk, "GROUP_M": 8}, num_stages=stages, num_warps=warps)
+        for bm, bn, bk, stages, warps in [
+            (128, 256, 128, 3, 8),
+            (256, 128, 128, 3, 8),
+            (128, 128, 128, 4, 8),
+            (128, 256, 64, 4, 8),
+            (256, 128, 64, 4, 8),
+            (128, 128, 64, 5, 4),
+        ]
+    ]
+
+    @triton.autotune(configs=configs, key=["M_BUCKET", "N", "K"])
+    @triton.jit
+    def int8_mm_kernel(
+        a_ptr, b_ptr, c_ptr, sa_ptr, sb_ptr, M, N, K, M_BUCKET, stride_am, stride_bn, stride_cm,
+        BM: tl.constexpr, BN: tl.constexpr, BK: tl.constexpr, GROUP_M: tl.constexpr,
+    ):  # fmt: skip
+        pid = tl.program_id(0)
+        num_m = tl.cdiv(M, BM)
+        num_n = tl.cdiv(N, BN)
+        group = GROUP_M * num_n
+        first_m = (pid // group) * GROUP_M
+        size_m = tl.minimum(num_m - first_m, GROUP_M)
+        pid_m = first_m + (pid % group) % size_m
+        pid_n = (pid % group) // size_m
+        rm = pid_m * BM + tl.arange(0, BM)
+        rn = pid_n * BN + tl.arange(0, BN)
+        rk = tl.arange(0, BK)
+        a_ptrs = a_ptr + rm.to(tl.int64)[:, None] * stride_am + rk[None, :]
+        b_ptrs = b_ptr + rn.to(tl.int64)[None, :] * stride_bn + rk[:, None]
+        acc = tl.zeros((BM, BN), dtype=tl.int32)
+        for k in range(0, K, BK):
+            a = tl.load(a_ptrs, mask=(rm[:, None] < M) & (k + rk[None, :] < K), other=0)
+            b = tl.load(b_ptrs, mask=(rn[None, :] < N) & (k + rk[:, None] < K), other=0)
+            acc += tl.dot(a, b, out_dtype=tl.int32)
+            a_ptrs += BK
+            b_ptrs += BK
+        sa = tl.load(sa_ptr + rm, mask=rm < M, other=0.0)
+        sb = tl.load(sb_ptr + rn, mask=rn < N, other=0.0)
+        c = (acc.to(tl.float32) * sa[:, None] * sb[None, :]).to(tl.bfloat16)
+        mask = (rm[:, None] < M) & (rn[None, :] < N)
+        tl.store(c_ptr + rm.to(tl.int64)[:, None] * stride_cm + rn[None, :], c, mask=mask)
+
+    @triton.jit
+    def rescale_kernel(acc_ptr, sa_ptr, sb_ptr, c_ptr, M, N, BM: tl.constexpr, BN: tl.constexpr):
+        rm = tl.program_id(0) * BM + tl.arange(0, BM)
+        rn = tl.program_id(1) * BN + tl.arange(0, BN)
+        mask = (rm[:, None] < M) & (rn[None, :] < N)
+        offsets = rm.to(tl.int64)[:, None] * N + rn[None, :]
+        acc = tl.load(acc_ptr + offsets, mask=mask, other=0)
+        sa = tl.load(sa_ptr + rm, mask=rm < M, other=0.0)
+        sb = tl.load(sb_ptr + rn, mask=rn < N, other=0.0)
+        tl.store(c_ptr + offsets, (acc.to(tl.float32) * sa[:, None] * sb[None, :]).to(tl.bfloat16), mask=mask)
+
+    def rotate_quant(x: Tensor, group: int) -> tuple[Tensor, Tensor]:
+        M, K = x.shape
+        q = torch.empty(M, K, device=x.device, dtype=torch.int8)
+        s = torch.empty(M, device=x.device, dtype=torch.float32)
+        factors = hadamard_factors(group) if KERNEL_OPTIONS["rotate_quant"] == "factored" else None
+        if factors is not None:
+            ga, gb = factors
+            ha, hb = _rotation(ga, x.device), _rotation(gb, x.device)
+            rotate_quant_factored_kernel[(triton.cdiv(M, FACTORED_BM),)](
+                x, ha, hb, q, s, M, K, x.stride(0), q.stride(0), GA=ga, GB=gb, BM=FACTORED_BM, num_warps=4,
+                num_stages=4,
+            )  # fmt: skip
+            return q, s
+        BN = rotate_tile(x.device.index, group)
+        rotate_quant_kernel[(triton.cdiv(M, ROTATE_BM),)](
+            x, _rotation(group, x.device), q, s, M, K, x.stride(0), q.stride(0), G=group, BM=ROTATE_BM, BN=BN,
+            num_warps=8, num_stages=1,
+        )  # fmt: skip
+        return q, s
+
+    def matmul_triton(a: Tensor, b: Tensor, sa: Tensor, sb: Tensor, c: Tensor) -> None:
+        M, K = a.shape
+        N = b.shape[0]
+        grid = lambda meta: (triton.cdiv(M, meta["BM"]) * triton.cdiv(N, meta["BN"]),)  # noqa: E731
+        m_bucket = 1 << max(M - 1, 1).bit_length()
+        int8_mm_kernel[grid](a, b, c, sa, sb, M, N, K, m_bucket, a.stride(0), b.stride(0), c.stride(0))
+
+    def matmul_cublas(a: Tensor, b: Tensor, sa: Tensor, sb: Tensor, c: Tensor) -> None:
+        # Row chunks keep the int32 accumulator near 1 GiB.
+        M, N = a.shape[0], b.shape[0]
+        chunk = max(128, ((1 << 28) // N) // 128 * 128)
+        for start in range(0, M, chunk):
+            stop = min(M, start + chunk)
+            if stop - start < _MIN_KERNEL_ROWS:
+                matmul_triton(a[start:stop], b, sa[start:stop], sb, c[start:stop])
+                continue
+            acc = torch._int_mm(a[start:stop], b.t())
+            grid = (triton.cdiv(stop - start, 64), triton.cdiv(N, 128))
+            rescale_kernel[grid](acc, sa[start:stop], sb, c[start:stop], stop - start, N, BM=64, BN=128, num_warps=8)
+
+    def matmul_bf16(a: Tensor, b: Tensor, sa: Tensor, sb: Tensor, c: Tensor) -> None:
+        M, N = a.shape[0], b.shape[0]
+        weight = b.to(torch.bfloat16)
+        chunk = max(128, ((1 << 28) // N) // 128 * 128)
+        for start in range(0, M, chunk):
+            stop = min(M, start + chunk)
+            acc = torch.mm(a[start:stop].to(torch.bfloat16), weight.t(), out_dtype=torch.float32)
+            grid = (triton.cdiv(stop - start, 64), triton.cdiv(N, 128))
+            rescale_kernel[grid](acc, sa[start:stop], sb, c[start:stop], stop - start, N, BM=64, BN=128, num_warps=8)
+
+    def int8_matmul(a: Tensor, b: Tensor, sa: Tensor, sb: Tensor) -> Tensor:
+        M, K = a.shape
+        N = b.shape[0]
+        gemm = resolve_int8_gemm(a.device)
+        if gemm == "sgl" and K % 16 == 0 and N % 16 == 0:
+            # b is the [N, K] row-major qweight; its transpose is the column-major [K, N] operand CUTLASS expects.
+            return _sgl_int8_scaled_mm()(a, b.t(), sa, sb, torch.bfloat16)
+        c = torch.empty(M, N, device=a.device, dtype=torch.bfloat16)
+        if gemm == "bf16":
+            matmul_bf16(a, b, sa, sb, c)
+        elif gemm == "cublas" and K % 8 == 0 and N % 8 == 0:
+            matmul_cublas(a, b, sa, sb, c)
+        else:
+            matmul_triton(a, b, sa, sb, c)
+        return c
+
+    jit = SimpleNamespace(
+        rotate_quant=rotate_quant_kernel,
+        rotate_quant_factored=rotate_quant_factored_kernel,
+        int8_mm=int8_mm_kernel,
+        rescale=rescale_kernel,
+    )
+    return SimpleNamespace(rotate_quant=rotate_quant, int8_matmul=int8_matmul, jit=jit, rotate_tile=rotate_tile)
+
+
+__all__ = [
+    "INT8_GEMMS",
+    "KERNEL_OPTIONS",
+    "ROTATE_QUANTS",
+    "Int8RotLinear",
+    "hadamard_factors",
+    "regular_hadamard",
+    "resolve_int8_gemm",
+    "set_kernel_options",
+]

ref2va_int8_convrot/hyperflow_h3_flash_transformer.py

@@ -0,0 +1,403 @@
+# Copyright 2026 The HyperFlow authors. Licensed under the Apache License, Version 2.0.
+#
+# `HyperFlowH3FlashTransformer3DModel.forward` is derived from `MiniMaxH3Transformer3DModel.forward` in diffusers
+# (`models/transformers/transformer_minimax_h3.py`), Copyright 2025 The MiniMax Team and The HuggingFace Team,
+# Apache License 2.0. See https://github.com/Video-Rebirth/hyperflow-h3-flash/blob/main/THIRD_PARTY_NOTICES.md.
+"""The HyperFlow H3 Flash transformer: MiniMax-H3, compressed and conditioned on two times.
+
+Four changes to `MiniMaxH3Transformer3DModel`, all driven by `config.json`:
+
+* **Two-time conditioning.** An `endpoint_time_embedder` embeds the step's endpoint ``r`` next to the current time
+  ``t``; every AdaLN reads ``temb(t, r) = emb_t(t) + hyperflow_gate * (emb_r(r) - emb_t(t))``.
+* **Dropped blocks** (`dropped_blocks`): those blocks are identities and have no weights.
+* **Low-rank linears** (`lowrank_ranks`): ``W ~= up @ down``, stored as `<name>.down.weight` (rank x in) and
+  `<name>.up.weight` (out x rank). Ranks are zero-padded to multiples of 64, in the checkpoint or at load
+  (`pad_lowrank_ranks`), so both GEMMs are aligned; `set_lowrank_rank_multiple` re-pads them at run time.
+* **INT8 linears** (`quantization`): W8A8 with a Hadamard-rotated input, see `int8.py`.
+
+Embeddings, AdaLN, the token refiner and the output heads are the base model's, in bf16 / fp32. The block
+feed-forwards run in token chunks (`ChunkedFeedForward`, `set_ff_chunk`), which bounds their activations;
+`set_block_chunk` slices every token-wise op of a block.
+"""
+
+from __future__ import annotations
+
+import sys
+from typing import Any, ClassVar
+
+import torch
+from diffusers import MiniMaxH3Transformer3DModel
+from diffusers.configuration_utils import register_to_config
+from diffusers.models.embeddings import TimestepEmbedding
+from diffusers.models.modeling_utils import get_parameter_dtype
+from diffusers.models.transformers.transformer_minimax_h3 import (
+    MINIMAX_H3_MODALITY_NUM,
+    MiniMaxH3TransformerOutput,
+)
+from diffusers.utils import apply_lora_scale
+from torch import Tensor, nn
+
+from .hyperflow_h3_flash_int8 import Int8RotLinear
+
+#: Tokens per feed-forward chunk; a 5.2 s clip has about 38k, a 14.4 s clip about 105k.
+FF_CHUNK_TOKENS = 16384
+#: Low-rank ranks are padded to multiples of this at load, so both GEMMs of a low-rank linear are aligned.
+RANK_MULTIPLE = 64
+
+
+class LowRankLinear(nn.Module):
+    """``y = up(down(x))``, the factorized replacement of a bias-free `nn.Linear`."""
+
+    def __init__(self, in_features: int, out_features: int, rank: int) -> None:
+        super().__init__()
+        self.in_features, self.out_features, self.rank = in_features, out_features, rank
+        self.down = nn.Linear(in_features, rank, bias=False)
+        self.up = nn.Linear(rank, out_features, bias=False)
+
+    def forward(self, x: Tensor) -> Tensor:
+        return self.up(self.down(x))
+
+    @torch.no_grad()
+    def pad_rank(self, multiple: int = RANK_MULTIPLE) -> bool:
+        """Zero-pad the rank up to a multiple of ``multiple``: new rows of ``down`` and columns of ``up`` are zero, so
+        ``up @ down`` is unchanged. Returns whether the rank changed."""
+        rank = -(-self.rank // multiple) * multiple
+        if rank == self.rank:
+            return False
+        extra = rank - self.rank
+        down, up = self.down.weight, self.up.weight
+        self.down.weight = nn.Parameter(torch.cat([down, down.new_zeros(extra, down.shape[1])]), requires_grad=False)
+        self.up.weight = nn.Parameter(torch.cat([up, up.new_zeros(up.shape[0], extra)], dim=1), requires_grad=False)
+        self.down.out_features = self.up.in_features = self.rank = rank
+        return True
+
+
+class ChunkedFeedForward(nn.Module):
+    """A block's feed-forward run over at most ``chunk_tokens`` tokens at a time; it keeps the block's ``net``.
+
+    The feed-forward is token-wise, so the chunks compute the same function; only the rounding of bf16 GEMMs can
+    change with their row count. It bounds the SwiGLU input, ``2 * ffn_dim`` wide per token (6 GiB for a 14.4 s
+    clip), to one chunk.
+    """
+
+    def __init__(self, net: nn.ModuleList, out_features: int, chunk_tokens: int) -> None:
+        super().__init__()
+        self.net, self.out_features, self.chunk_tokens = net, out_features, chunk_tokens
+
+    def _forward(self, hidden_states: Tensor) -> Tensor:
+        for module in self.net:
+            hidden_states = module(hidden_states)
+        return hidden_states
+
+    def forward(self, hidden_states: Tensor) -> Tensor:
+        if hidden_states.shape[-2] <= self.chunk_tokens:
+            return self._forward(hidden_states)
+        out = hidden_states.new_empty(*hidden_states.shape[:-1], self.out_features)
+        for start in range(0, hidden_states.shape[-2], self.chunk_tokens):
+            stop = start + self.chunk_tokens
+            out[..., start:stop, :] = self._forward(hidden_states[..., start:stop, :])
+        return out
+
+
+class DroppedBlock(nn.Module):
+    """A removed transformer block: the hidden states go straight to the next block.
+
+    The signature is the block's own: context parallel hooks address `adaln_indices` by name.
+    """
+
+    def forward(self, hidden_states: Tensor, temb=None, adaln_indices=None, rotary_emb=None, attention_mask=None):
+        return hidden_states
+
+
+def _replace_linear(model: nn.Module, name: str, make: type[nn.Module], *args: Any) -> None:
+    owner_name, _, attr = name.rpartition(".")
+    owner = model.get_submodule(owner_name)
+    dense = owner[int(attr)] if isinstance(owner, nn.ModuleList) else getattr(owner, attr)
+    if type(dense) is not nn.Linear or dense.bias is not None:
+        raise TypeError(f"{name} is not a bias-free nn.Linear")
+    replacement = make(dense.in_features, dense.out_features, *args)
+    if isinstance(owner, nn.ModuleList):
+        owner[int(attr)] = replacement
+    else:
+        setattr(owner, attr, replacement)
+
+
+class HyperFlowH3FlashTransformer3DModel(MiniMaxH3Transformer3DModel):
+    """`MiniMaxH3Transformer3DModel` conditioned on ``(t, r)``, with dropped blocks, low-rank and INT8 linears."""
+
+    # Matched against whole dotted name parts at load time, so `time_embedder` does not cover the endpoint embedder.
+    _keep_in_fp32_modules: ClassVar[list[str]] = [
+        *MiniMaxH3Transformer3DModel._keep_in_fp32_modules,
+        "endpoint_time_embedder",
+        "wscale",
+    ]
+
+    @register_to_config
+    def __init__(
+        self,
+        num_attention_heads: int = 56,
+        attention_head_dim: int = 128,
+        hidden_size: int = 5376,
+        num_layers: int = 50,
+        num_refiner_layers: int = 2,
+        ffn_dim: int = 14336,
+        in_channels: int = 24,
+        audio_in_channels: int = 32,
+        patch_size: tuple[int, int, int] = (1, 2, 2),
+        text_dim: int = 5120,
+        freq_dim: int = 256,
+        time_embed_hidden_dim: int = 5376,
+        time_embed_dim: int = 16,
+        rope_freq_dim: int = 16,
+        rope_theta: float = 10000.0,
+        norm_eps: float = 1e-5,
+        qk_norm_eps: float = 1e-5,
+        final_norm_eps: float = 1e-5,
+        hyperflow_gate: float = 0.25,
+        dropped_blocks: list[int] | None = None,
+        lowrank_ranks: dict[str, int] | None = None,
+        quantization: dict[str, Any] | None = None,
+    ) -> None:
+        super().__init__(
+            num_attention_heads=num_attention_heads,
+            attention_head_dim=attention_head_dim,
+            hidden_size=hidden_size,
+            num_layers=num_layers,
+            num_refiner_layers=num_refiner_layers,
+            ffn_dim=ffn_dim,
+            in_channels=in_channels,
+            audio_in_channels=audio_in_channels,
+            patch_size=patch_size,
+            text_dim=text_dim,
+            freq_dim=freq_dim,
+            time_embed_hidden_dim=time_embed_hidden_dim,
+            time_embed_dim=time_embed_dim,
+            rope_freq_dim=rope_freq_dim,
+            rope_theta=rope_theta,
+            norm_eps=norm_eps,
+            qk_norm_eps=qk_norm_eps,
+            final_norm_eps=final_norm_eps,
+        )
+        self.endpoint_time_embedder = TimestepEmbedding(
+            in_channels=freq_dim, time_embed_dim=time_embed_hidden_dim, out_dim=time_embed_dim
+        )
+
+        dropped = set(dropped_blocks or [])
+        for index, block in enumerate(self.transformer_blocks):
+            if index in dropped:
+                self.transformer_blocks[index] = DroppedBlock()
+            else:
+                block.ff = ChunkedFeedForward(block.ff.net, block.ff.net[-1].out_features, FF_CHUNK_TOKENS)
+        for name, rank in (lowrank_ranks or {}).items():
+            if int(name.split(".")[1]) not in dropped:
+                _replace_linear(self, name, LowRankLinear, int(rank))
+        if quantization:
+            if quantization.get("method") != "int8_convrot":
+                raise ValueError(f"unsupported quantization method {quantization.get('method')!r}")
+            for name in quantization["linears"]:
+                _replace_linear(self, name, Int8RotLinear, int(quantization["group"]))
+        for block in self.transformer_blocks:
+            attn = getattr(block, "attn", None)
+            for name in ("to_q", "to_k"):
+                if isinstance(layer := getattr(attn, name, None), Int8RotLinear):
+                    layer.keep_input = True
+
+    def pad_lowrank_ranks(self, multiple: int = RANK_MULTIPLE) -> int:
+        """Zero-pad every low-rank linear to a rank multiple of ``multiple`` (see `LowRankLinear.pad_rank`) for a
+        checkpoint stored at its unpadded ranks. Returns how many linears changed."""
+        ranks = dict(self.config.lowrank_ranks or {})
+        padded = 0
+        for name, module in self.named_modules():
+            if isinstance(module, LowRankLinear) and module.pad_rank(multiple):
+                ranks[name] = module.rank
+                padded += 1
+        if padded:
+            self.register_to_config(lowrank_ranks=ranks)
+        return padded
+
+    def time_embedding(self, timestep: Tensor, endpoint_timestep: Tensor) -> Tensor:
+        """``temb`` of every distinct ``(t, r)`` pair, at the time embedders' precision (fp32)."""
+        if endpoint_timestep.shape != timestep.shape:
+            raise ValueError(f"endpoint_timestep {list(endpoint_timestep.shape)} != timestep {list(timestep.shape)}")
+        dtype = get_parameter_dtype(self.time_embedder)
+        current = self.time_embedder(self.time_proj(timestep).to(dtype))
+        endpoint = self.endpoint_time_embedder(self.time_proj(endpoint_timestep).to(dtype))
+        return current + self.config.hyperflow_gate * (endpoint - current)
+
+    @apply_lora_scale("attention_kwargs")
+    def forward(
+        self,
+        hidden_states: Tensor,
+        audio_hidden_states: Tensor,
+        encoder_hidden_states: Tensor,
+        timestep: Tensor,
+        timestep_indices: Tensor,
+        token_tags: Tensor,
+        position_ids: Tensor,
+        video_indices: Tensor,
+        audio_indices: Tensor,
+        text_indices: Tensor,
+        endpoint_timestep: Tensor | None = None,
+        attention_kwargs: dict[str, Any] | None = None,
+        return_dict: bool = True,
+    ) -> MiniMaxH3TransformerOutput | tuple[Tensor, Tensor]:
+        """The base forward plus ``endpoint_timestep``: ``(timestep[i], endpoint_timestep[i])`` is the ``i``-th pair.
+
+        ``timestep_indices`` maps every row of the packed sequence to its pair. Build the pairs with
+        `hyperflow_h3_flash.schedule.build_row_time_pairs`, or run the model through `flash_blocks(...)`.
+        """
+        if endpoint_timestep is None:
+            raise ValueError(
+                "HyperFlow H3 Flash needs `endpoint_timestep`, the endpoint `r` of every `(t, r)` pair: it is a "
+                "few-step model and has no single-time mode. Run it through `hyperflow_h3_flash.flash_blocks(...)`."
+            )
+        if position_ids.ndim != 2 or position_ids.shape[-1] != 3:
+            raise ValueError(f"`position_ids` must be a `(seq_len, 3)` tensor, got {list(position_ids.shape)}.")
+        sequence_length = position_ids.shape[0]
+        if token_tags.shape != (sequence_length,) or timestep_indices.shape != (sequence_length,):
+            raise ValueError("`token_tags` and `timestep_indices` must both be `(seq_len,)` tensors.")
+
+        rotary_emb = self.rope(position_ids)
+
+        video_embeds = self.proj_in(hidden_states.to(get_parameter_dtype(self.proj_in)))
+        audio_embeds = self.audio_proj_in(audio_hidden_states.to(get_parameter_dtype(self.audio_proj_in)))
+        text_embeds = self.context_embedder(encoder_hidden_states.to(get_parameter_dtype(self.context_embedder)))
+        text_embeds = self.token_refiner(text_embeds)
+
+        hidden_states = text_embeds.new_zeros((text_embeds.shape[0], sequence_length, text_embeds.shape[-1]))
+        hidden_states = hidden_states.index_copy(1, text_indices, text_embeds)
+        hidden_states = hidden_states.index_copy(1, video_indices, video_embeds.to(text_embeds.dtype))
+        hidden_states = hidden_states.index_copy(1, audio_indices, audio_embeds.to(text_embeds.dtype))
+
+        temb = self.time_embedding(timestep, endpoint_timestep)
+        adaln_indices = timestep_indices * MINIMAX_H3_MODALITY_NUM + token_tags
+        for block in self.transformer_blocks:
+            hidden_states = block(hidden_states, temb, adaln_indices, rotary_emb)
+
+        hidden_states = self.norm_out(hidden_states, temb, timestep_indices).to(get_parameter_dtype(self.proj_out))
+        video_output = self.proj_out(hidden_states).index_select(1, video_indices)
+        audio_output = self.audio_proj_out(hidden_states).index_select(1, audio_indices)
+        if not return_dict:
+            return (video_output, audio_output)
+        return MiniMaxH3TransformerOutput(sample=video_output, audio_sample=audio_output)
+
+
+def _used_rank(layer: LowRankLinear) -> int:
+    """The rank without the zero padding (trailing zero rows of ``down`` and columns of ``up``)."""
+    used = (layer.down.weight.detach().abs().amax(1) != 0) | (layer.up.weight.detach().abs().amax(0) != 0)
+    nonzero = used.nonzero()
+    return int(nonzero[-1]) + 1 if len(nonzero) else 1
+
+
+def set_lowrank_rank_multiple(model: nn.Module, multiple: int) -> int:
+    """Re-pad every low-rank linear to a rank multiple of ``multiple`` (0: no padding); returns the layers changed.
+
+    Zero rows and columns add exact zeros, so any multiple computes the same function; the GEMM shapes change, and with
+    them the bf16 rounding (8 is 2 to 7 % faster than 64 on H200). Run time only: the config still lists the stored
+    ranks, so do not `save_pretrained` the re-padded model.
+    """
+    changed = 0
+    for layer in model.modules():
+        if not isinstance(layer, LowRankLinear):
+            continue
+        rank = _used_rank(layer)
+        target = -(-rank // multiple) * multiple if multiple > 0 else rank
+        if target == layer.rank:
+            continue
+        down, up = layer.down.weight.detach(), layer.up.weight.detach()
+        if target < layer.rank:
+            down, up = down[:target], up[:, :target]
+        else:
+            down = torch.cat([down, down.new_zeros(target - layer.rank, down.shape[1])])
+            up = torch.cat([up, up.new_zeros(up.shape[0], target - layer.rank)], dim=1)
+        layer.down.weight = nn.Parameter(down.contiguous(), requires_grad=False)
+        layer.up.weight = nn.Parameter(up.contiguous(), requires_grad=False)
+        layer.rank = layer.down.out_features = layer.up.in_features = target
+        changed += 1
+    return changed
+
+
+def set_ff_chunk(model: nn.Module, tokens: int) -> None:
+    """Run every feed-forward over at most ``tokens`` tokens at a time (0: whole sequence)."""
+    for module in model.modules():
+        if isinstance(module, ChunkedFeedForward):
+            module.chunk_tokens = tokens if tokens > 0 else sys.maxsize
+
+
+def chunk_block(block: nn.Module, chunk: int) -> None:
+    """Run every token-wise op of a transformer block over ``chunk``-token slices; only the attention sees all of q /
+    k / v.
+
+    The unchunked block holds about a dozen ``(seq, hidden)`` tensors at once (each modulation picked per token, the
+    modulated norm, the residual sums), 1.2 GB apiece at 108k tokens. Per-row results are the same up to the bf16 GEMM
+    rounding for the smaller row count. Apply before context parallelism and offloading, which hook the block.
+    """
+    import diffusers.models.transformers.transformer_minimax_h3 as h3
+
+    attn = block.attn
+
+    def forward(hidden_states, temb, adaln_indices, rotary_emb, attention_mask=None):
+        shift_msa, scale_msa, gate_msa, shift_mlp, scale_mlp, gate_mlp = block.adaln_proj(temb)
+        batch, seq, _ = hidden_states.shape
+        cos, sin = rotary_emb
+        qkv = None
+        for start in range(0, seq, chunk):
+            rows = slice(start, min(start + chunk, seq))
+            idx = adaln_indices[rows]
+            h = block.norm1(hidden_states[:, rows])
+            h = h * (1.0 + scale_msa.index_select(0, idx)) + shift_msa.index_select(0, idx)
+            parts = []
+            for proj, norm in ((attn.to_q, attn.norm_q), (attn.to_k, attn.norm_k), (attn.to_v, None)):
+                x = proj(h).unflatten(-1, (attn.heads, -1))
+                if norm is not None:
+                    x = h3._apply_rotary_emb(norm(x), cos[rows], sin[rows])
+                parts.append(x)
+            if qkv is None:
+                qkv = [x.new_empty((batch, seq, *x.shape[2:])) for x in parts]
+            for full, x in zip(qkv, parts, strict=True):
+                full[:, rows] = x
+            del h, parts, x
+        out = h3.dispatch_attention_fn(
+            *qkv,
+            attn_mask=attention_mask,
+            dropout_p=0.0,
+            is_causal=False,
+            backend=attn.processor._attention_backend,
+            parallel_config=attn.processor._parallel_config,
+        )
+        dtype = qkv[0].dtype
+        del qkv
+        result = torch.empty_like(hidden_states)
+        for start in range(0, seq, chunk):
+            rows = slice(start, min(start + chunk, seq))
+            idx = adaln_indices[rows]
+            a = attn.to_out[1](attn.to_out[0](out[:, rows].flatten(2, 3).to(dtype)))
+            h = hidden_states[:, rows] + gate_msa.index_select(0, idx) * a
+            n = block.norm2(h)
+            n = n * (1.0 + scale_mlp.index_select(0, idx)) + shift_mlp.index_select(0, idx)
+            result[:, rows] = h + gate_mlp.index_select(0, idx) * block.ff(n)
+            del a, h, n
+        return result
+
+    block.forward = forward
+
+
+def set_block_chunk(model: nn.Module, tokens: int) -> None:
+    """`chunk_block` on every kept block (0: off). Call once, before context parallelism and offloading."""
+    if tokens > 0:
+        for block in model.transformer_blocks:
+            if not isinstance(block, DroppedBlock):
+                chunk_block(block, tokens)
+
+
+__all__ = [
+    "RANK_MULTIPLE",
+    "ChunkedFeedForward",
+    "DroppedBlock",
+    "HyperFlowH3FlashTransformer3DModel",
+    "LowRankLinear",
+    "chunk_block",
+    "set_block_chunk",
+    "set_ff_chunk",
+    "set_lowrank_rank_multiple",
+]