Instructions to use hf-internal-testing/tiny-qwenimage21-modular with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Diffusers
How to use hf-internal-testing/tiny-qwenimage21-modular with Diffusers:
pip install -U diffusers transformers accelerate
import torch from diffusers import DiffusionPipeline # switch to "mps" for apple devices pipe = DiffusionPipeline.from_pretrained("hf-internal-testing/tiny-qwenimage21-modular", dtype=torch.bfloat16, device_map="cuda") prompt = "Astronaut in a jungle, cold color palette, muted colors, detailed, 8k" image = pipe(prompt).images[0] - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- Draw Things
- DiffusionBee
Copy of akshan-main/tiny-qwenimage21-modular-pipe@5a780beb for diffusers modular tests (PR #14976)
Browse files- .gitattributes +1 -0
- README.md +91 -0
- modular_model_index.json +76 -0
- processor/chat_template.jinja +7 -0
- processor/processor_config.json +60 -0
- processor/tokenizer.json +3 -0
- processor/tokenizer_config.json +36 -0
- scheduler/scheduler_config.json +18 -0
- text_encoder/config.json +60 -0
- text_encoder/generation_config.json +7 -0
- text_encoder/model.safetensors +3 -0
- transformer/config.json +20 -0
- transformer/diffusion_pytorch_model.safetensors +3 -0
- vae/config.json +50 -0
- vae/diffusion_pytorch_model.safetensors +3 -0
.gitattributes
CHANGED
|
@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
|
| 33 |
*.zip filter=lfs diff=lfs merge=lfs -text
|
| 34 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
|
|
|
|
|
| 33 |
*.zip filter=lfs diff=lfs merge=lfs -text
|
| 34 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
| 36 |
+
processor/tokenizer.json filter=lfs diff=lfs merge=lfs -text
|
README.md
ADDED
|
@@ -0,0 +1,91 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
library_name: diffusers
|
| 3 |
+
tags:
|
| 4 |
+
- modular-diffusers
|
| 5 |
+
- diffusers
|
| 6 |
+
- qwenimage21
|
| 7 |
+
- text-to-image
|
| 8 |
+
---
|
| 9 |
+
This is a modular diffusion pipeline built with 🧨 Diffusers' modular pipeline framework.
|
| 10 |
+
|
| 11 |
+
**Pipeline Type**: QwenImage21AutoBlocks
|
| 12 |
+
|
| 13 |
+
**Description**: Auto Modular pipeline for text-to-image and image-conditioned generation using Qwen-Image 2.1.
|
| 14 |
+
- for text-to-image generation, all you need to provide is `prompt`
|
| 15 |
+
- for image-conditioned generation, you need to provide `prompt` and `image` (one image or a list)
|
| 16 |
+
|
| 17 |
+
This pipeline uses a 4-block architecture that can be customized and extended.
|
| 18 |
+
|
| 19 |
+
## Example Usage
|
| 20 |
+
|
| 21 |
+
[TODO]
|
| 22 |
+
|
| 23 |
+
## Pipeline Architecture
|
| 24 |
+
|
| 25 |
+
This modular pipeline is composed of the following blocks:
|
| 26 |
+
|
| 27 |
+
1. **text_encoder** (`QwenImage21AutoTextEncoderStep`)
|
| 28 |
+
- Text encoder step that encodes the prompt, together with the condition images when there are any.
|
| 29 |
+
2. **vae_encoder** (`QwenImage21AutoVaeEncoderStep`)
|
| 30 |
+
- VAE encoder step that encodes the condition images into their latent representations.
|
| 31 |
+
3. **denoise** (`QwenImage21AutoCoreDenoiseStep`)
|
| 32 |
+
- Auto core denoise step that performs the denoising process.
|
| 33 |
+
4. **decode** (`QwenImage21DecodeStep`)
|
| 34 |
+
- Step that decodes the latents to RGBA images and postprocesses them
|
| 35 |
+
|
| 36 |
+
## Model Components
|
| 37 |
+
|
| 38 |
+
1. image_processor (`VaeImageProcessor`)
|
| 39 |
+
2. text_encoder (`Qwen3VLForConditionalGeneration`)
|
| 40 |
+
3. processor (`Qwen3VLProcessor`)
|
| 41 |
+
4. guider (`ClassifierFreeGuidance`)
|
| 42 |
+
5. vae (`AutoencoderKLQwenImage21`)
|
| 43 |
+
6. scheduler (`FlowMatchEulerDiscreteScheduler`)
|
| 44 |
+
7. transformer (`QwenImage21Transformer2DModel`)
|
| 45 |
+
|
| 46 |
+
## Configuration Parameters
|
| 47 |
+
|
| 48 |
+
sample_sigmas (default: None): Default sampling grid of the checkpoint, used when `sigmas` is not passed.
|
| 49 |
+
|
| 50 |
+
## Workflow Input Specification
|
| 51 |
+
|
| 52 |
+
<details>
|
| 53 |
+
<summary><strong>text2image</strong></summary>
|
| 54 |
+
|
| 55 |
+
- `prompt` (`str`): The prompt or prompts to guide image generation.
|
| 56 |
+
|
| 57 |
+
</details>
|
| 58 |
+
|
| 59 |
+
<details>
|
| 60 |
+
<summary><strong>image_conditioned</strong></summary>
|
| 61 |
+
|
| 62 |
+
- `image` (`Image | list`): Reference image(s) for denoising. Can be a single image or list of images.
|
| 63 |
+
- `prompt` (`str`): The prompt or prompts to guide image generation.
|
| 64 |
+
|
| 65 |
+
</details>
|
| 66 |
+
|
| 67 |
+
|
| 68 |
+
## Input/Output Specification
|
| 69 |
+
|
| 70 |
+
**Inputs:**
|
| 71 |
+
|
| 72 |
+
- `image` (`Image | list`, *optional*): Reference image(s) for denoising. Can be a single image or list of images.
|
| 73 |
+
- `output_resolution` (`int`, *optional*, defaults to `1024`): Target side length used to derive the output size and to resize condition images.
|
| 74 |
+
- `prompt` (`str`): The prompt or prompts to guide image generation.
|
| 75 |
+
- `negative_prompt` (`str`, *optional*): The prompt or prompts not to guide the image generation.
|
| 76 |
+
- `generator` (`Generator`, *optional*): Torch generator for deterministic generation.
|
| 77 |
+
- `num_images_per_prompt` (`int`, *optional*, defaults to `1`): The number of images to generate per prompt.
|
| 78 |
+
- `height` (`int`, *optional*): The height in pixels of the generated image.
|
| 79 |
+
- `width` (`int`, *optional*): The width in pixels of the generated image.
|
| 80 |
+
- `image_latents` (`list`, *optional*): Normalized latents of each condition image. Can be generated from vae_encoder step.
|
| 81 |
+
- `latents` (`Tensor`): Pre-generated noisy latents for image generation.
|
| 82 |
+
- `num_inference_steps` (`int`): The number of denoising steps.
|
| 83 |
+
- `sigmas` (`list`, *optional*): Custom sigmas for the denoising process.
|
| 84 |
+
- `use_kv_cache` (`bool`, *optional*, defaults to `True`): Cache the text and condition-image keys and values after the first step. Valid because `causal_condition` modulates those tokens from `t = 0`, making their activations step-independent. Toggling it does not reproduce the same image bit-for-bit in reduced precision.
|
| 85 |
+
- `attention_kwargs` (`dict`, *optional*): Additional kwargs for attention processors.
|
| 86 |
+
- `**denoiser_input_fields` (`None`, *optional*): conditional model inputs for the denoiser: e.g. prompt_embeds, negative_prompt_embeds, etc.
|
| 87 |
+
- `output_type` (`str`, *optional*, defaults to `pil`): Output format: 'pil', 'np', 'pt'.
|
| 88 |
+
|
| 89 |
+
**Outputs:**
|
| 90 |
+
|
| 91 |
+
- `images` (`list`): Generated images.
|
modular_model_index.json
ADDED
|
@@ -0,0 +1,76 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"_blocks_class_name": "QwenImage21AutoBlocks",
|
| 3 |
+
"_class_name": "QwenImage21ModularPipeline",
|
| 4 |
+
"_diffusers_version": "0.41.0.dev0",
|
| 5 |
+
"processor": [
|
| 6 |
+
"transformers",
|
| 7 |
+
"Qwen3VLProcessor",
|
| 8 |
+
{
|
| 9 |
+
"pretrained_model_name_or_path": "hf-internal-testing/tiny-qwenimage21-modular",
|
| 10 |
+
"revision": null,
|
| 11 |
+
"subfolder": "processor",
|
| 12 |
+
"type_hint": [
|
| 13 |
+
"transformers",
|
| 14 |
+
"Qwen3VLProcessor"
|
| 15 |
+
],
|
| 16 |
+
"variant": null
|
| 17 |
+
}
|
| 18 |
+
],
|
| 19 |
+
"sample_sigmas": null,
|
| 20 |
+
"scheduler": [
|
| 21 |
+
"diffusers",
|
| 22 |
+
"FlowMatchEulerDiscreteScheduler",
|
| 23 |
+
{
|
| 24 |
+
"pretrained_model_name_or_path": "hf-internal-testing/tiny-qwenimage21-modular",
|
| 25 |
+
"revision": null,
|
| 26 |
+
"subfolder": "scheduler",
|
| 27 |
+
"type_hint": [
|
| 28 |
+
"diffusers",
|
| 29 |
+
"FlowMatchEulerDiscreteScheduler"
|
| 30 |
+
],
|
| 31 |
+
"variant": null
|
| 32 |
+
}
|
| 33 |
+
],
|
| 34 |
+
"text_encoder": [
|
| 35 |
+
"transformers",
|
| 36 |
+
"Qwen3VLForConditionalGeneration",
|
| 37 |
+
{
|
| 38 |
+
"pretrained_model_name_or_path": "hf-internal-testing/tiny-qwenimage21-modular",
|
| 39 |
+
"revision": null,
|
| 40 |
+
"subfolder": "text_encoder",
|
| 41 |
+
"type_hint": [
|
| 42 |
+
"transformers",
|
| 43 |
+
"Qwen3VLForConditionalGeneration"
|
| 44 |
+
],
|
| 45 |
+
"variant": null
|
| 46 |
+
}
|
| 47 |
+
],
|
| 48 |
+
"transformer": [
|
| 49 |
+
"diffusers",
|
| 50 |
+
"QwenImage21Transformer2DModel",
|
| 51 |
+
{
|
| 52 |
+
"pretrained_model_name_or_path": "hf-internal-testing/tiny-qwenimage21-modular",
|
| 53 |
+
"revision": null,
|
| 54 |
+
"subfolder": "transformer",
|
| 55 |
+
"type_hint": [
|
| 56 |
+
"diffusers",
|
| 57 |
+
"QwenImage21Transformer2DModel"
|
| 58 |
+
],
|
| 59 |
+
"variant": null
|
| 60 |
+
}
|
| 61 |
+
],
|
| 62 |
+
"vae": [
|
| 63 |
+
"diffusers",
|
| 64 |
+
"AutoencoderKLQwenImage21",
|
| 65 |
+
{
|
| 66 |
+
"pretrained_model_name_or_path": "hf-internal-testing/tiny-qwenimage21-modular",
|
| 67 |
+
"revision": null,
|
| 68 |
+
"subfolder": "vae",
|
| 69 |
+
"type_hint": [
|
| 70 |
+
"diffusers",
|
| 71 |
+
"AutoencoderKLQwenImage21"
|
| 72 |
+
],
|
| 73 |
+
"variant": null
|
| 74 |
+
}
|
| 75 |
+
]
|
| 76 |
+
}
|
processor/chat_template.jinja
ADDED
|
@@ -0,0 +1,7 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{% set image_count = namespace(value=0) %}{% set video_count = namespace(value=0) %}{% for message in messages %}{% if loop.first and message['role'] != 'system' %}<|im_start|>system
|
| 2 |
+
You are a helpful assistant.<|im_end|>
|
| 3 |
+
{% endif %}<|im_start|>{{ message['role'] }}
|
| 4 |
+
{% if message['content'] is string %}{{ message['content'] }}<|im_end|>
|
| 5 |
+
{% else %}{% for content in message['content'] %}{% if content['type'] == 'image' or 'image' in content or 'image_url' in content %}{% set image_count.value = image_count.value + 1 %}{% if add_vision_id %}Picture {{ image_count.value }}: {% endif %}<|vision_start|><|image_pad|><|vision_end|>{% elif content['type'] == 'video' or 'video' in content %}{% set video_count.value = video_count.value + 1 %}{% if add_vision_id %}Video {{ video_count.value }}: {% endif %}<|vision_start|><|video_pad|><|vision_end|>{% elif 'text' in content %}{{ content['text'] }}{% endif %}{% endfor %}<|im_end|>
|
| 6 |
+
{% endif %}{% endfor %}{% if add_generation_prompt %}<|im_start|>assistant
|
| 7 |
+
{% endif %}
|
processor/processor_config.json
ADDED
|
@@ -0,0 +1,60 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"image_processor": {
|
| 3 |
+
"do_convert_rgb": true,
|
| 4 |
+
"do_normalize": true,
|
| 5 |
+
"do_rescale": true,
|
| 6 |
+
"do_resize": true,
|
| 7 |
+
"image_mean": [
|
| 8 |
+
0.48145466,
|
| 9 |
+
0.4578275,
|
| 10 |
+
0.40821073
|
| 11 |
+
],
|
| 12 |
+
"image_processor_type": "Qwen2VLImageProcessor",
|
| 13 |
+
"image_std": [
|
| 14 |
+
0.26862954,
|
| 15 |
+
0.26130258,
|
| 16 |
+
0.27577711
|
| 17 |
+
],
|
| 18 |
+
"merge_size": 2,
|
| 19 |
+
"patch_size": 16,
|
| 20 |
+
"resample": 3,
|
| 21 |
+
"rescale_factor": 0.00392156862745098,
|
| 22 |
+
"size": {
|
| 23 |
+
"longest_edge": 4096,
|
| 24 |
+
"shortest_edge": 1024
|
| 25 |
+
},
|
| 26 |
+
"temporal_patch_size": 2
|
| 27 |
+
},
|
| 28 |
+
"processor_class": "Qwen3VLProcessor",
|
| 29 |
+
"video_processor": {
|
| 30 |
+
"do_convert_rgb": true,
|
| 31 |
+
"do_normalize": true,
|
| 32 |
+
"do_rescale": true,
|
| 33 |
+
"do_resize": true,
|
| 34 |
+
"do_sample_frames": true,
|
| 35 |
+
"fps": 2,
|
| 36 |
+
"image_mean": [
|
| 37 |
+
0.5,
|
| 38 |
+
0.5,
|
| 39 |
+
0.5
|
| 40 |
+
],
|
| 41 |
+
"image_std": [
|
| 42 |
+
0.5,
|
| 43 |
+
0.5,
|
| 44 |
+
0.5
|
| 45 |
+
],
|
| 46 |
+
"max_frames": 768,
|
| 47 |
+
"merge_size": 2,
|
| 48 |
+
"min_frames": 4,
|
| 49 |
+
"patch_size": 16,
|
| 50 |
+
"resample": 3,
|
| 51 |
+
"rescale_factor": 0.00392156862745098,
|
| 52 |
+
"return_metadata": false,
|
| 53 |
+
"size": {
|
| 54 |
+
"longest_edge": 786432,
|
| 55 |
+
"shortest_edge": 131072
|
| 56 |
+
},
|
| 57 |
+
"temporal_patch_size": 2,
|
| 58 |
+
"video_processor_type": "Qwen3VLVideoProcessor"
|
| 59 |
+
}
|
| 60 |
+
}
|
processor/tokenizer.json
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:ff8cce547abc110590d19c6b5b6e0c6a7b4c8d1012d78b9c42131bae7f494a02
|
| 3 |
+
size 11420367
|
processor/tokenizer_config.json
ADDED
|
@@ -0,0 +1,36 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"add_prefix_space": false,
|
| 3 |
+
"backend": "tokenizers",
|
| 4 |
+
"bos_token": null,
|
| 5 |
+
"clean_up_tokenization_spaces": false,
|
| 6 |
+
"eos_token": "<|im_end|>",
|
| 7 |
+
"errors": "replace",
|
| 8 |
+
"extra_special_tokens": [
|
| 9 |
+
"<|im_start|>",
|
| 10 |
+
"<|im_end|>",
|
| 11 |
+
"<|object_ref_start|>",
|
| 12 |
+
"<|object_ref_end|>",
|
| 13 |
+
"<|box_start|>",
|
| 14 |
+
"<|box_end|>",
|
| 15 |
+
"<|quad_start|>",
|
| 16 |
+
"<|quad_end|>",
|
| 17 |
+
"<|vision_start|>",
|
| 18 |
+
"<|vision_end|>",
|
| 19 |
+
"<|vision_pad|>",
|
| 20 |
+
"<|image_pad|>",
|
| 21 |
+
"<|video_pad|>"
|
| 22 |
+
],
|
| 23 |
+
"is_local": true,
|
| 24 |
+
"local_files_only": false,
|
| 25 |
+
"max_length": null,
|
| 26 |
+
"model_max_length": 32768,
|
| 27 |
+
"pad_to_multiple_of": null,
|
| 28 |
+
"pad_token": "<|endoftext|>",
|
| 29 |
+
"pad_token_type_id": 0,
|
| 30 |
+
"padding_side": "left",
|
| 31 |
+
"processor_class": "Qwen3VLProcessor",
|
| 32 |
+
"split_special_tokens": false,
|
| 33 |
+
"tokenizer_class": "Qwen2Tokenizer",
|
| 34 |
+
"unk_token": null,
|
| 35 |
+
"variant": null
|
| 36 |
+
}
|
scheduler/scheduler_config.json
ADDED
|
@@ -0,0 +1,18 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"_class_name": "FlowMatchEulerDiscreteScheduler",
|
| 3 |
+
"_diffusers_version": "0.41.0.dev0",
|
| 4 |
+
"base_image_seq_len": 256,
|
| 5 |
+
"base_shift": 0.5,
|
| 6 |
+
"invert_sigmas": false,
|
| 7 |
+
"max_image_seq_len": 4096,
|
| 8 |
+
"max_shift": 1.15,
|
| 9 |
+
"num_train_timesteps": 1000,
|
| 10 |
+
"shift": 1.0,
|
| 11 |
+
"shift_terminal": null,
|
| 12 |
+
"stochastic_sampling": false,
|
| 13 |
+
"time_shift_type": "exponential",
|
| 14 |
+
"use_beta_sigmas": false,
|
| 15 |
+
"use_dynamic_shifting": false,
|
| 16 |
+
"use_exponential_sigmas": false,
|
| 17 |
+
"use_karras_sigmas": false
|
| 18 |
+
}
|
text_encoder/config.json
ADDED
|
@@ -0,0 +1,60 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"architectures": [
|
| 3 |
+
"Qwen3VLForConditionalGeneration"
|
| 4 |
+
],
|
| 5 |
+
"dtype": "float32",
|
| 6 |
+
"image_token_id": 151655,
|
| 7 |
+
"model_type": "qwen3_vl",
|
| 8 |
+
"text_config": {
|
| 9 |
+
"attention_bias": false,
|
| 10 |
+
"attention_dropout": 0.0,
|
| 11 |
+
"dtype": "float32",
|
| 12 |
+
"head_dim": 8,
|
| 13 |
+
"hidden_act": "silu",
|
| 14 |
+
"hidden_size": 16,
|
| 15 |
+
"initializer_range": 0.02,
|
| 16 |
+
"intermediate_size": 16,
|
| 17 |
+
"max_position_embeddings": 128000,
|
| 18 |
+
"model_type": "qwen3_vl_text",
|
| 19 |
+
"num_attention_heads": 2,
|
| 20 |
+
"num_hidden_layers": 2,
|
| 21 |
+
"num_key_value_heads": 2,
|
| 22 |
+
"pad_token_id": null,
|
| 23 |
+
"rms_norm_eps": 1e-06,
|
| 24 |
+
"rope_parameters": {
|
| 25 |
+
"mrope_section": [
|
| 26 |
+
1,
|
| 27 |
+
1,
|
| 28 |
+
2
|
| 29 |
+
],
|
| 30 |
+
"rope_theta": 1000000.0,
|
| 31 |
+
"rope_type": "default"
|
| 32 |
+
},
|
| 33 |
+
"use_cache": true,
|
| 34 |
+
"vocab_size": 151936
|
| 35 |
+
},
|
| 36 |
+
"tie_word_embeddings": false,
|
| 37 |
+
"transformers_version": "5.15.0",
|
| 38 |
+
"video_token_id": 151656,
|
| 39 |
+
"vision_config": {
|
| 40 |
+
"deepstack_visual_indexes": [
|
| 41 |
+
0
|
| 42 |
+
],
|
| 43 |
+
"depth": 2,
|
| 44 |
+
"dtype": "float32",
|
| 45 |
+
"hidden_act": "gelu_pytorch_tanh",
|
| 46 |
+
"hidden_size": 16,
|
| 47 |
+
"in_channels": 3,
|
| 48 |
+
"initializer_range": 0.02,
|
| 49 |
+
"intermediate_size": 16,
|
| 50 |
+
"model_type": "qwen3_vl_vision",
|
| 51 |
+
"num_heads": 2,
|
| 52 |
+
"num_position_embeddings": 64,
|
| 53 |
+
"out_hidden_size": 16,
|
| 54 |
+
"patch_size": 16,
|
| 55 |
+
"spatial_merge_size": 2,
|
| 56 |
+
"temporal_patch_size": 2
|
| 57 |
+
},
|
| 58 |
+
"vision_end_token_id": 151653,
|
| 59 |
+
"vision_start_token_id": 151652
|
| 60 |
+
}
|
text_encoder/generation_config.json
ADDED
|
@@ -0,0 +1,7 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"_from_model_config": true,
|
| 3 |
+
"output_attentions": false,
|
| 4 |
+
"output_hidden_states": false,
|
| 5 |
+
"transformers_version": "5.15.0",
|
| 6 |
+
"use_cache": true
|
| 7 |
+
}
|
text_encoder/model.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:65b388abe71f859ef783ff788d29836ea0a0513df3180f6f0ac4b476f6ea9d8b
|
| 3 |
+
size 19628032
|
transformer/config.json
ADDED
|
@@ -0,0 +1,20 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"_class_name": "QwenImage21Transformer2DModel",
|
| 3 |
+
"_diffusers_version": "0.41.0.dev0",
|
| 4 |
+
"_name_or_path": "/Volumes/T7 Shield/ai-lab/tiny/qwenimage21-modular/standard",
|
| 5 |
+
"attention_head_dim": 16,
|
| 6 |
+
"axes_dims_rope": [
|
| 7 |
+
4,
|
| 8 |
+
6,
|
| 9 |
+
6
|
| 10 |
+
],
|
| 11 |
+
"causal_condition": true,
|
| 12 |
+
"context_in_dim": 16,
|
| 13 |
+
"eps": 1e-06,
|
| 14 |
+
"in_channels": 8,
|
| 15 |
+
"mlp_ratio": 2,
|
| 16 |
+
"num_attention_heads": 2,
|
| 17 |
+
"num_layers": 2,
|
| 18 |
+
"out_channels": 8,
|
| 19 |
+
"patch_size": 1
|
| 20 |
+
}
|
transformer/diffusion_pytorch_model.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:96a310d836b05f743b29d1a1d5ea91d4b59f976dce7d44451b105b5bc4b569c9
|
| 3 |
+
size 150496
|
vae/config.json
ADDED
|
@@ -0,0 +1,50 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"_class_name": "AutoencoderKLQwenImage21",
|
| 3 |
+
"_diffusers_version": "0.41.0.dev0",
|
| 4 |
+
"_name_or_path": "/Volumes/T7 Shield/ai-lab/tiny/qwenimage21-modular/standard",
|
| 5 |
+
"attn_scales": [],
|
| 6 |
+
"base_dim": 4,
|
| 7 |
+
"decoder_base_dim": 4,
|
| 8 |
+
"dim_mult": [
|
| 9 |
+
1,
|
| 10 |
+
1,
|
| 11 |
+
1,
|
| 12 |
+
1,
|
| 13 |
+
1
|
| 14 |
+
],
|
| 15 |
+
"dropout": 0.0,
|
| 16 |
+
"in_channels": 4,
|
| 17 |
+
"is_residual": true,
|
| 18 |
+
"latents_mean": [
|
| 19 |
+
0.0,
|
| 20 |
+
0.0,
|
| 21 |
+
0.0,
|
| 22 |
+
0.0,
|
| 23 |
+
0.0,
|
| 24 |
+
0.0,
|
| 25 |
+
0.0,
|
| 26 |
+
0.0
|
| 27 |
+
],
|
| 28 |
+
"latents_std": [
|
| 29 |
+
1.0,
|
| 30 |
+
1.0,
|
| 31 |
+
1.0,
|
| 32 |
+
1.0,
|
| 33 |
+
1.0,
|
| 34 |
+
1.0,
|
| 35 |
+
1.0,
|
| 36 |
+
1.0
|
| 37 |
+
],
|
| 38 |
+
"num_res_blocks": 1,
|
| 39 |
+
"out_channels": 4,
|
| 40 |
+
"patch_size": null,
|
| 41 |
+
"scale_factor_spatial": 16,
|
| 42 |
+
"scale_factor_temporal": 8,
|
| 43 |
+
"temperal_downsample": [
|
| 44 |
+
false,
|
| 45 |
+
true,
|
| 46 |
+
true,
|
| 47 |
+
true
|
| 48 |
+
],
|
| 49 |
+
"z_dim": 8
|
| 50 |
+
}
|
vae/diffusion_pytorch_model.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:47b298bbf851ea4ec0b1b256c3a1480c3210485d7793bca35be919cbdfcfb393
|
| 3 |
+
size 52616
|