# P-H3-Image

P-H3-Image is an image model by Pruna AI based on [MiniMax-H3](./minimax_h3), for text-to-image generation and instruction-based image editing. You can find the checkpoint at [PrunaAI/P-H3-Image](https://huggingface.co/PrunaAI/P-H3-Image).

```py
import torch
from diffusers import ComponentsManager, ModularPipeline

manager = ComponentsManager()
manager.enable_auto_cpu_offload(device="cuda")  # or "mps", "xpu"

pipe = ModularPipeline.from_pretrained("PrunaAI/P-H3-Image", components_manager=manager)
pipe.load_components(dtype=torch.bfloat16)

prompt = (
    "A German Expressionist oil painting of a Berlin street at night in the manner of Ernst Ludwig Kirchner, "
    "with gaunt figures hurrying past a glowing gas lamp."
)
image = pipe(prompt=prompt, height=3360, width=2496, generator=torch.Generator().manual_seed(3), output="images")[0]
edited = pipe(prompt="let it snow on the street", image=image, output="images")[0]
```

Every request goes through the [prompt upsampler](#prompt-upsampling) first. With every component on the accelerator, the pipeline needs about 101 GB in bfloat16 at 8 megapixels, so the examples use auto offloading: the [ComponentsManager](/docs/diffusers/main/en/api/modular_diffusers/pipeline_components#diffusers.ComponentsManager) moves each component onto the accelerator when a block needs it, and the pipeline runs on an 80 GB GPU.

## Components

| Component | Class |
|---|---|
| `transformer` | [MiniMaxH3Transformer3DModel](/docs/diffusers/main/en/api/models/minimax_h3_transformer3d#diffusers.MiniMaxH3Transformer3DModel) |
| `text_encoder` | `Qwen3VLForConditionalGeneration` (Qwen3-VL-4B) |
| `text_connector` | [PH3ImageTextConnector](/docs/diffusers/main/en/api/pipelines/p_h3_image#diffusers.PH3ImageTextConnector) |
| `vae` | [AutoencoderKLPH3Image](/docs/diffusers/main/en/api/models/autoencoderkl_p_h3_image#diffusers.AutoencoderKLPH3Image) |
| `tokenizer`, `processor` | The text encoder's tokenizer, and a `Qwen3VLProcessor` with MiniMax-H3's image processor |
| `scheduler`, `audio_scheduler` | [MiniMaxH3Scheduler](/docs/diffusers/main/en/api/schedulers/minimax_h3#diffusers.MiniMaxH3Scheduler) |
| `prompt_upsampler`, `prompt_upsampler_processor` | `Qwen3_5ForConditionalGeneration` (based on Qwen3.5-9B) and its processor |

## Workflows

The workflow is selected on the `image` input:

- **`text2image`** generates from `prompt`. `height` and `width` default to 2912 and are rounded to multiples of 32. The model works best at 8 megapixels, for example 2912x2912, 3872x2176 or 2176x3872.
- **`image_conditioned`** edits `image` by the instruction in `prompt`, and works best at 4 or 8 megapixels. The image is encoded at about `vision_pixels` (one megapixel) for the text encoder and at its own size for the VAE, scaled up to `reference_min_pixels` (one megapixel) or down to `reference_max_pixels` (eight megapixels) when outside that range. Without `height` and `width`, the edit is generated at the size of the first image, which is scaled up to `canvas_min_pixels` (four megapixels) if smaller. Passed `height` and `width`, rounded to multiples of 32, change the canvas but not the images. `image` can also be a list of up to 8 images: the first is edited and the others are references, as `<Picture 2>`, `<Picture 3>` and so on.

## Prompt upsampling

P-H3-Image is meant to render full prompts rather than short requests, so the pipeline runs the prompt upsampler first on every request. It thinks about the request and then writes the prompt: a detailed text-to-image prompt for the aspect ratio of `height` and `width`, or, when editing, a brief of what to keep and what to change in the images. The prompt it wrote comes back as the `prompt` output.

```py
out = pipe(prompt=prompt, height=3360, width=2496, output=["images", "prompt"])
print(out["prompt"])
```

`max_new_tokens` caps the reasoning and the prompt together, 1024 tokens for text-to-image and 1536 for edits by default. If the upsampler does not return a usable prompt, the pipeline raises an error rather than rendering it.

To render `prompt` as given, pop the step from the blocks. The popped block also runs on its own, for example to show the prompt to a user before rendering it:

```py
manager = ComponentsManager()
manager.enable_auto_cpu_offload(device="cuda")

blocks = ModularPipeline.from_pretrained("PrunaAI/P-H3-Image").blocks
upsampler_block = blocks.sub_blocks.pop("prompt_upsampler")

upsampler = upsampler_block.init_pipeline("PrunaAI/P-H3-Image", components_manager=manager)
upsampler.load_components(dtype=torch.bfloat16)
upsampled = upsampler(prompt=prompt, height=3360, width=2496, output="prompt")

pipe = blocks.init_pipeline("PrunaAI/P-H3-Image", components_manager=manager)
pipe.load_components(dtype=torch.bfloat16)
image = pipe(prompt=upsampled, height=3360, width=2496, output="images")[0]
```

Without the step, an edit prompt in either form the upsampler writes, the six-section brief or a paragraph that starts with `Edit <Picture 1>.`, is encoded as written; any other instruction gets that prefix.

## Sampling

- The 8 steps run over the checkpoint's `sample_sigmas`. Pass `sigmas` to use another grid.
- The model is guidance-distilled, so there is no guider, `negative_prompt` or `guidance_scale`.

## PH3ImageModularPipeline[[diffusers.PH3ImageModularPipeline]]

#### diffusers.PH3ImageModularPipeline[[diffusers.PH3ImageModularPipeline]]

```python
diffusers.PH3ImageModularPipeline(blocks: diffusers.modular_pipelines.modular_pipeline.ModularPipelineBlocks | None = None, pretrained_model_name_or_path: str | os.PathLike | None = None, components_manager: diffusers.modular_pipelines.components_manager.ComponentsManager | None = None, collection: str | None = None, workflow: str | None = None, modular_config_dict: dict[str, typing.Any] | None = None, config_dict: dict[str, typing.Any] | None = None, **kwargs)
```

[Source](https://github.com/huggingface/diffusers/blob/main/src/diffusers/modular_pipelines/p_h3_image/modular_pipeline.py#L22)

A ModularPipeline for P-H3-Image, an image model based on MiniMax-H3, for text-to-image (`text2image`) and
instruction-based editing (`image_conditioned`).

The image is a single latent frame, denoised in one packed sequence with two audio latents that are discarded.

```py
import torch
from diffusers import ComponentsManager, ModularPipeline

manager = ComponentsManager()
manager.enable_auto_cpu_offload(device="cuda")

pipe = ModularPipeline.from_pretrained("PrunaAI/P-H3-Image", components_manager=manager)
pipe.load_components(dtype=torch.bfloat16)
prompt = (
    "A German Expressionist oil painting of a Berlin street at night in the manner of Ernst Ludwig Kirchner, "
    "with gaunt figures hurrying past a glowing gas lamp."
)
image = pipe(prompt=prompt, height=3360, width=2496, output="images")[0]
```

## PH3ImageBlocks[[diffusers.PH3ImageBlocks]]

#### diffusers.PH3ImageBlocks[[diffusers.PH3ImageBlocks]]

```python
diffusers.PH3ImageBlocks()
```

[Source](https://github.com/huggingface/diffusers/blob/main/src/diffusers/modular_pipelines/p_h3_image/modular_blocks_p_h3_image.py#L376)

Modular pipeline blocks for P-H3-Image: `text2image`, and instruction-based editing (`image_conditioned`) when
`image` is passed. `prompt` goes through the prompt upsampler first; pop the `prompt_upsampler` step to render it
as given.

Supported workflows:
- `text2image`: requires `prompt`
- `image_conditioned`: requires `prompt`, `image`

Components:
image_processor (`VaeImageProcessor`) prompt_upsampler (`Qwen3_5ForConditionalGeneration`): The prompt
upsampler. prompt_upsampler_processor (`AutoProcessor`): Its processor. text_encoder
(`Qwen3VLForConditionalGeneration`) text_connector (`PH3ImageTextConnector`) tokenizer (`Qwen2Tokenizer`)
processor (`Qwen3VLProcessor`) vae (`AutoencoderKLPH3Image`) scheduler (`MiniMaxH3Scheduler`) audio_scheduler
(`MiniMaxH3Scheduler`) transformer (`MiniMaxH3Transformer3DModel`)

Configs:
reference_min_pixels (default: 1048576) reference_max_pixels (default: 8388608) canvas_min_pixels (default:
4194304) vision_pixels (default: 1000000) sample_sigmas (default: [1.0, 0.984753, 0.967905, 0.951688,
0.925372, 0.881923, 0.810083, 0.528718, 0.0]): The checkpoint's sampling grid, used when `sigmas` is not
passed.

Inputs:
image (`Image | list`, *optional*):
The image to edit, or a list of at most 8 images: the first is edited, the others are references.
height (`int`, *optional*):
Height of the generated image in pixels, rounded to a multiple of 32. Defaults to 2912 for `text2image`,
which works best at 8 megapixels. An edit defaults to the first image's size, at least
`canvas_min_pixels`, and works best at 4 or 8 megapixels.
width (`int`, *optional*):
Width of the generated image in pixels, rounded to a multiple of 32. Defaults to 2912 for `text2image`,
which works best at 8 megapixels. An edit defaults to the first image's size, at least
`canvas_min_pixels`, and works best at 4 or 8 megapixels.
prompt (`str`):
The prompt, or the edit instruction when `image` is passed. A single string.
max_new_tokens (`int`, *optional*, defaults to 1536 or 1024, depending on the workflow):
The most tokens the upsampler may generate, its reasoning included.
vision_images (`list`, *optional*):
The images of the edit resized for the text encoder, the image to edit first.
reference_images (`list`, *optional*):
The images of the edit resized for the VAE, the image to edit first.
condition_latents (`list`, *optional*):
The conditioning latents of the images of the edit. Their shapes give the geometry of the reference
blocks.
generator (`Generator`, *optional*):
The generator of the request. The conditioning noise is drawn from it first, one draw per condition,
before the noise of the generated rows.
latents (`Tensor`):
Pre-generated image noise of shape `(1, latent_channels, 1, latent_height, latent_width)`, used instead
of the draw.
sigmas (`list`, *optional*):
A custom sampling grid, strictly decreasing and ending at 0. Defaults to the checkpoint's
`sample_sigmas`.
**denoiser_input_fields (`None`, *optional*):
The structural description of the packed sequence the transformer reads by name: `token_tags`,
`position_ids` and the three row-index tensors.
attention_kwargs (`dict`, *optional*):
Additional kwargs for attention processors.
output_type (`str`, *optional*, defaults to pil):
Output format: 'pil', 'np', 'pt'.

Outputs:
images (`list`):
Generated images.

## PH3ImageTextConnector[[diffusers.PH3ImageTextConnector]]

#### diffusers.PH3ImageTextConnector[[diffusers.PH3ImageTextConnector]]

```python
diffusers.PH3ImageTextConnector(hidden_state_indices: tuple = (13, 19, 25, 31, 36), text_hidden_size: int = 2560, out_channels: int = 5120)
```

[Source](https://github.com/huggingface/diffusers/blob/main/src/diffusers/models/condition_embedders/condition_embedder_p_h3_image.py#L22)

**Parameters:**

hidden_state_indices (`tuple[int, ...]`, defaults to `(13, 19, 25, 31, 36)`) : Which entries of the text encoder's `hidden_states` are read, in the numbering of `output_hidden_states` (`0` is the embedding output, `i` the output of decoder layer `i`, and the last one the final normed hidden state).

text_hidden_size (`int`, defaults to `2560`) : The hidden size of the text encoder.

out_channels (`int`, defaults to `5120`) : The number of channels of the conditioning MiniMax-H3's transformer reads.

Maps hidden states of P-H3-Image's Qwen3-VL text encoder to the text conditioning its transformer reads.

The hidden states listed in `hidden_state_indices` are concatenated, standardized, projected linearly and
destandardized. The first token is an attention sink with the same conditioning for every prompt, so it is replaced
by `sink_embedding`.

#### forward[[diffusers.PH3ImageTextConnector.forward]]

```python
forward(hidden_states: tuple)
```

[Source](https://github.com/huggingface/diffusers/blob/main/src/diffusers/models/condition_embedders/condition_embedder_p_h3_image.py#L61)

**Parameters:**

hidden_states (`tuple[torch.Tensor, ...]`) : The hidden states of the text encoder, as `output_hidden_states=True` returns them: one tensor of shape `(batch_size, num_tokens, text_hidden_size)` per entry.

**Returns:** `torch.Tensor`

the conditioning, of shape `(batch_size, num_tokens, out_channels)`, in the dtype of the
connector's parameters.