Simple-Workflow

#2
by darkenHUB - opened

Save the image and import it into ComfyUI
workflow

Got only noise, and ask chtgpt for help. Answer was

Correction: ModelSamplingSD3 has been removed. PAN2 already uses Shift 9 and multiplier 1.0. The model is now connected directly to the KSampler.

Also it should get a structured JSON prompt, so ChatGPT created a small sample worklfow. Should be included in mp4. Sample is T2V. all credits to darkenHub.

The workflow preset on 50 step , you can get same quality on 12 or 15 step

Found the cause of the noise. PAN2 reads timesteps from 0 to 1 (its model sampling uses multiplier 1.0) and clamps them to that range before the AdaLN table lookup. ModelSamplingSD3 hard-codes multiplier 1000, so every step arrives as t = 1 and the model denoises as if the latent were still pure noise. That's why removing the node works: the model already defaults to shift 9. To keep a shift knob, use ModelSamplingAuraFlow, which is the same node with multiplier 1.0.

The prompt still has to be one of IFM's structured JSON captions. Free text gives noise as well (kijai notes this in the ComfyUI PR: https://github.com/Comfy-Org/ComfyUI/pull/17755).

Attached is a T2V + I2V workflow, built from the Wan 2.2 5B TI2V template. Drag the PNG into ComfyUI. It uses IFM's defaults (704x1248, 24 fps, 241 frames, 50 steps, CFG 2.5, euler/simple), their swimmer example as the positive prompt and their fixed negative. For I2V, select Load Image and press Ctrl+B. I tested I2V at 704x1248 on a 4090.

PAN2_t2v_i2v_workflow

Thanks everyone for sharing the workflows and for tracking down the noise issue. Really helpful.

As noted above, PAN 2 expects a structured JSON caption rather than free text. For the best results, we recommend pairing it with a strong LLM, such as GPT-6 Astra, Claude or Kimi K3, to turn your plain-language idea into that format.

Our prompt rewriter instructions are open: https://github.com/ifm-ai/PAN2/tree/main/inference/pan-prompt-rewriter

  • Give the LLM system_prompt.md as its system prompt, then your scene description and duration. It returns the positive prompt.
  • Use assets/generic_negative.txt as the negative prompt, unchanged apart from the duration.

In ComfyUI, you can do this inside the workflow with the OpenAI node or the OpenRouter nodes if you'd prefer to use your own API key.
Full code and inference setup are in our GitHub repo: https://github.com/ifm-ai/PAN2

Let us know how it goes. And stay tuned: FastPAN2 is coming soon! 20x speedup with out quality loss.

Sign up or log in to comment