| --- |
| license: apache-2.0 |
| library_name: diffusers |
| pipeline_tag: text-to-image |
| tags: |
| - diffusion |
| - dit |
| - text-to-image |
| - flow-matching |
| - safetensors |
| - rectified-flow |
| - artistic |
| - custom-architecture |
| inference: false |
| --- |
| |
| # PRISM-1: Diffusion Transformer (DiT) Checkpoint |
|
|
| **PRISM-1** is a custom Diffusion Transformer (DiT) architecture designed for multi-domain text-to-image synthesis, featuring native spectral light modeling, authentic traditional/digital medium emulation, and high geometric consistency. |
|
|
| --- |
|
|
| ## Model Specifications |
|
|
| | Attribute | Specification | |
| | :--- | :--- | |
| | **Model Type** | Diffusion Transformer (`PrismDiT`) | |
| | **Parameter Count** | 815.46 Million (FP16) | |
| | **Weights File** | `PRISM-1.safetensors` (1.55 GB) | |
| | **Latent Dimension** | 16 channels ($8\times$ spatial downsampling) | |
| | **Patch Size** | $2 \times 2$ latent patches ($16 \times 16$ pixel equivalent) | |
| | **Text Conditioning** | Cross-Attention (1024-dim context, CLIP / T5-XXL) | |
| | **Diffusion Formulation** | Rectified Flow / Flow-Matching with Euler ODE solver | |
| | **Recommended Steps** | **12 – 25 steps** (optimal: 18–20) | |
| | **Recommended CFG** | **5.0 – 6.5** | |
| | **Default Sampler** | Euler | |
|
|
| --- |
|
|
| ## Key Capabilities |
|
|
| - **Zero Style-Bleed:** Seamlessly shifts between pure 2D traditional media (pencil, chalk pastel, aged oil), cinematic photography, and faceted geometric abstraction without carrying over unwanted stylistic tropes. |
| - **Prismatic Optics:** Built-in comprehension of chromatic dispersion, caustics, and prism flare. |
| - **Fast Rectified Flow:** High convergence rate requiring only **12 – 25 Euler steps** for full detail fidelity. |
|
|
| --- |
|
|
| ## Quickstart & Usage |
|
|
| ### 1. Installation |
| ```bash |
| pip install torch torchvision safetensors pillow huggingface_hub |
| ``` |
|
|
| ### 2. Python Inference Snippet |
| ```python |
| import torch |
| from model import PrismDiT |
| from pipeline import PrismPipeline |
| |
| # Initialize DiT architecture |
| dit = PrismDiT( |
| input_size=64, |
| patch_size=2, |
| in_channels=16, |
| hidden_size=1152, |
| depth=28, |
| num_heads=16, |
| context_dim=1024, |
| ) |
| |
| # Load pipeline with PRISM-1.safetensors |
| pipe = PrismPipeline.from_pretrained( |
| checkpoint_path="PRISM-1.safetensors", |
| device="cuda" if torch.cuda.is_available() else "cpu", |
| dtype=torch.float16, |
| ) |
| |
| # Generate image (12 - 25 flow steps) |
| images = pipe( |
| prompt="kaleidoscopic radial mandala, abstract geometric human figures, chromatic prism refraction", |
| negative_prompt="blurry, muddy colors, noise", |
| height=1024, |
| width=1024, |
| num_inference_steps=20, # Recommended: 12 - 25 steps |
| guidance_scale=6.5, |
| seed=42, |
| ) |
| |
| ### 3. Image Variation Snippet (Img2Img) |
| ```python |
| # Generate a variation from a reference photo |
| variations = pipe.generate_variation( |
| reference_image="path/to/reference.jpg", |
| prompt="same scene at night with neon ambient lighting", |
| strength=0.65, # 0.1 (micro-tweak) to 0.9 (broad reimagining) |
| num_inference_steps=20, # Recommended: 12 - 25 steps |
| guidance_scale=6.0, |
| ) |
| variations[0].save("variation_output.png") |
| ``` |
| |
| Or via CLI: |
| ```bash |
| python variation_demo.py --input "path/to/reference.jpg" --strength 0.65 --prompt "sunset lighting" |
| ``` |
|
|
| --- |
|
|
| ## Repository Files |
|
|
| - `PRISM-1.safetensors`: Main compiled FP16 model checkpoint (815M parameters). |
| - `config.json`: Architecture configuration for automated loading. |
| - `model.py`: Full PyTorch implementation of `PrismDiT` and `PrismDiTBlock`. |
| - `pipeline.py`: Inference pipeline with `PrismFlowScheduler` and CFG support. |
| - `benchmark_suite.py`: Standard 5-domain evaluation suite. |
| - `inspect_checkpoint.py`: Command-line inspection utility for safe tensor viewing. |
|
|
| --- |
|
|
| ## Citation & License |
|
|
| Released under the **Apache-2.0** License. |
|
|
