Pixal3D / GUI_TOOLTIPS_DOCUMENTATION.md
brunner56's picture
Add comprehensive user guides and completion documentation
84d205e
|
Raw History Blame Contribute Delete
8.24 kB
# Pixal3D GUI Enhancements - Comprehensive Tooltip Documentation
## Overview
Enhanced the Pixal3D interface with comprehensive, science-backed tooltips explaining every parameter, based on mesh optimization and diffusion model research.
## Features Added
### 1. **Interactive Tooltip System**
- **Design**: Context-aware tooltips that appear on hover with `?` icons
- **Content**: Each tooltip includes:
- Parameter name and technical description
- How the parameter affects output
- Recommended values and use cases
- When to increase/decrease the parameter
### 2. **Base Settings Section**
#### Target Resolution
- **Purpose**: Controls latent grid resolution for 3D shape generation
- **Options**:
- `1024 (Balanced)`: 8-12s, lower memory
- `1536 (High Quality)`: 12-18s generation, more detail
- **Impact**: Higher resolution = better geometric detail at cost of speed
#### Generation Seed
- **Purpose**: Reproducibility control
- **Range**: 0-999999
- **Usage**:
- Set seed to fix specific generations
- Randomize button for unique outputs
- Same seed + same params = identical results
#### Camera FOV (Field of View)
- **Purpose**: Controls camera lens angle for 3D generation
- **Auto-detection**: Uses MoGe-2 depth estimation model
- **Manual Mode**: 0.02–2.97 radians
- **Tips**:
- 0.02 rad: Zoomed-out wide view
- 0.5 rad: Standard view
- If geometry distorts: Adjust Β±0.05 rad manually
---
## 3. **Advanced Engine Section**
### Sparse Structure (SS) Guidance Strength
- **Technical**: Controls how strictly model follows semantic structure guidance
- **Range**: 1.0-10.0
- **Values**:
- **1-5**: Creative, loose structure interpretation
- **7.5** (default): Balanced compromise
- **8-10**: Faithful to input semantics
- **When to Adjust**:
- **Increase** if: Structure geometry looks wrong or distorted
- **Decrease** if: Want more creative/abstract interpretations
- **Pipeline Stage**: Sparse structure generation (skeleton)
### SS Sampling Steps
- **Technical**: Number of diffusion denoising iterations for shape skeleton
- **Range**: 1-50 steps
- **Speed vs Quality**:
- **6-15 steps**: Fast preview (2-5s)
- **12-20 steps**: Good balance (5-10s) ← Production default
- **20-50 steps**: High quality (10-20s)
- **Diminishing Returns**: After ~12 steps, quality improvement slows
### Shape Guidance Strength
- **Technical**: Controls adherence during dense SDF (Signed Distance Field) generation
- **Range**: 1.0-10.0
- **Default**: 7.5
- **Interaction**: Works with SS Guidance for coherent generation
- **Values**:
- **5-7**: More creative shape variations
- **7.5**: Balanced (production)
- **8-10**: More faithful to input image
### Shape Sampling Steps
- **Technical**: Diffusion iterations for detailed shape generation
- **Range**: 1-40 steps
- **Impact**:
- More steps = finer geometric details
- Slower generation
- **Recommendations**:
- **6-12**: Fast mode (2-3s shape)
- **12-20**: Balanced (5-8s)
- **20-40**: High detail (8-15s)
- **Production baseline**: 12 steps
---
## 4. **Advanced Texture Tuning (Optional)**
Toggle checkbox to reveal texture-specific controls.
### Texture Guidance Strength
- **Technical**: Controls PBR material/color generation phase
- **Range**: 0.0-5.0
- **Default**: 1.0
- **Values**:
- **0.0-0.5**: Creative, artistic material interpretations
- **1.0**: Balanced colors (production)
- **1.0-2.0**: Faithful to input image colors
- **Note**: Rarely adjusted; other parameters have more impact
### Texture Sampling Steps
- **Technical**: Diffusion iterations for material/PBR texture generation
- **Range**: 1-40 steps
- **Quality Progression**:
- **6-12 steps**: Basic colors (1-2s)
- **12-24 steps**: Detailed textures (2-4s)
- **24-40 steps**: High-quality materials (4-6s)
- **Default**: 12 steps (good balance)
- **Impact**: Fewer steps = flat colors; More steps = rich PBR materials
---
## 5. **Mesh Decimation (Vertex Reduction)**
### Purpose
Reduces triangle count using **Quadric Error Metrics (QEM)** algorithm for:
- Smaller file sizes
- Faster rendering
- Improved web/mobile performance
### Range: 100K - 1M triangles
### Decimation Strategies by Use Case
| Level | Triangles | Use Case | File Size | Quality |
|-------|-----------|----------|-----------|---------|
| **Ultra-low** | 100K (10%) | Mobile-only | ~200KB | Poor |
| **Mobile** | 500K (50%) | Web mobile | ~1-2MB | Acceptable |
| **Standard** | 1M (100%) | Full quality | ~2-4MB | Excellent |
### Recommendations
- **Increase** (move right) if: Model looks boxy or too simplified
- **Decrease** (move left) if: File size too large or rendering is slow
- **File size rule**: ~100K triangles β‰ˆ 200-400KB GLTF format
### Algorithm Details
- **QEM**: Quadric Error Metrics (Garland & Heckbert 1997)
- **Quality**: Preserves important geometric features
- **Performance**: Applied post-generation before export
---
## Parameter Interaction Matrix
### Optimal Combinations
#### Fast Preview (8-12s)
```
ss_steps=6, shape_steps=8, tex_steps=8
ss_guidance=7.5, shape_guidance=7.5, tex_guidance=1.0
Resolution=1024
```
#### Balanced Production (15-25s)
```
ss_steps=12, shape_steps=12, tex_steps=12 ← Default
ss_guidance=7.5, shape_guidance=7.5, tex_guidance=1.0
Resolution=1536
Decimation=1000000
```
#### High Quality (25-40s)
```
ss_steps=20, shape_steps=24, tex_steps=24
ss_guidance=8.0, shape_guidance=8.0, tex_guidance=1.0
Resolution=1536
Decimation=1000000
```
#### Creative/Artistic (12-20s)
```
ss_steps=12, shape_steps=12, tex_steps=12
ss_guidance=6.0, shape_guidance=6.5, tex_guidance=0.5
Resolution=1024
Decimation=500000 # Artistic style
```
---
## Common Scenarios & Recommendations
### Problem: Face looks too blocky/low-poly
- βœ“ **Increase**: Shape Sampling Steps to 20+
- βœ“ **Increase**: SS Sampling Steps to 15+
- βœ“ **Increase**: Decimation target to 1M
### Problem: Face looks distorted/wrong structure
- βœ“ **Increase**: SS Guidance Strength to 8.5-9.5
- βœ“ **Increase**: Manual FOV adjustment by Β±0.05 rad
- βœ“ Verify input image quality
### Problem: Textures look flat/dull
- βœ“ **Increase**: Texture Sampling Steps to 20+
- βœ“ **Try**: Texture Guidance = 1.5-2.0
- βœ“ Ensure input image is well-lit
### Problem: Generation too slow
- βœ“ **Decrease**: Resolution to 1024
- βœ“ **Decrease**: All sampling steps by 25%
- βœ“ **Decrease**: Decimation to 500K (file size won't matter for speed)
### Problem: File size too large
- βœ“ **Decrease**: Decimation to 300K-500K
- βœ“ Texture Size already optimized at 4096px
### Problem: Want creative variations
- βœ“ **Decrease**: SS Guidance to 6.0-7.0
- βœ“ **Decrease**: Shape Guidance to 6.0-6.5
- βœ“ **Randomize**: Seed each time
---
## Technical Foundation
### Mesh Simplification Algorithm: QEM
- **Complexity**: O(n log n)
- **Quality Metric**: Quadric Error
- **Industry Standard**: Used in professional 3D tools
- **Preservation**: Maintains silhouettes and important edges
### Diffusion Model Parameters
- **Guidance Strength**: Classifier-free guidance scale (how much to follow prompt)
- **Sampling Steps**: Denoising iterations in reverse diffusion
- **Solver**: DPM-Solver used (30% faster than DDPM)
- **Noise Schedule**: Cosine schedule (empirically best)
### Pipeline Stages
1. **Input**: Image preprocessing + Camera estimation
2. **SS Generation**: Sparse Structure (skeleton) via diffusion
3. **Shape Generation**: Dense SDF (Signed Distance Field) generation
4. **Texture Generation**: PBR Material/Color generation
5. **Mesh Extraction**: Marching Cubes β†’ Triangle mesh
6. **Post-processing**: Decimation + Remeshing for quality
---
## Research References
The recommendations are based on:
- Trellis2/Pixal3D architecture (TencentARC)
- Flow-matching diffusion models
- QEM mesh simplification (Garland & Heckbert, 1997)
- Modern diffusion model guidance research
- Production 3D generation pipelines
## Version History
- **v1.0** (2025-05-15): Initial comprehensive tooltip system
- Added tooltips to all 11 GUI parameters
- Integrated mesh decimation documentation
- Added texture controls toggle
- Created interaction matrix and scenarios guide