Updated README

README.md

@@ -11,4 +11,196 @@ pinned: false
 short_description: An improved portrait mode renderer for pictures.
 ---
 
-Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
+# Physically-Based Portrait Mode Engine
+
+A Gradio demo that simulates **shallow depth-of-field** (portrait mode) on ordinary photos. Instead of applying a flat Gaussian blur to the background, it estimates scene depth, builds a **circle-of-confusion (CoC)** map, and uses a trained neural renderer (**RendererNet**) to produce physically motivated defocus. In-focus regions are preserved by blending the render back onto the original image.
+
+The app is designed to run as a [Hugging Face Space](https://huggingface.co/docs/hub/spaces) and is self-contained: model definitions and inference code live in a single `app.py` file.
+
+## What it does
+
+Given an input photograph and camera parameters (f-stop, focal length), the pipeline:
+
+1. **Estimates relative depth** with [Depth Anything V2](https://huggingface.co/depth-anything/Depth-Anything-V2-Base-hf) via the Hugging Face `transformers` library.
+2. **Builds a pseudo CoC map** by measuring how far each pixel's depth deviates from the depth at the **image center** (the assumed focus point). Pixels closer to the focus depth get a CoC near zero; pixels farther away get larger CoC values.
+3. **Renders defocus with RendererNet**, a U-Net that takes the RGB image plus normalized f-stop, focal length, and CoC channels and outputs a blurred RGB image at the chosen aperture settings.
+4. **Blends the NN render onto the original** using a smooth CoC-based weight so that in-focus areas (CoC below a threshold) remain untouched.
+5. **Produces a non-NN baseline** for comparison: a flat Gaussian blur applied only where CoC exceeds a separate threshold.
+
+Three outputs are shown:
+
+| Output | Description |
+|--------|-------------|
+| **Rendered (NN)** | RendererNet defocus, blended with the original in in-focus regions |
+| **Gaussian baseline** | Simple background blur for comparison against the learned renderer |
+| **Pseudo CoC map** | Colorized circle-of-confusion visualization (inferno colormap) |
+
+## Pipeline overview
+
+```
+Input image
+    │
+    ▼
+Resize (longest side ≤ 1024 px)
+    │
+    ▼
+Depth Anything V2  ──►  relative depth [0, 1]
+    │
+    ▼
+Pseudo CoC map  (focus = image center, max 4 px)
+    │
+    ├──────────────────────────────┐
+    ▼                              ▼
+RendererNet (512×512)         Gaussian blur baseline
+f-stop + focal length + CoC   (CoC > threshold)
+    │                              │
+    ▼                              │
+CoC-weighted blend ◄───────────────┘
+    │
+    ▼
+Final outputs (NN render, baseline, CoC map)
+```
+
+### Pseudo CoC
+
+Because the demo has no interactive focus point, focus is fixed at the **center of the image** (`h // 2`, `w // 2`). The CoC at each pixel is proportional to the absolute difference between that pixel's relative depth and the depth at the focus point, scaled to a maximum of **4 px** (`COC_MAX_PX`).
+
+### RendererNet input
+
+RendererNet is a U-Net with **6 input channels** and **3 output channels** (RGB):
+
+| Channel(s) | Content |
+|------------|---------|
+| 0–2 | RGB image, resized to 512×512 |
+| 3 | f-stop map, normalized by `F_STOP_MAX` (22.0) |
+| 4 | Focal length map, normalized by `FOCAL_LENGTH_MM_MAX` (200.0) |
+| 5 | CoC map, clipped to [0, 25] px and normalized by `COC_PX_NORM` (25.0) |
+
+The network output is resized back to the working resolution and blended with the original using a smoothstep weight derived from the CoC map and the **in-focus CoC threshold** slider.
+
+### Blending
+
+The blend weight uses a smoothstep on CoC values above the focus threshold, so transitions between sharp and blurred regions are gradual rather than hard-edged. Where CoC is near zero, the original pixels are kept; where CoC is large, the RendererNet output dominates.
+
+## Models
+
+### Depth Anything V2
+
+- **Default checkpoint:** `depth-anything/Depth-Anything-V2-Base-hf`
+- Loaded at startup via `AutoImageProcessor` and `AutoModelForDepthEstimation` from `transformers`.
+- Returns relative (not metric) depth, normalized per image to [0, 1].
+- Requires **torch** and **torchvision**.
+
+Swap to a different variant via the `DEPTH_MODEL_ID` environment variable:
+
+| Model ID | Trade-off |
+|----------|-----------|
+| `depth-anything/Depth-Anything-V2-Small-hf` | Faster, lower quality — good for CPU Spaces |
+| `depth-anything/Depth-Anything-V2-Base-hf` | Default; balanced speed and quality |
+| `depth-anything/Depth-Anything-V2-Large-hf` | Best quality, slowest |
+
+### RendererNet
+
+- A U-Net architecture (inlined in `app.py`) trained to render defocus given RGB + camera parameters + CoC.
+- Weights are loaded from **`renderer/best_renderer.pth`** by default.
+- Can alternatively be fetched from the Hugging Face Hub when `RENDERER_REPO_ID` is set.
+
+## Using the demo
+
+1. Upload an image (JPEG or PNG).
+2. Adjust **f-stop** (0.95–22.0) — lower values produce stronger background blur.
+3. Adjust **focal length** (4–200 mm) — longer focal lengths increase the shallow-DoF effect.
+4. Tune **in-focus CoC threshold** — pixels with CoC below this value suppress the NN render and keep the original sharp.
+5. Optionally expand **Gaussian baseline** to configure the comparison blur (CoC threshold and sigma).
+6. Click **Render**.
+
+If a `cache/` directory with example images exists, sample images appear in the Examples panel.
+
+## Configuration
+
+All settings below can be overridden with environment variables (useful for Hugging Face Space **Settings → Repository secrets / Variables** without editing code).
+
+| Variable | Default | Description |
+|----------|---------|-------------|
+| `DEPTH_MODEL_ID` | `depth-anything/Depth-Anything-V2-Base-hf` | Hugging Face model ID for depth estimation |
+| `RENDERER_LOCAL_PATH` | `renderer/best_renderer.pth` | Path to RendererNet weights on disk |
+| `RENDERER_REPO_ID` | *(empty)* | Hugging Face repo to download weights from |
+| `RENDERER_FILENAME` | `best_renderer.pth` | Filename within the Hub repo |
+| `HF_TOKEN` | *(empty)* | Token for private Hub repos (only needed with `RENDERER_REPO_ID`) |
+
+### Internal constants
+
+These are fixed in code and must match RendererNet's training setup:
+
+| Constant | Value | Purpose |
+|----------|-------|---------|
+| `F_STOP_MAX` | 22.0 | f-stop normalization divisor |
+| `FOCAL_LENGTH_MM_MAX` | 200.0 | Focal length normalization divisor |
+| `COC_PX_NORM` | 25.0 | CoC channel clip and normalize |
+| `TARGET_SIZE` | 512 | RendererNet spatial resolution |
+| `COC_MAX_PX` | 4.0 | Maximum pseudo CoC magnitude |
+| `MAX_SIDE` | 1024 | Longest image side before inference |
+
+## Local development
+
+### Requirements
+
+- Python 3.10+ (3.13 on Hugging Face Spaces)
+- CUDA optional (falls back to CPU)
+
+### Install and run
+
+```bash
+git clone https://github.com/<your-username>/Physically-Based-Portrait-Mode-Engine.git
+cd Physically-Based-Portrait-Mode-Engine
+
+pip install -r requirements.txt
+
+# Ensure RendererNet weights are present
+# (renderer/best_renderer.pth is included in the repo)
+
+python app.py
+```
+
+Gradio will print a local URL (typically `http://127.0.0.1:7860`).
+
+### Dependencies
+
+```
+gradio>=4.44.0
+torch>=2.1.0
+torchvision>=0.16.0
+transformers>=4.45.0
+huggingface_hub>=0.24.0
+numpy>=1.26.0
+pillow>=10.0.0
+scikit-image>=0.22.0
+matplotlib>=3.7.0
+```
+
+On first run, Depth Anything V2 weights are downloaded from the Hugging Face Hub (~370 MB for the Base model).
+
+## Project structure
+
+```
+.
+├── app.py                  # Gradio app, RendererNet definition, full inference pipeline
+├── requirements.txt        # Python dependencies
+├── renderer/
+│   └── best_renderer.pth   # Trained RendererNet weights
+└── cache/                  # Optional example images for Gradio Examples
+```
+
+## Limitations
+
+- **Fixed focus point:** Focus is always the image center; there is no click-to-focus or subject detection.
+- **Relative depth only:** Depth Anything V2 outputs ordinal depth, not metric distances, so the CoC map is a heuristic rather than a physically exact optical simulation.
+- **Resolution cap:** Images are downscaled so the longest side is at most 1024 px to keep CPU inference responsive.
+- **RendererNet runs at 512×512:** Fine detail may be softened; output is upsampled to the working resolution.
+- **CPU Spaces are slow:** First inference after startup can take tens of seconds; consider the Small depth model or a GPU Space for faster turnaround.
+
+## Acknowledgments
+
+- [Depth Anything V2](https://github.com/DepthAnything/Depth-Anything-V2) — monocular depth estimation ([Lihe Yang et al.](https://huggingface.co/depth-anything/Depth-Anything-V2-Base-hf))
+- [Hugging Face Transformers](https://github.com/huggingface/transformers) — model loading and inference
+- [Gradio](https://gradio.app/) — web UI