Instructions to use RenderFormer/renderformer-v2 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use RenderFormer/renderformer-v2 with Transformers:
# pip install -U transformers accelerate # Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("RenderFormer/renderformer-v2", device_map="auto") - Notebooks
- Google Colab
- Kaggle
Document direct component loading
Browse files
README.md
CHANGED
|
@@ -10,14 +10,15 @@ tags:
|
|
| 10 |
|
| 11 |
# RenderFormer V2
|
| 12 |
|
| 13 |
-
> **Status:** This is a private pre-release bundle. Do not redistribute it or
|
| 14 |
-
> depend on its layout until the release is approved.
|
| 15 |
-
|
| 16 |
This repository bundles the two RenderFormer V2 renderer checkpoints and the
|
| 17 |
material-latent models used by the data pipeline. Source code, installation
|
| 18 |
instructions, scene conversion, and inference examples live in
|
| 19 |
[RenderFormer Studio](https://github.com/iamNCJ/RenderFormer-Studio).
|
| 20 |
|
|
|
|
|
|
|
|
|
|
|
|
|
| 21 |
## Components
|
| 22 |
|
| 23 |
| Path | Class | Resolution | Purpose |
|
|
@@ -35,37 +36,27 @@ external dependency and are not duplicated in this bundle.
|
|
| 35 |
|
| 36 |
## Load a renderer
|
| 37 |
|
| 38 |
-
|
| 39 |
-
resolution you need, then construct the shared V2 pipeline:
|
| 40 |
|
| 41 |
```python
|
| 42 |
-
from pathlib import Path
|
| 43 |
-
|
| 44 |
-
import torch
|
| 45 |
-
from huggingface_hub import snapshot_download
|
| 46 |
from renderformer import RenderFormerV2Pipeline
|
| 47 |
|
| 48 |
-
snapshot = Path(snapshot_download(
|
| 49 |
-
"RenderFormer/renderformer-v2",
|
| 50 |
-
allow_patterns=["transformer_512/*"],
|
| 51 |
-
))
|
| 52 |
pipeline = RenderFormerV2Pipeline.from_pretrained(
|
| 53 |
-
|
|
|
|
| 54 |
device="cuda",
|
| 55 |
-
torch_dtype=torch.float16,
|
| 56 |
)
|
|
|
|
| 57 |
```
|
| 58 |
|
| 59 |
-
|
| 60 |
-
|
| 61 |
-
precision, view batching, and tone
|
|
|
|
| 62 |
|
| 63 |
## Load material components
|
| 64 |
|
| 65 |
```python
|
| 66 |
-
from pathlib import Path
|
| 67 |
-
|
| 68 |
-
from huggingface_hub import snapshot_download
|
| 69 |
from renderformer.models.material import (
|
| 70 |
DiffuseSpecularToLatent,
|
| 71 |
MaterialAutoencoder,
|
|
@@ -73,26 +64,18 @@ from renderformer.models.material import (
|
|
| 73 |
PrincipledBRDFToLatentWithTransmission,
|
| 74 |
)
|
| 75 |
|
| 76 |
-
|
| 77 |
-
"RenderFormer/renderformer-v2",
|
| 78 |
-
allow_patterns=[
|
| 79 |
-
"material_autoencoder/*",
|
| 80 |
-
"diffspec_mapper/*",
|
| 81 |
-
"metallic_mapper/*",
|
| 82 |
-
"metallic_transmission_mapper/*",
|
| 83 |
-
],
|
| 84 |
-
))
|
| 85 |
material_autoencoder = MaterialAutoencoder.from_pretrained(
|
| 86 |
-
|
| 87 |
).eval()
|
| 88 |
diffspec_mapper = DiffuseSpecularToLatent.from_pretrained(
|
| 89 |
-
|
| 90 |
).eval()
|
| 91 |
metallic_mapper = PrincipledBRDFToLatent.from_pretrained(
|
| 92 |
-
|
| 93 |
).eval()
|
| 94 |
transmission_mapper = PrincipledBRDFToLatentWithTransmission.from_pretrained(
|
| 95 |
-
|
| 96 |
).eval()
|
| 97 |
```
|
| 98 |
|
|
|
|
| 10 |
|
| 11 |
# RenderFormer V2
|
| 12 |
|
|
|
|
|
|
|
|
|
|
| 13 |
This repository bundles the two RenderFormer V2 renderer checkpoints and the
|
| 14 |
material-latent models used by the data pipeline. Source code, installation
|
| 15 |
instructions, scene conversion, and inference examples live in
|
| 16 |
[RenderFormer Studio](https://github.com/iamNCJ/RenderFormer-Studio).
|
| 17 |
|
| 18 |
+
If this repository is access-protected during release staging, authenticate
|
| 19 |
+
with Hugging Face before loading it. One repository revision selects every
|
| 20 |
+
component below as an atomic, compatible set.
|
| 21 |
+
|
| 22 |
## Components
|
| 23 |
|
| 24 |
| Path | Class | Resolution | Purpose |
|
|
|
|
| 36 |
|
| 37 |
## Load a renderer
|
| 38 |
|
| 39 |
+
`resolution` automatically selects `transformer_512` or `transformer_2048`:
|
|
|
|
| 40 |
|
| 41 |
```python
|
|
|
|
|
|
|
|
|
|
|
|
|
| 42 |
from renderformer import RenderFormerV2Pipeline
|
| 43 |
|
|
|
|
|
|
|
|
|
|
|
|
|
| 44 |
pipeline = RenderFormerV2Pipeline.from_pretrained(
|
| 45 |
+
"RenderFormer/renderformer-v2",
|
| 46 |
+
resolution=512,
|
| 47 |
device="cuda",
|
|
|
|
| 48 |
)
|
| 49 |
+
result = pipeline("scene.h5", resolution=512, precision="fp16")
|
| 50 |
```
|
| 51 |
|
| 52 |
+
Advanced callers may select a component explicitly with
|
| 53 |
+
`subfolder="transformer_512"`. The CLI and pipeline documentation in
|
| 54 |
+
RenderFormer Studio cover H5 preparation, precision, view batching, and tone
|
| 55 |
+
mapping.
|
| 56 |
|
| 57 |
## Load material components
|
| 58 |
|
| 59 |
```python
|
|
|
|
|
|
|
|
|
|
| 60 |
from renderformer.models.material import (
|
| 61 |
DiffuseSpecularToLatent,
|
| 62 |
MaterialAutoencoder,
|
|
|
|
| 64 |
PrincipledBRDFToLatentWithTransmission,
|
| 65 |
)
|
| 66 |
|
| 67 |
+
repo = "RenderFormer/renderformer-v2"
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 68 |
material_autoencoder = MaterialAutoencoder.from_pretrained(
|
| 69 |
+
repo, subfolder="material_autoencoder", strict=True
|
| 70 |
).eval()
|
| 71 |
diffspec_mapper = DiffuseSpecularToLatent.from_pretrained(
|
| 72 |
+
repo, subfolder="diffspec_mapper", strict=True
|
| 73 |
).eval()
|
| 74 |
metallic_mapper = PrincipledBRDFToLatent.from_pretrained(
|
| 75 |
+
repo, subfolder="metallic_mapper", strict=True
|
| 76 |
).eval()
|
| 77 |
transmission_mapper = PrincipledBRDFToLatentWithTransmission.from_pretrained(
|
| 78 |
+
repo, subfolder="metallic_transmission_mapper", strict=True
|
| 79 |
).eval()
|
| 80 |
```
|
| 81 |
|