Spaces:
Running on Zero
Running on Zero
| title: Component Studio Reference | |
| emoji: 🧩 | |
| colorFrom: gray | |
| colorTo: blue | |
| sdk: gradio | |
| sdk_version: 6.27.0 | |
| python_version: "3.12" | |
| app_file: app.py | |
| pinned: false | |
| # Component Studio | |
| A new Gradio + uv pipeline targeting Hugging Face ZeroGPU. It accepts an image, | |
| a prompt, or both and produces a textured GLB with individually editable components. | |
| Add an animation prompt to run learned humanoid rigging, NVIDIA Kimodo motion generation, | |
| retargeting, video export, full-frame collision checks and Astra visual review. | |
| The animated workflow is under production verification. A completed export is not a quality | |
| pass: rejected models or animations retain their artifacts with `needs_review` status. | |
| ## Inputs | |
| | Input | Example | Behavior | | |
| | --- | --- | --- | | |
| | Image only | Upload a stool photograph | Reconstruct the supplied image; skip image generation | | |
| | Prompt only | “A brass desk lamp with a green glass shade” | Generate a reference with FLUX.2 Klein, then reconstruct it | | |
| | Both | Stool image + “Walnut seat and black metal legs; keep the shape” | Edit the image with FLUX.2 Klein, then reconstruct the edited reference | | |
| ## Pipeline stages | |
| 1. Validate inputs and prepare a reference. Images keep EXIF orientation and alpha. | |
| 2. TRELLIS.2 generates geometry at its 1024 cascade setting and a 1024px PBR atlas. Texture sampling | |
| uses twelve steps before component upscaling, giving refinement more source detail. | |
| 3. PartField predicts learned 3D features. Cluster those features into the requested number | |
| of parts, partition the original faces, crop component texture bounds and remap UVs. | |
| This is learned part segmentation, not merely splitting disconnected islands. The | |
| target count controls granularity; semantic correctness is not guaranteed. | |
| 4. Real-ESRGAN x4plus upscales each component's base color, with tiled overlap. Profile a | |
| maximal tile, inspect free RAM, CUDA memory and cgroup CPU limits, and admit the maximum | |
| workers fitting 90% of those resources. Recalculate each wave. Minimum concurrency is | |
| one; if one cannot fit safely, shrink tiles or fail instead of overriding the reserve. | |
| CUDA OOM halves concurrency and then tile size. GPU leases and requests are serialized; | |
| component workers run concurrently within the lease. Alpha is interpolated separately. | |
| Normal, metallic, roughness, occlusion and emission maps preserve their data and UVs. | |
| 5. Astra (`openai/gpt-6-astra`) reviews the reference and rendered views. An embedded fx | |
| agent inspects per-run evidence inside a fresh just-bash virtual filesystem and returns | |
| a validated repair plan. Its tools cannot access the host filesystem, shell or network. | |
| The host applies bounded repairs, renders again and preserves the original on regression. | |
| Quality acceptance is a separate decision from non-regression. | |
| 6. For an animation prompt, learn the humanoid skeleton and weights, transfer them onto | |
| the original textured components, and generate the requested motion with pinned native | |
| NVIDIA Kimodo. Retarget it in Blender and export the actual animated GLB and MP4. | |
| 7. Measure every exported frame for new triangle intersections and ground penetration; | |
| render chronological front/side evidence and ask Astra to judge the requested action, | |
| deformation, texture fidelity and contacts. A failing review remains `needs_review`. | |
| 8. Package the original inputs, candidate meshes, textures, rig, source motion, GLB, | |
| video, audit evidence and provenance. Downloads belong to one unique run directory. | |
| ## Local development | |
| ```sh | |
| uv sync --locked | |
| uv run pytest | |
| uv run app.py | |
| ``` | |
| On a Mac this starts the interface without loading CUDA models. Generation reports that | |
| the GPU runtime is required; it does not substitute sample meshes. `PORT=7861 uv run app.py` | |
| starts another local instance. No public sharing is enabled. | |
| ## Hugging Face ZeroGPU deployment | |
| Create a **Gradio Space**, select **ZeroGPU** hardware, and upload this checkout including | |
| `requirements.txt`, `requirements-hf.lock`, `requirements-gpu.txt`, `packages.txt`, `models.lock.json`, `uv.lock`, `vendor/fx/`, | |
| `studio/`, `scripts/`, and `assets/`. Do not upload `.runtime/`, credentials, or old outputs. | |
| Add `OPENROUTER_API_KEY` as a Space secret. Add `HF_TOKEN` from an account with access to TRELLIS’s gated DINOv3 and BRIA | |
| RMBG-2.0 dependencies. Obtain that access on the model pages before startup; setup | |
| does not accept model terms for you. The exact `openai/gpt-6-astra` route is used for review and fx repair planning. | |
| OpenRouter billing and availability apply; no substitute review model is selected. | |
| Native Kimodo also requires access to `meta-llama/Meta-Llama-3-8B-Instruct`. | |
| HF installs the CUDA dependencies and Blender at build time. On startup `app.py` detects | |
| `SPACE_ID`, downloads pinned source/models and installs the pinned embedded fx/Node runtime, then loads models before handling | |
| requests. GPU stages use `@spaces.GPU`; CPU exports, rendering and fx run outside leases. | |
| Only CPU inputs/results cross worker queues. Upscaled textures are returned explicitly | |
| to the parent process; models and UI callbacks are never serialized as worker arguments. | |
| The supported wheel target is **Linux x86_64, Python 3.12, torch 2.11, CUDA 13** and follows | |
| Microsoft's current TRELLIS.2 ZeroGPU Space. This is not a Docker Space. | |
| For a matching local NVIDIA machine: | |
| ```sh | |
| uv sync --locked | |
| uv pip install -r requirements-gpu.txt | |
| uv run --no-sync scripts/prepare_runtime.py | |
| STUDIO_NATIVE=1 uv run --no-sync app.py | |
| ``` | |
| Use `--no-sync` after adding the GPU requirements; normal uv sync intentionally manages | |
| only the portable development environment. GPU requirements are kept separate because | |
| the upstream CUDA wheels cannot install on macOS. Startup requires substantial model | |
| storage and RAM; downloads are cached. ZeroGPU quota expiry, unavailable models and memory | |
| failures are surfaced with retained intermediate files, never reported as completed runs. | |
| Refinement uses `libfx` with an explicit host-owned OpenRouter transport and a fresh | |
| in-memory just-bash filesystem for each attempt. Only the run's review evidence is mounted; | |
| credentials remain outside tools. Tool calls, model calls, memory, output size and elapsed | |
| time are bounded. Runtime dependencies and the upstream cleanup skill are pinned. | |
| ## Model choices and known limits | |
| See [model decisions](docs/models.md) for sources, revision pins and the quality/latency | |
| tradeoffs. “Newest” is not a claim of superior texture reconstruction. Neural upscaling | |
| cannot recover details that the source model never generated. | |
| The pinned fx source's custom OpenRouter adapter does not accept native image inputs. A separate | |
| Ling vision call supplies visual evidence to Ling inside fx. The fx stage is constrained | |
| to producing a validated plan; it does not execute arbitrary model-authored scripts in a | |
| public Space. Technical validation and a VLM verdict are not a guarantee of visual quality. | |
| The original code is in `stash@{0}` (`Archive pre-rebuild forge3d and Mercy experiments`). | |
| Ignored old outputs, weights and local secret pointers were moved to the sibling directory | |
| `../.3dgen-legacy-20260916/`. They are not runtime dependencies of this implementation. | |
| Use `git stash show --stat stash@{0}` to inspect the archived source without restoring it. | |
| The released fx v0.0.10 binary predates custom OpenRouter support. The Linux deployment uses a checksummed binary built from | |
| commit `b51be034fea6a26cf9ac7aaa647fbe4bf089914b` with Zig 0.16; source and rebuild details | |
| are in `vendor/fx/BUILD.md`. Other platforms build the same source during setup. | |
| See [verification status](docs/verification.md) for executed checks and the remaining | |
| ZeroGPU smoke test. Regenerate the Linux lock with: | |
| ```sh | |
| uv pip compile requirements-hf.in --python-version 3.12 --python-platform x86_64-manylinux_2_28 --emit-index-url -o requirements-hf.lock | |
| sed -e '/^-e \.$/d' -e 's/^torch==2.11.0+cu130$/torch==2.11.0/' -e 's/^torchvision==0.26.0+cu130$/torchvision==0.26.0/' requirements-hf.lock > requirements.txt | |
| ``` | |
| Archived source stash object: `0899c5725eddd965045cf645af45fbf9a60e10c2`. | |
| ## Continue an interrupted generation | |
| If a later GPU lease fails, retain `source.glb` and `reference.png` from that run. | |
| The **Continue an existing textured mesh** panel accepts those files and resumes | |
| PartField, texture upscaling, review and packaging in a fresh request. Its API is | |
| `/continue_asset(image, source_mesh, prompt, parts, seed)`. The prompt supplies the | |
| review brief; it does not regenerate the reference. The continued run records | |
| `continued_from_source` and preserves the supplied GLB bytes. | |
| The September 18 astronaut experiment is in `outputs/animated-astronaut/`. | |
| `scripts/animate_kimodo_astronaut.py` fits that specific character to an exported | |
| Kimodo NPZ/BVH pair in Blender; it is not a generic auto-rigger. Its delivered | |
| GLB retains eight components and a 21-bone skin. Revision 2 projects the reference | |
| into a 4096px front texture and constrains the Kimodo motion to a smaller elbow-led | |
| gesture. All 180 exported frames have zero new triangle intersections relative | |
| to rest; seven static-mesh intersections remain. See `docs/verification.md` and | |
| `outputs/animated-astronaut/final/collision-comparison.json` for the exact scope. | |