waveshaker's picture
Update README.md
d4bdcfb verified
|
Raw History Blame Contribute Delete
5.92 kB
---
title: Universal Model Character LoRA Studio
emoji: ๐ŸŽจ
colorFrom: gray
colorTo: green
sdk: gradio
hf_oauth: true
fullWidth: true
sdk_version: 6.31.0
python_version: '3.12'
app_file: app.py
pinned: true
short_description: "Run waveshaker's ๐— ๐—ข๐—ฆ๐—ฆ/๐—ž๐—ฟ๐—ฒ๐—ฎ๐Ÿฎ/๐—”๐—ป๐—ถ๐—บ๐—ฎ/๐—™๐—Ÿ๐—จ๐—ซ/๐—ฃ๐—ผ๐—ป๐˜† LoRAs"
---
---
<div align="center">
<h1>โšก Universal LoRA Studio</h1>
</div>
---
### โš™๏ธ System Architecture
This Space is a specialized, on-demand inference engine designed to run Waveshaker's character LoRAs on ZeroGPU. To overcome Hugging Face's cross-space CORS restrictions and ZeroGPU's billing constraints, I use a **decoupled "Proxy-Driven" architecture** with a blazing-fast native frontend.
#### ๐Ÿ—๏ธ System Topology
| Layer | Core Components | Role & Responsibility |
| :--- | :--- | :--- |
| **๐Ÿ–ฅ๏ธ Client** | โ€ข Native HTML/JS<br/>โ€ข REST API Calls | Authenticates users, provides a native UI, and submits generation parameters via a lightweight HTTP payload. |
| **๐Ÿง  Proxy & Scheduler Layer**<br/>(FastAPI + Gradio) | โ€ข **Dummy Probe**<br/>โ€ข **ContextVar**<br/>โ€ข **Singleton Executor**<br/>โ€ข Threading Lock (`_exec_lock`) | Bypasses the ZeroGPU scanner, passes the `x-ip-token` accurately, serializes incoming tasks, and enforces hard boundary limits (1536px). |
| **โšก Execution Layer**<br/>(ZeroGPU + ComfyUI) | โ€ข **Binary Tensor Fast-Path**<br/>โ€ข **RAM_PRESSURE Cache**<br/>โ€ข **WebP + EXIF Injection** | Executes the pipeline using custom zero-copy tensor loading. Relies entirely on ComfyUI's native LRU cache for multi-model hot-swapping without OOM. |
#### ๐Ÿ”„ Request Lifecycle
```text
[User] โ”€โ”€(OAuth Login)โ”€โ”€โ–บ [Native Studio UI]
โ”‚
โ”‚ (POST Model & LoRA Params)
โ–ผ
[FastAPI Routing & Thread Scheduler]
โ”œโ”€โ”€ HTTPException (Enforces 1536px limit)
โ”œโ”€โ”€ ContextVar (Binds x-ip-token accurately)
โ””โ”€โ”€ Background Thread (Appends to Queue)
โ”‚
โ”‚ (Task pops from queue)
โ–ผ
[Gradio + ZeroGPU Probe (@spaces.GPU duration=40)]
โ”œโ”€โ”€ Dummy Probe (Retains quota & allocates slot)
โ””โ”€โ”€ Threading Lock (Serializes jobs via _exec_lock)
โ”‚
โ”‚ (Passes to Singleton Executor)
โ–ผ
[ComfyUI Core Engine]
โ”œโ”€โ”€ Fast-Path Loader (Zero-copy numpy for INT8)
โ”œโ”€โ”€ RAM_PRESSURE Cache (Seamless hot-swapping)
โ””โ”€โ”€ WebP + Binary EXIF Injection
โ”‚
โ–ผ
[Static `/output/` Mount] โ”€โ”€โ–บ [Browser Renders Image]
```
#### ๐Ÿ“ One-Sentence Summary
> A decoupled "Proxy-Driven" architecture: a native frontend passes LoRA parameters via URL, while a Gradio proxy bypasses cross-space CORS and ZeroGPU billing constraints. It serves generated images as physical WebP URLs with custom-injected ComfyUI EXIF metadata, delivering accurate billing.
#### ๐Ÿ“œ Architecture Evolution
- **v0.1 (The FastAPI Initiation)**: The initial dual-track setup. FastAPI handled data routing, while Gradio rendered the UI. Due to cross-space CORS restrictions and a lack of `x-ip-token` handling, the frontend fell into the ZeroGPU anonymous pool.
- **v0.2 (The Gradio-Injection Era)**: A modular reconstruction (`core/`, `frontend/`, `ui/`). Introduced the `task_runner` for precise background scheduling and logging. To retain ZeroGPU billing, I hid Gradio components and injected native HTML/CSS/JS via Python. This successfully kept the GPU quota but introduced frontend performance bottlenecks and UI state sync issues.
- **v0.3 (The Decoupled Proxy UI)**: A new refactor. A 100% native static web UI is decoupled from the FastAPI backend. I implemented a dummy Gradio probe to trick the ZeroGPU scanner and used `ContextVar` to perfectly pass through the `x-ip-token`. Featuring a dual-Space sync mechanism (`frontend_cache`), it delivers faster loading,accurate ZeroGPU billing, and a pure native frontend experience.
- **v0.4 (The Performance Hardened Era)**: A brutal optimization phase to eliminate the 50s+ cold-start latency and multi-model memory leaks under ZeroGPU.
- **Binary Tensor Fast-Path**: Bypassed ComfyUI's native slow loading path by intercepting `load_torch_file` and implementing `numpy` zero-copy for large INT8/ConvRot tensors.
- **Lazy Execution & Native Cache**: Removed aggressive `empty_cache` and manual GC interventions. Relied entirely on ComfyUI's native `RAM_PRESSURE` (LRU 8, 16GB RAM limit) to orchestrate multi-model hot-swapping smoothly.
- **Queue-Aware Billing**: Adjusted `@spaces.GPU(duration=40)` to gracefully handle ZeroGPU queueing spikes during consecutive architecture switching.
- **Safety Boundaries**: Enforced hard pixel and edge limits (1536px) at both frontend and FastAPI levels to prevent OOM and timeout kills.
- **Result**: Cold-start load time dropped from ~24s to **~6.5s**. Cross-architecture hot-swapping (e.g., Flux 1536 โ†’ MOSS 1536) stabilized under **15s total**, completely eliminating the "ghost 35s" timeouts.
---
### ๐Ÿ”— Ecosystem Links
- ๐Ÿ–ผ๏ธ **[Browse the LoRA Gallery](https://huggingface.co/spaces/waveshaker/loras_gallery)** - Explore 1000+ community characters.
- ๐Ÿ“Š **[Dataset](https://huggingface.co/datasets/waveshaker/dataset_collections)**
<div align="center">
<sub>Engineered by <b>waveshaker</b>. Running on ZeroGPU. Duplicated and modified from <a href="https://huggingface.co/spaces/2i/pornmaster-krea2">2i/pornmaster-krea2</a>.</sub>
</div>