--- title: Workflow Studio emoji: ๐Ÿงฉ colorFrom: blue colorTo: purple sdk: gradio sdk_version: 6.28.0 python_version: '3.12' app_file: app.py pinned: false short_description: Self-contained H3 workflows with one automation API suggested_hardware: zero-a10g hf_oauth: true hf_oauth_scopes: - inference-api --- # Workflow Studio Last updated: 2026-10-22 An additive, self-contained H3 FL2VA workflow runner. **One Space and one API request perform local Qwen3-VL conditioning, H3 generation and output encoding.** It never invokes another Space. Presets select parameters and LoRA adapters. Ref2VA, the Qwen audio family, Music3, Stable Audio, and Phase-3 have not been migrated into this app. **Acceptance state:** experimental. Local tests do not establish successful GPU generation, output quality, speed, or memory fit. `/health` separates runtime initialization from resident model state. Inference is disabled by default. Existing production clients and deployments are not redirected. ## Presets and model state `presets.json` is the workflow registry. It includes stable, quality-20, quality-25, quality-32, fast, turbo-quality, rough, lightx8 and landscape-detail. Requests cannot supply arbitrary Python, a custom-node package, a model URL or an unrestricted checkpoint path. - The same pinned FL2VA base checkpoint serves every preset; each generation loads pristine weights. - Larry and LightX use non-fused low-rank forward hooks scoped to a serialized call. - Adapter hooks are removed even when inference fails. Base weights remain unchanged. - Scheduler configuration resets between requests. - Compiled AoTI execution is disabled. These are eager equivalents, not promises of the old compiled profiles' speed or identical rounding. - The FP32 low-rank mapping comes from the inspected owned deployments; actual transformer integration and GPU acceptance remain unverified. - Only one stage's required components are downloaded at a time, not the entire MiniMax repository or its Ref2VA/native duplicates. ## Local model lifecycle The BF16 Qwen encoder and H3 generator cannot reside together on one 96 GB GPU. This implementation stages them sequentially inside the same Space: 1. Reserve one GPU allocation for the entire workflow, before downloading either stage. Download pinned conditioner components into an isolated stage directory. 2. Load and run Qwen locally. Retain CPU tensors and release its model references and CUDA allocations, while keeping the same GPU reservation. 3. Remove only the completed stage's owned checkpoint directory. 4. Stage the generator and selected adapter. Run H3 within that same reservation, then release its model and checkpoint stage. 5. Return CPU frames/audio, release the GPU reservation, encode the MP4 and return it to the original API request. There are no intermediate tensor uploads, signing secrets, remote conditioner URLs, or server-to-server inference calls. Input and internal tensor/geometry validation remain enabled. The single GPU allocation belongs to the original request's caller. This bounded-storage cold path avoids ZeroGPU startup packing and simultaneous checkpoint residency. Model files are downloaded again on later requests, **inside the GPU allocation, consuming quota**. A two-allocation prototype completed Qwen but its proxy token expired before generator admission; the single allocation removes that second scheduling boundary. The estimator reserves both stage budgets plus 370 seconds for downloads and rejects estimates above 900 seconds rather than clamping. Hosted node speed varies about 6ร—: three runs used 224โ€“524 s for the whole allocation and 178โ€“243 s for downloads. The allowances keep at least 1.5ร— headroom over the slowest run. Every preset fits at 2 s (719โ€“827 s reserved). Longest admissible duration at the default canvas: 8 s for `stable`, `fast` and `rough`; 7 s for `turbo-quality` and `lightx8`; 6 s for `quality-20`; 5 s for `quality-25` and `quality-32`; 4 s for `landscape-detail`. That matters for quota as well as admission: ZeroGPU admits an xlarge job only when the caller has twice its reservation left, though it bills actual use. `/plan` validates model parameters, not GPU admission. Download and loading latency must be measured before performance claims. The staged cache is blocked from Gradio file serving. No laptop model downloads are needed. ## Workflow canvas The whole page is a Gradio 6.28 `WorkflowCanvas`, laid out like a `gr.Workflow` Space: graph title, **View API** and **Templates** on top, **Component** and **Run** at the bottom. **Templates** opens a gallery of 11 workflows from `workflow_templates.py`: text-to-video for each of the nine presets, plus first-frame and first+last-frame versions of `stable`. Each card shows its mode, canvas and reserved GPU seconds. Each graph wires prompt, seconds, seed and image nodes into one `generate_video ยท LOCAL` node with video and metadata outputs. Preset and canvas resolution are dropdowns on that node. - **Run** uses the Gradio queue (`/predict_fn_generate_video`, undocumented) and calls the same handler as `/generate` (`canvas_bridge.py`): same validation, file checks, `RUN_LOCK` and single GPU allocation. Neither route waits behind a running GPU job, because a long queue wait can outlive the caller's ZeroGPU identity token. A second run returns `runtime_busy; โ€ฆ nothing was submitted` immediately. - **ZeroGPU quota:** on huggingface.co the Space page passes your signed-in identity automatically. **Sign in with Hugging Face** (OAuth, `inference-api` scope) shows who you are on the canvas. On the direct `*.hf.space` URL, `canvas_launch.js` also sends your own token with each run, so the Hugging Face proxy bills your quota. API clients pass their token as usual. - A page left open for hours can hold an expired identity token. The run then stops before scheduling with `zerogpu_session_expired; reload the page โ€ฆ Nothing was submitted`. - Images are uploaded by the canvas. The bridge accepts only files inside Gradio's upload folder. - **View API** documents the real named endpoints below, with Gradio's placeholder values. - Model, dataset, pasted-token and history controls are hidden. They need `gr.Workflow`'s remote helpers, and this Space runs one local pipeline only. - Errors before submission say nothing was submitted. `generation_outcome_unknown` means the GPU job may still finish, so check `/health` before running again. - Canvas edits stay in the browser session. Stop only stops the browser graph; it does not cancel allocated GPU work, so it is hidden. ## Reference to video (Ref2VA) `reference_to_video` runs MiniMax-H3's Ref2VA mode on the stable base model. The model reads up to three reference images, one soundtrack and one 2โ€“15 s video, in that order, then the prompt. It uses the same one-allocation design: the Qwen conditioner stage with Ref2VA blocks, then the `transformer_ref` generator stage (same ~72 GiB size as the FL2VA generator), then CPU MP4 encoding. Both halves read images at a 1024 px short edge (the model default is 2048). The budget is computed from each reference's probed size before anything is scheduled: one image fits up to 8 s, two up to 5 s, three up to 3 s. A video reference adds tens of thousands of sequence rows; in practice it exceeds the 900 s one-allocation cap and is refused before any GPU use. Ref2VA is 10 steps with no adapter, and output quality is unreviewed. ## Custom workflows **Function** on the canvas's bottom bar adds local nodes you can wire into your own graphs: | Node | Inputs โ†’ outputs | Runs on | |---|---|---| | `generate_video` | preset, prompt, first/last frame, canvas, seconds, seed โ†’ video, metadata | GPU (one allocation) | | `reference_to_video` | prompt, images 1โ€“3, audio, video, canvas, seconds, seed โ†’ video, metadata | GPU (one allocation) | | `last_frame` / `first_frame` | video โ†’ image | CPU (ffmpeg) | | `concat_videos` | first video, second video โ†’ video (scaled to the first clip, 24 fps, AAC) | CPU (ffmpeg) | Each node runs through the queue as `/predict_fn_`. Media inputs must be files the Space already holds (uploads or earlier outputs). Templates adds **Reference to video ยท 2 images** and **Extend a clip ยท 2 shots** (text โ†’ last frame โ†’ image-to-video โ†’ join). Every GPU node is its own allocation, and the canvas runs nodes one after another, so a two-shot chain needs quota for two runs. Inputs fed by another node are marked required so a failed step stops the chain. On ports you wire yourself to an optional input such as `image_path`, a failed upstream node can still let the next GPU node run without that input. ## Stable API contract The live Gradio schema is `/gradio_api/info`. Endpoint names are explicit: | Endpoint | Purpose | |---|---| | `/presets` | Registry and current runtime readiness | | `/plan` | Validate preset, prompt, canvas, duration, seed and input roles without inference | | `/health` | Runtime blockers, local component identity, staging mode and busy state | | `/generate` | End-to-end local conditioning and generation; return MP4 and structured run metadata | | `/reference_to_video` | Ref2VA from reference images, audio and video; return MP4 and run metadata | `/plan` parameters, in order: `preset_id`, `prompt`, `canvas`, `duration`, `seed`, `has_first`, `has_last`. `/generate` parameters, in order: `preset_id`, `prompt`, `image_path`, `last_image_path`, `canvas`, `duration`, `seed`. `/reference_to_video` parameters, in order: `prompt`, `image_1`, `image_2`, `image_3`, `audio`, `video`, `canvas`, `duration`, `seed` (at least one image or video). Runtime/catalog/run-report schema version is **2**. The signed-input version-1 generation contract is intentionally replaced. Update old automation clients; there is no fallback to the old two-Space flow. Registry and planning data retain their existing version-1 format. API scalar values retain their JSON types. Fractional/boolean seeds and non-string prompts are rejected, not coerced by UI widgets. Leading or trailing prompt whitespace is rejected consistently. Internal whitespace and Unicode are preserved. Example planning call: ```python from gradio_client import Client client = Client("https://owenisas-workflow-studio.hf.space", verbose=False) print(client.predict(api_name="/presets")) plan = client.predict("fast", "A small robot walks across a desk", "", 2, 42, False, False, api_name="/plan") ``` Use the schema's actual file parameter types with `gradio_client.handle_file`. JSON paths alone are not file uploads. ## Automation The automation client connects only to Workflow Studio and submits `/generate` exactly once with the caller credential. The app does not use a deployment-owner token for inference or contact another Space. Input JSON: ```json { "preset_id": "stable", "prompt": "A small robot walks across a desk", "duration": 2, "seed": 42 } ``` `image` and `last_image` are optional local paths. The client stages input copies, verifies the approved Workflow Studio revision, records one submitted workflow and output hashes, and never automatically retries an ambiguous job. A failure after internal conditioning may still have consumed GPU time. The client requires an explicitly supplied inference-token environment variable and the existing prepaid-overage-disabled operator attestation. It does not use the saved deployment owner's token for generation. See `python client.py --help` for the current CLI contract. No paid fallback is configured. `run --max-wall-seconds` applies one deadline across caller/revision checks, client initialization, health/planning waits and inference waits. Timed-out preflight reads cannot advance to GPU submission; late-created clients close. Local staging, journal writes and bounded cleanup add overhead. A post-submission timeout remains an unknown outcome, not permission to retry. ## Runtime configuration Startup never downloads model weights. When enabled and imports initialize successfully, `/health` reports `runtime_initialized: true` and `status: ready`. `model_loaded: false` is expected between requests because models are scoped to GPU execution. This is not proof of successful model loading or inference. | Setting | Requirement | |---|---| | `WORKFLOW_STUDIO_ENABLE_INFERENCE` | Explicit `1` to initialize the local runtime | | `H3_LICENSE_AUTHORIZED` | Existing deployment gate must be resolved before enabling | | `H3_LICENSE_AUTHORIZATION_REF` | Non-secret verified authorization reference | | `H3_AOTI` | Must remain `0` | | `H3_TURBO_PROFILE` | Must remain `off`; the request selects its adapter | | `WORKFLOW_MODEL_CACHE` | Owned temporary stage root, default `.workflow-models` | | `GRADIO_SERVER_NAME` | `0.0.0.0` on the hosted Space; local default is loopback | The upstream model and adapter revisions are pinned in `app.py`. Their repositories are public, so the model loader does not need or receive a deployment-owner write token. The model uses the [MiniMax H3 Community License](https://huggingface.co/MiniMaxAI/MiniMax-H3/blob/main/LICENSE). Deployment approval is not proof of the separate authorization required by the existing deployment gate. ## Local verification Install Gradio 6.28.0 into an isolated environment for CPU-only API tests. Do not install the GPU requirements or download model weights onto a laptop just to test planning. ```bash python -m pytest -q tests WORKFLOW_STUDIO_ENABLE_INFERENCE=0 python app.py ``` Tests use tiny synthetic tensors and injected pipelines for adapter isolation, stage order, resource cleanup and internal tensor validation. Those fixtures are not model inference results. ## Acceptance before migration 1. Verify exact uploaded source hashes and the live Gradio schema. 2. Resolve readiness blockers and verify both local model stages load on the chosen ZeroGPU size. 3. Run approved short base โ†’ Larry โ†’ LightX โ†’ base jobs. Measure wall time, memory and effective quota separately. 4. Decode every output, verify geometry/audio and compare the final base run for adapter contamination. 5. Exercise interrupted and failed jobs without duplicate submission. 6. Retire or redirect old deployments only after their corresponding presets pass. No old Space is deleted by this project.