workflow-studio / README.md
owenisas's picture
Collect and freeze parent garbage before each worker fork (worker GC segfault); drop spacing
513c139 verified
|
Raw History Blame Contribute Delete
14.4 kB

A newer version of the Gradio SDK is available: 6.32.0

Upgrade
metadata
title: Workflow Studio
emoji: 🧩
colorFrom: blue
colorTo: purple
sdk: gradio
sdk_version: 6.28.0
python_version: '3.12'
app_file: app.py
pinned: false
short_description: Self-contained H3 workflows with one automation API
suggested_hardware: zero-a10g
hf_oauth: true
hf_oauth_scopes:
  - inference-api

Workflow Studio

Last updated: 2026-10-22

An additive, self-contained H3 FL2VA workflow runner. One Space and one API request perform local Qwen3-VL conditioning, H3 generation and output encoding. It never invokes another Space. Presets select parameters and LoRA adapters. Ref2VA, the Qwen audio family, Music3, Stable Audio, and Phase-3 have not been migrated into this app.

Acceptance state: experimental. Local tests do not establish successful GPU generation, output quality, speed, or memory fit. /health separates runtime initialization from resident model state. Inference is disabled by default. Existing production clients and deployments are not redirected.

Presets and model state

presets.json is the workflow registry. It includes stable, quality-20, quality-25, quality-32, fast, turbo-quality, rough, lightx8 and landscape-detail. Requests cannot supply arbitrary Python, a custom-node package, a model URL or an unrestricted checkpoint path.

  • The same pinned FL2VA base checkpoint serves every preset; each generation loads pristine weights.
  • Larry and LightX use non-fused low-rank forward hooks scoped to a serialized call.
  • Adapter hooks are removed even when inference fails. Base weights remain unchanged.
  • Scheduler configuration resets between requests.
  • Compiled AoTI execution is disabled. These are eager equivalents, not promises of the old compiled profiles' speed or identical rounding.
  • The FP32 low-rank mapping comes from the inspected owned deployments; actual transformer integration and GPU acceptance remain unverified.
  • Only one stage's required components are downloaded at a time, not the entire MiniMax repository or its Ref2VA/native duplicates.

Local model lifecycle

The BF16 Qwen encoder and H3 generator cannot reside together on one 96 GB GPU. This implementation stages them sequentially inside the same Space:

  1. Reserve one GPU allocation for the entire workflow, before downloading either stage. Download pinned conditioner components into an isolated stage directory.
  2. Load and run Qwen locally. Retain CPU tensors and release its model references and CUDA allocations, while keeping the same GPU reservation.
  3. Remove only the completed stage's owned checkpoint directory.
  4. Stage the generator and selected adapter. Run H3 within that same reservation, then release its model and checkpoint stage.
  5. Return CPU frames/audio, release the GPU reservation, encode the MP4 and return it to the original API request.

There are no intermediate tensor uploads, signing secrets, remote conditioner URLs, or server-to-server inference calls. Input and internal tensor/geometry validation remain enabled. The single GPU allocation belongs to the original request's caller.

This bounded-storage cold path avoids ZeroGPU startup packing and simultaneous checkpoint residency. Model files are downloaded again on later requests, inside the GPU allocation, consuming quota. A two-allocation prototype completed Qwen but its proxy token expired before generator admission; the single allocation removes that second scheduling boundary. The estimator reserves both stage budgets plus 370 seconds for downloads and rejects estimates above 900 seconds rather than clamping. Hosted node speed varies about 6Γ—: three runs used 224–524 s for the whole allocation and 178–243 s for downloads. The allowances keep at least 1.5Γ— headroom over the slowest run. Every preset fits at 2 s (719–827 s reserved). Longest admissible duration at the default canvas: 8 s for stable, fast and rough; 7 s for turbo-quality and lightx8; 6 s for quality-20; 5 s for quality-25 and quality-32; 4 s for landscape-detail. That matters for quota as well as admission: ZeroGPU admits an xlarge job only when the caller has twice its reservation left, though it bills actual use. /plan validates model parameters, not GPU admission. Download and loading latency must be measured before performance claims. The staged cache is blocked from Gradio file serving. No laptop model downloads are needed.

Workflow canvas

The whole page is a Gradio 6.28 WorkflowCanvas, laid out like a gr.Workflow Space: graph title, View API and Templates on top, Component and Run at the bottom. Templates opens a gallery of 11 workflows from workflow_templates.py: text-to-video for each of the nine presets, plus first-frame and first+last-frame versions of stable. Each card shows its mode, canvas and reserved GPU seconds. Each graph wires prompt, seconds, seed and image nodes into one generate_video Β· LOCAL node with video and metadata outputs. Preset and canvas resolution are dropdowns on that node.

  • Run uses the Gradio queue (/predict_fn_generate_video, undocumented) and calls the same handler as /generate (canvas_bridge.py): same validation, file checks, RUN_LOCK and single GPU allocation. Neither route waits behind a running GPU job, because a long queue wait can outlive the caller's ZeroGPU identity token. A second run returns runtime_busy; … nothing was submitted immediately.
  • ZeroGPU quota: on huggingface.co the Space page passes your signed-in identity automatically. Sign in with Hugging Face (OAuth, inference-api scope) shows who you are on the canvas. On the direct *.hf.space URL, canvas_launch.js also sends your own token with each run, so the Hugging Face proxy bills your quota. API clients pass their token as usual.
  • A page left open for hours can hold an expired identity token. The run then stops before scheduling with zerogpu_session_expired; reload the page … Nothing was submitted.
  • Images are uploaded by the canvas. The bridge accepts only files inside Gradio's upload folder.
  • View API documents the real named endpoints below, with Gradio's placeholder values.
  • Model, dataset, pasted-token and history controls are hidden. They need gr.Workflow's remote helpers, and this Space runs one local pipeline only.
  • Errors before submission say nothing was submitted. generation_outcome_unknown means the GPU job may still finish, so check /health before running again.
  • Canvas edits stay in the browser session. Stop only stops the browser graph; it does not cancel allocated GPU work, so it is hidden.

Reference to video (Ref2VA)

reference_to_video runs MiniMax-H3's Ref2VA mode on the stable base model. The model reads up to three reference images, one soundtrack and one 2–15 s video, in that order, then the prompt. It uses the same one-allocation design: the Qwen conditioner stage with Ref2VA blocks, then the transformer_ref generator stage (same ~72 GiB size as the FL2VA generator), then CPU MP4 encoding. Both halves read images at a 1024 px short edge (the model default is 2048). The budget is computed from each reference's probed size before anything is scheduled: one image fits up to 8 s, two up to 5 s, three up to 3 s. A video reference adds tens of thousands of sequence rows; in practice it exceeds the 900 s one-allocation cap and is refused before any GPU use. Ref2VA is 10 steps with no adapter, and output quality is unreviewed.

Custom workflows

Function on the canvas's bottom bar adds local nodes you can wire into your own graphs:

Node Inputs β†’ outputs Runs on
generate_video preset, prompt, first/last frame, canvas, seconds, seed β†’ video, metadata GPU (one allocation)
reference_to_video prompt, images 1–3, audio, video, canvas, seconds, seed β†’ video, metadata GPU (one allocation)
last_frame / first_frame video β†’ image CPU (ffmpeg)
concat_videos first video, second video β†’ video (scaled to the first clip, 24 fps, AAC) CPU (ffmpeg)

Each node runs through the queue as /predict_fn_<name>. Media inputs must be files the Space already holds (uploads or earlier outputs). Templates adds Reference to video Β· 2 images and Extend a clip Β· 2 shots (text β†’ last frame β†’ image-to-video β†’ join). Every GPU node is its own allocation, and the canvas runs nodes one after another, so a two-shot chain needs quota for two runs. Inputs fed by another node are marked required so a failed step stops the chain. On ports you wire yourself to an optional input such as image_path, a failed upstream node can still let the next GPU node run without that input.

Stable API contract

The live Gradio schema is /gradio_api/info. Endpoint names are explicit:

Endpoint Purpose
/presets Registry and current runtime readiness
/plan Validate preset, prompt, canvas, duration, seed and input roles without inference
/health Runtime blockers, local component identity, staging mode and busy state
/generate End-to-end local conditioning and generation; return MP4 and structured run metadata
/reference_to_video Ref2VA from reference images, audio and video; return MP4 and run metadata

/plan parameters, in order: preset_id, prompt, canvas, duration, seed, has_first, has_last.

/generate parameters, in order: preset_id, prompt, image_path, last_image_path, canvas, duration, seed.

/reference_to_video parameters, in order: prompt, image_1, image_2, image_3, audio, video, canvas, duration, seed (at least one image or video).

Runtime/catalog/run-report schema version is 2. The signed-input version-1 generation contract is intentionally replaced. Update old automation clients; there is no fallback to the old two-Space flow. Registry and planning data retain their existing version-1 format.

API scalar values retain their JSON types. Fractional/boolean seeds and non-string prompts are rejected, not coerced by UI widgets. Leading or trailing prompt whitespace is rejected consistently. Internal whitespace and Unicode are preserved.

Example planning call:

from gradio_client import Client
client = Client("https://owenisas-workflow-studio.hf.space", verbose=False)
print(client.predict(api_name="/presets"))
plan = client.predict("fast", "A small robot walks across a desk", "", 2, 42,
                      False, False, api_name="/plan")

Use the schema's actual file parameter types with gradio_client.handle_file. JSON paths alone are not file uploads.

Automation

The automation client connects only to Workflow Studio and submits /generate exactly once with the caller credential. The app does not use a deployment-owner token for inference or contact another Space.

Input JSON:

{
  "preset_id": "stable",
  "prompt": "A small robot walks across a desk",
  "duration": 2,
  "seed": 42
}

image and last_image are optional local paths. The client stages input copies, verifies the approved Workflow Studio revision, records one submitted workflow and output hashes, and never automatically retries an ambiguous job. A failure after internal conditioning may still have consumed GPU time.

The client requires an explicitly supplied inference-token environment variable and the existing prepaid-overage-disabled operator attestation. It does not use the saved deployment owner's token for generation. See python client.py --help for the current CLI contract. No paid fallback is configured.

run --max-wall-seconds applies one deadline across caller/revision checks, client initialization, health/planning waits and inference waits. Timed-out preflight reads cannot advance to GPU submission; late-created clients close. Local staging, journal writes and bounded cleanup add overhead. A post-submission timeout remains an unknown outcome, not permission to retry.

Runtime configuration

Startup never downloads model weights. When enabled and imports initialize successfully, /health reports runtime_initialized: true and status: ready. model_loaded: false is expected between requests because models are scoped to GPU execution. This is not proof of successful model loading or inference.

Setting Requirement
WORKFLOW_STUDIO_ENABLE_INFERENCE Explicit 1 to initialize the local runtime
H3_LICENSE_AUTHORIZED Existing deployment gate must be resolved before enabling
H3_LICENSE_AUTHORIZATION_REF Non-secret verified authorization reference
H3_AOTI Must remain 0
H3_TURBO_PROFILE Must remain off; the request selects its adapter
WORKFLOW_MODEL_CACHE Owned temporary stage root, default .workflow-models
GRADIO_SERVER_NAME 0.0.0.0 on the hosted Space; local default is loopback

The upstream model and adapter revisions are pinned in app.py. Their repositories are public, so the model loader does not need or receive a deployment-owner write token.

The model uses the MiniMax H3 Community License. Deployment approval is not proof of the separate authorization required by the existing deployment gate.

Local verification

Install Gradio 6.28.0 into an isolated environment for CPU-only API tests. Do not install the GPU requirements or download model weights onto a laptop just to test planning.

python -m pytest -q tests
WORKFLOW_STUDIO_ENABLE_INFERENCE=0 python app.py

Tests use tiny synthetic tensors and injected pipelines for adapter isolation, stage order, resource cleanup and internal tensor validation. Those fixtures are not model inference results.

Acceptance before migration

  1. Verify exact uploaded source hashes and the live Gradio schema.
  2. Resolve readiness blockers and verify both local model stages load on the chosen ZeroGPU size.
  3. Run approved short base β†’ Larry β†’ LightX β†’ base jobs. Measure wall time, memory and effective quota separately.
  4. Decode every output, verify geometry/audio and compare the final base run for adapter contamination.
  5. Exercise interrupted and failed jobs without duplicate submission.
  6. Retire or redirect old deployments only after their corresponding presets pass. No old Space is deleted by this project.