Instructions to use mlx-community/RelayVSR-bf16 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- MLX
How to use mlx-community/RelayVSR-bf16 with MLX:
# Download the model from the Hub pip install huggingface_hub[hf_xet] hf download mlx-community/RelayVSR-bf16 --local-dir RelayVSR-bf16
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- LM Studio
- Atomic Chat
|
Download README.md from mlx-community/RelayVSR-bf16: direct link, hf CLI and curl.
- Browser
- Download file 3.98 kB
-
https://huggingface.co/mlx-community/RelayVSR-bf16/resolve/main/README.md
- Command line
-
hf download hf://mlx-community/RelayVSR-bf16/README.md
-
curl -L -o README.md https://huggingface.co/mlx-community/RelayVSR-bf16/resolve/main/README.md
3.98 kB
| library_name: mlx | |
| license: apache-2.0 | |
| license_link: https://huggingface.co/kopper/RelayVSR/blob/main/LICENSE | |
| pipeline_tag: video-to-video | |
| base_model: kopper/RelayVSR | |
| tags: | |
| - mlx | |
| - apple-silicon | |
| - video-super-resolution | |
| - super-resolution | |
| - video-to-video | |
| - relayvsr | |
| # mlx-community/RelayVSR-bf16 | |
| **RelayVSR** ([kopper/RelayVSR](https://huggingface.co/kopper/RelayVSR), | |
| [kopperx/RelayVSR](https://github.com/kopperx/RelayVSR), [arXiv 2609.37850](https://arxiv.org/abs/2609.37850)) — | |
| streaming ×4 video super-resolution by large–small model collaboration — converted to MLX for the Swift/MLX port | |
| [`xocialize/mlx-relayvsr-swift`](https://github.com/xocialize/mlx-relayvsr-swift). | |
| A large generative model (the Wan2.2-TI2V-5B DiT, distilled to one step) runs on **one frame in fifteen** and | |
| proposes a reference latent. A 32 M-parameter transformer then super-resolves **every** frame from the | |
| low-resolution frame and the two reference latents around it. There is no VAE and no text encoder at runtime. | |
| ## What to expect | |
| - **Faithful.** It stays close to the source: defocus stays soft and a drawing stays a drawing, so it suits anime and | |
| graphics as well as live action. | |
| - **Soft.** It removes film grain and adds little high-frequency detail, and it does not repair codec damage. | |
| - **×4 only**; output lags input by up to 15 frames; every frame is kept. | |
| ## Files | |
| | file | component | params | notes | | |
| |---|---|---|---| | |
| | `generator_bf16.safetensors` | the reference generator: Wan2.2-TI2V-5B DiT (LoRA merged, one step at t = 1000) + the LQ projector | 5,031,059,136 | released dtype (bf16), bit for bit; `patch_embedding` Conv3d (O,D,H,W,I), `lq_proj` Conv2d (O,H,W,I) | | |
| | `decoder_fp32.safetensors` | the lightweight VSR network (upstream `flash.safetensors`, variant S, step 60000 EMA) | 31,934,208 | released dtype (fp32), bit for bit; Conv2d (O,H,W,I) | | |
| | `prompt_bf16.safetensors` | the fixed prompt embedding (1, 512, 4096) | — | split out of upstream's `wan.safetensors` | | |
| | `config.json` | both upstream configs, sampling constants, provenance (source sha256s) | | | | |
| Keys are upstream's verbatim; only conv layouts change (channels-last), and the two `lq_proj` norm weights are | |
| flattened from (C, 1, 1) to (C). Produced by the port's `oracle/convert_weights.py` from `kopper/RelayVSR @ 751954dc`. | |
| One lane serves both precisions: the port runs the generator in bf16 and the small network in fp16 by default, and | |
| upcasts the same files exactly for its fp32 parity lane. | |
| ## Parity with upstream | |
| Gated against upstream's own code executed verbatim on the CPU in fp32 (a 9-frame clip chosen so that both window | |
| masks bite and the generator's 4-keyframe cache wraps): | |
| - key contract: 833 + 200 tensors, no missing and no unused key; | |
| - every component within 3e-5 (max abs / reference max) of upstream; | |
| - the whole stream at least 120 dB per frame, on the CPU and on the GPU. | |
| The default half-precision setting stays 49–53 dB from the fp32 lane on real footage, level with upstream's own bf16. | |
| Not yet gated against a CUDA run of upstream's Triton attention kernels; the port uses dense attention under the same | |
| masks. | |
| ## Use | |
| ``` | |
| swift build -c release --build-system swiftbuild --product relayvsr-smoke | |
| .build/out/Products/Release/relayvsr-smoke run <lrFramesDir> <outDir> <this directory> | |
| ``` | |
| ```swift | |
| let pipe = try RelayVSRPipeline.load(directory: laneDirectory) | |
| try pipe.run(frames: lrFrames) { index, frame in … } // (4H, 4W, 3) in [−1, 1], in order | |
| ``` | |
| Through [MLXEngine](https://github.com/xocialize/mlx-engine-swift) the `MLXRelayVSR` package downloads this repository | |
| on first use. | |
| Measured on an M5 Max (GPU shared with other apps): 0.075 s per 1280×768 output frame and 0.245 s at 1920×1080; | |
| process peak 14.3 GB and 16.8 GB through the engine. | |
| ## Licence | |
| Apache-2.0, as released by the RelayVSR authors; the base model Wan2.2-TI2V-5B is Apache-2.0. The training data is | |
| not disclosed by the authors. | |