--- tags: - blackhole - diffrhythm - music-generation - p100a - tenstorrent - text-to-audio - tt-dit-server - tt-metal - tt-model-cache - tt-model-catalog - tt-model-container - tt-nn - ttnn pipeline_tag: text-to-audio license: apache-2.0 license_link: https://huggingface.co/ASLP-lab/DiffRhythm-1_2-full base_model: - ASLP-lab/DiffRhythm-1_2-full --- # diffrhythm-1.2-p100a DiffRhythm 1.2 Full, ported to one Tenstorrent Blackhole p100a. Weights: [ASLP-lab/DiffRhythm-1_2-full](https://huggingface.co/ASLP-lab/DiffRhythm-1_2-full) · Upstream code: [ASLP-lab/DiffRhythm](https://github.com/ASLP-lab/DiffRhythm) · Port: [SunandBean/tt-diffrhythm](https://github.com/SunandBean/tt-diffrhythm) Runs on **p100a** (mesh 1×1). The **CFM transformer and the stable-audio VAE decoder run on the card**; the style encoder, the sampler and the post-processing stay on the CPU. **What this port adds** — the CFM transformer and the stable-audio-tools Oobleck decoder on the card, sharing the worker, transport and host helpers with the sibling [tt-acestep](https://github.com/SunandBean/tt-acestep) port. **What it builds on** — DiffRhythm's own pipeline (Apache-2.0), which runs unchanged on the CPU; only those two stages move. **A 96-second song in 10.0 s** end to end — the CFM transformer 7.7 s across 62 calls, the audio VAE decode 2.0 s. The same song takes 9.7–10.1 s on an RTX 5070 Ti. ## Listen Exactly what the container returned on run 1 of the measurement below — `POST /predict` with `{"prompt": "warm lo-fi hip hop, mellow rhodes piano, soft vinyl crackle, relaxed drums", "instrumental": true, "duration": 96, "seed": 42}`, generated in **10.05 s** on the card. The file is the response's WAV bytes, unmodified: 96.0 s stereo 44.1 kHz 16-bit, sha256 `7e89a573e6d94e4f`. [`media/diffrhythm_96s_instrumental.wav`](media/diffrhythm_96s_instrumental.wav) (16 MB) ## How it is wired Two stages move to the card and nothing else changes: `remote.py` replaces `cfm.transformer` with a `RemoteDiffRhythmDiT` and decodes through a `RemoteVae`, so the upstream `CFM.sample` loop runs unmodified. Both shims reach the card through exactly one method, `worker.call(msg)`, and there are two things that answer it. **Two processes** (`TTWorker`), when DiffRhythm and tt-metal cannot share an interpreter — which is the deployment this port came from, where DiffRhythm resolved `torch 2.10.0+cu128` and `numpy>=2` against ttnn's `torch 2.11.0+cpu` and `numpy 1.x`: ``` DiffRhythm venv (CPU) TT image (p100a) ───────────────────── ──────────────── style encoder ──socket──► worker.py CFM sampler loop DiffRhythmDiT (on the card) post-processing OobleckTT (on the card) ``` Tensors cross as raw little-endian bytes with a dtype and a shape (`wire.py`), because the two sides cannot share a pickle of a torch tensor. **One process** (`LocalWorker`), when one environment satisfies both — which is what the container below is. `LocalWorker` answers the same `call(msg)` in-process, so the shims, their window arithmetic and their drop-in signatures are reused unchanged rather than reimplemented. Messages still carry `wire`-encoded tensors in that case even though both ends are one process. That is deliberate: it is what lets the shims run against either worker, and the encoding is a float32 copy of a few MB against card work measured in seconds. ## Quickstart Prerequisite: a tt-metal / ttnn environment for the worker, and DiffRhythm installed in its own environment for the host side. ```bash hf download sunandbean/diffrhythm-1.2-p100a --include "code/*" --local-dir diffrhythm-1.2-p100a && cd diffrhythm-1.2-p100a pip install -e code/ ``` Host side, inside the DiffRhythm environment: ```python from tt_diffrhythm import TTWorker, RemoteDiffRhythmDiT, RemoteVae worker = TTWorker(cfm_model_pt, model="diffrhythm", vae=vae_model_pt).start() worker.wait_ready() cfm.transformer = RemoteDiffRhythmDiT(worker, cfm.transformer) ... # upstream CFM.sample runs unchanged audio = RemoteVae(worker, upsample=2048)(latents) worker.close() ``` Worker side, inside the tt-metal image: ```bash python -m tt_diffrhythm.worker --socket /ipc/worker.sock \ --checkpoint /checkpoint/cfm_model.pt --vae /vae/vae_model.pt python -m tt_diffrhythm.warm_cache # compile every sequence bucket ahead of time ``` | | | |---|---| | **Duration** | 96–285 seconds | | **Lyrics** | Plain lyrics with estimated line timing, or a timestamped LRC | | **Guidance** | CFG: `pred + 4 (pred − null)` — every step runs two DiT calls | | **Shapes** | Song lengths fall into sequence buckets (`diffrhythm_host.seq_buckets`), both CFG branches, plus the VAE windows. `warm_cache` compiles all of them. | ## Performance Measured on an idle p100a. A 96-second song is 2067 latent frames, padded to a sequence of 2560. | | | |---|---:| | **DiT, whole 96-second song** | **8.9 s** (7.7 s at HiFi2) | | One DiT call at sequence 2560 | 121 ms | | Per call during the device check | 0.143–0.186 s | Where a 121 ms call goes (`dit_profile.py`, 156.8 ms profiled): | Op | Time | Calls | |---|---:|---:| | matmul | 66.6 ms | 66 | | SDPA | 21.9 ms | 16 | | add | 12.0 ms | 47 | | RMS norm | 7.7 ms | 32 | | slice | 7.3 ms | 62 | | rotary embedding | 5.2 ms | 32 | The audio VAE decoder, against a CPU decode of the same latents: | Latent frames | CPU | p100a (first / warm) | |---:|---:|---:| | 512 | 10.67 s | 33.94 / **0.33** s | | 1024 | — | 0.81 s | The first decode at a shape pays the kernel compilation; afterwards it is 30× faster than the CPU. ### From the published container The same image anyone can `tt-model pull`, reached over its own HTTP server on an idle card (2026-10-09). Three 96-second instrumentals, seed 42, from a cold start: | | run 1 | run 2 | run 3 | |---|---:|---:|---:| | **Generation** | **10.05 s** | 10.06 s | 10.04 s | | CFM transformer, 62 calls | 7.68 s | 7.68 s | 7.69 s | | VAE decode | 2.03 s | 2.05 s | 2.01 s | All three returned the same 96.0 s stereo 44.1 kHz WAV. Load to ready was 50 s, of which the VAE warm-up is 3.1 s; device DRAM after load is 2.9 GB of 29.9 — the card is nowhere near full, and the port holds that headroom deliberately so the sequence buckets can grow to 285 seconds. The whole request takes about 30 s of wall time, because the response carries the song: 17 MB of WAV, base64-encoded. Generation is the 10 s above; the rest is serialising and shipping it. ### Against an RTX 5070 Ti DiffRhythm's own GPU path with the deployment's stage-by-stage placement (one model in VRAM at a time, which is what keeps it inside 16 GB), same prompt and duration, a fresh seed per run: | | p100a (container) | RTX 5070 Ti | |---|---:|---:| | 96-second instrumental | 10.04–10.06 s | **9.73–10.05 s** | | First run | 10.05 s | 12.35 s | | Peak device memory | 2.9 GB of 29.9 | 2.5 GB of 15.5 | The same, within noise. The card reaches it with the CFM transformer and the audio VAE decoder in TTNN and everything else on the CPU; the GPU reaches it by moving three models in and out of VRAM. ### Accuracy **The torch reference in this port against upstream DiffRhythm** (float32, per CFM call): PCC **0.9999997–0.9999999**. For scale, upstream float32 against the GPU fp16 run is 0.999995–0.999999 — so the reimplementation is closer to upstream than upstream is to its own GPU run. **The card against this port's reference**: minimum PCC **0.99994** across the whole 96-second song. **Final latents against the GPU fp16 run**, which is what actually matters after CFG amplification: | | Final latent PCC | Relative RMS | |---|---:|---:| | CPU float32 | 0.99997 | 0.008 | | **p100a (HiFi4, the default)** | **0.9954** | 0.096 | | p100a (HiFi2) | 0.9931 | 0.118 | | CPU bf16 | 0.99129 | 0.132 | **The card beats a CPU bf16 run.** Decoded audio against the GPU: PCC 0.9721 and 2.06 dB spectral distance, against 0.9691 and 3.07 dB for CPU bf16 (CPU float32 is 0.9996 / 0.51 dB). The audio VAE decoder itself is PCC 0.99963 with 0.93 dB spectral distance against the CPU. CFG is why the margin matters: `pred + 4 (pred − null)` multiplies the difference between the two branches by four, so a small per-call error is amplified before it reaches the audio. `DRPrecision` therefore keeps the activations, the residual and the output in fp32 and defaults to HiFi4 matmuls — HiFi2 is about 14% faster and measurably worse. ## Caveats - **Single card only.** Mesh 1×1. - **Seeds do not reproduce across backends.** A song made on the p100a cannot be reproduced bit-for-bit on a GPU with the same seed. - **Two environments, or one.** Where DiffRhythm's environment and the tt-metal runtime cannot be merged, the host side runs inside DiffRhythm's own environment and the worker inside a tt-metal image. Where they can — which the container below is — `LocalWorker` runs both in one process. Which one you get is a property of the environment, not of the port. - **CFG doubles the work.** Every sampler step is two DiT calls, one conditional and one null. - The first song of each shape pays the kernel compilation, and so does the first VAE decode at a shape (33.9 s against 0.33 s warm). Run `warm_cache` after changing the code or the image. - A render holds the card for the length of the song. On a shared card that blocks everything else. ## Licensing - Weights: [ASLP-lab/DiffRhythm-1_2-full](https://huggingface.co/ASLP-lab/DiffRhythm-1_2-full), Apache-2.0. Not redistributed here. - Port code (`code/tt_diffrhythm/`): Apache-2.0, from [SunandBean/tt-diffrhythm](https://github.com/SunandBean/tt-diffrhythm). - `code/{model,infer,g2p,config,dataset}/` are **DiffRhythm itself** (Apache-2.0), at [`28ad63c`](https://github.com/ASLP-lab/DiffRhythm/commit/28ad63c0f096fe2ee258bcabbcf081d5d9366afd), unmodified, with the upstream Apache-2.0 licence at [`code/LICENSE-DiffRhythm.md`](code/LICENSE-DiffRhythm.md) (its `g2p/` carries Amphion's MIT headers in the files themselves). They ship here because the container runs them: upstream is a research checkout with no packaging, so it is vendored as source rather than resolved by pip. `code/models/` is [tt-metal](https://github.com/tenstorrent/tt-metal)'s `models/common` and `models/tt_dit` (Apache-2.0), for the same reason. - `code/tt_diffrhythm/common.py` and the worker, transport and audio VAE modules are shared with the sibling [tt-acestep](https://github.com/SunandBean/tt-acestep) port (Apache-2.0); `oobleck_vae.py` implements both DiffRhythm's stable-audio-tools decoder and ACE-Step's diffusers Oobleck decoder. - `code/THIRD-PARTY.md` lists every directory under `code/` that came from somewhere else, with its upstream commit and where its licence text sits in this repo. ## Run it as a container Everything below is generated by `tt-model package` from the manifest this image was built from. ```bash tt-model pull sunandbean/diffrhythm-1.2-p100a --with-weights tt-model serve sunandbean/diffrhythm-1.2-p100a ``` `pull --with-weights` downloads the Docker image and the [`ASLP-lab/DiffRhythm-1_2-full`](https://huggingface.co/ASLP-lab/DiffRhythm-1_2-full) weights (into your HF cache; they are not in the image). `serve` starts the model's own HTTP server on port 20000 (or the next free port, if that one is busy); the first start compiles kernels for your device, which takes several minutes, and the server is ready when it logs `Application startup complete`. ### Request shape `POST /predict` takes `{"prompt": str, "lyrics": str, "instrumental": bool, "duration": int, "seed": int}` and returns a base64 WAV with `timing_s` and the `lrc` the song was actually sung to. `prompt` is the style description. Vocals need `lyrics`: real LRC is used as given, and plain lines get approximate starts derived from their word counts -- a convenience, not learned alignment. `duration` is 96-285 seconds; use the ACE-Step port for longer songs. The schedule is fixed at **32 steps**; any other `inference_steps` is a 422. ### Provenance The exact sources the image was built from — the model code under `code/` is byte-identical to what is inside the image (`pyproject.toml`, `PYTHON.md` and `examples/` are packaging files this repo adds, and are not in the image): | component | built from | | --- | --- | | tt-metal | [`58648d44443223041148125843ca7c7d4780463e`](https://github.com/tenstorrent/tt-metal/commit/58648d44443223041148125843ca7c7d4780463e) | | `code/` digest | `322cdea4755e9a81` (sha256, first 16 hex digits) | | built | 2026-10-10T03:02:08+00:00 by tt-model 0.1.0 |