README.md

---
tags:
- blackhole
- diffrhythm
- music-generation
- p100a
- tenstorrent
- text-to-audio
- tt-dit-server
- tt-metal
- tt-model-cache
- tt-model-catalog
- tt-model-container
- tt-nn
- ttnn
pipeline_tag: text-to-audio
license: apache-2.0
license_link: https://huggingface.co/ASLP-lab/DiffRhythm-1_2-full
base_model:
- ASLP-lab/DiffRhythm-1_2-full
---

# diffrhythm-1.2-p100a

DiffRhythm 1.2 Full, ported to one Tenstorrent Blackhole p100a.
Weights: [ASLP-lab/DiffRhythm-1_2-full](https://huggingface.co/ASLP-lab/DiffRhythm-1_2-full) · Upstream code: [ASLP-lab/DiffRhythm](https://github.com/ASLP-lab/DiffRhythm) · Port: [SunandBean/tt-diffrhythm](https://github.com/SunandBean/tt-diffrhythm)

Runs on **p100a** (mesh 1×1). The **CFM transformer and the stable-audio VAE decoder run on the card**; the style encoder, the sampler and the post-processing stay on the CPU.

**What this port adds** — the CFM transformer and the stable-audio-tools Oobleck decoder on the card, sharing the worker, transport and host helpers with the sibling [tt-acestep](https://github.com/SunandBean/tt-acestep) port.
**What it builds on** — DiffRhythm's own pipeline (Apache-2.0), which runs unchanged on the CPU; only those two stages move.

**A 96-second song in 10.0 s** end to end — the CFM transformer 7.7 s across 62 calls, the audio VAE decode 2.0 s. The same song takes 9.7–10.1 s on an RTX 5070 Ti.

## Listen

Exactly what the container returned on run 1 of the measurement below — `POST /predict` with
`{"prompt": "warm lo-fi hip hop, mellow rhodes piano, soft vinyl crackle, relaxed drums",
"instrumental": true, "duration": 96, "seed": 42}`, generated in **10.05 s** on the card. The file is
the response's WAV bytes, unmodified: 96.0 s stereo 44.1 kHz 16-bit, sha256 `7e89a573e6d94e4f`.

<audio controls src="https://huggingface.co/sunandbean/diffrhythm-1.2-p100a/resolve/main/media/diffrhythm_96s_instrumental.wav"></audio>

[`media/diffrhythm_96s_instrumental.wav`](media/diffrhythm_96s_instrumental.wav) (16 MB)

## How it is wired

Two stages move to the card and nothing else changes: `remote.py` replaces `cfm.transformer` with a `RemoteDiffRhythmDiT` and decodes through a `RemoteVae`, so the upstream `CFM.sample` loop runs unmodified.

Both shims reach the card through exactly one method, `worker.call(msg)`, and there are two things that answer it.

**Two processes** (`TTWorker`), when DiffRhythm and tt-metal cannot share an interpreter — which is the deployment this port came from, where DiffRhythm resolved `torch 2.10.0+cu128` and `numpy>=2` against ttnn's `torch 2.11.0+cpu` and `numpy 1.x`:

```
  DiffRhythm venv (CPU)                    TT image (p100a)
  ─────────────────────                    ────────────────
  style encoder                ──socket──► worker.py
  CFM sampler loop                         DiffRhythmDiT  (on the card)
  post-processing                          OobleckTT      (on the card)
```

Tensors cross as raw little-endian bytes with a dtype and a shape (`wire.py`), because the two sides cannot share a pickle of a torch tensor.

**One process** (`LocalWorker`), when one environment satisfies both — which is what the container below is. `LocalWorker` answers the same `call(msg)` in-process, so the shims, their window arithmetic and their drop-in signatures are reused unchanged rather than reimplemented.

Messages still carry `wire`-encoded tensors in that case even though both ends are one process. That is deliberate: it is what lets the shims run against either worker, and the encoding is a float32 copy of a few MB against card work measured in seconds.

## Quickstart

Prerequisite: a tt-metal / ttnn environment for the worker, and DiffRhythm installed in its own environment for the host side.

```bash
hf download sunandbean/diffrhythm-1.2-p100a --include "code/*" --local-dir diffrhythm-1.2-p100a && cd diffrhythm-1.2-p100a
pip install -e code/
```

Host side, inside the DiffRhythm environment:

```python
from tt_diffrhythm import TTWorker, RemoteDiffRhythmDiT, RemoteVae

worker = TTWorker(cfm_model_pt, model="diffrhythm", vae=vae_model_pt).start()
worker.wait_ready()
cfm.transformer = RemoteDiffRhythmDiT(worker, cfm.transformer)
...                                          # upstream CFM.sample runs unchanged
audio = RemoteVae(worker, upsample=2048)(latents)
worker.close()
```

Worker side, inside the tt-metal image:

```bash
python -m tt_diffrhythm.worker --socket /ipc/worker.sock \
  --checkpoint /checkpoint/cfm_model.pt --vae /vae/vae_model.pt
python -m tt_diffrhythm.warm_cache     # compile every sequence bucket ahead of time
```

| | |
|---|---|
| **Duration** | 96–285 seconds |
| **Lyrics** | Plain lyrics with estimated line timing, or a timestamped LRC |
| **Guidance** | CFG: `pred + 4 (pred − null)` — every step runs two DiT calls |
| **Shapes** | Song lengths fall into sequence buckets (`diffrhythm_host.seq_buckets`), both CFG branches, plus the VAE windows. `warm_cache` compiles all of them. |

## Performance

Measured on an idle p100a. A 96-second song is 2067 latent frames, padded to a sequence of 2560.

| | |
|---|---:|
| **DiT, whole 96-second song** | **8.9 s** (7.7 s at HiFi2) |
| One DiT call at sequence 2560 | 121 ms |
| Per call during the device check | 0.143–0.186 s |

Where a 121 ms call goes (`dit_profile.py`, 156.8 ms profiled):

| Op | Time | Calls |
|---|---:|---:|
| matmul | 66.6 ms | 66 |
| SDPA | 21.9 ms | 16 |
| add | 12.0 ms | 47 |
| RMS norm | 7.7 ms | 32 |
| slice | 7.3 ms | 62 |
| rotary embedding | 5.2 ms | 32 |

The audio VAE decoder, against a CPU decode of the same latents:

| Latent frames | CPU | p100a (first / warm) |
|---:|---:|---:|
| 512 | 10.67 s | 33.94 / **0.33** s |
| 1024 | — | 0.81 s |

The first decode at a shape pays the kernel compilation; afterwards it is 30× faster than the CPU.

### From the published container

The same image anyone can `tt-model pull`, reached over its own HTTP server on an idle card
(2026-10-09). Three 96-second instrumentals, seed 42, from a cold start:

| | run 1 | run 2 | run 3 |
|---|---:|---:|---:|
| **Generation** | **10.05 s** | 10.06 s | 10.04 s |
| CFM transformer, 62 calls | 7.68 s | 7.68 s | 7.69 s |
| VAE decode | 2.03 s | 2.05 s | 2.01 s |

All three returned the same 96.0 s stereo 44.1 kHz WAV. Load to ready was 50 s, of which the VAE
warm-up is 3.1 s; device DRAM after load is 2.9 GB of 29.9 — the card is nowhere near full, and
the port holds that headroom deliberately so the sequence buckets can grow to 285 seconds.

The whole request takes about 30 s of wall time, because the response carries the song: 17 MB of
WAV, base64-encoded. Generation is the 10 s above; the rest is serialising and shipping it.

### Against an RTX 5070 Ti

DiffRhythm's own GPU path with the deployment's stage-by-stage placement (one model in VRAM at a
time, which is what keeps it inside 16 GB), same prompt and duration, a fresh seed per run:

| | p100a (container) | RTX 5070 Ti |
|---|---:|---:|
| 96-second instrumental | 10.04–10.06 s | **9.73–10.05 s** |
| First run | 10.05 s | 12.35 s |
| Peak device memory | 2.9 GB of 29.9 | 2.5 GB of 15.5 |

The same, within noise. The card reaches it with the CFM transformer and the audio VAE decoder in
TTNN and everything else on the CPU; the GPU reaches it by moving three models in and out of VRAM.

### Accuracy

**The torch reference in this port against upstream DiffRhythm** (float32, per CFM call): PCC **0.9999997–0.9999999**. For scale, upstream float32 against the GPU fp16 run is 0.999995–0.999999 — so the reimplementation is closer to upstream than upstream is to its own GPU run.

**The card against this port's reference**: minimum PCC **0.99994** across the whole 96-second song.

**Final latents against the GPU fp16 run**, which is what actually matters after CFG amplification:

| | Final latent PCC | Relative RMS |
|---|---:|---:|
| CPU float32 | 0.99997 | 0.008 |
| **p100a (HiFi4, the default)** | **0.9954** | 0.096 |
| p100a (HiFi2) | 0.9931 | 0.118 |
| CPU bf16 | 0.99129 | 0.132 |

**The card beats a CPU bf16 run.** Decoded audio against the GPU: PCC 0.9721 and 2.06 dB spectral distance, against 0.9691 and 3.07 dB for CPU bf16 (CPU float32 is 0.9996 / 0.51 dB). The audio VAE decoder itself is PCC 0.99963 with 0.93 dB spectral distance against the CPU.

CFG is why the margin matters: `pred + 4 (pred − null)` multiplies the difference between the two branches by four, so a small per-call error is amplified before it reaches the audio. `DRPrecision` therefore keeps the activations, the residual and the output in fp32 and defaults to HiFi4 matmuls — HiFi2 is about 14% faster and measurably worse.

## Caveats

- **Single card only.** Mesh 1×1.
- **Seeds do not reproduce across backends.** A song made on the p100a cannot be reproduced bit-for-bit on a GPU with the same seed.
- **Two environments, or one.** Where DiffRhythm's environment and the tt-metal runtime cannot be merged, the host side runs inside DiffRhythm's own environment and the worker inside a tt-metal image. Where they can — which the container below is — `LocalWorker` runs both in one process. Which one you get is a property of the environment, not of the port.
- **CFG doubles the work.** Every sampler step is two DiT calls, one conditional and one null.
- The first song of each shape pays the kernel compilation, and so does the first VAE decode at a shape (33.9 s against 0.33 s warm). Run `warm_cache` after changing the code or the image.
- A render holds the card for the length of the song. On a shared card that blocks everything else.

## Licensing

- Weights: [ASLP-lab/DiffRhythm-1_2-full](https://huggingface.co/ASLP-lab/DiffRhythm-1_2-full), Apache-2.0. Not redistributed here.
- Port code (`code/tt_diffrhythm/`): Apache-2.0, from [SunandBean/tt-diffrhythm](https://github.com/SunandBean/tt-diffrhythm).
- `code/{model,infer,g2p,config,dataset}/` are **DiffRhythm itself** (Apache-2.0), at [`28ad63c`](https://github.com/ASLP-lab/DiffRhythm/commit/28ad63c0f096fe2ee258bcabbcf081d5d9366afd), unmodified, with the upstream Apache-2.0 licence at [`code/LICENSE-DiffRhythm.md`](code/LICENSE-DiffRhythm.md) (its `g2p/` carries Amphion's MIT headers in the files themselves). They ship here because the container runs them: upstream is a research checkout with no packaging, so it is vendored as source rather than resolved by pip. `code/models/` is [tt-metal](https://github.com/tenstorrent/tt-metal)'s `models/common` and `models/tt_dit` (Apache-2.0), for the same reason.
- `code/tt_diffrhythm/common.py` and the worker, transport and audio VAE modules are shared with the sibling [tt-acestep](https://github.com/SunandBean/tt-acestep) port (Apache-2.0); `oobleck_vae.py` implements both DiffRhythm's stable-audio-tools decoder and ACE-Step's diffusers Oobleck decoder.
- `code/THIRD-PARTY.md` lists every directory under `code/` that came from somewhere else, with its upstream commit and where its licence text sits in this repo.

## Run it as a container

Everything below is generated by `tt-model package` from the manifest this image was built from.

```bash
tt-model pull  sunandbean/diffrhythm-1.2-p100a --with-weights
tt-model serve sunandbean/diffrhythm-1.2-p100a
```

`pull --with-weights` downloads the Docker image and the [`ASLP-lab/DiffRhythm-1_2-full`](https://huggingface.co/ASLP-lab/DiffRhythm-1_2-full) weights (into your HF cache; they are not in the image). `serve` starts the model's own HTTP server on port 20000 (or the next free port, if that one is busy); the first start compiles kernels for your device, which takes several minutes, and the server is ready when it logs `Application startup complete`.

### Request shape

`POST /predict` takes `{"prompt": str, "lyrics": str, "instrumental": bool,
"duration": int, "seed": int}` and returns a base64 WAV with `timing_s` and the `lrc` the
song was actually sung to.

`prompt` is the style description. Vocals need `lyrics`: real LRC is used as given, and
plain lines get approximate starts derived from their word counts -- a convenience, not
learned alignment. `duration` is 96-285 seconds; use the ACE-Step port for longer songs.
The schedule is fixed at **32 steps**; any other `inference_steps` is a 422.

### Provenance

The exact sources the image was built from — the model code under `code/` is byte-identical to what is inside the image (`pyproject.toml`, `PYTHON.md` and `examples/` are packaging files this repo adds, and are not in the image):

| component | built from |
| --- | --- |
| tt-metal | [`58648d44443223041148125843ca7c7d4780463e`](https://github.com/tenstorrent/tt-metal/commit/58648d44443223041148125843ca7c7d4780463e) |
| `code/` digest | `322cdea4755e9a81` (sha256, first 16 hex digits) |
| built | 2026-10-10T03:02:08+00:00 by tt-model 0.1.0 |