Instructions to use stabilityai/stable-audio-3-optimized with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Stable Audio 3
How to use stabilityai/stable-audio-3-optimized with Stable Audio 3:
# No code snippets available yet for this library. # To use this model, check the repository files and the library's documentation. # Want to help? PRs adding snippets are welcome at: # https://github.com/huggingface/huggingface.js
- Notebooks
- Google Colab
- Kaggle
tensorRT: document the runtime LoRA / DoRA engine
Browse files- tensorRT/README.md +10 -0
tensorRT/README.md
CHANGED
|
@@ -172,6 +172,16 @@ comparison — earlier builds of it degraded past about two minutes of audio bec
|
|
| 172 |
RoPE angle was computed in BF16, where one ULP exceeds 2π at long sequence lengths; the
|
| 173 |
published engine has the RoPE table baked in at FP32 precision.
|
| 174 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 175 |
**`dit_fp8.trt` is sm_90 only** and must be built weakly-typed; strong typing breaks
|
| 176 |
TensorRT's attention fusion. This is the *opposite* of the FP16-mixed rule, which requires
|
| 177 |
`STRONGLY_TYPED` or the builder re-casts the FP32 islands.
|
|
|
|
| 172 |
RoPE angle was computed in BF16, where one ULP exceeds 2π at long sequence lengths; the
|
| 173 |
published engine has the RoPE table baked in at FP32 precision.
|
| 174 |
|
| 175 |
+
**`dit_fp16_lora.trt` serves LoRA / DoRA adapters at runtime.** Same DiT, plus a low-rank
|
| 176 |
+
branch on each of 229 linears whose operands are *network inputs* rather than weights — so an
|
| 177 |
+
adapter loads in milliseconds, several stack, and each keeps its own strength, with no rebuild.
|
| 178 |
+
It costs a flat ~4–6 ms per step (+13% at L=4096, +37% at 1292) and you pay that with an empty
|
| 179 |
+
stack too, so use `dit_fp16.trt` until an adapter is actually wanted. **sm_90 only** for now;
|
| 180 |
+
other architectures build it locally in ~8 min with
|
| 181 |
+
`scripts/lora/build_branch.py --model sa3-m --download`. The runtime, the adapter baking step
|
| 182 |
+
and the layer map all live in the stable-audio-3 repo under
|
| 183 |
+
`optimized/tensorRT/scripts/lora/` — this file is only the compiled plan.
|
| 184 |
+
|
| 185 |
**`dit_fp8.trt` is sm_90 only** and must be built weakly-typed; strong typing breaks
|
| 186 |
TensorRT's attention fusion. This is the *opposite* of the FP16-mixed rule, which requires
|
| 187 |
`STRONGLY_TYPED` or the builder re-casts the FP32 islands.
|