Commit History
-
sm_120 SAME-L: AOT (graph-capturable) engines + tensorRT/README.md
Verified signatureReplaces the two `sm_120` SAME-L engines with AOT builds, and fills in the empty `tensorRT/README.md`. ## Why the engines change The `sm_120` SAME-L engines published here are JIT builds, and a JIT-built SWA plugin engine is **not stream-capturable on sm_120**. Verified against the file currently in this repo: ``` [1] capture with NO prior warmup enqueue -> False [3] capture AFTER warmup enqueue -> False "this TRT engine is not stream capturable" / "The CUDA Graph is empty" ``` `sa3_trt.py` captures the decoder inside the mega-graph (T5 -> DiT loop -> decoder -> PCM), which is the default path. So the decode is silently dropped from the graph, `enqueueV3` returns False, nothing raises, and the decoder returns the pre-capture warm-up decode of zero latents: a constant wash of noise, byte-identical for every prompt and seed, exit code 0. The replacements embed an ahead-of-time block-tiled tensor-core kernel, so no Python runs during inference and the engine captures.…1d21398
-
Upload onnx/sa3-m/dit_fp8lin.onnx.data with huggingface_hub
Verified signature9efa545 -
Upload onnx/sa3-m/dit_fp8.onnx with huggingface_hub
Verified signature21b6506 -
Upload tensorRT/sm_90/sa3-m/dit_fp8.trt with huggingface_hub
Verified signaturefeddb8b -
Fix the bf16 medium DiT: bake RoPE's tables into the graph (clipping 3.112% -> 0.014%) (#5)
- Fix the bf16 medium DiT: bake RoPE's tables into the graph (clipping 3.112% -> 0.014%) (59bd967d4635c6e32b0b2a677d56a21984654b9d)6152b1d -
Rebuild medium DiT fp16-mixed engines (sm_90 + new sm_120): attention now fuses, 4.3x faster (#4)
- Rebuild medium DiT fp16-mixed engines (sm_90 + new sm_120): attention now fuses, 4.3x faster (f3263ec28281132dda00915ad3a83920f3b98730) - Complete the sm_120 engine set (mirror sm_90's 15 files) (dcca4a7a8676aa3846ca92ecf7607d572706d685)06debc3 -
Fix medium DiT FP16-mixed ONNX: bound the RoPE island so attention fuses (4.3x faster, cos 1.0000) (#3)
- Fix medium DiT FP16-mixed ONNX: bound the RoPE island so attention fuses (4.3x faster, cos 1.0000) (b4300a99fc3b12b6c59e2d10218399f672087340)fb17708
-
Add MLX base-model DiT weights for LoRA training (#2)
- Add MLX base-model DiT weights for LoRA training (be6b0350d3677f5981e36fb9c4a1dd2c75307dd9) Co-authored-by: CJ Carr <cortexelus@users.noreply.huggingface.co>08c64b9 -
Add bf16 medium DiT TRT engine (sm_90) — FMHA-fused, medium default (#1)
- Add bf16 medium DiT TRT engine (sm_90) — FMHA-fused, medium default (c8cb8ba88ddea67804bcacbbb9265afcbff03b6f) Co-authored-by: CJ Carr <cortexelus@users.noreply.huggingface.co>9a38efb
-
tflite: replace naive same-l/enc_w8a8-dyn.tflite with GPTQ (torch-free, real-audio calibrated)
Verified signaturec2949a4 -
tflite: replace naive same-l/enc_w8a32.tflite with GPTQ (torch-free, real-audio calibrated)
Verified signatureaea1c33 -
tflite: replace naive same-s/enc_w8a8-dyn.tflite with GPTQ (torch-free, real-audio calibrated)
Verified signature6401d6c -
tflite: replace naive same-s/enc_w8a32.tflite with GPTQ (torch-free, real-audio calibrated)
Verified signature07885df
-
tflite: add same-l/enc_w8a8-dyn.tflite (quantized encoder, varlen)
Verified signaturea7970b2 -
tflite: add same-l/enc_w8a32.tflite (quantized encoder, varlen)
Verified signature7de7f11 -
tflite: add same-l/enc_w16a32.tflite (quantized encoder, varlen)
Verified signature4adfa45 -
tflite: add same-s/enc_w8a8-dyn.tflite (quantized encoder, varlen)
Verified signature151e748 -
tflite: add same-s/enc_w8a32.tflite (quantized encoder, varlen)
Verified signature2682028 -
tflite: add same-s/enc_w16a32.tflite (quantized encoder, varlen)
Verified signatured35a0c9 -
tflite: add sa3-m/dit_w16a32.tflite (quantized, varlen+varbatch)
Verified signature4ed3663 -
tflite: add sa3-m/dit_w8a32.tflite (quantized, varlen+varbatch)
Verified signature3e6eee6 -
tflite: add sa3-m/dit_w8a8-dyn.tflite (quantized, varlen+varbatch)
Verified signature78049a1 -
tflite: add same-l/dec_w16a32.tflite (quantized, varlen+varbatch)
Verified signaturef4cd71b -
tflite: add sa3-sm-sfx/dit_w16a32.tflite (quantized, varlen+varbatch)
Verified signature860ab42 -
tflite: add sa3-sm-music/dit_w16a32.tflite (quantized, varlen+varbatch)
Verified signaturee6e2e82 -
tflite: add same-l/dec_w8a32.tflite (quantized, varlen+varbatch)
Verified signaturecd3b3c2 -
tflite: add same-l/dec_w8a8-dyn.tflite (quantized, varlen+varbatch)
Verified signature65565ac -
tflite: add sa3-sm-sfx/dit_w8a32.tflite (quantized, varlen+varbatch)
Verified signatureb70b82a -
tflite: add sa3-sm-music/dit_w8a32.tflite (quantized, varlen+varbatch)
Verified signature62266ef -
tflite: add sa3-sm-sfx/dit_w8a8-dyn.tflite (quantized, varlen+varbatch)
Verified signature33af4b1 -
tflite: add sa3-sm-music/dit_w8a8-dyn.tflite (quantized, varlen+varbatch)
Verified signature20c6b1a -
tflite: add same-s/dec_w16a32.tflite (quantized, varlen+varbatch)
Verified signature142d6be -
tflite: add same-s/dec_w8a32.tflite (quantized, varlen+varbatch)
Verified signatureb9b2aed -
tflite: add same-s/dec_w8a8-dyn.tflite (quantized, varlen+varbatch)
Verified signature2bcd53f
-
varbatch DiT: dynamic batch axis (bit-exact at batch=1; enables batched CFG at batch=2)
Verified signature45a175a -
varbatch DiT: dynamic batch axis (bit-exact at batch=1; enables batched CFG at batch=2)
Verified signature07c665a -
varbatch DiT: dynamic batch axis (bit-exact at batch=1; enables batched CFG at batch=2)
Verified signaturee1bc60b
-
Upload tflite/README.md with huggingface_hub
Verified signature2b486c6 -
Upload tflite/t5gemma/encoder_fp16.tflite with huggingface_hub
Verified signatured4300ff -
Upload tflite/same-l/dec_fp32.tflite with huggingface_hub
Verified signature45796ea -
Upload tflite/same-l/enc_fp32.tflite with huggingface_hub
Verified signatured6b119e -
Upload tflite/same-s/dec_fp32.tflite with huggingface_hub
Verified signature65c576d -
Upload tflite/same-s/enc_fp32.tflite with huggingface_hub
Verified signature216858b -
Upload tflite/sa3-sm-sfx/dit_fp32.tflite with huggingface_hub
Verified signaturef36220a -
Upload tflite/sa3-m/dit_fp32.tflite with huggingface_hub
Verified signature58efbcd -
Upload tflite/sa3-sm-music/dit_fp32.tflite with huggingface_hub
Verified signaturedbb5aca
-
Fix missing 2π factor in seconds_freqs across all dit.onnx (FP32 source ONNX)
Cortexelus +1: Claude Opus 4.7 (1M context)The base FP32 dit.onnx files were exported by an older version of the producer (fast_load/build_dit.py) before the 2π fix was applied. The post-surgery dit_fp16mixed.onnx files DID get the fix re-injected (their seconds_freqs has max ≈ 62832 = 2π × 10000), but the source dit.onnx files were never re-uploaded and still have max=10000. Effect: any consumer building from dit.onnx (e.g., `--precision fp32` path that reads sa3-m/dit.onnx, or anyone re-running build_dit_fp16mixed locally) would inherit the silence-after-~5s bug — the seconds_total conditioner produces frequencies 2π× too low, so the Fourier features collapse and the model can't condition on duration beyond ~5s. Surgical fix: read the (128,) seconds_freqs initializer in each dit.onnx, multiply by 2π, write back. For sa3-m the buffer is in the .data sidecar at offset 3072+512 bytes — read it from there, multiply, and inline the result into the .onnx so the 5.8 GB .data file doesn't need to be re-uploaded. For sa3-sm-music/sf…ecc70ac -
Fix sa3-m DiT ONNX external_data ref (was pointing at wrong filename)
Cortexelus +1: Claude Opus 4.7 (1M context)The medium DiT FP16-mixed ONNX had 428 initializers whose external_data location field pointed at "dit_m_fp16mixed.onnx.data", but the actual sidecar on disk (and in this repo) is "dit_fp16mixed.onnx.data" — no "_m_" infix. The mismatch came from the producer naming convention at build time; the rename to the shipped name only touched the file, not the in-ONNX reference. Consequence: any consumer trying to compile a TRT engine from this ONNX (e.g., on a new GPU arch like sm_120 via build_from_onnx.py sa3-m) hit: [TRT] [E] WeightsContext.cpp:190: Failed to open file: ... /onnx/sa3-m/dit_m_fp16mixed.onnx.data [TRT] [E] Failed to import initializer: padding_embedding … and ended up either with no engine or a partial/garbage engine, manifesting as "medium DiT outputs ~5s of music then dead silence" on some setups. The pre-built sm_90 TRT engine in this repo is unaffected (it was built from the producer's ONNX at the moment the data file still had the original name). …3dbdcfa -
Add FP32 SAME-L + SAME-S TRT decoders
Cortexelus +1: Claude Opus 4.7 (1M context)Pair the existing FP32 DiT engines (sa3-{m,sm-music,sm-sfx}/dit_fp32.trt) with FP32 decoders so the consumer-side `--precision fp32` flag has a full FP32-throughout pipeline available, bit-equivalent to PyTorch eager. - same-l/dec_dynamic_fp32.trt — built by upcasting dec_dynamic_triton_swa.onnx initializers + value_info + Constant nodes + Cast.to from FP16 to FP32 (STRONGLY_TYPED). The Triton SWA plugin already runs on the FP32 side of the attention island, so the plugin contract is unchanged. ~1.7 GB. - same-s/dec_dynamic_fp32.trt — analogous FP32 engine for SAME-S. ~210 MB. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>773ac48 -
Decoder ONNX: clip+scale in FP32 before int32 cast
Cortexelus +1: Claude Opus 4.7 (1M context)The PCM-baked tail (audio → clamp → ×32767 → int32) ran in the engine's FP16/BF16 trunk, where 32767 is not representable (rounds up to 32768), causing the int32 output to exceed the int16 range. The consumer code then wrapped 32768 → -32768 on int16 downcast, producing audible clicks on loud transients. Fix: cast audio to FP32 before the clip/scale chain in both build_same_l_decoder.py and build_same_s_decoder.py producers. 32767 is exact in FP32. A trailing clamp(-32767, 32767) is added as belt-and-suspenders against any residual numerical drift. Affected: dec_dynamic_triton_swa.{onnx,trt} (SAME-L), dec_dynamic_bf16.{onnx,trt} (SAME-S). Verified: 0 samples outside [-32767, +32767] on real-music decode (was ~800 with the old engines). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>a429ffa