Commit History
-
Add MLX base-model DiT weights for LoRA trainingβ
Verified signaturedit_{sm-music,sm-sfx,medium}-base_f16.npz β the *-base (rectified_flow) DiT + baked conditioner in MLX layout, for LoRA training via optimized/mlx lora_train_mlx.py (registered in weights.TRAINING_BASE for lazy download).be6b035
-
tflite: replace naive same-l/enc_w8a8-dyn.tflite with GPTQ (torch-free, real-audio calibrated)
Verified signaturec2949a4 -
tflite: replace naive same-l/enc_w8a32.tflite with GPTQ (torch-free, real-audio calibrated)
Verified signatureaea1c33 -
tflite: replace naive same-s/enc_w8a8-dyn.tflite with GPTQ (torch-free, real-audio calibrated)
Verified signature6401d6c -
tflite: replace naive same-s/enc_w8a32.tflite with GPTQ (torch-free, real-audio calibrated)
Verified signature07885df
-
tflite: add same-l/enc_w8a8-dyn.tflite (quantized encoder, varlen)
Verified signaturea7970b2 -
tflite: add same-l/enc_w8a32.tflite (quantized encoder, varlen)
Verified signature7de7f11 -
tflite: add same-l/enc_w16a32.tflite (quantized encoder, varlen)
Verified signature4adfa45 -
tflite: add same-s/enc_w8a8-dyn.tflite (quantized encoder, varlen)
Verified signature151e748 -
tflite: add same-s/enc_w8a32.tflite (quantized encoder, varlen)
Verified signature2682028 -
tflite: add same-s/enc_w16a32.tflite (quantized encoder, varlen)
Verified signatured35a0c9 -
tflite: add sa3-m/dit_w16a32.tflite (quantized, varlen+varbatch)
Verified signature4ed3663 -
tflite: add sa3-m/dit_w8a32.tflite (quantized, varlen+varbatch)
Verified signature3e6eee6 -
tflite: add sa3-m/dit_w8a8-dyn.tflite (quantized, varlen+varbatch)
Verified signature78049a1 -
tflite: add same-l/dec_w16a32.tflite (quantized, varlen+varbatch)
Verified signaturef4cd71b -
tflite: add sa3-sm-sfx/dit_w16a32.tflite (quantized, varlen+varbatch)
Verified signature860ab42 -
tflite: add sa3-sm-music/dit_w16a32.tflite (quantized, varlen+varbatch)
Verified signaturee6e2e82 -
tflite: add same-l/dec_w8a32.tflite (quantized, varlen+varbatch)
Verified signaturecd3b3c2 -
tflite: add same-l/dec_w8a8-dyn.tflite (quantized, varlen+varbatch)
Verified signature65565ac -
tflite: add sa3-sm-sfx/dit_w8a32.tflite (quantized, varlen+varbatch)
Verified signatureb70b82a -
tflite: add sa3-sm-music/dit_w8a32.tflite (quantized, varlen+varbatch)
Verified signature62266ef -
tflite: add sa3-sm-sfx/dit_w8a8-dyn.tflite (quantized, varlen+varbatch)
Verified signature33af4b1 -
tflite: add sa3-sm-music/dit_w8a8-dyn.tflite (quantized, varlen+varbatch)
Verified signature20c6b1a -
tflite: add same-s/dec_w16a32.tflite (quantized, varlen+varbatch)
Verified signature142d6be -
tflite: add same-s/dec_w8a32.tflite (quantized, varlen+varbatch)
Verified signatureb9b2aed -
tflite: add same-s/dec_w8a8-dyn.tflite (quantized, varlen+varbatch)
Verified signature2bcd53f
-
varbatch DiT: dynamic batch axis (bit-exact at batch=1; enables batched CFG at batch=2)
Verified signature45a175a -
varbatch DiT: dynamic batch axis (bit-exact at batch=1; enables batched CFG at batch=2)
Verified signature07c665a -
varbatch DiT: dynamic batch axis (bit-exact at batch=1; enables batched CFG at batch=2)
Verified signaturee1bc60b
-
Upload tflite/README.md with huggingface_hub
Verified signature2b486c6 -
Upload tflite/t5gemma/encoder_fp16.tflite with huggingface_hub
Verified signatured4300ff -
Upload tflite/same-l/dec_fp32.tflite with huggingface_hub
Verified signature45796ea -
Upload tflite/same-l/enc_fp32.tflite with huggingface_hub
Verified signatured6b119e -
Upload tflite/same-s/dec_fp32.tflite with huggingface_hub
Verified signature65c576d -
Upload tflite/same-s/enc_fp32.tflite with huggingface_hub
Verified signature216858b -
Upload tflite/sa3-sm-sfx/dit_fp32.tflite with huggingface_hub
Verified signaturef36220a -
Upload tflite/sa3-m/dit_fp32.tflite with huggingface_hub
Verified signature58efbcd -
Upload tflite/sa3-sm-music/dit_fp32.tflite with huggingface_hub
Verified signaturedbb5aca
-
Fix missing 2Ο factor in seconds_freqs across all dit.onnx (FP32 source ONNX)β
Cortexelus +1: Claude Opus 4.7 (1M context)The base FP32 dit.onnx files were exported by an older version of the producer (fast_load/build_dit.py) before the 2Ο fix was applied. The post-surgery dit_fp16mixed.onnx files DID get the fix re-injected (their seconds_freqs has max β 62832 = 2Ο Γ 10000), but the source dit.onnx files were never re-uploaded and still have max=10000. Effect: any consumer building from dit.onnx (e.g., `--precision fp32` path that reads sa3-m/dit.onnx, or anyone re-running build_dit_fp16mixed locally) would inherit the silence-after-~5s bug β the seconds_total conditioner produces frequencies 2ΟΓ too low, so the Fourier features collapse and the model can't condition on duration beyond ~5s. Surgical fix: read the (128,) seconds_freqs initializer in each dit.onnx, multiply by 2Ο, write back. For sa3-m the buffer is in the .data sidecar at offset 3072+512 bytes β read it from there, multiply, and inline the result into the .onnx so the 5.8 GB .data file doesn't need to be re-uploaded. For sa3-sm-music/sfβ¦ecc70ac -
Fix sa3-m DiT ONNX external_data ref (was pointing at wrong filename)β
Cortexelus +1: Claude Opus 4.7 (1M context)The medium DiT FP16-mixed ONNX had 428 initializers whose external_data location field pointed at "dit_m_fp16mixed.onnx.data", but the actual sidecar on disk (and in this repo) is "dit_fp16mixed.onnx.data" β no "_m_" infix. The mismatch came from the producer naming convention at build time; the rename to the shipped name only touched the file, not the in-ONNX reference. Consequence: any consumer trying to compile a TRT engine from this ONNX (e.g., on a new GPU arch like sm_120 via build_from_onnx.py sa3-m) hit: [TRT] [E] WeightsContext.cpp:190: Failed to open file: ... /onnx/sa3-m/dit_m_fp16mixed.onnx.data [TRT] [E] Failed to import initializer: padding_embedding β¦ and ended up either with no engine or a partial/garbage engine, manifesting as "medium DiT outputs ~5s of music then dead silence" on some setups. The pre-built sm_90 TRT engine in this repo is unaffected (it was built from the producer's ONNX at the moment the data file still had the original name). β¦3dbdcfa -
Add FP32 SAME-L + SAME-S TRT decodersβ
Cortexelus +1: Claude Opus 4.7 (1M context)Pair the existing FP32 DiT engines (sa3-{m,sm-music,sm-sfx}/dit_fp32.trt) with FP32 decoders so the consumer-side `--precision fp32` flag has a full FP32-throughout pipeline available, bit-equivalent to PyTorch eager. - same-l/dec_dynamic_fp32.trt β built by upcasting dec_dynamic_triton_swa.onnx initializers + value_info + Constant nodes + Cast.to from FP16 to FP32 (STRONGLY_TYPED). The Triton SWA plugin already runs on the FP32 side of the attention island, so the plugin contract is unchanged. ~1.7 GB. - same-s/dec_dynamic_fp32.trt β analogous FP32 engine for SAME-S. ~210 MB. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>773ac48 -
Decoder ONNX: clip+scale in FP32 before int32 castβ
Cortexelus +1: Claude Opus 4.7 (1M context)The PCM-baked tail (audio β clamp β Γ32767 β int32) ran in the engine's FP16/BF16 trunk, where 32767 is not representable (rounds up to 32768), causing the int32 output to exceed the int16 range. The consumer code then wrapped 32768 β -32768 on int16 downcast, producing audible clicks on loud transients. Fix: cast audio to FP32 before the clip/scale chain in both build_same_l_decoder.py and build_same_s_decoder.py producers. 32767 is exact in FP32. A trailing clamp(-32767, 32767) is added as belt-and-suspenders against any residual numerical drift. Affected: dec_dynamic_triton_swa.{onnx,trt} (SAME-L), dec_dynamic_bf16.{onnx,trt} (SAME-S). Verified: 0 samples outside [-32767, +32767] on real-music decode (was ~800 with the old engines). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>a429ffa
-
Add pre-processed FP16-mixed ONNX for SA3 DiTsβ
Cortexelus +1: Claude Opus 4.7 (1M context)Ships dit_fp16mixed.onnx alongside the original FP32 dit.onnx for each SA3 DiT (sm-music, sm-sfx, sa3-m). Consumers compiling for a non-sm_90 architecture can now build the DiT engine with a plain `build_from_onnx.py sa3-sm-music` (STRONGLY_TYPED, no precision flags needed) β no onnx-graphsurgeon dependency required. The pre-processing was done once with build_dit_fp16mixed.py: - 140 RMSNorm chains wrapped in Cast(FP32)/Cast(FP16) islands - 40 attention Softmax nodes wrapped - ~186 RoPE-region nodes (reachable from Cast(to=FP32) feeding Cos/Sin) - Non-island weights converted to FP16 ONNX size shrinks roughly in half (1.84 GB β 921 MB for sm-music) because most weights are now FP16. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>e11ce65 -
DiT engines: replace BF16 with FP16-mixed (FP32 islands) + add FP32 variantsβ
Cortexelus +1: Claude Opus 4.7 (1M context)Drop dit_bf16.trt for all three SA3 DiTs (sm-music, sm-sfx, sa3-m). BF16 quantization error compounds over 8 pingpong sampling steps, drifting cos-sim vs PT FP32 from 0.99 (single step) to 0.81 (final latent) and producing audibly degraded output. Replace with dit_fp16mixed.trt (the canonical from now on): FP16 trunk with FP32 islands around every RMSNorm chain (Pow+ReduceMean+Sqrt+Mul), every attention Softmax, and the RoPE region (anything reachable from a Cast(to=FP32) feeding Cos/Sin/Einsum). 140 RMSNorms + 40 Softmaxes per sm-music block, more for medium. Built with STRONGLY_TYPED so TRT honors the explicit dtypes (no auto-promotion). Matches MLX's "FP16 with implicit FP32 reductions via fused kernels" recipe. Per-step cos-sim vs PT FP32: 0.99997 single-step, 0.998 over 8 steps. RMS-curve correlation: 0.998. Audio basically indistinguishable from FP32 PyTorch reference at 1/15 the wall time (43 ms vs 630 ms). Also adds dit_fp32.trt (1.8 GB sm-*, 5.8 GB medium) for users who wanβ¦97651ef
-
DiT TRT engines: rebuild with profile min=1 (sm_90)β
Cortexelus +1: Claude Opus 4.7 (1M context)Lowered the dynamic-shape lower bound on `x` and `local_add_cond` from 256 β 1 for all three DiT engines (sa3-m, sa3-sm-music, sa3-sm-sfx). This unlocks short-form output below 24 s (down to ~93 ms at L=1) at zero measured perf cost β TRT picks identical tactics across [1, 4096] when opt stays at 1292. Output quality below the trained range (L<256, ~23.8 s) is undefined but the engine runs. Also refreshes tokenizer.json content for the T5Gemma engine. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>5f27021
-
T5Gemma: FP16-mixed (FP32 attention island) β fixes BF16 numerical bugβ
CortexelusThe previous BF16 build had a numerical bug: cos similarity vs PyTorch FP32 reference dropped to 0.17 at specific tokens (the ' beautiful' token id 4964 output activations of magnitude 26 instead of 52). Some tokens silently produced bad conditioning for downstream DiT. Switch to FP16 trunk + FP32 attention island (STRONGLY_TYPED network), the same strategy that fixed SAME-L decoder's accuracy. Cos vs FP32 PyTorch is now 0.999998; the worst-token cos across our test prompts is 0.9987. Engine filename: t5gemma_bf16.trt β t5gemma_fp16mixed.trt. Engine size: 564 MB β 623 MB (+10%). Latency: 0.78 ms β 0.91 ms (negligible vs DiT's 50+ ms). ONNX: re-exported with mixed dtypes; STRONGLY_TYPED build respects them. Consumer code (sa3_trt.py, install.sh, build.py, build_from_onnx.py) updated in the github repo to point at the new filename.23f5c64 -
Decoder TRT engines: PCM-baked-in (sm_90)β
CortexelusCompanion to the previous commit's ONNX update β the published sm_90 .trt engines for SAME-S and SAME-L decoders now have the same clamp+scale+ cast(int32)+permute postprocess baked into the engine graph. Drop-in compatible: sa3_trt.py auto-detects engine flavor by output tensor name (pcm vs audio) and skips Stage 5 clip/cast/transpose when pcm-baked. Stage 5 wall time drops from ~110ms to ~21ms on sm-music + same-s, 30s audio.106e6fd -
Decoder ONNX: bake PCM postprocessing into the graphβ
CortexelusBoth SAME-S and SAME-L decoder graphs now include the final clamp+scale+ cast(int32)+permute step, so the engine emits PCM-ready int32 tensors of shape (1, T_samples, 2) instead of float32 (1, 2, T_samples) in [-1, 1]. Consumer (sa3_trt.py) detects the new flavor by output tensor name (pcm vs audio) and narrows int32βint16 with a single GPU op β Stage 5 of the inference pipeline goes from ~110ms to ~21ms. Renamed outputs in the .onnx graphs: - audio β pcm - dtype: float32 β int32 - shape: (1, 2, T) β (1, T, 2)459fd8d
-
Add ONNX files for per-arch TRT compilationβ
Cortexelus- onnx/t5gemma/encoder.onnx - onnx/same-s/{enc,dec}_dynamic_bf16.onnx - onnx/same-l/{enc,dec}_dynamic_triton_swa.onnx - onnx/sa3-{m,sm-music,sm-sfx}/dit.onnx opset 17, arch-independent. With these, anyone can compile fresh TRT engines for a new GPU architecture (sm_100, sm_120, ...) without SAT / source ckpts β just TensorRT + torch + huggingface-hub. The sa3-m DiT exceeds protobuf's 2GB limit so its weights live in a single external dit.onnx.data blob alongside the proto (also LFS-tracked).62164ab -
Drop L-range suffix from SAME-L decoder filenameβ
CortexelusRename: tensorRT/sm_90/same-l/dec_dynamic_triton_swa_min32_opt1292_max4096.trt β tensorRT/sm_90/same-l/dec_dynamic_triton_swa.trt The (min,opt,max) L profile is baked into the engine itself; the verbose filename was a historical leftover from when only the decoder was dynamic. The encoder + other dynamic engines never had it. Now they all match.633bd26