Commit History
-
Move the retired fp8_fast ONNX under onnx/same-s/legacy/β
Verified signatureThe SAME-S-only fp8_fast tier pushed fp8 into the attention projections as well, buying +0.08x (1.14 -> 1.22x) at a real quality cost, and was retired. Nothing builds from these graphs and --dec-precision no longer accepts the value.b5182df -
Rename the autoencoder ONNX to match the engine naming schemeβ
Verified signaturedec_*_chunkable_limiter.onnx -> dec_*_limiter.onnx {enc,dec}_dynamic_triton_swa.onnx -> {enc,dec}_fp16.onnx (SAME-L) {enc,dec}_dynamic_bf16.onnx -> {enc,dec}_bf16.onnx (SAME-S) 'chunkable' was never a property of the ONNX: it is the two optimization profiles the TRT builder adds, so the engines keep the name and the graphs drop it. 'dynamic_triton_swa' named the samel::diff_attn_swa plugin node rather than a precision, which put it out of step with every other file here. Engine filenames are unchanged.4ea87d6 -
Move the retired w8_bf16 ONNX under onnx/*/legacy/β
Verified signatureint8 weight-only storage. Its DequantizeLinear nodes sit on constant inputs, so TensorRT folds them at build time and the resulting engine is identical to the one built from the plain bf16 graph -- a file-size format, not a quantisation tier. No build target references them.74cd2b7
-
build-ready ONNX: limiter grafted + ceiling baked (decoders), amax-recalibrated activation scales (fp8 encoders)
Verified signatured070aac -
build-ready ONNX: limiter grafted + ceiling baked (decoders), amax-recalibrated activation scales (fp8 encoders)
Verified signatureb1400fd -
build-ready ONNX: limiter grafted + ceiling baked (decoders), amax-recalibrated activation scales (fp8 encoders)
Verified signature3f689ec -
build-ready ONNX: limiter grafted + ceiling baked (decoders), amax-recalibrated activation scales (fp8 encoders)
Verified signaturef29bd0e -
build-ready ONNX: limiter grafted + ceiling baked (decoders), amax-recalibrated activation scales (fp8 encoders)
Verified signature9aa2165 -
build-ready ONNX: limiter grafted + ceiling baked (decoders), amax-recalibrated activation scales (fp8 encoders)
Verified signaturedcaaf84
-
Delete old fp16mixed names + retired medium dit_bf16.trt (fp16 rename shipped)
Verified signature48e34e2 -
Rename fp16mixed->fp16 DiT/T5Gemma engines+onnx (copies; deletes follow after code merge)
Verified signature3f29675 -
Add quantized encoder+decoder tier ONNX (fp8 / w8_bf16, padded encoders)
Verified signature4168236
-
Add fp8 tier for sm-music / sm-sfx DiTs (clean fp8-linear graft on fp16mixed)
Verified signature308abaf -
Add fp8 tier for sm-music / sm-sfx DiTs (clean fp8-linear graft on fp16mixed)
Verified signatureac3a57e -
Add fp8 tier for sm-music / sm-sfx DiTs (clean fp8-linear graft on fp16mixed)
Verified signature5bcf0c7 -
Add fp8 tier for sm-music / sm-sfx DiTs (clean fp8-linear graft on fp16mixed)
Verified signature6a96f1b -
sa3-m fp8: calibrated dit_fp8.onnx (real-conditioning activation + per-channel weight scales, @ryanontheinside #47; refs unchanged dit_fp8lin.onnx.data)
Verified signature8383b7d
-
Upload onnx/sa3-m/dit_fp8lin.onnx.data with huggingface_hub
Verified signature9efa545 -
Upload onnx/sa3-m/dit_fp8.onnx with huggingface_hub
Verified signature21b6506 -
Fix the bf16 medium DiT: bake RoPE's tables into the graph (clipping 3.112% -> 0.014%) (#5)β
- Fix the bf16 medium DiT: bake RoPE's tables into the graph (clipping 3.112% -> 0.014%) (59bd967d4635c6e32b0b2a677d56a21984654b9d)6152b1d -
Fix medium DiT FP16-mixed ONNX: bound the RoPE island so attention fuses (4.3x faster, cos 1.0000) (#3)β
- Fix medium DiT FP16-mixed ONNX: bound the RoPE island so attention fuses (4.3x faster, cos 1.0000) (b4300a99fc3b12b6c59e2d10218399f672087340)fb17708
-
Fix missing 2Ο factor in seconds_freqs across all dit.onnx (FP32 source ONNX)β
Cortexelus +1: Claude Opus 4.7 (1M context)The base FP32 dit.onnx files were exported by an older version of the producer (fast_load/build_dit.py) before the 2Ο fix was applied. The post-surgery dit_fp16mixed.onnx files DID get the fix re-injected (their seconds_freqs has max β 62832 = 2Ο Γ 10000), but the source dit.onnx files were never re-uploaded and still have max=10000. Effect: any consumer building from dit.onnx (e.g., `--precision fp32` path that reads sa3-m/dit.onnx, or anyone re-running build_dit_fp16mixed locally) would inherit the silence-after-~5s bug β the seconds_total conditioner produces frequencies 2ΟΓ too low, so the Fourier features collapse and the model can't condition on duration beyond ~5s. Surgical fix: read the (128,) seconds_freqs initializer in each dit.onnx, multiply by 2Ο, write back. For sa3-m the buffer is in the .data sidecar at offset 3072+512 bytes β read it from there, multiply, and inline the result into the .onnx so the 5.8 GB .data file doesn't need to be re-uploaded. For sa3-sm-music/sfβ¦ecc70ac -
Fix sa3-m DiT ONNX external_data ref (was pointing at wrong filename)β
Cortexelus +1: Claude Opus 4.7 (1M context)The medium DiT FP16-mixed ONNX had 428 initializers whose external_data location field pointed at "dit_m_fp16mixed.onnx.data", but the actual sidecar on disk (and in this repo) is "dit_fp16mixed.onnx.data" β no "_m_" infix. The mismatch came from the producer naming convention at build time; the rename to the shipped name only touched the file, not the in-ONNX reference. Consequence: any consumer trying to compile a TRT engine from this ONNX (e.g., on a new GPU arch like sm_120 via build_from_onnx.py sa3-m) hit: [TRT] [E] WeightsContext.cpp:190: Failed to open file: ... /onnx/sa3-m/dit_m_fp16mixed.onnx.data [TRT] [E] Failed to import initializer: padding_embedding β¦ and ended up either with no engine or a partial/garbage engine, manifesting as "medium DiT outputs ~5s of music then dead silence" on some setups. The pre-built sm_90 TRT engine in this repo is unaffected (it was built from the producer's ONNX at the moment the data file still had the original name). β¦3dbdcfa -
Decoder ONNX: clip+scale in FP32 before int32 castβ
Cortexelus +1: Claude Opus 4.7 (1M context)The PCM-baked tail (audio β clamp β Γ32767 β int32) ran in the engine's FP16/BF16 trunk, where 32767 is not representable (rounds up to 32768), causing the int32 output to exceed the int16 range. The consumer code then wrapped 32768 β -32768 on int16 downcast, producing audible clicks on loud transients. Fix: cast audio to FP32 before the clip/scale chain in both build_same_l_decoder.py and build_same_s_decoder.py producers. 32767 is exact in FP32. A trailing clamp(-32767, 32767) is added as belt-and-suspenders against any residual numerical drift. Affected: dec_dynamic_triton_swa.{onnx,trt} (SAME-L), dec_dynamic_bf16.{onnx,trt} (SAME-S). Verified: 0 samples outside [-32767, +32767] on real-music decode (was ~800 with the old engines). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>a429ffa
-
Add pre-processed FP16-mixed ONNX for SA3 DiTsβ
Cortexelus +1: Claude Opus 4.7 (1M context)Ships dit_fp16mixed.onnx alongside the original FP32 dit.onnx for each SA3 DiT (sm-music, sm-sfx, sa3-m). Consumers compiling for a non-sm_90 architecture can now build the DiT engine with a plain `build_from_onnx.py sa3-sm-music` (STRONGLY_TYPED, no precision flags needed) β no onnx-graphsurgeon dependency required. The pre-processing was done once with build_dit_fp16mixed.py: - 140 RMSNorm chains wrapped in Cast(FP32)/Cast(FP16) islands - 40 attention Softmax nodes wrapped - ~186 RoPE-region nodes (reachable from Cast(to=FP32) feeding Cos/Sin) - Non-island weights converted to FP16 ONNX size shrinks roughly in half (1.84 GB β 921 MB for sm-music) because most weights are now FP16. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>e11ce65
-
T5Gemma: FP16-mixed (FP32 attention island) β fixes BF16 numerical bugβ
CortexelusThe previous BF16 build had a numerical bug: cos similarity vs PyTorch FP32 reference dropped to 0.17 at specific tokens (the ' beautiful' token id 4964 output activations of magnitude 26 instead of 52). Some tokens silently produced bad conditioning for downstream DiT. Switch to FP16 trunk + FP32 attention island (STRONGLY_TYPED network), the same strategy that fixed SAME-L decoder's accuracy. Cos vs FP32 PyTorch is now 0.999998; the worst-token cos across our test prompts is 0.9987. Engine filename: t5gemma_bf16.trt β t5gemma_fp16mixed.trt. Engine size: 564 MB β 623 MB (+10%). Latency: 0.78 ms β 0.91 ms (negligible vs DiT's 50+ ms). ONNX: re-exported with mixed dtypes; STRONGLY_TYPED build respects them. Consumer code (sa3_trt.py, install.sh, build.py, build_from_onnx.py) updated in the github repo to point at the new filename.23f5c64 -
Decoder ONNX: bake PCM postprocessing into the graphβ
CortexelusBoth SAME-S and SAME-L decoder graphs now include the final clamp+scale+ cast(int32)+permute step, so the engine emits PCM-ready int32 tensors of shape (1, T_samples, 2) instead of float32 (1, 2, T_samples) in [-1, 1]. Consumer (sa3_trt.py) detects the new flavor by output tensor name (pcm vs audio) and narrows int32βint16 with a single GPU op β Stage 5 of the inference pipeline goes from ~110ms to ~21ms. Renamed outputs in the .onnx graphs: - audio β pcm - dtype: float32 β int32 - shape: (1, 2, T) β (1, T, 2)459fd8d
-
Add ONNX files for per-arch TRT compilationβ
Cortexelus- onnx/t5gemma/encoder.onnx - onnx/same-s/{enc,dec}_dynamic_bf16.onnx - onnx/same-l/{enc,dec}_dynamic_triton_swa.onnx - onnx/sa3-{m,sm-music,sm-sfx}/dit.onnx opset 17, arch-independent. With these, anyone can compile fresh TRT engines for a new GPU architecture (sm_100, sm_120, ...) without SAT / source ckpts β just TensorRT + torch + huggingface-hub. The sa3-m DiT exceeds protobuf's 2GB limit so its weights live in a single external dit.onnx.data blob alongside the proto (also LFS-tracked).62164ab