Commit History
-
Move deprecated sm_90 TRT engines under legacy/
Verified signatureThese are not reachable from the runtime registry: every tier resolves to the chunkable engines (dec_*_chunkable_limiter / enc_*_chunkable). Kept rather than deleted so older checkouts and reproductions still have them, but moved out of the way so the live set is unambiguous. Copies are server-side; the ONNX sources are untouched, and four of them (enc_dynamic_triton_swa.onnx, enc_dynamic_bf16.onnx, both enc_fp8.onnx) remain load-bearing for build_autoencoders.py -- 'chunkable' is a profile choice for the encoders, not a graph change.ceaecd4
-
retire fp8_fast; remove the superseded sm_90 fp8 engines (miscalibrated encoder, 8192-ceiling decoders)
Verified signature3963f0e -
SAME-L/SAME-S: bake the limiter ceiling so the chunkable decoders keep the ('latent','pcm') signature; add the SAME-S fp8 chunkable pair
Verified signatureaea1cf0 -
SAME-L/SAME-S: bake the limiter ceiling so the chunkable decoders keep the ('latent','pcm') signature; add the SAME-S fp8 chunkable pair
Verified signature9a01fee -
SAME-L encoders: low band 64 -> 256 (1.3-1.5x faster, cos 0.99831 -> 0.99969)
Verified signature1a8702f -
SAME-L encoders: low band 64 -> 256 (1.3-1.5x faster, cos 0.99831 -> 0.99969)
Verified signatured0d486b
-
SAME-L: remove miscalibrated enc_fp8.trt (activation scales 1.3-28x too small; superseded by enc_fp8_chunkable.trt on sm_90)
Verified signaturee00128c -
SAME-L: add enc_fp8_chunkable.trt (fp16, two-profile chunkable)
Verified signature7036ca2 -
SAME-L: add dec_fp8_chunkable_limiter.trt (fp16, two-profile chunkable, runtime limiter ceiling)
Verified signature093c4d6 -
SAME-L: add enc_fp16_chunkable.trt (fp16, two-profile chunkable)
Verified signatureeb343c9 -
SAME-L: add dec_fp16_chunkable_limiter.trt (fp16, two-profile chunkable, runtime limiter ceiling)
Verified signature5dba282
-
Remove redundant w8_bf16 engines (int8 weights fold to bf16 at build → identical size/speed to the bf16 baseline)
Verified signature63cc70d -
Add sm_90 TRT engines (wide profile) for the quantized tiers
Verified signatured593f74
-
Add FP32 SAME-L + SAME-S TRT decoders
Cortexelus +1: Claude Opus 4.7 (1M context)Pair the existing FP32 DiT engines (sa3-{m,sm-music,sm-sfx}/dit_fp32.trt) with FP32 decoders so the consumer-side `--precision fp32` flag has a full FP32-throughout pipeline available, bit-equivalent to PyTorch eager. - same-l/dec_dynamic_fp32.trt — built by upcasting dec_dynamic_triton_swa.onnx initializers + value_info + Constant nodes + Cast.to from FP16 to FP32 (STRONGLY_TYPED). The Triton SWA plugin already runs on the FP32 side of the attention island, so the plugin contract is unchanged. ~1.7 GB. - same-s/dec_dynamic_fp32.trt — analogous FP32 engine for SAME-S. ~210 MB. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>773ac48 -
Decoder ONNX: clip+scale in FP32 before int32 cast
Cortexelus +1: Claude Opus 4.7 (1M context)The PCM-baked tail (audio → clamp → ×32767 → int32) ran in the engine's FP16/BF16 trunk, where 32767 is not representable (rounds up to 32768), causing the int32 output to exceed the int16 range. The consumer code then wrapped 32768 → -32768 on int16 downcast, producing audible clicks on loud transients. Fix: cast audio to FP32 before the clip/scale chain in both build_same_l_decoder.py and build_same_s_decoder.py producers. 32767 is exact in FP32. A trailing clamp(-32767, 32767) is added as belt-and-suspenders against any residual numerical drift. Affected: dec_dynamic_triton_swa.{onnx,trt} (SAME-L), dec_dynamic_bf16.{onnx,trt} (SAME-S). Verified: 0 samples outside [-32767, +32767] on real-music decode (was ~800 with the old engines). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>a429ffa
-
Decoder TRT engines: PCM-baked-in (sm_90)
CortexelusCompanion to the previous commit's ONNX update — the published sm_90 .trt engines for SAME-S and SAME-L decoders now have the same clamp+scale+ cast(int32)+permute postprocess baked into the engine graph. Drop-in compatible: sa3_trt.py auto-detects engine flavor by output tensor name (pcm vs audio) and skips Stage 5 clip/cast/transpose when pcm-baked. Stage 5 wall time drops from ~110ms to ~21ms on sm-music + same-s, 30s audio.106e6fd
-
Drop L-range suffix from SAME-L decoder filename
CortexelusRename: tensorRT/sm_90/same-l/dec_dynamic_triton_swa_min32_opt1292_max4096.trt → tensorRT/sm_90/same-l/dec_dynamic_triton_swa.trt The (min,opt,max) L profile is baked into the engine itself; the verbose filename was a historical leftover from when only the decoder was dynamic. The encoder + other dynamic engines never had it. Now they all match.633bd26 -
Reorganize tensorRT/ into per-arch subdirs; rebuild DiT engines with conditioner baked in
Cortexelus- Move all engines under tensorRT/sm_90/ (room for sm_100, sm_120 later) - Co-locate T5Gemma tokenizer.json with the engine at tensorRT/sm_90/t5gemma/ - DiT engines now bundle per-DiT padding_embedding + seconds_total Linear as graph Constants, eliminating the runtime/padding_embeddings/ and runtime/num_conditioner.* sidecar files08c1abe