Add beta7 int8 (learned rounding, ConvRot)
#30
by sandpies - opened
An int8 version of 10Eros_Max_h3_8stepTURBO_beta7_test1, made with silver's convert_to_quant (ctq 1.3.4) using learned (SVD/AdaRound) rounding, to sit alongside Qrusher's plain-rounded -int8.
- Only qkv, out, fc1 and fc2 in all 50 blocks are int8, with per-row scales and ConvRot at group size 256.
- The folded adaln, adaln_t_table, all norms, rope, the patch and condition projections, final_layer and token_refiner stay in bf16, so the source precision is untouched.
- All of beta7's original metadata (the Omni r64 and adaln-fold notes) is kept, plus _quantization_metadata.
- 20.97 GB. Tested in ComfyUI: it loads with the native mixed-precision ops and ran 8 steps at euler/simple with SLA and LoRAs without problems.
TenStrip changed pull request status to merged