Add beta7 int8 (learned rounding, ConvRot)

#30
by sandpies - opened

An int8 version of 10Eros_Max_h3_8stepTURBO_beta7_test1, made with silver's convert_to_quant (ctq 1.3.4) using learned (SVD/AdaRound) rounding, to sit alongside Qrusher's plain-rounded -int8.

  • Only qkv, out, fc1 and fc2 in all 50 blocks are int8, with per-row scales and ConvRot at group size 256.
  • The folded adaln, adaln_t_table, all norms, rope, the patch and condition projections, final_layer and token_refiner stay in bf16, so the source precision is untouched.
  • All of beta7's original metadata (the Omni r64 and adaln-fold notes) is kept, plus _quantization_metadata.
  • 20.97 GB. Tested in ComfyUI: it loads with the native mixed-precision ops and ran 8 steps at euler/simple with SLA and LoRAs without problems.
TenStrip changed pull request status to merged

Sign up or log in to comment