quant: publish the quantized encodings

README.md

@@ -6,6 +6,7 @@ license_name: stable-audio-community
 license_link: LICENSE.md
 pipeline_tag: text-to-audio
 base_model: stabilityai/stable-audio-3-medium-base
+base_model_relation: quantized
 tags:
 - audio-generation
 - diffusion
@@ -16,7 +17,7 @@ tags:
 
 # Stable Audio 3 Medium Base DiT — GGUF for sa3.cpp
 
-F16 and F32 GGUF conversions of the training DiT from
+F32, F16 and Q4_K_M GGUF conversions of the training DiT from
 [stabilityai/stable-audio-3-medium-base](https://huggingface.co/stabilityai/stable-audio-3-medium-base),
 for native LoRA/DoRA training with [sa3.cpp](https://github.com/betweentwomidnights/sa3.cpp).
 
@@ -26,11 +27,12 @@ Do not use the post-trained/ARC medium DiT as the training base.
 
 ## Files
 
-| file | purpose |
-|---|---|
-| `stable-audio-3-medium-base-dit-1.5B-v1.0-F16.gguf` | recommended frozen training base |
-| `stable-audio-3-medium-base-dit-1.5B-v1.0-F32.gguf` | CPU/reference validation |
-| `SHA256SUMS` | release checksums |
+| file | size | purpose |
+|---|---:|---|
+| `stable-audio-3-medium-base-dit-1.5B-v1.0-Q4_K_M.gguf` | 962 MB | smallest, and the fastest to train on |
+| `stable-audio-3-medium-base-dit-1.5B-v1.0-F16.gguf` | 2773 MB | the reference training base |
+| `stable-audio-3-medium-base-dit-1.5B-v1.0-F32.gguf` | 5545 MB | CPU/reference validation |
+| `SHA256SUMS` | | release checksums |
 
 The tokenizer, T5Gemma encoder, conditioner, SAME autoencoder, and inference DiT are fetched from
 the normal medium model set; they are intentionally not duplicated here.
@@ -40,11 +42,44 @@ python tools/download_models.py --variant medium --encoding f16 --training-base
 build-cuda/bin/sa3-train --model medium --models-dir models --dataset /path/to/dataset --out train-runs/example
 ```
 
+## Training on the Q4_K_M base
+
+Training against a quantized base is supported on **every** backend — CPU, CUDA, Vulkan and Metal.
+The frozen base only ever enters the adapter path as a `mul_mat` argument, and the one backward that
+needed it, `out_prod(W, transpose(grad))`, now accepts a quantized `src0` on all three GPU backends.
+
+It is *faster* than F16 rather than a tradeoff, on a 2.9x smaller file — steady state, medium,
+128 frames, `dora-rows`:
+
+| backend | Q4_K_M | F16 |
+|---|---:|---:|
+| CUDA | 0.93 s/step | 1.08 s/step |
+| Vulkan | 1.72 s/step | 1.90 s/step |
+| Metal (M4) | 2.95 s/step | 3.39 s/step |
+| CPU | 29.2 s/step | 46.9 s/step |
+
+A 2000-step DoRA trained on the Q4_K_M base is **audibly indistinguishable** from the same run on
+the F16 base; loss tracks 2.5-4% higher throughout, the offset a slightly degraded forward should
+produce. Peak VRAM drops from 5243 to 3414 MiB, which is the part that decides whether a small
+machine can train at all.
+
+```bash
+python tools/download_models.py --variant medium --encoding q4_k_m --training-base
+build-cuda/bin/sa3-train --model medium --models-dir models \
+  --dit models/stable-audio-3-medium-base-dit-1.5B-v1.0-Q4_K_M.gguf \
+  --dataset /path/to/dataset --adapter-type dora-rows --rank 16 --out train-runs/example
+```
+
+The resulting adapter is an ordinary GGUF LoRA — nothing about it is quantization-specific, so it
+applies to an F16 or a quantized inference DiT either way.
+
 ## Provenance
 
 - Source revision: `b32993f73c3bdc3864043a72d8032606bba737c8`
-- Conversion: `tools/convert_dit.py --variant medium --training-base`, then `tools/quantize_gguf.py` for F16
+- Conversion: `tools/convert_dit.py --variant medium --training-base`, then `tools/quantize_gguf.py`
+  for F16 and `sa3-quantize --mix q4_k_m` for the quant
 - Relationship: tensor rename/serialization and precision conversion only; no retraining
+- The Q4_K_M base passes `sa3-quant-check` against F16 with `below-threshold=0` at cosine `0.990`
 
 ## License and attribution
 

SHA256SUMS

@@ -1,2 +1,3 @@
 b6d672da618e9a3007250df4b44a2b295f429a60e3669ba60987943679a52d7d  stable-audio-3-medium-base-dit-1.5B-v1.0-F16.gguf
 b8901b021eeb15a187a69e224e594545ff5fe0ada57c864db4edf6420997ccb2  stable-audio-3-medium-base-dit-1.5B-v1.0-F32.gguf
+4b97f48b777cbe41c7f7b84bd95971e649ce5e899c97904f6b2a1caea0339719  stable-audio-3-medium-base-dit-1.5B-v1.0-Q4_K_M.gguf

stable-audio-3-medium-base-dit-1.5B-v1.0-Q4_K_M.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4b97f48b777cbe41c7f7b84bd95971e649ce5e899c97904f6b2a1caea0339719
+size 1009176288