SafeCompress recipes + loader for Tiny Aya Global
Layer sets and a loader for every arm we evaluated (recipes.json, load_safecompress.py), including the INT8-protected arms that cannot be saved as standard checkpoints. No weights here: the base model is gated.
INT8 on the listed layers, NF4 everywhere else. Reference rows: FP16 harmful refusal 0.655, MMLU 0.475, 6.70 GB; uniform 4-bit 0.605, 0.457, 2.53 GB.
| arm | INT8 layers | harmful refusal | first-token gap | MMLU | Belebele | GB |
|---|---|---|---|---|---|---|
S_C_refusalDrop_W8_W4 |
[7, 14, 16, 27, 35] | 0.650 | 2.31 | 0.470 | 0.755 | 2.73 |
S_X_Cons5_W8_W4 |
[0, 7, 14, 16, 35] | 0.655 | 2.26 | 0.463 | 0.750 | 2.71 |
S_CK_circuitkit_W8_W4 |
[0, 14, 16, 20, 35] | 0.650 | 2.10 | 0.478 | 0.735 | 2.74 |
Differences between these selections and FP16 are within overlapping 95% confidence intervals (200 harmful prompts, 600 MMLU questions); we do not claim any arm beats FP16. Usage: model = load_safecompress(<layers>, protect_bits=8, token=HF_TOKEN). Full table: results_table.csv.
Headline of the study: uniform 4-bit loses refusal on harmful prompts; protecting five well-chosen layers (at FP16 or INT8) recovers it.