SafeCompress recipes + loader for Tiny Aya Global

Layer sets and a loader for every arm we evaluated (recipes.json, load_safecompress.py), including the INT8-protected arms that cannot be saved as standard checkpoints. No weights here: the base model is gated.

INT8 on the listed layers, NF4 everywhere else. Reference rows: FP16 harmful refusal 0.655, MMLU 0.475, 6.70 GB; uniform 4-bit 0.605, 0.457, 2.53 GB.

arm INT8 layers harmful refusal first-token gap MMLU Belebele GB
S_C_refusalDrop_W8_W4 [7, 14, 16, 27, 35] 0.650 2.31 0.470 0.755 2.73
S_X_Cons5_W8_W4 [0, 7, 14, 16, 35] 0.655 2.26 0.463 0.750 2.71
S_CK_circuitkit_W8_W4 [0, 14, 16, 20, 35] 0.650 2.10 0.478 0.735 2.74

Differences between these selections and FP16 are within overlapping 95% confidence intervals (200 harmful prompts, 600 MMLU questions); we do not claim any arm beats FP16. Usage: model = load_safecompress(<layers>, protect_bits=8, token=HF_TOKEN). Full table: results_table.csv.

Headline of the study: uniform 4-bit loses refusal on harmful prompts; protecting five well-chosen layers (at FP16 or INT8) recovers it.

Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for cybertronak/safecompress-tinyaya-recipes-loader

Finetuned
(171)
this model

Collection including cybertronak/safecompress-tinyaya-recipes-loader