Mudit666/tiny-aya-global-qint4

Tiny Aya Global, every decoder linear at qint4 (plain-q4)

All 252 decoder linear layers of CohereLabs/tiny-aya-global are quantized to 4-bit (quanto qint4, weights only). It is the fully quantized baseline of our Study S, which asks whether 4-bit compression changes refusal behaviour and whether keeping an attribution-selected candidate refusal mask in bf16 helps. The other Study S models are Mudit666/tiny-aya-global-qint4-random77-bf16 and Mudit666/tiny-aya-global-qint4-refusalmask77-bf16.

Files and identity

File	Bytes	sha256
model.safetensors	2,568,072,456	dcc660f10b702d484e320862a8f82518ab19f4f521fca2910b14710bee1c31f1

• All 252 decoder linears are qint4. R1 counted 252 qint4 and 0 bf16 decoder linears out of 252; quanto_qmap.json has 252 entries.
• R1 also compared the 38 non-quantized parameter tensors (embeddings, norms and other excluded weights) with the source model. An independent AI auditor reviewed R1 (PASS 02:07). This sha256 is the new reference: no output hash was recorded at build time.
• tokenizer.json is byte-identical to Tiny Aya Global's at revision af89d219.

How to use

The weights are an optimum-quanto checkpoint. Load them with CircuitKIT's loader, as our evaluation did. It raises an error if the reloaded model contains no quanto modules.

from huggingface_hub import snapshot_download
from transformers import AutoTokenizer
from circuitkit.evaluation import load_quantized_checkpoint   # circuitkit 0.2.0, optimum-quanto 0.2.7

path = snapshot_download("Mudit666/tiny-aya-global-qint4")     # the loader needs a local folder
tok = AutoTokenizer.from_pretrained(path)
model = load_quantized_checkpoint(path).to("cuda").eval()

messages = [{"role": "user", "content": "<your question>"}]
inputs = tok.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt",
                                 return_dict=True).to(model.device)
out = model.generate(**inputs, max_new_tokens=256, do_sample=False, eos_token_id=[6, 3])
print(tok.decode(out[0, inputs["input_ids"].shape[1]:], skip_special_tokens=True))

Stop ids: always pass eos_token_id=[6, 3], as above. This repo's generation_config.json lists ids 6 and 3 (<|END_OF_TURN_TOKEN|>, <EOS_TOKEN>), the stops our evaluations used. It is an edited upload copy: the build saved one with id 3 only, and the weights are unchanged. But load_quantized_checkpoint does not apply that file. In our CPU load check of all five qint4 checkpoints (2026-11-13), the loaded model's generation_config.eos_token_id was still 3. Without the explicit argument, generation stops only at id 3, unlike our evaluations. Loading through plain Transformers is a different route, and this note does not cover it. Our evaluations ran on one NVIDIA A100.

Build tools on first load. optimum-quanto compiles a small C++ extension the first time it unpacks the 4-bit weights, so ninja and a C++ compiler must be on PATH; for GPU use, also a CUDA toolkit (CUDA_HOME). Without ninja the load stops with "Ninja is required to load C++ extensions". We checked this code path on CPU with C1's local checkpoint: with ninja on PATH it loaded and generated; without it, it failed with that error. Loading from the Hub is not yet checked.

How it was made

• Quantizer: CircuitKIT with optimum-quanto 0.2.7: qint4 weights only, groups of 128, no calibration data. Activations are not quantized.
• Source: Tiny Aya Global bf16, source hashes as R1 recorded them.
• Code: Mudit/scripts/make_baselines.py, random_quantize(hf, 36, high_fraction=0.0, seed=0). This is historical code, modified after the build. The current recipe is Mudit/scripts/build_qint4.py.

Evaluation (Study S)

Study S scoring: AuditKIT, greedy, stop ids [6, 3], max 256 new tokens. Refusal = the WildGuard judge's label (AuditKIT GuardJudge, ""-stripped profile, bf16), not a human rating. Δ = this model − the original bf16 Tiny Aya on the same prompts, 95% paired bootstrap by prompt (by question for MMLU), 10,000 resamples, seed 0. A CI covering 0 means no change detected, not unchanged. Harmful sets: higher refusal is better. XSTest-safe: higher refusal is over-refusal. Code repository: https://github.com/Muditkumar123/tiny-aya-unlearning-4bit.

Set (n)	Original refusal	This model	Δ vs original [95% CI]
HarmBench held-out (168)	0.792	0.821	+0.030 [−0.006, +0.065]
Aya red-team Arabic (50)	0.560	0.500	−0.060 [−0.180, +0.060]
Aya red-team Hindi (50)	0.460	0.440	−0.020 [−0.120, +0.060]
XSTest safe, over-refusal (250)	0.084	0.080	−0.004 [−0.028, +0.020]

Utility	Original	This model	Δ vs original (pp)
Global-MMLU English (500)	0.562 [0.518, 0.606]	0.548 [0.504, 0.590]	−1.4 [−4.4, +1.6]
Global-MMLU Hindi (200)	0.515 [0.445, 0.585]	0.495 [0.425, 0.565]	−2.0 [−6.5, +2.5]

No change was detected on any set compared with the original. That is not evidence that refusal is preserved: the intervals still include changes in both directions, for example [−0.006, +0.065] on HarmBench and [−0.180, +0.060] on Arabic.

Limits

• Judge labels. Refusal is a WildGuard label. Its accuracy on these Hindi and Arabic outputs has not been validated by human raters.
• Truncation. Answers that reach the 256-token cap were judged on cut text. Re-tokenizing the saved answers gives an estimate of how many reach the cap: 38/168 HarmBench, 17/50 Arabic, 31/50 Hindi, 95/250 XSTest. Actual truncation is not verified.
• Coarse tests. 168 HarmBench prompts and 50 per Aya language give wide intervals. No correction for multiple comparisons. One model, greedy decoding.
• Base revision. The source weights' hashes match Tiny Aya Global at af89d219 (R1), but the revision behind these historical builds cannot be proven from history.
• Size. The harness reports parameter memory (2449.0 MiB); the file on disk is 2,568,072,456 bytes.

Licence and use

• A derivative of CohereLabs/tiny-aya-global. It carries CC-BY-NC-4.0, non-commercial use only (licence agreement). Use also requires adhering to Cohere Labs' Acceptable Use Policy.
• Research use only: not for production, user-facing or agentic deployment.

Citation

@misc{salamanca2026tinyayabridgingscale,
  title={Tiny Aya: Bridging Scale and Multilingual Depth}, author={Alejandro R. Salamanca and others},
  year={2026}, eprint={2603.11510}, archivePrefix={arXiv}, url={https://arxiv.org/abs/2603.11510}
}