Tiny Aya Global: Pocket GGUF builds

Research builds of CohereLabs/tiny-aya-global used in Pocket, a study of whether Tiny Aya's safety survives GGUF quantization in English, Swahili, Yorรนbรก, Hausa and Igbo (Lexsi AI Safety & Alignment Hackathon).

These are evaluation artifacts, not recommended deployments. Every 3-bit and 2-bit file here refuses harmful requests much less often outside English than Cohere's official files, including the imatrix builds (the safety-calibrated imatrix did not restore refusal). For use in Swahili, Yorรนbรก, Hausa or Igbo, prefer Cohere's official q4_k_m or larger.

Files

File Size What it is Refuses harmful prompts (en / sw / yo / ha / ig)
tiny-aya-global-q3_k_m.gguf 1.8 GB uniform q3_k_m from the official f16, no imatrix 75 / 40 / 18 / 43 / 52 %
tiny-aya-global-q2_k.gguf 1.5 GB uniform q2_k from the official f16, no imatrix 75 / 29 / 2 / 20 / 21 %
imatrix-generic.gguf 7 MB importance matrix from generic text (Belebele passages not in the test) โ€”
imatrix-safety.gguf 7 MB importance matrix from Tiny Aya's own f16 answers to 600 held-out harmful and harmless prompts in 5 languages โ€”
tiny-aya-global-q3_k_m-imat-safety.gguf 1.8 GB q3_k_m with imatrix-safety (our fix attempt) 73 / 40 / 20 / 42 / 42 %
tiny-aya-global-q3_k_m-imat-generic.gguf 1.8 GB q3_k_m with imatrix-generic (control) 74 / 44 / 23 / 44 / 45 %

For reference, the official f16 file refuses 74 / 56 / 29 / 51 / 48 % and the official q4_k_m 73 / 59 / 32 / 56 / 56 %. Full results with judge scores and intervals: MOREN808/pocket-safety-evals. Browse every answer: Pocket Space.

How they were made

Code: github.com/neromtoobad/pocket.

llama.cpp b12403, from Cohere's official tiny-aya-global-f16.gguf:

llama-imatrix -m tiny-aya-global-f16.gguf -f calib_safety.txt -o imatrix-safety.gguf -ngl 99 -c 512 --chunks 400
llama-quantize tiny-aya-global-f16.gguf tiny-aya-global-q3_k_m.gguf q3_k_m
llama-quantize --imatrix imatrix-safety.gguf tiny-aya-global-f16.gguf tiny-aya-global-q3_k_m-imat-safety.gguf q3_k_m

Run with the model's own chat template (llama-server --jinja), which includes Cohere's default safety preamble.

License

Same as the base model: CC-BY-NC 4.0, and use must follow the Cohere Labs Acceptable Use Policy.

Downloads last month
239
GGUF
Model size
3B params
Architecture
cohere2
Hardware compatibility
Log In to add your hardware

2-bit

3-bit

Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for MOREN808/tiny-aya-global-pocket-gguf

Quantized
(29)
this model

Collection including MOREN808/tiny-aya-global-pocket-gguf