425GMM/llama-3.2-3b-qlora-aslg-gloss

Llama-3.2-3B-Instruct QLoRA adapter for English -> ASL gloss (ASLG-PC12)

A LoRA adapter (r=16, alpha=32, all attention and MLP projections) trained with QLoRA for one epoch on the first 20,000 of the 64,872 training pairs of the ASLG-PC12 English-ASL gloss parallel corpus. The subset was a compute budget (about 90 minutes on a 10 GB GPU), not a data selection; the full split was not used. The BART model from the same project was trained on the whole split. Input is a lowercased English sentence in a chat prompt; output is an upper-case ASL gloss sequence in the corpus's conventions (X- pronouns, DESC- modifiers).

The base model is unsloth/Llama-3.2-3B-Instruct (an ungated mirror of Meta's meta-llama/Llama-3.2-3B-Instruct; the adapter works on either). This repo contains only the adapter; the base weights are downloaded from the Hub.

This model is one of the systems in a small project comparing grammar-constrained decoding (xgrammar) with unconstrained decoding for gloss generation. Code, grammars, the exact system prompt and evaluation scripts: https://github.com/TheRealGioviok/nlpproj

Training

Base model	unsloth/Llama-3.2-3B-Instruct, 4-bit NF4 with double quantisation
LoRA	r=16, alpha=32, dropout 0.05, targets q/k/v/o/gate/up/down_proj
Data	first 20,000 of the 64,872 pairs in the training split of the Kaggle redistribution of ASLG-PC12 (87,710 pairs, no official split; deduplicated and re-split 80/10/10 with seed 42 in the project), chat format, loss on gloss tokens only
Epochs	1
Effective batch size	16 (2 x 8 accumulation)
Learning rate	2e-4
Max length	384 tokens
Steps	1,250

Results (1,000-sentence test sample, greedy decoding)

Decoding	BLEU	chrF	Exact match	Sequence validity
Unconstrained	98.1	99.3	92.0%	89.8%
Constrained (grammar v3)	96.7	98.6	87.9%	100%
Constrained + copy rule (v3)	98.0	99.3	92.2%	100%

"Sequence validity" is the share of outputs whose every token is in the closed gloss vocabulary. The constrained + copy condition is statistically indistinguishable from unconstrained decoding on BLEU and exact match (paired bootstrap) while guaranteeing valid output.

Usage

import torch
from transformers import AutoTokenizer, BitsAndBytesConfig
from peft import AutoPeftModelForCausalLM

repo = "425GMM/llama-3.2-3b-qlora-aslg-gloss"
tok = AutoTokenizer.from_pretrained(repo)
bnb = BitsAndBytesConfig(load_in_4bit=True, bnb_4bit_quant_type="nf4",
                         bnb_4bit_compute_dtype=torch.bfloat16, bnb_4bit_use_double_quant=True)
model = AutoPeftModelForCausalLM.from_pretrained(repo, quantization_config=bnb,
                                                 device_map="auto", dtype=torch.bfloat16)

system = "<the system prompt from the project repository>"
src = "we should behave outside european waters in just the same way as we do at home ."
msgs = [{"role": "system", "content": system},
        {"role": "user", "content": f"Translate to ASL gloss: {src}"}]
ids = tok.apply_chat_template(msgs, add_generation_prompt=True, return_tensors="pt").to(model.device)
out = model.generate(ids, max_new_tokens=128, do_sample=False)
print(tok.decode(out[0, ids.shape[1]:], skip_special_tokens=True))

The adapter was trained with a specific system prompt (zero-shot form, no demonstrations); use the same prompt for best results. It is in the project repository.

Limitations

ASLG-PC12 glosses are rule-generated from Europarl text, not produced by signers. The adapter learns those rules almost completely (92% exact match), which says more about the corpus than about ASL translation. Scores do not transfer to human-annotated gloss corpora.