LLM-as-Jev (Qwen3-0.6B, full fine-tune)

Qwen3-0.6B fine-tuned to output a probability distribution over numbered options. The prompt lists options as [1] ... [K] and the reply is prefilled with Best answer: [. Candidate k is scored as the log-probability of the suffix k]; probabilities are the softmax of those scores.

Training: Reason-9k (9 sets x 1,000 examples), tree-restricted likelihood plus mass/out/position anchors to the frozen base model (top-64 truncation), AdamW, lr 5e-6, global batch 32. Code: https://github.com/hasnain1241/llm-as-jev

Results

Metric Training-free Fine-tuned
External macro accuracy (6 sets) 0.5531 0.5729
External macro ECE 0.2624 0.1178
External macro Brier 0.6523 0.5303
Banking77 accuracy 0.1847 0.6026
Banking77 ECE 0.4597 0.1201
Text-mode parse failures 0 0
Text-mode samples 350 350
Downloads last month
41
Safetensors
Model size
0.6B params
Tensor type
BF16
ยท
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for Hasnain1241/llm-as-jev

Finetuned
Qwen/Qwen3-0.6B
Finetuned
(1,494)
this model