Multiple Choice
Transformers
Safetensors
qwen3
text-generation
decision-model
llm-as-jev
text-generation-inference
Instructions to use Hasnain1241/llm-as-jev with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use Hasnain1241/llm-as-jev with Transformers:
# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("Hasnain1241/llm-as-jev") model = AutoModelForCausalLM.from_pretrained("Hasnain1241/llm-as-jev", device_map="auto") - Notebooks
- Google Colab
- Kaggle
LLM-as-Jev (Qwen3-0.6B, full fine-tune)
Qwen3-0.6B fine-tuned to output a probability distribution over numbered options.
The prompt lists options as [1] ... [K] and the reply is prefilled with Best answer: [.
Candidate k is scored as the log-probability of the suffix k]; probabilities are the softmax of those scores.
Training: Reason-9k (9 sets x 1,000 examples), tree-restricted likelihood plus mass/out/position anchors to the frozen base model (top-64 truncation), AdamW, lr 5e-6, global batch 32. Code: https://github.com/hasnain1241/llm-as-jev
Results
| Metric | Training-free | Fine-tuned |
|---|---|---|
| External macro accuracy (6 sets) | 0.5531 | 0.5729 |
| External macro ECE | 0.2624 | 0.1178 |
| External macro Brier | 0.6523 | 0.5303 |
| Banking77 accuracy | 0.1847 | 0.6026 |
| Banking77 ECE | 0.4597 | 0.1201 |
| Text-mode parse failures | 0 | 0 |
| Text-mode samples | 350 | 350 |
- Downloads last month
- 41
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐ Ask for provider support