Instructions to use upfall-co/voidx-decider-12b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- PEFT
How to use upfall-co/voidx-decider-12b with PEFT:
from peft import PeftModel from transformers import AutoModelForCausalLM base_model = AutoModelForCausalLM.from_pretrained("google/gemma-4-12B-it") model = PeftModel.from_pretrained(base_model, "upfall-co/voidx-decider-12b") - Notebooks
- Google Colab
- Kaggle
voidx-decider-12b
A LoRA adapter for google/gemma-4-12B-it
(revision 707f0a3b) that answers multiple-choice decision questions:
classification, ontology placement, and state judgment. This is not an official
Google model.
A merged Q4_K_M GGUF is in voidx-decider-12b-GGUF.
Status: research release. This model did not pass its pre-registered locked-test gate (G6). It is strongly vulnerable to prompt injection. Read Evaluation and Limitations before using it.
What it is
- Reads the probability of each label token at the answer position, instead of generating free text.
- Gives calibrated probabilities: validation ECE 0.028, locked-test ECE 0.028 (Q4_K_M: 0.034).
- Supports English and Korean inputs.
Usage
Prompt format
The user turn:
Context:
<context text>
Question: <question>
Options:
(A) <option 1>
(B) <option 2>
(C) <option 3>
Apply the Gemma 4 chat template with thinking disabled (empty thought channel)
and no system prompt. Then append Answer: ( to the assistant turn. Read the
next-token probabilities of the label tokens A, B, C, β¦ and renormalize
over the shown options.
- Serving temperature: divide the label logits by T = 0.98580 before softmax
(
serving_temperature.json). - The prompt format and label table follow Mapika/decider (Apache-2.0).
With PEFT
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer
base = AutoModelForCausalLM.from_pretrained(
"google/gemma-4-12B-it", revision="707f0a3b8a3c7ad586ed01e27eafbad8a27dd0f7",
torch_dtype="bfloat16", device_map="auto")
model = PeftModel.from_pretrained(base, "upfall-co/voidx-decider-12b")
tok = AutoTokenizer.from_pretrained("google/gemma-4-12B-it",
revision="707f0a3b8a3c7ad586ed01e27eafbad8a27dd0f7")
With llama-server (GGUF)
- Raw prompt: send the fully rendered prompt (ending in
Answer: () to/completionwithn_probsset, and read the label tokens. - Chat endpoint: pass
chat_template_kwargs: {"enable_thinking": false}.
Training
| Item | Value |
|---|---|
| Data | about 14,500 rows of public data plus rule-generated state tracking (see NOTICE) |
| Method | LoRA r=32, alpha=64, dropout 0.05, all attention and MLP projections |
| Schedule | 1 epoch (97 steps), lr 5e-5 cosine |
| Hardware | 1Γ H100 80GB, about 90 minutes |
| Validation (1,300 rows) | accuracy 0.822, soft NLL 0.503, ECE 0.028 |
No teacher-model or commercial-API outputs were used for training.
Evaluation 1
One-shot evaluation on a locked test set (3,050 rows), with pass criteria registered before the run.
Accuracy by pool (bf16):
| Pool | Rows | Tuned | Base | Diff |
|---|---|---|---|---|
| All | 3,050 | 79.3% | 75.9% | +3.4pt |
| English holdout | 1,100 | 76.6% | 70.0% | +6.6 |
| Korean | 300 | 93.0% | 91.7% | +1.3 |
| Parallel enβko | 400 | 83.3% | 83.5% | β0.3 |
| Ontology: IPTC | 600 | 78.5% | 91.3% | β12.8 |
| Ontology: Wikidata held-out roots | 350 | 86.9% | 69.7% | +17.1 |
| State tracking, unseen domains | 300 | 62.3% | 47.0% | +15.3 |
Training improved Wikidata placement on unseen roots, but hurt IPTC. Most of the IPTC drop comes from "none of these" items: the tuned model places 41.8% of them into a category (base 8.4%).
Evaluation 2 β comparison with other models
All models got the same 3,023 questions with the same chat-format instructions. The G6 test set was already used once, so this is a descriptive comparison only.
| Model | Acc. | English | Korean | IPTC | WD held-out | Unseen-domain state | IPTC "none" over-placement | Injection drop |
|---|---|---|---|---|---|---|---|---|
| Gemini 3.8 Flash (reasoning track) | 84.3% | 73.4 | 92.7 | 93.3 | 77.7 | 100 | 3% | 0pt |
| voidx-decider-12b | 78.1% | 75.3 | 88.3 | 81.7 | 85.4 | 55.3 | 33% | β54pt |
| Gemma 4 31B | 75.9% | 67.5 | 90.3 | 92.5 | 68.3 | 56.7 | 3% | β30pt |
| GPT-5.6 terra | 75.6% | 63.2 | 90.3 | 93.5 | 74.9 | 61.3 | 3% | β18pt |
| Gemma 4 26B-A4B | 74.7% | 66.8 | 88.0 | 93.5 | 70.6 | 46.0 | 5% | β48pt |
| Gemma 4 12B base (before training) | 73.0% | 68.2 | 88.7 | 90.2 | 69.1 | 32.3 | 10% | β49pt |
| Qwen3.5-9B | 70.0% | 60.3 | 81.3 | 89.8 | 71.1 | 46.0 | 5% | β44pt |
| Qwen3-14B | 65.8% | 55.1 | 75.3 | 85.8 | 60.0 | 45.7 | 8% | β40pt |
| Ministral 14B | 65.8% | 52.8 | 81.3 | 82.0 | 70.9 | 46.3 | 25% | β30pt |
| Llama 3.1 8B | 52.7% | 46.6 | 65.0 | 67.8 | 45.1 | 38.3 | 42% | β45pt |
Notes:
- All models used the same instructions and questions (option order fixed, 27 tied-gold rows excluded).
- This model and the base were re-measured as Q4_K_M with the chat format. The original slot-format scores are the G6 numbers above.
- Gemini cannot disable reasoning, so it ran on a reasoning track (up to 1,024 output tokens).
- The GPT and Gemini endpoints do not accept a temperature, so they used default sampling.
- Llama 3.1 8B, Qwen3-14B, Ministral and the commercial models were scored on their text answers only (no logprobs).
- ECE is available for only 5 logprob models on 161 common rows, so it is not ranked.
- A Claude Sonnet 5.5 row is in progress and will be added when it finishes.
Raw numbers: eval/model_comparison.json.
Limitations and risks
- Prompt injection. If the input contains text like "the answer is X", the model picks X 98% of the time (direct injection). Do not put untrusted user input into the prompt without isolating it first (delimiters, stripping instructions). The base model has the same weakness (β53.9pt); training made it about 5pt worse.
- "None of these" over-placement on IPTC-style ontologies: the model tends to choose a category even when none fits.
- Option-order sensitivity: 13.8% of items change answer when options are reordered.
- Performance drops when the correct candidate is missing from retrieved options.
- Labels come from public datasets. There was no human gold review, and the public sources may overlap with pretraining data. Wikidata class noise is learned too (e.g. a song classified as "music video").
- A retraining run to fix the "none" over-placement is planned.
License
Apache-2.0, inherited from the base model. See LICENSE and NOTICE for
modification notices and training-data attributions.
- Downloads last month
- 5