TigreGotico/jevasio-0.8b

Jevásio-0.8B

Jevásio-0.8B is a decision model for European Portuguese (pt-PT). It reads a state (a text or a JSON object) and a set of typed questions about it. It returns a probability over the options of each question, in one forward pass and without generated text. The question types are choice (pick one option), score (an ordered level) and noul (yes or no).

It is Kev-0.8B fine-tuned on TigreGotico/jevasio-decisions, a set of typed decisions built from the amalia-llm data of the AMALIA project. The request format and the serving code are those of Kev, so a Kev or System One client works with it unchanged.

Model details

Backbone	Qwen/Qwen3.5-0.8B-Base (revision dc7cdfe2), frozen
Initialised from	jaredpalmer/kev-0.8b
Adapter	LoRA, rank 16, α 32, on the attention, MLP and DeltaNet projections (11.3M trainable parameters)
Head	Kev's pointer head: it scores each option's closing token against the question's final token
Calibration	One temperature, T = 1.66, stored in head.pt and applied at load time
Languages	European Portuguese values; English keys and option names
Licence	CC BY-NC-SA 4.0, for non-commercial research, because the training data is share-alike and non-commercial

How to use

Serve it with the Kev repository at commit 1f04cfa28e6a708485b28a9585010b9e436b776e:

git clone https://github.com/jaredpalmer/kev.git && cd kev && git checkout 1f04cfa28e6a708485b28a9585010b9e436b776e && uv sync --extra serve
uv run --extra serve python -m kev.serve --run TigreGotico/jevasio-0.8b --port 8008

from typesafe_sdk import Choice, Noul, TypeSafeClient

client = TypeSafeClient(api_key="local", base_url="http://127.0.0.1:8008", model="kev-latest")
response = client.system_one(
    state={"request": "Fui cobrado duas vezes pela encomenda 1182. Podem devolver uma das cobranças?"},
    questions={
        "team": Choice(instructions="Que equipa deve tratar deste pedido?",
                       criteria={"billing": "Cobranças e reembolsos", "shipping": "Entregas", "returns": "Trocas"}),
        "urgent": Noul(instructions="Este pedido precisa de resposta hoje?"),
    },
)
print(response.choices["team"].choice, response.nouls["urgent"].noul)

Write the keys in English and the values in Portuguese, as in the training data: the state's field names, the question ids and the option names are code identifiers. KEV_TEMPERATURE=1.0 returns the probabilities before calibration.

Training

The training data is the train split of TigreGotico/jevasio-decisions (147,254 requests), mixed with 12,576 English records of Kev's decision-v7 suite as replay. Records with a state over 2,048 tokens were dropped, which left 151,804 of 159,830 records.

Epochs	2
Learning rate	2e-05
Effective batch	64 (8 × 8 accumulation)
Precision	bf16 autocast, gradient checkpointing
Kev options	--p_none 0 --p_none_distract 0 --p_distract 0 (no English "none of the above" options or distractors), --length_sort 1
Optimizer steps	4,744
Wall time	4.3 hours on one RTX 4090

The temperature is fitted on Portuguese questions from both sides of training: the calib split, from the sources the model trained on, and the test questions of the evaluation-only sources, which it never saw. A fit on calib alone leaves the model overconfident on new tasks. A fit on the unseen sources alone leaves it underconfident on the trained ones. To keep the unseen figure honest, each evaluation-only source is left out in turn and scored at the temperature fitted without it:

	ECE, temperature from calib only (T = 1.23)	ECE, combined temperature
Unseen sources, each left out of its own fit	0.129	0.091
Trained sources, test split	0.004	0.037

scripts/refit_temperature.py in the kev-pt code does the fit, and Kev's scripts/calibrate_checkpoint.py --temperature writes it into head.pt.

Evaluation

The test split holds 17,332 requests with 18,546 questions. Every request was scored, for this model and for stock Kev-0.8B. Eight sources are evaluation-only, so the model never trained on them: pt_exams, alba_mcq, alba_rating, cultura_viva, pt_text_completion, xstest, harmbench and sst. English transfer is Kev's transfer-v4 test suite.

	Stock Kev-0.8B	Jevásio-0.8B
Accuracy, pt-PT test	0.564	0.721
ECE, pt-PT test	0.017	0.019
Brier, pt-PT test	0.533	0.354
Accuracy, English transfer-v4	0.653	0.660
ECE, English transfer-v4	0.044	0.047

Per source, on the pt-PT test split. The tuned ECE of the evaluation-only sources is partly in-sample, because their questions are in the temperature fit; Training gives the left-out figures.

Source	Questions	Stock acc	Tuned acc	Stock ECE	Tuned ECE
alba_mcq	237	0.447	0.481	0.057	0.060
alba_rating	719	0.316	0.323	0.061	0.226
bbh_pt	144	0.306	0.479	0.130	0.068
corege_fos	1,000	0.679	0.836	0.037	0.113
csqa_pt	421	0.511	0.653	0.026	0.104
cultura_viva	2,463	0.401	0.387	0.029	0.071
dpo_pt	1,000	0.542	0.970	0.043	0.019
dsl_tl	147	0.680	0.735	0.044	0.073
egsciqa	1,000	0.485	0.627	0.064	0.094
frmt	4	0.500	0.750	0.248	0.200
harmbench	300	0.547	0.643	0.111	0.090
openbookqa_pt	478	0.446	0.621	0.064	0.066
piqa_pt	542	0.600	0.668	0.038	0.022
pt_culture	1,000	0.678	0.945	0.090	0.018
pt_exams	1,346	0.476	0.516	0.028	0.030
pt_text_completion	138	0.536	0.543	0.191	0.085
ptradutor	987	0.563	1.000	0.132	0.006
rewrite_faithful	1,000	0.770	1.000	0.112	0.001
saudade	458	0.620	0.773	0.039	0.044
siqa_pt	685	0.512	0.648	0.079	0.041
sst	100	0.690	0.830	0.063	0.080
summary_faithful	994	0.717	1.000	0.060	0.001
tierc_everyday_conv	260	0.581	0.781	0.105	0.064
tierc_persona_general	343	0.700	0.869	0.123	0.055
tierc_persona_nemotron	335	0.558	0.869	0.085	0.097
tierc_wikipedia_conversations	319	0.696	0.831	0.133	0.105
toxichat	478	0.897	0.975	0.209	0.027
wildguardmix	305	0.685	0.895	0.047	0.064
winogrande_pt	396	0.528	0.606	0.118	0.089
wmt24pp	497	0.561	0.775	0.115	0.040
xstest	450	0.678	0.727	0.072	0.067

Limits

• One temperature does not fit every task. On a Portuguese task it never saw, the ECE is about 0.091, and on alba_rating, a five-level quality rating, it is 0.259. Before you set a threshold, fit a temperature on labelled data from your own workload (python -m kev.calibrate).
• The variety question leans towards Brazilian Portuguese. On the dsl_tl and FRMT test texts, the model names European Portuguese texts correctly 24 of 49 times, and Brazilian texts 87 of 102 times. The training texts are two-thirds Brazilian.
• Portuguese knowledge did not improve. Accuracy on cultura_viva and alba_rating, two native pt-PT sets the model never trained on, stayed level or fell.
• Some sources are too easy to measure. The faithfulness, translation and preference sources score near 1.0, because their wrong options come from other rows.
• Most knowledge sources are machine translations into Portuguese. The native pt-PT sources (saudade, pt_culture, corege_fos, egsciqa, pt_exams, alba, cultura_viva) are the better test of pt-PT ability.
• The conversation questions (tierc) have teacher labels. AMALIA-9B-1225-DPO labelled them, and nobody checked its answers.
• This is not a model for decisions with legal, medical, financial or employment consequences for people without human review. It does not generate text.

int8 weights

The int8/ folder holds the same model with weight-only int8 backbone weights: one bf16 scale per output row, with the LoRA merged in. It needs about half the memory of bf16. Load it with jevasio_int8.py from this repository:

from jevasio_int8 import load

tok, model = load("TigreGotico/jevasio-0.8b", "cuda")

On 300 test requests, its accuracy is the same as the bf16 model's, and 3 answers change.

Citation

The training data comes from the AMALIA project. Cite it, and Kev:

@inproceedings{simplicio-etal-2026-amalia,
    title = "{AMALIA}: A Fully Open Large Language Model for {E}uropean {P}ortuguese",
    author = "Simpl{\'i}cio, Afonso and Vinagre, Gon{\c{c}}alo and Ramos, Miguel Moura and Tavares, Diogo and Ferreira, Rafael and Attanasio, Giuseppe and Alves, Duarte M. and Calvo, In{\^e}s and others",
    booktitle = "Proceedings of the 17th International Conference on Computational Processing of {P}ortuguese ({PROPOR} 2026) - Vol. 1",
    year = "2026",
    address = "Salvador, Brazil",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2026.propor-1.38/",
    pages = "380--391"
}