TigreGotico/jevasio-0.8b-v2

Jevásio-0.8B-V2

Jevásio-0.8B-V2 is a decision model for European Portuguese (pt-PT). It reads a state (a text or a JSON object) and a set of typed questions about it. It returns a probability over the options of each question, in one forward pass and without generated text. The question types are choice (pick one option), score (an ordered level) and noul (yes or no).

It is Jevásio-0.8B trained further on TigreGotico/jevasio-decisions-v2. That dataset holds every record of the first version and adds 36 sources: the language variety of a text in six domains, intent and scenario, news section and genre, legal area, parliament votes, entailment and similarity, offensive language and hate speech, puns, generated pt-PT messages, and copies of the faithfulness pairs with one fact changed. The request format and the serving code are those of Kev, so a Kev or System One client works with it unchanged.

Model details

Backbone	Qwen/Qwen3.5-0.8B-Base (revision dc7cdfe2), frozen
Initialised from	TigreGotico/jevasio-0.8b
Adapter	LoRA, rank 16, α 32, on the attention, MLP and DeltaNet projections (11.3M trainable parameters)
Head	Kev's pointer head: it scores each option's closing token against the question's final token
Calibration	One temperature, T = 1.38, stored in head.pt and applied at load time
Languages	European Portuguese values; English keys and option names
Licence	CC BY-NC-SA 4.0, for non-commercial research, because the training data is share-alike and non-commercial

How to use

Serve it with the Kev repository at commit 1f04cfa28e6a708485b28a9585010b9e436b776e:

git clone https://github.com/jaredpalmer/kev.git && cd kev && git checkout 1f04cfa28e6a708485b28a9585010b9e436b776e && uv sync --extra serve
uv run --extra serve python -m kev.serve --run TigreGotico/jevasio-0.8b-v2 --port 8008

from typesafe_sdk import Choice, Noul, TypeSafeClient

client = TypeSafeClient(api_key="local", base_url="http://127.0.0.1:8008", model="kev-latest")
response = client.system_one(
    state={"request": "Fui cobrado duas vezes pela encomenda 1182. Podem devolver uma das cobranças?"},
    questions={
        "team": Choice(instructions="Que equipa deve tratar deste pedido?",
                       criteria={"billing": "Cobranças e reembolsos", "shipping": "Entregas", "returns": "Trocas"}),
        "urgent": Noul(instructions="Este pedido precisa de resposta hoje?"),
    },
)
print(response.choices["team"].choice, response.nouls["urgent"].noul)

Write the keys in English and the values in Portuguese, as in the training data: the state's field names, the question ids and the option names are code identifiers. KEV_TEMPERATURE=1.0 returns the probabilities before calibration.

Training

Training continues from Jevásio-0.8B and does not start again from Kev. It reads 267,950 of the 335,063 train requests of jevasio-decisions-v2:

• every request of the 36 new sources (196,762);
• every request of the 4 first-version sources that have edited copies, because each original is one side of a pair (38,008);
• a fixed third of the other first-version requests, chosen by a hash of the state, so the model keeps what it learned (33,180).

They are mixed with 12,576 English records of Kev's decision-v7 suite as replay. Records with a state over 2,048 tokens were dropped, which left 277,880 of 280,526 records.

Epochs	1
Learning rate	2e-05
Effective batch	64 (8 × 8 accumulation)
Precision	bf16 autocast, gradient checkpointing
Kev options	--p_none 0 --p_none_distract 0 --p_distract 0 (no English "none of the above" options or distractors), --length_sort 1
Optimizer steps	4,342
Wall time	4.3 hours on one RTX 4090

The temperature is fitted on Portuguese questions from both sides of training: the calib split, from the sources the model trained on, and the test questions of the evaluation-only sources, which it never saw. A fit on calib alone leaves the model overconfident on new tasks. A fit on the unseen sources alone leaves it underconfident on the trained ones. To keep the unseen figure honest, each evaluation-only source is left out in turn and scored at the temperature fitted without it:

	ECE, temperature from calib only (T = 1.15)	ECE, combined temperature
Unseen sources, each left out of its own fit	0.153	0.130
Trained sources, test split	0.012	0.023

scripts/refit_temperature.py in the kev-pt code does the fit, and Kev's scripts/calibrate_checkpoint.py --temperature writes it into head.pt.

Evaluation

The test split of jevasio-decisions-v2 holds 37,972 requests with 46,661 questions. Every request was scored, for this model and for Jevásio-0.8B, each at its own temperature. Nine sources are evaluation-only, so neither model trained on them: alba_mcq, alba_rating, cultura_viva, harmbench, porvid_gold, pt_exams, pt_text_completion, sst, xstest. English transfer is Kev's transfer-v4 test suite.

	Jevásio-0.8B	Jevásio-0.8B-V2
Accuracy, pt-PT test	0.660	0.775
ECE, pt-PT test	0.040	0.016
Brier, pt-PT test	0.441	0.304
Accuracy, English transfer-v4	0.660	0.637
ECE, English transfer-v4	0.047	0.106

Accuracy by kind of source, weighted by questions:

Sources	Questions	Jevásio-0.8B	Jevásio-0.8B-V2
New in V2, edited copies included	29,372	0.630	0.816
First-version sources it trained on	11,536	0.835	0.827
Evaluation-only, never trained	5,972	0.477	0.479

Per source, on the pt-PT test split. The ECE of the evaluation-only sources is partly in-sample, because their questions are in the temperature fit; Training gives the left-out figures.

Source	Questions	Stock acc	Tuned acc	Stock ECE	Tuned ECE
alba_mcq	237	0.481	0.481	0.060	0.123
alba_rating	719	0.323	0.328	0.226	0.281
assin2	2,000	0.539	0.749	0.112	0.016
assin_ptbr	2,000	0.449	0.771	0.114	0.041
assin_ptpt	2,000	0.451	0.742	0.093	0.053
bbh_pt	144	0.479	0.458	0.068	0.103
boolq	868	0.782	0.820	0.098	0.043
cb	13	0.692	0.846	0.185	0.156
copa	33	0.758	0.758	0.086	0.125
corege_fos	1,000	0.836	0.830	0.113	0.043
csqa_pt	421	0.653	0.646	0.104	0.027
cultura_viva	2,463	0.387	0.391	0.071	0.126
dpo_pt	1,000	0.970	0.972	0.019	0.014
dsl_tl	147	0.735	0.707	0.073	0.172
egsciqa	1,000	0.627	0.648	0.094	0.066
fortuna	279	0.735	0.753	0.073	0.016
frmt	4	0.750	0.750	0.200	0.272
harmbench	300	0.643	0.707	0.090	0.068
hatebr	344	0.692	0.866	0.053	0.060
massive	1,994	0.736	0.876	0.074	0.057
mrpc	142	0.732	0.775	0.107	0.046
olid_br	2,533	0.479	0.760	0.180	0.082
openbookqa_pt	478	0.621	0.619	0.066	0.057
parliament	272	0.375	0.496	0.167	0.049
piqa_pt	542	0.668	0.688	0.022	0.018
porvid_gold	219	0.804	0.886	0.075	0.053
porvid_journalistic	826	0.976	0.889	0.173	0.010
porvid_legal	917	0.692	0.698	0.036	0.233
porvid_literature	1,000	0.568	0.861	0.029	0.029
porvid_politics	985	0.936	0.975	0.157	0.014
porvid_social_media	697	0.654	0.623	0.056	0.249
porvid_web	996	0.981	0.959	0.120	0.019
pt_culture	1,000	0.945	0.946	0.018	0.007
pt_exams	1,346	0.516	0.488	0.030	0.071
pt_text_completion	138	0.543	0.536	0.085	0.113
ptradutor	987	1.000	0.973	0.005	0.043
ptradutor_hard	491	0.000	0.888	0.976	0.057
publico	1,812	0.647	0.855	0.068	0.077
puntuguese	1,000	0.543	0.665	0.195	0.079
rewrite_faithful	1,000	1.000	0.987	0.001	0.026
rewrite_faithful_hard	493	0.012	0.970	0.957	0.018
rte	101	0.673	0.812	0.188	0.077
saudade	458	0.773	0.779	0.044	0.032
siqa_pt	685	0.648	0.653	0.041	0.031
sst	100	0.830	0.900	0.080	0.104
sst2	298	0.866	0.872	0.032	0.034
stj	1,000	0.972	0.996	0.168	0.011
stsb	512	0.395	0.637	0.089	0.078
summary_faithful	994	1.000	0.959	0.001	0.016
summary_faithful_hard	508	0.122	0.917	0.817	0.026
synth_offensive	180	0.900	1.000	0.133	0.023
synth_sentiment	130	0.900	0.977	0.099	0.038
synth_support_route	139	0.777	0.892	0.140	0.049
synth_urgency	102	0.716	0.971	0.144	0.049
synth_variety	124	0.815	0.911	0.088	0.039
tierc_everyday_conv	260	0.781	0.796	0.064	0.058
tierc_persona_general	343	0.869	0.880	0.055	0.028
tierc_persona_nemotron	335	0.869	0.863	0.097	0.057
tierc_wikipedia_conversations	319	0.831	0.859	0.105	0.060
toxichat	478	0.975	0.977	0.027	0.027
tupy	2,000	0.802	0.850	0.021	0.083
tweet_sentiment	869	0.601	0.679	0.058	0.107
wildguardmix	305	0.895	0.898	0.064	0.052
winogrande_pt	396	0.606	0.596	0.089	0.133
wmt24pp	497	0.769	0.702	0.030	0.046
wmt24pp_hard	238	0.118	0.899	0.540	0.125
xstest	450	0.727	0.707	0.067	0.071

Limits

• The gain is on the tasks it trained on. On the nine evaluation-only sources, accuracy went from 0.477 to 0.479. Do not expect better answers on a new kind of question.
• Checking a fact costs some faithful answers. On the edited copies, which change one fact, accuracy went from 0.055 to 0.921. On the faithful originals it went from 0.967 to 0.934: the first version called almost every pair faithful.
• One temperature does not fit every task. On a Portuguese task it never saw, the ECE is about 0.130, and on alba_rating, a five-level quality rating, it is 0.300. Before you set a threshold, fit a temperature on labelled data from your own workload (python -m kev.calibrate).
• The variety answer is uneven between the two varieties. On the dsl_tl and FRMT test texts, the model names European Portuguese texts correctly 46 of 49 times, and Brazilian texts 61 of 102 times.
• Portuguese knowledge did not improve. Accuracy on cultura_viva went from 0.387 to 0.391, and on pt_exams from 0.516 to 0.488.
• English accuracy fell, from 0.660 to 0.637 on transfer-v4.
• Part of the data is silver or generated. The variety labels of the porvid sources come from the source of each text, and a blind label by Claude Haiku removed the texts it disagreed with. Claude Haiku wrote the edited copies. Claude Sonnet wrote the generated messages, and a blind label by Claude Haiku kept only those it agreed with. AMALIA-9B-1225-DPO labelled the conversation questions (tierc). Most knowledge sources are machine translations.
• This is not a model for decisions with legal, medical, financial or employment consequences for people without human review. It does not generate text.

int8 weights

The int8/ folder holds the same model with weight-only int8 backbone weights: one bf16 scale per output row, with the LoRA merged in. It needs about half the memory of bf16. Load it with jevasio_int8.py from this repository:

from jevasio_int8 import load

tok, model = load("TigreGotico/jevasio-0.8b-v2", "cuda")

On 300 test requests, its accuracy is the same as the bf16 model's, and 3 answers change.

Citation

The first-version data comes from the AMALIA project. Cite it, and Kev:

@inproceedings{simplicio-etal-2026-amalia,
    title = "{AMALIA}: A Fully Open Large Language Model for {E}uropean {P}ortuguese",
    author = "Simpl{\'i}cio, Afonso and Vinagre, Gon{\c{c}}alo and Ramos, Miguel Moura and Tavares, Diogo and Ferreira, Rafael and Attanasio, Giuseppe and Alves, Duarte M. and Calvo, In{\^e}s and others",
    booktitle = "Proceedings of the 17th International Conference on Computational Processing of {P}ortuguese ({PROPOR} 2026) - Vol. 1",
    year = "2026",
    address = "Salvador, Brazil",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2026.propor-1.38/",
    pages = "380--391"
}