yannelli/gliner2.5-decide-code-review-decisions

GLiNER2.5-Decide Code Review Decisions

A decision model that reviews JavaScript and TypeScript code. Give it a snippet and the task the snippet implements, and it answers 13 typed questions with probabilities: which of 10 code review flaws the code has, whether it has any flaw, which flaw matters most, and how mergeable it is. Each question is a single-label GLiNER2 classification over the question's options, so one snippet is 13 encoder sequences, read in one batch.

It is fastino/GLiNER2.5-Decide fine-tuned on JS/TS Code Review Decisions. On that dataset's test split it scores macro F1 0.687 and accuracy 0.863 over 15,600 decisions.

Base model	fastino/GLiNER2.5-Decide, GLiNER2 DeBERTa-v3 encoder with a classification head
Parameters	486M (fp32, 1.9 GB)
Input	A state with task, language and code, and named questions
Output	P(yes) for yes/no questions; a distribution over options for choice and score questions
Training data	JS/TS Code Review Decisions, decisions config, train split (9,600 snippets)
Test results	Macro F1 0.687 (cutoff 0.5), accuracy 0.863, ECE 0.039
License	Apache 2.0

Questions

The model answers the 13 questions of the dataset's decisions config. They are the same for every snippet and are stored in the dataset's manifest.json. Each question's criteria carry the flaw definition the labels follow.

Question	Type	Asks	Answer
unclear_naming	noul	Does this code have unclear naming?	P(yes)
noise_comments	noul	Does this code have noise comments?	P(yes)
dead_code	noul	Does this code have dead code?	P(yes)
inconsistent_style	noul	Does this code have inconsistent style?	P(yes)
over_abstracted	noul	Does this code have over-abstraction?	P(yes)
too_clever	noul	Does this code have overly clever code?	P(yes)
duplicated_logic	noul	Does this code have duplicated logic?	P(yes)
error_handling_problem	noul	Does this code have an error handling problem?	P(yes)
type_safety_hole	noul	Does this code have a type safety hole?	P(yes)
unsafe_input	noul	Does this code have unsafe input handling?	P(yes)
has_flaw	noul	Does this code have at least one code review flaw?	P(yes)
primary_flaw	choice	Which code review problem matters most in this code? Choose none if the code is clean.	Distribution over none and the 10 flaws
quality	score	How mergeable is this code as written?	Distribution over 5 levels, 0 "Should not merge." to 4 "Exemplary. Merge as is."

Quality level i is quality i + 1 in the dataset's default config.

Usage

pip install torch==2.14.1 gliner2==2.1.0 huggingface_hub pyarrow

import json
import sys

import torch
from huggingface_hub import hf_hub_download, snapshot_download

path = snapshot_download("yannelli/gliner2.5-decide-code-review-decisions")
sys.path.insert(0, f"{path}/training")
from gliner_decide import Prompts, answer_case, load_model

device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
model = load_model(path, device)

# The 13 questions the model was trained on, from the dataset's manifest.json
manifest = hf_hub_download("yannelli/js-ts-code-review-decisions", "manifest.json", repo_type="dataset")
questions = json.load(open(manifest))["questions"]
prompts = Prompts(model.processor, questions, max_len=512)

state = {
    "task": "Define payroll bank account fields and decode a JSON account record, "
            "accepting only a nine-digit routing number and a four-digit account suffix.",
    "language": "TypeScript",
    "code": """import { z } from 'zod';

const bankAccountSchema = z.object({
    routingNumber: z.string().regex(/^\\d{9}$/),
    accountSuffix: z.string().regex(/^\\d{4}$/)
});
export type PayrollBankAccount = z.infer<typeof bankAccountSchema>;
export function decodeBankAccount(serialized: string): PayrollBankAccount {
    const account_payload = JSON.parse(serialized);
    return account_payload;
}""",
}

answers = answer_case(model, prompts, {"state": state, "questions": questions}, device)
print(json.dumps(answers, indent=1))

The example is a test-split snippet the model did not see in training. Its gold labels are inconsistent_style (account_payload next to camelCase names) and unsafe_input (the JSON.parse result is returned without the schema defined above it). This model's answers, from the code above on CPU:

Question	Gold	This model
unclear_naming	no	P(yes) 0.00
noise_comments	no	P(yes) 0.00
dead_code	no	P(yes) 0.12
inconsistent_style	yes	P(yes) 0.06
over_abstracted	no	P(yes) 0.00
too_clever	no	P(yes) 0.01
duplicated_logic	no	P(yes) 0.03
error_handling_problem	no	P(yes) 0.01
type_safety_hole	no	P(yes) 0.02
unsafe_input	yes	P(yes) 0.32
has_flaw	yes	P(yes) 0.38
primary_flaw	unsafe_input	none (0.56)
quality	level 0	level 4 (0.72); expected level 3.39

Answers use the dataset's answers format: a noul answer is P(yes), and choice and score answers carry probabilities over every option. One {"id": ..., "answers": ...} line per snippet scores directly with the dataset's score_decisions.py. The quality answer above, rounded:

{
 "type": "score",
 "probabilities": {
  "0": 0.019,
  "1": 0.069,
  "2": 0.137,
  "3": 0.055,
  "4": 0.721
 }
}

Inference code is in training/gliner_decide.py, which ships with the model. answer_case reads one logit per option and returns full distributions; GLiNER2's classify_text returns only the top label. Set device to mps for Apple Silicon (not benchmarked).

The model accepts other questions in the same schema, but it was trained and evaluated only on these 13.

Results

Results cover the test split of the decisions config: 300 tasks, 1,200 snippets, and 15,600 decisions, scored with the dataset's score_decisions.py. Each snippet was one request with all 13 questions. The four fine-tuned models used the same data and scorer.

Model	Params	Macro F1 @ 0.5	Accuracy	Log loss	Brier	ECE	Clean judged clean
d1-3B	3.1B	0.917	0.940	0.172	0.072	0.017	0.950
d1-omni-600M	587M	0.771	0.882	0.321	0.127	0.028	0.827
GLiNER2.5-Decide (this model)	486M	0.687	0.863	0.398	0.149	0.039	0.683
NanoJev	596M	0.452	0.831	0.459	0.173	0.038	0.000
GLiNER2.5-Decide base model, zero-shot	486M	0.071	0.736	0.786	0.291	0.227	0.043
TF-IDF + logistic regression (dataset baseline)	-	0.502	0.830	0.449	0.172	0.030	0.307

Higher is better for macro F1, accuracy, and clean judged clean; lower is better for log loss, Brier, and ECE. Clean judged clean is the share of the 300 clean test snippets with P(has_flaw) below 0.5. The dataset card's Metrics section defines the other metrics.

95% intervals for this model, from 500 bootstrap resamples of test tasks: macro F1 0.660 to 0.709, accuracy 0.855 to 0.871, log loss 0.376 to 0.419, Brier 0.142 to 0.156. By question type its accuracy is 0.918 on the 11 nouls, 0.593 on primary_flaw, and 0.529 on quality.

Per question

Question	Type	Test positives	Accuracy	F1 @ 0.5	Log loss	ECE
unclear_naming	noul	139	0.983	0.926	0.062	0.010
noise_comments	noul	178	0.989	0.962	0.054	0.006
dead_code	noul	223	0.864	0.432	0.390	0.018
inconsistent_style	noul	147	0.912	0.478	0.293	0.024
over_abstracted	noul	145	0.960	0.831	0.135	0.020
too_clever	noul	164	0.919	0.657	0.244	0.031
duplicated_logic	noul	171	0.887	0.634	0.308	0.071
error_handling_problem	noul	140	0.956	0.780	0.163	0.018
type_safety_hole	noul	148	0.937	0.670	0.232	0.028
unsafe_input	noul	165	0.897	0.496	0.314	0.037
has_flaw	noul	900	0.796	0.860	0.410	0.066
primary_flaw	choice		0.593		1.368	0.102
quality	score		0.529		1.207	0.077

Calibration

Answers are the model's raw softmax (test ECE 0.039). temperatures.json holds one temperature per question, fitted on the validation split by log loss. Replacing each answer's distribution p with softmax(log p / T) moves test log loss from 0.398 to 0.384, Brier from 0.149 to 0.147 and ECE from 0.039 to 0.027. It never changes the top answer or the order of P(yes) values, so F1 and accuracy stay the same.

Speed

Median wall-clock time for one snippet with all 13 questions, batch size 1. The CPU row covers 52 test snippets.

Hardware	Runtime	p50 per snippet
One RTX PRO 6000 (Blackwell)	PyTorch, bf16 autocast	107.7 ms
16 Neoverse V3 CPU cores	PyTorch, fp32	6.0 s

Intended use

Flag code review problems in JavaScript and TypeScript while code is written or before it reaches review: in an editor, a pre-commit hook, or CI, as hints for a human reviewer. The probabilities let you pick a cutoff per flag and trade false positives against misses.

Training

• Data: the train split of the decisions config, 2,400 tasks and 9,600 snippets (124,800 decisions). The state is the task description, language and code. The training files match the SHA-256 hashes in the dataset's published manifest.json.
• The dataset keeps the 4 variants of a task in one split, so no test task was seen in training.

Setting	Value
Method	Full fine-tune of the encoder and classification head; word embeddings frozen
Objective	Cross entropy of the softmax over each question's label logits
Schedule	1 epoch, 3,900 steps of 32 (snippet, question) sequences
Optimizer	AdamW, learning rate 1e-5 (encoder) and 5e-4 (head)
Precision	bf16 autocast over fp32 weights
Hardware	One RTX PRO 6000 (Blackwell), 47 minutes
Validation loss	0.416 at the end of training (300 validation snippets)

training/ holds the scripts that trained and scored this model (gliner_decide.py, common.py, calibrate.py). To rerun it:

pip install "gliner2[train]==2.1.0" pyarrow numpy
hf download yannelli/js-ts-code-review-decisions --repo-type dataset --local-dir js-ts-code-review-decisions
export DECISIONS_DATA=js-ts-code-review-decisions
python training/gliner_decide.py train --ckpt ckpt --epochs 1
python training/gliner_decide.py predict --ckpt ckpt --split test --out test-answers.jsonl
python js-ts-code-review-decisions/score_decisions.py test-answers.jsonl --split test

Evaluation files

eval/test-answers.jsonl holds this model's answers to all 1,200 test snippets in the scorer's format, and eval/score.txt is the scorer's output for it. To rescore:

python js-ts-code-review-decisions/score_decisions.py eval/test-answers.jsonl --split test

Limitations

• Gold labels come from blind model review and a model audit that accepted 98.3% of findings, not from human reviewers. Scores measure agreement with those labels.
• The snippets are synthetic, 5 to 80 lines long, and 25% of them are clean, so false positive rates on real repositories will differ.
• Training covered only JavaScript, JSX, TypeScript and TSX snippets, each with a one-sentence English task; pass one with every snippet.
• Weakest flags by test F1: dead_code (0.43), inconsistent_style (0.48), unsafe_input (0.50). quality is the hardest question (accuracy 0.529); 59% of its wrong answers are one level off.
• unsafe_input follows the dataset's definition (external data used unchecked); it is not a vulnerability scanner.
• The encoder reads at most 512 tokens per (question, snippet) sequence. 656 of the 1,200 test snippets (55%) lose the end of their code in at least one question, and a flaw in the cut tail cannot be seen.

Other models trained on this dataset

Model	Base	Params	Macro F1 @ 0.5	Accuracy
d1-3B	LFM2.5-VL-3B with Liquid AI's d1 decision readout	3.1B	0.917	0.940
d1-omni-600M	LFM2.5 encoder with a decision head	587M	0.771	0.882
GLiNER2.5-Decide (this model)	GLiNER2 DeBERTa-v3 encoder with a classification head	486M	0.687	0.863
NanoJev	Qwen3-0.6B with a choice head	596M	0.452	0.831

yannelli/code-review-decisions-GGUF has GGUF builds of the two Liquid models for llama.cpp and Ollama.

License

Apache 2.0, the license of the base model fastino/GLiNER2.5-Decide.

The training data, JS/TS Code Review Decisions, is CC BY 4.0.

Citation

@misc{yannelli2026glinercodereview,
  title        = {GLiNER2.5-Decide Code Review Decisions},
  author       = {Ryan Yannelli},
  year         = {2026},
  publisher    = {Hugging Face},
  howpublished = {\url{https://huggingface.co/yannelli/gliner2.5-decide-code-review-decisions}}
}

@misc{yannelli2026jstscodereviewdecisions,
  title        = {JS/TS Code Review Decisions},
  author       = {Ryan Yannelli},
  year         = {2026},
  publisher    = {Hugging Face},
  howpublished = {\url{https://huggingface.co/datasets/yannelli/js-ts-code-review-decisions}}
}

Maintainer

Ryan Yannelli | 10/12/2026 | GitHub: @yannelli | Hugging Face: yannelli