aellies/vif-etincelle-beta-v0

Vif Étincelle — beta-v0

Early BETA. This is the first published checkpoint of Vif Étincelle, an open-source, bilingual (FR/EN) decision-making model: instead of generating text, it answers typed decisions — choice (pick one option from a free list), score (rate on ordered levels described in words), and yesno (binary question) — with a probability per candidate and a confidence score.

It is inspired by the typed decisions, not chat category pioneered by TypeSafe AI's Jev (not affiliated, no shared code, weights, or trademark).

Primary intended use: high-level autonomous decision-making in robotics (navigation choices, task prioritization, energy management, anomaly flagging) from sensor-derived state descriptions — never low-level safety control. General classification/agent use is also supported.

What this checkpoint is (and isn't)

This is a v0 beta, trained fast on a modest synthetic dataset to validate the full pipeline end-to-end (data → training → API → publishing) before scaling up. It will be superseded by better versions. Concretely:

• No temperature scaling / calibration applied yet — confidence scores are raw softmax outputs and are likely overconfident (e.g. near-1.0 on easy examples). Do not treat the confidence number as a true probability yet.
• No out-of-distribution detection implemented — the ood_flag in the reference API is currently always false.
• Trained almost entirely on synthetic data (see below), not on human-authored/curated real-world text.

Architecture

• Base encoder: EuroBERT/EuroBERT-210m (Apache 2.0, 210M params, trust_remote_code=True required).
• Cross-encoder scoring: state [SEP] question [SEP] candidate → mean-pooled hidden states → linear head → one scalar logit per candidate. All candidates for one question are scored together and turned into a probability distribution via softmax.
• choice and score use the user-supplied options/levels as candidates. yesno is internally treated as a 2-candidate choice (yes/no, or oui/non when the question is detected as French).
• Trained sequence length: 512 tokens (the base encoder supports up to 8192; not yet exploited).

Training data

3,394 examples (2,716 train / 339 val / 339 test), roughly balanced across English, French, and mixed-language pairs:

• 3,000 robotics examples, generated by a fully deterministic rule-based script (no LLM) covering 6 robot types (mobile, arm, drone, domestic, agricultural, warehouse) and 4 "hard case" categories (near-threshold values, conflicting signals, faulty/outlier sensor readings, no clearly good option). Ground-truth labels are computed by explicit rules, not guessed by a model.
• 394 general-purpose examples (customer support, agent planning, sentiment, scheduling), generated locally with Qwen/Qwen2.5-7B-Instruct (Apache 2.0, open-weight) — no proprietary API output was used anywhere in this pipeline.

Full source/license documentation: see data/SOURCES.md in the project repository (link to be added once the code repo is public).

Evaluation (held-out test set, 339 examples never seen during training)

Metric	Value
Overall accuracy	92.9%
choice / score / yesno	89.9% / 93.0% / 95.5%
English / French / mixed	86.7% / 96.6% / 95.4%
Hard cases (threshold, conflicting signals, faulty sensor, no good option)	88–100%

Trivial majority-class baselines for reference: score 21.7%, yesno 44.1%, choice 34.3% — the model clearly beats chance/majority, this is not an inflated number from an easy dataset.

Known weak spot, disclosed honestly: the customer_support domain (general/LLM-generated data, smaller sample) scores only 43.8% on held-out test — markedly worse than the robotics domains (95–100%). This is most likely due to fewer examples and noisier/less-structured synthetic data for this domain. Expect this model to be noticeably weaker outside the robotics use case it was primarily built for.

No calibration curves (ECE, reliability diagrams) are published yet for this checkpoint — planned for the next version.

Usage

Use decide.py from this repository — it handles the snapshot download, the config.json compatibility shim, the RoPE patch, and device selection (CUDA / Apple Silicon MPS / CPU) automatically:

from decide import load_model, decide

tokenizer, model, device = load_model("aellies/vif-etincelle-beta-v0")
result = decide(
    tokenizer, model, device,
    state="battery: 15% (below low threshold of 25%); obstacle_distance: 300cm",
    question="Should the robot return to base to recharge now?",
    candidates=["yes", "no"],
)
# {"answer": "yes", "confidence": 0.999..., "probabilities": {"yes": 0.999..., "no": 0.0006...}}

A simpler, shorter variant is also available in example_usage.py.

Important: this checkpoint is a custom architecture (EuroBERT encoder + a linear scoring head), not a standard Transformers model class. AutoModel.from_pretrained("aellies/vif-etincelle-beta-v0") will not work correctly on its own — config.json's model_type field is set only so naive tooling doesn't crash outright, it does not make this repo AutoModel-loadable. Always load through decide.py / example_usage.py.

Known compatibility issues

• EuroBERT/EuroBERT-210m's remote code crashes with KeyError: 'default' on transformers>=5 (the ROPE_INIT_FUNCTIONS registry dropped the plain "default" RoPE entry, moved into each native model). decide.py and example_usage.py both include a compatibility patch that restores it.
• On some recent CUDA GPUs (observed on an RTX 5060 Ti, Blackwell/sm_120), the optimized flash/mem-efficient SDPA attention kernels can silently return NaN with some PyTorch builds. decide.py disables them in favor of the slower but reliable "math" kernel on CUDA devices. This does not affect Apple Silicon (MPS) or CPU.

Intended use / out-of-scope use

This model is a probabilistic classifier. It can be confidently wrong, especially given the lack of calibration in this v0. It must not be the sole decision-maker in safety-critical systems: medical, legal/financial decisions affecting people, or direct control of vehicles/machinery.

For robotics, it is designed for high-level autonomous decision-making (what to do, in what order, how to assess a situation) — never for low-level safety. Any real deployment should follow the principle "the model proposes, a deterministic supervisor validates": braking, emergency stop, physical limits, no-go zones, and a watchdog must be handled by independent deterministic code that always has the final say.

License

Apache 2.0 (this checkpoint and associated code). Base encoder (EuroBERT-210m) is also Apache 2.0. Training data sources and licenses are documented separately; no non-commercial-licensed data was used in training this checkpoint.