Vinci-MLE-123B-1.0

Bring ML-engineering specialization to Devstral 2: open weights trained to inspect evidence, make bounded changes, and recognize when no change—or more evidence—is the right next step.

Canadian-developed by SimpleDirect · Mistral AI Modified MIT License · Research release

Vinci-MLE-123B-1.0 is a fine-tune of Mistral AI's Devstral 2 123B Instruct, developed by SimpleDirect, a Canadian AI lab. MLE stands for machine-learning engineering: inspecting experiment artifacts, diagnosing problems, making bounded changes, and reporting what the evidence supports. It is the largest of three Vinci MLE 1.0 sizes; the 8B and 30B models are built on IBM Granite and use a different licence.

Built around the ML engineering workflow

• Evidence before intervention. Training covers experiment artifacts, tool feedback, diagnosis, and structured reports across eight ML-engineering task families.
• Act, preserve, or investigate further. The specialization includes justified no-change and stop decisions alongside bounded repairs.
• Run in your own environment. Standalone merged BF16 weights include the tokenizer and chat template; connect them to your application’s tools and workspace.

Measured results: BFCL tool-calling performance falls within the preregistered retention margin relative to the Devstral parent. Paired GPQA-Diamond, GSM8K, AIME, MATH-500 and IFEval results, including both positive and negative differences, appear below. On MATH-500, this release scores below its parent (-4.80 pp; 95% interval [-7.80, -2.00], which excludes zero). These comparisons characterize the release; they do not establish ML-engineering task performance.

Evaluation scope: held-out MLE-task and LiveCodeBench coding results are not available for 123B 1.0. See the evaluation results and coverage table for the measured scope.

GGUF downloads — from 74.9 GB · Vinci MLE collection

Licence: Mistral AI’s Modified MIT License, including its revenue condition. The 8B/30B siblings have different terms; read Licence before use.

Specialization · MLE evaluation status · Quick start · General capability retention · Limitations · Licence

Choose your format

Distribution	Weight download	Documented runtime configuration
BF16 Safetensors	250.05 GB	vLLM; two 141 GB GPUs
GGUF Q4_K_M	74.9 GB	llama.cpp; one NVIDIA H200
GGUF Q5_K_M	88.3 GB	llama.cpp; one NVIDIA H200
GGUF Q8_0	132.9 GB	llama.cpp; two NVIDIA H200s

Choose BF16 for the Transformers-format weights used in the reported model evaluations, or GGUF for the documented llama.cpp workflow and smaller downloads. These are tested configurations, not minimum-memory requirements. Allow additional memory for context and runtime overhead. BF16 benchmark results do not establish GGUF capability retention.

Model at a glance

Property	Value
Developer	SimpleDirect, Canada
Parent	mistralai/Devstral-2-123B-Instruct-2512
Architecture	Ministral3ForCausalLM, 88 layers
Parameters	125,025,988,608 stored (published by Mistral AI as a 123B model)
Format	Full merged BF16 Safetensors; 27 weight shards
Weight-file size	250,052,071,424 bytes; approximately 250.05 GB / 232.88 GiB
Specialization	Supervised fine-tuning with LoRA, then deterministic merge
Training corpus	91 trajectories; 64 distinct task roots; 8 task families
Training / evaluation serving limit	16,384 tokens
Parent-configured position limit	262,144 tokens; not a validated MLE context-length claim
Evaluation serving runtime	vLLM 0.27.1; BF16; tensor parallel 2; mistral tool parser
License	Mistral AI Modified MIT License, with NOTICE

Mistral AI publishes the parent in FP8. Vinci trained and merged against a local BF16 conversion of those weights, not the native FP8 files; see Provenance. Weight-file size is not a runtime-memory requirement: allow additional memory for the KV cache, activations, runtime, context length, and concurrent requests.

ML engineering specialization

Vinci MLE is trained as a policy inside an instrumented ML workspace, not only on question-and-answer pairs. Its specialization examples combine file inspection, tool feedback, evidence-based diagnosis, bounded interventions, and structured reports of observations, changes, unknowns, and next evidence.

The frozen specialization corpus contains 91 trajectories across 64 distinct task roots in eight task families, the same trajectories used for the 8B and 30B models. The 123B model consumed its own tokenization of them (Devstral tokenizer), with its own LoRA targets and optimization recipe. The family names below follow the training manifest, with punctuation expanded for readability.

ML-engineering task family	Training scope
Duplicate data / data quality	Inspect duplication and data-quality problems
Experiment design / interpretation	Interpret experiment evidence and assess whether a change is justified
Failed resume	Diagnose failed training-resume behavior
Memory / batching / accumulation	Inspect memory, batching, and gradient-accumulation problems
Metric / evaluator defect	Diagnose defects in metrics and evaluators
NaN / divergence	Investigate numerical failures and divergence
Packaging / serving / reproducibility	Inspect packaging, serving, and reproducibility problems
Tokenizer / model configuration	Diagnose tokenizer and model-configuration problems

Act / No Change / Stop is central to the training design. An ML-engineering policy should not generate changes indiscriminately: preserving a correct setup or requesting missing evidence can be the appropriate response.

Trained response	Meaning in the specialization corpus	Trajectories
Act	Make a bounded change when the evidence supports a defect and an intervention	45
No Change	Preserve a correct setup and explain the evidence	23
Stop / Needs Evidence	Identify missing evidence or a blocker instead of inventing a repair	23

These counts describe training coverage, not success rates on new tasks.

MLE evaluation status

Vinci MLE 1.0 does not yet have a valid held-out ML-engineering task score.

The release documents its training design and measured general-capability comparisons. Held-out MLE and external MLE-bench evaluations were not completed for 1.0, so 1.0 does not claim measured improvement on ML-engineering tasks. Those evaluations require a qualified execution environment with the intended isolation and reproducibility controls; that environment was not ready for 1.0.

For Vinci MLE 1.1, we intend to run the held-out MLE evaluation and external MLE benchmark suite as release-gating measurements, alongside general capability-retention evaluations. Results will be reported when measured; this 1.0 card does not predict them or promise a release date.

Intended workflow

Illustration of training intent; tools are supplied by the application. This is not a captured model run.

A task supplies a problem statement, workspace files, and tool feedback. The policy inspects evidence, chooses an action or a justified no-change/stop response, and produces a structured final report. Applications must implement a bounded interaction loop, record tool outputs, and independently test proposed changes.

The weights alone are not the full Vinci MLE agent harness. This repository supplies the model and tokenizer. An application must provide and validate its own tools, sandbox, permissions, and workspace.

Tool schema and structured-report format

The workspace interface uses task-specific subsets of inspect_files, run_command (an argv array), and write_file (path and content). All 91 include run_command; 36 expose all three tools. Final reports use a fenced json FINAL block with decision, root_cause, evidence, changes, facts, unknowns, why_blocks, and next_evidence fields. Reproducing that workflow requires the matching tool schema and system instructions; a generic chat prompt is a different use of the model.

The stored final-report decision values are INTERVENED for Act, NO_CHANGE for No Change, and STOP for Stop / Needs Evidence.

Choosing a size

Choose 123B to explore the Vinci MLE training design on Devstral 2, with the infrastructure to serve its approximately 250 GB BF16 weights. Choose 8B or 30B for smaller weight footprints and their Apache-2.0 licence. All three use the same underlying specialization trajectories, with different parents or recipes. There is no measured held-out MLE-task ranking between them; size alone does not establish a capability advantage.

GGUF downloads are available in Q4_K_M, Q5_K_M, and Q8_0, starting at 74.9 GB. Visit the GGUF card for file sizes, verified download instructions, and the tested runtime configurations.

Quick start

Download

Install huggingface_hub, and authenticate with hf auth login if repository access requires it. The weights are about 250.05 GB. The example pins the verified model snapshot; later card-only revisions do not change those model files. This snapshot includes its historical README and matching checksums.

from huggingface_hub import snapshot_download

model_dir = snapshot_download(
    repo_id="simpledirect/Vinci-MLE-123B-1.0",
    revision="53c633850c711f8e1acbbccfc3e815eca12b4645",
    local_dir="./vinci-mle-123b",
)
print(model_dir)

Serve an OpenAI-compatible endpoint with vLLM

The release evaluation used vLLM 0.27.1 on two GPUs (tensor parallel 2), a 16,384-token limit, greedy decoding, the packaged chat template, and the mistral tool-call parser. After downloading the snapshot above, the following launch pattern preserves those core model settings; the local port and served alias are chosen for this example.

CUDA_VISIBLE_DEVICES=0,1 \
HF_HUB_OFFLINE=1 TRANSFORMERS_OFFLINE=1 \
python -m vllm.entrypoints.openai.api_server \
  --model ./vinci-mle-123b \
  --tokenizer ./vinci-mle-123b \
  --tokenizer-mode hf --config-format hf --load-format safetensors \
  --chat-template ./vinci-mle-123b/chat_template.jinja \
  --served-model-name vinci-mle-123b \
  --dtype bfloat16 \
  --tensor-parallel-size 2 \
  --max-model-len 16384 \
  --gpu-memory-utilization 0.93 \
  --seed 0 \
  --enforce-eager \
  --enable-auto-tool-choice \
  --tool-call-parser mistral \
  --host 127.0.0.1 --port 8000

The evaluation additionally served one sequence at a time (--max-num-seqs 1 --max-num-batched-tokens 1024 --enable-chunked-prefill --no-enable-prefix-caching). Tensor parallel 2 describes the evaluated setup on 141 GB GPUs; this card does not certify a minimum GPU model or all alternative parallel configurations.

Send requests to http://127.0.0.1:8000/v1/chat/completions, using model: "vinci-mle-123b", temperature: 0, and an explicit max_tokens budget. For tool use, supply the tools your application implements and validate every returned name and argument. The vLLM tool-calling documentation explains the API contract; the parser setting alone does not provide a tool executor.

123B tool-call smoke: all 12 cases passed in the fixed smoke run on the frozen release. This checks a small set of output-format cases; it does not establish reliable tool use on arbitrary tasks.

Send a first request

Once the server is ready, run this in a second terminal:

curl --fail-with-body http://127.0.0.1:8000/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
  "model": "vinci-mle-123b",
  "messages": [
    {
      "role": "user",
      "content": "A dataset contains multiple records per customer. Training and validation rows were split at random. What should I inspect before trusting the validation score? Separate observations from hypotheses and do not claim to have run checks."
    }
  ],
  "temperature": 0,
  "max_tokens": 512
}'

This is an illustrative text-only prompt, not a captured model result or benchmark. It does not execute tools or reproduce the training harness. Review the response before using it in an experiment.

This request block has not been run literally against the released BF16 weights. The recorded attempt was stopped by a host memory safeguard before the server started, and it was not retried. The GGUF release's equivalent quick-start was run literally.

General capability retention

The evaluations below compare this release with its pinned Devstral parent. They are not substitutes for a held-out ML-engineering benchmark. Only the tool-calling comparison has a preregistered retention pass criterion; the other measurements are descriptive and retain their individual caveats. These are measurements of the BF16 merged release, not of a quantized conversion.

Tool calling and coding

BFCL measures tool-call retention. The preregistered criterion permits a score drop of at most 3 percentage points from the parent under the same harness. Scores below are on a 0–100 scale; the difference is this model minus its parent, in percentage points.

Benchmark	Devstral 123B parent	Vinci MLE 123B	Difference (pp)	Retention margin
BFCL	69.4064	69.0868	-0.3196	Within margin
LiveCodeBench	Not measured	Not measured	—	No verdict

BFCL covers 11 categories and excludes the executable-code category. BFCL multi-turn calls execute in the benchmark's in-process simulated Python environments, without a separate sandbox, as in the upstream benchmark design.

Coding evaluation was not completed. No score is reported or inferred.

General knowledge and reasoning

Descriptive; no multiplicity correction; not a retention verdict. These paired comparisons have no preregistered pass rule. Scores and intervals are shown as percentages / percentage points, converted from the recorded fractions. The interval is the paired bootstrap 95% interval for the difference, not an interval around either model's score.

Benchmark	Items	Parent (%)	Vinci MLE (%)	Difference (pp)	95% interval for difference (pp)
GPQA-Diamond	198	57.07	63.64	+6.57	[+1.01, +12.12]
GSM8K	1,319	96.06	95.53	-0.53	[-1.36, +0.30]
AIME 2024 + 2025	60	28.33	31.67	+3.33	[-5.00, +13.33]
MATH-500	500	81.20	76.40	-4.80	[-7.80, -2.00]
IFEval	541	78.37	78.74	+0.37	[-1.85, +2.59]

GPQA-Diamond uses exact match with its locked flexible-extract rule; GSM8K uses exact match with its locked strict-match rule. The 8B and 30B comparisons also covered AIME, BBH, IFEval, MATH-500 and MMLU-Pro. For 123B, BBH and MMLU-Pro are not measured in the release record used here. MATH-500 uses math_verify; IFEval uses strict prompt-level accuracy. On MATH-500, this release scores below its parent (-4.80 pp; 95% interval [-7.80, -2.00], which excludes zero). On IFEval, the difference from the parent is positive but its interval includes zero (+0.37 pp; 95% interval [-1.85, +2.59]). AIME pools AIME 2024 and AIME 2025 (30 + 30 = 60 items) and uses exact match with no filter, 0-shot, with up to 12,288 generated tokens; with only 60 items, its intervals are wide by construction. Compare only runs using the same task definitions, prompts, and scoring configuration. An interval excluding zero in this descriptive table is not a multiplicity-adjusted superiority claim.

Safety: refusal-pattern measurements

Descriptive; no multiplicity correction; not a retention verdict. A9 compares refusal patterns between Vinci MLE and its pinned parent (Devstral-2-123B-Instruct-2512) on XSTest and StrongREJECT using two registered refusal-phrase windows: the start of the normalized response and the first 300 normalized characters.

Classifier audit. In a blind review of 90 responses per model by independent AI reviewers, with model identity and classifier output hidden, neither classifier window labelled a compliant answer as a refusal in the audited sample. The start-of-response window identified 25/60 parent and 24/59 Vinci MLE refusals identified by the first reviewer; the first-300-character window identified 30/60 and 28/59, respectively. Detection was comparable between the parent and Vinci MLE, so the paired differences below are like-for-like; because many refusals go undetected, absolute detected rates are not reported.

Detected refusal-pattern differences — start of response

Prompt set	Paired items	Difference (pp)	95% interval for difference (pp)
XSTest safe	250	+0.80	[+0.00, +2.00]
XSTest unsafe	200	+0.00	[-2.50, +2.50]
StrongREJECT	313	-1.92	[-4.15, +0.32]

First-300-character window

Prompt set	Paired items	Difference (pp)	95% interval for difference (pp)
XSTest safe	250	+0.00	[-1.60, +1.60]
XSTest unsafe	200	+0.00	[-2.50, +2.50]
StrongREJECT	313	-1.60	[-3.83, +0.64]

On StrongREJECT, Vinci MLE shows fewer detected refusals than its parent in both windows; the intervals include zero.

On safe XSTest prompts, a higher detected-refusal rate can indicate increased over-refusal. On unsafe prompts and StrongREJECT, refusal-phrase detection is only a behavioral proxy: it does not by itself establish whether the complete response is safe or harmful.

These measurements characterize refusal-pattern differences relative to the parent model; they are not a general safety evaluation.

Evaluation scope: what was not run

Held-out MLE and coding evaluation were not completed for 1.0. The detailed coverage and release record follow.

Evaluation coverage and release record

Evaluation	1.0 status for 123B
Sealed internal MLE evaluation	Not run
48-task held-out MLE evaluation	Not run
MLE-bench release subset and MLE-bench Lite	Not run
LiveCodeBench coding retention	Not measured (see above)
BBH, MMLU-Pro	Not measured
Agentic evaluation (tau2)	Not run
Long-context synthetic retrieval (RULER)	Not run
Repeated-run variance studies	Not run
Behavioral fidelity of merged release versus adapter	Not measured

This research release includes the measured components above. The overall tool-calling-and-coding retention criterion remains unevaluated because its coding component is missing. Release proceeded under a documented exception with these evaluation gaps disclosed.

The full preregistered evaluation protocol was not completed for 1.0. Component results do not constitute a completed release-evaluation result, and no incomplete gate is reported as passed.

Training

Vinci applied supervised fine-tuning to a local BF16 conversion of the pinned Devstral parent using ordinary LoRA, then merged the terminal adapter into those weights for distribution. This recipe uses neither DoRA nor rank-stabilized LoRA.

Training recipe and corpus statistics

Setting	Value
Examples / distinct task roots	91 / 64
Trajectory composition	45 direct action / 23 no change / 23 stop
Task families	8, with 8 distinct roots each
Epochs / optimizer steps	1 / 23
Selected checkpoint	Terminal checkpoint-23; the sole final candidate
LoRA rank / alpha / dropout	32 / 64 / 0.0
Target modules	q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
Trainable adapter parameters	559,415,296
Learning rate	5 × 10⁻⁶
Schedule / warmup ratio	Cosine / 0.03
Weight decay	0.0
Microbatch / gradient accumulation	1 / 4
Sequence ceiling	16,384 tokens; no packing or truncation
Training seed	18427
Tokenized corpus	700,791 total tokens; 59,336 supervised tokens (Devstral tokenizer)
Longest training example	14,405 tokens

These receipts describe the specialization data, not the Devstral parent's pretraining corpus. The token counts differ from the 8B/30B models because the tokenizer differs; the underlying trajectories are the same.

Training record and checkpoint admission

The trainer completed its final optimizer step and then failed while writing its run receipt, so no trainer-generated receipt exists. The terminal checkpoint was admitted by a documented release decision on the basis of a reconstruction from durable evidence, independently reviewed before admission. There was no rerun and no quality result was used to choose it. Some run-time details that the receipt would have recorded are permanently unrecorded.

Selective assistant supervision. The loss includes designated assistant reasoning, successful tool calls, final answers, and justified-stop answers. System/user context, tool results, scaffolding, and failed tool actions are masked. Verifier feedback is excluded from the rendered prompt. This describes the supervision scheme; it does not establish that the resulting model always chooses a correct action or stop decision.

Limitations

• No completed MLE-specific evaluation. General tool-calling and reasoning scores cannot fill that gap.
• Lower MATH-500 score than the parent. On MATH-500 this release scores 4.80 percentage points below its Devstral parent (76.40% versus 81.20%; 95% interval for the difference [-7.80, -2.00] pp). If mathematical problem solving matters for your use, compare against the parent before choosing this release.
• Fewer measured comparisons than 8B and 30B. No coding (LiveCodeBench), agentic, long-context or variance result exists for 123B; only the general-capability suites shown above were measured.
• Small specialization corpus. One epoch over 91 trajectories does not establish broad coverage of ML frameworks, datasets, experiment types, or failure modes.
• Output and tool errors remain possible. Validate final-report fields, tool names, arguments, proposed code, and observed effects.
• No trainer run receipt. See Training.
• Merge integrity is not behavioral equivalence. The release export passed mechanical verification and standalone loading. Behavioral fidelity to the original adapter was not measured.
• Parent representation. Training and evaluation used a local BF16 conversion of the FP8 parent, not the upstream files byte-for-byte.
• Context scope is bounded. The 262,144-position parent configuration is not proof of reliable MLE behavior at that length; training and evaluation used a 16,384-token limit.
• Serving footprint. The evaluated setup used two 141 GB GPUs at one sequence at a time. Greedy decoding under vLLM is not bitwise deterministic, so reruns can produce different item-level outputs.
• No safety certification or unattended-deployment validation. The refusal-pattern measurements above do not establish a safety property. Use human review and an isolated, permission-limited environment for executable actions.

Contamination audit

The training-data audit covered 19 benchmark/subset entries. It recorded zero primary-field 13-gram matches and zero substring matches, with one MMLU-Pro secondary-field 13-gram match. Primary-field 8-gram matches were AIME 2025 (1), BFCL simple_java (1), GPQA (1), MATH-500 (1), and MMLU-Pro (4). The upstream source texts used to construct trajectories were not audited, and LiveCodeBench was outside the audit's scope. These checks do not establish a contamination-free evaluation.

Provenance and file integrity

The repository includes PROVENANCE.json, SHA256SUMS, LICENSE, and NOTICE. The provenance file identifies the pinned parent, training manifest, export, and release artifact. Load the merged weights directly and keep their matching configuration, tokenizer, and template together.

Identity	Value
Parent	mistralai/Devstral-2-123B-Instruct-2512 at revision 1613bf01adb5e1c6fdc196b46e6b173eae75eb4a (FP8)
Training parent	local BF16 conversion of that revision (deterministic-bf16-build-A)
Candidate	mle1-final-123b-g1-a2-release
Frozen release manifest SHA-256	c883335d3d3450b0d434651091d7f77b8925f0bf7da944667eea7016decbf2d2
Frozen model artifact tree SHA-256	6c8a05c28b392083e7e917af0aacbb15d28d22a94e947b44f49a26187980df8f
Training manifest SHA-256	e48f81c217f15234ba6c6b6b033f6ea056019e9ad40ba366efa6cf472606d4a1
Tokenized training examples SHA-256	34b298ef9400a3b18b5bcc2dff055a66362e7607353dacb6de44b6356e697539

In PROVENANCE.json, the files listed as unmodified are byte-identical to the local BF16 conversion. The tokenizer, chat template, generation configuration, params.json and LICENSE are also byte-identical to the upstream repository's files; the model configuration and weight index differ from upstream because of the BF16 conversion.

After downloading one complete revision, run this from its directory:

sha256sum -c SHA256SUMS
# macOS alternative:
shasum -a 256 -c SHA256SUMS

Checksums detect differences against the recorded reference; they do not independently authenticate its publisher. This repository does not include a detached MLE release signature.

Evaluation receipt identifiers

These hashes identify the internal records used for the tables. The full evaluator environment and per-item receipts are not distributed in this model repository, so hashes alone are not a public reproduction package.

Record	SHA-256
BFCL paired receipt	910551387116d486a819486ac9ca99e18c5179a213d5ca582a259efdd6c76791
General-capability paired record (GPQA-Diamond)	e5eb56ca275c564880f4ff840222210d8048b1cc77ac5b9dd2da4f75e62c824b
General-capability paired record (GSM8K)	9be8fe3bc3e42509c5533dc62b9a621775b0363f690e7479c3c0561eb29328da
General-capability paired record (AIME)	207e17c6d77548e4ded5077bfe3be212615b8f15af82c64a716b4c41ec3f65c4
General-capability paired record (MATH-500 and IFEval)	e6ca4f8083efeb6d6230b7c9a42da9244bfa442c3b998858b4e65d25e6083d0d
Card numbers receipt	3019b3901321e098672a3d2e5e1072960f7fcc8ce6cba140a73c387cf05dd229
Card numbers receipt (AIME)	5e3c97a119934cede119fad88e8c9667b33f14f37d7d6d4beda85d8d648cca81
Card numbers receipt (MATH-500 and IFEval)	80f434bcb34eae13358809d461df67f085043cdd76a5f6ee7e82f94911517e9e

Licence

This model is not Apache-2.0. Vinci MLE 8B and 30B are built on IBM Granite and use Apache-2.0. This 123B model is built on Devstral 2, so it carries Mistral AI's Modified MIT License (attribution notice: 2025 – Mistral AI). The full text is in LICENSE.

The revenue restriction. The licence does not grant any rights if the global consolidated monthly revenue of your company, or of your employer, exceeded $20 million (or the equivalent in another currency) in the preceding month. The licence applies this restriction to the model and to any derivatives, modifications or combined works built on it, including this model.

• The test is the preceding month's actual consolidated revenue. It is not annual revenue divided by twelve, and it is not profit or revenue from the model. An organization under $240 million a year can still exceed $20 million in a single month.
• If the restriction applies to you, or you are unsure, contact Mistral AI at sales@mistral.ai. A commercial licence is at Mistral AI's discretion.
• If you redistribute the model, including quantized copies, keep the attribution and permission notice. The model is provided without warranty.

The licence covers the model weights and associated documentation. Vinci's private training data is not included or licensed for redistribution. Mistral AI has not reviewed or endorsed this model; see NOTICE. This summary is not legal advice. The LICENSE file governs.

Citation

@misc{vinci_mle_1_0_123b,
  title  = {Vinci MLE 1.0 (123B)},
  author = {SimpleDirect},
  year   = {2026},
  url    = {https://huggingface.co/simpledirect/Vinci-MLE-123B-1.0}
}

Copyright 2026 SimpleDirect.

SimpleDirect · Vinci models