| --- |
| license: apache-2.0 |
| language: |
| - en |
| base_model: |
| - deepseek-ai/DeepSeek-V4-Flash |
| --- |
| # SLAI T-Rex-Flash (DeepSeek-V4-Flash-OR) |
|
|
| SLAI T-Rex-Flash is an Operations Research (OR)–specialized model built from DeepSeek-V4-Flash through full-parameter continued pre-training (CPT) followed by supervised fine-tuning (SFT). It is designed for solver-oriented mathematical modeling, including formulation, Gurobi program generation, feasibility-oriented tasks, and structure-sensitive optimization modeling. |
|
|
| - Paper: [SLAI T-Rex: Full-Parameter Post-training of the DeepSeek-V4 Family on Ascend SuperPOD](https://arxiv.org/abs/2607.20145) |
| - Paper PDF: [arXiv:2607.20145](https://arxiv.org/pdf/2607.20145) |
| - Code and training recipes: [SLAI-AITP/SLAI-T-Rex](https://github.com/SLAI-AITP/SLAI-T-Rex) |
|
|
| ## Model Summary |
|
|
| | Item | Description | |
| | --- | --- | |
| | Model name | SLAI T-Rex-Flash | |
| | Base model | DeepSeek-V4-Flash | |
| | Domain | Operations Research and mathematical programming | |
| | Training | Full-parameter OR-oriented CPT followed by Clean-CoT SFT | |
| | Primary output | Mathematical formulation and/or solver-facing Python code | |
| | Training hardware | Ascend CloudMatrix384 SuperPOD with Ascend 910C NPUs | |
|
|
| The CPT stage supplies OR terminology, formulation patterns, solver APIs, and structural priors. The SFT stage aligns these capabilities with task instructions, output contracts, mathematical explanations, and executable Gurobi-style implementations. The reported SFT recipe uses an 8,192-token sequence length, AdamW, a global batch size of 128, and full-parameter optimization. |
|
|
| ## Download |
|
|
| Install ModelScope and download the checkpoint: |
|
|
| ```bash |
| python3 -m pip install -U modelscope |
| ``` |
|
|
| ```python |
| from modelscope import snapshot_download |
| |
| model_dir = snapshot_download("SLAIAITP/DeepSeek-V4-Flash-OR") |
| print(model_dir) |
| ``` |
|
|
| The checkpoint should be served with a runtime that supports the DeepSeek-V4 architecture and the checkpoint format in this repository. Distributed serving flags depend on the target hardware, runtime version, and available memory. Check the repository configuration files before deployment. |
|
|
| ## Intended Use |
|
|
| SLAI T-Rex-Flash is intended for research and development involving: |
|
|
| - translating natural-language OR problems into mathematical programs; |
| - identifying variables, objectives, and constraints; |
| - generating Gurobi-style Python implementations; |
| - producing solver-compatible models under strict output contracts; |
| - studying domain-adaptive CPT and SFT for structured reasoning. |
|
|
| A typical prompt should state the optimization problem, all required data, the expected solver behavior, and the output contract. For tagged modeling tasks, the training data commonly use `<think>`, `<model>`, and `<python>` sections. Code-only and LP-writing tasks use their task-specific contracts instead. |
|
|
| ## Evaluation |
|
|
| The following results are reported in the technical report under zero-shot Pass@1 evaluation. `Overall` is the unweighted mean of the four OR benchmarks. |
|
|
| | Model | NL4OPT | OptiBench | B4O-Feasible | B4O-ORGEval | Overall | |
| | --- | ---: | ---: | ---: | ---: | ---: | |
| | DeepSeek-V4-Flash | 84.08 | 63.33 | 60.47 | 34.26 | 60.54 | |
| | SLAI T-Rex-Flash | **89.52** | **67.12** | **71.22** | **59.39** | **71.81** | |
|
|
| These results correspond to the complete CPT+SFT pipeline. The reported overall score improves by 11.27 percentage points over the original DeepSeek-V4-Flash checkpoint. The largest absolute gain is on B4O-ORGEval, which evaluates structural equivalence rather than execution success alone. |
|
|
| The report also evaluates general-capability retention: |
|
|
| | Benchmark | SLAI T-Rex-Flash | Base | |
| | --- | ---: | ---: | |
| | MMLU, 5-shot | 88.5 | 87.6 | |
| | MMLU-Pro, 5-shot | 69.2 | 71.0 | |
| | CMMLU, 5-shot | 92.4 | 92.1 | |
| | HumanEval, 0-shot | 68.8 | 69.4 | |
| | GSM8K, 8-shot | 90.3 | 89.8 | |
| | MATH, 4-shot | 56.7 | 58.4 | |
|
|
| Evaluation numbers should be compared only under the same prompt templates, decoding budgets, benchmark versions, solver environment, and scoring implementation described in the paper. |
|
|
| ## Citation |
|
|
| If you use this model, please cite the technical report: |
|
|
| ```bibtex |
| @misc{li2026slaitrexfullparameterposttraining, |
| title = {SLAI T-Rex: Full-Parameter Post-training of the DeepSeek-V4 Family on Ascend SuperPOD}, |
| author = {Dongfang Li and others}, |
| year = {2026}, |
| eprint = {2607.20145}, |
| archivePrefix= {arXiv}, |
| url = {https://arxiv.org/abs/2607.20145} |
| } |
| ``` |
