| --- |
| license: apache-2.0 |
| language: |
| - en |
| base_model: |
| - deepseek-ai/DeepSeek-V4-Pro |
| --- |
| # SLAI T-Rex-Pro |
|
|
| SLAI T-Rex-Pro is the trillion-parameter scale-up of the SLAI T-Rex Operations Research (OR) post-training workflow. Starting from DeepSeek-V4-Pro, it applies full-parameter OR-oriented continued pre-training (CPT) followed by Clean-CoT supervised fine-tuning (SFT) on an optimized Ascend SuperPOD training stack. |
|
|
| - Paper: [SLAI T-Rex: Full-Parameter Post-training of the DeepSeek-V4 Family on Ascend SuperPOD](https://arxiv.org/abs/2607.20145) |
| - Paper PDF: [arXiv:2607.20145](https://arxiv.org/pdf/2607.20145) |
| - Code and training recipes: [SLAI-AITP/SLAI-T-Rex](https://github.com/SLAI-AITP/SLAI-T-Rex) |
|
|
| ## Model Summary |
|
|
| | Item | Description | |
| | --- | --- | |
| | Model name | SLAI T-Rex-Pro | |
| | Base model | DeepSeek-V4-Pro | |
| | Domain | Operations Research and mathematical programming | |
| | Training | Full-parameter OR-oriented CPT followed by Clean-CoT SFT | |
| | Primary output | Mathematical formulation and/or solver-facing Python code | |
| | Training hardware | Ascend CloudMatrix384 SuperPOD with Ascend 910C NPUs | |
|
|
| SLAI T-Rex-Pro validates that the CPT+SFT workflow developed on DeepSeek-V4-Flash transfers to the substantially larger Pro model. The system-side optimizations reported for DeepSeek-V4-Pro increase Model FLOPs Utilization from 11.67% to 34.22%, a 2.93× improvement over the open-source baseline recipe. This is a training-system result rather than an inference-speed claim. |
|
|
| ## Download and Deployment |
|
|
| Download the files from this repository using the repository platform's standard snapshot or Git LFS workflow. Because this is a very large MoE checkpoint, verify all of the following before deployment: |
|
|
| - checkpoint precision and shard index integrity; |
| - storage capacity and host-memory requirements; |
| - tokenizer and DeepSeek-V4 architecture support; |
| - tensor, pipeline, data, and expert parallel layout; |
| - runtime support for the checkpoint's attention and MoE operators. |
|
|
| The public code repository documents the checkpoint-preparation and training-side conversion boundary. Serving commands are intentionally hardware- and runtime-specific; use a DeepSeek-V4-compatible serving stack and validate a small deterministic request before scaling out. |
|
|
| ## Intended Use |
|
|
| SLAI T-Rex-Pro is intended for research and development involving: |
|
|
| - natural-language-to-optimization modeling; |
| - mathematical formulation and structural reasoning; |
| - Gurobi-style solver program generation; |
| - feasibility- and equivalence-oriented OR tasks; |
| - research on full-parameter post-training of very large MoE models. |
|
|
| Prompts should provide complete problem data, explicitly state the expected solver action, and define the required response format. Generated code and formulations must be independently checked before use. |
|
|
| ## Evaluation |
|
|
| The following scale-up results are reported under zero-shot Pass@4 evaluation. `Overall` is the unweighted mean of the four OR benchmarks. |
|
|
| | Model | NL4OPT | OptiBench | B4O-Feasible | B4O-ORGEval | Overall | Gain vs. Base | |
| | --- | ---: | ---: | ---: | ---: | ---: | ---: | |
| | DeepSeek-V4-Pro | 86.51 | 70.00 | 80.23 | 43.91 | 70.16 | — | |
| | DeepSeek-V4-Pro + SFT | 89.97 | 69.50 | **86.04** | 61.43 | 76.74 | +6.58 | |
| | SLAI T-Rex-Pro | **92.04** | **70.17** | 85.17 | **61.93** | **77.33** | **+7.17** | |
|
|
| The complete pipeline obtains the strongest aggregate OR result. It does not dominate SFT-only on every benchmark: SFT-only scores 86.04 on B4O-Feasible, compared with 85.17 for SLAI T-Rex-Pro. The largest improvement over the base checkpoint is on B4O-ORGEval. |
|
|
| The report also provides a complementary general-capability check: |
|
|
| | Model | AIME 2024 | AIME 2025 | CMMLU | HumanEval | LiveCodeBench | |
| | --- | ---: | ---: | ---: | ---: | ---: | |
| | DeepSeek-V4-Pro | 83.33 | 70.00 | **93.88** | **100.00** | 55.00 | |
| | DeepSeek-V4-Pro + SFT | 83.33 | 76.67 | 93.75 | **100.00** | **72.00** | |
| | SLAI T-Rex-Pro | **90.00** | **86.67** | 93.12 | **100.00** | 67.00 | |
|
|
| Evaluation numbers should be compared only under the same prompts, decoding budgets, benchmark versions, solver environment, and scoring implementation described in the paper. |
|
|
|
|
| ## Citation |
|
|
| If you use this model, please cite the technical report: |
|
|
| ```bibtex |
| @misc{li2026slaitrexfullparameterposttraining, |
| title = {SLAI T-Rex: Full-Parameter Post-training of the DeepSeek-V4 Family on Ascend SuperPOD}, |
| author = {Dongfang Li and others}, |
| year = {2026}, |
| eprint = {2607.20145}, |
| archivePrefix= {arXiv}, |
| url = {https://arxiv.org/abs/2607.20145} |
| } |
| ``` |
