Browang1 Rookie-Liu commited on
Commit
6bb7bf8
·
0 Parent(s):

Duplicate from inclusionAI/Ling-3.0-tiny-UBTree

Browse files

Co-authored-by: Huabin <Rookie-Liu@users.noreply.huggingface.co>

Files changed (5) hide show
  1. .gitattributes +36 -0
  2. README.md +117 -0
  3. assets/ubtree_pipeline.png +3 -0
  4. config.json +95 -0
  5. model.safetensors +3 -0
.gitattributes ADDED
@@ -0,0 +1,36 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ *.7z filter=lfs diff=lfs merge=lfs -text
2
+ *.arrow filter=lfs diff=lfs merge=lfs -text
3
+ *.bin filter=lfs diff=lfs merge=lfs -text
4
+ *.bz2 filter=lfs diff=lfs merge=lfs -text
5
+ *.ckpt filter=lfs diff=lfs merge=lfs -text
6
+ *.ftz filter=lfs diff=lfs merge=lfs -text
7
+ *.gz filter=lfs diff=lfs merge=lfs -text
8
+ *.h5 filter=lfs diff=lfs merge=lfs -text
9
+ *.joblib filter=lfs diff=lfs merge=lfs -text
10
+ *.lfs.* filter=lfs diff=lfs merge=lfs -text
11
+ *.mlmodel filter=lfs diff=lfs merge=lfs -text
12
+ *.model filter=lfs diff=lfs merge=lfs -text
13
+ *.msgpack filter=lfs diff=lfs merge=lfs -text
14
+ *.npy filter=lfs diff=lfs merge=lfs -text
15
+ *.npz filter=lfs diff=lfs merge=lfs -text
16
+ *.onnx filter=lfs diff=lfs merge=lfs -text
17
+ *.ot filter=lfs diff=lfs merge=lfs -text
18
+ *.parquet filter=lfs diff=lfs merge=lfs -text
19
+ *.pb filter=lfs diff=lfs merge=lfs -text
20
+ *.pickle filter=lfs diff=lfs merge=lfs -text
21
+ *.pkl filter=lfs diff=lfs merge=lfs -text
22
+ *.pt filter=lfs diff=lfs merge=lfs -text
23
+ *.pth filter=lfs diff=lfs merge=lfs -text
24
+ *.rar filter=lfs diff=lfs merge=lfs -text
25
+ *.safetensors filter=lfs diff=lfs merge=lfs -text
26
+ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
27
+ *.tar.* filter=lfs diff=lfs merge=lfs -text
28
+ *.tar filter=lfs diff=lfs merge=lfs -text
29
+ *.tflite filter=lfs diff=lfs merge=lfs -text
30
+ *.tgz filter=lfs diff=lfs merge=lfs -text
31
+ *.wasm filter=lfs diff=lfs merge=lfs -text
32
+ *.xz filter=lfs diff=lfs merge=lfs -text
33
+ *.zip filter=lfs diff=lfs merge=lfs -text
34
+ *.zst filter=lfs diff=lfs merge=lfs -text
35
+ *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ assets/ubtree_pipeline.png filter=lfs diff=lfs merge=lfs -text
README.md ADDED
@@ -0,0 +1,117 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ library_name: sglang
3
+ pipeline_tag: text-generation
4
+ license: mit
5
+ base_model:
6
+ - inclusionAI/Ling-3.0-tiny
7
+ tags:
8
+ - speculative-decoding
9
+ - tree-speculative-decoding
10
+ - ubtree
11
+ - dflash
12
+ - sglang
13
+ - ling3
14
+ - arxiv:2609.39972
15
+ language:
16
+ - zh
17
+ - en
18
+ ---
19
+
20
+ # Ling3-Tiny-UBTree-B16
21
+
22
+ [Paper](https://arxiv.org/abs/2609.39972) · [Target model](https://huggingface.co/inclusionAI/Ling-3.0-tiny)
23
+
24
+ **Ling3-Tiny-UBTree-B16** is a UBTree speculative-decoding draft model for
25
+ [inclusionAI/Ling-3.0-tiny](https://huggingface.co/inclusionAI/Ling-3.0-tiny). It combines a
26
+ five-layer DFlash backbone with a learned bigram selector to propose multiple
27
+ candidate continuations for target-model verification.
28
+
29
+ > **SGLang integration — stay tuned!** Our UBTree adaptation for Ling3 has not
30
+ > yet been released to the open-source SGLang community. Public installation
31
+ > instructions and serving recipes will be added here when it becomes available.
32
+
33
+ This checkpoint is used alongside the matching target model; it cannot serve
34
+ as a standalone text-generation model.
35
+
36
+ ## How UBTree works
37
+
38
+ ![UBTree architecture and tree-native training pipeline](assets/ubtree_pipeline.png)
39
+
40
+ *UBTree pipeline*
41
+
42
+ UBTree pairs a **unigram proposer** with a **bigram selector**:
43
+
44
+ 1. A DFlash backbone proposes candidate tokens for future positions in parallel.
45
+ 2. The selector scores transitions between adjacent candidates in parallel.
46
+ 3. Tree construction retains promising branches within a verification budget.
47
+ 4. The target verifies the tree and commits the accepted continuation.
48
+
49
+ The paper also introduces tree-native training with renormalized KL supervision
50
+ on high-temperature target traces, encouraging plausible alternative branches.
51
+ See the [paper](https://arxiv.org/abs/2609.39972) for the method and experiments.
52
+
53
+ ## Model specifications
54
+
55
+ | Item | Value |
56
+ |---|---|
57
+ | Target model | [inclusionAI/Ling-3.0-tiny](https://huggingface.co/inclusionAI/Ling-3.0-tiny) |
58
+ | Draft architecture | `Ling3UBTreeModel` |
59
+ | Draft backbone | 5 full-attention Transformer layers |
60
+ | Parameters stored in this checkpoint | 183,127,326 (183.1M) |
61
+ | Weight dtype | BF16 |
62
+ | Hidden size | 1,536 |
63
+ | Attention heads / KV heads | 16 / 16 |
64
+ | Target auxiliary feature layers (zero-based) | 4, 8, 12, 16, 22 |
65
+ | Draft block size | 16 (anchor-inclusive runtime convention) |
66
+ | Candidate top-k | 64 per draft position |
67
+ | Default verification tree budget | 64 non-root nodes + 1 anchor |
68
+ | Training | DFlash warm start, then joint backbone + UBTree selector training |
69
+
70
+ The parameter count covers tensors distributed in this repository. The draft
71
+ shares token embeddings and the language-model head with the target; these
72
+ shared weights are not included here. B16 denotes the draft block size, not
73
+ the verification tree's node budget.
74
+
75
+ ## Serving with SGLang
76
+
77
+ UBTree support for Ling3 in SGLang is coming soon. We will share installation
78
+ instructions and serving examples once the integration is publicly available.
79
+ **Stay tuned!**
80
+
81
+ ## Benchmark results
82
+
83
+ All methods use **block size 16**, TP1, concurrency 1, BF16, and greedy
84
+ no-thinking decoding. UBTree uses a 49-node tree.
85
+
86
+ Each cell shows **acceptance length / E2E speedup over target-only AR**.
87
+
88
+ | Dataset | UBTree | DFlash | DSpark |
89
+ |---|---:|---:|---:|
90
+ | GSM8K | **9.53 / 3.055×** | 6.57 / 2.665× | 7.53 / 2.718× |
91
+ | MATH-500 | **9.86 / 3.518×** | 5.74 / 2.643× | 6.96 / 2.826× |
92
+ | AIME 2025 | **8.44 / 3.152×** | 4.76 / 2.261× | 5.37 / 2.263× |
93
+ | HumanEval | **10.04 / 3.411×** | 6.39 / 2.770× | 7.59 / 2.914× |
94
+ | MBPP | **9.35 / 3.062×** | 5.87 / 2.473× | 7.16 / 2.678× |
95
+ | LiveCodeBench | **7.46 / 2.794×** | 4.11 / 1.948× | 4.84 / 2.030× |
96
+
97
+ AL and throughput use three-round medians; the matched AR baseline uses one round.
98
+
99
+ ## Citation
100
+
101
+ If you use UBTree in your work, please cite:
102
+
103
+ ```bibtex
104
+ @article{liang2026ubtree,
105
+ title = {UBTree: Parallel Tree Drafting via Unigram and Bigram Models for Speculative Decoding},
106
+ author = {Liang, Chumeng and Wang, Linxuan and Peng, Xinyu and Liu, Huabin and Chen, Yuxin and Liu, Ge and Lin, Guang and Song, Qifan and Li, Jianguo},
107
+ journal = {arXiv preprint arXiv:2609.39972},
108
+ year = {2026},
109
+ url = {https://arxiv.org/abs/2609.39972}
110
+ }
111
+ ```
112
+
113
+ ## License
114
+
115
+ The model weights are released under the **MIT License**. The pipeline figure
116
+ is from [Liang et al., Figure 2](https://arxiv.org/html/2609.39972v1#S2.F2),
117
+ reproduced without modification under [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/).
assets/ubtree_pipeline.png ADDED

Git LFS Details

  • SHA256: 3233e0c7a3d9be0edf8e93a303f662a97d0b5f6eb92fa4fe3857eca11a4225e7
  • Pointer size: 131 Bytes
  • Size of remote file: 231 kB
config.json ADDED
@@ -0,0 +1,95 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "architectures": [
3
+ "Ling3UBTreeModel"
4
+ ],
5
+ "attention_bias": false,
6
+ "attention_dropout": 0.0,
7
+ "block_size": 16,
8
+ "bos_token_id": null,
9
+ "dflash_config": {
10
+ "block_size": 16,
11
+ "contract": "flashflow_anchor_inclusive_v1",
12
+ "mask_token_id": 156892,
13
+ "num_target_layers": 24,
14
+ "target_layer_ids": [
15
+ 4,
16
+ 8,
17
+ 12,
18
+ 16,
19
+ 22
20
+ ]
21
+ },
22
+ "dflash_contract": "flashflow_anchor_inclusive_v1",
23
+ "dtype": "bfloat16",
24
+ "eos_token_id": 156895,
25
+ "head_dim": 128,
26
+ "hidden_act": "silu",
27
+ "hidden_size": 1536,
28
+ "initializer_range": 0.02,
29
+ "intermediate_size": 4608,
30
+ "layer_types": [
31
+ "full_attention",
32
+ "full_attention",
33
+ "full_attention",
34
+ "full_attention",
35
+ "full_attention"
36
+ ],
37
+ "mask_token_id": 156892,
38
+ "max_position_embeddings": 131072,
39
+ "max_window_layers": 20,
40
+ "model_type": "qwen3",
41
+ "num_attention_heads": 16,
42
+ "num_hidden_layers": 5,
43
+ "num_key_value_heads": 16,
44
+ "num_target_layers": 24,
45
+ "pad_token_id": 156892,
46
+ "partial_rotary_factor": 1.0,
47
+ "rms_norm_eps": 1e-06,
48
+ "rope_parameters": {
49
+ "partial_rotary_factor": 1.0,
50
+ "rope_theta": 6000000,
51
+ "rope_type": "default"
52
+ },
53
+ "rope_theta": 6000000.0,
54
+ "sliding_window": null,
55
+ "target_layer_ids": [
56
+ 4,
57
+ 8,
58
+ 12,
59
+ 16,
60
+ 22
61
+ ],
62
+ "tie_word_embeddings": false,
63
+ "transformers_version": "5.12.1",
64
+ "ubtree_config": {
65
+ "adds_unary_logits": true,
66
+ "architecture": "cp",
67
+ "candidate_k": 64,
68
+ "candidate_source": "frozen_dflash_unary_topk_ids_only",
69
+ "depth_condition_count": 15,
70
+ "depth_conditioning": "unary_correction_two_scale",
71
+ "depth_scale_parameterization": "exp_log_scale",
72
+ "factorizer_hidden_width": 512,
73
+ "hidden_projection_depth_count": 15,
74
+ "hidden_projection_type": "shared",
75
+ "inference_hidden_projection": "materialized_per_depth_affine",
76
+ "kl_temperature": 1.0,
77
+ "objective": "kl",
78
+ "rank": 256,
79
+ "target_embedding_dim": 1536,
80
+ "token_factorizer": "target_embedding_mlp",
81
+ "train_backbone": true,
82
+ "trained": true,
83
+ "training_mode": "joint_backbone_selector",
84
+ "tree_budget": 64,
85
+ "tree_builder": "pruned",
86
+ "tree_depth_bonus": 0.0,
87
+ "tree_supertree_width": 12,
88
+ "type": "residual_bigram",
89
+ "version": 1
90
+ },
91
+ "use_cache": true,
92
+ "use_qk_norm": true,
93
+ "use_sliding_window": false,
94
+ "vocab_size": 157184
95
+ }
model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c0820dfec7492fd6543dc170be8db2de076bc88524cf7d3dce1de37fe3173dca
3
+ size 366261868