HongjoonAhn commited on
Commit
673e390
·
verified ·
1 Parent(s): 51bc952

Enrich card: accuracy vs FLARE base + c=1 speed & commit-rate tables

Browse files
Files changed (1) hide show
  1. README.md +41 -14
README.md CHANGED
@@ -12,28 +12,55 @@ tags:
12
 
13
  *English | [한국어](./README_ko.md)*
14
 
15
- Qwen3.5-4B FLARE two-stream **block-diffusion** checkpoint (research). Continued from the v6 SFT
16
- base (step_18000) with the **full-mask recipe**: 100% of each block masked (single view, no
17
  complementary mask) at block size **bd=8**, fresh optimizer, LR 5e-5, ~2k steps.
18
 
19
  ## How to serve
20
 
21
  Best served **self-speculative** (draft with the diffusion stream, verify with the clean/AR stream):
22
- verify-guaranteed AR quality at ~2.6 accepted tokens/forward — the fastest self-spec of our
23
- checkpoints at concurrency 1. It also serves in plain **AR/causal** mode. **Pure iterative diffusion
24
- decoding is not supported by this recipe** (the shift decoder's partial-mask states are untrained).
 
 
25
 
26
- Runs on the HybridDiffusion SGLang fork. `mask_id = 248077`, vocab 248078.
27
 
28
- ## Eval (concurrency 1, threshold 0.9)
 
29
 
30
- | | self-spec | AR |
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
31
  |---|---|---|
32
- | GSM8K | 84.2 | 89.3 |
33
- | MMLU-Pro | 63.0 | — |
34
- | IFEval (prompt-strict) | 69.7 | — |
35
- | BFCL (Non-Live AST) | 65.2 | — |
36
- | Ko-AgentBench L1/L2 | 0.909 / 1.00 | — |
37
- | FunctionChat call/compl | 0.85 / 0.88 | — |
 
 
 
 
38
 
39
  Internal research checkpoint — trillionlabs.
 
12
 
13
  *English | [한국어](./README_ko.md)*
14
 
15
+ Qwen3.5-4B FLARE two-stream **block-diffusion** checkpoint (research). Continued from the FLARE-4B
16
+ base (v6 SFT, step_18000) with the **full-mask recipe**: 100% of each block masked (single view, no
17
  complementary mask) at block size **bd=8**, fresh optimizer, LR 5e-5, ~2k steps.
18
 
19
  ## How to serve
20
 
21
  Best served **self-speculative** (draft with the diffusion stream, verify with the clean/AR stream):
22
+ verify-guaranteed AR quality, and — because full-mask training shapes exactly the single-shot
23
+ full-block prediction a self-spec draft needs — a **higher acceptance length than the base**, so it's
24
+ the fastest self-spec at low concurrency. It also serves in plain **AR/causal** mode. **Pure iterative
25
+ diffusion decoding is not supported by this recipe** (the shift decoder's partial-mask states are
26
+ untrained). Runs on the HybridDiffusion SGLang fork. `mask_id = 248077`, vocab 248078.
27
 
28
+ ## Accuracy
29
 
30
+ Concurrency 1, threshold 0.9, reasoning on. Compared against the **FLARE-4B base** (the v6 SFT
31
+ checkpoint Trida2.0 continues from). *A vanilla Qwen3.5-4B baseline is being added.*
32
 
33
+ | benchmark | Trida2.0 · self-spec | Trida2.0 · AR | FLARE base · self-spec |
34
+ |---|---|---|---|
35
+ | GSM8K | **84.2** | **89.3** | 83.2 |
36
+ | MMLU-Pro | 63.0 | — | 63.5 |
37
+ | IFEval (prompt-strict) | 69.7 | — | 70.1 |
38
+ | GPQA-Diamond † | 36.4 | 38.4 | 40.9 |
39
+ | BFCL (Non-Live AST) | **65.2** | — | 53.2 |
40
+ | Ko-AgentBench L1 / L2 | **0.909 / 1.00** | — | 0.909 / 0.933 |
41
+ | FunctionChat call / compl | 0.85 / 0.88 | — | 0.92 / 0.95 |
42
+ | tau2 retail / air / tel | 0.40 / 0.50 / 0.80 | — | 0.65 / 0.55 / 0.90 |
43
+
44
+ † GPQA-Diamond at a 16k token cap is heavily truncation-limited (long chain-of-thought); these are
45
+ floors — a higher-cap rerun is pending.
46
+
47
+ ## Serving speed & parallel commit (concurrency 1)
48
+
49
+ Block-diffusion self-spec commits **multiple tokens per forward pass** at verify-guaranteed AR
50
+ quality. Trida2.0's full-mask training makes it a better drafter, so it accepts more per step and runs
51
+ faster than the base.
52
+
53
+ | model · decoding | tok/s (per-request) | tokens / forward |
54
  |---|---|---|
55
+ | **Trida2.0 · self-spec** | **243** | **2.59** |
56
+ | Trida2.0 · AR | 216 | 1.00 |
57
+ | FLARE base · self-spec | 188 | ~2.0 |
58
+ | FLARE base · AR | 217 | 1.00 |
59
+ | FLARE base · diffusion | 189 | ~2.5 |
60
+
61
+ Trida2.0 self-spec is **~30% faster** than the base self-spec at equal-or-better accuracy. Per-benchmark
62
+ commit rate (Trida2.0 self-spec, tokens/forward): **GSM8K 3.10, BFCL 2.33, MMLU-Pro 2.33**.
63
+
64
+ *Concurrency-4 throughput and a vanilla Qwen3.5-4B baseline are being added.*
65
 
66
  Internal research checkpoint — trillionlabs.