Enrich card: accuracy vs FLARE base + c=1 speed & commit-rate tables
Browse files
README.md
CHANGED
|
@@ -12,28 +12,55 @@ tags:
|
|
| 12 |
|
| 13 |
*English | [한국어](./README_ko.md)*
|
| 14 |
|
| 15 |
-
Qwen3.5-4B FLARE two-stream **block-diffusion** checkpoint (research). Continued from the
|
| 16 |
-
base (step_18000) with the **full-mask recipe**: 100% of each block masked (single view, no
|
| 17 |
complementary mask) at block size **bd=8**, fresh optimizer, LR 5e-5, ~2k steps.
|
| 18 |
|
| 19 |
## How to serve
|
| 20 |
|
| 21 |
Best served **self-speculative** (draft with the diffusion stream, verify with the clean/AR stream):
|
| 22 |
-
verify-guaranteed AR quality
|
| 23 |
-
|
| 24 |
-
|
|
|
|
|
|
|
| 25 |
|
| 26 |
-
|
| 27 |
|
| 28 |
-
|
|
|
|
| 29 |
|
| 30 |
-
| | self-spec | AR |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 31 |
|---|---|---|
|
| 32 |
-
|
|
| 33 |
-
|
|
| 34 |
-
|
|
| 35 |
-
|
|
| 36 |
-
|
|
| 37 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
| 38 |
|
| 39 |
Internal research checkpoint — trillionlabs.
|
|
|
|
| 12 |
|
| 13 |
*English | [한국어](./README_ko.md)*
|
| 14 |
|
| 15 |
+
Qwen3.5-4B FLARE two-stream **block-diffusion** checkpoint (research). Continued from the FLARE-4B
|
| 16 |
+
base (v6 SFT, step_18000) with the **full-mask recipe**: 100% of each block masked (single view, no
|
| 17 |
complementary mask) at block size **bd=8**, fresh optimizer, LR 5e-5, ~2k steps.
|
| 18 |
|
| 19 |
## How to serve
|
| 20 |
|
| 21 |
Best served **self-speculative** (draft with the diffusion stream, verify with the clean/AR stream):
|
| 22 |
+
verify-guaranteed AR quality, and — because full-mask training shapes exactly the single-shot
|
| 23 |
+
full-block prediction a self-spec draft needs — a **higher acceptance length than the base**, so it's
|
| 24 |
+
the fastest self-spec at low concurrency. It also serves in plain **AR/causal** mode. **Pure iterative
|
| 25 |
+
diffusion decoding is not supported by this recipe** (the shift decoder's partial-mask states are
|
| 26 |
+
untrained). Runs on the HybridDiffusion SGLang fork. `mask_id = 248077`, vocab 248078.
|
| 27 |
|
| 28 |
+
## Accuracy
|
| 29 |
|
| 30 |
+
Concurrency 1, threshold 0.9, reasoning on. Compared against the **FLARE-4B base** (the v6 SFT
|
| 31 |
+
checkpoint Trida2.0 continues from). *A vanilla Qwen3.5-4B baseline is being added.*
|
| 32 |
|
| 33 |
+
| benchmark | Trida2.0 · self-spec | Trida2.0 · AR | FLARE base · self-spec |
|
| 34 |
+
|---|---|---|---|
|
| 35 |
+
| GSM8K | **84.2** | **89.3** | 83.2 |
|
| 36 |
+
| MMLU-Pro | 63.0 | — | 63.5 |
|
| 37 |
+
| IFEval (prompt-strict) | 69.7 | — | 70.1 |
|
| 38 |
+
| GPQA-Diamond † | 36.4 | 38.4 | 40.9 |
|
| 39 |
+
| BFCL (Non-Live AST) | **65.2** | — | 53.2 |
|
| 40 |
+
| Ko-AgentBench L1 / L2 | **0.909 / 1.00** | — | 0.909 / 0.933 |
|
| 41 |
+
| FunctionChat call / compl | 0.85 / 0.88 | — | 0.92 / 0.95 |
|
| 42 |
+
| tau2 retail / air / tel | 0.40 / 0.50 / 0.80 | — | 0.65 / 0.55 / 0.90 |
|
| 43 |
+
|
| 44 |
+
† GPQA-Diamond at a 16k token cap is heavily truncation-limited (long chain-of-thought); these are
|
| 45 |
+
floors — a higher-cap rerun is pending.
|
| 46 |
+
|
| 47 |
+
## Serving speed & parallel commit (concurrency 1)
|
| 48 |
+
|
| 49 |
+
Block-diffusion self-spec commits **multiple tokens per forward pass** at verify-guaranteed AR
|
| 50 |
+
quality. Trida2.0's full-mask training makes it a better drafter, so it accepts more per step and runs
|
| 51 |
+
faster than the base.
|
| 52 |
+
|
| 53 |
+
| model · decoding | tok/s (per-request) | tokens / forward |
|
| 54 |
|---|---|---|
|
| 55 |
+
| **Trida2.0 · self-spec** | **243** | **2.59** |
|
| 56 |
+
| Trida2.0 · AR | 216 | 1.00 |
|
| 57 |
+
| FLARE base · self-spec | 188 | ~2.0 |
|
| 58 |
+
| FLARE base · AR | 217 | 1.00 |
|
| 59 |
+
| FLARE base · diffusion | 189 | ~2.5 |
|
| 60 |
+
|
| 61 |
+
Trida2.0 self-spec is **~30% faster** than the base self-spec at equal-or-better accuracy. Per-benchmark
|
| 62 |
+
commit rate (Trida2.0 self-spec, tokens/forward): **GSM8K 3.10, BFCL 2.33, MMLU-Pro 2.33**.
|
| 63 |
+
|
| 64 |
+
*Concurrency-4 throughput and a vanilla Qwen3.5-4B baseline are being added.*
|
| 65 |
|
| 66 |
Internal research checkpoint — trillionlabs.
|