ProCreations/Nanbeige4.3-33B-DFlash2

Nanbeige4.3-33B DFlash2

DFlash2 block-diffusion draft model (3 layers, block 8) for Nanbeige4.3-33B.
Batch-1 decode speedups on RTX PRO 6000 (Modal 600 W), SGLang 0.5.21: BF16 1.76× greedy / 1.63× T1; FP8 1.65× / 1.49×; NVFP4-A16 1.51× / 1.39×.
Native exllamav3 with the EXL3 quants (Vega RTX PRO 6000, 580 W), exllamav3 branch: 4.05bpw 1.51× / 1.46×; 5.05bpw 1.55× / 1.47×.
Agentic coding ~2.3-2.6×, code/math ~1.7-2.1×, chat/Chinese ~1.2-1.3×.

v2 with more training data is planned.

This is a speculative draft model for Nanbeige4.3-33B, not a standalone language model. It shares the serving target's frozen embedding and untied output head. Neither large matrix is duplicated in the drafter checkpoint. Use the target's tokenizer and chat template.

• Base revision:c1093c501cc6460427c0ac634db7223d0450c0ee.
• Primary deployment: SGLang 0.5.21 with the BF16 base or the published FP8. The NVFP4-A16 and EXL3 targets are published.
• Native exllamav3 v1.6.4: use the exllamav3 branch with the published EXL3 target. Vega (580 W) batch-1 decode: 4.05bpw 1.51× greedy / 1.46× T1; 5.05bpw 1.55× / 1.47×.
• Recommended sampling follows the target: greedy, or temperature 0.6 for general reasoning and 1.0 for agentic tools, top_p 0.95. Keep thinking/tool-history settings appropriate to the task.

Architecture and checkpoint format

Item	Value
Decoder layers / parameters	3 /979,731,200
Hidden / MLP width	7168 /8192
Query / KV heads	28 /4, head dimension128
Draft block	8 tokens including one verified anchor
Attention	Full bidirectional attention inside the proposal block
Target features	Post-layer residuals1,25,49, before final RMSNorm
Feature fusion	Concat21504 → learned7168 projection → hidden RMSNorm
Dynamic convolution	Two taps, channel groups16, around attention and MLP
Selector	Rank128, top-k32, conditional predecessor/successor scores
Mask token	0 (<unk>), no tokenizer modification
RoPE / RMS epsilon	Neox, theta120000000 /1e-5
Stored weights	BF16 safetensors; separate q/k/v and gate/up tensors

Tap IDs are zero-based layer outputs, equivalent to HF hidden_states[2,26,50]. They are not inputs to layers1,25,49. The checkpoint has root fc, hidden_norm, norm, layers.* and bare selector codebooks. Native EXL3 config pins tap_shift=0. Both runtimes attach the target head/embedding directly.

14,484 of 14,533 planned updates (99.66%); final 49 updates at the LR floor not run; selection_complete=false by orchestrator decision

Training data and recipe

Only approved allow sources at their exact pinned revisions are admitted under the accompanying source policy. Prompt histories are rendered with Nanbeige's own template and continued on-policy by the pinned Nanbeige BF16 weights, using native token IDs, top_p0.95 and temperature0.6/1.0. Selected generation precision is BF16 weights with16,612,362 response tokens using FP8E4M3 KV and2,734,682 using BF16 KV. Precision comes from the authenticated final per-run lookup, checked against available row fields; legacy per-ID sidecars remain immutable. Generation precision is distinct from the training teacher's precision.

The first full selection contains 11,554 training records,19,347,044 response tokens and100,819,182 supplied-history tokens, with36 fixed family-disjoint validation records. Supplied history means the native prompt IDs plus response IDs in each generated row. The main run completed all11 frozen windows and11,286 optimizer updates with2.25 shuffled view passes; these are unique selected-data counts, not a claim that every token was sampled as a supervised label on every pass. There is no training on sealed evaluation data or priority-generated reference data. Family aliases survive stage changes; selected train/validation family-key overlap is zero.

The selected rows include10,785 stopped responses (12,645,988 response tokens) and769 length-capped responses (6,701,056 response tokens). Capped rows are incomplete native continuations, not EOS-complete examples. Their original finish reasons and available completion/truncation flags remain in immutable source rows;1,258 legacy rows lack the newer Boolean flags. No synthetic EOS is added. The final generated pool contains70,759,625 response tokens, including25,866,289 agentic and16,169,620 code tokens; these pool totals are not the selected or trained token counts. Its FINAL_READY receipt binds SUMMARY SHA25604bf46590b49da810ecec16902146cb00f5156a9bdfac3fe36cf384e466a9947 and MANIFEST SHA256a4c018e66e397849b5f14f266633ee801009a1d9868be5b8f1a84c8feaf2df9f.

Domain	Response tokens	Share
agentic	9,003,113	46.53%
code	4,003,484	20.69%
math	1,683,454	8.70%
chat	1,680,691	8.69%
chinese	1,121,044	5.79%
multilingual	707,757	3.66%
structured	702,769	3.63%
longctx	433,324	2.24%
stem	11,408	0.06%

Source revisions, licenses, exact domain/source counts, finish reasons and immutable selection fingerprints are in training-selection.json. Prompt sources include restricted approved OpenHands and synthetic terminal components, coding/reasoning/chat/Chinese/multilingual/structured/long-context datasets; see NOTICE.md and the policy for component restrictions. Source data was re-rendered and new target continuations were generated. Original dataset licenses and attribution remain applicable.

The main teacher is the project's FP8-channel calibrated-KV training artifact, with full-file manifest SHA25671c3d153cabd97a6483ff14a4dbb97f8150ff2e11727280caa60da1fb718da2a, FP8E4M3 KV, stock SGLang0.5.21. This training-stage teacher is not a recommendation to serve that FP8 artifact. Every supplied prompt and response token is processed before retaining response features and trailing context. Features remain BF16 in sealed local RAM; no feature compression is used.

• 3-layer draft trained data-parallel on4B200s, global sequence batch4,2048-token views with1536 stride.
• 2.25 consecutive shuffled view passes per bounded feature window, up to64 sampled valid anchors/view and seven successor slots. This is not a claim of2.25 complete supervised-label passes.
• AdamW FP32 masters/BF16 autocast, peakLR3e-4,100-step warmup/global cosine, betas0.9/0.95, weight decay0.01, clip1; activation checkpointing.
• Global model/optimizer/rank RNG and learning-rate progress continue across windows. Feature deletion follows a durable model/optimizer/RNG receipt.
• Main objective:90% soft served-teacher distribution/10% replay hard labels plus selector loss. Implementation:SpecForge commitfddcb4ad37733a821708170df28cac2f7bb6ea05 with the full noncausal block mask.

This release (v1)

The broad retrain warm-starts the main11286 weights and preserves the3-layer/block8/top-k32/mask0 architecture. It uses the same frozen FP8 teacher and full-history capture setup. Its13-window selection contains9,941 rows and30,035,818 response tokens, including24,664,716 new tokens and5,371,102 replay tokens. There are44,011,760 distinct-record response tokens across the main and broad selections. Counts below describe selected data; completion and actual optimization exposure are bound to the final checkpoint receipt.

Domain	Selected response tokens	Use
agentic	5,252,408	Replay
code	8,000,263	New
chat	6,003,654	New
math	5,001,735	New
multilingual	3,751,802	New
chinese	1,506,170	New, two view copies
structured	401,092	New
longctx	107,286	Replay
stem	11,408	Replay

Fresh AdamW uses peakLR1e-4,100 warmup steps and a global cosine schedule with15% floor, global batch4. The plan has14,533 updates and2.25 shuffled view passes, with Chinese views doubled. A bounded exact continuation restores the same optimizer, learning rate, rank RNG and data cursor; recaptured feature manifests must match before training resumes. The fixed200-record validation set has no train-family overlap. Per-slot oracle-prefix rejection overlap is reported separately from real serving AL. The broader selection has8,521 stopped and1,420 length-capped rows; capped native continuations receive no appended EOS or stop target.

A separate 1,000-step A16 adapter did not improve acceptance, so this release uses the broader FP8-teacher retrain.

Measured acceptance and speed

Targets are the pinned BF16 base, the published ProCreations FP8, and the published NVFP4-A16 target. The training teacher is a separate historical artifact. Exact runtime target identities accompany each result.

Initial measurements: Modal RTX PRO 6000 Server Edition (600 W). Raw measured rates are not normalised.

The one-case plain-A16 hardware comparison was 7.45% slower than Vega; this is context, not a correction factor. Vega measurements will be added after the 01:15 booking.

SGLang uses stochastic conditional-q rejection. AL includes the bonus target token and excludes the first prefill token. T1 uses top_p0.95; greedy is T0. Decode and end-to-end rates are distinct.

Target	T	Domain	AL	Decode tok/s	E2E tok/s	Matched plain decode tok/s	Speedup
a16	0	overall	1.8984	98.87	96.51	65.42	1.511x
a16	0	agentic	2.9020	151.24	136.84	65.31	2.316x
a16	0	chat	1.5288	79.43	78.38	65.45	1.214x
a16	0	chinese	1.4391	75.05	74.27	65.47	1.146x
a16	0	code	2.3655	123.69	120.03	65.41	1.891x
a16	0	math	2.1832	113.40	111.69	65.45	1.733x
a16	1	overall	1.7691	90.71	88.74	65.15	1.392x
a16	1	agentic	2.5943	133.79	123.03	65.02	2.058x
a16	1	chat	1.4182	71.48	70.59	65.18	1.097x
a16	1	chinese	1.3799	71.10	70.45	65.17	1.091x
a16	1	code	2.1832	112.70	109.63	65.14	1.730x
a16	1	math	1.9953	103.09	101.71	65.18	1.582x
bf16	0	overall	1.8834	38.99	38.53	22.18	1.758x
bf16	0	agentic	2.8977	60.03	57.56	22.17	2.708x
bf16	0	chat	1.5251	31.49	31.28	22.19	1.419x
bf16	0	chinese	1.4167	29.37	29.20	22.19	1.324x
bf16	0	code	2.2566	46.73	46.08	22.18	2.107x
bf16	0	math	2.2368	46.35	45.86	22.19	2.089x
bf16	1	overall	1.7498	36.03	35.68	22.15	1.627x
bf16	1	agentic	2.5587	52.68	50.91	22.14	2.380x
bf16	1	chat	1.4057	28.93	28.77	22.15	1.306x
bf16	1	chinese	1.3478	27.77	27.64	22.15	1.253x
bf16	1	code	2.2135	45.55	44.99	22.15	2.057x
bf16	1	math	1.9706	40.59	40.26	22.15	1.832x
fp8	0	overall	1.8904	67.78	67.12	41.08	1.650x
fp8	0	agentic	2.9721	106.30	102.01	41.00	2.593x
fp8	0	chat	1.4947	53.62	53.35	41.11	1.304x
fp8	0	chinese	1.4182	50.89	50.69	41.11	1.238x
fp8	0	code	2.3699	84.91	83.84	41.07	2.067x
fp8	0	math	2.2290	79.93	79.29	41.09	1.945x
fp8	1	overall	1.7175	60.99	60.47	40.99	1.488x
fp8	1	agentic	2.7128	96.16	92.55	40.93	2.349x
fp8	1	chat	1.4135	50.11	49.88	41.00	1.222x
fp8	1	chinese	1.3506	48.00	47.81	41.00	1.171x
fp8	1	code	1.9646	69.81	69.16	40.99	1.703x
fp8	1	math	1.9737	70.17	69.72	41.00	1.711x

Target and measurement fingerprints:

Published NVFP4-A16 target.

• bf16: hf_source_full_manifest_sha256:d5a411046b3e235f3e284d06cf4cb1a9a255ee032083660fc6f9d6caa441483a, KV bfloat16, runtime e00930c5489053f26d86b179cee0d087f846acbb, image lmsysorg/sglang:v0.5.21;stock-e00930c. Report SHA256 b855f432b0b71816851bface5adbbe5b1c18e1d79989938200075da5b414c80f, export receipt 48d5dd410f2d6ef8e2e6fca171acdf08f365fef73546afb3dc637564bb2d4e0f.
• fp8: manifest_sha256:5f3d0a025474f103d963541841d934f619b917f47d12fd547a3ed0f5d474ae98, KV bfloat16, runtime e00930c5489053f26d86b179cee0d087f846acbb, image lmsysorg/sglang:v0.5.21;stock-e00930c. Report SHA256 9298bb497717b9bbe383dbdc37699614f157ef701e383edecd1a2ff1b05da361, export receipt 2dcba15ac533c90999164d3234bee4415e5c12e548f0ee2517237682d5a27bce.
• a16: manifest_sha256:8279fa02e52ec35c70f12b5be55d27808c8b79ae1ddbe546f1b0d8913a49eaf6, KV bfloat16, runtime e00930c5489053f26d86b179cee0d087f846acbb, image lmsysorg/sglang:v0.5.21;stock-e00930c. Report SHA256 6abbc466f3adeec88ea493f11ff32278029f3b4a18301bd8571973f4ba08ff2c, export receipt f62a38c9c56e98eab30dc101d2d23ebfa60d2a1fe68b06ad583cabc98194e9a0.

Exact measured commands

bf16

Recorded worker argv; only artifact mount paths use placeholders. These batch-1 measurement flags are retained for reproduction.

/opt/sglang/bin/python -m sglang.launch_server --model-path '<target-dir>' --served-model-name nb43 --host 127.0.0.1 --port 30143 --kv-cache-dtype bfloat16 --attention-backend flashinfer --mem-fraction-static 0.88 --context-length 8192 --max-total-tokens 16384 --max-running-requests 1 --cuda-graph-max-bs-decode 1 --cuda-graph-backend-prefill disabled --chunked-prefill-size 2048 --dtype bfloat16 --reasoning-parser nanbeige --tool-call-parser nanbeige --disable-radix-cache --speculative-algorithm DFLASH --speculative-draft-model-path '<draft-dir>' --speculative-num-draft-tokens 8 --speculative-draft-attention-backend flashinfer --speculative-draft-kv-cache-dtype bfloat16

fp8

Recorded worker argv; only artifact mount paths use placeholders. These batch-1 measurement flags are retained for reproduction.

/opt/sglang/bin/python -m sglang.launch_server --model-path '<target-dir>' --served-model-name nb43 --host 127.0.0.1 --port 30143 --kv-cache-dtype bfloat16 --attention-backend flashinfer --mem-fraction-static 0.88 --context-length 8192 --max-total-tokens 16384 --max-running-requests 1 --cuda-graph-max-bs-decode 1 --cuda-graph-backend-prefill disabled --chunked-prefill-size 2048 --dtype bfloat16 --reasoning-parser nanbeige --tool-call-parser nanbeige --disable-radix-cache --speculative-algorithm DFLASH --speculative-draft-model-path '<draft-dir>' --speculative-num-draft-tokens 8 --speculative-draft-attention-backend flashinfer --speculative-draft-kv-cache-dtype bfloat16

a16

Recorded worker argv; only artifact mount paths use placeholders. These batch-1 measurement flags are retained for reproduction.

/opt/sglang/bin/python -m sglang.launch_server --model-path '<target-dir>' --served-model-name nb43 --host 127.0.0.1 --port 30143 --kv-cache-dtype bfloat16 --attention-backend flashinfer --mem-fraction-static 0.88 --context-length 8192 --max-total-tokens 16384 --max-running-requests 1 --cuda-graph-max-bs-decode 1 --cuda-graph-backend-prefill disabled --chunked-prefill-size 2048 --dtype bfloat16 --reasoning-parser nanbeige --tool-call-parser nanbeige --disable-radix-cache --speculative-algorithm DFLASH --speculative-draft-model-path '<draft-dir>' --speculative-num-draft-tokens 8 --speculative-draft-attention-backend flashinfer --speculative-draft-kv-cache-dtype bfloat16

On the matched A16 bank, main→v1 AL improved 1.730→1.898 at T0 and 1.611→1.769 at T1/top_p0.95.

Speedups above were measured at batch 1. Four/eight-stream and long-context results are separate measurements and are not inferred from batch 1. These fixed-bank results are not a guarantee on every workload.

Speculative-vs-plain behaviour and length

On the published FP8 target, all160 paired trials per arm completed the fixed-key screen. The observed protocol-token ratio was0.96813 [0.82789,1.13424]; graded accuracy was136/152 versus134/152, paired difference+1.32pp [-3.95,+6.58]. Unfinished counts were9 versus8 of160 and loops0 in both arms. No paired-CI difference was detected under the fixed short-cap protocol; this is not a statistical equivalence claim or unrestricted completion-length proof.

Greedy numerical behaviour

Speculative greedy output is not guaranteed token-identical to plain greedy decoding. Every observed first divergence (18/25 prompts) occurred at a 0-1 BF16-ULP logit tie, the same numerical effect seen when plain decoding changes batch size (14/25 prompts diverge between 1 and 8 concurrent requests). The stochastic rejection rule is exact; differences come from floating-point non-associativity across GEMM shapes, not from accepting wrong tokens.

Observed on the25-prompt clean-A16/BF16-KV controls with stockSGLang0.5.21. Evidence receipt:1fc35c42613cc26180cf254487a23a4cd49393255a6a73807431bd5645e3a05f.

Quick start: SGLang 0.5.21

Download the published FP8 target and this draft into local directories:

huggingface-cli download ProCreations/Nanbeige4.3-33B-FP8 --local-dir ./target
huggingface-cli download ProCreations/Nanbeige4.3-33B-DFlash2 --local-dir ./draft

For the BF16 base, use huggingface-cli download Nanbeige/Nanbeige4.3-33B --local-dir ./target instead. Replace <target-dir> and <draft-dir> below with those local directories:

python -m sglang.launch_server \
  --model-path "<target-dir>" \
  --speculative-draft-model-path "<draft-dir>" \
  --speculative-algorithm DFLASH \
  --speculative-num-draft-tokens 8 \
  --dtype bfloat16 --kv-cache-dtype bfloat16 \
  --attention-backend flashinfer \
  --speculative-draft-attention-backend flashinfer \
  --speculative-draft-kv-cache-dtype bfloat16 \
  --mem-fraction-static 0.88 --chunked-prefill-size 2048 \
  --reasoning-parser nanbeige --tool-call-parser nanbeige

Use --dtype bfloat16, as tested. Release weights and config are BF16; the measured original config is linked to this dtype-only correction by an all-tensor CPU bit-exact proof. Weight bytes are unchanged. Stock source commit:e00930c5489053f26d86b179cee0d087f846acbb; no model/kernel overlays were used.

The quick start omits batch-1 benchmark limits. The exact measured commands above retain those limits and use placeholders only for artifact mount paths. Send temperature0/0.6/1.0 and top_p0.95 per request. Preserve thinking and tool history for multi-turn agent work. The displayed results cover T0 andT1/top_p0.95; T0.6, higher concurrency and optional FP8 KV require separate measurements.

Native EXL3 support

Use the exllamav3 branch with ProCreations/Nanbeige4.3-33B-EXL3. On Vega, an RTX PRO 6000 Workstation Edition at 580 W, native batch-1 decode is 1.51× faster at temperature 0 and 1.46× at temperature 1 with 4.05bpw, and 1.55× / 1.47× with 5.05bpw. Agentic coding reaches about 2.0–2.25×. Every measured domain is faster than plain decoding on both rungs at both temperatures. The branch card has the tested stock v1.6.4 install, a runnable quickstart and per-domain raw rates (5 prompts per domain, Q8 target KV, FP16 draft KV, top_p 0.95).

The branch holds the unchanged v1 weights; there is no EXL3 adaptation. exllamav3 samples the target and accepts matching draft tokens, while SGLang uses stochastic rejection, and native AL also counts the initial/final boundary rounds. Keep native and SGLang measurements separate.

License, provenance and limitations

Weights inherit Apache-2.0 from the base model. Training code and runtime notices retain their respective licenses; dataset sources retain their own licenses. Only the approved permissive source/component policy was used; excluded and evaluation families are not training data. No source corpus is redistributed in this model repository.

Acceptance can change with target weights, quantization, head/KV precision, context and runtime arithmetic. Train-time teacher diagnostics do not predict real throughput reliably. The final model and target fingerprints, completed exposure, behaviour/length results and serving commands must be reviewed together. Reported results cover the tested targets, contexts, samplers and request limits.

Sibling target: FP8.