README.md

metadata

title: LLaDA-8B-Instruct
emoji: 🌊
colorFrom: blue
colorTo: purple
sdk: gradio
sdk_version: 5.49.1
python_version: '3.12'
app_file: app.py
pinned: false
license: other
startup_duration_timeout: 1h
short_description: Native LLaDA with diffusion and PMI Viterbi decoding
models:
  - GSAI-ML/LLaDA-8B-Instruct

LLaDA-8B-Instruct

Loads the official GSAI-ML/LLaDA-8B-Instruct directly in BF16 on ZeroGPU. Model and tokenizer revision: 08b83a6feb34df1a6011b80c3c00c7563e963b07.

The interface shares the BYOD Gemma demo's decoding controls: temperature, top-k, block diffusion, confidence-guided or random retention, PMI Viterbi, ending delay, early stopping, and the live token visualization. The model uses its native bidirectional forward pass, chat template, and mask token 126336. This is a comparison using the shared app decoder, not a claim to reproduce the official benchmark sampler.

PMI uses a separate scorer trained on all 980,000 responses in the existing training split, retokenized with LLaDA's tokenizer. It covers all 126,464 model output IDs, including output padding beyond the tokenizer vocabulary. Scoring is PMI, retention confidence is diffusion probability, minimum count is 1, and clipping is ±2. Edit the constants near the top of app.py to change these defaults. Temperature and top-k are shared controls; bigram weight is fixed at 0.5. Ending-delay strength remains adjustable.

LLaDA's available strategies are confidence-guided, random, and PMI Viterbi. The adapter-based autoregressive confidence strategy applies to BYOD models. The first request may take longer while ZeroGPU allocates hardware.

Optional repetition-aware PMI Viterbi

Select PMI Viterbi + Repetition Penalty to use the block-local transition score 0.5 * temperature * PMI(a,b) - gamma * C(a,b). Gamma defaults to 1.0; 0 reproduces the original PMI decoder. C accumulates earlier expected bigram occurrences from full-vocabulary diffusion probabilities, updated only after scoring each position. EOS/EOT pairs are neither penalized nor counted. Retained tokens remain fixed constraints. Counts reset each iteration and block, without crossing prompt/block boundaries. There are no additional model forward passes. The original Bigram Viterbi option and generation defaults remain available.

Adaptive Parallel Decoding

The Adaptive Parallel Decoding (APD) checkbox defaults OFF. APD is a commitment layer after the selected sampling or sequence decoder. Each block uses a cosine confidence threshold from 0.90 to 0.50, with a 15% warmup. It commits all qualifying masked positions using their selected token's original full-vocabulary diffusion probability. Remaining positions stay masked; a best-confidence fallback makes progress and the final step resolves all masks. Completed blocks exit early, independently of the convergence-stop checkbox. Existing EOS/EOT retention delay is respected, with endings retained after ordinary predictions; whole-answer EOS/EOT stopping still follows Early stopping. Committed values stay fixed. Optional AR retention scoring and retained-token revision are bypassed under APD. No additional model forwards, token budgets, position factors, or grinding-phase detection are used. OFF retains the original decoding configuration. This implements the requested APD subset, inspired by CAI-dLLM, rather than its complete controller.

Optional sequence sampling and candidate-occurrence counts

Both checkboxes default OFF, preserving existing decoding. Sample bigram-guided sequences replaces the maximum-path decision with exact forward-filter/backward sampling from exp(total path score) on the same candidate lattice. Diffusion, PMI, boundaries and any repetition scores are unchanged; no score perturbations or extra model forwards are used. Seed controls reproducibility. This applies to either bigram decoding strategy, independently of APD.

Count every candidate bigram occurrence applies only with the repetition penalty strategy. Each earlier adjacent pair of candidate sets containing (a,b) adds 1 to C(a,b), irrespective of probability. Duplicate fixed-token columns count once; EOS/EOT pairs are excluded. Counts update after scoring each edge and reset each block/iteration. Gamma still subtracts gamma*C in log-score units; this is not a percentage-parameter remapping. Gamma=0 restores unpenalized results for the selected maximum-path or sampling mode. Ordinary decoding ignores these checkboxes; the original temperature weighting is unchanged.

Independent temperature and bigram weight

The LLaDA Space now uses S(y) = sum log p_t(y_t) + w sum PMI - gamma sum C. Bigram weight ranges from 0 to 5, default 1. A weight of 1 gives equal coefficients, not necessarily equal numerical influence. Weight 0 disables PMI but does not disable repetition penalties. The previous fixed 0.5*temperature PMI coefficient described above is superseded for this Space.

Sample bigram-guided sequences samples P(y) proportional to exp(S(y)/T). Temperature controls the entropy of the whole sequence score, including repetition, without changing the ratios of score coefficients. T=0 selects the maximum path; positive T is floored to 1e-6 for numerical stability. Maximum-path Viterbi ignores temperature. Raw full-vocabulary diffusion probabilities still determine expected repetition counts and retention/APD confidence. Default T is now 1, w is 1, matching the former T=2 PMI coefficient and sequence distribution. Ordinary diffusion retains its usual temperature behavior. No extra model forwards.

Unseen pairs have a neutral PMI correction of zero, never a forbidden transition. Conditional scoring instead uses smoothed finite probabilities. Expected repetition counts remain the default; the optional candidate-set count mode is unchanged.

Optional repetition-risk penalty

With PMI Viterbi + Repetition Penalty, enable Repetition-risk penalty (default OFF) to estimate earlier repetition risk from joint diffusion+PMI sequence probabilities. This overrides the candidate-count checkbox if both are enabled; OFF preserves the existing expected/candidate-count choice.

A forward/backward sum-product pass on the unpenalized candidate lattice yields q_s(a,b). Earlier risk is approximated as r_t=1-prod_(s<t)(1-q_s). The raw log-score correction is -gamma*log((1-r+9r)/(1-r)). At T=1, gamma=1, risk 0.1 gives multiplier 0.5; risk 0.5 gives multiplier 0.1. Sequence temperature subsequently scales the entire corrected score as usual. This is an approximate risk heuristic, not the exact conditional probability of repetition or a guarantee on normalized output probabilities. Overlapping occurrences are dependent.

Risk uses the same temperature as sequence sampling; maximum-path decoding uses a T=1 soft distribution, including when sampling temperature is zero. Fixed positions and existing PMI boundary terms are included. Duplicate fixed columns have zero marginal. Counts reset per iteration/block and do not cross prompt or block boundaries. EOS/EOT pairs neither seed risk nor receive penalties. Gamma=0 matches the unpenalized selected decoding mode exactly. A 1e-7 survival floor keeps near-certain risk finite. Marginals are computed once before risk adjustment; no iterative fixed-point calculation is performed.

There are no extra model forwards, external language models, or training changes. The extra decoder passes add overhead. Existing defaults and other Spaces remain unchanged. Independent temperature/bigram weight controls are preserved.

Early-iteration repetition-penalty fade

Fade repetition penalty over early iterations is ON by default in the UI for LLaDA. It only affects the repetition strategy, including expected-count, candidate-count and risk modes. Turn it OFF to retain constant strength. API calls that omit decay_repetition_penalty retain constant strength for compatibility; pass true explicitly to match the new UI.

Gamma_t = gamma * f(t), with zero-based iteration t within each block. Let L=6, c=(L-1)/2 and g(t)=1/(1+exp(1.6*(t-c))). Then f(t)=(g(t)-g(L-1))/(g(0)-g(L-1)) for 0<=t<L-1, and zero afterward. The first iteration is exactly full strength. From iteration 6 the penalty is exactly zero, not merely asymptotically small, so the original unpenalized sequence decoder runs and risk/count computations are bypassed.

The active window is absolute and does not stretch with total NFE. A one-step block receives full strength, while longer blocks spend their later iterations without repetition penalties. The schedule resets at each new block. APD early exit does not rescale it. Edit REPETITION_FADE_ITERATIONS in app.py to change the window. The sigmoid slope is fixed at 1.6. Sampling, diffusion confidence, PMI, temperature and ending retention keep their existing scoring behavior.

There are no extra model forwards. All existing penalty modes and the constant option remain available; improvement across budgets still needs evaluation.

Tunable APD aggressiveness

With Adaptive Parallel Decoding (APD) ON, APD aggressiveness ranges from 0 to 1 and defaults to 0. The original warmup/cosine threshold schedule is multiplied by (1-a). Thus a=0 retains 0.90 to 0.50, a=0.5 gives 0.45 to 0.25, a=0.9 gives 0.09 to 0.05, and a=1 gives zero threshold, committing all finite selected-token predictions in one forward per block. Higher values can reduce passes but may reduce generation quality by locking less-confident predictions.

The control is interactive only while APD is enabled. API parameter is apd_aggressiveness, default 0. APD OFF ignores it. The zero default reproduces existing APD paths, thresholds and status text. Selected-token confidence stays full-vocabulary and untempered, including when Viterbi selects a non-argmax token. Progress fallback, final cleanup, MASK validity fallback, fixed positions, ending retention delay, and early block exit remain unchanged. No extra model forwards.

Works with all existing decoding strategies, sequence sampling, repetition-risk penalties and their fade. It changes commitment only. Repetition fading and all other settings remain unchanged. Only this LLaDA Space was updated.

APD minimum commitment per iteration

APD now commits at least ceil(block_length / allocated_block_iterations) pending positions on each pass, capped by the number still masked. For a 64-token block with 16 iterations, the minimum is 4 tokens per iteration. Nondivisible lengths round up: a 65-token block at 16 iterations has a minimum of 5 until fewer remain. This avoids leaving a large unresolved tail for the final pass. This is a fixed per-step floor, not a reduced cumulative target after an earlier large commit.

All threshold-qualified pending positions still commit, even above the quota. If they are fewer than the quota, fill the deficit by selected-token diffusion confidence, without changing the selected IDs or adding model calls. Frozen rows are excluded. With EOS/EOT delay, fill ordinary rows first; if fewer ordinary rows remain than needed, fill endings only after all ordinary rows have been selected. Final cleanup and early block exit remain. Each block uses its own length and allocated iteration budget; unequal final blocks are handled independently.

No new UI control is added. APD OFF is unchanged. At aggressiveness 0 the original threshold curve remains, but the quota is now guaranteed; the previous claim of identical full APD behavior at zero is superseded by this intended commitment change. Aggressiveness still lowers thresholds and can commit more than the quota. All decoding strategies, sequence sampling, risk/expected/candidate penalties, repetition decay and ending safeguards remain available. Other Spaces were not updated. No extra model forwards or training changes are introduced.

Optional expanded-corpus bigram and trigram guidance

N-gram guidance defaults to Original bigram, preserving the existing checkpoint and behavior. Expanded bigram uses the new corpus; Expanded bigram + trigram adds a second-order correction to that same expanded bigram. These choices apply to both bigram decoding strategies, including sequence sampling, repetition penalties/risk/decay, APD and its minimum quota. Ordinary independent diffusion ignores this setting. Existing weights and model settings are unchanged; no extra diffusion-model forwards or language model are needed.

The new corpus adds 800M LLaDA-tokenized tokens: 720M FineWeb-Edu English, 40M educational Python and 40M FineMath-4+, plus all original training responses. The added-token ratio is 90% / 5% / 5%, not the initially proposed 50/25/25 mix. Exact per-source text duplicates are skipped. Document boundaries are preserved; EOS is appended to complete raw documents and nontruncated original responses. The final sampled raw document may be cut to its exact token budget. Both new scorers use the same corpus and exact tokenizer, not separately sampled data.

Sources and attribution:

• FineWeb-Edu, sample-10BT.
• Python-Edu, via the text-resolved mirror. The upstream publishes blob metadata, not file text; the mirror resolves it.
• FineMath-4+. Pinned revisions, exact file paths, token/document counts and hashes accompany these checkpoints in the local experiment/training records.

Score: sum log p_t + wsum(PMI bigram) + 0.5w*sum(delta trigram) - repetition. Delta = log P(c|a,b) - log P(c|b), with alpha=50 bigram-backoff smoothing, minimum trigram count 2, and delta clipped to [-2,2]. Unseen/singleton triples give zero correction (backoff), never forbidden transitions. All bigram counts are retained (PMI minimum count 1). The deployed trigram checkpoint stores triples observed at least twice and their required contexts, retaining full prefix totals including singleton contributions. Scorers are downloaded at startup from a pinned model-repository revision, avoiding the Space's 1 GB storage limit. The conditional bigram is the same smoothed backoff model used by the bigram checkpoint. Edit TRIGRAM_RELATIVE_WEIGHT/MIN_COUNT/CLIP in app.py to tune it.

Trigrams are evaluated only wholly inside the current block; existing bigram boundary terms remain. Fixed positions are singleton constraints. Exact max-path, forward/backward sampling, path margins and joint bigram marginals use the second-order chain. In risk mode, earlier-repetition estimates include trigram scores as well as diffusion and PMI. Sequence temperature scales the complete score; bigram weight 0 disables both n-gram corrections. For blocks of 1–2 tokens there is no trigram term. The default Original bigram path is unchanged.

GPU candidate dimensions are vectorized, but trigram cost grows as K^3 rather than K^2. Start at Top-k 16. At K=16, B=64 the extra FP32 trigram-potential table is about 1 MiB; checkpoints and temporary buffers add memory. Longer corpus matching does not check factual relevance to the question. Quality benefits and latency overhead must be evaluated rather than assumed.