Apply numerically validated dialogue step250 runtime patch

README.md

@@ -1,128 +1,72 @@
 ---
 license: other
 license_link: https://huggingface.co/LiquidAI/LFM2.5-24B-A2B/blob/main/LICENSE
-language:
-- en
-- ja
+language: [en, ar, zh, fr, de, it, ja, ko, pt, es, vi, th, id, hi, ru, pl]
 pipeline_tag: text-generation
-tags:
-- diffusion
-- lfm2.5
-- experimental
-- moe
+tags: [diffusion, lfm2.5, experimental, moe]
 base_model: LiquidAI/LFM2.5-24B-A2B
 ---
+# Diffusion LFM2.5-24B-A2B — experimental dialogue step250 patch
 
-# pre
-# Diffusion LFM2.5-24B-A2B — experimental merged checkpoint
+**This is an experimental candidate, not a quality-approved conversational model.**
+Post-training conversation evaluation is incomplete. Word truncation, malformed text and weak instruction
+following were observed in earlier tests. There is no verified claim of correct conversation at1000tokens/s.
+This is not an official Liquid AI release.
 
-This repository contains an experimental block masked-diffusion fine-tune of
-[`LiquidAI/LFM2.5-24B-A2B`](https://huggingface.co/LiquidAI/LFM2.5-24B-A2B),
-not an official Liquid AI release. It is intended for research and evaluation.
+## Current packaging
 
-## Important quality notice
+The BF16 weight-merge attempt did not meet the declared fidelity gate. Main retains the original base weights and applies the step250 adapter at runtime through the companion loader. This preserves the split-GEMM adapter computation instead of calling an approximate merge equivalent.
 
-**The model is not suitable for reliable general-purpose use.** Tested outputs
-show severe repetition, mixed-language fragments, malformed Python and token
-loops. The checkpoint must not be described as a quality-preserving or generally
-usable 600-token/s model. The approximately 600 token/s measurement was on one
-Colab A100 80GB with input 256 tokens, actual output 1024 tokens, a 1024-token
-canvas and 12 denoising steps. A repeated-text health screen failed badly. Short
-questions have a different output length and measured a different rate.
+The step250 adapter was trained against repository revision `2d791e03b14bb42b4323049fbfd832415d32b055`.
+Its SHA256 is `83ffc6f5bff2622a90e635629b03855e5e35456ac1c78870d7859ab567a55244`. The learned mask embedding is included in
+`diffusion.safetensors`. See `merge-verification.json` for actual fixed-probe comparisons, thresholds,
+input-embedding preservation and serialization checks. BF16 rounding can change logits and generated tokens.
+These tests do not certify factual correctness, fluency, all-language support or general output equivalence.
 
-The 600 token/s recommendation is only a latency experiment configuration; it
-is **not** a quality recommendation. Prefer autoregressive decoding from the
-upstream base model for general answers until this checkpoint passes broader
-independent evaluations.
+## Diffusion inference
 
-## Measured experimental settings
-
-| Goal | Setting | Measured result |
-|---|---|---|
-| Closest tested speed to 600, not quality | batch 1, input 256, canvas 1024, 12 denoising steps, max output 1024 | 601.95 output tokens/s median, 3 warmups + 5 timed repeats, A100-SXM4-80GB |
-| Alternative latency point, not quality | batch 1, input 256, canvas 256, 4 denoising steps, max output 1024 | 583.93 output tokens/s median, same A100 procedure |
-| Alternative latency point, not quality | batch 1, input 256, canvas 512, 7 denoising steps, max output 1024 | 639.16 output tokens/s median, same A100 procedure |
-
-No high-quality setting was found in the tested 1–15 denoising-step sweep. For
-latency reproduction only, use a 1024-token canvas and 12 steps. Expect output
-to be poor and output-length-dependent. Do not call this the highest-quality
-setting. More denoising steps did not repair the learned degeneration. The
-earlier 256-canvas/3-step timing was from the pre-merge adapter and does not
-describe this merged repository.
-
-## Training and evaluation provenance
-
-- Upstream model revision: `3dc4cd09c9a2a99e1e819693e79d3aba5e9bb4f2`.
-- LoRA rank 64, alpha 128, merged into BF16 base weights and packed MoE expert
-  weights, based on the 3100-update checkpoint selected by fixed validation
-  masked-token NLL.
-- Training: Alpaca train examples and ARC-Easy train; validation records and
-  final test records excluded from training. Data license terms remain applicable.
-- On 64 fixed validation examples, half-mask NLL changed 3.0261 → 2.7118
-  (10.39% reduction); full-mask NLL changed 5.8993 → 5.6291 (4.58% reduction).
-  The jointly agreed 10% target on both corruption levels was not met.
-- On the separate 256-example final test, half-mask NLL changed 3.0005 → 2.7312
-  (8.97% reduction), full-mask NLL 5.8205 → 5.6774 (2.46% reduction).
-- These losses are denoising cross-entropy, not autoregressive perplexity and
-  not a measure of free-generation correctness.
-
-## Inference
-
-The checkpoint uses a custom block sampler and an attention mode distinct from
-ordinary causal generation; stock `generate()` is **not** the diffusion sampler.
-See `inference_recipe.json` and `modeling_diffusion_lfm.py` for the tested sampler.
-Example:
+Download `modeling_diffusion_lfm.py` next to your script, then use the companion sampler:
 
 ```python
+import torch
 from transformers import AutoTokenizer
-from modeling_diffusion_lfm import from_pretrained
-
-repo = "summerAIO/diffusion-lfm2.5-24b-a2b-merged"
+from modeling_diffusion_lfm import from_pretrained, prepare_prompt
+repo = "summerAIO/diffusion-lfm2.5-24b-a2b"
 tokenizer = AutoTokenizer.from_pretrained(repo)
-model = from_pretrained(repo, torch_dtype="bfloat16", device_map="auto")
-prompt = tokenizer.apply_chat_template(
-    [{"role": "user", "content": "Explain masked diffusion briefly."}],
-    tokenize=True, add_generation_prompt=True, return_tensors="pt",
-).to(model.causal_model.device)
-tokens, stats = model.generate(prompt, max_new_tokens=1024, block_size=1024, steps=12)
+sampler = from_pretrained(repo, dtype=torch.bfloat16, device_map={"": 0})
+prompt = prepare_prompt(tokenizer, [{"role":"user", "content":"日本語で短く自己紹介してください。"}],
+                        think="plain", device=sampler.causal_model.device)
+tokens, stats = sampler.generate(prompt, max_new_tokens=128, block_size=16, steps=16)
 print(tokenizer.decode(tokens, skip_special_tokens=True), stats)
 ```
 
-`AutoModelForCausalLM.generate()` alone invokes causal autoregressive decoding,
-not the diffusion sampler. Model loading needs enough GPU
-memory for the BF16 checkpoint; measured packaging/inference target was A100
-80GB. Merged BF16 weights are approximately 48GB before runtime overhead.
-
-## Limitations
+Stock `AutoModelForCausalLM.generate()` is autoregressive decoding, not the diffusion sampler.
+The companion uses causal attention within the prefix and bidirectional attention within the canvas.
+It retains the last256prefix tokens. Runtime/mask downloads respect the requested revision.
+The default plain assistant prefill follows this repair's training format.
 
-- Severe degeneration and invalid free-form output observed.
-- No broad language, code, multilingual or safety benchmark establishes quality.
-- Single A100 setup; latency depends on prompt, output length and hardware.
-- No claim of superiority over the upstream model.
-- The merged standard Transformers checkpoint loads and runs, but a strict
-  masked-logit comparison against the unmerged adapter failed (maximum absolute
-  difference 8.44; top-1 mismatch). Treat the merged export as a distinct,
-  experimentally verified artifact rather than an exact numerically equivalent
-  conversion of the adapter.
-- A fixed three-prompt sweep at 1–15 denoising steps found **no setting** that
-  passed the tested arithmetic-mean code check, exact short-answer check and
-  repetition screen together. Therefore no high-quality inference setting is
-  recommended. The 1024-canvas/12-step setting is listed only to reproduce latency.
+BF16 weights require an80GB-class GPU with runtime headroom. Use Transformers5.0.0 or a tested compatible
+version. No throughput result for this patched checkpoint is certified. Legacy latency measurements
+belong to the previous revision and must not be presented as quality-preserving results for this patch.
 
-## License
+## Training and languages
 
-The upstream checkpoint uses the [LFM Open License v1.0](https://huggingface.co/LiquidAI/LFM2.5-24B-A2B/blob/main/LICENSE).
-It permits redistribution of derivative works with attribution and a copy of
-the license, and includes a commercial-use threshold: commercial use by an
-entity with annual revenue at or above USD 10M is not licensed by this grant.
+The repair uses rank16, scale2 custom LoRA and a learned mask vector. It includes OASST2 dialogue data
+and Aya replay targeting all16official upstream languages. Japanese dialogue was emphasized. Dataset
+coverage and language metadata do not certify fluency; several language-specific issues remain.
+The saved250-update candidate reduced the small fixed validation Japanese next-mask NLL to1.578991 and
+multilingual span NLL to5.471355. These are denoising losses, not autoregressive perplexity or accuracy.
 
-Training data: [Alpaca](https://huggingface.co/datasets/tatsu-lab/alpaca) is CC-BY-NC-4.0 and [ARC](https://huggingface.co/datasets/allenai/ai2_arc) is CC-BY-SA-4.0. These data
-licenses may impose additional restrictions on use and distribution of derived
-artifacts; the upstream model license does not override them. This model card
-does not relicense the base model or training data. Review the linked license
-texts and determine redistribution/use permissions for your circumstances.
+## Provenance and limitations
 
-## References
+- Source candidate training run: a27d5a26-edfb-400c-84c0-a27a47a09e54; post-training conversation retest incomplete.
+- Merge/publication run: a7de0438-35ea-4afc-a940-fc0ed80618d1.
+- Staging branch: `dialogue-step250-merge-bb4aea9b`.
+- Earlier base export's failure to match the historical unmerged step3100 adapter remains a separate issue.
+- The preserved `dialogue_step250/` tensors use this project's custom LoRA format, not a PEFT drop-in package.
+- The original model license is LFM Open License v1.0. Original Alpaca CC-BY-NC4.0 and ARC CC-BY-SA4.0
+  data provenance remains applicable; the repair additionally uses Dolly CC-BY-SA3.0 and OASST2/Aya Apache2.0.
+  Existing license notices are retained. This card does not relicense upstream weights or data.
 
-- Base model: https://huggingface.co/LiquidAI/LFM2.5-24B-A2B
-- Experimental code and measured logs are in this project's reproducibility archive.
+Historical results and files are available at the previous pinned revision:
+https://huggingface.co/summerAIO/diffusion-lfm2.5-24b-a2b/tree/2d791e03b14bb42b4323049fbfd832415d32b055

config.json

@@ -1,15 +1,23 @@
 {
-  "architectures": [
-    "Lfm2MoeForCausalLM"
-  ],
-  "bos_token_id": 124894,
-  "conv_L_cache": 3,
-  "conv_bias": false,
-  "dtype": "bfloat16",
-  "eos_token_id": 124900,
+  "vocab_size": 128000,
   "hidden_size": 2048,
-  "initializer_range": 0.02,
   "intermediate_size": 11776,
+  "num_hidden_layers": 40,
+  "max_position_embeddings": 131072,
+  "initializer_range": 0.02,
+  "use_cache": true,
+  "norm_eps": 1e-05,
+  "num_attention_heads": 32,
+  "num_key_value_heads": 8,
+  "conv_bias": false,
+  "conv_L_cache": 3,
+  "num_dense_layers": 2,
+  "moe_intermediate_size": 1536,
+  "num_experts_per_tok": 4,
+  "num_experts": 64,
+  "use_expert_bias": true,
+  "routed_scaling_factor": 1.0,
+  "norm_topk_prob": true,
   "layer_types": [
     "conv",
     "conv",
@@ -52,26 +60,37 @@
     "full_attention",
     "conv"
   ],
-  "max_position_embeddings": 131072,
-  "model_type": "lfm2_moe",
-  "moe_intermediate_size": 1536,
-  "norm_eps": 1e-05,
-  "norm_topk_prob": true,
-  "num_attention_heads": 32,
-  "num_dense_layers": 2,
-  "num_experts": 64,
-  "num_experts_per_tok": 4,
-  "num_hidden_layers": 40,
-  "num_key_value_heads": 8,
-  "pad_token_id": 124893,
   "rope_parameters": {
     "rope_theta": 10000000.0,
     "rope_type": "default"
   },
-  "routed_scaling_factor": 1.0,
   "tie_word_embeddings": true,
+  "pad_token_id": 124893,
+  "bos_token_id": 124894,
+  "eos_token_id": 124900,
+  "return_dict": true,
+  "output_hidden_states": false,
+  "dtype": "bfloat16",
+  "chunk_size_feed_forward": 0,
+  "is_encoder_decoder": false,
+  "architectures": [
+    "Lfm2MoeForCausalLM"
+  ],
+  "id2label": {
+    "0": "LABEL_0",
+    "1": "LABEL_1"
+  },
+  "label2id": {
+    "LABEL_0": 0,
+    "LABEL_1": 1
+  },
+  "problem_type": null,
+  "_name_or_path": "summerAIO/diffusion-lfm2.5-24b-a2b",
   "transformers_version": "5.0.0",
-  "use_cache": true,
-  "use_expert_bias": true,
-  "vocab_size": 128000
-}
+  "model_type": "lfm2_moe",
+  "output_attentions": false,
+  "dialogue_adapter_merged": false,
+  "dialogue_runtime_patch": true,
+  "dialogue_adapter_path": "dialogue_step250",
+  "dialogue_source_base_revision": "2d791e03b14bb42b4323049fbfd832415d32b055"
+}

dialogue_step250/adapter.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:83ffc6f5bff2622a90e635629b03855e5e35456ac1c78870d7859ab567a55244
+size 45775632

dialogue_step250/adapter_config.json

@@ -0,0 +1,158 @@
+{
+  "rank": 16,
+  "scale": 2.0,
+  "base_model": "summerAIO/diffusion-lfm2.5-24b-a2b",
+  "base_revision": "2d791e03b14bb42b4323049fbfd832415d32b055",
+  "parent_adapter_sha256": "e63a39a1bb579e5712599c9d18d3059458ee5085605a90b5f93d9b5395312331",
+  "data": {
+    "tokenizer_revision": "2d791e03b14bb42b4323049fbfd832415d32b055",
+    "oasst2_revision": "179dd21fc55192153d94adb0e0ce8f69e222bf75",
+    "aya_revision": "f9ea04583f02a8f86404ff6c58bf75fe637df8a2",
+    "aya_original_split_fingerprint": "5062fb630a48fb952f8f2eb50169d7c0cff839a7e6d9626c14f443ca60c00d2b",
+    "oasst2_license": "Apache-2.0",
+    "aya_license": "Apache-2.0",
+    "annotator_user_ids_retained": false,
+    "conversation_counts": {
+      "train": {
+        "en": 256,
+        "ar": 12,
+        "zh": 256,
+        "fr": 256,
+        "de": 256,
+        "it": 97,
+        "ja": 108,
+        "ko": 3,
+        "pt": 0,
+        "es": 256,
+        "vi": 26,
+        "th": 256,
+        "id": 2,
+        "hi": 0,
+        "ru": 256,
+        "pl": 49
+      },
+      "validation": {
+        "en": 16,
+        "ar": 1,
+        "zh": 16,
+        "fr": 16,
+        "de": 16,
+        "it": 14,
+        "ja": 13,
+        "ko": 0,
+        "pt": 0,
+        "es": 16,
+        "vi": 6,
+        "th": 16,
+        "id": 0,
+        "hi": 0,
+        "ru": 16,
+        "pl": 5
+      },
+      "test": {
+        "en": 16,
+        "ar": 0,
+        "zh": 16,
+        "fr": 16,
+        "de": 16,
+        "it": 10,
+        "ja": 16,
+        "ko": 0,
+        "pt": 0,
+        "es": 16,
+        "vi": 7,
+        "th": 16,
+        "id": 0,
+        "hi": 0,
+        "ru": 16,
+        "pl": 4
+      }
+    },
+    "replay_counts": {
+      "train": {
+        "en": 31,
+        "ar": 27,
+        "zh": 27,
+        "fr": 32,
+        "de": 29,
+        "it": 30,
+        "ja": 15,
+        "ko": 29,
+        "pt": 28,
+        "es": 30,
+        "vi": 31,
+        "th": 29,
+        "id": 30,
+        "hi": 30,
+        "ru": 31,
+        "pl": 30
+      },
+      "validation": {
+        "en": 8,
+        "ar": 8,
+        "zh": 8,
+        "fr": 7,
+        "de": 8,
+        "it": 8,
+        "ja": 2,
+        "ko": 8,
+        "pt": 7,
+        "es": 8,
+        "vi": 8,
+        "th": 6,
+        "id": 7,
+        "hi": 8,
+        "ru": 7,
+        "pl": 8
+      },
+      "test": {
+        "en": 7,
+        "ar": 8,
+        "zh": 7,
+        "fr": 8,
+        "de": 8,
+        "it": 7,
+        "ja": 4,
+        "ko": 8,
+        "pt": 7,
+        "es": 8,
+        "vi": 8,
+        "th": 8,
+        "id": 8,
+        "hi": 8,
+        "ru": 8,
+        "pl": 8
+      }
+    },
+    "forbidden_evaluation_prompts": 94
+  },
+  "ancestor_data_licenses": [
+    "CC-BY-SA-3.0 Dolly",
+    "upstream model and data terms"
+  ],
+  "quality_approved": false,
+  "step": 250,
+  "validation": {
+    "japanese_next_nll": 1.5789912194013596,
+    "multilingual_span_nll": 5.471354529261589,
+    "per_language": {
+      "en": 2.9787462949752808,
+      "ar": 6.015078067779541,
+      "zh": 4.236312627792358,
+      "fr": 5.8818418979644775,
+      "de": 6.675511121749878,
+      "it": 5.073833584785461,
+      "ja": 4.965384483337402,
+      "ko": 5.386567115783691,
+      "pt": 4.24447226524353,
+      "es": 3.6444900035858154,
+      "vi": 7.3604865074157715,
+      "th": 5.754176378250122,
+      "id": 5.001194477081299,
+      "hi": 6.649858236312866,
+      "ru": 7.102121353149414,
+      "pl": 6.571598052978516
+    },
+    "selection_score": 3.5251728743314743
+  }
+}

diffusion.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:aac26b643d6f07f4f68057c9ab8fb6d56ac1e9303af6d6775c8306047aa89f40
-size 4184
+oid sha256:8a2f034120cded559c16b298b88f155ad4506cdc40b480d364cba556c3ab09ff
+size 8280

inference_recipe.json

@@ -1,22 +1,14 @@
 {
   "model_type": "lfm2_moe",
-  "method": "block masked diffusion (experimental)",
-  "model_id": "LiquidAI/LFM2.5-24B-A2B",
-  "revision": "3dc4cd09c9a2a99e1e819693e79d3aba5e9bb4f2",
-  "checkpoint_total_updates": 3100,
-  "lora_rank_merged": 64,
-  "mask_embedding_file": "diffusion.safetensors",
+  "packaging": "exact_runtime_adapter_patch",
   "sampler_source": "modeling_diffusion_lfm.py",
-  "warning": "This configuration is for latency reproduction only; generated output is severely repetitive and low quality.",
-  "latency_reproduction_setting": {
-    "batch_size": 1,
-    "input_tokens": 256,
-    "canvas_tokens": 1024,
-    "denoising_steps": 12,
-    "max_new_tokens": 1024,
-    "measured_median_tokens_per_second": 601.9543854074814,
-    "hardware": "NVIDIA A100-SXM4-80GB",
-    "quality_approved": false,
-    "warning": "This setting reproduces latency, not useful output quality."
-  }
+  "mask_embedding_file": "diffusion.safetensors",
+  "raw_sampler": {
+    "block_size": 16,
+    "steps": 16,
+    "prefix_tokens": 256
+  },
+  "quality_approved": false,
+  "current_revision_speed_measured": false,
+  "legacy_results_revision": "2d791e03b14bb42b4323049fbfd832415d32b055"
 }

merge-equivalence-warning.json

@@ -1,6 +1,7 @@
 {
-  "max_abs_logit_diff": 8.4375,
-  "mean_abs_logit_diff": 0.3472544550895691,
-  "top1_match": false,
-  "strict_equivalence_pass": false
+  "strict_bitwise_equivalence": false,
+  "approximate_bf16_gate": false,
+  "packaging": "exact_runtime_adapter_patch",
+  "warning": "Merge fidelity tests are limited to fixed probes. Natural conversation quality and1000tok/s are not verified.",
+  "base_issue": "Historical step3100 export equivalence is a separate unresolved issue."
 }

modeling_diffusion_lfm.py

@@ -1,90 +1,138 @@
-"""Research sampler companion for the merged LFM2.5 masked-diffusion checkpoint."""
+"""Masked-diffusion companion for experimental LFM dialogue step250 packages."""
 import torch
 from torch import nn
+import torch.nn.functional as F
+
+
+class DialogueLinear(nn.Module):
+    def __init__(self,base,a,b,scale):
+        super().__init__()
+        self.base = base
+        self.lora_A = nn.Parameter(a.to(device=base.weight.device,dtype=torch.float32),requires_grad=False)
+        self.lora_B = nn.Parameter(b.to(device=base.weight.device,dtype=torch.float32),requires_grad=False)
+        self.scale = float(scale)
+    @property
+    def weight(self):
+        return self.base.weight
+    def forward(self,x):
+        delta = F.linear(F.linear(x.float(),self.lora_A),self.lora_B)
+        return self.base(x)+(delta*self.scale).to(x.dtype)
+
+
+def apply_dialogue_patch(model,state,scale):
+    """Exact split-GEMM runtime representation; not a BF16 weight merge."""
+    state = dict(state)
+    mask = state.pop('mask_embedding')
+    names = sorted(k[:-7] for k in state if k.endswith('.lora_A'))
+    for name in names:
+        base = model.get_submodule(name)
+        a,b = state.pop(name+'.lora_A'),state.pop(name+'.lora_B')
+        if not isinstance(base,nn.Linear) or a.shape[1]!=base.in_features or b.shape!=(base.out_features,a.shape[0]):
+            raise ValueError('Invalid dialogue adapter: '+name)
+        parent,_,leaf = name.rpartition('.')
+        setattr(model.get_submodule(parent) if parent else model,leaf,DialogueLinear(base,a,b,scale))
+    if state:
+        raise ValueError('Unexpected dialogue adapter tensors')
+    return mask
 
 
 class DiffusionLFM(nn.Module):
-    """Use bidirectional canvas attention and the saved learned mask embedding."""
-    def __init__(self, causal_model, mask_embedding):
+    def __init__(self,causal_model,mask_embedding):
         super().__init__()
         self.model = causal_model
-        self.mask_embedding = nn.Parameter(mask_embedding.detach().clone(), requires_grad=False)
+        self.mask_embedding = nn.Parameter(mask_embedding.detach().clone(),requires_grad=False)
         for layer in self.model.model.layers:
             if layer.is_attention_layer:
                 layer.self_attn.is_causal = False
-
     @property
     def causal_model(self):
         return self.model
-
-    def hidden(self, ids, masked, valid, prefix_lengths):
-        m=self.model.model
-        hidden=m.embed_tokens(ids)
-        hidden=torch.where(masked[...,None],self.mask_embedding.to(hidden.dtype),hidden)
-        positions=torch.arange(ids.shape[1],device=ids.device)[None]
-        rotary=m.pos_emb(hidden,positions)
-        length=ids.shape[1]
-        q=torch.arange(length,device=ids.device)[None,:,None]
-        k=torch.arange(length,device=ids.device)[None,None,:]
-        p=prefix_lengths[:,None,None]
-        allowed=torch.where(q<p,k<=q,torch.ones_like(k<=q)) & valid[:,None,:].bool()
-        attn=torch.zeros_like(allowed,dtype=hidden.dtype).masked_fill(~allowed,torch.finfo(hidden.dtype).min)[:,None]
+    def hidden(self,ids,masked,valid,prefix_lengths):
+        m = self.model.model
+        h = m.embed_tokens(ids)
+        h = torch.where(masked[...,None],self.mask_embedding.to(h.dtype),h)
+        positions = torch.arange(ids.shape[1],device=ids.device)[None]
+        rotary = m.pos_emb(h,positions)
+        q,k = positions[:,:,None],positions[:,None,:]
+        allowed = ((q>=prefix_lengths[:,None,None]) | (k<=q)) & valid[:,None,:].bool()
+        attn = torch.zeros_like(allowed,dtype=h.dtype).masked_fill(~allowed,torch.finfo(h.dtype).min)[:,None]
         for layer in m.layers:
-            mask=attn if layer.is_attention_layer else valid
-            hidden=layer(hidden,position_embeddings=rotary,attention_mask=mask,
-                         position_ids=positions,past_key_values=None)
-        return m.embedding_norm(hidden)
-
+            h = layer(h,position_embeddings=rotary,attention_mask=attn if layer.is_attention_layer else valid,
+                      position_ids=positions,past_key_values=None)
+        return m.embedding_norm(h)
     @torch.inference_mode()
-    def generate(self,prompt_ids,*,max_new_tokens=128,block_size=256,steps=3,temperature=0.0):
-        if prompt_ids.ndim!=2 or prompt_ids.shape[0]!=1 or min(max_new_tokens,block_size,steps)<1:
-            raise ValueError("one prompt and positive generation settings required")
-        tokenizer_pad=getattr(self.model.config,"pad_token_id",None)
-        eos=self.model.config.eos_token_id
-        generated=[];calls=0
+    def generate(self,prompt_ids,*,max_new_tokens=128,block_size=16,steps=16,temperature=0.0):
+        if prompt_ids.ndim!=2 or prompt_ids.shape[0]!=1 or prompt_ids.shape[1]<1 or min(max_new_tokens,block_size,steps)<1:
+            raise ValueError('One nonempty prompt and positive generation settings required')
+        if temperature<0:
+            raise ValueError('Temperature must be nonnegative')
+        pad,eos = self.model.config.pad_token_id,self.model.config.eos_token_id
+        eos_ids = {eos} if isinstance(eos,int) else set(eos or [])
+        generated,calls = [],0
         while len(generated)<max_new_tokens:
-            count=min(block_size,max_new_tokens-len(generated))
-            hist=prompt_ids.new_tensor([generated])
-            prefix=torch.cat((prompt_ids,hist),1)[:,-256:]
-            p=prefix.shape[1]
-            ids=torch.cat((prefix,prefix.new_full((1,count),tokenizer_pad)),1)
-            masked=torch.zeros_like(ids,dtype=torch.bool);masked[:,p:]=True
-            valid=torch.ones_like(masked)
-            for s in range(steps):
-                h=self.hidden(ids,masked,valid,prompt_ids.new_tensor([p]))
-                logits=self.model.lm_head(h[masked]).float()
-                logits[:,tokenizer_pad]=-torch.inf
+            count = min(block_size,max_new_tokens-len(generated))
+            prefix = torch.cat((prompt_ids,prompt_ids.new_tensor([generated])),1)[:,-256:]
+            p = prefix.shape[1]
+            ids = torch.cat((prefix,prefix.new_full((1,count),pad)),1)
+            masked = torch.zeros_like(ids,dtype=torch.bool)
+            masked[:,p:] = True
+            valid = torch.ones_like(masked)
+            for step in range(steps):
+                if not bool(masked.any()):
+                    break
+                h = self.hidden(ids,masked,valid,prefix.new_tensor([p]))
+                logits = self.model.lm_head(h[masked]).float()
+                logits[:,pad] = -torch.inf
                 if temperature:
-                    probs=(logits/temperature).softmax(-1)
-                    tokens=torch.multinomial(probs,1).squeeze(-1)
-                    conf=probs.gather(1,tokens[:,None]).squeeze(1)
+                    probabilities = (logits/temperature).softmax(-1)
+                    tokens = torch.multinomial(probabilities,1).squeeze(-1)
+                    confidence = probabilities.gather(1,tokens[:,None]).squeeze(1)
                 else:
-                    tokens=logits.argmax(-1)
-                    conf=logits.log_softmax(-1).amax(-1)
-                if s==steps-1:
-                    ids[masked]=tokens
+                    tokens = logits.argmax(-1)
+                    confidence = logits.log_softmax(-1).amax(-1)
+                remaining = count*(steps-step-1)//steps
+                commit = int(masked.sum())-remaining
+                if not remaining:
+                    ids[masked] = tokens
                     masked.zero_()
                 else:
-                    remaining=count*(steps-s-1)//steps
-                    commit=int(masked.sum())-remaining
-                    chosen=conf.topk(commit).indices
-                    positions=masked[0].nonzero().squeeze(1)[chosen]
-                    ids[0,positions]=tokens[chosen]
-                    masked[0,positions]=False
-                calls+=1
-            canvas=ids[0,p:].tolist()
-            if eos in canvas:
-                generated.extend(canvas[:canvas.index(eos)])
+                    chosen = confidence.topk(commit).indices
+                    position = masked[0].nonzero().squeeze(1)[chosen]
+                    ids[0,position] = tokens[chosen]
+                    masked[0,position] = False
+                calls += 1
+            canvas = ids[0,p:].tolist()
+            end = next((i for i,t in enumerate(canvas) if t in eos_ids),len(canvas))
+            generated.extend(canvas[:end])
+            if end!=len(canvas):
                 break
-            generated.extend(canvas)
-        return generated,{"forward_calls":calls,"tokens":len(generated)}
+        return generated,{'forward_calls':calls,'tokens':len(generated)}
 
 
-def from_pretrained(model_id,**kwargs):
-    from safetensors.torch import load_file
+def prepare_prompt(tokenizer,messages,*,think='plain',device=None):
+    text = tokenizer.apply_chat_template(messages,tokenize=False,add_generation_prompt=True)
+    if think=='plain' and text.endswith('<think>'):
+        text = text[:-len('<think>')]
+    elif think not in {'plain','keep'}:
+        raise ValueError('think must be plain or keep')
+    ids = tokenizer(text,add_special_tokens=False,return_tensors='pt').input_ids
+    return ids.to(device) if device is not None else ids
+
+
+def from_pretrained(model_id,*,revision=None,**kwargs):
+    import json
+    from pathlib import Path
     from huggingface_hub import hf_hub_download
+    from safetensors.torch import load_file
     from transformers import AutoModelForCausalLM
-    model=AutoModelForCausalLM.from_pretrained(model_id,**kwargs)
-    weights=load_file(hf_hub_download(model_id,"diffusion.safetensors"),device=str(model.device))
-    return DiffusionLFM(model,weights["mask_embedding"])
-
+    kwargs.setdefault('attn_implementation','sdpa')
+    model = AutoModelForCausalLM.from_pretrained(model_id,revision=revision,**kwargs)
+    device = str(model.get_input_embeddings().weight.device)
+    if getattr(model.config,'dialogue_runtime_patch',False) and not getattr(model.config,'dialogue_adapter_merged',False):
+        folder = model.config.dialogue_adapter_path
+        config = json.loads(Path(hf_hub_download(model_id,folder+'/adapter_config.json',revision=revision)).read_text())
+        state = load_file(hf_hub_download(model_id,folder+'/adapter.safetensors',revision=revision),device=device)
+        mask = apply_dialogue_patch(model,state,config['scale'])
+    else:
+        mask = load_file(hf_hub_download(model_id,'diffusion.safetensors',revision=revision),device=device)['mask_embedding']
+    return DiffusionLFM(model,mask).eval()

quality-selection-summary.json

@@ -1,96 +1,29 @@
 {
-  "steps_tested": 15,
-  "quality_approved_steps": [],
-  "best_quality_setting": null,
-  "reason": "All tested settings failed at least one held-out code, exact answer, or generation-health check.",
-  "verified_speed_500_to_700_tokps": [
-    {
-      "block_size": 256,
-      "steps": 4,
-      "median_tokps": 583.931875195536,
-      "long_health": {
-        "alphabetic_fraction": 0.05560791705937795,
-        "distinct_4gram_ratio": 0.04897159647404505,
-        "longest_identical_token_run": 526,
-        "pass_screen": false
-      }
+  "quality_approved": false,
+  "posttraining_conversation_evaluation_complete": false,
+  "source_step": 250,
+  "validation": {
+    "japanese_next_nll": 1.5789912194013596,
+    "multilingual_span_nll": 5.471354529261589,
+    "per_language": {
+      "en": 2.9787462949752808,
+      "ar": 6.015078067779541,
+      "zh": 4.236312627792358,
+      "fr": 5.8818418979644775,
+      "de": 6.675511121749878,
+      "it": 5.073833584785461,
+      "ja": 4.965384483337402,
+      "ko": 5.386567115783691,
+      "pt": 4.24447226524353,
+      "es": 3.6444900035858154,
+      "vi": 7.3604865074157715,
+      "th": 5.754176378250122,
+      "id": 5.001194477081299,
+      "hi": 6.649858236312866,
+      "ru": 7.102121353149414,
+      "pl": 6.571598052978516
     },
-    {
-      "block_size": 512,
-      "steps": 7,
-      "median_tokps": 639.1644566581542,
-      "long_health": {
-        "alphabetic_fraction": 0.1011743450767841,
-        "distinct_4gram_ratio": 0.04995102840352596,
-        "longest_identical_token_run": 914,
-        "pass_screen": false
-      }
-    },
-    {
-      "block_size": 512,
-      "steps": 8,
-      "median_tokps": 559.6046788602476,
-      "long_health": {
-        "alphabetic_fraction": 0.0947176684881603,
-        "distinct_4gram_ratio": 0.04211557296767875,
-        "longest_identical_token_run": 927,
-        "pass_screen": false
-      }
-    },
-    {
-      "block_size": 512,
-      "steps": 9,
-      "median_tokps": 497.3790361543517,
-      "long_health": {
-        "alphabetic_fraction": 0.1055004508566276,
-        "distinct_4gram_ratio": 0.04897159647404505,
-        "longest_identical_token_run": 916,
-        "pass_screen": false
-      }
-    },
-    {
-      "block_size": 1024,
-      "steps": 11,
-      "median_tokps": 656.0337779697369,
-      "long_health": {
-        "alphabetic_fraction": 0.10171017101710171,
-        "distinct_4gram_ratio": 0.03330068560235064,
-        "longest_identical_token_run": 923,
-        "pass_screen": false
-      }
-    },
-    {
-      "block_size": 1024,
-      "steps": 12,
-      "median_tokps": 601.9543854074814,
-      "long_health": {
-        "alphabetic_fraction": 0.10743061772605192,
-        "distinct_4gram_ratio": 0.03819784524975514,
-        "longest_identical_token_run": 924,
-        "pass_screen": false
-      }
-    },
-    {
-      "block_size": 1024,
-      "steps": 13,
-      "median_tokps": 555.3291700396725,
-      "long_health": {
-        "alphabetic_fraction": 0.13823272090988625,
-        "distinct_4gram_ratio": 0.058765915768854066,
-        "longest_identical_token_run": 944,
-        "pass_screen": false
-      }
-    },
-    {
-      "block_size": 1024,
-      "steps": 14,
-      "median_tokps": 514.9606330424041,
-      "long_health": {
-        "alphabetic_fraction": 0.13127753303964756,
-        "distinct_4gram_ratio": 0.05582761998041136,
-        "longest_identical_token_run": 908,
-        "pass_screen": false
-      }
-    }
-  ]
+    "selection_score": 3.5251728743314743
+  },
+  "merge_test_is_not_a_quality_benchmark": true
 }

recommended_settings.json

@@ -1,208 +1,14 @@
 {
-  "candidate_steps_1_to_15": [
-    {
-      "steps": 1,
-      "code_correct": false,
-      "capital_correct": false,
-      "explanations_nondegenerate": false,
-      "distinct_4gram_ratio": 0.21774193548387097
-    },
-    {
-      "steps": 2,
-      "code_correct": false,
-      "capital_correct": false,
-      "explanations_nondegenerate": false,
-      "distinct_4gram_ratio": 0.3306451612903226
-    },
-    {
-      "steps": 3,
-      "code_correct": false,
-      "capital_correct": false,
-      "explanations_nondegenerate": false,
-      "distinct_4gram_ratio": 0.33870967741935487
-    },
-    {
-      "steps": 4,
-      "code_correct": false,
-      "capital_correct": false,
-      "explanations_nondegenerate": false,
-      "distinct_4gram_ratio": 0.29838709677419356
-    },
-    {
-      "steps": 5,
-      "code_correct": false,
-      "capital_correct": false,
-      "explanations_nondegenerate": false,
-      "distinct_4gram_ratio": 0.3709677419354839
-    },
-    {
-      "steps": 6,
-      "code_correct": false,
-      "capital_correct": false,
-      "explanations_nondegenerate": false,
-      "distinct_4gram_ratio": 0.3951612903225807
-    },
-    {
-      "steps": 7,
-      "code_correct": false,
-      "capital_correct": false,
-      "explanations_nondegenerate": false,
-      "distinct_4gram_ratio": 0.4381720430107527
-    },
-    {
-      "steps": 8,
-      "code_correct": false,
-      "capital_correct": false,
-      "explanations_nondegenerate": false,
-      "distinct_4gram_ratio": 0.510752688172043
-    },
-    {
-      "steps": 9,
-      "code_correct": false,
-      "capital_correct": false,
-      "explanations_nondegenerate": true,
-      "distinct_4gram_ratio": 0.41666666666666663
-    },
-    {
-      "steps": 10,
-      "code_correct": false,
-      "capital_correct": false,
-      "explanations_nondegenerate": true,
-      "distinct_4gram_ratio": 0.5752688172043011
-    },
-    {
-      "steps": 11,
-      "code_correct": false,
-      "capital_correct": false,
-      "explanations_nondegenerate": true,
-      "distinct_4gram_ratio": 0.5672043010752689
-    },
-    {
-      "steps": 12,
-      "code_correct": false,
-      "capital_correct": false,
-      "explanations_nondegenerate": true,
-      "distinct_4gram_ratio": 0.532258064516129
-    },
-    {
-      "steps": 13,
-      "code_correct": false,
-      "capital_correct": false,
-      "explanations_nondegenerate": true,
-      "distinct_4gram_ratio": 0.532258064516129
-    },
-    {
-      "steps": 14,
-      "code_correct": false,
-      "capital_correct": false,
-      "explanations_nondegenerate": true,
-      "distinct_4gram_ratio": 0.6129032258064516
-    },
-    {
-      "steps": 15,
-      "code_correct": false,
-      "capital_correct": false,
-      "explanations_nondegenerate": true,
-      "distinct_4gram_ratio": 0.5564516129032258
-    }
-  ],
-  "verified_speed_500_to_700_tokps": [
-    {
-      "block_size": 256,
-      "steps": 4,
-      "median_tokps": 583.931875195536,
-      "long_health": {
-        "alphabetic_fraction": 0.05560791705937795,
-        "distinct_4gram_ratio": 0.04897159647404505,
-        "longest_identical_token_run": 526,
-        "pass_screen": false
-      }
-    },
-    {
-      "block_size": 512,
-      "steps": 7,
-      "median_tokps": 639.1644566581542,
-      "long_health": {
-        "alphabetic_fraction": 0.1011743450767841,
-        "distinct_4gram_ratio": 0.04995102840352596,
-        "longest_identical_token_run": 914,
-        "pass_screen": false
-      }
-    },
-    {
-      "block_size": 512,
-      "steps": 8,
-      "median_tokps": 559.6046788602476,
-      "long_health": {
-        "alphabetic_fraction": 0.0947176684881603,
-        "distinct_4gram_ratio": 0.04211557296767875,
-        "longest_identical_token_run": 927,
-        "pass_screen": false
-      }
-    },
-    {
-      "block_size": 512,
-      "steps": 9,
-      "median_tokps": 497.3790361543517,
-      "long_health": {
-        "alphabetic_fraction": 0.1055004508566276,
-        "distinct_4gram_ratio": 0.04897159647404505,
-        "longest_identical_token_run": 916,
-        "pass_screen": false
-      }
-    },
-    {
-      "block_size": 1024,
-      "steps": 11,
-      "median_tokps": 656.0337779697369,
-      "long_health": {
-        "alphabetic_fraction": 0.10171017101710171,
-        "distinct_4gram_ratio": 0.03330068560235064,
-        "longest_identical_token_run": 923,
-        "pass_screen": false
-      }
-    },
-    {
-      "block_size": 1024,
-      "steps": 12,
-      "median_tokps": 601.9543854074814,
-      "long_health": {
-        "alphabetic_fraction": 0.10743061772605192,
-        "distinct_4gram_ratio": 0.03819784524975514,
-        "longest_identical_token_run": 924,
-        "pass_screen": false
-      }
-    },
-    {
-      "block_size": 1024,
-      "steps": 13,
-      "median_tokps": 555.3291700396725,
-      "long_health": {
-        "alphabetic_fraction": 0.13823272090988625,
-        "distinct_4gram_ratio": 0.058765915768854066,
-        "longest_identical_token_run": 944,
-        "pass_screen": false
-      }
-    },
-    {
-      "block_size": 1024,
-      "steps": 14,
-      "median_tokps": 514.9606330424041,
-      "long_health": {
-        "alphabetic_fraction": 0.13127753303964756,
-        "distinct_4gram_ratio": 0.05582761998041136,
-        "longest_identical_token_run": 908,
-        "pass_screen": false
-      }
-    }
-  ],
-  "best_quality_setting": null,
-  "reason": "No tested step count passed all fixed quality checks; speed-only settings are not endorsed as high quality.",
-  "latency_reproduction_setting": {
-    "canvas": 1024,
-    "steps": 12,
-    "median_tokps_on_A100": 601.9543854074814,
-    "quality_warning": "Severely repetitive/invalid generation; latency reproduction only, not a quality recommendation."
+  "model_type": "lfm2_moe",
+  "packaging": "exact_runtime_adapter_patch",
+  "sampler_source": "modeling_diffusion_lfm.py",
+  "mask_embedding_file": "diffusion.safetensors",
+  "raw_sampler": {
+    "block_size": 16,
+    "steps": 16,
+    "prefix_tokens": 256
   },
-  "quality_status": "No configuration in the 1-15 step sweep passed all checked quality criteria."
+  "quality_approved": false,
+  "current_revision_speed_measured": false,
+  "legacy_results_revision": "2d791e03b14bb42b4323049fbfd832415d32b055"
 }