--- license: other license_name: tencent-hy license_link: LICENSE base_model: tencent/Hy4-flash tags: - tencent - hy4-flash - reference-logits - quantization - evaluation - kld --- # `tencent/Hy4-flash` Pure-BF16 Reference Logits & Teacher Hidden States Exact pure-`BF16` teacher reference logits and post-`RMSNorm` hidden states from **`tencent/Hy4-flash`** (`220.25B` total / `155.5B` backbone, `fp8_exact` Engram, `120,832` vocab) for evaluating quantized checkpoints (`KLD`, `Top-1`, `Top-5`, `Delta PPL`) without needing `440+ GB` of VRAM/disk to run the full BF16 teacher. ## Included Files | File | Size | Contents | | :--- | :---: | :--- | | `blind_val_2k_logits_bf16.safetensors` | `3.96 GB` | Direct pre-materialized `logits_bf16 [8, 2048, 120832]` (`bfloat16`) + `input_ids [8, 2048]` (`int64`). Rows `0:2` = WikiText-2 `test` prose, `2:4` = Held-Out Code, `4:8` = On-Policy Math/Code CoT. | | `blind_val_8k_logits_bf16.safetensors` | `3.96 GB` | Direct pre-materialized `logits_bf16 [2, 8192, 120832]` (`bfloat16`) + `input_ids [2, 8192]` (`int64`, WikiText-2 `test` 8K long-context). | | `factorized_fp32_exact.safetensors` | `2.72 GB` | **Zero-loss `FP32` teacher factorization** (`logits_f32 = hidden_f32 @ lm_head_weight_bf16.float().T`, `0.0` bit error vs full BF16 teacher): `blind_val_2k_{ids,hidden_f32}`, `blind_val_8k_{ids,hidden_f32}`, `opt_64x2048_{ids,hidden_f32}` (`131,072` disjoint calibration tokens: `25%` C4 + `25%` WT2 + `25%` Code + `25%` CoT), and `lm_head_weight_bf16 [120832, 3072]`. | | `bf16_reference_metadata.json` | `< 4 KB` | Verified `BF16` PPL baselines, slice boundaries, and tensor shapes. | | `tokenizer.json` / `tokenizer_config.json` / `chat_template.jinja` | `9.4 MB` | Official `tencent/Hy4-flash` tokenizer files for convenience. | ## Verified Pure-BF16 Teacher Baselines - **Blind-Val `2K` (`8 x 2,048`, `16,376` next-token targets):** `BF16 PPL = 13.2992` (`WT2 Prose = 29.2078`, `Code = 42.3502`, `Self-Gen Math/Code = 5.1036`) - **Blind-Val `8K` (`2 x 8,192`, `16,382` next-token targets):** `BF16 PPL = 34.8908` (`Early 0..512 = 11.9276`, `Mid 4096..4608 = 52.4458`, `Late 7679..8191 = 33.1118`) ## Quickstart (`safetensors` + PyTorch) ### Option A: Load Direct `[B, S, 120832]` `BF16` Logits ```python import torch import torch.nn.functional as F from safetensors.torch import load_file d2k = load_file("blind_val_2k_logits_bf16.safetensors") input_ids = d2k["input_ids"] # [8, 2048] int64 tea_logits = d2k["logits_bf16"] # [8, 2048, 120832] bfloat16 ``` ### Option B: Exact `FP32` Logits (`0.0` Bit Loss, Includes `64 x 2,048` Calibration Set) ```python import math, torch import torch.nn.functional as F from safetensors.torch import load_file fac = load_file("factorized_fp32_exact.safetensors") W_lm = fac["lm_head_weight_bf16"].cuda().float() # [120832, 3072] hid = fac["blind_val_2k_hidden_f32"] # [8, 2048, 3072] float32 ids = fac["blind_val_2k_ids"] # [8, 2048] int64 # Streamed exact FP32 KLD(P_bf16 || P_quant) & Top-1 agreement against your student logits `stu_logits [8, 2048, 120832]` kld_sum, top1_hits, n_tok = 0.0, 0, 0 for b in range(hid.shape[0]): for s0 in range(0, hid.shape[1] - 1, 256): s1 = min(hid.shape[1] - 1, s0 + 256) t_lg = hid[b, s0:s1].cuda() @ W_lm.T s_lg = stu_logits[b, s0:s1].cuda().float() lpt = F.log_softmax(t_lg, dim=-1) lps = F.log_softmax(s_lg, dim=-1) kld_sum += float((lpt.exp() * (lpt - lps)).sum().item()) top1_hits += int((t_lg.argmax(-1) == s_lg.argmax(-1)).sum().item()) n_tok += (s1 - s0) print(f"Blind-Val 2K KLD = {kld_sum / n_tok:.5f} | Top-1 = {100.0 * top1_hits / n_tok:.2f}%") ```