Grawnch's picture
Publish tencent/Hy4-flash pure-BF16 reference logits (2K + 8K BF16 safetensors + exact FP32 factorized + OPT 64x2048)
cfd1f91 verified
|
Raw History Blame Contribute Delete
3.74 kB
metadata
license: other
license_name: tencent-hy
license_link: LICENSE
base_model: tencent/Hy4-flash
tags:
  - tencent
  - hy4-flash
  - reference-logits
  - quantization
  - evaluation
  - kld

tencent/Hy4-flash Pure-BF16 Reference Logits & Teacher Hidden States

Exact pure-BF16 teacher reference logits and post-RMSNorm hidden states from tencent/Hy4-flash (220.25B total / 155.5B backbone, fp8_exact Engram, 120,832 vocab) for evaluating quantized checkpoints (KLD, Top-1, Top-5, Delta PPL) without needing 440+ GB of VRAM/disk to run the full BF16 teacher.

Included Files

File Size Contents
blind_val_2k_logits_bf16.safetensors 3.96 GB Direct pre-materialized logits_bf16 [8, 2048, 120832] (bfloat16) + input_ids [8, 2048] (int64). Rows 0:2 = WikiText-2 test prose, 2:4 = Held-Out Code, 4:8 = On-Policy Math/Code CoT.
blind_val_8k_logits_bf16.safetensors 3.96 GB Direct pre-materialized logits_bf16 [2, 8192, 120832] (bfloat16) + input_ids [2, 8192] (int64, WikiText-2 test 8K long-context).
factorized_fp32_exact.safetensors 2.72 GB Zero-loss FP32 teacher factorization (logits_f32 = hidden_f32 @ lm_head_weight_bf16.float().T, 0.0 bit error vs full BF16 teacher): blind_val_2k_{ids,hidden_f32}, blind_val_8k_{ids,hidden_f32}, opt_64x2048_{ids,hidden_f32} (131,072 disjoint calibration tokens: 25% C4 + 25% WT2 + 25% Code + 25% CoT), and lm_head_weight_bf16 [120832, 3072].
bf16_reference_metadata.json < 4 KB Verified BF16 PPL baselines, slice boundaries, and tensor shapes.
tokenizer.json / tokenizer_config.json / chat_template.jinja 9.4 MB Official tencent/Hy4-flash tokenizer files for convenience.

Verified Pure-BF16 Teacher Baselines

  • Blind-Val 2K (8 x 2,048, 16,376 next-token targets): BF16 PPL = 13.2992 (WT2 Prose = 29.2078, Code = 42.3502, Self-Gen Math/Code = 5.1036)
  • Blind-Val 8K (2 x 8,192, 16,382 next-token targets): BF16 PPL = 34.8908 (Early 0..512 = 11.9276, Mid 4096..4608 = 52.4458, Late 7679..8191 = 33.1118)

Quickstart (safetensors + PyTorch)

Option A: Load Direct [B, S, 120832] BF16 Logits

import torch
import torch.nn.functional as F
from safetensors.torch import load_file

d2k = load_file("blind_val_2k_logits_bf16.safetensors")
input_ids = d2k["input_ids"]          # [8, 2048] int64
tea_logits = d2k["logits_bf16"]       # [8, 2048, 120832] bfloat16

Option B: Exact FP32 Logits (0.0 Bit Loss, Includes 64 x 2,048 Calibration Set)

import math, torch
import torch.nn.functional as F
from safetensors.torch import load_file

fac = load_file("factorized_fp32_exact.safetensors")
W_lm = fac["lm_head_weight_bf16"].cuda().float()   # [120832, 3072]
hid  = fac["blind_val_2k_hidden_f32"]              # [8, 2048, 3072] float32
ids  = fac["blind_val_2k_ids"]                     # [8, 2048] int64

# Streamed exact FP32 KLD(P_bf16 || P_quant) & Top-1 agreement against your student logits `stu_logits [8, 2048, 120832]`
kld_sum, top1_hits, n_tok = 0.0, 0, 0
for b in range(hid.shape[0]):
    for s0 in range(0, hid.shape[1] - 1, 256):
        s1 = min(hid.shape[1] - 1, s0 + 256)
        t_lg = hid[b, s0:s1].cuda() @ W_lm.T
        s_lg = stu_logits[b, s0:s1].cuda().float()
        lpt = F.log_softmax(t_lg, dim=-1)
        lps = F.log_softmax(s_lg, dim=-1)
        kld_sum += float((lpt.exp() * (lpt - lps)).sum().item())
        top1_hits += int((t_lg.argmax(-1) == s_lg.argmax(-1)).sum().item())
        n_tok += (s1 - s0)

print(f"Blind-Val 2K KLD = {kld_sum / n_tok:.5f} | Top-1 = {100.0 * top1_hits / n_tok:.2f}%")