|
Download README.md from Grawnch/Hy4-Flash-BF16-Reference-Logits: direct link, hf CLI and curl.
- Browser
- Download file 3.74 kB
-
https://huggingface.co/Grawnch/Hy4-Flash-BF16-Reference-Logits/resolve/main/README.md
- Command line
-
hf download hf://Grawnch/Hy4-Flash-BF16-Reference-Logits/README.md
-
curl -L -o README.md https://huggingface.co/Grawnch/Hy4-Flash-BF16-Reference-Logits/resolve/main/README.md
3.74 kB
| license: other | |
| license_name: tencent-hy | |
| license_link: LICENSE | |
| base_model: tencent/Hy4-flash | |
| tags: | |
| - tencent | |
| - hy4-flash | |
| - reference-logits | |
| - quantization | |
| - evaluation | |
| - kld | |
| # `tencent/Hy4-flash` Pure-BF16 Reference Logits & Teacher Hidden States | |
| Exact pure-`BF16` teacher reference logits and post-`RMSNorm` hidden states from **`tencent/Hy4-flash`** (`220.25B` total / `155.5B` backbone, `fp8_exact` Engram, `120,832` vocab) for evaluating quantized checkpoints (`KLD`, `Top-1`, `Top-5`, `Delta PPL`) without needing `440+ GB` of VRAM/disk to run the full BF16 teacher. | |
| ## Included Files | |
| | File | Size | Contents | | |
| | :--- | :---: | :--- | | |
| | `blind_val_2k_logits_bf16.safetensors` | `3.96 GB` | Direct pre-materialized `logits_bf16 [8, 2048, 120832]` (`bfloat16`) + `input_ids [8, 2048]` (`int64`). Rows `0:2` = WikiText-2 `test` prose, `2:4` = Held-Out Code, `4:8` = On-Policy Math/Code CoT. | | |
| | `blind_val_8k_logits_bf16.safetensors` | `3.96 GB` | Direct pre-materialized `logits_bf16 [2, 8192, 120832]` (`bfloat16`) + `input_ids [2, 8192]` (`int64`, WikiText-2 `test` 8K long-context). | | |
| | `factorized_fp32_exact.safetensors` | `2.72 GB` | **Zero-loss `FP32` teacher factorization** (`logits_f32 = hidden_f32 @ lm_head_weight_bf16.float().T`, `0.0` bit error vs full BF16 teacher): `blind_val_2k_{ids,hidden_f32}`, `blind_val_8k_{ids,hidden_f32}`, `opt_64x2048_{ids,hidden_f32}` (`131,072` disjoint calibration tokens: `25%` C4 + `25%` WT2 + `25%` Code + `25%` CoT), and `lm_head_weight_bf16 [120832, 3072]`. | | |
| | `bf16_reference_metadata.json` | `< 4 KB` | Verified `BF16` PPL baselines, slice boundaries, and tensor shapes. | | |
| | `tokenizer.json` / `tokenizer_config.json` / `chat_template.jinja` | `9.4 MB` | Official `tencent/Hy4-flash` tokenizer files for convenience. | | |
| ## Verified Pure-BF16 Teacher Baselines | |
| - **Blind-Val `2K` (`8 x 2,048`, `16,376` next-token targets):** `BF16 PPL = 13.2992` (`WT2 Prose = 29.2078`, `Code = 42.3502`, `Self-Gen Math/Code = 5.1036`) | |
| - **Blind-Val `8K` (`2 x 8,192`, `16,382` next-token targets):** `BF16 PPL = 34.8908` (`Early 0..512 = 11.9276`, `Mid 4096..4608 = 52.4458`, `Late 7679..8191 = 33.1118`) | |
| ## Quickstart (`safetensors` + PyTorch) | |
| ### Option A: Load Direct `[B, S, 120832]` `BF16` Logits | |
| ```python | |
| import torch | |
| import torch.nn.functional as F | |
| from safetensors.torch import load_file | |
| d2k = load_file("blind_val_2k_logits_bf16.safetensors") | |
| input_ids = d2k["input_ids"] # [8, 2048] int64 | |
| tea_logits = d2k["logits_bf16"] # [8, 2048, 120832] bfloat16 | |
| ``` | |
| ### Option B: Exact `FP32` Logits (`0.0` Bit Loss, Includes `64 x 2,048` Calibration Set) | |
| ```python | |
| import math, torch | |
| import torch.nn.functional as F | |
| from safetensors.torch import load_file | |
| fac = load_file("factorized_fp32_exact.safetensors") | |
| W_lm = fac["lm_head_weight_bf16"].cuda().float() # [120832, 3072] | |
| hid = fac["blind_val_2k_hidden_f32"] # [8, 2048, 3072] float32 | |
| ids = fac["blind_val_2k_ids"] # [8, 2048] int64 | |
| # Streamed exact FP32 KLD(P_bf16 || P_quant) & Top-1 agreement against your student logits `stu_logits [8, 2048, 120832]` | |
| kld_sum, top1_hits, n_tok = 0.0, 0, 0 | |
| for b in range(hid.shape[0]): | |
| for s0 in range(0, hid.shape[1] - 1, 256): | |
| s1 = min(hid.shape[1] - 1, s0 + 256) | |
| t_lg = hid[b, s0:s1].cuda() @ W_lm.T | |
| s_lg = stu_logits[b, s0:s1].cuda().float() | |
| lpt = F.log_softmax(t_lg, dim=-1) | |
| lps = F.log_softmax(s_lg, dim=-1) | |
| kld_sum += float((lpt.exp() * (lpt - lps)).sum().item()) | |
| top1_hits += int((t_lg.argmax(-1) == s_lg.argmax(-1)).sum().item()) | |
| n_tok += (s1 - s0) | |
| print(f"Blind-Val 2K KLD = {kld_sum / n_tok:.5f} | Top-1 = {100.0 * top1_hits / n_tok:.2f}%") | |
| ``` | |