Hugging Face's logo

kimgreg
/
efficient-lm-head-reliability

research-artifacts
output-head
efficient-inference
Model card Files
xet
 Community
efficient-lm-head-reliability / README.md
kimgreg's picture
kimgreg
Link organization research repository
f0c4139 verified
preview code
|
Raw History Blame Contribute Delete
2.69 kB
metadata 
base_model:
  - embedl/Qwen3-0.6B-FlashHead
  - embedl/Qwen3-1.7B-FlashHead
  - embedl/Llama-3.2-1B-Instruct-FlashHead
  - embedl/gemma-3-1b-it-FlashHead
tags:
  - research-artifacts
  - output-head
  - efficient-inference

Efficient LM-head reliability: compressed heads and reference states

This repository contains auxiliary research tensors, not a standalone language model. The three heads/<model>/head.pt files are packed ARCHead output heads built from the pinned FlashHead checkpoints. The ten reference_states/<model>/<source>/*.pt files are dense-model hidden states used for matched-state candidate-retrieval experiments. The heads and states are tied to the source models and tokenizers; they cannot be loaded with AutoModel.from_pretrained on their own.

Contents

Folder Models Purpose
heads/ Qwen3 0.6B and 1.7B, Llama 3.2 1B Packed ARCHead heads, with build manifests
reference_states/*/states/ All four models Dense natural/tool trajectories
reference_states/*/bfcl_stage2/ Qwen3 and Llama Dense BFCL trajectories
reference_states/*/forced_formats_h/ Qwen3 and Llama Selected format-probe states

The BFCL, natural-prompt and format-probe metadata, experiment code, and per-example results are maintained in the companion research repository. That repository is currently private. The full released FlashHead checkpoints and HNSW indexes are downloaded from their upstream repositories; they are not duplicated here.

Source checkpoints

Model key Released checkpoint revision
qwen3-0.6b embedl/Qwen3-0.6B-FlashHead@a3ed38667e683dcf90c38f5a23d8ef139494badb
qwen3-1.7b embedl/Qwen3-1.7B-FlashHead@399929e7f68464fa43395fa3b28416725bfd7c44
llama-3.2-1b embedl/Llama-3.2-1B-Instruct-FlashHead@927ce57183e92f16a569cc705e6d806fc548d154
gemma-3-1b embedl/gemma-3-1b-it-FlashHead@f19052159425d8db5154c1068e7add46d653332b

The ARCHead files use the released code at suayptalha/archead@29002720037fa4215822933e3ec50636882a7cc2. Each head's manifest.json records its calibration sources, compression configuration and SHA-256 digest.

Download and integrity 

hf download kimgreg/efficient-lm-head-reliability --local-dir ./hf-assets
cd hf-assets
sha256sum -c SHA256SUMS

The companion repository's fetch_research_data.sh restores the same files to the paths expected by its experiment scripts. SHA256SUMS covers the 13 large tensors. Verify the upstream checkpoint terms before reusing a derived head with a different distribution or purpose.