---
base_model:
  - embedl/Qwen3-0.6B-FlashHead
  - embedl/Qwen3-1.7B-FlashHead
  - embedl/Llama-3.2-1B-Instruct-FlashHead
  - embedl/gemma-3-1b-it-FlashHead
tags:
  - research-artifacts
  - output-head
  - efficient-inference
---

# Efficient LM-head reliability: compressed heads and reference states

This repository contains **auxiliary research tensors**, not a standalone
language model. The three `heads/<model>/head.pt` files are packed ARCHead
output heads built from the pinned FlashHead checkpoints. The ten
`reference_states/<model>/<source>/*.pt` files are dense-model hidden states
used for matched-state candidate-retrieval experiments. The heads and states
are tied to the source models and tokenizers; they cannot be loaded with
`AutoModel.from_pretrained` on their own.

## Contents

| Folder | Models | Purpose |
|---|---|---|
| `heads/` | Qwen3 0.6B and 1.7B, Llama 3.2 1B | Packed ARCHead heads, with build manifests |
| `reference_states/*/states/` | All four models | Dense natural/tool trajectories |
| `reference_states/*/bfcl_stage2/` | Qwen3 and Llama | Dense BFCL trajectories |
| `reference_states/*/forced_formats_h/` | Qwen3 and Llama | Selected format-probe states |

The BFCL, natural-prompt and format-probe metadata, experiment code, and
per-example results are maintained in the companion
[research repository](https://github.com/Yonsei-EASY-LAB/efficient-lm-head-reliability).
That repository is currently private. The full released FlashHead checkpoints
and HNSW indexes are downloaded from their upstream repositories; they are not
duplicated here.

## Source checkpoints

| Model key | Released checkpoint revision |
|---|---|
| `qwen3-0.6b` | `embedl/Qwen3-0.6B-FlashHead@a3ed38667e683dcf90c38f5a23d8ef139494badb` |
| `qwen3-1.7b` | `embedl/Qwen3-1.7B-FlashHead@399929e7f68464fa43395fa3b28416725bfd7c44` |
| `llama-3.2-1b` | `embedl/Llama-3.2-1B-Instruct-FlashHead@927ce57183e92f16a569cc705e6d806fc548d154` |
| `gemma-3-1b` | `embedl/gemma-3-1b-it-FlashHead@f19052159425d8db5154c1068e7add46d653332b` |

The ARCHead files use the released code at
`suayptalha/archead@29002720037fa4215822933e3ec50636882a7cc2`.
Each head's `manifest.json` records its calibration sources, compression
configuration and SHA-256 digest.

## Download and integrity

```bash
hf download kimgreg/efficient-lm-head-reliability --local-dir ./hf-assets
cd hf-assets
sha256sum -c SHA256SUMS
```

The companion repository's `fetch_research_data.sh` restores the same files
to the paths expected by its experiment scripts. `SHA256SUMS` covers the 13
large tensors. Verify the upstream checkpoint terms before reusing a derived
head with a different distribution or purpose.