kimgreg's picture
Link organization research repository
f0c4139 verified
|
Raw History Blame Contribute Delete
2.69 kB
---
base_model:
- embedl/Qwen3-0.6B-FlashHead
- embedl/Qwen3-1.7B-FlashHead
- embedl/Llama-3.2-1B-Instruct-FlashHead
- embedl/gemma-3-1b-it-FlashHead
tags:
- research-artifacts
- output-head
- efficient-inference
---
# Efficient LM-head reliability: compressed heads and reference states
This repository contains **auxiliary research tensors**, not a standalone
language model. The three `heads/<model>/head.pt` files are packed ARCHead
output heads built from the pinned FlashHead checkpoints. The ten
`reference_states/<model>/<source>/*.pt` files are dense-model hidden states
used for matched-state candidate-retrieval experiments. The heads and states
are tied to the source models and tokenizers; they cannot be loaded with
`AutoModel.from_pretrained` on their own.
## Contents
| Folder | Models | Purpose |
|---|---|---|
| `heads/` | Qwen3 0.6B and 1.7B, Llama 3.2 1B | Packed ARCHead heads, with build manifests |
| `reference_states/*/states/` | All four models | Dense natural/tool trajectories |
| `reference_states/*/bfcl_stage2/` | Qwen3 and Llama | Dense BFCL trajectories |
| `reference_states/*/forced_formats_h/` | Qwen3 and Llama | Selected format-probe states |
The BFCL, natural-prompt and format-probe metadata, experiment code, and
per-example results are maintained in the companion
[research repository](https://github.com/Yonsei-EASY-LAB/efficient-lm-head-reliability).
That repository is currently private. The full released FlashHead checkpoints
and HNSW indexes are downloaded from their upstream repositories; they are not
duplicated here.
## Source checkpoints
| Model key | Released checkpoint revision |
|---|---|
| `qwen3-0.6b` | `embedl/Qwen3-0.6B-FlashHead@a3ed38667e683dcf90c38f5a23d8ef139494badb` |
| `qwen3-1.7b` | `embedl/Qwen3-1.7B-FlashHead@399929e7f68464fa43395fa3b28416725bfd7c44` |
| `llama-3.2-1b` | `embedl/Llama-3.2-1B-Instruct-FlashHead@927ce57183e92f16a569cc705e6d806fc548d154` |
| `gemma-3-1b` | `embedl/gemma-3-1b-it-FlashHead@f19052159425d8db5154c1068e7add46d653332b` |
The ARCHead files use the released code at
`suayptalha/archead@29002720037fa4215822933e3ec50636882a7cc2`.
Each head's `manifest.json` records its calibration sources, compression
configuration and SHA-256 digest.
## Download and integrity
```bash
hf download kimgreg/efficient-lm-head-reliability --local-dir ./hf-assets
cd hf-assets
sha256sum -c SHA256SUMS
```
The companion repository's `fetch_research_data.sh` restores the same files
to the paths expected by its experiment scripts. `SHA256SUMS` covers the 13
large tensors. Verify the upstream checkpoint terms before reusing a derived
head with a different distribution or purpose.