|
Download README.md from TinyLamda/llm-quant-artifacts: direct link, hf CLI and curl.
- Browser
- Download file 3.29 kB
-
https://huggingface.co/TinyLamda/llm-quant-artifacts/resolve/main/README.md
- Command line
-
hf download hf://TinyLamda/llm-quant-artifacts/README.md
-
curl -L -o README.md https://huggingface.co/TinyLamda/llm-quant-artifacts/resolve/main/README.md
3.29 kB
| license: other | |
| license_name: per-model | |
| # llm-quant-artifacts | |
| - **What**: fake-quantized LLM checkpoints (weights 4-bit, group 128) used in a quantization study, packed losslessly. | |
| - **Use**: transfer between our own machines; restore with the companion tool, which decodes `.lqz` back to the original | |
| `model.pt` and checks every tensor's sha256 against `manifest.json`. | |
| - **Layout** | |
| - `<model>/g_<method>/model.lqz`: one packed checkpoint per quantization method | |
| - `<model>/g_qoq/smooth.pt`: per-channel smoothing scales used by that method at runtime | |
| - `<model>/bases/`, `<model>/pca/`: per-layer key-side bases and weight-side PCA bases used at runtime | |
| - `<model>/qoq/`: per-layer key-side smoothing bases of a SmoothAttention-style baseline (scales tied over rotary pairs) | |
| - `isca1106_kv_materials.json`: sha256 of the key-side bases of the edge models and the KV-store settings they are used with | |
| - `<model>/manifest.json`: file list, sha256 of every file and tensor, restore paths | |
| - `<model>/ppl/`: tokenized WikiText-2 / C4 evaluation windows (int32 token ids) with their `manifest.json` | |
| - `<model>/moments/`: calibration second-moment statistics the key-side bases were fitted from | |
| - `<model>/mask_locality/cache/`: inputs and intermediate products of a pruning-mask locality study (channel permutations from calibration, per-channel keep-frequency profiles); sha256 of every file is recorded in the study's `ARTIFACTS.json` | |
| - `<model>/mask_locality/cache/bases/beta1104/`, `<model>/mask_locality/cache/graded/`: key-side bases with a fixed exponent of 1/2 and the per-channel key read-error statistics fitted in them (metadata paths are relative; each statistics file stores the sha256 of its basis file) | |
| - **Format (.lqz)**: a 4-bit group of 128 values holds at most 16 distinct fp16 values, stored as a 16-entry fp16 table plus | |
| 128 4-bit indices; all other tensors are stored raw. Decoding is exact (bit-identical tensors). | |
| - **Not** an inference-ready release: research artifacts only, no quality guarantee. | |
| ## Models and licenses | |
| Each folder carries the license of the model it is derived from. | |
| | Folder | Derived from | License | | |
| |---|---|---| | |
| | `q3_1p7b/` | `Qwen/Qwen3-1.7B` | Apache-2.0 | | |
| | `q3_4b/` | `Qwen/Qwen3-4B` | Apache-2.0 | | |
| | `q3_8b/` | `Qwen/Qwen3-8B` | Apache-2.0 | | |
| | `l31i/` | `meta-llama/Llama-3.1-8B-Instruct` | Llama 3.1 Community License (`l31i/LICENSE`, `l31i/USE_POLICY.md`) | | |
| | `l31/` | `meta-llama/Llama-3.1-8B` | Llama 3.1 Community License (`l31/LICENSE`, `l31/USE_POLICY.md`) | | |
| | `l32i/` | `meta-llama/Llama-3.2-3B-Instruct` | Llama 3.2 Community License (`l32i/LICENSE.txt`, `l32i/USE_POLICY.md`) | | |
| | `l2/` | `meta-llama/Llama-2-7b-hf` | Llama 2 Community License (`l2/LICENSE.txt`, `l2/USE_POLICY.md`) | | |
| **Built with Llama.** Llama 3.1 is licensed under the Llama 3.1 Community License, Copyright © Meta Platforms, Inc. All Rights | |
| Reserved. Llama 3.2 is licensed under the Llama 3.2 Community License, Copyright © Meta Platforms, Inc. All Rights Reserved. Llama 2 is licensed under the LLAMA 2 Community License, Copyright (c) Meta Platforms, Inc. All Rights Reserved. | |
| Use of the `l31i/`, `l31/`, `l32i/`, and `l2/` artifacts is subject to those licenses and to Meta's Acceptable Use Policies included alongside. | |