File size: 3,028 Bytes
40ce80b cdfcfcf 40ce80b cdfcfcf 40ce80b cdfcfcf | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 | ---
library_name: pytorch
license: apache-2.0
tags:
- pruning
- sparsity
- optimization
- higher-order
- transformers
---
# Higher-Order Pruner
A reusable Python pruning method hosted in a Hugging Face **model repository**.
This repository is not a pruned checkpoint and does not contain benchmark
results. It supplies a direct code API for pruning compatible unquantized
PyTorch models.
The method combines calibration-derived KFAC-diagonal second-order information
with a clipped diagonal higher-order finite-difference estimate, exact global
sparsity allocation, and monotone held-out refinement. It does not initialize
from Wanda, SparseGPT, magnitude pruning, or a previously pruned checkpoint.
## Install from this model repository
```bash
pip install "git+https://huggingface.co/OWNER/REPOSITORY"
```
Then load the method configuration from the Hub and use it in code:
```python
from higher_order_pruner import HigherOrderPruner
pruner = HigherOrderPruner.from_pretrained("OWNER/REPOSITORY")
result = pruner.prune_hf_model(
"Qwen/Qwen3-4B-Base",
calibration_texts=my_calibration_texts,
validation_texts=my_heldout_texts,
sparsity=0.50,
output_dir="./pruned-model",
sequence_length=512,
device="cuda",
)
print(result["report"]["achieved_sparsity"])
```
`pruned-model/checkpoint` is a normal Hugging Face checkpoint saved with
`save_pretrained` and safetensors.
## Generic PyTorch API
The core is not restricted to language models. Supply any differentiable
scalar loss and batches already placed on the desired device:
```python
from higher_order_pruner import HigherOrderPruner
pruner = HigherOrderPruner.from_pretrained("OWNER/REPOSITORY")
def loss_fn(model, batch):
x, target = batch
return (model(x) - target).square().mean()
report = pruner.prune(
model,
calibration_batches,
loss_fn,
sparsity=0.50,
validation_batches=heldout_batches,
progress_dir="./pruning-progress",
)
```
For nonstandard architectures, pass explicit `module_names=[...]`. The method
prunes unique floating-point `torch.nn.Linear.weight` tensors. It refuses
quantized weights and does not promise that arbitrary custom kernels, tied
non-Linear tensors, or remote-code-only architectures are compatible.
## Important behavior
- Calibration statistics and higher-order estimates are computed from the
caller's model and data at runtime; no Qwen/Falcon statistics are embedded.
- The requested zero count is enforced exactly across selected weights.
- Existing exact zeros are included in that budget.
- The HF adapter auto-tunes its measured curvature kernel for throughput under
a 95% VRAM ceiling. It does not require a specific GPU model.
- Unstructured zeros require an appropriate sparse inference kernel to produce
inference speedups.
## Local development
```bash
pip install -e ".[test]"
pytest -q
```
`upload_to_hub.py` creates a new Hugging Face **model** repository and uploads
this package. It refuses to overwrite an existing repository.
|