---
library_name: pytorch
license: apache-2.0
tags:
  - pruning
  - sparsity
  - optimization
  - higher-order
  - transformers
---

# Higher-Order Pruner

A reusable Python pruning method hosted in a Hugging Face **model repository**.
This repository is not a pruned checkpoint and does not contain benchmark
results. It supplies a direct code API for pruning compatible unquantized
PyTorch models.

The method combines calibration-derived KFAC-diagonal second-order information
with a clipped diagonal higher-order finite-difference estimate, exact global
sparsity allocation, and monotone held-out refinement. It does not initialize
from Wanda, SparseGPT, magnitude pruning, or a previously pruned checkpoint.

## Install from this model repository

```bash
pip install "git+https://huggingface.co/OWNER/REPOSITORY"
```

Then load the method configuration from the Hub and use it in code:

```python
from higher_order_pruner import HigherOrderPruner

pruner = HigherOrderPruner.from_pretrained("OWNER/REPOSITORY")

result = pruner.prune_hf_model(
    "Qwen/Qwen3-4B-Base",
    calibration_texts=my_calibration_texts,
    validation_texts=my_heldout_texts,
    sparsity=0.50,
    output_dir="./pruned-model",
    sequence_length=512,
    device="cuda",
)
print(result["report"]["achieved_sparsity"])
```

`pruned-model/checkpoint` is a normal Hugging Face checkpoint saved with
`save_pretrained` and safetensors.

## Generic PyTorch API

The core is not restricted to language models. Supply any differentiable
scalar loss and batches already placed on the desired device:

```python
from higher_order_pruner import HigherOrderPruner

pruner = HigherOrderPruner.from_pretrained("OWNER/REPOSITORY")

def loss_fn(model, batch):
    x, target = batch
    return (model(x) - target).square().mean()

report = pruner.prune(
    model,
    calibration_batches,
    loss_fn,
    sparsity=0.50,
    validation_batches=heldout_batches,
    progress_dir="./pruning-progress",
)
```

For nonstandard architectures, pass explicit `module_names=[...]`. The method
prunes unique floating-point `torch.nn.Linear.weight` tensors. It refuses
quantized weights and does not promise that arbitrary custom kernels, tied
non-Linear tensors, or remote-code-only architectures are compatible.

## Important behavior

- Calibration statistics and higher-order estimates are computed from the
  caller's model and data at runtime; no Qwen/Falcon statistics are embedded.
- The requested zero count is enforced exactly across selected weights.
- Existing exact zeros are included in that budget.
- The HF adapter auto-tunes its measured curvature kernel for throughput under
  a 95% VRAM ceiling. It does not require a specific GPU model.
- Unstructured zeros require an appropriate sparse inference kernel to produce
  inference speedups.

## Local development

```bash
pip install -e ".[test]"
pytest -q
```

`upload_to_hub.py` creates a new Hugging Face **model** repository and uploads
this package. It refuses to overwrite an existing repository.