scripts/README.md

# Scripts

Tooling used to produce this checkpoint and the numbers in the model card. None
of it is needed to use the model.

Install the dependencies first:

```bash
uv venv --python 3.12 .venv
uv pip install --python .venv/bin/python torch transformers bitsandbytes accelerate sentencepiece
```

## `quantize.py`

Rebuilds the 4-bit NF4 checkpoint from the original fp32 weights and writes a
self-contained repository (weights, tokenizer files, `.gitattributes`) that can be
uploaded directly. Optional argument: output directory, defaulting to
`../models/gradient-ai-text-detector-4bit`.

```bash
python scripts/quantize.py
```

It keeps the classifier head in fp32, because bitsandbytes' packed CPU kernel
requires each quantized layer's output dimension to divide evenly by its block
size and the head is `[1, 1024]`. It then asserts that no quantized layer would
break that kernel, and prints a reload sanity value.

## `bench_quant.py`

Compares fp32, bf16, and NF4 4-bit on CPU and Apple Silicon MPS, measuring
resident memory, batch latency, and the maximum probability change against the
fp32 reference.

```bash
python scripts/bench_quant.py
```

## `eval_quant.py`

Measures how much quantization moves individual scores: max and mean absolute
change in P(AI) against fp32, and how many verdicts flip at the 0.5 threshold.

```bash
python scripts/eval_quant.py
```