README.md

---
license: mit
library_name: binprov
tags:
- binary-analysis
- compilation-provenance
- optimization-level-classification
- pytorch
- roberta
metrics:
- accuracy
---

# BinProv Opt4

`binprov` classifies the compiler optimization level of x86-64 ELF
binaries from raw `.text` bytes: O0, O1, O2, O3.
It follows the BinProv method and does not disassemble the input.

This is a BinProv-native PyTorch checkpoint. Its byte encoder is RoBERTa-shaped,
but its pooling and classification head are custom, so it is not compatible
with `transformers.AutoModel`. Use the bundled `inference.py`; no remote code or
`trust_remote_code` is involved.

## Model details

- Architecture: RoBERTa-style byte encoder, seq 2048 bytes, 12 layers x 768 hidden; 261-token byte vocabulary
- Pooling: `border` with 4 heads
- Input: 2048-byte windows at a 512-byte evaluation stride
- Output: four probabilities corresponding to O0, O1, O2, and O3
- Framework: PyTorch with Hugging Face Transformers components
- License: MIT

The model follows [BinProv: Binary Code Provenance Identification without
Disassembly](https://doi.org/10.1145/3545948.3545956), RAID 2022. This checkpoint
comes from a clean reproduction run of the repository's best single-model
recipe. Source code and full reproduction scripts are available at
[Viewer-HX/BinProv](https://github.com/Viewer-HX/BinProv).
Evaluation (this export): sequence accuracy **84.2255%**. Binary-level accuracy **93.8830%**. A fresh training run's numbers vary by ~1-2 points with seed.

## Direct use

```bash
python -m venv .venv
source .venv/bin/activate                    # Windows: .venv\Scripts\activate
pip install -r requirements.txt

python inference.py --model model --elf /path/to/binary.elf
python inference.py --model model --text-bytes raw_text.bin --json
```

The script extracts `.text` from an ELF or accepts already extracted bytes. It
evaluates overlapping 2048-byte inputs every 512 bytes, then averages the
per-window probabilities for one binary. It automatically selects CUDA, Apple
MPS, or CPU. Use `--batch-size` to trade memory for speed.

## Training data

The model was trained and evaluated on the x86-64 subset of BinKit Normal,
covering GCC and Clang binaries at optimization levels O0 through O3. Splits are
grouped by source program so the same program does not cross train and test.
Exact training arguments and environment versions are in `metadata.json` and
`evidence/args.json`.

## Evaluation

| metric | value |
|---|---:|
| sequence accuracy | **84.2255%** |
| balanced sequence accuracy | **83.0951%** |
| binary soft-vote accuracy | **93.8830%** |
| test windows | 116,321 |
| test binaries | 376 |

The sequence metric uses 512-byte target positions with 2048-byte overlapping
model inputs. Binary accuracy averages the saved per-window probabilities. The
recomputed evaluation in `evidence/evaluation.json` matches the recorded test
accuracy. Results can vary with retraining and should be compared on the same
program-grouped split.

## Intended use and limitations

This checkpoint is intended for research and analysis of compiler optimization
provenance in ordinary x86-64 ELF code produced by toolchains represented in
BinKit. It has not been validated as a general malware, authorship, attribution,
or security-decision system. Packed, obfuscated, self-modifying, non-ELF, other
architecture, and out-of-distribution toolchain inputs may be unreliable. The
softmax scores are not calibrated confidence estimates.

## Files

- `model/` — `model.pt`, `binprov_config.json`, `head.json` (BinProv checkpoint)
- `binprov/` — minimal source package needed by `inference.py`
- `requirements.txt` — runtime dependencies
- `labels.json` — class id -> label
- `metadata.json` — exact training args + eval results + environment
- `environment.json` — runtime versions
- `evidence/` — args, result, evaluation report, and saved probabilities
- `LICENSE` — repository license

## License

The model weights, bundled inference code, and documentation are released under
the [MIT License](LICENSE).

## Citation

```bibtex
@inproceedings{he2022binprov,
  title={BinProv: Binary Code Provenance Identification without Disassembly},
  author={He, Xu and Wang, Shu and Xing, Yunlong and Feng, Pengbin and Wang, Haining and Li, Qi and Chen, Songqing and Sun, Kun},
  booktitle={RAID},
  year={2022},
  doi={10.1145/3545948.3545956}
}
```