README.md

---
license: mit
library_name: transformers
pipeline_tag: text-generation
tags:
  - trustworthy-ml
  - model-verification
  - model-substitution
  - research-fixture
  - gpt2
---

# Assay reproducibility bundle — the seeded M / M' pair and a calibrated band

This is **not a model you should generate text with.** It is the fixture that makes
[Assay](https://github.com/NagaYu/assay)'s model-substitution experiments reproducible
offline, in seconds, with no training and no downloads of anything large.

- **Code**: [github.com/NagaYu/assay](https://github.com/NagaYu/assay)
- **Demo**: [NagaYu/assay-demo](https://huggingface.co/spaces/NagaYu/assay-demo)
- **Receipt corpus**: [NagaYu/assay-receipts](https://huggingface.co/datasets/NagaYu/assay-receipts)

## Contents

| Path | What it is |
|---|---|
| `tiny_pair/M/` | The "real" model **M**: a seeded, randomly-initialized GPT-2, 4 layers, hidden 64, vocab 256 |
| `tiny_pair/Mp/` | The "cheap" substitute **M'**: 2 layers, same hidden size and vocab, with an untied and permuted output head |
| `calibration.json` | The sketch protocol, the tolerance band calibrated on **gpt2**, and the demo HMAC key |

## Why M' has a permuted head

This detail matters more than it looks. A randomly-initialized GPT-2 is dominated by the
residual copy of the last prompt token, so two independently seeded models both greedily echo
that token — they are *behaviourally identical*, and a "substitution" between them is one
nobody could notice in the output. Permuting the tied embedding does not help either: the
echo circuit reads and writes through the same matrix, so the permutation cancels.

So M' here has its output head **untied and permuted**. It stays a well-formed model, it
produces genuinely different text, and the substitution becomes something worth detecting.
Anything that reads the emitted tokens — including Assay's generation-consistency check and
the forgery it defends against — is meaningless without this.

## Using it

```python
from transformers import GPT2LMHeadModel
from huggingface_hub import snapshot_download

path = snapshot_download("NagaYu/assay-tiny-pair")
M  = GPT2LMHeadModel.from_pretrained(f"{path}/tiny_pair/M").eval()
Mp = GPT2LMHeadModel.from_pretrained(f"{path}/tiny_pair/Mp").eval()
```

The identical pair is also constructed on the fly by the library, with no download at all:

```python
from assay.hooks import tiny_random_pair
M, Mp = tiny_random_pair(seed=0)   # bit-identical to the weights published here
```

Both are published because the copy here pins the fixture: if the constructor ever changes,
these weights still reproduce the numbers in the paper-style README and the receipt corpus.

## `calibration.json`

The tolerance band Assay's verifier compares against, measured on honest **gpt2** traffic
with injected numerical jitter, plus the sketch protocol parameters:

```json
{
  "sketch": {"seed": ..., "layers": [3, 6, 8, 11], "n_positions": 6, "sketch_dim": 32},
  "calibration": {"mu": {...}, "sigma": {...}, "gen_mismatch_rate": ...},
  "verifier": {"deep_check_rate": 0.25, "alpha": 0.01}
}
```

`gen_mismatch_rate` is the rate at which an honest greedy provider's emitted token differs
from M's own argmax at an inspected position — the null rate of the check that catches a
forged receipt. It is calibrated rather than assumed to be zero, because numerical jitter can
flip a near-tie.

The HMAC key in that file is published deliberately: without it the signatures in the receipt
corpus cannot be checked. It is a demo key for a research prototype, not a secret, and not a
TEE attestation key.

## Intended use and limits

Intended for reproducing and probing substitution-detection experiments. These weights are
random — M and M' have never been trained and their text is meaningless. Do not use them for
anything other than the verification experiments they exist for.

## Citation

```bibtex
@software{assay2026,
  title  = {Assay: per-request internal receipts for model-substitution detection},
  author = {Nagao, Yuta},
  year   = {2026},
  url    = {https://github.com/NagaYu/assay},
  note   = {Research prototype}
}
```