# Creative-bench methodology

Writing-voice evidence for two model files. Creative generations run at
temperature 0.7: capability discipline (greedy, deterministic) would
strangle voice; writing needs room. Seed fixed per file so runs are
reproducible short of sampler noise.
## Prompts

32 prompts from EQ-Bench Creative Writing v3
(`creative-bench/prompts.json`, vendored from
[EQ-bench/creative-writing-bench](https://github.com/EQ-bench/creative-writing-bench),
used with citation). One generation per prompt per file, 6000-token budget
(generous on purpose: every char is data), user turn only — some models
support no system role.

## What is measured (no judge model)

- **Shared trigram vocabulary** per prompt (Gryphe-style): size of
  trigram-set intersection over union, file A vs. file B. Lower means more
  distinct voices. Reported as mean and max over paired prompts.
- **Cliche density**: substring hits per 100 words against a plain-text
  list (`audition/cliches-starter.txt` documents its own provenance;
  bring your own list for serious work).
- **Texture stats**: lengths, sentence rhythm, lexical diversity,
  top-trigram fingerprint per file.
- **Blind pairs**: every prompt exported with both texts labeled A/B only.
  The reader judges; nothing is scored.

## What is archived

Per file, `creative-bench/runs/file-<x>-creative.jsonl`: prompt metadata,
full response text, full reasoning trace, finish reason, temp/seed/budget.
No personal information: server model ids are replaced with run labels
before saving.

## Reproduce

```bash
BENCH_LABEL=file-c python3 creative-bench/generate.py \
    --prompts creative-bench/prompts.json \
    --out creative-bench/runs/file-c-creative.jsonl
python3 audition/voice-taste.py creative-bench/runs/file-a-creative.jsonl \
    creative-bench/runs/file-b-creative.jsonl --out taste-out
```