serve/README.md

# amos-serve

The serving code of the Amos 1 decision models (`amos-1-large`, `amos-1-medium`, `amos-1-small`, `amos-1-mini`): one
HTTP server that takes a case and typed questions and answers every question from one forward pass, each with a
probability. No text is generated. This is the code behind api.amos.scira.ai and the code the Decision Index runs.

Install into a fresh environment (Python 3.12 or newer, an NVIDIA GPU):

```bash
pip install torch==2.14.0 --index-url https://download.pytorch.org/whl/cu130
pip install .
```

The dependencies are pinned (torch 2.14.0, transformers 5.17.0, flash-linear-attention 0.5.2) because they are the
versions every published number was measured with: with them the package answers byte for byte like our own servers.
Other versions run, but probabilities can move by up to 0.02.

Serve a model directory (a Hugging Face checkout with `model/`, `config.json` and `calibration.json`):

```bash
python -m amos.serve amos-1-large --device cuda --port 8000 --rotations 1 --max-len 131072 --strict --no-state-cache
```

- `--strict` keeps question and option text exactly as sent and refuses requests over `--max-len` tokens instead of
  truncating them.
- `--rotations N` averages each choice question over N option orders (1 is what the Index latency rule allows).
- `--no-state-cache` encodes every request from scratch.
- `GET /v1/info` reports the model name; `POST /v1/systemone` takes the System One request shape:

```bash
curl -s localhost:8000/v1/systemone -H 'content-type: application/json' -d '{
  "state": "Invoice INV-2231 for 306.00 was charged twice this morning. Please refund one of them today.",
  "questions": {
    "refund": {"type": "noul", "instructions": "Is the customer asking for a refund?"},
    "team": {"type": "choice", "instructions": "Which team should handle this?",
             "criteria": {"billing": null, "technical": null, "sales": null}},
    "urgency": {"type": "score", "instructions": "How urgent is this?", "criteria": ["Can wait", "This week", "Today"]}
  }
}'
```

Choice questions take 2 to 255 options and are read in one pass. Photos go in `images` as base64 data URLs; every
question sees them. `span` (quote the case) and `region` (point at the photo) need the model's `heads.pt`.

`sources.json` lists every file with the repository commit and sha256 it was taken from. `decoder.py` and
`calibrate.py` are the repository's modules cut to the definitions the server reaches; the other modules are verbatim.