serve/README.md

amos-serve

The serving code of the Amos 1 decision models (amos-1-large, amos-1-medium, amos-1-small, amos-1-mini): one HTTP server that takes a case and typed questions and answers every question from one forward pass, each with a probability. No text is generated. This is the code behind api.amos.scira.ai and the code the Decision Index runs.

Install into a fresh environment (Python 3.12 or newer, an NVIDIA GPU):

pip install torch==2.14.0 --index-url https://download.pytorch.org/whl/cu130
pip install .

The dependencies are pinned (torch 2.14.0, transformers 5.17.0, flash-linear-attention 0.5.2) because they are the versions every published number was measured with: with them the package answers byte for byte like our own servers. Other versions run, but probabilities can move by up to 0.02.

Serve a model directory (a Hugging Face checkout with model/, config.json and calibration.json):

python -m amos.serve amos-1-large --device cuda --port 8000 --rotations 1 --max-len 131072 --strict --no-state-cache

• --strict keeps question and option text exactly as sent and refuses requests over --max-len tokens instead of truncating them.
• --rotations N averages each choice question over N option orders (1 is what the Index latency rule allows).
• --no-state-cache encodes every request from scratch.
• GET /v1/info reports the model name; POST /v1/systemone takes the System One request shape:

curl -s localhost:8000/v1/systemone -H 'content-type: application/json' -d '{
  "state": "Invoice INV-2231 for 306.00 was charged twice this morning. Please refund one of them today.",
  "questions": {
    "refund": {"type": "noul", "instructions": "Is the customer asking for a refund?"},
    "team": {"type": "choice", "instructions": "Which team should handle this?",
             "criteria": {"billing": null, "technical": null, "sales": null}},
    "urgency": {"type": "score", "instructions": "How urgent is this?", "criteria": ["Can wait", "This week", "Today"]}
  }
}'

Choice questions take 2 to 255 options and are read in one pass. Photos go in images as base64 data URLs; every question sees them. span (quote the case) and region (point at the photo) need the model's heads.pt.

sources.json lists every file with the repository commit and sha256 it was taken from. decoder.py and calibrate.py are the repository's modules cut to the definitions the server reaches; the other modules are verbatim.