# Local inference

Pluto-1 is a two-encoder ensemble. The supported runtime is the PyTorch package
under `code/`; it loads both encoders, the per-question heads, and the ensemble
calibration.

## Tested environment

- **CPU, clean virtual environment:** Python 3.13.13, `torch 2.15.0+cpu`,
  `transformers 5.22.0`, `huggingface_hub 2.4.0`, `fastapi 0.162.0`,
  `uvicorn 0.54.0`.
- **CUDA:** Windows, NVIDIA GeForce RTX 5060 Ti 16GB, driver 610.74,
  Python 3.13.13, `torch 2.11.0+cu128`, `transformers 5.21.0`,
  `huggingface_hub 2.3.0`.

No performance or memory figures are claimed here.

## Install

The model is private during release preparation. Set `HF_TOKEN` to a token with
read access to `omida-ai/pluto-1`. Once the repository is public, the token and
`HF_TOKEN` lines can be omitted.

### Windows PowerShell (tested)

```powershell
python -m venv .venv
.venv\Scripts\python -m pip install --upgrade pip
.venv\Scripts\python -m pip install huggingface_hub
$env:HF_TOKEN="hf_..."
.venv\Scripts\hf.exe download omida-ai/pluto-1 --local-dir pluto-1
.venv\Scripts\python -m pip install -e "pluto-1/code[serve]"
.venv\Scripts\pluto.exe --model pluto-1 info
```

### Linux/macOS

These are the same package commands using the venv’s `bin/` paths; they were not
exercised on the Windows test host.

```bash
python3 -m venv .venv
.venv/bin/python -m pip install --upgrade pip
.venv/bin/python -m pip install huggingface_hub
export HF_TOKEN="hf_..."
.venv/bin/hf download omida-ai/pluto-1 --local-dir pluto-1
.venv/bin/python -m pip install -e "pluto-1/code[serve]"
.venv/bin/pluto --model pluto-1 info
```

## Device selection

The loader chooses CUDA automatically when a working CUDA PyTorch build is
available. Force CPU with `--device cpu`; force CUDA with `--device cuda`.

```powershell
.venv\Scripts\pluto.exe --model pluto-1 --device cpu info
.venv\Scripts\pluto.exe --model pluto-1 --device cuda info
```

For CUDA, use a CUDA PyTorch build before installing the runtime. The build used
in testing was:

```powershell
.venv\Scripts\python -m pip install torch --index-url https://download.pytorch.org/whl/cu128
```

The default PyPI wheel in the clean CPU test was `torch 2.15.0+cpu`.

## Python

From the parent directory that contains `pluto-1/`:

```python
from pluto.ensemble import EnsembleEngine

engine = EnsembleEngine("pluto-1")
result = engine.predict(
    "Invoice #4411 was charged twice.",
    {
        "action": {
            "type": "choice",
            "instructions": "What should happen?",
            "criteria": {
                "continue": "Proceed.",
                "human_review": "Escalate.",
                "stop": "Stop.",
            },
        },
        "needs_review": {"type": "noul", "instructions": "Requires review."},
        "urgency": {
            "type": "score",
            "instructions": "How urgent?",
            "criteria": ["low", "medium", "high"],
        },
    },
)
print(result["answers"])
```

Loading directly from the Hub, without a local checkout:

```python
from pluto.ensemble import EnsembleEngine

engine = EnsembleEngine.from_pretrained("omida-ai/pluto-1")
```

`from_pretrained` accepts `token=` when the repository is private.

## CLI

Create `questions.json`:

```json
{
  "action": {
    "type": "choice",
    "instructions": "What should the system do?",
    "criteria": {"continue": "Proceed.", "human_review": "Escalate.", "stop": "Stop."}
  },
  "needs_review": {"type": "noul", "instructions": "Requires review."},
  "urgency": {
    "type": "score",
    "instructions": "How urgent?",
    "criteria": ["low", "medium", "high"]
  }
}
```

Run a decision:

```powershell
.venv\Scripts\pluto.exe --model pluto-1 --device cpu predict `
  --state "Trace: tool error 1, no irreversible actions, 7 steps." `
  --questions questions.json
```

The output contains `answers.action` (choice), `answers.needs_review.noul`
(probability of true), and `answers.urgency.score` (expected level).

## HTTP server

Start the Jev-compatible server:

```powershell
.venv\Scripts\pluto.exe --model pluto-1 --device cpu serve --host 127.0.0.1 --port 8000
```

Check it:

```powershell
curl.exe -s http://127.0.0.1:8000/health
```

Send a decision request (`request.json` contains `state` plus the
`questions` object shown above):

```powershell
curl.exe -s -H "Content-Type: application/json" --data-binary "@request.json" `
  http://127.0.0.1:8000/v1/systemone
```

## Hub-only CLI load

```powershell
.venv\Scripts\pluto.exe --model omida-ai/pluto-1 --device cpu info
```

The first Hub-only load downloads the repository. Later loads reuse the local
Hugging Face cache.

## Supported question names

Pluto-1 is fitted for these fixed Typed Decisions question names:

`action`, `needs_review`, `outcome`, `risk`, `urgency`, `category`,
`churn_risk`, `needs_human`, `discrepancy_severity`, `duplicate`,
`matches_order`, `credential_compromise`, `severity`, `true_positive`.

Arbitrary new question names are not supported by this specialist checkpoint.