Release prep: ensemble-aware CLI/server, clean-install validation, compact Run locally section and local inference doc

README.md

@@ -1,222 +1,252 @@
----
-license: cc-by-nc-4.0
-license_name: cc-by-nc-4.0
-license_link: https://creativecommons.org/licenses/by-nc/4.0/
-language:
-- en
-library_name: pytorch
-pipeline_tag: text-classification
-tags:
-- decision-model
-- system-one
-- calibrated-decisions
-- typed-decisions
-- choice
-- noul
-- score
-- ensemble
-- modernbert
-- deberta
-- pytorch
-- safetensors
----
-
-# Pluto-1
-
-### A 333M decision model that beats 27B-class Jev 1.13 on Typed Decisions at 1/80th the size and 13x lower latency.
-
-Pluto-1 is a dual-encoder decision model. Give it a state and typed questions; it
-returns calibrated probability distributions for `choice`, `noul`, and `score`
-questions. It never generates text. There is nothing to parse, nothing to retry,
-and nothing to hallucinate.
-
-Built solo on one **RTX 5060 Ti 16GB**.
-
-**Headline:** 0.7515 accuracy on the independent Typed Decisions test split,
-beating published Jev 1.13 (0.727) and Liquid d1 (0.742), at **54 ms p50** and
-**332.8M parameters**. Laya-typed, measured by us on the same rows, remains 1.65
-points higher on raw accuracy; Pluto-1 wins on calibration, latency, and size.
-
----
-
-## The receipts
-
-400 held-out cases / 2,000 probabilistic decisions. Higher accuracy, lower KL,
-Brier, and ECE are better.
-
-| Model | Params | Mode | Accuracy | KL | Brier | ECE | p50 |
-|---|---:|---|---:|---:|---:|---:|---:|
-| **Pluto-1** | **332.8M** | fitted on train | **0.7515** | **0.1183** | **0.0649** | **0.1400** | **54.2 ms** |
-| Pluto-1 Air (ablation) | 149.1M | fitted on train | 0.7275 | 0.1391 | 0.0765 | 0.1067 | 26.9 ms |
-| Laya-typed | 421.3M | fitted on train | **0.7680** | 0.1171 | 0.0615 | 0.2152 | 66.2 ms |
-| Jev 1.13 | ~27B | zero-shot (published) | 0.727 | 1.442 | 0.148 | 0.144 | ~710 ms |
-| Liquid d1 | ~3B | zero-shot (published) | 0.742 | 0.475 | 0.155 | 0.124 | ~525 ms |
-
-**What this means**
-- **Beats Jev 1.13 and d1 on accuracy.** 1/80th Jev's size, ~13x lower latency.
-- **Within 1.65 points of Laya-typed.** 35% lower ECE, 18% lower latency, 21% fewer parameters.
-- **Calibration is the point.** 0.140 ECE on 2,000 decisions; 0.107 for the 149M single.
-- **No token bill at inference.** One encoder pass per specialist, no generated tokens.
-
-![Accuracy](assets/benchmark_accuracy.png)
-
-![Accuracy vs latency](assets/accuracy_vs_latency.png)
-
-![Calibration](assets/calibration.png)
-
-![Ensemble payoff](assets/per_kind.png)
-
-*The per-primitive chart is an ablation against the 149M single checkpoint; that
-single checkpoint is not included in this repository.*
-
-### Comparison hygiene
-
-Pluto-1 and Laya-typed are fitted on the benchmark's `train` split. Jev 1.13 and
-d1 numbers are published zero-shot values. Laya-typed was run by us with
-`laya 0.4.17` on the same test rows and scored with the public metric definition.
-This is a descriptive comparison, not a controlled head-to-head across every
-system and every task. The benchmark `test` split was never used for training,
-early stopping, model selection, or calibration.
-
----
-
-## Architecture
-
-![Architecture](assets/architecture.png)
-
-```
-state + typed questions
-        |
-  ModernBERT specialist (149M)     DeBERTa specialist (184M)
-        |                                  |
-        +-------- calibrated 0.5/0.5 ------+
-                       |
-          choice / noul / score distributions
-                  + confidence + escalate
-```
-
-- Two specialist encoders, each with 13 per-question heads.
-- The ModernBERT branch is initialized from the general Pluto-149M backbone.
-- The DeBERTa-v3-base branch adds architectural diversity.
-- Probabilities are averaged and temperature-calibrated per primitive.
-- One case = five questions, one pass through each encoder.
-- No autoregressive decoding.
-
----
-
-## Quickstart
-
-```bash
-git clone https://huggingface.co/omida-ai/pluto-1
-pip install -e pluto-1/code
-```
-
-```python
-import json
-from pluto.ensemble import EnsembleEngine
-
-engine = EnsembleEngine("pluto-1")  # or EnsembleEngine("omida-ai/pluto-1")
-
-state = json.dumps({
-    "agent": {"autonomy": "checkpointed", "model": "internal-agent-v4"},
-    "task": "Delete personal data for the accounts in the erasure queue.",
-    "trace_summary": {"steps": 7, "tool_errors": 1, "irreversible_actions": 0},
-})
-
-questions = {
-    "action": {
-        "type": "choice",
-        "instructions": "What should the observability system do with this trace?",
-        "criteria": {
-            "continue": "Let the agent proceed without interruption.",
-            "human_review": "Queue this trace for a human to review.",
-            "observe": "Keep running, but flag the trace for later sampling.",
-            "stop": "Halt the agent now.",
-        },
-    },
-    "needs_review": {
-        "type": "noul",
-        "instructions": "This trace requires human review.",
-    },
-    "risk": {
-        "type": "score",
-        "instructions": "How risky was the agent's behaviour in this trace?",
-        "criteria": ["Benign", "Low", "Moderate", "High"],
-    },
-}
-
-result = engine.predict(state, questions)
-print(result["answers"]["action"]["choice"])
-print(result["answers"]["needs_review"]["noul"])
-print(result["answers"]["risk"]["score"])
-print(result["escalate"])
-```
-
-Supported question names: `action`, `needs_review`, `outcome`, `risk`, `urgency`,
-`category`, `churn_risk`, `needs_human`, `discrepancy_severity`, `duplicate`,
-`matches_order`, `credential_compromise`, `severity`, `true_positive`.
-
-This is a **specialist** for the Typed Decisions workflows. It is not a general
-chat model and does not accept arbitrary new question names.
-
----
-
-## Training
-
-- 1,200 Typed Decisions training cases. 1,080 fit / 60 validation / 60 calibration.
-- 5,400 fitted decisions, 13 per-question heads, 8 epochs, freeze 8 backbone layers.
-- ModernBERT-base 149.1M + DeBERTa-v3-base 183.8M, 0.5/0.5 probability average.
-- Per-primitive temperatures fitted on the 60-case calibration split.
-- Test split untouched until final scoring.
-- General backbone initialized from Pluto-149M, trained on 655,806 calibrated
-  decisions from `SargeDev/jev-distill-corpus-v3`.
-
-Data and base-model attribution is in `NOTICE`.
-
----
-
-## What Pluto-1 is not
-
-- It is not a reasoning model. It does not compete on GPQA, MMLU-Pro, HLE, or
-  open-ended generation.
-- It is not Laya-typed on raw accuracy: Laya-typed is 1.65 points higher on this
-  test split. Pluto-1 wins calibration, latency, and parameter count.
-- It is not a general arbitrary-schema model. The 13 question names are fixed.
-- It has no certified risk-controlled abstention gate. `escalate` is a useful
-  confidence signal, not a guarantee. Validate thresholds on your data.
-- It is English-only.
-- It is synthetic-benchmark-validated. Production reliability requires production
-  evaluation.
-
----
-
-## License and commercial use
-
-Pluto-1 is released for **non-commercial** use under
-[CC BY-NC 4.0](https://creativecommons.org/licenses/by-nc/4.0/). You can use,
-modify, and build non-commercial derivatives with attribution.
-
-**Commercial use requires a separate license from Omida AI.** Hosted APIs, paid
-products, internal business operations, consulting deliverables, and paid
-redistribution are not permitted under CC BY-NC 4.0.
-
-See `COMMERCIAL_LICENSE.md` for the commercial licensing path.
-
-The runtime code under `code/` is Apache 2.0; that does not grant commercial
-rights to the model weights. Upstream components remain under their original
-licenses (see `NOTICE`).
-
----
-
-## Citation
-
-```bibtex
-@misc{omida2026pluto1,
-  title  = {Pluto-1: A Compact Calibrated Decision Model},
-  author = {Omida AI},
-  year   = {2026},
-  url    = {https://huggingface.co/omida-ai/pluto-1}
-}
-```
-
-Built by **Omida AI** on one 16GB consumer GPU.
+---
+license: cc-by-nc-4.0
+license_name: cc-by-nc-4.0
+license_link: https://creativecommons.org/licenses/by-nc/4.0/
+language:
+- en
+library_name: pytorch
+pipeline_tag: text-classification
+tags:
+- decision-model
+- system-one
+- calibrated-decisions
+- typed-decisions
+- choice
+- noul
+- score
+- ensemble
+- modernbert
+- deberta
+- pytorch
+- safetensors
+---
+
+# Pluto-1
+
+### A 333M decision model that beats 27B-class Jev 1.13 on Typed Decisions at 1/80th the size and 13x lower latency.
+
+Pluto-1 is a dual-encoder decision model. Give it a state and typed questions; it
+returns calibrated probability distributions for `choice`, `noul`, and `score`
+questions. It never generates text. There is nothing to parse, nothing to retry,
+and nothing to hallucinate.
+
+Built solo on one **RTX 5060 Ti 16GB**.
+
+**Headline:** 0.7515 accuracy on the independent Typed Decisions test split,
+beating published Jev 1.13 (0.727) and Liquid d1 (0.742), at **54 ms p50** and
+**332.8M parameters**. Laya-typed, measured by us on the same rows, remains 1.65
+points higher on raw accuracy; Pluto-1 wins on calibration, latency, and size.
+
+---
+
+## The receipts
+
+400 held-out cases / 2,000 probabilistic decisions. Higher accuracy, lower KL,
+Brier, and ECE are better.
+
+| Model | Params | Mode | Accuracy | KL | Brier | ECE | p50 |
+|---|---:|---|---:|---:|---:|---:|---:|
+| **Pluto-1** | **332.8M** | fitted on train | **0.7515** | **0.1183** | **0.0649** | **0.1400** | **54.2 ms** |
+| Pluto-1 Air (ablation) | 149.1M | fitted on train | 0.7275 | 0.1391 | 0.0765 | 0.1067 | 26.9 ms |
+| Laya-typed | 421.3M | fitted on train | **0.7680** | 0.1171 | 0.0615 | 0.2152 | 66.2 ms |
+| Jev 1.13 | ~27B | zero-shot (published) | 0.727 | 1.442 | 0.148 | 0.144 | ~710 ms |
+| Liquid d1 | ~3B | zero-shot (published) | 0.742 | 0.475 | 0.155 | 0.124 | ~525 ms |
+
+**What this means**
+- **Beats Jev 1.13 and d1 on accuracy.** 1/80th Jev's size, ~13x lower latency.
+- **Within 1.65 points of Laya-typed.** 35% lower ECE, 18% lower latency, 21% fewer parameters.
+- **Calibration is the point.** 0.140 ECE on 2,000 decisions; 0.107 for the 149M single.
+- **No token bill at inference.** One encoder pass per specialist, no generated tokens.
+
+![Accuracy](assets/benchmark_accuracy.png)
+
+![Accuracy vs latency](assets/accuracy_vs_latency.png)
+
+![Calibration](assets/calibration.png)
+
+![Ensemble payoff](assets/per_kind.png)
+
+*The per-primitive chart is an ablation against the 149M single checkpoint; that
+single checkpoint is not included in this repository.*
+
+### Comparison hygiene
+
+Pluto-1 and Laya-typed are fitted on the benchmark's `train` split. Jev 1.13 and
+d1 numbers are published zero-shot values. Laya-typed was run by us with
+`laya 0.4.17` on the same test rows and scored with the public metric definition.
+This is a descriptive comparison, not a controlled head-to-head across every
+system and every task. The benchmark `test` split was never used for training,
+early stopping, model selection, or calibration.
+
+---
+
+## Architecture
+
+![Architecture](assets/architecture.png)
+
+```
+state + typed questions
+        |
+  ModernBERT specialist (149M)     DeBERTa specialist (184M)
+        |                                  |
+        +-------- calibrated 0.5/0.5 ------+
+                       |
+          choice / noul / score distributions
+                  + confidence + escalate
+```
+
+- Two specialist encoders, each with 13 per-question heads.
+- The ModernBERT branch is initialized from the general Pluto-149M backbone.
+- The DeBERTa-v3-base branch adds architectural diversity.
+- Probabilities are averaged and temperature-calibrated per primitive.
+- One case = five questions, one pass through each encoder.
+- No autoregressive decoding.
+
+---
+
+## Quickstart
+
+```bash
+git clone https://huggingface.co/omida-ai/pluto-1
+pip install -e pluto-1/code
+```
+
+```python
+import json
+from pluto.ensemble import EnsembleEngine
+
+engine = EnsembleEngine("pluto-1")  # or EnsembleEngine("omida-ai/pluto-1")
+
+state = json.dumps({
+    "agent": {"autonomy": "checkpointed", "model": "internal-agent-v4"},
+    "task": "Delete personal data for the accounts in the erasure queue.",
+    "trace_summary": {"steps": 7, "tool_errors": 1, "irreversible_actions": 0},
+})
+
+questions = {
+    "action": {
+        "type": "choice",
+        "instructions": "What should the observability system do with this trace?",
+        "criteria": {
+            "continue": "Let the agent proceed without interruption.",
+            "human_review": "Queue this trace for a human to review.",
+            "observe": "Keep running, but flag the trace for later sampling.",
+            "stop": "Halt the agent now.",
+        },
+    },
+    "needs_review": {
+        "type": "noul",
+        "instructions": "This trace requires human review.",
+    },
+    "risk": {
+        "type": "score",
+        "instructions": "How risky was the agent's behaviour in this trace?",
+        "criteria": ["Benign", "Low", "Moderate", "High"],
+    },
+}
+
+result = engine.predict(state, questions)
+print(result["answers"]["action"]["choice"])
+print(result["answers"]["needs_review"]["noul"])
+print(result["answers"]["risk"]["score"])
+print(result["escalate"])
+```
+
+Supported question names: `action`, `needs_review`, `outcome`, `risk`, `urgency`,
+`category`, `churn_risk`, `needs_human`, `discrepancy_severity`, `duplicate`,
+`matches_order`, `credential_compromise`, `severity`, `true_positive`.
+
+This is a **specialist** for the Typed Decisions workflows. It is not a general
+chat model and does not accept arbitrary new question names.
+
+---
+
+## Training
+
+- 1,200 Typed Decisions training cases. 1,080 fit / 60 validation / 60 calibration.
+- 5,400 fitted decisions, 13 per-question heads, 8 epochs, freeze 8 backbone layers.
+- ModernBERT-base 149.1M + DeBERTa-v3-base 183.8M, 0.5/0.5 probability average.
+- Per-primitive temperatures fitted on the 60-case calibration split.
+- Test split untouched until final scoring.
+- General backbone initialized from Pluto-149M, trained on 655,806 calibrated
+  decisions from `SargeDev/jev-distill-corpus-v3`.
+
+Data and base-model attribution is in `NOTICE`.
+
+---
+
+## What Pluto-1 is not
+
+- It is not a reasoning model. It does not compete on GPQA, MMLU-Pro, HLE, or
+  open-ended generation.
+- It is not Laya-typed on raw accuracy: Laya-typed is 1.65 points higher on this
+  test split. Pluto-1 wins calibration, latency, and parameter count.
+- It is not a general arbitrary-schema model. The 13 question names are fixed.
+- It has no certified risk-controlled abstention gate. `escalate` is a useful
+  confidence signal, not a guarantee. Validate thresholds on your data.
+- It is English-only.
+- It is synthetic-benchmark-validated. Production reliability requires production
+  evaluation.
+
+---
+
+
+## Run locally
+
+Verified with the repository runtime on CPU and CUDA.
+
+```bash
+# Set HF_TOKEN in your environment while this repository is private.
+pip install huggingface_hub
+hf download omida-ai/pluto-1 --local-dir pluto-1
+pip install -e "pluto-1/code[serve]"
+pluto --model pluto-1 info
+```
+
+```python
+from pluto.ensemble import EnsembleEngine
+
+engine = EnsembleEngine("pluto-1")
+result = engine.predict(
+    "Invoice #4411 was charged twice.",
+    {
+        "action": {"type": "choice", "instructions": "What should happen?", "criteria": {"continue": "Proceed.", "human_review": "Escalate.", "stop": "Stop."}},
+        "needs_review": {"type": "noul", "instructions": "Requires review."},
+        "urgency": {"type": "score", "instructions": "How urgent?", "criteria": ["low", "medium", "high"]},
+    },
+)
+print(result["answers"])
+```
+
+Full CPU, CUDA, CLI, and HTTP instructions: [docs/LOCAL_INFERENCE.md](docs/LOCAL_INFERENCE.md).
+
+## License and commercial use
+
+Pluto-1 is released for **non-commercial** use under
+[CC BY-NC 4.0](https://creativecommons.org/licenses/by-nc/4.0/). You can use,
+modify, and build non-commercial derivatives with attribution.
+
+**Commercial use requires a separate license from Omida AI.** Hosted APIs, paid
+products, internal business operations, consulting deliverables, and paid
+redistribution are not permitted under CC BY-NC 4.0.
+
+See `COMMERCIAL_LICENSE.md` for the commercial licensing path.
+
+The runtime code under `code/` is Apache 2.0; that does not grant commercial
+rights to the model weights. Upstream components remain under their original
+licenses (see `NOTICE`).
+
+---
+
+## Citation
+
+```bibtex
+@misc{omida2026pluto1,
+  title  = {Pluto-1: A Compact Calibrated Decision Model},
+  author = {Omida AI},
+  year   = {2026},
+  url    = {https://huggingface.co/omida-ai/pluto-1}
+}
+```
+
+Built by **Omida AI** on one 16GB consumer GPU.

code/pluto/__init__.py

@@ -6,11 +6,13 @@ from .schema import Question, SchemaError, parse_question, parse_questions
 from .engine import PlutoEngine
 from .task_specialist import TaskSpecialistEngine
 from .ensemble import EnsembleEngine
+from .loader import load_engine
 
 __all__ = [
     "PlutoEngine",
     "TaskSpecialistEngine",
     "EnsembleEngine",
+    "load_engine",
     "Question",
     "SchemaError",
     "parse_question",

code/pluto/cli.py

@@ -1,62 +1,63 @@
-"""Command line interface for Pluto."""
-
-from __future__ import annotations
-
-import argparse
-import json
-import os
-import sys
-
-from .engine import PlutoEngine
-
-
-def _resolve_questions(value: str) -> dict:
-    if os.path.exists(value):
-        with open(value, "r", encoding="utf-8") as f:
-            return json.load(f)
-    return json.loads(value)
-
-
-def main(argv: list[str] | None = None) -> None:
-    ap = argparse.ArgumentParser(prog="pluto", description="Pluto decision models")
-    ap.add_argument("--model", default=os.environ.get("PLUTO_MODEL", "artifacts/pluto-149m-v3"))
-    sub = ap.add_subparsers(dest="cmd", required=True)
-
-    sub.add_parser("info", help="show checkpoint metadata")
-
-    p = sub.add_parser("predict", help="answer typed questions about a state")
-    p.add_argument("--state", help="state text")
-    p.add_argument("--state-file", help="read state from a file")
-    p.add_argument("--questions", required=True, help="questions JSON or path")
-    p.add_argument("--max-len", type=int, default=None)
-
-    s = sub.add_parser("serve", help="start the HTTP server")
-    s.add_argument("--host", default="0.0.0.0")
-    s.add_argument("--port", type=int, default=8000)
-
-    args = ap.parse_args(argv)
-    if args.cmd == "info":
-        print(json.dumps(PlutoEngine(args.model).info, indent=2))
-        return
-    if args.cmd == "predict":
-        if args.state_file:
-            with open(args.state_file, "r", encoding="utf-8") as f:
-                state = f.read()
-        elif args.state is not None:
-            state = args.state
-        else:
-            state = sys.stdin.read()
-        engine = PlutoEngine(args.model)
-        result = engine.predict(state, _resolve_questions(args.questions), max_len=args.max_len)
-        print(json.dumps(result, indent=2))
-        return
-    if args.cmd == "serve":
-        import uvicorn
-
-        from .serve import create_app
-
-        uvicorn.run(create_app(args.model), host=args.host, port=args.port)
-
-
-if __name__ == "__main__":
-    main()
+"""Command line interface for Pluto."""
+
+from __future__ import annotations
+
+import argparse
+import json
+import os
+import sys
+
+from .loader import load_engine
+
+
+def _resolve_questions(value: str) -> dict:
+    if os.path.exists(value):
+        with open(value, "r", encoding="utf-8") as f:
+            return json.load(f)
+    return json.loads(value)
+
+
+def main(argv: list[str] | None = None) -> None:
+    ap = argparse.ArgumentParser(prog="pluto", description="Pluto decision models")
+    ap.add_argument("--model", default=os.environ.get("PLUTO_MODEL", "omida-ai/pluto-1"))
+    ap.add_argument("--device", default=None, help="cpu, cuda, or mps; default is automatic")
+    sub = ap.add_subparsers(dest="cmd", required=True)
+
+    sub.add_parser("info", help="show checkpoint metadata")
+
+    p = sub.add_parser("predict", help="answer typed questions about a state")
+    p.add_argument("--state", help="state text")
+    p.add_argument("--state-file", help="read state from a file")
+    p.add_argument("--questions", required=True, help="questions JSON or path")
+    p.add_argument("--max-len", type=int, default=None)
+
+    s = sub.add_parser("serve", help="start the HTTP server")
+    s.add_argument("--host", default="0.0.0.0")
+    s.add_argument("--port", type=int, default=8000)
+
+    args = ap.parse_args(argv)
+    if args.cmd == "info":
+        print(json.dumps(load_engine(args.model, device=args.device).info, indent=2))
+        return
+    if args.cmd == "predict":
+        if args.state_file:
+            with open(args.state_file, "r", encoding="utf-8") as f:
+                state = f.read()
+        elif args.state is not None:
+            state = args.state
+        else:
+            state = sys.stdin.read()
+        engine = load_engine(args.model, device=args.device)
+        result = engine.predict(state, _resolve_questions(args.questions), max_len=args.max_len)
+        print(json.dumps(result, indent=2))
+        return
+    if args.cmd == "serve":
+        import uvicorn
+
+        from .serve import create_app
+
+        uvicorn.run(create_app(args.model, device=args.device), host=args.host, port=args.port)
+
+
+if __name__ == "__main__":
+    main()

code/pluto/ensemble.py

@@ -49,6 +49,7 @@ class EnsembleEngine:
             "models": [e.info["name"] for e in self.engines],
             "weights": self.weights,
             "temperatures": self.temperatures,
+            "device": str(self.engines[0].device) if self.engines else str(self.device),
             "params": sum(e.info["params"] for e in self.engines),
         }
 

code/pluto/loader.py

@@ -0,0 +1,63 @@
+"""Model-kind detection and loading for the CLI and HTTP server."""
+
+from __future__ import annotations
+
+import json
+import os
+
+from .engine import PlutoEngine
+from .ensemble import EnsembleEngine
+from .task_specialist import TaskSpecialistEngine
+
+
+def resolve_model_path(model: str, token: str | None = None) -> str:
+    """Return a local path, downloading a Hub repository if needed."""
+    if os.path.exists(model):
+        return model
+    from huggingface_hub import snapshot_download
+
+    return snapshot_download(model, token=token)
+
+
+def _ensemble_config(directory: str) -> str | None:
+    preferred = os.path.join(directory, "ensemble_config.json")
+    if os.path.exists(preferred):
+        return preferred
+    legacy = os.path.join(directory, "config.json")
+    if os.path.exists(legacy):
+        try:
+            data = json.load(open(legacy, encoding="utf-8"))
+        except Exception:
+            return None
+        if isinstance(data, dict) and "models" in data and "weights" in data:
+            return legacy
+    return None
+
+
+def load_engine(model: str, device: str | None = None, token: str | None = None):
+    """Load an ensemble, specialist, or general checkpoint by its layout."""
+    path = resolve_model_path(model, token=token)
+    if os.path.isdir(path):
+        ensemble = _ensemble_config(path)
+        if ensemble:
+            return EnsembleEngine(ensemble, device=device, token=token)
+        if os.path.exists(os.path.join(path, "pluto_task_config.json")):
+            return TaskSpecialistEngine(path, device=device)
+        if os.path.exists(os.path.join(path, "pluto_config.json")):
+            return PlutoEngine(path, device=device)
+        raise ValueError(
+            f"{path!r} is not a Pluto checkpoint: expected ensemble_config.json, "
+            "pluto_task_config.json, or pluto_config.json"
+        )
+    if os.path.isfile(path):
+        base = os.path.dirname(path)
+        name = os.path.basename(path)
+        if name in ("ensemble_config.json", "config.json"):
+            ensemble = _ensemble_config(base)
+            if ensemble:
+                return EnsembleEngine(ensemble, device=device, token=token)
+        if name == "pluto_task_config.json":
+            return TaskSpecialistEngine(base, device=device)
+        if name == "pluto_config.json":
+            return PlutoEngine(base, device=device)
+    raise ValueError(f"unsupported model path: {path!r}")

code/pluto/serve.py

@@ -8,7 +8,7 @@ from typing import Any
 from fastapi import FastAPI, Header, HTTPException
 from pydantic import BaseModel, Field
 
-from .engine import PlutoEngine
+from .loader import load_engine
 from .schema import SchemaError
 
 
@@ -25,9 +25,9 @@ class Health(BaseModel):
     device: str
 
 
-def create_app(model_path: str | None = None) -> FastAPI:
-    path = model_path or os.environ.get("PLUTO_MODEL", "artifacts/pluto-149m-v3")
-    engine = PlutoEngine(path)
+def create_app(model_path: str | None = None, device: str | None = None) -> FastAPI:
+    path = model_path or os.environ.get("PLUTO_MODEL", "omida-ai/pluto-1")
+    engine = load_engine(path, device=device or os.environ.get("PLUTO_DEVICE"))
     api_key = os.environ.get("PLUTO_API_KEY")
     app = FastAPI(title="Pluto Decision API", version="0.1.0")
 

docs/LOCAL_INFERENCE.md

@@ -0,0 +1,181 @@
+# Local inference
+
+Pluto-1 is a two-encoder ensemble. The supported runtime is the PyTorch package
+under `code/`; it loads both encoders, the per-question heads, and the ensemble
+calibration.
+
+## Tested environment
+
+- **CPU, clean virtual environment:** Python 3.13.13, `torch 2.15.0+cpu`,
+  `transformers 5.22.0`, `huggingface_hub 2.4.0`, `fastapi 0.162.0`,
+  `uvicorn 0.54.0`.
+- **CUDA:** Windows, NVIDIA GeForce RTX 5060 Ti 16GB, driver 610.74,
+  Python 3.13.13, `torch 2.11.0+cu128`, `transformers 5.21.0`,
+  `huggingface_hub 2.3.0`.
+
+No performance or memory figures are claimed here.
+
+## Install
+
+The model is private during release preparation. Set `HF_TOKEN` to a token with
+read access to `omida-ai/pluto-1`. Once the repository is public, the token and
+`HF_TOKEN` lines can be omitted.
+
+### Windows PowerShell (tested)
+
+```powershell
+python -m venv .venv
+.venv\Scripts\python -m pip install --upgrade pip
+.venv\Scripts\python -m pip install huggingface_hub
+$env:HF_TOKEN="hf_..."
+.venv\Scripts\hf.exe download omida-ai/pluto-1 --local-dir pluto-1
+.venv\Scripts\python -m pip install -e "pluto-1/code[serve]"
+.venv\Scripts\pluto.exe --model pluto-1 info
+```
+
+### Linux/macOS
+
+These are the same package commands using the venv’s `bin/` paths; they were not
+exercised on the Windows test host.
+
+```bash
+python3 -m venv .venv
+.venv/bin/python -m pip install --upgrade pip
+.venv/bin/python -m pip install huggingface_hub
+export HF_TOKEN="hf_..."
+.venv/bin/hf download omida-ai/pluto-1 --local-dir pluto-1
+.venv/bin/python -m pip install -e "pluto-1/code[serve]"
+.venv/bin/pluto --model pluto-1 info
+```
+
+## Device selection
+
+The loader chooses CUDA automatically when a working CUDA PyTorch build is
+available. Force CPU with `--device cpu`; force CUDA with `--device cuda`.
+
+```powershell
+.venv\Scripts\pluto.exe --model pluto-1 --device cpu info
+.venv\Scripts\pluto.exe --model pluto-1 --device cuda info
+```
+
+For CUDA, use a CUDA PyTorch build before installing the runtime. The build used
+in testing was:
+
+```powershell
+.venv\Scripts\python -m pip install torch --index-url https://download.pytorch.org/whl/cu128
+```
+
+The default PyPI wheel in the clean CPU test was `torch 2.15.0+cpu`.
+
+## Python
+
+From the parent directory that contains `pluto-1/`:
+
+```python
+from pluto.ensemble import EnsembleEngine
+
+engine = EnsembleEngine("pluto-1")
+result = engine.predict(
+    "Invoice #4411 was charged twice.",
+    {
+        "action": {
+            "type": "choice",
+            "instructions": "What should happen?",
+            "criteria": {
+                "continue": "Proceed.",
+                "human_review": "Escalate.",
+                "stop": "Stop.",
+            },
+        },
+        "needs_review": {"type": "noul", "instructions": "Requires review."},
+        "urgency": {
+            "type": "score",
+            "instructions": "How urgent?",
+            "criteria": ["low", "medium", "high"],
+        },
+    },
+)
+print(result["answers"])
+```
+
+Loading directly from the Hub, without a local checkout:
+
+```python
+from pluto.ensemble import EnsembleEngine
+
+engine = EnsembleEngine.from_pretrained("omida-ai/pluto-1")
+```
+
+`from_pretrained` accepts `token=` when the repository is private.
+
+## CLI
+
+Create `questions.json`:
+
+```json
+{
+  "action": {
+    "type": "choice",
+    "instructions": "What should the system do?",
+    "criteria": {"continue": "Proceed.", "human_review": "Escalate.", "stop": "Stop."}
+  },
+  "needs_review": {"type": "noul", "instructions": "Requires review."},
+  "urgency": {
+    "type": "score",
+    "instructions": "How urgent?",
+    "criteria": ["low", "medium", "high"]
+  }
+}
+```
+
+Run a decision:
+
+```powershell
+.venv\Scripts\pluto.exe --model pluto-1 --device cpu predict `
+  --state "Trace: tool error 1, no irreversible actions, 7 steps." `
+  --questions questions.json
+```
+
+The output contains `answers.action` (choice), `answers.needs_review.noul`
+(probability of true), and `answers.urgency.score` (expected level).
+
+## HTTP server
+
+Start the Jev-compatible server:
+
+```powershell
+.venv\Scripts\pluto.exe --model pluto-1 --device cpu serve --host 127.0.0.1 --port 8000
+```
+
+Check it:
+
+```powershell
+curl.exe -s http://127.0.0.1:8000/health
+```
+
+Send a decision request (`request.json` contains `state` plus the
+`questions` object shown above):
+
+```powershell
+curl.exe -s -H "Content-Type: application/json" --data-binary "@request.json" `
+  http://127.0.0.1:8000/v1/systemone
+```
+
+## Hub-only CLI load
+
+```powershell
+.venv\Scripts\pluto.exe --model omida-ai/pluto-1 --device cpu info
+```
+
+The first Hub-only load downloads the repository. Later loads reuse the local
+Hugging Face cache.
+
+## Supported question names
+
+Pluto-1 is fitted for these fixed Typed Decisions question names:
+
+`action`, `needs_review`, `outcome`, `risk`, `urgency`, `category`,
+`churn_risk`, `needs_human`, `discrepancy_severity`, `duplicate`,
+`matches_order`, `credential_compromise`, `severity`, `true_positive`.
+
+Arbitrary new question names are not supported by this specialist checkpoint.