BaseDecision-Dyno / SERVING.md
onlyaady's picture
Publish calibrated BaseDecision-Dyno with portable HTTP serving
3e88dcc verified
|
Raw History Blame Contribute Delete
3.07 kB

Serve BaseDecision-Dyno

The server is included in the model download. It needs Python and a compatible NVIDIA CUDA/PyTorch environment; no HPC account, project checkout, or project environment variables are required. The tested stack used Python 3.12, PyTorch 2.14.0+cu130, Transformers 5.17.0 and PEFT 0.20.0. The BF16 checkpoint also needs accelerate and safetensors. This is the CUDA serving profile.

Download and start:

hf download onlyaady/BaseDecision-Dyno --local-dir BaseDecision-Dyno
python BaseDecision-Dyno/serve_typesafe.py

The model loads from the directory containing serve_typesafe.py, regardless of the current working directory. Default: read twice, with its fitted temperature applied internally. The local endpoint is http://127.0.0.1:8000.

Send a request from another terminal:

curl http://127.0.0.1:8000/v1/systemone \
  -H 'Content-Type: application/json' \
  -d '{"model":"onlyaady/BaseDecision-Dyno","state":"The invoice has a duplicated charge.","questions":{"decision":{"type":"choice","instructions":"Which team should handle this?","criteria":{"billing":"Invoices and refunds","technical":"Integration problems"}}}}'

Single-read flash mode requires explicit selection:

python BaseDecision-Dyno/serve_typesafe.py --mode flash

Flash uses its own fitted temperature and identifies itself as onlyaady/BaseDecision-Dyno-flash. A server process has a fixed mode. The request may use onlyaady/BaseDecision-Dyno or BaseDecision-Dyno as an alias for the model that process serves; the response reports the actual mode.

Optional arguments: --model /path/to/snapshot, --model onlyaady/BaseDecision-Dyno, --revision COMMIT_SHA, --gpu 0, --host 0.0.0.0, --port 8000. GET /health shows readiness, mode, calibration status and release id. GET /v1/models lists the served model.

The server supports choice (2–255 options), noul, and score (2–10 ordered levels). It returns the model's probabilities; noul is the probability under true, and score is the expected zero-based level. It does not generate answer text.

The limit is 16,384 rendered tokens per question, including repetition. Over-limit requests return HTTP 400/max_tokens_exceeded. There is no truncation or automatic flash fallback. Questions are independent forward passes. One request is processed at a time; concurrent requests receive HTTP 529. JSON and gzip JSON are accepted.

For remote access, bind an appropriate host, supply your HTTPS reverse proxy, and optionally set DYNO_API_KEY before startup; clients then send an Authorization: Bearer header. This optional API key is the only project-named environment setting. The server does not log request bodies or provision public hosting.

Benchmark Heaven can use its published typesafe adapter, base endpoint http://127.0.0.1:8000, model onlyaady/BaseDecision-Dyno, and no key for a local server without authentication. Pin the Hugging Face commit for an evaluation. The evaluator still needs to review the adapter and run its official evaluation.