python/README.md

TinyDecide for Python

The TinyDecide engine in Python. It needs only numpy. On the shared conformance set it gives the same token ids, picks and spans as the JavaScript engine, and its probabilities differ by about 1e-6.

Install

Install it from a downloaded copy of the model repo.

hf download TheREZOR/TinyDecide --local-dir TinyDecide --exclude "assets/*"
pip install ./TinyDecide/python            # add [hub] for TinyDecide.from_pretrained()

The package is not on PyPI yet.

Use

from tinydecide import TinyDecide

model = TinyDecide.load("TinyDecide")           # folder with meta.json + model.bin
# model = TinyDecide.from_pretrained()          # downloads from the Hub (needs huggingface_hub)
# model = TinyDecide.from_pretrained(subfolder="full")   # the 13.8M build

r = model.answer(
    "Book a table for 4 at an Italian place near the station on Friday at 7:30",
    [
        {"type": "choice", "text": "Which app should handle this?",
         "options": ["reminders", "music", "calendar", "restaurants", "weather"]},
        {"type": "noul", "text": "The message is urgent."},
        {"type": "score", "text": "How positive is the tone?", "options": ["negative", "neutral", "positive"]},
        {"type": "span", "text": "Extract the time."},
    ],
)
app, urgent, tone, time = r["answers"]
app["pick"], urgent["p"], tone["score"], time["text"]    # 3, 0.22, 0.52, "7:30"

python examples/quickstart.py runs this example.

Answers

type	fields
choice	probs (one per option), pick, confidence (1 - normalised entropy), qvec, z0
score	the same, plus score from 0 (first level) to 1 (last level)
noul	p, the probability the statement is true; qvec, z0
span	text, p_present, p_span, tok (first and last state token), char

char is a [start, end) pair of Python string indices into the message, so message[start:end] is the span before trimming. The JavaScript engine counts UTF-16 units instead. The two agree unless the message has characters outside the BMP, such as emoji.

The result also has tokens, ids, ms and truncated. The engine reads the first 127 tokens of a message and sets truncated when it cuts the rest. A question longer than 192 tokens raises ValueError, and so does a choice or score question without 2 to 32 options.

Corrections

Store the answer's qvec and z0 under the option a person says is right, and pass them back as protos. Nothing retrains the model. Also keep the mean qvec of every message asked with the question, and pass it as center. Without it, the centre is the mean of the examples, and a single example has no effect.

import numpy as np
from tinydecide import make_protos

q = {"type": "choice", "text": "What kind of note is this?", "options": ["shopping", "todo", "event"]}
lists, seen = [[], [], []], []                        # examples per option (noul: [false, true])
for msg, k in [("buy oat milk", 0), ("call the plumber", 1), ("dentist thursday 4pm", 2)]:
    a = model.answer(msg, [q])["answers"][0]
    seen.append(a["qvec"])
    lists[k].append({"v": a["qvec"], "z": a["z0"]})  # a person said option k was right
protos = make_protos(q["type"], lists, model.meta["beta"], center=np.mean(seen, axis=0))
model.answer("get coffee beans", [q], protos=[protos])

Test

python tests/test_conformance.py        # inside the downloaded repo: checks model.bin and full/
python tests/test_conformance.py <dir with meta.json + model.bin> <full build dir> <conformance dir>

This checks the engine against the JavaScript reference outputs in sdk/conformance.