The Python API and the HTTP server now open the device with dispatch on ETH cores, 1 command queue and a 12x10 compute grid by default (the p150 target). Worker dispatch stays only as an explicit opt-in (11x10 on a p150). Card numbers re-measured in this configuration and independently verified; outputs unchanged. tt-model.yaml and SERVING.md are unchanged; they still describe the container image.
24 days ago
Python API: warm-up so the first real call is fast (warmup_variants, model.warmup()), quiet logs, install extras
from_pretrained() now warms up the common per-request variants with dummy inputs (device and host), so the first real call is within about 10% of later calls (independently verified in fresh processes; outputs bit-identical; steady-state latency unchanged). Adds model.warmup(...) for other variants, verbose=False by default, and documents the [server,test] extras and repo-root paths. tt-model.yaml and SERVING.md are unchanged.
code/ adds an installable Python package with a from_pretrained()/model(...) API, examples and tests (independently verified from a clean shell; outputs equal the server; no numerics change unless stated in the card). README usage is now Python-first; HTTP serving moved below. tt-model.yaml and SERVING.md are unchanged; they still describe the container image.
Optimized build (2026-10-03): model call 19.0 ms, trace 17.7 ms
code/ updated to the independently verified optimized build (12x10 compute grid). README Demo & Performances, GPU_COMPARISON.md update, OPT_REPORT/OPT_BASELINE, VERIFICATION_2026-10-03.md and patches/tt-metal-eth-dispatch.patch. tt-model.yaml and SERVING.md are unchanged; they still describe the container image.
