Phase A serving speedup: zero-copy state-cache expand (no deepcopy per decision) dd605dc verified guychuk commited on 18 days ago
Serving-path speedup: state-KV LRU cache + suffix diet (see inference/bench_serve.py) c552a5b verified guychuk commited on 18 days ago
typical-medium (tm1b): weights, evals, inference package, manifest 970f2df verified guychuk commited on 19 days ago