Add inference metrics + bench script, tune threads/batch for 2-vCPU HF Space, backoff on readiness polling

Enforce max_tokens upstream, prune request registry, reap dead workers,

Log detailed error bodies for UI failures

fix build bugs

Refactor the C++ LLM manager into modular components, moves Python modules under python/, and keeps the current control-plane behavior intact. The C++ server now has clearer separation for config, model lifecycle, runtime services, request parsing, HTTP helpers, and server routing, while Docker build/runtime paths were updated to compile multiple C++ files and load Python code from the new package folder