Spaces:

ResearchEngineering
/

AGI

Running

App Files Files Community

AGI / cpp

88.9 kB

Ctrl+K

2 contributors

History: 18 commits

Dmitry Beresnev

Add inference metrics + bench script, tune threads/batch for 2-vCPU HF Space, backoff on readiness polling

27c1c3c 18 days ago

config.cpp

10.2 kB
Enforce max_tokens upstream, prune request registry, reap dead workers, 18 days ago
config.h

448 Bytes
Refactor the C++ LLM manager into modular components, moves Python modules under python/, and keeps the current control-plane behavior intact. The C++ server now has clearer separation for config, model lifecycle, runtime services, request parsing, HTTP helpers, and server routing, while Docker build/runtime paths were updated to compile multiple C++ files and load Python code from the new package folder 4 months ago
http_helpers.cpp

3.46 kB
Refactor the C++ LLM manager into modular components, moves Python modules under python/, and keeps the current control-plane behavior intact. The C++ server now has clearer separation for config, model lifecycle, runtime services, request parsing, HTTP helpers, and server routing, while Docker build/runtime paths were updated to compile multiple C++ files and load Python code from the new package folder 4 months ago
http_helpers.h

511 Bytes
Refactor the C++ LLM manager into modular components, moves Python modules under python/, and keeps the current control-plane behavior intact. The C++ server now has clearer separation for config, model lifecycle, runtime services, request parsing, HTTP helpers, and server routing, while Docker build/runtime paths were updated to compile multiple C++ files and load Python code from the new package folder 4 months ago
llm_manager.cpp

2.2 kB
Enforce max_tokens upstream, prune request registry, reap dead workers, 18 days ago
llm_manager_types.cpp

600 Bytes
Refactor the C++ LLM manager into modular components, moves Python modules under python/, and keeps the current control-plane behavior intact. The C++ server now has clearer separation for config, model lifecycle, runtime services, request parsing, HTTP helpers, and server routing, while Docker build/runtime paths were updated to compile multiple C++ files and load Python code from the new package folder 4 months ago
llm_manager_types.h

3.71 kB
Enforce max_tokens upstream, prune request registry, reap dead workers, 18 days ago
model_manager.cpp

9.26 kB
Add inference metrics + bench script, tune threads/batch for 2-vCPU HF Space, backoff on readiness polling 18 days ago
model_manager.h

1.24 kB
Enforce max_tokens upstream, prune request registry, reap dead workers, 18 days ago
request_parsing.cpp

2.97 kB
fix request parsing 4 months ago
request_parsing.h

269 Bytes
Refactor the C++ LLM manager into modular components, moves Python modules under python/, and keeps the current control-plane behavior intact. The C++ server now has clearer separation for config, model lifecycle, runtime services, request parsing, HTTP helpers, and server routing, while Docker build/runtime paths were updated to compile multiple C++ files and load Python code from the new package folder 4 months ago
runtime_components.cpp

17.8 kB
Add inference metrics + bench script, tune threads/batch for 2-vCPU HF Space, backoff on readiness polling 18 days ago
runtime_components.h

5.24 kB
Add inference metrics + bench script, tune threads/batch for 2-vCPU HF Space, backoff on readiness polling 18 days ago
server.cpp

30.1 kB
Enforce max_tokens upstream, prune request registry, reap dead workers, 18 days ago
server.h

846 Bytes
Refactor the C++ LLM manager into modular components, moves Python modules under python/, and keeps the current control-plane behavior intact. The C++ server now has clearer separation for config, model lifecycle, runtime services, request parsing, HTTP helpers, and server routing, while Docker build/runtime paths were updated to compile multiple C++ files and load Python code from the new package folder 4 months ago