Spaces:
Sleeping
Sleeping
Commit History
5 days ago
-
fix gitignore
Dmitry Beresnev94d7e22 -
exclude config toml
Dmitry Beresnev7a18d61 -
fix security volnurabilities
Dmitry Beresnev70dee29
6 days ago
-
fix other security volnurabilities
Dmitry Beresnev6dc320d -
fix critical secutity issues
Dmitry Beresnev6471a06
-
Add inference metrics + bench script, tune threads/batch for 2-vCPU HF Space, backoff on readiness polling
Dmitry Beresnev27c1c3c
-
Enforce max_tokens upstream, prune request registry, reap dead workers,
Dmitry Beresnevfork-safe spawn, honor keep-alive, auth on /slots and /metrics, loopback worker bind, /health readiness probe, port reuse, CAS cancellation, socket timeouts, accurate Retry-After, etc30c6ca2
-
fix exceptions
Dmitry Beresnevdc32982
-
fix error format wrapping now applies to /v1/chat/completions and generation stats
Dmitry Beresnev470e737 -
change timeouts
Dmitry Beresnev2c31416 -
add token generation speed to ui
Dmitry Beresneve8080f5 -
fix request parsing
Dmitry Beresnev3634ca6
-
Log detailed error bodies for UI failures
Dmitry Beresnev7caa6ba -
Fix 400 for llama.cpp web UI completion requests
Dmitry Beresnev677456b -
Fix web UI chat by adding buffered SSE fallback
Dmitry Beresnev6379bd0 -
Fix Docker build for modular llm-manager
Dmitry Beresnev58d70b1 -
fix description
Dmitry Beresnev952d357 -
fix build bugs
Dmitry Beresnevacdc6c1 -
fix readme file
Dmitry Beresnevfe156b2 -
add new architectural diagram to readme file
Dmitry Beresneva51a89f -
Refactor the C++ LLM manager into modular components, moves Python modules under python/, and keeps the current control-plane behavior intact. The C++ server now has clearer separation for config, model lifecycle, runtime services, request parsing, HTTP helpers, and server routing, while Docker build/runtime paths were updated to compile multiple C++ files and load Python code from the new package folder
Dmitry Beresnev332826f
-
add auth, token policy, queue scheduler, and cancel flow, etc
Dmitry Beresnevd9ce859
-
add new endpoint to cancel all processing prompts
Dmitry Beresnev8ef326a
-
add new build profile
Dmitry Beresneva97386f -
fix encoding
Dmitry Beresnevd211568 -
fix model config
Dmitry Beresnev057edf0
-
fix proxied response in llm manager
Dmitry Beresnev53e9f39 -
fix routing in llm manager
Dmitry Beresneva4ee76d -
add cpp server
Dmitry Beresnevfc0860f
-
change llm model
Dmitry Beresnevf41621b -
change llm model
Dmitry Beresnev4f2dffc -
change model to Qwen2.5-Math-7B-Instruct-GGUF
Dmitry Beresnevcca3c7b -
change llm model to qwen2 math
Dmitry Beresnevfe7089d -
change llm model to mistral
Dmitry Beresnev97d9520 -
fix dockerfile
Dmitry Beresnevc33410f
-
change compilation flags
Dmitry Beresnev0e913e4 -
change compilation flags
Dmitry Beresnev1a4efad -
reduce context and batch
Dmitry Beresnev34775a7 -
fix repo name of model
Dmitry Beresnevdc883f9 -
fix repo of model
Dmitry Beresnevc7c8563 -
fix cmd in dockerfile
Dmitry Beresnev0fbce92 -
fix dockerfile
Dmitry Beresnevc261631 -
switch to qwen model via cpp server
Dmitry Beresnev9a590ac -
Log elapsed time and token rate when the response arrives.
Dmitry BeresnevAdd a hard timeout (e.g., 300s) and return a friendly 504. Add a rule to drop system prompts above a cap unless explicitly allowed.a8f6b6b -
a “slow request” logging, log when a request exceeds the budgeted prompt and gets compacted
Dmitry Beresnev130d9e3 -
add simple compacting
Dmitry Beresnev6381e7f -
fix context window size
Dmitry Beresnev62a5a49 -
fix payload processing
Dmitry Beresneve1e4b82 -
fix logger
Dmitry Beresneve9b8569 -
fix app to handle exceptions
Dmitry Beresnev7d65cc9