Fetching metadata from the HF Docker repository...
-
Update start.sh
-
add mmproj vision support
-
update custom llama cpp support
-
increase batch size
-
bring verbosity back
-
some experimental
-
increase batch size
-
bring back flash attention
-
experimental flash attention
-
fix verbose
-
fix balanced profile
-
fix balanced profile
-
update batch size
-
fix flash attention
-
max optimisation
-
optimised cpu
-
minor fixes
-
experimental boosting changes
-
remove parallel support for faster inference
-
Fix llama-server startup and enable LangSearch via shell tool
-
Optimize inference performance: add continuous-batching, NO_WARMUP support, improve thread/batch recommendations
-
Update start.sh
-
Add LangSearch web search tool for small model knowledge augmentation
-
Add REASONING env toggle for llama-server reasoning mode
-
Use temp download dir and robust rmtree handler to avoid permission errors on cleanup
-
Save models using repo filename, set MODEL_PATH from marker, ensure HF xet log dirs writable
-
Allow MODEL_FILE override; improve diagnostics when no .gguf files present
-
Auto-tune ctx-size, batch and cache types for low-memory (<=16GB) Spaces; export CACHE_TYPE_{K,V}
-
Add QUANT_PREFERENCE and PERF_PROFILE (low_latency/balanced/throughput) with server defaults
-
Add startup phase logs and avoid recursive permission setup
-
Copy llama.cpp shared libraries into runtime image
-
Add model list and download progress logging
-
Speed up model download by selecting one GGUF
-
Speed up build (ccache) and fix /data permissions: create appuser and run downloader/server as appuser
-
Support CONTEXT_LENGTH env var (default 4096) for ctx-size
-
Use Python venv in runtime to avoid PEP 668 pip install error
-
Fix llama.cpp math compiler flags
-
Fix llama.cpp Debian slim build deps
-
Add optimized llama.cpp CPU inference backend
-
initial commit
