inference / README.md

Commit History

  1. some experimental

    57426fb
  2. fix flash attention

    3a9ad3f
  3. max optimisation

    0c43530
  4. optimised cpu

    d83933c
  5. minor fixes

    edf2ef0
  6. experimental boosting changes

    434da24
  7. remove parallel support for faster inference

    e22f356
  8. Fix llama-server startup and enable LangSearch via shell tool

    1420bd1
  9. Optimize inference performance: add continuous-batching, NO_WARMUP support, improve thread/batch recommendations

    392efee
  10. Add LangSearch web search tool for small model knowledge augmentation

    c459436
  11. Add REASONING env toggle for llama-server reasoning mode

    a6b5235
  12. Add optimized llama.cpp CPU inference backend

    8ebf335
  13. initial commit

    Verified signature
    957406c