Instructions to use anilsuthar2004/qwen-bot with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use anilsuthar2004/qwen-bot with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf anilsuthar2004/qwen-bot # Run inference directly in the terminal: llama cli -hf anilsuthar2004/qwen-bot
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf anilsuthar2004/qwen-bot # Run inference directly in the terminal: llama cli -hf anilsuthar2004/qwen-bot
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf anilsuthar2004/qwen-bot # Run inference directly in the terminal: ./build/bin/llama cli -hf anilsuthar2004/qwen-bot
Use Docker
docker model run hf.co/anilsuthar2004/qwen-bot
- Llama new
- LM Studio
- Jan
- Ollama
How to use anilsuthar2004/qwen-bot with Ollama:
ollama run hf.co/anilsuthar2004/qwen-bot
- Unsloth Desktop
- Docker Model Runner
How to use anilsuthar2004/qwen-bot with Docker Model Runner:
docker model run hf.co/anilsuthar2004/qwen-bot
- Lemonade
How to use anilsuthar2004/qwen-bot with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull anilsuthar2004/qwen-bot
Run and chat with the model
lemonade run user.qwen-bot-{{QUANT_TAG}}List all available models
lemonade list
- Atomic Chat
Download llama.cpp/docs/build-profiling.md from anilsuthar2004/qwen-bot: direct link, hf CLI and curl.
- Browser
- Download file 4.52 kB
-
https://huggingface.co/anilsuthar2004/qwen-bot/resolve/main/llama.cpp/docs/build-profiling.md
- Command line
-
hf download hf://anilsuthar2004/qwen-bot/llama.cpp/docs/build-profiling.md
-
curl -L -o build-profiling.md https://huggingface.co/anilsuthar2004/qwen-bot/resolve/main/llama.cpp/docs/build-profiling.md
Build profiling
This page is a working document for analyzing the current build and try to identify ways to improve the build time.
Requirements
The profiling script requires clang to be used as the compiler tool chain and also requires that ClangBuildAnalyzer is installed.
Mac:
brew install clang-build-analyzer
Linux:
git clone https://github.com/aras-p/ClangBuildAnalyzer.git
cd ClangBuildAnalyzer
cmake -B build -DCMAKE_BUILD_TYPE=Release
cmake --build build -j$(nproc)
sudo cp build/ClangBuildAnalyzer /usr/local/bin/
Windows: install LLVM/clang and Ninja (e.g. via the
LLVM releases page and
winget install Ninja-build.Ninja), then build ClangBuildAnalyzer the same
way as on Linux:
git clone https://github.com/aras-p/ClangBuildAnalyzer.git
cd ClangBuildAnalyzer
cmake -B build -G Ninja -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++ -DCMAKE_BUILD_TYPE=Release
cmake --build build --config Release
Then add ClangBuildAnalyzer\build to PATH.
Usage
Mac/Linux:
$ ./scripts/build-profile.sh
Windows:
> .\scripts\build-profile.ps1
Both accept --full/-Full (include Server, Tools, and Tests) and a jobs
override (-jN / -Jobs N).
Note: on Windows, cmake defaults to the Visual Studio generator, which
ignores CMAKE_C_COMPILER/CMAKE_CXX_COMPILER and silently falls back to
MSVC. build-profile.ps1 passes -G Ninja so clang is actually used, this
is required on ARM64.
Linux (Ubuntu 24.04)
Environment:
- Clang: 18.1.3 (Ubuntu clang version 18.1.3 (1ubuntu1))
- libstdc++: GCC 13.3.0 (Ubuntu 13.3.0-6ubuntu2~24.04.1)
- Target: x86_64-pc-linux-gnu
+------------------------+-----+------------+------------+------------+
| Build | TUs | Frontend | Backend | Total |
+------------------------+-----+------------+------------+------------+
| Minimal, master | 249 | 468.2 s | 270.3 s | 738.5 s |
| Minimal, with PCH | 253 | 177.1 s | 265.8 s | 442.9 s |
| Full, master | 396 | 811.0 s | 692.2 s | 1,503.2 s |
| Full, with PCH | 405 | 380.0 s | 664.7 s | 1,044.7 s |
| Full, with PCH + UB | 264 | 357.7 s | 635.7 s | 993.4 s |
+------------------------+-----+------------+------------+------------+
PCH = precompiled header.
Full = includes building Server, Tools, and Tests.
UB = unity build for models
Note that the number of translation units (TUs) increases when using precompiled headers — each PCH target adds one extra TU for the precompilation step itself.
Mac (Apple M3)
Environment:
- Clang: Apple clang version 17.0.0 (clang-1700.3.19.1)
- libc++: ships with Apple clang 17.0.0 (Xcode toolchain)
- Target: arm64-apple-macosx15.6
+------------------------+-----+------------+------------+------------+
| Build | TUs | Frontend | Backend | Total |
+------------------------+-----+------------+------------+------------+
| Minimal, master | 256 | 154.5 s | 94.8 s | 249.3 s |
| Minimal, with PCH | 261 | 65.9 s | 90.0 s | 155.9 s |
| Full, master | 407 | 265.7 s | 209.7 s | 475.4 s |
| Full, with PCH | 414 | 154.6 s | 197.5 s | 352.1 s |
| Full, with PCH + UB | 274 | 143.0 s | 192.2 s | 335.2 s |
+------------------------+-----+------------+------------+------------+
PCH = precompiled header.
Full = includes building Server, Tools, and Tests.
UB = unity build for models
Windows (ARM64)
Environment:
- Clang: clang version 22.1.8 (LLVM,
C:\Program Files\LLVM) - STL: MSVC STL (Visual Studio 2022 Build Tools 14.44.35207)
- Target: aarch64-pc-windows-msvc
+------------------------+-----+------------+------------+------------+
| Build | TUs | Frontend | Backend | Total |
+------------------------+-----+------------+------------+------------+
| Minimal, master | 249 | 159.4 s | 82.2 s | 241.6 s |
| Full, master | 373 | 337.2 s | 167.4 s | 504.6 s |
| Minimal, with PCH + UB | 113 | 62.3 s | 82.4 s | 144.7 s |
| Full, with PCH + UB | 240 | 233.0 s | 185.1 s | 418.1 s |
+------------------------+-----+------------+------------+------------+
PCH = precompiled header.
Full = includes building Server, Tools, and Tests.
UB = unity build for models