Instructions to use anilsuthar2004/qwen-bot with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use anilsuthar2004/qwen-bot with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf anilsuthar2004/qwen-bot # Run inference directly in the terminal: llama cli -hf anilsuthar2004/qwen-bot
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf anilsuthar2004/qwen-bot # Run inference directly in the terminal: llama cli -hf anilsuthar2004/qwen-bot
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf anilsuthar2004/qwen-bot # Run inference directly in the terminal: ./build/bin/llama cli -hf anilsuthar2004/qwen-bot
Use Docker
docker model run hf.co/anilsuthar2004/qwen-bot
- Llama new
- LM Studio
- Jan
- Ollama
How to use anilsuthar2004/qwen-bot with Ollama:
ollama run hf.co/anilsuthar2004/qwen-bot
- Unsloth Desktop
- Docker Model Runner
How to use anilsuthar2004/qwen-bot with Docker Model Runner:
docker model run hf.co/anilsuthar2004/qwen-bot
- Lemonade
How to use anilsuthar2004/qwen-bot with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull anilsuthar2004/qwen-bot
Run and chat with the model
lemonade run user.qwen-bot-{{QUANT_TAG}}List all available models
lemonade list
- Atomic Chat
File size: 4,518 Bytes
ed107dd | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 | ## Build profiling
This page is a working document for analyzing the current build and try to
identify ways to improve the build time.
### Requirements
The profiling script requires clang to be used as the compiler tool chain and
also requires that ClangBuildAnalyzer is installed.
Mac:
```console
brew install clang-build-analyzer
```
Linux:
```console
git clone https://github.com/aras-p/ClangBuildAnalyzer.git
cd ClangBuildAnalyzer
cmake -B build -DCMAKE_BUILD_TYPE=Release
cmake --build build -j$(nproc)
sudo cp build/ClangBuildAnalyzer /usr/local/bin/
```
Windows: install LLVM/clang and Ninja (e.g. via the
[LLVM releases page](https://github.com/llvm/llvm-project/releases) and
`winget install Ninja-build.Ninja`), then build ClangBuildAnalyzer the same
way as on Linux:
```console
git clone https://github.com/aras-p/ClangBuildAnalyzer.git
cd ClangBuildAnalyzer
cmake -B build -G Ninja -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++ -DCMAKE_BUILD_TYPE=Release
cmake --build build --config Release
```
Then add `ClangBuildAnalyzer\build` to `PATH`.
### Usage
Mac/Linux:
```console
$ ./scripts/build-profile.sh
```
Windows:
```console
> .\scripts\build-profile.ps1
```
Both accept `--full`/`-Full` (include Server, Tools, and Tests) and a jobs
override (`-jN` / `-Jobs N`).
Note: on Windows, `cmake` defaults to the Visual Studio generator, which
ignores `CMAKE_C_COMPILER`/`CMAKE_CXX_COMPILER` and silently falls back to
MSVC. `build-profile.ps1` passes `-G Ninja` so clang is actually used, this
is required on ARM64.
### Linux (Ubuntu 24.04)
Environment:
- Clang: 18.1.3 (Ubuntu clang version 18.1.3 (1ubuntu1))
- libstdc++: GCC 13.3.0 (Ubuntu 13.3.0-6ubuntu2~24.04.1)
- Target: x86_64-pc-linux-gnu
```console
+------------------------+-----+------------+------------+------------+
| Build | TUs | Frontend | Backend | Total |
+------------------------+-----+------------+------------+------------+
| Minimal, master | 249 | 468.2 s | 270.3 s | 738.5 s |
| Minimal, with PCH | 253 | 177.1 s | 265.8 s | 442.9 s |
| Full, master | 396 | 811.0 s | 692.2 s | 1,503.2 s |
| Full, with PCH | 405 | 380.0 s | 664.7 s | 1,044.7 s |
| Full, with PCH + UB | 264 | 357.7 s | 635.7 s | 993.4 s |
+------------------------+-----+------------+------------+------------+
PCH = precompiled header.
Full = includes building Server, Tools, and Tests.
UB = unity build for models
```
Note that the number of translation units (TUs) increases when using precompiled
headers — each PCH target adds one extra TU for the precompilation step itself.
### Mac (Apple M3)
Environment:
- Clang: Apple clang version 17.0.0 (clang-1700.3.19.1)
- libc++: ships with Apple clang 17.0.0 (Xcode toolchain)
- Target: arm64-apple-macosx15.6
```console
+------------------------+-----+------------+------------+------------+
| Build | TUs | Frontend | Backend | Total |
+------------------------+-----+------------+------------+------------+
| Minimal, master | 256 | 154.5 s | 94.8 s | 249.3 s |
| Minimal, with PCH | 261 | 65.9 s | 90.0 s | 155.9 s |
| Full, master | 407 | 265.7 s | 209.7 s | 475.4 s |
| Full, with PCH | 414 | 154.6 s | 197.5 s | 352.1 s |
| Full, with PCH + UB | 274 | 143.0 s | 192.2 s | 335.2 s |
+------------------------+-----+------------+------------+------------+
PCH = precompiled header.
Full = includes building Server, Tools, and Tests.
UB = unity build for models
```
### Windows (ARM64)
Environment:
- Clang: clang version 22.1.8 (LLVM, `C:\Program Files\LLVM`)
- STL: MSVC STL (Visual Studio 2022 Build Tools 14.44.35207)
- Target: aarch64-pc-windows-msvc
```console
+------------------------+-----+------------+------------+------------+
| Build | TUs | Frontend | Backend | Total |
+------------------------+-----+------------+------------+------------+
| Minimal, master | 249 | 159.4 s | 82.2 s | 241.6 s |
| Full, master | 373 | 337.2 s | 167.4 s | 504.6 s |
| Minimal, with PCH + UB | 113 | 62.3 s | 82.4 s | 144.7 s |
| Full, with PCH + UB | 240 | 233.0 s | 185.1 s | 418.1 s |
+------------------------+-----+------------+------------+------------+
PCH = precompiled header.
Full = includes building Server, Tools, and Tests.
UB = unity build for models
```
|