Instructions to use anilsuthar2004/qwen-bot with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use anilsuthar2004/qwen-bot with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf anilsuthar2004/qwen-bot # Run inference directly in the terminal: llama cli -hf anilsuthar2004/qwen-bot
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf anilsuthar2004/qwen-bot # Run inference directly in the terminal: llama cli -hf anilsuthar2004/qwen-bot
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf anilsuthar2004/qwen-bot # Run inference directly in the terminal: ./build/bin/llama cli -hf anilsuthar2004/qwen-bot
Use Docker
docker model run hf.co/anilsuthar2004/qwen-bot
- Llama new
- LM Studio
- Jan
- Ollama
How to use anilsuthar2004/qwen-bot with Ollama:
ollama run hf.co/anilsuthar2004/qwen-bot
- Unsloth Desktop
- Docker Model Runner
How to use anilsuthar2004/qwen-bot with Docker Model Runner:
docker model run hf.co/anilsuthar2004/qwen-bot
- Lemonade
How to use anilsuthar2004/qwen-bot with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull anilsuthar2004/qwen-bot
Run and chat with the model
lemonade run user.qwen-bot-{{QUANT_TAG}}List all available models
lemonade list
- Atomic Chat
File size: 7,716 Bytes
572ef52 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 | #!/bin/bash
# Run all pre-release checks and determine the release version.
#
# Usage: make-release-checks.sh [--dry-run]
# --dry-run: warn on failures instead of aborting
#
# Env (when running in GitHub Actions):
# GH_TOKEN, GITHUB_REPOSITORY, GITHUB_OUTPUT
# RELEASE_BRANCH: when set, HEAD must belong to origin/RELEASE_BRANCH and must
# not be older than 3 days from the branch HEAD (skipped when unset)
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
REPO_ROOT="$(cd "$SCRIPT_DIR/.." && pwd)"
DRY_RUN=false
CHECKS_PASSED=true
for arg in "$@"; do
case "$arg" in
--dry-run) DRY_RUN=true ;;
*) echo "Unknown argument: $arg"; exit 1 ;;
esac
done
MAJOR=$(grep "set(LLAMA_VERSION_MAJOR" "$REPO_ROOT/CMakeLists.txt" | sed 's/.*MAJOR \([0-9]*\).*/\1/')
MINOR=$(grep "set(LLAMA_VERSION_MINOR" "$REPO_ROOT/CMakeLists.txt" | sed 's/.*MINOR \([0-9]*\).*/\1/')
PATCH=$(grep "set(LLAMA_VERSION_PATCH" "$REPO_ROOT/CMakeLists.txt" | sed 's/.*PATCH \([0-9]*\).*/\1/')
VERSION="v${MAJOR}.${MINOR}.${PATCH}"
echo "Determined version: ${VERSION}"
if [[ -n "${GITHUB_OUTPUT:-}" ]]; then
echo "version=${VERSION}" >> "$GITHUB_OUTPUT"
fi
SHA=$(git rev-parse HEAD)
echo "Checking that commit ${SHA} belongs to the release branch..."
if [[ -z "${RELEASE_BRANCH:-}" ]]; then
echo "Warning: RELEASE_BRANCH not set - skipping commit check (local run)"
else
TIP="origin/${RELEASE_BRANCH}"
COMMIT_ERR=""
if ! git rev-parse --verify "${TIP}" >/dev/null 2>&1; then
COMMIT_ERR="branch ${RELEASE_BRANCH} not found on remote"
elif ! git merge-base --is-ancestor "${SHA}" "${TIP}"; then
COMMIT_ERR="commit ${SHA} is not part of branch ${RELEASE_BRANCH}"
else
COMMIT_TS=$(git show -s --format=%ct "${SHA}")
TIP_TS=$(git show -s --format=%ct "${TIP}")
AGE_DAYS=$(( (TIP_TS - COMMIT_TS) / 86400 ))
if (( TIP_TS - COMMIT_TS > 3 * 86400 )); then
COMMIT_ERR="commit ${SHA} is ${AGE_DAYS} day(s) older than the HEAD of ${RELEASE_BRANCH} (max: 3)"
fi
fi
if [[ -n "${COMMIT_ERR}" ]]; then
if [[ "$DRY_RUN" == "true" ]]; then
echo "Warning: ${COMMIT_ERR} (dry run, continuing)."
CHECKS_PASSED=false
else
echo "Error: ${COMMIT_ERR}"
exit 1
fi
else
echo "Commit ${SHA} is on branch ${RELEASE_BRANCH} and within 3 days of its HEAD - OK"
fi
fi
echo "Checking that tag ${VERSION} does not already exist..."
if git ls-remote --tags origin "${VERSION}" | grep -q "${VERSION}"; then
echo "Error: tag ${VERSION} already exists on remote"
exit 1
fi
echo "Tag ${VERSION} does not exist on remote - OK"
echo "Checking release.yml status for commit ${SHA}..."
if [[ -z "${GITHUB_REPOSITORY:-}" ]]; then
echo "Warning: GITHUB_REPOSITORY not set - skipping CI check (local run)"
else
RUNS=$(gh api "repos/${GITHUB_REPOSITORY}/actions/workflows/release.yml/runs?per_page=100" \
--jq "[.workflow_runs[] | select(.head_sha == \"${SHA}\" and .conclusion == \"success\")] | length")
if [[ "$RUNS" -eq 0 ]]; then
if [[ "$DRY_RUN" == "true" ]]; then
echo "Warning: no successful release.yml run found for HEAD (${SHA}) (dry run, continuing)."
CHECKS_PASSED=false
else
echo "Error: no successful release.yml run found for HEAD (${SHA})"
echo "The nightly build must complete successfully before making a release."
exit 1
fi
else
echo "Found successful release.yml run for HEAD."
fi
fi
MAJOR=$(grep "set(GGML_VERSION_MAJOR" "$REPO_ROOT/ggml/CMakeLists.txt" | sed 's/.*MAJOR \([0-9]*\).*/\1/')
MINOR=$(grep "set(GGML_VERSION_MINOR" "$REPO_ROOT/ggml/CMakeLists.txt" | sed 's/.*MINOR \([0-9]*\).*/\1/')
PATCH=$(grep "set(GGML_VERSION_PATCH" "$REPO_ROOT/ggml/CMakeLists.txt" | sed 's/.*PATCH \([0-9]*\).*/\1/')
GGML_VERSION="v${MAJOR}.${MINOR}.${PATCH}"
echo "Local ggml version: ${GGML_VERSION}"
if ! git clone --depth 1 --branch "${GGML_VERSION}" https://github.com/ggml-org/ggml.git upstream-ggml 2>/dev/null; then
echo "Warning: tag ${GGML_VERSION} not found in upstream ggml - skipping comparison"
else
echo "Comparing local ggml/ src and include with upstream ${GGML_VERSION}..."
DIFF=$(diff -rq "$REPO_ROOT/ggml/src" upstream-ggml/src 2>&1 || true)
DIFF+=$(diff -rq "$REPO_ROOT/ggml/include" upstream-ggml/include 2>&1 || true)
DIFF+=$(diff "$REPO_ROOT/ggml/CMakeLists.txt" upstream-ggml/CMakeLists.txt 2>&1 || true)
rm -rf upstream-ggml
if [[ -n "$DIFF" ]]; then
echo "local ggml/ differs from upstream ${GGML_VERSION}:"
echo "$DIFF"
if [[ "$DRY_RUN" == "true" ]]; then
echo "Warning: would abort release due to ggml mismatch (dry run, continuing)."
CHECKS_PASSED=false
else
echo "Error: ggml must match upstream before making a release."
exit 1
fi
else
echo "local ggml/ matches upstream ${GGML_VERSION}"
fi
fi
echo "Checking container images for commit ${SHA}..."
NIGHTLY_TAG="$(git tag --points-at "${SHA}" | grep -E '(^|-)b[0-9]+(-[0-9a-f]{7})?$' | head -n 1 || true)"
if [[ -z "${NIGHTLY_TAG}" ]]; then
echo "Warning: no nightly tag points at ${SHA} - skipping container image check"
elif [[ -z "${GITHUB_REPOSITORY:-}" ]]; then
echo "Warning: GITHUB_REPOSITORY not set - skipping container image check (local run)"
else
CONTAINER_REPO="${GITHUB_REPOSITORY,,}" # lower-case owner/repo for ghcr.io
GHCR_TOKEN="$(curl -fsSL \
"https://ghcr.io/token?scope=repository:${CONTAINER_REPO}:pull&service=ghcr.io" \
| grep -oP '"token"\s*:\s*"\K[^"]+')"
VARIANTS=("" "-cuda" "-cuda13" "-vulkan" "-rocm" "-intel" "-musa" "-openvino")
TYPES=("full" "light" "server")
CONTAINER_ERR=""
for type in "${TYPES[@]}"; do
for variant in "${VARIANTS[@]}"; do
tag="${type}${variant}-${NIGHTLY_TAG}"
STATUS="$(curl -s -o /dev/null -w "%{http_code}" \
-H "Authorization: Bearer ${GHCR_TOKEN}" \
-H "Accept: application/vnd.oci.image.index.v1+json,application/vnd.docker.distribution.manifest.list.v2+json" \
"https://ghcr.io/v2/${CONTAINER_REPO}/manifests/${tag}")"
if [[ "${STATUS}" == "200" ]]; then
echo " ${tag} - OK"
else
echo " ${tag} - MISSING"
CONTAINER_ERR+=" ${tag}"
fi
done
done
if [[ -n "${CONTAINER_ERR}" ]]; then
if [[ "$DRY_RUN" == "true" ]]; then
echo "Warning: missing container images for ${NIGHTLY_TAG}:${CONTAINER_ERR} (dry run, continuing)."
CHECKS_PASSED=false
else
echo "Error: missing container images for ${NIGHTLY_TAG}:${CONTAINER_ERR}"
echo "The Docker workflow must complete successfully before making a release."
exit 1
fi
else
echo "All container images found for ${NIGHTLY_TAG} - OK"
fi
fi
echo "Checking API/ABI compatibility..."
set +e
bash "$SCRIPT_DIR/check-release-apiabi.sh"
APIABI_RESULT=$?
set -e
if [[ $APIABI_RESULT -ne 0 ]]; then
if [[ "$DRY_RUN" == "true" ]]; then
echo "Warning: API/ABI check found backwards-incompatible changes (dry run, continuing)."
CHECKS_PASSED=false
else
echo "Error: API/ABI check found backwards-incompatible changes."
exit 1
fi
else
echo "API/ABI compatibility check passed - OK"
fi
if [[ -n "${GITHUB_OUTPUT:-}" ]]; then
echo "checks_passed=${CHECKS_PASSED}" >> "$GITHUB_OUTPUT"
fi
|