Instructions to use aelgendy/QModel with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use aelgendy/QModel with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf aelgendy/QModel:Q8_0 # Run inference directly in the terminal: llama cli -hf aelgendy/QModel:Q8_0
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf aelgendy/QModel:Q8_0 # Run inference directly in the terminal: llama cli -hf aelgendy/QModel:Q8_0
Use pre-built binary
# Download pre-built binary from: # https://github.com/ggerganov/llama.cpp/releases # Start a local OpenAI-compatible server with a web UI: ./llama-server -hf aelgendy/QModel:Q8_0 # Run inference directly in the terminal: ./llama-cli -hf aelgendy/QModel:Q8_0
Build from source code
git clone https://github.com/ggerganov/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-server llama-cli # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama-server -hf aelgendy/QModel:Q8_0 # Run inference directly in the terminal: ./build/bin/llama-cli -hf aelgendy/QModel:Q8_0
Use Docker
docker model run hf.co/aelgendy/QModel:Q8_0
- LM Studio
- Jan
- Ollama
How to use aelgendy/QModel with Ollama:
ollama run hf.co/aelgendy/QModel:Q8_0
- Unsloth Desktop
- Pi
How to use aelgendy/QModel with Pi:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf aelgendy/QModel:Q8_0
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "llama-cpp": { "baseUrl": "http://localhost:8080/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "aelgendy/QModel:Q8_0" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Docker Model Runner
How to use aelgendy/QModel with Docker Model Runner:
docker model run hf.co/aelgendy/QModel:Q8_0
- Lemonade
How to use aelgendy/QModel with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull aelgendy/QModel:Q8_0
Run and chat with the model
lemonade run user.QModel-Q8_0
List all available models
lemonade list
- Hermes Agent
How to use aelgendy/QModel with Hermes Agent:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf aelgendy/QModel:Q8_0
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:8080/v1 hermes config set model.default aelgendy/QModel:Q8_0
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use aelgendy/QModel with OpenClaw:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf aelgendy/QModel:Q8_0
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:8080/v1 \ --custom-model-id "aelgendy/QModel:Q8_0" \ --custom-provider-id llama-cpp \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
File size: 2,634 Bytes
c038254 15f1210 20edea9 15f1210 20edea9 e7c1485 15f1210 e7c1485 15f1210 e7c1485 15f1210 e7c1485 15f1210 c038254 15f1210 c038254 15f1210 c038254 15f1210 c038254 15f1210 e7c1485 20edea9 e7c1485 15f1210 20edea9 e7c1485 15f1210 20edea9 e7c1485 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 | # name: logicsoft
# QModel 7 - Islamic RAG API
# =============================
# Dockerfile for QModel API
# Supports both Ollama and HuggingFace backends via .env configuration
#
# Build: docker build -t qmodel .
# Run: docker run -p 8000:8000 --env-file .env qmodel
# ββ Builder stage: compile deps, keep toolchain out of the final image ββ
FROM python:3.11-slim AS builder
ENV PYTHONDONTWRITEBYTECODE=1 \
PYTHONUNBUFFERED=1 \
PIP_NO_CACHE_DIR=1
# - build-essential/cmake: for compiling llama-cpp-python and other native deps
# - libopenblas-dev: for numerical operations (FAISS, numpy)
# - libomp-dev: for OpenMP (FAISS parallelization)
RUN apt-get update && apt-get install -y --no-install-recommends \
build-essential \
cmake \
libopenblas-dev \
libomp-dev \
&& rm -rf /var/lib/apt/lists/*
RUN python -m venv /opt/venv
ENV PATH="/opt/venv/bin:$PATH"
COPY requirements.txt .
# faiss-cpu means this deployment doesn't use GPU acceleration β install the
# CPU-only torch wheel first so requirements.txt doesn't pull the default
# CUDA-enabled build (several GB of unused NVIDIA libraries).
RUN pip install --default-timeout=1000 --retries 10 --no-cache-dir \
--extra-index-url https://download.pytorch.org/whl/cpu torch \
&& pip install --default-timeout=1000 --retries 10 --no-cache-dir -r requirements.txt
# ββ Final stage: slim runtime image βββββββββββββββββββββββββββββββββββββ
FROM python:3.11-slim
LABEL name="Logicsoft"
LABEL author="Abdullah Elgendy"
LABEL maintainer="Abdullah Elgendy"
LABEL description="QModel v8 - Quran, Hadith, Tafsir and Fiqh RAG API"
LABEL version="8.0.0"
ENV PYTHONDONTWRITEBYTECODE=1 \
PYTHONUNBUFFERED=1 \
PATH="/opt/venv/bin:$PATH"
WORKDIR /app
# Runtime-only system deps (no compiler toolchain here)
RUN apt-get update && apt-get install -y --no-install-recommends \
libopenblas0-pthread \
libomp5 \
libgomp1 \
curl \
&& rm -rf /var/lib/apt/lists/*
COPY --from=builder /opt/venv /opt/venv
# Copy application code
COPY . .
# Run as non-root
RUN useradd --create-home --uid 1000 appuser \
&& chown -R appuser:appuser /app
USER appuser
# Expose port for API
EXPOSE 8000
# Health check β /health returns 503 until the RAG pipeline finishes loading
HEALTHCHECK --interval=30s --timeout=10s --start-period=60s --retries=3 \
CMD curl -f http://localhost:8000/health || exit 1
# Start application
# Configure via .env: LLM_BACKEND=ollama or LLM_BACKEND=hf
CMD ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8000"]
|