Spaces:

Nithesharaja
/

Software_Bug_Assistant

Sleeping

Enable Local LLM: Pre-download Gemma model in Docker build, add background fallback, support @local , and periodic status refresh

125b392 about 1 month ago

Raw History Blame Contribute Delete

728 Bytes

# Use a Python image with uv pre-installed
FROM ghcr.io/astral-sh/uv:python3.11-bookworm-slim

# Install only minimal runtime system dependencies
RUN apt-get update && apt-get install -y \
    libgomp1 \
    && rm -rf /var/lib/apt/lists/*

WORKDIR /app

ENV UV_LINK_MODE=copy
ENV PYTHONUNBUFFERED=1

ADD . /app

# uv is configured in pyproject.toml to fetch:
# - llama-cpp-python: pre-built CPU binary (no C++ compilation)
# - torch: CPU-only wheel (avoids ~2GB of CUDA packages)
RUN uv sync --no-install-project --no-dev

# Pre-download Gemma-2B GGUF model during build so Local LLM is available immediately at startup
RUN uv run python download_gemma.py

ENV PORT=7860
EXPOSE 7860

CMD ["sh", "-c", "uv run python web_ui.py"]

	# Use a Python image with uv pre-installed
	FROM ghcr.io/astral-sh/uv:python3.11-bookworm-slim

	# Install only minimal runtime system dependencies
	RUN apt-get update && apt-get install -y \
	libgomp1 \
	&& rm -rf /var/lib/apt/lists/*

	WORKDIR /app

	ENV UV_LINK_MODE=copy
	ENV PYTHONUNBUFFERED=1

	ADD . /app

	# uv is configured in pyproject.toml to fetch:
	# - llama-cpp-python: pre-built CPU binary (no C++ compilation)
	# - torch: CPU-only wheel (avoids ~2GB of CUDA packages)
	RUN uv sync --no-install-project --no-dev

	# Pre-download Gemma-2B GGUF model during build so Local LLM is available immediately at startup
	RUN uv run python download_gemma.py

	ENV PORT=7860
	EXPOSE 7860

	CMD ["sh", "-c", "uv run python web_ui.py"]