Model_Zoo / Dockerfile.simple
Prashanth Ramakrishna
Switch to Qwen2.5-72B-Instruct with 4xL4 GPU tensor parallelism and NF4 quantization
82e4ccc
Raw History Blame Contribute Delete
435 Bytes
# Simplified TGI deployment - if main Dockerfile still has issues
FROM ghcr.io/huggingface/text-generation-inference:1.4
# Completely disable JIT compilation to avoid all caching issues
ENV NUMBA_DISABLE_JIT=1
# Basic configuration
ENV PORT=7860
ENV MODEL_ID=mistralai/Mistral-7B-Instruct-v0.2
ENV MAX_INPUT_LENGTH=2048
ENV MAX_TOTAL_TOKENS=4096
ENV TRUST_REMOTE_CODE=true
ENV HUGGINGFACE_HUB_CACHE=/tmp
# Run from /tmp
WORKDIR /tmp