Spaces:
Runtime error
Runtime error
Prashanth Ramakrishna
Switch to Qwen2.5-72B-Instruct with 4xL4 GPU tensor parallelism and NF4 quantization
82e4ccc Download Dockerfile.simple from Praz314159/Model_Zoo: direct link, hf CLI and curl.
- Browser
- Download file 435 Bytes
-
https://huggingface.co/spaces/Praz314159/Model_Zoo/resolve/main/Dockerfile.simple
- Command line
-
hf download hf://spaces/Praz314159/Model_Zoo/Dockerfile.simple
-
curl -L -o Dockerfile.simple https://huggingface.co/spaces/Praz314159/Model_Zoo/resolve/main/Dockerfile.simple
435 Bytes
| # Simplified TGI deployment - if main Dockerfile still has issues | |
| FROM ghcr.io/huggingface/text-generation-inference:1.4 | |
| # Completely disable JIT compilation to avoid all caching issues | |
| ENV NUMBA_DISABLE_JIT=1 | |
| # Basic configuration | |
| ENV PORT=7860 | |
| ENV MODEL_ID=mistralai/Mistral-7B-Instruct-v0.2 | |
| ENV MAX_INPUT_LENGTH=2048 | |
| ENV MAX_TOTAL_TOKENS=4096 | |
| ENV TRUST_REMOTE_CODE=true | |
| ENV HUGGINGFACE_HUB_CACHE=/tmp | |
| # Run from /tmp | |
| WORKDIR /tmp |