runtime error
Exit code: 134. Reason: ========== == CUDA == ========== CUDA Version 12.6.0 Container image Copyright (c) 2016-2023, NVIDIA CORPORATION & AFFILIATES. All rights reserved. This container image and its contents are governed by the NVIDIA Deep Learning Container License. By pulling and using the container, you accept the terms and conditions of this license: https://developer.nvidia.com/ngc/nvidia-deep-learning-container-license A copy of this license is made available in this container at /NGC-DL-CONTAINER-LICENSE for your convenience. Downloading model. mkdir: cannot create directory '/data': Permission denied /data/models/mixtral-8x7b-instruct-v0.1.Q5_K_M.gguf: No such file or directory Model downloaded. INFO [ main] build info | tid="140697139920896" timestamp=1784949857 build=3465 commit="41cd47caa" INFO [ main] system info | tid="140697139920896" timestamp=1784949857 n_threads=48 n_threads_batch=-1 total_threads=96 system_info="AVX = 1 | AVX_VNNI = 0 | AVX2 = 1 | AVX512 = 1 | AVX512_VBMI = 1 | AVX512_VNNI = 1 | AVX512_BF16 = 0 | FMA = 1 | NEON = 0 | SVE = 0 | ARM_FMA = 0 | F16C = 1 | FP16_VA = 0 | WASM_SIMD = 0 | BLAS = 1 | SSE3 = 1 | SSSE3 = 1 | VSX = 0 | MATMUL_INT8 = 0 | LLAMAFILE = 1 | " gguf_init_from_file: failed to open '/data/models/mixtral-8x7b-instruct-v0.1.Q5_K_M.gguf': 'No such file or directory' llama_model_load: error loading model: llama_model_loader: failed to load model from /data/models/mixtral-8x7b-instruct-v0.1.Q5_K_M.gguf llama_load_model_from_file: failed to load model llama_init_from_gpt_params: error: failed to load model '/data/models/mixtral-8x7b-instruct-v0.1.Q5_K_M.gguf' ERR [ load_model] unable to load model | tid="140697139920896" timestamp=1784949857 model="/data/models/mixtral-8x7b-instruct-v0.1.Q5_K_M.gguf" free(): invalid pointer ./run.sh: line 31: 74 Aborted (core dumped) ./server -m "$MODEL_PATH" -c $CONTEXT --port $PORT --host 0.0.0.0 --n-gpu-layers $N_GPU_LAYERS --path "/app/public" -fa
Container logs:
Fetching error logs...