journal de demarrage

etat/6tqc2otcxvsry3.log

@@ -1,4 +1,4 @@
-=== bootstrap v82-humming-extras-cu13 | 15:55:51 UTC ===
+=== bootstrap v82-humming-extras-cu13 | 15:56:19 UTC ===
 [VL] 15:45:50 bootstrap v82-humming-extras-cu13
 [VL] 15:45:51 pilote 550.54.15, CUDA runtime 13.0
 [VL] 15:45:51 compat CUDA 13 active (LD_LIBRARY_PATH)
@@ -18,7 +18,7 @@
 [VL] 15:46:49 journal distant : https://huggingface.co/patdev/k3-a40-bootstrap/resolve/main/etat/6tqc2otcxvsry3.log
 
 === nvidia-smi ===
-20213 MiB, 81920 MiB
+75441 MiB, 81920 MiB
 
 === vllm.log (fin) ===
 (APIServer pid=872) INFO 08-23 15:47:06 [api_utils.py:345] 
@@ -147,3 +147,18 @@
 (EngineCore pid=1604) INFO 08-23 15:53:52 [ssu_dispatch.py:310] Using flashinfer Mamba SSU backend.
 (EngineCore pid=1604) INFO 08-23 15:53:52 [flashinfer.py:824] FlashInfer resolved query dtypes: prefill=torch.bfloat16, decode=torch.bfloat16, decode_backend=flashinfer-native, kv_cache_dtype=torch.float8_e4m3fn, arch=sm80
 (EngineCore pid=1604) INFO 08-23 15:53:52 [gpu_model_runner.py:6681] Profiling CUDA graph memory: PIECEWISE=19 (largest=128), FULL=11 (largest=64)
+(EngineCore pid=1604) INFO 08-23 15:55:59 [gpu_model_runner.py:6806] Estimated CUDA graph memory: 0.77 GiB total
+(EngineCore pid=1604) INFO 08-23 15:55:59 [gpu_worker.py:563] Available KV cache memory: 53.81 GiB
+(EngineCore pid=1604) INFO 08-23 15:55:59 [gpu_worker.py:578] CUDA graph memory profiling is enabled (default since v0.21.0). The current --gpu-memory-utilization=0.9300 is equivalent to --gpu-memory-utilization=0.9203 without CUDA graph memory profiling. To maintain the same effective KV cache size as before, increase --gpu-memory-utilization to 0.9397. To disable, set VLLM_MEMORY_PROFILER_ESTIMATE_CUDAGRAPHS=0.
+(EngineCore pid=1604) WARNING 08-23 15:55:59 [kv_cache_utils.py:1261] Add 1 padding layers, may waste at most 4.35% KV cache memory
+(EngineCore pid=1604) INFO 08-23 15:55:59 [kv_cache_utils.py:2235] GPU KV cache size: 18,495,541 tokens
+(EngineCore pid=1604) INFO 08-23 15:55:59 [kv_cache_utils.py:2236] Maximum concurrency for 1,048,576 tokens per request: 17.64x
+(EngineCore pid=1604) INFO 08-23 15:56:00 [gpu_model_runner.py:6867] Rank 0: Torch profiler disabled for CUDA graph capture
+(EngineCore pid=1604) 
+(EngineCore pid=1604) 
+(EngineCore pid=1604) INFO 08-23 15:56:07 [gpu_model_runner.py:6913] Graph capturing finished in 7 secs, took 0.76 GiB
+(EngineCore pid=1604) INFO 08-23 15:56:07 [gpu_worker.py:726] CUDA graph pool memory: 0.76 GiB (actual), 0.77 GiB (estimated), difference: 0.01 GiB (1.8%).
+(EngineCore pid=1604) INFO 08-23 15:56:07 [gpu_worker.py:789] Free memory on device (78.72/79.14 GiB) on startup. Desired GPU memory utilization is (0.93, 73.6 GiB). Actual usage is 18.65 GiB for consumed memory (weights + non-torch), 1.14 GiB for peak activation, and 0.76 GiB for CUDAGraph memory. Replace gpu_memory_utilization config with `--kv-cache-memory=56806828626` (52.91 GiB) to fit into requested memory, or `--kv-cache-memory=62309380608` (58.03 GiB) to fully utilize gpu memory. Current kv cache memory in use is 53.81 GiB.
+(EngineCore pid=1604) INFO 08-23 15:56:10 [jit_monitor.py:79] Kernel JIT monitor activated; monitored JIT compilations during inference will use mode=warn.
+(EngineCore pid=1604) INFO 08-23 15:56:11 [core.py:348] init engine (profile, create kv cache, warmup model) took 273.96 s (compilation: 23.71 s)
+(EngineCore pid=1604) [fastokens] patch_transformers: successfully patched transformers v5.15.1