Buckets:

Mercity
/

Desicion-Model-Custom

Files
xet
Mercity/Desicion-Model-Custom / run3_2b /logs /v2b_driver.log
Pranav2748
11 days ago
download
raw
281 kB
{"time": "2026-10-31T23:59:53Z", "event": "start", "stage": "2b_probe_qwen"}
(MainProcess pid=140642) INFO 11-01 00:00:05 [api_utils.py:286] non-default args: {'dtype': 'bfloat16', 'seed': 17, 'max_model_len': 32768, 'enable_prefix_caching': True, 'max_num_batched_tokens': 32768, 'max_num_seqs': 256, 'max_logprobs': 26, 'logprobs_mode': 'processed_logprobs', 'model': 'Qwen/Qwen3.5-2B'}
(MainProcess pid=140642) INFO 11-01 00:00:06 [model.py:733] Resolved architecture: Qwen3_5ForConditionalGeneration
(MainProcess pid=140642) INFO 11-01 00:00:06 [model.py:2152] Using max model len 32768

Fetching 12 files:   0%|          | 0/12 [00:00<?, ?it/s]
Fetching 12 files: 100%|██████████| 12/12 [00:00<00:00, 844.45it/s]
(MainProcess pid=140642) INFO 11-01 00:00:08 [scheduler.py:307] Chunked prefill is enabled with max_num_batched_tokens=32768.
(MainProcess pid=140642) INFO 11-01 00:00:08 [config.py:688] Mamba cache mode is set to 'align' for Qwen3_5ForConditionalGeneration by default when prefix caching is enabled

Parse safetensors files:   0%|          | 0/1 [00:00<?, ?it/s]
Parse safetensors files: 100%|██████████| 1/1 [00:00<00:00,  4.88it/s]
Parse safetensors files: 100%|██████████| 1/1 [00:00<00:00,  4.87it/s]
(MainProcess pid=140642) INFO 11-01 00:00:08 [kernel.py:467] Final IR op priority after setting platform defaults: IrOpPriorityConfig(rms_norm=['native'], fused_add_rms_norm=['native'], gelu_and_mul_sparse=['triton', 'native'])
(MainProcess pid=140642) WARNING 11-01 00:00:20 [system_utils.py:153] We must use the `spawn` multiprocessing start method. Overriding VLLM_WORKER_MULTIPROC_METHOD to 'spawn'. See https://docs.vllm.ai/en/latest/usage/troubleshooting.html#python-multiprocessing for more information. Reasons: CUDA is initialized
(EngineCore pid=141070) INFO 11-01 00:00:32 [core.py:133] Initializing a V1 LLM engine (v0.32.0) with config: model='Qwen/Qwen3.5-2B', speculative_config=None, tokenizer='Qwen/Qwen3.5-2B', skip_tokenizer_init=False, tokenizer_mode=auto, revision=main, tokenizer_revision=main, trust_remote_code=False, dtype=torch.bfloat16, max_seq_len=32768, download_dir=None, load_format=auto, tensor_parallel_size=1, pipeline_parallel_size=1, data_parallel_size=1, decode_context_parallel_size=1, dcp_comm_backend=ag_rs, disable_custom_all_reduce=False, quantization=None, quantization_config=None, enforce_eager=False, aux_output_config=AuxOutputConfig(enable_return_routed_experts=False, max_bytes=None), kv_cache_dtype=auto, device_config=cuda, structured_outputs_config=StructuredOutputsConfig(backend='auto', disable_any_whitespace=False, disable_additional_properties=False, reasoning_parser='', reasoning_parser_plugin='', enable_in_reasoning=False), observability_config=ObservabilityConfig(show_hidden_metrics_for_version=None, otlp_traces_endpoint=None, collect_detailed_traces=None, per_request_spec_decode_metrics='none', kv_cache_metrics=False, kv_cache_metrics_sample=0.01, custom_histogram_buckets=None, cudagraph_metrics=False, enable_layerwise_nvtx_tracing=False, enable_mfu_metrics=False, enable_mm_processor_stats=False, enable_logging_iteration_details=False, jit_monitor_mode='warn', jit_monitor_verbose=False), seed=17, served_model_name=Qwen/Qwen3.5-2B, enable_prefix_caching=True, enable_chunked_prefill=True, pooler_config=None, compilation_config={'mode': <CompilationMode.VLLM_COMPILE: 3>, 'debug_dump_path': None, 'cache_dir': '', 'compile_cache_save_format': 'binary', 'backend': 'inductor', 'custom_ops': ['none'], 'ir_enable_torch_wrap': True, 'splitting_ops': ['vllm::unified_attention_with_output', 'vllm::unified_mla_attention_with_output', 'vllm::mamba_mixer2', 'vllm::mamba_mixer', 'vllm::short_conv', 'vllm::qwen4_exp_ple_short_conv', 'vllm::qwen4_exp_qsa_with_output', 'vllm::qwen4_exp_qsa_prepare_with_output', 'vllm::linear_attention', 'vllm::qwen_gdn_attention_core', 'vllm::qwen_gdn_attention_core_fused_norm_packed', 'vllm::gdn_attention_core_xpu', 'vllm::olmo_hybrid_gdn_full_forward', 'vllm::sparse_attn_indexer', 'vllm::rocm_aiter_sparse_attn_indexer', 'vllm::deepseek_v4_attention', 'vllm::hpc_rope_norm_forward', 'vllm::unified_kv_cache_update', 'vllm::unified_mla_kv_cache_update'], 'compile_mm_encoder': False, 'cudagraph_mm_encoder': False, 'encoder_cudagraph_token_budgets': [], 'encoder_cudagraph_max_vision_items_per_batch': 0, 'encoder_cudagraph_max_frames_per_batch': None, 'cudagraph_decoder_replay': True, 'decoder_replay_cudagraph_capture_sizes': [], 'decoder_replay_trim_threshold': 768, 'compile_sizes': [], 'compile_ranges_endpoints': [32768], 'inductor_compile_config': {'enable_auto_functionalized_v2': False, 'combo_kernels': True, 'benchmark_combo_kernel': True}, 'inductor_passes': {}, 'cudagraph_mode': <CUDAGraphMode.FULL_AND_PIECEWISE: (2, 1)>, 'cudagraph_num_of_warmups': 1, 'cudagraph_capture_sizes': [1, 2, 4, 8, 16, 24, 32, 40, 48, 56, 64, 72, 80, 88, 96, 104, 112, 120, 128, 136, 144, 152, 160, 168, 176, 184, 192, 200, 208, 216, 224, 232, 240, 248, 256, 272, 288, 304, 320, 336, 352, 368, 384, 400, 416, 432, 448, 464, 480, 496, 512], 'cudagraph_copy_inputs': False, 'cudagraph_specialize_lora': True, 'use_inductor_graph_partition': False, 'pass_config': {'fuse_norm_quant': False, 'fuse_act_quant': False, 'fuse_attn_quant': False, 'fuse_allreduce_rms': False, 'enable_qk_norm_rope_fusion': False, 'fuse_rope_kvcache_cat_mla': False, 'fuse_act_padding': False, 'fuse_qk_norm_rope_kvcache': False}, 'max_cudagraph_capture_size': 512, 'dynamic_shapes_config': {'type': <DynamicShapesType.BACKED: 'backed'>, 'evaluate_guards': False, 'assume_32_bit_indexing': False}, 'local_cache_dir': None, 'fast_moe_cold_start': False, 'static_all_moe_layers': []}, kernel_config=KernelConfig(ir_op_priority=IrOpPriorityConfig(rms_norm=['native'], fused_add_rms_norm=['native'], gelu_and_mul_sparse=['triton', 'native']), enable_flashinfer_autotune=True, enable_rocm_segmented_attn_autotune=False, enable_cutedsl_warmup=True, enable_jit_warmup=True, moe_backend='auto', sparse_indexer_topk_backend='auto', linear_backend='auto', linear_backend_per_quant=None)
(EngineCore pid=141070) INFO 11-01 00:00:34 [parallel_state.py:1906] world_size=1 rank=0 local_rank=0 distributed_init_method=file:///tmp/vllm_dist_0a381f21c4b443a3b856e7bf920500a5 backend=nccl
(EngineCore pid=141070) INFO 11-01 00:00:34 [parallel_state.py:2350] rank 0 in world size 1 is assigned as DP rank 0, PP rank 0, PCP rank 0, TP rank 0, ETP rank 0, EP rank N/A, EPLB rank N/A
(EngineCore pid=141070) INFO 11-01 00:00:34 [gpu_worker.py:518] Using V2 Model Runner
(EngineCore pid=141070) INFO 11-01 00:00:35 [model_runner.py:392] Loading model from scratch...

Fetching 12 files:   0%|          | 0/12 [00:00<?, ?it/s]
Fetching 12 files: 100%|██████████| 12/12 [00:00<00:00, 948.15it/s]
(EngineCore pid=141070) INFO 11-01 00:00:36 [cuda.py:565] Using backend AttentionBackendEnum.FLASH_ATTN for vit attention
(EngineCore pid=141070) INFO 11-01 00:00:36 [mm_encoder_attention.py:372] Using AttentionBackendEnum.FLASH_ATTN for MMEncoderAttention.
(EngineCore pid=141070) INFO 11-01 00:00:36 [qwen_gdn_linear_attn.py:221] Using Triton/FLA GDN prefill kernel (requested=auto, head_k_dim=128).
(EngineCore pid=141070) INFO 11-01 00:00:37 [cuda.py:506] Using FLASH_ATTN attention backend out of potential backends: ['FLASH_ATTN', 'FLASHINFER', 'TRITON_ATTN', 'FLEX_ATTENTION'].
(EngineCore pid=141070) INFO 11-01 00:00:37 [flash_attn.py:1117] Using FlashAttention version 2
(EngineCore pid=141070) INFO 11-01 00:00:37 [base_loader.py:82] Initializing model took 2.108418 seconds
(EngineCore pid=141070) INFO 11-01 00:00:37 [weight_utils.py:1108] Filesystem type for checkpoints: XFS. Checkpoint size: 4.24 GiB. Available RAM: 133.04 GiB.
(EngineCore pid=141070) INFO 11-01 00:00:37 [weight_utils.py:1131] Auto-prefetch is disabled because the filesystem (XFS) is not a recognized network FS (NFS/Lustre). If you want to force prefetching, start vLLM with --safetensors-load-strategy=prefetch.

Loading safetensors checkpoint shards:   0% Completed | 0/1 [00:00<?, ?it/s]
(EngineCore pid=141070) WARNING 11-01 00:00:37 [utils.py:333] Attempted to load weight layers.0.linear_attn.norm.weight with dtype torch.float32 into parameter with dtype torch.bfloat16
(EngineCore pid=141070) WARNING 11-01 00:00:38 [utils.py:333] Attempted to load weight layers.1.linear_attn.norm.weight with dtype torch.float32 into parameter with dtype torch.bfloat16
(EngineCore pid=141070) WARNING 11-01 00:00:38 [utils.py:333] Attempted to load weight layers.10.linear_attn.norm.weight with dtype torch.float32 into parameter with dtype torch.bfloat16
(EngineCore pid=141070) WARNING 11-01 00:00:38 [utils.py:333] Attempted to load weight layers.12.linear_attn.norm.weight with dtype torch.float32 into parameter with dtype torch.bfloat16
(EngineCore pid=141070) WARNING 11-01 00:00:38 [utils.py:333] Attempted to load weight layers.13.linear_attn.norm.weight with dtype torch.float32 into parameter with dtype torch.bfloat16
(EngineCore pid=141070) WARNING 11-01 00:00:38 [utils.py:333] Attempted to load weight layers.14.linear_attn.norm.weight with dtype torch.float32 into parameter with dtype torch.bfloat16
(EngineCore pid=141070) WARNING 11-01 00:00:38 [utils.py:333] Attempted to load weight layers.16.linear_attn.norm.weight with dtype torch.float32 into parameter with dtype torch.bfloat16
(EngineCore pid=141070) WARNING 11-01 00:00:38 [utils.py:333] Attempted to load weight layers.17.linear_attn.norm.weight with dtype torch.float32 into parameter with dtype torch.bfloat16
(EngineCore pid=141070) WARNING 11-01 00:00:38 [utils.py:333] Attempted to load weight layers.18.linear_attn.norm.weight with dtype torch.float32 into parameter with dtype torch.bfloat16
(EngineCore pid=141070) WARNING 11-01 00:00:38 [utils.py:333] Attempted to load weight layers.2.linear_attn.norm.weight with dtype torch.float32 into parameter with dtype torch.bfloat16
(EngineCore pid=141070) WARNING 11-01 00:00:38 [utils.py:333] Attempted to load weight layers.20.linear_attn.norm.weight with dtype torch.float32 into parameter with dtype torch.bfloat16
(EngineCore pid=141070) WARNING 11-01 00:00:38 [utils.py:333] Attempted to load weight layers.21.linear_attn.norm.weight with dtype torch.float32 into parameter with dtype torch.bfloat16
(EngineCore pid=141070) WARNING 11-01 00:00:38 [utils.py:333] Attempted to load weight layers.22.linear_attn.norm.weight with dtype torch.float32 into parameter with dtype torch.bfloat16
(EngineCore pid=141070) WARNING 11-01 00:00:38 [utils.py:333] Attempted to load weight layers.4.linear_attn.norm.weight with dtype torch.float32 into parameter with dtype torch.bfloat16
(EngineCore pid=141070) WARNING 11-01 00:00:38 [utils.py:333] Attempted to load weight layers.5.linear_attn.norm.weight with dtype torch.float32 into parameter with dtype torch.bfloat16
(EngineCore pid=141070) WARNING 11-01 00:00:38 [utils.py:333] Attempted to load weight layers.6.linear_attn.norm.weight with dtype torch.float32 into parameter with dtype torch.bfloat16
(EngineCore pid=141070) WARNING 11-01 00:00:38 [utils.py:333] Attempted to load weight layers.8.linear_attn.norm.weight with dtype torch.float32 into parameter with dtype torch.bfloat16
(EngineCore pid=141070) WARNING 11-01 00:00:38 [utils.py:333] Attempted to load weight layers.9.linear_attn.norm.weight with dtype torch.float32 into parameter with dtype torch.bfloat16

Loading safetensors checkpoint shards: 100% Completed | 1/1 [00:00<00:00,  1.84it/s]

Loading safetensors checkpoint shards: 100% Completed | 1/1 [00:00<00:00,  1.84it/s]

(EngineCore pid=141070) INFO 11-01 00:00:38 [default_loader.py:501] Loading weights took 0.55 seconds
(EngineCore pid=141070) INFO 11-01 00:00:39 [model_runner.py:424] Model loading took 4.18 GiB memory and 4.588218 seconds
(EngineCore pid=141070) INFO 11-01 00:00:40 [topk_topp_sampler.py:124] Using FlashInfer for top-p & top-k sampling.
(EngineCore pid=141070) INFO 11-01 00:00:40 [torch_utils.py:295] Reducing Torch threads from 128 to 1 for serving. Set OMP_NUM_THREADS in the external environment to override.
(EngineCore pid=141070) INFO 11-01 00:00:40 [interface.py:1096] Setting attention block size to 544 tokens to ensure that attention page size is >= mamba page size.
(EngineCore pid=141070) INFO 11-01 00:00:40 [interface.py:1120] Padding mamba page size by 2.64% to ensure that mamba page size and attention page size are exactly equal.
(EngineCore pid=141070) INFO 11-01 00:00:40 [utils.py:347] Using LBNHC KV cache layout.
[transformers] Qwen3VL video processing does not apply the per-frame pixel cap the reference implementation (qwen-vl-utils) applies, so some videos cost far more tokens than they would there. In v5.22 the capped behavior will become the default and `cap_pixels_per_frame` will be removed. Pass `cap_pixels_per_frame=True` to adopt the reference behavior now, or `False` to keep the current behavior and silence this warning.
(MainProcess pid=140642) INFO 11-01 00:00:40 [base.py:280] Multi-modal warmup completed in 20.757s
(EngineCore pid=141070) INFO 11-01 00:00:47 [encoder_runner.py:131] Encoder cache will be initialized with a budget of 32768 tokens, and profiled with 2 image items of the maximum feature size.
(EngineCore pid=141070) INFO 11-01 00:00:53 [monitor.py:40] [backbone] Starting torch.compile
(EngineCore pid=141070) INFO 11-01 00:00:56 [backends.py:1104] [backbone] Using cache directory: /root/.cache/vllm/torch_compile_cache/ca01c80216/rank_0_0/backbone for vLLM's torch.compile
(EngineCore pid=141070) INFO 11-01 00:00:56 [backends.py:1166] [backbone] Dynamo bytecode transform time: 3.21 s
(EngineCore pid=141070) INFO 11-01 00:01:12 [backends.py:398] [backbone] Compiling a graph for compile range (1, 32768) takes 15.88 s
(EngineCore pid=141070) INFO 11-01 00:01:15 [backends.py:927] collected artifacts: 25 entries, 11 artifacts, 16110167 bytes total
(EngineCore pid=141070) INFO 11-01 00:01:15 [decorators.py:743] [backbone] saved AOT compiled function to /root/.cache/vllm/torch_compile_cache/torch_aot_compile/c806750065577d4cde6c4f91318ecde69c22cc1dde7e90363ece61d97269bb77/rank_0_0/model
(EngineCore pid=141070) INFO 11-01 00:01:15 [monitor.py:62] [backbone] torch.compile took 21.90 s in total
(EngineCore pid=141070) INFO 11-01 00:02:40 [monitor.py:91] [backbone] Initial profiling/warmup run took 85.21 s

Capturing CUDA graphs (PIECEWISE):   0%|          | 0/51 [00:00<?, ?it/s]
Capturing CUDA graphs (PIECEWISE):   2%|▏         | 1/51 [00:02<02:21,  2.84s/it]
Capturing CUDA graphs (PIECEWISE):   4%|▍         | 2/51 [00:03<01:19,  1.63s/it]
Capturing CUDA graphs (PIECEWISE):   6%|▌         | 3/51 [00:03<00:46,  1.03it/s]
Capturing CUDA graphs (PIECEWISE):   8%|▊         | 4/51 [00:04<00:39,  1.19it/s]
Capturing CUDA graphs (PIECEWISE):  10%|▉         | 5/51 [00:04<00:31,  1.47it/s]
Capturing CUDA graphs (PIECEWISE):  12%|█▏        | 6/51 [00:05<00:24,  1.82it/s]
Capturing CUDA graphs (PIECEWISE):  14%|█▎        | 7/51 [00:05<00:18,  2.35it/s]
Capturing CUDA graphs (PIECEWISE):  16%|█▌        | 8/51 [00:05<00:18,  2.26it/s]
Capturing CUDA graphs (PIECEWISE):  18%|█▊        | 9/51 [00:05<00:15,  2.70it/s]
Capturing CUDA graphs (PIECEWISE):  20%|█▉        | 10/51 [00:06<00:16,  2.43it/s]
Capturing CUDA graphs (PIECEWISE):  22%|██▏       | 11/51 [00:06<00:13,  2.91it/s]
Capturing CUDA graphs (PIECEWISE):  24%|██▎       | 12/51 [00:06<00:11,  3.49it/s]
Capturing CUDA graphs (PIECEWISE):  25%|██▌       | 13/51 [00:07<00:12,  2.95it/s]
Capturing CUDA graphs (PIECEWISE):  27%|██▋       | 14/51 [00:07<00:10,  3.65it/s]
Capturing CUDA graphs (PIECEWISE):  29%|██▉       | 15/51 [00:07<00:08,  4.40it/s]
Capturing CUDA graphs (PIECEWISE):  31%|███▏      | 16/51 [00:07<00:06,  5.26it/s]
Capturing CUDA graphs (PIECEWISE):  33%|███▎      | 17/51 [00:08<00:16,  2.04it/s]
Capturing CUDA graphs (PIECEWISE):  35%|███▌      | 18/51 [00:09<00:13,  2.43it/s]
Capturing CUDA graphs (PIECEWISE):  41%|████      | 21/51 [00:09<00:06,  4.90it/s]
Capturing CUDA graphs (PIECEWISE):  45%|████▌     | 23/51 [00:09<00:05,  5.22it/s]
Capturing CUDA graphs (PIECEWISE):  51%|█████     | 26/51 [00:09<00:03,  7.88it/s]
Capturing CUDA graphs (PIECEWISE):  57%|█████▋    | 29/51 [00:09<00:02, 10.69it/s]
Capturing CUDA graphs (PIECEWISE):  63%|██████▎   | 32/51 [00:09<00:01, 11.83it/s]
Capturing CUDA graphs (PIECEWISE):  67%|██████▋   | 34/51 [00:10<00:01, 10.60it/s]
Capturing CUDA graphs (PIECEWISE):  73%|███████▎  | 37/51 [00:10<00:01, 13.43it/s]
Capturing CUDA graphs (PIECEWISE):  78%|███████▊  | 40/51 [00:10<00:00, 16.11it/s]
Capturing CUDA graphs (PIECEWISE):  84%|████████▍ | 43/51 [00:10<00:00, 18.30it/s]
Capturing CUDA graphs (PIECEWISE):  90%|█████████ | 46/51 [00:10<00:00, 13.02it/s]
Capturing CUDA graphs (PIECEWISE):  96%|█████████▌| 49/51 [00:11<00:00, 15.44it/s]
Capturing CUDA graphs (PIECEWISE): 100%|██████████| 51/51 [00:11<00:00,  4.45it/s]

Capturing CUDA graphs (FULL):   0%|          | 0/2 [00:00<?, ?it/s]
Capturing CUDA graphs (FULL):  50%|█████     | 1/2 [00:00<00:00,  8.97it/s]
Capturing CUDA graphs (FULL): 100%|██████████| 2/2 [00:00<00:00, 14.09it/s]
(EngineCore pid=141070) INFO 11-01 00:02:55 [model_runner.py:1127] Graph capturing finished in 13 secs, took 0.31 GiB
(EngineCore pid=141070) INFO 11-01 00:02:55 [gpu_worker.py:759] Available KV cache memory: 62.23 GiB
(EngineCore pid=141070) INFO 11-01 00:02:55 [gpu_worker.py:774] CUDA graph memory profiling is enabled (default since v0.21.0). The current --gpu-memory-utilization=0.9200 is equivalent to --gpu-memory-utilization=0.9154 without CUDA graph memory profiling. To maintain the same effective KV cache size as before, increase --gpu-memory-utilization to 0.9246. To disable, set VLLM_MEMORY_PROFILER_ESTIMATE_CUDAGRAPHS=0.
(EngineCore pid=141070) INFO 11-01 00:02:55 [kv_cache_utils.py:2327] GPU KV cache size: 4,888,789 tokens, Maximum concurrency for 32,768 tokens per request: 149.19x
(EngineCore pid=141070) INFO 11-01 00:02:55 [kv_cache_utils.py:853] kv cache group sizes (544, 544, 544, 544)
(EngineCore pid=141070) INFO 11-01 00:02:55 [kv_cache_utils.py:854] kv lcm block sizes 544
(EngineCore pid=141070) INFO 11-01 00:02:56 [kernel_warmup.py:176] JIT kernel warmup starting.
(EngineCore pid=141070) INFO 11-01 00:02:56 [kernel_warmup.py:189] JIT kernel warmup finished in 0.00s.
(EngineCore pid=141070) INFO 11-01 00:02:56 [qwen_triton_warmup.py:296] Warming up Qwen GDN Triton kernels for model_type=qwen3_5_text.
(EngineCore pid=141070) INFO 11-01 00:02:57 [qwen_vl_triton_warmup.py:57] Warmed position embedding and vision rotary kernels on grids=[(1, 16, 16), (1, 16, 2), (1, 2, 16), (1, 2, 2)].
(EngineCore pid=141070) INFO 11-01 00:02:57 [mamba_triton_warmup.py:42] Warmed Mamba batch_memcpy_kernel.

Capturing CUDA graphs (PIECEWISE):   0%|          | 0/51 [00:00<?, ?it/s]
Capturing CUDA graphs (PIECEWISE):   2%|▏         | 1/51 [00:00<00:13,  3.80it/s]
Capturing CUDA graphs (PIECEWISE):   4%|▍         | 2/51 [00:00<00:15,  3.25it/s]
Capturing CUDA graphs (PIECEWISE):   6%|▌         | 3/51 [00:00<00:14,  3.38it/s]
Capturing CUDA graphs (PIECEWISE):   8%|▊         | 4/51 [00:01<00:13,  3.57it/s]
Capturing CUDA graphs (PIECEWISE):  10%|▉         | 5/51 [00:01<00:15,  3.05it/s]
Capturing CUDA graphs (PIECEWISE):  12%|█▏        | 6/51 [00:01<00:12,  3.51it/s]
Capturing CUDA graphs (PIECEWISE):  14%|█▎        | 7/51 [00:02<00:14,  3.00it/s]
Capturing CUDA graphs (PIECEWISE):  16%|█▌        | 8/51 [00:02<00:11,  3.59it/s]
Capturing CUDA graphs (PIECEWISE):  18%|█▊        | 9/51 [00:02<00:14,  2.97it/s]
Capturing CUDA graphs (PIECEWISE):  20%|█▉        | 10/51 [00:03<00:12,  3.32it/s]
Capturing CUDA graphs (PIECEWISE):  22%|██▏       | 11/51 [00:03<00:10,  3.73it/s]
Capturing CUDA graphs (PIECEWISE):  24%|██▎       | 12/51 [00:03<00:11,  3.39it/s]
Capturing CUDA graphs (PIECEWISE):  25%|██▌       | 13/51 [00:03<00:09,  3.98it/s]
Capturing CUDA graphs (PIECEWISE):  27%|██▋       | 14/51 [00:03<00:08,  4.60it/s]
Capturing CUDA graphs (PIECEWISE):  29%|██▉       | 15/51 [00:04<00:09,  3.81it/s]
Capturing CUDA graphs (PIECEWISE):  31%|███▏      | 16/51 [00:04<00:07,  4.68it/s]
Capturing CUDA graphs (PIECEWISE):  37%|███▋      | 19/51 [00:04<00:03,  8.78it/s]
Capturing CUDA graphs (PIECEWISE):  43%|████▎     | 22/51 [00:04<00:02, 12.48it/s]
Capturing CUDA graphs (PIECEWISE):  47%|████▋     | 24/51 [00:04<00:02,  9.63it/s]
Capturing CUDA graphs (PIECEWISE):  53%|█████▎    | 27/51 [00:04<00:01, 12.81it/s]
Capturing CUDA graphs (PIECEWISE):  59%|█████▉    | 30/51 [00:05<00:01, 15.95it/s]
Capturing CUDA graphs (PIECEWISE):  65%|██████▍   | 33/51 [00:05<00:00, 18.68it/s]
Capturing CUDA graphs (PIECEWISE):  71%|███████   | 36/51 [00:05<00:01, 13.40it/s]
Capturing CUDA graphs (PIECEWISE):  76%|███████▋  | 39/51 [00:05<00:00, 16.23it/s]
Capturing CUDA graphs (PIECEWISE):  82%|████████▏ | 42/51 [00:05<00:00, 18.86it/s]
Capturing CUDA graphs (PIECEWISE):  88%|████████▊ | 45/51 [00:05<00:00, 21.19it/s]
Capturing CUDA graphs (PIECEWISE):  94%|█████████▍| 48/51 [00:06<00:00, 17.17it/s]
Capturing CUDA graphs (PIECEWISE): 100%|██████████| 51/51 [00:06<00:00, 16.65it/s]
Capturing CUDA graphs (PIECEWISE): 100%|██████████| 51/51 [00:06<00:00,  8.08it/s]

Capturing CUDA graphs (FULL):   0%|          | 0/35 [00:00<?, ?it/s]
Capturing CUDA graphs (FULL):  11%|█▏        | 4/35 [00:00<00:00, 35.24it/s]
Capturing CUDA graphs (FULL):  23%|██▎       | 8/35 [00:00<00:00, 35.86it/s]
Capturing CUDA graphs (FULL):  34%|███▍      | 12/35 [00:00<00:00, 26.15it/s]
Capturing CUDA graphs (FULL):  43%|████▎     | 15/35 [00:00<00:00, 20.82it/s]
Capturing CUDA graphs (FULL):  54%|█████▍    | 19/35 [00:00<00:00, 25.03it/s]
Capturing CUDA graphs (FULL):  66%|██████▌   | 23/35 [00:00<00:00, 28.18it/s]
Capturing CUDA graphs (FULL):  77%|███████▋  | 27/35 [00:01<00:00, 25.03it/s]
Capturing CUDA graphs (FULL):  86%|████████▌ | 30/35 [00:03<00:01,  4.24it/s]
Capturing CUDA graphs (FULL):  97%|█████████▋| 34/35 [00:03<00:00,  5.99it/s]
Capturing CUDA graphs (FULL): 100%|██████████| 35/35 [00:03<00:00,  9.92it/s]
(EngineCore pid=141070) INFO 11-01 00:03:27 [model_runner.py:1127] Graph capturing finished in 11 secs, took 0.21 GiB
(EngineCore pid=141070) INFO 11-01 00:03:27 [gpu_worker.py:971] CUDA graph pool memory: 0.21 GiB (actual), 0.37 GiB (estimated), difference: 0.15 GiB (73.1%).
(EngineCore pid=141070) INFO 11-01 00:03:27 [gpu_worker.py:1034] Free memory on device (78.84/79.25 GiB) on startup. Desired GPU memory utilization is (0.92, 72.91 GiB). Actual usage is 7.11 GiB for consumed memory (weights + non-torch), 3.57 GiB for peak activation, and 0.21 GiB for CUDAGraph memory. Replace gpu_memory_utilization config with `--kv-cache-memory=66436249826` (61.87 GiB) to fit into requested memory, or `--kv-cache-memory=72800419328` (67.8 GiB) to fully utilize gpu memory. Current kv cache memory in use is 62.23 GiB.
(EngineCore pid=141070) INFO 11-01 00:03:28 [jit_monitor.py:84] Kernel JIT monitor activated; monitored JIT compilations during inference will use mode=warn.
(EngineCore pid=141070) INFO 11-01 00:03:28 [core.py:395] init engine (profile, create kv cache, warmup model) took 168.45 s (compilation: 21.90 s)

Parse safetensors files:   0%|          | 0/1 [00:00<?, ?it/s]
Parse safetensors files: 100%|██████████| 1/1 [00:00<00:00,  5.71it/s]
Parse safetensors files: 100%|██████████| 1/1 [00:00<00:00,  5.70it/s]
(EngineCore pid=141070) INFO 11-01 00:03:28 [kernel.py:467] Final IR op priority after setting platform defaults: IrOpPriorityConfig(rms_norm=['native'], fused_add_rms_norm=['native'], gelu_and_mul_sparse=['triton', 'native'])
(MainProcess pid=140642) INFO 11-01 00:03:29 [hf.py:648] Detected the chat template content format to be 'openai'. You can set `--chat-template-content-format` to override this.
(EngineCore pid=141070) WARNING 11-01 00:03:30 [jit_monitor.py:140] Triton kernel JIT compilation during inference: _topk_log_softmax_kernel. This causes a latency spike; consider extending warmup to cover this shape/config.
(MainProcess pid=140642) INFO 11-01 00:03:47 [loggers.py:335] Engine 000: Avg prompt throughput: 57671.2 tokens/s, Avg generation throughput: 115.9 tokens/s, Running: 87 reqs, Waiting: 3152 reqs, GPU KV cache usage: 4.2%, Prefix cache hit rate: 0.0%
(MainProcess pid=140642) INFO 11-01 00:03:57 [loggers.py:335] Engine 000: Avg prompt throughput: 66507.1 tokens/s, Avg generation throughput: 178.7 tokens/s, Running: 25 reqs, Waiting: 1381 reqs, GPU KV cache usage: 1.2%, Prefix cache hit rate: 1.2%
(MainProcess pid=140642) INFO 11-01 00:04:07 [loggers.py:335] Engine 000: Avg prompt throughput: 59431.1 tokens/s, Avg generation throughput: 138.4 tokens/s, Running: 1 reqs, Waiting: 511 reqs, GPU KV cache usage: 0.1%, Prefix cache hit rate: 0.0%
(EngineCore pid=141070) WARNING 11-01 00:04:08 [jit_monitor.py:140] Triton kernel JIT compilation during inference: _topp_sb_stats_kernel. This causes a latency spike; consider extending warmup to cover this shape/config.
(EngineCore pid=141070) WARNING 11-01 00:04:08 [jit_monitor.py:140] Triton kernel JIT compilation during inference: _topp_sb_step_kernel. This causes a latency spike; consider extending warmup to cover this shape/config.
(EngineCore pid=141070) WARNING 11-01 00:04:10 [jit_monitor.py:140] Triton kernel JIT compilation during inference: _topp_sb_mask_kernel. This causes a latency spike; consider extending warmup to cover this shape/config.
(MainProcess pid=140642) INFO 11-01 00:04:17 [loggers.py:335] Engine 000: Avg prompt throughput: 7664.2 tokens/s, Avg generation throughput: 8559.5 tokens/s, Running: 256 reqs, Waiting: 255 reqs, GPU KV cache usage: 11.9%, Prefix cache hit rate: 11.6%
(MainProcess pid=140642) INFO 11-01 00:04:27 [loggers.py:335] Engine 000: Avg prompt throughput: 1251.2 tokens/s, Avg generation throughput: 13294.4 tokens/s, Running: 256 reqs, Waiting: 212 reqs, GPU KV cache usage: 13.8%, Prefix cache hit rate: 13.6%
(MainProcess pid=140642) INFO 11-01 00:04:38 [loggers.py:335] Engine 000: Avg prompt throughput: 452.0 tokens/s, Avg generation throughput: 13895.5 tokens/s, Running: 256 reqs, Waiting: 193 reqs, GPU KV cache usage: 16.2%, Prefix cache hit rate: 13.4%
(MainProcess pid=140642) INFO 11-01 00:04:48 [loggers.py:335] Engine 000: Avg prompt throughput: 553.4 tokens/s, Avg generation throughput: 13133.6 tokens/s, Running: 256 reqs, Waiting: 171 reqs, GPU KV cache usage: 17.8%, Prefix cache hit rate: 13.6%
(MainProcess pid=140642) INFO 11-01 00:04:58 [loggers.py:335] Engine 000: Avg prompt throughput: 408.1 tokens/s, Avg generation throughput: 12777.0 tokens/s, Running: 256 reqs, Waiting: 150 reqs, GPU KV cache usage: 19.7%, Prefix cache hit rate: 14.9%
(MainProcess pid=140642) INFO 11-01 00:05:08 [loggers.py:335] Engine 000: Avg prompt throughput: 532.8 tokens/s, Avg generation throughput: 12422.6 tokens/s, Running: 256 reqs, Waiting: 133 reqs, GPU KV cache usage: 21.4%, Prefix cache hit rate: 15.1%
(MainProcess pid=140642) INFO 11-01 00:05:18 [loggers.py:335] Engine 000: Avg prompt throughput: 658.6 tokens/s, Avg generation throughput: 11863.4 tokens/s, Running: 256 reqs, Waiting: 112 reqs, GPU KV cache usage: 22.8%, Prefix cache hit rate: 16.2%
(MainProcess pid=140642) INFO 11-01 00:05:28 [loggers.py:335] Engine 000: Avg prompt throughput: 999.9 tokens/s, Avg generation throughput: 11460.1 tokens/s, Running: 256 reqs, Waiting: 85 reqs, GPU KV cache usage: 24.1%, Prefix cache hit rate: 17.0%
(MainProcess pid=140642) INFO 11-01 00:05:38 [loggers.py:335] Engine 000: Avg prompt throughput: 673.8 tokens/s, Avg generation throughput: 11408.4 tokens/s, Running: 256 reqs, Waiting: 66 reqs, GPU KV cache usage: 24.9%, Prefix cache hit rate: 19.0%
(MainProcess pid=140642) INFO 11-01 00:05:48 [loggers.py:335] Engine 000: Avg prompt throughput: 426.4 tokens/s, Avg generation throughput: 11283.6 tokens/s, Running: 256 reqs, Waiting: 48 reqs, GPU KV cache usage: 25.6%, Prefix cache hit rate: 19.9%
(MainProcess pid=140642) INFO 11-01 00:05:58 [loggers.py:335] Engine 000: Avg prompt throughput: 735.2 tokens/s, Avg generation throughput: 11056.1 tokens/s, Running: 256 reqs, Waiting: 26 reqs, GPU KV cache usage: 26.5%, Prefix cache hit rate: 20.1%
(MainProcess pid=140642) INFO 11-01 00:06:08 [loggers.py:335] Engine 000: Avg prompt throughput: 622.0 tokens/s, Avg generation throughput: 10933.9 tokens/s, Running: 256 reqs, Waiting: 3 reqs, GPU KV cache usage: 27.1%, Prefix cache hit rate: 21.8%
(MainProcess pid=140642) INFO 11-01 00:06:18 [loggers.py:335] Engine 000: Avg prompt throughput: 2.7 tokens/s, Avg generation throughput: 11349.8 tokens/s, Running: 226 reqs, Waiting: 0 reqs, GPU KV cache usage: 25.7%, Prefix cache hit rate: 22.1%
(MainProcess pid=140642) INFO 11-01 00:06:28 [loggers.py:335] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 10738.3 tokens/s, Running: 213 reqs, Waiting: 0 reqs, GPU KV cache usage: 26.3%, Prefix cache hit rate: 22.1%
(MainProcess pid=140642) INFO 11-01 00:06:38 [loggers.py:335] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 10182.9 tokens/s, Running: 200 reqs, Waiting: 0 reqs, GPU KV cache usage: 26.7%, Prefix cache hit rate: 22.1%
(MainProcess pid=140642) INFO 11-01 00:06:48 [loggers.py:335] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 9673.4 tokens/s, Running: 179 reqs, Waiting: 0 reqs, GPU KV cache usage: 25.4%, Prefix cache hit rate: 22.1%
(MainProcess pid=140642) INFO 11-01 00:06:58 [loggers.py:335] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 9025.7 tokens/s, Running: 164 reqs, Waiting: 0 reqs, GPU KV cache usage: 25.1%, Prefix cache hit rate: 22.1%
(MainProcess pid=140642) INFO 11-01 00:07:08 [loggers.py:335] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 8428.9 tokens/s, Running: 148 reqs, Waiting: 0 reqs, GPU KV cache usage: 24.1%, Prefix cache hit rate: 22.1%
(MainProcess pid=140642) INFO 11-01 00:07:18 [loggers.py:335] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 7905.3 tokens/s, Running: 138 reqs, Waiting: 0 reqs, GPU KV cache usage: 23.7%, Prefix cache hit rate: 22.1%
(MainProcess pid=140642) INFO 11-01 00:07:28 [loggers.py:335] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 7502.4 tokens/s, Running: 128 reqs, Waiting: 0 reqs, GPU KV cache usage: 23.4%, Prefix cache hit rate: 22.1%
(MainProcess pid=140642) INFO 11-01 00:07:38 [loggers.py:335] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 7323.4 tokens/s, Running: 122 reqs, Waiting: 0 reqs, GPU KV cache usage: 23.9%, Prefix cache hit rate: 22.1%
(MainProcess pid=140642) INFO 11-01 00:07:48 [loggers.py:335] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 6996.7 tokens/s, Running: 118 reqs, Waiting: 0 reqs, GPU KV cache usage: 24.5%, Prefix cache hit rate: 22.1%
(MainProcess pid=140642) INFO 11-01 00:07:58 [loggers.py:335] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 6668.0 tokens/s, Running: 114 reqs, Waiting: 0 reqs, GPU KV cache usage: 24.8%, Prefix cache hit rate: 22.1%
(MainProcess pid=140642) INFO 11-01 00:08:08 [loggers.py:335] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 6368.2 tokens/s, Running: 110 reqs, Waiting: 0 reqs, GPU KV cache usage: 25.4%, Prefix cache hit rate: 22.1%
(MainProcess pid=140642) INFO 11-01 00:08:18 [loggers.py:335] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 6136.6 tokens/s, Running: 108 reqs, Waiting: 0 reqs, GPU KV cache usage: 26.1%, Prefix cache hit rate: 22.1%
(MainProcess pid=140642) INFO 11-01 00:08:28 [loggers.py:335] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 5990.0 tokens/s, Running: 107 reqs, Waiting: 0 reqs, GPU KV cache usage: 27.2%, Prefix cache hit rate: 22.1%
(MainProcess pid=140642) INFO 11-01 00:08:38 [loggers.py:335] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 5837.0 tokens/s, Running: 107 reqs, Waiting: 0 reqs, GPU KV cache usage: 28.2%, Prefix cache hit rate: 22.1%
(MainProcess pid=140642) INFO 11-01 00:08:48 [loggers.py:335] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 5702.8 tokens/s, Running: 106 reqs, Waiting: 0 reqs, GPU KV cache usage: 28.9%, Prefix cache hit rate: 22.1%
(MainProcess pid=140642) INFO 11-01 00:08:58 [loggers.py:335] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 5583.8 tokens/s, Running: 106 reqs, Waiting: 0 reqs, GPU KV cache usage: 29.9%, Prefix cache hit rate: 22.1%
(MainProcess pid=140642) INFO 11-01 00:09:08 [loggers.py:335] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 5473.9 tokens/s, Running: 106 reqs, Waiting: 0 reqs, GPU KV cache usage: 31.0%, Prefix cache hit rate: 22.1%
(MainProcess pid=140642) INFO 11-01 00:09:18 [loggers.py:335] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 5374.5 tokens/s, Running: 104 reqs, Waiting: 0 reqs, GPU KV cache usage: 31.3%, Prefix cache hit rate: 22.1%
(MainProcess pid=140642) INFO 11-01 00:09:28 [loggers.py:335] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 5253.5 tokens/s, Running: 103 reqs, Waiting: 0 reqs, GPU KV cache usage: 32.0%, Prefix cache hit rate: 22.1%
(MainProcess pid=140642) INFO 11-01 00:09:38 [loggers.py:335] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 5150.2 tokens/s, Running: 103 reqs, Waiting: 0 reqs, GPU KV cache usage: 32.9%, Prefix cache hit rate: 22.1%
(MainProcess pid=140642) INFO 11-01 00:09:48 [loggers.py:335] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 5038.7 tokens/s, Running: 102 reqs, Waiting: 0 reqs, GPU KV cache usage: 33.7%, Prefix cache hit rate: 22.1%
(MainProcess pid=140642) INFO 11-01 00:09:58 [loggers.py:335] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 4953.9 tokens/s, Running: 102 reqs, Waiting: 0 reqs, GPU KV cache usage: 34.7%, Prefix cache hit rate: 22.1%
(MainProcess pid=140642) INFO 11-01 00:10:08 [loggers.py:335] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 4875.5 tokens/s, Running: 102 reqs, Waiting: 0 reqs, GPU KV cache usage: 35.4%, Prefix cache hit rate: 22.1%
(MainProcess pid=140642) INFO 11-01 00:10:18 [loggers.py:335] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 4786.0 tokens/s, Running: 101 reqs, Waiting: 0 reqs, GPU KV cache usage: 36.1%, Prefix cache hit rate: 22.1%
(MainProcess pid=140642) INFO 11-01 00:10:28 [loggers.py:335] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 4701.4 tokens/s, Running: 101 reqs, Waiting: 0 reqs, GPU KV cache usage: 37.0%, Prefix cache hit rate: 22.1%
(MainProcess pid=140642) INFO 11-01 00:10:38 [loggers.py:335] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 4634.3 tokens/s, Running: 101 reqs, Waiting: 0 reqs, GPU KV cache usage: 37.8%, Prefix cache hit rate: 22.1%
(MainProcess pid=140642) INFO 11-01 00:10:48 [loggers.py:335] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 4566.4 tokens/s, Running: 101 reqs, Waiting: 0 reqs, GPU KV cache usage: 38.6%, Prefix cache hit rate: 22.1%
(MainProcess pid=140642) INFO 11-01 00:10:58 [loggers.py:335] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 4503.8 tokens/s, Running: 101 reqs, Waiting: 0 reqs, GPU KV cache usage: 39.4%, Prefix cache hit rate: 22.1%
(MainProcess pid=140642) INFO 11-01 00:11:08 [loggers.py:335] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 4443.0 tokens/s, Running: 101 reqs, Waiting: 0 reqs, GPU KV cache usage: 40.3%, Prefix cache hit rate: 22.1%
(MainProcess pid=140642) INFO 11-01 00:11:18 [loggers.py:335] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 4381.9 tokens/s, Running: 100 reqs, Waiting: 0 reqs, GPU KV cache usage: 40.7%, Prefix cache hit rate: 22.1%
(MainProcess pid=140642) INFO 11-01 00:11:28 [loggers.py:335] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 4307.7 tokens/s, Running: 99 reqs, Waiting: 0 reqs, GPU KV cache usage: 41.2%, Prefix cache hit rate: 22.1%
(MainProcess pid=140642) INFO 11-01 00:11:38 [loggers.py:335] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 4237.0 tokens/s, Running: 99 reqs, Waiting: 0 reqs, GPU KV cache usage: 42.0%, Prefix cache hit rate: 22.1%
(MainProcess pid=140642) INFO 11-01 00:11:48 [loggers.py:335] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 4185.2 tokens/s, Running: 99 reqs, Waiting: 0 reqs, GPU KV cache usage: 42.7%, Prefix cache hit rate: 22.1%
(MainProcess pid=140642) INFO 11-01 00:11:58 [loggers.py:335] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 4134.5 tokens/s, Running: 99 reqs, Waiting: 0 reqs, GPU KV cache usage: 43.5%, Prefix cache hit rate: 22.1%
(MainProcess pid=140642) INFO 11-01 00:12:08 [loggers.py:335] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 4084.6 tokens/s, Running: 99 reqs, Waiting: 0 reqs, GPU KV cache usage: 44.3%, Prefix cache hit rate: 22.1%
(MainProcess pid=140642) INFO 11-01 00:12:18 [loggers.py:335] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 4035.9 tokens/s, Running: 99 reqs, Waiting: 0 reqs, GPU KV cache usage: 45.0%, Prefix cache hit rate: 22.1%
(MainProcess pid=140642) INFO 11-01 00:12:28 [loggers.py:335] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 3989.2 tokens/s, Running: 99 reqs, Waiting: 0 reqs, GPU KV cache usage: 45.7%, Prefix cache hit rate: 22.1%
(MainProcess pid=140642) INFO 11-01 00:12:38 [loggers.py:335] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 3943.8 tokens/s, Running: 99 reqs, Waiting: 0 reqs, GPU KV cache usage: 46.4%, Prefix cache hit rate: 22.1%
(MainProcess pid=140642) INFO 11-01 00:12:48 [loggers.py:335] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 3900.0 tokens/s, Running: 99 reqs, Waiting: 0 reqs, GPU KV cache usage: 47.2%, Prefix cache hit rate: 22.1%
(MainProcess pid=140642) INFO 11-01 00:12:58 [loggers.py:335] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 3857.7 tokens/s, Running: 99 reqs, Waiting: 0 reqs, GPU KV cache usage: 47.9%, Prefix cache hit rate: 22.1%
(MainProcess pid=140642) INFO 11-01 00:13:08 [loggers.py:335] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 3817.0 tokens/s, Running: 99 reqs, Waiting: 0 reqs, GPU KV cache usage: 48.6%, Prefix cache hit rate: 22.1%
(MainProcess pid=140642) INFO 11-01 00:13:18 [loggers.py:335] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 3776.8 tokens/s, Running: 99 reqs, Waiting: 0 reqs, GPU KV cache usage: 49.3%, Prefix cache hit rate: 22.1%
(MainProcess pid=140642) INFO 11-01 00:13:28 [loggers.py:335] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 3737.7 tokens/s, Running: 99 reqs, Waiting: 0 reqs, GPU KV cache usage: 50.0%, Prefix cache hit rate: 22.1%
(MainProcess pid=140642) INFO 11-01 00:13:38 [loggers.py:335] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 3700.4 tokens/s, Running: 99 reqs, Waiting: 0 reqs, GPU KV cache usage: 50.6%, Prefix cache hit rate: 22.1%
(MainProcess pid=140642) INFO 11-01 00:13:48 [loggers.py:335] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 3664.6 tokens/s, Running: 99 reqs, Waiting: 0 reqs, GPU KV cache usage: 51.3%, Prefix cache hit rate: 22.1%
(MainProcess pid=140642) INFO 11-01 00:13:58 [loggers.py:335] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 3629.7 tokens/s, Running: 99 reqs, Waiting: 0 reqs, GPU KV cache usage: 52.0%, Prefix cache hit rate: 22.1%
(MainProcess pid=140642) INFO 11-01 00:14:08 [loggers.py:335] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 3594.6 tokens/s, Running: 99 reqs, Waiting: 0 reqs, GPU KV cache usage: 52.7%, Prefix cache hit rate: 22.1%
(MainProcess pid=140642) INFO 11-01 00:14:18 [loggers.py:335] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 3561.0 tokens/s, Running: 99 reqs, Waiting: 0 reqs, GPU KV cache usage: 53.3%, Prefix cache hit rate: 22.1%
(MainProcess pid=140642) INFO 11-01 00:14:28 [loggers.py:335] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 3528.9 tokens/s, Running: 99 reqs, Waiting: 0 reqs, GPU KV cache usage: 53.9%, Prefix cache hit rate: 22.1%
(MainProcess pid=140642) INFO 11-01 00:14:38 [loggers.py:335] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 3497.8 tokens/s, Running: 99 reqs, Waiting: 0 reqs, GPU KV cache usage: 54.6%, Prefix cache hit rate: 22.1%
(MainProcess pid=140642) INFO 11-01 00:14:48 [loggers.py:335] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 3468.7 tokens/s, Running: 99 reqs, Waiting: 0 reqs, GPU KV cache usage: 55.2%, Prefix cache hit rate: 22.1%
(MainProcess pid=140642) INFO 11-01 00:14:58 [loggers.py:335] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 3445.2 tokens/s, Running: 97 reqs, Waiting: 0 reqs, GPU KV cache usage: 54.6%, Prefix cache hit rate: 22.1%
(MainProcess pid=140642) INFO 11-01 00:15:08 [loggers.py:335] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 3455.6 tokens/s, Running: 97 reqs, Waiting: 0 reqs, GPU KV cache usage: 55.3%, Prefix cache hit rate: 22.1%
(MainProcess pid=140642) INFO 11-01 00:15:18 [loggers.py:335] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 3425.9 tokens/s, Running: 97 reqs, Waiting: 0 reqs, GPU KV cache usage: 55.9%, Prefix cache hit rate: 22.1%
(MainProcess pid=140642) INFO 11-01 00:15:28 [loggers.py:335] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 3396.6 tokens/s, Running: 97 reqs, Waiting: 0 reqs, GPU KV cache usage: 56.6%, Prefix cache hit rate: 22.1%
(MainProcess pid=140642) INFO 11-01 00:15:38 [loggers.py:335] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 3368.0 tokens/s, Running: 97 reqs, Waiting: 0 reqs, GPU KV cache usage: 57.1%, Prefix cache hit rate: 22.1%
(MainProcess pid=140642) INFO 11-01 00:15:48 [loggers.py:335] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 3330.3 tokens/s, Running: 89 reqs, Waiting: 0 reqs, GPU KV cache usage: 52.8%, Prefix cache hit rate: 22.1%
(MainProcess pid=140642) INFO 11-01 00:15:58 [loggers.py:335] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 3197.4 tokens/s, Running: 83 reqs, Waiting: 0 reqs, GPU KV cache usage: 49.5%, Prefix cache hit rate: 22.1%
(MainProcess pid=140642) INFO 11-01 00:16:08 [loggers.py:335] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 2521.6 tokens/s, Running: 64 reqs, Waiting: 0 reqs, GPU KV cache usage: 37.9%, Prefix cache hit rate: 22.1%
(MainProcess pid=140642) INFO 11-01 00:16:18 [loggers.py:335] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 2857.1 tokens/s, Running: 41 reqs, Waiting: 0 reqs, GPU KV cache usage: 23.8%, Prefix cache hit rate: 22.1%
(MainProcess pid=140642) INFO 11-01 00:16:28 [loggers.py:335] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 2747.7 tokens/s, Running: 40 reqs, Waiting: 0 reqs, GPU KV cache usage: 23.7%, Prefix cache hit rate: 22.1%
(MainProcess pid=140642) INFO 11-01 00:16:38 [loggers.py:335] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 2623.4 tokens/s, Running: 34 reqs, Waiting: 0 reqs, GPU KV cache usage: 20.4%, Prefix cache hit rate: 22.1%
(MainProcess pid=140642) INFO 11-01 00:16:48 [loggers.py:335] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 2520.9 tokens/s, Running: 29 reqs, Waiting: 0 reqs, GPU KV cache usage: 17.6%, Prefix cache hit rate: 22.1%
(MainProcess pid=140642) INFO 11-01 00:16:58 [loggers.py:335] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 1769.0 tokens/s, Running: 19 reqs, Waiting: 0 reqs, GPU KV cache usage: 11.7%, Prefix cache hit rate: 22.1%
(MainProcess pid=140642) INFO 11-01 00:17:08 [loggers.py:335] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 1260.1 tokens/s, Running: 11 reqs, Waiting: 0 reqs, GPU KV cache usage: 6.9%, Prefix cache hit rate: 22.1%
(MainProcess pid=140642) INFO 11-01 00:17:19 [loggers.py:335] Engine 000: Avg prompt throughput: 14372.0 tokens/s, Avg generation throughput: 465.2 tokens/s, Running: 17 reqs, Waiting: 61 reqs, GPU KV cache usage: 2.3%, Prefix cache hit rate: 21.3%
{
 "model": "Qwen/Qwen3.5-2B",
 "readout_seconds": 37.46240019798279,
 "readout_items": 5442,
 "readout_acc": {
  "eval_proc": 0.40304182509505704,
  "eval_ts": 0.5091819699499165,
  "opendecisions": 0.5637755102040817,
  "localllama": 0.5104325699745547
 },
 "od_raw": {
  "capability": 48.70640726673241,
  "intelligence": 28.102108595620972,
  "calibration": 69.31070593784385
 },
 "think": {
  "items": 128,
  "samples": 512,
  "seconds": 788.4473285675049,
  "gen_tok_per_s": 5874.26050185857,
  "tokens_median": 3949.0,
  "tokens_p90": 32162.0,
  "tokens_max": 32665,
  "parse_rate": 0.806640625,
  "looped": 98
 },
 "pool_acc": {
  "train_proc": {
   "readout": 0.5,
   "think1": 0.6612903225806451,
   "maj4": 0.8225806451612904,
   "n": 62
  },
  "train_ts": {
   "readout": 0.5909090909090909,
   "think1": 0.6666666666666666,
   "maj4": 0.6818181818181818,
   "n": 66
  }
 },
 "post_readout_acc": 0.8055555555555556,
 "post_readout_agrees_with_generated_letter": 0.9629629629629629
}
(MainProcess pid=140642) INFO 11-01 00:17:23 [utils.py:628] [shutdown] Process manager: send sigterm to process EngineCore
(EngineCore pid=141070) INFO 11-01 00:17:23 [core.py:1454] [shutdown] EngineCore: trigger received signal=SIGTERM
(EngineCore pid=141070) INFO 11-01 00:17:23 [core.py:1635] [shutdown] EngineCore: start mode=abort timeout=0s
(EngineCore pid=141070) INFO 11-01 00:17:23 [core.py:1666] [shutdown] EngineCore: request processing complete; starting resource teardown
(EngineCore pid=141070) INFO 11-01 00:17:23 [core.py:1499] [shutdown] EngineCore: exiting busy loop
{"time": "2026-11-01T00:17:31Z", "event": "done", "stage": "2b_probe_qwen"}
{"time": "2026-11-01T00:17:42Z", "event": "start", "stage": "2b_probe_minicpm"}
(MainProcess pid=143946) INFO 11-01 00:17:53 [api_utils.py:286] non-default args: {'dtype': 'bfloat16', 'seed': 17, 'max_model_len': 32768, 'enable_prefix_caching': True, 'max_num_batched_tokens': 32768, 'max_num_seqs': 256, 'max_logprobs': 26, 'logprobs_mode': 'processed_logprobs', 'model': 'openbmb/MiniCPM5-2B'}
(MainProcess pid=143946) INFO 11-01 00:18:08 [model.py:733] Resolved architecture: LlamaForCausalLM
(MainProcess pid=143946) INFO 11-01 00:18:08 [model.py:2152] Using max model len 32768
(MainProcess pid=143946) INFO 11-01 00:18:08 [scheduler.py:307] Chunked prefill is enabled with max_num_batched_tokens=32768.
(MainProcess pid=143946) INFO 11-01 00:18:08 [kernel.py:467] Final IR op priority after setting platform defaults: IrOpPriorityConfig(rms_norm=['native'], fused_add_rms_norm=['native'], gelu_and_mul_sparse=['triton', 'native'])

Fetching 10 files:   0%|          | 0/10 [00:00<?, ?it/s]
Fetching 10 files: 100%|██████████| 10/10 [00:00<00:00, 929.38it/s]
(EngineCore pid=144491) INFO 11-01 00:18:10 [core.py:133] Initializing a V1 LLM engine (v0.32.0) with config: model='openbmb/MiniCPM5-2B', speculative_config=None, tokenizer='openbmb/MiniCPM5-2B', skip_tokenizer_init=False, tokenizer_mode=auto, revision=main, tokenizer_revision=main, trust_remote_code=False, dtype=torch.bfloat16, max_seq_len=32768, download_dir=None, load_format=auto, tensor_parallel_size=1, pipeline_parallel_size=1, data_parallel_size=1, decode_context_parallel_size=1, dcp_comm_backend=ag_rs, disable_custom_all_reduce=False, quantization=None, quantization_config=None, enforce_eager=False, aux_output_config=AuxOutputConfig(enable_return_routed_experts=False, max_bytes=None), kv_cache_dtype=auto, device_config=cuda, structured_outputs_config=StructuredOutputsConfig(backend='auto', disable_any_whitespace=False, disable_additional_properties=False, reasoning_parser='', reasoning_parser_plugin='', enable_in_reasoning=False), observability_config=ObservabilityConfig(show_hidden_metrics_for_version=None, otlp_traces_endpoint=None, collect_detailed_traces=None, per_request_spec_decode_metrics='none', kv_cache_metrics=False, kv_cache_metrics_sample=0.01, custom_histogram_buckets=None, cudagraph_metrics=False, enable_layerwise_nvtx_tracing=False, enable_mfu_metrics=False, enable_mm_processor_stats=False, enable_logging_iteration_details=False, jit_monitor_mode='warn', jit_monitor_verbose=False), seed=17, served_model_name=openbmb/MiniCPM5-2B, enable_prefix_caching=True, enable_chunked_prefill=True, pooler_config=None, compilation_config={'mode': <CompilationMode.VLLM_COMPILE: 3>, 'debug_dump_path': None, 'cache_dir': '', 'compile_cache_save_format': 'binary', 'backend': 'inductor', 'custom_ops': ['none'], 'ir_enable_torch_wrap': True, 'splitting_ops': ['vllm::unified_attention_with_output', 'vllm::unified_mla_attention_with_output', 'vllm::mamba_mixer2', 'vllm::mamba_mixer', 'vllm::short_conv', 'vllm::qwen4_exp_ple_short_conv', 'vllm::qwen4_exp_qsa_with_output', 'vllm::qwen4_exp_qsa_prepare_with_output', 'vllm::linear_attention', 'vllm::qwen_gdn_attention_core', 'vllm::qwen_gdn_attention_core_fused_norm_packed', 'vllm::gdn_attention_core_xpu', 'vllm::olmo_hybrid_gdn_full_forward', 'vllm::sparse_attn_indexer', 'vllm::rocm_aiter_sparse_attn_indexer', 'vllm::deepseek_v4_attention', 'vllm::hpc_rope_norm_forward', 'vllm::unified_kv_cache_update', 'vllm::unified_mla_kv_cache_update'], 'compile_mm_encoder': False, 'cudagraph_mm_encoder': False, 'encoder_cudagraph_token_budgets': [], 'encoder_cudagraph_max_vision_items_per_batch': 0, 'encoder_cudagraph_max_frames_per_batch': None, 'cudagraph_decoder_replay': True, 'decoder_replay_cudagraph_capture_sizes': [], 'decoder_replay_trim_threshold': 768, 'compile_sizes': [], 'compile_ranges_endpoints': [32768], 'inductor_compile_config': {'enable_auto_functionalized_v2': False, 'combo_kernels': True, 'benchmark_combo_kernel': True}, 'inductor_passes': {}, 'cudagraph_mode': <CUDAGraphMode.FULL_AND_PIECEWISE: (2, 1)>, 'cudagraph_num_of_warmups': 1, 'cudagraph_capture_sizes': [1, 2, 4, 8, 16, 24, 32, 40, 48, 56, 64, 72, 80, 88, 96, 104, 112, 120, 128, 136, 144, 152, 160, 168, 176, 184, 192, 200, 208, 216, 224, 232, 240, 248, 256, 272, 288, 304, 320, 336, 352, 368, 384, 400, 416, 432, 448, 464, 480, 496, 512], 'cudagraph_copy_inputs': False, 'cudagraph_specialize_lora': True, 'use_inductor_graph_partition': False, 'pass_config': {'fuse_norm_quant': False, 'fuse_act_quant': False, 'fuse_attn_quant': False, 'fuse_allreduce_rms': False, 'enable_qk_norm_rope_fusion': False, 'fuse_rope_kvcache_cat_mla': False, 'fuse_act_padding': False, 'fuse_qk_norm_rope_kvcache': False}, 'max_cudagraph_capture_size': 512, 'dynamic_shapes_config': {'type': <DynamicShapesType.BACKED: 'backed'>, 'evaluate_guards': False, 'assume_32_bit_indexing': False}, 'local_cache_dir': None, 'fast_moe_cold_start': False, 'static_all_moe_layers': []}, kernel_config=KernelConfig(ir_op_priority=IrOpPriorityConfig(rms_norm=['native'], fused_add_rms_norm=['native'], gelu_and_mul_sparse=['triton', 'native']), enable_flashinfer_autotune=True, enable_rocm_segmented_attn_autotune=False, enable_cutedsl_warmup=True, enable_jit_warmup=True, moe_backend='auto', sparse_indexer_topk_backend='auto', linear_backend='auto', linear_backend_per_quant=None)
(EngineCore pid=144491) INFO 11-01 00:18:11 [parallel_state.py:1906] world_size=1 rank=0 local_rank=0 distributed_init_method=file:///tmp/vllm_dist_252d4a1317914b1cba10decf60fc84a1 backend=nccl
(EngineCore pid=144491) INFO 11-01 00:18:11 [parallel_state.py:2350] rank 0 in world size 1 is assigned as DP rank 0, PP rank 0, PCP rank 0, TP rank 0, ETP rank 0, EP rank N/A, EPLB rank N/A
(EngineCore pid=144491) INFO 11-01 00:18:11 [gpu_worker.py:518] Using V2 Model Runner
(EngineCore pid=144491) INFO 11-01 00:18:12 [model_runner.py:392] Loading model from scratch...
(EngineCore pid=144491) INFO 11-01 00:18:14 [cuda.py:506] Using FLASH_ATTN attention backend out of potential backends: ['FLASH_ATTN', 'FLASHINFER', 'TRITON_ATTN', 'FLEX_ATTENTION'].
(EngineCore pid=144491) INFO 11-01 00:18:14 [flash_attn.py:1117] Using FlashAttention version 2
(EngineCore pid=144491) INFO 11-01 00:18:14 [base_loader.py:82] Initializing model took 1.759125 seconds
(EngineCore pid=144491) INFO 11-01 00:18:14 [weight_utils.py:1108] Filesystem type for checkpoints: XFS. Checkpoint size: 4.69 GiB. Available RAM: 134.81 GiB.
(EngineCore pid=144491) INFO 11-01 00:18:14 [weight_utils.py:1131] Auto-prefetch is disabled because the filesystem (XFS) is not a recognized network FS (NFS/Lustre). If you want to force prefetching, start vLLM with --safetensors-load-strategy=prefetch.

Loading safetensors checkpoint shards:   0% Completed | 0/1 [00:00<?, ?it/s]

Loading safetensors checkpoint shards: 100% Completed | 1/1 [00:00<00:00,  2.12it/s]

Loading safetensors checkpoint shards: 100% Completed | 1/1 [00:00<00:00,  2.12it/s]

(EngineCore pid=144491) INFO 11-01 00:18:15 [default_loader.py:501] Loading weights took 0.48 seconds
(EngineCore pid=144491) INFO 11-01 00:18:17 [model_runner.py:424] Model loading took 4.75 GiB memory and 4.675047 seconds
(EngineCore pid=144491) INFO 11-01 00:18:17 [topk_topp_sampler.py:124] Using FlashInfer for top-p & top-k sampling.
(EngineCore pid=144491) INFO 11-01 00:18:17 [torch_utils.py:295] Reducing Torch threads from 128 to 1 for serving. Set OMP_NUM_THREADS in the external environment to override.
(EngineCore pid=144491) INFO 11-01 00:18:17 [utils.py:347] Using LBNHC KV cache layout.
(EngineCore pid=144491) INFO 11-01 00:18:18 [monitor.py:40] [backbone] Starting torch.compile
(EngineCore pid=144491) INFO 11-01 00:18:26 [backends.py:1104] [backbone] Using cache directory: /root/.cache/vllm/torch_compile_cache/04798c4eef/rank_0_0/backbone for vLLM's torch.compile
(EngineCore pid=144491) INFO 11-01 00:18:26 [backends.py:1166] [backbone] Dynamo bytecode transform time: 8.84 s
(EngineCore pid=144491) INFO 11-01 00:18:35 [backends.py:398] [backbone] Compiling a graph for compile range (1, 32768) takes 7.92 s
(EngineCore pid=144491) INFO 11-01 00:18:40 [backends.py:927] collected artifacts: 43 entries, 3 artifacts, 4489589 bytes total
(EngineCore pid=144491) INFO 11-01 00:18:40 [decorators.py:743] [backbone] saved AOT compiled function to /root/.cache/vllm/torch_compile_cache/torch_aot_compile/b5dc5e04701a3cf6fd20d9922eb06fcda1ba1df72a6ff929f3731df693799890/rank_0_0/model
(EngineCore pid=144491) INFO 11-01 00:18:40 [monitor.py:62] [backbone] torch.compile took 22.68 s in total
(EngineCore pid=144491) INFO 11-01 00:18:40 [monitor.py:91] [backbone] Initial profiling/warmup run took 0.18 s

Capturing CUDA graphs (PIECEWISE):   0%|          | 0/51 [00:00<?, ?it/s]
Capturing CUDA graphs (PIECEWISE):   2%|▏         | 1/51 [00:00<00:10,  4.80it/s]
Capturing CUDA graphs (PIECEWISE):   6%|▌         | 3/51 [00:00<00:05,  9.18it/s]
Capturing CUDA graphs (PIECEWISE):  10%|▉         | 5/51 [00:00<00:07,  6.38it/s]
Capturing CUDA graphs (PIECEWISE):  12%|█▏        | 6/51 [00:00<00:07,  6.07it/s]
Capturing CUDA graphs (PIECEWISE):  16%|█▌        | 8/51 [00:01<00:05,  7.97it/s]
Capturing CUDA graphs (PIECEWISE):  20%|█▉        | 10/51 [00:01<00:04,  9.59it/s]
Capturing CUDA graphs (PIECEWISE):  24%|██▎       | 12/51 [00:01<00:06,  6.28it/s]
Capturing CUDA graphs (PIECEWISE):  27%|██▋       | 14/51 [00:01<00:04,  7.73it/s]
Capturing CUDA graphs (PIECEWISE):  31%|███▏      | 16/51 [00:02<00:03,  9.14it/s]
Capturing CUDA graphs (PIECEWISE):  35%|███▌      | 18/51 [00:02<00:05,  5.97it/s]
Capturing CUDA graphs (PIECEWISE):  39%|███▉      | 20/51 [00:02<00:04,  7.28it/s]
Capturing CUDA graphs (PIECEWISE):  43%|████▎     | 22/51 [00:02<00:03,  8.37it/s]
Capturing CUDA graphs (PIECEWISE):  47%|████▋     | 24/51 [00:03<00:04,  6.27it/s]
Capturing CUDA graphs (PIECEWISE):  51%|█████     | 26/51 [00:03<00:03,  7.55it/s]
Capturing CUDA graphs (PIECEWISE):  55%|█████▍    | 28/51 [00:03<00:02,  8.54it/s]
Capturing CUDA graphs (PIECEWISE):  59%|█████▉    | 30/51 [00:04<00:03,  6.41it/s]
Capturing CUDA graphs (PIECEWISE):  63%|██████▎   | 32/51 [00:04<00:02,  7.68it/s]
Capturing CUDA graphs (PIECEWISE):  67%|██████▋   | 34/51 [00:04<00:01,  8.74it/s]
Capturing CUDA graphs (PIECEWISE):  71%|███████   | 36/51 [00:05<00:02,  6.33it/s]
Capturing CUDA graphs (PIECEWISE):  75%|███████▍  | 38/51 [00:05<00:01,  7.61it/s]
Capturing CUDA graphs (PIECEWISE):  78%|███████▊  | 40/51 [00:05<00:01,  8.02it/s]
Capturing CUDA graphs (PIECEWISE):  82%|████████▏ | 42/51 [00:05<00:01,  6.54it/s]
Capturing CUDA graphs (PIECEWISE):  86%|████████▋ | 44/51 [00:05<00:00,  7.82it/s]
Capturing CUDA graphs (PIECEWISE):  90%|█████████ | 46/51 [00:06<00:00,  8.17it/s]
Capturing CUDA graphs (PIECEWISE):  92%|█████████▏| 47/51 [00:06<00:00,  6.06it/s]
Capturing CUDA graphs (PIECEWISE):  96%|█████████▌| 49/51 [00:06<00:00,  7.26it/s]
Capturing CUDA graphs (PIECEWISE): 100%|██████████| 51/51 [00:06<00:00,  8.15it/s]
Capturing CUDA graphs (PIECEWISE): 100%|██████████| 51/51 [00:06<00:00,  7.40it/s]

Capturing CUDA graphs (FULL):   0%|          | 0/2 [00:00<?, ?it/s]
Capturing CUDA graphs (FULL): 100%|██████████| 2/2 [00:00<00:00, 13.48it/s]
Capturing CUDA graphs (FULL): 100%|██████████| 2/2 [00:00<00:00, 13.47it/s]
(EngineCore pid=144491) INFO 11-01 00:18:51 [model_runner.py:1127] Graph capturing finished in 8 secs, took 0.42 GiB
(EngineCore pid=144491) INFO 11-01 00:18:52 [gpu_worker.py:759] Available KV cache memory: 65.84 GiB
(EngineCore pid=144491) INFO 11-01 00:18:52 [gpu_worker.py:774] CUDA graph memory profiling is enabled (default since v0.21.0). The current --gpu-memory-utilization=0.9200 is equivalent to --gpu-memory-utilization=0.9140 without CUDA graph memory profiling. To maintain the same effective KV cache size as before, increase --gpu-memory-utilization to 0.9260. To disable, set VLLM_MEMORY_PROFILER_ESTIMATE_CUDAGRAPHS=0.
(EngineCore pid=144491) INFO 11-01 00:18:52 [kv_cache_utils.py:2327] GPU KV cache size: 1,643,712 tokens, Maximum concurrency for 32,768 tokens per request: 50.16x
(EngineCore pid=144491) INFO 11-01 00:18:52 [kernel_warmup.py:176] JIT kernel warmup starting.
(EngineCore pid=144491) INFO 11-01 00:18:52 [kernel_warmup.py:189] JIT kernel warmup finished in 0.00s.

Capturing CUDA graphs (PIECEWISE):   0%|          | 0/51 [00:00<?, ?it/s]
Capturing CUDA graphs (PIECEWISE):   2%|▏         | 1/51 [00:00<00:06,  7.71it/s]
Capturing CUDA graphs (PIECEWISE):   6%|▌         | 3/51 [00:00<00:04, 11.95it/s]
Capturing CUDA graphs (PIECEWISE):  10%|▉         | 5/51 [00:00<00:03, 13.27it/s]
Capturing CUDA graphs (PIECEWISE):  14%|█▎        | 7/51 [00:00<00:06,  6.55it/s]
Capturing CUDA graphs (PIECEWISE):  18%|█▊        | 9/51 [00:01<00:05,  8.03it/s]
Capturing CUDA graphs (PIECEWISE):  22%|██▏       | 11/51 [00:01<00:04,  9.51it/s]
Capturing CUDA graphs (PIECEWISE):  25%|██▌       | 13/51 [00:01<00:05,  6.50it/s]
Capturing CUDA graphs (PIECEWISE):  29%|██▉       | 15/51 [00:01<00:04,  7.73it/s]
Capturing CUDA graphs (PIECEWISE):  33%|███▎      | 17/51 [00:02<00:03,  8.84it/s]
Capturing CUDA graphs (PIECEWISE):  37%|███▋      | 19/51 [00:02<00:05,  6.36it/s]
Capturing CUDA graphs (PIECEWISE):  41%|████      | 21/51 [00:02<00:03,  7.61it/s]
Capturing CUDA graphs (PIECEWISE):  45%|████▌     | 23/51 [00:02<00:03,  8.89it/s]
Capturing CUDA graphs (PIECEWISE):  49%|████▉     | 25/51 [00:03<00:04,  6.04it/s]
Capturing CUDA graphs (PIECEWISE):  51%|█████     | 26/51 [00:05<00:14,  1.70it/s]
Capturing CUDA graphs (PIECEWISE):  53%|█████▎    | 27/51 [00:06<00:12,  1.85it/s]
Capturing CUDA graphs (PIECEWISE):  57%|█████▋    | 29/51 [00:06<00:08,  2.67it/s]
Capturing CUDA graphs (PIECEWISE):  61%|██████    | 31/51 [00:06<00:06,  3.16it/s]
Capturing CUDA graphs (PIECEWISE):  63%|██████▎   | 32/51 [00:07<00:05,  3.44it/s]
Capturing CUDA graphs (PIECEWISE):  67%|██████▋   | 34/51 [00:07<00:03,  4.73it/s]
Capturing CUDA graphs (PIECEWISE):  71%|███████   | 36/51 [00:07<00:02,  6.10it/s]
Capturing CUDA graphs (PIECEWISE):  75%|███████▍  | 38/51 [00:07<00:02,  5.10it/s]
Capturing CUDA graphs (PIECEWISE):  78%|███████▊  | 40/51 [00:08<00:01,  6.38it/s]
Capturing CUDA graphs (PIECEWISE):  82%|████████▏ | 42/51 [00:08<00:01,  7.71it/s]
Capturing CUDA graphs (PIECEWISE):  86%|████████▋ | 44/51 [00:08<00:01,  5.85it/s]
Capturing CUDA graphs (PIECEWISE):  90%|█████████ | 46/51 [00:08<00:00,  7.13it/s]
Capturing CUDA graphs (PIECEWISE):  94%|█████████▍| 48/51 [00:08<00:00,  8.37it/s]
Capturing CUDA graphs (PIECEWISE):  98%|█████████▊| 50/51 [00:09<00:00,  6.14it/s]
Capturing CUDA graphs (PIECEWISE): 100%|██████████| 51/51 [00:09<00:00,  5.34it/s]

Capturing CUDA graphs (FULL):   0%|          | 0/35 [00:00<?, ?it/s]
Capturing CUDA graphs (FULL):   9%|▊         | 3/35 [00:00<00:01, 24.25it/s]
Capturing CUDA graphs (FULL):  17%|█▋        | 6/35 [00:00<00:01, 24.00it/s]
Capturing CUDA graphs (FULL):  26%|██▌       | 9/35 [00:00<00:01, 18.35it/s]
Capturing CUDA graphs (FULL):  31%|███▏      | 11/35 [00:00<00:01, 15.11it/s]
Capturing CUDA graphs (FULL):  40%|████      | 14/35 [00:00<00:01, 17.42it/s]
Capturing CUDA graphs (FULL):  49%|████▊     | 17/35 [00:00<00:00, 19.51it/s]
Capturing CUDA graphs (FULL):  57%|█████▋    | 20/35 [00:01<00:00, 21.13it/s]
Capturing CUDA graphs (FULL):  66%|██████▌   | 23/35 [00:01<00:00, 20.32it/s]
Capturing CUDA graphs (FULL):  74%|███████▍  | 26/35 [00:01<00:00, 17.64it/s]
Capturing CUDA graphs (FULL):  80%|████████  | 28/35 [00:01<00:00, 17.15it/s]
Capturing CUDA graphs (FULL):  89%|████████▊ | 31/35 [00:01<00:00, 19.24it/s]
Capturing CUDA graphs (FULL):  97%|█████████▋| 34/35 [00:01<00:00, 20.65it/s]
Capturing CUDA graphs (FULL): 100%|██████████| 35/35 [00:01<00:00, 19.44it/s]
(EngineCore pid=144491) INFO 11-01 00:19:22 [model_runner.py:1127] Graph capturing finished in 13 secs, took 0.26 GiB
(EngineCore pid=144491) INFO 11-01 00:19:22 [gpu_worker.py:971] CUDA graph pool memory: 0.26 GiB (actual), 0.47 GiB (estimated), difference: 0.22 GiB (84.1%).
(EngineCore pid=144491) INFO 11-01 00:19:22 [gpu_worker.py:1034] Free memory on device (78.84/79.25 GiB) on startup. Desired GPU memory utilization is (0.92, 72.91 GiB). Actual usage is 5.1 GiB for consumed memory (weights + non-torch), 1.97 GiB for peak activation, and 0.26 GiB for CUDAGraph memory. Replace gpu_memory_utilization config with `--kv-cache-memory=70259336418` (65.43 GiB) to fit into requested memory, or `--kv-cache-memory=76623505920` (71.36 GiB) to fully utilize gpu memory. Current kv cache memory in use is 65.84 GiB.
(EngineCore pid=144491) INFO 11-01 00:19:22 [jit_monitor.py:84] Kernel JIT monitor activated; monitored JIT compilations during inference will use mode=warn.
(EngineCore pid=144491) INFO 11-01 00:19:23 [core.py:395] init engine (profile, create kv cache, warmup model) took 65.93 s (compilation: 22.68 s)
(EngineCore pid=144491) INFO 11-01 00:19:23 [kernel.py:467] Final IR op priority after setting platform defaults: IrOpPriorityConfig(rms_norm=['native'], fused_add_rms_norm=['native'], gelu_and_mul_sparse=['triton', 'native'])
(MainProcess pid=143946) INFO 11-01 00:19:24 [hf.py:648] Detected the chat template content format to be 'string'. You can set `--chat-template-content-format` to override this.
(EngineCore pid=144491) WARNING 11-01 00:19:25 [jit_monitor.py:140] Triton kernel JIT compilation during inference: _topk_log_softmax_kernel. This causes a latency spike; consider extending warmup to cover this shape/config.
(MainProcess pid=143946) INFO 11-01 00:19:41 [loggers.py:335] Engine 000: Avg prompt throughput: 46452.8 tokens/s, Avg generation throughput: 105.9 tokens/s, Running: 40 reqs, Waiting: 3461 reqs, GPU KV cache usage: 2.1%, Prefix cache hit rate: 8.5%
(MainProcess pid=143946) INFO 11-01 00:19:52 [loggers.py:335] Engine 000: Avg prompt throughput: 53698.2 tokens/s, Avg generation throughput: 140.2 tokens/s, Running: 167 reqs, Waiting: 1862 reqs, GPU KV cache usage: 2.1%, Prefix cache hit rate: 27.6%
(MainProcess pid=143946) INFO 11-01 00:20:02 [loggers.py:335] Engine 000: Avg prompt throughput: 35225.1 tokens/s, Avg generation throughput: 3547.1 tokens/s, Running: 255 reqs, Waiting: 237 reqs, GPU KV cache usage: 4.0%, Prefix cache hit rate: 66.7%
(MainProcess pid=143946) INFO 11-01 00:20:12 [loggers.py:335] Engine 000: Avg prompt throughput: 1610.9 tokens/s, Avg generation throughput: 10428.0 tokens/s, Running: 256 reqs, Waiting: 79 reqs, GPU KV cache usage: 8.5%, Prefix cache hit rate: 68.5%
(MainProcess pid=143946) INFO 11-01 00:20:22 [loggers.py:335] Engine 000: Avg prompt throughput: 775.9 tokens/s, Avg generation throughput: 11250.2 tokens/s, Running: 217 reqs, Waiting: 0 reqs, GPU KV cache usage: 11.5%, Prefix cache hit rate: 70.9%
(MainProcess pid=143946) INFO 11-01 00:20:32 [loggers.py:335] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 10329.2 tokens/s, Running: 97 reqs, Waiting: 0 reqs, GPU KV cache usage: 9.5%, Prefix cache hit rate: 70.9%
(EngineCore pid=144491) WARNING 11-01 00:20:38 [jit_monitor.py:140] Triton kernel JIT compilation during inference: _topp_sb_stats_kernel. This causes a latency spike; consider extending warmup to cover this shape/config.
(EngineCore pid=144491) WARNING 11-01 00:20:38 [jit_monitor.py:140] Triton kernel JIT compilation during inference: _topp_sb_step_kernel. This causes a latency spike; consider extending warmup to cover this shape/config.
(EngineCore pid=144491) WARNING 11-01 00:20:40 [jit_monitor.py:140] Triton kernel JIT compilation during inference: _topp_sb_mask_kernel. This causes a latency spike; consider extending warmup to cover this shape/config.
(MainProcess pid=143946) INFO 11-01 00:20:43 [loggers.py:335] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 4708.3 tokens/s, Running: 54 reqs, Waiting: 0 reqs, GPU KV cache usage: 7.9%, Prefix cache hit rate: 70.9%
(MainProcess pid=143946) INFO 11-01 00:20:53 [loggers.py:335] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 3467.7 tokens/s, Running: 26 reqs, Waiting: 0 reqs, GPU KV cache usage: 5.4%, Prefix cache hit rate: 70.9%
(MainProcess pid=143946) INFO 11-01 00:21:03 [loggers.py:335] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 2505.0 tokens/s, Running: 14 reqs, Waiting: 0 reqs, GPU KV cache usage: 4.0%, Prefix cache hit rate: 70.9%
(MainProcess pid=143946) INFO 11-01 00:21:14 [loggers.py:335] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 881.7 tokens/s, Running: 6 reqs, Waiting: 0 reqs, GPU KV cache usage: 2.2%, Prefix cache hit rate: 70.9%
(MainProcess pid=143946) INFO 11-01 00:21:24 [loggers.py:335] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 809.0 tokens/s, Running: 5 reqs, Waiting: 0 reqs, GPU KV cache usage: 2.3%, Prefix cache hit rate: 70.9%
(MainProcess pid=143946) INFO 11-01 00:21:34 [loggers.py:335] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 643.7 tokens/s, Running: 2 reqs, Waiting: 0 reqs, GPU KV cache usage: 1.2%, Prefix cache hit rate: 70.9%
(MainProcess pid=143946) INFO 11-01 00:21:44 [loggers.py:335] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 312.9 tokens/s, Running: 2 reqs, Waiting: 0 reqs, GPU KV cache usage: 1.3%, Prefix cache hit rate: 70.9%
(MainProcess pid=143946) INFO 11-01 00:21:54 [loggers.py:335] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 309.4 tokens/s, Running: 2 reqs, Waiting: 0 reqs, GPU KV cache usage: 1.5%, Prefix cache hit rate: 70.9%
(MainProcess pid=143946) INFO 11-01 00:22:04 [loggers.py:335] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 303.3 tokens/s, Running: 2 reqs, Waiting: 0 reqs, GPU KV cache usage: 1.7%, Prefix cache hit rate: 70.9%
(MainProcess pid=143946) INFO 11-01 00:22:14 [loggers.py:335] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 297.8 tokens/s, Running: 2 reqs, Waiting: 0 reqs, GPU KV cache usage: 1.9%, Prefix cache hit rate: 70.9%
(MainProcess pid=143946) INFO 11-01 00:22:24 [loggers.py:335] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 295.8 tokens/s, Running: 2 reqs, Waiting: 0 reqs, GPU KV cache usage: 2.1%, Prefix cache hit rate: 70.9%
(MainProcess pid=143946) INFO 11-01 00:22:34 [loggers.py:335] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 293.9 tokens/s, Running: 2 reqs, Waiting: 0 reqs, GPU KV cache usage: 2.3%, Prefix cache hit rate: 70.9%
(MainProcess pid=143946) INFO 11-01 00:22:44 [loggers.py:335] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 292.2 tokens/s, Running: 2 reqs, Waiting: 0 reqs, GPU KV cache usage: 2.4%, Prefix cache hit rate: 70.9%
(MainProcess pid=143946) INFO 11-01 00:22:54 [loggers.py:335] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 284.6 tokens/s, Running: 2 reqs, Waiting: 0 reqs, GPU KV cache usage: 2.6%, Prefix cache hit rate: 70.9%
(MainProcess pid=143946) INFO 11-01 00:23:04 [loggers.py:335] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 282.9 tokens/s, Running: 2 reqs, Waiting: 0 reqs, GPU KV cache usage: 2.8%, Prefix cache hit rate: 70.9%
(MainProcess pid=143946) INFO 11-01 00:23:14 [loggers.py:335] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 280.9 tokens/s, Running: 2 reqs, Waiting: 0 reqs, GPU KV cache usage: 2.9%, Prefix cache hit rate: 70.9%
(MainProcess pid=143946) INFO 11-01 00:23:24 [loggers.py:335] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 279.3 tokens/s, Running: 2 reqs, Waiting: 0 reqs, GPU KV cache usage: 3.1%, Prefix cache hit rate: 70.9%
(MainProcess pid=143946) INFO 11-01 00:23:34 [loggers.py:335] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 276.5 tokens/s, Running: 2 reqs, Waiting: 0 reqs, GPU KV cache usage: 3.3%, Prefix cache hit rate: 70.9%
(MainProcess pid=143946) INFO 11-01 00:23:44 [loggers.py:335] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 271.7 tokens/s, Running: 2 reqs, Waiting: 0 reqs, GPU KV cache usage: 3.5%, Prefix cache hit rate: 70.9%
(MainProcess pid=143946) INFO 11-01 00:23:55 [loggers.py:335] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 268.3 tokens/s, Running: 2 reqs, Waiting: 0 reqs, GPU KV cache usage: 3.6%, Prefix cache hit rate: 70.9%
(MainProcess pid=143946) INFO 11-01 00:24:05 [loggers.py:335] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 267.7 tokens/s, Running: 2 reqs, Waiting: 0 reqs, GPU KV cache usage: 3.8%, Prefix cache hit rate: 70.9%
(MainProcess pid=143946) INFO 11-01 00:24:15 [loggers.py:335] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 266.1 tokens/s, Running: 2 reqs, Waiting: 0 reqs, GPU KV cache usage: 3.9%, Prefix cache hit rate: 70.9%
{
 "model": "openbmb/MiniCPM5-2B",
 "readout_seconds": 33.92064571380615,
 "readout_items": 5442,
 "readout_acc": {
  "eval_proc": 0.4068441064638783,
  "eval_ts": 0.5058430717863105,
  "opendecisions": 0.5535714285714286,
  "localllama": 0.46615776081424937
 },
 "od_raw": {
  "capability": 29.539803811254448,
  "intelligence": 25.083511738925665,
  "calibration": 33.99609588358323
 },
 "think": {
  "items": 128,
  "samples": 512,
  "seconds": 258.7961437702179,
  "gen_tok_per_s": 2086.6075982935245,
  "tokens_median": 532.0,
  "tokens_p90": 2269.5000000000005,
  "tokens_max": 31708,
  "parse_rate": 0.99609375,
  "looped": 2
 },
 "pool_acc": {
  "train_proc": {
   "readout": 0.4838709677419355,
   "think1": 0.9354838709677419,
   "maj4": 0.9516129032258065,
   "n": 62
  },
  "train_ts": {
   "readout": 0.5151515151515151,
   "think1": 0.6818181818181818,
   "maj4": 0.7121212121212122,
   "n": 66
  }
 },
 "post_readout_acc": 0.8110236220472441,
 "post_readout_agrees_with_generated_letter": 1.0
}
(MainProcess pid=143946) INFO 11-01 00:24:18 [utils.py:628] [shutdown] Process manager: send sigterm to process EngineCore
(EngineCore pid=144491) INFO 11-01 00:24:18 [core.py:1454] [shutdown] EngineCore: trigger received signal=SIGTERM
(EngineCore pid=144491) INFO 11-01 00:24:18 [core.py:1635] [shutdown] EngineCore: start mode=abort timeout=0s
(EngineCore pid=144491) INFO 11-01 00:24:18 [core.py:1666] [shutdown] EngineCore: request processing complete; starting resource teardown
(EngineCore pid=144491) INFO 11-01 00:24:18 [core.py:1499] [shutdown] EngineCore: exiting busy loop
{"time": "2026-11-01T00:24:23Z", "event": "done", "stage": "2b_probe_minicpm"}
{"time": "2026-11-01T00:24:32Z", "event": "chosen", "stage": "2b_model=openbmb/MiniCPM5-2B"}
{"time": "2026-11-01T00:24:32Z", "event": "start", "stage": "2b_smoke"}
{"examples": 12474, "tokens": 25945529, "readout_from_votes": 259, "readout_eval_order": 720, "readout_gold_only": 1241, "reasoner_chains_dropped_invented_recall": 2193, "reasoner_chains_dropped_repetition": 11, "reasoner_cw_stripped": 1532, "reasoner_soft_letter": 10972, "reasoner_examples": 10974, "reasoner_items": 5779, "reasoner_no_correct_chain": 976, "reasoner_too_few_correct": 257, "reasoner_letter_not_found": 2}

Loading weights:   0%|          | 0/381 [00:00<?, ?it/s]
Loading weights:  97%|█████████▋| 368/381 [00:00<00:00, 3647.69it/s]
Loading weights: 100%|██████████| 381/381 [00:00<00:00, 3706.37it/s]
trainable params: 50,233,344 || all params: 2,566,989,824 || trainable%: 1.9569
/workspace/decision-reasoner/runs/v2_2b/train.py:231: UserWarning: Converting a tensor with requires_grad=True to a scalar may lead to unexpected behavior.
Consider using tensor.detach() first. (Triggered internally at /__w/pytorch/pytorch/torch/csrc/autograd/generated/python_variable_methods.cpp:822.)
  stats["chain_loss"] += float(loss)
{"step": 1, "of": 716, "lr": 0.0002, "grad_norm": 8.223553657531738, "examples": 8, "readout_examples": 0, "reasoner_examples": 8, "step_tokens": 38940, "chain_tokens": 32757, "micro_batches": 6, "tok_per_s": 4622.245860980052, "elapsed_min": 0.14040793577829996, "eta_min": 100.39167408148448, "chain_loss": 0.42687782645225525, "readout_loss": null, "readout_argmax_matches_target": null, "readout_mean_top_prob": null, "readout_target_entropy": null, "letter_loss": 0.30469766257115793, "letter_top_prob": 0.999989852309227, "mem_gb": 47.253861888}
{"step": 2, "of": 716, "lr": 0.0, "grad_norm": 5.36545991897583, "examples": 11, "readout_examples": 0, "reasoner_examples": 11, "step_tokens": 38819, "chain_tokens": 32243, "micro_batches": 6, "tok_per_s": 5514.857421849702, "elapsed_min": 0.23499852021535236, "eta_min": 83.8944717168808, "chain_loss": 0.3356162431565198, "readout_loss": null, "readout_argmax_matches_target": null, "readout_mean_top_prob": null, "readout_target_entropy": null, "letter_loss": 0.6917720816351886, "letter_top_prob": 0.9999892820011486, "mem_gb": 48.18899968}
{"dev_n": 999, "dev_acc": 0.4004004004004004, "dev_nll": 2.48420600401401, "dev_ece15": 0.36437308648088435}
{"dev_n": 999, "dev_acc": 0.4004004004004004, "dev_nll": 2.48420600401401, "dev_ece15": 0.36437308648088435, "v2_options": {"readout_pool": "all", "subset": "data/sweep_ids.txt", "min_correct": 2, "second_chain_min": 3, "rep_max": 20, "post_soft": true, "eval_order_frac": 0.5, "drop_recall": true, "strip_cw": true}, "speed_options": {"liger": false, "ckpt_above": 8192, "micro_tokens": 8192, "no_probe": true}, "mode": "joint", "alpha": 0.5, "lr": 0.0002, "rank": 32, "lora_alpha": 64, "seed": 17, "examples": 12474, "steps_done": 2, "steps_planned": 716, "stopped_at_deadline": true, "train_tokens": 77759, "train_seconds": 14.167759656906128, "tok_per_s": 5488.44714217721, "counts": {"readout_from_votes": 259, "readout_eval_order": 720, "readout_gold_only": 1241, "reasoner_chains_dropped_invented_recall": 2193, "reasoner_chains_dropped_repetition": 11, "reasoner_cw_stripped": 1532, "reasoner_soft_letter": 10972, "reasoner_examples": 10974, "reasoner_items": 5779, "reasoner_no_correct_chain": 976, "reasoner_too_few_correct": 257, "reasoner_letter_not_found": 2}, "reasoner_examples": 10974, "readout_examples": 1500}
{"time": "2026-11-01T00:26:56Z", "event": "done", "stage": "2b_smoke"}
{"time": "2026-11-01T00:27:06Z", "event": "start", "stage": "2b_sweep_1e-4"}
{"examples": 1500, "tokens": 742814, "readout_from_votes": 259, "readout_eval_order": 720, "readout_gold_only": 1241}

Loading weights:   0%|          | 0/381 [00:00<?, ?it/s]
Loading weights:  94%|█████████▍| 358/381 [00:00<00:00, 3578.02it/s]
Loading weights: 100%|██████████| 381/381 [00:00<00:00, 3579.99it/s]
trainable params: 50,233,344 || all params: 2,566,989,824 || trainable%: 1.9569
/workspace/decision-reasoner/runs/v2_2b/train.py:215: UserWarning: Converting a tensor with requires_grad=True to a scalar may lead to unexpected behavior.
Consider using tensor.detach() first. (Triggered internally at /__w/pytorch/pytorch/torch/csrc/autograd/generated/python_variable_methods.cpp:822.)
  stats["readout_loss"] += float(loss)
{"step": 1, "of": 20, "lr": 0.0001, "grad_norm": 47.893653869628906, "examples": 54, "readout_examples": 54, "reasoner_examples": 0, "step_tokens": 37151, "chain_tokens": 0, "micro_batches": 5, "tok_per_s": 4921.436857058764, "elapsed_min": 0.12581352790196737, "eta_min": 2.3904570301373798, "chain_loss": null, "readout_loss": 2.9318235466357523, "readout_argmax_matches_target": 0.4444444444444444, "readout_mean_top_prob": 0.8359314054250717, "readout_target_entropy": 0.07517363943366541, "letter_loss": null, "letter_top_prob": null, "mem_gb": 42.871113216}
{"step": 2, "of": 20, "lr": 9.931806517013612e-05, "grad_norm": 37.132781982421875, "examples": 78, "readout_examples": 78, "reasoner_examples": 0, "step_tokens": 38032, "chain_tokens": 0, "micro_batches": 5, "tok_per_s": 6168.388031736784, "elapsed_min": 0.20314059257507325, "eta_min": 1.828265333175659, "chain_loss": null, "readout_loss": 2.800655071254322, "readout_argmax_matches_target": 0.41025641025641024, "readout_mean_top_prob": 0.8179513659232702, "readout_target_entropy": 0.09658416920604232, "letter_loss": null, "letter_top_prob": null, "mem_gb": 43.257649664}
{"step": 3, "of": 20, "lr": 9.729086208503174e-05, "grad_norm": 7.457327365875244, "examples": 64, "readout_examples": 64, "reasoner_examples": 0, "step_tokens": 36426, "chain_tokens": 0, "micro_batches": 5, "tok_per_s": 6769.63216999632, "elapsed_min": 0.2747785925865173, "eta_min": 1.5570786913235983, "chain_loss": null, "readout_loss": 1.44439245526155, "readout_argmax_matches_target": 0.40625, "readout_mean_top_prob": 0.5706920898519456, "readout_target_entropy": 0.09218526375479996, "letter_loss": null, "letter_top_prob": null, "mem_gb": 43.257649664}
{"step": 10, "of": 20, "lr": 5.4128967273616625e-05, "grad_norm": 5.378308296203613, "examples": 41, "readout_examples": 41, "reasoner_examples": 0, "step_tokens": 35004, "chain_tokens": 0, "micro_batches": 5, "tok_per_s": 7487.791877797799, "elapsed_min": 0.8286510586738587, "eta_min": 0.8286510586738587, "chain_loss": null, "readout_loss": 1.1393815309899609, "readout_argmax_matches_target": 0.5609756097560976, "readout_mean_top_prob": 0.5089296167943536, "readout_target_entropy": 0.04584710990510336, "letter_loss": null, "letter_top_prob": null, "mem_gb": 44.100466176}
{"step": 20, "of": 20, "lr": 0.0, "grad_norm": 7.187343597412109, "examples": 41, "readout_examples": 41, "reasoner_examples": 0, "step_tokens": 31234, "chain_tokens": 0, "micro_batches": 4, "tok_per_s": 7897.066086848658, "elapsed_min": 1.5677003582318625, "eta_min": 0.0, "chain_loss": null, "readout_loss": 1.195220783716295, "readout_argmax_matches_target": 0.4878048780487805, "readout_mean_top_prob": 0.5573934465646744, "readout_target_entropy": 0.009950448917906458, "letter_loss": null, "letter_top_prob": null, "mem_gb": 44.301522944}
{"dev_n": 500, "dev_acc": 0.536, "dev_nll": 1.0756603895192967, "dev_ece15": 0.02956324663758277}
{"dev_n": 500, "dev_acc": 0.536, "dev_nll": 1.0756603895192967, "dev_ece15": 0.02956324663758277, "v2_options": {"readout_pool": "all", "subset": "data/sweep_ids.txt", "min_correct": 2, "second_chain_min": 3, "rep_max": 20, "post_soft": true, "eval_order_frac": 0.5, "drop_recall": true, "strip_cw": true}, "speed_options": {"liger": false, "ckpt_above": 8192, "micro_tokens": 8192, "no_probe": true}, "mode": "readout_only", "alpha": 0.5, "lr": 0.0001, "rank": 32, "lora_alpha": 64, "seed": 17, "examples": 1500, "steps_done": 20, "steps_planned": 20, "stopped_at_deadline": false, "train_tokens": 742814, "train_seconds": 94.12405157089233, "tok_per_s": 7891.861725061075, "counts": {"readout_from_votes": 259, "readout_eval_order": 720, "readout_gold_only": 1241}, "reasoner_examples": 0, "readout_examples": 1500}
{"time": "2026-11-01T00:29:17Z", "event": "done", "stage": "2b_sweep_1e-4"}
{"time": "2026-11-01T00:29:34Z", "event": "start", "stage": "2b_sweep_2e-4"}
{"examples": 1500, "tokens": 742814, "readout_from_votes": 259, "readout_eval_order": 720, "readout_gold_only": 1241}

Loading weights:   0%|          | 0/381 [00:00<?, ?it/s]
Loading weights:  97%|█████████▋| 370/381 [00:00<00:00, 3652.26it/s]
Loading weights: 100%|██████████| 381/381 [00:00<00:00, 3690.77it/s]
trainable params: 50,233,344 || all params: 2,566,989,824 || trainable%: 1.9569
/workspace/decision-reasoner/runs/v2_2b/train.py:215: UserWarning: Converting a tensor with requires_grad=True to a scalar may lead to unexpected behavior.
Consider using tensor.detach() first. (Triggered internally at /__w/pytorch/pytorch/torch/csrc/autograd/generated/python_variable_methods.cpp:822.)
  stats["readout_loss"] += float(loss)
{"step": 1, "of": 20, "lr": 0.0002, "grad_norm": 47.8929557800293, "examples": 54, "readout_examples": 54, "reasoner_examples": 0, "step_tokens": 37151, "chain_tokens": 0, "micro_batches": 5, "tok_per_s": 6951.826981527774, "elapsed_min": 0.08906771341959635, "eta_min": 1.6922865549723307, "chain_loss": null, "readout_loss": 2.9318235466357523, "readout_argmax_matches_target": 0.4444444444444444, "readout_mean_top_prob": 0.8359314054250717, "readout_target_entropy": 0.07517363943366541, "letter_loss": null, "letter_top_prob": null, "mem_gb": 42.871113216}
{"step": 2, "of": 20, "lr": 0.00019863613034027224, "grad_norm": 37.131141662597656, "examples": 78, "readout_examples": 78, "reasoner_examples": 0, "step_tokens": 38032, "chain_tokens": 0, "micro_batches": 5, "tok_per_s": 7531.746802993979, "elapsed_min": 0.16636910835901897, "eta_min": 1.4973219752311706, "chain_loss": null, "readout_loss": 2.800655071254322, "readout_argmax_matches_target": 0.41025641025641024, "readout_mean_top_prob": 0.8179513659232702, "readout_target_entropy": 0.09658416920604232, "letter_loss": null, "letter_top_prob": null, "mem_gb": 43.257649664}
{"step": 3, "of": 20, "lr": 0.00019458172417006347, "grad_norm": 4.937190532684326, "examples": 64, "readout_examples": 64, "reasoner_examples": 0, "step_tokens": 36426, "chain_tokens": 0, "micro_batches": 5, "tok_per_s": 7814.172991689074, "elapsed_min": 0.23804822365442913, "eta_min": 1.348939934041765, "chain_loss": null, "readout_loss": 1.3845821488648653, "readout_argmax_matches_target": 0.359375, "readout_mean_top_prob": 0.4604114727117121, "readout_target_entropy": 0.09218526375479996, "letter_loss": null, "letter_top_prob": null, "mem_gb": 43.257649664}
{"step": 10, "of": 20, "lr": 0.00010825793454723325, "grad_norm": 3.8807525634765625, "examples": 41, "readout_examples": 41, "reasoner_examples": 0, "step_tokens": 35004, "chain_tokens": 0, "micro_batches": 5, "tok_per_s": 7926.88821371395, "elapsed_min": 0.7827493588129679, "eta_min": 0.7827493588129679, "chain_loss": null, "readout_loss": 1.1432294176846016, "readout_argmax_matches_target": 0.4634146341463415, "readout_mean_top_prob": 0.501617315702322, "readout_target_entropy": 0.04584710990510336, "letter_loss": null, "letter_top_prob": null, "mem_gb": 44.100466176}
{"step": 20, "of": 20, "lr": 0.0, "grad_norm": 6.041870594024658, "examples": 41, "readout_examples": 41, "reasoner_examples": 0, "step_tokens": 31234, "chain_tokens": 0, "micro_batches": 4, "tok_per_s": 8138.003964860756, "elapsed_min": 1.5212862253189088, "eta_min": 0.0, "chain_loss": null, "readout_loss": 1.131969743203826, "readout_argmax_matches_target": 0.5121951219512195, "readout_mean_top_prob": 0.5944744010524052, "readout_target_entropy": 0.009950448917906458, "letter_loss": null, "letter_top_prob": null, "mem_gb": 44.301522944}
{"dev_n": 500, "dev_acc": 0.57, "dev_nll": 1.0392292883340268, "dev_ece15": 0.06858765184879302}
{"dev_n": 500, "dev_acc": 0.57, "dev_nll": 1.0392292883340268, "dev_ece15": 0.06858765184879302, "v2_options": {"readout_pool": "all", "subset": "data/sweep_ids.txt", "min_correct": 2, "second_chain_min": 3, "rep_max": 20, "post_soft": true, "eval_order_frac": 0.5, "drop_recall": true, "strip_cw": true}, "speed_options": {"liger": false, "ckpt_above": 8192, "micro_tokens": 8192, "no_probe": true}, "mode": "readout_only", "alpha": 0.5, "lr": 0.0002, "rank": 32, "lora_alpha": 64, "seed": 17, "examples": 1500, "steps_done": 20, "steps_planned": 20, "stopped_at_deadline": false, "train_tokens": 742814, "train_seconds": 91.34080481529236, "tok_per_s": 8132.3347380407295, "counts": {"readout_from_votes": 259, "readout_eval_order": 720, "readout_gold_only": 1241}, "reasoner_examples": 0, "readout_examples": 1500}
{"time": "2026-11-01T00:31:41Z", "event": "done", "stage": "2b_sweep_2e-4"}
{"time": "2026-11-01T00:31:57Z", "event": "start", "stage": "2b_sweep_3e-4"}
{"examples": 1500, "tokens": 742814, "readout_from_votes": 259, "readout_eval_order": 720, "readout_gold_only": 1241}

Loading weights:   0%|          | 0/381 [00:00<?, ?it/s]
Loading weights:  96%|█████████▋| 367/381 [00:00<00:00, 3640.41it/s]
Loading weights: 100%|██████████| 381/381 [00:00<00:00, 3687.44it/s]
trainable params: 50,233,344 || all params: 2,566,989,824 || trainable%: 1.9569
/workspace/decision-reasoner/runs/v2_2b/train.py:215: UserWarning: Converting a tensor with requires_grad=True to a scalar may lead to unexpected behavior.
Consider using tensor.detach() first. (Triggered internally at /__w/pytorch/pytorch/torch/csrc/autograd/generated/python_variable_methods.cpp:822.)
  stats["readout_loss"] += float(loss)
{"step": 1, "of": 20, "lr": 0.0003, "grad_norm": 47.89808654785156, "examples": 54, "readout_examples": 54, "reasoner_examples": 0, "step_tokens": 37151, "chain_tokens": 0, "micro_batches": 5, "tok_per_s": 3624.0760969846383, "elapsed_min": 0.1708527406056722, "eta_min": 3.246202071507772, "chain_loss": null, "readout_loss": 2.9318235466357523, "readout_argmax_matches_target": 0.4444444444444444, "readout_mean_top_prob": 0.8359314054250717, "readout_target_entropy": 0.07517363943366541, "letter_loss": null, "letter_top_prob": null, "mem_gb": 42.871113216}
{"step": 2, "of": 20, "lr": 0.00029795419551040833, "grad_norm": 37.137794494628906, "examples": 78, "readout_examples": 78, "reasoner_examples": 0, "step_tokens": 38032, "chain_tokens": 0, "micro_batches": 5, "tok_per_s": 5046.783967723153, "elapsed_min": 0.24828683137893676, "eta_min": 2.234581482410431, "chain_loss": null, "readout_loss": 2.800655071254322, "readout_argmax_matches_target": 0.41025641025641024, "readout_mean_top_prob": 0.8179513659232702, "readout_target_entropy": 0.09658416920604232, "letter_loss": null, "letter_top_prob": null, "mem_gb": 43.257649664}
{"step": 3, "of": 20, "lr": 0.00029187258625509513, "grad_norm": 4.618750095367432, "examples": 64, "readout_examples": 64, "reasoner_examples": 0, "step_tokens": 36426, "chain_tokens": 0, "micro_batches": 5, "tok_per_s": 5813.482581852887, "elapsed_min": 0.31997171640396116, "eta_min": 1.8131730596224467, "chain_loss": null, "readout_loss": 1.4517000822816044, "readout_argmax_matches_target": 0.328125, "readout_mean_top_prob": 0.43882185546681285, "readout_target_entropy": 0.09218526375479996, "letter_loss": null, "letter_top_prob": null, "mem_gb": 43.257649664}
{"step": 10, "of": 20, "lr": 0.00016238690182084986, "grad_norm": 2.8892664909362793, "examples": 41, "readout_examples": 41, "reasoner_examples": 0, "step_tokens": 35004, "chain_tokens": 0, "micro_batches": 5, "tok_per_s": 7180.066390452092, "elapsed_min": 0.8641656398773193, "eta_min": 0.8641656398773193, "chain_loss": null, "readout_loss": 1.2214370006468238, "readout_argmax_matches_target": 0.34146341463414637, "readout_mean_top_prob": 0.45861235969677205, "readout_target_entropy": 0.04584710990510336, "letter_loss": null, "letter_top_prob": null, "mem_gb": 44.100466176}
{"step": 20, "of": 20, "lr": 0.0, "grad_norm": 7.878785133361816, "examples": 41, "readout_examples": 41, "reasoner_examples": 0, "step_tokens": 31234, "chain_tokens": 0, "micro_batches": 4, "tok_per_s": 7723.178792930174, "elapsed_min": 1.602997116247813, "eta_min": 0.0, "chain_loss": null, "readout_loss": 1.3647387646047808, "readout_argmax_matches_target": 0.5609756097560976, "readout_mean_top_prob": 0.6429943914820508, "readout_target_entropy": 0.009950448917906458, "letter_loss": null, "letter_top_prob": null, "mem_gb": 44.301522944}
{"dev_n": 500, "dev_acc": 0.586, "dev_nll": 1.0575983835346996, "dev_ece15": 0.08453042723238469}
{"dev_n": 500, "dev_acc": 0.586, "dev_nll": 1.0575983835346996, "dev_ece15": 0.08453042723238469, "v2_options": {"readout_pool": "all", "subset": "data/sweep_ids.txt", "min_correct": 2, "second_chain_min": 3, "rep_max": 20, "post_soft": true, "eval_order_frac": 0.5, "drop_recall": true, "strip_cw": true}, "speed_options": {"liger": false, "ckpt_above": 8192, "micro_tokens": 8192, "no_probe": true}, "mode": "readout_only", "alpha": 0.5, "lr": 0.0003, "rank": 32, "lora_alpha": 64, "seed": 17, "examples": 1500, "steps_done": 20, "steps_planned": 20, "stopped_at_deadline": false, "train_tokens": 742814, "train_seconds": 96.24385499954224, "tok_per_s": 7718.040803784647, "counts": {"readout_from_votes": 259, "readout_eval_order": 720, "readout_gold_only": 1241}, "reasoner_examples": 0, "readout_examples": 1500}
{"time": "2026-11-01T00:34:09Z", "event": "done", "stage": "2b_sweep_3e-4"}
{"time": "2026-11-01T00:34:29Z", "event": "chosen", "stage": "2b_lr=2e-4"}
{"time": "2026-11-01T00:34:29Z", "event": "start", "stage": "2b_train_joint"}
{"examples": 52968, "tokens": 46011760, "readout_from_votes": 7320, "readout_eval_order": 20989, "readout_gold_fallback": 13, "readout_gold_only": 34661, "reasoner_chains_dropped_invented_recall": 2193, "reasoner_chains_dropped_repetition": 11, "reasoner_cw_stripped": 1532, "reasoner_soft_letter": 10972, "reasoner_examples": 10974, "reasoner_items": 5779, "reasoner_no_correct_chain": 976, "reasoner_too_few_correct": 257, "reasoner_letter_not_found": 2}

Loading weights:   0%|          | 0/381 [00:00<?, ?it/s]
Loading weights:  97%|█████████▋| 369/381 [00:00<00:00, 3668.84it/s]
Loading weights: 100%|██████████| 381/381 [00:00<00:00, 3725.69it/s]
trainable params: 50,233,344 || all params: 2,566,989,824 || trainable%: 1.9569
/workspace/decision-reasoner/runs/v2_2b/train.py:231: UserWarning: Converting a tensor with requires_grad=True to a scalar may lead to unexpected behavior.
Consider using tensor.detach() first. (Triggered internally at /__w/pytorch/pytorch/torch/csrc/autograd/generated/python_variable_methods.cpp:822.)
  stats["chain_loss"] += float(loss)
{"probe_tok_per_s": 7681.167337961819, "fit_tokens": 108072236.28579576, "have_tokens": 46011760}
{"step": 1, "of": 1265, "lr": 5.405405405405406e-06, "grad_norm": 21.291322708129883, "examples": 98, "readout_examples": 86, "reasoner_examples": 12, "step_tokens": 37340, "chain_tokens": 14512, "micro_batches": 5, "tok_per_s": 7657.8396784100005, "elapsed_min": 0.08126748005549113, "eta_min": 102.72209479014079, "chain_loss": 0.5624861195683479, "readout_loss": 1.5472678220658198, "readout_argmax_matches_target": 0.5697674418604651, "readout_mean_top_prob": 0.8610499924005464, "readout_target_entropy": 0.17835095004979953, "letter_loss": 0.3118489583333333, "letter_top_prob": 1.0, "mem_gb": 48.998109184}
{"step": 2, "of": 1265, "lr": 1.0810810810810812e-05, "grad_norm": 36.30150604248047, "examples": 71, "readout_examples": 65, "reasoner_examples": 6, "step_tokens": 38488, "chain_tokens": 5917, "micro_batches": 5, "tok_per_s": 7931.84193840338, "elapsed_min": 0.15933247407277426, "eta_min": 100.61845737695694, "chain_loss": 0.4291130354007085, "readout_loss": 2.762804669116248, "readout_argmax_matches_target": 0.36923076923076925, "readout_mean_top_prob": 0.8421665833546566, "readout_target_entropy": 0.14026222870900082, "letter_loss": 1.0132764316495013e-06, "letter_top_prob": 0.9999989867210388, "mem_gb": 48.998109184}
{"step": 3, "of": 1265, "lr": 1.6216216216216218e-05, "grad_norm": 35.765933990478516, "examples": 50, "readout_examples": 37, "reasoner_examples": 13, "step_tokens": 35675, "chain_tokens": 10508, "micro_batches": 5, "tok_per_s": 8156.948127404515, "elapsed_min": 0.2278282642364502, "eta_min": 95.83975648880005, "chain_loss": 0.4602410403581766, "readout_loss": 2.863382283984597, "readout_argmax_matches_target": 0.43243243243243246, "readout_mean_top_prob": 0.8327744135985503, "readout_target_entropy": 0.030366834756490345, "letter_loss": 6.418936537172087e-07, "letter_top_prob": 0.9999993581038255, "mem_gb": 48.998109184}
{"step": 10, "of": 1265, "lr": 5.405405405405406e-05, "grad_norm": 3.7627806663513184, "examples": 57, "readout_examples": 48, "reasoner_examples": 9, "step_tokens": 38808, "chain_tokens": 7653, "micro_batches": 5, "tok_per_s": 8391.081828464214, "elapsed_min": 0.7353918671607971, "eta_min": 92.29167932868003, "chain_loss": 0.4526560935709212, "readout_loss": 1.1054809419438243, "readout_argmax_matches_target": 0.4791666666666667, "readout_mean_top_prob": 0.46926008475323516, "readout_target_entropy": 0.10648553636080275, "letter_loss": 0.00042358011207473584, "letter_top_prob": 0.9995768136448331, "mem_gb": 48.998109184}
{"step": 20, "of": 1265, "lr": 0.00010810810810810812, "grad_norm": 3.038816213607788, "examples": 56, "readout_examples": 50, "reasoner_examples": 6, "step_tokens": 35212, "chain_tokens": 4418, "micro_batches": 5, "tok_per_s": 7823.945561328473, "elapsed_min": 1.5543670694033305, "eta_min": 96.75935007035733, "chain_loss": 0.4627371405561765, "readout_loss": 0.9923751824349165, "readout_argmax_matches_target": 0.6, "readout_mean_top_prob": 0.556278792321682, "readout_target_entropy": 0.09835200190544129, "letter_loss": 0.004409538271526496, "letter_top_prob": 0.9956045945485433, "mem_gb": 48.998109184}
{"step": 30, "of": 1265, "lr": 0.00016216216216216218, "grad_norm": 4.995848178863525, "examples": 9, "readout_examples": 2, "reasoner_examples": 7, "step_tokens": 39291, "chain_tokens": 31059, "micro_batches": 5, "tok_per_s": 7753.268158440445, "elapsed_min": 2.345843021074931, "eta_min": 96.570537700918, "chain_loss": 0.447035299880164, "readout_loss": 0.9334807097911835, "readout_argmax_matches_target": 1.0, "readout_mean_top_prob": 0.3945961445569992, "readout_target_entropy": 0.0, "letter_loss": 0.35150652752989636, "letter_top_prob": 0.9582420928137643, "mem_gb": 48.998109184}
{"step": 40, "of": 1265, "lr": 0.00019999705480948108, "grad_norm": 3.5189712047576904, "examples": 47, "readout_examples": 42, "reasoner_examples": 5, "step_tokens": 33677, "chain_tokens": 12959, "micro_batches": 5, "tok_per_s": 7934.828554143678, "elapsed_min": 3.056113080183665, "eta_min": 93.59346308062474, "chain_loss": 0.4505480766296387, "readout_loss": 1.0507358697553475, "readout_argmax_matches_target": 0.6190476190476191, "readout_mean_top_prob": 0.6070795648154759, "readout_target_entropy": 0.05872804636047, "letter_loss": 0.1668595630123491, "letter_top_prob": 0.9987679123878479, "mem_gb": 48.998109184}
{"step": 50, "of": 1265, "lr": 0.0001999447006928918, "grad_norm": 4.770105838775635, "examples": 18, "readout_examples": 7, "reasoner_examples": 11, "step_tokens": 33457, "chain_tokens": 17652, "micro_batches": 4, "tok_per_s": 7859.190984200391, "elapsed_min": 3.861177746454875, "eta_min": 93.82661923885347, "chain_loss": 0.4047818861224435, "readout_loss": 1.5986695289611816, "readout_argmax_matches_target": 0.42857142857142855, "readout_mean_top_prob": 0.38174386961119516, "readout_target_entropy": 0.0, "letter_loss": 0.0033463287479455835, "letter_top_prob": 0.9966839335181497, "mem_gb": 48.998109184}
{"step": 60, "of": 1265, "lr": 0.0001998269373372708, "grad_norm": 3.585799217224121, "examples": 31, "readout_examples": 21, "reasoner_examples": 10, "step_tokens": 34793, "chain_tokens": 11843, "micro_batches": 5, "tok_per_s": 7943.258024414499, "elapsed_min": 4.5870073994000755, "eta_min": 92.12239860461818, "chain_loss": 0.4812329590320587, "readout_loss": 1.1337367565858931, "readout_argmax_matches_target": 0.5714285714285714, "readout_mean_top_prob": 0.46096934378147125, "readout_target_entropy": 0.0, "letter_loss": 0.01958112744614482, "letter_top_prob": 0.9808444023132324, "mem_gb": 48.998109184}
{"step": 70, "of": 1265, "lr": 0.00019964384181335237, "grad_norm": 3.37990140914917, "examples": 19, "readout_examples": 11, "reasoner_examples": 8, "step_tokens": 38459, "chain_tokens": 14559, "micro_batches": 6, "tok_per_s": 7903.43740527931, "elapsed_min": 5.382103315989176, "eta_min": 91.88019232295808, "chain_loss": 0.5066429134458303, "readout_loss": 1.3662986972115256, "readout_argmax_matches_target": 0.45454545454545453, "readout_mean_top_prob": 0.43790887973525305, "readout_target_entropy": 0.0, "letter_loss": 0.12034690557629801, "letter_top_prob": 0.9911489486694336, "mem_gb": 48.998109184}
{"step": 80, "of": 1265, "lr": 0.00019939553394878926, "grad_norm": 6.021086692810059, "examples": 81, "readout_examples": 72, "reasoner_examples": 9, "step_tokens": 40638, "chain_tokens": 9807, "micro_batches": 6, "tok_per_s": 7953.065316964369, "elapsed_min": 6.1452594757080075, "eta_min": 91.02665598392487, "chain_loss": 0.47117888927459717, "readout_loss": 1.1104155298995062, "readout_argmax_matches_target": 0.5833333333333334, "readout_mean_top_prob": 0.6829469458510479, "readout_target_entropy": 0.07735397869853639, "letter_loss": 0.006613435741100047, "letter_top_prob": 0.9934494031800164, "mem_gb": 48.998109184}
{"step": 90, "of": 1265, "lr": 0.00019908217624973099, "grad_norm": 15.1987886428833, "examples": 14, "readout_examples": 10, "reasoner_examples": 4, "step_tokens": 35018, "chain_tokens": 19031, "micro_batches": 5, "tok_per_s": 7946.038525099906, "elapsed_min": 6.932586574554444, "eta_min": 90.50876916779411, "chain_loss": 0.4527457654476166, "readout_loss": 1.455775325745344, "readout_argmax_matches_target": 0.4, "readout_mean_top_prob": 0.6944580852985383, "readout_target_entropy": 0.0, "letter_loss": 0.5123660237622971, "letter_top_prob": 0.9993530213832855, "mem_gb": 48.998109184}
{"step": 100, "of": 1265, "lr": 0.00019870397379447073, "grad_norm": 11.548027038574219, "examples": 51, "readout_examples": 44, "reasoner_examples": 7, "step_tokens": 37211, "chain_tokens": 6890, "micro_batches": 5, "tok_per_s": 7997.098835683234, "elapsed_min": 7.670769270261129, "eta_min": 89.36446199854215, "chain_loss": 0.5163723485810416, "readout_loss": 1.0005323693727737, "readout_argmax_matches_target": 0.6818181818181818, "readout_mean_top_prob": 0.755150977183472, "readout_target_entropy": 0.08628951775079424, "letter_loss": 0.004628722113661102, "letter_top_prob": 0.9954304354531425, "mem_gb": 48.998109184}
{"step": 110, "of": 1265, "lr": 0.0001982611740992313, "grad_norm": 4.961239337921143, "examples": 32, "readout_examples": 15, "reasoner_examples": 17, "step_tokens": 35450, "chain_tokens": 15900, "micro_batches": 5, "tok_per_s": 7951.240054877178, "elapsed_min": 8.483758449554443, "eta_min": 89.07946372032166, "chain_loss": 0.4308522545239505, "readout_loss": 1.014942733446757, "readout_argmax_matches_target": 0.7333333333333333, "readout_mean_top_prob": 0.4876279403765996, "readout_target_entropy": 0.0, "letter_loss": 0.016166174864766243, "letter_top_prob": 0.9840582854607526, "mem_gb": 48.998109184}
{"step": 120, "of": 1265, "lr": 0.00019775406695617679, "grad_norm": 2.803405284881592, "examples": 83, "readout_examples": 81, "reasoner_examples": 2, "step_tokens": 34359, "chain_tokens": 8475, "micro_batches": 5, "tok_per_s": 7973.193172862414, "elapsed_min": 9.254985014597574, "eta_min": 88.30798201428519, "chain_loss": 0.4511786103248596, "readout_loss": 0.9156633419456122, "readout_argmax_matches_target": 0.6049382716049383, "readout_mean_top_prob": 0.6063305444923448, "readout_target_entropy": 0.13503729818779744, "letter_loss": 0.016417873557657003, "letter_top_prob": 0.9837702810764313, "mem_gb": 48.998109184}
{"step": 130, "of": 1265, "lr": 0.00019718298424375706, "grad_norm": 9.634334564208984, "examples": 15, "readout_examples": 6, "reasoner_examples": 9, "step_tokens": 33172, "chain_tokens": 16471, "micro_batches": 5, "tok_per_s": 7978.8084132336035, "elapsed_min": 9.988151848316193, "eta_min": 87.20424882952985, "chain_loss": 0.41346968130932915, "readout_loss": 1.962234598894914, "readout_argmax_matches_target": 0.16666666666666666, "readout_mean_top_prob": 0.557808518409729, "readout_target_entropy": 0.0, "letter_loss": 0.033566793219910726, "letter_top_prob": 0.9693252709176805, "mem_gb": 48.998109184}
{"step": 140, "of": 1265, "lr": 0.00019654829970950835, "grad_norm": 3.4977359771728516, "examples": 15, "readout_examples": 11, "reasoner_examples": 4, "step_tokens": 35031, "chain_tokens": 16734, "micro_batches": 5, "tok_per_s": 7960.22396548233, "elapsed_min": 10.741230444113414, "eta_min": 86.31345892591136, "chain_loss": 0.5036605224013329, "readout_loss": 0.7937580089677464, "readout_argmax_matches_target": 0.6363636363636364, "readout_mean_top_prob": 0.6809038736603477, "readout_target_entropy": 0.0, "letter_loss": 0.43927349557634443, "letter_top_prob": 0.9742569327354431, "mem_gb": 48.998109184}
{"step": 150, "of": 1265, "lr": 0.00019585042872545264, "grad_norm": 3.4093151092529297, "examples": 84, "readout_examples": 82, "reasoner_examples": 2, "step_tokens": 38191, "chain_tokens": 13626, "micro_batches": 5, "tok_per_s": 7947.663705316984, "elapsed_min": 11.511005421479544, "eta_min": 85.5651402996646, "chain_loss": 0.35993171483278275, "readout_loss": 0.7360942773748098, "readout_argmax_matches_target": 0.6829268292682927, "readout_mean_top_prob": 0.7025086091059011, "readout_target_entropy": 0.03924543842128137, "letter_loss": 0.18534580990672112, "letter_top_prob": 0.8355291485786438, "mem_gb": 48.998109184}
{"step": 160, "of": 1265, "lr": 0.00019508982801625557, "grad_norm": 5.306968688964844, "examples": 37, "readout_examples": 31, "reasoner_examples": 6, "step_tokens": 37452, "chain_tokens": 14841, "micro_batches": 5, "tok_per_s": 7936.4287258699005, "elapsed_min": 12.298522845904033, "eta_min": 84.93667340452473, "chain_loss": 0.3951815292239189, "readout_loss": 1.0523923106731907, "readout_argmax_matches_target": 0.4838709677419355, "readout_mean_top_prob": 0.5545561222299453, "readout_target_entropy": 0.019768621442058393, "letter_loss": 0.003142542819356701, "letter_top_prob": 0.9968806207180023, "mem_gb": 48.998109184}
{"step": 170, "of": 1265, "lr": 0.0001942669953603211, "grad_norm": 3.194342613220215, "examples": 79, "readout_examples": 77, "reasoner_examples": 2, "step_tokens": 38479, "chain_tokens": 12418, "micro_batches": 5, "tok_per_s": 7959.110151964575, "elapsed_min": 13.04045486052831, "eta_min": 83.99587101340293, "chain_loss": 0.3641641288995743, "readout_loss": 1.176336587176888, "readout_argmax_matches_target": 0.5194805194805194, "readout_mean_top_prob": 0.5855060382322832, "readout_target_entropy": 0.08427876001828677, "letter_loss": 0.287887807469815, "letter_top_prob": 0.9706372320652008, "mem_gb": 48.998109184}
{"step": 180, "of": 1265, "lr": 0.00019338246926401827, "grad_norm": 3.20951509475708, "examples": 43, "readout_examples": 37, "reasoner_examples": 6, "step_tokens": 35240, "chain_tokens": 11042, "micro_batches": 5, "tok_per_s": 7966.0282139056135, "elapsed_min": 13.793086556593577, "eta_min": 83.14166063280017, "chain_loss": 0.49818134928743046, "readout_loss": 0.9017678243485657, "readout_argmax_matches_target": 0.5675675675675675, "readout_mean_top_prob": 0.5680735219974775, "readout_target_entropy": 0.04443542699556093, "letter_loss": 0.13264506065752357, "letter_top_prob": 0.9936838547388712, "mem_gb": 48.998109184}
{"step": 190, "of": 1265, "lr": 0.00019243682860925338, "grad_norm": 11.065648078918457, "examples": 35, "readout_examples": 28, "reasoner_examples": 7, "step_tokens": 37341, "chain_tokens": 17480, "micro_batches": 5, "tok_per_s": 7981.268798438459, "elapsed_min": 14.515010448296865, "eta_min": 82.124401220627, "chain_loss": 0.5978791138955525, "readout_loss": 0.9620228488596955, "readout_argmax_matches_target": 0.6071428571428571, "readout_mean_top_prob": 0.6735226990921157, "readout_target_entropy": 0.017871513962745667, "letter_loss": 0.002560372805289392, "letter_top_prob": 0.9974506497383118, "mem_gb": 48.998109184}
{"step": 200, "of": 1265, "lr": 0.0001914306922746183, "grad_norm": 9.302431106567383, "examples": 33, "readout_examples": 22, "reasoner_examples": 11, "step_tokens": 35672, "chain_tokens": 17751, "micro_batches": 5, "tok_per_s": 7970.264081305359, "elapsed_min": 15.291392115751902, "eta_min": 81.42666301637888, "chain_loss": 0.5028096017512408, "readout_loss": 1.0210751932622357, "readout_argmax_matches_target": 0.5454545454545454, "readout_mean_top_prob": 0.5536447153850035, "readout_target_entropy": 0.09119014712897214, "letter_loss": 0.06723745470199818, "letter_top_prob": 0.9952863292260603, "mem_gb": 48.998109184}
{"step": 210, "of": 1265, "lr": 0.0001903647187303626, "grad_norm": 7.326548099517822, "examples": 35, "readout_examples": 24, "reasoner_examples": 11, "step_tokens": 38888, "chain_tokens": 14956, "micro_batches": 6, "tok_per_s": 7993.41357272219, "elapsed_min": 16.02414481639862, "eta_min": 80.5022513395264, "chain_loss": 0.43198015879500995, "readout_loss": 1.10798840985323, "readout_argmax_matches_target": 0.5, "readout_mean_top_prob": 0.5220442221810421, "readout_target_entropy": 0.0, "letter_loss": 0.018638436936519363, "letter_top_prob": 0.9816458658738569, "mem_gb": 48.998109184}
{"step": 220, "of": 1265, "lr": 0.00018923960560745495, "grad_norm": 3.7683262825012207, "examples": 19, "readout_examples": 11, "reasoner_examples": 8, "step_tokens": 33025, "chain_tokens": 11410, "micro_batches": 5, "tok_per_s": 8002.952187645739, "elapsed_min": 16.761889469623565, "eta_min": 79.61897498071194, "chain_loss": 0.3955907691270113, "readout_loss": 0.9780506329102949, "readout_argmax_matches_target": 0.5454545454545454, "readout_mean_top_prob": 0.61612269282341, "readout_target_entropy": 0.07886938615278764, "letter_loss": 0.09476142513449304, "letter_top_prob": 0.991264320909977, "mem_gb": 48.998109184}
{"step": 230, "of": 1265, "lr": 0.00018805608924101551, "grad_norm": 4.429680824279785, "examples": 18, "readout_examples": 8, "reasoner_examples": 10, "step_tokens": 36811, "chain_tokens": 24231, "micro_batches": 5, "tok_per_s": 8002.740285690973, "elapsed_min": 17.51878044207891, "eta_min": 78.8345119893551, "chain_loss": 0.46571054607629775, "readout_loss": 0.8100521834567189, "readout_argmax_matches_target": 0.75, "readout_mean_top_prob": 0.6581276617944241, "readout_target_entropy": 0.0027693229203578085, "letter_loss": 0.06171538841444999, "letter_top_prob": 0.989974969625473, "mem_gb": 48.998109184}
{"step": 240, "of": 1265, "lr": 0.00018681494418841825, "grad_norm": 4.877554416656494, "examples": 33, "readout_examples": 27, "reasoner_examples": 6, "step_tokens": 35304, "chain_tokens": 8714, "micro_batches": 5, "tok_per_s": 8019.371817393245, "elapsed_min": 18.27233728567759, "eta_min": 78.03810715758138, "chain_loss": 0.405504730840524, "readout_loss": 0.6646551796131663, "readout_argmax_matches_target": 0.6666666666666666, "readout_mean_top_prob": 0.6356907829090401, "readout_target_entropy": 0.0, "letter_loss": 0.002266594802980156, "letter_top_prob": 0.9977389276027679, "mem_gb": 48.998109184}
{"step": 250, "of": 1265, "lr": 0.0001855169827223785, "grad_norm": 8.534375190734863, "examples": 41, "readout_examples": 38, "reasoner_examples": 3, "step_tokens": 38593, "chain_tokens": 9559, "micro_batches": 5, "tok_per_s": 8018.649677997189, "elapsed_min": 19.044316204388938, "eta_min": 77.31992378981909, "chain_loss": 0.4887751042842865, "readout_loss": 0.8828030819024303, "readout_argmax_matches_target": 0.6578947368421053, "readout_mean_top_prob": 0.7044179031723424, "readout_target_entropy": 0.057794769343576934, "letter_loss": 0.018580345281710226, "letter_top_prob": 0.9817383289337158, "mem_gb": 48.998109184}
{"step": 260, "of": 1265, "lr": 0.00018416305429935775, "grad_norm": 5.730326175689697, "examples": 31, "readout_examples": 16, "reasoner_examples": 15, "step_tokens": 35160, "chain_tokens": 18260, "micro_batches": 4, "tok_per_s": 8006.59863231727, "elapsed_min": 19.820372414588928, "eta_min": 76.61336260254568, "chain_loss": 0.46120381355285645, "readout_loss": 0.8106653401628137, "readout_argmax_matches_target": 0.75, "readout_mean_top_prob": 0.5906499531120062, "readout_target_entropy": 7.29479143046774e-05, "letter_loss": 0.08322643022596216, "letter_top_prob": 0.981232217947642, "mem_gb": 48.998109184}
{"step": 270, "of": 1265, "lr": 0.0001827540450036329, "grad_norm": 3.6267623901367188, "examples": 21, "readout_examples": 14, "reasoner_examples": 7, "step_tokens": 38157, "chain_tokens": 21429, "micro_batches": 5, "tok_per_s": 8019.252182631179, "elapsed_min": 20.55646789073944, "eta_min": 75.75439093068793, "chain_loss": 0.4161131807735988, "readout_loss": 0.9640071147254535, "readout_argmax_matches_target": 0.5714285714285714, "readout_mean_top_prob": 0.5509189601455416, "readout_target_entropy": 0.05401360349995749, "letter_loss": 0.15269374075744832, "letter_top_prob": 0.9880248308181763, "mem_gb": 48.998109184}
{"step": 280, "of": 1265, "lr": 0.00018129087696739496, "grad_norm": 5.047298908233643, "examples": 29, "readout_examples": 19, "reasoner_examples": 10, "step_tokens": 34892, "chain_tokens": 12123, "micro_batches": 5, "tok_per_s": 8023.4858143138335, "elapsed_min": 21.29916314284007, "eta_min": 74.92741319891952, "chain_loss": 0.46823691129684447, "readout_loss": 0.705706131693564, "readout_argmax_matches_target": 0.7894736842105263, "readout_mean_top_prob": 0.7406379486385145, "readout_target_entropy": 0.17239210088002055, "letter_loss": 0.00507200435095001, "letter_top_prob": 0.9949511229991913, "mem_gb": 48.998109184}
{"step": 290, "of": 1265, "lr": 0.00017977450776725538, "grad_norm": 2.6097795963287354, "examples": 41, "readout_examples": 23, "reasoner_examples": 18, "step_tokens": 35732, "chain_tokens": 17644, "micro_batches": 5, "tok_per_s": 8030.793704809131, "elapsed_min": 22.014474097887675, "eta_min": 74.01418015669131, "chain_loss": 0.42299748543235993, "readout_loss": 0.9350532601061075, "readout_argmax_matches_target": 0.6086956521739131, "readout_mean_top_prob": 0.6355033555756444, "readout_target_entropy": 0.030136834020199982, "letter_loss": 0.012300946743279282, "letter_top_prob": 0.9879203571213616, "mem_gb": 48.998109184}
{"step": 300, "of": 1265, "lr": 0.00017820592979755605, "grad_norm": 2.575838565826416, "examples": 45, "readout_examples": 35, "reasoner_examples": 10, "step_tokens": 35812, "chain_tokens": 21077, "micro_batches": 5, "tok_per_s": 8029.327258926791, "elapsed_min": 22.772173488140105, "eta_min": 73.25049138685068, "chain_loss": 0.4223234042525291, "readout_loss": 1.130329034078334, "readout_argmax_matches_target": 0.5714285714285714, "readout_mean_top_prob": 0.5542155742645264, "readout_target_entropy": 0.11961956364767892, "letter_loss": 0.3252038318692939, "letter_top_prob": 0.9990169703960419, "mem_gb": 48.998109184}
{"step": 310, "of": 1265, "lr": 0.00017658616962089261, "grad_norm": 13.889756202697754, "examples": 27, "readout_examples": 17, "reasoner_examples": 10, "step_tokens": 35601, "chain_tokens": 20038, "micro_batches": 5, "tok_per_s": 8033.599540060292, "elapsed_min": 23.540748957792918, "eta_min": 72.52069436997496, "chain_loss": 0.3506860494613647, "readout_loss": 1.1580836594323902, "readout_argmax_matches_target": 0.23529411764705882, "readout_mean_top_prob": 0.6319954482948079, "readout_target_entropy": 0.0, "letter_loss": 0.11118427867618266, "letter_top_prob": 0.9997533440589905, "mem_gb": 48.998109184}
{"step": 320, "of": 1265, "lr": 0.0001749162872962761, "grad_norm": 4.469111442565918, "examples": 48, "readout_examples": 37, "reasoner_examples": 11, "step_tokens": 34110, "chain_tokens": 18487, "micro_batches": 5, "tok_per_s": 8033.775854809462, "elapsed_min": 24.284187185764313, "eta_min": 71.71424028296023, "chain_loss": 0.39212077314203436, "readout_loss": 1.0541124157104138, "readout_argmax_matches_target": 0.4594594594594595, "readout_mean_top_prob": 0.6641654903824264, "readout_target_entropy": 0.0596419371463157, "letter_loss": 0.0003198782925954385, "letter_top_prob": 0.9996802102435719, "mem_gb": 48.998109184}
{"step": 330, "of": 1265, "lr": 0.00017319737568537258, "grad_norm": 8.860196113586426, "examples": 28, "readout_examples": 25, "reasoner_examples": 3, "step_tokens": 33410, "chain_tokens": 4351, "micro_batches": 5, "tok_per_s": 8041.045963039871, "elapsed_min": 25.001179480552672, "eta_min": 70.83667519489924, "chain_loss": 0.543813039859136, "readout_loss": 0.9608212189376354, "readout_argmax_matches_target": 0.56, "readout_mean_top_prob": 0.6932376313209534, "readout_target_entropy": 0.02709030479192734, "letter_loss": 0.0028336693067103624, "letter_top_prob": 0.9971705476442972, "mem_gb": 48.998109184}
{"step": 340, "of": 1265, "lr": 0.00017143055973727552, "grad_norm": 9.89608383178711, "examples": 43, "readout_examples": 31, "reasoner_examples": 12, "step_tokens": 35530, "chain_tokens": 17487, "micro_batches": 6, "tok_per_s": 8042.1541580260555, "elapsed_min": 25.751446183522543, "eta_min": 70.05908152870103, "chain_loss": 0.5127026401460171, "readout_loss": 0.9594720473603136, "readout_argmax_matches_target": 0.6774193548387096, "readout_mean_top_prob": 0.8225337555331569, "readout_target_entropy": 0.07922321041264842, "letter_loss": 0.07064314409702395, "letter_top_prob": 0.9964270889759064, "mem_gb": 48.998109184}
{"step": 350, "of": 1265, "lr": 0.00016961699575227765, "grad_norm": 8.629547119140625, "examples": 32, "readout_examples": 25, "reasoner_examples": 7, "step_tokens": 33667, "chain_tokens": 13566, "micro_batches": 5, "tok_per_s": 8038.082770493153, "elapsed_min": 26.502022445201874, "eta_min": 69.28385867817062, "chain_loss": 0.46341364298548016, "readout_loss": 0.7280855852365494, "readout_argmax_matches_target": 0.68, "readout_mean_top_prob": 0.5896827340126037, "readout_target_entropy": 0.02249340534210205, "letter_loss": 0.08518661998095922, "letter_top_prob": 0.995189334665026, "mem_gb": 48.998109184}
{"step": 360, "of": 1265, "lr": 0.00016775787062512555, "grad_norm": 4.925978183746338, "examples": 41, "readout_examples": 33, "reasoner_examples": 8, "step_tokens": 37795, "chain_tokens": 17298, "micro_batches": 5, "tok_per_s": 8034.245465221568, "elapsed_min": 27.265783818562827, "eta_min": 68.54315098833155, "chain_loss": 0.42863524332642555, "readout_loss": 0.8953548528705583, "readout_argmax_matches_target": 0.6363636363636364, "readout_mean_top_prob": 0.6666106703606519, "readout_target_entropy": 0.15228995409878818, "letter_loss": 0.13236612695618533, "letter_top_prob": 0.9946164563298225, "mem_gb": 48.998109184}
{"step": 370, "of": 1265, "lr": 0.00016585440106825104, "grad_norm": 2.113783836364746, "examples": 31, "readout_examples": 13, "reasoner_examples": 18, "step_tokens": 36872, "chain_tokens": 17921, "micro_batches": 5, "tok_per_s": 8035.101356323279, "elapsed_min": 28.01547734340032, "eta_min": 67.76716816849537, "chain_loss": 0.4075136093629731, "readout_loss": 0.8595684950168316, "readout_argmax_matches_target": 0.7692307692307693, "readout_mean_top_prob": 0.7410692251645602, "readout_target_entropy": 0.05331901403573843, "letter_loss": 0.05272259472662376, "letter_top_prob": 0.9926672412289513, "mem_gb": 48.998109184}
{"step": 380, "of": 1265, "lr": 0.00016390783281548864, "grad_norm": 4.553898334503174, "examples": 43, "readout_examples": 39, "reasoner_examples": 4, "step_tokens": 38240, "chain_tokens": 19271, "micro_batches": 6, "tok_per_s": 8028.246458833449, "elapsed_min": 28.835759405295054, "eta_min": 67.15696598338454, "chain_loss": 0.5590483918786049, "readout_loss": 0.8876548629039183, "readout_argmax_matches_target": 0.5897435897435898, "readout_mean_top_prob": 0.7272852598092495, "readout_target_entropy": 0.0, "letter_loss": 0.1941649488871917, "letter_top_prob": 0.9894874393939972, "mem_gb": 48.998109184}
{"step": 390, "of": 1265, "lr": 0.00016191943980679976, "grad_norm": 7.794270992279053, "examples": 39, "readout_examples": 31, "reasoner_examples": 8, "step_tokens": 34333, "chain_tokens": 8922, "micro_batches": 5, "tok_per_s": 8034.444330465204, "elapsed_min": 29.561083948612215, "eta_min": 66.32294475650176, "chain_loss": 0.47520510107278824, "readout_loss": 1.3191430977876148, "readout_argmax_matches_target": 0.41935483870967744, "readout_mean_top_prob": 0.622562036879601, "readout_target_entropy": 0.014344353829660723, "letter_loss": 0.003381587397598196, "letter_top_prob": 0.9966266378760338, "mem_gb": 48.998109184}
{"step": 400, "of": 1265, "lr": 0.00015989052335453697, "grad_norm": 4.597210884094238, "examples": 44, "readout_examples": 40, "reasoner_examples": 4, "step_tokens": 35429, "chain_tokens": 18704, "micro_batches": 5, "tok_per_s": 8020.518835297113, "elapsed_min": 30.358973236878715, "eta_min": 65.65127962475022, "chain_loss": 0.30252647027373314, "readout_loss": 1.4178386185318232, "readout_argmax_matches_target": 0.45, "readout_mean_top_prob": 0.4938524834811687, "readout_target_entropy": 0.1764489896595478, "letter_loss": 0.18081270344555378, "letter_top_prob": 0.8433936387300491, "mem_gb": 48.998109184}
{"step": 410, "of": 1265, "lr": 0.0001578224112917948, "grad_norm": 7.734204292297363, "examples": 21, "readout_examples": 17, "reasoner_examples": 4, "step_tokens": 35308, "chain_tokens": 18469, "micro_batches": 5, "tok_per_s": 8020.177933916521, "elapsed_min": 31.105024774869282, "eta_min": 64.86535654271522, "chain_loss": 0.3800939805805683, "readout_loss": 1.1866265072980349, "readout_argmax_matches_target": 0.6470588235294118, "readout_mean_top_prob": 0.7229062380159602, "readout_target_entropy": 0.00012982019004138077, "letter_loss": 0.01571058656554669, "letter_top_prob": 0.9844712764024734, "mem_gb": 48.998109184}
{"step": 420, "of": 1265, "lr": 0.0001557164571034036, "grad_norm": 3.556270122528076, "examples": 84, "readout_examples": 81, "reasoner_examples": 3, "step_tokens": 36825, "chain_tokens": 7290, "micro_batches": 5, "tok_per_s": 8024.568459444875, "elapsed_min": 31.85167957941691, "eta_min": 64.08254582049354, "chain_loss": 0.5440772374471029, "readout_loss": 0.951935681661039, "readout_argmax_matches_target": 0.6419753086419753, "readout_mean_top_prob": 0.7252369753004592, "readout_target_entropy": 0.06779502718536942, "letter_loss": 0.00023719747453772774, "letter_top_prob": 0.9997628331184387, "mem_gb": 48.998109184}
{"step": 430, "of": 1265, "lr": 0.00015357403904013545, "grad_norm": 5.73386287689209, "examples": 37, "readout_examples": 25, "reasoner_examples": 12, "step_tokens": 35834, "chain_tokens": 8375, "micro_batches": 5, "tok_per_s": 8029.449209593128, "elapsed_min": 32.58598771890004, "eta_min": 63.277441268096595, "chain_loss": 0.384562112390995, "readout_loss": 1.0781944815814495, "readout_argmax_matches_target": 0.56, "readout_mean_top_prob": 0.6803220498561859, "readout_target_entropy": 0.06529311060905457, "letter_loss": 0.06484965813190986, "letter_top_prob": 0.9983032693465551, "mem_gb": 48.998109184}
{"step": 440, "of": 1265, "lr": 0.00015139655921670212, "grad_norm": 38.68296432495117, "examples": 15, "readout_examples": 6, "reasoner_examples": 9, "step_tokens": 33352, "chain_tokens": 13537, "micro_batches": 5, "tok_per_s": 8028.442153022057, "elapsed_min": 33.36393016576767, "eta_min": 62.55736906081439, "chain_loss": 0.40542023215028977, "readout_loss": 2.4014605741637447, "readout_argmax_matches_target": 0.3333333333333333, "readout_mean_top_prob": 0.750525618592898, "readout_target_entropy": 0.0, "letter_loss": 0.22988845143845538, "letter_top_prob": 0.9992785983615451, "mem_gb": 48.998109184}
{"step": 450, "of": 1265, "lr": 0.0001491854426941351, "grad_norm": 5.74164342880249, "examples": 14, "readout_examples": 3, "reasoner_examples": 11, "step_tokens": 38527, "chain_tokens": 23706, "micro_batches": 6, "tok_per_s": 8027.9344216946765, "elapsed_min": 34.109826465447746, "eta_min": 61.77668570964425, "chain_loss": 0.4490593292496421, "readout_loss": 1.3544598420461018, "readout_argmax_matches_target": 0.3333333333333333, "readout_mean_top_prob": 0.5711831847826639, "readout_target_entropy": 0.0, "letter_loss": 0.005126288986172189, "letter_top_prob": 0.9948923858729276, "mem_gb": 48.998109184}
{"step": 460, "of": 1265, "lr": 0.00014694213654714816, "grad_norm": 6.628501892089844, "examples": 34, "readout_examples": 22, "reasoner_examples": 12, "step_tokens": 39359, "chain_tokens": 8314, "micro_batches": 5, "tok_per_s": 8037.0350285943705, "elapsed_min": 34.82912736733754, "eta_min": 60.9509728928407, "chain_loss": 0.3685354106128216, "readout_loss": 1.095756919948723, "readout_argmax_matches_target": 0.6363636363636364, "readout_mean_top_prob": 0.699453355236487, "readout_target_entropy": 0.06731321594931862, "letter_loss": 0.002219279694448536, "letter_top_prob": 0.9977861493825912, "mem_gb": 48.998109184}
{"step": 470, "of": 1265, "lr": 0.0001446681089170932, "grad_norm": 3.3514511585235596, "examples": 78, "readout_examples": 71, "reasoner_examples": 7, "step_tokens": 35944, "chain_tokens": 10154, "micro_batches": 5, "tok_per_s": 8032.8569266651, "elapsed_min": 35.57914109627406, "eta_min": 60.18173866284655, "chain_loss": 0.5132326356002263, "readout_loss": 0.8000141130104451, "readout_argmax_matches_target": 0.5492957746478874, "readout_mean_top_prob": 0.6613431186743186, "readout_target_entropy": 0.058308451948031574, "letter_loss": 0.01892724568356893, "letter_top_prob": 0.9814622742789132, "mem_gb": 48.998109184}
{"step": 480, "of": 1265, "lr": 0.00014236484805112877, "grad_norm": 7.315610408782959, "examples": 24, "readout_examples": 17, "reasoner_examples": 7, "step_tokens": 36381, "chain_tokens": 15838, "micro_batches": 5, "tok_per_s": 8032.424733450828, "elapsed_min": 36.34183031320572, "eta_min": 59.43403499138852, "chain_loss": 0.2671012708118984, "readout_loss": 0.8603813052314389, "readout_argmax_matches_target": 0.5294117647058824, "readout_mean_top_prob": 0.681814828339745, "readout_target_entropy": 0.10675729898845449, "letter_loss": 0.0015682702775978083, "letter_top_prob": 0.9984334622110639, "mem_gb": 48.998109184}
{"step": 490, "of": 1265, "lr": 0.0001400338613282304, "grad_norm": 3.3681087493896484, "examples": 57, "readout_examples": 48, "reasoner_examples": 9, "step_tokens": 35614, "chain_tokens": 18245, "micro_batches": 5, "tok_per_s": 8044.820949433271, "elapsed_min": 37.049117505550385, "eta_min": 58.598094013880704, "chain_loss": 0.411130675011211, "readout_loss": 0.9132980862632394, "readout_argmax_matches_target": 0.5416666666666666, "readout_mean_top_prob": 0.6195759394516548, "readout_target_entropy": 0.1605813099692265, "letter_loss": 0.13199361018551928, "letter_top_prob": 0.9922196732627021, "mem_gb": 48.998109184}
{"step": 500, "of": 1265, "lr": 0.00013767667427267975, "grad_norm": 8.37640380859375, "examples": 22, "readout_examples": 12, "reasoner_examples": 10, "step_tokens": 34515, "chain_tokens": 12669, "micro_batches": 5, "tok_per_s": 8046.7018419721735, "elapsed_min": 37.779526233673096, "eta_min": 57.802675137519834, "chain_loss": 0.4799667403101921, "readout_loss": 1.7407254986464977, "readout_argmax_matches_target": 0.25, "readout_mean_top_prob": 0.637783316274484, "readout_target_entropy": 0.0, "letter_loss": 0.09249872320506256, "letter_top_prob": 0.9958008408546448, "mem_gb": 48.998109184}
{"step": 510, "of": 1265, "lr": 0.00013529482955567895, "grad_norm": 5.675610542297363, "examples": 45, "readout_examples": 35, "reasoner_examples": 10, "step_tokens": 35465, "chain_tokens": 10431, "micro_batches": 5, "tok_per_s": 8045.22759045879, "elapsed_min": 38.5231861392657, "eta_min": 57.029422617932546, "chain_loss": 0.3893231824040413, "readout_loss": 1.0535955530724355, "readout_argmax_matches_target": 0.5714285714285714, "readout_mean_top_prob": 0.6505087281976427, "readout_target_entropy": 0.05999919346400669, "letter_loss": 0.13321510501264128, "letter_top_prob": 0.998040395975113, "mem_gb": 48.998109184}
{"step": 520, "of": 1265, "lr": 0.00013288988598574258, "grad_norm": 2.3114101886749268, "examples": 45, "readout_examples": 43, "reasoner_examples": 2, "step_tokens": 34027, "chain_tokens": 9258, "micro_batches": 5, "tok_per_s": 8047.144702026642, "elapsed_min": 39.255223854382834, "eta_min": 56.24065725291387, "chain_loss": 0.4503622353076935, "readout_loss": 0.723660483176625, "readout_argmax_matches_target": 0.7209302325581395, "readout_mean_top_prob": 0.6742946852085202, "readout_target_entropy": 0.2019678072700667, "letter_loss": 0.004758385010063648, "letter_top_prob": 0.9952535629272461, "mem_gb": 48.998109184}
{"step": 530, "of": 1265, "lr": 0.0001304634174885289, "grad_norm": 4.010870456695557, "examples": 11, "readout_examples": 2, "reasoner_examples": 9, "step_tokens": 38629, "chain_tokens": 29018, "micro_batches": 5, "tok_per_s": 8035.483870012033, "elapsed_min": 40.07736665805181, "eta_min": 55.57898961069449, "chain_loss": 0.4383817497226927, "readout_loss": 1.2936642169952393, "readout_argmax_matches_target": 0.0, "readout_mean_top_prob": 0.6216872781515121, "readout_target_entropy": 0.0, "letter_loss": 0.10034354927807322, "letter_top_prob": 0.9811490906609429, "mem_gb": 48.998109184}
{"step": 540, "of": 1265, "lr": 0.00012801701207677729, "grad_norm": 4.417247295379639, "examples": 90, "readout_examples": 83, "reasoner_examples": 7, "step_tokens": 37054, "chain_tokens": 6486, "micro_batches": 5, "tok_per_s": 8043.5481854335085, "elapsed_min": 40.79238943258921, "eta_min": 54.767559886346625, "chain_loss": 0.25894745439291, "readout_loss": 0.8964976224072098, "readout_argmax_matches_target": 0.6626506024096386, "readout_mean_top_prob": 0.697377392685557, "readout_target_entropy": 0.0576083782207535, "letter_loss": 0.002731396841617035, "letter_top_prob": 0.9972738538469587, "mem_gb": 48.998109184}
{"step": 550, "of": 1265, "lr": 0.00012555227081102662, "grad_norm": 3.1506271362304688, "examples": 32, "readout_examples": 23, "reasoner_examples": 9, "step_tokens": 38147, "chain_tokens": 15397, "micro_batches": 6, "tok_per_s": 8049.466895642149, "elapsed_min": 41.5158611536026, "eta_min": 53.97061949968339, "chain_loss": 0.4449106868770387, "readout_loss": 0.7904043341379451, "readout_argmax_matches_target": 0.6521739130434783, "readout_mean_top_prob": 0.6164253302242445, "readout_target_entropy": 0.02706465772960497, "letter_loss": 0.0723840761962088, "letter_top_prob": 0.983151740498013, "mem_gb": 48.998109184}
{"step": 560, "of": 1265, "lr": 0.0001230708067517942, "grad_norm": 3.711010694503784, "examples": 38, "readout_examples": 29, "reasoner_examples": 9, "step_tokens": 34086, "chain_tokens": 10941, "micro_batches": 5, "tok_per_s": 8053.483881259493, "elapsed_min": 42.229630680878955, "eta_min": 53.16408862503512, "chain_loss": 0.32343366079860264, "readout_loss": 0.7871129620691826, "readout_argmax_matches_target": 0.7586206896551724, "readout_mean_top_prob": 0.6626973648009629, "readout_target_entropy": 0.078535879608886, "letter_loss": 0.000981309789899064, "letter_top_prob": 0.9990207420455085, "mem_gb": 48.998109184}
{"step": 570, "of": 1265, "lr": 0.00012057424390390142, "grad_norm": 3.4454126358032227, "examples": 43, "readout_examples": 42, "reasoner_examples": 1, "step_tokens": 39142, "chain_tokens": 6981, "micro_batches": 5, "tok_per_s": 8055.3532813807215, "elapsed_min": 42.99144157965978, "eta_min": 52.419389294497456, "chain_loss": 0.3644760251045227, "readout_loss": 0.7278859535775458, "readout_argmax_matches_target": 0.7857142857142857, "readout_mean_top_prob": 0.7236678859307653, "readout_target_entropy": 0.08319721125770316, "letter_loss": 0.0012456761905923486, "letter_top_prob": 0.9987550973892212, "mem_gb": 48.998109184}
{"step": 580, "of": 1265, "lr": 0.00011806421615363686, "grad_norm": 5.791473865509033, "examples": 23, "readout_examples": 18, "reasoner_examples": 5, "step_tokens": 37047, "chain_tokens": 13846, "micro_batches": 6, "tok_per_s": 8049.17486534137, "elapsed_min": 43.76613825559616, "eta_min": 51.68931845704029, "chain_loss": 0.42509670853614806, "readout_loss": 1.1245977191461458, "readout_argmax_matches_target": 0.6111111111111112, "readout_mean_top_prob": 0.718599850932757, "readout_target_entropy": 0.06679358747270372, "letter_loss": 0.0033428418159019203, "letter_top_prob": 0.9966727375984192, "mem_gb": 48.998109184}
{"step": 590, "of": 1265, "lr": 0.00011554236619945229, "grad_norm": 4.552141189575195, "examples": 28, "readout_examples": 23, "reasoner_examples": 5, "step_tokens": 35357, "chain_tokens": 8936, "micro_batches": 5, "tok_per_s": 8052.689364982013, "elapsed_min": 44.48672161102295, "eta_min": 50.89582557193304, "chain_loss": 0.3819133087992668, "readout_loss": 0.6656102423110734, "readout_argmax_matches_target": 0.782608695652174, "readout_mean_top_prob": 0.6650154383286185, "readout_target_entropy": 0.04846581373282749, "letter_loss": 0.004360212036408484, "letter_top_prob": 0.9956524968147278, "mem_gb": 48.998109184}
{"step": 600, "of": 1265, "lr": 0.00011301034447689169, "grad_norm": 3.9055988788604736, "examples": 35, "readout_examples": 28, "reasoner_examples": 7, "step_tokens": 39026, "chain_tokens": 3977, "micro_batches": 4, "tok_per_s": 8045.376567975319, "elapsed_min": 45.28583966890971, "eta_min": 50.1918056330416, "chain_loss": 0.3713390678167343, "readout_loss": 0.6889863631555012, "readout_argmax_matches_target": 0.7142857142857143, "readout_mean_top_prob": 0.6539920206580844, "readout_target_entropy": 0.10865921234445912, "letter_loss": 0.006746575535674181, "letter_top_prob": 0.9932936174528939, "mem_gb": 48.998109184}
{"step": 610, "of": 1265, "lr": 0.0001104698080784566, "grad_norm": 9.250932693481445, "examples": 9, "readout_examples": 3, "reasoner_examples": 6, "step_tokens": 33127, "chain_tokens": 23955, "micro_batches": 5, "tok_per_s": 8042.99724267226, "elapsed_min": 46.04013762871424, "eta_min": 49.43654122427513, "chain_loss": 0.47873008499542874, "readout_loss": 1.1643880407015483, "readout_argmax_matches_target": 0.3333333333333333, "readout_mean_top_prob": 0.640039841334025, "readout_target_entropy": 0.0, "letter_loss": 0.3136096994082133, "letter_top_prob": 0.8557995855808258, "mem_gb": 48.998109184}
{"step": 620, "of": 1265, "lr": 0.00010792241966911471, "grad_norm": 3.934204578399658, "examples": 24, "readout_examples": 15, "reasoner_examples": 9, "step_tokens": 37075, "chain_tokens": 18010, "micro_batches": 5, "tok_per_s": 8044.774309917553, "elapsed_min": 46.798609320322676, "eta_min": 48.68565001872278, "chain_loss": 0.3869883269071579, "readout_loss": 0.584146864215533, "readout_argmax_matches_target": 0.8666666666666667, "readout_mean_top_prob": 0.6462648590405782, "readout_target_entropy": 0.025118011236190795, "letter_loss": 0.08142244725281166, "letter_top_prob": 0.9839308990372552, "mem_gb": 48.998109184}
{"step": 630, "of": 1265, "lr": 0.00010536984639816183, "grad_norm": 3.532978057861328, "examples": 69, "readout_examples": 60, "reasoner_examples": 9, "step_tokens": 34542, "chain_tokens": 17988, "micro_batches": 4, "tok_per_s": 8045.089432003265, "elapsed_min": 47.56138344605764, "eta_min": 47.93885474324857, "chain_loss": 0.46901485986179775, "readout_loss": 0.6453814744735913, "readout_argmax_matches_target": 0.7166666666666667, "readout_mean_top_prob": 0.7294117997090022, "readout_target_entropy": 0.1608464548829943, "letter_loss": 0.2817244960654837, "letter_top_prob": 0.9853031833966573, "mem_gb": 48.998109184}
{"step": 640, "of": 1265, "lr": 0.00010281375880814863, "grad_norm": 3.0277047157287598, "examples": 62, "readout_examples": 48, "reasoner_examples": 14, "step_tokens": 37967, "chain_tokens": 20738, "micro_batches": 5, "tok_per_s": 8044.285057093785, "elapsed_min": 48.3091636578242, "eta_min": 47.17691763459394, "chain_loss": 0.37504149441208157, "readout_loss": 0.8006623729985828, "readout_argmax_matches_target": 0.7083333333333334, "readout_mean_top_prob": 0.6382019650191069, "readout_target_entropy": 0.06274935354789098, "letter_loss": 0.07202131795098207, "letter_top_prob": 0.9827006076063428, "mem_gb": 48.998109184}
{"step": 650, "of": 1265, "lr": 0.00010025582974158709, "grad_norm": 3.3589155673980713, "examples": 50, "readout_examples": 45, "reasoner_examples": 5, "step_tokens": 37082, "chain_tokens": 12535, "micro_batches": 5, "tok_per_s": 8049.889912269021, "elapsed_min": 49.013608173529306, "eta_min": 46.37441388726235, "chain_loss": 0.44413272738456727, "readout_loss": 0.9654278370034364, "readout_argmax_matches_target": 0.6444444444444445, "readout_mean_top_prob": 0.6815178434054057, "readout_target_entropy": 0.09738036062982347, "letter_loss": 0.009449530486017465, "letter_top_prob": 0.9906108498573303, "mem_gb": 48.998109184}
{"step": 660, "of": 1265, "lr": 9.769773324615132e-05, "grad_norm": 2.339282512664795, "examples": 86, "readout_examples": 77, "reasoner_examples": 9, "step_tokens": 33391, "chain_tokens": 12037, "micro_batches": 5, "tok_per_s": 8050.5534104940625, "elapsed_min": 49.75985660950343, "eta_min": 45.613201892044806, "chain_loss": 0.38039566493696636, "readout_loss": 0.6783350033733946, "readout_argmax_matches_target": 0.6363636363636364, "readout_mean_top_prob": 0.7136279259409223, "readout_target_entropy": 0.02808033258883984, "letter_loss": 0.008710476537493782, "letter_top_prob": 0.9913778172598945, "mem_gb": 48.998109184}
{"step": 670, "of": 1265, "lr": 9.514114347909012e-05, "grad_norm": 2.9081006050109863, "examples": 34, "readout_examples": 17, "reasoner_examples": 17, "step_tokens": 39822, "chain_tokens": 18827, "micro_batches": 6, "tok_per_s": 8052.905958364912, "elapsed_min": 50.51035639842351, "eta_min": 44.85621202546565, "chain_loss": 0.32746372678700614, "readout_loss": 0.5217314039576141, "readout_argmax_matches_target": 0.8235294117647058, "readout_mean_top_prob": 0.8613155729630414, "readout_target_entropy": 0.2375143520972308, "letter_loss": 0.0014026007154504495, "letter_top_prob": 0.9986004619037404, "mem_gb": 48.998109184}
{"step": 680, "of": 1265, "lr": 9.258773361156725e-05, "grad_norm": 4.343477725982666, "examples": 31, "readout_examples": 19, "reasoner_examples": 12, "step_tokens": 40012, "chain_tokens": 15953, "micro_batches": 6, "tok_per_s": 8049.125083216423, "elapsed_min": 51.283733208974205, "eta_min": 44.11909401066162, "chain_loss": 0.4463430618246396, "readout_loss": 1.0894361013840688, "readout_argmax_matches_target": 0.5263157894736842, "readout_mean_top_prob": 0.6432628913929588, "readout_target_entropy": 0.05723344338567633, "letter_loss": 0.008811855698392415, "letter_top_prob": 0.9912568579117457, "mem_gb": 48.998109184}
{"step": 690, "of": 1265, "lr": 9.003917473364774e-05, "grad_norm": 9.856346130371094, "examples": 14, "readout_examples": 6, "reasoner_examples": 8, "step_tokens": 40083, "chain_tokens": 31614, "micro_batches": 5, "tok_per_s": 8045.1918825501425, "elapsed_min": 52.06421294212341, "eta_min": 43.38684411843618, "chain_loss": 0.41996799036860466, "readout_loss": 1.7895590911308925, "readout_argmax_matches_target": 0.5, "readout_mean_top_prob": 0.7625309775273005, "readout_target_entropy": 0.2163462440172831, "letter_loss": 0.14988292317138985, "letter_top_prob": 0.9822407066822052, "mem_gb": 48.998109184}
{"step": 700, "of": 1265, "lr": 8.749713476064547e-05, "grad_norm": 4.564524173736572, "examples": 39, "readout_examples": 33, "reasoner_examples": 6, "step_tokens": 33026, "chain_tokens": 10875, "micro_batches": 5, "tok_per_s": 8046.981918517131, "elapsed_min": 52.79760009845098, "eta_min": 42.615205793749716, "chain_loss": 0.4264892190694809, "readout_loss": 0.41954562578449084, "readout_argmax_matches_target": 0.8484848484848485, "readout_mean_top_prob": 0.8306758033506798, "readout_target_entropy": 0.0, "letter_loss": 0.005687944445526227, "letter_top_prob": 0.9943510591983795, "mem_gb": 48.998109184}
{"step": 710, "of": 1265, "lr": 8.496327734154904e-05, "grad_norm": 3.0763652324676514, "examples": 20, "readout_examples": 9, "reasoner_examples": 11, "step_tokens": 40216, "chain_tokens": 22275, "micro_batches": 6, "tok_per_s": 8045.366877070892, "elapsed_min": 53.5389506260554, "eta_min": 41.850869855578516, "chain_loss": 0.39156817170706665, "readout_loss": 0.8325423436342843, "readout_argmax_matches_target": 0.7777777777777778, "readout_mean_top_prob": 0.6243095662858751, "readout_target_entropy": 0.0, "letter_loss": 0.06030695557340302, "letter_top_prob": 0.9919082414020192, "mem_gb": 48.998109184}
{"step": 720, "of": 1265, "lr": 8.243926077023946e-05, "grad_norm": 4.498336315155029, "examples": 38, "readout_examples": 29, "reasoner_examples": 9, "step_tokens": 34133, "chain_tokens": 13285, "micro_batches": 5, "tok_per_s": 8050.894661763188, "elapsed_min": 54.23855795065562, "eta_min": 41.05557511542682, "chain_loss": 0.4257984177933799, "readout_loss": 0.26131908317770935, "readout_argmax_matches_target": 0.896551724137931, "readout_mean_top_prob": 0.8790416563379353, "readout_target_entropy": 0.010794512156782479, "letter_loss": 0.017524739384599444, "letter_top_prob": 0.98319896724489, "mem_gb": 48.998109184}
{"step": 730, "of": 1265, "lr": 7.992673690021261e-05, "grad_norm": 2.909829616546631, "examples": 34, "readout_examples": 29, "reasoner_examples": 5, "step_tokens": 34491, "chain_tokens": 12213, "micro_batches": 5, "tok_per_s": 8052.255286466545, "elapsed_min": 54.98087193171183, "eta_min": 40.29420066228196, "chain_loss": 0.43287441730499265, "readout_loss": 0.7330555939276157, "readout_argmax_matches_target": 0.6896551724137931, "readout_mean_top_prob": 0.6428000721438177, "readout_target_entropy": 0.09491217958516088, "letter_loss": 0.021179878944531084, "letter_top_prob": 0.9794036507606506, "mem_gb": 48.998109184}
{"step": 740, "of": 1265, "lr": 7.742735006351656e-05, "grad_norm": 3.6088004112243652, "examples": 16, "readout_examples": 6, "reasoner_examples": 10, "step_tokens": 33740, "chain_tokens": 17646, "micro_batches": 5, "tok_per_s": 8053.886461636339, "elapsed_min": 55.73078523079554, "eta_min": 39.53873276509144, "chain_loss": 0.3492144137620926, "readout_loss": 0.4243944484042004, "readout_argmax_matches_target": 0.8333333333333334, "readout_mean_top_prob": 0.886324147383372, "readout_target_entropy": 0.0, "letter_loss": 0.009725558062200435, "letter_top_prob": 0.9903809666633606, "mem_gb": 48.998109184}
{"step": 750, "of": 1265, "lr": 7.494273599461152e-05, "grad_norm": 4.868605136871338, "examples": 16, "readout_examples": 4, "reasoner_examples": 12, "step_tokens": 33191, "chain_tokens": 20790, "micro_batches": 5, "tok_per_s": 8052.4735047911145, "elapsed_min": 56.4897646745046, "eta_min": 38.78963840982649, "chain_loss": 0.3960484986503919, "readout_loss": 0.6605105213820934, "readout_argmax_matches_target": 0.75, "readout_mean_top_prob": 0.6249298006296158, "readout_target_entropy": 0.0, "letter_loss": 0.0495629370198003, "letter_top_prob": 0.9869318604469299, "mem_gb": 48.998109184}
{"step": 760, "of": 1265, "lr": 7.247452075985623e-05, "grad_norm": 3.946183204650879, "examples": 28, "readout_examples": 16, "reasoner_examples": 12, "step_tokens": 37785, "chain_tokens": 18533, "micro_batches": 5, "tok_per_s": 8055.289665348868, "elapsed_min": 57.238740317026775, "eta_min": 38.03363665802438, "chain_loss": 0.31436194231112796, "readout_loss": 0.7978903135299333, "readout_argmax_matches_target": 0.6875, "readout_mean_top_prob": 0.8313737101852894, "readout_target_entropy": 0.031275149434804916, "letter_loss": 0.08365421576733449, "letter_top_prob": 0.9983816196521124, "mem_gb": 48.998109184}
{"step": 770, "of": 1265, "lr": 7.002431969332196e-05, "grad_norm": 3.3163440227508545, "examples": 48, "readout_examples": 31, "reasoner_examples": 17, "step_tokens": 38304, "chain_tokens": 14211, "micro_batches": 5, "tok_per_s": 8056.844725858517, "elapsed_min": 57.968536802132924, "eta_min": 37.26548794422831, "chain_loss": 0.44468148140346303, "readout_loss": 0.4782811972184009, "readout_argmax_matches_target": 0.9032258064516129, "readout_mean_top_prob": 0.7769731369710737, "readout_target_entropy": 0.04038947147707785, "letter_loss": 0.002068810839987504, "letter_top_prob": 0.9979618121595943, "mem_gb": 48.998109184}
{"step": 780, "of": 1265, "lr": 6.75937363396302e-05, "grad_norm": 6.9885663986206055, "examples": 19, "readout_examples": 8, "reasoner_examples": 11, "step_tokens": 38372, "chain_tokens": 21430, "micro_batches": 7, "tok_per_s": 8059.672524507344, "elapsed_min": 58.726292153199516, "eta_min": 36.51570730038688, "chain_loss": 0.4833578345450488, "readout_loss": 0.8969714004342677, "readout_argmax_matches_target": 0.75, "readout_mean_top_prob": 0.8300432823598385, "readout_target_entropy": 0.0, "letter_loss": 0.001074854234254665, "letter_top_prob": 0.9989265528592196, "mem_gb": 48.998109184}
{"step": 790, "of": 1265, "lr": 6.518436140450579e-05, "grad_norm": 2.0473921298980713, "examples": 96, "readout_examples": 91, "reasoner_examples": 5, "step_tokens": 35867, "chain_tokens": 7274, "micro_batches": 5, "tok_per_s": 8066.036251595357, "elapsed_min": 59.427088479201, "eta_min": 35.731477250152494, "chain_loss": 0.3043776899576187, "readout_loss": 0.5917909850339549, "readout_argmax_matches_target": 0.7692307692307693, "readout_mean_top_prob": 0.7360354426142933, "readout_target_entropy": 0.07008548473918831, "letter_loss": 0.0015713708533439785, "letter_top_prob": 0.9984319090843201, "mem_gb": 48.998109184}
{"step": 800, "of": 1265, "lr": 6.279777171373303e-05, "grad_norm": 3.296260356903076, "examples": 50, "readout_examples": 44, "reasoner_examples": 6, "step_tokens": 33092, "chain_tokens": 14351, "micro_batches": 5, "tok_per_s": 8068.006565914208, "elapsed_min": 60.16459737618764, "eta_min": 34.97067222490907, "chain_loss": 0.42929694180687267, "readout_loss": 0.8658040716916069, "readout_argmax_matches_target": 0.6136363636363636, "readout_mean_top_prob": 0.5931446369398724, "readout_target_entropy": 0.12227508493445137, "letter_loss": 0.10092056892851058, "letter_top_prob": 0.9890721837679545, "mem_gb": 48.998109184}
{"step": 810, "of": 1265, "lr": 6.043552918119499e-05, "grad_norm": 3.6478734016418457, "examples": 34, "readout_examples": 17, "reasoner_examples": 17, "step_tokens": 35603, "chain_tokens": 15264, "micro_batches": 5, "tok_per_s": 8067.658671540246, "elapsed_min": 60.910119032859804, "eta_min": 34.21494340734717, "chain_loss": 0.4011727422475815, "readout_loss": 0.5346615211730001, "readout_argmax_matches_target": 0.7058823529411765, "readout_mean_top_prob": 0.7596218936583575, "readout_target_entropy": 0.0, "letter_loss": 0.051140338700326325, "letter_top_prob": 0.9939884929095998, "mem_gb": 48.998109184}
{"step": 820, "of": 1265, "lr": 5.8099179786672367e-05, "grad_norm": 36.155494689941406, "examples": 9, "readout_examples": 3, "reasoner_examples": 6, "step_tokens": 32999, "chain_tokens": 21420, "micro_batches": 5, "tok_per_s": 8065.009351054444, "elapsed_min": 61.671222150325775, "eta_min": 33.4679193376768, "chain_loss": 0.44138769308725995, "readout_loss": 1.7522906263669331, "readout_argmax_matches_target": 0.6666666666666666, "readout_mean_top_prob": 0.6234972278277079, "readout_target_entropy": 0.0, "letter_loss": 0.10235812225922321, "letter_top_prob": 0.991463283697764, "mem_gb": 48.998109184}
{"step": 830, "of": 1265, "lr": 5.579025256407038e-05, "grad_norm": 1.356512188911438, "examples": 21, "readout_examples": 9, "reasoner_examples": 12, "step_tokens": 36556, "chain_tokens": 19922, "micro_batches": 5, "tok_per_s": 8060.147958086696, "elapsed_min": 62.470726668834686, "eta_min": 32.740682049329024, "chain_loss": 0.45903381581107777, "readout_loss": 0.3722540993152911, "readout_argmax_matches_target": 0.7777777777777778, "readout_mean_top_prob": 0.8971936570273505, "readout_target_entropy": 0.1283403757131762, "letter_loss": 0.1086630361508772, "letter_top_prob": 0.9910347610712051, "mem_gb": 48.998109184}
{"step": 840, "of": 1265, "lr": 5.351025860073604e-05, "grad_norm": 1.2599338293075562, "examples": 84, "readout_examples": 77, "reasoner_examples": 7, "step_tokens": 33398, "chain_tokens": 13584, "micro_batches": 5, "tok_per_s": 8057.223841881237, "elapsed_min": 63.22603707313537, "eta_min": 31.989363995336355, "chain_loss": 0.41138141070093426, "readout_loss": 0.6202671522903559, "readout_argmax_matches_target": 0.7142857142857143, "readout_mean_top_prob": 0.6567547890273008, "readout_target_entropy": 0.07355961203575134, "letter_loss": 0.012384052463208459, "letter_top_prob": 0.9877553411892482, "mem_gb": 48.998109184}
{"step": 850, "of": 1265, "lr": 5.126069004852032e-05, "grad_norm": 1.7121193408966064, "examples": 33, "readout_examples": 16, "reasoner_examples": 17, "step_tokens": 35485, "chain_tokens": 12626, "micro_batches": 5, "tok_per_s": 8053.980277957101, "elapsed_min": 64.00883565743764, "eta_min": 31.251372703337204, "chain_loss": 0.3946536034345627, "readout_loss": 0.9739418643148383, "readout_argmax_matches_target": 0.5625, "readout_mean_top_prob": 0.607111640740186, "readout_target_entropy": 0.1801468525081873, "letter_loss": 0.058263481464510894, "letter_top_prob": 0.9838269037358901, "mem_gb": 48.998109184}
{"step": 860, "of": 1265, "lr": 4.904301914723315e-05, "grad_norm": 3.9712953567504883, "examples": 28, "readout_examples": 25, "reasoner_examples": 3, "step_tokens": 33587, "chain_tokens": 8476, "micro_batches": 5, "tok_per_s": 8049.910565302877, "elapsed_min": 64.7865148027738, "eta_min": 30.509928482701607, "chain_loss": 0.4187633991241455, "readout_loss": 0.7645608798391186, "readout_argmax_matches_target": 0.68, "readout_mean_top_prob": 0.673112725019455, "readout_target_entropy": 0.029424877166748048, "letter_loss": 0.037084852966169514, "letter_top_prob": 0.9637856880823771, "mem_gb": 49.085901312}
{"step": 870, "of": 1265, "lr": 4.685869726112962e-05, "grad_norm": 3.2910537719726562, "examples": 54, "readout_examples": 50, "reasoner_examples": 4, "step_tokens": 39624, "chain_tokens": 19152, "micro_batches": 6, "tok_per_s": 8048.99732636422, "elapsed_min": 65.55823190609614, "eta_min": 29.76494437115859, "chain_loss": 0.3067982792854309, "readout_loss": 0.7785743452631868, "readout_argmax_matches_target": 0.66, "readout_mean_top_prob": 0.7066619503498077, "readout_target_entropy": 0.08654972910881042, "letter_loss": 0.012393771205097437, "letter_top_prob": 0.9876905679702759, "mem_gb": 49.085901312}
{"step": 880, "of": 1265, "lr": 4.470915392905861e-05, "grad_norm": 2.9124317169189453, "examples": 14, "readout_examples": 3, "reasoner_examples": 11, "step_tokens": 37150, "chain_tokens": 20805, "micro_batches": 6, "tok_per_s": 8050.313764814882, "elapsed_min": 66.30948170820872, "eta_min": 29.010398247341314, "chain_loss": 0.25566338070414285, "readout_loss": 1.0678492163618405, "readout_argmax_matches_target": 0.6666666666666666, "readout_mean_top_prob": 0.5838317771752676, "readout_target_entropy": 0.0, "letter_loss": 0.0052802345116188835, "letter_top_prob": 0.994774417443709, "mem_gb": 49.085901312}
{"step": 890, "of": 1265, "lr": 4.259579592889463e-05, "grad_norm": 1.9569847583770752, "examples": 107, "readout_examples": 95, "reasoner_examples": 12, "step_tokens": 39452, "chain_tokens": 6531, "micro_batches": 5, "tok_per_s": 8051.058259130217, "elapsed_min": 67.05116883913676, "eta_min": 28.25189698278234, "chain_loss": 0.2567315685252349, "readout_loss": 0.6238736384738187, "readout_argmax_matches_target": 0.6947368421052632, "readout_mean_top_prob": 0.6777332644713553, "readout_target_entropy": 0.07516505247668216, "letter_loss": 0.0009329870990768541, "letter_top_prob": 0.9990678131580353, "mem_gb": 49.085901312}
{"step": 900, "of": 1265, "lr": 4.052000635686627e-05, "grad_norm": 4.9884114265441895, "examples": 54, "readout_examples": 45, "reasoner_examples": 9, "step_tokens": 40169, "chain_tokens": 11128, "micro_batches": 5, "tok_per_s": 8048.936255394086, "elapsed_min": 67.84152994553249, "eta_min": 27.513509366799283, "chain_loss": 0.4653955598672231, "readout_loss": 0.6054493606620882, "readout_argmax_matches_target": 0.7111111111111111, "readout_mean_top_prob": 0.755985951423645, "readout_target_entropy": 7.090431431101428e-05, "letter_loss": 0.0029682426215408165, "letter_top_prob": 0.9970427619086372, "mem_gb": 49.085901312}
{"step": 910, "of": 1265, "lr": 3.848314372238272e-05, "grad_norm": 6.225015163421631, "examples": 57, "readout_examples": 48, "reasoner_examples": 9, "step_tokens": 35771, "chain_tokens": 8927, "micro_batches": 5, "tok_per_s": 8050.621760220115, "elapsed_min": 68.57616920073828, "eta_min": 26.75224183105724, "chain_loss": 0.28674530651834274, "readout_loss": 0.6517809024025306, "readout_argmax_matches_target": 0.7291666666666666, "readout_mean_top_prob": 0.7502789770563444, "readout_target_entropy": 0.06259051710367203, "letter_loss": 0.1028695660287566, "letter_top_prob": 0.9986946847703722, "mem_gb": 49.085901312}
{"step": 920, "of": 1265, "lr": 3.648654105895169e-05, "grad_norm": 3.033160448074341, "examples": 29, "readout_examples": 19, "reasoner_examples": 10, "step_tokens": 39661, "chain_tokens": 13805, "micro_batches": 6, "tok_per_s": 8051.122964701033, "elapsed_min": 69.3287635087967, "eta_min": 25.99828631579876, "chain_loss": 0.39889142885804174, "readout_loss": 0.7062022140323135, "readout_argmax_matches_target": 0.7368421052631579, "readout_mean_top_prob": 0.6848449895256444, "readout_target_entropy": 0.1202700917461985, "letter_loss": 0.0034279127008630893, "letter_top_prob": 0.996590530872345, "mem_gb": 49.085901312}
{"step": 930, "of": 1265, "lr": 3.4531505051769664e-05, "grad_norm": 2.133617877960205, "examples": 69, "readout_examples": 66, "reasoner_examples": 3, "step_tokens": 34703, "chain_tokens": 6911, "micro_batches": 5, "tok_per_s": 8051.03201904443, "elapsed_min": 70.05181430975595, "eta_min": 25.23371805781532, "chain_loss": 0.21666033814350763, "readout_loss": 0.638873227864899, "readout_argmax_matches_target": 0.6818181818181818, "readout_mean_top_prob": 0.684919076887044, "readout_target_entropy": 0.08299873588663159, "letter_loss": 0.007468095796260362, "letter_top_prob": 0.9926109512646993, "mem_gb": 49.085901312}
{"step": 940, "of": 1265, "lr": 3.261931518255623e-05, "grad_norm": 1.285915493965149, "examples": 10, "readout_examples": 3, "reasoner_examples": 7, "step_tokens": 35823, "chain_tokens": 22170, "micro_batches": 6, "tok_per_s": 8048.779321516066, "elapsed_min": 70.83242819309234, "eta_min": 24.48993527952661, "chain_loss": 0.3561262935400009, "readout_loss": 0.06527276205209394, "readout_argmax_matches_target": 1.0, "readout_mean_top_prob": 0.9403157234191895, "readout_target_entropy": 0.0, "letter_loss": 0.003897185769996473, "letter_top_prob": 0.9961185199873788, "mem_gb": 49.085901312}
{"step": 950, "of": 1265, "lr": 3.075122289219181e-05, "grad_norm": 0.3287191092967987, "examples": 7, "readout_examples": 0, "reasoner_examples": 7, "step_tokens": 34594, "chain_tokens": 25174, "micro_batches": 5, "tok_per_s": 8048.854195239664, "elapsed_min": 71.59689474900564, "eta_min": 23.740022995722924, "chain_loss": 0.3604563219206674, "readout_loss": null, "readout_argmax_matches_target": null, "readout_mean_top_prob": null, "readout_target_entropy": null, "letter_loss": 0.0051744816342501766, "letter_top_prob": 0.9948543991361346, "mem_gb": 49.085901312}
{"step": 960, "of": 1265, "lr": 2.892845076170685e-05, "grad_norm": 3.1011157035827637, "examples": 83, "readout_examples": 78, "reasoner_examples": 5, "step_tokens": 37844, "chain_tokens": 7982, "micro_batches": 5, "tok_per_s": 8050.602025272211, "elapsed_min": 72.32483336925506, "eta_min": 22.978202268357077, "chain_loss": 0.39354124665260315, "readout_loss": 0.728924237443164, "readout_argmax_matches_target": 0.7435897435897436, "readout_mean_top_prob": 0.7181558845899044, "readout_target_entropy": 0.11000505662881412, "letter_loss": 0.0036493964056717233, "letter_top_prob": 0.9963659882545471, "mem_gb": 49.085901312}
{"step": 970, "of": 1265, "lr": 2.7152191712158203e-05, "grad_norm": 7.379469871520996, "examples": 17, "readout_examples": 6, "reasoner_examples": 11, "step_tokens": 38627, "chain_tokens": 20514, "micro_batches": 5, "tok_per_s": 8045.301245136312, "elapsed_min": 73.14723614056905, "eta_min": 22.245808929348318, "chain_loss": 0.29185087843374774, "readout_loss": 0.5374331063649151, "readout_argmax_matches_target": 0.8333333333333334, "readout_mean_top_prob": 0.7496121327082316, "readout_target_entropy": 0.14965762694676718, "letter_loss": 0.010403713767035779, "letter_top_prob": 0.9900210987437855, "mem_gb": 49.085901312}
{"step": 980, "of": 1265, "lr": 2.5423608223917016e-05, "grad_norm": 4.799283027648926, "examples": 9, "readout_examples": 3, "reasoner_examples": 6, "step_tokens": 36490, "chain_tokens": 17205, "micro_batches": 6, "tok_per_s": 8044.303766264325, "elapsed_min": 73.90356794993083, "eta_min": 21.49236414870437, "chain_loss": 0.4142346456646919, "readout_loss": 1.277634824315707, "readout_argmax_matches_target": 0.6666666666666666, "readout_mean_top_prob": 0.694540003935496, "readout_target_entropy": 0.0, "letter_loss": 0.012177172776622077, "letter_top_prob": 0.9879326919714609, "mem_gb": 49.085901312}
{"step": 990, "of": 1265, "lr": 2.374383157587835e-05, "grad_norm": 3.321228265762329, "examples": 33, "readout_examples": 27, "reasoner_examples": 6, "step_tokens": 40004, "chain_tokens": 13807, "micro_batches": 6, "tok_per_s": 8046.863317139438, "elapsed_min": 74.64424041112264, "eta_min": 20.73451122531184, "chain_loss": 0.368628869454066, "readout_loss": 0.4146596913439377, "readout_argmax_matches_target": 0.8518518518518519, "readout_mean_top_prob": 0.8084547906010239, "readout_target_entropy": 0.05695366225915711, "letter_loss": 0.007282450717563431, "letter_top_prob": 0.9927787482738495, "mem_gb": 49.085901312}
{"step": 1000, "of": 1265, "lr": 2.211396110509093e-05, "grad_norm": 2.1638987064361572, "examples": 10, "readout_examples": 2, "reasoner_examples": 8, "step_tokens": 36331, "chain_tokens": 25193, "micro_batches": 6, "tok_per_s": 8047.633858036304, "elapsed_min": 75.39339645703633, "eta_min": 19.97925006111463, "chain_loss": 0.4594302624464035, "readout_loss": 0.628044705837965, "readout_argmax_matches_target": 0.5, "readout_mean_top_prob": 0.7635778188705444, "readout_target_entropy": 0.0, "letter_loss": 0.02500084537314251, "letter_top_prob": 0.9757007360458374, "mem_gb": 49.085901312}
{"step": 1010, "of": 1265, "lr": 2.053506348729117e-05, "grad_norm": 2.3508639335632324, "examples": 30, "readout_examples": 21, "reasoner_examples": 9, "step_tokens": 37141, "chain_tokens": 12477, "micro_batches": 5, "tok_per_s": 8046.006141282269, "elapsed_min": 76.16001519759496, "eta_min": 19.228518688501698, "chain_loss": 0.49266184700859916, "readout_loss": 0.4411755955689365, "readout_argmax_matches_target": 0.8095238095238095, "readout_mean_top_prob": 0.8235042251291729, "readout_target_entropy": 0.001374046007792155, "letter_loss": 0.003923356717374797, "letter_top_prob": 0.9960926837391324, "mem_gb": 49.085901312}
{"step": 1020, "of": 1265, "lr": 1.9008172038812742e-05, "grad_norm": 6.071223735809326, "examples": 27, "readout_examples": 19, "reasoner_examples": 8, "step_tokens": 35095, "chain_tokens": 18286, "micro_batches": 5, "tok_per_s": 8051.495752161782, "elapsed_min": 76.86063588460287, "eta_min": 18.461623325223236, "chain_loss": 0.3731641061604023, "readout_loss": 0.6681745174298014, "readout_argmax_matches_target": 0.8421052631578947, "readout_mean_top_prob": 0.7777105114961925, "readout_target_entropy": 0.16452010681754664, "letter_loss": 0.08080897703985102, "letter_top_prob": 0.9963511154055595, "mem_gb": 49.085901312}
{"step": 1030, "of": 1265, "lr": 1.753428604032821e-05, "grad_norm": 2.6937646865844727, "examples": 23, "readout_examples": 10, "reasoner_examples": 13, "step_tokens": 33795, "chain_tokens": 16774, "micro_batches": 5, "tok_per_s": 8052.106464598482, "elapsed_min": 77.58971966902415, "eta_min": 17.702508856524926, "chain_loss": 0.34575681216441667, "readout_loss": 0.34604724273085596, "readout_argmax_matches_target": 1.0, "readout_mean_top_prob": 0.7510302156209946, "readout_target_entropy": 0.05443160086870193, "letter_loss": 0.011897628441847, "letter_top_prob": 0.9883932608824509, "mem_gb": 49.085901312}
{"step": 1040, "of": 1265, "lr": 1.611437008286555e-05, "grad_norm": 5.739242076873779, "examples": 14, "readout_examples": 5, "reasoner_examples": 9, "step_tokens": 33323, "chain_tokens": 22530, "micro_batches": 5, "tok_per_s": 8056.52842179399, "elapsed_min": 78.30336885054906, "eta_min": 16.940632684013018, "chain_loss": 0.3048822068505817, "readout_loss": 0.6133501123636961, "readout_argmax_matches_target": 0.6, "readout_mean_top_prob": 0.7638502717018127, "readout_target_entropy": 0.20593061447143554, "letter_loss": 0.0072462048289404874, "letter_top_prob": 0.9928215940793356, "mem_gb": 49.085901312}
{"step": 1050, "of": 1265, "lr": 1.4749353436527358e-05, "grad_norm": 1.3421555757522583, "examples": 22, "readout_examples": 4, "reasoner_examples": 18, "step_tokens": 37013, "chain_tokens": 21794, "micro_batches": 5, "tok_per_s": 8051.975340528596, "elapsed_min": 79.08367281357447, "eta_min": 16.193323480874774, "chain_loss": 0.31868697702884674, "readout_loss": 0.3240493295015767, "readout_argmax_matches_target": 1.0, "readout_mean_top_prob": 0.7430003583431244, "readout_target_entropy": 0.0, "letter_loss": 0.04385782483344277, "letter_top_prob": 0.9927237927913666, "mem_gb": 49.085901312}
{"step": 1060, "of": 1265, "lr": 1.3440129442326044e-05, "grad_norm": 1.7185641527175903, "examples": 106, "readout_examples": 101, "reasoner_examples": 5, "step_tokens": 37257, "chain_tokens": 12374, "micro_batches": 5, "tok_per_s": 8053.60564159352, "elapsed_min": 79.81705072323481, "eta_min": 15.43631641345579, "chain_loss": 0.33213708400726316, "readout_loss": 0.6216468746003944, "readout_argmax_matches_target": 0.7326732673267327, "readout_mean_top_prob": 0.7010211732127879, "readout_target_entropy": 0.0835984461378343, "letter_loss": 0.015056019532494247, "letter_top_prob": 0.9851915955543518, "mem_gb": 49.085901312}
{"step": 1070, "of": 1265, "lr": 1.2187554927532962e-05, "grad_norm": 2.7859435081481934, "examples": 72, "readout_examples": 64, "reasoner_examples": 8, "step_tokens": 37171, "chain_tokens": 12740, "micro_batches": 5, "tok_per_s": 8056.4165336566, "elapsed_min": 80.53442420164744, "eta_min": 14.676834317122664, "chain_loss": 0.3845534361898899, "readout_loss": 0.9578909110696259, "readout_argmax_matches_target": 0.65625, "readout_mean_top_prob": 0.7326262439601123, "readout_target_entropy": 0.12116605043411255, "letter_loss": 0.01045613362111908, "letter_top_prob": 0.989652581512928, "mem_gb": 49.085901312}
{"step": 1080, "of": 1265, "lr": 1.0992449644924185e-05, "grad_norm": 4.167708873748779, "examples": 89, "readout_examples": 83, "reasoner_examples": 6, "step_tokens": 37300, "chain_tokens": 7129, "micro_batches": 5, "tok_per_s": 8055.040351958843, "elapsed_min": 81.30852295160294, "eta_min": 13.927848838931984, "chain_loss": 0.40595656633377075, "readout_loss": 0.7269629453279426, "readout_argmax_matches_target": 0.6265060240963856, "readout_mean_top_prob": 0.7263272939676262, "readout_target_entropy": 0.06724838873915688, "letter_loss": 0.01089194547967054, "letter_top_prob": 0.9894038041432699, "mem_gb": 49.085901312}
{"step": 1090, "of": 1265, "lr": 9.855595736289668e-06, "grad_norm": 6.130951881408691, "examples": 20, "readout_examples": 14, "reasoner_examples": 6, "step_tokens": 39208, "chain_tokens": 16832, "micro_batches": 6, "tok_per_s": 8059.545563275308, "elapsed_min": 82.02531745036443, "eta_min": 13.169202342948417, "chain_loss": 0.45355189839998883, "readout_loss": 0.6774868188825037, "readout_argmax_matches_target": 0.6428571428571429, "readout_mean_top_prob": 0.8333845095975059, "readout_target_entropy": 0.0, "letter_loss": 0.02625805129840349, "letter_top_prob": 0.9743980964024862, "mem_gb": 49.085901312}
{"step": 1100, "of": 1265, "lr": 8.77773722055727e-06, "grad_norm": 1.8853235244750977, "examples": 33, "readout_examples": 21, "reasoner_examples": 12, "step_tokens": 39007, "chain_tokens": 13518, "micro_batches": 5, "tok_per_s": 8059.744199792919, "elapsed_min": 82.77211411794026, "eta_min": 12.41581711769104, "chain_loss": 0.39413764886558056, "readout_loss": 0.3584233541984022, "readout_argmax_matches_target": 0.8095238095238095, "readout_mean_top_prob": 0.822179111696425, "readout_target_entropy": 0.04998961091041565, "letter_loss": 0.010892751777040152, "letter_top_prob": 0.9893272817134857, "mem_gb": 49.085901312}
{"step": 1110, "of": 1265, "lr": 7.75957950686631e-06, "grad_norm": 4.373437404632568, "examples": 12, "readout_examples": 2, "reasoner_examples": 10, "step_tokens": 34049, "chain_tokens": 22640, "micro_batches": 5, "tok_per_s": 8059.834786427059, "elapsed_min": 83.520421675841, "eta_min": 11.662761585365187, "chain_loss": 0.32314553707838056, "readout_loss": 1.1868495345115662, "readout_argmax_matches_target": 1.0, "readout_mean_top_prob": 0.34210580587387085, "readout_target_entropy": 0.0, "letter_loss": 0.014150628063362092, "letter_top_prob": 0.9861014246940613, "mem_gb": 49.085901312}
{"step": 1120, "of": 1265, "lr": 6.801788932909647e-06, "grad_norm": 1.9532170295715332, "examples": 35, "readout_examples": 27, "reasoner_examples": 8, "step_tokens": 36374, "chain_tokens": 18324, "micro_batches": 6, "tok_per_s": 8057.876104854316, "elapsed_min": 84.30079975922902, "eta_min": 10.913942825971615, "chain_loss": 0.3300303425639868, "readout_loss": 0.5201270010762123, "readout_argmax_matches_target": 0.8518518518518519, "readout_mean_top_prob": 0.8120992868034927, "readout_target_entropy": 0.03696646844899213, "letter_loss": 0.018520904661272652, "letter_top_prob": 0.9819151014089584, "mem_gb": 49.085901312}
{"step": 1130, "of": 1265, "lr": 5.904992328845893e-06, "grad_norm": 5.578059673309326, "examples": 21, "readout_examples": 9, "reasoner_examples": 12, "step_tokens": 37230, "chain_tokens": 18202, "micro_batches": 6, "tok_per_s": 8057.640949381612, "elapsed_min": 85.05516328811646, "eta_min": 10.161457560969666, "chain_loss": 0.4858916774392128, "readout_loss": 1.2995579284357115, "readout_argmax_matches_target": 0.4444444444444444, "readout_mean_top_prob": 0.6731488870249854, "readout_target_entropy": 0.0, "letter_loss": 0.0862893446101225, "letter_top_prob": 0.9900408238172531, "mem_gb": 49.085901312}
{"step": 1140, "of": 1265, "lr": 5.069776607067944e-06, "grad_norm": 4.103229999542236, "examples": 18, "readout_examples": 11, "reasoner_examples": 7, "step_tokens": 37589, "chain_tokens": 14178, "micro_batches": 5, "tok_per_s": 8057.182624777948, "elapsed_min": 85.82279921770096, "eta_min": 9.4103946510637, "chain_loss": 0.3559493805680956, "readout_loss": 0.5294860364572908, "readout_argmax_matches_target": 0.8181818181818182, "readout_mean_top_prob": 0.8296557095917788, "readout_target_entropy": 0.12988810105757279, "letter_loss": 0.010166539477982692, "letter_top_prob": 0.9899805869374957, "mem_gb": 49.085901312}
{"step": 1150, "of": 1265, "lr": 4.296688378095493e-06, "grad_norm": 3.0649187564849854, "examples": 47, "readout_examples": 38, "reasoner_examples": 9, "step_tokens": 37821, "chain_tokens": 16303, "micro_batches": 5, "tok_per_s": 8059.149730640706, "elapsed_min": 86.57853164672852, "eta_min": 8.65785316467285, "chain_loss": 0.4534454213248359, "readout_loss": 0.5485138190258447, "readout_argmax_matches_target": 0.8421052631578947, "readout_mean_top_prob": 0.7272992596814507, "readout_target_entropy": 0.02264940895532307, "letter_loss": 0.09545596828360835, "letter_top_prob": 0.9931415319442749, "mem_gb": 49.085901312}
{"step": 1160, "of": 1265, "lr": 3.5862335928435465e-06, "grad_norm": 3.1632611751556396, "examples": 33, "readout_examples": 25, "reasoner_examples": 8, "step_tokens": 37359, "chain_tokens": 15808, "micro_batches": 5, "tok_per_s": 8060.033415585075, "elapsed_min": 87.30511364539464, "eta_min": 7.902618045488309, "chain_loss": 0.33072615414857864, "readout_loss": 0.7439891670597717, "readout_argmax_matches_target": 0.76, "readout_mean_top_prob": 0.81377510368824, "readout_target_entropy": 0.027725887298583985, "letter_loss": 0.08271156485716347, "letter_top_prob": 0.9824133217334747, "mem_gb": 49.085901312}
{"step": 1170, "of": 1265, "lr": 2.938877211500546e-06, "grad_norm": 0.9908670783042908, "examples": 11, "readout_examples": 2, "reasoner_examples": 9, "step_tokens": 40064, "chain_tokens": 29271, "micro_batches": 6, "tok_per_s": 8055.766184807012, "elapsed_min": 88.12296480735144, "eta_min": 7.155283467263577, "chain_loss": 0.45080602500173783, "readout_loss": 0.13635975867509842, "readout_argmax_matches_target": 1.0, "readout_mean_top_prob": 0.8779710829257965, "readout_target_entropy": 0.0, "letter_loss": 0.1419721823444383, "letter_top_prob": 0.9709002905421786, "mem_gb": 49.085901312}
{"step": 1180, "of": 1265, "lr": 2.3550428992332506e-06, "grad_norm": 1.4924250841140747, "examples": 19, "readout_examples": 4, "reasoner_examples": 15, "step_tokens": 36395, "chain_tokens": 22529, "micro_batches": 5, "tok_per_s": 8055.195922291031, "elapsed_min": 88.86468521753947, "eta_min": 6.401269697873606, "chain_loss": 0.3043092419703802, "readout_loss": 0.5462045148015022, "readout_argmax_matches_target": 0.75, "readout_mean_top_prob": 0.7112569212913513, "readout_target_entropy": 0.16539081931114197, "letter_loss": 0.03436813084699679, "letter_top_prob": 0.9913729826609293, "mem_gb": 49.085901312}
{"step": 1190, "of": 1265, "lr": 1.8351127489172226e-06, "grad_norm": 5.79235315322876, "examples": 42, "readout_examples": 38, "reasoner_examples": 4, "step_tokens": 37644, "chain_tokens": 7107, "micro_batches": 5, "tok_per_s": 8056.239669226979, "elapsed_min": 89.60604197978974, "eta_min": 5.647439620574984, "chain_loss": 0.37832217290997505, "readout_loss": 0.6434532039968058, "readout_argmax_matches_target": 0.8157894736842105, "readout_mean_top_prob": 0.703374351325788, "readout_target_entropy": 0.048321322599229846, "letter_loss": 0.0025060261905309744, "letter_top_prob": 0.9975001364946365, "mem_gb": 49.085901312}
{"step": 1200, "of": 1265, "lr": 1.379427031074554e-06, "grad_norm": 2.064044952392578, "examples": 81, "readout_examples": 64, "reasoner_examples": 17, "step_tokens": 38192, "chain_tokens": 10509, "micro_batches": 5, "tok_per_s": 8054.144564408208, "elapsed_min": 90.38831219275792, "eta_min": 4.896033577107721, "chain_loss": 0.4403546066845165, "readout_loss": 0.6421352873076103, "readout_argmax_matches_target": 0.765625, "readout_mean_top_prob": 0.7010885861236602, "readout_target_entropy": 0.1633277479559183, "letter_loss": 0.00207881023251342, "letter_top_prob": 0.9979262001374188, "mem_gb": 49.085901312}
{"step": 1210, "of": 1265, "lr": 9.882839711822467e-07, "grad_norm": 2.392390727996826, "examples": 81, "readout_examples": 72, "reasoner_examples": 9, "step_tokens": 35715, "chain_tokens": 9890, "micro_batches": 5, "tok_per_s": 8052.970942983869, "elapsed_min": 91.15629356304804, "eta_min": 4.1434678892294565, "chain_loss": 0.39544809195730424, "readout_loss": 0.5687624681886518, "readout_argmax_matches_target": 0.7083333333333334, "readout_mean_top_prob": 0.7494589547730155, "readout_target_entropy": 0.09730610903352499, "letter_loss": 0.003947750235157501, "letter_top_prob": 0.9960670934783088, "mem_gb": 49.085901312}
{"step": 1220, "of": 1265, "lr": 6.619395544974039e-07, "grad_norm": 4.667299747467041, "examples": 37, "readout_examples": 33, "reasoner_examples": 4, "step_tokens": 40586, "chain_tokens": 11081, "micro_batches": 6, "tok_per_s": 8050.378453752091, "elapsed_min": 91.93861765861512, "eta_min": 3.3911785201948197, "chain_loss": 0.4290136434137821, "readout_loss": 0.42765523941665323, "readout_argmax_matches_target": 0.8484848484848485, "readout_mean_top_prob": 0.8945745735457449, "readout_target_entropy": 0.0771788295471307, "letter_loss": 0.030948145082220435, "letter_top_prob": 0.9698023647069931, "mem_gb": 49.085901312}
{"step": 1230, "of": 1265, "lr": 4.0060735852647246e-07, "grad_norm": 2.376401662826538, "examples": 20, "readout_examples": 14, "reasoner_examples": 6, "step_tokens": 38571, "chain_tokens": 20641, "micro_batches": 5, "tok_per_s": 8051.443494831318, "elapsed_min": 92.66920693318049, "eta_min": 2.6369286525701767, "chain_loss": 0.41912971436977386, "readout_loss": 0.2691517783594983, "readout_argmax_matches_target": 0.9285714285714286, "readout_mean_top_prob": 0.9098524429968425, "readout_target_entropy": 0.038288891315460205, "letter_loss": 0.3876329220123201, "letter_top_prob": 0.9737654129664103, "mem_gb": 49.085901312}
{"step": 1240, "of": 1265, "lr": 2.0445841324856497e-07, "grad_norm": 8.630610466003418, "examples": 27, "readout_examples": 17, "reasoner_examples": 10, "step_tokens": 35489, "chain_tokens": 13771, "micro_batches": 5, "tok_per_s": 8047.5345517580345, "elapsed_min": 93.47099145650864, "eta_min": 1.8844957954941257, "chain_loss": 0.35963755697011945, "readout_loss": 0.5681035924559076, "readout_argmax_matches_target": 0.7058823529411765, "readout_mean_top_prob": 0.7491015774362227, "readout_target_entropy": 0.0, "letter_loss": 0.0059401757433079185, "letter_top_prob": 0.9941039979457855, "mem_gb": 49.085901312}
{"step": 1250, "of": 1265, "lr": 7.362108918418375e-08, "grad_norm": 7.756507396697998, "examples": 25, "readout_examples": 19, "reasoner_examples": 6, "step_tokens": 37640, "chain_tokens": 19228, "micro_batches": 6, "tok_per_s": 8044.817601586722, "elapsed_min": 94.24801831642786, "eta_min": 1.1309762197971345, "chain_loss": 0.38918523242076236, "readout_loss": 1.1456879539917664, "readout_argmax_matches_target": 0.42105263157894735, "readout_mean_top_prob": 0.6685221591278127, "readout_target_entropy": 0.10061370071611907, "letter_loss": 0.14739222017427286, "letter_top_prob": 0.9793843825658163, "mem_gb": 49.085901312}
{"step": 1260, "of": 1265, "lr": 8.181013382524416e-09, "grad_norm": 3.3705642223358154, "examples": 31, "readout_examples": 20, "reasoner_examples": 11, "step_tokens": 33646, "chain_tokens": 15171, "micro_batches": 4, "tok_per_s": 8044.421187067305, "elapsed_min": 94.99523942073186, "eta_min": 0.376965235796555, "chain_loss": 0.369529741731557, "readout_loss": 0.4503404173978197, "readout_argmax_matches_target": 0.85, "readout_mean_top_prob": 0.7986559122800827, "readout_target_entropy": 0.09077194482088088, "letter_loss": 0.08147452998012093, "letter_top_prob": 0.9912727908654646, "mem_gb": 49.085901312}
{"dev_n": 999, "dev_acc": 0.7417417417417418, "dev_nll": 0.6170839262493034, "dev_ece15": 0.038537513177554744}
{"dev_n": 999, "dev_acc": 0.7417417417417418, "dev_nll": 0.6170839262493034, "dev_ece15": 0.038537513177554744, "v2_options": {"readout_pool": "all", "subset": null, "min_correct": 2, "second_chain_min": 3, "rep_max": 20, "post_soft": true, "eval_order_frac": 0.5, "drop_recall": true, "strip_cw": true}, "speed_options": {"liger": false, "ckpt_above": 8192, "micro_tokens": 8192, "no_probe": false}, "mode": "joint", "alpha": 0.5, "lr": 0.0002, "rank": 32, "lora_alpha": 64, "seed": 17, "examples": 52968, "steps_done": 1265, "steps_planned": 1265, "stopped_at_deadline": false, "train_tokens": 46011760, "train_seconds": 5720.039484739304, "tok_per_s": 8043.9584591603625, "counts": {"readout_from_votes": 7320, "readout_eval_order": 20989, "readout_gold_fallback": 13, "readout_gold_only": 34661, "reasoner_chains_dropped_invented_recall": 2193, "reasoner_chains_dropped_repetition": 11, "reasoner_cw_stripped": 1532, "reasoner_soft_letter": 10972, "reasoner_examples": 10974, "reasoner_items": 5779, "reasoner_no_correct_chain": 976, "reasoner_too_few_correct": 257, "reasoner_letter_not_found": 2}, "reasoner_examples": 10974, "readout_examples": 41994}
{"time": "2026-11-01T02:13:57Z", "event": "done", "stage": "2b_train_joint"}
{"time": "2026-11-01T02:14:27Z", "event": "start", "stage": "2b_merge_joint"}

Loading weights:   0%|          | 0/381 [00:00<?, ?it/s]
Loading weights:  96%|█████████▌| 366/381 [00:00<00:00, 3658.90it/s]
Loading weights: 100%|██████████| 381/381 [00:00<00:00, 3724.91it/s]

Writing model shards:   0%|          | 0/1 [00:00<?, ?it/s]
Writing model shards: 100%|██████████| 1/1 [00:02<00:00,  2.32s/it]
Writing model shards: 100%|██████████| 1/1 [00:02<00:00,  2.32s/it]
merged results/adapters/v2b_joint into checkpoints/v2b_joint_merged
{"time": "2026-11-01T02:15:19Z", "event": "done", "stage": "2b_merge_joint"}
{"time": "2026-11-01T02:15:45Z", "event": "start", "stage": "2b_eval_base"}
(MainProcess pid=152146) INFO 11-01 02:15:57 [model.py:733] Resolved architecture: LlamaForCausalLM
(MainProcess pid=152146) INFO 11-01 02:15:57 [model.py:2152] Using max model len 32768
(MainProcess pid=152146) INFO 11-01 02:15:57 [scheduler.py:307] Chunked prefill is enabled with max_num_batched_tokens=2048.
(MainProcess pid=152146) INFO 11-01 02:15:57 [kernel.py:467] Final IR op priority after setting platform defaults: IrOpPriorityConfig(rms_norm=['native'], fused_add_rms_norm=['native'], gelu_and_mul_sparse=['triton', 'native'])

Fetching 10 files:   0%|          | 0/10 [00:00<?, ?it/s]
Fetching 10 files: 100%|██████████| 10/10 [00:00<00:00, 992.34it/s]
(EngineCore pid=152420) INFO 11-01 02:16:00 [core.py:133] Initializing a V1 LLM engine (v0.32.0) with config: model='openbmb/MiniCPM5-2B', speculative_config=None, tokenizer='openbmb/MiniCPM5-2B', skip_tokenizer_init=False, tokenizer_mode=auto, revision=main, tokenizer_revision=main, trust_remote_code=False, dtype=torch.bfloat16, max_seq_len=32768, download_dir=None, load_format=auto, tensor_parallel_size=1, pipeline_parallel_size=1, data_parallel_size=1, decode_context_parallel_size=1, dcp_comm_backend=ag_rs, disable_custom_all_reduce=False, quantization=None, quantization_config=None, enforce_eager=False, aux_output_config=AuxOutputConfig(enable_return_routed_experts=False, max_bytes=None), kv_cache_dtype=auto, device_config=cuda, structured_outputs_config=StructuredOutputsConfig(backend='auto', disable_any_whitespace=False, disable_additional_properties=False, reasoning_parser='', reasoning_parser_plugin='', enable_in_reasoning=False), observability_config=ObservabilityConfig(show_hidden_metrics_for_version=None, otlp_traces_endpoint=None, collect_detailed_traces=None, per_request_spec_decode_metrics='none', kv_cache_metrics=False, kv_cache_metrics_sample=0.01, custom_histogram_buckets=None, cudagraph_metrics=False, enable_layerwise_nvtx_tracing=False, enable_mfu_metrics=False, enable_mm_processor_stats=False, enable_logging_iteration_details=False, jit_monitor_mode='warn', jit_monitor_verbose=False), seed=17, served_model_name=openbmb/MiniCPM5-2B, enable_prefix_caching=True, enable_chunked_prefill=True, pooler_config=None, compilation_config={'mode': <CompilationMode.VLLM_COMPILE: 3>, 'debug_dump_path': None, 'cache_dir': '', 'compile_cache_save_format': 'binary', 'backend': 'inductor', 'custom_ops': ['none'], 'ir_enable_torch_wrap': True, 'splitting_ops': ['vllm::unified_attention_with_output', 'vllm::unified_mla_attention_with_output', 'vllm::mamba_mixer2', 'vllm::mamba_mixer', 'vllm::short_conv', 'vllm::qwen4_exp_ple_short_conv', 'vllm::qwen4_exp_qsa_with_output', 'vllm::qwen4_exp_qsa_prepare_with_output', 'vllm::linear_attention', 'vllm::qwen_gdn_attention_core', 'vllm::qwen_gdn_attention_core_fused_norm_packed', 'vllm::gdn_attention_core_xpu', 'vllm::olmo_hybrid_gdn_full_forward', 'vllm::sparse_attn_indexer', 'vllm::rocm_aiter_sparse_attn_indexer', 'vllm::deepseek_v4_attention', 'vllm::hpc_rope_norm_forward', 'vllm::unified_kv_cache_update', 'vllm::unified_mla_kv_cache_update'], 'compile_mm_encoder': False, 'cudagraph_mm_encoder': False, 'encoder_cudagraph_token_budgets': [], 'encoder_cudagraph_max_vision_items_per_batch': 0, 'encoder_cudagraph_max_frames_per_batch': None, 'cudagraph_decoder_replay': True, 'decoder_replay_cudagraph_capture_sizes': [], 'decoder_replay_trim_threshold': 768, 'compile_sizes': [], 'compile_ranges_endpoints': [2048], 'inductor_compile_config': {'enable_auto_functionalized_v2': False, 'combo_kernels': True, 'benchmark_combo_kernel': True}, 'inductor_passes': {}, 'cudagraph_mode': <CUDAGraphMode.FULL_AND_PIECEWISE: (2, 1)>, 'cudagraph_num_of_warmups': 1, 'cudagraph_capture_sizes': [1, 2, 4, 8, 16, 24, 32, 40, 48, 56, 64, 72, 80, 88, 96, 104, 112, 120, 128, 136, 144, 152, 160, 168, 176, 184, 192, 200, 208, 216, 224, 232, 240, 248, 256, 272, 288, 304, 320, 336, 352, 368, 384, 400, 416, 432, 448, 464, 480, 496, 512], 'cudagraph_copy_inputs': False, 'cudagraph_specialize_lora': True, 'use_inductor_graph_partition': False, 'pass_config': {'fuse_norm_quant': False, 'fuse_act_quant': False, 'fuse_attn_quant': False, 'fuse_allreduce_rms': False, 'enable_qk_norm_rope_fusion': False, 'fuse_rope_kvcache_cat_mla': False, 'fuse_act_padding': False, 'fuse_qk_norm_rope_kvcache': False}, 'max_cudagraph_capture_size': 512, 'dynamic_shapes_config': {'type': <DynamicShapesType.BACKED: 'backed'>, 'evaluate_guards': False, 'assume_32_bit_indexing': False}, 'local_cache_dir': None, 'fast_moe_cold_start': False, 'static_all_moe_layers': []}, kernel_config=KernelConfig(ir_op_priority=IrOpPriorityConfig(rms_norm=['native'], fused_add_rms_norm=['native'], gelu_and_mul_sparse=['triton', 'native']), enable_flashinfer_autotune=True, enable_rocm_segmented_attn_autotune=False, enable_cutedsl_warmup=True, enable_jit_warmup=True, moe_backend='auto', sparse_indexer_topk_backend='auto', linear_backend='auto', linear_backend_per_quant=None)
(EngineCore pid=152420) INFO 11-01 02:16:01 [parallel_state.py:1906] world_size=1 rank=0 local_rank=0 distributed_init_method=file:///tmp/vllm_dist_9f6fbeb7f44b4f5cb90eb8ffc540bf4b backend=nccl
(EngineCore pid=152420) INFO 11-01 02:16:01 [parallel_state.py:2350] rank 0 in world size 1 is assigned as DP rank 0, PP rank 0, PCP rank 0, TP rank 0, ETP rank 0, EP rank N/A, EPLB rank N/A
(EngineCore pid=152420) INFO 11-01 02:16:01 [gpu_worker.py:518] Using V2 Model Runner
(EngineCore pid=152420) INFO 11-01 02:16:02 [model_runner.py:392] Loading model from scratch...
(EngineCore pid=152420) INFO 11-01 02:16:03 [cuda.py:506] Using FLASH_ATTN attention backend out of potential backends: ['FLASH_ATTN', 'FLASHINFER', 'TRITON_ATTN', 'FLEX_ATTENTION'].
(EngineCore pid=152420) INFO 11-01 02:16:03 [flash_attn.py:1117] Using FlashAttention version 2
(EngineCore pid=152420) INFO 11-01 02:16:06 [base_loader.py:82] Initializing model took 3.765978 seconds
(EngineCore pid=152420) INFO 11-01 02:16:06 [weight_utils.py:1108] Filesystem type for checkpoints: XFS. Checkpoint size: 4.69 GiB. Available RAM: 129.82 GiB.
(EngineCore pid=152420) INFO 11-01 02:16:06 [weight_utils.py:1131] Auto-prefetch is disabled because the filesystem (XFS) is not a recognized network FS (NFS/Lustre). If you want to force prefetching, start vLLM with --safetensors-load-strategy=prefetch.

Loading safetensors checkpoint shards:   0% Completed | 0/1 [00:00<?, ?it/s]

Loading safetensors checkpoint shards: 100% Completed | 1/1 [00:00<00:00,  1.71it/s]

Loading safetensors checkpoint shards: 100% Completed | 1/1 [00:00<00:00,  1.71it/s]
(EngineCore pid=152420) INFO 11-01 02:16:07 [default_loader.py:501] Loading weights took 0.59 seconds
(EngineCore pid=152420) INFO 11-01 02:16:08 [model_runner.py:424] Model loading took 4.75 GiB memory and 6.755927 seconds
(EngineCore pid=152420) INFO 11-01 02:16:09 [topk_topp_sampler.py:124] Using FlashInfer for top-p & top-k sampling.
(EngineCore pid=152420) INFO 11-01 02:16:09 [torch_utils.py:295] Reducing Torch threads from 128 to 1 for serving. Set OMP_NUM_THREADS in the external environment to override.
(EngineCore pid=152420) INFO 11-01 02:16:09 [utils.py:347] Using LBNHC KV cache layout.
(EngineCore pid=152420) INFO 11-01 02:16:09 [monitor.py:40] [backbone] Starting torch.compile
(EngineCore pid=152420) INFO 11-01 02:16:18 [backends.py:1104] [backbone] Using cache directory: /root/.cache/vllm/torch_compile_cache/4ae45673a2/rank_0_0/backbone for vLLM's torch.compile
(EngineCore pid=152420) INFO 11-01 02:16:18 [backends.py:1166] [backbone] Dynamo bytecode transform time: 8.98 s
(EngineCore pid=152420) INFO 11-01 02:16:26 [backends.py:398] [backbone] Compiling a graph for compile range (1, 2048) takes 7.47 s
(EngineCore pid=152420) INFO 11-01 02:16:32 [backends.py:927] collected artifacts: 43 entries, 3 artifacts, 4889897 bytes total
(EngineCore pid=152420) INFO 11-01 02:16:32 [decorators.py:743] [backbone] saved AOT compiled function to /root/.cache/vllm/torch_compile_cache/torch_aot_compile/8af31eb29db492efb42d43f0b7d6f9e21786afa38ef612607b7e13dce12a0d17/rank_0_0/model
(EngineCore pid=152420) INFO 11-01 02:16:32 [monitor.py:62] [backbone] torch.compile took 22.15 s in total
(EngineCore pid=152420) INFO 11-01 02:16:32 [monitor.py:91] [backbone] Initial profiling/warmup run took 0.20 s

Capturing CUDA graphs (PIECEWISE):   0%|          | 0/51 [00:00<?, ?it/s]
Capturing CUDA graphs (PIECEWISE):   4%|▍         | 2/51 [00:00<00:03, 12.49it/s]
Capturing CUDA graphs (PIECEWISE):   8%|▊         | 4/51 [00:00<00:03, 12.78it/s]
Capturing CUDA graphs (PIECEWISE):  12%|█▏        | 6/51 [00:00<00:03, 13.49it/s]
Capturing CUDA graphs (PIECEWISE):  16%|█▌        | 8/51 [00:00<00:03, 13.71it/s]
Capturing CUDA graphs (PIECEWISE):  20%|█▉        | 10/51 [00:00<00:03, 13.63it/s]
Capturing CUDA graphs (PIECEWISE):  24%|██▎       | 12/51 [00:00<00:02, 13.95it/s]
Capturing CUDA graphs (PIECEWISE):  27%|██▋       | 14/51 [00:01<00:02, 13.90it/s]
Capturing CUDA graphs (PIECEWISE):  31%|███▏      | 16/51 [00:01<00:02, 14.18it/s]
Capturing CUDA graphs (PIECEWISE):  35%|███▌      | 18/51 [00:01<00:02, 12.96it/s]
Capturing CUDA graphs (PIECEWISE):  39%|███▉      | 20/51 [00:01<00:02, 12.88it/s]
Capturing CUDA graphs (PIECEWISE):  43%|████▎     | 22/51 [00:01<00:02, 13.13it/s]
Capturing CUDA graphs (PIECEWISE):  47%|████▋     | 24/51 [00:01<00:02, 13.10it/s]
Capturing CUDA graphs (PIECEWISE):  51%|█████     | 26/51 [00:01<00:01, 13.39it/s]
Capturing CUDA graphs (PIECEWISE):  55%|█████▍    | 28/51 [00:02<00:01, 13.53it/s]
Capturing CUDA graphs (PIECEWISE):  59%|█████▉    | 30/51 [00:02<00:01, 13.38it/s]
Capturing CUDA graphs (PIECEWISE):  63%|██████▎   | 32/51 [00:02<00:01, 13.53it/s]
Capturing CUDA graphs (PIECEWISE):  67%|██████▋   | 34/51 [00:02<00:01, 13.35it/s]
Capturing CUDA graphs (PIECEWISE):  71%|███████   | 36/51 [00:02<00:01, 13.62it/s]
Capturing CUDA graphs (PIECEWISE):  75%|███████▍  | 38/51 [00:02<00:00, 13.50it/s]
Capturing CUDA graphs (PIECEWISE):  78%|███████▊  | 40/51 [00:03<00:00, 12.62it/s]
Capturing CUDA graphs (PIECEWISE):  82%|████████▏ | 42/51 [00:03<00:00, 13.11it/s]
Capturing CUDA graphs (PIECEWISE):  86%|████████▋ | 44/51 [00:03<00:00, 13.50it/s]
Capturing CUDA graphs (PIECEWISE):  90%|█████████ | 46/51 [00:03<00:00, 13.62it/s]
Capturing CUDA graphs (PIECEWISE):  94%|█████████▍| 48/51 [00:03<00:00, 13.12it/s]
Capturing CUDA graphs (PIECEWISE):  98%|█████████▊| 50/51 [00:03<00:00, 13.30it/s]
Capturing CUDA graphs (PIECEWISE): 100%|██████████| 51/51 [00:03<00:00, 13.27it/s]

Capturing CUDA graphs (FULL):   0%|          | 0/2 [00:00<?, ?it/s]
Capturing CUDA graphs (FULL): 100%|██████████| 2/2 [00:00<00:00, 23.17it/s]
(EngineCore pid=152420) INFO 11-01 02:16:39 [model_runner.py:1127] Graph capturing finished in 5 secs, took 0.40 GiB
(EngineCore pid=152420) INFO 11-01 02:16:39 [gpu_worker.py:759] Available KV cache memory: 65.26 GiB
(EngineCore pid=152420) INFO 11-01 02:16:39 [gpu_worker.py:774] CUDA graph memory profiling is enabled (default since v0.21.0). The current --gpu-memory-utilization=0.9000 is equivalent to --gpu-memory-utilization=0.8943 without CUDA graph memory profiling. To maintain the same effective KV cache size as before, increase --gpu-memory-utilization to 0.9057. To disable, set VLLM_MEMORY_PROFILER_ESTIMATE_CUDAGRAPHS=0.
(EngineCore pid=152420) INFO 11-01 02:16:39 [kv_cache_utils.py:2327] GPU KV cache size: 1,629,360 tokens, Maximum concurrency for 32,768 tokens per request: 49.72x
(EngineCore pid=152420) INFO 11-01 02:16:39 [kernel_warmup.py:176] JIT kernel warmup starting.
(EngineCore pid=152420) INFO 11-01 02:16:39 [kernel_warmup.py:189] JIT kernel warmup finished in 0.00s.

Capturing CUDA graphs (PIECEWISE):   0%|          | 0/51 [00:00<?, ?it/s]
Capturing CUDA graphs (PIECEWISE):   4%|▍         | 2/51 [00:00<00:04, 12.02it/s]
Capturing CUDA graphs (PIECEWISE):   8%|▊         | 4/51 [00:00<00:03, 13.16it/s]
Capturing CUDA graphs (PIECEWISE):  12%|█▏        | 6/51 [00:00<00:03, 13.83it/s]
Capturing CUDA graphs (PIECEWISE):  16%|█▌        | 8/51 [00:00<00:03, 14.06it/s]
Capturing CUDA graphs (PIECEWISE):  20%|█▉        | 10/51 [00:00<00:02, 14.47it/s]
Capturing CUDA graphs (PIECEWISE):  24%|██▎       | 12/51 [00:00<00:02, 14.86it/s]
Capturing CUDA graphs (PIECEWISE):  27%|██▋       | 14/51 [00:00<00:02, 15.13it/s]
Capturing CUDA graphs (PIECEWISE):  31%|███▏      | 16/51 [00:01<00:02, 15.35it/s]
Capturing CUDA graphs (PIECEWISE):  35%|███▌      | 18/51 [00:01<00:02, 14.47it/s]
Capturing CUDA graphs (PIECEWISE):  39%|███▉      | 20/51 [00:01<00:02, 14.22it/s]
Capturing CUDA graphs (PIECEWISE):  43%|████▎     | 22/51 [00:01<00:02, 14.34it/s]
Capturing CUDA graphs (PIECEWISE):  47%|████▋     | 24/51 [00:01<00:01, 14.16it/s]
Capturing CUDA graphs (PIECEWISE):  51%|█████     | 26/51 [00:01<00:01, 14.20it/s]
Capturing CUDA graphs (PIECEWISE):  55%|█████▍    | 28/51 [00:01<00:01, 14.18it/s]
Capturing CUDA graphs (PIECEWISE):  59%|█████▉    | 30/51 [00:02<00:01, 14.18it/s]
Capturing CUDA graphs (PIECEWISE):  63%|██████▎   | 32/51 [00:02<00:01, 14.19it/s]
Capturing CUDA graphs (PIECEWISE):  67%|██████▋   | 34/51 [00:02<00:01, 14.24it/s]
Capturing CUDA graphs (PIECEWISE):  71%|███████   | 36/51 [00:02<00:01, 14.24it/s]
Capturing CUDA graphs (PIECEWISE):  75%|███████▍  | 38/51 [00:02<00:00, 14.26it/s]
Capturing CUDA graphs (PIECEWISE):  78%|███████▊  | 40/51 [00:02<00:00, 14.20it/s]
Capturing CUDA graphs (PIECEWISE):  82%|████████▏ | 42/51 [00:02<00:00, 14.25it/s]
Capturing CUDA graphs (PIECEWISE):  86%|████████▋ | 44/51 [00:03<00:00, 14.33it/s]
Capturing CUDA graphs (PIECEWISE):  90%|█████████ | 46/51 [00:03<00:00, 14.37it/s]
Capturing CUDA graphs (PIECEWISE):  94%|█████████▍| 48/51 [00:03<00:00, 14.34it/s]
Capturing CUDA graphs (PIECEWISE):  98%|█████████▊| 50/51 [00:03<00:00, 14.26it/s]
Capturing CUDA graphs (PIECEWISE): 100%|██████████| 51/51 [00:03<00:00, 14.29it/s]

Capturing CUDA graphs (FULL):   0%|          | 0/35 [00:00<?, ?it/s]
Capturing CUDA graphs (FULL):   9%|▊         | 3/35 [00:00<00:01, 24.49it/s]
Capturing CUDA graphs (FULL):  17%|█▋        | 6/35 [00:00<00:01, 24.04it/s]
Capturing CUDA graphs (FULL):  26%|██▌       | 9/35 [00:00<00:01, 24.13it/s]
Capturing CUDA graphs (FULL):  34%|███▍      | 12/35 [00:00<00:00, 24.35it/s]
Capturing CUDA graphs (FULL):  43%|████▎     | 15/35 [00:00<00:00, 24.52it/s]
Capturing CUDA graphs (FULL):  51%|█████▏    | 18/35 [00:00<00:00, 24.75it/s]
Capturing CUDA graphs (FULL):  60%|██████    | 21/35 [00:00<00:00, 24.73it/s]
Capturing CUDA graphs (FULL):  69%|██████▊   | 24/35 [00:00<00:00, 24.43it/s]
Capturing CUDA graphs (FULL):  77%|███████▋  | 27/35 [00:01<00:00, 24.79it/s]
Capturing CUDA graphs (FULL):  86%|████████▌ | 30/35 [00:01<00:00, 25.19it/s]
Capturing CUDA graphs (FULL):  94%|█████████▍| 33/35 [00:01<00:00, 24.95it/s]
Capturing CUDA graphs (FULL): 100%|██████████| 35/35 [00:01<00:00, 24.74it/s]
(EngineCore pid=152420) INFO 11-01 02:17:04 [model_runner.py:1127] Graph capturing finished in 6 secs, took 0.23 GiB
(EngineCore pid=152420) INFO 11-01 02:17:04 [gpu_worker.py:971] CUDA graph pool memory: 0.23 GiB (actual), 0.46 GiB (estimated), difference: 0.23 GiB (100.9%).
(EngineCore pid=152420) INFO 11-01 02:17:04 [gpu_worker.py:1034] Free memory on device (78.84/79.25 GiB) on startup. Desired GPU memory utilization is (0.9, 71.33 GiB). Actual usage is 5.1 GiB for consumed memory (weights + non-torch), 0.96 GiB for peak activation, and 0.23 GiB for CUDAGraph memory. Replace gpu_memory_utilization config with `--kv-cache-memory=69675340698` (64.89 GiB) to fit into requested memory, or `--kv-cache-memory=77741406720` (72.4 GiB) to fully utilize gpu memory. Current kv cache memory in use is 65.26 GiB.
(EngineCore pid=152420) INFO 11-01 02:17:04 [jit_monitor.py:84] Kernel JIT monitor activated; monitored JIT compilations during inference will use mode=warn.
(EngineCore pid=152420) INFO 11-01 02:17:05 [core.py:395] init engine (profile, create kv cache, warmup model) took 56.14 s (compilation: 22.15 s)
(EngineCore pid=152420) INFO 11-01 02:17:05 [kernel.py:467] Final IR op priority after setting platform defaults: IrOpPriorityConfig(rms_norm=['native'], fused_add_rms_norm=['native'], gelu_and_mul_sparse=['triton', 'native'])
(MainProcess pid=152146) WARNING 11-01 02:17:06 [input_processor.py:408] Passing raw prompts to InputProcessor is deprecated and will be removed in the future. You should instead pass the outputs of Renderer.render_cmpl() or Renderer.render_chat().
(EngineCore pid=152420) WARNING 11-01 02:17:06 [jit_monitor.py:140] Triton kernel JIT compilation during inference: _topk_log_softmax_kernel. This causes a latency spike; consider extending warmup to cover this shape/config.
readout base2b: 5957 items in 47s
(MainProcess pid=152146) INFO 11-01 02:17:52 [core_client.py:770] [shutdown] MPClient: start timeout=default
(MainProcess pid=152146) INFO 11-01 02:17:52 [core_client.py:772] [shutdown] MPClient: stopping engine manager
(MainProcess pid=152146) INFO 11-01 02:17:52 [utils.py:628] [shutdown] Process manager: send sigterm to process EngineCore
(EngineCore pid=152420) INFO 11-01 02:17:52 [core.py:1454] [shutdown] EngineCore: trigger received signal=SIGTERM
(EngineCore pid=152420) INFO 11-01 02:17:52 [core.py:1635] [shutdown] EngineCore: start mode=abort timeout=0s
(EngineCore pid=152420) INFO 11-01 02:17:52 [core.py:1666] [shutdown] EngineCore: request processing complete; starting resource teardown
(EngineCore pid=152420) INFO 11-01 02:17:52 [core.py:1499] [shutdown] EngineCore: exiting busy loop
(MainProcess pid=152146) INFO 11-01 02:17:57 [core_client.py:774] [shutdown] MPClient: engine manager stopped
(MainProcess pid=152146) INFO 11-01 02:17:57 [core_client.py:775] [shutdown] MPClient: cleaning up background resources
(MainProcess pid=152146) INFO 11-01 02:17:57 [core_client.py:777] [shutdown] MPClient: complete
{"time": "2026-11-01T02:17:59Z", "event": "done", "stage": "2b_eval_base"}
{"time": "2026-11-01T02:18:29Z", "event": "start", "stage": "2b_eval_joint"}
(MainProcess pid=153522) INFO 11-01 02:18:40 [model.py:733] Resolved architecture: LlamaForCausalLM
(MainProcess pid=153522) INFO 11-01 02:18:40 [model.py:2152] Using max model len 32768
(MainProcess pid=153522) INFO 11-01 02:18:40 [scheduler.py:307] Chunked prefill is enabled with max_num_batched_tokens=2048.
(MainProcess pid=153522) INFO 11-01 02:18:40 [kernel.py:467] Final IR op priority after setting platform defaults: IrOpPriorityConfig(rms_norm=['native'], fused_add_rms_norm=['native'], gelu_and_mul_sparse=['triton', 'native'])
(EngineCore pid=153795) INFO 11-01 02:18:43 [core.py:133] Initializing a V1 LLM engine (v0.32.0) with config: model='checkpoints/v2b_joint_merged', speculative_config=None, tokenizer='checkpoints/v2b_joint_merged', skip_tokenizer_init=False, tokenizer_mode=auto, revision=None, tokenizer_revision=None, trust_remote_code=False, dtype=torch.bfloat16, max_seq_len=32768, download_dir=None, load_format=auto, tensor_parallel_size=1, pipeline_parallel_size=1, data_parallel_size=1, decode_context_parallel_size=1, dcp_comm_backend=ag_rs, disable_custom_all_reduce=False, quantization=None, quantization_config=None, enforce_eager=False, aux_output_config=AuxOutputConfig(enable_return_routed_experts=False, max_bytes=None), kv_cache_dtype=auto, device_config=cuda, structured_outputs_config=StructuredOutputsConfig(backend='auto', disable_any_whitespace=False, disable_additional_properties=False, reasoning_parser='', reasoning_parser_plugin='', enable_in_reasoning=False), observability_config=ObservabilityConfig(show_hidden_metrics_for_version=None, otlp_traces_endpoint=None, collect_detailed_traces=None, per_request_spec_decode_metrics='none', kv_cache_metrics=False, kv_cache_metrics_sample=0.01, custom_histogram_buckets=None, cudagraph_metrics=False, enable_layerwise_nvtx_tracing=False, enable_mfu_metrics=False, enable_mm_processor_stats=False, enable_logging_iteration_details=False, jit_monitor_mode='warn', jit_monitor_verbose=False), seed=17, served_model_name=checkpoints/v2b_joint_merged, enable_prefix_caching=True, enable_chunked_prefill=True, pooler_config=None, compilation_config={'mode': <CompilationMode.VLLM_COMPILE: 3>, 'debug_dump_path': None, 'cache_dir': '', 'compile_cache_save_format': 'binary', 'backend': 'inductor', 'custom_ops': ['none'], 'ir_enable_torch_wrap': True, 'splitting_ops': ['vllm::unified_attention_with_output', 'vllm::unified_mla_attention_with_output', 'vllm::mamba_mixer2', 'vllm::mamba_mixer', 'vllm::short_conv', 'vllm::qwen4_exp_ple_short_conv', 'vllm::qwen4_exp_qsa_with_output', 'vllm::qwen4_exp_qsa_prepare_with_output', 'vllm::linear_attention', 'vllm::qwen_gdn_attention_core', 'vllm::qwen_gdn_attention_core_fused_norm_packed', 'vllm::gdn_attention_core_xpu', 'vllm::olmo_hybrid_gdn_full_forward', 'vllm::sparse_attn_indexer', 'vllm::rocm_aiter_sparse_attn_indexer', 'vllm::deepseek_v4_attention', 'vllm::hpc_rope_norm_forward', 'vllm::unified_kv_cache_update', 'vllm::unified_mla_kv_cache_update'], 'compile_mm_encoder': False, 'cudagraph_mm_encoder': False, 'encoder_cudagraph_token_budgets': [], 'encoder_cudagraph_max_vision_items_per_batch': 0, 'encoder_cudagraph_max_frames_per_batch': None, 'cudagraph_decoder_replay': True, 'decoder_replay_cudagraph_capture_sizes': [], 'decoder_replay_trim_threshold': 768, 'compile_sizes': [], 'compile_ranges_endpoints': [2048], 'inductor_compile_config': {'enable_auto_functionalized_v2': False, 'combo_kernels': True, 'benchmark_combo_kernel': True}, 'inductor_passes': {}, 'cudagraph_mode': <CUDAGraphMode.FULL_AND_PIECEWISE: (2, 1)>, 'cudagraph_num_of_warmups': 1, 'cudagraph_capture_sizes': [1, 2, 4, 8, 16, 24, 32, 40, 48, 56, 64, 72, 80, 88, 96, 104, 112, 120, 128, 136, 144, 152, 160, 168, 176, 184, 192, 200, 208, 216, 224, 232, 240, 248, 256, 272, 288, 304, 320, 336, 352, 368, 384, 400, 416, 432, 448, 464, 480, 496, 512], 'cudagraph_copy_inputs': False, 'cudagraph_specialize_lora': True, 'use_inductor_graph_partition': False, 'pass_config': {'fuse_norm_quant': False, 'fuse_act_quant': False, 'fuse_attn_quant': False, 'fuse_allreduce_rms': False, 'enable_qk_norm_rope_fusion': False, 'fuse_rope_kvcache_cat_mla': False, 'fuse_act_padding': False, 'fuse_qk_norm_rope_kvcache': False}, 'max_cudagraph_capture_size': 512, 'dynamic_shapes_config': {'type': <DynamicShapesType.BACKED: 'backed'>, 'evaluate_guards': False, 'assume_32_bit_indexing': False}, 'local_cache_dir': None, 'fast_moe_cold_start': False, 'static_all_moe_layers': []}, kernel_config=KernelConfig(ir_op_priority=IrOpPriorityConfig(rms_norm=['native'], fused_add_rms_norm=['native'], gelu_and_mul_sparse=['triton', 'native']), enable_flashinfer_autotune=True, enable_rocm_segmented_attn_autotune=False, enable_cutedsl_warmup=True, enable_jit_warmup=True, moe_backend='auto', sparse_indexer_topk_backend='auto', linear_backend='auto', linear_backend_per_quant=None)
(EngineCore pid=153795) INFO 11-01 02:18:44 [parallel_state.py:1906] world_size=1 rank=0 local_rank=0 distributed_init_method=file:///tmp/vllm_dist_024de2e71b2a4f748d89558f7b36c0e6 backend=nccl
(EngineCore pid=153795) INFO 11-01 02:18:45 [parallel_state.py:2350] rank 0 in world size 1 is assigned as DP rank 0, PP rank 0, PCP rank 0, TP rank 0, ETP rank 0, EP rank N/A, EPLB rank N/A
(EngineCore pid=153795) INFO 11-01 02:18:45 [gpu_worker.py:518] Using V2 Model Runner
(EngineCore pid=153795) INFO 11-01 02:18:46 [model_runner.py:392] Loading model from scratch...
(EngineCore pid=153795) INFO 11-01 02:18:47 [cuda.py:506] Using FLASH_ATTN attention backend out of potential backends: ['FLASH_ATTN', 'FLASHINFER', 'TRITON_ATTN', 'FLEX_ATTENTION'].
(EngineCore pid=153795) INFO 11-01 02:18:47 [flash_attn.py:1117] Using FlashAttention version 2
(EngineCore pid=153795) INFO 11-01 02:18:47 [base_loader.py:82] Initializing model took 1.548803 seconds
(EngineCore pid=153795) INFO 11-01 02:18:47 [weight_utils.py:1108] Filesystem type for checkpoints: XFS. Checkpoint size: 4.69 GiB. Available RAM: 129.06 GiB.
(EngineCore pid=153795) INFO 11-01 02:18:47 [weight_utils.py:1131] Auto-prefetch is disabled because the filesystem (XFS) is not a recognized network FS (NFS/Lustre). If you want to force prefetching, start vLLM with --safetensors-load-strategy=prefetch.

Loading safetensors checkpoint shards:   0% Completed | 0/1 [00:00<?, ?it/s]

Loading safetensors checkpoint shards: 100% Completed | 1/1 [00:00<00:00,  1.35it/s]

Loading safetensors checkpoint shards: 100% Completed | 1/1 [00:00<00:00,  1.35it/s]

(EngineCore pid=153795) INFO 11-01 02:18:48 [default_loader.py:501] Loading weights took 0.76 seconds
(EngineCore pid=153795) INFO 11-01 02:18:50 [model_runner.py:424] Model loading took 4.75 GiB memory and 4.677984 seconds
(EngineCore pid=153795) INFO 11-01 02:18:50 [topk_topp_sampler.py:124] Using FlashInfer for top-p & top-k sampling.
(EngineCore pid=153795) INFO 11-01 02:18:50 [torch_utils.py:295] Reducing Torch threads from 128 to 1 for serving. Set OMP_NUM_THREADS in the external environment to override.
(EngineCore pid=153795) INFO 11-01 02:18:50 [utils.py:347] Using LBNHC KV cache layout.
(EngineCore pid=153795) INFO 11-01 02:18:51 [monitor.py:40] [backbone] Starting torch.compile
(EngineCore pid=153795) INFO 11-01 02:19:01 [backends.py:1104] [backbone] Using cache directory: /root/.cache/vllm/torch_compile_cache/b830caab8c/rank_0_0/backbone for vLLM's torch.compile
(EngineCore pid=153795) INFO 11-01 02:19:01 [backends.py:1166] [backbone] Dynamo bytecode transform time: 9.29 s
(EngineCore pid=153795) INFO 11-01 02:19:08 [backends.py:398] [backbone] Compiling a graph for compile range (1, 2048) takes 7.49 s
(EngineCore pid=153795) INFO 11-01 02:19:14 [backends.py:927] collected artifacts: 43 entries, 3 artifacts, 4889803 bytes total
(EngineCore pid=153795) INFO 11-01 02:19:14 [decorators.py:743] [backbone] saved AOT compiled function to /root/.cache/vllm/torch_compile_cache/torch_aot_compile/8b11e22f345a5dc7763ac9fdd8eae668df4da66c816459ebdb605500ec426e7a/rank_0_0/model
(EngineCore pid=153795) INFO 11-01 02:19:14 [monitor.py:62] [backbone] torch.compile took 22.52 s in total
(EngineCore pid=153795) INFO 11-01 02:19:14 [monitor.py:91] [backbone] Initial profiling/warmup run took 0.19 s

Capturing CUDA graphs (PIECEWISE):   0%|          | 0/51 [00:00<?, ?it/s]
Capturing CUDA graphs (PIECEWISE):   2%|▏         | 1/51 [00:00<00:08,  6.04it/s]
Capturing CUDA graphs (PIECEWISE):   6%|▌         | 3/51 [00:00<00:04, 10.22it/s]
Capturing CUDA graphs (PIECEWISE):  10%|▉         | 5/51 [00:00<00:04, 11.17it/s]
Capturing CUDA graphs (PIECEWISE):  14%|█▎        | 7/51 [00:00<00:04,  9.90it/s]
Capturing CUDA graphs (PIECEWISE):  18%|█▊        | 9/51 [00:00<00:03, 11.21it/s]
Capturing CUDA graphs (PIECEWISE):  22%|██▏       | 11/51 [00:01<00:03, 10.63it/s]
Capturing CUDA graphs (PIECEWISE):  25%|██▌       | 13/51 [00:01<00:03, 10.91it/s]
Capturing CUDA graphs (PIECEWISE):  29%|██▉       | 15/51 [00:01<00:03, 11.85it/s]
Capturing CUDA graphs (PIECEWISE):  33%|███▎      | 17/51 [00:01<00:03,  9.73it/s]
Capturing CUDA graphs (PIECEWISE):  37%|███▋      | 19/51 [00:01<00:02, 10.79it/s]
Capturing CUDA graphs (PIECEWISE):  41%|████      | 21/51 [00:01<00:02, 11.22it/s]
Capturing CUDA graphs (PIECEWISE):  45%|████▌     | 23/51 [00:02<00:02,  9.90it/s]
Capturing CUDA graphs (PIECEWISE):  49%|████▉     | 25/51 [00:02<00:02, 10.66it/s]
Capturing CUDA graphs (PIECEWISE):  53%|█████▎    | 27/51 [00:02<00:02,  9.80it/s]
Capturing CUDA graphs (PIECEWISE):  57%|█████▋    | 29/51 [00:02<00:02, 10.69it/s]
Capturing CUDA graphs (PIECEWISE):  61%|██████    | 31/51 [00:02<00:01, 11.14it/s]
Capturing CUDA graphs (PIECEWISE):  65%|██████▍   | 33/51 [00:03<00:01, 10.76it/s]
Capturing CUDA graphs (PIECEWISE):  69%|██████▊   | 35/51 [00:03<00:01, 11.63it/s]
Capturing CUDA graphs (PIECEWISE):  73%|███████▎  | 37/51 [00:03<00:01, 11.93it/s]
Capturing CUDA graphs (PIECEWISE):  76%|███████▋  | 39/51 [00:03<00:01, 10.62it/s]
Capturing CUDA graphs (PIECEWISE):  80%|████████  | 41/51 [00:03<00:00, 11.45it/s]
Capturing CUDA graphs (PIECEWISE):  84%|████████▍ | 43/51 [00:04<00:00, 10.31it/s]
Capturing CUDA graphs (PIECEWISE):  88%|████████▊ | 45/51 [00:04<00:00, 11.12it/s]
Capturing CUDA graphs (PIECEWISE):  92%|█████████▏| 47/51 [00:04<00:00, 11.83it/s]
Capturing CUDA graphs (PIECEWISE):  96%|█████████▌| 49/51 [00:04<00:00, 10.06it/s]
Capturing CUDA graphs (PIECEWISE): 100%|██████████| 51/51 [00:04<00:00, 10.52it/s]
Capturing CUDA graphs (PIECEWISE): 100%|██████████| 51/51 [00:04<00:00, 10.71it/s]

Capturing CUDA graphs (FULL):   0%|          | 0/2 [00:00<?, ?it/s]
Capturing CUDA graphs (FULL): 100%|██████████| 2/2 [00:00<00:00, 22.50it/s]
(EngineCore pid=153795) INFO 11-01 02:19:22 [model_runner.py:1127] Graph capturing finished in 6 secs, took 0.40 GiB
(EngineCore pid=153795) INFO 11-01 02:19:22 [gpu_worker.py:759] Available KV cache memory: 66.85 GiB
(EngineCore pid=153795) INFO 11-01 02:19:22 [gpu_worker.py:774] CUDA graph memory profiling is enabled (default since v0.21.0). The current --gpu-memory-utilization=0.9200 is equivalent to --gpu-memory-utilization=0.9143 without CUDA graph memory profiling. To maintain the same effective KV cache size as before, increase --gpu-memory-utilization to 0.9257. To disable, set VLLM_MEMORY_PROFILER_ESTIMATE_CUDAGRAPHS=0.
(EngineCore pid=153795) INFO 11-01 02:19:22 [kv_cache_utils.py:2327] GPU KV cache size: 1,668,928 tokens, Maximum concurrency for 32,768 tokens per request: 50.93x
(EngineCore pid=153795) INFO 11-01 02:19:22 [kernel_warmup.py:176] JIT kernel warmup starting.
(EngineCore pid=153795) INFO 11-01 02:19:22 [kernel_warmup.py:189] JIT kernel warmup finished in 0.00s.

Capturing CUDA graphs (PIECEWISE):   0%|          | 0/51 [00:00<?, ?it/s]
Capturing CUDA graphs (PIECEWISE):   4%|▍         | 2/51 [00:00<00:03, 12.89it/s]
Capturing CUDA graphs (PIECEWISE):   8%|▊         | 4/51 [00:00<00:03, 12.60it/s]
Capturing CUDA graphs (PIECEWISE):  12%|█▏        | 6/51 [00:00<00:04, 10.24it/s]
Capturing CUDA graphs (PIECEWISE):  16%|█▌        | 8/51 [00:00<00:03, 11.34it/s]
Capturing CUDA graphs (PIECEWISE):  20%|█▉        | 10/51 [00:00<00:04, 10.16it/s]
Capturing CUDA graphs (PIECEWISE):  24%|██▎       | 12/51 [00:01<00:03, 11.18it/s]
Capturing CUDA graphs (PIECEWISE):  27%|██▋       | 14/51 [00:01<00:03, 12.21it/s]
Capturing CUDA graphs (PIECEWISE):  31%|███▏      | 16/51 [00:01<00:03, 10.52it/s]
Capturing CUDA graphs (PIECEWISE):  35%|███▌      | 18/51 [00:01<00:03, 10.75it/s]
Capturing CUDA graphs (PIECEWISE):  39%|███▉      | 20/51 [00:01<00:02, 11.29it/s]
Capturing CUDA graphs (PIECEWISE):  43%|████▎     | 22/51 [00:01<00:02, 10.90it/s]
Capturing CUDA graphs (PIECEWISE):  47%|████▋     | 24/51 [00:02<00:02, 11.67it/s]
Capturing CUDA graphs (PIECEWISE):  51%|█████     | 26/51 [00:02<00:02, 10.45it/s]
Capturing CUDA graphs (PIECEWISE):  55%|█████▍    | 28/51 [00:02<00:02, 11.15it/s]
Capturing CUDA graphs (PIECEWISE):  59%|█████▉    | 30/51 [00:02<00:01, 11.91it/s]
Capturing CUDA graphs (PIECEWISE):  63%|██████▎   | 32/51 [00:02<00:01, 10.19it/s]
Capturing CUDA graphs (PIECEWISE):  67%|██████▋   | 34/51 [00:03<00:01, 11.10it/s]
Capturing CUDA graphs (PIECEWISE):  71%|███████   | 36/51 [00:03<00:01, 11.92it/s]
Capturing CUDA graphs (PIECEWISE):  75%|███████▍  | 38/51 [00:03<00:01, 12.23it/s]
Capturing CUDA graphs (PIECEWISE):  78%|███████▊  | 40/51 [00:03<00:00, 12.63it/s]
Capturing CUDA graphs (PIECEWISE):  82%|████████▏ | 42/51 [00:03<00:00, 13.04it/s]
Capturing CUDA graphs (PIECEWISE):  86%|████████▋ | 44/51 [00:03<00:00, 13.05it/s]
Capturing CUDA graphs (PIECEWISE):  90%|█████████ | 46/51 [00:03<00:00, 12.85it/s]
Capturing CUDA graphs (PIECEWISE):  94%|█████████▍| 48/51 [00:04<00:00, 13.15it/s]
Capturing CUDA graphs (PIECEWISE):  98%|█████████▊| 50/51 [00:04<00:00, 11.02it/s]
Capturing CUDA graphs (PIECEWISE): 100%|██████████| 51/51 [00:04<00:00, 11.51it/s]

Capturing CUDA graphs (FULL):   0%|          | 0/35 [00:00<?, ?it/s]
Capturing CUDA graphs (FULL):   9%|▊         | 3/35 [00:00<00:01, 24.27it/s]
Capturing CUDA graphs (FULL):  17%|█▋        | 6/35 [00:00<00:01, 23.19it/s]
Capturing CUDA graphs (FULL):  26%|██▌       | 9/35 [00:00<00:01, 19.64it/s]
Capturing CUDA graphs (FULL):  34%|███▍      | 12/35 [00:00<00:01, 21.30it/s]
Capturing CUDA graphs (FULL):  43%|████▎     | 15/35 [00:00<00:00, 21.75it/s]
Capturing CUDA graphs (FULL):  51%|█████▏    | 18/35 [00:00<00:00, 21.47it/s]
Capturing CUDA graphs (FULL):  60%|██████    | 21/35 [00:00<00:00, 22.08it/s]
Capturing CUDA graphs (FULL):  69%|██████▊   | 24/35 [00:01<00:00, 22.48it/s]
Capturing CUDA graphs (FULL):  77%|███████▋  | 27/35 [00:01<00:00, 23.25it/s]
Capturing CUDA graphs (FULL):  86%|████████▌ | 30/35 [00:01<00:00, 21.86it/s]
Capturing CUDA graphs (FULL):  94%|█████████▍| 33/35 [00:01<00:00, 22.56it/s]
Capturing CUDA graphs (FULL): 100%|██████████| 35/35 [00:01<00:00, 22.30it/s]
(EngineCore pid=153795) INFO 11-01 02:19:47 [model_runner.py:1127] Graph capturing finished in 7 secs, took 0.23 GiB
(EngineCore pid=153795) INFO 11-01 02:19:47 [gpu_worker.py:971] CUDA graph pool memory: 0.23 GiB (actual), 0.46 GiB (estimated), difference: 0.23 GiB (100.9%).
(EngineCore pid=153795) INFO 11-01 02:19:47 [gpu_worker.py:1034] Free memory on device (78.84/79.25 GiB) on startup. Desired GPU memory utilization is (0.92, 72.91 GiB). Actual usage is 5.1 GiB for consumed memory (weights + non-torch), 0.96 GiB for peak activation, and 0.23 GiB for CUDAGraph memory. Replace gpu_memory_utilization config with `--kv-cache-memory=71377237218` (66.48 GiB) to fit into requested memory, or `--kv-cache-memory=77741406720` (72.4 GiB) to fully utilize gpu memory. Current kv cache memory in use is 66.85 GiB.
(EngineCore pid=153795) INFO 11-01 02:19:48 [jit_monitor.py:84] Kernel JIT monitor activated; monitored JIT compilations during inference will use mode=warn.
(EngineCore pid=153795) INFO 11-01 02:19:48 [core.py:395] init engine (profile, create kv cache, warmup model) took 57.97 s (compilation: 22.52 s)
(EngineCore pid=153795) INFO 11-01 02:19:48 [kernel.py:467] Final IR op priority after setting platform defaults: IrOpPriorityConfig(rms_norm=['native'], fused_add_rms_norm=['native'], gelu_and_mul_sparse=['triton', 'native'])
(MainProcess pid=153522) WARNING 11-01 02:19:50 [input_processor.py:408] Passing raw prompts to InputProcessor is deprecated and will be removed in the future. You should instead pass the outputs of Renderer.render_cmpl() or Renderer.render_chat().
(EngineCore pid=153795) WARNING 11-01 02:19:50 [jit_monitor.py:140] Triton kernel JIT compilation during inference: _topk_log_softmax_kernel. This causes a latency spike; consider extending warmup to cover this shape/config.
readout v2b: 5957 items in 48s
(EngineCore pid=153795) WARNING 11-01 02:20:37 [jit_monitor.py:140] Triton kernel JIT compilation during inference: _topp_sb_stats_kernel. This causes a latency spike; consider extending warmup to cover this shape/config.
(EngineCore pid=153795) WARNING 11-01 02:20:38 [jit_monitor.py:140] Triton kernel JIT compilation during inference: _topp_sb_step_kernel. This causes a latency spike; consider extending warmup to cover this shape/config.
(EngineCore pid=153795) WARNING 11-01 02:20:39 [jit_monitor.py:140] Triton kernel JIT compilation during inference: _topp_sb_mask_kernel. This causes a latency spike; consider extending warmup to cover this shape/config.
progress {"time": "02:21:36", "elapsed_min": 1.0, "submitted": 420, "finished": 164, "in_flight": 256, "gen_tokens": 117898, "tok_per_s": 1965, "chain_tokens_median": 686.0, "chain_tokens_p90": 1076.6000000000004, "chain_tokens_max": 1319, "parse_rate": 1.0, "looped": 0, "minutes_to_deadline": Infinity}
(MainProcess pid=153522) INFO 11-01 02:21:36 [loggers.py:335] Engine 000: Avg prompt throughput: 20610.7 tokens/s, Avg generation throughput: 3204.8 tokens/s, Running: 256 reqs, Waiting: 0 reqs, GPU KV cache usage: 22.6%, Prefix cache hit rate: 37.1%
progress {"time": "02:22:36", "elapsed_min": 2.0, "submitted": 579, "finished": 323, "in_flight": 256, "gen_tokens": 286033, "tok_per_s": 2383, "chain_tokens_median": 740.0, "chain_tokens_p90": 1548.2, "chain_tokens_max": 2677, "parse_rate": 1.0, "looped": 0, "minutes_to_deadline": Infinity}
(MainProcess pid=153522) INFO 11-01 02:22:36 [loggers.py:335] Engine 000: Avg prompt throughput: 730.7 tokens/s, Avg generation throughput: 5817.3 tokens/s, Running: 256 reqs, Waiting: 0 reqs, GPU KV cache usage: 32.2%, Prefix cache hit rate: 61.0%
progress {"time": "02:23:36", "elapsed_min": 3.0, "submitted": 689, "finished": 433, "in_flight": 256, "gen_tokens": 434469, "tok_per_s": 2414, "chain_tokens_median": 762.0, "chain_tokens_p90": 2057.0000000000005, "chain_tokens_max": 3750, "parse_rate": 1.0, "looped": 0, "minutes_to_deadline": Infinity}
(MainProcess pid=153522) INFO 11-01 02:23:36 [loggers.py:335] Engine 000: Avg prompt throughput: 742.7 tokens/s, Avg generation throughput: 4608.8 tokens/s, Running: 255 reqs, Waiting: 0 reqs, GPU KV cache usage: 39.8%, Prefix cache hit rate: 71.9%
progress {"time": "02:24:36", "elapsed_min": 4.0, "submitted": 812, "finished": 556, "in_flight": 256, "gen_tokens": 621314, "tok_per_s": 2589, "chain_tokens_median": 776.5, "chain_tokens_p90": 2610.0, "chain_tokens_max": 4534, "parse_rate": 1.0, "looped": 0, "minutes_to_deadline": Infinity}
(MainProcess pid=153522) INFO 11-01 02:24:36 [loggers.py:335] Engine 000: Avg prompt throughput: 947.5 tokens/s, Avg generation throughput: 4377.7 tokens/s, Running: 254 reqs, Waiting: 0 reqs, GPU KV cache usage: 44.9%, Prefix cache hit rate: 81.8%
progress {"time": "02:25:36", "elapsed_min": 5.0, "submitted": 919, "finished": 663, "in_flight": 256, "gen_tokens": 813730, "tok_per_s": 2712, "chain_tokens_median": 802.0, "chain_tokens_p90": 2806.2000000000007, "chain_tokens_max": 5653, "parse_rate": 1.0, "looped": 0, "minutes_to_deadline": Infinity}
(MainProcess pid=153522) INFO 11-01 02:25:36 [loggers.py:335] Engine 000: Avg prompt throughput: 638.4 tokens/s, Avg generation throughput: 4067.4 tokens/s, Running: 256 reqs, Waiting: 0 reqs, GPU KV cache usage: 48.2%, Prefix cache hit rate: 83.3%
progress {"time": "02:26:36", "elapsed_min": 6.0, "submitted": 1031, "finished": 775, "in_flight": 256, "gen_tokens": 967494, "tok_per_s": 2687, "chain_tokens_median": 796.0, "chain_tokens_p90": 2893.2000000000007, "chain_tokens_max": 6657, "parse_rate": 1.0, "looped": 0, "minutes_to_deadline": Infinity}
(MainProcess pid=153522) INFO 11-01 02:26:36 [loggers.py:335] Engine 000: Avg prompt throughput: 480.9 tokens/s, Avg generation throughput: 3759.9 tokens/s, Running: 256 reqs, Waiting: 0 reqs, GPU KV cache usage: 52.0%, Prefix cache hit rate: 83.9%
progress {"time": "02:27:36", "elapsed_min": 7.0, "submitted": 1126, "finished": 870, "in_flight": 256, "gen_tokens": 1164015, "tok_per_s": 2771, "chain_tokens_median": 810.5, "chain_tokens_p90": 3232.1, "chain_tokens_max": 7374, "parse_rate": 1.0, "looped": 0, "minutes_to_deadline": Infinity}
(MainProcess pid=153522) INFO 11-01 02:27:36 [loggers.py:335] Engine 000: Avg prompt throughput: 310.0 tokens/s, Avg generation throughput: 3901.8 tokens/s, Running: 256 reqs, Waiting: 0 reqs, GPU KV cache usage: 53.5%, Prefix cache hit rate: 86.2%
progress {"time": "02:28:36", "elapsed_min": 8.0, "submitted": 1184, "finished": 928, "in_flight": 256, "gen_tokens": 1276993, "tok_per_s": 2660, "chain_tokens_median": 832.5, "chain_tokens_p90": 3288.5000000000005, "chain_tokens_max": 8336, "parse_rate": 1.0, "looped": 0, "minutes_to_deadline": Infinity}
(MainProcess pid=153522) INFO 11-01 02:28:36 [loggers.py:335] Engine 000: Avg prompt throughput: 217.4 tokens/s, Avg generation throughput: 4386.3 tokens/s, Running: 255 reqs, Waiting: 0 reqs, GPU KV cache usage: 62.3%, Prefix cache hit rate: 87.1%
progress {"time": "02:29:36", "elapsed_min": 9.0, "submitted": 1246, "finished": 990, "in_flight": 256, "gen_tokens": 1397915, "tok_per_s": 2589, "chain_tokens_median": 852.5, "chain_tokens_p90": 3357.500000000001, "chain_tokens_max": 8790, "parse_rate": 0.998989898989899, "looped": 0, "minutes_to_deadline": Infinity}
(MainProcess pid=153522) INFO 11-01 02:29:36 [loggers.py:335] Engine 000: Avg prompt throughput: 310.6 tokens/s, Avg generation throughput: 3797.3 tokens/s, Running: 256 reqs, Waiting: 0 reqs, GPU KV cache usage: 68.8%, Prefix cache hit rate: 88.0%
progress {"time": "02:30:36", "elapsed_min": 10.0, "submitted": 1306, "finished": 1050, "in_flight": 256, "gen_tokens": 1530805, "tok_per_s": 2551, "chain_tokens_median": 864.0, "chain_tokens_p90": 3453.5, "chain_tokens_max": 9535, "parse_rate": 0.9990476190476191, "looped": 0, "minutes_to_deadline": Infinity}
(MainProcess pid=153522) INFO 11-01 02:30:36 [loggers.py:335] Engine 000: Avg prompt throughput: 281.0 tokens/s, Avg generation throughput: 3507.4 tokens/s, Running: 256 reqs, Waiting: 0 reqs, GPU KV cache usage: 73.3%, Prefix cache hit rate: 89.4%
progress {"time": "02:31:36", "elapsed_min": 11.0, "submitted": 1362, "finished": 1106, "in_flight": 256, "gen_tokens": 1638603, "tok_per_s": 2483, "chain_tokens_median": 874.0, "chain_tokens_p90": 3577.0, "chain_tokens_max": 9535, "parse_rate": 0.9990958408679927, "looped": 0, "minutes_to_deadline": Infinity}
(MainProcess pid=153522) INFO 11-01 02:31:36 [loggers.py:335] Engine 000: Avg prompt throughput: 265.5 tokens/s, Avg generation throughput: 3383.8 tokens/s, Running: 256 reqs, Waiting: 0 reqs, GPU KV cache usage: 79.0%, Prefix cache hit rate: 90.0%
progress {"time": "02:32:36", "elapsed_min": 12.0, "submitted": 1409, "finished": 1153, "in_flight": 256, "gen_tokens": 1745814, "tok_per_s": 2425, "chain_tokens_median": 886.0, "chain_tokens_p90": 3725.599999999999, "chain_tokens_max": 9535, "parse_rate": 0.9991326973113617, "looped": 0, "minutes_to_deadline": Infinity}
(MainProcess pid=153522) INFO 11-01 02:32:36 [loggers.py:335] Engine 000: Avg prompt throughput: 217.3 tokens/s, Avg generation throughput: 3354.6 tokens/s, Running: 256 reqs, Waiting: 0 reqs, GPU KV cache usage: 84.8%, Prefix cache hit rate: 90.5%
progress {"time": "02:33:36", "elapsed_min": 13.0, "submitted": 1455, "finished": 1199, "in_flight": 256, "gen_tokens": 1860784, "tok_per_s": 2386, "chain_tokens_median": 894.0, "chain_tokens_p90": 3836.2, "chain_tokens_max": 9535, "parse_rate": 0.9991659716430359, "looped": 0, "minutes_to_deadline": Infinity}
(MainProcess pid=153522) INFO 11-01 02:33:36 [loggers.py:335] Engine 000: Avg prompt throughput: 190.9 tokens/s, Avg generation throughput: 3218.1 tokens/s, Running: 256 reqs, Waiting: 0 reqs, GPU KV cache usage: 89.4%, Prefix cache hit rate: 90.5%
progress {"time": "02:34:36", "elapsed_min": 14.0, "submitted": 1493, "finished": 1237, "in_flight": 256, "gen_tokens": 1939041, "tok_per_s": 2308, "chain_tokens_median": 905.0, "chain_tokens_p90": 3837.4, "chain_tokens_max": 9535, "parse_rate": 0.9991915925626516, "looped": 0, "minutes_to_deadline": Infinity}
(MainProcess pid=153522) INFO 11-01 02:34:36 [loggers.py:335] Engine 000: Avg prompt throughput: 158.9 tokens/s, Avg generation throughput: 3274.1 tokens/s, Running: 256 reqs, Waiting: 0 reqs, GPU KV cache usage: 96.4%, Prefix cache hit rate: 90.7%
progress {"time": "02:35:36", "elapsed_min": 15.0, "submitted": 1511, "finished": 1255, "in_flight": 256, "gen_tokens": 1970161, "tok_per_s": 2189, "chain_tokens_median": 908.0, "chain_tokens_p90": 3830.800000000002, "chain_tokens_max": 9535, "parse_rate": 0.999203187250996, "looped": 0, "minutes_to_deadline": Infinity}
(MainProcess pid=153522) INFO 11-01 02:35:36 [loggers.py:335] Engine 000: Avg prompt throughput: 68.4 tokens/s, Avg generation throughput: 2857.3 tokens/s, Running: 211 reqs, Waiting: 45 reqs, GPU KV cache usage: 100.0%, Prefix cache hit rate: 90.9%
progress {"time": "02:36:36", "elapsed_min": 16.0, "submitted": 1511, "finished": 1255, "in_flight": 256, "gen_tokens": 1970161, "tok_per_s": 2052, "chain_tokens_median": 908.0, "chain_tokens_p90": 3830.800000000002, "chain_tokens_max": 9535, "parse_rate": 0.999203187250996, "looped": 0, "minutes_to_deadline": Infinity}
(MainProcess pid=153522) INFO 11-01 02:36:36 [loggers.py:335] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 2514.8 tokens/s, Running: 159 reqs, Waiting: 97 reqs, GPU KV cache usage: 99.9%, Prefix cache hit rate: 90.9%
progress {"time": "02:37:36", "elapsed_min": 17.0, "submitted": 1511, "finished": 1255, "in_flight": 256, "gen_tokens": 1970161, "tok_per_s": 1931, "chain_tokens_median": 908.0, "chain_tokens_p90": 3830.800000000002, "chain_tokens_max": 9535, "parse_rate": 0.999203187250996, "looped": 0, "minutes_to_deadline": Infinity}
(MainProcess pid=153522) INFO 11-01 02:37:36 [loggers.py:335] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 2360.2 tokens/s, Running: 130 reqs, Waiting: 126 reqs, GPU KV cache usage: 99.8%, Prefix cache hit rate: 90.9%
progress {"time": "02:38:36", "elapsed_min": 18.0, "submitted": 1511, "finished": 1255, "in_flight": 256, "gen_tokens": 1970161, "tok_per_s": 1824, "chain_tokens_median": 908.0, "chain_tokens_p90": 3830.800000000002, "chain_tokens_max": 9535, "parse_rate": 0.999203187250996, "looped": 0, "minutes_to_deadline": Infinity}
(MainProcess pid=153522) INFO 11-01 02:38:36 [loggers.py:335] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 2223.1 tokens/s, Running: 111 reqs, Waiting: 145 reqs, GPU KV cache usage: 99.7%, Prefix cache hit rate: 90.9%
progress {"time": "02:39:36", "elapsed_min": 19.0, "submitted": 1511, "finished": 1255, "in_flight": 256, "gen_tokens": 1970161, "tok_per_s": 1728, "chain_tokens_median": 908.0, "chain_tokens_p90": 3830.800000000002, "chain_tokens_max": 9535, "parse_rate": 0.999203187250996, "looped": 0, "minutes_to_deadline": Infinity}
(MainProcess pid=153522) INFO 11-01 02:39:36 [loggers.py:335] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 1966.1 tokens/s, Running: 100 reqs, Waiting: 156 reqs, GPU KV cache usage: 99.8%, Prefix cache hit rate: 90.9%
progress {"time": "02:40:36", "elapsed_min": 20.0, "submitted": 1511, "finished": 1255, "in_flight": 256, "gen_tokens": 1970161, "tok_per_s": 1642, "chain_tokens_median": 908.0, "chain_tokens_p90": 3830.800000000002, "chain_tokens_max": 9535, "parse_rate": 0.999203187250996, "looped": 0, "minutes_to_deadline": Infinity}
(MainProcess pid=153522) INFO 11-01 02:40:36 [loggers.py:335] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 1777.2 tokens/s, Running: 91 reqs, Waiting: 165 reqs, GPU KV cache usage: 99.3%, Prefix cache hit rate: 90.9%
progress {"time": "02:41:36", "elapsed_min": 21.0, "submitted": 1511, "finished": 1255, "in_flight": 256, "gen_tokens": 1970161, "tok_per_s": 1564, "chain_tokens_median": 908.0, "chain_tokens_p90": 3830.800000000002, "chain_tokens_max": 9535, "parse_rate": 0.999203187250996, "looped": 0, "minutes_to_deadline": Infinity}
(MainProcess pid=153522) INFO 11-01 02:41:36 [loggers.py:335] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 1611.5 tokens/s, Running: 85 reqs, Waiting: 171 reqs, GPU KV cache usage: 99.6%, Prefix cache hit rate: 90.9%
progress {"time": "02:42:36", "elapsed_min": 22.0, "submitted": 1511, "finished": 1255, "in_flight": 256, "gen_tokens": 1970161, "tok_per_s": 1492, "chain_tokens_median": 908.0, "chain_tokens_p90": 3830.800000000002, "chain_tokens_max": 9535, "parse_rate": 0.999203187250996, "looped": 0, "minutes_to_deadline": Infinity}
(MainProcess pid=153522) INFO 11-01 02:42:36 [loggers.py:335] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 1517.0 tokens/s, Running: 79 reqs, Waiting: 177 reqs, GPU KV cache usage: 99.0%, Prefix cache hit rate: 90.9%
progress {"time": "02:43:36", "elapsed_min": 23.0, "submitted": 1511, "finished": 1255, "in_flight": 256, "gen_tokens": 1970161, "tok_per_s": 1428, "chain_tokens_median": 908.0, "chain_tokens_p90": 3830.800000000002, "chain_tokens_max": 9535, "parse_rate": 0.999203187250996, "looped": 0, "minutes_to_deadline": Infinity}
(MainProcess pid=153522) INFO 11-01 02:43:36 [loggers.py:335] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 1564.7 tokens/s, Running: 74 reqs, Waiting: 182 reqs, GPU KV cache usage: 99.0%, Prefix cache hit rate: 90.9%
progress {"time": "02:44:36", "elapsed_min": 24.0, "submitted": 1511, "finished": 1255, "in_flight": 256, "gen_tokens": 1970161, "tok_per_s": 1368, "chain_tokens_median": 908.0, "chain_tokens_p90": 3830.800000000002, "chain_tokens_max": 9535, "parse_rate": 0.999203187250996, "looped": 0, "minutes_to_deadline": Infinity}
(MainProcess pid=153522) INFO 11-01 02:44:36 [loggers.py:335] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 1461.5 tokens/s, Running: 70 reqs, Waiting: 186 reqs, GPU KV cache usage: 99.3%, Prefix cache hit rate: 90.9%
progress {"time": "02:45:36", "elapsed_min": 25.0, "submitted": 1511, "finished": 1255, "in_flight": 256, "gen_tokens": 1970161, "tok_per_s": 1313, "chain_tokens_median": 908.0, "chain_tokens_p90": 3830.800000000002, "chain_tokens_max": 9535, "parse_rate": 0.999203187250996, "looped": 0, "minutes_to_deadline": Infinity}
(MainProcess pid=153522) INFO 11-01 02:45:36 [loggers.py:335] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 1371.9 tokens/s, Running: 66 reqs, Waiting: 190 reqs, GPU KV cache usage: 99.0%, Prefix cache hit rate: 90.9%
progress {"time": "02:46:36", "elapsed_min": 26.0, "submitted": 1511, "finished": 1255, "in_flight": 256, "gen_tokens": 1970161, "tok_per_s": 1263, "chain_tokens_median": 908.0, "chain_tokens_p90": 3830.800000000002, "chain_tokens_max": 9535, "parse_rate": 0.999203187250996, "looped": 0, "minutes_to_deadline": Infinity}
(MainProcess pid=153522) INFO 11-01 02:46:36 [loggers.py:335] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 1300.3 tokens/s, Running: 63 reqs, Waiting: 193 reqs, GPU KV cache usage: 99.4%, Prefix cache hit rate: 90.9%
progress {"time": "02:47:36", "elapsed_min": 27.0, "submitted": 1511, "finished": 1255, "in_flight": 256, "gen_tokens": 1970161, "tok_per_s": 1216, "chain_tokens_median": 908.0, "chain_tokens_p90": 3830.800000000002, "chain_tokens_max": 9535, "parse_rate": 0.999203187250996, "looped": 0, "minutes_to_deadline": Infinity}
(MainProcess pid=153522) INFO 11-01 02:47:36 [loggers.py:335] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 1234.3 tokens/s, Running: 60 reqs, Waiting: 196 reqs, GPU KV cache usage: 99.4%, Prefix cache hit rate: 90.9%
progress {"time": "02:48:36", "elapsed_min": 28.0, "submitted": 1512, "finished": 1256, "in_flight": 256, "gen_tokens": 1997627, "tok_per_s": 1189, "chain_tokens_median": 908.5, "chain_tokens_p90": 3836.5, "chain_tokens_max": 27466, "parse_rate": 0.9984076433121019, "looped": 1, "minutes_to_deadline": Infinity}
(MainProcess pid=153522) INFO 11-01 02:48:36 [loggers.py:335] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 1077.8 tokens/s, Running: 58 reqs, Waiting: 198 reqs, GPU KV cache usage: 99.7%, Prefix cache hit rate: 90.9%
progress {"time": "02:49:36", "elapsed_min": 29.0, "submitted": 1514, "finished": 1258, "in_flight": 256, "gen_tokens": 2057009, "tok_per_s": 1182, "chain_tokens_median": 910.0, "chain_tokens_p90": 3845.199999999999, "chain_tokens_max": 29721, "parse_rate": 0.9968203497615262, "looped": 3, "minutes_to_deadline": Infinity}
(MainProcess pid=153522) INFO 11-01 02:49:36 [loggers.py:335] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 1043.2 tokens/s, Running: 56 reqs, Waiting: 200 reqs, GPU KV cache usage: 98.3%, Prefix cache hit rate: 90.9%
progress {"time": "02:50:36", "elapsed_min": 30.0, "submitted": 1515, "finished": 1259, "in_flight": 256, "gen_tokens": 2087154, "tok_per_s": 1159, "chain_tokens_median": 911.0, "chain_tokens_p90": 3862.6000000000004, "chain_tokens_max": 30145, "parse_rate": 0.9960285941223193, "looped": 4, "minutes_to_deadline": Infinity}
(MainProcess pid=153522) INFO 11-01 02:50:36 [loggers.py:335] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 985.9 tokens/s, Running: 54 reqs, Waiting: 202 reqs, GPU KV cache usage: 99.5%, Prefix cache hit rate: 90.9%
progress {"time": "02:51:36", "elapsed_min": 31.0, "submitted": 1520, "finished": 1264, "in_flight": 256, "gen_tokens": 2244331, "tok_per_s": 1207, "chain_tokens_median": 912.5, "chain_tokens_p90": 3932.5000000000027, "chain_tokens_max": 31589, "parse_rate": 0.9920886075949367, "looped": 9, "minutes_to_deadline": Infinity}
(MainProcess pid=153522) INFO 11-01 02:51:36 [loggers.py:335] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 677.9 tokens/s, Running: 54 reqs, Waiting: 202 reqs, GPU KV cache usage: 98.7%, Prefix cache hit rate: 90.9%
progress {"time": "02:52:36", "elapsed_min": 32.0, "submitted": 1529, "finished": 1273, "in_flight": 256, "gen_tokens": 2529648, "tok_per_s": 1317, "chain_tokens_median": 917.0, "chain_tokens_p90": 4095.5999999999995, "chain_tokens_max": 31819, "parse_rate": 0.9850746268656716, "looped": 18, "minutes_to_deadline": Infinity}
(MainProcess pid=153522) INFO 11-01 02:52:36 [loggers.py:335] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 180.2 tokens/s, Running: 58 reqs, Waiting: 198 reqs, GPU KV cache usage: 99.5%, Prefix cache hit rate: 90.9%
progress {"time": "02:53:36", "elapsed_min": 33.0, "submitted": 1538, "finished": 1282, "in_flight": 256, "gen_tokens": 2817363, "tok_per_s": 1423, "chain_tokens_median": 919.5, "chain_tokens_p90": 4277.000000000003, "chain_tokens_max": 32104, "parse_rate": 0.9781591263650546, "looped": 27, "minutes_to_deadline": Infinity}
(MainProcess pid=153522) INFO 11-01 02:53:36 [loggers.py:335] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 275.1 tokens/s, Running: 66 reqs, Waiting: 190 reqs, GPU KV cache usage: 99.4%, Prefix cache hit rate: 90.9%
progress {"time": "02:54:36", "elapsed_min": 34.0, "submitted": 1544, "finished": 1288, "in_flight": 256, "gen_tokens": 3009311, "tok_per_s": 1475, "chain_tokens_median": 924.5, "chain_tokens_p90": 4316.6, "chain_tokens_max": 32550, "parse_rate": 0.9736024844720497, "looped": 33, "minutes_to_deadline": Infinity}
(MainProcess pid=153522) INFO 11-01 02:54:36 [loggers.py:335] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 667.2 tokens/s, Running: 73 reqs, Waiting: 183 reqs, GPU KV cache usage: 99.4%, Prefix cache hit rate: 90.9%
progress {"time": "02:55:36", "elapsed_min": 35.0, "submitted": 1549, "finished": 1293, "in_flight": 256, "gen_tokens": 3170975, "tok_per_s": 1510, "chain_tokens_median": 925.0, "chain_tokens_p90": 4362.4, "chain_tokens_max": 32579, "parse_rate": 0.9698375870069605, "looped": 38, "minutes_to_deadline": Infinity}
(MainProcess pid=153522) INFO 11-01 02:55:36 [loggers.py:335] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 1042.4 tokens/s, Running: 75 reqs, Waiting: 181 reqs, GPU KV cache usage: 97.3%, Prefix cache hit rate: 90.9%
progress {"time": "02:56:36", "elapsed_min": 36.0, "submitted": 1558, "finished": 1302, "in_flight": 256, "gen_tokens": 3463067, "tok_per_s": 1603, "chain_tokens_median": 929.5, "chain_tokens_p90": 4480.6, "chain_tokens_max": 32635, "parse_rate": 0.9631336405529954, "looped": 47, "minutes_to_deadline": Infinity}
(MainProcess pid=153522) INFO 11-01 02:56:36 [loggers.py:335] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 333.0 tokens/s, Running: 140 reqs, Waiting: 116 reqs, GPU KV cache usage: 96.7%, Prefix cache hit rate: 90.9%
progress {"time": "02:57:36", "elapsed_min": 37.0, "submitted": 1606, "finished": 1350, "in_flight": 256, "gen_tokens": 3705205, "tok_per_s": 1669, "chain_tokens_median": 970.0, "chain_tokens_p90": 4722.900000000007, "chain_tokens_max": 32635, "parse_rate": 0.9629629629629629, "looped": 48, "minutes_to_deadline": Infinity}
(MainProcess pid=153522) INFO 11-01 02:57:36 [loggers.py:335] Engine 000: Avg prompt throughput: 4511.9 tokens/s, Avg generation throughput: 439.7 tokens/s, Running: 244 reqs, Waiting: 12 reqs, GPU KV cache usage: 99.2%, Prefix cache hit rate: 75.1%
progress {"time": "02:58:36", "elapsed_min": 38.0, "submitted": 1634, "finished": 1378, "in_flight": 256, "gen_tokens": 3868105, "tok_per_s": 1696, "chain_tokens_median": 968.0, "chain_tokens_p90": 4794.9, "chain_tokens_max": 32635, "parse_rate": 0.9608127721335269, "looped": 52, "minutes_to_deadline": Infinity}
(MainProcess pid=153522) INFO 11-01 02:58:36 [loggers.py:335] Engine 000: Avg prompt throughput: 750.0 tokens/s, Avg generation throughput: 2239.7 tokens/s, Running: 255 reqs, Waiting: 0 reqs, GPU KV cache usage: 98.9%, Prefix cache hit rate: 72.3%
progress {"time": "02:59:36", "elapsed_min": 39.0, "submitted": 1686, "finished": 1430, "in_flight": 256, "gen_tokens": 4080801, "tok_per_s": 1744, "chain_tokens_median": 949.5, "chain_tokens_p90": 4794.3, "chain_tokens_max": 32658, "parse_rate": 0.9587412587412587, "looped": 57, "minutes_to_deadline": Infinity}
(MainProcess pid=153522) INFO 11-01 02:59:36 [loggers.py:335] Engine 000: Avg prompt throughput: 684.8 tokens/s, Avg generation throughput: 2023.2 tokens/s, Running: 253 reqs, Waiting: 0 reqs, GPU KV cache usage: 94.4%, Prefix cache hit rate: 70.0%
progress {"time": "03:00:36", "elapsed_min": 40.0, "submitted": 1730, "finished": 1474, "in_flight": 256, "gen_tokens": 4142149, "tok_per_s": 1726, "chain_tokens_median": 964.0, "chain_tokens_p90": 4701.900000000002, "chain_tokens_max": 32658, "parse_rate": 0.9599728629579376, "looped": 57, "minutes_to_deadline": Infinity}
(MainProcess pid=153522) INFO 11-01 03:00:36 [loggers.py:335] Engine 000: Avg prompt throughput: 359.1 tokens/s, Avg generation throughput: 2121.7 tokens/s, Running: 254 reqs, Waiting: 0 reqs, GPU KV cache usage: 98.5%, Prefix cache hit rate: 68.7%
progress {"time": "03:01:36", "elapsed_min": 41.0, "submitted": 1768, "finished": 1512, "in_flight": 256, "gen_tokens": 4216273, "tok_per_s": 1714, "chain_tokens_median": 954.0, "chain_tokens_p90": 4651.5, "chain_tokens_max": 32658, "parse_rate": 0.9603174603174603, "looped": 58, "minutes_to_deadline": Infinity}
(MainProcess pid=153522) INFO 11-01 03:01:36 [loggers.py:335] Engine 000: Avg prompt throughput: 288.6 tokens/s, Avg generation throughput: 2120.4 tokens/s, Running: 235 reqs, Waiting: 20 reqs, GPU KV cache usage: 99.8%, Prefix cache hit rate: 68.0%
progress {"time": "03:02:36", "elapsed_min": 42.0, "submitted": 1768, "finished": 1512, "in_flight": 256, "gen_tokens": 4216273, "tok_per_s": 1673, "chain_tokens_median": 954.0, "chain_tokens_p90": 4651.5, "chain_tokens_max": 32658, "parse_rate": 0.9603174603174603, "looped": 58, "minutes_to_deadline": Infinity}
(MainProcess pid=153522) INFO 11-01 03:02:36 [loggers.py:335] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 2283.0 tokens/s, Running: 159 reqs, Waiting: 97 reqs, GPU KV cache usage: 99.8%, Prefix cache hit rate: 68.0%
progress {"time": "03:03:36", "elapsed_min": 43.0, "submitted": 1768, "finished": 1512, "in_flight": 256, "gen_tokens": 4216273, "tok_per_s": 1634, "chain_tokens_median": 954.0, "chain_tokens_p90": 4651.5, "chain_tokens_max": 32658, "parse_rate": 0.9603174603174603, "looped": 58, "minutes_to_deadline": Infinity}
(MainProcess pid=153522) INFO 11-01 03:03:36 [loggers.py:335] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 1646.3 tokens/s, Running: 134 reqs, Waiting: 122 reqs, GPU KV cache usage: 99.9%, Prefix cache hit rate: 68.0%
progress {"time": "03:04:36", "elapsed_min": 44.0, "submitted": 1770, "finished": 1514, "in_flight": 256, "gen_tokens": 4280996, "tok_per_s": 1621, "chain_tokens_median": 956.0, "chain_tokens_p90": 4668.4, "chain_tokens_max": 32658, "parse_rate": 0.9590488771466315, "looped": 60, "minutes_to_deadline": Infinity}
(MainProcess pid=153522) INFO 11-01 03:04:36 [loggers.py:335] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 1441.4 tokens/s, Running: 127 reqs, Waiting: 128 reqs, GPU KV cache usage: 99.9%, Prefix cache hit rate: 68.0%
progress {"time": "03:05:36", "elapsed_min": 45.0, "submitted": 1771, "finished": 1515, "in_flight": 256, "gen_tokens": 4313358, "tok_per_s": 1597, "chain_tokens_median": 956.0, "chain_tokens_p90": 4697.200000000006, "chain_tokens_max": 32658, "parse_rate": 0.9584158415841584, "looped": 61, "minutes_to_deadline": Infinity}
(MainProcess pid=153522) INFO 11-01 03:05:36 [loggers.py:335] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 1519.2 tokens/s, Running: 109 reqs, Waiting: 147 reqs, GPU KV cache usage: 98.6%, Prefix cache hit rate: 68.0%
progress {"time": "03:06:36", "elapsed_min": 46.0, "submitted": 1771, "finished": 1515, "in_flight": 256, "gen_tokens": 4313358, "tok_per_s": 1563, "chain_tokens_median": 956.0, "chain_tokens_p90": 4697.200000000006, "chain_tokens_max": 32658, "parse_rate": 0.9584158415841584, "looped": 61, "minutes_to_deadline": Infinity}
(MainProcess pid=153522) INFO 11-01 03:06:36 [loggers.py:335] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 1377.4 tokens/s, Running: 100 reqs, Waiting: 156 reqs, GPU KV cache usage: 99.9%, Prefix cache hit rate: 68.0%
progress {"time": "03:07:36", "elapsed_min": 47.0, "submitted": 1774, "finished": 1518, "in_flight": 256, "gen_tokens": 4410249, "tok_per_s": 1564, "chain_tokens_median": 959.0, "chain_tokens_p90": 4775.0999999999985, "chain_tokens_max": 32658, "parse_rate": 0.9565217391304348, "looped": 64, "minutes_to_deadline": Infinity}
(MainProcess pid=153522) INFO 11-01 03:07:36 [loggers.py:335] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 1109.0 tokens/s, Running: 101 reqs, Waiting: 155 reqs, GPU KV cache usage: 99.7%, Prefix cache hit rate: 68.0%
progress {"time": "03:08:36", "elapsed_min": 48.0, "submitted": 1775, "finished": 1519, "in_flight": 256, "gen_tokens": 4442844, "tok_per_s": 1543, "chain_tokens_median": 960.0, "chain_tokens_p90": 4794.6, "chain_tokens_max": 32658, "parse_rate": 0.955892034233048, "looped": 65, "minutes_to_deadline": Infinity}
(MainProcess pid=153522) INFO 11-01 03:08:36 [loggers.py:335] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 1312.2 tokens/s, Running: 94 reqs, Waiting: 162 reqs, GPU KV cache usage: 99.7%, Prefix cache hit rate: 68.0%
progress {"time": "03:09:36", "elapsed_min": 49.0, "submitted": 1777, "finished": 1521, "in_flight": 256, "gen_tokens": 4507652, "tok_per_s": 1533, "chain_tokens_median": 964.0, "chain_tokens_p90": 4804.0, "chain_tokens_max": 32658, "parse_rate": 0.9546351084812623, "looped": 67, "minutes_to_deadline": Infinity}
(MainProcess pid=153522) INFO 11-01 03:09:36 [loggers.py:335] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 1156.9 tokens/s, Running: 92 reqs, Waiting: 164 reqs, GPU KV cache usage: 100.0%, Prefix cache hit rate: 68.0%
progress {"time": "03:10:36", "elapsed_min": 50.0, "submitted": 1779, "finished": 1523, "in_flight": 256, "gen_tokens": 4572309, "tok_per_s": 1524, "chain_tokens_median": 964.0, "chain_tokens_p90": 4822.2, "chain_tokens_max": 32658, "parse_rate": 0.953381483913329, "looped": 69, "minutes_to_deadline": Infinity}
(MainProcess pid=153522) INFO 11-01 03:10:36 [loggers.py:335] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 1112.6 tokens/s, Running: 89 reqs, Waiting: 167 reqs, GPU KV cache usage: 99.6%, Prefix cache hit rate: 68.0%
progress {"time": "03:11:36", "elapsed_min": 51.0, "submitted": 1780, "finished": 1524, "in_flight": 256, "gen_tokens": 4603861, "tok_per_s": 1504, "chain_tokens_median": 965.5, "chain_tokens_p90": 4831.700000000001, "chain_tokens_max": 32658, "parse_rate": 0.952755905511811, "looped": 70, "minutes_to_deadline": Infinity}
(MainProcess pid=153522) INFO 11-01 03:11:36 [loggers.py:335] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 1226.9 tokens/s, Running: 84 reqs, Waiting: 172 reqs, GPU KV cache usage: 99.7%, Prefix cache hit rate: 68.0%
progress {"time": "03:12:36", "elapsed_min": 52.0, "submitted": 1782, "finished": 1526, "in_flight": 256, "gen_tokens": 4668171, "tok_per_s": 1496, "chain_tokens_median": 968.0, "chain_tokens_p90": 4907.5, "chain_tokens_max": 32658, "parse_rate": 0.9515072083879423, "looped": 72, "minutes_to_deadline": Infinity}
(MainProcess pid=153522) INFO 11-01 03:12:36 [loggers.py:335] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 1087.4 tokens/s, Running: 82 reqs, Waiting: 174 reqs, GPU KV cache usage: 99.6%, Prefix cache hit rate: 68.0%
progress {"time": "03:13:36", "elapsed_min": 53.0, "submitted": 1784, "finished": 1528, "in_flight": 256, "gen_tokens": 4732298, "tok_per_s": 1488, "chain_tokens_median": 970.0, "chain_tokens_p90": 4914.5, "chain_tokens_max": 32658, "parse_rate": 0.9502617801047121, "looped": 74, "minutes_to_deadline": Infinity}
(MainProcess pid=153522) INFO 11-01 03:13:36 [loggers.py:335] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 1263.2 tokens/s, Running: 79 reqs, Waiting: 177 reqs, GPU KV cache usage: 99.6%, Prefix cache hit rate: 68.0%
progress {"time": "03:14:36", "elapsed_min": 54.0, "submitted": 1788, "finished": 1532, "in_flight": 256, "gen_tokens": 4859630, "tok_per_s": 1500, "chain_tokens_median": 973.0, "chain_tokens_p90": 4960.400000000001, "chain_tokens_max": 32658, "parse_rate": 0.9477806788511749, "looped": 78, "minutes_to_deadline": Infinity}
(MainProcess pid=153522) INFO 11-01 03:14:36 [loggers.py:335] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 950.0 tokens/s, Running: 83 reqs, Waiting: 173 reqs, GPU KV cache usage: 99.6%, Prefix cache hit rate: 68.0%
progress {"time": "03:15:36", "elapsed_min": 55.0, "submitted": 1790, "finished": 1534, "in_flight": 256, "gen_tokens": 4923514, "tok_per_s": 1492, "chain_tokens_median": 976.0, "chain_tokens_p90": 5007.600000000002, "chain_tokens_max": 32658, "parse_rate": 0.9465449804432855, "looped": 80, "minutes_to_deadline": Infinity}
(MainProcess pid=153522) INFO 11-01 03:15:36 [loggers.py:335] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 1265.7 tokens/s, Running: 80 reqs, Waiting: 176 reqs, GPU KV cache usage: 99.3%, Prefix cache hit rate: 68.0%
progress {"time": "03:16:36", "elapsed_min": 56.0, "submitted": 1792, "finished": 1536, "in_flight": 256, "gen_tokens": 4988128, "tok_per_s": 1484, "chain_tokens_median": 978.0, "chain_tokens_p90": 5035.0, "chain_tokens_max": 32658, "parse_rate": 0.9453125, "looped": 82, "minutes_to_deadline": Infinity}
(MainProcess pid=153522) INFO 11-01 03:16:36 [loggers.py:335] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 1276.7 tokens/s, Running: 77 reqs, Waiting: 179 reqs, GPU KV cache usage: 99.9%, Prefix cache hit rate: 68.0%
progress {"time": "03:17:36", "elapsed_min": 57.0, "submitted": 1795, "finished": 1539, "in_flight": 256, "gen_tokens": 5084311, "tok_per_s": 1487, "chain_tokens_median": 979.0, "chain_tokens_p90": 5107.6, "chain_tokens_max": 32658, "parse_rate": 0.9434697855750487, "looped": 85, "minutes_to_deadline": Infinity}
(MainProcess pid=153522) INFO 11-01 03:17:36 [loggers.py:335] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 1150.7 tokens/s, Running: 78 reqs, Waiting: 178 reqs, GPU KV cache usage: 100.0%, Prefix cache hit rate: 68.0%
progress {"time": "03:18:36", "elapsed_min": 58.0, "submitted": 1798, "finished": 1542, "in_flight": 256, "gen_tokens": 5179843, "tok_per_s": 1488, "chain_tokens_median": 979.5, "chain_tokens_p90": 5145.6, "chain_tokens_max": 32658, "parse_rate": 0.9416342412451362, "looped": 88, "minutes_to_deadline": Infinity}
(MainProcess pid=153522) INFO 11-01 03:18:36 [loggers.py:335] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 1076.2 tokens/s, Running: 80 reqs, Waiting: 176 reqs, GPU KV cache usage: 100.0%, Prefix cache hit rate: 68.0%
progress {"time": "03:19:36", "elapsed_min": 59.0, "submitted": 1801, "finished": 1545, "in_flight": 256, "gen_tokens": 5276087, "tok_per_s": 1490, "chain_tokens_median": 982.0, "chain_tokens_p90": 5187.800000000007, "chain_tokens_max": 32658, "parse_rate": 0.9398058252427185, "looped": 91, "minutes_to_deadline": Infinity}
(MainProcess pid=153522) INFO 11-01 03:19:36 [loggers.py:335] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 1053.2 tokens/s, Running: 86 reqs, Waiting: 170 reqs, GPU KV cache usage: 99.9%, Prefix cache hit rate: 68.0%
progress {"time": "03:20:36", "elapsed_min": 60.0, "submitted": 1802, "finished": 1546, "in_flight": 256, "gen_tokens": 5308643, "tok_per_s": 1475, "chain_tokens_median": 983.0, "chain_tokens_p90": 5212.5, "chain_tokens_max": 32658, "parse_rate": 0.9391979301423027, "looped": 92, "minutes_to_deadline": Infinity}
(MainProcess pid=153522) INFO 11-01 03:20:36 [loggers.py:335] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 1334.1 tokens/s, Running: 75 reqs, Waiting: 181 reqs, GPU KV cache usage: 99.9%, Prefix cache hit rate: 68.0%
progress {"time": "03:21:36", "elapsed_min": 61.0, "submitted": 1806, "finished": 1550, "in_flight": 256, "gen_tokens": 5437217, "tok_per_s": 1485, "chain_tokens_median": 984.5, "chain_tokens_p90": 5286.500000000002, "chain_tokens_max": 32658, "parse_rate": 0.9367741935483871, "looped": 96, "minutes_to_deadline": Infinity}
(MainProcess pid=153522) INFO 11-01 03:21:36 [loggers.py:335] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 943.9 tokens/s, Running: 92 reqs, Waiting: 164 reqs, GPU KV cache usage: 99.9%, Prefix cache hit rate: 68.0%
progress {"time": "03:22:36", "elapsed_min": 62.0, "submitted": 1808, "finished": 1552, "in_flight": 256, "gen_tokens": 5501727, "tok_per_s": 1479, "chain_tokens_median": 985.0, "chain_tokens_p90": 5399.00000000001, "chain_tokens_max": 32658, "parse_rate": 0.9355670103092784, "looped": 98, "minutes_to_deadline": Infinity}
(MainProcess pid=153522) INFO 11-01 03:22:36 [loggers.py:335] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 1075.8 tokens/s, Running: 94 reqs, Waiting: 162 reqs, GPU KV cache usage: 99.9%, Prefix cache hit rate: 68.0%
progress {"time": "03:23:36", "elapsed_min": 63.0, "submitted": 1862, "finished": 1606, "in_flight": 256, "gen_tokens": 5706358, "tok_per_s": 1510, "chain_tokens_median": 1010.0, "chain_tokens_p90": 5355.0, "chain_tokens_max": 32658, "parse_rate": 0.935865504358655, "looped": 101, "minutes_to_deadline": Infinity}
(MainProcess pid=153522) INFO 11-01 03:23:36 [loggers.py:335] Engine 000: Avg prompt throughput: 2210.4 tokens/s, Avg generation throughput: 671.1 tokens/s, Running: 133 reqs, Waiting: 122 reqs, GPU KV cache usage: 99.3%, Prefix cache hit rate: 60.9%
{"time": "2026-11-01T03:24:27Z", "event": "skip", "stage": "2b_probe_qwen"}
{"time": "2026-11-01T03:24:27Z", "event": "skip", "stage": "2b_probe_minicpm"}
{"time": "2026-11-01T03:24:27Z", "event": "chosen", "stage": "2b_model=openbmb/MiniCPM5-2B"}
{"time": "2026-11-01T03:24:27Z", "event": "skip", "stage": "2b_smoke"}
{"time": "2026-11-01T03:24:27Z", "event": "skip", "stage": "2b_sweep_1e-4"}
{"time": "2026-11-01T03:24:27Z", "event": "skip", "stage": "2b_sweep_2e-4"}
{"time": "2026-11-01T03:24:27Z", "event": "skip", "stage": "2b_sweep_3e-4"}
{"time": "2026-11-01T03:24:27Z", "event": "chosen", "stage": "2b_lr=2e-4"}
{"time": "2026-11-01T03:24:27Z", "event": "skip", "stage": "2b_train_joint"}
{"time": "2026-11-01T03:24:27Z", "event": "skip", "stage": "2b_merge_joint"}
{"time": "2026-11-01T03:24:27Z", "event": "skip", "stage": "2b_eval_base"}
{"time": "2026-11-01T03:24:27Z", "event": "start", "stage": "2b_eval_joint"}
(MainProcess pid=156107) INFO 11-01 03:24:39 [model.py:733] Resolved architecture: LlamaForCausalLM
(MainProcess pid=156107) INFO 11-01 03:24:39 [model.py:2152] Using max model len 32768
(MainProcess pid=156107) INFO 11-01 03:24:39 [scheduler.py:307] Chunked prefill is enabled with max_num_batched_tokens=2048.
(MainProcess pid=156107) INFO 11-01 03:24:39 [kernel.py:467] Final IR op priority after setting platform defaults: IrOpPriorityConfig(rms_norm=['native'], fused_add_rms_norm=['native'], gelu_and_mul_sparse=['triton', 'native'])
(EngineCore pid=156379) INFO 11-01 03:24:41 [core.py:133] Initializing a V1 LLM engine (v0.32.0) with config: model='checkpoints/v2b_joint_merged', speculative_config=None, tokenizer='checkpoints/v2b_joint_merged', skip_tokenizer_init=False, tokenizer_mode=auto, revision=None, tokenizer_revision=None, trust_remote_code=False, dtype=torch.bfloat16, max_seq_len=32768, download_dir=None, load_format=auto, tensor_parallel_size=1, pipeline_parallel_size=1, data_parallel_size=1, decode_context_parallel_size=1, dcp_comm_backend=ag_rs, disable_custom_all_reduce=False, quantization=None, quantization_config=None, enforce_eager=False, aux_output_config=AuxOutputConfig(enable_return_routed_experts=False, max_bytes=None), kv_cache_dtype=auto, device_config=cuda, structured_outputs_config=StructuredOutputsConfig(backend='auto', disable_any_whitespace=False, disable_additional_properties=False, reasoning_parser='', reasoning_parser_plugin='', enable_in_reasoning=False), observability_config=ObservabilityConfig(show_hidden_metrics_for_version=None, otlp_traces_endpoint=None, collect_detailed_traces=None, per_request_spec_decode_metrics='none', kv_cache_metrics=False, kv_cache_metrics_sample=0.01, custom_histogram_buckets=None, cudagraph_metrics=False, enable_layerwise_nvtx_tracing=False, enable_mfu_metrics=False, enable_mm_processor_stats=False, enable_logging_iteration_details=False, jit_monitor_mode='warn', jit_monitor_verbose=False), seed=17, served_model_name=checkpoints/v2b_joint_merged, enable_prefix_caching=True, enable_chunked_prefill=True, pooler_config=None, compilation_config={'mode': <CompilationMode.VLLM_COMPILE: 3>, 'debug_dump_path': None, 'cache_dir': '', 'compile_cache_save_format': 'binary', 'backend': 'inductor', 'custom_ops': ['none'], 'ir_enable_torch_wrap': True, 'splitting_ops': ['vllm::unified_attention_with_output', 'vllm::unified_mla_attention_with_output', 'vllm::mamba_mixer2', 'vllm::mamba_mixer', 'vllm::short_conv', 'vllm::qwen4_exp_ple_short_conv', 'vllm::qwen4_exp_qsa_with_output', 'vllm::qwen4_exp_qsa_prepare_with_output', 'vllm::linear_attention', 'vllm::qwen_gdn_attention_core', 'vllm::qwen_gdn_attention_core_fused_norm_packed', 'vllm::gdn_attention_core_xpu', 'vllm::olmo_hybrid_gdn_full_forward', 'vllm::sparse_attn_indexer', 'vllm::rocm_aiter_sparse_attn_indexer', 'vllm::deepseek_v4_attention', 'vllm::hpc_rope_norm_forward', 'vllm::unified_kv_cache_update', 'vllm::unified_mla_kv_cache_update'], 'compile_mm_encoder': False, 'cudagraph_mm_encoder': False, 'encoder_cudagraph_token_budgets': [], 'encoder_cudagraph_max_vision_items_per_batch': 0, 'encoder_cudagraph_max_frames_per_batch': None, 'cudagraph_decoder_replay': True, 'decoder_replay_cudagraph_capture_sizes': [], 'decoder_replay_trim_threshold': 768, 'compile_sizes': [], 'compile_ranges_endpoints': [2048], 'inductor_compile_config': {'enable_auto_functionalized_v2': False, 'combo_kernels': True, 'benchmark_combo_kernel': True}, 'inductor_passes': {}, 'cudagraph_mode': <CUDAGraphMode.FULL_AND_PIECEWISE: (2, 1)>, 'cudagraph_num_of_warmups': 1, 'cudagraph_capture_sizes': [1, 2, 4, 8, 16, 24, 32, 40, 48, 56, 64, 72, 80, 88, 96, 104, 112, 120, 128, 136, 144, 152, 160, 168, 176, 184, 192], 'cudagraph_copy_inputs': False, 'cudagraph_specialize_lora': True, 'use_inductor_graph_partition': False, 'pass_config': {'fuse_norm_quant': False, 'fuse_act_quant': False, 'fuse_attn_quant': False, 'fuse_allreduce_rms': False, 'enable_qk_norm_rope_fusion': False, 'fuse_rope_kvcache_cat_mla': False, 'fuse_act_padding': False, 'fuse_qk_norm_rope_kvcache': False}, 'max_cudagraph_capture_size': 192, 'dynamic_shapes_config': {'type': <DynamicShapesType.BACKED: 'backed'>, 'evaluate_guards': False, 'assume_32_bit_indexing': False}, 'local_cache_dir': None, 'fast_moe_cold_start': False, 'static_all_moe_layers': []}, kernel_config=KernelConfig(ir_op_priority=IrOpPriorityConfig(rms_norm=['native'], fused_add_rms_norm=['native'], gelu_and_mul_sparse=['triton', 'native']), enable_flashinfer_autotune=True, enable_rocm_segmented_attn_autotune=False, enable_cutedsl_warmup=True, enable_jit_warmup=True, moe_backend='auto', sparse_indexer_topk_backend='auto', linear_backend='auto', linear_backend_per_quant=None)
(EngineCore pid=156379) INFO 11-01 03:24:43 [parallel_state.py:1906] world_size=1 rank=0 local_rank=0 distributed_init_method=file:///tmp/vllm_dist_5ff3fc7a187d4427a2292bb32bac974b backend=nccl
(EngineCore pid=156379) INFO 11-01 03:24:43 [parallel_state.py:2350] rank 0 in world size 1 is assigned as DP rank 0, PP rank 0, PCP rank 0, TP rank 0, ETP rank 0, EP rank N/A, EPLB rank N/A
(EngineCore pid=156379) INFO 11-01 03:24:43 [gpu_worker.py:518] Using V2 Model Runner
(EngineCore pid=156379) INFO 11-01 03:24:44 [model_runner.py:392] Loading model from scratch...
(EngineCore pid=156379) INFO 11-01 03:24:45 [cuda.py:506] Using FLASH_ATTN attention backend out of potential backends: ['FLASH_ATTN', 'FLASHINFER', 'TRITON_ATTN', 'FLEX_ATTENTION'].
(EngineCore pid=156379) INFO 11-01 03:24:45 [flash_attn.py:1117] Using FlashAttention version 2
(EngineCore pid=156379) INFO 11-01 03:24:45 [base_loader.py:82] Initializing model took 1.454567 seconds
(EngineCore pid=156379) INFO 11-01 03:24:45 [weight_utils.py:1108] Filesystem type for checkpoints: XFS. Checkpoint size: 4.69 GiB. Available RAM: 129.07 GiB.
(EngineCore pid=156379) INFO 11-01 03:24:45 [weight_utils.py:1131] Auto-prefetch is disabled because the filesystem (XFS) is not a recognized network FS (NFS/Lustre). If you want to force prefetching, start vLLM with --safetensors-load-strategy=prefetch.

Loading safetensors checkpoint shards:   0% Completed | 0/1 [00:00<?, ?it/s]

Loading safetensors checkpoint shards: 100% Completed | 1/1 [00:00<00:00,  1.43it/s]

Loading safetensors checkpoint shards: 100% Completed | 1/1 [00:00<00:00,  1.43it/s]

(EngineCore pid=156379) INFO 11-01 03:24:46 [default_loader.py:501] Loading weights took 0.72 seconds
(EngineCore pid=156379) INFO 11-01 03:24:48 [model_runner.py:424] Model loading took 4.75 GiB memory and 3.939792 seconds
(EngineCore pid=156379) INFO 11-01 03:24:48 [topk_topp_sampler.py:124] Using FlashInfer for top-p & top-k sampling.
(EngineCore pid=156379) INFO 11-01 03:24:48 [torch_utils.py:295] Reducing Torch threads from 128 to 1 for serving. Set OMP_NUM_THREADS in the external environment to override.
(EngineCore pid=156379) INFO 11-01 03:24:48 [utils.py:347] Using LBNHC KV cache layout.
(EngineCore pid=156379) INFO 11-01 03:24:49 [monitor.py:40] [backbone] Starting torch.compile
(EngineCore pid=156379) INFO 11-01 03:24:58 [backends.py:1104] [backbone] Using cache directory: /root/.cache/vllm/torch_compile_cache/96dd63e7a0/rank_0_0/backbone for vLLM's torch.compile
(EngineCore pid=156379) INFO 11-01 03:24:58 [backends.py:1166] [backbone] Dynamo bytecode transform time: 9.24 s
(EngineCore pid=156379) INFO 11-01 03:25:07 [backends.py:398] [backbone] Compiling a graph for compile range (1, 2048) takes 8.80 s
(EngineCore pid=156379) INFO 11-01 03:25:12 [backends.py:927] collected artifacts: 43 entries, 3 artifacts, 4889380 bytes total
(EngineCore pid=156379) INFO 11-01 03:25:13 [decorators.py:743] [backbone] saved AOT compiled function to /root/.cache/vllm/torch_compile_cache/torch_aot_compile/aa3f47b3a73c9a7c129db57924b313997a3010322f7d686911c02255ebf37710/rank_0_0/model
(EngineCore pid=156379) INFO 11-01 03:25:13 [monitor.py:62] [backbone] torch.compile took 23.91 s in total
(EngineCore pid=156379) INFO 11-01 03:25:13 [monitor.py:91] [backbone] Initial profiling/warmup run took 0.22 s

Capturing CUDA graphs (PIECEWISE):   0%|          | 0/27 [00:00<?, ?it/s]
Capturing CUDA graphs (PIECEWISE):   7%|▋         | 2/27 [00:00<00:01, 17.03it/s]
Capturing CUDA graphs (PIECEWISE):  19%|█▊        | 5/27 [00:00<00:01, 19.96it/s]
Capturing CUDA graphs (PIECEWISE):  30%|██▉       | 8/27 [00:00<00:00, 21.30it/s]
Capturing CUDA graphs (PIECEWISE):  41%|████      | 11/27 [00:00<00:00, 21.92it/s]
Capturing CUDA graphs (PIECEWISE):  52%|█████▏    | 14/27 [00:00<00:00, 21.31it/s]
Capturing CUDA graphs (PIECEWISE):  63%|██████▎   | 17/27 [00:00<00:00, 21.08it/s]
Capturing CUDA graphs (PIECEWISE):  74%|███████▍  | 20/27 [00:00<00:00, 20.81it/s]
Capturing CUDA graphs (PIECEWISE):  85%|████████▌ | 23/27 [00:01<00:00, 20.69it/s]
Capturing CUDA graphs (PIECEWISE):  96%|█████████▋| 26/27 [00:01<00:00, 19.45it/s]
Capturing CUDA graphs (PIECEWISE): 100%|██████████| 27/27 [00:01<00:00, 20.04it/s]

Capturing CUDA graphs (FULL):   0%|          | 0/2 [00:00<?, ?it/s]
Capturing CUDA graphs (FULL): 100%|██████████| 2/2 [00:00<00:00, 25.39it/s]
(EngineCore pid=156379) INFO 11-01 03:25:17 [model_runner.py:1127] Graph capturing finished in 2 secs, took 0.24 GiB
(EngineCore pid=156379) INFO 11-01 03:25:18 [gpu_worker.py:759] Available KV cache memory: 67.09 GiB
(EngineCore pid=156379) INFO 11-01 03:25:18 [gpu_worker.py:774] CUDA graph memory profiling is enabled (default since v0.21.0). The current --gpu-memory-utilization=0.9200 is equivalent to --gpu-memory-utilization=0.9164 without CUDA graph memory profiling. To maintain the same effective KV cache size as before, increase --gpu-memory-utilization to 0.9236. To disable, set VLLM_MEMORY_PROFILER_ESTIMATE_CUDAGRAPHS=0.
(EngineCore pid=156379) INFO 11-01 03:25:18 [kv_cache_utils.py:2327] GPU KV cache size: 1,675,072 tokens, Maximum concurrency for 32,768 tokens per request: 51.12x
(EngineCore pid=156379) INFO 11-01 03:25:18 [kernel_warmup.py:176] JIT kernel warmup starting.
(EngineCore pid=156379) INFO 11-01 03:25:18 [kernel_warmup.py:189] JIT kernel warmup finished in 0.00s.

Capturing CUDA graphs (PIECEWISE):   0%|          | 0/27 [00:00<?, ?it/s]
Capturing CUDA graphs (PIECEWISE):  11%|█         | 3/27 [00:00<00:01, 21.47it/s]
Capturing CUDA graphs (PIECEWISE):  22%|██▏       | 6/27 [00:00<00:00, 22.55it/s]
Capturing CUDA graphs (PIECEWISE):  33%|███▎      | 9/27 [00:00<00:00, 22.91it/s]
Capturing CUDA graphs (PIECEWISE):  44%|████▍     | 12/27 [00:00<00:00, 23.14it/s]
Capturing CUDA graphs (PIECEWISE):  56%|█████▌    | 15/27 [00:00<00:00, 22.57it/s]
Capturing CUDA graphs (PIECEWISE):  67%|██████▋   | 18/27 [00:00<00:00, 22.25it/s]
Capturing CUDA graphs (PIECEWISE):  78%|███████▊  | 21/27 [00:00<00:00, 21.95it/s]
Capturing CUDA graphs (PIECEWISE):  89%|████████▉ | 24/27 [00:01<00:00, 21.57it/s]
Capturing CUDA graphs (PIECEWISE): 100%|██████████| 27/27 [00:01<00:00, 20.76it/s]
Capturing CUDA graphs (PIECEWISE): 100%|██████████| 27/27 [00:01<00:00, 21.72it/s]

Capturing CUDA graphs (FULL):   0%|          | 0/15 [00:00<?, ?it/s]
Capturing CUDA graphs (FULL):  20%|██        | 3/15 [00:00<00:00, 24.65it/s]
Capturing CUDA graphs (FULL):  40%|████      | 6/15 [00:00<00:00, 24.72it/s]
Capturing CUDA graphs (FULL):  60%|██████    | 9/15 [00:00<00:00, 24.41it/s]
Capturing CUDA graphs (FULL):  80%|████████  | 12/15 [00:00<00:00, 24.91it/s]
Capturing CUDA graphs (FULL): 100%|██████████| 15/15 [00:00<00:00, 25.29it/s]
Capturing CUDA graphs (FULL): 100%|██████████| 15/15 [00:00<00:00, 25.01it/s]
(EngineCore pid=156379) INFO 11-01 03:25:38 [model_runner.py:1127] Graph capturing finished in 3 secs, took 0.12 GiB
(EngineCore pid=156379) INFO 11-01 03:25:38 [gpu_worker.py:971] CUDA graph pool memory: 0.12 GiB (actual), 0.29 GiB (estimated), difference: 0.17 GiB (146.7%).
(EngineCore pid=156379) INFO 11-01 03:25:38 [gpu_worker.py:1034] Free memory on device (78.84/79.25 GiB) on startup. Desired GPU memory utilization is (0.92, 72.91 GiB). Actual usage is 5.02 GiB for consumed memory (weights + non-torch), 0.79 GiB for peak activation, and 0.12 GiB for CUDAGraph memory. Replace gpu_memory_utilization config with `--kv-cache-memory=71758921954` (66.83 GiB) to fit into requested memory, or `--kv-cache-memory=78123091456` (72.76 GiB) to fully utilize gpu memory. Current kv cache memory in use is 67.09 GiB.
(EngineCore pid=156379) INFO 11-01 03:25:38 [jit_monitor.py:84] Kernel JIT monitor activated; monitored JIT compilations during inference will use mode=warn.
(EngineCore pid=156379) INFO 11-01 03:25:39 [core.py:395] init engine (profile, create kv cache, warmup model) took 51.20 s (compilation: 23.91 s)
(EngineCore pid=156379) INFO 11-01 03:25:39 [kernel.py:467] Final IR op priority after setting platform defaults: IrOpPriorityConfig(rms_norm=['native'], fused_add_rms_norm=['native'], gelu_and_mul_sparse=['triton', 'native'])
readout v2b: exists
(MainProcess pid=156107) WARNING 11-01 03:25:40 [input_processor.py:408] Passing raw prompts to InputProcessor is deprecated and will be removed in the future. You should instead pass the outputs of Renderer.render_cmpl() or Renderer.render_chat().
(EngineCore pid=156379) WARNING 11-01 03:25:41 [jit_monitor.py:140] Triton kernel JIT compilation during inference: _topp_sb_stats_kernel. This causes a latency spike; consider extending warmup to cover this shape/config.
(EngineCore pid=156379) WARNING 11-01 03:25:41 [jit_monitor.py:140] Triton kernel JIT compilation during inference: _topp_sb_step_kernel. This causes a latency spike; consider extending warmup to cover this shape/config.
(EngineCore pid=156379) WARNING 11-01 03:25:42 [jit_monitor.py:140] Triton kernel JIT compilation during inference: _topp_sb_mask_kernel. This causes a latency spike; consider extending warmup to cover this shape/config.
progress {"time": "03:26:39", "elapsed_min": 1.0, "submitted": 256, "finished": 0, "in_flight": 256, "gen_tokens": 0, "tok_per_s": 0, "chain_tokens_median": 0.0, "chain_tokens_p90": 0.0, "chain_tokens_max": 0, "parse_rate": null, "looped": 0, "minutes_to_deadline": Infinity}
(MainProcess pid=156107) INFO 11-01 03:26:39 [loggers.py:335] Engine 000: Avg prompt throughput: 1432.9 tokens/s, Avg generation throughput: 4778.9 tokens/s, Running: 96 reqs, Waiting: 159 reqs, GPU KV cache usage: 16.4%, Prefix cache hit rate: 7.4%
progress {"time": "03:27:39", "elapsed_min": 2.0, "submitted": 256, "finished": 0, "in_flight": 256, "gen_tokens": 0, "tok_per_s": 0, "chain_tokens_median": 0.0, "chain_tokens_p90": 0.0, "chain_tokens_max": 0, "parse_rate": null, "looped": 0, "minutes_to_deadline": Infinity}
(MainProcess pid=156107) INFO 11-01 03:27:39 [loggers.py:335] Engine 000: Avg prompt throughput: 597.1 tokens/s, Avg generation throughput: 4105.7 tokens/s, Running: 96 reqs, Waiting: 160 reqs, GPU KV cache usage: 22.7%, Prefix cache hit rate: 7.5%
(EngineCore pid=156379) WARNING 11-01 03:28:00 [jit_monitor.py:140] Triton kernel JIT compilation during inference: _topk_log_softmax_kernel. This causes a latency spike; consider extending warmup to cover this shape/config.
progress {"time": "03:28:39", "elapsed_min": 3.0, "submitted": 391, "finished": 162, "in_flight": 229, "gen_tokens": 313542, "tok_per_s": 1742, "chain_tokens_median": 1244.5, "chain_tokens_p90": 4499.6, "chain_tokens_max": 6212, "parse_rate": 1.0, "looped": 0, "minutes_to_deadline": Infinity}
(MainProcess pid=156107) INFO 11-01 03:28:39 [loggers.py:335] Engine 000: Avg prompt throughput: 509.6 tokens/s, Avg generation throughput: 3066.4 tokens/s, Running: 96 reqs, Waiting: 133 reqs, GPU KV cache usage: 25.2%, Prefix cache hit rate: 72.2%
progress {"time": "03:29:39", "elapsed_min": 4.0, "submitted": 391, "finished": 164, "in_flight": 227, "gen_tokens": 314699, "tok_per_s": 1311, "chain_tokens_median": 1233.5, "chain_tokens_p90": 4494.8, "chain_tokens_max": 6212, "parse_rate": 1.0, "looped": 0, "minutes_to_deadline": Infinity}
(MainProcess pid=156107) INFO 11-01 03:29:39 [loggers.py:335] Engine 000: Avg prompt throughput: 206.0 tokens/s, Avg generation throughput: 3050.2 tokens/s, Running: 96 reqs, Waiting: 131 reqs, GPU KV cache usage: 29.6%, Prefix cache hit rate: 70.8%
progress {"time": "03:30:39", "elapsed_min": 5.0, "submitted": 391, "finished": 164, "in_flight": 227, "gen_tokens": 314699, "tok_per_s": 1049, "chain_tokens_median": 1233.5, "chain_tokens_p90": 4494.8, "chain_tokens_max": 6212, "parse_rate": 1.0, "looped": 0, "minutes_to_deadline": Infinity}
(MainProcess pid=156107) INFO 11-01 03:30:39 [loggers.py:335] Engine 000: Avg prompt throughput: 177.7 tokens/s, Avg generation throughput: 2770.6 tokens/s, Running: 96 reqs, Waiting: 131 reqs, GPU KV cache usage: 36.9%, Prefix cache hit rate: 69.5%
progress {"time": "03:31:39", "elapsed_min": 6.0, "submitted": 391, "finished": 298, "in_flight": 93, "gen_tokens": 552070, "tok_per_s": 1533, "chain_tokens_median": 969.5, "chain_tokens_p90": 4459.0, "chain_tokens_max": 12053, "parse_rate": 1.0, "looped": 0, "minutes_to_deadline": Infinity}
(MainProcess pid=156107) INFO 11-01 03:31:39 [loggers.py:335] Engine 000: Avg prompt throughput: 84.8 tokens/s, Avg generation throughput: 2085.0 tokens/s, Running: 93 reqs, Waiting: 0 reqs, GPU KV cache usage: 41.8%, Prefix cache hit rate: 79.4%
progress {"time": "03:32:39", "elapsed_min": 7.0, "submitted": 391, "finished": 318, "in_flight": 73, "gen_tokens": 627336, "tok_per_s": 1494, "chain_tokens_median": 1053.5, "chain_tokens_p90": 4782.0, "chain_tokens_max": 12832, "parse_rate": 1.0, "looped": 0, "minutes_to_deadline": Infinity}
(MainProcess pid=156107) INFO 11-01 03:32:39 [loggers.py:335] Engine 000: Avg prompt throughput: 3.1 tokens/s, Avg generation throughput: 2675.8 tokens/s, Running: 73 reqs, Waiting: 0 reqs, GPU KV cache usage: 46.4%, Prefix cache hit rate: 81.2%
progress {"time": "03:33:39", "elapsed_min": 8.0, "submitted": 391, "finished": 323, "in_flight": 68, "gen_tokens": 658558, "tok_per_s": 1372, "chain_tokens_median": 1073.0, "chain_tokens_p90": 4898.8, "chain_tokens_max": 12832, "parse_rate": 1.0, "looped": 0, "minutes_to_deadline": Infinity}
(MainProcess pid=156107) INFO 11-01 03:33:39 [loggers.py:335] Engine 000: Avg prompt throughput: 0.8 tokens/s, Avg generation throughput: 2495.2 tokens/s, Running: 68 reqs, Waiting: 0 reqs, GPU KV cache usage: 53.4%, Prefix cache hit rate: 81.8%
progress {"time": "03:34:39", "elapsed_min": 9.0, "submitted": 391, "finished": 323, "in_flight": 68, "gen_tokens": 658558, "tok_per_s": 1220, "chain_tokens_median": 1073.0, "chain_tokens_p90": 4898.8, "chain_tokens_max": 12832, "parse_rate": 1.0, "looped": 0, "minutes_to_deadline": Infinity}
(MainProcess pid=156107) INFO 11-01 03:34:39 [loggers.py:335] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 2178.2 tokens/s, Running: 68 reqs, Waiting: 0 reqs, GPU KV cache usage: 61.2%, Prefix cache hit rate: 81.8%
progress {"time": "03:35:39", "elapsed_min": 10.0, "submitted": 391, "finished": 323, "in_flight": 68, "gen_tokens": 658558, "tok_per_s": 1098, "chain_tokens_median": 1073.0, "chain_tokens_p90": 4898.8, "chain_tokens_max": 12832, "parse_rate": 1.0, "looped": 0, "minutes_to_deadline": Infinity}
(MainProcess pid=156107) INFO 11-01 03:35:39 [loggers.py:335] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 1977.6 tokens/s, Running: 68 reqs, Waiting: 0 reqs, GPU KV cache usage: 68.2%, Prefix cache hit rate: 81.8%
progress {"time": "03:36:39", "elapsed_min": 11.0, "submitted": 391, "finished": 323, "in_flight": 68, "gen_tokens": 658558, "tok_per_s": 998, "chain_tokens_median": 1073.0, "chain_tokens_p90": 4898.8, "chain_tokens_max": 12832, "parse_rate": 1.0, "looped": 0, "minutes_to_deadline": Infinity}
(MainProcess pid=156107) INFO 11-01 03:36:39 [loggers.py:335] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 1823.5 tokens/s, Running: 68 reqs, Waiting: 0 reqs, GPU KV cache usage: 74.8%, Prefix cache hit rate: 81.8%
progress {"time": "03:37:39", "elapsed_min": 12.0, "submitted": 391, "finished": 323, "in_flight": 68, "gen_tokens": 658558, "tok_per_s": 915, "chain_tokens_median": 1073.0, "chain_tokens_p90": 4898.8, "chain_tokens_max": 12832, "parse_rate": 1.0, "looped": 0, "minutes_to_deadline": Infinity}
(MainProcess pid=156107) INFO 11-01 03:37:39 [loggers.py:335] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 1701.1 tokens/s, Running: 68 reqs, Waiting: 0 reqs, GPU KV cache usage: 80.9%, Prefix cache hit rate: 81.8%
progress {"time": "03:38:39", "elapsed_min": 13.0, "submitted": 391, "finished": 323, "in_flight": 68, "gen_tokens": 658558, "tok_per_s": 844, "chain_tokens_median": 1073.0, "chain_tokens_p90": 4898.8, "chain_tokens_max": 12832, "parse_rate": 1.0, "looped": 0, "minutes_to_deadline": Infinity}
(MainProcess pid=156107) INFO 11-01 03:38:39 [loggers.py:335] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 1601.3 tokens/s, Running: 68 reqs, Waiting: 0 reqs, GPU KV cache usage: 86.6%, Prefix cache hit rate: 81.8%
progress {"time": "03:39:39", "elapsed_min": 14.0, "submitted": 391, "finished": 323, "in_flight": 68, "gen_tokens": 658558, "tok_per_s": 784, "chain_tokens_median": 1073.0, "chain_tokens_p90": 4898.8, "chain_tokens_max": 12832, "parse_rate": 1.0, "looped": 0, "minutes_to_deadline": Infinity}
(MainProcess pid=156107) INFO 11-01 03:39:39 [loggers.py:335] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 1516.4 tokens/s, Running: 68 reqs, Waiting: 0 reqs, GPU KV cache usage: 92.0%, Prefix cache hit rate: 81.8%
progress {"time": "03:40:39", "elapsed_min": 15.0, "submitted": 391, "finished": 323, "in_flight": 68, "gen_tokens": 658558, "tok_per_s": 732, "chain_tokens_median": 1073.0, "chain_tokens_p90": 4898.8, "chain_tokens_max": 12832, "parse_rate": 1.0, "looped": 0, "minutes_to_deadline": Infinity}
(MainProcess pid=156107) INFO 11-01 03:40:39 [loggers.py:335] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 1443.8 tokens/s, Running: 68 reqs, Waiting: 0 reqs, GPU KV cache usage: 97.2%, Prefix cache hit rate: 81.8%
progress {"time": "03:41:39", "elapsed_min": 16.0, "submitted": 391, "finished": 324, "in_flight": 67, "gen_tokens": 683401, "tok_per_s": 712, "chain_tokens_median": 1078.5, "chain_tokens_p90": 4991.299999999998, "chain_tokens_max": 24843, "parse_rate": 0.9969135802469136, "looped": 1, "minutes_to_deadline": Infinity}
(MainProcess pid=156107) INFO 11-01 03:41:39 [loggers.py:335] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 1383.8 tokens/s, Running: 66 reqs, Waiting: 1 reqs, GPU KV cache usage: 99.2%, Prefix cache hit rate: 81.8%
progress {"time": "03:42:39", "elapsed_min": 17.0, "submitted": 391, "finished": 325, "in_flight": 66, "gen_tokens": 708376, "tok_per_s": 694, "chain_tokens_median": 1084.0, "chain_tokens_p90": 5057.000000000001, "chain_tokens_max": 24975, "parse_rate": 0.9938461538461538, "looped": 2, "minutes_to_deadline": Infinity}
(MainProcess pid=156107) INFO 11-01 03:42:39 [loggers.py:335] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 1215.9 tokens/s, Running: 63 reqs, Waiting: 3 reqs, GPU KV cache usage: 99.5%, Prefix cache hit rate: 81.8%
progress {"time": "03:43:39", "elapsed_min": 18.0, "submitted": 391, "finished": 325, "in_flight": 66, "gen_tokens": 708376, "tok_per_s": 656, "chain_tokens_median": 1084.0, "chain_tokens_p90": 5057.000000000001, "chain_tokens_max": 24975, "parse_rate": 0.9938461538461538, "looped": 2, "minutes_to_deadline": Infinity}
(MainProcess pid=156107) INFO 11-01 03:43:39 [loggers.py:335] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 1246.0 tokens/s, Running: 59 reqs, Waiting: 7 reqs, GPU KV cache usage: 99.0%, Prefix cache hit rate: 81.8%
progress {"time": "03:44:39", "elapsed_min": 19.0, "submitted": 391, "finished": 331, "in_flight": 60, "gen_tokens": 896960, "tok_per_s": 787, "chain_tokens_median": 1103.0, "chain_tokens_p90": 5533.0, "chain_tokens_max": 31672, "parse_rate": 0.9758308157099698, "looped": 8, "minutes_to_deadline": Infinity}
(MainProcess pid=156107) INFO 11-01 03:44:39 [loggers.py:335] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 738.0 tokens/s, Running: 60 reqs, Waiting: 0 reqs, GPU KV cache usage: 98.0%, Prefix cache hit rate: 81.8%
progress {"time": "03:45:39", "elapsed_min": 20.0, "submitted": 391, "finished": 348, "in_flight": 43, "gen_tokens": 1443453, "tok_per_s": 1203, "chain_tokens_median": 1220.5, "chain_tokens_p90": 6386.600000000001, "chain_tokens_max": 32609, "parse_rate": 0.9281609195402298, "looped": 25, "minutes_to_deadline": Infinity}
(MainProcess pid=156107) INFO 11-01 03:45:39 [loggers.py:335] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 1182.2 tokens/s, Running: 43 reqs, Waiting: 0 reqs, GPU KV cache usage: 69.0%, Prefix cache hit rate: 81.8%
progress {"time": "03:46:39", "elapsed_min": 21.0, "submitted": 391, "finished": 351, "in_flight": 40, "gen_tokens": 1538679, "tok_per_s": 1221, "chain_tokens_median": 1240.0, "chain_tokens_p90": 6943.0, "chain_tokens_max": 32609, "parse_rate": 0.9202279202279202, "looped": 28, "minutes_to_deadline": Infinity}
(MainProcess pid=156107) INFO 11-01 03:46:39 [loggers.py:335] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 1146.6 tokens/s, Running: 40 reqs, Waiting: 0 reqs, GPU KV cache usage: 67.2%, Prefix cache hit rate: 81.8%
progress {"time": "03:47:39", "elapsed_min": 22.0, "submitted": 391, "finished": 360, "in_flight": 31, "gen_tokens": 1829804, "tok_per_s": 1386, "chain_tokens_median": 1271.5, "chain_tokens_p90": 24856.200000000004, "chain_tokens_max": 32653, "parse_rate": 0.8972222222222223, "looped": 37, "minutes_to_deadline": Infinity}
(MainProcess pid=156107) INFO 11-01 03:47:39 [loggers.py:335] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 1088.9 tokens/s, Running: 31 reqs, Waiting: 0 reqs, GPU KV cache usage: 53.6%, Prefix cache hit rate: 81.8%
progress {"time": "03:48:39", "elapsed_min": 23.0, "submitted": 391, "finished": 369, "in_flight": 22, "gen_tokens": 2121216, "tok_per_s": 1537, "chain_tokens_median": 1378.0, "chain_tokens_p90": 31490.2, "chain_tokens_max": 32653, "parse_rate": 0.8753387533875339, "looped": 46, "minutes_to_deadline": Infinity}
(MainProcess pid=156107) INFO 11-01 03:48:39 [loggers.py:335] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 981.7 tokens/s, Running: 22 reqs, Waiting: 0 reqs, GPU KV cache usage: 39.5%, Prefix cache hit rate: 81.8%
progress {"time": "03:49:39", "elapsed_min": 24.0, "submitted": 391, "finished": 386, "in_flight": 5, "gen_tokens": 2672795, "tok_per_s": 1856, "chain_tokens_median": 1582.5, "chain_tokens_p90": 32301.0, "chain_tokens_max": 32654, "parse_rate": 0.8367875647668394, "looped": 63, "minutes_to_deadline": Infinity}
(MainProcess pid=156107) INFO 11-01 03:49:39 [loggers.py:335] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 711.5 tokens/s, Running: 5 reqs, Waiting: 0 reqs, GPU KV cache usage: 8.8%, Prefix cache hit rate: 81.8%
reason v2b: {"submitted": 391, "finished": 391, "gen_tokens": 2835202, "t0": 1793503539.7389824, "seconds": 1485.1973476409912, "tok_per_s": 1908.9732448709829}
reason v2b: post-chain readout 1844 taken at generation, 0 by prefill
reason v2b: 2013 items, post-readout in 1486s total
latency v2b: {"readout_ms": {"median": 21.76104485988617, "p90": 30.89566342532635, "n": 100}, "reasoning_ms": {"median": 6353.157198056579, "p90": 209582.13100507855, "n": 20}}
(MainProcess pid=156107) INFO 11-01 04:03:07 [core_client.py:770] [shutdown] MPClient: start timeout=default
(MainProcess pid=156107) INFO 11-01 04:03:07 [core_client.py:772] [shutdown] MPClient: stopping engine manager
(MainProcess pid=156107) INFO 11-01 04:03:07 [utils.py:628] [shutdown] Process manager: send sigterm to process EngineCore
(EngineCore pid=156379) INFO 11-01 04:03:07 [core.py:1454] [shutdown] EngineCore: trigger received signal=SIGTERM
(EngineCore pid=156379) INFO 11-01 04:03:07 [core.py:1635] [shutdown] EngineCore: start mode=abort timeout=0s
(EngineCore pid=156379) INFO 11-01 04:03:07 [core.py:1666] [shutdown] EngineCore: request processing complete; starting resource teardown
(EngineCore pid=156379) INFO 11-01 04:03:07 [core.py:1499] [shutdown] EngineCore: exiting busy loop
(MainProcess pid=156107) INFO 11-01 04:03:10 [core_client.py:774] [shutdown] MPClient: engine manager stopped
(MainProcess pid=156107) INFO 11-01 04:03:10 [core_client.py:775] [shutdown] MPClient: cleaning up background resources
(MainProcess pid=156107) INFO 11-01 04:03:10 [core_client.py:777] [shutdown] MPClient: complete
{"time": "2026-11-01T04:03:12Z", "event": "done", "stage": "2b_eval_joint"}
{"time": "2026-11-01T04:03:37Z", "event": "start", "stage": "2b_metrics"}
readout base         all 0.627  OD Cap 61.8  ECE 0.036  NLL 0.861  targets {'readout_acc_all': False, 'od_capability': False, 'agree_all': False, 'post_ece': True, 'reasoning_vs_base': True, 'loops_vs_base': True}
readout v1_joint     all 0.695  OD Cap 66.7  ECE 0.030  NLL 0.715  targets {'readout_acc_all': False, 'od_capability': False, 'agree_all': False, 'post_ece': True, 'reasoning_vs_base': True, 'loops_vs_base': False}
readout v2_joint     all 0.727  OD Cap 67.3  ECE 0.033  NLL 0.669  targets {'readout_acc_all': True, 'od_capability': False, 'agree_all': True, 'post_ece': True, 'reasoning_vs_base': False, 'loops_vs_base': False}
readout base2b       all 0.502  OD Cap 52.4  ECE 0.049  NLL 1.062  targets {'readout_acc_all': False, 'od_capability': False}
readout v2b          all 0.651  OD Cap 56.1  ECE 0.089  NLL 0.857  targets {'readout_acc_all': False, 'od_capability': False, 'agree_all': False, 'post_ece': False, 'reasoning_vs_base': False, 'loops_vs_base': False}
reason  base         all 0.840  tokens 1578  loops 13  post-ECE 0.050 (raw 0.149)
reason  v1_joint     all 0.844  tokens 1323  loops 21  post-ECE 0.023 (raw 0.144)
reason  v2_joint     all 0.812  tokens 1340  loops 76  post-ECE 0.043 (raw 0.125)
reason  v2b          all 0.717  tokens 1056  loops 111  post-ECE 0.089 (raw 0.210)
pair    base         agree 0.657  dis-AUROC 0.773  gap@0.3 0.50
pair    v1_joint     agree 0.757  dis-AUROC 0.780  gap@0.3 0.52
pair    v2_joint     agree 0.783  dis-AUROC 0.839  gap@0.3 0.70
pair    v2b          agree 0.751  dis-AUROC 0.809  gap@0.3 0.72
{"time": "2026-11-01T04:03:45Z", "event": "done", "stage": "2b_metrics"}
{"time": "2026-11-01T04:04:10Z", "event": "finished", "stage": "2b_phase_2b"}
DRIVER_EXIT 0
Xet Storage Details

Size:: 281 kB
Xet hash:: 7ae577fa11b7a4a5eeca49ec5de2e74aaf87f3f31a01d8edb657683c0c0a65c7
Xet efficiently stores files, intelligently splitting them into unique chunks and accelerating uploads and downloads. More info.